2024-09-14T23:45:42.151 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-14T23:45:42.166 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-14T23:45:42.254 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904803 branch: squid description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nocache} email: ceph-qa@ceph.com first_in_suite: false job_id: '7904803' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: conf: client: rbd cache: false global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluestore block size: 96636764160 bluestore compression algorithm: snappy bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - noscrub - nodeep-scrub sha1: ecab7be58963a5bd03910cd73eef66405d7fdd50 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: ecab7be58963a5bd03910cd73eef66405d7fdd50 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: ecab7be58963a5bd03910cd73eef66405d7fdd50 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 4271 sha1: ecab7be58963a5bd03910cd73eef66405d7fdd50 sleep_before_teardown: 0 subset: 97/128 suite: rbd suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_ecab7be58963a5bd03910cd73eef66405d7fdd50/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: ecab7be58963a5bd03910cd73eef66405d7fdd50 targets: smithi042.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDb3OL/nC1WjLPH+EZA18Mx7L9b0q+LMSqIFVWM1efshryjeK83pNoMNh38bJdpj8moukGrjZYn6T1QMNUWEDOw= smithi043.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLazenSCK9k57rjGeexoS3vIBbfy0qN3INEm2BDxrd7uuRvutxr20LZrUYuIBXBL6azuh/WGr3KYXsIzZlkbMwo= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 ops: 6000 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-14_21:16:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-14T23:45:42.254 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_ecab7be58963a5bd03910cd73eef66405d7fdd50/qa; will attempt to use it 2024-09-14T23:45:42.255 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_ecab7be58963a5bd03910cd73eef66405d7fdd50/qa/tasks 2024-09-14T23:45:42.255 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-14T23:45:42.257 INFO:teuthology.task.internal:Checking packages... 2024-09-14T23:45:42.277 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'ecab7be58963a5bd03910cd73eef66405d7fdd50' 2024-09-14T23:45:42.277 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-14T23:45:42.277 INFO:teuthology.packaging:ref: None 2024-09-14T23:45:42.277 INFO:teuthology.packaging:tag: None 2024-09-14T23:45:42.277 INFO:teuthology.packaging:branch: squid 2024-09-14T23:45:42.277 INFO:teuthology.packaging:sha1: ecab7be58963a5bd03910cd73eef66405d7fdd50 2024-09-14T23:45:42.278 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-14T23:45:42.508 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-324.gecab7be5 2024-09-14T23:45:42.509 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-14T23:45:42.516 INFO:teuthology.task.internal:no buildpackages task found 2024-09-14T23:45:42.516 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-14T23:45:42.536 INFO:teuthology.task.internal:Saving configuration 2024-09-14T23:45:42.549 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-14T23:45:42.587 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-14T23:45:42.613 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi042.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904803', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-14 23:41:00.419817', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDb3OL/nC1WjLPH+EZA18Mx7L9b0q+LMSqIFVWM1efshryjeK83pNoMNh38bJdpj8moukGrjZYn6T1QMNUWEDOw='} 2024-09-14T23:45:42.636 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi043.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904803', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-14 23:41:00.421324', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLazenSCK9k57rjGeexoS3vIBbfy0qN3INEm2BDxrd7uuRvutxr20LZrUYuIBXBL6azuh/WGr3KYXsIzZlkbMwo='} 2024-09-14T23:45:42.637 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-14T23:45:42.643 INFO:teuthology.task.internal:roles: ubuntu@smithi042.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-09-14T23:45:42.643 INFO:teuthology.task.internal:roles: ubuntu@smithi043.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-09-14T23:45:42.643 INFO:teuthology.run_tasks:Running task console_log... 2024-09-14T23:45:42.732 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f003c1fb250>, signals=[15]) 2024-09-14T23:45:42.732 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-14T23:45:42.739 INFO:teuthology.task.internal:Opening connections... 2024-09-14T23:45:42.739 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi042.front.sepia.ceph.com 2024-09-14T23:45:42.740 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T23:45:42.814 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi043.front.sepia.ceph.com 2024-09-14T23:45:42.815 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T23:45:42.886 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-14T23:45:42.893 DEBUG:teuthology.orchestra.run.smithi042:> uname -m 2024-09-14T23:45:42.910 INFO:teuthology.orchestra.run.smithi042.stdout:x86_64 2024-09-14T23:45:42.910 DEBUG:teuthology.orchestra.run.smithi042:> cat /etc/os-release 2024-09-14T23:45:42.967 INFO:teuthology.orchestra.run.smithi042.stdout:NAME="CentOS Stream" 2024-09-14T23:45:42.968 INFO:teuthology.orchestra.run.smithi042.stdout:VERSION="9" 2024-09-14T23:45:42.968 INFO:teuthology.orchestra.run.smithi042.stdout:ID="centos" 2024-09-14T23:45:42.968 INFO:teuthology.orchestra.run.smithi042.stdout:ID_LIKE="rhel fedora" 2024-09-14T23:45:42.968 INFO:teuthology.orchestra.run.smithi042.stdout:VERSION_ID="9" 2024-09-14T23:45:42.968 INFO:teuthology.orchestra.run.smithi042.stdout:PLATFORM_ID="platform:el9" 2024-09-14T23:45:42.968 INFO:teuthology.orchestra.run.smithi042.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-14T23:45:42.968 INFO:teuthology.orchestra.run.smithi042.stdout:ANSI_COLOR="0;31" 2024-09-14T23:45:42.968 INFO:teuthology.orchestra.run.smithi042.stdout:LOGO="fedora-logo-icon" 2024-09-14T23:45:42.968 INFO:teuthology.orchestra.run.smithi042.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-14T23:45:42.968 INFO:teuthology.orchestra.run.smithi042.stdout:HOME_URL="https://centos.org/" 2024-09-14T23:45:42.968 INFO:teuthology.orchestra.run.smithi042.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-14T23:45:42.968 INFO:teuthology.orchestra.run.smithi042.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-14T23:45:42.969 INFO:teuthology.orchestra.run.smithi042.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-14T23:45:42.969 INFO:teuthology.lock.ops:Updating smithi042.front.sepia.ceph.com on lock server 2024-09-14T23:45:42.989 DEBUG:teuthology.orchestra.run.smithi043:> uname -m 2024-09-14T23:45:43.007 INFO:teuthology.orchestra.run.smithi043.stdout:x86_64 2024-09-14T23:45:43.008 DEBUG:teuthology.orchestra.run.smithi043:> cat /etc/os-release 2024-09-14T23:45:43.064 INFO:teuthology.orchestra.run.smithi043.stdout:NAME="CentOS Stream" 2024-09-14T23:45:43.065 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION="9" 2024-09-14T23:45:43.065 INFO:teuthology.orchestra.run.smithi043.stdout:ID="centos" 2024-09-14T23:45:43.065 INFO:teuthology.orchestra.run.smithi043.stdout:ID_LIKE="rhel fedora" 2024-09-14T23:45:43.065 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION_ID="9" 2024-09-14T23:45:43.065 INFO:teuthology.orchestra.run.smithi043.stdout:PLATFORM_ID="platform:el9" 2024-09-14T23:45:43.065 INFO:teuthology.orchestra.run.smithi043.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-14T23:45:43.065 INFO:teuthology.orchestra.run.smithi043.stdout:ANSI_COLOR="0;31" 2024-09-14T23:45:43.065 INFO:teuthology.orchestra.run.smithi043.stdout:LOGO="fedora-logo-icon" 2024-09-14T23:45:43.065 INFO:teuthology.orchestra.run.smithi043.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-14T23:45:43.065 INFO:teuthology.orchestra.run.smithi043.stdout:HOME_URL="https://centos.org/" 2024-09-14T23:45:43.065 INFO:teuthology.orchestra.run.smithi043.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-14T23:45:43.065 INFO:teuthology.orchestra.run.smithi043.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-14T23:45:43.066 INFO:teuthology.orchestra.run.smithi043.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-14T23:45:43.066 INFO:teuthology.lock.ops:Updating smithi043.front.sepia.ceph.com on lock server 2024-09-14T23:45:43.089 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-14T23:45:43.098 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-14T23:45:43.159 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-14T23:45:43.159 DEBUG:teuthology.orchestra.run.smithi042:> test '!' -e /home/ubuntu/cephtest 2024-09-14T23:45:43.162 DEBUG:teuthology.orchestra.run.smithi043:> test '!' -e /home/ubuntu/cephtest 2024-09-14T23:45:43.178 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-14T23:45:43.188 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-14T23:45:43.188 DEBUG:teuthology.orchestra.run.smithi042:> test -z $(ls -A /var/lib/ceph) 2024-09-14T23:45:43.219 DEBUG:teuthology.orchestra.run.smithi043:> test -z $(ls -A /var/lib/ceph) 2024-09-14T23:45:43.273 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-14T23:45:43.333 INFO:teuthology.run_tasks:Running task kernel... 2024-09-14T23:45:43.347 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-14T23:45:43.347 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-14T23:45:43.347 DEBUG:teuthology.orchestra.run.smithi042:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-14T23:45:43.348 DEBUG:teuthology.orchestra.run.smithi043:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-14T23:45:43.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T23:45:43.364 DEBUG:teuthology.orchestra.run.smithi043:> uname -r 2024-09-14T23:45:43.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T23:45:43.365 DEBUG:teuthology.orchestra.run.smithi042:> uname -r 2024-09-14T23:45:43.421 INFO:teuthology.orchestra.run.smithi043.stdout:5.14.0-503.el9.x86_64 2024-09-14T23:45:43.421 INFO:teuthology.task.kernel:Running kernel on smithi043: 5.14.0-503.el9.x86_64 2024-09-14T23:45:43.421 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum install -y kernel 2024-09-14T23:45:43.422 INFO:teuthology.orchestra.run.smithi042.stdout:5.14.0-503.el9.x86_64 2024-09-14T23:45:43.423 INFO:teuthology.task.kernel:Running kernel on smithi042: 5.14.0-503.el9.x86_64 2024-09-14T23:45:43.423 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum install -y kernel 2024-09-14T23:45:45.706 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.2 MB 00:00 2024-09-14T23:45:47.191 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - BaseOS 4.0 MB/s | 8.2 MB 00:02 2024-09-14T23:45:47.773 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - AppStream 22 MB/s | 20 MB 00:00 2024-09-14T23:45:49.000 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - AppStream 32 MB/s | 20 MB 00:00 2024-09-14T23:45:52.232 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.5 MB 00:00 2024-09-14T23:45:53.751 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - CRB 8.8 MB/s | 6.5 MB 00:00 2024-09-14T23:45:53.774 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - Extras packages 62 kB/s | 19 kB 00:00 2024-09-14T23:45:54.970 INFO:teuthology.orchestra.run.smithi043.stdout:Extra Packages for Enterprise Linux 21 MB/s | 23 MB 00:01 2024-09-14T23:45:56.052 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - Extras packages 18 kB/s | 19 kB 00:01 2024-09-14T23:45:56.823 INFO:teuthology.orchestra.run.smithi042.stdout:Extra Packages for Enterprise Linux 36 MB/s | 23 MB 00:00 2024-09-14T23:46:00.584 INFO:teuthology.orchestra.run.smithi043.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-14T23:46:02.700 INFO:teuthology.orchestra.run.smithi043.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-14T23:46:02.700 INFO:teuthology.orchestra.run.smithi043.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-14T23:46:02.712 INFO:teuthology.orchestra.run.smithi042.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-14T23:46:02.788 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-14T23:46:02.793 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-14T23:46:02.793 INFO:teuthology.orchestra.run.smithi043.stdout: Package Architecture Version Repository Size 2024-09-14T23:46:02.793 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-14T23:46:02.793 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-14T23:46:02.793 INFO:teuthology.orchestra.run.smithi043.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-14T23:46:02.793 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-14T23:46:02.793 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-14T23:46:02.794 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-14T23:46:02.794 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-14T23:46:02.794 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:46:02.794 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-14T23:46:02.794 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-14T23:46:02.794 INFO:teuthology.orchestra.run.smithi043.stdout:Install 4 Packages 2024-09-14T23:46:02.794 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:46:02.794 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 79 M 2024-09-14T23:46:02.794 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 126 M 2024-09-14T23:46:02.794 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-14T23:46:03.003 INFO:teuthology.orchestra.run.smithi043.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 251 kB/s | 23 kB 00:00 2024-09-14T23:46:03.395 INFO:teuthology.orchestra.run.smithi043.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 32 MB/s | 16 MB 00:00 2024-09-14T23:46:03.845 INFO:teuthology.orchestra.run.smithi043.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 37 MB/s | 35 MB 00:00 2024-09-14T23:46:04.412 INFO:teuthology.orchestra.run.smithi043.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 20 MB/s | 29 MB 00:01 2024-09-14T23:46:04.412 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-14T23:46:04.412 INFO:teuthology.orchestra.run.smithi043.stdout:Total 49 MB/s | 79 MB 00:01 2024-09-14T23:46:04.843 INFO:teuthology.orchestra.run.smithi042.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-14T23:46:04.843 INFO:teuthology.orchestra.run.smithi042.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-14T23:46:04.939 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-14T23:46:04.950 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-14T23:46:04.950 INFO:teuthology.orchestra.run.smithi042.stdout: Package Architecture Version Repository Size 2024-09-14T23:46:04.950 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-14T23:46:04.950 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-09-14T23:46:04.950 INFO:teuthology.orchestra.run.smithi042.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-14T23:46:04.950 INFO:teuthology.orchestra.run.smithi042.stdout:Installing dependencies: 2024-09-14T23:46:04.950 INFO:teuthology.orchestra.run.smithi042.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-14T23:46:04.950 INFO:teuthology.orchestra.run.smithi042.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-14T23:46:04.950 INFO:teuthology.orchestra.run.smithi042.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-14T23:46:04.950 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:46:04.950 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-09-14T23:46:04.951 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-14T23:46:04.951 INFO:teuthology.orchestra.run.smithi042.stdout:Install 4 Packages 2024-09-14T23:46:04.951 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:46:04.951 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 79 M 2024-09-14T23:46:04.951 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 126 M 2024-09-14T23:46:04.951 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-09-14T23:46:05.125 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-14T23:46:05.510 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-14T23:46:05.511 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-14T23:46:05.530 INFO:teuthology.orchestra.run.smithi042.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 76 kB/s | 23 kB 00:00 2024-09-14T23:46:06.511 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-14T23:46:06.511 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-14T23:46:06.622 INFO:teuthology.orchestra.run.smithi042.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-14T23:46:07.364 INFO:teuthology.orchestra.run.smithi042.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 16 MB/s | 35 MB 00:02 2024-09-14T23:46:07.631 INFO:teuthology.orchestra.run.smithi042.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 14 MB/s | 29 MB 00:02 2024-09-14T23:46:07.631 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-09-14T23:46:07.631 INFO:teuthology.orchestra.run.smithi042.stdout:Total 29 MB/s | 79 MB 00:02 2024-09-14T23:46:08.342 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-09-14T23:46:08.731 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-09-14T23:46:08.732 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-09-14T23:46:09.081 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-14T23:46:09.747 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-09-14T23:46:09.748 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-09-14T23:46:09.749 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-14T23:46:09.848 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-14T23:46:12.319 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-09-14T23:46:12.665 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-14T23:46:12.791 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-14T23:46:12.999 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-14T23:46:13.082 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-14T23:46:15.870 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-14T23:46:15.995 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-14T23:46:17.912 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-14T23:46:17.950 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-14T23:46:21.122 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-14T23:46:21.188 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-14T23:46:23.084 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-14T23:46:26.331 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-14T23:47:02.204 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-14T23:47:02.245 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-14T23:47:02.634 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-14T23:47:02.634 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-14T23:47:02.634 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-14T23:47:02.635 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-14T23:47:03.517 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-14T23:47:03.517 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:47:03.517 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-14T23:47:03.517 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-14T23:47:03.517 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-14T23:47:03.517 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-14T23:47:03.517 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-14T23:47:03.517 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:47:03.517 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-14T23:47:03.874 DEBUG:teuthology.orchestra.run.smithi043:> rpm -q kernel | sort -rV | head -n 1 2024-09-14T23:47:03.960 INFO:teuthology.orchestra.run.smithi043.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-14T23:47:03.960 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-14T23:47:03.961 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-14T23:47:03.961 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-14T23:47:03.961 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-14T23:47:03.961 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-14T23:47:03.961 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi043.front.sepia.ceph.com, path=None, version=distro) 2024-09-14T23:47:03.961 DEBUG:teuthology.orchestra.run.smithi043:> rpm -q kernel | sort -rV | head -n 1 2024-09-14T23:47:04.019 INFO:teuthology.orchestra.run.smithi043.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-14T23:47:04.019 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-14T23:47:04.020 DEBUG:teuthology.orchestra.run.smithi043:> sudo rpm -qi grub2-tools 2024-09-14T23:47:04.066 INFO:teuthology.orchestra.run.smithi043.stdout:Name : grub2-tools 2024-09-14T23:47:04.066 INFO:teuthology.orchestra.run.smithi043.stdout:Epoch : 1 2024-09-14T23:47:04.066 INFO:teuthology.orchestra.run.smithi043.stdout:Version : 2.06 2024-09-14T23:47:04.066 INFO:teuthology.orchestra.run.smithi043.stdout:Release : 82.el9 2024-09-14T23:47:04.066 INFO:teuthology.orchestra.run.smithi043.stdout:Architecture: x86_64 2024-09-14T23:47:04.066 INFO:teuthology.orchestra.run.smithi043.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-14T23:47:04.067 INFO:teuthology.orchestra.run.smithi043.stdout:Group : Unspecified 2024-09-14T23:47:04.067 INFO:teuthology.orchestra.run.smithi043.stdout:Size : 8274670 2024-09-14T23:47:04.067 INFO:teuthology.orchestra.run.smithi043.stdout:License : GPLv3+ 2024-09-14T23:47:04.067 INFO:teuthology.orchestra.run.smithi043.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-14T23:47:04.067 INFO:teuthology.orchestra.run.smithi043.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-14T23:47:04.067 INFO:teuthology.orchestra.run.smithi043.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-14T23:47:04.067 INFO:teuthology.orchestra.run.smithi043.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-14T23:47:04.067 INFO:teuthology.orchestra.run.smithi043.stdout:Packager : builder@centos.org 2024-09-14T23:47:04.067 INFO:teuthology.orchestra.run.smithi043.stdout:Vendor : CentOS 2024-09-14T23:47:04.067 INFO:teuthology.orchestra.run.smithi043.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-14T23:47:04.067 INFO:teuthology.orchestra.run.smithi043.stdout:Summary : Support tools for GRUB. 2024-09-14T23:47:04.067 INFO:teuthology.orchestra.run.smithi043.stdout:Description : 2024-09-14T23:47:04.067 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:47:04.067 INFO:teuthology.orchestra.run.smithi043.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-14T23:47:04.068 INFO:teuthology.orchestra.run.smithi043.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-14T23:47:04.068 INFO:teuthology.orchestra.run.smithi043.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-14T23:47:04.068 INFO:teuthology.orchestra.run.smithi043.stdout:hardware devices. 2024-09-14T23:47:04.068 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:47:04.068 INFO:teuthology.orchestra.run.smithi043.stdout:This subpackage provides tools for support of all platforms. 2024-09-14T23:47:04.069 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-14T23:47:04.069 INFO:teuthology.task.kernel:Updating grub on smithi043 to boot 5.14.0-505.el9.x86_64 2024-09-14T23:47:04.069 DEBUG:teuthology.orchestra.run.smithi043:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-14T23:47:04.767 INFO:teuthology.orchestra.run.smithi043.stderr:Generating grub configuration file ... 2024-09-14T23:47:05.542 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-14T23:47:05.583 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-14T23:47:05.891 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-14T23:47:05.891 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-14T23:47:05.891 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-14T23:47:05.891 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-14T23:47:06.154 INFO:teuthology.orchestra.run.smithi043.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-14T23:47:06.171 INFO:teuthology.orchestra.run.smithi043.stderr:done 2024-09-14T23:47:06.173 DEBUG:teuthology.orchestra.run.smithi043:> mktemp 2024-09-14T23:47:06.188 INFO:teuthology.orchestra.run.smithi043.stdout:/tmp/tmp.JQl5E9RFWf 2024-09-14T23:47:06.188 DEBUG:teuthology.orchestra.run.smithi043:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.JQl5E9RFWf 2024-09-14T23:47:06.255 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 0666 /tmp/tmp.JQl5E9RFWf 2024-09-14T23:47:06.400 DEBUG:teuthology.orchestra.remote:smithi043:/tmp/tmp.JQl5E9RFWf is 6KB 2024-09-14T23:47:06.420 DEBUG:teuthology.orchestra.run.smithi043:> rm -fr /tmp/tmp.JQl5E9RFWf 2024-09-14T23:47:06.435 DEBUG:teuthology.orchestra.run.smithi043:> sudo /bin/ls /boot/loader/entries || true 2024-09-14T23:47:06.506 INFO:teuthology.orchestra.run.smithi043.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-14T23:47:06.506 INFO:teuthology.orchestra.run.smithi043.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-14T23:47:06.506 INFO:teuthology.orchestra.run.smithi043.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-14T23:47:06.506 INFO:teuthology.orchestra.run.smithi043.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-14T23:47:06.507 DEBUG:teuthology.orchestra.run.smithi043:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-14T23:47:06.722 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-14T23:47:06.722 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:47:06.722 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-09-14T23:47:06.722 INFO:teuthology.orchestra.run.smithi042.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-14T23:47:06.722 INFO:teuthology.orchestra.run.smithi042.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-14T23:47:06.722 INFO:teuthology.orchestra.run.smithi042.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-14T23:47:06.722 INFO:teuthology.orchestra.run.smithi042.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-14T23:47:06.723 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:47:06.723 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-14T23:47:06.784 DEBUG:teuthology.orchestra.run.smithi043:> sudo shutdown -r now 2024-09-14T23:47:07.048 DEBUG:teuthology.orchestra.run.smithi042:> rpm -q kernel | sort -rV | head -n 1 2024-09-14T23:47:07.184 INFO:teuthology.orchestra.run.smithi042.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-14T23:47:07.184 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-14T23:47:07.184 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-14T23:47:07.184 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-14T23:47:07.185 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-14T23:47:07.185 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-14T23:47:07.185 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi042.front.sepia.ceph.com, path=None, version=distro) 2024-09-14T23:47:07.185 DEBUG:teuthology.orchestra.run.smithi042:> rpm -q kernel | sort -rV | head -n 1 2024-09-14T23:47:07.243 INFO:teuthology.orchestra.run.smithi042.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-14T23:47:07.243 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-14T23:47:07.244 DEBUG:teuthology.orchestra.run.smithi042:> sudo rpm -qi grub2-tools 2024-09-14T23:47:07.282 INFO:teuthology.orchestra.run.smithi042.stdout:Name : grub2-tools 2024-09-14T23:47:07.282 INFO:teuthology.orchestra.run.smithi042.stdout:Epoch : 1 2024-09-14T23:47:07.282 INFO:teuthology.orchestra.run.smithi042.stdout:Version : 2.06 2024-09-14T23:47:07.282 INFO:teuthology.orchestra.run.smithi042.stdout:Release : 82.el9 2024-09-14T23:47:07.282 INFO:teuthology.orchestra.run.smithi042.stdout:Architecture: x86_64 2024-09-14T23:47:07.282 INFO:teuthology.orchestra.run.smithi042.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-14T23:47:07.282 INFO:teuthology.orchestra.run.smithi042.stdout:Group : Unspecified 2024-09-14T23:47:07.282 INFO:teuthology.orchestra.run.smithi042.stdout:Size : 8274670 2024-09-14T23:47:07.282 INFO:teuthology.orchestra.run.smithi042.stdout:License : GPLv3+ 2024-09-14T23:47:07.282 INFO:teuthology.orchestra.run.smithi042.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-14T23:47:07.283 INFO:teuthology.orchestra.run.smithi042.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-14T23:47:07.283 INFO:teuthology.orchestra.run.smithi042.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-14T23:47:07.283 INFO:teuthology.orchestra.run.smithi042.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-14T23:47:07.283 INFO:teuthology.orchestra.run.smithi042.stdout:Packager : builder@centos.org 2024-09-14T23:47:07.283 INFO:teuthology.orchestra.run.smithi042.stdout:Vendor : CentOS 2024-09-14T23:47:07.283 INFO:teuthology.orchestra.run.smithi042.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-14T23:47:07.283 INFO:teuthology.orchestra.run.smithi042.stdout:Summary : Support tools for GRUB. 2024-09-14T23:47:07.283 INFO:teuthology.orchestra.run.smithi042.stdout:Description : 2024-09-14T23:47:07.283 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:47:07.283 INFO:teuthology.orchestra.run.smithi042.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-14T23:47:07.283 INFO:teuthology.orchestra.run.smithi042.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-14T23:47:07.283 INFO:teuthology.orchestra.run.smithi042.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-14T23:47:07.283 INFO:teuthology.orchestra.run.smithi042.stdout:hardware devices. 2024-09-14T23:47:07.283 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:47:07.284 INFO:teuthology.orchestra.run.smithi042.stdout:This subpackage provides tools for support of all platforms. 2024-09-14T23:47:07.284 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-14T23:47:07.285 INFO:teuthology.task.kernel:Updating grub on smithi042 to boot 5.14.0-505.el9.x86_64 2024-09-14T23:47:07.285 DEBUG:teuthology.orchestra.run.smithi042:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-14T23:47:07.980 INFO:teuthology.orchestra.run.smithi042.stderr:Generating grub configuration file ... 2024-09-14T23:47:09.303 INFO:teuthology.orchestra.run.smithi042.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-14T23:47:09.332 INFO:teuthology.orchestra.run.smithi042.stderr:done 2024-09-14T23:47:09.338 DEBUG:teuthology.orchestra.run.smithi042:> mktemp 2024-09-14T23:47:09.353 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/tmp.3DPegSuuH4 2024-09-14T23:47:09.354 DEBUG:teuthology.orchestra.run.smithi042:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.3DPegSuuH4 2024-09-14T23:47:09.420 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 0666 /tmp/tmp.3DPegSuuH4 2024-09-14T23:47:09.557 DEBUG:teuthology.orchestra.remote:smithi042:/tmp/tmp.3DPegSuuH4 is 6KB 2024-09-14T23:47:09.606 DEBUG:teuthology.orchestra.run.smithi042:> rm -fr /tmp/tmp.3DPegSuuH4 2024-09-14T23:47:09.621 DEBUG:teuthology.orchestra.run.smithi042:> sudo /bin/ls /boot/loader/entries || true 2024-09-14T23:47:09.683 INFO:teuthology.orchestra.run.smithi042.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-14T23:47:09.684 INFO:teuthology.orchestra.run.smithi042.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-14T23:47:09.684 INFO:teuthology.orchestra.run.smithi042.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-14T23:47:09.684 INFO:teuthology.orchestra.run.smithi042.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-14T23:47:09.685 DEBUG:teuthology.orchestra.run.smithi042:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-14T23:47:09.825 DEBUG:teuthology.orchestra.run.smithi042:> sudo shutdown -r now 2024-09-14T23:47:36.805 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-14T23:47:36.805 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-09-14T23:47:36.806 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T23:47:39.830 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-14T23:47:39.831 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-09-14T23:47:39.831 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T23:47:55.325 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.43 2024-09-14T23:48:04.333 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-09-14T23:48:04.333 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T23:48:39.869 DEBUG:teuthology.orchestra.remote:timed out 2024-09-14T23:48:48.877 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-09-14T23:48:48.878 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T23:48:49.167 DEBUG:teuthology.orchestra.run.smithi042:> true 2024-09-14T23:48:49.586 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-09-14T23:48:49.586 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-14T23:48:49.587 DEBUG:teuthology.orchestra.run.smithi042:> uname -r 2024-09-14T23:48:49.602 INFO:teuthology.orchestra.run.smithi042.stdout:5.14.0-505.el9.x86_64 2024-09-14T23:48:49.602 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-14T23:48:49.602 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-14T23:48:49.602 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-14T23:48:50.603 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-14T23:48:50.604 DEBUG:teuthology.orchestra.run.smithi042:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-14T23:48:50.704 INFO:teuthology.orchestra.run.smithi042.stdout:ttyS1 2024-09-14T23:48:50.733 DEBUG:teuthology.parallel:result is None 2024-09-14T23:49:04.335 DEBUG:teuthology.orchestra.remote:timed out 2024-09-14T23:49:16.336 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-09-14T23:49:16.337 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T23:49:16.666 DEBUG:teuthology.orchestra.run.smithi043:> true 2024-09-14T23:49:17.018 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-09-14T23:49:17.018 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-14T23:49:17.018 DEBUG:teuthology.orchestra.run.smithi043:> uname -r 2024-09-14T23:49:17.035 INFO:teuthology.orchestra.run.smithi043.stdout:5.14.0-505.el9.x86_64 2024-09-14T23:49:17.035 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-14T23:49:17.035 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-14T23:49:17.035 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-14T23:49:18.036 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-14T23:49:18.036 DEBUG:teuthology.orchestra.run.smithi043:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-14T23:49:18.115 INFO:teuthology.orchestra.run.smithi043.stdout:ttyS1 2024-09-14T23:49:18.138 DEBUG:teuthology.parallel:result is None 2024-09-14T23:49:18.138 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-14T23:49:18.146 INFO:teuthology.task.internal:Creating test directory... 2024-09-14T23:49:18.146 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-14T23:49:18.150 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-14T23:49:18.169 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-14T23:49:18.199 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-14T23:49:18.258 INFO:teuthology.task.internal:Creating archive directory... 2024-09-14T23:49:18.258 DEBUG:teuthology.orchestra.run.smithi042:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-14T23:49:18.261 DEBUG:teuthology.orchestra.run.smithi043:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-14T23:49:18.293 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-14T23:49:18.440 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-14T23:49:18.440 DEBUG:teuthology.orchestra.run.smithi042:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-14T23:49:18.443 DEBUG:teuthology.orchestra.run.smithi043:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-14T23:49:18.473 INFO:teuthology.orchestra.run.smithi042.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-14T23:49:18.481 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-14T23:49:18.485 INFO:teuthology.orchestra.run.smithi042.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-14T23:49:18.493 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-14T23:49:18.495 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-14T23:49:18.502 INFO:teuthology.task.internal:Configuring sudo... 2024-09-14T23:49:18.502 DEBUG:teuthology.orchestra.run.smithi042:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-14T23:49:18.530 DEBUG:teuthology.orchestra.run.smithi043:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-14T23:49:18.564 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-14T23:49:18.573 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-14T23:49:18.573 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-14T23:49:18.600 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-14T23:49:18.621 DEBUG:teuthology.orchestra.run.smithi042:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-14T23:49:18.700 DEBUG:teuthology.orchestra.run.smithi042:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-14T23:49:18.752 DEBUG:teuthology.orchestra.run.smithi042:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-14T23:49:18.837 DEBUG:teuthology.orchestra.run.smithi042:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-14T23:49:18.881 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-14T23:49:18.881 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-14T23:49:18.948 DEBUG:teuthology.orchestra.run.smithi043:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-14T23:49:18.996 DEBUG:teuthology.orchestra.run.smithi043:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-14T23:49:19.045 DEBUG:teuthology.orchestra.run.smithi043:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-14T23:49:19.129 DEBUG:teuthology.orchestra.run.smithi043:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-14T23:49:19.173 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-14T23:49:19.173 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-14T23:49:19.240 DEBUG:teuthology.orchestra.run.smithi042:> sudo service rsyslog restart 2024-09-14T23:49:19.244 DEBUG:teuthology.orchestra.run.smithi043:> sudo service rsyslog restart 2024-09-14T23:49:19.297 INFO:teuthology.orchestra.run.smithi042.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-14T23:49:19.320 INFO:teuthology.orchestra.run.smithi043.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-14T23:49:19.809 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-14T23:49:19.816 INFO:teuthology.task.internal:Starting timer... 2024-09-14T23:49:19.816 INFO:teuthology.run_tasks:Running task pcp... 2024-09-14T23:49:19.868 INFO:teuthology.run_tasks:Running task selinux... 2024-09-14T23:49:19.883 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-09-14T23:49:19.883 DEBUG:teuthology.orchestra.run.smithi042:> sudo service auditd rotate 2024-09-14T23:49:19.929 INFO:teuthology.orchestra.run.smithi042.stdout:Rotating logs: 2024-09-14T23:49:19.931 DEBUG:teuthology.orchestra.run.smithi043:> sudo service auditd rotate 2024-09-14T23:49:19.986 INFO:teuthology.orchestra.run.smithi043.stdout:Rotating logs: 2024-09-14T23:49:19.988 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-14T23:49:19.988 DEBUG:teuthology.orchestra.run.smithi042:> /usr/sbin/getenforce 2024-09-14T23:49:20.011 INFO:teuthology.orchestra.run.smithi042.stdout:Permissive 2024-09-14T23:49:20.012 DEBUG:teuthology.orchestra.run.smithi043:> /usr/sbin/getenforce 2024-09-14T23:49:20.038 INFO:teuthology.orchestra.run.smithi043.stdout:Permissive 2024-09-14T23:49:20.038 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi042.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi043.front.sepia.ceph.com': 'permissive'} 2024-09-14T23:49:20.038 DEBUG:teuthology.orchestra.run.smithi042:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-14T23:49:20.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T23:49:20.081 DEBUG:teuthology.orchestra.run.smithi043:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-14T23:49:20.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T23:49:20.110 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-14T23:49:20.110 DEBUG:teuthology.orchestra.run.smithi042:> sudo /usr/sbin/setenforce permissive 2024-09-14T23:49:20.151 DEBUG:teuthology.orchestra.run.smithi043:> sudo /usr/sbin/setenforce permissive 2024-09-14T23:49:20.186 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-14T23:49:20.195 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-14T23:49:20.206 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-14T23:49:20.207 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi042.front.sepia.ceph.com,smithi043.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-14T23:53:10.960 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi042.front.sepia.ceph.com'), Remote(name='ubuntu@smithi043.front.sepia.ceph.com')] 2024-09-14T23:53:10.962 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-09-14T23:53:10.963 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T23:53:11.040 DEBUG:teuthology.orchestra.run.smithi042:> true 2024-09-14T23:53:11.086 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-09-14T23:53:11.086 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-09-14T23:53:11.087 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T23:53:11.160 DEBUG:teuthology.orchestra.run.smithi043:> true 2024-09-14T23:53:11.241 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-09-14T23:53:11.242 INFO:teuthology.run_tasks:Running task clock... 2024-09-14T23:53:11.252 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-14T23:53:11.253 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-14T23:53:11.253 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-14T23:53:11.256 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-14T23:53:11.256 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-14T23:53:11.289 INFO:teuthology.orchestra.run.smithi042.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-14T23:53:11.305 INFO:teuthology.orchestra.run.smithi042.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-14T23:53:11.314 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-14T23:53:11.329 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-14T23:53:11.332 INFO:teuthology.orchestra.run.smithi042.stderr:sudo: ntpd: command not found 2024-09-14T23:53:11.344 INFO:teuthology.orchestra.run.smithi042.stdout:506 Cannot talk to daemon 2024-09-14T23:53:11.356 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: ntpd: command not found 2024-09-14T23:53:11.359 INFO:teuthology.orchestra.run.smithi042.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-14T23:53:11.368 INFO:teuthology.orchestra.run.smithi043.stdout:506 Cannot talk to daemon 2024-09-14T23:53:11.374 INFO:teuthology.orchestra.run.smithi042.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-14T23:53:11.383 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-14T23:53:11.398 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-14T23:53:11.429 INFO:teuthology.orchestra.run.smithi042.stderr:bash: line 1: ntpq: command not found 2024-09-14T23:53:11.432 INFO:teuthology.orchestra.run.smithi042.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-14T23:53:11.432 INFO:teuthology.orchestra.run.smithi042.stdout:=============================================================================== 2024-09-14T23:53:11.432 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-14T23:53:11.432 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-14T23:53:11.432 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-14T23:53:11.432 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-14T23:53:11.452 INFO:teuthology.orchestra.run.smithi043.stderr:bash: line 1: ntpq: command not found 2024-09-14T23:53:11.455 INFO:teuthology.orchestra.run.smithi043.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-14T23:53:11.455 INFO:teuthology.orchestra.run.smithi043.stdout:=============================================================================== 2024-09-14T23:53:11.456 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-14T23:53:11.456 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-14T23:53:11.456 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-14T23:53:11.456 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-14T23:53:11.457 INFO:teuthology.run_tasks:Running task install... 2024-09-14T23:53:11.465 DEBUG:teuthology.task.install:project ceph 2024-09-14T23:53:11.465 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50'}} 2024-09-14T23:53:11.466 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50'} 2024-09-14T23:53:11.466 INFO:teuthology.task.install:Using flavor: default 2024-09-14T23:53:11.471 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-14T23:53:11.471 INFO:teuthology.task.install:extra packages: [] 2024-09-14T23:53:11.471 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50', 'tag': None, 'wait_for_package': False} 2024-09-14T23:53:11.472 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ecab7be58963a5bd03910cd73eef66405d7fdd50 2024-09-14T23:53:11.474 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50', 'tag': None, 'wait_for_package': False} 2024-09-14T23:53:11.474 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ecab7be58963a5bd03910cd73eef66405d7fdd50 2024-09-14T23:53:11.621 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/centos/9/flavors/default/ 2024-09-14T23:53:11.621 INFO:teuthology.task.install.rpm:Package version is 19.1.1-324.gecab7be5 2024-09-14T23:53:11.625 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/centos/9/flavors/default/ 2024-09-14T23:53:11.625 INFO:teuthology.task.install.rpm:Package version is 19.1.1-324.gecab7be5 2024-09-14T23:53:11.731 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-14T23:53:11.731 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-14T23:53:11.732 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-14T23:53:11.737 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-14T23:53:11.737 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-14T23:53:11.737 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-14T23:53:11.760 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-14T23:53:11.760 DEBUG:teuthology.orchestra.run.smithi043:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ecab7be58963a5bd03910cd73eef66405d7fdd50/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-14T23:53:11.764 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-14T23:53:11.764 DEBUG:teuthology.orchestra.run.smithi042:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ecab7be58963a5bd03910cd73eef66405d7fdd50/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-14T23:53:11.828 DEBUG:teuthology.orchestra.run.smithi043:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-14T23:53:11.832 DEBUG:teuthology.orchestra.run.smithi042:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-14T23:53:11.906 DEBUG:teuthology.orchestra.run.smithi043:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-14T23:53:11.910 DEBUG:teuthology.orchestra.run.smithi042:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-14T23:53:11.971 INFO:teuthology.orchestra.run.smithi043.stdout:check_obsoletes = 1 2024-09-14T23:53:11.973 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum clean all 2024-09-14T23:53:11.975 INFO:teuthology.orchestra.run.smithi042.stdout:check_obsoletes = 1 2024-09-14T23:53:11.977 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum clean all 2024-09-14T23:53:12.222 INFO:teuthology.orchestra.run.smithi042.stdout:45 files removed 2024-09-14T23:53:12.223 INFO:teuthology.orchestra.run.smithi043.stdout:45 files removed 2024-09-14T23:53:12.249 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-radosgw 2024-09-14T23:53:12.251 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-radosgw 2024-09-14T23:53:12.821 INFO:teuthology.orchestra.run.smithi043.stdout:ceph packages for x86_64 308 kB/s | 84 kB 00:00 2024-09-14T23:53:12.829 INFO:teuthology.orchestra.run.smithi042.stdout:ceph packages for x86_64 298 kB/s | 84 kB 00:00 2024-09-14T23:53:13.221 INFO:teuthology.orchestra.run.smithi042.stdout:ceph noarch packages 34 kB/s | 12 kB 00:00 2024-09-14T23:53:13.222 INFO:teuthology.orchestra.run.smithi043.stdout:ceph noarch packages 33 kB/s | 12 kB 00:00 2024-09-14T23:53:13.614 INFO:teuthology.orchestra.run.smithi043.stdout:ceph source packages 5.3 kB/s | 1.9 kB 00:00 2024-09-14T23:53:13.622 INFO:teuthology.orchestra.run.smithi042.stdout:ceph source packages 5.2 kB/s | 1.9 kB 00:00 2024-09-14T23:53:15.380 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - BaseOS 4.8 MB/s | 8.2 MB 00:01 2024-09-14T23:53:15.705 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - BaseOS 4.0 MB/s | 8.2 MB 00:02 2024-09-14T23:53:17.448 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - AppStream 22 MB/s | 20 MB 00:00 2024-09-14T23:53:18.990 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - AppStream 9.6 MB/s | 20 MB 00:02 2024-09-14T23:53:21.924 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.5 MB 00:00 2024-09-14T23:53:23.400 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - Extras packages 90 kB/s | 19 kB 00:00 2024-09-14T23:53:23.766 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - CRB 7.9 MB/s | 6.5 MB 00:00 2024-09-14T23:53:24.537 INFO:teuthology.orchestra.run.smithi042.stdout:Extra Packages for Enterprise Linux 23 MB/s | 23 MB 00:01 2024-09-14T23:53:25.450 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - Extras packages 48 kB/s | 19 kB 00:00 2024-09-14T23:53:26.279 INFO:teuthology.orchestra.run.smithi043.stdout:Extra Packages for Enterprise Linux 32 MB/s | 23 MB 00:00 2024-09-14T23:53:30.402 INFO:teuthology.orchestra.run.smithi042.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-14T23:53:31.968 INFO:teuthology.orchestra.run.smithi043.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-14T23:53:32.873 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-14T23:53:32.886 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-14T23:53:32.886 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repository Size 2024-09-14T23:53:32.886 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-14T23:53:32.886 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-09-14T23:53:32.886 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-radosgw x86_64 2:19.1.1-324.gecab7be5.el9 ceph 11 M 2024-09-14T23:53:32.886 INFO:teuthology.orchestra.run.smithi042.stdout:Upgrading: 2024-09-14T23:53:32.887 INFO:teuthology.orchestra.run.smithi042.stdout: librados2 x86_64 2:19.1.1-324.gecab7be5.el9 ceph 3.4 M 2024-09-14T23:53:32.887 INFO:teuthology.orchestra.run.smithi042.stdout: librbd1 x86_64 2:19.1.1-324.gecab7be5.el9 ceph 3.2 M 2024-09-14T23:53:32.887 INFO:teuthology.orchestra.run.smithi042.stdout:Installing dependencies: 2024-09-14T23:53:32.887 INFO:teuthology.orchestra.run.smithi042.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-14T23:53:32.887 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-base x86_64 2:19.1.1-324.gecab7be5.el9 ceph 5.4 M 2024-09-14T23:53:32.887 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-common x86_64 2:19.1.1-324.gecab7be5.el9 ceph 21 M 2024-09-14T23:53:32.887 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-selinux x86_64 2:19.1.1-324.gecab7be5.el9 ceph 25 k 2024-09-14T23:53:32.887 INFO:teuthology.orchestra.run.smithi042.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-14T23:53:32.887 INFO:teuthology.orchestra.run.smithi042.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-14T23:53:32.887 INFO:teuthology.orchestra.run.smithi042.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-14T23:53:32.887 INFO:teuthology.orchestra.run.smithi042.stdout: libcephfs2 x86_64 2:19.1.1-324.gecab7be5.el9 ceph 752 k 2024-09-14T23:53:32.887 INFO:teuthology.orchestra.run.smithi042.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-14T23:53:32.887 INFO:teuthology.orchestra.run.smithi042.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-14T23:53:32.887 INFO:teuthology.orchestra.run.smithi042.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-14T23:53:32.888 INFO:teuthology.orchestra.run.smithi042.stdout: libradosstriper1 x86_64 2:19.1.1-324.gecab7be5.el9 ceph 507 k 2024-09-14T23:53:32.888 INFO:teuthology.orchestra.run.smithi042.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-14T23:53:32.888 INFO:teuthology.orchestra.run.smithi042.stdout: librgw2 x86_64 2:19.1.1-324.gecab7be5.el9 ceph 5.3 M 2024-09-14T23:53:32.888 INFO:teuthology.orchestra.run.smithi042.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-14T23:53:32.888 INFO:teuthology.orchestra.run.smithi042.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-14T23:53:32.888 INFO:teuthology.orchestra.run.smithi042.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-14T23:53:32.888 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-argparse x86_64 2:19.1.1-324.gecab7be5.el9 ceph 45 k 2024-09-14T23:53:32.888 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-common x86_64 2:19.1.1-324.gecab7be5.el9 ceph 136 k 2024-09-14T23:53:32.888 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cephfs x86_64 2:19.1.1-324.gecab7be5.el9 ceph 162 k 2024-09-14T23:53:32.888 INFO:teuthology.orchestra.run.smithi042.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-14T23:53:32.888 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rados x86_64 2:19.1.1-324.gecab7be5.el9 ceph 321 k 2024-09-14T23:53:32.888 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rbd x86_64 2:19.1.1-324.gecab7be5.el9 ceph 301 k 2024-09-14T23:53:32.889 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rgw x86_64 2:19.1.1-324.gecab7be5.el9 ceph 99 k 2024-09-14T23:53:32.889 INFO:teuthology.orchestra.run.smithi042.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-14T23:53:32.889 INFO:teuthology.orchestra.run.smithi042.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-14T23:53:32.889 INFO:teuthology.orchestra.run.smithi042.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-14T23:53:32.889 INFO:teuthology.orchestra.run.smithi042.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-14T23:53:32.889 INFO:teuthology.orchestra.run.smithi042.stdout:Installing weak dependencies: 2024-09-14T23:53:32.889 INFO:teuthology.orchestra.run.smithi042.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-14T23:53:32.889 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:53:32.889 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-09-14T23:53:32.889 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-14T23:53:32.889 INFO:teuthology.orchestra.run.smithi042.stdout:Install 30 Packages 2024-09-14T23:53:32.889 INFO:teuthology.orchestra.run.smithi042.stdout:Upgrade 2 Packages 2024-09-14T23:53:32.889 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:53:32.890 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 60 M 2024-09-14T23:53:32.890 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-09-14T23:53:33.928 INFO:teuthology.orchestra.run.smithi042.stdout:(1/32): ceph-base-19.1.1-324.gecab7be5.el9.x86_ 8.6 MB/s | 5.4 MB 00:00 2024-09-14T23:53:34.070 INFO:teuthology.orchestra.run.smithi042.stdout:(2/32): ceph-radosgw-19.1.1-324.gecab7be5.el9.x 14 MB/s | 11 MB 00:00 2024-09-14T23:53:34.095 INFO:teuthology.orchestra.run.smithi042.stdout:(3/32): ceph-selinux-19.1.1-324.gecab7be5.el9.x 150 kB/s | 25 kB 00:00 2024-09-14T23:53:34.129 INFO:teuthology.orchestra.run.smithi042.stdout:(4/32): libcephfs2-19.1.1-324.gecab7be5.el9.x86 13 MB/s | 752 kB 00:00 2024-09-14T23:53:34.162 INFO:teuthology.orchestra.run.smithi042.stdout:(5/32): libradosstriper1-19.1.1-324.gecab7be5.e 7.4 MB/s | 507 kB 00:00 2024-09-14T23:53:34.213 INFO:teuthology.orchestra.run.smithi042.stdout:(6/32): python3-ceph-argparse-19.1.1-324.gecab7 905 kB/s | 45 kB 00:00 2024-09-14T23:53:34.296 INFO:teuthology.orchestra.run.smithi042.stdout:(7/32): python3-ceph-common-19.1.1-324.gecab7be 1.6 MB/s | 136 kB 00:00 2024-09-14T23:53:34.388 INFO:teuthology.orchestra.run.smithi042.stdout:(8/32): librgw2-19.1.1-324.gecab7be5.el9.x86_64 21 MB/s | 5.3 MB 00:00 2024-09-14T23:53:34.413 INFO:teuthology.orchestra.run.smithi042.stdout:(9/32): python3-cephfs-19.1.1-324.gecab7be5.el9 1.4 MB/s | 162 kB 00:00 2024-09-14T23:53:34.432 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-14T23:53:34.439 INFO:teuthology.orchestra.run.smithi042.stdout:(10/32): python3-rados-19.1.1-324.gecab7be5.el9 6.3 MB/s | 321 kB 00:00 2024-09-14T23:53:34.455 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-14T23:53:34.455 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-14T23:53:34.455 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-14T23:53:34.455 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-14T23:53:34.455 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-radosgw x86_64 2:19.1.1-324.gecab7be5.el9 ceph 11 M 2024-09-14T23:53:34.456 INFO:teuthology.orchestra.run.smithi043.stdout:Upgrading: 2024-09-14T23:53:34.456 INFO:teuthology.orchestra.run.smithi043.stdout: librados2 x86_64 2:19.1.1-324.gecab7be5.el9 ceph 3.4 M 2024-09-14T23:53:34.456 INFO:teuthology.orchestra.run.smithi043.stdout: librbd1 x86_64 2:19.1.1-324.gecab7be5.el9 ceph 3.2 M 2024-09-14T23:53:34.456 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-14T23:53:34.456 INFO:teuthology.orchestra.run.smithi043.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-14T23:53:34.456 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-base x86_64 2:19.1.1-324.gecab7be5.el9 ceph 5.4 M 2024-09-14T23:53:34.456 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-common x86_64 2:19.1.1-324.gecab7be5.el9 ceph 21 M 2024-09-14T23:53:34.456 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-selinux x86_64 2:19.1.1-324.gecab7be5.el9 ceph 25 k 2024-09-14T23:53:34.456 INFO:teuthology.orchestra.run.smithi043.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-14T23:53:34.456 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-14T23:53:34.456 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-14T23:53:34.456 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs2 x86_64 2:19.1.1-324.gecab7be5.el9 ceph 752 k 2024-09-14T23:53:34.456 INFO:teuthology.orchestra.run.smithi043.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-14T23:53:34.457 INFO:teuthology.orchestra.run.smithi043.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-14T23:53:34.457 INFO:teuthology.orchestra.run.smithi043.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-14T23:53:34.457 INFO:teuthology.orchestra.run.smithi043.stdout: libradosstriper1 x86_64 2:19.1.1-324.gecab7be5.el9 ceph 507 k 2024-09-14T23:53:34.457 INFO:teuthology.orchestra.run.smithi043.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-14T23:53:34.457 INFO:teuthology.orchestra.run.smithi043.stdout: librgw2 x86_64 2:19.1.1-324.gecab7be5.el9 ceph 5.3 M 2024-09-14T23:53:34.457 INFO:teuthology.orchestra.run.smithi043.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-14T23:53:34.457 INFO:teuthology.orchestra.run.smithi043.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-14T23:53:34.457 INFO:teuthology.orchestra.run.smithi043.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-14T23:53:34.457 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-argparse x86_64 2:19.1.1-324.gecab7be5.el9 ceph 45 k 2024-09-14T23:53:34.457 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-common x86_64 2:19.1.1-324.gecab7be5.el9 ceph 136 k 2024-09-14T23:53:34.457 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cephfs x86_64 2:19.1.1-324.gecab7be5.el9 ceph 162 k 2024-09-14T23:53:34.457 INFO:teuthology.orchestra.run.smithi043.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-14T23:53:34.457 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rados x86_64 2:19.1.1-324.gecab7be5.el9 ceph 321 k 2024-09-14T23:53:34.458 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rbd x86_64 2:19.1.1-324.gecab7be5.el9 ceph 301 k 2024-09-14T23:53:34.458 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rgw x86_64 2:19.1.1-324.gecab7be5.el9 ceph 99 k 2024-09-14T23:53:34.458 INFO:teuthology.orchestra.run.smithi043.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-14T23:53:34.458 INFO:teuthology.orchestra.run.smithi043.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-14T23:53:34.458 INFO:teuthology.orchestra.run.smithi043.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-14T23:53:34.458 INFO:teuthology.orchestra.run.smithi043.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-14T23:53:34.458 INFO:teuthology.orchestra.run.smithi043.stdout:Installing weak dependencies: 2024-09-14T23:53:34.458 INFO:teuthology.orchestra.run.smithi043.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-14T23:53:34.458 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:53:34.458 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-14T23:53:34.458 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-14T23:53:34.458 INFO:teuthology.orchestra.run.smithi043.stdout:Install 30 Packages 2024-09-14T23:53:34.458 INFO:teuthology.orchestra.run.smithi043.stdout:Upgrade 2 Packages 2024-09-14T23:53:34.458 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:53:34.459 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 60 M 2024-09-14T23:53:34.459 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-14T23:53:34.472 INFO:teuthology.orchestra.run.smithi042.stdout:(11/32): python3-rbd-19.1.1-324.gecab7be5.el9.x 5.0 MB/s | 301 kB 00:00 2024-09-14T23:53:34.506 INFO:teuthology.orchestra.run.smithi042.stdout:(12/32): python3-rgw-19.1.1-324.gecab7be5.el9.x 1.4 MB/s | 99 kB 00:00 2024-09-14T23:53:34.723 INFO:teuthology.orchestra.run.smithi042.stdout:(13/32): ceph-common-19.1.1-324.gecab7be5.el9.x 15 MB/s | 21 MB 00:01 2024-09-14T23:53:34.765 INFO:teuthology.orchestra.run.smithi042.stdout:(14/32): boost-program-options-1.75.0-8.el9.x86 366 kB/s | 107 kB 00:00 2024-09-14T23:53:34.832 INFO:teuthology.orchestra.run.smithi042.stdout:(15/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 492 kB/s | 160 kB 00:00 2024-09-14T23:53:34.849 INFO:teuthology.orchestra.run.smithi042.stdout:(16/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 361 kB/s | 45 kB 00:00 2024-09-14T23:53:34.891 INFO:teuthology.orchestra.run.smithi042.stdout:(17/32): librdkafka-1.6.1-102.el9.x86_64.rpm 5.1 MB/s | 662 kB 00:00 2024-09-14T23:53:34.916 INFO:teuthology.orchestra.run.smithi042.stdout:(18/32): python3-prettytable-0.7.2-27.el9.noarc 630 kB/s | 42 kB 00:00 2024-09-14T23:53:34.941 INFO:teuthology.orchestra.run.smithi042.stdout:(19/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-09-14T23:53:34.966 INFO:teuthology.orchestra.run.smithi042.stdout:(20/32): qatlib-service-24.02.0-1.el9.x86_64.rp 715 kB/s | 36 kB 00:00 2024-09-14T23:53:34.992 INFO:teuthology.orchestra.run.smithi042.stdout:(21/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-09-14T23:53:35.025 INFO:teuthology.orchestra.run.smithi042.stdout:(22/32): qatlib-24.02.0-1.el9.x86_64.rpm 1.6 MB/s | 221 kB 00:00 2024-09-14T23:53:35.084 INFO:teuthology.orchestra.run.smithi042.stdout:(23/32): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.6 MB/s | 308 kB 00:00 2024-09-14T23:53:35.110 INFO:teuthology.orchestra.run.smithi042.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 2.1 MB/s | 49 kB 00:00 2024-09-14T23:53:35.135 INFO:teuthology.orchestra.run.smithi042.stdout:(25/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 281 kB/s | 26 kB 00:00 2024-09-14T23:53:35.160 INFO:teuthology.orchestra.run.smithi042.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-14T23:53:35.185 INFO:teuthology.orchestra.run.smithi042.stdout:(27/32): re2-20211101-20.el9.x86_64.rpm 7.6 MB/s | 191 kB 00:00 2024-09-14T23:53:35.236 INFO:teuthology.orchestra.run.smithi042.stdout:(28/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.2 MB/s | 839 kB 00:00 2024-09-14T23:53:35.286 INFO:teuthology.orchestra.run.smithi042.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-14T23:53:35.369 INFO:teuthology.orchestra.run.smithi043.stdout:(1/32): ceph-base-19.1.1-324.gecab7be5.el9.x86_ 8.6 MB/s | 5.4 MB 00:00 2024-09-14T23:53:35.370 INFO:teuthology.orchestra.run.smithi042.stdout:(30/32): libarrow-9.0.0-11.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-09-14T23:53:35.453 INFO:teuthology.orchestra.run.smithi043.stdout:(2/32): ceph-selinux-19.1.1-324.gecab7be5.el9.x 302 kB/s | 25 kB 00:00 2024-09-14T23:53:35.586 INFO:teuthology.orchestra.run.smithi043.stdout:(3/32): ceph-radosgw-19.1.1-324.gecab7be5.el9.x 13 MB/s | 11 MB 00:00 2024-09-14T23:53:35.645 INFO:teuthology.orchestra.run.smithi043.stdout:(4/32): libradosstriper1-19.1.1-324.gecab7be5.e 8.5 MB/s | 507 kB 00:00 2024-09-14T23:53:35.653 INFO:teuthology.orchestra.run.smithi042.stdout:(31/32): librbd1-19.1.1-324.gecab7be5.el9.x86_6 8.6 MB/s | 3.2 MB 00:00 2024-09-14T23:53:35.712 INFO:teuthology.orchestra.run.smithi042.stdout:(32/32): librados2-19.1.1-324.gecab7be5.el9.x86 7.2 MB/s | 3.4 MB 00:00 2024-09-14T23:53:35.716 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-09-14T23:53:35.717 INFO:teuthology.orchestra.run.smithi042.stdout:Total 21 MB/s | 60 MB 00:02 2024-09-14T23:53:35.822 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-09-14T23:53:35.862 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-09-14T23:53:35.863 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-09-14T23:53:35.937 INFO:teuthology.orchestra.run.smithi043.stdout:(5/32): ceph-common-19.1.1-324.gecab7be5.el9.x8 18 MB/s | 21 MB 00:01 2024-09-14T23:53:35.979 INFO:teuthology.orchestra.run.smithi043.stdout:(6/32): libcephfs2-19.1.1-324.gecab7be5.el9.x86 1.4 MB/s | 752 kB 00:00 2024-09-14T23:53:36.004 INFO:teuthology.orchestra.run.smithi043.stdout:(7/32): python3-ceph-argparse-19.1.1-324.gecab7 678 kB/s | 45 kB 00:00 2024-09-14T23:53:36.029 INFO:teuthology.orchestra.run.smithi043.stdout:(8/32): python3-ceph-common-19.1.1-324.gecab7be 2.7 MB/s | 136 kB 00:00 2024-09-14T23:53:36.063 INFO:teuthology.orchestra.run.smithi043.stdout:(9/32): python3-cephfs-19.1.1-324.gecab7be5.el9 2.7 MB/s | 162 kB 00:00 2024-09-14T23:53:36.088 INFO:teuthology.orchestra.run.smithi043.stdout:(10/32): python3-rados-19.1.1-324.gecab7be5.el9 5.4 MB/s | 321 kB 00:00 2024-09-14T23:53:36.113 INFO:teuthology.orchestra.run.smithi043.stdout:(11/32): python3-rbd-19.1.1-324.gecab7be5.el9.x 5.9 MB/s | 301 kB 00:00 2024-09-14T23:53:36.139 INFO:teuthology.orchestra.run.smithi043.stdout:(12/32): python3-rgw-19.1.1-324.gecab7be5.el9.x 1.9 MB/s | 99 kB 00:00 2024-09-14T23:53:36.223 INFO:teuthology.orchestra.run.smithi043.stdout:(13/32): librgw2-19.1.1-324.gecab7be5.el9.x86_6 9.3 MB/s | 5.3 MB 00:00 2024-09-14T23:53:36.314 INFO:teuthology.orchestra.run.smithi043.stdout:(14/32): boost-program-options-1.75.0-8.el9.x86 532 kB/s | 107 kB 00:00 2024-09-14T23:53:36.340 INFO:teuthology.orchestra.run.smithi043.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 387 kB/s | 45 kB 00:00 2024-09-14T23:53:36.365 INFO:teuthology.orchestra.run.smithi043.stdout:(16/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 709 kB/s | 160 kB 00:00 2024-09-14T23:53:36.407 INFO:teuthology.orchestra.run.smithi043.stdout:(17/32): librdkafka-1.6.1-102.el9.x86_64.rpm 7.0 MB/s | 662 kB 00:00 2024-09-14T23:53:36.423 INFO:teuthology.orchestra.run.smithi043.stdout:(18/32): python3-prettytable-0.7.2-27.el9.noarc 724 kB/s | 42 kB 00:00 2024-09-14T23:53:36.449 INFO:teuthology.orchestra.run.smithi043.stdout:(19/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-09-14T23:53:36.466 INFO:teuthology.orchestra.run.smithi043.stdout:(20/32): qatlib-service-24.02.0-1.el9.x86_64.rp 853 kB/s | 36 kB 00:00 2024-09-14T23:53:36.491 INFO:teuthology.orchestra.run.smithi043.stdout:(21/32): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-09-14T23:53:36.508 INFO:teuthology.orchestra.run.smithi043.stdout:(22/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 807 kB/s | 47 kB 00:00 2024-09-14T23:53:36.567 INFO:teuthology.orchestra.run.smithi043.stdout:(23/32): gperftools-libs-2.9.1-3.el9.x86_64.rpm 3.0 MB/s | 308 kB 00:00 2024-09-14T23:53:36.584 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-09-14T23:53:36.584 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-09-14T23:53:36.592 INFO:teuthology.orchestra.run.smithi043.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 2.1 MB/s | 49 kB 00:00 2024-09-14T23:53:36.617 INFO:teuthology.orchestra.run.smithi043.stdout:(25/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 276 kB/s | 26 kB 00:00 2024-09-14T23:53:36.643 INFO:teuthology.orchestra.run.smithi043.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-14T23:53:36.668 INFO:teuthology.orchestra.run.smithi043.stdout:(27/32): re2-20211101-20.el9.x86_64.rpm 7.5 MB/s | 191 kB 00:00 2024-09-14T23:53:36.718 INFO:teuthology.orchestra.run.smithi043.stdout:(28/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.1 MB/s | 839 kB 00:00 2024-09-14T23:53:36.769 INFO:teuthology.orchestra.run.smithi043.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-14T23:53:36.853 INFO:teuthology.orchestra.run.smithi043.stdout:(30/32): libarrow-9.0.0-11.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-09-14T23:53:37.005 INFO:teuthology.orchestra.run.smithi043.stdout:(31/32): librbd1-19.1.1-324.gecab7be5.el9.x86_6 14 MB/s | 3.2 MB 00:00 2024-09-14T23:53:37.061 INFO:teuthology.orchestra.run.smithi043.stdout:(32/32): librados2-19.1.1-324.gecab7be5.el9.x86 10 MB/s | 3.4 MB 00:00 2024-09-14T23:53:37.066 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-14T23:53:37.066 INFO:teuthology.orchestra.run.smithi043.stdout:Total 23 MB/s | 60 MB 00:02 2024-09-14T23:53:37.170 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-14T23:53:37.211 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-14T23:53:37.211 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-14T23:53:37.893 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-14T23:53:37.894 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-14T23:53:38.125 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-09-14T23:53:38.209 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-09-14T23:53:38.242 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-09-14T23:53:38.267 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-09-14T23:53:38.289 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-14T23:53:38.319 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-14T23:53:38.519 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-14T23:53:38.572 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-14T23:53:38.980 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-09-14T23:53:39.035 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : librados2-2:19.1.1-324.gecab7be5.el9.x86_64 7/34 2024-09-14T23:53:39.120 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librados2-2:19.1.1-324.gecab7be5.el9.x86_64 7/34 2024-09-14T23:53:39.218 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-rados-2:19.1.1-324.gecab7be5.el9.x86_64 8/34 2024-09-14T23:53:39.263 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libcephfs2-2:19.1.1-324.gecab7be5.el9.x86_64 9/34 2024-09-14T23:53:39.308 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libcephfs2-2:19.1.1-324.gecab7be5.el9.x86_64 9/34 2024-09-14T23:53:39.337 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-09-14T23:53:39.390 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-ceph-argparse-2:19.1.1-324.gecab7be5.el9.x 11/34 2024-09-14T23:53:39.459 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cephfs-2:19.1.1-324.gecab7be5.el9.x86_64 12/34 2024-09-14T23:53:39.481 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libradosstriper1-2:19.1.1-324.gecab7be5.el9.x86_64 13/34 2024-09-14T23:53:39.545 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libradosstriper1-2:19.1.1-324.gecab7be5.el9.x86_64 13/34 2024-09-14T23:53:39.579 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-09-14T23:53:39.593 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-14T23:53:39.685 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-09-14T23:53:39.690 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/34 2024-09-14T23:53:39.717 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-09-14T23:53:39.722 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-09-14T23:53:39.742 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-09-14T23:53:39.764 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-14T23:53:39.776 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-09-14T23:53:39.792 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-14T23:53:39.812 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/34 2024-09-14T23:53:39.842 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-09-14T23:53:39.876 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-09-14T23:53:39.976 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-14T23:53:40.029 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-14T23:53:40.205 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-09-14T23:53:40.245 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : librbd1-2:19.1.1-324.gecab7be5.el9.x86_64 22/34 2024-09-14T23:53:40.345 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librbd1-2:19.1.1-324.gecab7be5.el9.x86_64 22/34 2024-09-14T23:53:40.375 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-rbd-2:19.1.1-324.gecab7be5.el9.x86_64 23/34 2024-09-14T23:53:40.378 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-09-14T23:53:40.428 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-09-14T23:53:40.484 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : librados2-2:19.1.1-324.gecab7be5.el9.x86_64 7/34 2024-09-14T23:53:40.586 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librados2-2:19.1.1-324.gecab7be5.el9.x86_64 7/34 2024-09-14T23:53:40.685 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rados-2:19.1.1-324.gecab7be5.el9.x86_64 8/34 2024-09-14T23:53:40.720 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libcephfs2-2:19.1.1-324.gecab7be5.el9.x86_64 9/34 2024-09-14T23:53:40.783 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libcephfs2-2:19.1.1-324.gecab7be5.el9.x86_64 9/34 2024-09-14T23:53:40.828 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-09-14T23:53:40.873 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-ceph-argparse-2:19.1.1-324.gecab7be5.el9.x 11/34 2024-09-14T23:53:40.938 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-09-14T23:53:40.942 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cephfs-2:19.1.1-324.gecab7be5.el9.x86_64 12/34 2024-09-14T23:53:40.971 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libradosstriper1-2:19.1.1-324.gecab7be5.el9.x86_64 13/34 2024-09-14T23:53:40.972 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : librgw2-2:19.1.1-324.gecab7be5.el9.x86_64 26/34 2024-09-14T23:53:41.036 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libradosstriper1-2:19.1.1-324.gecab7be5.el9.x86_64 13/34 2024-09-14T23:53:41.044 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librgw2-2:19.1.1-324.gecab7be5.el9.x86_64 26/34 2024-09-14T23:53:41.070 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-09-14T23:53:41.125 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-rgw-2:19.1.1-324.gecab7be5.el9.x86_64 27/34 2024-09-14T23:53:41.167 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-ceph-common-2:19.1.1-324.gecab7be5.el9.x86 28/34 2024-09-14T23:53:41.180 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/34 2024-09-14T23:53:41.212 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-09-14T23:53:41.270 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-09-14T23:53:41.303 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/34 2024-09-14T23:53:41.332 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-09-14T23:53:41.365 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-09-14T23:53:41.700 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-09-14T23:53:41.752 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : librbd1-2:19.1.1-324.gecab7be5.el9.x86_64 22/34 2024-09-14T23:53:41.836 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librbd1-2:19.1.1-324.gecab7be5.el9.x86_64 22/34 2024-09-14T23:53:41.865 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rbd-2:19.1.1-324.gecab7be5.el9.x86_64 23/34 2024-09-14T23:53:41.902 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-09-14T23:53:42.409 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-09-14T23:53:42.438 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librgw2-2:19.1.1-324.gecab7be5.el9.x86_64 26/34 2024-09-14T23:53:42.509 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librgw2-2:19.1.1-324.gecab7be5.el9.x86_64 26/34 2024-09-14T23:53:42.589 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rgw-2:19.1.1-324.gecab7be5.el9.x86_64 27/34 2024-09-14T23:53:42.624 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-ceph-common-2:19.1.1-324.gecab7be5.el9.x86 28/34 2024-09-14T23:53:43.602 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-common-2:19.1.1-324.gecab7be5.el9.x86_64 29/34 2024-09-14T23:53:43.663 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-common-2:19.1.1-324.gecab7be5.el9.x86_64 29/34 2024-09-14T23:53:44.257 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-common-2:19.1.1-324.gecab7be5.el9.x86_64 29/34 2024-09-14T23:53:44.300 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-base-2:19.1.1-324.gecab7be5.el9.x86_64 30/34 2024-09-14T23:53:44.391 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-base-2:19.1.1-324.gecab7be5.el9.x86_64 30/34 2024-09-14T23:53:44.391 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-14T23:53:44.392 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-14T23:53:44.392 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:53:44.435 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-selinux-2:19.1.1-324.gecab7be5.el9.x86_64 31/34 2024-09-14T23:53:45.056 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-common-2:19.1.1-324.gecab7be5.el9.x86_64 29/34 2024-09-14T23:53:45.103 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-common-2:19.1.1-324.gecab7be5.el9.x86_64 29/34 2024-09-14T23:53:45.692 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-common-2:19.1.1-324.gecab7be5.el9.x86_64 29/34 2024-09-14T23:53:45.732 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-base-2:19.1.1-324.gecab7be5.el9.x86_64 30/34 2024-09-14T23:53:45.808 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-base-2:19.1.1-324.gecab7be5.el9.x86_64 30/34 2024-09-14T23:53:45.808 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-14T23:53:45.808 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-14T23:53:45.808 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:53:45.841 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-selinux-2:19.1.1-324.gecab7be5.el9.x86_64 31/34 2024-09-14T23:53:55.406 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-selinux-2:19.1.1-324.gecab7be5.el9.x86_64 31/34 2024-09-14T23:53:55.406 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /sys 2024-09-14T23:53:55.406 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /proc 2024-09-14T23:53:55.406 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /mnt 2024-09-14T23:53:55.406 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /var/tmp 2024-09-14T23:53:55.406 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /home 2024-09-14T23:53:55.406 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /root 2024-09-14T23:53:55.407 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /tmp 2024-09-14T23:53:55.407 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:53:56.408 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-radosgw-2:19.1.1-324.gecab7be5.el9.x86_64 32/34 2024-09-14T23:53:56.435 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-324.gecab7be5.el9.x86_64 32/34 2024-09-14T23:53:56.435 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T23:53:56.435 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-14T23:53:56.435 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-14T23:53:56.435 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-14T23:53:56.435 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:53:56.437 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-14T23:53:56.480 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-selinux-2:19.1.1-324.gecab7be5.el9.x86_64 31/34 2024-09-14T23:53:56.480 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /sys 2024-09-14T23:53:56.480 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /proc 2024-09-14T23:53:56.480 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /mnt 2024-09-14T23:53:56.480 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /var/tmp 2024-09-14T23:53:56.480 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /home 2024-09-14T23:53:56.480 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /root 2024-09-14T23:53:56.481 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /tmp 2024-09-14T23:53:56.481 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:53:56.544 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-14T23:53:56.544 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-14T23:53:57.491 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-radosgw-2:19.1.1-324.gecab7be5.el9.x86_64 32/34 2024-09-14T23:53:57.519 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-324.gecab7be5.el9.x86_64 32/34 2024-09-14T23:53:57.519 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T23:53:57.519 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-14T23:53:57.519 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-14T23:53:57.519 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-14T23:53:57.519 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:53:57.522 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-14T23:53:57.610 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-14T23:53:57.610 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-14T23:53:57.909 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-14T23:53:57.909 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-base-2:19.1.1-324.gecab7be5.el9.x86_64 1/34 2024-09-14T23:53:57.909 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-common-2:19.1.1-324.gecab7be5.el9.x86_64 2/34 2024-09-14T23:53:57.910 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-radosgw-2:19.1.1-324.gecab7be5.el9.x86_64 3/34 2024-09-14T23:53:57.910 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-selinux-2:19.1.1-324.gecab7be5.el9.x86_64 4/34 2024-09-14T23:53:57.910 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libcephfs2-2:19.1.1-324.gecab7be5.el9.x86_64 5/34 2024-09-14T23:53:57.910 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libradosstriper1-2:19.1.1-324.gecab7be5.el9.x86_64 6/34 2024-09-14T23:53:57.910 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librgw2-2:19.1.1-324.gecab7be5.el9.x86_64 7/34 2024-09-14T23:53:57.910 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-ceph-argparse-2:19.1.1-324.gecab7be5.el9.x 8/34 2024-09-14T23:53:57.910 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-ceph-common-2:19.1.1-324.gecab7be5.el9.x86 9/34 2024-09-14T23:53:57.910 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cephfs-2:19.1.1-324.gecab7be5.el9.x86_64 10/34 2024-09-14T23:53:57.910 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-rados-2:19.1.1-324.gecab7be5.el9.x86_64 11/34 2024-09-14T23:53:57.910 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-rbd-2:19.1.1-324.gecab7be5.el9.x86_64 12/34 2024-09-14T23:53:57.910 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-rgw-2:19.1.1-324.gecab7be5.el9.x86_64 13/34 2024-09-14T23:53:57.910 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-09-14T23:53:57.910 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-09-14T23:53:57.911 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-09-14T23:53:57.911 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-09-14T23:53:57.911 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-09-14T23:53:57.911 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-09-14T23:53:57.911 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-09-14T23:53:57.911 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-09-14T23:53:57.911 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-09-14T23:53:57.911 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 23/34 2024-09-14T23:53:57.911 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-09-14T23:53:57.911 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-09-14T23:53:57.911 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-09-14T23:53:57.911 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-09-14T23:53:57.911 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-09-14T23:53:57.912 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : re2-1:20211101-20.el9.x86_64 29/34 2024-09-14T23:53:57.912 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-09-14T23:53:57.912 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librados2-2:19.1.1-324.gecab7be5.el9.x86_64 31/34 2024-09-14T23:53:57.912 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-09-14T23:53:57.912 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librbd1-2:19.1.1-324.gecab7be5.el9.x86_64 33/34 2024-09-14T23:53:58.896 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-09-14T23:53:58.896 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:53:58.896 INFO:teuthology.orchestra.run.smithi042.stdout:Upgraded: 2024-09-14T23:53:58.896 INFO:teuthology.orchestra.run.smithi042.stdout: librados2-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T23:53:58.896 INFO:teuthology.orchestra.run.smithi042.stdout: librbd1-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T23:53:58.896 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-09-14T23:53:58.896 INFO:teuthology.orchestra.run.smithi042.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-14T23:53:58.896 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-base-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T23:53:58.897 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-common-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T23:53:58.897 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-radosgw-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T23:53:58.897 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-selinux-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T23:53:58.897 INFO:teuthology.orchestra.run.smithi042.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-14T23:53:58.897 INFO:teuthology.orchestra.run.smithi042.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-14T23:53:58.897 INFO:teuthology.orchestra.run.smithi042.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-14T23:53:58.897 INFO:teuthology.orchestra.run.smithi042.stdout: libcephfs2-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T23:53:58.897 INFO:teuthology.orchestra.run.smithi042.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-14T23:53:58.897 INFO:teuthology.orchestra.run.smithi042.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-14T23:53:58.897 INFO:teuthology.orchestra.run.smithi042.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-14T23:53:58.897 INFO:teuthology.orchestra.run.smithi042.stdout: libradosstriper1-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T23:53:58.897 INFO:teuthology.orchestra.run.smithi042.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-14T23:53:58.897 INFO:teuthology.orchestra.run.smithi042.stdout: librgw2-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T23:53:58.898 INFO:teuthology.orchestra.run.smithi042.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-14T23:53:58.898 INFO:teuthology.orchestra.run.smithi042.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-14T23:53:58.898 INFO:teuthology.orchestra.run.smithi042.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-14T23:53:58.898 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-argparse-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T23:53:58.898 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-common-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T23:53:58.898 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cephfs-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T23:53:58.898 INFO:teuthology.orchestra.run.smithi042.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-14T23:53:58.898 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rados-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T23:53:58.898 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rbd-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T23:53:58.898 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rgw-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T23:53:58.898 INFO:teuthology.orchestra.run.smithi042.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-14T23:53:58.898 INFO:teuthology.orchestra.run.smithi042.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-14T23:53:58.898 INFO:teuthology.orchestra.run.smithi042.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-14T23:53:58.899 INFO:teuthology.orchestra.run.smithi042.stdout: re2-1:20211101-20.el9.x86_64 2024-09-14T23:53:58.899 INFO:teuthology.orchestra.run.smithi042.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-14T23:53:58.899 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:53:58.899 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-14T23:53:58.932 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-14T23:53:58.932 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-base-2:19.1.1-324.gecab7be5.el9.x86_64 1/34 2024-09-14T23:53:58.932 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-common-2:19.1.1-324.gecab7be5.el9.x86_64 2/34 2024-09-14T23:53:58.932 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-radosgw-2:19.1.1-324.gecab7be5.el9.x86_64 3/34 2024-09-14T23:53:58.932 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-selinux-2:19.1.1-324.gecab7be5.el9.x86_64 4/34 2024-09-14T23:53:58.933 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libcephfs2-2:19.1.1-324.gecab7be5.el9.x86_64 5/34 2024-09-14T23:53:58.933 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libradosstriper1-2:19.1.1-324.gecab7be5.el9.x86_64 6/34 2024-09-14T23:53:58.933 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librgw2-2:19.1.1-324.gecab7be5.el9.x86_64 7/34 2024-09-14T23:53:58.933 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-ceph-argparse-2:19.1.1-324.gecab7be5.el9.x 8/34 2024-09-14T23:53:58.933 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-ceph-common-2:19.1.1-324.gecab7be5.el9.x86 9/34 2024-09-14T23:53:58.933 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cephfs-2:19.1.1-324.gecab7be5.el9.x86_64 10/34 2024-09-14T23:53:58.933 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rados-2:19.1.1-324.gecab7be5.el9.x86_64 11/34 2024-09-14T23:53:58.933 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rbd-2:19.1.1-324.gecab7be5.el9.x86_64 12/34 2024-09-14T23:53:58.933 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rgw-2:19.1.1-324.gecab7be5.el9.x86_64 13/34 2024-09-14T23:53:58.933 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-09-14T23:53:58.933 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-09-14T23:53:58.933 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-09-14T23:53:58.933 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-09-14T23:53:58.933 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-09-14T23:53:58.934 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-09-14T23:53:58.934 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-09-14T23:53:58.934 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-09-14T23:53:58.934 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-09-14T23:53:58.934 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 23/34 2024-09-14T23:53:58.934 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-09-14T23:53:58.934 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-09-14T23:53:58.934 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-09-14T23:53:58.934 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-09-14T23:53:58.935 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-09-14T23:53:58.935 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : re2-1:20211101-20.el9.x86_64 29/34 2024-09-14T23:53:58.935 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-09-14T23:53:58.935 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librados2-2:19.1.1-324.gecab7be5.el9.x86_64 31/34 2024-09-14T23:53:58.935 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-09-14T23:53:58.935 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librbd1-2:19.1.1-324.gecab7be5.el9.x86_64 33/34 2024-09-14T23:53:59.266 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-test 2024-09-14T23:53:59.860 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:00:29 ago on Sat 14 Sep 2024 11:53:30 PM UTC. 2024-09-14T23:53:59.961 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-09-14T23:53:59.961 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:53:59.961 INFO:teuthology.orchestra.run.smithi043.stdout:Upgraded: 2024-09-14T23:53:59.961 INFO:teuthology.orchestra.run.smithi043.stdout: librados2-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T23:53:59.961 INFO:teuthology.orchestra.run.smithi043.stdout: librbd1-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T23:53:59.961 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-14T23:53:59.961 INFO:teuthology.orchestra.run.smithi043.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-14T23:53:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-base-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T23:53:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-common-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T23:53:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-radosgw-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T23:53:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-selinux-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T23:53:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-14T23:53:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-14T23:53:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-14T23:53:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs2-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T23:53:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-14T23:53:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-14T23:53:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-14T23:53:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: libradosstriper1-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T23:53:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-14T23:53:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: librgw2-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T23:53:59.963 INFO:teuthology.orchestra.run.smithi043.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-14T23:53:59.963 INFO:teuthology.orchestra.run.smithi043.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-14T23:53:59.963 INFO:teuthology.orchestra.run.smithi043.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-14T23:53:59.963 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-argparse-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T23:53:59.963 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-common-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T23:53:59.963 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cephfs-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T23:53:59.963 INFO:teuthology.orchestra.run.smithi043.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-14T23:53:59.963 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rados-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T23:53:59.963 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rbd-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T23:53:59.963 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rgw-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T23:53:59.963 INFO:teuthology.orchestra.run.smithi043.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-14T23:53:59.963 INFO:teuthology.orchestra.run.smithi043.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-14T23:53:59.963 INFO:teuthology.orchestra.run.smithi043.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-14T23:53:59.963 INFO:teuthology.orchestra.run.smithi043.stdout: re2-1:20211101-20.el9.x86_64 2024-09-14T23:53:59.963 INFO:teuthology.orchestra.run.smithi043.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-14T23:53:59.964 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:53:59.964 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-14T23:54:00.001 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-14T23:54:00.001 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-14T23:54:00.001 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repository Size 2024-09-14T23:54:00.001 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-14T23:54:00.002 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-09-14T23:54:00.002 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-test x86_64 2:19.1.1-324.gecab7be5.el9 ceph 49 M 2024-09-14T23:54:00.002 INFO:teuthology.orchestra.run.smithi042.stdout:Installing dependencies: 2024-09-14T23:54:00.002 INFO:teuthology.orchestra.run.smithi042.stdout: libcephsqlite x86_64 2:19.1.1-324.gecab7be5.el9 ceph 163 k 2024-09-14T23:54:00.002 INFO:teuthology.orchestra.run.smithi042.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-14T23:54:00.002 INFO:teuthology.orchestra.run.smithi042.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-14T23:54:00.002 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:54:00.002 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-09-14T23:54:00.002 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-14T23:54:00.002 INFO:teuthology.orchestra.run.smithi042.stdout:Install 4 Packages 2024-09-14T23:54:00.002 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:54:00.003 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 49 M 2024-09-14T23:54:00.003 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 209 M 2024-09-14T23:54:00.003 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-09-14T23:54:00.176 INFO:teuthology.orchestra.run.smithi042.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 2.0 MB/s | 304 kB 00:00 2024-09-14T23:54:00.218 INFO:teuthology.orchestra.run.smithi042.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-14T23:54:00.268 INFO:teuthology.orchestra.run.smithi042.stdout:(3/4): libcephsqlite-19.1.1-324.gecab7be5.el9.x 672 kB/s | 163 kB 00:00 2024-09-14T23:54:00.309 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-test 2024-09-14T23:54:00.897 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:00:29 ago on Sat 14 Sep 2024 11:53:31 PM UTC. 2024-09-14T23:54:01.039 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-14T23:54:01.040 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-14T23:54:01.040 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-14T23:54:01.040 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-14T23:54:01.040 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-14T23:54:01.040 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-test x86_64 2:19.1.1-324.gecab7be5.el9 ceph 49 M 2024-09-14T23:54:01.040 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-14T23:54:01.040 INFO:teuthology.orchestra.run.smithi043.stdout: libcephsqlite x86_64 2:19.1.1-324.gecab7be5.el9 ceph 163 k 2024-09-14T23:54:01.041 INFO:teuthology.orchestra.run.smithi043.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-14T23:54:01.041 INFO:teuthology.orchestra.run.smithi043.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-14T23:54:01.041 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:54:01.041 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-14T23:54:01.041 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-14T23:54:01.041 INFO:teuthology.orchestra.run.smithi043.stdout:Install 4 Packages 2024-09-14T23:54:01.041 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:54:01.042 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 49 M 2024-09-14T23:54:01.042 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 209 M 2024-09-14T23:54:01.042 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-14T23:54:01.190 INFO:teuthology.orchestra.run.smithi043.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 2.4 MB/s | 304 kB 00:00 2024-09-14T23:54:01.273 INFO:teuthology.orchestra.run.smithi043.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 765 kB/s | 64 kB 00:00 2024-09-14T23:54:01.298 INFO:teuthology.orchestra.run.smithi043.stdout:(3/4): libcephsqlite-19.1.1-324.gecab7be5.el9.x 701 kB/s | 163 kB 00:00 2024-09-14T23:54:01.510 INFO:teuthology.orchestra.run.smithi042.stdout:(4/4): ceph-test-19.1.1-324.gecab7be5.el9.x86_6 33 MB/s | 49 MB 00:01 2024-09-14T23:54:01.512 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-09-14T23:54:01.513 INFO:teuthology.orchestra.run.smithi042.stdout:Total 33 MB/s | 49 MB 00:01 2024-09-14T23:54:01.541 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-09-14T23:54:01.564 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-09-14T23:54:01.564 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-09-14T23:54:01.976 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-09-14T23:54:01.976 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-09-14T23:54:02.520 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-09-14T23:54:02.585 INFO:teuthology.orchestra.run.smithi043.stdout:(4/4): ceph-test-19.1.1-324.gecab7be5.el9.x86_6 32 MB/s | 49 MB 00:01 2024-09-14T23:54:02.588 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-14T23:54:02.588 INFO:teuthology.orchestra.run.smithi043.stdout:Total 32 MB/s | 49 MB 00:01 2024-09-14T23:54:02.599 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-14T23:54:02.617 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-14T23:54:02.639 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-14T23:54:02.639 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-14T23:54:02.715 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-14T23:54:02.813 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libcephsqlite-2:19.1.1-324.gecab7be5.el9.x86_64 3/4 2024-09-14T23:54:03.061 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-14T23:54:03.062 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-14T23:54:03.608 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-14T23:54:03.684 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-14T23:54:03.742 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-14T23:54:03.767 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libcephsqlite-2:19.1.1-324.gecab7be5.el9.x86_64 3/4 2024-09-14T23:54:07.712 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libcephsqlite-2:19.1.1-324.gecab7be5.el9.x86_64 3/4 2024-09-14T23:54:07.789 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-test-2:19.1.1-324.gecab7be5.el9.x86_64 4/4 2024-09-14T23:54:08.628 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libcephsqlite-2:19.1.1-324.gecab7be5.el9.x86_64 3/4 2024-09-14T23:54:08.702 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-test-2:19.1.1-324.gecab7be5.el9.x86_64 4/4 2024-09-14T23:54:10.155 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-test-2:19.1.1-324.gecab7be5.el9.x86_64 4/4 2024-09-14T23:54:10.155 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-test-2:19.1.1-324.gecab7be5.el9.x86_64 1/4 2024-09-14T23:54:10.155 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libcephsqlite-2:19.1.1-324.gecab7be5.el9.x86_64 2/4 2024-09-14T23:54:10.155 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-14T23:54:10.498 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-14T23:54:10.498 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:54:10.498 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-09-14T23:54:10.498 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-test-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T23:54:10.499 INFO:teuthology.orchestra.run.smithi042.stdout: libcephsqlite-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T23:54:10.499 INFO:teuthology.orchestra.run.smithi042.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-14T23:54:10.499 INFO:teuthology.orchestra.run.smithi042.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-14T23:54:10.499 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:54:10.499 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-14T23:54:10.777 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph 2024-09-14T23:54:11.180 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-test-2:19.1.1-324.gecab7be5.el9.x86_64 4/4 2024-09-14T23:54:11.181 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-test-2:19.1.1-324.gecab7be5.el9.x86_64 1/4 2024-09-14T23:54:11.181 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libcephsqlite-2:19.1.1-324.gecab7be5.el9.x86_64 2/4 2024-09-14T23:54:11.181 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-14T23:54:11.362 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:00:41 ago on Sat 14 Sep 2024 11:53:30 PM UTC. 2024-09-14T23:54:11.504 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-14T23:54:11.506 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-14T23:54:11.506 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:54:11.506 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-14T23:54:11.506 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-test-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T23:54:11.506 INFO:teuthology.orchestra.run.smithi043.stdout: libcephsqlite-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T23:54:11.506 INFO:teuthology.orchestra.run.smithi043.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-14T23:54:11.506 INFO:teuthology.orchestra.run.smithi043.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-14T23:54:11.506 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:54:11.507 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-14T23:54:11.507 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-14T23:54:11.507 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repository Size 2024-09-14T23:54:11.507 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-14T23:54:11.507 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-09-14T23:54:11.507 INFO:teuthology.orchestra.run.smithi042.stdout: ceph x86_64 2:19.1.1-324.gecab7be5.el9 ceph 6.5 k 2024-09-14T23:54:11.507 INFO:teuthology.orchestra.run.smithi042.stdout:Installing dependencies: 2024-09-14T23:54:11.507 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mds x86_64 2:19.1.1-324.gecab7be5.el9 ceph 2.4 M 2024-09-14T23:54:11.507 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr x86_64 2:19.1.1-324.gecab7be5.el9 ceph 1.5 M 2024-09-14T23:54:11.507 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-modules-core noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 249 k 2024-09-14T23:54:11.508 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mon x86_64 2:19.1.1-324.gecab7be5.el9 ceph 4.7 M 2024-09-14T23:54:11.508 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-osd x86_64 2:19.1.1-324.gecab7be5.el9 ceph 17 M 2024-09-14T23:54:11.508 INFO:teuthology.orchestra.run.smithi042.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-14T23:54:11.508 INFO:teuthology.orchestra.run.smithi042.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-14T23:54:11.508 INFO:teuthology.orchestra.run.smithi042.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-14T23:54:11.508 INFO:teuthology.orchestra.run.smithi042.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-14T23:54:11.508 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-14T23:54:11.508 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-14T23:54:11.508 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-14T23:54:11.508 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-14T23:54:11.508 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-14T23:54:11.508 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-14T23:54:11.509 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-14T23:54:11.509 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-14T23:54:11.509 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-14T23:54:11.509 INFO:teuthology.orchestra.run.smithi042.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-14T23:54:11.509 INFO:teuthology.orchestra.run.smithi042.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-14T23:54:11.509 INFO:teuthology.orchestra.run.smithi042.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-14T23:54:11.509 INFO:teuthology.orchestra.run.smithi042.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-14T23:54:11.509 INFO:teuthology.orchestra.run.smithi042.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-14T23:54:11.509 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-14T23:54:11.509 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-14T23:54:11.510 INFO:teuthology.orchestra.run.smithi042.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-14T23:54:11.510 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-14T23:54:11.510 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-14T23:54:11.510 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-14T23:54:11.510 INFO:teuthology.orchestra.run.smithi042.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-14T23:54:11.510 INFO:teuthology.orchestra.run.smithi042.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-14T23:54:11.510 INFO:teuthology.orchestra.run.smithi042.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-14T23:54:11.510 INFO:teuthology.orchestra.run.smithi042.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-14T23:54:11.510 INFO:teuthology.orchestra.run.smithi042.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-14T23:54:11.510 INFO:teuthology.orchestra.run.smithi042.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-14T23:54:11.510 INFO:teuthology.orchestra.run.smithi042.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-14T23:54:11.510 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:54:11.510 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-09-14T23:54:11.511 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-14T23:54:11.511 INFO:teuthology.orchestra.run.smithi042.stdout:Install 37 Packages 2024-09-14T23:54:11.511 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:54:11.511 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 30 M 2024-09-14T23:54:11.511 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 108 M 2024-09-14T23:54:11.511 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-09-14T23:54:11.847 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph 2024-09-14T23:54:12.357 INFO:teuthology.orchestra.run.smithi042.stdout:(1/37): ceph-19.1.1-324.gecab7be5.el9.x86_64.rp 54 kB/s | 6.5 kB 00:00 2024-09-14T23:54:12.435 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:00:41 ago on Sat 14 Sep 2024 11:53:31 PM UTC. 2024-09-14T23:54:12.566 INFO:teuthology.orchestra.run.smithi042.stdout:(2/37): ceph-mgr-19.1.1-324.gecab7be5.el9.x86_6 4.6 MB/s | 1.5 MB 00:00 2024-09-14T23:54:12.578 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-14T23:54:12.581 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-14T23:54:12.581 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-14T23:54:12.581 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-14T23:54:12.581 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-14T23:54:12.581 INFO:teuthology.orchestra.run.smithi043.stdout: ceph x86_64 2:19.1.1-324.gecab7be5.el9 ceph 6.5 k 2024-09-14T23:54:12.581 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-14T23:54:12.581 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mds x86_64 2:19.1.1-324.gecab7be5.el9 ceph 2.4 M 2024-09-14T23:54:12.581 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr x86_64 2:19.1.1-324.gecab7be5.el9 ceph 1.5 M 2024-09-14T23:54:12.581 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-modules-core noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 249 k 2024-09-14T23:54:12.581 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mon x86_64 2:19.1.1-324.gecab7be5.el9 ceph 4.7 M 2024-09-14T23:54:12.582 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-osd x86_64 2:19.1.1-324.gecab7be5.el9 ceph 17 M 2024-09-14T23:54:12.582 INFO:teuthology.orchestra.run.smithi043.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-14T23:54:12.582 INFO:teuthology.orchestra.run.smithi043.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-14T23:54:12.582 INFO:teuthology.orchestra.run.smithi043.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-14T23:54:12.582 INFO:teuthology.orchestra.run.smithi043.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-14T23:54:12.582 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-14T23:54:12.582 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-14T23:54:12.582 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-14T23:54:12.582 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-14T23:54:12.582 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-14T23:54:12.582 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-14T23:54:12.582 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-14T23:54:12.583 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-14T23:54:12.583 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-14T23:54:12.583 INFO:teuthology.orchestra.run.smithi043.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-14T23:54:12.583 INFO:teuthology.orchestra.run.smithi043.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-14T23:54:12.583 INFO:teuthology.orchestra.run.smithi043.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-14T23:54:12.583 INFO:teuthology.orchestra.run.smithi043.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-14T23:54:12.583 INFO:teuthology.orchestra.run.smithi043.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-14T23:54:12.583 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-14T23:54:12.583 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-14T23:54:12.583 INFO:teuthology.orchestra.run.smithi043.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-14T23:54:12.583 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-14T23:54:12.583 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-14T23:54:12.583 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-14T23:54:12.584 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-14T23:54:12.584 INFO:teuthology.orchestra.run.smithi043.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-14T23:54:12.584 INFO:teuthology.orchestra.run.smithi043.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-14T23:54:12.584 INFO:teuthology.orchestra.run.smithi043.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-14T23:54:12.584 INFO:teuthology.orchestra.run.smithi043.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-14T23:54:12.584 INFO:teuthology.orchestra.run.smithi043.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-14T23:54:12.584 INFO:teuthology.orchestra.run.smithi043.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-14T23:54:12.584 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:54:12.584 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-14T23:54:12.584 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-14T23:54:12.584 INFO:teuthology.orchestra.run.smithi043.stdout:Install 37 Packages 2024-09-14T23:54:12.584 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:54:12.585 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 30 M 2024-09-14T23:54:12.585 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 108 M 2024-09-14T23:54:12.585 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-14T23:54:12.616 INFO:teuthology.orchestra.run.smithi042.stdout:(3/37): ceph-mds-19.1.1-324.gecab7be5.el9.x86_6 6.3 MB/s | 2.4 MB 00:00 2024-09-14T23:54:12.700 INFO:teuthology.orchestra.run.smithi042.stdout:(4/37): ceph-mgr-modules-core-19.1.1-324.gecab7 2.9 MB/s | 249 kB 00:00 2024-09-14T23:54:12.783 INFO:teuthology.orchestra.run.smithi042.stdout:(5/37): ceph-mon-19.1.1-324.gecab7be5.el9.x86_6 11 MB/s | 4.7 MB 00:00 2024-09-14T23:54:12.859 INFO:teuthology.orchestra.run.smithi042.stdout:(6/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.6 MB/s | 253 kB 00:00 2024-09-14T23:54:12.900 INFO:teuthology.orchestra.run.smithi042.stdout:(7/37): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-14T23:54:12.942 INFO:teuthology.orchestra.run.smithi042.stdout:(8/37): python3-pycparser-2.20-6.el9.noarch.rpm 3.2 MB/s | 135 kB 00:00 2024-09-14T23:54:12.949 INFO:teuthology.orchestra.run.smithi043.stdout:(1/37): ceph-19.1.1-324.gecab7be5.el9.x86_64.rp 44 kB/s | 6.5 kB 00:00 2024-09-14T23:54:12.984 INFO:teuthology.orchestra.run.smithi042.stdout:(9/37): python3-cryptography-36.0.1-4.el9.x86_6 6.1 MB/s | 1.2 MB 00:00 2024-09-14T23:54:13.010 INFO:teuthology.orchestra.run.smithi042.stdout:(10/37): python3-requests-2.25.1-8.el9.noarch.r 1.8 MB/s | 125 kB 00:00 2024-09-14T23:54:13.035 INFO:teuthology.orchestra.run.smithi042.stdout:(11/37): python3-urllib3-1.26.5-6.el9.noarch.rp 4.2 MB/s | 215 kB 00:00 2024-09-14T23:54:13.108 INFO:teuthology.orchestra.run.smithi043.stdout:(2/37): ceph-mgr-19.1.1-324.gecab7be5.el9.x86_6 5.0 MB/s | 1.5 MB 00:00 2024-09-14T23:54:13.176 INFO:teuthology.orchestra.run.smithi043.stdout:(3/37): ceph-mds-19.1.1-324.gecab7be5.el9.x86_6 6.4 MB/s | 2.4 MB 00:00 2024-09-14T23:54:13.219 INFO:teuthology.orchestra.run.smithi042.stdout:(12/37): ceph-osd-19.1.1-324.gecab7be5.el9.x86_ 26 MB/s | 17 MB 00:00 2024-09-14T23:54:13.259 INFO:teuthology.orchestra.run.smithi043.stdout:(4/37): ceph-mgr-modules-core-19.1.1-324.gecab7 2.9 MB/s | 249 kB 00:00 2024-09-14T23:54:13.302 INFO:teuthology.orchestra.run.smithi042.stdout:(13/37): lua-5.4.4-4.el9.x86_64.rpm 645 kB/s | 188 kB 00:00 2024-09-14T23:54:13.327 INFO:teuthology.orchestra.run.smithi042.stdout:(14/37): python3-markupsafe-1.1.1-12.el9.x86_64 321 kB/s | 35 kB 00:00 2024-09-14T23:54:13.353 INFO:teuthology.orchestra.run.smithi042.stdout:(15/37): python3-mako-1.1.4-6.el9.noarch.rpm 542 kB/s | 172 kB 00:00 2024-09-14T23:54:13.378 INFO:teuthology.orchestra.run.smithi042.stdout:(16/37): python3-packaging-20.9-5.el9.noarch.rp 1.0 MB/s | 77 kB 00:00 2024-09-14T23:54:13.403 INFO:teuthology.orchestra.run.smithi042.stdout:(17/37): python3-pytz-2021.1-5.el9.noarch.rpm 676 kB/s | 51 kB 00:00 2024-09-14T23:54:13.428 INFO:teuthology.orchestra.run.smithi042.stdout:(18/37): python3-toml-0.10.2-6.el9.noarch.rpm 832 kB/s | 42 kB 00:00 2024-09-14T23:54:13.479 INFO:teuthology.orchestra.run.smithi042.stdout:(19/37): luarocks-3.9.2-5.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-09-14T23:54:13.504 INFO:teuthology.orchestra.run.smithi042.stdout:(20/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 578 kB/s | 43 kB 00:00 2024-09-14T23:54:13.529 INFO:teuthology.orchestra.run.smithi042.stdout:(21/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 3.5 MB/s | 172 kB 00:00 2024-09-14T23:54:13.554 INFO:teuthology.orchestra.run.smithi042.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 429 kB/s | 11 kB 00:00 2024-09-14T23:54:13.580 INFO:teuthology.orchestra.run.smithi042.stdout:(23/37): python3-jaraco-classes-3.2.1-5.el9.noa 706 kB/s | 18 kB 00:00 2024-09-14T23:54:13.596 INFO:teuthology.orchestra.run.smithi042.stdout:(24/37): python3-jaraco-collections-3.0.0-8.el9 1.4 MB/s | 23 kB 00:00 2024-09-14T23:54:13.614 INFO:teuthology.orchestra.run.smithi042.stdout:(25/37): python3-jaraco-functools-3.5.0-2.el9.n 1.1 MB/s | 19 kB 00:00 2024-09-14T23:54:13.639 INFO:teuthology.orchestra.run.smithi042.stdout:(26/37): python3-cherrypy-18.6.1-2.el9.noarch.r 2.6 MB/s | 358 kB 00:00 2024-09-14T23:54:13.656 INFO:teuthology.orchestra.run.smithi042.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 471 kB/s | 20 kB 00:00 2024-09-14T23:54:13.673 INFO:teuthology.orchestra.run.smithi042.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 1.4 MB/s | 46 kB 00:00 2024-09-14T23:54:13.694 INFO:teuthology.orchestra.run.smithi043.stdout:(5/37): ceph-osd-19.1.1-324.gecab7be5.el9.x86_6 29 MB/s | 17 MB 00:00 2024-09-14T23:54:13.698 INFO:teuthology.orchestra.run.smithi042.stdout:(29/37): python3-more-itertools-8.12.0-2.el9.no 1.8 MB/s | 79 kB 00:00 2024-09-14T23:54:13.732 INFO:teuthology.orchestra.run.smithi042.stdout:(30/37): lua-devel-5.4.4-4.el9.x86_64.rpm 63 kB/s | 22 kB 00:00 2024-09-14T23:54:13.757 INFO:teuthology.orchestra.run.smithi042.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 279 kB/s | 16 kB 00:00 2024-09-14T23:54:13.768 INFO:teuthology.orchestra.run.smithi043.stdout:(6/37): ceph-mon-19.1.1-324.gecab7be5.el9.x86_6 5.8 MB/s | 4.7 MB 00:00 2024-09-14T23:54:13.791 INFO:teuthology.orchestra.run.smithi042.stdout:(32/37): python3-tempora-5.0.0-2.el9.noarch.rpm 1.4 MB/s | 36 kB 00:00 2024-09-14T23:54:13.816 INFO:teuthology.orchestra.run.smithi042.stdout:(33/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.9 MB/s | 272 kB 00:00 2024-09-14T23:54:13.841 INFO:teuthology.orchestra.run.smithi042.stdout:(34/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 825 kB/s | 90 kB 00:00 2024-09-14T23:54:13.866 INFO:teuthology.orchestra.run.smithi042.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 802 kB/s | 20 kB 00:00 2024-09-14T23:54:13.892 INFO:teuthology.orchestra.run.smithi042.stdout:(36/37): python3-webob-1.8.8-2.el9.noarch.rpm 2.3 MB/s | 230 kB 00:00 2024-09-14T23:54:13.917 INFO:teuthology.orchestra.run.smithi042.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.2 MB/s | 427 kB 00:00 2024-09-14T23:54:13.922 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-09-14T23:54:13.922 INFO:teuthology.orchestra.run.smithi042.stdout:Total 13 MB/s | 30 MB 00:02 2024-09-14T23:54:14.082 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-09-14T23:54:14.126 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-09-14T23:54:14.127 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-09-14T23:54:14.168 INFO:teuthology.orchestra.run.smithi043.stdout:(7/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 279 kB/s | 253 kB 00:00 2024-09-14T23:54:14.502 INFO:teuthology.orchestra.run.smithi043.stdout:(8/37): python3-ply-3.11-14.el9.noarch.rpm 145 kB/s | 106 kB 00:00 2024-09-14T23:54:14.623 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-09-14T23:54:14.623 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-09-14T23:54:14.644 INFO:teuthology.orchestra.run.smithi043.stdout:(9/37): python3-cryptography-36.0.1-4.el9.x86_6 1.3 MB/s | 1.2 MB 00:00 2024-09-14T23:54:14.864 INFO:teuthology.orchestra.run.smithi043.stdout:(10/37): python3-pycparser-2.20-6.el9.noarch.rp 194 kB/s | 135 kB 00:00 2024-09-14T23:54:15.031 INFO:teuthology.orchestra.run.smithi043.stdout:(11/37): lua-5.4.4-4.el9.x86_64.rpm 1.1 MB/s | 188 kB 00:00 2024-09-14T23:54:15.115 INFO:teuthology.orchestra.run.smithi043.stdout:(12/37): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-09-14T23:54:15.165 INFO:teuthology.orchestra.run.smithi043.stdout:(13/37): python3-markupsafe-1.1.1-12.el9.x86_64 695 kB/s | 35 kB 00:00 2024-09-14T23:54:15.190 INFO:teuthology.orchestra.run.smithi043.stdout:(14/37): python3-requests-2.25.1-8.el9.noarch.r 182 kB/s | 125 kB 00:00 2024-09-14T23:54:15.224 INFO:teuthology.orchestra.run.smithi043.stdout:(15/37): python3-packaging-20.9-5.el9.noarch.rp 1.3 MB/s | 77 kB 00:00 2024-09-14T23:54:15.274 INFO:teuthology.orchestra.run.smithi043.stdout:(16/37): python3-toml-0.10.2-6.el9.noarch.rpm 833 kB/s | 42 kB 00:00 2024-09-14T23:54:15.332 INFO:teuthology.orchestra.run.smithi043.stdout:(17/37): python3-pytz-2021.1-5.el9.noarch.rpm 359 kB/s | 51 kB 00:00 2024-09-14T23:54:15.358 INFO:teuthology.orchestra.run.smithi043.stdout:(18/37): lua-devel-5.4.4-4.el9.x86_64.rpm 267 kB/s | 22 kB 00:00 2024-09-14T23:54:15.408 INFO:teuthology.orchestra.run.smithi043.stdout:(19/37): luarocks-3.9.2-5.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-09-14T23:54:15.433 INFO:teuthology.orchestra.run.smithi043.stdout:(20/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 577 kB/s | 43 kB 00:00 2024-09-14T23:54:15.458 INFO:teuthology.orchestra.run.smithi043.stdout:(21/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 3.5 MB/s | 172 kB 00:00 2024-09-14T23:54:15.492 INFO:teuthology.orchestra.run.smithi043.stdout:(22/37): python3-urllib3-1.26.5-6.el9.noarch.rp 253 kB/s | 215 kB 00:00 2024-09-14T23:54:15.518 INFO:teuthology.orchestra.run.smithi043.stdout:(23/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 186 kB/s | 11 kB 00:00 2024-09-14T23:54:15.551 INFO:teuthology.orchestra.run.smithi043.stdout:(24/37): python3-jaraco-collections-3.0.0-8.el9 927 kB/s | 23 kB 00:00 2024-09-14T23:54:15.576 INFO:teuthology.orchestra.run.smithi043.stdout:(25/37): python3-jaraco-functools-3.5.0-2.el9.n 853 kB/s | 19 kB 00:00 2024-09-14T23:54:15.601 INFO:teuthology.orchestra.run.smithi043.stdout:(26/37): python3-jaraco-classes-3.2.1-5.el9.noa 162 kB/s | 18 kB 00:00 2024-09-14T23:54:15.627 INFO:teuthology.orchestra.run.smithi043.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 392 kB/s | 20 kB 00:00 2024-09-14T23:54:15.652 INFO:teuthology.orchestra.run.smithi043.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 922 kB/s | 46 kB 00:00 2024-09-14T23:54:15.677 INFO:teuthology.orchestra.run.smithi043.stdout:(29/37): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-09-14T23:54:15.703 INFO:teuthology.orchestra.run.smithi043.stdout:(30/37): python3-cherrypy-18.6.1-2.el9.noarch.r 1.3 MB/s | 358 kB 00:00 2024-09-14T23:54:15.728 INFO:teuthology.orchestra.run.smithi043.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 327 kB/s | 16 kB 00:00 2024-09-14T23:54:15.753 INFO:teuthology.orchestra.run.smithi043.stdout:(32/37): python3-tempora-5.0.0-2.el9.noarch.rpm 1.4 MB/s | 36 kB 00:00 2024-09-14T23:54:15.778 INFO:teuthology.orchestra.run.smithi043.stdout:(33/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.2 MB/s | 90 kB 00:00 2024-09-14T23:54:15.804 INFO:teuthology.orchestra.run.smithi043.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-09-14T23:54:15.829 INFO:teuthology.orchestra.run.smithi043.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 805 kB/s | 20 kB 00:00 2024-09-14T23:54:15.854 INFO:teuthology.orchestra.run.smithi043.stdout:(36/37): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-14T23:54:15.896 INFO:teuthology.orchestra.run.smithi043.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.6 MB/s | 427 kB 00:00 2024-09-14T23:54:15.901 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-14T23:54:15.901 INFO:teuthology.orchestra.run.smithi043.stdout:Total 9.1 MB/s | 30 MB 00:03 2024-09-14T23:54:16.019 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-09-14T23:54:16.222 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-14T23:54:16.222 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-14T23:54:16.222 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-14T23:54:16.222 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-14T23:54:16.223 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-14T23:54:16.223 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-14T23:54:16.247 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-14T23:54:16.289 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-14T23:54:16.321 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-14T23:54:16.416 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-14T23:54:16.502 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-14T23:54:16.549 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-14T23:54:16.584 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-14T23:54:16.613 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-14T23:54:16.622 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-14T23:54:16.623 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-14T23:54:16.648 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-14T23:54:16.681 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-14T23:54:16.723 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-14T23:54:16.766 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-14T23:54:16.800 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-14T23:54:16.849 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-14T23:54:16.890 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-14T23:54:16.965 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-14T23:54:17.117 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-14T23:54:17.200 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-14T23:54:17.251 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-14T23:54:17.289 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-14T23:54:17.348 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-14T23:54:17.411 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-14T23:54:17.606 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-14T23:54:17.660 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-14T23:54:17.741 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-14T23:54:17.838 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-14T23:54:17.875 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-14T23:54:17.997 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-14T23:54:18.000 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-14T23:54:18.066 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-14T23:54:18.108 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-14T23:54:18.135 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-14T23:54:18.162 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-324.gecab7be5.el9.n 32/37 2024-09-14T23:54:18.194 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-2:19.1.1-324.gecab7be5.el9.x86_64 33/37 2024-09-14T23:54:18.211 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-14T23:54:18.221 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-2:19.1.1-324.gecab7be5.el9.x86_64 33/37 2024-09-14T23:54:18.221 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T23:54:18.222 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-14T23:54:18.222 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-14T23:54:18.222 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-14T23:54:18.222 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:54:18.260 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-14T23:54:18.293 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-14T23:54:18.389 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-14T23:54:18.474 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-14T23:54:18.521 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-14T23:54:18.556 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-14T23:54:18.610 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-14T23:54:18.644 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-14T23:54:18.678 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-14T23:54:18.727 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-14T23:54:18.770 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-14T23:54:18.804 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-14T23:54:18.862 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-14T23:54:18.902 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-14T23:54:18.980 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-14T23:54:19.130 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-14T23:54:19.214 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-14T23:54:19.274 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-14T23:54:19.319 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-14T23:54:19.377 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-14T23:54:19.449 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-14T23:54:19.645 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-14T23:54:19.697 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-14T23:54:19.744 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-osd-2:19.1.1-324.gecab7be5.el9.x86_64 34/37 2024-09-14T23:54:19.773 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-osd-2:19.1.1-324.gecab7be5.el9.x86_64 34/37 2024-09-14T23:54:19.773 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T23:54:19.773 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-14T23:54:19.773 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-14T23:54:19.773 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-14T23:54:19.774 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:54:19.778 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-14T23:54:19.876 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-14T23:54:19.913 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-14T23:54:20.037 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-14T23:54:20.200 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-324.gecab7be5.el9.n 32/37 2024-09-14T23:54:20.237 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mon-2:19.1.1-324.gecab7be5.el9.x86_64 35/37 2024-09-14T23:54:20.239 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-2:19.1.1-324.gecab7be5.el9.x86_64 33/37 2024-09-14T23:54:20.262 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mon-2:19.1.1-324.gecab7be5.el9.x86_64 35/37 2024-09-14T23:54:20.262 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T23:54:20.263 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-14T23:54:20.263 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-14T23:54:20.263 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-14T23:54:20.263 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:54:20.267 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-2:19.1.1-324.gecab7be5.el9.x86_64 33/37 2024-09-14T23:54:20.267 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T23:54:20.267 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-14T23:54:20.268 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-14T23:54:20.268 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-14T23:54:20.268 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:54:20.504 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mds-2:19.1.1-324.gecab7be5.el9.x86_64 36/37 2024-09-14T23:54:20.530 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mds-2:19.1.1-324.gecab7be5.el9.x86_64 36/37 2024-09-14T23:54:20.530 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T23:54:20.530 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-14T23:54:20.530 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-14T23:54:20.530 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-14T23:54:20.530 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:54:20.595 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-2:19.1.1-324.gecab7be5.el9.x86_64 37/37 2024-09-14T23:54:21.790 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-osd-2:19.1.1-324.gecab7be5.el9.x86_64 34/37 2024-09-14T23:54:21.819 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-osd-2:19.1.1-324.gecab7be5.el9.x86_64 34/37 2024-09-14T23:54:21.819 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T23:54:21.819 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-14T23:54:21.819 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-14T23:54:21.820 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-14T23:54:21.820 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:54:22.215 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-2:19.1.1-324.gecab7be5.el9.x86_64 37/37 2024-09-14T23:54:22.216 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-2:19.1.1-324.gecab7be5.el9.x86_64 1/37 2024-09-14T23:54:22.216 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mds-2:19.1.1-324.gecab7be5.el9.x86_64 2/37 2024-09-14T23:54:22.216 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-2:19.1.1-324.gecab7be5.el9.x86_64 3/37 2024-09-14T23:54:22.216 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mon-2:19.1.1-324.gecab7be5.el9.x86_64 4/37 2024-09-14T23:54:22.216 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-osd-2:19.1.1-324.gecab7be5.el9.x86_64 5/37 2024-09-14T23:54:22.216 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-324.gecab7be5.el9.n 6/37 2024-09-14T23:54:22.216 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-14T23:54:22.216 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-14T23:54:22.216 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-14T23:54:22.216 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-14T23:54:22.216 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-14T23:54:22.216 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-14T23:54:22.217 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-14T23:54:22.217 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-14T23:54:22.217 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-14T23:54:22.217 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-14T23:54:22.217 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-14T23:54:22.217 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-14T23:54:22.217 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-14T23:54:22.217 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-14T23:54:22.217 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-14T23:54:22.217 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-14T23:54:22.217 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-14T23:54:22.218 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-14T23:54:22.218 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-14T23:54:22.218 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-14T23:54:22.218 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-14T23:54:22.218 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-14T23:54:22.218 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-14T23:54:22.218 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-14T23:54:22.218 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-14T23:54:22.218 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-14T23:54:22.218 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-14T23:54:22.218 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-14T23:54:22.218 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-14T23:54:22.218 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-14T23:54:22.283 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mon-2:19.1.1-324.gecab7be5.el9.x86_64 35/37 2024-09-14T23:54:22.308 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mon-2:19.1.1-324.gecab7be5.el9.x86_64 35/37 2024-09-14T23:54:22.308 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T23:54:22.308 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-14T23:54:22.308 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-14T23:54:22.308 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-14T23:54:22.308 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:54:22.550 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mds-2:19.1.1-324.gecab7be5.el9.x86_64 36/37 2024-09-14T23:54:22.576 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mds-2:19.1.1-324.gecab7be5.el9.x86_64 36/37 2024-09-14T23:54:22.576 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T23:54:22.576 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-14T23:54:22.576 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-14T23:54:22.576 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-14T23:54:22.576 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:54:22.642 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-2:19.1.1-324.gecab7be5.el9.x86_64 37/37 2024-09-14T23:54:23.093 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-14T23:54:23.093 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:54:23.093 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-09-14T23:54:23.093 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T23:54:23.093 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mds-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T23:54:23.093 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T23:54:23.093 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-modules-core-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-14T23:54:23.094 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mon-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T23:54:23.094 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-osd-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T23:54:23.094 INFO:teuthology.orchestra.run.smithi042.stdout: lua-5.4.4-4.el9.x86_64 2024-09-14T23:54:23.094 INFO:teuthology.orchestra.run.smithi042.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-14T23:54:23.094 INFO:teuthology.orchestra.run.smithi042.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-14T23:54:23.094 INFO:teuthology.orchestra.run.smithi042.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-14T23:54:23.094 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-14T23:54:23.094 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-14T23:54:23.094 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-14T23:54:23.094 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-14T23:54:23.094 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-14T23:54:23.094 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-14T23:54:23.094 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-14T23:54:23.094 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-14T23:54:23.095 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-14T23:54:23.095 INFO:teuthology.orchestra.run.smithi042.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-14T23:54:23.095 INFO:teuthology.orchestra.run.smithi042.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-14T23:54:23.095 INFO:teuthology.orchestra.run.smithi042.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-14T23:54:23.095 INFO:teuthology.orchestra.run.smithi042.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-14T23:54:23.095 INFO:teuthology.orchestra.run.smithi042.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-14T23:54:23.095 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-14T23:54:23.095 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ply-3.11-14.el9.noarch 2024-09-14T23:54:23.095 INFO:teuthology.orchestra.run.smithi042.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-14T23:54:23.095 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-14T23:54:23.095 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-14T23:54:23.095 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-14T23:54:23.095 INFO:teuthology.orchestra.run.smithi042.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-14T23:54:23.095 INFO:teuthology.orchestra.run.smithi042.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-14T23:54:23.096 INFO:teuthology.orchestra.run.smithi042.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-14T23:54:23.096 INFO:teuthology.orchestra.run.smithi042.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-14T23:54:23.096 INFO:teuthology.orchestra.run.smithi042.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-14T23:54:23.096 INFO:teuthology.orchestra.run.smithi042.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-14T23:54:23.096 INFO:teuthology.orchestra.run.smithi042.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-14T23:54:23.096 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:54:23.096 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-14T23:54:23.461 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-base 2024-09-14T23:54:24.053 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:00:54 ago on Sat 14 Sep 2024 11:53:30 PM UTC. 2024-09-14T23:54:24.147 INFO:teuthology.orchestra.run.smithi042.stdout:Package ceph-base-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-14T23:54:24.192 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-14T23:54:24.193 INFO:teuthology.orchestra.run.smithi042.stdout:Nothing to do. 2024-09-14T23:54:24.193 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-14T23:54:24.277 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install cephadm 2024-09-14T23:54:24.587 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-2:19.1.1-324.gecab7be5.el9.x86_64 37/37 2024-09-14T23:54:24.587 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-2:19.1.1-324.gecab7be5.el9.x86_64 1/37 2024-09-14T23:54:24.587 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mds-2:19.1.1-324.gecab7be5.el9.x86_64 2/37 2024-09-14T23:54:24.588 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-2:19.1.1-324.gecab7be5.el9.x86_64 3/37 2024-09-14T23:54:24.588 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mon-2:19.1.1-324.gecab7be5.el9.x86_64 4/37 2024-09-14T23:54:24.588 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-osd-2:19.1.1-324.gecab7be5.el9.x86_64 5/37 2024-09-14T23:54:24.588 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-324.gecab7be5.el9.n 6/37 2024-09-14T23:54:24.588 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-14T23:54:24.588 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-14T23:54:24.588 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-14T23:54:24.588 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-14T23:54:24.588 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-14T23:54:24.588 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-14T23:54:24.588 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-14T23:54:24.588 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-14T23:54:24.588 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-14T23:54:24.588 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-14T23:54:24.589 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-14T23:54:24.589 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-14T23:54:24.589 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-14T23:54:24.589 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-14T23:54:24.589 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-14T23:54:24.589 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-14T23:54:24.589 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-14T23:54:24.589 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-14T23:54:24.589 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-14T23:54:24.589 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-14T23:54:24.589 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-14T23:54:24.590 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-14T23:54:24.590 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-14T23:54:24.590 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-14T23:54:24.590 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-14T23:54:24.591 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-14T23:54:24.591 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-14T23:54:24.591 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-14T23:54:24.591 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-14T23:54:24.591 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-14T23:54:24.867 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:00:54 ago on Sat 14 Sep 2024 11:53:30 PM UTC. 2024-09-14T23:54:25.005 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-14T23:54:25.006 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-14T23:54:25.006 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repository Size 2024-09-14T23:54:25.006 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-14T23:54:25.006 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-09-14T23:54:25.006 INFO:teuthology.orchestra.run.smithi042.stdout: cephadm noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 765 k 2024-09-14T23:54:25.007 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:54:25.007 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-09-14T23:54:25.007 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-14T23:54:25.007 INFO:teuthology.orchestra.run.smithi042.stdout:Install 1 Package 2024-09-14T23:54:25.007 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:54:25.007 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 765 k 2024-09-14T23:54:25.007 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 771 k 2024-09-14T23:54:25.007 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-09-14T23:54:25.289 INFO:teuthology.orchestra.run.smithi042.stdout:cephadm-19.1.1-324.gecab7be5.el9.noarch.rpm 2.7 MB/s | 765 kB 00:00 2024-09-14T23:54:25.289 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-09-14T23:54:25.290 INFO:teuthology.orchestra.run.smithi042.stdout:Total 2.6 MB/s | 765 kB 00:00 2024-09-14T23:54:25.290 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-09-14T23:54:25.296 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-09-14T23:54:25.296 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-09-14T23:54:25.302 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-09-14T23:54:25.302 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-09-14T23:54:25.377 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-09-14T23:54:25.577 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-14T23:54:25.577 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:54:25.577 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-14T23:54:25.577 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T23:54:25.577 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mds-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T23:54:25.577 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T23:54:25.577 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-modules-core-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-14T23:54:25.578 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mon-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T23:54:25.578 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-osd-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T23:54:25.578 INFO:teuthology.orchestra.run.smithi043.stdout: lua-5.4.4-4.el9.x86_64 2024-09-14T23:54:25.578 INFO:teuthology.orchestra.run.smithi043.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-14T23:54:25.578 INFO:teuthology.orchestra.run.smithi043.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-14T23:54:25.578 INFO:teuthology.orchestra.run.smithi043.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-14T23:54:25.578 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-14T23:54:25.578 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-14T23:54:25.578 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-14T23:54:25.578 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-14T23:54:25.578 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-14T23:54:25.578 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-14T23:54:25.578 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-14T23:54:25.578 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-14T23:54:25.579 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-14T23:54:25.579 INFO:teuthology.orchestra.run.smithi043.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-14T23:54:25.579 INFO:teuthology.orchestra.run.smithi043.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-14T23:54:25.579 INFO:teuthology.orchestra.run.smithi043.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-14T23:54:25.579 INFO:teuthology.orchestra.run.smithi043.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-14T23:54:25.579 INFO:teuthology.orchestra.run.smithi043.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-14T23:54:25.579 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-14T23:54:25.579 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ply-3.11-14.el9.noarch 2024-09-14T23:54:25.579 INFO:teuthology.orchestra.run.smithi043.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-14T23:54:25.579 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-14T23:54:25.579 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-14T23:54:25.579 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-14T23:54:25.579 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-14T23:54:25.580 INFO:teuthology.orchestra.run.smithi043.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-14T23:54:25.580 INFO:teuthology.orchestra.run.smithi043.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-14T23:54:25.580 INFO:teuthology.orchestra.run.smithi043.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-14T23:54:25.580 INFO:teuthology.orchestra.run.smithi043.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-14T23:54:25.580 INFO:teuthology.orchestra.run.smithi043.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-14T23:54:25.580 INFO:teuthology.orchestra.run.smithi043.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-14T23:54:25.580 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:54:25.580 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-14T23:54:25.859 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: cephadm-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-14T23:54:25.904 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-base 2024-09-14T23:54:25.954 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : cephadm-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-14T23:54:26.310 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: cephadm-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-14T23:54:26.497 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:00:55 ago on Sat 14 Sep 2024 11:53:31 PM UTC. 2024-09-14T23:54:26.556 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : cephadm-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-14T23:54:26.556 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:54:26.556 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-09-14T23:54:26.556 INFO:teuthology.orchestra.run.smithi042.stdout: cephadm-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-14T23:54:26.556 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:54:26.556 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-14T23:54:26.591 INFO:teuthology.orchestra.run.smithi043.stdout:Package ceph-base-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-14T23:54:26.635 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-14T23:54:26.636 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-09-14T23:54:26.636 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-14T23:54:26.707 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install cephadm 2024-09-14T23:54:26.761 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-immutable-object-cache 2024-09-14T23:54:27.293 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:00:56 ago on Sat 14 Sep 2024 11:53:31 PM UTC. 2024-09-14T23:54:27.349 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:00:57 ago on Sat 14 Sep 2024 11:53:30 PM UTC. 2024-09-14T23:54:27.432 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-14T23:54:27.432 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-14T23:54:27.432 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-14T23:54:27.432 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-14T23:54:27.432 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-14T23:54:27.432 INFO:teuthology.orchestra.run.smithi043.stdout: cephadm noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 765 k 2024-09-14T23:54:27.433 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:54:27.433 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-14T23:54:27.433 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-14T23:54:27.433 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-09-14T23:54:27.433 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:54:27.433 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 765 k 2024-09-14T23:54:27.433 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 771 k 2024-09-14T23:54:27.434 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-14T23:54:27.487 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-14T23:54:27.488 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-14T23:54:27.488 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repo Size 2024-09-14T23:54:27.488 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-14T23:54:27.488 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-09-14T23:54:27.488 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-324.gecab7be5.el9 ceph 145 k 2024-09-14T23:54:27.488 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:54:27.488 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-09-14T23:54:27.488 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-14T23:54:27.488 INFO:teuthology.orchestra.run.smithi042.stdout:Install 1 Package 2024-09-14T23:54:27.488 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:54:27.489 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 145 k 2024-09-14T23:54:27.489 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 435 k 2024-09-14T23:54:27.489 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-09-14T23:54:27.671 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-immutable-object-cache-19.1.1-324.gecab7be 798 kB/s | 145 kB 00:00 2024-09-14T23:54:27.671 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-09-14T23:54:27.671 INFO:teuthology.orchestra.run.smithi042.stdout:Total 792 kB/s | 145 kB 00:00 2024-09-14T23:54:27.672 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-09-14T23:54:27.680 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-09-14T23:54:27.680 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-09-14T23:54:27.692 INFO:teuthology.orchestra.run.smithi043.stdout:cephadm-19.1.1-324.gecab7be5.el9.noarch.rpm 2.9 MB/s | 765 kB 00:00 2024-09-14T23:54:27.692 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-14T23:54:27.693 INFO:teuthology.orchestra.run.smithi043.stdout:Total 2.9 MB/s | 765 kB 00:00 2024-09-14T23:54:27.693 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-14T23:54:27.699 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-14T23:54:27.699 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-14T23:54:27.705 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-14T23:54:27.705 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-14T23:54:27.740 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-09-14T23:54:27.741 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-09-14T23:54:27.782 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-14T23:54:27.891 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-09-14T23:54:27.947 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-324.gecab7be5.e 1/1 2024-09-14T23:54:27.973 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-324.gecab7be5.e 1/1 2024-09-14T23:54:27.973 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T23:54:27.973 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-14T23:54:27.973 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:54:28.274 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: cephadm-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-14T23:54:28.375 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : cephadm-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-14T23:54:28.773 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: cephadm-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-14T23:54:28.926 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-324.gecab7be5.e 1/1 2024-09-14T23:54:28.926 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:54:28.926 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-09-14T23:54:28.927 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-immutable-object-cache-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T23:54:28.927 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:54:28.927 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-14T23:54:29.030 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : cephadm-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-14T23:54:29.031 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:54:29.031 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-14T23:54:29.031 INFO:teuthology.orchestra.run.smithi043.stdout: cephadm-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-14T23:54:29.031 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:54:29.031 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-14T23:54:29.167 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-mgr 2024-09-14T23:54:29.231 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-immutable-object-cache 2024-09-14T23:54:29.754 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:00:59 ago on Sat 14 Sep 2024 11:53:30 PM UTC. 2024-09-14T23:54:29.820 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:00:58 ago on Sat 14 Sep 2024 11:53:31 PM UTC. 2024-09-14T23:54:29.848 INFO:teuthology.orchestra.run.smithi042.stdout:Package ceph-mgr-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-14T23:54:29.892 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-14T23:54:29.893 INFO:teuthology.orchestra.run.smithi042.stdout:Nothing to do. 2024-09-14T23:54:29.894 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-14T23:54:29.956 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-14T23:54:29.957 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-14T23:54:29.957 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repo Size 2024-09-14T23:54:29.957 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-14T23:54:29.957 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-14T23:54:29.957 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-324.gecab7be5.el9 ceph 145 k 2024-09-14T23:54:29.957 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:54:29.957 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-14T23:54:29.957 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-14T23:54:29.958 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-09-14T23:54:29.958 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:54:29.958 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 145 k 2024-09-14T23:54:29.958 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 435 k 2024-09-14T23:54:29.958 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-14T23:54:29.974 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-mgr-dashboard 2024-09-14T23:54:30.176 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-immutable-object-cache-19.1.1-324.gecab7be 676 kB/s | 145 kB 00:00 2024-09-14T23:54:30.176 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-14T23:54:30.177 INFO:teuthology.orchestra.run.smithi043.stdout:Total 670 kB/s | 145 kB 00:00 2024-09-14T23:54:30.177 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-14T23:54:30.183 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-14T23:54:30.183 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-14T23:54:30.244 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-14T23:54:30.244 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-14T23:54:30.396 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-14T23:54:30.491 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-324.gecab7be5.e 1/1 2024-09-14T23:54:30.517 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-324.gecab7be5.e 1/1 2024-09-14T23:54:30.517 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T23:54:30.517 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-14T23:54:30.517 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:54:30.562 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:00 ago on Sat 14 Sep 2024 11:53:30 PM UTC. 2024-09-14T23:54:30.700 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-14T23:54:30.701 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-14T23:54:30.701 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repository Size 2024-09-14T23:54:30.701 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-14T23:54:30.701 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-09-14T23:54:30.701 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-dashboard noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 3.6 M 2024-09-14T23:54:30.702 INFO:teuthology.orchestra.run.smithi042.stdout:Installing dependencies: 2024-09-14T23:54:30.702 INFO:teuthology.orchestra.run.smithi042.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-14T23:54:30.702 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-grafana-dashboards noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 32 k 2024-09-14T23:54:30.702 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-prometheus-alerts noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 16 k 2024-09-14T23:54:30.702 INFO:teuthology.orchestra.run.smithi042.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-14T23:54:30.702 INFO:teuthology.orchestra.run.smithi042.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-14T23:54:30.702 INFO:teuthology.orchestra.run.smithi042.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-14T23:54:30.702 INFO:teuthology.orchestra.run.smithi042.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-14T23:54:30.702 INFO:teuthology.orchestra.run.smithi042.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-14T23:54:30.702 INFO:teuthology.orchestra.run.smithi042.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-14T23:54:30.702 INFO:teuthology.orchestra.run.smithi042.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-14T23:54:30.702 INFO:teuthology.orchestra.run.smithi042.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-14T23:54:30.702 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:54:30.702 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-09-14T23:54:30.703 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-14T23:54:30.703 INFO:teuthology.orchestra.run.smithi042.stdout:Install 12 Packages 2024-09-14T23:54:30.703 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:54:30.703 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 8.7 M 2024-09-14T23:54:30.703 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 104 M 2024-09-14T23:54:30.703 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-09-14T23:54:31.420 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-324.gecab7be5.e 1/1 2024-09-14T23:54:31.420 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:54:31.420 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-14T23:54:31.420 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-immutable-object-cache-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T23:54:31.420 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:54:31.421 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-14T23:54:31.441 INFO:teuthology.orchestra.run.smithi042.stdout:(1/12): ceph-grafana-dashboards-19.1.1-324.geca 201 kB/s | 32 kB 00:00 2024-09-14T23:54:31.467 INFO:teuthology.orchestra.run.smithi042.stdout:(2/12): ceph-prometheus-alerts-19.1.1-324.gecab 90 kB/s | 16 kB 00:00 2024-09-14T23:54:31.583 INFO:teuthology.orchestra.run.smithi042.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 2.3 MB/s | 269 kB 00:00 2024-09-14T23:54:31.634 INFO:teuthology.orchestra.run.smithi042.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 5.3 MB/s | 1.0 MB 00:00 2024-09-14T23:54:31.687 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-mgr 2024-09-14T23:54:31.717 INFO:teuthology.orchestra.run.smithi042.stdout:(5/12): ceph-mgr-dashboard-19.1.1-324.gecab7be5 8.2 MB/s | 3.6 MB 00:00 2024-09-14T23:54:31.751 INFO:teuthology.orchestra.run.smithi042.stdout:(6/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 4.6 MB/s | 548 kB 00:00 2024-09-14T23:54:31.785 INFO:teuthology.orchestra.run.smithi042.stdout:(7/12): grpc-data-1.46.7-10.el9.noarch.rpm 290 kB/s | 19 kB 00:00 2024-09-14T23:54:31.818 INFO:teuthology.orchestra.run.smithi042.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 4.2 MB/s | 144 kB 00:00 2024-09-14T23:54:31.844 INFO:teuthology.orchestra.run.smithi042.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 1.2 MB/s | 31 kB 00:00 2024-09-14T23:54:31.877 INFO:teuthology.orchestra.run.smithi042.stdout:(10/12): protobuf-compiler-3.14.0-13.el9.x86_64 2.9 MB/s | 863 kB 00:00 2024-09-14T23:54:31.902 INFO:teuthology.orchestra.run.smithi042.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 3.2 MB/s | 188 kB 00:00 2024-09-14T23:54:31.953 INFO:teuthology.orchestra.run.smithi042.stdout:(12/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 10 MB/s | 2.0 MB 00:00 2024-09-14T23:54:31.957 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-09-14T23:54:31.957 INFO:teuthology.orchestra.run.smithi042.stdout:Total 6.9 MB/s | 8.7 MB 00:01 2024-09-14T23:54:32.007 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-09-14T23:54:32.026 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-09-14T23:54:32.026 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-09-14T23:54:32.251 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-09-14T23:54:32.251 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-09-14T23:54:32.282 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:01 ago on Sat 14 Sep 2024 11:53:31 PM UTC. 2024-09-14T23:54:32.375 INFO:teuthology.orchestra.run.smithi043.stdout:Package ceph-mgr-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-14T23:54:32.420 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-14T23:54:32.421 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-09-14T23:54:32.421 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-14T23:54:32.497 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-mgr-dashboard 2024-09-14T23:54:32.795 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-09-14T23:54:32.862 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-14T23:54:32.897 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-14T23:54:32.945 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-14T23:54:32.974 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-14T23:54:33.076 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-14T23:54:33.079 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:02 ago on Sat 14 Sep 2024 11:53:31 PM UTC. 2024-09-14T23:54:33.213 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-14T23:54:33.221 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-14T23:54:33.222 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-14T23:54:33.222 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-14T23:54:33.222 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-14T23:54:33.222 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-14T23:54:33.222 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-dashboard noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 3.6 M 2024-09-14T23:54:33.223 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-14T23:54:33.223 INFO:teuthology.orchestra.run.smithi043.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-14T23:54:33.223 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-grafana-dashboards noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 32 k 2024-09-14T23:54:33.223 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-prometheus-alerts noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 16 k 2024-09-14T23:54:33.223 INFO:teuthology.orchestra.run.smithi043.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-14T23:54:33.223 INFO:teuthology.orchestra.run.smithi043.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-14T23:54:33.223 INFO:teuthology.orchestra.run.smithi043.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-14T23:54:33.223 INFO:teuthology.orchestra.run.smithi043.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-14T23:54:33.223 INFO:teuthology.orchestra.run.smithi043.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-14T23:54:33.223 INFO:teuthology.orchestra.run.smithi043.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-14T23:54:33.223 INFO:teuthology.orchestra.run.smithi043.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-14T23:54:33.223 INFO:teuthology.orchestra.run.smithi043.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-14T23:54:33.224 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:54:33.224 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-14T23:54:33.224 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-14T23:54:33.224 INFO:teuthology.orchestra.run.smithi043.stdout:Install 12 Packages 2024-09-14T23:54:33.224 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:54:33.224 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 8.7 M 2024-09-14T23:54:33.224 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 104 M 2024-09-14T23:54:33.225 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-14T23:54:33.307 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-14T23:54:33.354 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-14T23:54:33.399 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-14T23:54:33.430 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-324.gecab7be5.el9. 10/12 2024-09-14T23:54:34.078 INFO:teuthology.orchestra.run.smithi043.stdout:(1/12): ceph-grafana-dashboards-19.1.1-324.geca 202 kB/s | 32 kB 00:00 2024-09-14T23:54:34.103 INFO:teuthology.orchestra.run.smithi043.stdout:(2/12): ceph-prometheus-alerts-19.1.1-324.gecab 90 kB/s | 16 kB 00:00 2024-09-14T23:54:34.303 INFO:teuthology.orchestra.run.smithi043.stdout:(3/12): ceph-mgr-dashboard-19.1.1-324.gecab7be5 9.3 MB/s | 3.6 MB 00:00 2024-09-14T23:54:34.329 INFO:teuthology.orchestra.run.smithi043.stdout:(4/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.2 MB/s | 269 kB 00:00 2024-09-14T23:54:34.387 INFO:teuthology.orchestra.run.smithi043.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 9.2 MB/s | 548 kB 00:00 2024-09-14T23:54:34.412 INFO:teuthology.orchestra.run.smithi043.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 787 kB/s | 19 kB 00:00 2024-09-14T23:54:34.454 INFO:teuthology.orchestra.run.smithi043.stdout:(7/12): protobuf-3.14.0-13.el9.x86_64.rpm 2.7 MB/s | 1.0 MB 00:00 2024-09-14T23:54:34.521 INFO:teuthology.orchestra.run.smithi043.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 2.1 MB/s | 144 kB 00:00 2024-09-14T23:54:34.544 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-324.gecab7be5.el9 11/12 2024-09-14T23:54:34.546 INFO:teuthology.orchestra.run.smithi043.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 1.2 MB/s | 31 kB 00:00 2024-09-14T23:54:34.580 INFO:teuthology.orchestra.run.smithi043.stdout:(10/12): protobuf-compiler-3.14.0-13.el9.x86_64 3.0 MB/s | 863 kB 00:00 2024-09-14T23:54:34.590 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-324.gecab7be5.el9.noar 12/12 2024-09-14T23:54:34.605 INFO:teuthology.orchestra.run.smithi043.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 3.1 MB/s | 188 kB 00:00 2024-09-14T23:54:34.656 INFO:teuthology.orchestra.run.smithi043.stdout:(12/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 8.4 MB/s | 2.0 MB 00:00 2024-09-14T23:54:34.660 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-14T23:54:34.660 INFO:teuthology.orchestra.run.smithi043.stdout:Total 6.0 MB/s | 8.7 MB 00:01 2024-09-14T23:54:34.712 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-14T23:54:34.732 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-14T23:54:34.732 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-14T23:54:34.958 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-14T23:54:34.958 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-14T23:54:35.527 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-14T23:54:35.599 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-14T23:54:35.642 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-14T23:54:35.698 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-14T23:54:35.743 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-14T23:54:35.857 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-14T23:54:35.922 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-324.gecab7be5.el9.noar 12/12 2024-09-14T23:54:35.922 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-324.gecab7be5.el9 1/12 2024-09-14T23:54:35.922 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-324.gecab7be5.el9.noar 2/12 2024-09-14T23:54:35.922 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-324.gecab7be5.el9. 3/12 2024-09-14T23:54:35.922 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-14T23:54:35.922 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-14T23:54:35.922 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-14T23:54:35.922 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-14T23:54:35.923 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-14T23:54:35.923 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-14T23:54:35.923 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-14T23:54:35.923 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-14T23:54:36.001 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-14T23:54:36.092 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-14T23:54:36.141 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-14T23:54:36.169 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-14T23:54:36.200 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-324.gecab7be5.el9. 10/12 2024-09-14T23:54:36.253 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-14T23:54:36.254 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:54:36.254 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-09-14T23:54:36.254 INFO:teuthology.orchestra.run.smithi042.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-14T23:54:36.254 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-grafana-dashboards-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-14T23:54:36.254 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-dashboard-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-14T23:54:36.254 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-prometheus-alerts-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-14T23:54:36.254 INFO:teuthology.orchestra.run.smithi042.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-14T23:54:36.254 INFO:teuthology.orchestra.run.smithi042.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-14T23:54:36.254 INFO:teuthology.orchestra.run.smithi042.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-14T23:54:36.254 INFO:teuthology.orchestra.run.smithi042.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-14T23:54:36.254 INFO:teuthology.orchestra.run.smithi042.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-14T23:54:36.254 INFO:teuthology.orchestra.run.smithi042.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-14T23:54:36.255 INFO:teuthology.orchestra.run.smithi042.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-14T23:54:36.255 INFO:teuthology.orchestra.run.smithi042.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-14T23:54:36.255 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:54:36.255 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-14T23:54:36.483 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-14T23:54:37.066 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:07 ago on Sat 14 Sep 2024 11:53:30 PM UTC. 2024-09-14T23:54:37.208 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-14T23:54:37.209 INFO:teuthology.orchestra.run.smithi042.stdout:====================================================================================== 2024-09-14T23:54:37.209 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repository Size 2024-09-14T23:54:37.209 INFO:teuthology.orchestra.run.smithi042.stdout:====================================================================================== 2024-09-14T23:54:37.210 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-09-14T23:54:37.210 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 7.4 M 2024-09-14T23:54:37.210 INFO:teuthology.orchestra.run.smithi042.stdout:Installing dependencies: 2024-09-14T23:54:37.210 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-14T23:54:37.210 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-14T23:54:37.210 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-14T23:54:37.210 INFO:teuthology.orchestra.run.smithi042.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-14T23:54:37.210 INFO:teuthology.orchestra.run.smithi042.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-14T23:54:37.210 INFO:teuthology.orchestra.run.smithi042.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-14T23:54:37.210 INFO:teuthology.orchestra.run.smithi042.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-14T23:54:37.210 INFO:teuthology.orchestra.run.smithi042.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-14T23:54:37.210 INFO:teuthology.orchestra.run.smithi042.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-14T23:54:37.211 INFO:teuthology.orchestra.run.smithi042.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-14T23:54:37.211 INFO:teuthology.orchestra.run.smithi042.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-14T23:54:37.211 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:54:37.211 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-09-14T23:54:37.211 INFO:teuthology.orchestra.run.smithi042.stdout:====================================================================================== 2024-09-14T23:54:37.211 INFO:teuthology.orchestra.run.smithi042.stdout:Install 12 Packages 2024-09-14T23:54:37.211 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:54:37.212 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 38 M 2024-09-14T23:54:37.212 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 212 M 2024-09-14T23:54:37.212 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-09-14T23:54:37.311 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-324.gecab7be5.el9 11/12 2024-09-14T23:54:37.351 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-324.gecab7be5.el9.noar 12/12 2024-09-14T23:54:37.510 INFO:teuthology.orchestra.run.smithi042.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.2 MB/s | 191 kB 00:00 2024-09-14T23:54:37.552 INFO:teuthology.orchestra.run.smithi042.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 4.0 MB/s | 801 kB 00:00 2024-09-14T23:54:37.578 INFO:teuthology.orchestra.run.smithi042.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 488 kB/s | 33 kB 00:00 2024-09-14T23:54:37.619 INFO:teuthology.orchestra.run.smithi042.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 438 kB/s | 18 kB 00:00 2024-09-14T23:54:37.661 INFO:teuthology.orchestra.run.smithi042.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 923 kB/s | 38 kB 00:00 2024-09-14T23:54:37.796 INFO:teuthology.orchestra.run.smithi042.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 12 MB/s | 3.0 MB 00:00 2024-09-14T23:54:37.904 INFO:teuthology.orchestra.run.smithi042.stdout:(7/12): ceph-mgr-diskprediction-local-19.1.1-32 13 MB/s | 7.4 MB 00:00 2024-09-14T23:54:37.929 INFO:teuthology.orchestra.run.smithi042.stdout:(8/12): python3-devel-3.9.19-8.el9.x86_64.rpm 1.8 MB/s | 245 kB 00:00 2024-09-14T23:54:38.013 INFO:teuthology.orchestra.run.smithi042.stdout:(9/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 14 MB/s | 4.9 MB 00:00 2024-09-14T23:54:38.038 INFO:teuthology.orchestra.run.smithi042.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 2.3 MB/s | 250 kB 00:00 2024-09-14T23:54:38.338 INFO:teuthology.orchestra.run.smithi042.stdout:(11/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 50 MB/s | 16 MB 00:00 2024-09-14T23:54:38.446 INFO:teuthology.orchestra.run.smithi042.stdout:(12/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 9.4 MB/s | 5.1 MB 00:00 2024-09-14T23:54:38.447 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-09-14T23:54:38.448 INFO:teuthology.orchestra.run.smithi042.stdout:Total 31 MB/s | 38 MB 00:01 2024-09-14T23:54:38.681 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-324.gecab7be5.el9.noar 12/12 2024-09-14T23:54:38.681 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-324.gecab7be5.el9 1/12 2024-09-14T23:54:38.681 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-324.gecab7be5.el9.noar 2/12 2024-09-14T23:54:38.681 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-324.gecab7be5.el9. 3/12 2024-09-14T23:54:38.681 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-14T23:54:38.681 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-14T23:54:38.681 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-14T23:54:38.681 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-14T23:54:38.681 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-14T23:54:38.681 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-14T23:54:38.681 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-14T23:54:38.681 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-14T23:54:38.814 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-09-14T23:54:38.840 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-09-14T23:54:38.840 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-09-14T23:54:39.036 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-14T23:54:39.037 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:54:39.037 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-14T23:54:39.037 INFO:teuthology.orchestra.run.smithi043.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-14T23:54:39.040 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-grafana-dashboards-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-14T23:54:39.040 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-dashboard-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-14T23:54:39.040 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-prometheus-alerts-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-14T23:54:39.040 INFO:teuthology.orchestra.run.smithi043.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-14T23:54:39.040 INFO:teuthology.orchestra.run.smithi043.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-14T23:54:39.040 INFO:teuthology.orchestra.run.smithi043.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-14T23:54:39.040 INFO:teuthology.orchestra.run.smithi043.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-14T23:54:39.040 INFO:teuthology.orchestra.run.smithi043.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-14T23:54:39.040 INFO:teuthology.orchestra.run.smithi043.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-14T23:54:39.040 INFO:teuthology.orchestra.run.smithi043.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-14T23:54:39.041 INFO:teuthology.orchestra.run.smithi043.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-14T23:54:39.041 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:54:39.041 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-14T23:54:39.263 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-14T23:54:39.292 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-09-14T23:54:39.292 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-09-14T23:54:39.849 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:08 ago on Sat 14 Sep 2024 11:53:31 PM UTC. 2024-09-14T23:54:39.990 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-14T23:54:39.991 INFO:teuthology.orchestra.run.smithi043.stdout:====================================================================================== 2024-09-14T23:54:39.991 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-14T23:54:39.991 INFO:teuthology.orchestra.run.smithi043.stdout:====================================================================================== 2024-09-14T23:54:39.991 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-14T23:54:39.991 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 7.4 M 2024-09-14T23:54:39.991 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-14T23:54:39.991 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-14T23:54:39.991 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-14T23:54:39.991 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-14T23:54:39.991 INFO:teuthology.orchestra.run.smithi043.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-14T23:54:39.991 INFO:teuthology.orchestra.run.smithi043.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-14T23:54:39.992 INFO:teuthology.orchestra.run.smithi043.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-14T23:54:39.992 INFO:teuthology.orchestra.run.smithi043.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-14T23:54:39.992 INFO:teuthology.orchestra.run.smithi043.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-14T23:54:39.992 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-14T23:54:39.992 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-14T23:54:39.992 INFO:teuthology.orchestra.run.smithi043.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-14T23:54:39.992 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:54:39.992 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-14T23:54:39.992 INFO:teuthology.orchestra.run.smithi043.stdout:====================================================================================== 2024-09-14T23:54:39.992 INFO:teuthology.orchestra.run.smithi043.stdout:Install 12 Packages 2024-09-14T23:54:39.992 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:54:39.993 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 38 M 2024-09-14T23:54:39.993 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 212 M 2024-09-14T23:54:39.993 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-14T23:54:40.205 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-09-14T23:54:40.287 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-14T23:54:40.345 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-14T23:54:40.451 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-14T23:54:40.488 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-14T23:54:40.497 INFO:teuthology.orchestra.run.smithi043.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.6 MB/s | 191 kB 00:00 2024-09-14T23:54:40.555 INFO:teuthology.orchestra.run.smithi043.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 4.5 MB/s | 801 kB 00:00 2024-09-14T23:54:40.614 INFO:teuthology.orchestra.run.smithi043.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 279 kB/s | 33 kB 00:00 2024-09-14T23:54:40.682 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-14T23:54:40.689 INFO:teuthology.orchestra.run.smithi043.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 243 kB/s | 18 kB 00:00 2024-09-14T23:54:40.712 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-14T23:54:40.765 INFO:teuthology.orchestra.run.smithi043.stdout:(5/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 14 MB/s | 3.0 MB 00:00 2024-09-14T23:54:40.790 INFO:teuthology.orchestra.run.smithi043.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 384 kB/s | 38 kB 00:00 2024-09-14T23:54:40.792 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-14T23:54:40.882 INFO:teuthology.orchestra.run.smithi043.stdout:(7/12): ceph-mgr-diskprediction-local-19.1.1-32 15 MB/s | 7.4 MB 00:00 2024-09-14T23:54:40.924 INFO:teuthology.orchestra.run.smithi043.stdout:(8/12): python3-devel-3.9.19-8.el9.x86_64.rpm 1.8 MB/s | 245 kB 00:00 2024-09-14T23:54:40.999 INFO:teuthology.orchestra.run.smithi043.stdout:(9/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 21 MB/s | 4.9 MB 00:00 2024-09-14T23:54:41.024 INFO:teuthology.orchestra.run.smithi043.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 2.4 MB/s | 250 kB 00:00 2024-09-14T23:54:41.191 INFO:teuthology.orchestra.run.smithi043.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 5.1 MB 00:00 2024-09-14T23:54:41.400 INFO:teuthology.orchestra.run.smithi043.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 41 MB/s | 16 MB 00:00 2024-09-14T23:54:41.400 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-14T23:54:41.401 INFO:teuthology.orchestra.run.smithi043.stdout:Total 27 MB/s | 38 MB 00:01 2024-09-14T23:54:41.478 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-14T23:54:41.595 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-14T23:54:41.766 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-14T23:54:41.793 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-14T23:54:41.793 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-14T23:54:42.244 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-14T23:54:42.245 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-14T23:54:42.891 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-14T23:54:43.058 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-14T23:54:43.132 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-14T23:54:43.178 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-14T23:54:43.285 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-14T23:54:43.324 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-14T23:54:43.519 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-14T23:54:43.548 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-14T23:54:43.627 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-14T23:54:43.981 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-14T23:54:44.012 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-324.gecab7b 12/12 2024-09-14T23:54:44.321 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-14T23:54:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-14T23:54:45.753 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-14T23:54:46.526 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-324.gecab7b 12/12 2024-09-14T23:54:46.526 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-324.gecab7b 1/12 2024-09-14T23:54:46.526 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-14T23:54:46.526 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-14T23:54:46.526 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-14T23:54:46.526 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-14T23:54:46.527 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-14T23:54:46.527 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-14T23:54:46.527 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-14T23:54:46.527 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-14T23:54:46.527 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-14T23:54:46.527 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-14T23:54:46.816 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-14T23:54:46.858 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-324.gecab7b 12/12 2024-09-14T23:54:46.922 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-14T23:54:46.922 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:54:46.922 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-09-14T23:54:46.923 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-diskprediction-local-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-14T23:54:46.923 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-14T23:54:46.923 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-14T23:54:46.923 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-14T23:54:46.923 INFO:teuthology.orchestra.run.smithi042.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-14T23:54:46.923 INFO:teuthology.orchestra.run.smithi042.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-14T23:54:46.923 INFO:teuthology.orchestra.run.smithi042.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-14T23:54:46.923 INFO:teuthology.orchestra.run.smithi042.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-14T23:54:46.923 INFO:teuthology.orchestra.run.smithi042.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-14T23:54:46.923 INFO:teuthology.orchestra.run.smithi042.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-14T23:54:46.923 INFO:teuthology.orchestra.run.smithi042.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-14T23:54:46.923 INFO:teuthology.orchestra.run.smithi042.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-14T23:54:46.923 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:54:46.923 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-14T23:54:47.344 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-mgr-rook 2024-09-14T23:54:47.936 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:17 ago on Sat 14 Sep 2024 11:53:30 PM UTC. 2024-09-14T23:54:48.076 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-14T23:54:48.077 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-14T23:54:48.077 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repository Size 2024-09-14T23:54:48.077 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-14T23:54:48.077 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-09-14T23:54:48.077 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-rook noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 49 k 2024-09-14T23:54:48.077 INFO:teuthology.orchestra.run.smithi042.stdout:Installing dependencies: 2024-09-14T23:54:48.078 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-14T23:54:48.078 INFO:teuthology.orchestra.run.smithi042.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-14T23:54:48.078 INFO:teuthology.orchestra.run.smithi042.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-14T23:54:48.078 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-14T23:54:48.078 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-14T23:54:48.078 INFO:teuthology.orchestra.run.smithi042.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-14T23:54:48.078 INFO:teuthology.orchestra.run.smithi042.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-14T23:54:48.078 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-14T23:54:48.078 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-14T23:54:48.078 INFO:teuthology.orchestra.run.smithi042.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-14T23:54:48.078 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-14T23:54:48.079 INFO:teuthology.orchestra.run.smithi042.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-14T23:54:48.079 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:54:48.079 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-09-14T23:54:48.079 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-14T23:54:48.079 INFO:teuthology.orchestra.run.smithi042.stdout:Install 13 Packages 2024-09-14T23:54:48.079 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:54:48.080 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 2.2 M 2024-09-14T23:54:48.080 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 26 M 2024-09-14T23:54:48.080 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-09-14T23:54:48.470 INFO:teuthology.orchestra.run.smithi042.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 230 kB/s | 19 kB 00:00 2024-09-14T23:54:48.496 INFO:teuthology.orchestra.run.smithi042.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 236 kB/s | 26 kB 00:00 2024-09-14T23:54:48.554 INFO:teuthology.orchestra.run.smithi042.stdout:(3/13): ceph-mgr-rook-19.1.1-324.gecab7be5.el9. 294 kB/s | 49 kB 00:00 2024-09-14T23:54:48.580 INFO:teuthology.orchestra.run.smithi042.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.0 MB/s | 222 kB 00:00 2024-09-14T23:54:48.605 INFO:teuthology.orchestra.run.smithi042.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-09-14T23:54:48.630 INFO:teuthology.orchestra.run.smithi042.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.0 MB/s | 54 kB 00:00 2024-09-14T23:54:48.672 INFO:teuthology.orchestra.run.smithi042.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 482 kB/s | 32 kB 00:00 2024-09-14T23:54:48.697 INFO:teuthology.orchestra.run.smithi042.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 211 kB/s | 14 kB 00:00 2024-09-14T23:54:48.723 INFO:teuthology.orchestra.run.smithi042.stdout:(9/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.6 MB/s | 279 kB 00:00 2024-09-14T23:54:48.756 INFO:teuthology.orchestra.run.smithi042.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 2.6 MB/s | 220 kB 00:00 2024-09-14T23:54:48.782 INFO:teuthology.orchestra.run.smithi042.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 3.8 MB/s | 90 kB 00:00 2024-09-14T23:54:48.807 INFO:teuthology.orchestra.run.smithi042.stdout:(12/13): python3-rsa-4.9-2.el9.noarch.rpm 707 kB/s | 59 kB 00:00 2024-09-14T23:54:48.848 INFO:teuthology.orchestra.run.smithi042.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 6.8 MB/s | 1.0 MB 00:00 2024-09-14T23:54:48.851 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-09-14T23:54:48.851 INFO:teuthology.orchestra.run.smithi042.stdout:Total 2.9 MB/s | 2.2 MB 00:00 2024-09-14T23:54:48.920 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-09-14T23:54:48.937 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-09-14T23:54:48.937 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-09-14T23:54:49.033 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-09-14T23:54:49.034 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-09-14T23:54:49.383 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-324.gecab7b 12/12 2024-09-14T23:54:49.383 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-324.gecab7b 1/12 2024-09-14T23:54:49.383 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-14T23:54:49.383 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-14T23:54:49.383 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-14T23:54:49.383 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-14T23:54:49.383 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-14T23:54:49.383 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-14T23:54:49.383 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-14T23:54:49.384 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-14T23:54:49.384 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-14T23:54:49.384 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-14T23:54:49.551 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-09-14T23:54:49.730 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-14T23:54:49.785 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-14T23:54:49.849 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-14T23:54:49.854 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-14T23:54:49.854 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:54:49.854 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-14T23:54:49.854 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-diskprediction-local-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-14T23:54:49.854 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-14T23:54:49.854 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-14T23:54:49.854 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-14T23:54:49.854 INFO:teuthology.orchestra.run.smithi043.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-14T23:54:49.854 INFO:teuthology.orchestra.run.smithi043.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-14T23:54:49.855 INFO:teuthology.orchestra.run.smithi043.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-14T23:54:49.855 INFO:teuthology.orchestra.run.smithi043.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-14T23:54:49.855 INFO:teuthology.orchestra.run.smithi043.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-14T23:54:49.855 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-14T23:54:49.855 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-14T23:54:49.855 INFO:teuthology.orchestra.run.smithi043.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-14T23:54:49.855 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:54:49.855 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-14T23:54:49.893 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-14T23:54:49.935 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-14T23:54:50.041 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-14T23:54:50.151 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-14T23:54:50.212 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-14T23:54:50.218 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-mgr-rook 2024-09-14T23:54:50.810 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:19 ago on Sat 14 Sep 2024 11:53:31 PM UTC. 2024-09-14T23:54:50.951 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-14T23:54:50.953 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-14T23:54:50.953 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-14T23:54:50.953 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-14T23:54:50.953 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-14T23:54:50.953 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-rook noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 49 k 2024-09-14T23:54:50.953 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-14T23:54:50.953 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-14T23:54:50.953 INFO:teuthology.orchestra.run.smithi043.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-14T23:54:50.953 INFO:teuthology.orchestra.run.smithi043.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-14T23:54:50.953 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-14T23:54:50.953 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-14T23:54:50.954 INFO:teuthology.orchestra.run.smithi043.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-14T23:54:50.954 INFO:teuthology.orchestra.run.smithi043.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-14T23:54:50.954 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-14T23:54:50.954 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-14T23:54:50.954 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-14T23:54:50.954 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-14T23:54:50.954 INFO:teuthology.orchestra.run.smithi043.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-14T23:54:50.954 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:54:50.954 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-14T23:54:50.954 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-14T23:54:50.954 INFO:teuthology.orchestra.run.smithi043.stdout:Install 13 Packages 2024-09-14T23:54:50.954 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:54:50.955 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 2.2 M 2024-09-14T23:54:50.955 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 26 M 2024-09-14T23:54:50.955 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-14T23:54:50.975 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-14T23:54:51.033 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-14T23:54:51.067 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-14T23:54:51.125 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-14T23:54:51.153 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-rook-2:19.1.1-324.gecab7be5.el9.noarch 13/13 2024-09-14T23:54:51.390 INFO:teuthology.orchestra.run.smithi043.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 201 kB/s | 19 kB 00:00 2024-09-14T23:54:51.415 INFO:teuthology.orchestra.run.smithi043.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 212 kB/s | 26 kB 00:00 2024-09-14T23:54:51.475 INFO:teuthology.orchestra.run.smithi043.stdout:(3/13): ceph-mgr-rook-19.1.1-324.gecab7be5.el9. 275 kB/s | 49 kB 00:00 2024-09-14T23:54:51.524 INFO:teuthology.orchestra.run.smithi043.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.6 MB/s | 222 kB 00:00 2024-09-14T23:54:51.549 INFO:teuthology.orchestra.run.smithi043.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-09-14T23:54:51.583 INFO:teuthology.orchestra.run.smithi043.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 918 kB/s | 54 kB 00:00 2024-09-14T23:54:51.616 INFO:teuthology.orchestra.run.smithi043.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 481 kB/s | 32 kB 00:00 2024-09-14T23:54:51.633 INFO:teuthology.orchestra.run.smithi043.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 292 kB/s | 14 kB 00:00 2024-09-14T23:54:51.658 INFO:teuthology.orchestra.run.smithi043.stdout:(9/13): python3-google-auth-2.34.0-1.el9.noarch 5.4 MB/s | 220 kB 00:00 2024-09-14T23:54:51.684 INFO:teuthology.orchestra.run.smithi043.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-14T23:54:51.709 INFO:teuthology.orchestra.run.smithi043.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 3.5 MB/s | 90 kB 00:00 2024-09-14T23:54:51.734 INFO:teuthology.orchestra.run.smithi043.stdout:(12/13): python3-pyasn1-modules-0.4.8-6.el9.noa 1.0 MB/s | 279 kB 00:00 2024-09-14T23:54:51.776 INFO:teuthology.orchestra.run.smithi043.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 7.2 MB/s | 1.0 MB 00:00 2024-09-14T23:54:51.778 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-14T23:54:51.778 INFO:teuthology.orchestra.run.smithi043.stdout:Total 2.7 MB/s | 2.2 MB 00:00 2024-09-14T23:54:51.848 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-14T23:54:51.865 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-14T23:54:51.865 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-14T23:54:51.962 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-14T23:54:51.963 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-14T23:54:52.000 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-324.gecab7be5.el9.noarch 13/13 2024-09-14T23:54:52.001 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-rook-2:19.1.1-324.gecab7be5.el9.noarch 1/13 2024-09-14T23:54:52.001 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-14T23:54:52.001 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-14T23:54:52.001 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-14T23:54:52.001 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-14T23:54:52.001 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-14T23:54:52.001 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-14T23:54:52.001 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-14T23:54:52.001 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-14T23:54:52.001 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-14T23:54:52.001 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-14T23:54:52.002 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-14T23:54:52.336 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-14T23:54:52.336 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:54:52.336 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-09-14T23:54:52.336 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-rook-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-14T23:54:52.336 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-14T23:54:52.336 INFO:teuthology.orchestra.run.smithi042.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-14T23:54:52.336 INFO:teuthology.orchestra.run.smithi042.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-14T23:54:52.336 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-14T23:54:52.336 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-14T23:54:52.336 INFO:teuthology.orchestra.run.smithi042.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-14T23:54:52.336 INFO:teuthology.orchestra.run.smithi042.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-14T23:54:52.337 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-14T23:54:52.337 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-14T23:54:52.337 INFO:teuthology.orchestra.run.smithi042.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-14T23:54:52.337 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-14T23:54:52.337 INFO:teuthology.orchestra.run.smithi042.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-14T23:54:52.337 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:54:52.337 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-14T23:54:52.487 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-14T23:54:52.640 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-mgr-cephadm 2024-09-14T23:54:52.668 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-14T23:54:52.738 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-14T23:54:52.803 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-14T23:54:52.838 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-14T23:54:52.880 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-14T23:54:52.995 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-14T23:54:53.122 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-14T23:54:53.174 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-14T23:54:53.229 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:23 ago on Sat 14 Sep 2024 11:53:30 PM UTC. 2024-09-14T23:54:53.369 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-14T23:54:53.370 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-14T23:54:53.370 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repository Size 2024-09-14T23:54:53.370 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-14T23:54:53.370 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-09-14T23:54:53.370 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-cephadm noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 146 k 2024-09-14T23:54:53.370 INFO:teuthology.orchestra.run.smithi042.stdout:Installing dependencies: 2024-09-14T23:54:53.370 INFO:teuthology.orchestra.run.smithi042.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-14T23:54:53.371 INFO:teuthology.orchestra.run.smithi042.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-14T23:54:53.371 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-14T23:54:53.371 INFO:teuthology.orchestra.run.smithi042.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-14T23:54:53.371 INFO:teuthology.orchestra.run.smithi042.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-14T23:54:53.371 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:54:53.371 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-09-14T23:54:53.371 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-14T23:54:53.371 INFO:teuthology.orchestra.run.smithi042.stdout:Install 6 Packages 2024-09-14T23:54:53.371 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:54:53.372 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 7.0 M 2024-09-14T23:54:53.372 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 33 M 2024-09-14T23:54:53.372 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-09-14T23:54:53.842 INFO:teuthology.orchestra.run.smithi042.stdout:(1/6): ceph-mgr-cephadm-19.1.1-324.gecab7be5.el 680 kB/s | 146 kB 00:00 2024-09-14T23:54:53.909 INFO:teuthology.orchestra.run.smithi042.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 10 MB/s | 548 kB 00:00 2024-09-14T23:54:53.926 INFO:teuthology.orchestra.run.smithi042.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 3.4 MB/s | 58 kB 00:00 2024-09-14T23:54:53.945 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-14T23:54:53.951 INFO:teuthology.orchestra.run.smithi042.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 3.0 MB/s | 74 kB 00:00 2024-09-14T23:54:54.003 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-14T23:54:54.037 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-14T23:54:54.094 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-14T23:54:54.131 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-rook-2:19.1.1-324.gecab7be5.el9.noarch 13/13 2024-09-14T23:54:54.502 INFO:teuthology.orchestra.run.smithi042.stdout:(5/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 284 kB/s | 248 kB 00:00 2024-09-14T23:54:54.909 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-324.gecab7be5.el9.noarch 13/13 2024-09-14T23:54:54.909 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-rook-2:19.1.1-324.gecab7be5.el9.noarch 1/13 2024-09-14T23:54:54.909 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-14T23:54:54.909 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-14T23:54:54.909 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-14T23:54:54.909 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-14T23:54:54.909 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-14T23:54:54.909 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-14T23:54:54.909 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-14T23:54:54.909 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-14T23:54:54.910 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-14T23:54:54.910 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-14T23:54:54.910 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-14T23:54:55.259 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-14T23:54:55.259 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:54:55.259 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-14T23:54:55.259 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-rook-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-14T23:54:55.259 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-14T23:54:55.259 INFO:teuthology.orchestra.run.smithi043.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-14T23:54:55.260 INFO:teuthology.orchestra.run.smithi043.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-14T23:54:55.260 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-14T23:54:55.260 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-14T23:54:55.260 INFO:teuthology.orchestra.run.smithi043.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-14T23:54:55.260 INFO:teuthology.orchestra.run.smithi043.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-14T23:54:55.260 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-14T23:54:55.260 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-14T23:54:55.260 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-14T23:54:55.260 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-14T23:54:55.260 INFO:teuthology.orchestra.run.smithi043.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-14T23:54:55.260 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:54:55.260 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-14T23:54:55.517 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-mgr-cephadm 2024-09-14T23:54:56.107 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:25 ago on Sat 14 Sep 2024 11:53:31 PM UTC. 2024-09-14T23:54:56.248 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-14T23:54:56.248 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-14T23:54:56.248 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-14T23:54:56.248 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-14T23:54:56.249 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-14T23:54:56.249 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-cephadm noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 146 k 2024-09-14T23:54:56.249 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-14T23:54:56.249 INFO:teuthology.orchestra.run.smithi043.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-14T23:54:56.249 INFO:teuthology.orchestra.run.smithi043.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-14T23:54:56.249 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-14T23:54:56.249 INFO:teuthology.orchestra.run.smithi043.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-14T23:54:56.249 INFO:teuthology.orchestra.run.smithi043.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-14T23:54:56.249 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:54:56.249 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-14T23:54:56.249 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-14T23:54:56.249 INFO:teuthology.orchestra.run.smithi043.stdout:Install 6 Packages 2024-09-14T23:54:56.249 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:54:56.250 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 7.0 M 2024-09-14T23:54:56.250 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 33 M 2024-09-14T23:54:56.250 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-14T23:54:56.685 INFO:teuthology.orchestra.run.smithi042.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 2.0 MB/s | 6.0 MB 00:03 2024-09-14T23:54:56.688 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-09-14T23:54:56.688 INFO:teuthology.orchestra.run.smithi042.stdout:Total 2.1 MB/s | 7.0 MB 00:03 2024-09-14T23:54:56.764 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-09-14T23:54:56.770 INFO:teuthology.orchestra.run.smithi043.stdout:(1/6): ceph-mgr-cephadm-19.1.1-324.gecab7be5.el 701 kB/s | 146 kB 00:00 2024-09-14T23:54:56.777 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-09-14T23:54:56.777 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-09-14T23:54:56.837 INFO:teuthology.orchestra.run.smithi043.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 8.0 MB/s | 548 kB 00:00 2024-09-14T23:54:56.863 INFO:teuthology.orchestra.run.smithi043.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 2.3 MB/s | 58 kB 00:00 2024-09-14T23:54:56.885 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-09-14T23:54:56.885 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-09-14T23:54:56.888 INFO:teuthology.orchestra.run.smithi043.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 3.0 MB/s | 74 kB 00:00 2024-09-14T23:54:57.217 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-09-14T23:54:57.363 INFO:teuthology.orchestra.run.smithi043.stdout:(5/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 310 kB/s | 248 kB 00:00 2024-09-14T23:54:57.400 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-14T23:54:57.455 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-14T23:54:57.722 INFO:teuthology.orchestra.run.smithi043.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 5.1 MB/s | 6.0 MB 00:01 2024-09-14T23:54:57.724 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-14T23:54:57.724 INFO:teuthology.orchestra.run.smithi043.stdout:Total 4.8 MB/s | 7.0 MB 00:01 2024-09-14T23:54:57.800 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-14T23:54:57.813 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-14T23:54:57.813 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-14T23:54:57.921 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-14T23:54:57.921 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-14T23:54:58.008 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-14T23:54:58.101 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-14T23:54:58.172 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-14T23:54:58.198 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-324.gecab7be5.el9.noarch 6/6 2024-09-14T23:54:58.247 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-14T23:54:58.438 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-14T23:54:58.483 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-14T23:54:59.015 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-324.gecab7be5.el9.noarch 6/6 2024-09-14T23:54:59.016 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-324.gecab7be5.el9.noarch 1/6 2024-09-14T23:54:59.016 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-14T23:54:59.016 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-14T23:54:59.016 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-14T23:54:59.016 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-14T23:54:59.063 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-14T23:54:59.163 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-14T23:54:59.243 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-14T23:54:59.269 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-324.gecab7be5.el9.noarch 6/6 2024-09-14T23:54:59.293 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-14T23:54:59.293 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:54:59.293 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-09-14T23:54:59.293 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-cephadm-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-14T23:54:59.293 INFO:teuthology.orchestra.run.smithi042.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-14T23:54:59.293 INFO:teuthology.orchestra.run.smithi042.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-14T23:54:59.293 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-14T23:54:59.293 INFO:teuthology.orchestra.run.smithi042.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-14T23:54:59.293 INFO:teuthology.orchestra.run.smithi042.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-14T23:54:59.293 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:54:59.293 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-14T23:54:59.551 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-fuse 2024-09-14T23:55:00.039 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-324.gecab7be5.el9.noarch 6/6 2024-09-14T23:55:00.039 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-324.gecab7be5.el9.noarch 1/6 2024-09-14T23:55:00.039 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-14T23:55:00.039 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-14T23:55:00.039 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-14T23:55:00.039 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-14T23:55:00.144 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:30 ago on Sat 14 Sep 2024 11:53:30 PM UTC. 2024-09-14T23:55:00.284 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-14T23:55:00.285 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-14T23:55:00.285 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repository Size 2024-09-14T23:55:00.285 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-14T23:55:00.285 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-09-14T23:55:00.285 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-fuse x86_64 2:19.1.1-324.gecab7be5.el9 ceph 896 k 2024-09-14T23:55:00.285 INFO:teuthology.orchestra.run.smithi042.stdout:Installing dependencies: 2024-09-14T23:55:00.285 INFO:teuthology.orchestra.run.smithi042.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-14T23:55:00.285 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:00.285 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-09-14T23:55:00.285 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-14T23:55:00.285 INFO:teuthology.orchestra.run.smithi042.stdout:Install 2 Packages 2024-09-14T23:55:00.286 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:00.286 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 976 k 2024-09-14T23:55:00.286 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 2.9 M 2024-09-14T23:55:00.286 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-09-14T23:55:00.324 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-14T23:55:00.324 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:00.324 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-14T23:55:00.324 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-cephadm-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-14T23:55:00.324 INFO:teuthology.orchestra.run.smithi043.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-14T23:55:00.324 INFO:teuthology.orchestra.run.smithi043.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-14T23:55:00.324 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-14T23:55:00.324 INFO:teuthology.orchestra.run.smithi043.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-14T23:55:00.324 INFO:teuthology.orchestra.run.smithi043.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-14T23:55:00.325 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:00.325 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-14T23:55:00.537 INFO:teuthology.orchestra.run.smithi042.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 660 kB/s | 80 kB 00:00 2024-09-14T23:55:00.580 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-fuse 2024-09-14T23:55:00.687 INFO:teuthology.orchestra.run.smithi042.stdout:(2/2): ceph-fuse-19.1.1-324.gecab7be5.el9.x86_6 3.2 MB/s | 896 kB 00:00 2024-09-14T23:55:00.687 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-09-14T23:55:00.687 INFO:teuthology.orchestra.run.smithi042.stdout:Total 2.4 MB/s | 976 kB 00:00 2024-09-14T23:55:00.700 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-09-14T23:55:00.724 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-09-14T23:55:00.724 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-09-14T23:55:00.811 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-09-14T23:55:00.811 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-09-14T23:55:01.010 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-09-14T23:55:01.151 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-14T23:55:01.169 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:30 ago on Sat 14 Sep 2024 11:53:31 PM UTC. 2024-09-14T23:55:01.223 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 2/2 2024-09-14T23:55:01.309 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-14T23:55:01.310 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-14T23:55:01.310 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-14T23:55:01.310 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-14T23:55:01.311 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-14T23:55:01.311 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-fuse x86_64 2:19.1.1-324.gecab7be5.el9 ceph 896 k 2024-09-14T23:55:01.311 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-14T23:55:01.311 INFO:teuthology.orchestra.run.smithi043.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-14T23:55:01.311 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:01.311 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-14T23:55:01.311 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-14T23:55:01.311 INFO:teuthology.orchestra.run.smithi043.stdout:Install 2 Packages 2024-09-14T23:55:01.311 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:01.311 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 976 k 2024-09-14T23:55:01.311 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 2.9 M 2024-09-14T23:55:01.311 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-14T23:55:01.515 INFO:teuthology.orchestra.run.smithi043.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 519 kB/s | 80 kB 00:00 2024-09-14T23:55:01.649 INFO:teuthology.orchestra.run.smithi043.stdout:(2/2): ceph-fuse-19.1.1-324.gecab7be5.el9.x86_6 3.0 MB/s | 896 kB 00:00 2024-09-14T23:55:01.649 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-14T23:55:01.649 INFO:teuthology.orchestra.run.smithi043.stdout:Total 2.8 MB/s | 976 kB 00:00 2024-09-14T23:55:01.661 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-14T23:55:01.685 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-14T23:55:01.685 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-14T23:55:01.770 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-14T23:55:01.771 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-14T23:55:01.931 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 2/2 2024-09-14T23:55:01.931 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 1/2 2024-09-14T23:55:01.969 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-14T23:55:02.114 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-14T23:55:02.193 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 2/2 2024-09-14T23:55:02.214 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-14T23:55:02.214 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:02.215 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-09-14T23:55:02.215 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-14T23:55:02.215 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:02.215 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-14T23:55:02.509 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-volume 2024-09-14T23:55:02.841 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 2/2 2024-09-14T23:55:02.841 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 1/2 2024-09-14T23:55:03.110 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:33 ago on Sat 14 Sep 2024 11:53:30 PM UTC. 2024-09-14T23:55:03.149 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-14T23:55:03.150 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:03.150 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-14T23:55:03.150 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-14T23:55:03.150 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:03.150 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-14T23:55:03.251 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-14T23:55:03.252 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-14T23:55:03.252 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repository Size 2024-09-14T23:55:03.252 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-14T23:55:03.253 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-09-14T23:55:03.253 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-volume noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 285 k 2024-09-14T23:55:03.253 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:03.253 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-09-14T23:55:03.253 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-14T23:55:03.253 INFO:teuthology.orchestra.run.smithi042.stdout:Install 1 Package 2024-09-14T23:55:03.253 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:03.253 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 285 k 2024-09-14T23:55:03.253 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 1.3 M 2024-09-14T23:55:03.253 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-09-14T23:55:03.420 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-volume 2024-09-14T23:55:03.474 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-volume-19.1.1-324.gecab7be5.el9.noarch.rpm 1.3 MB/s | 285 kB 00:00 2024-09-14T23:55:03.475 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-09-14T23:55:03.475 INFO:teuthology.orchestra.run.smithi042.stdout:Total 1.3 MB/s | 285 kB 00:00 2024-09-14T23:55:03.475 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-09-14T23:55:03.485 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-09-14T23:55:03.485 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-09-14T23:55:03.513 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-09-14T23:55:03.514 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-09-14T23:55:03.746 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-09-14T23:55:03.818 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-volume-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-14T23:55:03.836 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-volume-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-14T23:55:03.836 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T23:55:03.836 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-14T23:55:03.836 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:04.023 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:33 ago on Sat 14 Sep 2024 11:53:31 PM UTC. 2024-09-14T23:55:04.164 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-14T23:55:04.165 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-14T23:55:04.165 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-14T23:55:04.165 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-14T23:55:04.165 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-14T23:55:04.165 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-volume noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 285 k 2024-09-14T23:55:04.165 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:04.165 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-14T23:55:04.166 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-14T23:55:04.166 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-09-14T23:55:04.166 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:04.166 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 285 k 2024-09-14T23:55:04.166 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 1.3 M 2024-09-14T23:55:04.166 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-14T23:55:04.378 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-volume-19.1.1-324.gecab7be5.el9.noarch.rpm 1.3 MB/s | 285 kB 00:00 2024-09-14T23:55:04.379 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-14T23:55:04.379 INFO:teuthology.orchestra.run.smithi043.stdout:Total 1.3 MB/s | 285 kB 00:00 2024-09-14T23:55:04.379 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-14T23:55:04.389 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-14T23:55:04.389 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-14T23:55:04.421 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-14T23:55:04.421 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-14T23:55:04.651 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-14T23:55:04.714 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-volume-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-14T23:55:04.732 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-volume-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-14T23:55:04.732 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T23:55:04.732 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-14T23:55:04.732 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:04.770 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-volume-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-14T23:55:04.771 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:04.771 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-09-14T23:55:04.771 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-volume-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-14T23:55:04.771 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:04.771 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-14T23:55:05.081 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install librados-devel 2024-09-14T23:55:05.678 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:35 ago on Sat 14 Sep 2024 11:53:30 PM UTC. 2024-09-14T23:55:05.791 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-volume-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-14T23:55:05.791 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:05.791 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-14T23:55:05.791 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-volume-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-14T23:55:05.792 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:05.792 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-14T23:55:05.818 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-14T23:55:05.819 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-14T23:55:05.819 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repo Size 2024-09-14T23:55:05.819 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-14T23:55:05.819 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-09-14T23:55:05.819 INFO:teuthology.orchestra.run.smithi042.stdout: librados-devel x86_64 2:19.1.1-324.gecab7be5.el9 ceph 127 k 2024-09-14T23:55:05.820 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:05.820 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-09-14T23:55:05.820 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-14T23:55:05.820 INFO:teuthology.orchestra.run.smithi042.stdout:Install 1 Package 2024-09-14T23:55:05.820 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:05.820 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 127 k 2024-09-14T23:55:05.820 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 456 k 2024-09-14T23:55:05.820 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-09-14T23:55:06.013 INFO:teuthology.orchestra.run.smithi042.stdout:librados-devel-19.1.1-324.gecab7be5.el9.x86_64. 660 kB/s | 127 kB 00:00 2024-09-14T23:55:06.013 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-09-14T23:55:06.013 INFO:teuthology.orchestra.run.smithi042.stdout:Total 655 kB/s | 127 kB 00:00 2024-09-14T23:55:06.014 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-09-14T23:55:06.019 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-09-14T23:55:06.019 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-09-14T23:55:06.088 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-09-14T23:55:06.088 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-09-14T23:55:06.111 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install librados-devel 2024-09-14T23:55:06.247 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-09-14T23:55:06.381 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : librados-devel-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T23:55:06.711 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:35 ago on Sat 14 Sep 2024 11:53:31 PM UTC. 2024-09-14T23:55:06.826 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librados-devel-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T23:55:06.851 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-14T23:55:06.852 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-14T23:55:06.852 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repo Size 2024-09-14T23:55:06.852 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-14T23:55:06.852 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-14T23:55:06.852 INFO:teuthology.orchestra.run.smithi043.stdout: librados-devel x86_64 2:19.1.1-324.gecab7be5.el9 ceph 127 k 2024-09-14T23:55:06.852 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:06.852 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-14T23:55:06.852 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-14T23:55:06.852 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-09-14T23:55:06.852 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:06.852 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 127 k 2024-09-14T23:55:06.852 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 456 k 2024-09-14T23:55:06.853 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-14T23:55:07.075 INFO:teuthology.orchestra.run.smithi043.stdout:librados-devel-19.1.1-324.gecab7be5.el9.x86_64. 571 kB/s | 127 kB 00:00 2024-09-14T23:55:07.076 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-14T23:55:07.076 INFO:teuthology.orchestra.run.smithi043.stdout:Total 567 kB/s | 127 kB 00:00 2024-09-14T23:55:07.076 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-14T23:55:07.081 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-14T23:55:07.082 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-14T23:55:07.087 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librados-devel-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T23:55:07.088 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:07.088 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-09-14T23:55:07.088 INFO:teuthology.orchestra.run.smithi042.stdout: librados-devel-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T23:55:07.088 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:07.088 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-14T23:55:07.150 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-14T23:55:07.150 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-14T23:55:07.309 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-14T23:55:07.358 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install libcephfs2 2024-09-14T23:55:07.409 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librados-devel-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T23:55:07.841 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librados-devel-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T23:55:07.957 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:37 ago on Sat 14 Sep 2024 11:53:30 PM UTC. 2024-09-14T23:55:08.053 INFO:teuthology.orchestra.run.smithi042.stdout:Package libcephfs2-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-14T23:55:08.099 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-14T23:55:08.100 INFO:teuthology.orchestra.run.smithi042.stdout:Nothing to do. 2024-09-14T23:55:08.100 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-14T23:55:08.115 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librados-devel-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T23:55:08.115 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:08.115 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-14T23:55:08.115 INFO:teuthology.orchestra.run.smithi043.stdout: librados-devel-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T23:55:08.115 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:08.115 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-14T23:55:08.177 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install libcephfs-devel 2024-09-14T23:55:08.369 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install libcephfs2 2024-09-14T23:55:08.766 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:38 ago on Sat 14 Sep 2024 11:53:30 PM UTC. 2024-09-14T23:55:08.906 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-14T23:55:08.906 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-14T23:55:08.907 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repo Size 2024-09-14T23:55:08.907 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-14T23:55:08.907 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-09-14T23:55:08.907 INFO:teuthology.orchestra.run.smithi042.stdout: libcephfs-devel x86_64 2:19.1.1-324.gecab7be5.el9 ceph 32 k 2024-09-14T23:55:08.907 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:08.907 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-09-14T23:55:08.907 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-14T23:55:08.907 INFO:teuthology.orchestra.run.smithi042.stdout:Install 1 Package 2024-09-14T23:55:08.907 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:08.907 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 32 k 2024-09-14T23:55:08.907 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 142 k 2024-09-14T23:55:08.907 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-09-14T23:55:08.967 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:37 ago on Sat 14 Sep 2024 11:53:31 PM UTC. 2024-09-14T23:55:09.058 INFO:teuthology.orchestra.run.smithi042.stdout:libcephfs-devel-19.1.1-324.gecab7be5.el9.x86_64 211 kB/s | 32 kB 00:00 2024-09-14T23:55:09.059 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-09-14T23:55:09.059 INFO:teuthology.orchestra.run.smithi042.stdout:Total 209 kB/s | 32 kB 00:00 2024-09-14T23:55:09.059 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-09-14T23:55:09.062 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-09-14T23:55:09.062 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-09-14T23:55:09.063 INFO:teuthology.orchestra.run.smithi043.stdout:Package libcephfs2-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-14T23:55:09.082 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-09-14T23:55:09.083 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-09-14T23:55:09.109 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-14T23:55:09.110 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-09-14T23:55:09.110 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-14T23:55:09.167 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-09-14T23:55:09.173 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install libcephfs-devel 2024-09-14T23:55:09.293 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libcephfs-devel-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T23:55:09.697 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libcephfs-devel-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T23:55:09.761 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:38 ago on Sat 14 Sep 2024 11:53:31 PM UTC. 2024-09-14T23:55:09.901 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-14T23:55:09.901 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-14T23:55:09.902 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repo Size 2024-09-14T23:55:09.902 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-14T23:55:09.902 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-14T23:55:09.902 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs-devel x86_64 2:19.1.1-324.gecab7be5.el9 ceph 32 k 2024-09-14T23:55:09.902 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:09.902 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-14T23:55:09.902 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-14T23:55:09.902 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-09-14T23:55:09.902 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:09.902 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 32 k 2024-09-14T23:55:09.902 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 142 k 2024-09-14T23:55:09.902 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-14T23:55:09.933 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libcephfs-devel-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T23:55:09.933 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:09.934 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-09-14T23:55:09.934 INFO:teuthology.orchestra.run.smithi042.stdout: libcephfs-devel-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T23:55:09.934 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:09.934 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-14T23:55:10.054 INFO:teuthology.orchestra.run.smithi043.stdout:libcephfs-devel-19.1.1-324.gecab7be5.el9.x86_64 210 kB/s | 32 kB 00:00 2024-09-14T23:55:10.054 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-14T23:55:10.054 INFO:teuthology.orchestra.run.smithi043.stdout:Total 208 kB/s | 32 kB 00:00 2024-09-14T23:55:10.054 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-14T23:55:10.058 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-14T23:55:10.058 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-14T23:55:10.077 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-14T23:55:10.077 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-14T23:55:10.134 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install librados2 2024-09-14T23:55:10.160 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-14T23:55:10.287 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libcephfs-devel-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T23:55:10.632 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libcephfs-devel-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T23:55:10.738 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:40 ago on Sat 14 Sep 2024 11:53:30 PM UTC. 2024-09-14T23:55:10.836 INFO:teuthology.orchestra.run.smithi042.stdout:Package librados2-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-14T23:55:10.884 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-14T23:55:10.885 INFO:teuthology.orchestra.run.smithi042.stdout:Nothing to do. 2024-09-14T23:55:10.885 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-14T23:55:10.966 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install librbd1 2024-09-14T23:55:11.072 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libcephfs-devel-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T23:55:11.072 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:11.072 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-14T23:55:11.072 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs-devel-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T23:55:11.072 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:11.072 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-14T23:55:11.252 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install librados2 2024-09-14T23:55:11.562 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:41 ago on Sat 14 Sep 2024 11:53:30 PM UTC. 2024-09-14T23:55:11.657 INFO:teuthology.orchestra.run.smithi042.stdout:Package librbd1-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-14T23:55:11.704 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-14T23:55:11.705 INFO:teuthology.orchestra.run.smithi042.stdout:Nothing to do. 2024-09-14T23:55:11.705 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-14T23:55:11.785 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install python3-rados 2024-09-14T23:55:11.848 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:40 ago on Sat 14 Sep 2024 11:53:31 PM UTC. 2024-09-14T23:55:11.942 INFO:teuthology.orchestra.run.smithi043.stdout:Package librados2-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-14T23:55:11.988 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-14T23:55:11.989 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-09-14T23:55:11.989 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-14T23:55:12.056 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install librbd1 2024-09-14T23:55:12.375 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:42 ago on Sat 14 Sep 2024 11:53:30 PM UTC. 2024-09-14T23:55:12.469 INFO:teuthology.orchestra.run.smithi042.stdout:Package python3-rados-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-14T23:55:12.516 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-14T23:55:12.517 INFO:teuthology.orchestra.run.smithi042.stdout:Nothing to do. 2024-09-14T23:55:12.517 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-14T23:55:12.596 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install python3-rgw 2024-09-14T23:55:12.642 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:41 ago on Sat 14 Sep 2024 11:53:31 PM UTC. 2024-09-14T23:55:12.735 INFO:teuthology.orchestra.run.smithi043.stdout:Package librbd1-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-14T23:55:12.782 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-14T23:55:12.783 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-09-14T23:55:12.783 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-14T23:55:12.860 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install python3-rados 2024-09-14T23:55:13.186 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:43 ago on Sat 14 Sep 2024 11:53:30 PM UTC. 2024-09-14T23:55:13.280 INFO:teuthology.orchestra.run.smithi042.stdout:Package python3-rgw-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-14T23:55:13.325 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-14T23:55:13.326 INFO:teuthology.orchestra.run.smithi042.stdout:Nothing to do. 2024-09-14T23:55:13.327 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-14T23:55:13.409 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install python3-cephfs 2024-09-14T23:55:13.453 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:42 ago on Sat 14 Sep 2024 11:53:31 PM UTC. 2024-09-14T23:55:13.546 INFO:teuthology.orchestra.run.smithi043.stdout:Package python3-rados-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-14T23:55:13.592 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-14T23:55:13.593 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-09-14T23:55:13.593 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-14T23:55:13.667 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install python3-rgw 2024-09-14T23:55:14.000 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:43 ago on Sat 14 Sep 2024 11:53:30 PM UTC. 2024-09-14T23:55:14.095 INFO:teuthology.orchestra.run.smithi042.stdout:Package python3-cephfs-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-14T23:55:14.141 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-14T23:55:14.142 INFO:teuthology.orchestra.run.smithi042.stdout:Nothing to do. 2024-09-14T23:55:14.142 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-14T23:55:14.228 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install python3-rbd 2024-09-14T23:55:14.261 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:43 ago on Sat 14 Sep 2024 11:53:31 PM UTC. 2024-09-14T23:55:14.355 INFO:teuthology.orchestra.run.smithi043.stdout:Package python3-rgw-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-14T23:55:14.401 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-14T23:55:14.402 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-09-14T23:55:14.403 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-14T23:55:14.476 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install python3-cephfs 2024-09-14T23:55:14.822 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:44 ago on Sat 14 Sep 2024 11:53:30 PM UTC. 2024-09-14T23:55:14.916 INFO:teuthology.orchestra.run.smithi042.stdout:Package python3-rbd-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-14T23:55:14.963 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-14T23:55:14.965 INFO:teuthology.orchestra.run.smithi042.stdout:Nothing to do. 2024-09-14T23:55:14.965 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-14T23:55:15.044 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install rbd-fuse 2024-09-14T23:55:15.070 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:44 ago on Sat 14 Sep 2024 11:53:31 PM UTC. 2024-09-14T23:55:15.164 INFO:teuthology.orchestra.run.smithi043.stdout:Package python3-cephfs-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-14T23:55:15.211 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-14T23:55:15.212 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-09-14T23:55:15.212 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-14T23:55:15.290 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install python3-rbd 2024-09-14T23:55:15.634 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:45 ago on Sat 14 Sep 2024 11:53:30 PM UTC. 2024-09-14T23:55:15.776 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-14T23:55:15.777 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-14T23:55:15.777 INFO:teuthology.orchestra.run.smithi042.stdout: Package Architecture Version Repository Size 2024-09-14T23:55:15.777 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-14T23:55:15.777 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-09-14T23:55:15.777 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-fuse x86_64 2:19.1.1-324.gecab7be5.el9 ceph 85 k 2024-09-14T23:55:15.778 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:15.778 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-09-14T23:55:15.778 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-14T23:55:15.778 INFO:teuthology.orchestra.run.smithi042.stdout:Install 1 Package 2024-09-14T23:55:15.778 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:15.778 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 85 k 2024-09-14T23:55:15.778 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 231 k 2024-09-14T23:55:15.778 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-09-14T23:55:15.883 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:44 ago on Sat 14 Sep 2024 11:53:31 PM UTC. 2024-09-14T23:55:15.978 INFO:teuthology.orchestra.run.smithi043.stdout:Package python3-rbd-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-14T23:55:15.979 INFO:teuthology.orchestra.run.smithi042.stdout:rbd-fuse-19.1.1-324.gecab7be5.el9.x86_64.rpm 425 kB/s | 85 kB 00:00 2024-09-14T23:55:15.979 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-09-14T23:55:15.979 INFO:teuthology.orchestra.run.smithi042.stdout:Total 422 kB/s | 85 kB 00:00 2024-09-14T23:55:15.979 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-09-14T23:55:15.987 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-09-14T23:55:15.987 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-09-14T23:55:16.024 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-14T23:55:16.025 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-09-14T23:55:16.025 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-14T23:55:16.054 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-09-14T23:55:16.054 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-09-14T23:55:16.098 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install rbd-fuse 2024-09-14T23:55:16.216 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-09-14T23:55:16.329 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : rbd-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T23:55:16.687 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:45 ago on Sat 14 Sep 2024 11:53:31 PM UTC. 2024-09-14T23:55:16.789 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: rbd-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T23:55:16.829 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-14T23:55:16.829 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-14T23:55:16.830 INFO:teuthology.orchestra.run.smithi043.stdout: Package Architecture Version Repository Size 2024-09-14T23:55:16.830 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-14T23:55:16.830 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-14T23:55:16.830 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-fuse x86_64 2:19.1.1-324.gecab7be5.el9 ceph 85 k 2024-09-14T23:55:16.830 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:16.830 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-14T23:55:16.830 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-14T23:55:16.830 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-09-14T23:55:16.830 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:16.830 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 85 k 2024-09-14T23:55:16.830 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 231 k 2024-09-14T23:55:16.830 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-14T23:55:17.023 INFO:teuthology.orchestra.run.smithi043.stdout:rbd-fuse-19.1.1-324.gecab7be5.el9.x86_64.rpm 443 kB/s | 85 kB 00:00 2024-09-14T23:55:17.023 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-14T23:55:17.024 INFO:teuthology.orchestra.run.smithi043.stdout:Total 439 kB/s | 85 kB 00:00 2024-09-14T23:55:17.024 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-14T23:55:17.031 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-14T23:55:17.032 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-14T23:55:17.069 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : rbd-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T23:55:17.069 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:17.069 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-09-14T23:55:17.069 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T23:55:17.069 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:17.069 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-14T23:55:17.098 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-14T23:55:17.099 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-14T23:55:17.257 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-14T23:55:17.323 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install rbd-mirror 2024-09-14T23:55:17.349 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : rbd-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T23:55:17.725 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: rbd-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T23:55:17.922 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:47 ago on Sat 14 Sep 2024 11:53:30 PM UTC. 2024-09-14T23:55:18.006 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : rbd-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T23:55:18.006 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:18.007 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-14T23:55:18.007 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T23:55:18.007 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:18.007 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-14T23:55:18.064 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-14T23:55:18.064 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-14T23:55:18.065 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repo Size 2024-09-14T23:55:18.065 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-14T23:55:18.065 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-09-14T23:55:18.065 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-mirror x86_64 2:19.1.1-324.gecab7be5.el9 ceph 3.1 M 2024-09-14T23:55:18.065 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:18.065 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-09-14T23:55:18.065 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-14T23:55:18.065 INFO:teuthology.orchestra.run.smithi042.stdout:Install 1 Package 2024-09-14T23:55:18.065 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:18.065 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 3.1 M 2024-09-14T23:55:18.065 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 13 M 2024-09-14T23:55:18.065 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-09-14T23:55:18.281 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install rbd-mirror 2024-09-14T23:55:18.382 INFO:teuthology.orchestra.run.smithi042.stdout:rbd-mirror-19.1.1-324.gecab7be5.el9.x86_64.rpm 9.8 MB/s | 3.1 MB 00:00 2024-09-14T23:55:18.382 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-09-14T23:55:18.383 INFO:teuthology.orchestra.run.smithi042.stdout:Total 9.8 MB/s | 3.1 MB 00:00 2024-09-14T23:55:18.383 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-09-14T23:55:18.393 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-09-14T23:55:18.393 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-09-14T23:55:18.478 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-09-14T23:55:18.478 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-09-14T23:55:18.887 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:47 ago on Sat 14 Sep 2024 11:53:31 PM UTC. 2024-09-14T23:55:18.934 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-09-14T23:55:18.999 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : rbd-mirror-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T23:55:19.026 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: rbd-mirror-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T23:55:19.026 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T23:55:19.027 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-14T23:55:19.027 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-14T23:55:19.027 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-14T23:55:19.027 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:19.030 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-14T23:55:19.031 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-14T23:55:19.031 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repo Size 2024-09-14T23:55:19.031 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-14T23:55:19.031 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-14T23:55:19.031 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-mirror x86_64 2:19.1.1-324.gecab7be5.el9 ceph 3.1 M 2024-09-14T23:55:19.031 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:19.031 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-14T23:55:19.031 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-14T23:55:19.031 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-09-14T23:55:19.031 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:19.032 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 3.1 M 2024-09-14T23:55:19.032 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 13 M 2024-09-14T23:55:19.032 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-14T23:55:19.369 INFO:teuthology.orchestra.run.smithi043.stdout:rbd-mirror-19.1.1-324.gecab7be5.el9.x86_64.rpm 9.2 MB/s | 3.1 MB 00:00 2024-09-14T23:55:19.370 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-14T23:55:19.370 INFO:teuthology.orchestra.run.smithi043.stdout:Total 9.2 MB/s | 3.1 MB 00:00 2024-09-14T23:55:19.370 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-14T23:55:19.381 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-14T23:55:19.381 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-14T23:55:19.467 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-14T23:55:19.468 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-14T23:55:19.920 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-14T23:55:19.986 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : rbd-mirror-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T23:55:20.014 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: rbd-mirror-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T23:55:20.014 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T23:55:20.014 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-14T23:55:20.014 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-14T23:55:20.014 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-14T23:55:20.014 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:20.207 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : rbd-mirror-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T23:55:20.207 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:20.207 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-09-14T23:55:20.207 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-mirror-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T23:55:20.207 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:20.207 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-14T23:55:20.470 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install rbd-nbd 2024-09-14T23:55:21.077 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:51 ago on Sat 14 Sep 2024 11:53:30 PM UTC. 2024-09-14T23:55:21.171 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : rbd-mirror-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T23:55:21.172 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:21.172 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-14T23:55:21.172 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-mirror-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T23:55:21.172 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:21.172 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-14T23:55:21.218 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-14T23:55:21.219 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-14T23:55:21.219 INFO:teuthology.orchestra.run.smithi042.stdout: Package Architecture Version Repository Size 2024-09-14T23:55:21.219 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-14T23:55:21.219 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-09-14T23:55:21.219 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-nbd x86_64 2:19.1.1-324.gecab7be5.el9 ceph 170 k 2024-09-14T23:55:21.219 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:21.219 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-09-14T23:55:21.219 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-14T23:55:21.220 INFO:teuthology.orchestra.run.smithi042.stdout:Install 1 Package 2024-09-14T23:55:21.220 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:21.220 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 170 k 2024-09-14T23:55:21.220 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 494 k 2024-09-14T23:55:21.220 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-09-14T23:55:21.434 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install rbd-nbd 2024-09-14T23:55:21.435 INFO:teuthology.orchestra.run.smithi042.stdout:rbd-nbd-19.1.1-324.gecab7be5.el9.x86_64.rpm 792 kB/s | 170 kB 00:00 2024-09-14T23:55:21.437 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-09-14T23:55:21.437 INFO:teuthology.orchestra.run.smithi042.stdout:Total 786 kB/s | 170 kB 00:00 2024-09-14T23:55:21.437 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-09-14T23:55:21.445 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-09-14T23:55:21.445 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-09-14T23:55:21.513 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-09-14T23:55:21.513 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-09-14T23:55:21.677 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-09-14T23:55:21.786 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : rbd-nbd-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T23:55:22.033 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:51 ago on Sat 14 Sep 2024 11:53:31 PM UTC. 2024-09-14T23:55:22.174 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-14T23:55:22.175 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-14T23:55:22.175 INFO:teuthology.orchestra.run.smithi043.stdout: Package Architecture Version Repository Size 2024-09-14T23:55:22.175 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-14T23:55:22.175 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-14T23:55:22.175 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-nbd x86_64 2:19.1.1-324.gecab7be5.el9 ceph 170 k 2024-09-14T23:55:22.176 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:22.176 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-14T23:55:22.176 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-14T23:55:22.176 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-09-14T23:55:22.176 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:22.176 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 170 k 2024-09-14T23:55:22.176 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 494 k 2024-09-14T23:55:22.176 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-14T23:55:22.207 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: rbd-nbd-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T23:55:22.389 INFO:teuthology.orchestra.run.smithi043.stdout:rbd-nbd-19.1.1-324.gecab7be5.el9.x86_64.rpm 803 kB/s | 170 kB 00:00 2024-09-14T23:55:22.389 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-14T23:55:22.390 INFO:teuthology.orchestra.run.smithi043.stdout:Total 796 kB/s | 170 kB 00:00 2024-09-14T23:55:22.390 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-14T23:55:22.398 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-14T23:55:22.398 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-14T23:55:22.464 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : rbd-nbd-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T23:55:22.464 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:22.464 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-09-14T23:55:22.464 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-nbd-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T23:55:22.464 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:22.465 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-14T23:55:22.466 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-14T23:55:22.466 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-14T23:55:22.627 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-14T23:55:22.731 DEBUG:teuthology.parallel:result is None 2024-09-14T23:55:22.739 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : rbd-nbd-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T23:55:23.157 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: rbd-nbd-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T23:55:23.455 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : rbd-nbd-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T23:55:23.455 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:23.455 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-14T23:55:23.455 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-nbd-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T23:55:23.455 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:23.455 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-14T23:55:23.695 DEBUG:teuthology.parallel:result is None 2024-09-14T23:55:23.696 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ecab7be58963a5bd03910cd73eef66405d7fdd50 2024-09-14T23:55:23.842 DEBUG:teuthology.orchestra.run.smithi042:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-14T23:55:23.868 INFO:teuthology.orchestra.run.smithi042.stdout:19.1.1-324.gecab7be5.el9 2024-09-14T23:55:23.869 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-324.gecab7be5.el9 2024-09-14T23:55:23.869 INFO:teuthology.task.install:The correct ceph version 19.1.1-324.gecab7be5 is installed. 2024-09-14T23:55:23.871 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ecab7be58963a5bd03910cd73eef66405d7fdd50 2024-09-14T23:55:24.017 DEBUG:teuthology.orchestra.run.smithi043:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-14T23:55:24.040 INFO:teuthology.orchestra.run.smithi043.stdout:19.1.1-324.gecab7be5.el9 2024-09-14T23:55:24.040 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-324.gecab7be5.el9 2024-09-14T23:55:24.040 INFO:teuthology.task.install:The correct ceph version 19.1.1-324.gecab7be5 is installed. 2024-09-14T23:55:24.042 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-14T23:55:24.042 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-14T23:55:24.043 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-14T23:55:24.075 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-14T23:55:24.075 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-14T23:55:24.109 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-14T23:55:24.109 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-14T23:55:24.109 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/usr/bin/daemon-helper 2024-09-14T23:55:24.145 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-14T23:55:24.213 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-14T23:55:24.213 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/daemon-helper 2024-09-14T23:55:24.240 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-14T23:55:24.306 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-14T23:55:24.306 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-14T23:55:24.307 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-14T23:55:24.334 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-14T23:55:24.402 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-14T23:55:24.402 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-14T23:55:24.439 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-14T23:55:24.505 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-14T23:55:24.506 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-14T23:55:24.506 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/usr/bin/stdin-killer 2024-09-14T23:55:24.533 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-14T23:55:24.601 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-14T23:55:24.601 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/stdin-killer 2024-09-14T23:55:24.627 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-14T23:55:24.694 INFO:teuthology.run_tasks:Running task ceph... 2024-09-14T23:55:24.764 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-14T23:55:24.764 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 777 /var/log/ceph 2024-09-14T23:55:24.767 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 777 /var/log/ceph 2024-09-14T23:55:24.793 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-14T23:55:24.793 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-14T23:55:24.835 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-14T23:55:24.861 INFO:tasks.ceph:Creating extra log directories... 2024-09-14T23:55:24.861 DEBUG:teuthology.orchestra.run.smithi042:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-14T23:55:24.903 DEBUG:teuthology.orchestra.run.smithi043:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-14T23:55:24.938 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-14T23:55:24.938 INFO:tasks.ceph:config {'conf': {'client': {'rbd cache': False}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-14T23:55:24.938 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904803', 'branch': 'squid', 'description': 'rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nocache}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7904803', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'client': {'rbd cache': False}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 4271, 'sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50', 'sleep_before_teardown': 0, 'subset': '97/128', 'suite': 'rbd', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_ecab7be58963a5bd03910cd73eef66405d7fdd50/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50', 'targets': {'smithi042.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDb3OL/nC1WjLPH+EZA18Mx7L9b0q+LMSqIFVWM1efshryjeK83pNoMNh38bJdpj8moukGrjZYn6T1QMNUWEDOw=', 'smithi043.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLazenSCK9k57rjGeexoS3vIBbfy0qN3INEm2BDxrd7uuRvutxr20LZrUYuIBXBL6azuh/WGr3KYXsIzZlkbMwo='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'ops': 6000}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-14_21:16:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-14T23:55:24.938 DEBUG:teuthology.orchestra.run.smithi042:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-14T23:55:24.971 DEBUG:teuthology.orchestra.run.smithi043:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-14T23:55:24.987 DEBUG:teuthology.orchestra.run.smithi042:> sudo install -d -m0777 -- /var/run/ceph 2024-09-14T23:55:25.029 DEBUG:teuthology.orchestra.run.smithi043:> sudo install -d -m0777 -- /var/run/ceph 2024-09-14T23:55:25.054 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-14T23:55:25.054 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/scratch_devs of=/dev/stdout 2024-09-14T23:55:25.111 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-14T23:55:25.112 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_1 2024-09-14T23:55:25.168 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-14T23:55:25.168 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T23:55:25.168 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-14T23:55:25.168 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T23:55:25.168 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:device_t:s0 2024-09-14T23:55:25.168 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-09-14 23:55:19.298916430 +0000 2024-09-14T23:55:25.169 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-09-14 23:52:50.597813463 +0000 2024-09-14T23:55:25.169 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-09-14 23:52:50.598813455 +0000 2024-09-14T23:55:25.169 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: 2024-09-14 23:52:50.597813463 +0000 2024-09-14T23:55:25.169 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-14T23:55:25.234 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-09-14T23:55:25.234 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-09-14T23:55:25.234 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.000103539 s, 4.9 MB/s 2024-09-14T23:55:25.235 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-14T23:55:25.293 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_2 2024-09-14T23:55:25.350 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-14T23:55:25.350 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T23:55:25.350 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-14T23:55:25.350 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T23:55:25.350 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:device_t:s0 2024-09-14T23:55:25.350 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-09-14 23:55:19.299916422 +0000 2024-09-14T23:55:25.350 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-09-14 23:52:50.935810907 +0000 2024-09-14T23:55:25.350 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-09-14 23:52:50.935810907 +0000 2024-09-14T23:55:25.350 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: 2024-09-14 23:52:50.935810907 +0000 2024-09-14T23:55:25.351 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-14T23:55:25.414 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-09-14T23:55:25.415 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-09-14T23:55:25.415 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.000103071 s, 5.0 MB/s 2024-09-14T23:55:25.416 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-14T23:55:25.474 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_3 2024-09-14T23:55:25.529 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-14T23:55:25.529 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T23:55:25.529 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-14T23:55:25.529 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T23:55:25.529 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:device_t:s0 2024-09-14T23:55:25.530 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-09-14 23:55:19.299916422 +0000 2024-09-14T23:55:25.530 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-09-14 23:52:51.278808313 +0000 2024-09-14T23:55:25.530 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-09-14 23:52:51.278808313 +0000 2024-09-14T23:55:25.530 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: 2024-09-14 23:52:51.278808313 +0000 2024-09-14T23:55:25.530 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-14T23:55:25.593 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-09-14T23:55:25.593 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-09-14T23:55:25.593 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 9.6238e-05 s, 5.3 MB/s 2024-09-14T23:55:25.594 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-14T23:55:25.651 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_4 2024-09-14T23:55:25.707 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-14T23:55:25.707 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T23:55:25.707 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-14T23:55:25.707 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T23:55:25.707 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:device_t:s0 2024-09-14T23:55:25.707 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-09-14 23:55:19.300916414 +0000 2024-09-14T23:55:25.707 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-09-14 23:52:51.611805795 +0000 2024-09-14T23:55:25.707 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-09-14 23:52:51.611805795 +0000 2024-09-14T23:55:25.708 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: 2024-09-14 23:52:51.611805795 +0000 2024-09-14T23:55:25.708 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-14T23:55:25.771 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-09-14T23:55:25.771 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-09-14T23:55:25.771 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.000101363 s, 5.1 MB/s 2024-09-14T23:55:25.772 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-14T23:55:25.829 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-14T23:55:25.829 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-14T23:55:25.829 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/scratch_devs of=/dev/stdout 2024-09-14T23:55:25.850 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-14T23:55:25.850 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_1 2024-09-14T23:55:25.909 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-14T23:55:25.910 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T23:55:25.910 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-14T23:55:25.910 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T23:55:25.910 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-09-14T23:55:25.910 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-09-14 23:55:20.273564633 +0000 2024-09-14T23:55:25.910 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-09-14 23:52:50.971285702 +0000 2024-09-14T23:55:25.910 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-09-14 23:52:50.971285702 +0000 2024-09-14T23:55:25.910 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-09-14 23:52:50.971285702 +0000 2024-09-14T23:55:25.910 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-14T23:55:25.980 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-09-14T23:55:25.980 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-09-14T23:55:25.980 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.00012378 s, 4.1 MB/s 2024-09-14T23:55:25.981 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-14T23:55:26.041 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_2 2024-09-14T23:55:26.099 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-14T23:55:26.100 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T23:55:26.100 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-14T23:55:26.100 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T23:55:26.100 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-09-14T23:55:26.100 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-09-14 23:55:20.273564633 +0000 2024-09-14T23:55:26.100 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-09-14 23:52:51.296282352 +0000 2024-09-14T23:55:26.100 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-09-14 23:52:51.296282352 +0000 2024-09-14T23:55:26.100 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-09-14 23:52:51.296282352 +0000 2024-09-14T23:55:26.100 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-14T23:55:26.168 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-09-14T23:55:26.168 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-09-14T23:55:26.168 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000127298 s, 4.0 MB/s 2024-09-14T23:55:26.169 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-14T23:55:26.229 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_3 2024-09-14T23:55:26.288 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-14T23:55:26.288 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T23:55:26.288 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-14T23:55:26.288 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T23:55:26.288 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-09-14T23:55:26.288 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-09-14 23:55:20.274564622 +0000 2024-09-14T23:55:26.288 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-09-14 23:52:51.619279023 +0000 2024-09-14T23:55:26.288 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-09-14 23:52:51.619279023 +0000 2024-09-14T23:55:26.288 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-09-14 23:52:51.619279023 +0000 2024-09-14T23:55:26.289 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-14T23:55:26.356 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-09-14T23:55:26.356 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-09-14T23:55:26.356 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 9.8448e-05 s, 5.2 MB/s 2024-09-14T23:55:26.358 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-14T23:55:26.417 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_4 2024-09-14T23:55:26.474 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-14T23:55:26.474 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T23:55:26.475 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-14T23:55:26.475 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T23:55:26.475 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-09-14T23:55:26.475 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-09-14 23:55:20.274564622 +0000 2024-09-14T23:55:26.475 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-09-14 23:52:51.918275941 +0000 2024-09-14T23:55:26.475 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-09-14 23:52:51.918275941 +0000 2024-09-14T23:55:26.475 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-09-14 23:52:51.918275941 +0000 2024-09-14T23:55:26.475 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-14T23:55:26.540 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-09-14T23:55:26.540 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-09-14T23:55:26.541 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000108534 s, 4.7 MB/s 2024-09-14T23:55:26.542 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-14T23:55:26.600 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-14T23:55:26.600 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi042.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi043.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-14T23:55:26.600 INFO:tasks.ceph:Generating config... 2024-09-14T23:55:26.601 INFO:tasks.ceph:[client] rbd cache = False 2024-09-14T23:55:26.601 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-14T23:55:26.601 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-14T23:55:26.601 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-14T23:55:26.601 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-14T23:55:26.602 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-14T23:55:26.602 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-14T23:55:26.602 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-14T23:55:26.602 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-14T23:55:26.602 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-14T23:55:26.602 INFO:tasks.ceph:[osd] bluestore compression algorithm = snappy 2024-09-14T23:55:26.602 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-09-14T23:55:26.602 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-14T23:55:26.602 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-14T23:55:26.602 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-14T23:55:26.602 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-14T23:55:26.602 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-14T23:55:26.602 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-14T23:55:26.603 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-14T23:55:26.603 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-14T23:55:26.603 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-14T23:55:26.603 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-14T23:55:26.603 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-14T23:55:26.603 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-14T23:55:26.603 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-14T23:55:26.603 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-09-14T23:55:26.603 INFO:tasks.ceph:Setting up mon.a... 2024-09-14T23:55:26.603 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-14T23:55:26.655 INFO:teuthology.orchestra.run.smithi042.stdout:creating /etc/ceph/ceph.keyring 2024-09-14T23:55:26.658 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-14T23:55:26.704 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-14T23:55:26.730 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.42'), ('mon.c', '[v2:172.21.15.42:3301,v1:172.21.15.42:6790]'), ('mon.b', '172.21.15.43')] 2024-09-14T23:55:26.731 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.42,[v2:172.21.15.42:3301,v1:172.21.15.42:6790],172.21.15.43', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'rbd cache': False}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-14T23:55:26.732 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-14T23:55:26.732 DEBUG:teuthology.orchestra.run.smithi042:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-14T23:55:26.788 DEBUG:teuthology.orchestra.run.smithi042:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.42 --addv c '[v2:172.21.15.42:3301,v1:172.21.15.42:6790]' --add b 172.21.15.43 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-14T23:55:26.876 INFO:teuthology.orchestra.run.smithi042.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-14T23:55:26.876 INFO:teuthology.orchestra.run.smithi042.stdout:monmaptool: generated fsid 21916fcc-f95c-4bfa-8b8b-11ca45845613 2024-09-14T23:55:26.876 INFO:teuthology.orchestra.run.smithi042.stdout:setting min_mon_release = quincy 2024-09-14T23:55:26.876 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 0 2024-09-14T23:55:26.876 INFO:teuthology.orchestra.run.smithi042.stdout:fsid 21916fcc-f95c-4bfa-8b8b-11ca45845613 2024-09-14T23:55:26.876 INFO:teuthology.orchestra.run.smithi042.stdout:last_changed 2024-09-14T23:55:26.875387+0000 2024-09-14T23:55:26.876 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-09-14T23:55:26.875387+0000 2024-09-14T23:55:26.877 INFO:teuthology.orchestra.run.smithi042.stdout:min_mon_release 17 (quincy) 2024-09-14T23:55:26.877 INFO:teuthology.orchestra.run.smithi042.stdout:election_strategy: 1 2024-09-14T23:55:26.877 INFO:teuthology.orchestra.run.smithi042.stdout:0: [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon.a 2024-09-14T23:55:26.877 INFO:teuthology.orchestra.run.smithi042.stdout:1: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.b 2024-09-14T23:55:26.877 INFO:teuthology.orchestra.run.smithi042.stdout:2: [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] mon.c 2024-09-14T23:55:26.877 INFO:teuthology.orchestra.run.smithi042.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-14T23:55:26.878 DEBUG:teuthology.orchestra.run.smithi042:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-14T23:55:26.893 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 21916fcc-f95c-4bfa-8b8b-11ca45845613... 2024-09-14T23:55:26.894 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-14T23:55:26.936 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-14T23:55:26.978 INFO:teuthology.orchestra.run.smithi042.stdout:[global] 2024-09-14T23:55:26.978 INFO:teuthology.orchestra.run.smithi042.stdout: chdir = "" 2024-09-14T23:55:26.978 INFO:teuthology.orchestra.run.smithi042.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-14T23:55:26.978 INFO:teuthology.orchestra.run.smithi042.stdout: auth supported = cephx 2024-09-14T23:55:26.978 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:26.978 INFO:teuthology.orchestra.run.smithi042.stdout: filestore xattr use omap = true 2024-09-14T23:55:26.978 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:26.978 INFO:teuthology.orchestra.run.smithi042.stdout: mon clock drift allowed = 1.000 2024-09-14T23:55:26.978 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:26.978 INFO:teuthology.orchestra.run.smithi042.stdout: osd crush chooseleaf type = 0 2024-09-14T23:55:26.978 INFO:teuthology.orchestra.run.smithi042.stdout: auth debug = true 2024-09-14T23:55:26.978 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:26.979 INFO:teuthology.orchestra.run.smithi042.stdout: ms die on old message = true 2024-09-14T23:55:26.979 INFO:teuthology.orchestra.run.smithi042.stdout: ms die on bug = true 2024-09-14T23:55:26.979 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:26.979 INFO:teuthology.orchestra.run.smithi042.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-14T23:55:26.979 INFO:teuthology.orchestra.run.smithi042.stdout: mon pg warn max object skew = 0 2024-09-14T23:55:26.979 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:26.979 INFO:teuthology.orchestra.run.smithi042.stdout: # disable pg_autoscaler by default for new pools 2024-09-14T23:55:26.979 INFO:teuthology.orchestra.run.smithi042.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-14T23:55:26.979 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:26.979 INFO:teuthology.orchestra.run.smithi042.stdout: osd pool default size = 2 2024-09-14T23:55:26.979 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:26.980 INFO:teuthology.orchestra.run.smithi042.stdout: mon osd allow primary affinity = true 2024-09-14T23:55:26.980 INFO:teuthology.orchestra.run.smithi042.stdout: mon osd allow pg remap = true 2024-09-14T23:55:26.980 INFO:teuthology.orchestra.run.smithi042.stdout: mon warn on legacy crush tunables = false 2024-09-14T23:55:26.980 INFO:teuthology.orchestra.run.smithi042.stdout: mon warn on crush straw calc version zero = false 2024-09-14T23:55:26.980 INFO:teuthology.orchestra.run.smithi042.stdout: mon warn on no sortbitwise = false 2024-09-14T23:55:26.980 INFO:teuthology.orchestra.run.smithi042.stdout: mon warn on osd down out interval zero = false 2024-09-14T23:55:26.980 INFO:teuthology.orchestra.run.smithi042.stdout: mon warn on too few osds = false 2024-09-14T23:55:26.980 INFO:teuthology.orchestra.run.smithi042.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-14T23:55:26.980 INFO:teuthology.orchestra.run.smithi042.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-14T23:55:26.980 INFO:teuthology.orchestra.run.smithi042.stdout: mon_allow_pool_size_one = true 2024-09-14T23:55:26.980 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:26.980 INFO:teuthology.orchestra.run.smithi042.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-14T23:55:26.981 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:26.981 INFO:teuthology.orchestra.run.smithi042.stdout: osd default data pool replay window = 5 2024-09-14T23:55:26.981 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:26.981 INFO:teuthology.orchestra.run.smithi042.stdout: mon allow pool delete = true 2024-09-14T23:55:26.981 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:26.981 INFO:teuthology.orchestra.run.smithi042.stdout: mon cluster log file level = debug 2024-09-14T23:55:26.981 INFO:teuthology.orchestra.run.smithi042.stdout: debug asserts on shutdown = true 2024-09-14T23:55:26.981 INFO:teuthology.orchestra.run.smithi042.stdout: mon health detail to clog = false 2024-09-14T23:55:26.981 INFO:teuthology.orchestra.run.smithi042.stdout: mon host = "172.21.15.42,[v2:172.21.15.42:3301,v1:172.21.15.42:6790],172.21.15.43" 2024-09-14T23:55:26.981 INFO:teuthology.orchestra.run.smithi042.stdout: mon client directed command retry = 5 2024-09-14T23:55:26.981 INFO:teuthology.orchestra.run.smithi042.stdout: mon warn on pool no app = False 2024-09-14T23:55:26.981 INFO:teuthology.orchestra.run.smithi042.stdout: ms inject socket failures = 5000 2024-09-14T23:55:26.981 INFO:teuthology.orchestra.run.smithi042.stdout: fsid = 21916fcc-f95c-4bfa-8b8b-11ca45845613 2024-09-14T23:55:26.982 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:26.982 INFO:teuthology.orchestra.run.smithi042.stdout:[osd] 2024-09-14T23:55:26.982 INFO:teuthology.orchestra.run.smithi042.stdout: osd journal size = 100 2024-09-14T23:55:26.982 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:26.982 INFO:teuthology.orchestra.run.smithi042.stdout: osd scrub load threshold = 5.0 2024-09-14T23:55:26.982 INFO:teuthology.orchestra.run.smithi042.stdout: osd scrub max interval = 600 2024-09-14T23:55:26.982 INFO:teuthology.orchestra.run.smithi042.stdout: osd mclock profile = high_recovery_ops 2024-09-14T23:55:26.982 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:26.982 INFO:teuthology.orchestra.run.smithi042.stdout: osd recover clone overlap = true 2024-09-14T23:55:26.982 INFO:teuthology.orchestra.run.smithi042.stdout: osd recovery max chunk = 1048576 2024-09-14T23:55:26.982 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:26.982 INFO:teuthology.orchestra.run.smithi042.stdout: osd debug shutdown = true 2024-09-14T23:55:26.982 INFO:teuthology.orchestra.run.smithi042.stdout: osd debug op order = true 2024-09-14T23:55:26.982 INFO:teuthology.orchestra.run.smithi042.stdout: osd debug verify stray on activate = true 2024-09-14T23:55:26.983 INFO:teuthology.orchestra.run.smithi042.stdout: osd debug trim objects = true 2024-09-14T23:55:26.983 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:26.983 INFO:teuthology.orchestra.run.smithi042.stdout: osd open classes on start = true 2024-09-14T23:55:26.983 INFO:teuthology.orchestra.run.smithi042.stdout: osd debug pg log writeout = true 2024-09-14T23:55:26.983 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:26.983 INFO:teuthology.orchestra.run.smithi042.stdout: osd deep scrub update digest min age = 30 2024-09-14T23:55:26.983 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:26.983 INFO:teuthology.orchestra.run.smithi042.stdout: osd map max advance = 10 2024-09-14T23:55:26.983 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:26.983 INFO:teuthology.orchestra.run.smithi042.stdout: journal zero on create = true 2024-09-14T23:55:26.983 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:26.983 INFO:teuthology.orchestra.run.smithi042.stdout: filestore ondisk finisher threads = 3 2024-09-14T23:55:26.984 INFO:teuthology.orchestra.run.smithi042.stdout: filestore apply finisher threads = 3 2024-09-14T23:55:26.984 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:26.984 INFO:teuthology.orchestra.run.smithi042.stdout: bdev debug aio = true 2024-09-14T23:55:26.984 INFO:teuthology.orchestra.run.smithi042.stdout: osd debug misdirected ops = true 2024-09-14T23:55:26.984 INFO:teuthology.orchestra.run.smithi042.stdout: bluestore block size = 96636764160 2024-09-14T23:55:26.984 INFO:teuthology.orchestra.run.smithi042.stdout: bluestore compression algorithm = snappy 2024-09-14T23:55:26.984 INFO:teuthology.orchestra.run.smithi042.stdout: bluestore compression mode = aggressive 2024-09-14T23:55:26.984 INFO:teuthology.orchestra.run.smithi042.stdout: bluestore fsck on mount = True 2024-09-14T23:55:26.984 INFO:teuthology.orchestra.run.smithi042.stdout: debug bluefs = 1/20 2024-09-14T23:55:26.984 INFO:teuthology.orchestra.run.smithi042.stdout: debug bluestore = 1/20 2024-09-14T23:55:26.984 INFO:teuthology.orchestra.run.smithi042.stdout: debug ms = 1 2024-09-14T23:55:26.984 INFO:teuthology.orchestra.run.smithi042.stdout: debug osd = 20 2024-09-14T23:55:26.984 INFO:teuthology.orchestra.run.smithi042.stdout: debug rocksdb = 4/10 2024-09-14T23:55:26.984 INFO:teuthology.orchestra.run.smithi042.stdout: mon osd backfillfull_ratio = 0.85 2024-09-14T23:55:26.985 INFO:teuthology.orchestra.run.smithi042.stdout: mon osd full ratio = 0.9 2024-09-14T23:55:26.985 INFO:teuthology.orchestra.run.smithi042.stdout: mon osd nearfull ratio = 0.8 2024-09-14T23:55:26.985 INFO:teuthology.orchestra.run.smithi042.stdout: osd blocked scrub grace period = 3600 2024-09-14T23:55:26.985 INFO:teuthology.orchestra.run.smithi042.stdout: osd failsafe full ratio = 0.95 2024-09-14T23:55:26.985 INFO:teuthology.orchestra.run.smithi042.stdout: osd max markdown count = 1000 2024-09-14T23:55:26.985 INFO:teuthology.orchestra.run.smithi042.stdout: osd objectstore = bluestore 2024-09-14T23:55:26.985 INFO:teuthology.orchestra.run.smithi042.stdout: osd shutdown pgref assert = True 2024-09-14T23:55:26.985 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:26.985 INFO:teuthology.orchestra.run.smithi042.stdout:[mgr] 2024-09-14T23:55:26.985 INFO:teuthology.orchestra.run.smithi042.stdout: debug ms = 1 2024-09-14T23:55:26.985 INFO:teuthology.orchestra.run.smithi042.stdout: debug mgr = 20 2024-09-14T23:55:26.985 INFO:teuthology.orchestra.run.smithi042.stdout: debug mon = 20 2024-09-14T23:55:26.985 INFO:teuthology.orchestra.run.smithi042.stdout: debug auth = 20 2024-09-14T23:55:26.985 INFO:teuthology.orchestra.run.smithi042.stdout: mon reweight min pgs per osd = 4 2024-09-14T23:55:26.986 INFO:teuthology.orchestra.run.smithi042.stdout: mon reweight min bytes per osd = 10 2024-09-14T23:55:26.986 INFO:teuthology.orchestra.run.smithi042.stdout: mgr/telemetry/nag = false 2024-09-14T23:55:26.986 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:26.986 INFO:teuthology.orchestra.run.smithi042.stdout:[mon] 2024-09-14T23:55:26.986 INFO:teuthology.orchestra.run.smithi042.stdout: debug ms = 1 2024-09-14T23:55:26.987 INFO:teuthology.orchestra.run.smithi042.stdout: debug mon = 20 2024-09-14T23:55:26.987 INFO:teuthology.orchestra.run.smithi042.stdout: debug paxos = 20 2024-09-14T23:55:26.987 INFO:teuthology.orchestra.run.smithi042.stdout: debug auth = 20 2024-09-14T23:55:26.987 INFO:teuthology.orchestra.run.smithi042.stdout: mon data avail warn = 5 2024-09-14T23:55:26.987 INFO:teuthology.orchestra.run.smithi042.stdout: mon mgr mkfs grace = 240 2024-09-14T23:55:26.987 INFO:teuthology.orchestra.run.smithi042.stdout: mon reweight min pgs per osd = 4 2024-09-14T23:55:26.987 INFO:teuthology.orchestra.run.smithi042.stdout: mon osd reporter subtree level = osd 2024-09-14T23:55:26.987 INFO:teuthology.orchestra.run.smithi042.stdout: mon osd prime pg temp = true 2024-09-14T23:55:26.987 INFO:teuthology.orchestra.run.smithi042.stdout: mon reweight min bytes per osd = 10 2024-09-14T23:55:26.987 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:26.987 INFO:teuthology.orchestra.run.smithi042.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-14T23:55:26.987 INFO:teuthology.orchestra.run.smithi042.stdout: auth mon ticket ttl = 660 # 11m 2024-09-14T23:55:26.987 INFO:teuthology.orchestra.run.smithi042.stdout: auth service ticket ttl = 240 # 4m 2024-09-14T23:55:26.987 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:26.987 INFO:teuthology.orchestra.run.smithi042.stdout: # don't complain about insecure global_id in the test suite 2024-09-14T23:55:26.988 INFO:teuthology.orchestra.run.smithi042.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-14T23:55:26.988 INFO:teuthology.orchestra.run.smithi042.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-14T23:55:26.988 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:26.988 INFO:teuthology.orchestra.run.smithi042.stdout: # 1m isn't quite enough 2024-09-14T23:55:26.988 INFO:teuthology.orchestra.run.smithi042.stdout: mon_down_mkfs_grace = 2m 2024-09-14T23:55:26.988 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:26.988 INFO:teuthology.orchestra.run.smithi042.stdout: mon_warn_on_filestore_osds = false 2024-09-14T23:55:26.988 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:55:26.988 INFO:teuthology.orchestra.run.smithi042.stdout:[client] 2024-09-14T23:55:26.988 INFO:teuthology.orchestra.run.smithi042.stdout: rgw cache enabled = true 2024-09-14T23:55:26.988 INFO:teuthology.orchestra.run.smithi042.stdout: rgw enable ops log = true 2024-09-14T23:55:26.988 INFO:teuthology.orchestra.run.smithi042.stdout: rgw enable usage log = true 2024-09-14T23:55:26.988 INFO:teuthology.orchestra.run.smithi042.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-14T23:55:26.988 INFO:teuthology.orchestra.run.smithi042.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-14T23:55:26.989 INFO:teuthology.orchestra.run.smithi042.stdout: rbd cache = False 2024-09-14T23:55:26.989 INFO:teuthology.orchestra.run.smithi042.stdout:[mon.a] 2024-09-14T23:55:26.989 INFO:teuthology.orchestra.run.smithi042.stdout:[mon.c] 2024-09-14T23:55:26.989 INFO:teuthology.orchestra.run.smithi042.stdout:[mon.b] 2024-09-14T23:55:26.989 INFO:teuthology.orchestra.run.smithi043.stdout:[global] 2024-09-14T23:55:26.989 INFO:teuthology.orchestra.run.smithi043.stdout: chdir = "" 2024-09-14T23:55:26.989 INFO:teuthology.orchestra.run.smithi043.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-14T23:55:26.989 INFO:teuthology.orchestra.run.smithi043.stdout: auth supported = cephx 2024-09-14T23:55:26.990 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:26.990 INFO:teuthology.orchestra.run.smithi043.stdout: filestore xattr use omap = true 2024-09-14T23:55:26.990 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:26.990 INFO:teuthology.orchestra.run.smithi043.stdout: mon clock drift allowed = 1.000 2024-09-14T23:55:26.991 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:26.991 INFO:teuthology.orchestra.run.smithi043.stdout: osd crush chooseleaf type = 0 2024-09-14T23:55:26.991 INFO:teuthology.orchestra.run.smithi043.stdout: auth debug = true 2024-09-14T23:55:26.991 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:26.991 INFO:teuthology.orchestra.run.smithi043.stdout: ms die on old message = true 2024-09-14T23:55:26.991 INFO:teuthology.orchestra.run.smithi043.stdout: ms die on bug = true 2024-09-14T23:55:26.991 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:26.991 INFO:teuthology.orchestra.run.smithi043.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-14T23:55:26.991 INFO:teuthology.orchestra.run.smithi043.stdout: mon pg warn max object skew = 0 2024-09-14T23:55:26.991 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:26.991 INFO:teuthology.orchestra.run.smithi043.stdout: # disable pg_autoscaler by default for new pools 2024-09-14T23:55:26.991 INFO:teuthology.orchestra.run.smithi043.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-14T23:55:26.991 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:26.992 INFO:teuthology.orchestra.run.smithi043.stdout: osd pool default size = 2 2024-09-14T23:55:26.992 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:26.992 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd allow primary affinity = true 2024-09-14T23:55:26.992 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd allow pg remap = true 2024-09-14T23:55:26.992 INFO:teuthology.orchestra.run.smithi043.stdout: mon warn on legacy crush tunables = false 2024-09-14T23:55:26.992 INFO:teuthology.orchestra.run.smithi043.stdout: mon warn on crush straw calc version zero = false 2024-09-14T23:55:26.992 INFO:teuthology.orchestra.run.smithi043.stdout: mon warn on no sortbitwise = false 2024-09-14T23:55:26.992 INFO:teuthology.orchestra.run.smithi043.stdout: mon warn on osd down out interval zero = false 2024-09-14T23:55:26.992 INFO:teuthology.orchestra.run.smithi043.stdout: mon warn on too few osds = false 2024-09-14T23:55:26.992 INFO:teuthology.orchestra.run.smithi043.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-14T23:55:26.992 INFO:teuthology.orchestra.run.smithi043.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-14T23:55:26.992 INFO:teuthology.orchestra.run.smithi043.stdout: mon_allow_pool_size_one = true 2024-09-14T23:55:26.992 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:26.993 INFO:teuthology.orchestra.run.smithi043.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-14T23:55:26.993 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:26.993 INFO:teuthology.orchestra.run.smithi043.stdout: osd default data pool replay window = 5 2024-09-14T23:55:26.993 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:26.993 INFO:teuthology.orchestra.run.smithi043.stdout: mon allow pool delete = true 2024-09-14T23:55:26.993 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:26.993 INFO:teuthology.orchestra.run.smithi043.stdout: mon cluster log file level = debug 2024-09-14T23:55:26.993 INFO:teuthology.orchestra.run.smithi043.stdout: debug asserts on shutdown = true 2024-09-14T23:55:26.993 INFO:teuthology.orchestra.run.smithi043.stdout: mon health detail to clog = false 2024-09-14T23:55:26.993 INFO:teuthology.orchestra.run.smithi043.stdout: mon host = "172.21.15.42,[v2:172.21.15.42:3301,v1:172.21.15.42:6790],172.21.15.43" 2024-09-14T23:55:26.993 INFO:teuthology.orchestra.run.smithi043.stdout: mon client directed command retry = 5 2024-09-14T23:55:26.993 INFO:teuthology.orchestra.run.smithi043.stdout: mon warn on pool no app = False 2024-09-14T23:55:26.993 INFO:teuthology.orchestra.run.smithi043.stdout: ms inject socket failures = 5000 2024-09-14T23:55:26.993 INFO:teuthology.orchestra.run.smithi043.stdout: fsid = 21916fcc-f95c-4bfa-8b8b-11ca45845613 2024-09-14T23:55:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:26.994 INFO:teuthology.orchestra.run.smithi043.stdout:[osd] 2024-09-14T23:55:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: osd journal size = 100 2024-09-14T23:55:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: osd scrub load threshold = 5.0 2024-09-14T23:55:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: osd scrub max interval = 600 2024-09-14T23:55:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: osd mclock profile = high_recovery_ops 2024-09-14T23:55:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: osd recover clone overlap = true 2024-09-14T23:55:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: osd recovery max chunk = 1048576 2024-09-14T23:55:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: osd debug shutdown = true 2024-09-14T23:55:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: osd debug op order = true 2024-09-14T23:55:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: osd debug verify stray on activate = true 2024-09-14T23:55:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: osd debug trim objects = true 2024-09-14T23:55:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: osd open classes on start = true 2024-09-14T23:55:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: osd debug pg log writeout = true 2024-09-14T23:55:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: osd deep scrub update digest min age = 30 2024-09-14T23:55:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: osd map max advance = 10 2024-09-14T23:55:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: journal zero on create = true 2024-09-14T23:55:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: filestore ondisk finisher threads = 3 2024-09-14T23:55:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: filestore apply finisher threads = 3 2024-09-14T23:55:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:26.996 INFO:teuthology.orchestra.run.smithi043.stdout: bdev debug aio = true 2024-09-14T23:55:26.996 INFO:teuthology.orchestra.run.smithi043.stdout: osd debug misdirected ops = true 2024-09-14T23:55:26.996 INFO:teuthology.orchestra.run.smithi043.stdout: bluestore block size = 96636764160 2024-09-14T23:55:26.996 INFO:teuthology.orchestra.run.smithi043.stdout: bluestore compression algorithm = snappy 2024-09-14T23:55:26.996 INFO:teuthology.orchestra.run.smithi043.stdout: bluestore compression mode = aggressive 2024-09-14T23:55:26.996 INFO:teuthology.orchestra.run.smithi043.stdout: bluestore fsck on mount = True 2024-09-14T23:55:26.996 INFO:teuthology.orchestra.run.smithi043.stdout: debug bluefs = 1/20 2024-09-14T23:55:26.996 INFO:teuthology.orchestra.run.smithi043.stdout: debug bluestore = 1/20 2024-09-14T23:55:26.996 INFO:teuthology.orchestra.run.smithi043.stdout: debug ms = 1 2024-09-14T23:55:26.996 INFO:teuthology.orchestra.run.smithi043.stdout: debug osd = 20 2024-09-14T23:55:26.996 INFO:teuthology.orchestra.run.smithi043.stdout: debug rocksdb = 4/10 2024-09-14T23:55:26.996 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd backfillfull_ratio = 0.85 2024-09-14T23:55:26.996 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd full ratio = 0.9 2024-09-14T23:55:26.996 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd nearfull ratio = 0.8 2024-09-14T23:55:26.997 INFO:teuthology.orchestra.run.smithi043.stdout: osd blocked scrub grace period = 3600 2024-09-14T23:55:26.997 INFO:teuthology.orchestra.run.smithi043.stdout: osd failsafe full ratio = 0.95 2024-09-14T23:55:26.997 INFO:teuthology.orchestra.run.smithi043.stdout: osd max markdown count = 1000 2024-09-14T23:55:26.997 INFO:teuthology.orchestra.run.smithi043.stdout: osd objectstore = bluestore 2024-09-14T23:55:26.997 INFO:teuthology.orchestra.run.smithi043.stdout: osd shutdown pgref assert = True 2024-09-14T23:55:26.997 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:26.997 INFO:teuthology.orchestra.run.smithi043.stdout:[mgr] 2024-09-14T23:55:26.997 INFO:teuthology.orchestra.run.smithi043.stdout: debug ms = 1 2024-09-14T23:55:26.997 INFO:teuthology.orchestra.run.smithi043.stdout: debug mgr = 20 2024-09-14T23:55:26.997 INFO:teuthology.orchestra.run.smithi043.stdout: debug mon = 20 2024-09-14T23:55:26.997 INFO:teuthology.orchestra.run.smithi043.stdout: debug auth = 20 2024-09-14T23:55:26.997 INFO:teuthology.orchestra.run.smithi043.stdout: mon reweight min pgs per osd = 4 2024-09-14T23:55:26.997 INFO:teuthology.orchestra.run.smithi043.stdout: mon reweight min bytes per osd = 10 2024-09-14T23:55:26.997 INFO:teuthology.orchestra.run.smithi043.stdout: mgr/telemetry/nag = false 2024-09-14T23:55:26.997 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:26.998 INFO:teuthology.orchestra.run.smithi043.stdout:[mon] 2024-09-14T23:55:26.998 INFO:teuthology.orchestra.run.smithi043.stdout: debug ms = 1 2024-09-14T23:55:26.998 INFO:teuthology.orchestra.run.smithi043.stdout: debug mon = 20 2024-09-14T23:55:26.998 INFO:teuthology.orchestra.run.smithi043.stdout: debug paxos = 20 2024-09-14T23:55:26.998 INFO:teuthology.orchestra.run.smithi043.stdout: debug auth = 20 2024-09-14T23:55:26.998 INFO:teuthology.orchestra.run.smithi043.stdout: mon data avail warn = 5 2024-09-14T23:55:26.998 INFO:teuthology.orchestra.run.smithi043.stdout: mon mgr mkfs grace = 240 2024-09-14T23:55:26.998 INFO:teuthology.orchestra.run.smithi043.stdout: mon reweight min pgs per osd = 4 2024-09-14T23:55:26.998 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd reporter subtree level = osd 2024-09-14T23:55:26.998 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd prime pg temp = true 2024-09-14T23:55:26.998 INFO:teuthology.orchestra.run.smithi043.stdout: mon reweight min bytes per osd = 10 2024-09-14T23:55:26.998 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:26.998 INFO:teuthology.orchestra.run.smithi043.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-14T23:55:26.998 INFO:teuthology.orchestra.run.smithi043.stdout: auth mon ticket ttl = 660 # 11m 2024-09-14T23:55:26.999 INFO:teuthology.orchestra.run.smithi043.stdout: auth service ticket ttl = 240 # 4m 2024-09-14T23:55:26.999 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:26.999 INFO:teuthology.orchestra.run.smithi043.stdout: # don't complain about insecure global_id in the test suite 2024-09-14T23:55:26.999 INFO:teuthology.orchestra.run.smithi043.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-14T23:55:26.999 INFO:teuthology.orchestra.run.smithi043.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-14T23:55:26.999 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:26.999 INFO:teuthology.orchestra.run.smithi043.stdout: # 1m isn't quite enough 2024-09-14T23:55:26.999 INFO:teuthology.orchestra.run.smithi043.stdout: mon_down_mkfs_grace = 2m 2024-09-14T23:55:26.999 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:26.999 INFO:teuthology.orchestra.run.smithi043.stdout: mon_warn_on_filestore_osds = false 2024-09-14T23:55:26.999 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-14T23:55:26.999 INFO:teuthology.orchestra.run.smithi043.stdout:[client] 2024-09-14T23:55:26.999 INFO:teuthology.orchestra.run.smithi043.stdout: rgw cache enabled = true 2024-09-14T23:55:26.999 INFO:teuthology.orchestra.run.smithi043.stdout: rgw enable ops log = true 2024-09-14T23:55:26.999 INFO:teuthology.orchestra.run.smithi043.stdout: rgw enable usage log = true 2024-09-14T23:55:27.000 INFO:teuthology.orchestra.run.smithi043.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-14T23:55:27.000 INFO:teuthology.orchestra.run.smithi043.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-14T23:55:27.000 INFO:teuthology.orchestra.run.smithi043.stdout: rbd cache = False 2024-09-14T23:55:27.000 INFO:teuthology.orchestra.run.smithi043.stdout:[mon.a] 2024-09-14T23:55:27.000 INFO:teuthology.orchestra.run.smithi043.stdout:[mon.c] 2024-09-14T23:55:27.000 INFO:teuthology.orchestra.run.smithi043.stdout:[mon.b] 2024-09-14T23:55:27.001 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-14T23:55:27.002 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-14T23:55:27.097 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-14T23:55:27.097 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-14T23:55:27.097 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-14T23:55:27.113 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-14T23:55:27.113 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-14T23:55:27.168 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi042.front.sepia.ceph.com 2024-09-14T23:55:27.168 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-14T23:55:27.168 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-14T23:55:27.168 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-14T23:55:27.244 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-14T23:55:27.244 DEBUG:teuthology.orchestra.run.smithi042:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-14T23:55:27.300 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi043.front.sepia.ceph.com 2024-09-14T23:55:27.300 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-14T23:55:27.301 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-14T23:55:27.301 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-14T23:55:27.343 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-14T23:55:27.343 DEBUG:teuthology.orchestra.run.smithi043:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-14T23:55:27.403 INFO:tasks.ceph:Setting up mon nodes... 2024-09-14T23:55:27.403 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-14T23:55:27.403 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-14T23:55:27.456 INFO:teuthology.orchestra.run.smithi042.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-14T23:55:27.458 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-14T23:55:27.530 INFO:teuthology.orchestra.run.smithi043.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-14T23:55:27.533 INFO:tasks.ceph:Setting up mds nodes... 2024-09-14T23:55:27.533 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-14T23:55:27.533 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-14T23:55:27.576 INFO:teuthology.orchestra.run.smithi042.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-14T23:55:27.589 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-14T23:55:27.634 INFO:teuthology.orchestra.run.smithi043.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-14T23:55:27.648 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-14T23:55:27.648 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi042.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi043.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-14T23:55:27.648 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-14T23:55:27.673 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-14T23:55:27.673 INFO:tasks.ceph:role: osd.0 2024-09-14T23:55:27.673 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi042.front.sepia.ceph.com 2024-09-14T23:55:27.674 DEBUG:teuthology.orchestra.run.smithi042:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-14T23:55:27.738 INFO:teuthology.orchestra.run.smithi042.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T23:55:27.766 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T23:55:27.766 INFO:teuthology.orchestra.run.smithi042.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T23:55:27.766 INFO:teuthology.orchestra.run.smithi042.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-14T23:55:27.766 INFO:teuthology.orchestra.run.smithi042.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T23:55:27.766 INFO:teuthology.orchestra.run.smithi042.stdout: = sunit=0 swidth=0 blks 2024-09-14T23:55:27.767 INFO:teuthology.orchestra.run.smithi042.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T23:55:27.767 INFO:teuthology.orchestra.run.smithi042.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-14T23:55:27.767 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T23:55:27.767 INFO:teuthology.orchestra.run.smithi042.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T23:55:27.772 INFO:teuthology.orchestra.run.smithi042.stdout:Discarding blocks...Done. 2024-09-14T23:55:27.774 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi042.front.sepia.ceph.com -o noatime 2024-09-14T23:55:27.774 DEBUG:teuthology.orchestra.run.smithi042:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-14T23:55:27.845 DEBUG:teuthology.orchestra.run.smithi042:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-14T23:55:27.915 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-14T23:55:27.982 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-14T23:55:27.982 INFO:tasks.ceph:role: osd.1 2024-09-14T23:55:27.982 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi042.front.sepia.ceph.com 2024-09-14T23:55:27.982 DEBUG:teuthology.orchestra.run.smithi042:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-14T23:55:28.047 INFO:teuthology.orchestra.run.smithi042.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T23:55:28.047 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T23:55:28.048 INFO:teuthology.orchestra.run.smithi042.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T23:55:28.048 INFO:teuthology.orchestra.run.smithi042.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-14T23:55:28.048 INFO:teuthology.orchestra.run.smithi042.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T23:55:28.048 INFO:teuthology.orchestra.run.smithi042.stdout: = sunit=0 swidth=0 blks 2024-09-14T23:55:28.048 INFO:teuthology.orchestra.run.smithi042.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T23:55:28.049 INFO:teuthology.orchestra.run.smithi042.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-14T23:55:28.049 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T23:55:28.049 INFO:teuthology.orchestra.run.smithi042.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T23:55:28.080 INFO:teuthology.orchestra.run.smithi042.stdout:Discarding blocks...Done. 2024-09-14T23:55:28.082 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi042.front.sepia.ceph.com -o noatime 2024-09-14T23:55:28.082 DEBUG:teuthology.orchestra.run.smithi042:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-14T23:55:28.153 DEBUG:teuthology.orchestra.run.smithi042:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-14T23:55:28.226 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-14T23:55:28.293 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-14T23:55:28.293 INFO:tasks.ceph:role: osd.2 2024-09-14T23:55:28.293 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi042.front.sepia.ceph.com 2024-09-14T23:55:28.293 DEBUG:teuthology.orchestra.run.smithi042:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-14T23:55:28.359 INFO:teuthology.orchestra.run.smithi042.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T23:55:28.359 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T23:55:28.359 INFO:teuthology.orchestra.run.smithi042.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T23:55:28.359 INFO:teuthology.orchestra.run.smithi042.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-14T23:55:28.359 INFO:teuthology.orchestra.run.smithi042.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T23:55:28.359 INFO:teuthology.orchestra.run.smithi042.stdout: = sunit=0 swidth=0 blks 2024-09-14T23:55:28.359 INFO:teuthology.orchestra.run.smithi042.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T23:55:28.359 INFO:teuthology.orchestra.run.smithi042.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-14T23:55:28.360 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T23:55:28.360 INFO:teuthology.orchestra.run.smithi042.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T23:55:28.392 INFO:teuthology.orchestra.run.smithi042.stdout:Discarding blocks...Done. 2024-09-14T23:55:28.393 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi042.front.sepia.ceph.com -o noatime 2024-09-14T23:55:28.394 DEBUG:teuthology.orchestra.run.smithi042:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-14T23:55:28.463 DEBUG:teuthology.orchestra.run.smithi042:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-14T23:55:28.536 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-14T23:55:28.603 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-14T23:55:28.603 INFO:tasks.ceph:role: osd.3 2024-09-14T23:55:28.604 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi042.front.sepia.ceph.com 2024-09-14T23:55:28.604 DEBUG:teuthology.orchestra.run.smithi042:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-14T23:55:28.669 INFO:teuthology.orchestra.run.smithi042.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T23:55:28.670 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T23:55:28.670 INFO:teuthology.orchestra.run.smithi042.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T23:55:28.670 INFO:teuthology.orchestra.run.smithi042.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-14T23:55:28.670 INFO:teuthology.orchestra.run.smithi042.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T23:55:28.670 INFO:teuthology.orchestra.run.smithi042.stdout: = sunit=0 swidth=0 blks 2024-09-14T23:55:28.670 INFO:teuthology.orchestra.run.smithi042.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T23:55:28.671 INFO:teuthology.orchestra.run.smithi042.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-14T23:55:28.671 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T23:55:28.671 INFO:teuthology.orchestra.run.smithi042.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T23:55:28.702 INFO:teuthology.orchestra.run.smithi042.stdout:Discarding blocks...Done. 2024-09-14T23:55:28.704 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi042.front.sepia.ceph.com -o noatime 2024-09-14T23:55:28.704 DEBUG:teuthology.orchestra.run.smithi042:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-14T23:55:28.774 DEBUG:teuthology.orchestra.run.smithi042:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-14T23:55:28.845 DEBUG:teuthology.orchestra.run.smithi042:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T23:55:28.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-14T23:55:28.941+0000 7f071820e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-14T23:55:28.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-14T23:55:28.942+0000 7f071820e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-14T23:55:28.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-14T23:55:28.942+0000 7f071820e640 -1 bdev(0x55b3f422b800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-14T23:55:28.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-14T23:55:28.942+0000 7f071820e640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-14T23:55:31.863 DEBUG:teuthology.orchestra.run.smithi042:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-14T23:55:31.894 DEBUG:teuthology.orchestra.run.smithi042:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T23:55:31.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-14T23:55:31.986+0000 7feefa230640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-14T23:55:31.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-14T23:55:31.986+0000 7feefa230640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-14T23:55:31.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-14T23:55:31.986+0000 7feefa230640 -1 bdev(0x55896c97f800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-14T23:55:31.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-14T23:55:31.986+0000 7feefa230640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-14T23:55:34.920 DEBUG:teuthology.orchestra.run.smithi042:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-14T23:55:34.952 DEBUG:teuthology.orchestra.run.smithi042:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T23:55:35.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-14T23:55:35.042+0000 7f1d237dd640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-14T23:55:35.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-14T23:55:35.042+0000 7f1d237dd640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-14T23:55:35.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-14T23:55:35.042+0000 7f1d237dd640 -1 bdev(0x561e19043800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-14T23:55:35.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-14T23:55:35.042+0000 7f1d237dd640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-14T23:55:37.955 DEBUG:teuthology.orchestra.run.smithi042:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-14T23:55:37.986 DEBUG:teuthology.orchestra.run.smithi042:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T23:55:38.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-14T23:55:38.076+0000 7f8d1ac13640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-14T23:55:38.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-14T23:55:38.076+0000 7f8d1ac13640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-14T23:55:38.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-14T23:55:38.077+0000 7f8d1ac13640 -1 bdev(0x555e0223f800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-14T23:55:38.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-14T23:55:38.077+0000 7f8d1ac13640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-14T23:55:41.240 DEBUG:teuthology.orchestra.run.smithi042:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-14T23:55:41.270 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-14T23:55:41.302 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-14T23:55:41.302 INFO:tasks.ceph:role: osd.4 2024-09-14T23:55:41.302 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi043.front.sepia.ceph.com 2024-09-14T23:55:41.302 DEBUG:teuthology.orchestra.run.smithi043:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-14T23:55:41.372 INFO:teuthology.orchestra.run.smithi043.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T23:55:41.373 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T23:55:41.373 INFO:teuthology.orchestra.run.smithi043.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T23:55:41.373 INFO:teuthology.orchestra.run.smithi043.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-14T23:55:41.373 INFO:teuthology.orchestra.run.smithi043.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T23:55:41.373 INFO:teuthology.orchestra.run.smithi043.stdout: = sunit=0 swidth=0 blks 2024-09-14T23:55:41.373 INFO:teuthology.orchestra.run.smithi043.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T23:55:41.373 INFO:teuthology.orchestra.run.smithi043.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-14T23:55:41.373 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T23:55:41.373 INFO:teuthology.orchestra.run.smithi043.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T23:55:41.405 INFO:teuthology.orchestra.run.smithi043.stdout:Discarding blocks...Done. 2024-09-14T23:55:41.407 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi043.front.sepia.ceph.com -o noatime 2024-09-14T23:55:41.407 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-14T23:55:41.480 DEBUG:teuthology.orchestra.run.smithi043:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-14T23:55:41.551 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-14T23:55:41.617 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-14T23:55:41.617 INFO:tasks.ceph:role: osd.5 2024-09-14T23:55:41.617 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi043.front.sepia.ceph.com 2024-09-14T23:55:41.617 DEBUG:teuthology.orchestra.run.smithi043:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-14T23:55:41.682 INFO:teuthology.orchestra.run.smithi043.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T23:55:41.682 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T23:55:41.683 INFO:teuthology.orchestra.run.smithi043.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T23:55:41.683 INFO:teuthology.orchestra.run.smithi043.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-14T23:55:41.683 INFO:teuthology.orchestra.run.smithi043.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T23:55:41.683 INFO:teuthology.orchestra.run.smithi043.stdout: = sunit=0 swidth=0 blks 2024-09-14T23:55:41.683 INFO:teuthology.orchestra.run.smithi043.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T23:55:41.683 INFO:teuthology.orchestra.run.smithi043.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-14T23:55:41.683 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T23:55:41.683 INFO:teuthology.orchestra.run.smithi043.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T23:55:41.716 INFO:teuthology.orchestra.run.smithi043.stdout:Discarding blocks...Done. 2024-09-14T23:55:41.718 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi043.front.sepia.ceph.com -o noatime 2024-09-14T23:55:41.718 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-14T23:55:41.788 DEBUG:teuthology.orchestra.run.smithi043:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-14T23:55:41.860 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-14T23:55:41.926 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-14T23:55:41.926 INFO:tasks.ceph:role: osd.6 2024-09-14T23:55:41.926 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi043.front.sepia.ceph.com 2024-09-14T23:55:41.926 DEBUG:teuthology.orchestra.run.smithi043:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-14T23:55:41.992 INFO:teuthology.orchestra.run.smithi043.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T23:55:41.992 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T23:55:41.993 INFO:teuthology.orchestra.run.smithi043.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T23:55:41.993 INFO:teuthology.orchestra.run.smithi043.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-14T23:55:41.993 INFO:teuthology.orchestra.run.smithi043.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T23:55:41.993 INFO:teuthology.orchestra.run.smithi043.stdout: = sunit=0 swidth=0 blks 2024-09-14T23:55:41.993 INFO:teuthology.orchestra.run.smithi043.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T23:55:41.993 INFO:teuthology.orchestra.run.smithi043.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-14T23:55:41.993 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T23:55:41.993 INFO:teuthology.orchestra.run.smithi043.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T23:55:42.025 INFO:teuthology.orchestra.run.smithi043.stdout:Discarding blocks...Done. 2024-09-14T23:55:42.027 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi043.front.sepia.ceph.com -o noatime 2024-09-14T23:55:42.027 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-14T23:55:42.097 DEBUG:teuthology.orchestra.run.smithi043:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-14T23:55:42.167 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-14T23:55:42.233 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-14T23:55:42.233 INFO:tasks.ceph:role: osd.7 2024-09-14T23:55:42.233 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi043.front.sepia.ceph.com 2024-09-14T23:55:42.233 DEBUG:teuthology.orchestra.run.smithi043:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-14T23:55:42.298 INFO:teuthology.orchestra.run.smithi043.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T23:55:42.299 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T23:55:42.299 INFO:teuthology.orchestra.run.smithi043.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T23:55:42.299 INFO:teuthology.orchestra.run.smithi043.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-14T23:55:42.299 INFO:teuthology.orchestra.run.smithi043.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T23:55:42.299 INFO:teuthology.orchestra.run.smithi043.stdout: = sunit=0 swidth=0 blks 2024-09-14T23:55:42.299 INFO:teuthology.orchestra.run.smithi043.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T23:55:42.299 INFO:teuthology.orchestra.run.smithi043.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-14T23:55:42.299 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T23:55:42.299 INFO:teuthology.orchestra.run.smithi043.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T23:55:42.331 INFO:teuthology.orchestra.run.smithi043.stdout:Discarding blocks...Done. 2024-09-14T23:55:42.333 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi043.front.sepia.ceph.com -o noatime 2024-09-14T23:55:42.333 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-14T23:55:42.402 DEBUG:teuthology.orchestra.run.smithi043:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-14T23:55:42.472 DEBUG:teuthology.orchestra.run.smithi043:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T23:55:42.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-14T23:55:42.569+0000 7fd1700b2640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-14T23:55:42.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-14T23:55:42.570+0000 7fd1700b2640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-14T23:55:42.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-14T23:55:42.570+0000 7fd1700b2640 -1 bdev(0x55c6fb37b800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-14T23:55:42.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-14T23:55:42.570+0000 7fd1700b2640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-14T23:55:45.484 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-14T23:55:45.514 DEBUG:teuthology.orchestra.run.smithi043:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T23:55:45.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-14T23:55:45.606+0000 7f251fe33640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-14T23:55:45.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-14T23:55:45.606+0000 7f251fe33640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-14T23:55:45.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-14T23:55:45.606+0000 7f251fe33640 -1 bdev(0x55d9840c7800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-14T23:55:45.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-14T23:55:45.607+0000 7f251fe33640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-14T23:55:48.774 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-14T23:55:48.805 DEBUG:teuthology.orchestra.run.smithi043:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T23:55:48.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-14T23:55:48.896+0000 7f7d15c15640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-14T23:55:48.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-14T23:55:48.896+0000 7f7d15c15640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-14T23:55:48.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-14T23:55:48.896+0000 7f7d15c15640 -1 bdev(0x560c56895800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-14T23:55:48.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-14T23:55:48.897+0000 7f7d15c15640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-14T23:55:52.558 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-14T23:55:52.589 DEBUG:teuthology.orchestra.run.smithi043:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T23:55:52.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-14T23:55:52.681+0000 7f2aff414640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-14T23:55:52.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-14T23:55:52.681+0000 7f2aff414640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-14T23:55:52.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-14T23:55:52.681+0000 7f2aff414640 -1 bdev(0x558d38eb5800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-14T23:55:52.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-14T23:55:52.681+0000 7f2aff414640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-14T23:55:55.850 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-14T23:55:55.881 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-14T23:55:55.881 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-14T23:55:55.881 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-14T23:55:55.912 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-14T23:55:55.913 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-14T23:55:55.982 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-14T23:55:55.982 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-14T23:55:56.050 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-14T23:55:56.051 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-14T23:55:56.118 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-14T23:55:56.118 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-14T23:55:56.185 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-14T23:55:56.185 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-14T23:55:56.216 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-14T23:55:56.216 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-14T23:55:56.286 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-14T23:55:56.286 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-14T23:55:56.356 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-14T23:55:56.356 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-14T23:55:56.425 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-14T23:55:56.426 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-14T23:55:56.494 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-14T23:55:56.494 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-14T23:55:56.514 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-14T23:55:56.515 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-14T23:55:56.551 INFO:tasks.ceph:Adding keys to all mons... 2024-09-14T23:55:56.552 DEBUG:teuthology.orchestra.run.smithi042:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-14T23:55:56.557 DEBUG:teuthology.orchestra.run.smithi043:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-14T23:55:56.594 INFO:teuthology.orchestra.run.smithi042.stdout:[mgr.y] 2024-09-14T23:55:56.594 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQBvIuZmh3AoGxAA8umuut2ALZpOreiOZcWdgw== 2024-09-14T23:55:56.595 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.0] 2024-09-14T23:55:56.595 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQBwIuZmk10yOBAA+NVuQBTyc7FeLgbfp2AtSA== 2024-09-14T23:55:56.595 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.1] 2024-09-14T23:55:56.595 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQBzIuZmtCjVOhAAUeg00fJzcvZeWBVE3vztMw== 2024-09-14T23:55:56.595 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.2] 2024-09-14T23:55:56.595 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQB3IuZm7t6XAhAAoHvNbV5WE2mZtdOLyf62+A== 2024-09-14T23:55:56.595 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.3] 2024-09-14T23:55:56.595 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQB6IuZmIqmgBBAAGicbFFtQcABVg05xkNEK5g== 2024-09-14T23:55:56.595 INFO:teuthology.orchestra.run.smithi042.stdout:[mgr.x] 2024-09-14T23:55:56.595 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQBvIuZmOleVHxAAdO0jd9/zQzl+pi81rfNqeA== 2024-09-14T23:55:56.595 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.4] 2024-09-14T23:55:56.596 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQB+IuZmckMDIhAADHFVNYXkpaghON9J3XpVpQ== 2024-09-14T23:55:56.596 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.5] 2024-09-14T23:55:56.596 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQCBIuZmf1s0JBAAeHuYza/qAi5BYtzVCYVl3g== 2024-09-14T23:55:56.596 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.6] 2024-09-14T23:55:56.596 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQCEIuZmX/t9NRAA+xiRbTAJUPUPEAAFB+FetQ== 2024-09-14T23:55:56.596 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.7] 2024-09-14T23:55:56.596 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQCIIuZmMVilKBAAB6CI7DTCxn8yvyQDmBoQoQ== 2024-09-14T23:55:56.596 INFO:teuthology.orchestra.run.smithi042.stdout:[client.0] 2024-09-14T23:55:56.596 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQBvIuZm2MJQIhAAS0y5tWHwEDLsEl1x0zzaAw== 2024-09-14T23:55:56.596 INFO:teuthology.orchestra.run.smithi042.stdout:[client.1] 2024-09-14T23:55:56.596 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQBvIuZmmcPIJRAAlKkd0nWsH8Tf5a2urMmvbA== 2024-09-14T23:55:56.617 INFO:teuthology.orchestra.run.smithi043.stdout:[mgr.y] 2024-09-14T23:55:56.617 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQBvIuZmh3AoGxAA8umuut2ALZpOreiOZcWdgw== 2024-09-14T23:55:56.617 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.0] 2024-09-14T23:55:56.617 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQBwIuZmk10yOBAA+NVuQBTyc7FeLgbfp2AtSA== 2024-09-14T23:55:56.617 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.1] 2024-09-14T23:55:56.617 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQBzIuZmtCjVOhAAUeg00fJzcvZeWBVE3vztMw== 2024-09-14T23:55:56.617 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.2] 2024-09-14T23:55:56.617 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQB3IuZm7t6XAhAAoHvNbV5WE2mZtdOLyf62+A== 2024-09-14T23:55:56.617 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.3] 2024-09-14T23:55:56.617 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQB6IuZmIqmgBBAAGicbFFtQcABVg05xkNEK5g== 2024-09-14T23:55:56.617 INFO:teuthology.orchestra.run.smithi043.stdout:[mgr.x] 2024-09-14T23:55:56.618 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQBvIuZmOleVHxAAdO0jd9/zQzl+pi81rfNqeA== 2024-09-14T23:55:56.618 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.4] 2024-09-14T23:55:56.618 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQB+IuZmckMDIhAADHFVNYXkpaghON9J3XpVpQ== 2024-09-14T23:55:56.618 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.5] 2024-09-14T23:55:56.618 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQCBIuZmf1s0JBAAeHuYza/qAi5BYtzVCYVl3g== 2024-09-14T23:55:56.618 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.6] 2024-09-14T23:55:56.618 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQCEIuZmX/t9NRAA+xiRbTAJUPUPEAAFB+FetQ== 2024-09-14T23:55:56.618 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.7] 2024-09-14T23:55:56.618 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQCIIuZmMVilKBAAB6CI7DTCxn8yvyQDmBoQoQ== 2024-09-14T23:55:56.618 INFO:teuthology.orchestra.run.smithi043.stdout:[client.0] 2024-09-14T23:55:56.618 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQBvIuZm2MJQIhAAS0y5tWHwEDLsEl1x0zzaAw== 2024-09-14T23:55:56.618 INFO:teuthology.orchestra.run.smithi043.stdout:[client.1] 2024-09-14T23:55:56.618 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQBvIuZmmcPIJRAAlKkd0nWsH8Tf5a2urMmvbA== 2024-09-14T23:55:56.619 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-14T23:55:56.640 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-14T23:55:56.714 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T23:55:56.716 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T23:55:56.769 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T23:55:56.772 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T23:55:56.824 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T23:55:56.826 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T23:55:56.877 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T23:55:56.880 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T23:55:56.932 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-14T23:55:56.934 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-14T23:55:56.986 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T23:55:56.989 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T23:55:57.040 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T23:55:57.043 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T23:55:57.095 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T23:55:57.097 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T23:55:57.149 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T23:55:57.151 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T23:55:57.203 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-14T23:55:57.206 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-14T23:55:57.261 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-14T23:55:57.263 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-14T23:55:57.318 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-14T23:55:57.318 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-14T23:55:57.343 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-14T23:55:57.441 DEBUG:teuthology.orchestra.run.smithi042:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-14T23:55:57.466 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-14T23:55:57.532 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-14T23:55:57.629 DEBUG:teuthology.orchestra.run.smithi042:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-14T23:55:57.654 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-14T23:55:57.681 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-14T23:55:57.780 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-14T23:55:57.805 DEBUG:teuthology.orchestra.run.smithi042:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-14T23:55:57.808 DEBUG:teuthology.orchestra.run.smithi043:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-14T23:55:57.861 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-14T23:55:57.861 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-14T23:55:57.861 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-14T23:55:57.864 INFO:tasks.ceph.mon.a:Started 2024-09-14T23:55:57.864 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-14T23:55:57.864 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-14T23:55:57.868 INFO:tasks.ceph.mon.c:Started 2024-09-14T23:55:57.869 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-14T23:55:57.869 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-14T23:55:57.905 INFO:tasks.ceph.mon.b:Started 2024-09-14T23:55:57.905 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-14T23:55:57.905 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-14T23:55:57.905 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-14T23:55:57.908 INFO:tasks.ceph.mgr.y:Started 2024-09-14T23:55:57.908 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-14T23:55:57.908 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-14T23:55:57.910 INFO:tasks.ceph.mgr.x:Started 2024-09-14T23:55:57.910 INFO:tasks.ceph:Setting crush tunables to default 2024-09-14T23:55:57.910 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --cluster ceph osd crush tunables default 2024-09-14T23:56:03.261 INFO:teuthology.orchestra.run.smithi042.stderr:adjusted tunables profile to default 2024-09-14T23:56:03.274 INFO:tasks.ceph:check_enable_crimson: False 2024-09-14T23:56:03.274 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-14T23:56:03.274 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-14T23:56:03.274 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-14T23:56:03.301 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-14T23:56:03.301 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-14T23:56:03.302 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-14T23:56:03.300+0000 7f88e9c16040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-14T23:56:03.327 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-14T23:56:03.327 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-14T23:56:03.346 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-14T23:56:03.344+0000 7f88e9c16040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-14T23:56:03.371 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-14T23:56:03.371 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-14T23:56:03.438 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-14T23:56:03.438 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-14T23:56:03.470 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-14T23:56:03.470 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-14T23:56:03.493 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-14T23:56:03.491+0000 7f88e9c16040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-14T23:56:03.541 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-14T23:56:03.541 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-14T23:56:03.546 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-14T23:56:03.544+0000 7f88e9c16040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-14T23:56:03.611 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-14T23:56:03.612 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-14T23:56:03.681 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph osd new db9d0c67-2d60-4592-9ad3-f99e5294f93f 0 2024-09-14T23:56:03.716 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-14T23:56:03.714+0000 7f88e9c16040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-14T23:56:03.831 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-14T23:56:03.830+0000 7f88e9c16040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-14T23:56:03.914 INFO:teuthology.orchestra.run.smithi043.stdout:0 2024-09-14T23:56:03.925 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph osd new 6c1a2765-c591-4c57-8ab3-ee6645970aa5 1 2024-09-14T23:56:04.096 INFO:tasks.ceph.mgr.x.smithi043.stderr:2024-09-14T23:56:04.094+0000 7fc1d485c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-14T23:56:04.121 INFO:teuthology.orchestra.run.smithi043.stdout:1 2024-09-14T23:56:04.132 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph osd new 930db893-7be6-4645-a7f1-e043c5ec95eb 2 2024-09-14T23:56:04.139 INFO:tasks.ceph.mgr.x.smithi043.stderr:2024-09-14T23:56:04.138+0000 7fc1d485c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-14T23:56:04.167 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-14T23:56:04.165+0000 7f88e9c16040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-14T23:56:04.224 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-14T23:56:04.221+0000 7f88e9c16040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-14T23:56:04.293 INFO:tasks.ceph.mgr.x.smithi043.stderr:2024-09-14T23:56:04.291+0000 7fc1d485c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-14T23:56:04.316 INFO:teuthology.orchestra.run.smithi043.stdout:2 2024-09-14T23:56:04.328 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph osd new 45bb3b84-b1c4-4f87-a7d1-0eb77b81741b 3 2024-09-14T23:56:04.347 INFO:tasks.ceph.mgr.x.smithi043.stderr:2024-09-14T23:56:04.346+0000 7fc1d485c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-14T23:56:04.381 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-14T23:56:04.379+0000 7f88e9c16040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-14T23:56:04.523 INFO:tasks.ceph.mgr.x.smithi043.stderr:2024-09-14T23:56:04.522+0000 7fc1d485c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-14T23:56:04.530 INFO:teuthology.orchestra.run.smithi043.stdout:3 2024-09-14T23:56:04.542 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph osd new a812e38f-c0f9-4d33-9373-1910fb09103c 4 2024-09-14T23:56:04.642 INFO:tasks.ceph.mgr.x.smithi043.stderr:2024-09-14T23:56:04.641+0000 7fc1d485c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-14T23:56:04.726 INFO:teuthology.orchestra.run.smithi043.stdout:4 2024-09-14T23:56:04.737 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph osd new 0de87e5b-7a21-4fb4-9543-d96c065786f8 5 2024-09-14T23:56:04.920 INFO:teuthology.orchestra.run.smithi043.stdout:5 2024-09-14T23:56:04.931 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph osd new c58c1d59-ad02-40f0-98e3-9078ef7e622b 6 2024-09-14T23:56:04.996 INFO:tasks.ceph.mgr.x.smithi043.stderr:2024-09-14T23:56:04.995+0000 7fc1d485c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-14T23:56:05.056 INFO:tasks.ceph.mgr.x.smithi043.stderr:2024-09-14T23:56:05.054+0000 7fc1d485c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-14T23:56:05.114 INFO:teuthology.orchestra.run.smithi043.stdout:6 2024-09-14T23:56:05.126 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph osd new 389bdb04-20e3-4a7a-8504-687e034bd7dc 7 2024-09-14T23:56:05.217 INFO:tasks.ceph.mgr.x.smithi043.stderr:2024-09-14T23:56:05.215+0000 7fc1d485c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-14T23:56:05.309 INFO:teuthology.orchestra.run.smithi043.stdout:7 2024-09-14T23:56:05.321 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-14T23:56:05.321 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-14T23:56:05.324 INFO:tasks.ceph.osd.0:Started 2024-09-14T23:56:05.324 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-14T23:56:05.324 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-14T23:56:05.326 INFO:tasks.ceph.osd.1:Started 2024-09-14T23:56:05.326 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-14T23:56:05.326 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-14T23:56:05.329 INFO:tasks.ceph.osd.2:Started 2024-09-14T23:56:05.329 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-14T23:56:05.329 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-14T23:56:05.331 INFO:tasks.ceph.osd.3:Started 2024-09-14T23:56:05.331 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-14T23:56:05.331 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-14T23:56:05.333 INFO:tasks.ceph.osd.4:Started 2024-09-14T23:56:05.334 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-14T23:56:05.334 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-14T23:56:05.336 INFO:tasks.ceph.osd.5:Started 2024-09-14T23:56:05.336 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-14T23:56:05.336 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-14T23:56:05.338 INFO:tasks.ceph.osd.6:Started 2024-09-14T23:56:05.338 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-14T23:56:05.339 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-14T23:56:05.341 INFO:tasks.ceph.osd.7:Started 2024-09-14T23:56:05.341 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-14T23:56:05.442 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-14T23:56:05.440+0000 7f88e9c16040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-14T23:56:05.499 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-14T23:56:05.498+0000 7f88e9c16040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-14T23:56:05.532 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:56:05.532 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":10,"fsid":"21916fcc-f95c-4bfa-8b8b-11ca45845613","created":"2024-09-14T23:56:03.167948+0000","modified":"2024-09-14T23:56:05.307021+0000","last_up_change":"0.000000","last_in_change":"2024-09-14T23:56:05.307021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"db9d0c67-2d60-4592-9ad3-f99e5294f93f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6c1a2765-c591-4c57-8ab3-ee6645970aa5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"930db893-7be6-4645-a7f1-e043c5ec95eb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"45bb3b84-b1c4-4f87-a7d1-0eb77b81741b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a812e38f-c0f9-4d33-9373-1910fb09103c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"0de87e5b-7a21-4fb4-9543-d96c065786f8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"c58c1d59-ad02-40f0-98e3-9078ef7e622b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"389bdb04-20e3-4a7a-8504-687e034bd7dc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T23:56:05.544 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-14T23:56:05.544 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-14T23:56:05.889 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-14T23:56:05.888+0000 7f88e9c16040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-14T23:56:05.979 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-14T23:56:05.977+0000 7f88e9c16040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-14T23:56:06.091 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-14T23:56:06.090+0000 7f88e9c16040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-14T23:56:06.140 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-14T23:56:06.138+0000 7f88e9c16040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-14T23:56:06.259 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-14T23:56:06.258+0000 7f88e9c16040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-14T23:56:06.264 INFO:tasks.ceph.mgr.x.smithi043.stderr:2024-09-14T23:56:06.263+0000 7fc1d485c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-14T23:56:06.288 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:56:06.287+0000 7f1617e0b640 -1 Falling back to public interface 2024-09-14T23:56:06.306 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:56:06.304+0000 7facb8b43640 -1 Falling back to public interface 2024-09-14T23:56:06.313 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:56:06.311+0000 7f34af5b2640 -1 Falling back to public interface 2024-09-14T23:56:06.315 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-14T23:56:06.314+0000 7f88e9c16040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-14T23:56:06.320 INFO:tasks.ceph.mgr.x.smithi043.stderr:2024-09-14T23:56:06.319+0000 7fc1d485c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-14T23:56:06.349 DEBUG:teuthology.orchestra.run.smithi042:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-14T23:56:06.363 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-14T23:56:06.362+0000 7f88e9c16040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-14T23:56:06.485 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-14T23:56:06.483+0000 7f88e9c16040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-14T23:56:06.510 INFO:teuthology.misc.health.smithi042.stdout: 2024-09-14T23:56:06.510 INFO:teuthology.misc.health.smithi042.stdout:{"epoch":10,"fsid":"21916fcc-f95c-4bfa-8b8b-11ca45845613","created":"2024-09-14T23:56:03.167948+0000","modified":"2024-09-14T23:56:05.307021+0000","last_up_change":"0.000000","last_in_change":"2024-09-14T23:56:05.307021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"db9d0c67-2d60-4592-9ad3-f99e5294f93f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6c1a2765-c591-4c57-8ab3-ee6645970aa5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"930db893-7be6-4645-a7f1-e043c5ec95eb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"45bb3b84-b1c4-4f87-a7d1-0eb77b81741b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a812e38f-c0f9-4d33-9373-1910fb09103c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"0de87e5b-7a21-4fb4-9543-d96c065786f8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"c58c1d59-ad02-40f0-98e3-9078ef7e622b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"389bdb04-20e3-4a7a-8504-687e034bd7dc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T23:56:06.520 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-14T23:56:06.538 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-14T23:56:06.536+0000 7f88e9c16040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-14T23:56:06.550 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:56:06.548+0000 7f9876a1d640 -1 Falling back to public interface 2024-09-14T23:56:06.593 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-14T23:56:06.591+0000 7f88e9c16040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-14T23:56:06.663 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-14T23:56:06.662+0000 7f88e9c16040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-14T23:56:06.706 INFO:tasks.ceph.mgr.x.smithi043.stderr:2024-09-14T23:56:06.705+0000 7fc1d485c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-14T23:56:06.710 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-14T23:56:06.708+0000 7f88e9c16040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-14T23:56:06.756 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-14T23:56:06.754+0000 7f88e9c16040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-14T23:56:06.787 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:56:06.785+0000 7ff2dab17640 -1 Falling back to public interface 2024-09-14T23:56:06.795 INFO:tasks.ceph.mgr.x.smithi043.stderr:2024-09-14T23:56:06.794+0000 7fc1d485c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-14T23:56:06.803 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:56:06.802+0000 7ff0c0615640 -1 Falling back to public interface 2024-09-14T23:56:06.911 INFO:tasks.ceph.mgr.x.smithi043.stderr:2024-09-14T23:56:06.909+0000 7fc1d485c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-14T23:56:06.959 INFO:tasks.ceph.mgr.x.smithi043.stderr:2024-09-14T23:56:06.957+0000 7fc1d485c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-14T23:56:07.034 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:56:07.032+0000 7f3319cef640 -1 Falling back to public interface 2024-09-14T23:56:07.077 INFO:tasks.ceph.mgr.x.smithi043.stderr:2024-09-14T23:56:07.076+0000 7fc1d485c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-14T23:56:07.133 INFO:tasks.ceph.mgr.x.smithi043.stderr:2024-09-14T23:56:07.132+0000 7fc1d485c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-14T23:56:07.180 INFO:tasks.ceph.mgr.x.smithi043.stderr:2024-09-14T23:56:07.179+0000 7fc1d485c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-14T23:56:07.278 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:56:07.276+0000 7face5019640 -1 Falling back to public interface 2024-09-14T23:56:07.301 INFO:tasks.ceph.mgr.x.smithi043.stderr:2024-09-14T23:56:07.296+0000 7fc1d485c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-14T23:56:07.350 INFO:tasks.ceph.mgr.x.smithi043.stderr:2024-09-14T23:56:07.349+0000 7fc1d485c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-14T23:56:07.404 INFO:tasks.ceph.mgr.x.smithi043.stderr:2024-09-14T23:56:07.403+0000 7fc1d485c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-14T23:56:07.444 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-14T23:56:07.442+0000 7f88e9c16040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-14T23:56:07.477 INFO:tasks.ceph.mgr.x.smithi043.stderr:2024-09-14T23:56:07.476+0000 7fc1d485c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-14T23:56:07.524 INFO:tasks.ceph.mgr.x.smithi043.stderr:2024-09-14T23:56:07.523+0000 7fc1d485c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-14T23:56:07.570 INFO:tasks.ceph.mgr.x.smithi043.stderr:2024-09-14T23:56:07.569+0000 7fc1d485c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-14T23:56:08.255 INFO:tasks.ceph.mgr.x.smithi043.stderr:2024-09-14T23:56:08.253+0000 7fc1d485c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-14T23:56:08.458 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-14T23:56:08.457+0000 7f888989b640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-14T23:56:08.459 INFO:tasks.ceph.mgr.y.smithi042.stderr:2024-09-14T23:56:08.457+0000 7f888989b640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-14T23:56:09.871 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:56:09.869+0000 7f34af5b2640 -1 osd.2 0 log_to_monitors true 2024-09-14T23:56:10.055 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:56:10.054+0000 7f9876a1d640 -1 osd.0 0 log_to_monitors true 2024-09-14T23:56:10.617 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:56:10.615+0000 7facb8b43640 -1 osd.1 0 log_to_monitors true 2024-09-14T23:56:10.622 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:56:10.621+0000 7f1617e0b640 -1 osd.6 0 log_to_monitors true 2024-09-14T23:56:10.822 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:56:10.820+0000 7ff2dab17640 -1 osd.4 0 log_to_monitors true 2024-09-14T23:56:10.875 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:56:10.874+0000 7ff0c0615640 -1 osd.5 0 log_to_monitors true 2024-09-14T23:56:11.040 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:56:11.038+0000 7face5019640 -1 osd.7 0 log_to_monitors true 2024-09-14T23:56:11.273 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:56:11.272+0000 7f3319cef640 -1 osd.3 0 log_to_monitors true 2024-09-14T23:56:12.176 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:56:12.175+0000 7f34ab541640 -1 osd.2 0 waiting for initial osdmap 2024-09-14T23:56:12.183 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:56:12.181+0000 7f9872408640 -1 osd.0 0 waiting for initial osdmap 2024-09-14T23:56:12.185 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:56:12.184+0000 7f34a3361640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T23:56:12.193 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:56:12.192+0000 7f986a228640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T23:56:12.947 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:56:12.945+0000 7facb4ad2640 -1 osd.1 0 waiting for initial osdmap 2024-09-14T23:56:12.965 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:56:12.964+0000 7facac8f2640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T23:56:13.313 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:56:13.311+0000 7ff0bc709640 -1 osd.5 0 waiting for initial osdmap 2024-09-14T23:56:13.321 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:56:13.319+0000 7f1613fa6640 -1 osd.6 0 waiting for initial osdmap 2024-09-14T23:56:13.325 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:56:13.322+0000 7ff2d6aa6640 -1 osd.4 0 waiting for initial osdmap 2024-09-14T23:56:13.325 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:56:13.323+0000 7ff0b3d15640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T23:56:13.327 DEBUG:teuthology.orchestra.run.smithi042:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-14T23:56:13.332 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:56:13.330+0000 7f160b5b2640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T23:56:13.339 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:56:13.336+0000 7face0906640 -1 osd.7 0 waiting for initial osdmap 2024-09-14T23:56:13.339 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:56:13.337+0000 7ff2ce8c6640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T23:56:13.350 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:56:13.348+0000 7facd8726640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T23:56:13.765 INFO:teuthology.misc.health.smithi042.stdout: 2024-09-14T23:56:13.852 INFO:teuthology.misc.health.smithi042.stdout:{"epoch":14,"fsid":"21916fcc-f95c-4bfa-8b8b-11ca45845613","created":"2024-09-14T23:56:03.167948+0000","modified":"2024-09-14T23:56:13.188849+0000","last_up_change":"2024-09-14T23:56:13.188849+0000","last_in_change":"2024-09-14T23:56:05.307021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"db9d0c67-2d60-4592-9ad3-f99e5294f93f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":314865173},{"type":"v1","addr":"172.21.15.42:6817","nonce":314865173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":314865173},{"type":"v1","addr":"172.21.15.42:6819","nonce":314865173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":314865173},{"type":"v1","addr":"172.21.15.42:6823","nonce":314865173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":314865173},{"type":"v1","addr":"172.21.15.42:6821","nonce":314865173}]},"public_addr":"172.21.15.42:6817/314865173","cluster_addr":"172.21.15.42:6819/314865173","heartbeat_back_addr":"172.21.15.42:6823/314865173","heartbeat_front_addr":"172.21.15.42:6821/314865173","state":["exists","up"]},{"osd":1,"uuid":"6c1a2765-c591-4c57-8ab3-ee6645970aa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":3197912310},{"type":"v1","addr":"172.21.15.42:6801","nonce":3197912310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":3197912310},{"type":"v1","addr":"172.21.15.42:6803","nonce":3197912310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":3197912310},{"type":"v1","addr":"172.21.15.42:6807","nonce":3197912310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":3197912310},{"type":"v1","addr":"172.21.15.42:6805","nonce":3197912310}]},"public_addr":"172.21.15.42:6801/3197912310","cluster_addr":"172.21.15.42:6803/3197912310","heartbeat_back_addr":"172.21.15.42:6807/3197912310","heartbeat_front_addr":"172.21.15.42:6805/3197912310","state":["exists","up"]},{"osd":2,"uuid":"930db893-7be6-4645-a7f1-e043c5ec95eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":3241153660},{"type":"v1","addr":"172.21.15.42:6809","nonce":3241153660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":3241153660},{"type":"v1","addr":"172.21.15.42:6811","nonce":3241153660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":3241153660},{"type":"v1","addr":"172.21.15.42:6815","nonce":3241153660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":3241153660},{"type":"v1","addr":"172.21.15.42:6813","nonce":3241153660}]},"public_addr":"172.21.15.42:6809/3241153660","cluster_addr":"172.21.15.42:6811/3241153660","heartbeat_back_addr":"172.21.15.42:6815/3241153660","heartbeat_front_addr":"172.21.15.42:6813/3241153660","state":["exists","up"]},{"osd":3,"uuid":"45bb3b84-b1c4-4f87-a7d1-0eb77b81741b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a812e38f-c0f9-4d33-9373-1910fb09103c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"0de87e5b-7a21-4fb4-9543-d96c065786f8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"c58c1d59-ad02-40f0-98e3-9078ef7e622b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"389bdb04-20e3-4a7a-8504-687e034bd7dc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T23:56:10.895490+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T23:56:13.854 DEBUG:teuthology.misc:3 of 8 OSDs are up 2024-09-14T23:56:14.049 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:56:14.047+0000 7f3315c7e640 -1 osd.3 0 waiting for initial osdmap 2024-09-14T23:56:14.063 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:56:14.062+0000 7f330da9e640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T23:56:20.661 DEBUG:teuthology.orchestra.run.smithi042:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-14T23:56:20.956 INFO:teuthology.misc.health.smithi042.stdout: 2024-09-14T23:56:20.956 INFO:teuthology.misc.health.smithi042.stdout:{"epoch":17,"fsid":"21916fcc-f95c-4bfa-8b8b-11ca45845613","created":"2024-09-14T23:56:03.167948+0000","modified":"2024-09-14T23:56:16.408319+0000","last_up_change":"2024-09-14T23:56:14.322882+0000","last_in_change":"2024-09-14T23:56:05.307021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-14T23:56:13.500197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"db9d0c67-2d60-4592-9ad3-f99e5294f93f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":314865173},{"type":"v1","addr":"172.21.15.42:6817","nonce":314865173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":314865173},{"type":"v1","addr":"172.21.15.42:6819","nonce":314865173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":314865173},{"type":"v1","addr":"172.21.15.42:6823","nonce":314865173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":314865173},{"type":"v1","addr":"172.21.15.42:6821","nonce":314865173}]},"public_addr":"172.21.15.42:6817/314865173","cluster_addr":"172.21.15.42:6819/314865173","heartbeat_back_addr":"172.21.15.42:6823/314865173","heartbeat_front_addr":"172.21.15.42:6821/314865173","state":["exists","up"]},{"osd":1,"uuid":"6c1a2765-c591-4c57-8ab3-ee6645970aa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":3197912310},{"type":"v1","addr":"172.21.15.42:6801","nonce":3197912310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":3197912310},{"type":"v1","addr":"172.21.15.42:6803","nonce":3197912310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":3197912310},{"type":"v1","addr":"172.21.15.42:6807","nonce":3197912310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":3197912310},{"type":"v1","addr":"172.21.15.42:6805","nonce":3197912310}]},"public_addr":"172.21.15.42:6801/3197912310","cluster_addr":"172.21.15.42:6803/3197912310","heartbeat_back_addr":"172.21.15.42:6807/3197912310","heartbeat_front_addr":"172.21.15.42:6805/3197912310","state":["exists","up"]},{"osd":2,"uuid":"930db893-7be6-4645-a7f1-e043c5ec95eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":3241153660},{"type":"v1","addr":"172.21.15.42:6809","nonce":3241153660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":3241153660},{"type":"v1","addr":"172.21.15.42:6811","nonce":3241153660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":3241153660},{"type":"v1","addr":"172.21.15.42:6815","nonce":3241153660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":3241153660},{"type":"v1","addr":"172.21.15.42:6813","nonce":3241153660}]},"public_addr":"172.21.15.42:6809/3241153660","cluster_addr":"172.21.15.42:6811/3241153660","heartbeat_back_addr":"172.21.15.42:6815/3241153660","heartbeat_front_addr":"172.21.15.42:6813/3241153660","state":["exists","up"]},{"osd":3,"uuid":"45bb3b84-b1c4-4f87-a7d1-0eb77b81741b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":1736416326},{"type":"v1","addr":"172.21.15.42:6825","nonce":1736416326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1736416326},{"type":"v1","addr":"172.21.15.42:6827","nonce":1736416326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1736416326},{"type":"v1","addr":"172.21.15.42:6831","nonce":1736416326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1736416326},{"type":"v1","addr":"172.21.15.42:6829","nonce":1736416326}]},"public_addr":"172.21.15.42:6825/1736416326","cluster_addr":"172.21.15.42:6827/1736416326","heartbeat_back_addr":"172.21.15.42:6831/1736416326","heartbeat_front_addr":"172.21.15.42:6829/1736416326","state":["exists","up"]},{"osd":4,"uuid":"a812e38f-c0f9-4d33-9373-1910fb09103c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1755154427},{"type":"v1","addr":"172.21.15.43:6809","nonce":1755154427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1755154427},{"type":"v1","addr":"172.21.15.43:6811","nonce":1755154427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1755154427},{"type":"v1","addr":"172.21.15.43:6815","nonce":1755154427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1755154427},{"type":"v1","addr":"172.21.15.43:6813","nonce":1755154427}]},"public_addr":"172.21.15.43:6809/1755154427","cluster_addr":"172.21.15.43:6811/1755154427","heartbeat_back_addr":"172.21.15.43:6815/1755154427","heartbeat_front_addr":"172.21.15.43:6813/1755154427","state":["exists","up"]},{"osd":5,"uuid":"0de87e5b-7a21-4fb4-9543-d96c065786f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":3033769028},{"type":"v1","addr":"172.21.15.43:6817","nonce":3033769028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":3033769028},{"type":"v1","addr":"172.21.15.43:6819","nonce":3033769028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":3033769028},{"type":"v1","addr":"172.21.15.43:6823","nonce":3033769028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":3033769028},{"type":"v1","addr":"172.21.15.43:6821","nonce":3033769028}]},"public_addr":"172.21.15.43:6817/3033769028","cluster_addr":"172.21.15.43:6819/3033769028","heartbeat_back_addr":"172.21.15.43:6823/3033769028","heartbeat_front_addr":"172.21.15.43:6821/3033769028","state":["exists","up"]},{"osd":6,"uuid":"c58c1d59-ad02-40f0-98e3-9078ef7e622b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3314282416},{"type":"v1","addr":"172.21.15.43:6801","nonce":3314282416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3314282416},{"type":"v1","addr":"172.21.15.43:6803","nonce":3314282416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":3314282416},{"type":"v1","addr":"172.21.15.43:6807","nonce":3314282416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3314282416},{"type":"v1","addr":"172.21.15.43:6805","nonce":3314282416}]},"public_addr":"172.21.15.43:6801/3314282416","cluster_addr":"172.21.15.43:6803/3314282416","heartbeat_back_addr":"172.21.15.43:6807/3314282416","heartbeat_front_addr":"172.21.15.43:6805/3314282416","state":["exists","up"]},{"osd":7,"uuid":"389bdb04-20e3-4a7a-8504-687e034bd7dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1481350118},{"type":"v1","addr":"172.21.15.43:6825","nonce":1481350118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":1481350118},{"type":"v1","addr":"172.21.15.43:6827","nonce":1481350118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":1481350118},{"type":"v1","addr":"172.21.15.43:6831","nonce":1481350118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1481350118},{"type":"v1","addr":"172.21.15.43:6829","nonce":1481350118}]},"public_addr":"172.21.15.43:6825/1481350118","cluster_addr":"172.21.15.43:6827/1481350118","heartbeat_back_addr":"172.21.15.43:6831/1481350118","heartbeat_front_addr":"172.21.15.43:6829/1481350118","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T23:56:11.077810+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T23:56:11.578838+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T23:56:10.895490+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T23:56:12.290821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T23:56:11.800581+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T23:56:11.883155+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T23:56:11.604790+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T23:56:12.067759+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T23:56:20.967 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-14T23:56:20.967 INFO:tasks.ceph:Creating RBD pool 2024-09-14T23:56:20.967 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-14T23:56:21.517 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'rbd' created 2024-09-14T23:56:21.532 DEBUG:teuthology.orchestra.run.smithi042:> rbd --cluster ceph pool init rbd 2024-09-14T23:56:24.544 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-14T23:56:24.545 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-14T23:56:24.545 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-14T23:56:24.545 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-14T23:56:24.545 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-14T23:56:24.900 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:56:24.910 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":5,"flags":0,"active_gid":4104,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":292069327},{"type":"v1","addr":"172.21.15.42:6833","nonce":292069327}]},"active_addr":"172.21.15.42:6833/292069327","active_change":"2024-09-14T23:56:07.444505+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4120,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.42:0","nonce":2172445555}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.42:0","nonce":1617902543}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.42:0","nonce":2542269203}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.42:0","nonce":108263539}]}]} 2024-09-14T23:56:24.918 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-14T23:56:24.918 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-14T23:56:24.918 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-14T23:56:25.212 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:56:25.212 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":21,"fsid":"21916fcc-f95c-4bfa-8b8b-11ca45845613","created":"2024-09-14T23:56:03.167948+0000","modified":"2024-09-14T23:56:24.527428+0000","last_up_change":"2024-09-14T23:56:14.322882+0000","last_in_change":"2024-09-14T23:56:05.307021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-14T23:56:13.500197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-14T23:56:21.284705+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"db9d0c67-2d60-4592-9ad3-f99e5294f93f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":314865173},{"type":"v1","addr":"172.21.15.42:6817","nonce":314865173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":314865173},{"type":"v1","addr":"172.21.15.42:6819","nonce":314865173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":314865173},{"type":"v1","addr":"172.21.15.42:6823","nonce":314865173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":314865173},{"type":"v1","addr":"172.21.15.42:6821","nonce":314865173}]},"public_addr":"172.21.15.42:6817/314865173","cluster_addr":"172.21.15.42:6819/314865173","heartbeat_back_addr":"172.21.15.42:6823/314865173","heartbeat_front_addr":"172.21.15.42:6821/314865173","state":["exists","up"]},{"osd":1,"uuid":"6c1a2765-c591-4c57-8ab3-ee6645970aa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":3197912310},{"type":"v1","addr":"172.21.15.42:6801","nonce":3197912310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":3197912310},{"type":"v1","addr":"172.21.15.42:6803","nonce":3197912310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":3197912310},{"type":"v1","addr":"172.21.15.42:6807","nonce":3197912310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":3197912310},{"type":"v1","addr":"172.21.15.42:6805","nonce":3197912310}]},"public_addr":"172.21.15.42:6801/3197912310","cluster_addr":"172.21.15.42:6803/3197912310","heartbeat_back_addr":"172.21.15.42:6807/3197912310","heartbeat_front_addr":"172.21.15.42:6805/3197912310","state":["exists","up"]},{"osd":2,"uuid":"930db893-7be6-4645-a7f1-e043c5ec95eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":3241153660},{"type":"v1","addr":"172.21.15.42:6809","nonce":3241153660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":3241153660},{"type":"v1","addr":"172.21.15.42:6811","nonce":3241153660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":3241153660},{"type":"v1","addr":"172.21.15.42:6815","nonce":3241153660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":3241153660},{"type":"v1","addr":"172.21.15.42:6813","nonce":3241153660}]},"public_addr":"172.21.15.42:6809/3241153660","cluster_addr":"172.21.15.42:6811/3241153660","heartbeat_back_addr":"172.21.15.42:6815/3241153660","heartbeat_front_addr":"172.21.15.42:6813/3241153660","state":["exists","up"]},{"osd":3,"uuid":"45bb3b84-b1c4-4f87-a7d1-0eb77b81741b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":1736416326},{"type":"v1","addr":"172.21.15.42:6825","nonce":1736416326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1736416326},{"type":"v1","addr":"172.21.15.42:6827","nonce":1736416326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1736416326},{"type":"v1","addr":"172.21.15.42:6831","nonce":1736416326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1736416326},{"type":"v1","addr":"172.21.15.42:6829","nonce":1736416326}]},"public_addr":"172.21.15.42:6825/1736416326","cluster_addr":"172.21.15.42:6827/1736416326","heartbeat_back_addr":"172.21.15.42:6831/1736416326","heartbeat_front_addr":"172.21.15.42:6829/1736416326","state":["exists","up"]},{"osd":4,"uuid":"a812e38f-c0f9-4d33-9373-1910fb09103c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1755154427},{"type":"v1","addr":"172.21.15.43:6809","nonce":1755154427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1755154427},{"type":"v1","addr":"172.21.15.43:6811","nonce":1755154427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1755154427},{"type":"v1","addr":"172.21.15.43:6815","nonce":1755154427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1755154427},{"type":"v1","addr":"172.21.15.43:6813","nonce":1755154427}]},"public_addr":"172.21.15.43:6809/1755154427","cluster_addr":"172.21.15.43:6811/1755154427","heartbeat_back_addr":"172.21.15.43:6815/1755154427","heartbeat_front_addr":"172.21.15.43:6813/1755154427","state":["exists","up"]},{"osd":5,"uuid":"0de87e5b-7a21-4fb4-9543-d96c065786f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":3033769028},{"type":"v1","addr":"172.21.15.43:6817","nonce":3033769028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":3033769028},{"type":"v1","addr":"172.21.15.43:6819","nonce":3033769028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":3033769028},{"type":"v1","addr":"172.21.15.43:6823","nonce":3033769028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":3033769028},{"type":"v1","addr":"172.21.15.43:6821","nonce":3033769028}]},"public_addr":"172.21.15.43:6817/3033769028","cluster_addr":"172.21.15.43:6819/3033769028","heartbeat_back_addr":"172.21.15.43:6823/3033769028","heartbeat_front_addr":"172.21.15.43:6821/3033769028","state":["exists","up"]},{"osd":6,"uuid":"c58c1d59-ad02-40f0-98e3-9078ef7e622b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3314282416},{"type":"v1","addr":"172.21.15.43:6801","nonce":3314282416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3314282416},{"type":"v1","addr":"172.21.15.43:6803","nonce":3314282416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":3314282416},{"type":"v1","addr":"172.21.15.43:6807","nonce":3314282416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3314282416},{"type":"v1","addr":"172.21.15.43:6805","nonce":3314282416}]},"public_addr":"172.21.15.43:6801/3314282416","cluster_addr":"172.21.15.43:6803/3314282416","heartbeat_back_addr":"172.21.15.43:6807/3314282416","heartbeat_front_addr":"172.21.15.43:6805/3314282416","state":["exists","up"]},{"osd":7,"uuid":"389bdb04-20e3-4a7a-8504-687e034bd7dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1481350118},{"type":"v1","addr":"172.21.15.43:6825","nonce":1481350118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":1481350118},{"type":"v1","addr":"172.21.15.43:6827","nonce":1481350118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":1481350118},{"type":"v1","addr":"172.21.15.43:6831","nonce":1481350118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1481350118},{"type":"v1","addr":"172.21.15.43:6829","nonce":1481350118}]},"public_addr":"172.21.15.43:6825/1481350118","cluster_addr":"172.21.15.43:6827/1481350118","heartbeat_back_addr":"172.21.15.43:6831/1481350118","heartbeat_front_addr":"172.21.15.43:6829/1481350118","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T23:56:11.077810+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T23:56:11.578838+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T23:56:10.895490+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T23:56:12.290821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T23:56:11.800581+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T23:56:11.883155+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T23:56:11.604790+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T23:56:12.067759+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T23:56:25.225 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-14T23:56:25.225 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-14T23:56:25.516 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:56:25.516 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":21,"fsid":"21916fcc-f95c-4bfa-8b8b-11ca45845613","created":"2024-09-14T23:56:03.167948+0000","modified":"2024-09-14T23:56:24.527428+0000","last_up_change":"2024-09-14T23:56:14.322882+0000","last_in_change":"2024-09-14T23:56:05.307021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-14T23:56:13.500197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-14T23:56:21.284705+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"db9d0c67-2d60-4592-9ad3-f99e5294f93f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":314865173},{"type":"v1","addr":"172.21.15.42:6817","nonce":314865173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":314865173},{"type":"v1","addr":"172.21.15.42:6819","nonce":314865173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":314865173},{"type":"v1","addr":"172.21.15.42:6823","nonce":314865173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":314865173},{"type":"v1","addr":"172.21.15.42:6821","nonce":314865173}]},"public_addr":"172.21.15.42:6817/314865173","cluster_addr":"172.21.15.42:6819/314865173","heartbeat_back_addr":"172.21.15.42:6823/314865173","heartbeat_front_addr":"172.21.15.42:6821/314865173","state":["exists","up"]},{"osd":1,"uuid":"6c1a2765-c591-4c57-8ab3-ee6645970aa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":3197912310},{"type":"v1","addr":"172.21.15.42:6801","nonce":3197912310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":3197912310},{"type":"v1","addr":"172.21.15.42:6803","nonce":3197912310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":3197912310},{"type":"v1","addr":"172.21.15.42:6807","nonce":3197912310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":3197912310},{"type":"v1","addr":"172.21.15.42:6805","nonce":3197912310}]},"public_addr":"172.21.15.42:6801/3197912310","cluster_addr":"172.21.15.42:6803/3197912310","heartbeat_back_addr":"172.21.15.42:6807/3197912310","heartbeat_front_addr":"172.21.15.42:6805/3197912310","state":["exists","up"]},{"osd":2,"uuid":"930db893-7be6-4645-a7f1-e043c5ec95eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":3241153660},{"type":"v1","addr":"172.21.15.42:6809","nonce":3241153660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":3241153660},{"type":"v1","addr":"172.21.15.42:6811","nonce":3241153660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":3241153660},{"type":"v1","addr":"172.21.15.42:6815","nonce":3241153660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":3241153660},{"type":"v1","addr":"172.21.15.42:6813","nonce":3241153660}]},"public_addr":"172.21.15.42:6809/3241153660","cluster_addr":"172.21.15.42:6811/3241153660","heartbeat_back_addr":"172.21.15.42:6815/3241153660","heartbeat_front_addr":"172.21.15.42:6813/3241153660","state":["exists","up"]},{"osd":3,"uuid":"45bb3b84-b1c4-4f87-a7d1-0eb77b81741b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":1736416326},{"type":"v1","addr":"172.21.15.42:6825","nonce":1736416326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1736416326},{"type":"v1","addr":"172.21.15.42:6827","nonce":1736416326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1736416326},{"type":"v1","addr":"172.21.15.42:6831","nonce":1736416326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1736416326},{"type":"v1","addr":"172.21.15.42:6829","nonce":1736416326}]},"public_addr":"172.21.15.42:6825/1736416326","cluster_addr":"172.21.15.42:6827/1736416326","heartbeat_back_addr":"172.21.15.42:6831/1736416326","heartbeat_front_addr":"172.21.15.42:6829/1736416326","state":["exists","up"]},{"osd":4,"uuid":"a812e38f-c0f9-4d33-9373-1910fb09103c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1755154427},{"type":"v1","addr":"172.21.15.43:6809","nonce":1755154427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1755154427},{"type":"v1","addr":"172.21.15.43:6811","nonce":1755154427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1755154427},{"type":"v1","addr":"172.21.15.43:6815","nonce":1755154427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1755154427},{"type":"v1","addr":"172.21.15.43:6813","nonce":1755154427}]},"public_addr":"172.21.15.43:6809/1755154427","cluster_addr":"172.21.15.43:6811/1755154427","heartbeat_back_addr":"172.21.15.43:6815/1755154427","heartbeat_front_addr":"172.21.15.43:6813/1755154427","state":["exists","up"]},{"osd":5,"uuid":"0de87e5b-7a21-4fb4-9543-d96c065786f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":3033769028},{"type":"v1","addr":"172.21.15.43:6817","nonce":3033769028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":3033769028},{"type":"v1","addr":"172.21.15.43:6819","nonce":3033769028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":3033769028},{"type":"v1","addr":"172.21.15.43:6823","nonce":3033769028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":3033769028},{"type":"v1","addr":"172.21.15.43:6821","nonce":3033769028}]},"public_addr":"172.21.15.43:6817/3033769028","cluster_addr":"172.21.15.43:6819/3033769028","heartbeat_back_addr":"172.21.15.43:6823/3033769028","heartbeat_front_addr":"172.21.15.43:6821/3033769028","state":["exists","up"]},{"osd":6,"uuid":"c58c1d59-ad02-40f0-98e3-9078ef7e622b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3314282416},{"type":"v1","addr":"172.21.15.43:6801","nonce":3314282416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3314282416},{"type":"v1","addr":"172.21.15.43:6803","nonce":3314282416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":3314282416},{"type":"v1","addr":"172.21.15.43:6807","nonce":3314282416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3314282416},{"type":"v1","addr":"172.21.15.43:6805","nonce":3314282416}]},"public_addr":"172.21.15.43:6801/3314282416","cluster_addr":"172.21.15.43:6803/3314282416","heartbeat_back_addr":"172.21.15.43:6807/3314282416","heartbeat_front_addr":"172.21.15.43:6805/3314282416","state":["exists","up"]},{"osd":7,"uuid":"389bdb04-20e3-4a7a-8504-687e034bd7dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1481350118},{"type":"v1","addr":"172.21.15.43:6825","nonce":1481350118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":1481350118},{"type":"v1","addr":"172.21.15.43:6827","nonce":1481350118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":1481350118},{"type":"v1","addr":"172.21.15.43:6831","nonce":1481350118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1481350118},{"type":"v1","addr":"172.21.15.43:6829","nonce":1481350118}]},"public_addr":"172.21.15.43:6825/1481350118","cluster_addr":"172.21.15.43:6827/1481350118","heartbeat_back_addr":"172.21.15.43:6831/1481350118","heartbeat_front_addr":"172.21.15.43:6829/1481350118","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T23:56:11.077810+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T23:56:11.578838+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T23:56:10.895490+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T23:56:12.290821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T23:56:11.800581+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T23:56:11.883155+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T23:56:11.604790+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T23:56:12.067759+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T23:56:25.532 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-14T23:56:25.532 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-14T23:56:25.533 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-14T23:56:25.533 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-14T23:56:25.534 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-14T23:56:25.534 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-14T23:56:25.535 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-14T23:56:25.535 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-14T23:56:25.780 INFO:teuthology.orchestra.run.smithi042.stdout:64424509446 2024-09-14T23:56:25.780 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-14T23:56:25.784 INFO:teuthology.orchestra.run.smithi042.stdout:64424509447 2024-09-14T23:56:25.784 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-14T23:56:25.789 INFO:teuthology.orchestra.run.smithi042.stdout:55834574852 2024-09-14T23:56:25.790 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-14T23:56:25.790 INFO:teuthology.orchestra.run.smithi042.stdout:64424509445 2024-09-14T23:56:25.791 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-14T23:56:25.792 INFO:teuthology.orchestra.run.smithi042.stdout:60129542148 2024-09-14T23:56:25.792 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-14T23:56:25.793 INFO:teuthology.orchestra.run.smithi042.stdout:64424509445 2024-09-14T23:56:25.793 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-14T23:56:25.794 INFO:teuthology.orchestra.run.smithi042.stdout:60129542149 2024-09-14T23:56:25.795 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-14T23:56:25.796 INFO:teuthology.orchestra.run.smithi042.stdout:64424509447 2024-09-14T23:56:25.796 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-14T23:56:26.387 INFO:teuthology.orchestra.run.smithi042.stdout:64424509445 2024-09-14T23:56:26.397 INFO:teuthology.orchestra.run.smithi042.stdout:60129542147 2024-09-14T23:56:26.406 INFO:teuthology.orchestra.run.smithi042.stdout:64424509446 2024-09-14T23:56:26.406 INFO:teuthology.orchestra.run.smithi042.stdout:64424509446 2024-09-14T23:56:26.407 INFO:teuthology.orchestra.run.smithi042.stdout:64424509444 2024-09-14T23:56:26.408 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.6 2024-09-14T23:56:26.413 INFO:teuthology.orchestra.run.smithi042.stdout:64424509444 2024-09-14T23:56:26.421 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.0 2024-09-14T23:56:26.426 INFO:teuthology.orchestra.run.smithi042.stdout:55834574851 2024-09-14T23:56:26.427 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.7 2024-09-14T23:56:26.428 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.4 2024-09-14T23:56:26.429 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509444 for osd.5 2024-09-14T23:56:26.433 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509444 for osd.3 2024-09-14T23:56:26.434 INFO:teuthology.orchestra.run.smithi042.stdout:60129542148 2024-09-14T23:56:26.444 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.2 2024-09-14T23:56:26.449 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.1 2024-09-14T23:56:27.409 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-14T23:56:27.421 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-14T23:56:27.429 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-14T23:56:27.429 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-14T23:56:27.430 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-14T23:56:27.434 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-14T23:56:27.445 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-14T23:56:27.450 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-14T23:56:28.005 INFO:teuthology.orchestra.run.smithi042.stdout:64424509446 2024-09-14T23:56:28.027 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.6 2024-09-14T23:56:28.028 DEBUG:teuthology.parallel:result is None 2024-09-14T23:56:28.044 INFO:teuthology.orchestra.run.smithi042.stdout:64424509447 2024-09-14T23:56:28.058 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.7 2024-09-14T23:56:28.058 DEBUG:teuthology.parallel:result is None 2024-09-14T23:56:28.069 INFO:teuthology.orchestra.run.smithi042.stdout:60129542148 2024-09-14T23:56:28.070 INFO:teuthology.orchestra.run.smithi042.stdout:64424509445 2024-09-14T23:56:28.075 INFO:teuthology.orchestra.run.smithi042.stdout:64424509445 2024-09-14T23:56:28.083 INFO:teuthology.orchestra.run.smithi042.stdout:60129542149 2024-09-14T23:56:28.084 INFO:teuthology.orchestra.run.smithi042.stdout:55834574853 2024-09-14T23:56:28.086 INFO:teuthology.orchestra.run.smithi042.stdout:64424509447 2024-09-14T23:56:28.092 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542148 for osd.0 2024-09-14T23:56:28.092 DEBUG:teuthology.parallel:result is None 2024-09-14T23:56:28.093 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509445 for osd.5 2024-09-14T23:56:28.093 DEBUG:teuthology.parallel:result is None 2024-09-14T23:56:28.098 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509445 for osd.3 2024-09-14T23:56:28.098 DEBUG:teuthology.parallel:result is None 2024-09-14T23:56:28.103 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574853 for osd.2 2024-09-14T23:56:28.103 DEBUG:teuthology.parallel:result is None 2024-09-14T23:56:28.104 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.4 2024-09-14T23:56:28.105 DEBUG:teuthology.parallel:result is None 2024-09-14T23:56:28.105 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.1 2024-09-14T23:56:28.105 DEBUG:teuthology.parallel:result is None 2024-09-14T23:56:28.105 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-14T23:56:28.105 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T23:56:28.431 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:56:28.432 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-14T23:56:28.443 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-14T23:56:27.954840+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":251296,"kb_used_data":1756,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754723424,"statfs":{"total":773094113280,"available":772836786176,"internally_reserved":0,"allocated":1798144,"data_stored":1745830,"data_compressed":64594,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.435177"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-14T23:56:24.727061+0000","last_change":"2024-09-14T23:56:24.727061+0000","last_active":"2024-09-14T23:56:24.727061+0000","last_peered":"2024-09-14T23:56:24.727061+0000","last_clean":"2024-09-14T23:56:24.727061+0000","last_became_active":"2024-09-14T23:56:22.526915+0000","last_became_peered":"2024-09-14T23:56:22.526915+0000","last_unstale":"2024-09-14T23:56:24.727061+0000","last_undegraded":"2024-09-14T23:56:24.727061+0000","last_fullsized":"2024-09-14T23:56:24.727061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:43:15.359266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045105900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-14T23:56:24.534313+0000","last_change":"2024-09-14T23:56:24.534632+0000","last_active":"2024-09-14T23:56:24.534313+0000","last_peered":"2024-09-14T23:56:24.534313+0000","last_clean":"2024-09-14T23:56:24.534313+0000","last_became_active":"2024-09-14T23:56:22.526355+0000","last_became_peered":"2024-09-14T23:56:22.526355+0000","last_unstale":"2024-09-14T23:56:24.534313+0000","last_undegraded":"2024-09-14T23:56:24.534313+0000","last_fullsized":"2024-09-14T23:56:24.534313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:02:16.059627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049910000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-14T23:56:25.093112+0000","last_change":"2024-09-14T23:56:25.093112+0000","last_active":"2024-09-14T23:56:25.093112+0000","last_peered":"2024-09-14T23:56:25.093112+0000","last_clean":"2024-09-14T23:56:25.093112+0000","last_became_active":"2024-09-14T23:56:22.528211+0000","last_became_peered":"2024-09-14T23:56:22.528211+0000","last_unstale":"2024-09-14T23:56:25.093112+0000","last_undegraded":"2024-09-14T23:56:25.093112+0000","last_fullsized":"2024-09-14T23:56:25.093112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:14:11.495286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000459686,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-14T23:56:24.534238+0000","last_change":"2024-09-14T23:56:24.534510+0000","last_active":"2024-09-14T23:56:24.534238+0000","last_peered":"2024-09-14T23:56:24.534238+0000","last_clean":"2024-09-14T23:56:24.534238+0000","last_became_active":"2024-09-14T23:56:22.527611+0000","last_became_peered":"2024-09-14T23:56:22.527611+0000","last_unstale":"2024-09-14T23:56:24.534238+0000","last_undegraded":"2024-09-14T23:56:24.534238+0000","last_fullsized":"2024-09-14T23:56:24.534238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:15:45.606026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049916299999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-14T23:56:24.538477+0000","last_change":"2024-09-14T23:56:24.537808+0000","last_active":"2024-09-14T23:56:24.538477+0000","last_peered":"2024-09-14T23:56:24.538477+0000","last_clean":"2024-09-14T23:56:24.538477+0000","last_became_active":"2024-09-14T23:56:22.526607+0000","last_became_peered":"2024-09-14T23:56:22.526607+0000","last_unstale":"2024-09-14T23:56:24.538477+0000","last_undegraded":"2024-09-14T23:56:24.538477+0000","last_fullsized":"2024-09-14T23:56:24.538477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:19:47.096566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096572499999999996,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-14T23:56:24.909098+0000","last_change":"2024-09-14T23:56:24.909260+0000","last_active":"2024-09-14T23:56:24.909098+0000","last_peered":"2024-09-14T23:56:24.909098+0000","last_clean":"2024-09-14T23:56:24.909098+0000","last_became_active":"2024-09-14T23:56:22.822598+0000","last_became_peered":"2024-09-14T23:56:22.822598+0000","last_unstale":"2024-09-14T23:56:24.909098+0000","last_undegraded":"2024-09-14T23:56:24.909098+0000","last_fullsized":"2024-09-14T23:56:24.909098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:32:32.568028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026863400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-14T23:56:25.093090+0000","last_change":"2024-09-14T23:56:25.093090+0000","last_active":"2024-09-14T23:56:25.093090+0000","last_peered":"2024-09-14T23:56:25.093090+0000","last_clean":"2024-09-14T23:56:25.093090+0000","last_became_active":"2024-09-14T23:56:22.528115+0000","last_became_peered":"2024-09-14T23:56:22.528115+0000","last_unstale":"2024-09-14T23:56:25.093090+0000","last_undegraded":"2024-09-14T23:56:25.093090+0000","last_fullsized":"2024-09-14T23:56:25.093090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:57:36.610730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036520999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-14T23:56:24.537302+0000","last_change":"2024-09-14T23:56:24.537302+0000","last_active":"2024-09-14T23:56:24.537302+0000","last_peered":"2024-09-14T23:56:24.537302+0000","last_clean":"2024-09-14T23:56:24.537302+0000","last_became_active":"2024-09-14T23:56:22.526493+0000","last_became_peered":"2024-09-14T23:56:22.526493+0000","last_unstale":"2024-09-14T23:56:24.537302+0000","last_undegraded":"2024-09-14T23:56:24.537302+0000","last_fullsized":"2024-09-14T23:56:24.537302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:08:16.451680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044655999999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-14T23:56:25.092625+0000","last_change":"2024-09-14T23:56:15.332936+0000","last_active":"2024-09-14T23:56:25.092625+0000","last_peered":"2024-09-14T23:56:25.092625+0000","last_clean":"2024-09-14T23:56:25.092625+0000","last_became_active":"2024-09-14T23:56:15.331994+0000","last_became_peered":"2024-09-14T23:56:15.331994+0000","last_unstale":"2024-09-14T23:56:25.092625+0000","last_undegraded":"2024-09-14T23:56:25.092625+0000","last_fullsized":"2024-09-14T23:56:25.092625+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:14.322882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:14.322882+0000","last_clean_scrub_stamp":"2024-09-14T23:56:14.322882+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:36:12.639000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64594,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31444,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340396,"statfs":{"total":96636764160,"available":96604565504,"internally_reserved":0,"allocated":262144,"data_stored":655897,"data_compressed":32297,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":208896,"data_stored":71361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":212992,"data_stored":71380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31400,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":208896,"data_stored":71361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":208896,"data_stored":71361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":212992,"data_stored":71380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542148,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":274432,"data_stored":661729,"data_compressed":32297,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":208896,"data_stored":71361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32297,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32297,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T23:56:28.444 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T23:56:28.730 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:56:28.731 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-14T23:56:28.742 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-14T23:56:27.954840+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":251296,"kb_used_data":1756,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754723424,"statfs":{"total":773094113280,"available":772836786176,"internally_reserved":0,"allocated":1798144,"data_stored":1745830,"data_compressed":64594,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.435177"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-14T23:56:24.727061+0000","last_change":"2024-09-14T23:56:24.727061+0000","last_active":"2024-09-14T23:56:24.727061+0000","last_peered":"2024-09-14T23:56:24.727061+0000","last_clean":"2024-09-14T23:56:24.727061+0000","last_became_active":"2024-09-14T23:56:22.526915+0000","last_became_peered":"2024-09-14T23:56:22.526915+0000","last_unstale":"2024-09-14T23:56:24.727061+0000","last_undegraded":"2024-09-14T23:56:24.727061+0000","last_fullsized":"2024-09-14T23:56:24.727061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:43:15.359266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045105900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-14T23:56:24.534313+0000","last_change":"2024-09-14T23:56:24.534632+0000","last_active":"2024-09-14T23:56:24.534313+0000","last_peered":"2024-09-14T23:56:24.534313+0000","last_clean":"2024-09-14T23:56:24.534313+0000","last_became_active":"2024-09-14T23:56:22.526355+0000","last_became_peered":"2024-09-14T23:56:22.526355+0000","last_unstale":"2024-09-14T23:56:24.534313+0000","last_undegraded":"2024-09-14T23:56:24.534313+0000","last_fullsized":"2024-09-14T23:56:24.534313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:02:16.059627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049910000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-14T23:56:25.093112+0000","last_change":"2024-09-14T23:56:25.093112+0000","last_active":"2024-09-14T23:56:25.093112+0000","last_peered":"2024-09-14T23:56:25.093112+0000","last_clean":"2024-09-14T23:56:25.093112+0000","last_became_active":"2024-09-14T23:56:22.528211+0000","last_became_peered":"2024-09-14T23:56:22.528211+0000","last_unstale":"2024-09-14T23:56:25.093112+0000","last_undegraded":"2024-09-14T23:56:25.093112+0000","last_fullsized":"2024-09-14T23:56:25.093112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:14:11.495286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000459686,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-14T23:56:24.534238+0000","last_change":"2024-09-14T23:56:24.534510+0000","last_active":"2024-09-14T23:56:24.534238+0000","last_peered":"2024-09-14T23:56:24.534238+0000","last_clean":"2024-09-14T23:56:24.534238+0000","last_became_active":"2024-09-14T23:56:22.527611+0000","last_became_peered":"2024-09-14T23:56:22.527611+0000","last_unstale":"2024-09-14T23:56:24.534238+0000","last_undegraded":"2024-09-14T23:56:24.534238+0000","last_fullsized":"2024-09-14T23:56:24.534238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:15:45.606026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049916299999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-14T23:56:24.538477+0000","last_change":"2024-09-14T23:56:24.537808+0000","last_active":"2024-09-14T23:56:24.538477+0000","last_peered":"2024-09-14T23:56:24.538477+0000","last_clean":"2024-09-14T23:56:24.538477+0000","last_became_active":"2024-09-14T23:56:22.526607+0000","last_became_peered":"2024-09-14T23:56:22.526607+0000","last_unstale":"2024-09-14T23:56:24.538477+0000","last_undegraded":"2024-09-14T23:56:24.538477+0000","last_fullsized":"2024-09-14T23:56:24.538477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:19:47.096566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096572499999999996,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-14T23:56:24.909098+0000","last_change":"2024-09-14T23:56:24.909260+0000","last_active":"2024-09-14T23:56:24.909098+0000","last_peered":"2024-09-14T23:56:24.909098+0000","last_clean":"2024-09-14T23:56:24.909098+0000","last_became_active":"2024-09-14T23:56:22.822598+0000","last_became_peered":"2024-09-14T23:56:22.822598+0000","last_unstale":"2024-09-14T23:56:24.909098+0000","last_undegraded":"2024-09-14T23:56:24.909098+0000","last_fullsized":"2024-09-14T23:56:24.909098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:32:32.568028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026863400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-14T23:56:25.093090+0000","last_change":"2024-09-14T23:56:25.093090+0000","last_active":"2024-09-14T23:56:25.093090+0000","last_peered":"2024-09-14T23:56:25.093090+0000","last_clean":"2024-09-14T23:56:25.093090+0000","last_became_active":"2024-09-14T23:56:22.528115+0000","last_became_peered":"2024-09-14T23:56:22.528115+0000","last_unstale":"2024-09-14T23:56:25.093090+0000","last_undegraded":"2024-09-14T23:56:25.093090+0000","last_fullsized":"2024-09-14T23:56:25.093090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:57:36.610730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036520999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-14T23:56:24.537302+0000","last_change":"2024-09-14T23:56:24.537302+0000","last_active":"2024-09-14T23:56:24.537302+0000","last_peered":"2024-09-14T23:56:24.537302+0000","last_clean":"2024-09-14T23:56:24.537302+0000","last_became_active":"2024-09-14T23:56:22.526493+0000","last_became_peered":"2024-09-14T23:56:22.526493+0000","last_unstale":"2024-09-14T23:56:24.537302+0000","last_undegraded":"2024-09-14T23:56:24.537302+0000","last_fullsized":"2024-09-14T23:56:24.537302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:08:16.451680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044655999999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-14T23:56:25.092625+0000","last_change":"2024-09-14T23:56:15.332936+0000","last_active":"2024-09-14T23:56:25.092625+0000","last_peered":"2024-09-14T23:56:25.092625+0000","last_clean":"2024-09-14T23:56:25.092625+0000","last_became_active":"2024-09-14T23:56:15.331994+0000","last_became_peered":"2024-09-14T23:56:15.331994+0000","last_unstale":"2024-09-14T23:56:25.092625+0000","last_undegraded":"2024-09-14T23:56:25.092625+0000","last_fullsized":"2024-09-14T23:56:25.092625+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:14.322882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:14.322882+0000","last_clean_scrub_stamp":"2024-09-14T23:56:14.322882+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:36:12.639000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64594,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31444,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340396,"statfs":{"total":96636764160,"available":96604565504,"internally_reserved":0,"allocated":262144,"data_stored":655897,"data_compressed":32297,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":208896,"data_stored":71361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":212992,"data_stored":71380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31400,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":208896,"data_stored":71361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":208896,"data_stored":71361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":212992,"data_stored":71380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542148,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":274432,"data_stored":661729,"data_compressed":32297,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":208896,"data_stored":71361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32297,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32297,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T23:56:28.743 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-14T23:56:28.743 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-14T23:56:28.743 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-14T23:56:28.744 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-14T23:56:29.070 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:56:29.070 INFO:teuthology.orchestra.run.smithi042.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-14T23:56:29.083 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-14T23:56:29.084 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-14T23:56:29.094 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-14T23:56:29.094 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'client': {'rbd cache': False}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50'}} 2024-09-14T23:56:29.094 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-14T23:56:29.094 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-14T23:56:29.400 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-14T23:56:29.400 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-14T23:56:29.400 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T23:56:29.687 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:56:29.688 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-14T23:56:29.699 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-14T23:56:29.453317+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":251296,"kb_used_data":1756,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754723424,"statfs":{"total":773094113280,"available":772836786176,"internally_reserved":0,"allocated":1798144,"data_stored":1745830,"data_compressed":64594,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001055"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-14T23:56:24.727061+0000","last_change":"2024-09-14T23:56:24.727061+0000","last_active":"2024-09-14T23:56:24.727061+0000","last_peered":"2024-09-14T23:56:24.727061+0000","last_clean":"2024-09-14T23:56:24.727061+0000","last_became_active":"2024-09-14T23:56:22.526915+0000","last_became_peered":"2024-09-14T23:56:22.526915+0000","last_unstale":"2024-09-14T23:56:24.727061+0000","last_undegraded":"2024-09-14T23:56:24.727061+0000","last_fullsized":"2024-09-14T23:56:24.727061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:43:15.359266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045105900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T23:56:28.023584+0000","last_change":"2024-09-14T23:56:24.534632+0000","last_active":"2024-09-14T23:56:28.023584+0000","last_peered":"2024-09-14T23:56:28.023584+0000","last_clean":"2024-09-14T23:56:28.023584+0000","last_became_active":"2024-09-14T23:56:22.526355+0000","last_became_peered":"2024-09-14T23:56:22.526355+0000","last_unstale":"2024-09-14T23:56:28.023584+0000","last_undegraded":"2024-09-14T23:56:28.023584+0000","last_fullsized":"2024-09-14T23:56:28.023584+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:02:16.059627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049910000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-14T23:56:25.093112+0000","last_change":"2024-09-14T23:56:25.093112+0000","last_active":"2024-09-14T23:56:25.093112+0000","last_peered":"2024-09-14T23:56:25.093112+0000","last_clean":"2024-09-14T23:56:25.093112+0000","last_became_active":"2024-09-14T23:56:22.528211+0000","last_became_peered":"2024-09-14T23:56:22.528211+0000","last_unstale":"2024-09-14T23:56:25.093112+0000","last_undegraded":"2024-09-14T23:56:25.093112+0000","last_fullsized":"2024-09-14T23:56:25.093112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:14:11.495286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000459686,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T23:56:28.023697+0000","last_change":"2024-09-14T23:56:24.534510+0000","last_active":"2024-09-14T23:56:28.023697+0000","last_peered":"2024-09-14T23:56:28.023697+0000","last_clean":"2024-09-14T23:56:28.023697+0000","last_became_active":"2024-09-14T23:56:22.527611+0000","last_became_peered":"2024-09-14T23:56:22.527611+0000","last_unstale":"2024-09-14T23:56:28.023697+0000","last_undegraded":"2024-09-14T23:56:28.023697+0000","last_fullsized":"2024-09-14T23:56:28.023697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:15:45.606026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049916299999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-14T23:56:24.538477+0000","last_change":"2024-09-14T23:56:24.537808+0000","last_active":"2024-09-14T23:56:24.538477+0000","last_peered":"2024-09-14T23:56:24.538477+0000","last_clean":"2024-09-14T23:56:24.538477+0000","last_became_active":"2024-09-14T23:56:22.526607+0000","last_became_peered":"2024-09-14T23:56:22.526607+0000","last_unstale":"2024-09-14T23:56:24.538477+0000","last_undegraded":"2024-09-14T23:56:24.538477+0000","last_fullsized":"2024-09-14T23:56:24.538477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:19:47.096566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096572499999999996,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-14T23:56:24.909098+0000","last_change":"2024-09-14T23:56:24.909260+0000","last_active":"2024-09-14T23:56:24.909098+0000","last_peered":"2024-09-14T23:56:24.909098+0000","last_clean":"2024-09-14T23:56:24.909098+0000","last_became_active":"2024-09-14T23:56:22.822598+0000","last_became_peered":"2024-09-14T23:56:22.822598+0000","last_unstale":"2024-09-14T23:56:24.909098+0000","last_undegraded":"2024-09-14T23:56:24.909098+0000","last_fullsized":"2024-09-14T23:56:24.909098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:32:32.568028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026863400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-14T23:56:25.093090+0000","last_change":"2024-09-14T23:56:25.093090+0000","last_active":"2024-09-14T23:56:25.093090+0000","last_peered":"2024-09-14T23:56:25.093090+0000","last_clean":"2024-09-14T23:56:25.093090+0000","last_became_active":"2024-09-14T23:56:22.528115+0000","last_became_peered":"2024-09-14T23:56:22.528115+0000","last_unstale":"2024-09-14T23:56:25.093090+0000","last_undegraded":"2024-09-14T23:56:25.093090+0000","last_fullsized":"2024-09-14T23:56:25.093090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:57:36.610730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036520999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-14T23:56:24.537302+0000","last_change":"2024-09-14T23:56:24.537302+0000","last_active":"2024-09-14T23:56:24.537302+0000","last_peered":"2024-09-14T23:56:24.537302+0000","last_clean":"2024-09-14T23:56:24.537302+0000","last_became_active":"2024-09-14T23:56:22.526493+0000","last_became_peered":"2024-09-14T23:56:22.526493+0000","last_unstale":"2024-09-14T23:56:24.537302+0000","last_undegraded":"2024-09-14T23:56:24.537302+0000","last_fullsized":"2024-09-14T23:56:24.537302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:08:16.451680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044655999999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-14T23:56:25.092625+0000","last_change":"2024-09-14T23:56:15.332936+0000","last_active":"2024-09-14T23:56:25.092625+0000","last_peered":"2024-09-14T23:56:25.092625+0000","last_clean":"2024-09-14T23:56:25.092625+0000","last_became_active":"2024-09-14T23:56:15.331994+0000","last_became_peered":"2024-09-14T23:56:15.331994+0000","last_unstale":"2024-09-14T23:56:25.092625+0000","last_undegraded":"2024-09-14T23:56:25.092625+0000","last_fullsized":"2024-09-14T23:56:25.092625+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:14.322882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:14.322882+0000","last_clean_scrub_stamp":"2024-09-14T23:56:14.322882+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:36:12.639000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64594,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31444,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340396,"statfs":{"total":96636764160,"available":96604565504,"internally_reserved":0,"allocated":262144,"data_stored":655897,"data_compressed":32297,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":208896,"data_stored":71361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":212992,"data_stored":71380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31400,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":208896,"data_stored":71361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":208896,"data_stored":71361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":212992,"data_stored":71380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":274432,"data_stored":661729,"data_compressed":32297,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":208896,"data_stored":71361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32297,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32297,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T23:56:29.700 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T23:56:29.989 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:56:29.990 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-14T23:56:30.001 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-14T23:56:29.453317+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":251296,"kb_used_data":1756,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754723424,"statfs":{"total":773094113280,"available":772836786176,"internally_reserved":0,"allocated":1798144,"data_stored":1745830,"data_compressed":64594,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001055"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-14T23:56:24.727061+0000","last_change":"2024-09-14T23:56:24.727061+0000","last_active":"2024-09-14T23:56:24.727061+0000","last_peered":"2024-09-14T23:56:24.727061+0000","last_clean":"2024-09-14T23:56:24.727061+0000","last_became_active":"2024-09-14T23:56:22.526915+0000","last_became_peered":"2024-09-14T23:56:22.526915+0000","last_unstale":"2024-09-14T23:56:24.727061+0000","last_undegraded":"2024-09-14T23:56:24.727061+0000","last_fullsized":"2024-09-14T23:56:24.727061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:43:15.359266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045105900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T23:56:28.023584+0000","last_change":"2024-09-14T23:56:24.534632+0000","last_active":"2024-09-14T23:56:28.023584+0000","last_peered":"2024-09-14T23:56:28.023584+0000","last_clean":"2024-09-14T23:56:28.023584+0000","last_became_active":"2024-09-14T23:56:22.526355+0000","last_became_peered":"2024-09-14T23:56:22.526355+0000","last_unstale":"2024-09-14T23:56:28.023584+0000","last_undegraded":"2024-09-14T23:56:28.023584+0000","last_fullsized":"2024-09-14T23:56:28.023584+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:02:16.059627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049910000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-14T23:56:25.093112+0000","last_change":"2024-09-14T23:56:25.093112+0000","last_active":"2024-09-14T23:56:25.093112+0000","last_peered":"2024-09-14T23:56:25.093112+0000","last_clean":"2024-09-14T23:56:25.093112+0000","last_became_active":"2024-09-14T23:56:22.528211+0000","last_became_peered":"2024-09-14T23:56:22.528211+0000","last_unstale":"2024-09-14T23:56:25.093112+0000","last_undegraded":"2024-09-14T23:56:25.093112+0000","last_fullsized":"2024-09-14T23:56:25.093112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:14:11.495286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000459686,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T23:56:28.023697+0000","last_change":"2024-09-14T23:56:24.534510+0000","last_active":"2024-09-14T23:56:28.023697+0000","last_peered":"2024-09-14T23:56:28.023697+0000","last_clean":"2024-09-14T23:56:28.023697+0000","last_became_active":"2024-09-14T23:56:22.527611+0000","last_became_peered":"2024-09-14T23:56:22.527611+0000","last_unstale":"2024-09-14T23:56:28.023697+0000","last_undegraded":"2024-09-14T23:56:28.023697+0000","last_fullsized":"2024-09-14T23:56:28.023697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:15:45.606026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049916299999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-14T23:56:24.538477+0000","last_change":"2024-09-14T23:56:24.537808+0000","last_active":"2024-09-14T23:56:24.538477+0000","last_peered":"2024-09-14T23:56:24.538477+0000","last_clean":"2024-09-14T23:56:24.538477+0000","last_became_active":"2024-09-14T23:56:22.526607+0000","last_became_peered":"2024-09-14T23:56:22.526607+0000","last_unstale":"2024-09-14T23:56:24.538477+0000","last_undegraded":"2024-09-14T23:56:24.538477+0000","last_fullsized":"2024-09-14T23:56:24.538477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:19:47.096566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096572499999999996,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-14T23:56:24.909098+0000","last_change":"2024-09-14T23:56:24.909260+0000","last_active":"2024-09-14T23:56:24.909098+0000","last_peered":"2024-09-14T23:56:24.909098+0000","last_clean":"2024-09-14T23:56:24.909098+0000","last_became_active":"2024-09-14T23:56:22.822598+0000","last_became_peered":"2024-09-14T23:56:22.822598+0000","last_unstale":"2024-09-14T23:56:24.909098+0000","last_undegraded":"2024-09-14T23:56:24.909098+0000","last_fullsized":"2024-09-14T23:56:24.909098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:32:32.568028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026863400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-14T23:56:25.093090+0000","last_change":"2024-09-14T23:56:25.093090+0000","last_active":"2024-09-14T23:56:25.093090+0000","last_peered":"2024-09-14T23:56:25.093090+0000","last_clean":"2024-09-14T23:56:25.093090+0000","last_became_active":"2024-09-14T23:56:22.528115+0000","last_became_peered":"2024-09-14T23:56:22.528115+0000","last_unstale":"2024-09-14T23:56:25.093090+0000","last_undegraded":"2024-09-14T23:56:25.093090+0000","last_fullsized":"2024-09-14T23:56:25.093090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:57:36.610730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036520999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-14T23:56:24.537302+0000","last_change":"2024-09-14T23:56:24.537302+0000","last_active":"2024-09-14T23:56:24.537302+0000","last_peered":"2024-09-14T23:56:24.537302+0000","last_clean":"2024-09-14T23:56:24.537302+0000","last_became_active":"2024-09-14T23:56:22.526493+0000","last_became_peered":"2024-09-14T23:56:22.526493+0000","last_unstale":"2024-09-14T23:56:24.537302+0000","last_undegraded":"2024-09-14T23:56:24.537302+0000","last_fullsized":"2024-09-14T23:56:24.537302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:08:16.451680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044655999999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-14T23:56:25.092625+0000","last_change":"2024-09-14T23:56:15.332936+0000","last_active":"2024-09-14T23:56:25.092625+0000","last_peered":"2024-09-14T23:56:25.092625+0000","last_clean":"2024-09-14T23:56:25.092625+0000","last_became_active":"2024-09-14T23:56:15.331994+0000","last_became_peered":"2024-09-14T23:56:15.331994+0000","last_unstale":"2024-09-14T23:56:25.092625+0000","last_undegraded":"2024-09-14T23:56:25.092625+0000","last_fullsized":"2024-09-14T23:56:25.092625+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:14.322882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:14.322882+0000","last_clean_scrub_stamp":"2024-09-14T23:56:14.322882+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:36:12.639000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64594,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31444,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340396,"statfs":{"total":96636764160,"available":96604565504,"internally_reserved":0,"allocated":262144,"data_stored":655897,"data_compressed":32297,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":208896,"data_stored":71361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":212992,"data_stored":71380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31400,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":208896,"data_stored":71361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":208896,"data_stored":71361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":212992,"data_stored":71380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":274432,"data_stored":661729,"data_compressed":32297,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":208896,"data_stored":71361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32297,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32297,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T23:56:30.002 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-14T23:56:30.002 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-14T23:56:30.292 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 22 2024-09-14T23:56:30.292 INFO:teuthology.orchestra.run.smithi042.stdout:fsid 21916fcc-f95c-4bfa-8b8b-11ca45845613 2024-09-14T23:56:30.292 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-09-14T23:56:03.167948+0000 2024-09-14T23:56:30.292 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-09-14T23:56:27.952051+0000 2024-09-14T23:56:30.292 INFO:teuthology.orchestra.run.smithi042.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-14T23:56:30.292 INFO:teuthology.orchestra.run.smithi042.stdout:crush_version 6 2024-09-14T23:56:30.292 INFO:teuthology.orchestra.run.smithi042.stdout:full_ratio 0.95 2024-09-14T23:56:30.292 INFO:teuthology.orchestra.run.smithi042.stdout:backfillfull_ratio 0.9 2024-09-14T23:56:30.292 INFO:teuthology.orchestra.run.smithi042.stdout:nearfull_ratio 0.85 2024-09-14T23:56:30.293 INFO:teuthology.orchestra.run.smithi042.stdout:require_min_compat_client luminous 2024-09-14T23:56:30.293 INFO:teuthology.orchestra.run.smithi042.stdout:min_compat_client jewel 2024-09-14T23:56:30.293 INFO:teuthology.orchestra.run.smithi042.stdout:require_osd_release squid 2024-09-14T23:56:30.293 INFO:teuthology.orchestra.run.smithi042.stdout:stretch_mode_enabled false 2024-09-14T23:56:30.293 INFO:teuthology.orchestra.run.smithi042.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 17 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 8.00 2024-09-14T23:56:30.293 INFO:teuthology.orchestra.run.smithi042.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 21 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 1.99 2024-09-14T23:56:30.293 INFO:teuthology.orchestra.run.smithi042.stdout:max_osd 8 2024-09-14T23:56:30.293 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6816/314865173,v1:172.21.15.42:6817/314865173] [v2:172.21.15.42:6818/314865173,v1:172.21.15.42:6819/314865173] exists,up db9d0c67-2d60-4592-9ad3-f99e5294f93f 2024-09-14T23:56:30.293 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 up in weight 1 up_from 14 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6800/3197912310,v1:172.21.15.42:6801/3197912310] [v2:172.21.15.42:6802/3197912310,v1:172.21.15.42:6803/3197912310] exists,up 6c1a2765-c591-4c57-8ab3-ee6645970aa5 2024-09-14T23:56:30.293 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6808/3241153660,v1:172.21.15.42:6809/3241153660] [v2:172.21.15.42:6810/3241153660,v1:172.21.15.42:6811/3241153660] exists,up 930db893-7be6-4645-a7f1-e043c5ec95eb 2024-09-14T23:56:30.293 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6824/1736416326,v1:172.21.15.42:6825/1736416326] [v2:172.21.15.42:6826/1736416326,v1:172.21.15.42:6827/1736416326] exists,up 45bb3b84-b1c4-4f87-a7d1-0eb77b81741b 2024-09-14T23:56:30.293 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.43:6808/1755154427,v1:172.21.15.43:6809/1755154427] [v2:172.21.15.43:6810/1755154427,v1:172.21.15.43:6811/1755154427] exists,up a812e38f-c0f9-4d33-9373-1910fb09103c 2024-09-14T23:56:30.293 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 up in weight 1 up_from 15 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.43:6816/3033769028,v1:172.21.15.43:6817/3033769028] [v2:172.21.15.43:6818/3033769028,v1:172.21.15.43:6819/3033769028] exists,up 0de87e5b-7a21-4fb4-9543-d96c065786f8 2024-09-14T23:56:30.294 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6 up in weight 1 up_from 15 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.43:6800/3314282416,v1:172.21.15.43:6801/3314282416] [v2:172.21.15.43:6802/3314282416,v1:172.21.15.43:6803/3314282416] exists,up c58c1d59-ad02-40f0-98e3-9078ef7e622b 2024-09-14T23:56:30.294 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7 up in weight 1 up_from 15 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.43:6824/1481350118,v1:172.21.15.43:6825/1481350118] [v2:172.21.15.43:6826/1481350118,v1:172.21.15.43:6827/1481350118] exists,up 389bdb04-20e3-4a7a-8504-687e034bd7dc 2024-09-14T23:56:30.304 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6816/314865173,v1:172.21.15.42:6817/314865173] [v2:172.21.15.42:6818/314865173,v1:172.21.15.42:6819/314865173] exists,up db9d0c67-2d60-4592-9ad3-f99e5294f93f', 'osd.1 up in weight 1 up_from 14 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6800/3197912310,v1:172.21.15.42:6801/3197912310] [v2:172.21.15.42:6802/3197912310,v1:172.21.15.42:6803/3197912310] exists,up 6c1a2765-c591-4c57-8ab3-ee6645970aa5', 'osd.2 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6808/3241153660,v1:172.21.15.42:6809/3241153660] [v2:172.21.15.42:6810/3241153660,v1:172.21.15.42:6811/3241153660] exists,up 930db893-7be6-4645-a7f1-e043c5ec95eb', 'osd.3 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6824/1736416326,v1:172.21.15.42:6825/1736416326] [v2:172.21.15.42:6826/1736416326,v1:172.21.15.42:6827/1736416326] exists,up 45bb3b84-b1c4-4f87-a7d1-0eb77b81741b', 'osd.4 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.43:6808/1755154427,v1:172.21.15.43:6809/1755154427] [v2:172.21.15.43:6810/1755154427,v1:172.21.15.43:6811/1755154427] exists,up a812e38f-c0f9-4d33-9373-1910fb09103c', 'osd.5 up in weight 1 up_from 15 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.43:6816/3033769028,v1:172.21.15.43:6817/3033769028] [v2:172.21.15.43:6818/3033769028,v1:172.21.15.43:6819/3033769028] exists,up 0de87e5b-7a21-4fb4-9543-d96c065786f8', 'osd.6 up in weight 1 up_from 15 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.43:6800/3314282416,v1:172.21.15.43:6801/3314282416] [v2:172.21.15.43:6802/3314282416,v1:172.21.15.43:6803/3314282416] exists,up c58c1d59-ad02-40f0-98e3-9078ef7e622b', 'osd.7 up in weight 1 up_from 15 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.43:6824/1481350118,v1:172.21.15.43:6825/1481350118] [v2:172.21.15.43:6826/1481350118,v1:172.21.15.43:6827/1481350118] exists,up 389bdb04-20e3-4a7a-8504-687e034bd7dc'] 2024-09-14T23:56:30.305 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-14T23:56:30.407 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:56:30.407 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mon.a", 2024-09-14T23:56:30.408 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster": "ceph", 2024-09-14T23:56:30.408 INFO:teuthology.orchestra.run.smithi042.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-14T23:56:30.408 INFO:teuthology.orchestra.run.smithi042.stdout: "admin_socket_mode": "", 2024-09-14T23:56:30.408 INFO:teuthology.orchestra.run.smithi042.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-14T23:56:30.408 INFO:teuthology.orchestra.run.smithi042.stdout: "auth_client_required": "cephx, none", 2024-09-14T23:56:30.408 INFO:teuthology.orchestra.run.smithi042.stdout: "auth_cluster_required": "cephx", 2024-09-14T23:56:30.408 INFO:teuthology.orchestra.run.smithi042.stdout: "auth_debug": "true", 2024-09-14T23:56:30.408 INFO:teuthology.orchestra.run.smithi042.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-14T23:56:30.408 INFO:teuthology.orchestra.run.smithi042.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-14T23:56:30.408 INFO:teuthology.orchestra.run.smithi042.stdout: "auth_service_required": "cephx", 2024-09-14T23:56:30.408 INFO:teuthology.orchestra.run.smithi042.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-14T23:56:30.408 INFO:teuthology.orchestra.run.smithi042.stdout: "auth_supported": "cephx", 2024-09-14T23:56:30.409 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_aio": "true", 2024-09-14T23:56:30.409 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-14T23:56:30.409 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_aio_poll_ms": "250", 2024-09-14T23:56:30.409 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_aio_reap_max": "16", 2024-09-14T23:56:30.409 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_async_discard": "false", 2024-09-14T23:56:30.409 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_block_size": "4096", 2024-09-14T23:56:30.409 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_debug_aio": "false", 2024-09-14T23:56:30.409 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-14T23:56:30.409 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-14T23:56:30.409 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_debug_inflight_ios": "false", 2024-09-14T23:56:30.409 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_enable_discard": "false", 2024-09-14T23:56:30.409 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_flock_retry": "3", 2024-09-14T23:56:30.409 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-14T23:56:30.410 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_inject_crash": "0", 2024-09-14T23:56:30.410 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-14T23:56:30.410 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_ioring": "false", 2024-09-14T23:56:30.410 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_ioring_hipri": "false", 2024-09-14T23:56:30.410 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-14T23:56:30.410 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-14T23:56:30.410 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-14T23:56:30.410 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-14T23:56:30.410 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_stalled_read_warn_lifetime": "86400", 2024-09-14T23:56:30.410 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_stalled_read_warn_threshold": "1", 2024-09-14T23:56:30.410 INFO:teuthology.orchestra.run.smithi042.stdout: "bdev_type": "", 2024-09-14T23:56:30.410 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_alloc_size": "1048576", 2024-09-14T23:56:30.410 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_allocator": "hybrid", 2024-09-14T23:56:30.410 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_buffered_io": "true", 2024-09-14T23:56:30.411 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_check_for_zeros": "false", 2024-09-14T23:56:30.411 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-14T23:56:30.411 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-14T23:56:30.411 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_compact_log_sync": "false", 2024-09-14T23:56:30.411 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-14T23:56:30.411 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-14T23:56:30.411 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-14T23:56:30.411 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-14T23:56:30.411 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_max_log_runway": "4194304", 2024-09-14T23:56:30.411 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_max_prefetch": "1048576", 2024-09-14T23:56:30.411 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_min_flush_size": "524288", 2024-09-14T23:56:30.411 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_min_log_runway": "1048576", 2024-09-14T23:56:30.411 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_replay_recovery": "false", 2024-09-14T23:56:30.412 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-14T23:56:30.412 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-14T23:56:30.412 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_sync_write": "false", 2024-09-14T23:56:30.412 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-14T23:56:30.412 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-14T23:56:30.412 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-14T23:56:30.412 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_allocation_from_file": "true", 2024-09-14T23:56:30.412 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_allocator": "hybrid", 2024-09-14T23:56:30.412 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-14T23:56:30.412 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-14T23:56:30.412 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-14T23:56:30.412 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-14T23:56:30.412 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_bdev_label_multi": "true", 2024-09-14T23:56:30.413 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_bdev_label_multi_upgrade": "false", 2024-09-14T23:56:30.413 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_bdev_label_require_all": "true", 2024-09-14T23:56:30.413 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-14T23:56:30.413 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-14T23:56:30.413 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-14T23:56:30.413 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_block_create": "true", 2024-09-14T23:56:30.413 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_block_db_create": "false", 2024-09-14T23:56:30.413 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_block_db_path": "", 2024-09-14T23:56:30.413 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_block_db_size": "0", 2024-09-14T23:56:30.413 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_block_path": "", 2024-09-14T23:56:30.413 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_block_preallocate_file": "false", 2024-09-14T23:56:30.413 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_block_size": "107374182400", 2024-09-14T23:56:30.413 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_block_wal_create": "false", 2024-09-14T23:56:30.414 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_block_wal_path": "", 2024-09-14T23:56:30.414 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_block_wal_size": "100663296", 2024-09-14T23:56:30.414 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_bluefs": "true", 2024-09-14T23:56:30.414 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-14T23:56:30.414 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-14T23:56:30.414 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-14T23:56:30.414 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-14T23:56:30.414 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-14T23:56:30.414 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-14T23:56:30.414 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-14T23:56:30.414 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-14T23:56:30.414 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_autotune": "true", 2024-09-14T23:56:30.414 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-14T23:56:30.414 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-14T23:56:30.415 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-14T23:56:30.415 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-14T23:56:30.415 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_size": "0", 2024-09-14T23:56:30.415 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-14T23:56:30.415 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-14T23:56:30.415 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-14T23:56:30.415 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-14T23:56:30.415 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_type": "2q", 2024-09-14T23:56:30.415 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-14T23:56:30.415 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_clone_cow": "true", 2024-09-14T23:56:30.415 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-14T23:56:30.415 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-14T23:56:30.415 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-14T23:56:30.416 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-14T23:56:30.416 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-14T23:56:30.416 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-14T23:56:30.416 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-14T23:56:30.416 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_compression_mode": "none", 2024-09-14T23:56:30.416 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-14T23:56:30.416 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_csum_type": "crc32c", 2024-09-14T23:56:30.416 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-14T23:56:30.416 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_freelist": "false", 2024-09-14T23:56:30.416 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-14T23:56:30.416 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-14T23:56:30.416 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-14T23:56:30.416 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-14T23:56:30.417 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-14T23:56:30.417 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-14T23:56:30.417 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-14T23:56:30.417 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-14T23:56:30.417 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-14T23:56:30.417 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-14T23:56:30.417 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-14T23:56:30.417 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-14T23:56:30.417 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_small_allocations": "0", 2024-09-14T23:56:30.417 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-14T23:56:30.417 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_default_buffered_read": "true", 2024-09-14T23:56:30.417 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_default_buffered_write": "false", 2024-09-14T23:56:30.417 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-14T23:56:30.417 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-14T23:56:30.418 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-14T23:56:30.418 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_elastic_shared_blobs": "true", 2024-09-14T23:56:30.418 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-14T23:56:30.418 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-14T23:56:30.418 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-14T23:56:30.418 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-14T23:56:30.418 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-14T23:56:30.418 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fail_eio": "false", 2024-09-14T23:56:30.418 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-14T23:56:30.418 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-14T23:56:30.418 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-14T23:56:30.418 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-14T23:56:30.418 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-14T23:56:30.418 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-14T23:56:30.419 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_on_mount": "false", 2024-09-14T23:56:30.419 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-14T23:56:30.419 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_on_umount": "false", 2024-09-14T23:56:30.419 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-14T23:56:30.419 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-14T23:56:30.419 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-14T23:56:30.419 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-14T23:56:30.419 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-14T23:56:30.419 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-14T23:56:30.419 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-14T23:56:30.419 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-14T23:56:30.419 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_ignore_data_csum": "false", 2024-09-14T23:56:30.419 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-14T23:56:30.420 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-14T23:56:30.420 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-14T23:56:30.420 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-14T23:56:30.420 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_log_op_age": "5.000000", 2024-09-14T23:56:30.420 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_max_alloc_size": "0", 2024-09-14T23:56:30.420 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_max_blob_size": "0", 2024-09-14T23:56:30.420 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-14T23:56:30.420 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-14T23:56:30.420 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-14T23:56:30.420 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_max_deferred_txc": "32", 2024-09-14T23:56:30.420 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_min_alloc_size": "0", 2024-09-14T23:56:30.420 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-14T23:56:30.420 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-14T23:56:30.420 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_nid_prealloc": "1024", 2024-09-14T23:56:30.421 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-14T23:56:30.421 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-14T23:56:30.421 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-14T23:56:30.421 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-14T23:56:30.421 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_retry_disk_reads": "3", 2024-09-14T23:56:30.421 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_rocksdb_cf": "true", 2024-09-14T23:56:30.421 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-14T23:56:30.421 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-14T23:56:30.421 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-14T23:56:30.421 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_slow_ops_warn_lifetime": "86400", 2024-09-14T23:56:30.421 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_slow_ops_warn_threshold": "1", 2024-09-14T23:56:30.422 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-14T23:56:30.422 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-14T23:56:30.422 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-14T23:56:30.422 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_spdk_mem": "512", 2024-09-14T23:56:30.422 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-14T23:56:30.422 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-14T23:56:30.422 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-14T23:56:30.422 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-14T23:56:30.422 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-14T23:56:30.422 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-14T23:56:30.422 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-14T23:56:30.422 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_tracing": "false", 2024-09-14T23:56:30.422 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-14T23:56:30.423 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-14T23:56:30.423 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-14T23:56:30.423 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-14T23:56:30.423 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-14T23:56:30.423 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-14T23:56:30.423 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-14T23:56:30.423 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-14T23:56:30.423 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-14T23:56:30.423 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_zero_block_detection": "false", 2024-09-14T23:56:30.423 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-14T23:56:30.423 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-14T23:56:30.423 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-14T23:56:30.423 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-14T23:56:30.423 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-14T23:56:30.423 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-14T23:56:30.424 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-14T23:56:30.424 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-14T23:56:30.424 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-14T23:56:30.424 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-14T23:56:30.424 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-14T23:56:30.424 INFO:teuthology.orchestra.run.smithi042.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-14T23:56:30.424 INFO:teuthology.orchestra.run.smithi042.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-14T23:56:30.424 INFO:teuthology.orchestra.run.smithi042.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-14T23:56:30.424 INFO:teuthology.orchestra.run.smithi042.stdout: "cephx_cluster_require_signatures": "false", 2024-09-14T23:56:30.424 INFO:teuthology.orchestra.run.smithi042.stdout: "cephx_cluster_require_version": "2", 2024-09-14T23:56:30.424 INFO:teuthology.orchestra.run.smithi042.stdout: "cephx_require_signatures": "false", 2024-09-14T23:56:30.424 INFO:teuthology.orchestra.run.smithi042.stdout: "cephx_require_version": "2", 2024-09-14T23:56:30.424 INFO:teuthology.orchestra.run.smithi042.stdout: "cephx_service_require_signatures": "false", 2024-09-14T23:56:30.425 INFO:teuthology.orchestra.run.smithi042.stdout: "cephx_service_require_version": "2", 2024-09-14T23:56:30.425 INFO:teuthology.orchestra.run.smithi042.stdout: "cephx_sign_messages": "true", 2024-09-14T23:56:30.425 INFO:teuthology.orchestra.run.smithi042.stdout: "chdir": "", 2024-09-14T23:56:30.425 INFO:teuthology.orchestra.run.smithi042.stdout: "client_acl_type": "", 2024-09-14T23:56:30.425 INFO:teuthology.orchestra.run.smithi042.stdout: "client_asio_thread_count": "2", 2024-09-14T23:56:30.425 INFO:teuthology.orchestra.run.smithi042.stdout: "client_cache_mid": "0.750000", 2024-09-14T23:56:30.425 INFO:teuthology.orchestra.run.smithi042.stdout: "client_cache_size": "16384", 2024-09-14T23:56:30.425 INFO:teuthology.orchestra.run.smithi042.stdout: "client_caps_release_delay": "5", 2024-09-14T23:56:30.425 INFO:teuthology.orchestra.run.smithi042.stdout: "client_check_pool_perm": "true", 2024-09-14T23:56:30.425 INFO:teuthology.orchestra.run.smithi042.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-14T23:56:30.425 INFO:teuthology.orchestra.run.smithi042.stdout: "client_debug_force_sync_read": "false", 2024-09-14T23:56:30.425 INFO:teuthology.orchestra.run.smithi042.stdout: "client_debug_getattr_caps": "false", 2024-09-14T23:56:30.425 INFO:teuthology.orchestra.run.smithi042.stdout: "client_debug_inject_features": "", 2024-09-14T23:56:30.426 INFO:teuthology.orchestra.run.smithi042.stdout: "client_debug_inject_tick_delay": "0", 2024-09-14T23:56:30.426 INFO:teuthology.orchestra.run.smithi042.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-14T23:56:30.426 INFO:teuthology.orchestra.run.smithi042.stdout: "client_die_on_failed_remount": "false", 2024-09-14T23:56:30.426 INFO:teuthology.orchestra.run.smithi042.stdout: "client_dirsize_rbytes": "true", 2024-09-14T23:56:30.426 INFO:teuthology.orchestra.run.smithi042.stdout: "client_force_lazyio": "false", 2024-09-14T23:56:30.426 INFO:teuthology.orchestra.run.smithi042.stdout: "client_fs": "", 2024-09-14T23:56:30.426 INFO:teuthology.orchestra.run.smithi042.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-14T23:56:30.426 INFO:teuthology.orchestra.run.smithi042.stdout: "client_inject_release_failure": "false", 2024-09-14T23:56:30.426 INFO:teuthology.orchestra.run.smithi042.stdout: "client_max_inline_size": "4096", 2024-09-14T23:56:30.426 INFO:teuthology.orchestra.run.smithi042.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-14T23:56:30.426 INFO:teuthology.orchestra.run.smithi042.stdout: "client_mds_namespace": "", 2024-09-14T23:56:30.426 INFO:teuthology.orchestra.run.smithi042.stdout: "client_metadata": "", 2024-09-14T23:56:30.426 INFO:teuthology.orchestra.run.smithi042.stdout: "client_mount_gid": "-1", 2024-09-14T23:56:30.426 INFO:teuthology.orchestra.run.smithi042.stdout: "client_mount_timeout": "300", 2024-09-14T23:56:30.427 INFO:teuthology.orchestra.run.smithi042.stdout: "client_mount_uid": "-1", 2024-09-14T23:56:30.427 INFO:teuthology.orchestra.run.smithi042.stdout: "client_mountpoint": "/", 2024-09-14T23:56:30.427 INFO:teuthology.orchestra.run.smithi042.stdout: "client_notify_timeout": "10", 2024-09-14T23:56:30.427 INFO:teuthology.orchestra.run.smithi042.stdout: "client_oc": "true", 2024-09-14T23:56:30.427 INFO:teuthology.orchestra.run.smithi042.stdout: "client_oc_max_dirty": "104857600", 2024-09-14T23:56:30.427 INFO:teuthology.orchestra.run.smithi042.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-14T23:56:30.427 INFO:teuthology.orchestra.run.smithi042.stdout: "client_oc_max_objects": "1000", 2024-09-14T23:56:30.427 INFO:teuthology.orchestra.run.smithi042.stdout: "client_oc_size": "209715200", 2024-09-14T23:56:30.427 INFO:teuthology.orchestra.run.smithi042.stdout: "client_oc_target_dirty": "8388608", 2024-09-14T23:56:30.427 INFO:teuthology.orchestra.run.smithi042.stdout: "client_permissions": "true", 2024-09-14T23:56:30.427 INFO:teuthology.orchestra.run.smithi042.stdout: "client_quota": "true", 2024-09-14T23:56:30.427 INFO:teuthology.orchestra.run.smithi042.stdout: "client_quota_df": "true", 2024-09-14T23:56:30.427 INFO:teuthology.orchestra.run.smithi042.stdout: "client_readahead_max_bytes": "0", 2024-09-14T23:56:30.427 INFO:teuthology.orchestra.run.smithi042.stdout: "client_readahead_max_periods": "4", 2024-09-14T23:56:30.427 INFO:teuthology.orchestra.run.smithi042.stdout: "client_readahead_min": "131072", 2024-09-14T23:56:30.428 INFO:teuthology.orchestra.run.smithi042.stdout: "client_reconnect_stale": "false", 2024-09-14T23:56:30.428 INFO:teuthology.orchestra.run.smithi042.stdout: "client_shutdown_timeout": "30", 2024-09-14T23:56:30.428 INFO:teuthology.orchestra.run.smithi042.stdout: "client_snapdir": ".snap", 2024-09-14T23:56:30.428 INFO:teuthology.orchestra.run.smithi042.stdout: "client_tick_interval": "1", 2024-09-14T23:56:30.428 INFO:teuthology.orchestra.run.smithi042.stdout: "client_trace": "", 2024-09-14T23:56:30.428 INFO:teuthology.orchestra.run.smithi042.stdout: "client_try_dentry_invalidate": "false", 2024-09-14T23:56:30.428 INFO:teuthology.orchestra.run.smithi042.stdout: "client_use_faked_inos": "false", 2024-09-14T23:56:30.428 INFO:teuthology.orchestra.run.smithi042.stdout: "client_use_random_mds": "false", 2024-09-14T23:56:30.428 INFO:teuthology.orchestra.run.smithi042.stdout: "clog_to_graylog": "false", 2024-09-14T23:56:30.428 INFO:teuthology.orchestra.run.smithi042.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-14T23:56:30.428 INFO:teuthology.orchestra.run.smithi042.stdout: "clog_to_graylog_port": "12201", 2024-09-14T23:56:30.428 INFO:teuthology.orchestra.run.smithi042.stdout: "clog_to_monitors": "default=true", 2024-09-14T23:56:30.428 INFO:teuthology.orchestra.run.smithi042.stdout: "clog_to_syslog": "false", 2024-09-14T23:56:30.428 INFO:teuthology.orchestra.run.smithi042.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-14T23:56:30.429 INFO:teuthology.orchestra.run.smithi042.stdout: "clog_to_syslog_level": "info", 2024-09-14T23:56:30.429 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "-", 2024-09-14T23:56:30.429 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_network": "", 2024-09-14T23:56:30.429 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_network_interface": "", 2024-09-14T23:56:30.429 INFO:teuthology.orchestra.run.smithi042.stdout: "compressor_zlib_isal": "false", 2024-09-14T23:56:30.429 INFO:teuthology.orchestra.run.smithi042.stdout: "compressor_zlib_level": "5", 2024-09-14T23:56:30.429 INFO:teuthology.orchestra.run.smithi042.stdout: "compressor_zlib_winsize": "-15", 2024-09-14T23:56:30.429 INFO:teuthology.orchestra.run.smithi042.stdout: "compressor_zstd_level": "1", 2024-09-14T23:56:30.429 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-14T23:56:30.429 INFO:teuthology.orchestra.run.smithi042.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-14T23:56:30.429 INFO:teuthology.orchestra.run.smithi042.stdout: "crimson_alien_op_num_threads": "6", 2024-09-14T23:56:30.429 INFO:teuthology.orchestra.run.smithi042.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-14T23:56:30.429 INFO:teuthology.orchestra.run.smithi042.stdout: "crimson_osd_obc_lru_size": "512", 2024-09-14T23:56:30.430 INFO:teuthology.orchestra.run.smithi042.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-14T23:56:30.430 INFO:teuthology.orchestra.run.smithi042.stdout: "crimson_osd_stat_interval": "0", 2024-09-14T23:56:30.430 INFO:teuthology.orchestra.run.smithi042.stdout: "crimson_seastar_cpu_cores": "", 2024-09-14T23:56:30.430 INFO:teuthology.orchestra.run.smithi042.stdout: "crimson_seastar_num_threads": "0", 2024-09-14T23:56:30.430 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_location": "", 2024-09-14T23:56:30.430 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_location_hook": "", 2024-09-14T23:56:30.430 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_location_hook_timeout": "10", 2024-09-14T23:56:30.430 INFO:teuthology.orchestra.run.smithi042.stdout: "daemonize": "false", 2024-09-14T23:56:30.430 INFO:teuthology.orchestra.run.smithi042.stdout: "daos_pool": "tank", 2024-09-14T23:56:30.430 INFO:teuthology.orchestra.run.smithi042.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-14T23:56:30.430 INFO:teuthology.orchestra.run.smithi042.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-14T23:56:30.430 INFO:teuthology.orchestra.run.smithi042.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-14T23:56:30.430 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_alienstore": "0/5", 2024-09-14T23:56:30.430 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_allow_any_pool_priority": "false", 2024-09-14T23:56:30.430 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_asok": "1/5", 2024-09-14T23:56:30.431 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_asok_assert_abort": "false", 2024-09-14T23:56:30.431 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_asserts_on_shutdown": "true", 2024-09-14T23:56:30.431 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_auth": "20/20", 2024-09-14T23:56:30.431 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_bdev": "1/3", 2024-09-14T23:56:30.431 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_bluefs": "1/5", 2024-09-14T23:56:30.431 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_bluestore": "1/5", 2024-09-14T23:56:30.431 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_buffer": "0/1", 2024-09-14T23:56:30.431 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_ceph_exporter": "1/5", 2024-09-14T23:56:30.431 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_cephfs_mirror": "0/5", 2024-09-14T23:56:30.431 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_cephsqlite": "0/5", 2024-09-14T23:56:30.431 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_client": "0/5", 2024-09-14T23:56:30.431 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_compressor": "1/5", 2024-09-14T23:56:30.431 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_context": "0/1", 2024-09-14T23:56:30.431 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_crimson_interrupt": "0/5", 2024-09-14T23:56:30.431 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_crush": "1/1", 2024-09-14T23:56:30.432 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_crypto": "1/5", 2024-09-14T23:56:30.432 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_cyanstore": "0/5", 2024-09-14T23:56:30.432 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_deliberately_leak_memory": "false", 2024-09-14T23:56:30.432 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_disable_randomized_ping": "false", 2024-09-14T23:56:30.432 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_dpdk": "1/5", 2024-09-14T23:56:30.432 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_eventtrace": "1/5", 2024-09-14T23:56:30.432 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_filer": "0/1", 2024-09-14T23:56:30.432 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_filestore": "1/3", 2024-09-14T23:56:30.432 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_finisher": "1/1", 2024-09-14T23:56:30.432 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_fuse": "1/5", 2024-09-14T23:56:30.432 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_heartbeat_testing_span": "0", 2024-09-14T23:56:30.432 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_heartbeatmap": "1/5", 2024-09-14T23:56:30.432 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-14T23:56:30.433 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_javaclient": "1/5", 2024-09-14T23:56:30.433 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_journal": "1/3", 2024-09-14T23:56:30.433 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_journaler": "0/5", 2024-09-14T23:56:30.433 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_kstore": "1/5", 2024-09-14T23:56:30.433 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_lockdep": "0/1", 2024-09-14T23:56:30.433 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_mclock": "1/5", 2024-09-14T23:56:30.433 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_mds": "1/5", 2024-09-14T23:56:30.433 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_mds_balancer": "1/5", 2024-09-14T23:56:30.433 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_mds_locker": "1/5", 2024-09-14T23:56:30.433 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_mds_log": "1/5", 2024-09-14T23:56:30.433 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_mds_log_expire": "1/5", 2024-09-14T23:56:30.433 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_mds_migrator": "1/5", 2024-09-14T23:56:30.434 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_mds_quiesce": "3/5", 2024-09-14T23:56:30.434 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_memstore": "1/5", 2024-09-14T23:56:30.434 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_mgr": "2/5", 2024-09-14T23:56:30.434 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_mgrc": "1/5", 2024-09-14T23:56:30.434 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_mon": "20/20", 2024-09-14T23:56:30.434 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_monc": "0/10", 2024-09-14T23:56:30.434 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_ms": "1/1", 2024-09-14T23:56:30.434 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_none": "0/5", 2024-09-14T23:56:30.434 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_objclass": "0/5", 2024-09-14T23:56:30.434 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_objectcacher": "0/5", 2024-09-14T23:56:30.434 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_objecter": "0/1", 2024-09-14T23:56:30.434 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_optracker": "0/5", 2024-09-14T23:56:30.434 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_osd": "1/5", 2024-09-14T23:56:30.435 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_paxos": "20/20", 2024-09-14T23:56:30.435 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_perfcounter": "1/5", 2024-09-14T23:56:30.435 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_prioritycache": "1/5", 2024-09-14T23:56:30.435 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_rados": "0/5", 2024-09-14T23:56:30.435 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_rbd": "0/5", 2024-09-14T23:56:30.435 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_rbd_mirror": "0/5", 2024-09-14T23:56:30.435 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_rbd_pwl": "0/5", 2024-09-14T23:56:30.435 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_rbd_replay": "0/5", 2024-09-14T23:56:30.435 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_refs": "0/0", 2024-09-14T23:56:30.435 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_reserver": "1/1", 2024-09-14T23:56:30.435 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_rgw": "1/5", 2024-09-14T23:56:30.435 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_rgw_access": "1/5", 2024-09-14T23:56:30.435 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_rgw_datacache": "1/5", 2024-09-14T23:56:30.436 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_rgw_dbstore": "1/5", 2024-09-14T23:56:30.436 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_rgw_flight": "1/5", 2024-09-14T23:56:30.436 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_rgw_lifecycle": "1/5", 2024-09-14T23:56:30.436 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_rgw_sync": "1/5", 2024-09-14T23:56:30.436 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_rocksdb": "4/5", 2024-09-14T23:56:30.436 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_seastore": "0/5", 2024-09-14T23:56:30.436 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_seastore_backref": "0/5", 2024-09-14T23:56:30.436 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_seastore_cache": "0/5", 2024-09-14T23:56:30.436 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_seastore_cleaner": "0/5", 2024-09-14T23:56:30.436 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_seastore_device": "0/5", 2024-09-14T23:56:30.436 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_seastore_epm": "0/5", 2024-09-14T23:56:30.436 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-14T23:56:30.436 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_seastore_journal": "0/5", 2024-09-14T23:56:30.437 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_seastore_lba": "0/5", 2024-09-14T23:56:30.437 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_seastore_odata": "0/5", 2024-09-14T23:56:30.437 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_seastore_omap": "0/5", 2024-09-14T23:56:30.437 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_seastore_onode": "0/5", 2024-09-14T23:56:30.437 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_seastore_t": "0/5", 2024-09-14T23:56:30.437 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_seastore_tm": "0/5", 2024-09-14T23:56:30.437 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_striper": "0/1", 2024-09-14T23:56:30.437 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_test": "0/5", 2024-09-14T23:56:30.437 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_throttle": "1/1", 2024-09-14T23:56:30.437 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_timer": "0/1", 2024-09-14T23:56:30.437 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_tp": "0/5", 2024-09-14T23:56:30.437 INFO:teuthology.orchestra.run.smithi042.stdout: "debug_trace": "1/5", 2024-09-14T23:56:30.437 INFO:teuthology.orchestra.run.smithi042.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-14T23:56:30.437 INFO:teuthology.orchestra.run.smithi042.stdout: "device_failure_prediction_mode": "none", 2024-09-14T23:56:30.438 INFO:teuthology.orchestra.run.smithi042.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-14T23:56:30.438 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-09-14T23:56:30.438 INFO:teuthology.orchestra.run.smithi042.stdout: "err_to_graylog": "false", 2024-09-14T23:56:30.438 INFO:teuthology.orchestra.run.smithi042.stdout: "err_to_journald": "false", 2024-09-14T23:56:30.438 INFO:teuthology.orchestra.run.smithi042.stdout: "err_to_stderr": "true", 2024-09-14T23:56:30.438 INFO:teuthology.orchestra.run.smithi042.stdout: "err_to_syslog": "false", 2024-09-14T23:56:30.438 INFO:teuthology.orchestra.run.smithi042.stdout: "event_tracing": "false", 2024-09-14T23:56:30.438 INFO:teuthology.orchestra.run.smithi042.stdout: "exporter_addr": "0.0.0.0", 2024-09-14T23:56:30.438 INFO:teuthology.orchestra.run.smithi042.stdout: "exporter_http_port": "9926", 2024-09-14T23:56:30.438 INFO:teuthology.orchestra.run.smithi042.stdout: "exporter_prio_limit": "5", 2024-09-14T23:56:30.438 INFO:teuthology.orchestra.run.smithi042.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-14T23:56:30.438 INFO:teuthology.orchestra.run.smithi042.stdout: "exporter_sort_metrics": "true", 2024-09-14T23:56:30.438 INFO:teuthology.orchestra.run.smithi042.stdout: "exporter_stats_period": "5", 2024-09-14T23:56:30.439 INFO:teuthology.orchestra.run.smithi042.stdout: "fake_statfs_for_testing": "0", 2024-09-14T23:56:30.439 INFO:teuthology.orchestra.run.smithi042.stdout: "fatal_signal_handlers": "true", 2024-09-14T23:56:30.439 INFO:teuthology.orchestra.run.smithi042.stdout: "filer_max_purge_ops": "10", 2024-09-14T23:56:30.439 INFO:teuthology.orchestra.run.smithi042.stdout: "filer_max_truncate_ops": "128", 2024-09-14T23:56:30.439 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_apply_finisher_threads": "1", 2024-09-14T23:56:30.439 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_blackhole": "false", 2024-09-14T23:56:30.439 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_btrfs_clone_range": "true", 2024-09-14T23:56:30.439 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_btrfs_snap": "true", 2024-09-14T23:56:30.439 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_caller_concurrency": "10", 2024-09-14T23:56:30.439 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_collect_device_partition_information": "true", 2024-09-14T23:56:30.439 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_commit_timeout": "600.000000", 2024-09-14T23:56:30.439 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_debug_inject_read_err": "false", 2024-09-14T23:56:30.439 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_debug_omap_check": "false", 2024-09-14T23:56:30.439 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-14T23:56:30.440 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_debug_verify_split": "false", 2024-09-14T23:56:30.440 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_dump_file": "", 2024-09-14T23:56:30.440 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-14T23:56:30.440 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-14T23:56:30.440 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_fadvise": "true", 2024-09-14T23:56:30.440 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_fail_eio": "true", 2024-09-14T23:56:30.440 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_fd_cache_shards": "16", 2024-09-14T23:56:30.440 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_fd_cache_size": "128", 2024-09-14T23:56:30.440 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_fiemap": "false", 2024-09-14T23:56:30.440 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_fiemap_threshold": "4096", 2024-09-14T23:56:30.440 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-14T23:56:30.440 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-14T23:56:30.440 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_inject_stall": "0", 2024-09-14T23:56:30.440 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_journal_parallel": "false", 2024-09-14T23:56:30.441 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_journal_trailing": "false", 2024-09-14T23:56:30.441 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_journal_writeahead": "false", 2024-09-14T23:56:30.441 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_kill_at": "0", 2024-09-14T23:56:30.441 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-14T23:56:30.441 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-14T23:56:30.441 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-14T23:56:30.441 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-14T23:56:30.441 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-14T23:56:30.441 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_max_inline_xattrs": "0", 2024-09-14T23:56:30.441 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-14T23:56:30.441 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-14T23:56:30.441 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-14T23:56:30.441 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-14T23:56:30.442 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_max_xattr_value_size": "0", 2024-09-14T23:56:30.442 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-14T23:56:30.442 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-14T23:56:30.442 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-14T23:56:30.442 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_merge_threshold": "-10", 2024-09-14T23:56:30.442 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-14T23:56:30.442 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_odsync_write": "false", 2024-09-14T23:56:30.442 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_omap_backend": "rocksdb", 2024-09-14T23:56:30.442 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_omap_backend_path": "", 2024-09-14T23:56:30.442 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-14T23:56:30.442 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-14T23:56:30.442 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-14T23:56:30.442 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_op_thread_timeout": "60", 2024-09-14T23:56:30.442 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_op_threads": "2", 2024-09-14T23:56:30.442 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_punch_hole": "false", 2024-09-14T23:56:30.443 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-14T23:56:30.443 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-14T23:56:30.443 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-14T23:56:30.443 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-14T23:56:30.443 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-14T23:56:30.443 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-14T23:56:30.443 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-14T23:56:30.443 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-14T23:56:30.443 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-14T23:56:30.443 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_queue_max_ops": "50", 2024-09-14T23:56:30.443 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-14T23:56:30.443 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_seek_data_hole": "false", 2024-09-14T23:56:30.443 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_sloppy_crc": "false", 2024-09-14T23:56:30.443 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-14T23:56:30.444 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_splice": "false", 2024-09-14T23:56:30.444 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_split_multiple": "2", 2024-09-14T23:56:30.444 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_split_rand_factor": "20", 2024-09-14T23:56:30.444 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_update_to": "1000", 2024-09-14T23:56:30.444 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-14T23:56:30.444 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-14T23:56:30.444 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-14T23:56:30.444 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-14T23:56:30.444 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-14T23:56:30.444 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-14T23:56:30.444 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_wbthrottle_enable": "true", 2024-09-14T23:56:30.444 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-14T23:56:30.444 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-14T23:56:30.444 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-14T23:56:30.445 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-14T23:56:30.445 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-14T23:56:30.445 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-14T23:56:30.445 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_xfs_extsize": "false", 2024-09-14T23:56:30.445 INFO:teuthology.orchestra.run.smithi042.stdout: "filestore_zfs_snap": "false", 2024-09-14T23:56:30.445 INFO:teuthology.orchestra.run.smithi042.stdout: "fio_dir": "/tmp/fio", 2024-09-14T23:56:30.445 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "21916fcc-f95c-4bfa-8b8b-11ca45845613", 2024-09-14T23:56:30.445 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_allow_other": "true", 2024-09-14T23:56:30.445 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_atomic_o_trunc": "true", 2024-09-14T23:56:30.445 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_big_writes": "true", 2024-09-14T23:56:30.445 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_debug": "false", 2024-09-14T23:56:30.445 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_default_permissions": "false", 2024-09-14T23:56:30.445 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_disable_pagecache": "false", 2024-09-14T23:56:30.446 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_max_write": "0", 2024-09-14T23:56:30.446 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_multithreaded": "true", 2024-09-14T23:56:30.446 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_require_active_mds": "true", 2024-09-14T23:56:30.446 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_set_user_groups": "true", 2024-09-14T23:56:30.446 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_splice_move": "true", 2024-09-14T23:56:30.446 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_splice_read": "true", 2024-09-14T23:56:30.446 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_splice_write": "true", 2024-09-14T23:56:30.446 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-14T23:56:30.446 INFO:teuthology.orchestra.run.smithi042.stdout: "fuse_use_invalidate_cb": "true", 2024-09-14T23:56:30.448 INFO:teuthology.orchestra.run.smithi042.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-14T23:56:30.448 INFO:teuthology.orchestra.run.smithi042.stdout: "gss_target_name": "ceph", 2024-09-14T23:56:30.448 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_file": "", 2024-09-14T23:56:30.448 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_inject_failure": "0", 2024-09-14T23:56:30.448 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_interval": "5", 2024-09-14T23:56:30.449 INFO:teuthology.orchestra.run.smithi042.stdout: "host": "smithi042", 2024-09-14T23:56:30.449 INFO:teuthology.orchestra.run.smithi042.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-14T23:56:30.449 INFO:teuthology.orchestra.run.smithi042.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-14T23:56:30.449 INFO:teuthology.orchestra.run.smithi042.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-14T23:56:30.449 INFO:teuthology.orchestra.run.smithi042.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-14T23:56:30.449 INFO:teuthology.orchestra.run.smithi042.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-14T23:56:30.449 INFO:teuthology.orchestra.run.smithi042.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-14T23:56:30.449 INFO:teuthology.orchestra.run.smithi042.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-14T23:56:30.449 INFO:teuthology.orchestra.run.smithi042.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-14T23:56:30.449 INFO:teuthology.orchestra.run.smithi042.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-14T23:56:30.449 INFO:teuthology.orchestra.run.smithi042.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-14T23:56:30.449 INFO:teuthology.orchestra.run.smithi042.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-14T23:56:30.449 INFO:teuthology.orchestra.run.smithi042.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-14T23:56:30.449 INFO:teuthology.orchestra.run.smithi042.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-14T23:56:30.450 INFO:teuthology.orchestra.run.smithi042.stdout: "inject_early_sigterm": "false", 2024-09-14T23:56:30.450 INFO:teuthology.orchestra.run.smithi042.stdout: "jaeger_agent_port": "6799", 2024-09-14T23:56:30.450 INFO:teuthology.orchestra.run.smithi042.stdout: "jaeger_tracing_enable": "false", 2024-09-14T23:56:30.450 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_aio": "true", 2024-09-14T23:56:30.450 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_align_min_size": "65536", 2024-09-14T23:56:30.450 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_block_align": "true", 2024-09-14T23:56:30.450 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_block_size": "4096", 2024-09-14T23:56:30.450 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_dio": "true", 2024-09-14T23:56:30.450 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_discard": "false", 2024-09-14T23:56:30.450 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_force_aio": "false", 2024-09-14T23:56:30.450 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_ignore_corruption": "false", 2024-09-14T23:56:30.450 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_max_write_bytes": "10485760", 2024-09-14T23:56:30.450 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_max_write_entries": "100", 2024-09-14T23:56:30.450 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_replay_from": "0", 2024-09-14T23:56:30.451 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-14T23:56:30.451 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-14T23:56:30.451 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-14T23:56:30.451 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-14T23:56:30.451 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_write_header_frequency": "0", 2024-09-14T23:56:30.451 INFO:teuthology.orchestra.run.smithi042.stdout: "journal_zero_on_create": "false", 2024-09-14T23:56:30.451 INFO:teuthology.orchestra.run.smithi042.stdout: "journaler_prefetch_periods": "10", 2024-09-14T23:56:30.451 INFO:teuthology.orchestra.run.smithi042.stdout: "journaler_prezero_periods": "5", 2024-09-14T23:56:30.451 INFO:teuthology.orchestra.run.smithi042.stdout: "journaler_write_head_interval": "15", 2024-09-14T23:56:30.451 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "", 2024-09-14T23:56:30.451 INFO:teuthology.orchestra.run.smithi042.stdout: "keyfile": "", 2024-09-14T23:56:30.451 INFO:teuthology.orchestra.run.smithi042.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-14T23:56:30.451 INFO:teuthology.orchestra.run.smithi042.stdout: "kstore_backend": "rocksdb", 2024-09-14T23:56:30.451 INFO:teuthology.orchestra.run.smithi042.stdout: "kstore_default_stripe_size": "65536", 2024-09-14T23:56:30.451 INFO:teuthology.orchestra.run.smithi042.stdout: "kstore_fsck_on_mount": "false", 2024-09-14T23:56:30.451 INFO:teuthology.orchestra.run.smithi042.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-14T23:56:30.452 INFO:teuthology.orchestra.run.smithi042.stdout: "kstore_max_bytes": "67108864", 2024-09-14T23:56:30.452 INFO:teuthology.orchestra.run.smithi042.stdout: "kstore_max_ops": "512", 2024-09-14T23:56:30.452 INFO:teuthology.orchestra.run.smithi042.stdout: "kstore_nid_prealloc": "1024", 2024-09-14T23:56:30.452 INFO:teuthology.orchestra.run.smithi042.stdout: "kstore_onode_map_size": "1024", 2024-09-14T23:56:30.452 INFO:teuthology.orchestra.run.smithi042.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-14T23:56:30.452 INFO:teuthology.orchestra.run.smithi042.stdout: "kstore_sync_submit_transaction": "false", 2024-09-14T23:56:30.452 INFO:teuthology.orchestra.run.smithi042.stdout: "kstore_sync_transaction": "false", 2024-09-14T23:56:30.452 INFO:teuthology.orchestra.run.smithi042.stdout: "librados_thread_count": "2", 2024-09-14T23:56:30.452 INFO:teuthology.orchestra.run.smithi042.stdout: "lockdep": "false", 2024-09-14T23:56:30.452 INFO:teuthology.orchestra.run.smithi042.stdout: "lockdep_force_backtrace": "false", 2024-09-14T23:56:30.452 INFO:teuthology.orchestra.run.smithi042.stdout: "log_coarse_timestamps": "true", 2024-09-14T23:56:30.452 INFO:teuthology.orchestra.run.smithi042.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-14T23:56:30.452 INFO:teuthology.orchestra.run.smithi042.stdout: "log_flush_on_exit": "false", 2024-09-14T23:56:30.453 INFO:teuthology.orchestra.run.smithi042.stdout: "log_graylog_host": "127.0.0.1", 2024-09-14T23:56:30.453 INFO:teuthology.orchestra.run.smithi042.stdout: "log_graylog_port": "12201", 2024-09-14T23:56:30.453 INFO:teuthology.orchestra.run.smithi042.stdout: "log_max_new": "1000", 2024-09-14T23:56:30.453 INFO:teuthology.orchestra.run.smithi042.stdout: "log_max_recent": "10000", 2024-09-14T23:56:30.453 INFO:teuthology.orchestra.run.smithi042.stdout: "log_stderr_prefix": "", 2024-09-14T23:56:30.453 INFO:teuthology.orchestra.run.smithi042.stdout: "log_stop_at_utilization": "0.970000", 2024-09-14T23:56:30.453 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": "true", 2024-09-14T23:56:30.453 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_graylog": "false", 2024-09-14T23:56:30.453 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_journald": "false", 2024-09-14T23:56:30.453 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_stderr": "false", 2024-09-14T23:56:30.453 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_syslog": "false", 2024-09-14T23:56:30.453 INFO:teuthology.orchestra.run.smithi042.stdout: "max_rotating_auth_attempts": "10", 2024-09-14T23:56:30.453 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-14T23:56:30.454 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_action_on_write_error": "1", 2024-09-14T23:56:30.454 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_alternate_name_max": "8192", 2024-09-14T23:56:30.454 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_asio_thread_count": "2", 2024-09-14T23:56:30.454 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_export_pin": "true", 2024-09-14T23:56:30.454 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_fragment_dirs": "true", 2024-09-14T23:56:30.454 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-14T23:56:30.454 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_fragment_interval": "5", 2024-09-14T23:56:30.454 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-14T23:56:30.454 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-14T23:56:30.454 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_interval": "10", 2024-09-14T23:56:30.454 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_max": "-1", 2024-09-14T23:56:30.454 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_max_until": "-1", 2024-09-14T23:56:30.454 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_merge_size": "50", 2024-09-14T23:56:30.455 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_midchunk": "0.300000", 2024-09-14T23:56:30.455 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-14T23:56:30.455 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_min_start": "0.200000", 2024-09-14T23:56:30.455 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_minchunk": "0.001000", 2024-09-14T23:56:30.455 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_mode": "0", 2024-09-14T23:56:30.455 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_need_max": "1.200000", 2024-09-14T23:56:30.455 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_need_min": "0.800000", 2024-09-14T23:56:30.455 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_overload_epochs": "2", 2024-09-14T23:56:30.455 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-14T23:56:30.455 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-14T23:56:30.455 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_split_bits": "3", 2024-09-14T23:56:30.455 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-14T23:56:30.455 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_split_size": "10000", 2024-09-14T23:56:30.456 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-14T23:56:30.456 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_target_decay": "10.000000", 2024-09-14T23:56:30.456 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-14T23:56:30.456 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_beacon_grace": "15.000000", 2024-09-14T23:56:30.456 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_beacon_interval": "4.000000", 2024-09-14T23:56:30.456 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-14T23:56:30.456 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-14T23:56:30.456 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_cache_mid": "0.700000", 2024-09-14T23:56:30.456 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-09-14T23:56:30.456 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_cache_quiesce_delay": "0", 2024-09-14T23:56:30.456 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_cache_quiesce_sleep": "200", 2024-09-14T23:56:30.456 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_cache_quiesce_splitauth": "true", 2024-09-14T23:56:30.456 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_cache_quiesce_threshold": "524288", 2024-09-14T23:56:30.456 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_cache_release_free_interval": "10", 2024-09-14T23:56:30.457 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_cache_reservation": "0.050000", 2024-09-14T23:56:30.457 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-14T23:56:30.457 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_cache_trim_interval": "1", 2024-09-14T23:56:30.457 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_cache_trim_threshold": "262144", 2024-09-14T23:56:30.457 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-14T23:56:30.457 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-14T23:56:30.457 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-14T23:56:30.457 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_client_prealloc_inos": "1000", 2024-09-14T23:56:30.457 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-14T23:56:30.457 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_connect_bootstrapping": "false", 2024-09-14T23:56:30.457 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_damage_table_max_entries": "10000", 2024-09-14T23:56:30.457 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-14T23:56:30.457 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_debug_auth_pins": "false", 2024-09-14T23:56:30.458 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_debug_frag": "false", 2024-09-14T23:56:30.458 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_debug_scatterstat": "false", 2024-09-14T23:56:30.458 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_debug_subtrees": "false", 2024-09-14T23:56:30.458 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_decay_halflife": "5.000000", 2024-09-14T23:56:30.458 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_default_dir_hash": "2", 2024-09-14T23:56:30.458 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_defer_session_stale": "true", 2024-09-14T23:56:30.458 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_deny_all_reconnect": "false", 2024-09-14T23:56:30.458 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_dir_keys_per_op": "16384", 2024-09-14T23:56:30.458 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_dir_max_commit_size": "10", 2024-09-14T23:56:30.458 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_dir_max_entries": "0", 2024-09-14T23:56:30.458 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_dir_prefetch": "true", 2024-09-14T23:56:30.458 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-14T23:56:30.458 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-14T23:56:30.458 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_dump_cache_on_map": "false", 2024-09-14T23:56:30.459 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-14T23:56:30.459 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-14T23:56:30.459 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_early_reply": "true", 2024-09-14T23:56:30.459 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_enable_op_tracker": "true", 2024-09-14T23:56:30.459 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_enforce_unique_name": "true", 2024-09-14T23:56:30.459 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-14T23:56:30.459 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-14T23:56:30.459 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_export_ephemeral_random": "true", 2024-09-14T23:56:30.459 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-14T23:56:30.459 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-14T23:56:30.459 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-14T23:56:30.459 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-14T23:56:30.459 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-14T23:56:30.459 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-14T23:56:30.460 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-14T23:56:30.460 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-14T23:56:30.460 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_health_summarize_threshold": "10", 2024-09-14T23:56:30.460 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-14T23:56:30.460 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-14T23:56:30.460 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_inject_health_dummy": "false", 2024-09-14T23:56:30.460 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-14T23:56:30.460 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_inject_migrator_session_race": "false", 2024-09-14T23:56:30.460 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-14T23:56:30.460 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-14T23:56:30.460 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-14T23:56:30.460 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_join_fs": "", 2024-09-14T23:56:30.460 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_journal_format": "1", 2024-09-14T23:56:30.461 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-14T23:56:30.461 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_kill_create_at": "0", 2024-09-14T23:56:30.461 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_kill_export_at": "0", 2024-09-14T23:56:30.461 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_kill_import_at": "0", 2024-09-14T23:56:30.461 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_kill_journal_at": "0", 2024-09-14T23:56:30.461 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_kill_journal_expire_at": "0", 2024-09-14T23:56:30.461 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_kill_journal_replay_at": "0", 2024-09-14T23:56:30.461 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_kill_link_at": "0", 2024-09-14T23:56:30.461 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_kill_mdstable_at": "0", 2024-09-14T23:56:30.461 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_kill_openc_at": "0", 2024-09-14T23:56:30.461 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_kill_rename_at": "0", 2024-09-14T23:56:30.461 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_kill_shutdown_at": "0", 2024-09-14T23:56:30.462 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_log_event_large_threshold": "524288", 2024-09-14T23:56:30.462 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_log_events_per_segment": "1024", 2024-09-14T23:56:30.462 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_log_major_segment_event_ratio": "12", 2024-09-14T23:56:30.462 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_log_max_events": "-1", 2024-09-14T23:56:30.462 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_log_max_segments": "128", 2024-09-14T23:56:30.462 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_log_pause": "false", 2024-09-14T23:56:30.462 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_log_segment_size": "0", 2024-09-14T23:56:30.462 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-14T23:56:30.462 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_log_skip_unbounded_events": "false", 2024-09-14T23:56:30.462 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-09-14T23:56:30.462 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_log_trim_threshold": "128", 2024-09-14T23:56:30.462 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-09-14T23:56:30.462 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_log_warn_factor": "2.000000", 2024-09-14T23:56:30.462 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_max_caps_per_client": "1048576", 2024-09-14T23:56:30.463 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_max_completed_flushes": "100000", 2024-09-14T23:56:30.463 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_max_completed_requests": "100000", 2024-09-14T23:56:30.463 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_max_export_size": "20971520", 2024-09-14T23:56:30.463 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_max_file_recover": "32", 2024-09-14T23:56:30.463 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_max_purge_files": "64", 2024-09-14T23:56:30.463 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_max_purge_ops": "8192", 2024-09-14T23:56:30.463 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-14T23:56:30.463 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-14T23:56:30.463 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_max_snaps_per_dir": "100", 2024-09-14T23:56:30.463 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_metrics_update_interval": "2", 2024-09-14T23:56:30.463 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_min_caps_per_client": "100", 2024-09-14T23:56:30.463 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_min_caps_working_set": "10000", 2024-09-14T23:56:30.463 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-14T23:56:30.463 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_numa_node": "-1", 2024-09-14T23:56:30.464 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-14T23:56:30.464 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_op_complaint_time": "30.000000", 2024-09-14T23:56:30.464 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_op_history_duration": "600", 2024-09-14T23:56:30.464 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_op_history_size": "20", 2024-09-14T23:56:30.464 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_op_history_slow_op_size": "20", 2024-09-14T23:56:30.464 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-14T23:56:30.464 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_op_log_threshold": "5", 2024-09-14T23:56:30.464 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_ping_grace": "15", 2024-09-14T23:56:30.464 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_ping_interval": "5", 2024-09-14T23:56:30.464 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-14T23:56:30.464 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-14T23:56:30.464 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_recall_max_caps": "30000", 2024-09-14T23:56:30.464 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-14T23:56:30.464 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-14T23:56:30.465 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-14T23:56:30.465 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_recall_warning_threshold": "262144", 2024-09-14T23:56:30.465 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-14T23:56:30.465 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_replay_interval": "1.000000", 2024-09-14T23:56:30.465 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-14T23:56:30.465 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-14T23:56:30.465 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_root_ino_gid": "0", 2024-09-14T23:56:30.465 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_root_ino_uid": "0", 2024-09-14T23:56:30.465 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-14T23:56:30.465 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-14T23:56:30.465 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-14T23:56:30.465 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-14T23:56:30.465 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-14T23:56:30.466 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-14T23:56:30.466 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-14T23:56:30.466 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-14T23:56:30.466 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-14T23:56:30.466 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-14T23:56:30.466 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_shutdown_check": "0", 2024-09-14T23:56:30.466 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_skip_ino": "0", 2024-09-14T23:56:30.466 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-14T23:56:30.466 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_snap_max_uid": "4294967294", 2024-09-14T23:56:30.466 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_snap_min_uid": "0", 2024-09-14T23:56:30.466 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_snap_rstat": "false", 2024-09-14T23:56:30.466 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_standby_replay_damaged": "false", 2024-09-14T23:56:30.466 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_symlink_recovery": "true", 2024-09-14T23:56:30.467 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-14T23:56:30.467 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_thrash_exports": "0", 2024-09-14T23:56:30.467 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_thrash_fragments": "0", 2024-09-14T23:56:30.467 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_tick_interval": "5.000000", 2024-09-14T23:56:30.467 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_valgrind_exit": "false", 2024-09-14T23:56:30.467 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_verify_backtrace": "1", 2024-09-14T23:56:30.467 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_verify_scatter": "false", 2024-09-14T23:56:30.467 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-14T23:56:30.467 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_wipe_sessions": "false", 2024-09-14T23:56:30.467 INFO:teuthology.orchestra.run.smithi042.stdout: "mempool_debug": "false", 2024-09-14T23:56:30.467 INFO:teuthology.orchestra.run.smithi042.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-14T23:56:30.467 INFO:teuthology.orchestra.run.smithi042.stdout: "memstore_device_bytes": "1073741824", 2024-09-14T23:56:30.467 INFO:teuthology.orchestra.run.smithi042.stdout: "memstore_page_set": "false", 2024-09-14T23:56:30.468 INFO:teuthology.orchestra.run.smithi042.stdout: "memstore_page_size": "65536", 2024-09-14T23:56:30.468 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_client_bytes": "134217728", 2024-09-14T23:56:30.468 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_client_messages": "512", 2024-09-14T23:56:30.468 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-14T23:56:30.468 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-14T23:56:30.468 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-14T23:56:30.468 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-14T23:56:30.468 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_disabled_modules": "", 2024-09-14T23:56:30.468 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_enable_op_tracker": "true", 2024-09-14T23:56:30.468 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-14T23:56:30.468 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_max_pg_num_change": "128", 2024-09-14T23:56:30.468 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_mds_bytes": "134217728", 2024-09-14T23:56:30.469 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_mds_messages": "128", 2024-09-14T23:56:30.469 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-14T23:56:30.469 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_mon_bytes": "134217728", 2024-09-14T23:56:30.469 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_mon_messages": "128", 2024-09-14T23:56:30.469 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_num_op_tracker_shard": "32", 2024-09-14T23:56:30.469 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_op_complaint_time": "30.000000", 2024-09-14T23:56:30.469 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_op_history_duration": "600", 2024-09-14T23:56:30.469 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_op_history_size": "20", 2024-09-14T23:56:30.469 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_op_history_slow_op_size": "20", 2024-09-14T23:56:30.469 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2024-09-14T23:56:30.469 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_op_log_threshold": "5", 2024-09-14T23:56:30.469 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_osd_bytes": "536870912", 2024-09-14T23:56:30.469 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_osd_messages": "8192", 2024-09-14T23:56:30.469 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_pool": "true", 2024-09-14T23:56:30.470 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-14T23:56:30.470 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_standby_modules": "true", 2024-09-14T23:56:30.470 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_stats_period": "5", 2024-09-14T23:56:30.470 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_stats_threshold": "5", 2024-09-14T23:56:30.470 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_tick_period": "2", 2024-09-14T23:56:30.470 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-14T23:56:30.470 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-14T23:56:30.470 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_allow_pool_delete": "true", 2024-09-14T23:56:30.470 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_allow_pool_size_one": "true", 2024-09-14T23:56:30.470 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_auth_validate_all_caps": "true", 2024-09-14T23:56:30.470 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-14T23:56:30.470 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-14T23:56:30.470 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_client_bytes": "104857600", 2024-09-14T23:56:30.471 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_client_directed_command_retry": "5", 2024-09-14T23:56:30.471 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-14T23:56:30.471 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-14T23:56:30.471 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-14T23:56:30.471 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-14T23:56:30.471 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_client_hunt_parallel": "3", 2024-09-14T23:56:30.471 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_client_log_interval": "1.000000", 2024-09-14T23:56:30.471 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-14T23:56:30.471 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_client_ping_interval": "10.000000", 2024-09-14T23:56:30.471 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-14T23:56:30.471 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_client_target_rank": "-1", 2024-09-14T23:56:30.471 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-14T23:56:30.471 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-14T23:56:30.471 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-14T23:56:30.472 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_cluster_log_file_level": "debug", 2024-09-14T23:56:30.472 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_cluster_log_to_file": "true", 2024-09-14T23:56:30.472 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-14T23:56:30.472 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-14T23:56:30.472 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-14T23:56:30.472 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_cluster_log_to_journald": "false", 2024-09-14T23:56:30.472 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-14T23:56:30.472 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-14T23:56:30.472 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-14T23:56:30.472 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-09-14T23:56:30.472 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_compact_on_bootstrap": "false", 2024-09-14T23:56:30.472 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_compact_on_start": "false", 2024-09-14T23:56:30.472 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_compact_on_trim": "true", 2024-09-14T23:56:30.473 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-14T23:56:30.473 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-14T23:56:30.473 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-14T23:56:30.473 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_cpu_threads": "4", 2024-09-14T23:56:30.473 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_crush_min_required_version": "hammer", 2024-09-14T23:56:30.473 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_daemon_bytes": "419430400", 2024-09-14T23:56:30.473 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-14T23:56:30.473 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_data_avail_crit": "5", 2024-09-14T23:56:30.473 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_data_avail_warn": "5", 2024-09-14T23:56:30.473 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_data_size_warn": "16106127360", 2024-09-14T23:56:30.473 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-14T23:56:30.473 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-14T23:56:30.473 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_debug_dump_json": "false", 2024-09-14T23:56:30.474 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-14T23:56:30.474 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_debug_dump_transactions": "false", 2024-09-14T23:56:30.474 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_debug_extra_checks": "false", 2024-09-14T23:56:30.474 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-14T23:56:30.474 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-14T23:56:30.474 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_debug_no_require_reef": "false", 2024-09-14T23:56:30.474 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_debug_no_require_squid": "false", 2024-09-14T23:56:30.474 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-14T23:56:30.474 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-14T23:56:30.474 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-14T23:56:30.474 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_down_mkfs_grace": "120", 2024-09-14T23:56:30.474 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_down_uptime_grace": "60", 2024-09-14T23:56:30.474 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_election_default_strategy": "1", 2024-09-14T23:56:30.475 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_election_timeout": "5.000000", 2024-09-14T23:56:30.475 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-14T23:56:30.475 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_elector_ping_divisor": "2", 2024-09-14T23:56:30.475 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-14T23:56:30.475 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_enable_op_tracker": "true", 2024-09-14T23:56:30.475 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_fake_pool_delete": "false", 2024-09-14T23:56:30.475 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_force_quorum_join": "false", 2024-09-14T23:56:30.475 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_fsmap_prune_threshold": "300", 2024-09-14T23:56:30.475 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_globalid_prealloc": "10000", 2024-09-14T23:56:30.475 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_health_detail_to_clog": "false", 2024-09-14T23:56:30.475 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_health_log_update_period": "5", 2024-09-14T23:56:30.475 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_health_max_detail": "50", 2024-09-14T23:56:30.475 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_health_to_clog": "true", 2024-09-14T23:56:30.475 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_health_to_clog_interval": "600", 2024-09-14T23:56:30.476 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-14T23:56:30.476 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_host": "172.21.15.42,[v2:172.21.15.42:3301,v1:172.21.15.42:6790],172.21.15.43", 2024-09-14T23:56:30.476 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_host_override": "", 2024-09-14T23:56:30.476 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_initial_members": "", 2024-09-14T23:56:30.476 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-14T23:56:30.476 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-14T23:56:30.476 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-14T23:56:30.476 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-14T23:56:30.476 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-14T23:56:30.476 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_lease": "5.000000", 2024-09-14T23:56:30.476 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-14T23:56:30.476 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-14T23:56:30.476 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_log_full_interval": "50", 2024-09-14T23:56:30.476 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_log_max": "10000", 2024-09-14T23:56:30.477 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_log_max_summary": "50", 2024-09-14T23:56:30.477 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-14T23:56:30.477 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_max_log_epochs": "500", 2024-09-14T23:56:30.477 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-14T23:56:30.477 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-14T23:56:30.477 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_max_osd": "10000", 2024-09-14T23:56:30.477 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_max_pg_per_osd": "10000", 2024-09-14T23:56:30.477 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_max_pool_pg_num": "65536", 2024-09-14T23:56:30.477 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-14T23:56:30.477 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-14T23:56:30.477 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_mds_force_trim_to": "0", 2024-09-14T23:56:30.477 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_mds_skip_sanity": "false", 2024-09-14T23:56:30.477 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_memory_autotune": "true", 2024-09-14T23:56:30.477 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_memory_target": "2147483648", 2024-09-14T23:56:30.478 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_mgr_beacon_grace": "30", 2024-09-14T23:56:30.478 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-14T23:56:30.478 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_mgr_digest_period": "5", 2024-09-14T23:56:30.478 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_mgr_inactive_grace": "60", 2024-09-14T23:56:30.478 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-14T23:56:30.478 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-14T23:56:30.478 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_min_osdmap_epochs": "500", 2024-09-14T23:56:30.478 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_op_complaint_time": "30", 2024-09-14T23:56:30.478 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_op_history_duration": "600", 2024-09-14T23:56:30.478 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_op_history_size": "20", 2024-09-14T23:56:30.478 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_op_history_slow_op_size": "20", 2024-09-14T23:56:30.478 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-14T23:56:30.478 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_op_log_threshold": "5", 2024-09-14T23:56:30.479 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-14T23:56:30.479 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-14T23:56:30.479 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-14T23:56:30.479 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_auto_mark_in": "false", 2024-09-14T23:56:30.479 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-14T23:56:30.479 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-14T23:56:30.479 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-14T23:56:30.479 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_cache_size": "500", 2024-09-14T23:56:30.479 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-14T23:56:30.479 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-14T23:56:30.479 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-14T23:56:30.479 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_down_out_interval": "600", 2024-09-14T23:56:30.479 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-14T23:56:30.479 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-14T23:56:30.479 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_force_trim_to": "0", 2024-09-14T23:56:30.480 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-14T23:56:30.480 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-14T23:56:30.480 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-14T23:56:30.480 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-14T23:56:30.480 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-14T23:56:30.480 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-14T23:56:30.480 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-14T23:56:30.480 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-14T23:56:30.480 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_min_down_reporters": "2", 2024-09-14T23:56:30.480 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-14T23:56:30.480 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-14T23:56:30.480 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-14T23:56:30.481 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-14T23:56:30.481 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-14T23:56:30.481 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-14T23:56:30.481 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_report_timeout": "900", 2024-09-14T23:56:30.481 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-14T23:56:30.481 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-14T23:56:30.481 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-14T23:56:30.481 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-14T23:56:30.481 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-14T23:56:30.481 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-14T23:56:30.481 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-14T23:56:30.481 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-14T23:56:30.481 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-14T23:56:30.481 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_pg_stuck_threshold": "60", 2024-09-14T23:56:30.482 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-14T23:56:30.482 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-14T23:56:30.482 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-14T23:56:30.482 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-14T23:56:30.482 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-14T23:56:30.482 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-14T23:56:30.482 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_probe_timeout": "2.000000", 2024-09-14T23:56:30.482 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_reweight_max_change": "0.050000", 2024-09-14T23:56:30.482 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_reweight_max_osds": "4", 2024-09-14T23:56:30.482 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-14T23:56:30.482 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-14T23:56:30.482 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-14T23:56:30.482 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-14T23:56:30.483 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-14T23:56:30.483 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_scrub_interval": "86400", 2024-09-14T23:56:30.483 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_scrub_max_keys": "100", 2024-09-14T23:56:30.483 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_scrub_timeout": "300", 2024-09-14T23:56:30.483 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_session_timeout": "300", 2024-09-14T23:56:30.483 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_smart_report_timeout": "5", 2024-09-14T23:56:30.483 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_stat_smooth_intervals": "6", 2024-09-14T23:56:30.483 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-14T23:56:30.483 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_stretch_pool_min_size": "2", 2024-09-14T23:56:30.483 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_stretch_pool_size": "4", 2024-09-14T23:56:30.483 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-14T23:56:30.483 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-14T23:56:30.483 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_sync_debug": "false", 2024-09-14T23:56:30.483 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-14T23:56:30.484 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-14T23:56:30.484 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_sync_provider_kill_at": "0", 2024-09-14T23:56:30.484 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_sync_requester_kill_at": "0", 2024-09-14T23:56:30.484 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_sync_timeout": "60.000000", 2024-09-14T23:56:30.484 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_target_pg_per_osd": "100", 2024-09-14T23:56:30.484 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_tick_interval": "5", 2024-09-14T23:56:30.484 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_timecheck_interval": "300.000000", 2024-09-14T23:56:30.484 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-14T23:56:30.484 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_use_min_delay_socket": "false", 2024-09-14T23:56:30.484 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_older_version_delay": "604800", 2024-09-14T23:56:30.484 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-14T23:56:30.484 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-14T23:56:30.484 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-14T23:56:30.484 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-14T23:56:30.485 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-14T23:56:30.485 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-14T23:56:30.485 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-14T23:56:30.485 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_misplaced": "false", 2024-09-14T23:56:30.485 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-14T23:56:30.485 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_older_version": "true", 2024-09-14T23:56:30.485 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-14T23:56:30.485 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-14T23:56:30.485 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-14T23:56:30.485 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-14T23:56:30.485 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-14T23:56:30.485 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-14T23:56:30.485 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-14T23:56:30.485 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-14T23:56:30.486 INFO:teuthology.orchestra.run.smithi042.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-14T23:56:30.486 INFO:teuthology.orchestra.run.smithi042.stdout: "monmap": "", 2024-09-14T23:56:30.486 INFO:teuthology.orchestra.run.smithi042.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-14T23:56:30.486 INFO:teuthology.orchestra.run.smithi042.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-14T23:56:30.486 INFO:teuthology.orchestra.run.smithi042.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-14T23:56:30.486 INFO:teuthology.orchestra.run.smithi042.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-14T23:56:30.486 INFO:teuthology.orchestra.run.smithi042.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-14T23:56:30.486 INFO:teuthology.orchestra.run.smithi042.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-14T23:56:30.486 INFO:teuthology.orchestra.run.smithi042.stdout: "motr_tracing_enabled": "false", 2024-09-14T23:56:30.486 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_op_threads": "3", 2024-09-14T23:56:30.486 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-14T23:56:30.486 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_cm": "false", 2024-09-14T23:56:30.486 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_device_name": "", 2024-09-14T23:56:30.487 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_dscp": "96", 2024-09-14T23:56:30.487 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-14T23:56:30.487 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-14T23:56:30.487 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_local_gid": "", 2024-09-14T23:56:30.487 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-14T23:56:30.487 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_port_num": "1", 2024-09-14T23:56:30.487 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-14T23:56:30.487 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-14T23:56:30.487 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-14T23:56:30.487 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-14T23:56:30.487 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_sl": "3", 2024-09-14T23:56:30.487 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_support_srq": "true", 2024-09-14T23:56:30.487 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_rdma_type": "ib", 2024-09-14T23:56:30.487 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_async_reap_threshold": "5", 2024-09-14T23:56:30.488 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_bind_before_connect": "false", 2024-09-14T23:56:30.488 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_bind_ipv4": "true", 2024-09-14T23:56:30.488 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_bind_ipv6": "false", 2024-09-14T23:56:30.488 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_bind_msgr1": "true", 2024-09-14T23:56:30.488 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_bind_msgr2": "true", 2024-09-14T23:56:30.488 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_bind_port_max": "7568", 2024-09-14T23:56:30.488 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_bind_port_min": "6800", 2024-09-14T23:56:30.488 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-14T23:56:30.488 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_bind_retry_count": "3", 2024-09-14T23:56:30.488 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_bind_retry_delay": "5", 2024-09-14T23:56:30.488 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_blackhole_client": "false", 2024-09-14T23:56:30.488 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_blackhole_mds": "false", 2024-09-14T23:56:30.488 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_blackhole_mgr": "false", 2024-09-14T23:56:30.488 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_blackhole_mon": "false", 2024-09-14T23:56:30.489 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_blackhole_osd": "false", 2024-09-14T23:56:30.489 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_client_mode": "crc secure", 2024-09-14T23:56:30.489 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_cluster_mode": "crc secure", 2024-09-14T23:56:30.489 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_cluster_type": "", 2024-09-14T23:56:30.489 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_compress_secure": "false", 2024-09-14T23:56:30.489 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_connection_idle_timeout": "900", 2024-09-14T23:56:30.489 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_connection_ready_timeout": "10", 2024-09-14T23:56:30.489 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_crc_data": "true", 2024-09-14T23:56:30.489 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_crc_header": "true", 2024-09-14T23:56:30.489 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_die_on_bad_msg": "false", 2024-09-14T23:56:30.489 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_die_on_bug": "true", 2024-09-14T23:56:30.489 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_die_on_old_message": "true", 2024-09-14T23:56:30.489 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_die_on_skipped_message": "false", 2024-09-14T23:56:30.490 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-14T23:56:30.490 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-14T23:56:30.490 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_coremask": "0xF", 2024-09-14T23:56:30.490 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-14T23:56:30.490 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-14T23:56:30.490 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_enable_tso": "true", 2024-09-14T23:56:30.490 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-14T23:56:30.490 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-14T23:56:30.490 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_hugepages": "", 2024-09-14T23:56:30.490 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-14T23:56:30.490 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-14T23:56:30.490 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_lro": "true", 2024-09-14T23:56:30.490 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_memory_channel": "4", 2024-09-14T23:56:30.490 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-14T23:56:30.491 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_pmd": "", 2024-09-14T23:56:30.491 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_port_id": "0", 2024-09-14T23:56:30.491 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-14T23:56:30.491 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-14T23:56:30.491 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_dump_on_send": "false", 2024-09-14T23:56:30.491 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_initial_backoff": "0.200000", 2024-09-14T23:56:30.491 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_inject_delay_max": "1.000000", 2024-09-14T23:56:30.491 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-14T23:56:30.491 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_inject_delay_type": "", 2024-09-14T23:56:30.491 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-14T23:56:30.491 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_inject_network_congestion": "0", 2024-09-14T23:56:30.491 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_inject_socket_failures": "5000", 2024-09-14T23:56:30.491 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_learn_addr_from_peer": "true", 2024-09-14T23:56:30.492 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_max_accept_failures": "4", 2024-09-14T23:56:30.492 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_max_backoff": "15.000000", 2024-09-14T23:56:30.492 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_mon_client_mode": "secure crc", 2024-09-14T23:56:30.492 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-14T23:56:30.492 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_mon_service_mode": "secure crc", 2024-09-14T23:56:30.492 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_osd_compress_min_size": "1024", 2024-09-14T23:56:30.492 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_osd_compress_mode": "none", 2024-09-14T23:56:30.492 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-14T23:56:30.492 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-14T23:56:30.492 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_pq_min_cost": "65536", 2024-09-14T23:56:30.492 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_public_type": "", 2024-09-14T23:56:30.492 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_service_mode": "crc secure", 2024-09-14T23:56:30.492 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_tcp_listen_backlog": "512", 2024-09-14T23:56:30.493 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_tcp_nodelay": "true", 2024-09-14T23:56:30.493 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-14T23:56:30.493 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_tcp_rcvbuf": "0", 2024-09-14T23:56:30.493 INFO:teuthology.orchestra.run.smithi042.stdout: "ms_type": "async+posix", 2024-09-14T23:56:30.493 INFO:teuthology.orchestra.run.smithi042.stdout: "no_config_file": "false", 2024-09-14T23:56:30.493 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter_completion_locks_per_session": "32", 2024-09-14T23:56:30.493 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-14T23:56:30.493 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-14T23:56:30.493 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter_inflight_ops": "1024", 2024-09-14T23:56:30.493 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-14T23:56:30.493 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-14T23:56:30.493 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter_tick_interval": "5.000000", 2024-09-14T23:56:30.493 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter_timeout": "10.000000", 2024-09-14T23:56:30.493 INFO:teuthology.orchestra.run.smithi042.stdout: "objectstore_blackhole": "false", 2024-09-14T23:56:30.494 INFO:teuthology.orchestra.run.smithi042.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-09-14T23:56:30.494 INFO:teuthology.orchestra.run.smithi042.stdout: "openssl_engine_opts": "", 2024-09-14T23:56:30.494 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_agent_delay_time": "5.000000", 2024-09-14T23:56:30.494 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_agent_hist_halflife": "1000", 2024-09-14T23:56:30.494 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_agent_max_low_ops": "2", 2024-09-14T23:56:30.494 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_agent_max_ops": "4", 2024-09-14T23:56:30.494 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-14T23:56:30.494 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-14T23:56:30.494 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_agent_slop": "0.020000", 2024-09-14T23:56:30.494 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-14T23:56:30.494 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-14T23:56:30.494 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_asio_thread_count": "2", 2024-09-14T23:56:30.494 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_async_recovery_min_cost": "100", 2024-09-14T23:56:30.495 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-14T23:56:30.495 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-14T23:56:30.495 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_backfill_scan_max": "512", 2024-09-14T23:56:30.495 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_backfill_scan_min": "64", 2024-09-14T23:56:30.495 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_backoff_on_degraded": "false", 2024-09-14T23:56:30.495 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_backoff_on_peering": "false", 2024-09-14T23:56:30.495 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_backoff_on_unfound": "true", 2024-09-14T23:56:30.495 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_beacon_report_interval": "300", 2024-09-14T23:56:30.495 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bench_duration": "30", 2024-09-14T23:56:30.495 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-14T23:56:30.495 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bench_max_block_size": "67108864", 2024-09-14T23:56:30.495 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-14T23:56:30.495 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_blkin_trace_all": "false", 2024-09-14T23:56:30.496 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-14T23:56:30.496 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-14T23:56:30.496 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-14T23:56:30.496 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-14T23:56:30.496 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_check_for_log_corruption": "false", 2024-09-14T23:56:30.496 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-14T23:56:30.496 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-14T23:56:30.496 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-09-14T23:56:30.496 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-14T23:56:30.496 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_class_update_on_start": "true", 2024-09-14T23:56:30.496 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_client_message_cap": "256", 2024-09-14T23:56:30.496 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_client_message_size_cap": "524288000", 2024-09-14T23:56:30.496 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_client_op_priority": "63", 2024-09-14T23:56:30.497 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_client_watch_timeout": "30", 2024-09-14T23:56:30.497 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_command_max_records": "256", 2024-09-14T23:56:30.497 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_compact_on_start": "false", 2024-09-14T23:56:30.497 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-14T23:56:30.497 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-14T23:56:30.497 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-14T23:56:30.497 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_crush_update_on_start": "true", 2024-09-14T23:56:30.497 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_crush_update_weight_set": "true", 2024-09-14T23:56:30.497 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-14T23:56:30.497 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-14T23:56:30.497 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-14T23:56:30.497 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-14T23:56:30.498 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-14T23:56:30.498 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_feed_pullee": "-1", 2024-09-14T23:56:30.498 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-14T23:56:30.498 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-14T23:56:30.498 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-14T23:56:30.498 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_misdirected_ops": "false", 2024-09-14T23:56:30.498 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_no_acting_change": "false", 2024-09-14T23:56:30.498 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_no_purge_strays": "false", 2024-09-14T23:56:30.498 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_op_order": "false", 2024-09-14T23:56:30.498 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-14T23:56:30.498 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-14T23:56:30.498 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-14T23:56:30.498 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-14T23:56:30.498 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_shutdown": "false", 2024-09-14T23:56:30.499 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-14T23:56:30.499 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-14T23:56:30.499 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_trim_objects": "false", 2024-09-14T23:56:30.499 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-14T23:56:30.499 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-14T23:56:30.499 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_verify_snaps": "false", 2024-09-14T23:56:30.499 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-14T23:56:30.499 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-14T23:56:30.499 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_deep_scrub_keys": "1024", 2024-09-14T23:56:30.499 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-14T23:56:30.499 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-14T23:56:30.499 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-14T23:56:30.499 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_deep_scrub_stride": "524288", 2024-09-14T23:56:30.500 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-14T23:56:30.500 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-14T23:56:30.500 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_default_notify_timeout": "30", 2024-09-14T23:56:30.500 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_delete_sleep": "0.000000", 2024-09-14T23:56:30.500 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-14T23:56:30.500 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-14T23:56:30.500 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-14T23:56:30.500 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_discard_disconnected_ops": "true", 2024-09-14T23:56:30.500 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_enable_op_tracker": "true", 2024-09-14T23:56:30.500 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-14T23:56:30.500 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-14T23:56:30.500 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-14T23:56:30.500 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-14T23:56:30.501 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_fast_info": "true", 2024-09-14T23:56:30.501 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_fast_shutdown": "true", 2024-09-14T23:56:30.501 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-14T23:56:30.501 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-14T23:56:30.501 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-14T23:56:30.501 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-14T23:56:30.501 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-14T23:56:30.501 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_function_tracing": "false", 2024-09-14T23:56:30.501 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_heartbeat_grace": "20", 2024-09-14T23:56:30.501 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_heartbeat_interval": "6", 2024-09-14T23:56:30.501 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-14T23:56:30.501 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_heartbeat_min_peers": "10", 2024-09-14T23:56:30.501 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_heartbeat_min_size": "2000", 2024-09-14T23:56:30.502 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_heartbeat_stale": "600", 2024-09-14T23:56:30.502 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-14T23:56:30.502 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_hit_set_max_size": "100000", 2024-09-14T23:56:30.502 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_hit_set_min_size": "1000", 2024-09-14T23:56:30.502 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-14T23:56:30.502 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-14T23:56:30.502 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-14T23:56:30.502 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-14T23:56:30.502 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-14T23:56:30.502 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-14T23:56:30.502 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_journal_size": "5120", 2024-09-14T23:56:30.502 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_kill_backfill_at": "0", 2024-09-14T23:56:30.502 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-14T23:56:30.502 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_map_cache_size": "50", 2024-09-14T23:56:30.503 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_map_dedup": "true", 2024-09-14T23:56:30.503 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_map_message_max": "40", 2024-09-14T23:56:30.503 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-14T23:56:30.503 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_map_share_max_epochs": "40", 2024-09-14T23:56:30.503 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_attr_name_len": "100", 2024-09-14T23:56:30.503 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_attr_size": "0", 2024-09-14T23:56:30.503 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_backfills": "1", 2024-09-14T23:56:30.503 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_markdown_count": "5", 2024-09-14T23:56:30.503 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_markdown_period": "600", 2024-09-14T23:56:30.503 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_object_name_len": "2048", 2024-09-14T23:56:30.503 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_object_namespace_len": "256", 2024-09-14T23:56:30.503 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_object_size": "134217728", 2024-09-14T23:56:30.503 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-14T23:56:30.504 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-14T23:56:30.504 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_pg_blocked_by": "16", 2024-09-14T23:56:30.504 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_pg_log_entries": "10000", 2024-09-14T23:56:30.504 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-14T23:56:30.504 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_pgls": "1024", 2024-09-14T23:56:30.504 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_push_cost": "8388608", 2024-09-14T23:56:30.504 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_push_objects": "10", 2024-09-14T23:56:30.504 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_scrubs": "3", 2024-09-14T23:56:30.504 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-14T23:56:30.504 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_trimming_pgs": "2", 2024-09-14T23:56:30.504 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_write_op_reply_len": "64", 2024-09-14T23:56:30.504 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_max_write_size": "90", 2024-09-14T23:56:30.504 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-14T23:56:30.504 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-14T23:56:30.505 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-14T23:56:30.505 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-14T23:56:30.505 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-14T23:56:30.505 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-14T23:56:30.505 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-14T23:56:30.505 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-14T23:56:30.505 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_profile": "balanced", 2024-09-14T23:56:30.505 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-14T23:56:30.505 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-14T23:56:30.505 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-14T23:56:30.505 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-14T23:56:30.505 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-14T23:56:30.505 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-14T23:56:30.506 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-14T23:56:30.506 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-14T23:56:30.506 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-14T23:56:30.506 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-14T23:56:30.506 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-14T23:56:30.506 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_memory_base": "805306368", 2024-09-14T23:56:30.506 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_memory_cache_min": "134217728", 2024-09-14T23:56:30.506 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-14T23:56:30.506 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-14T23:56:30.506 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_memory_target": "4294967296", 2024-09-14T23:56:30.506 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_memory_target_autotune": "false", 2024-09-14T23:56:30.506 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-14T23:56:30.506 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_min_pg_log_entries": "250", 2024-09-14T23:56:30.506 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_min_recovery_priority": "0", 2024-09-14T23:56:30.507 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-14T23:56:30.507 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-14T23:56:30.507 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mon_report_interval": "5", 2024-09-14T23:56:30.507 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-14T23:56:30.507 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-14T23:56:30.507 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_num_cache_shards": "32", 2024-09-14T23:56:30.507 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_num_op_tracker_shard": "32", 2024-09-14T23:56:30.507 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_numa_auto_affinity": "true", 2024-09-14T23:56:30.507 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_numa_node": "-1", 2024-09-14T23:56:30.507 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_numa_prefer_iface": "true", 2024-09-14T23:56:30.507 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-14T23:56:30.507 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_objecter_finishers": "1", 2024-09-14T23:56:30.507 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_objectstore": "bluestore", 2024-09-14T23:56:30.507 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_objectstore_fuse": "false", 2024-09-14T23:56:30.508 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_objectstore_tracing": "false", 2024-09-14T23:56:30.508 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_complaint_time": "30.000000", 2024-09-14T23:56:30.508 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_history_duration": "600", 2024-09-14T23:56:30.508 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_history_size": "20", 2024-09-14T23:56:30.508 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_history_slow_op_size": "20", 2024-09-14T23:56:30.508 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-14T23:56:30.508 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_log_threshold": "5", 2024-09-14T23:56:30.508 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_num_shards": "0", 2024-09-14T23:56:30.508 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_num_shards_hdd": "5", 2024-09-14T23:56:30.508 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_num_shards_ssd": "8", 2024-09-14T23:56:30.508 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-14T23:56:30.508 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-14T23:56:30.508 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-14T23:56:30.509 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-14T23:56:30.509 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_pq_min_cost": "65536", 2024-09-14T23:56:30.509 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-14T23:56:30.509 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_queue_cut_off": "high", 2024-09-14T23:56:30.509 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-14T23:56:30.509 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_op_thread_timeout": "15", 2024-09-14T23:56:30.509 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_open_classes_on_start": "true", 2024-09-14T23:56:30.509 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_os_flags": "0", 2024-09-14T23:56:30.509 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_peering_op_priority": "255", 2024-09-14T23:56:30.509 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pg_delete_cost": "1048576", 2024-09-14T23:56:30.509 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pg_delete_priority": "5", 2024-09-14T23:56:30.509 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-14T23:56:30.509 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-14T23:56:30.509 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-14T23:56:30.510 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pg_log_trim_max": "10000", 2024-09-14T23:56:30.510 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pg_log_trim_min": "100", 2024-09-14T23:56:30.510 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-14T23:56:30.510 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-14T23:56:30.510 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-09-14T23:56:30.510 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-09-14T23:56:30.510 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-14T23:56:30.510 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-14T23:56:30.510 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-14T23:56:30.510 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-14T23:56:30.510 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-14T23:56:30.510 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-14T23:56:30.510 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_crimson": "false", 2024-09-14T23:56:30.510 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-14T23:56:30.511 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-14T23:56:30.511 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-14T23:56:30.511 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-14T23:56:30.511 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-14T23:56:30.511 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-14T23:56:30.511 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-14T23:56:30.511 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-14T23:56:30.511 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_flags": "0", 2024-09-14T23:56:30.511 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-14T23:56:30.511 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_min_size": "0", 2024-09-14T23:56:30.511 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-14T23:56:30.511 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_pg_num": "32", 2024-09-14T23:56:30.511 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_pgp_num": "0", 2024-09-14T23:56:30.512 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-14T23:56:30.512 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_read_ratio": "70", 2024-09-14T23:56:30.512 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_size": "2", 2024-09-14T23:56:30.512 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_default_type": "replicated", 2024-09-14T23:56:30.512 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-14T23:56:30.512 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-14T23:56:30.512 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_push_per_object_cost": "1000", 2024-09-14T23:56:30.512 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-14T23:56:30.512 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recover_clone_overlap": "true", 2024-09-14T23:56:30.512 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-14T23:56:30.512 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_cost": "20971520", 2024-09-14T23:56:30.512 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-14T23:56:30.512 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_max_active": "0", 2024-09-14T23:56:30.512 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-14T23:56:30.513 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-14T23:56:30.513 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-14T23:56:30.513 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-14T23:56:30.513 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_max_single_start": "1", 2024-09-14T23:56:30.513 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_op_priority": "3", 2024-09-14T23:56:30.513 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-14T23:56:30.513 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_priority": "5", 2024-09-14T23:56:30.513 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-14T23:56:30.513 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_sleep": "0.000000", 2024-09-14T23:56:30.513 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-14T23:56:30.513 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-14T23:56:30.513 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-14T23:56:30.513 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_repair_during_recovery": "false", 2024-09-14T23:56:30.514 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_requested_scrub_priority": "120", 2024-09-14T23:56:30.514 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-14T23:56:30.514 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-14T23:56:30.514 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_auto_repair": "false", 2024-09-14T23:56:30.514 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-14T23:56:30.514 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-14T23:56:30.514 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_begin_hour": "0", 2024-09-14T23:56:30.514 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_begin_week_day": "0", 2024-09-14T23:56:30.514 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_chunk_max": "25", 2024-09-14T23:56:30.514 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_chunk_min": "5", 2024-09-14T23:56:30.514 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_cost": "52428800", 2024-09-14T23:56:30.514 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-09-14T23:56:30.514 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_during_recovery": "false", 2024-09-14T23:56:30.514 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_end_hour": "0", 2024-09-14T23:56:30.515 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_end_week_day": "0", 2024-09-14T23:56:30.515 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_event_cost": "4096", 2024-09-14T23:56:30.515 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-14T23:56:30.515 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-14T23:56:30.515 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_invalid_stats": "true", 2024-09-14T23:56:30.515 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-14T23:56:30.515 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-14T23:56:30.515 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_max_preemptions": "5", 2024-09-14T23:56:30.515 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-14T23:56:30.515 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_priority": "5", 2024-09-14T23:56:30.515 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_scrub_sleep": "0.000000", 2024-09-14T23:56:30.515 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-14T23:56:30.515 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-14T23:56:30.516 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-14T23:56:30.516 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_skip_data_digest": "false", 2024-09-14T23:56:30.516 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_smart_report_timeout": "5", 2024-09-14T23:56:30.516 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_snap_trim_cost": "1048576", 2024-09-14T23:56:30.516 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_snap_trim_priority": "5", 2024-09-14T23:56:30.516 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-14T23:56:30.516 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-14T23:56:30.516 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-14T23:56:30.516 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-14T23:56:30.516 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-14T23:56:30.516 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-14T23:56:30.516 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-14T23:56:30.516 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_target_transaction_size": "30", 2024-09-14T23:56:30.517 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-14T23:56:30.517 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-14T23:56:30.517 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-14T23:56:30.517 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-14T23:56:30.517 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-14T23:56:30.517 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-14T23:56:30.517 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-14T23:56:30.517 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-14T23:56:30.517 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-14T23:56:30.517 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-14T23:56:30.517 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_tracing": "false", 2024-09-14T23:56:30.517 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_use_stale_snap": "false", 2024-09-14T23:56:30.517 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-14T23:56:30.517 INFO:teuthology.orchestra.run.smithi042.stdout: "osdc_blkin_trace_all": "false", 2024-09-14T23:56:30.518 INFO:teuthology.orchestra.run.smithi042.stdout: "paxos_kill_at": "0", 2024-09-14T23:56:30.518 INFO:teuthology.orchestra.run.smithi042.stdout: "paxos_max_join_drift": "10", 2024-09-14T23:56:30.518 INFO:teuthology.orchestra.run.smithi042.stdout: "paxos_min": "500", 2024-09-14T23:56:30.518 INFO:teuthology.orchestra.run.smithi042.stdout: "paxos_min_wait": "0.050000", 2024-09-14T23:56:30.518 INFO:teuthology.orchestra.run.smithi042.stdout: "paxos_propose_interval": "1.000000", 2024-09-14T23:56:30.518 INFO:teuthology.orchestra.run.smithi042.stdout: "paxos_service_trim_max": "500", 2024-09-14T23:56:30.518 INFO:teuthology.orchestra.run.smithi042.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-14T23:56:30.518 INFO:teuthology.orchestra.run.smithi042.stdout: "paxos_service_trim_min": "250", 2024-09-14T23:56:30.518 INFO:teuthology.orchestra.run.smithi042.stdout: "paxos_stash_full_interval": "25", 2024-09-14T23:56:30.518 INFO:teuthology.orchestra.run.smithi042.stdout: "paxos_trim_max": "500", 2024-09-14T23:56:30.518 INFO:teuthology.orchestra.run.smithi042.stdout: "paxos_trim_min": "250", 2024-09-14T23:56:30.518 INFO:teuthology.orchestra.run.smithi042.stdout: "perf": "true", 2024-09-14T23:56:30.518 INFO:teuthology.orchestra.run.smithi042.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-14T23:56:30.519 INFO:teuthology.orchestra.run.smithi042.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-14T23:56:30.519 INFO:teuthology.orchestra.run.smithi042.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-09-14T23:56:30.519 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "-", 2024-09-14T23:56:30.519 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrv": "", 2024-09-14T23:56:30.519 INFO:teuthology.orchestra.run.smithi042.stdout: "public_bind_addr": "-", 2024-09-14T23:56:30.519 INFO:teuthology.orchestra.run.smithi042.stdout: "public_network": "", 2024-09-14T23:56:30.519 INFO:teuthology.orchestra.run.smithi042.stdout: "public_network_interface": "", 2024-09-14T23:56:30.519 INFO:teuthology.orchestra.run.smithi042.stdout: "qat_compressor_busy_polling": "false", 2024-09-14T23:56:30.519 INFO:teuthology.orchestra.run.smithi042.stdout: "qat_compressor_enabled": "false", 2024-09-14T23:56:30.519 INFO:teuthology.orchestra.run.smithi042.stdout: "qat_compressor_session_max_number": "256", 2024-09-14T23:56:30.519 INFO:teuthology.orchestra.run.smithi042.stdout: "rados_mon_op_timeout": "0", 2024-09-14T23:56:30.519 INFO:teuthology.orchestra.run.smithi042.stdout: "rados_osd_op_timeout": "0", 2024-09-14T23:56:30.519 INFO:teuthology.orchestra.run.smithi042.stdout: "rados_tracing": "false", 2024-09-14T23:56:30.520 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_atime_update_interval": "60", 2024-09-14T23:56:30.520 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-14T23:56:30.520 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_balance_parent_reads": "false", 2024-09-14T23:56:30.520 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_balance_snap_reads": "false", 2024-09-14T23:56:30.520 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_blkin_trace_all": "false", 2024-09-14T23:56:30.520 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-14T23:56:30.520 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-14T23:56:30.520 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_cache": "true", 2024-09-14T23:56:30.520 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-14T23:56:30.520 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-14T23:56:30.520 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-14T23:56:30.520 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-14T23:56:30.520 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_cache_policy": "writearound", 2024-09-14T23:56:30.520 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_cache_size": "33554432", 2024-09-14T23:56:30.521 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-14T23:56:30.521 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-14T23:56:30.521 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_clone_copy_on_read": "false", 2024-09-14T23:56:30.521 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_compression_hint": "none", 2024-09-14T23:56:30.521 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_concurrent_management_ops": "10", 2024-09-14T23:56:30.521 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-14T23:56:30.521 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_default_clone_format": "auto", 2024-09-14T23:56:30.521 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_default_data_pool": "", 2024-09-14T23:56:30.521 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_default_features": "61", 2024-09-14T23:56:30.521 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_default_format": "2", 2024-09-14T23:56:30.521 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_default_map_options": "", 2024-09-14T23:56:30.521 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_default_order": "22", 2024-09-14T23:56:30.521 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_default_pool": "rbd", 2024-09-14T23:56:30.522 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-14T23:56:30.522 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_default_stripe_count": "0", 2024-09-14T23:56:30.522 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_default_stripe_unit": "0", 2024-09-14T23:56:30.522 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-14T23:56:30.522 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-14T23:56:30.522 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-14T23:56:30.522 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_enable_alloc_hint": "true", 2024-09-14T23:56:30.522 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-14T23:56:30.522 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_io_scheduler": "simple", 2024-09-14T23:56:30.522 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-14T23:56:30.522 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-14T23:56:30.522 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-14T23:56:30.522 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-14T23:56:30.523 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-14T23:56:30.523 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-14T23:56:30.523 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-14T23:56:30.523 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-14T23:56:30.523 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-14T23:56:30.523 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_journal_order": "24", 2024-09-14T23:56:30.523 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_journal_pool": "", 2024-09-14T23:56:30.523 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_journal_splay_width": "4", 2024-09-14T23:56:30.523 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_localize_parent_reads": "false", 2024-09-14T23:56:30.523 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_localize_snap_reads": "false", 2024-09-14T23:56:30.523 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-14T23:56:30.523 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-14T23:56:30.523 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-14T23:56:30.524 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-14T23:56:30.524 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-14T23:56:30.524 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-14T23:56:30.524 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-14T23:56:30.524 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-14T23:56:30.524 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-14T23:56:30.524 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-14T23:56:30.524 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-14T23:56:30.524 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-14T23:56:30.524 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-14T23:56:30.524 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-14T23:56:30.524 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-14T23:56:30.524 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-14T23:56:30.525 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-14T23:56:30.525 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-14T23:56:30.525 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-14T23:56:30.525 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-14T23:56:30.525 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-14T23:56:30.525 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-14T23:56:30.525 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-14T23:56:30.525 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-14T23:56:30.525 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-14T23:56:30.525 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-14T23:56:30.525 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-14T23:56:30.525 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-14T23:56:30.525 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-14T23:56:30.526 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-14T23:56:30.526 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-14T23:56:30.526 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_mtime_update_interval": "60", 2024-09-14T23:56:30.526 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_non_blocking_aio": "true", 2024-09-14T23:56:30.526 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_op_thread_timeout": "60", 2024-09-14T23:56:30.526 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_op_threads": "1", 2024-09-14T23:56:30.526 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_parent_cache_enabled": "false", 2024-09-14T23:56:30.526 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-14T23:56:30.526 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-14T23:56:30.526 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-14T23:56:30.526 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_plugins": "", 2024-09-14T23:56:30.526 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_bps_burst": "0", 2024-09-14T23:56:30.526 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-14T23:56:30.527 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_bps_limit": "0", 2024-09-14T23:56:30.527 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_exclude_ops": "0", 2024-09-14T23:56:30.527 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_iops_burst": "0", 2024-09-14T23:56:30.527 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-14T23:56:30.527 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_iops_limit": "0", 2024-09-14T23:56:30.527 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-14T23:56:30.527 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-14T23:56:30.527 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-14T23:56:30.527 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-14T23:56:30.527 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-14T23:56:30.527 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-14T23:56:30.527 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-14T23:56:30.527 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-14T23:56:30.527 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-14T23:56:30.528 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-14T23:56:30.528 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-14T23:56:30.528 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-14T23:56:30.528 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-14T23:56:30.528 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-14T23:56:30.528 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_read_from_replica_policy": "default", 2024-09-14T23:56:30.528 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-14T23:56:30.528 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-14T23:56:30.528 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-14T23:56:30.528 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-14T23:56:30.528 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_skip_partial_discard": "true", 2024-09-14T23:56:30.528 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-14T23:56:30.528 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_tracing": "false", 2024-09-14T23:56:30.529 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_validate_names": "true", 2024-09-14T23:56:30.529 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_validate_pool": "true", 2024-09-14T23:56:30.529 INFO:teuthology.orchestra.run.smithi042.stdout: "restapi_base_url": "", 2024-09-14T23:56:30.529 INFO:teuthology.orchestra.run.smithi042.stdout: "restapi_log_level": "", 2024-09-14T23:56:30.529 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-09-14T23:56:30.529 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_account_default_quota_max_size": "-1", 2024-09-14T23:56:30.529 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_acl_grants_max_num": "100", 2024-09-14T23:56:30.529 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_admin_entry": "admin", 2024-09-14T23:56:30.529 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-14T23:56:30.529 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_backend_store": "rados", 2024-09-14T23:56:30.529 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_barbican_url": "", 2024-09-14T23:56:30.529 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_beast_enable_async": "true", 2024-09-14T23:56:30.529 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_bucket_counters_cache": "false", 2024-09-14T23:56:30.530 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-09-14T23:56:30.530 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-14T23:56:30.530 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-14T23:56:30.530 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-14T23:56:30.530 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-14T23:56:30.530 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-14T23:56:30.530 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-14T23:56:30.530 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-14T23:56:30.530 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_cache_enabled": "true", 2024-09-14T23:56:30.530 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_cache_expiry_interval": "900", 2024-09-14T23:56:30.530 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_cache_lru_size": "25000", 2024-09-14T23:56:30.530 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_config_store": "rados", 2024-09-14T23:56:30.530 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_content_length_compat": "false", 2024-09-14T23:56:30.530 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_copy_obj_progress": "true", 2024-09-14T23:56:30.530 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-14T23:56:30.531 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_cors_rules_max_num": "100", 2024-09-14T23:56:30.531 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_cross_domain_policy": "", 2024-09-14T23:56:30.531 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-14T23:56:30.531 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_kmip_addr": "", 2024-09-14T23:56:30.531 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-14T23:56:30.531 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-14T23:56:30.531 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-14T23:56:30.531 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-14T23:56:30.531 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_kmip_password": "", 2024-09-14T23:56:30.531 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-14T23:56:30.531 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_kmip_username": "", 2024-09-14T23:56:30.531 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_require_ssl": "true", 2024-09-14T23:56:30.531 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-14T23:56:30.531 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-14T23:56:30.532 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-14T23:56:30.532 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-14T23:56:30.532 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-14T23:56:30.532 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-14T23:56:30.532 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-14T23:56:30.532 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-14T23:56:30.532 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-14T23:56:30.532 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-14T23:56:30.532 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-14T23:56:30.532 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-14T23:56:30.532 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-14T23:56:30.532 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-14T23:56:30.532 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-14T23:56:30.532 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_vault_addr": "", 2024-09-14T23:56:30.533 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_vault_auth": "token", 2024-09-14T23:56:30.533 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_vault_namespace": "", 2024-09-14T23:56:30.533 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_vault_prefix": "", 2024-09-14T23:56:30.533 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-14T23:56:30.533 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-14T23:56:30.533 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-14T23:56:30.533 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-14T23:56:30.533 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_vault_token_file": "", 2024-09-14T23:56:30.533 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-14T23:56:30.533 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_curl_buffersize": "524288", 2024-09-14T23:56:30.533 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-14T23:56:30.533 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_curl_low_speed_time": "300", 2024-09-14T23:56:30.533 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-14T23:56:30.534 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-14T23:56:30.534 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-14T23:56:30.534 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-14T23:56:30.534 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-14T23:56:30.534 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-14T23:56:30.534 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-14T23:56:30.534 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-14T23:56:30.534 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-14T23:56:30.534 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-14T23:56:30.534 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_d4n_host": "127.0.0.1", 2024-09-14T23:56:30.534 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_d4n_port": "6379", 2024-09-14T23:56:30.534 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-14T23:56:30.534 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_data_log_changes_size": "1000", 2024-09-14T23:56:30.535 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_data_log_num_shards": "128", 2024-09-14T23:56:30.535 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-14T23:56:30.535 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_data_log_window": "30", 2024-09-14T23:56:30.535 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-14T23:56:30.535 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-14T23:56:30.535 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-14T23:56:30.535 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-14T23:56:30.535 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-14T23:56:30.535 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-14T23:56:30.535 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-14T23:56:30.535 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-14T23:56:30.535 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-14T23:56:30.535 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-14T23:56:30.535 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-14T23:56:30.535 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-14T23:56:30.536 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-14T23:56:30.536 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-14T23:56:30.536 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-14T23:56:30.536 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-14T23:56:30.536 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-14T23:56:30.536 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-14T23:56:30.536 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-14T23:56:30.536 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-14T23:56:30.536 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-14T23:56:30.536 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-14T23:56:30.536 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-14T23:56:30.536 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-14T23:56:30.536 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-14T23:56:30.536 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dns_name": "", 2024-09-14T23:56:30.537 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dns_s3website_name": "", 2024-09-14T23:56:30.537 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_dynamic_resharding": "true", 2024-09-14T23:56:30.537 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-14T23:56:30.537 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_enable_gc_threads": "true", 2024-09-14T23:56:30.537 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_enable_lc_threads": "true", 2024-09-14T23:56:30.537 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_enable_ops_log": "false", 2024-09-14T23:56:30.537 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_enable_quota_threads": "true", 2024-09-14T23:56:30.537 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_enable_static_website": "false", 2024-09-14T23:56:30.537 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_enable_usage_log": "false", 2024-09-14T23:56:30.537 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_enforce_swift_acls": "true", 2024-09-14T23:56:30.537 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_exit_timeout_secs": "120", 2024-09-14T23:56:30.537 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_expose_bucket": "false", 2024-09-14T23:56:30.537 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_extended_http_attrs": "", 2024-09-14T23:56:30.538 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_filter": "none", 2024-09-14T23:56:30.538 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-14T23:56:30.538 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_frontends": "beast port=7480", 2024-09-14T23:56:30.538 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-14T23:56:30.538 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_gc_max_deferred": "50", 2024-09-14T23:56:30.538 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-14T23:56:30.538 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_gc_max_objs": "32", 2024-09-14T23:56:30.538 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-14T23:56:30.538 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-14T23:56:30.538 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-14T23:56:30.538 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-14T23:56:30.538 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_gc_processor_period": "3600", 2024-09-14T23:56:30.538 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-14T23:56:30.538 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-14T23:56:30.539 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-14T23:56:30.539 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-14T23:56:30.539 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_init_timeout": "300", 2024-09-14T23:56:30.539 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-14T23:56:30.539 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-14T23:56:30.539 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_kafka_connection_idle": "300", 2024-09-14T23:56:30.539 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-14T23:56:30.539 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_kafka_sleep_timeout": "10", 2024-09-14T23:56:30.539 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-14T23:56:30.539 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-09-14T23:56:30.539 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-14T23:56:30.539 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_admin_domain": "", 2024-09-14T23:56:30.539 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_admin_password": "", 2024-09-14T23:56:30.539 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_admin_password_path": "", 2024-09-14T23:56:30.540 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_admin_project": "", 2024-09-14T23:56:30.540 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_admin_tenant": "", 2024-09-14T23:56:30.540 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_admin_token": "", 2024-09-14T23:56:30.540 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_admin_token_path": "", 2024-09-14T23:56:30.540 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_admin_user": "", 2024-09-14T23:56:30.540 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_api_version": "2", 2024-09-14T23:56:30.540 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_barbican_domain": "", 2024-09-14T23:56:30.540 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_barbican_password": "", 2024-09-14T23:56:30.540 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_barbican_project": "", 2024-09-14T23:56:30.540 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-14T23:56:30.540 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_barbican_user": "", 2024-09-14T23:56:30.540 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-14T23:56:30.540 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-14T23:56:30.541 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-14T23:56:30.541 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-14T23:56:30.541 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-14T23:56:30.541 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_url": "", 2024-09-14T23:56:30.541 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-14T23:56:30.541 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_lc_debug_interval": "-1", 2024-09-14T23:56:30.541 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_lc_lock_max_time": "90", 2024-09-14T23:56:30.541 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_lc_max_objs": "32", 2024-09-14T23:56:30.541 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_lc_max_rules": "1000", 2024-09-14T23:56:30.541 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_lc_max_worker": "3", 2024-09-14T23:56:30.541 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-14T23:56:30.541 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_lc_thread_delay": "0", 2024-09-14T23:56:30.541 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-14T23:56:30.541 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_ldap_dnattr": "uid", 2024-09-14T23:56:30.542 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-14T23:56:30.542 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_ldap_searchfilter": "", 2024-09-14T23:56:30.542 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-14T23:56:30.542 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_ldap_uri": "", 2024-09-14T23:56:30.542 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-14T23:56:30.542 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-14T23:56:30.542 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-14T23:56:30.542 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_log_http_headers": "", 2024-09-14T23:56:30.542 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-14T23:56:30.542 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-14T23:56:30.542 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_log_object_name_utc": "false", 2024-09-14T23:56:30.542 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-09-14T23:56:30.542 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-09-14T23:56:30.542 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_max_attr_name_len": "0", 2024-09-14T23:56:30.543 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_max_attr_size": "0", 2024-09-14T23:56:30.543 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-14T23:56:30.543 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_max_chunk_size": "4194304", 2024-09-14T23:56:30.543 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-14T23:56:30.543 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-14T23:56:30.543 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-14T23:56:30.543 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_max_listing_results": "1000", 2024-09-14T23:56:30.543 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_max_notify_retries": "10", 2024-09-14T23:56:30.543 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-14T23:56:30.543 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_max_put_param_size": "1048576", 2024-09-14T23:56:30.543 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_max_put_size": "5368709120", 2024-09-14T23:56:30.543 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_max_slo_entries": "1000", 2024-09-14T23:56:30.543 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_md_log_max_shards": "64", 2024-09-14T23:56:30.543 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-14T23:56:30.544 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-14T23:56:30.544 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-14T23:56:30.544 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-14T23:56:30.544 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_mp_lock_max_time": "600", 2024-09-14T23:56:30.544 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-14T23:56:30.544 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-14T23:56:30.544 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-14T23:56:30.544 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-14T23:56:30.544 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-14T23:56:30.544 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-14T23:56:30.544 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-14T23:56:30.544 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-14T23:56:30.544 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_nfs_max_gc": "300", 2024-09-14T23:56:30.545 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-14T23:56:30.545 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-14T23:56:30.545 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-14T23:56:30.545 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-14T23:56:30.545 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-14T23:56:30.545 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-14T23:56:30.545 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-14T23:56:30.545 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_num_async_rados_threads": "32", 2024-09-14T23:56:30.545 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_num_control_oids": "8", 2024-09-14T23:56:30.545 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_numa_node": "-1", 2024-09-14T23:56:30.545 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-14T23:56:30.545 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-14T23:56:30.545 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_objexp_chunk_size": "100", 2024-09-14T23:56:30.546 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_objexp_gc_interval": "600", 2024-09-14T23:56:30.546 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-14T23:56:30.546 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-14T23:56:30.546 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-14T23:56:30.546 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_op_thread_timeout": "600", 2024-09-14T23:56:30.546 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_op_tracing": "false", 2024-09-14T23:56:30.546 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_opa_token": "", 2024-09-14T23:56:30.546 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_opa_url": "", 2024-09-14T23:56:30.546 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_opa_verify_ssl": "true", 2024-09-14T23:56:30.546 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-14T23:56:30.546 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-14T23:56:30.546 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_ops_log_rados": "false", 2024-09-14T23:56:30.546 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_ops_log_socket_path": "", 2024-09-14T23:56:30.546 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-14T23:56:30.547 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-14T23:56:30.547 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-14T23:56:30.547 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_period_push_interval": "2.000000", 2024-09-14T23:56:30.547 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-14T23:56:30.547 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-14T23:56:30.547 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-14T23:56:30.547 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-09-14T23:56:30.547 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_posix_cache_lanes": "3", 2024-09-14T23:56:30.547 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-09-14T23:56:30.547 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_posix_cache_max_buckets": "100", 2024-09-14T23:56:30.547 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_posix_cache_partitions": "3", 2024-09-14T23:56:30.547 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-09-14T23:56:30.547 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_print_continue": "true", 2024-09-14T23:56:30.548 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-14T23:56:30.548 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-14T23:56:30.548 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-14T23:56:30.548 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-14T23:56:30.548 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-14T23:56:30.548 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_rados_tracing": "false", 2024-09-14T23:56:30.548 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_realm": "", 2024-09-14T23:56:30.548 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_realm_id": "", 2024-09-14T23:56:30.548 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-14T23:56:30.548 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_region": "", 2024-09-14T23:56:30.548 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-14T23:56:30.548 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-14T23:56:30.548 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-14T23:56:30.548 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-14T23:56:30.549 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_request_uri": "", 2024-09-14T23:56:30.549 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_reshard_batch_size": "64", 2024-09-14T23:56:30.549 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-14T23:56:30.549 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_reshard_max_aio": "128", 2024-09-14T23:56:30.549 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_reshard_num_logs": "16", 2024-09-14T23:56:30.549 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_reshard_thread_interval": "600", 2024-09-14T23:56:30.549 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_resolve_cname": "false", 2024-09-14T23:56:30.549 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-14T23:56:30.549 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_run_sync_thread": "true", 2024-09-14T23:56:30.549 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-14T23:56:30.549 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-14T23:56:30.549 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-14T23:56:30.549 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-14T23:56:30.549 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-14T23:56:30.550 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-14T23:56:30.550 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-14T23:56:30.550 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-14T23:56:30.550 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-14T23:56:30.550 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_scheduler_type": "throttler", 2024-09-14T23:56:30.550 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_script_uri": "", 2024-09-14T23:56:30.550 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_service_provider_name": "", 2024-09-14T23:56:30.550 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-14T23:56:30.550 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sts_client_id": "", 2024-09-14T23:56:30.550 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sts_client_secret": "", 2024-09-14T23:56:30.550 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sts_entry": "sts", 2024-09-14T23:56:30.550 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sts_key": "sts", 2024-09-14T23:56:30.550 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-14T23:56:30.551 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sts_min_session_duration": "900", 2024-09-14T23:56:30.551 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sts_token_introspection_url": "", 2024-09-14T23:56:30.551 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_swift_account_in_url": "false", 2024-09-14T23:56:30.551 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_swift_auth_entry": "auth", 2024-09-14T23:56:30.551 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_swift_auth_url": "", 2024-09-14T23:56:30.551 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_swift_custom_header": "", 2024-09-14T23:56:30.551 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-14T23:56:30.551 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_swift_need_stats": "true", 2024-09-14T23:56:30.551 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_swift_tenant_name": "", 2024-09-14T23:56:30.551 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_swift_token_expiration": "86400", 2024-09-14T23:56:30.551 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_swift_url": "", 2024-09-14T23:56:30.551 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_swift_url_prefix": "swift", 2024-09-14T23:56:30.551 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-14T23:56:30.551 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-14T23:56:30.552 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-14T23:56:30.552 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sync_lease_period": "120", 2024-09-14T23:56:30.552 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-14T23:56:30.552 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-14T23:56:30.552 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-14T23:56:30.552 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-14T23:56:30.552 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-14T23:56:30.552 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-14T23:56:30.552 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-14T23:56:30.552 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-14T23:56:30.552 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-14T23:56:30.552 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_thread_pool_size": "512", 2024-09-14T23:56:30.552 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_topic_persistency_max_retries": "0", 2024-09-14T23:56:30.552 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-09-14T23:56:30.553 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-09-14T23:56:30.553 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_topic_require_publish_policy": "false", 2024-09-14T23:56:30.553 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_torrent_comment": "", 2024-09-14T23:56:30.553 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_torrent_createby": "", 2024-09-14T23:56:30.553 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_torrent_encoding": "", 2024-09-14T23:56:30.553 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_torrent_flag": "false", 2024-09-14T23:56:30.553 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_torrent_max_size": "5368709120", 2024-09-14T23:56:30.553 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_torrent_origin": "", 2024-09-14T23:56:30.553 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-14T23:56:30.553 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_torrent_tracker": "", 2024-09-14T23:56:30.553 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_trust_forwarded_https": "false", 2024-09-14T23:56:30.553 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-14T23:56:30.553 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-14T23:56:30.554 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_usage_max_shards": "32", 2024-09-14T23:56:30.554 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_usage_max_user_shards": "1", 2024-09-14T23:56:30.554 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_use_opa_authz": "false", 2024-09-14T23:56:30.554 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_user_counters_cache": "false", 2024-09-14T23:56:30.554 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_user_counters_cache_size": "10000", 2024-09-14T23:56:30.554 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-14T23:56:30.554 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-14T23:56:30.554 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_user_max_buckets": "1000", 2024-09-14T23:56:30.554 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_user_policies_max_num": "100", 2024-09-14T23:56:30.554 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-14T23:56:30.554 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-14T23:56:30.554 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-14T23:56:30.554 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-14T23:56:30.555 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_user_unique_email": "true", 2024-09-14T23:56:30.555 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_verify_ssl": "true", 2024-09-14T23:56:30.555 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-14T23:56:30.555 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_zone": "", 2024-09-14T23:56:30.555 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_zone_id": "", 2024-09-14T23:56:30.555 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-14T23:56:30.555 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_zonegroup": "", 2024-09-14T23:56:30.555 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_zonegroup_id": "", 2024-09-14T23:56:30.555 INFO:teuthology.orchestra.run.smithi042.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-14T23:56:30.555 INFO:teuthology.orchestra.run.smithi042.stdout: "rgwlc_auto_session_clear": "true", 2024-09-14T23:56:30.555 INFO:teuthology.orchestra.run.smithi042.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-14T23:56:30.555 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_block_size": "4096", 2024-09-14T23:56:30.555 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-14T23:56:30.556 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-14T23:56:30.556 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-14T23:56:30.556 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-14T23:56:30.556 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-14T23:56:30.556 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_cache_size": "536870912", 2024-09-14T23:56:30.556 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-14T23:56:30.556 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-14T23:56:30.556 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-14T23:56:30.556 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-14T23:56:30.556 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-14T23:56:30.556 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-14T23:56:30.556 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-14T23:56:30.556 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-14T23:56:30.556 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_index_type": "binary_search", 2024-09-14T23:56:30.557 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-14T23:56:30.557 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-14T23:56:30.557 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_partition_filters": "false", 2024-09-14T23:56:30.557 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_perf": "false", 2024-09-14T23:56:30.557 INFO:teuthology.orchestra.run.smithi042.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-14T23:56:30.557 INFO:teuthology.orchestra.run.smithi042.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-14T23:56:30.557 INFO:teuthology.orchestra.run.smithi042.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-14T23:56:30.557 INFO:teuthology.orchestra.run.smithi042.stdout: "run_dir": "/var/run/ceph", 2024-09-14T23:56:30.557 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_block_create": "true", 2024-09-14T23:56:30.557 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_cache_lru_size": "67108864", 2024-09-14T23:56:30.557 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-14T23:56:30.557 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_data_delta_based_overwrite": "0", 2024-09-14T23:56:30.557 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_default_max_object_size": "16777216", 2024-09-14T23:56:30.557 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-14T23:56:30.558 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_device_size": "53687091200", 2024-09-14T23:56:30.558 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_disable_end_to_end_data_protection": "true", 2024-09-14T23:56:30.558 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_full_integrity_check": "false", 2024-09-14T23:56:30.558 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_journal_batch_capacity": "16", 2024-09-14T23:56:30.558 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-14T23:56:30.558 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-14T23:56:30.558 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-14T23:56:30.558 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_main_device_type": "SSD", 2024-09-14T23:56:30.558 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-14T23:56:30.558 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_max_data_allocation_size": "32768", 2024-09-14T23:56:30.558 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-14T23:56:30.558 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-14T23:56:30.558 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-14T23:56:30.558 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-14T23:56:30.558 INFO:teuthology.orchestra.run.smithi042.stdout: "seastore_segment_size": "67108864", 2024-09-14T23:56:30.559 INFO:teuthology.orchestra.run.smithi042.stdout: "set_keepcaps": "false", 2024-09-14T23:56:30.559 INFO:teuthology.orchestra.run.smithi042.stdout: "setgroup": "", 2024-09-14T23:56:30.559 INFO:teuthology.orchestra.run.smithi042.stdout: "setuser": "", 2024-09-14T23:56:30.559 INFO:teuthology.orchestra.run.smithi042.stdout: "setuser_match_path": "", 2024-09-14T23:56:30.559 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-14T23:56:30.559 INFO:teuthology.orchestra.run.smithi042.stdout: "thp": "false", 2024-09-14T23:56:30.559 INFO:teuthology.orchestra.run.smithi042.stdout: "threadpool_default_timeout": "60", 2024-09-14T23:56:30.559 INFO:teuthology.orchestra.run.smithi042.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-14T23:56:30.559 INFO:teuthology.orchestra.run.smithi042.stdout: "throttler_perf_counter": "true" 2024-09-14T23:56:30.559 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:56:30.561 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-14T23:56:30.677 INFO:teuthology.orchestra.run.smithi042.stdout:mon.a: {} 2024-09-14T23:56:30.678 INFO:teuthology.orchestra.run.smithi042.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-14T23:56:30.687 INFO:teuthology.orchestra.run.smithi042.stdout:mon.b: {} 2024-09-14T23:56:30.687 INFO:teuthology.orchestra.run.smithi042.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-14T23:56:30.696 INFO:teuthology.orchestra.run.smithi042.stdout:mon.c: {} 2024-09-14T23:56:30.696 INFO:teuthology.orchestra.run.smithi042.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-14T23:56:30.707 DEBUG:teuthology.orchestra.run.smithi042:> type ceph-objectstore-tool 2024-09-14T23:56:30.721 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-14T23:56:30.722 DEBUG:teuthology.orchestra.run.smithi043:> type ceph-objectstore-tool 2024-09-14T23:56:30.741 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-14T23:56:30.741 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-09-14T23:56:30.751 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-09-14T23:56:30.751 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-14T23:56:30.751 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T23:56:30.751 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T23:56:30.752 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-14T23:56:30.752 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-09-14T23:56:30.752 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-14T23:56:30.752 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-14T23:56:30.753 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T23:56:30.753 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-14T23:56:30.753 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T23:56:30.753 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-14T23:56:30.753 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T23:56:30.754 DEBUG:teuthology.orchestra.run.smithi042:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 1 -w 1 -t 1 -h 1 -l 250000000 -S 0 -N 6000 pool_client.0 image_client.0 2024-09-14T23:56:30.763 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:56:30.761+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:56:30.801 INFO:teuthology.orchestra.run.smithi042.stdout:mapped writes DISABLED 2024-09-14T23:56:30.801 INFO:teuthology.orchestra.run.smithi042.stdout:mapped reads DISABLED 2024-09-14T23:56:30.801 INFO:teuthology.orchestra.run.smithi042.stdout:Seed set to 7085 2024-09-14T23:56:30.856 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:56:30.852+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:56:30.881 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:56:30.881 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:56:30.881 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:56:30.881 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:56:30.893 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T23:56:30.913 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:56:30.913 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:56:30.917 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-09-14T23:56:30.931 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:56:30.931 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:56:30.948 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:56:30.948 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:56:30.954 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:56:30.953+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:56:30.971 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:56:30.971 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:56:31.000 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:56:31.000 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:56:31.008 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:31.008 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:31.017 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T23:56:31.033 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:56:31.033 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:56:31.050 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:56:31.051 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:56:31.055 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:56:31.053+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:56:31.070 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:56:31.070 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:56:31.100 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-09-14T23:56:31.143 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:31.143 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:31.152 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T23:56:31.194 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:56:31.192+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:56:31.257 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:56:31.254+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:56:31.303 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-09-14T23:56:31.321 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:31.322 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:31.331 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T23:56:31.357 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:56:31.355+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:56:31.392 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is set 2024-09-14T23:56:31.451 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-14T23:56:31.486 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:31.486 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:31.487 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:56:31.485+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:56:31.495 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T23:56:31.559 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:56:31.556+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:56:31.627 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-09-14T23:56:31.646 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:31.646 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:31.655 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T23:56:31.697 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:56:31.695+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:56:31.760 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:56:31.759+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:56:31.767 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 reset_purged_snaps_last 2024-09-14T23:56:31.805 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:31.805 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:31.813 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T23:56:31.861 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:56:31.859+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:56:31.945 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 reset_purged_snaps_last 2024-09-14T23:56:31.962 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:56:31.960+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:56:31.964 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:31.964 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:31.974 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T23:56:32.063 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:56:32.062+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:56:32.122 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:32.123 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:32.131 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T23:56:32.172 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:56:32.171+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:56:32.264 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:56:32.263+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:56:32.279 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:32.279 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:32.288 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T23:56:32.366 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:56:32.364+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:56:32.442 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:32.442 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:32.451 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T23:56:32.467 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:56:32.465+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:56:32.559 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:32.559 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:32.567 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T23:56:32.568 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:56:32.566+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:56:32.669 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:56:32.667+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:56:32.689 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:32.689 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:32.700 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T23:56:32.769 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:56:32.768+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:56:32.809 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:32.809 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:32.817 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T23:56:32.870 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:56:32.869+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:56:32.924 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:32.924 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:32.933 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T23:56:32.976 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:56:32.974+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:56:33.041 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:33.041 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:33.050 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T23:56:33.072 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:56:33.071+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:56:33.082 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T23:56:33.173 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:56:33.170+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:56:33.196 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:33.197 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:33.206 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T23:56:33.221 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:56:33.222 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:56:33.239 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:56:33.239 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:56:33.255 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:56:33.255 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:56:33.272 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:56:33.272 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:56:33.273 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:56:33.272+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:56:33.290 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:56:33.290 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:56:33.306 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:56:33.307 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:56:33.324 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:56:33.325 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:56:33.341 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:56:33.341 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:56:33.357 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:56:33.357 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:56:33.357 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:56:33.357 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:56:33.357 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.358 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4284.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-14T23:56:33.358 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:56:33.358 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 10.831581062, 2024-09-14T23:56:33.358 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.004942086, 2024-09-14T23:56:33.358 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:33.358 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:33.358 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:33.358 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:33.358 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4284", 2024-09-14T23:56:33.358 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/4278347926", 2024-09-14T23:56:33.358 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1 2024-09-14T23:56:33.358 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.358 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:33.358 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.359 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:33.359 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:56:33.359 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:33.359 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.359 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.359 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:33.359 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:56:33.359 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:33.359 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.359 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.359 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:33.360 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524331+0000", 2024-09-14T23:56:33.360 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.383e-05 2024-09-14T23:56:33.360 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.360 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.360 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:33.360 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524339+0000", 2024-09-14T23:56:33.360 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8290000000000008e-06 2024-09-14T23:56:33.360 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.360 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.360 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:33.360 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524359+0000", 2024-09-14T23:56:33.361 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9998000000000001e-05 2024-09-14T23:56:33.361 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.361 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.361 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:33.361 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524373+0000", 2024-09-14T23:56:33.361 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4304000000000001e-05 2024-09-14T23:56:33.361 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.361 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.361 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:33.361 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524488+0000", 2024-09-14T23:56:33.361 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114663 2024-09-14T23:56:33.362 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.362 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.362 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:56:33.362 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524501+0000", 2024-09-14T23:56:33.362 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3349999999999999e-05 2024-09-14T23:56:33.362 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.362 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.362 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:33.362 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.526868+0000", 2024-09-14T23:56:33.362 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023670090000000002 2024-09-14T23:56:33.362 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.362 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.362 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:33.362 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.527041+0000", 2024-09-14T23:56:33.362 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000172597 2024-09-14T23:56:33.363 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.363 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.363 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-14T23:56:33.363 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.527384+0000", 2024-09-14T23:56:33.363 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00034309500000000002 2024-09-14T23:56:33.363 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.363 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.363 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:33.363 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529103+0000", 2024-09-14T23:56:33.363 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017185220000000001 2024-09-14T23:56:33.363 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.363 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.363 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:56:33.363 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529188+0000", 2024-09-14T23:56:33.363 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5591000000000001e-05 2024-09-14T23:56:33.364 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.364 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.364 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:33.364 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529227+0000", 2024-09-14T23:56:33.364 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8563000000000003e-05 2024-09-14T23:56:33.364 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.364 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.364 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:33.364 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529259+0000", 2024-09-14T23:56:33.364 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2734999999999997e-05 2024-09-14T23:56:33.364 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:33.364 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:33.364 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:33.364 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.365 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.365 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.4284.0:1 2.3 e19/18 ondisk, result = 0)", 2024-09-14T23:56:33.365 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:22.528997+0000", 2024-09-14T23:56:33.365 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 10.826901081000001, 2024-09-14T23:56:33.365 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000113931, 2024-09-14T23:56:33.365 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:33.365 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:33.365 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:33.365 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:33.365 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.365 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:33.365 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.528997+0000", 2024-09-14T23:56:33.365 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:33.365 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.366 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.366 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:33.366 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.528997+0000", 2024-09-14T23:56:33.366 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:33.366 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.366 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.366 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:33.366 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.528998+0000", 2024-09-14T23:56:33.366 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6799999999999999e-07 2024-09-14T23:56:33.366 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.366 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.366 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:33.366 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529005+0000", 2024-09-14T23:56:33.366 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9319999999999999e-06 2024-09-14T23:56:33.366 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.367 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.367 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:33.367 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529019+0000", 2024-09-14T23:56:33.367 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3549e-05 2024-09-14T23:56:33.367 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.367 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.367 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:33.367 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529026+0000", 2024-09-14T23:56:33.367 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7600000000000002e-06 2024-09-14T23:56:33.367 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.367 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.367 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:33.367 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529080+0000", 2024-09-14T23:56:33.367 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3566999999999997e-05 2024-09-14T23:56:33.367 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.368 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.368 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:33.368 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529096+0000", 2024-09-14T23:56:33.368 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5874000000000001e-05 2024-09-14T23:56:33.368 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.368 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.368 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:33.368 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529111+0000", 2024-09-14T23:56:33.368 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5381000000000001e-05 2024-09-14T23:56:33.368 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:33.368 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:33.368 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:33.368 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.368 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.368 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4284.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e19)", 2024-09-14T23:56:33.369 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:22.532053+0000", 2024-09-14T23:56:33.369 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 10.823845485, 2024-09-14T23:56:33.369 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00019815999999999999, 2024-09-14T23:56:33.369 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:33.369 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:33.369 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:56:33.369 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:33.369 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4284", 2024-09-14T23:56:33.369 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/4278347926", 2024-09-14T23:56:33.369 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 2 2024-09-14T23:56:33.369 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.370 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:33.370 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.370 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:33.370 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.532053+0000", 2024-09-14T23:56:33.370 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:33.370 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.370 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.370 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:33.370 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.532053+0000", 2024-09-14T23:56:33.370 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:33.371 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.371 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.371 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:33.371 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.532054+0000", 2024-09-14T23:56:33.371 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.421e-06 2024-09-14T23:56:33.371 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.371 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.371 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:33.371 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.532060+0000", 2024-09-14T23:56:33.371 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9279999999999998e-06 2024-09-14T23:56:33.371 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.371 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.371 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:33.371 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.532074+0000", 2024-09-14T23:56:33.371 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.397e-05 2024-09-14T23:56:33.372 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.372 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.372 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:33.372 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.532080+0000", 2024-09-14T23:56:33.372 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1639999999999998e-06 2024-09-14T23:56:33.372 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.372 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.372 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:33.372 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.532145+0000", 2024-09-14T23:56:33.373 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4888999999999995e-05 2024-09-14T23:56:33.373 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.373 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.373 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:33.373 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.532251+0000", 2024-09-14T23:56:33.373 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105788 2024-09-14T23:56:33.373 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:33.373 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:33.373 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:33.373 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.373 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.373 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4284.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e19)", 2024-09-14T23:56:33.373 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:22.532446+0000", 2024-09-14T23:56:33.373 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 10.823452802, 2024-09-14T23:56:33.374 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000161714, 2024-09-14T23:56:33.374 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:33.374 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:33.374 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:56:33.374 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:33.374 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4284", 2024-09-14T23:56:33.374 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/4278347926", 2024-09-14T23:56:33.374 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3 2024-09-14T23:56:33.374 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.374 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:33.374 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.374 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:33.374 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.532446+0000", 2024-09-14T23:56:33.375 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:33.375 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.375 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.375 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:33.375 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.532446+0000", 2024-09-14T23:56:33.375 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:33.375 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.375 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.375 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:33.375 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.532447+0000", 2024-09-14T23:56:33.375 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.540000000000001e-07 2024-09-14T23:56:33.375 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.375 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.376 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:33.376 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.532452+0000", 2024-09-14T23:56:33.376 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1830000000000002e-06 2024-09-14T23:56:33.376 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.376 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.376 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:33.376 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.532463+0000", 2024-09-14T23:56:33.376 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0859999999999999e-05 2024-09-14T23:56:33.376 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.376 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.376 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:33.376 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.532473+0000", 2024-09-14T23:56:33.376 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0113e-05 2024-09-14T23:56:33.376 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.377 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.377 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:33.377 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.532527+0000", 2024-09-14T23:56:33.377 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4345999999999998e-05 2024-09-14T23:56:33.377 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.377 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.377 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:33.377 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.532607+0000", 2024-09-14T23:56:33.377 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0258000000000003e-05 2024-09-14T23:56:33.377 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:33.377 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:33.377 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:33.377 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.377 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.378 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4284.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-14T23:56:33.378 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:23.530683+0000", 2024-09-14T23:56:33.378 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.8252153970000009, 2024-09-14T23:56:33.378 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0036133110000000001, 2024-09-14T23:56:33.378 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:33.378 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:33.378 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:33.378 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:33.378 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4284", 2024-09-14T23:56:33.378 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/4278347926", 2024-09-14T23:56:33.378 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5 2024-09-14T23:56:33.378 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.378 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:33.378 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.379 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:33.379 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530683+0000", 2024-09-14T23:56:33.379 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:33.379 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.379 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.379 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:33.379 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530683+0000", 2024-09-14T23:56:33.379 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:33.379 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.379 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.379 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:33.379 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530685+0000", 2024-09-14T23:56:33.379 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.075e-06 2024-09-14T23:56:33.380 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.380 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.380 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:33.380 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530695+0000", 2024-09-14T23:56:33.380 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0261999999999999e-05 2024-09-14T23:56:33.380 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.380 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.380 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:33.380 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530715+0000", 2024-09-14T23:56:33.381 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9930000000000001e-05 2024-09-14T23:56:33.381 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.381 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.381 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:33.381 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530724+0000", 2024-09-14T23:56:33.381 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.901e-06 2024-09-14T23:56:33.381 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.381 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.381 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:33.381 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530800+0000", 2024-09-14T23:56:33.381 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5873999999999999e-05 2024-09-14T23:56:33.381 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.382 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.382 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-14T23:56:33.382 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530807+0000", 2024-09-14T23:56:33.382 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2389999999999996e-06 2024-09-14T23:56:33.382 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.382 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.382 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:33.382 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.532494+0000", 2024-09-14T23:56:33.382 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016871830000000001 2024-09-14T23:56:33.382 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.382 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.382 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:33.383 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.532599+0000", 2024-09-14T23:56:33.383 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010472 2024-09-14T23:56:33.383 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.383 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.383 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1", 2024-09-14T23:56:33.383 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.532715+0000", 2024-09-14T23:56:33.383 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011568199999999999 2024-09-14T23:56:33.383 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.383 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.383 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:56:33.383 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.533460+0000", 2024-09-14T23:56:33.383 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00074494999999999997 2024-09-14T23:56:33.383 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.384 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.384 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:33.384 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534230+0000", 2024-09-14T23:56:33.384 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00077036599999999998 2024-09-14T23:56:33.384 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.384 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.384 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:33.384 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534266+0000", 2024-09-14T23:56:33.384 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5725000000000001e-05 2024-09-14T23:56:33.384 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.384 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.384 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:33.384 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534296+0000", 2024-09-14T23:56:33.384 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0403999999999999e-05 2024-09-14T23:56:33.385 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:33.385 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:33.385 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:33.385 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.385 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.385 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.4284.0:5 2.2 e19/18 ondisk, result = 0)", 2024-09-14T23:56:33.385 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:23.534124+0000", 2024-09-14T23:56:33.385 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.8217743280000001, 2024-09-14T23:56:33.385 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017988999999999999, 2024-09-14T23:56:33.385 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:33.385 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:33.385 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:33.385 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:33.386 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.386 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:33.386 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534124+0000", 2024-09-14T23:56:33.386 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:33.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.386 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.386 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:33.386 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534124+0000", 2024-09-14T23:56:33.386 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:33.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.386 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.386 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:33.386 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534125+0000", 2024-09-14T23:56:33.387 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1710000000000001e-06 2024-09-14T23:56:33.387 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.387 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.387 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:33.387 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534131+0000", 2024-09-14T23:56:33.387 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.066e-06 2024-09-14T23:56:33.387 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.387 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.387 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:33.387 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534145+0000", 2024-09-14T23:56:33.387 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.348e-05 2024-09-14T23:56:33.387 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.387 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.387 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:33.388 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534151+0000", 2024-09-14T23:56:33.388 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3910000000000003e-06 2024-09-14T23:56:33.388 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.388 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.388 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:33.388 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534219+0000", 2024-09-14T23:56:33.388 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7679e-05 2024-09-14T23:56:33.388 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.388 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.388 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:33.388 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534227+0000", 2024-09-14T23:56:33.388 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7740000000000004e-06 2024-09-14T23:56:33.388 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.389 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.389 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:33.389 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534304+0000", 2024-09-14T23:56:33.389 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7329000000000005e-05 2024-09-14T23:56:33.389 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:33.389 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:33.389 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:33.389 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.389 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.389 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4284.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-14T23:56:33.389 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:24.535160+0000", 2024-09-14T23:56:33.389 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.8207387560000008, 2024-09-14T23:56:33.389 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033473650000000001, 2024-09-14T23:56:33.390 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:33.390 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:33.390 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:33.390 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:33.390 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4284", 2024-09-14T23:56:33.390 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/4278347926", 2024-09-14T23:56:33.390 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 7 2024-09-14T23:56:33.390 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.390 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:33.390 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.390 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:33.390 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535160+0000", 2024-09-14T23:56:33.390 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:33.390 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.391 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.391 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:33.391 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535160+0000", 2024-09-14T23:56:33.391 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:33.391 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.391 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.391 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:33.391 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535162+0000", 2024-09-14T23:56:33.391 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3369999999999998e-06 2024-09-14T23:56:33.391 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.392 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.392 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:33.392 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535171+0000", 2024-09-14T23:56:33.392 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7700000000000007e-06 2024-09-14T23:56:33.392 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.392 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.392 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:33.392 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535189+0000", 2024-09-14T23:56:33.392 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7938e-05 2024-09-14T23:56:33.392 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.392 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.392 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:33.392 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535197+0000", 2024-09-14T23:56:33.392 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5639999999999997e-06 2024-09-14T23:56:33.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.393 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.393 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:33.393 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535266+0000", 2024-09-14T23:56:33.393 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8926999999999999e-05 2024-09-14T23:56:33.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.393 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.393 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-14T23:56:33.393 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535271+0000", 2024-09-14T23:56:33.393 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0329999999999999e-06 2024-09-14T23:56:33.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.393 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.393 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:33.394 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.537064+0000", 2024-09-14T23:56:33.394 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001792929 2024-09-14T23:56:33.394 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.394 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.394 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:33.394 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.537131+0000", 2024-09-14T23:56:33.394 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6605000000000004e-05 2024-09-14T23:56:33.394 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.394 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.394 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1", 2024-09-14T23:56:33.394 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.537271+0000", 2024-09-14T23:56:33.394 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013992400000000001 2024-09-14T23:56:33.394 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.394 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.395 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:56:33.395 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.538093+0000", 2024-09-14T23:56:33.395 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00082210200000000001 2024-09-14T23:56:33.395 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.395 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.395 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:33.395 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.538439+0000", 2024-09-14T23:56:33.395 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00034595999999999997 2024-09-14T23:56:33.395 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.395 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.395 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:33.396 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.538476+0000", 2024-09-14T23:56:33.396 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6770000000000002e-05 2024-09-14T23:56:33.396 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.396 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.396 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:33.396 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.538507+0000", 2024-09-14T23:56:33.396 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1506e-05 2024-09-14T23:56:33.396 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:33.396 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:33.396 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:33.396 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.397 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.397 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.4284.0:7 2.2 e21/18 ondisk, result = 0)", 2024-09-14T23:56:33.397 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:24.538327+0000", 2024-09-14T23:56:33.397 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.8175716350000002, 2024-09-14T23:56:33.397 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018775700000000001, 2024-09-14T23:56:33.397 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:33.398 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:33.398 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:33.398 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:33.398 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.398 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:33.398 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.538327+0000", 2024-09-14T23:56:33.398 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:33.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.399 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.399 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:33.399 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.538327+0000", 2024-09-14T23:56:33.399 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:33.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.399 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.400 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:33.400 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.538328+0000", 2024-09-14T23:56:33.400 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.392e-06 2024-09-14T23:56:33.400 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.400 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.400 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:33.400 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.538333+0000", 2024-09-14T23:56:33.401 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5029999999999999e-06 2024-09-14T23:56:33.401 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.401 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.401 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:33.401 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.538345+0000", 2024-09-14T23:56:33.401 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2109e-05 2024-09-14T23:56:33.402 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.402 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.402 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:33.402 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.538352+0000", 2024-09-14T23:56:33.402 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1770000000000001e-06 2024-09-14T23:56:33.402 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.403 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.403 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:33.403 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.538423+0000", 2024-09-14T23:56:33.403 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1477999999999996e-05 2024-09-14T23:56:33.403 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.403 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.404 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:33.404 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.538433+0000", 2024-09-14T23:56:33.404 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0950000000000002e-06 2024-09-14T23:56:33.404 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:33.404 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:33.404 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:33.405 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.538515+0000", 2024-09-14T23:56:33.405 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2002999999999994e-05 2024-09-14T23:56:33.405 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:33.405 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:33.405 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:33.406 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:33.406 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:33.406 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:56:33.408 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T23:56:33.408 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:56:33.406+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:56:33.413 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T23:56:33.508 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:56:33.507+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:56:33.556 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:56:33.556 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:56:33.556 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:56:33.556 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:56:33.565 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T23:56:33.609 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:56:33.607+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:56:33.672 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:56:33.672 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:56:33.672 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:56:33.673 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:56:33.673 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:56:33.681 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T23:56:33.710 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:56:33.708+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:56:33.788 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:56:33.788 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:56:33.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:56:33.788 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [] 2024-09-14T23:56:33.788 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:56:33.797 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T23:56:33.811 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:56:33.810+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:56:33.904 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:56:33.904 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:56:33.904 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:56:33.905 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:56:33.912 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:56:33.910+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:56:33.914 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T23:56:34.013 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:56:34.011+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:56:34.060 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:56:34.061 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:56:34.061 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:56:34.061 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:56:34.061 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:56:34.069 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T23:56:34.154 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:56:34.151+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:56:34.215 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:56:34.213+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:56:34.219 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:56:34.219 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:56:34.219 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:56:34.219 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:56:34.219 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.219 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4209.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-14T23:56:34.220 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:15.357728+0000", 2024-09-14T23:56:34.220 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.858434246000002, 2024-09-14T23:56:34.220 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0043495850000000004, 2024-09-14T23:56:34.220 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:34.220 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:34.220 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:34.220 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:34.220 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4209", 2024-09-14T23:56:34.220 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/1617902543", 2024-09-14T23:56:34.220 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 37 2024-09-14T23:56:34.220 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.220 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:34.221 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.221 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:34.221 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.357728+0000", 2024-09-14T23:56:34.221 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:34.221 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.221 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.221 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:34.222 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.357728+0000", 2024-09-14T23:56:34.222 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:34.222 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.222 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.223 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:34.223 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.357728+0000", 2024-09-14T23:56:34.223 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1e-07 2024-09-14T23:56:34.223 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.223 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.223 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:34.223 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.357730+0000", 2024-09-14T23:56:34.223 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1720000000000001e-06 2024-09-14T23:56:34.223 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.223 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.223 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:34.224 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.357733+0000", 2024-09-14T23:56:34.224 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5300000000000001e-06 2024-09-14T23:56:34.224 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.224 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.224 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:34.224 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.357736+0000", 2024-09-14T23:56:34.224 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5900000000000002e-06 2024-09-14T23:56:34.224 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.224 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.224 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:34.224 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.361176+0000", 2024-09-14T23:56:34.224 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0034402899999999999 2024-09-14T23:56:34.225 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.225 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.225 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:34.225 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.361228+0000", 2024-09-14T23:56:34.225 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1561999999999999e-05 2024-09-14T23:56:34.225 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.225 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.225 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:56:34.225 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.361326+0000", 2024-09-14T23:56:34.225 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8369999999999995e-05 2024-09-14T23:56:34.225 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.225 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.226 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:56:34.226 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.361744+0000", 2024-09-14T23:56:34.226 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000417279 2024-09-14T23:56:34.226 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.226 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.226 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:34.226 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.362030+0000", 2024-09-14T23:56:34.226 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000285965 2024-09-14T23:56:34.226 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.226 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.226 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:34.226 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.362058+0000", 2024-09-14T23:56:34.226 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8479999999999998e-05 2024-09-14T23:56:34.227 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.227 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.227 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:34.227 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.362077+0000", 2024-09-14T23:56:34.227 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9137e-05 2024-09-14T23:56:34.227 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:34.227 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:34.227 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:34.227 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.227 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.227 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4209.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-14T23:56:34.227 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:15.397958+0000", 2024-09-14T23:56:34.227 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.818204085000001, 2024-09-14T23:56:34.227 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0043849279999999997, 2024-09-14T23:56:34.227 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:34.228 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:34.228 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:34.228 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:34.228 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4209", 2024-09-14T23:56:34.228 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/1617902543", 2024-09-14T23:56:34.228 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 87 2024-09-14T23:56:34.228 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.228 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:34.228 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.228 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:34.228 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.397958+0000", 2024-09-14T23:56:34.229 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:34.229 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.229 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.229 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:34.229 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.397958+0000", 2024-09-14T23:56:34.229 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:34.229 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.229 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.229 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:34.229 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.397958+0000", 2024-09-14T23:56:34.229 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7800000000000002e-07 2024-09-14T23:56:34.229 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.230 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.230 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:34.230 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.398008+0000", 2024-09-14T23:56:34.230 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0309000000000002e-05 2024-09-14T23:56:34.230 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.230 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.230 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:34.230 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.398014+0000", 2024-09-14T23:56:34.230 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.823e-06 2024-09-14T23:56:34.230 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.230 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.230 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:34.230 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.398018+0000", 2024-09-14T23:56:34.230 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.206e-06 2024-09-14T23:56:34.231 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.231 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.231 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:34.231 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.400798+0000", 2024-09-14T23:56:34.231 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0027797329999999999 2024-09-14T23:56:34.231 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.231 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.231 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:34.231 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.400877+0000", 2024-09-14T23:56:34.231 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9295999999999994e-05 2024-09-14T23:56:34.231 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.231 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.231 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:56:34.231 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.401011+0000", 2024-09-14T23:56:34.232 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013304699999999999 2024-09-14T23:56:34.232 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.232 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.232 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:56:34.232 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.401940+0000", 2024-09-14T23:56:34.232 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00092952400000000004 2024-09-14T23:56:34.232 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.232 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.232 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:34.232 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.402244+0000", 2024-09-14T23:56:34.232 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00030433700000000001 2024-09-14T23:56:34.232 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.232 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.232 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:34.233 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.402300+0000", 2024-09-14T23:56:34.233 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6051999999999998e-05 2024-09-14T23:56:34.233 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.233 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.233 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:34.233 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.402343+0000", 2024-09-14T23:56:34.233 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2222999999999999e-05 2024-09-14T23:56:34.233 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:34.233 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:34.233 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:34.233 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.233 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.233 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4209.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-14T23:56:34.233 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:15.398027+0000", 2024-09-14T23:56:34.234 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.818134947000001, 2024-09-14T23:56:34.234 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0045217030000000002, 2024-09-14T23:56:34.234 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:34.234 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:34.234 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:34.234 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:34.234 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4209", 2024-09-14T23:56:34.234 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/1617902543", 2024-09-14T23:56:34.234 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 88 2024-09-14T23:56:34.234 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.235 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:34.235 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.235 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:34.235 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.398027+0000", 2024-09-14T23:56:34.235 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:34.235 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.235 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.235 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:34.235 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.398027+0000", 2024-09-14T23:56:34.235 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:34.235 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.235 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.236 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:34.236 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.398027+0000", 2024-09-14T23:56:34.236 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9000000000000002e-07 2024-09-14T23:56:34.236 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.236 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.236 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:34.236 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.398029+0000", 2024-09-14T23:56:34.236 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1600000000000001e-06 2024-09-14T23:56:34.236 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.236 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.236 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:34.236 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.398049+0000", 2024-09-14T23:56:34.236 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9745000000000002e-05 2024-09-14T23:56:34.236 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.237 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.237 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:34.237 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.398061+0000", 2024-09-14T23:56:34.237 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1832e-05 2024-09-14T23:56:34.237 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.237 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.237 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:34.237 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.401556+0000", 2024-09-14T23:56:34.237 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0034946280000000001 2024-09-14T23:56:34.237 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.237 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.237 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:34.237 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.401600+0000", 2024-09-14T23:56:34.237 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4512999999999999e-05 2024-09-14T23:56:34.237 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.238 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.238 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:56:34.238 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.401690+0000", 2024-09-14T23:56:34.238 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9420999999999996e-05 2024-09-14T23:56:34.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.238 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.238 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:56:34.238 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.402355+0000", 2024-09-14T23:56:34.238 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00066586200000000003 2024-09-14T23:56:34.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.238 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.238 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:34.238 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.402499+0000", 2024-09-14T23:56:34.238 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000143098 2024-09-14T23:56:34.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.239 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.239 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:34.239 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.402527+0000", 2024-09-14T23:56:34.239 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8433e-05 2024-09-14T23:56:34.239 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.239 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.239 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:34.239 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.402549+0000", 2024-09-14T23:56:34.239 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1620999999999998e-05 2024-09-14T23:56:34.239 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:34.239 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:34.240 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:34.240 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.240 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.240 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4209.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-14T23:56:34.240 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:15.572684+0000", 2024-09-14T23:56:34.240 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.643477927999999, 2024-09-14T23:56:34.240 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.004979804, 2024-09-14T23:56:34.240 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:34.240 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:34.240 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:34.240 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:34.240 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4209", 2024-09-14T23:56:34.240 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/1617902543", 2024-09-14T23:56:34.241 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 109 2024-09-14T23:56:34.241 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.241 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:34.241 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.241 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:34.241 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.572684+0000", 2024-09-14T23:56:34.241 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:34.241 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.241 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.241 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:34.241 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.572684+0000", 2024-09-14T23:56:34.241 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:34.242 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.242 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.242 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:34.242 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.572684+0000", 2024-09-14T23:56:34.242 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8599999999999998e-07 2024-09-14T23:56:34.242 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.242 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.242 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:34.242 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.572765+0000", 2024-09-14T23:56:34.242 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0195e-05 2024-09-14T23:56:34.242 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.243 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.243 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:34.243 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.572778+0000", 2024-09-14T23:56:34.243 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3209e-05 2024-09-14T23:56:34.243 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.243 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.243 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:34.243 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.572793+0000", 2024-09-14T23:56:34.243 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4884e-05 2024-09-14T23:56:34.243 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.243 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.243 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:34.243 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.575300+0000", 2024-09-14T23:56:34.243 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0025069390000000001 2024-09-14T23:56:34.243 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.244 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.244 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:34.244 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.575346+0000", 2024-09-14T23:56:34.244 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6718000000000002e-05 2024-09-14T23:56:34.244 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.244 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.244 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:56:34.244 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.575423+0000", 2024-09-14T23:56:34.244 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7071999999999997e-05 2024-09-14T23:56:34.244 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.244 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.244 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:34.244 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.577057+0000", 2024-09-14T23:56:34.245 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016335259999999999 2024-09-14T23:56:34.245 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.245 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.245 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:56:34.245 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.577584+0000", 2024-09-14T23:56:34.245 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00052741500000000005 2024-09-14T23:56:34.245 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.245 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.245 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:34.245 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.577639+0000", 2024-09-14T23:56:34.245 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4623000000000002e-05 2024-09-14T23:56:34.245 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.245 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.245 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:34.246 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.577664+0000", 2024-09-14T23:56:34.246 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4637000000000001e-05 2024-09-14T23:56:34.246 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:34.246 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:34.246 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:34.246 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.246 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.246 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4209.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-14T23:56:34.246 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:15.572821+0000", 2024-09-14T23:56:34.246 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.643341114999998, 2024-09-14T23:56:34.247 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0053598150000000004, 2024-09-14T23:56:34.247 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:34.247 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:34.247 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:34.247 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:34.247 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4209", 2024-09-14T23:56:34.247 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/1617902543", 2024-09-14T23:56:34.247 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 110 2024-09-14T23:56:34.247 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.247 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:34.247 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.248 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:34.248 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.572821+0000", 2024-09-14T23:56:34.248 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:34.248 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.248 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.248 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:34.248 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.572821+0000", 2024-09-14T23:56:34.248 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:34.248 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.248 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.248 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:34.248 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.572821+0000", 2024-09-14T23:56:34.248 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5000000000000002e-07 2024-09-14T23:56:34.248 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.248 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.249 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:34.249 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.572826+0000", 2024-09-14T23:56:34.249 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9570000000000001e-06 2024-09-14T23:56:34.249 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.249 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.249 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:34.249 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.572837+0000", 2024-09-14T23:56:34.249 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0766e-05 2024-09-14T23:56:34.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.250 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.250 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:34.250 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.572844+0000", 2024-09-14T23:56:34.250 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4710000000000004e-06 2024-09-14T23:56:34.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.250 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.250 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:34.250 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.576100+0000", 2024-09-14T23:56:34.250 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003256292 2024-09-14T23:56:34.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.250 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.250 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:34.251 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.576190+0000", 2024-09-14T23:56:34.251 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0000000000000006e-05 2024-09-14T23:56:34.251 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.251 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.251 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:56:34.251 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.576330+0000", 2024-09-14T23:56:34.251 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014050400000000001 2024-09-14T23:56:34.251 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.251 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.251 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:56:34.251 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.577683+0000", 2024-09-14T23:56:34.251 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0013524030000000001 2024-09-14T23:56:34.251 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.252 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.252 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:34.252 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.578126+0000", 2024-09-14T23:56:34.252 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00044277599999999998 2024-09-14T23:56:34.252 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.252 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.252 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:34.252 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.578159+0000", 2024-09-14T23:56:34.252 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3254999999999999e-05 2024-09-14T23:56:34.252 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.252 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.252 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:34.252 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.578181+0000", 2024-09-14T23:56:34.253 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1741000000000001e-05 2024-09-14T23:56:34.253 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:34.253 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:34.253 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:34.253 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.253 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.253 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4209.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-14T23:56:34.253 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:15.572859+0000", 2024-09-14T23:56:34.253 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.643302862999999, 2024-09-14T23:56:34.253 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0054419050000000004, 2024-09-14T23:56:34.253 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:34.253 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:34.253 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:34.253 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:34.254 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4209", 2024-09-14T23:56:34.254 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/1617902543", 2024-09-14T23:56:34.254 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 111 2024-09-14T23:56:34.254 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.254 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:34.254 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.254 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:34.254 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.572859+0000", 2024-09-14T23:56:34.254 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:34.254 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.254 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.254 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:34.254 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.572859+0000", 2024-09-14T23:56:34.254 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:34.254 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.255 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:34.255 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.572860+0000", 2024-09-14T23:56:34.255 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9200000000000001e-07 2024-09-14T23:56:34.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.255 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:34.255 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.572864+0000", 2024-09-14T23:56:34.255 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.408e-06 2024-09-14T23:56:34.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.255 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:34.255 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.572876+0000", 2024-09-14T23:56:34.255 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2378000000000001e-05 2024-09-14T23:56:34.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.256 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:34.256 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.572883+0000", 2024-09-14T23:56:34.256 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4400000000000002e-06 2024-09-14T23:56:34.256 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.256 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.256 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:34.256 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.576636+0000", 2024-09-14T23:56:34.256 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003752891 2024-09-14T23:56:34.256 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.256 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.256 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:34.256 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.576692+0000", 2024-09-14T23:56:34.256 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6042999999999999e-05 2024-09-14T23:56:34.256 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.257 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.257 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:56:34.257 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.576765+0000", 2024-09-14T23:56:34.257 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3731000000000003e-05 2024-09-14T23:56:34.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.257 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.257 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:56:34.257 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.578188+0000", 2024-09-14T23:56:34.257 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001422399 2024-09-14T23:56:34.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.257 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.258 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:34.258 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.578219+0000", 2024-09-14T23:56:34.258 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0694000000000001e-05 2024-09-14T23:56:34.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.258 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.258 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:34.258 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.578262+0000", 2024-09-14T23:56:34.258 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3511999999999999e-05 2024-09-14T23:56:34.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.259 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.259 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:34.259 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.578301+0000", 2024-09-14T23:56:34.259 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8816999999999998e-05 2024-09-14T23:56:34.259 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:34.259 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:34.259 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:34.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.259 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.259 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4209.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-14T23:56:34.259 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:15.572919+0000", 2024-09-14T23:56:34.259 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.643242759, 2024-09-14T23:56:34.259 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0055260450000000003, 2024-09-14T23:56:34.260 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:34.260 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:34.260 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:34.260 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:34.260 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4209", 2024-09-14T23:56:34.260 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/1617902543", 2024-09-14T23:56:34.260 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 112 2024-09-14T23:56:34.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.260 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:34.261 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.261 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:34.261 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.572919+0000", 2024-09-14T23:56:34.261 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:34.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.261 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.261 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:34.261 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.572919+0000", 2024-09-14T23:56:34.261 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:34.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.261 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.261 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:34.261 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.572920+0000", 2024-09-14T23:56:34.262 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8999999999999995e-07 2024-09-14T23:56:34.262 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.262 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.262 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:34.262 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.572981+0000", 2024-09-14T23:56:34.262 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1577999999999999e-05 2024-09-14T23:56:34.262 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.262 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.262 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:34.262 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.572989+0000", 2024-09-14T23:56:34.262 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5970000000000003e-06 2024-09-14T23:56:34.262 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.262 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.262 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:34.263 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.572993+0000", 2024-09-14T23:56:34.263 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3529999999999997e-06 2024-09-14T23:56:34.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.263 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:34.263 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.577086+0000", 2024-09-14T23:56:34.263 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0040926670000000004 2024-09-14T23:56:34.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.263 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:34.263 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.577142+0000", 2024-09-14T23:56:34.263 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5875999999999998e-05 2024-09-14T23:56:34.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.264 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:56:34.264 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.577219+0000", 2024-09-14T23:56:34.264 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6652999999999999e-05 2024-09-14T23:56:34.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.264 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.264 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:34.264 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.578356+0000", 2024-09-14T23:56:34.264 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0011372789999999999 2024-09-14T23:56:34.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.264 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.264 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:56:34.264 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.578372+0000", 2024-09-14T23:56:34.264 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6529e-05 2024-09-14T23:56:34.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.265 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:34.265 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.578422+0000", 2024-09-14T23:56:34.265 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9317999999999998e-05 2024-09-14T23:56:34.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.265 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:34.265 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.578445+0000", 2024-09-14T23:56:34.265 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3405e-05 2024-09-14T23:56:34.265 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:34.265 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:34.265 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:34.266 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.266 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.266 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4209.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-14T23:56:34.266 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:15.573005+0000", 2024-09-14T23:56:34.266 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.643156813000001, 2024-09-14T23:56:34.266 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0066227630000000003, 2024-09-14T23:56:34.266 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:34.266 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:34.266 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:34.266 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:34.266 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4209", 2024-09-14T23:56:34.266 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/1617902543", 2024-09-14T23:56:34.267 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 113 2024-09-14T23:56:34.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.267 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:34.267 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.267 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:34.267 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573005+0000", 2024-09-14T23:56:34.267 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:34.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.267 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.267 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:34.267 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573005+0000", 2024-09-14T23:56:34.267 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:34.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.268 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.268 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:34.268 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573005+0000", 2024-09-14T23:56:34.268 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0900000000000002e-07 2024-09-14T23:56:34.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.268 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.268 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:34.268 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573009+0000", 2024-09-14T23:56:34.268 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1080000000000001e-06 2024-09-14T23:56:34.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.268 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.268 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:34.269 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573023+0000", 2024-09-14T23:56:34.269 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4103e-05 2024-09-14T23:56:34.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.269 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.269 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:34.269 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573027+0000", 2024-09-14T23:56:34.269 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.929e-06 2024-09-14T23:56:34.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.269 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.269 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:34.269 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.577716+0000", 2024-09-14T23:56:34.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0046899380000000003 2024-09-14T23:56:34.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.270 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.270 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:34.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.577763+0000", 2024-09-14T23:56:34.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6261999999999999e-05 2024-09-14T23:56:34.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.270 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.270 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:56:34.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.577841+0000", 2024-09-14T23:56:34.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7906999999999998e-05 2024-09-14T23:56:34.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.271 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.271 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:56:34.271 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.579521+0000", 2024-09-14T23:56:34.271 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016806480000000001 2024-09-14T23:56:34.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.271 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.271 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:34.271 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.579571+0000", 2024-09-14T23:56:34.271 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9700000000000002e-05 2024-09-14T23:56:34.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.271 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.271 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:34.272 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.579601+0000", 2024-09-14T23:56:34.272 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0340000000000001e-05 2024-09-14T23:56:34.272 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.272 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.272 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:34.272 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.579628+0000", 2024-09-14T23:56:34.272 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6418999999999999e-05 2024-09-14T23:56:34.272 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:34.272 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:34.272 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:34.272 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.272 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.272 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4209.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-14T23:56:34.272 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:15.573032+0000", 2024-09-14T23:56:34.273 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.643129341000002, 2024-09-14T23:56:34.273 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0076935190000000002, 2024-09-14T23:56:34.273 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:34.273 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:34.273 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:34.273 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:34.273 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4209", 2024-09-14T23:56:34.273 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/1617902543", 2024-09-14T23:56:34.273 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 114 2024-09-14T23:56:34.273 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.273 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:34.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.274 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:34.274 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573032+0000", 2024-09-14T23:56:34.274 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:34.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.274 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:34.274 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573032+0000", 2024-09-14T23:56:34.274 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:34.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.274 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:34.275 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573033+0000", 2024-09-14T23:56:34.275 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8000000000000002e-07 2024-09-14T23:56:34.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.275 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.275 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:34.275 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573035+0000", 2024-09-14T23:56:34.275 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.627e-06 2024-09-14T23:56:34.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.275 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.275 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:34.275 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573041+0000", 2024-09-14T23:56:34.275 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.237e-06 2024-09-14T23:56:34.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.276 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.276 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:34.276 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573049+0000", 2024-09-14T23:56:34.276 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3930000000000008e-06 2024-09-14T23:56:34.276 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.276 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.276 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:34.276 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.578474+0000", 2024-09-14T23:56:34.276 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0054253160000000003 2024-09-14T23:56:34.277 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.277 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.277 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:34.277 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.578532+0000", 2024-09-14T23:56:34.277 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8146999999999997e-05 2024-09-14T23:56:34.277 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.277 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.277 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:56:34.277 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.578646+0000", 2024-09-14T23:56:34.277 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000113428 2024-09-14T23:56:34.277 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.277 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.278 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:56:34.278 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.580086+0000", 2024-09-14T23:56:34.278 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001439957 2024-09-14T23:56:34.278 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.278 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.278 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:34.278 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.580661+0000", 2024-09-14T23:56:34.278 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000575393 2024-09-14T23:56:34.278 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.278 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.278 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:34.278 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.580703+0000", 2024-09-14T23:56:34.278 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1359999999999997e-05 2024-09-14T23:56:34.279 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.279 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.279 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:34.279 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.580726+0000", 2024-09-14T23:56:34.279 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3380999999999999e-05 2024-09-14T23:56:34.279 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:34.279 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:34.279 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:34.279 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.279 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.279 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4209.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-14T23:56:34.279 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:15.573151+0000", 2024-09-14T23:56:34.279 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.643011237, 2024-09-14T23:56:34.279 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0081104469999999998, 2024-09-14T23:56:34.280 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:34.280 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:34.280 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:34.280 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:34.280 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4209", 2024-09-14T23:56:34.280 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/1617902543", 2024-09-14T23:56:34.280 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 115 2024-09-14T23:56:34.280 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.280 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:34.280 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.280 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:34.281 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573151+0000", 2024-09-14T23:56:34.281 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:34.281 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.281 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.281 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:34.281 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573151+0000", 2024-09-14T23:56:34.281 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:34.281 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.281 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.281 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:34.281 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573151+0000", 2024-09-14T23:56:34.281 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9599999999999999e-07 2024-09-14T23:56:34.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.282 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.282 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:34.282 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573245+0000", 2024-09-14T23:56:34.282 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3423000000000003e-05 2024-09-14T23:56:34.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.282 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.282 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:34.282 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573251+0000", 2024-09-14T23:56:34.282 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5819999999999998e-06 2024-09-14T23:56:34.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.282 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.283 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:34.283 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573255+0000", 2024-09-14T23:56:34.283 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6899999999999998e-06 2024-09-14T23:56:34.283 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.283 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.283 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:34.283 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.578943+0000", 2024-09-14T23:56:34.283 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0056883999999999997 2024-09-14T23:56:34.283 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.283 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.283 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:34.283 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.578991+0000", 2024-09-14T23:56:34.284 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8049e-05 2024-09-14T23:56:34.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.284 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.284 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:56:34.284 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.579079+0000", 2024-09-14T23:56:34.284 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7886000000000005e-05 2024-09-14T23:56:34.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.284 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.284 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:56:34.285 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.580095+0000", 2024-09-14T23:56:34.285 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001016213 2024-09-14T23:56:34.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.285 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.285 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:34.285 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.581209+0000", 2024-09-14T23:56:34.285 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001113677 2024-09-14T23:56:34.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.285 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.285 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:34.285 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.581238+0000", 2024-09-14T23:56:34.285 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9189000000000001e-05 2024-09-14T23:56:34.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.285 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.286 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:34.286 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.581261+0000", 2024-09-14T23:56:34.286 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2742e-05 2024-09-14T23:56:34.286 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:34.286 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:34.286 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:34.286 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.286 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.286 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4209.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-14T23:56:34.286 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:15.573263+0000", 2024-09-14T23:56:34.286 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.642898984999999, 2024-09-14T23:56:34.286 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0080966040000000003, 2024-09-14T23:56:34.286 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:34.286 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:34.287 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:34.287 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:34.287 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4209", 2024-09-14T23:56:34.287 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/1617902543", 2024-09-14T23:56:34.287 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 116 2024-09-14T23:56:34.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.287 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:34.287 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.287 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:34.287 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573263+0000", 2024-09-14T23:56:34.287 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:34.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.287 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.287 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:34.288 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573263+0000", 2024-09-14T23:56:34.288 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:34.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.288 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.288 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:34.288 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573263+0000", 2024-09-14T23:56:34.288 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3200000000000001e-07 2024-09-14T23:56:34.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.288 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.288 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:34.288 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573272+0000", 2024-09-14T23:56:34.288 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4889999999999995e-06 2024-09-14T23:56:34.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.288 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.288 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:34.289 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573279+0000", 2024-09-14T23:56:34.289 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2899999999999997e-06 2024-09-14T23:56:34.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.289 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.289 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:34.289 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573288+0000", 2024-09-14T23:56:34.289 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1810000000000007e-06 2024-09-14T23:56:34.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.289 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.289 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:34.289 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.579675+0000", 2024-09-14T23:56:34.289 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0063870799999999998 2024-09-14T23:56:34.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.289 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.290 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:34.290 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.579743+0000", 2024-09-14T23:56:34.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8256000000000005e-05 2024-09-14T23:56:34.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.290 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.290 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:56:34.290 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.579847+0000", 2024-09-14T23:56:34.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000103805 2024-09-14T23:56:34.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.290 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.290 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:56:34.290 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.580743+0000", 2024-09-14T23:56:34.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00089532000000000001 2024-09-14T23:56:34.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.290 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.290 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:34.291 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.581301+0000", 2024-09-14T23:56:34.291 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00055803999999999999 2024-09-14T23:56:34.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.291 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.291 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:34.291 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.581335+0000", 2024-09-14T23:56:34.291 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4022000000000001e-05 2024-09-14T23:56:34.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.291 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.291 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:34.291 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.581359+0000", 2024-09-14T23:56:34.291 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4788999999999999e-05 2024-09-14T23:56:34.292 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:34.292 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:34.292 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:34.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.292 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.292 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4209.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-14T23:56:34.292 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:15.573293+0000", 2024-09-14T23:56:34.292 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.642868405000002, 2024-09-14T23:56:34.292 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0082988439999999997, 2024-09-14T23:56:34.292 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:34.292 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:34.292 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:34.292 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:34.292 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4209", 2024-09-14T23:56:34.293 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/1617902543", 2024-09-14T23:56:34.293 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 117 2024-09-14T23:56:34.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.293 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:34.293 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.293 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:34.293 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573293+0000", 2024-09-14T23:56:34.293 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:34.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.293 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.293 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:34.293 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573293+0000", 2024-09-14T23:56:34.293 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:34.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.294 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.294 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:34.294 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573294+0000", 2024-09-14T23:56:34.294 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2999999999999999e-07 2024-09-14T23:56:34.294 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.294 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.294 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:34.294 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573296+0000", 2024-09-14T23:56:34.294 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3319999999999999e-06 2024-09-14T23:56:34.294 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.294 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.294 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:34.294 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573300+0000", 2024-09-14T23:56:34.294 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7670000000000001e-06 2024-09-14T23:56:34.294 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.294 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.295 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:34.295 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573304+0000", 2024-09-14T23:56:34.295 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1620000000000001e-06 2024-09-14T23:56:34.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.295 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.295 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:34.295 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.580145+0000", 2024-09-14T23:56:34.295 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0068414859999999999 2024-09-14T23:56:34.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.295 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.295 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:34.295 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.580231+0000", 2024-09-14T23:56:34.295 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5997999999999999e-05 2024-09-14T23:56:34.296 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.296 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.296 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:56:34.296 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.580372+0000", 2024-09-14T23:56:34.296 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000140889 2024-09-14T23:56:34.296 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.296 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.296 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:56:34.296 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.580946+0000", 2024-09-14T23:56:34.296 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00057388100000000004 2024-09-14T23:56:34.296 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.296 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.296 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:34.296 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.581545+0000", 2024-09-14T23:56:34.297 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00059883899999999999 2024-09-14T23:56:34.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.297 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.297 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:34.297 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.581573+0000", 2024-09-14T23:56:34.297 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7596000000000001e-05 2024-09-14T23:56:34.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.297 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.297 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:34.297 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.581592+0000", 2024-09-14T23:56:34.297 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9664e-05 2024-09-14T23:56:34.297 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:34.297 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:34.298 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:34.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.298 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.298 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4209.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-14T23:56:34.298 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:16.002477+0000", 2024-09-14T23:56:34.298 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.213684970999999, 2024-09-14T23:56:34.298 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0043788669999999998, 2024-09-14T23:56:34.298 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:34.298 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:34.298 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:34.298 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:34.298 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4209", 2024-09-14T23:56:34.299 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/1617902543", 2024-09-14T23:56:34.299 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 134 2024-09-14T23:56:34.299 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.299 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:34.299 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.299 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:34.299 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002477+0000", 2024-09-14T23:56:34.299 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:34.299 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.299 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.299 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:34.299 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002477+0000", 2024-09-14T23:56:34.299 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:34.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.300 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.300 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:34.300 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002477+0000", 2024-09-14T23:56:34.300 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1699999999999998e-07 2024-09-14T23:56:34.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.300 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.300 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:34.300 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002564+0000", 2024-09-14T23:56:34.300 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7051000000000004e-05 2024-09-14T23:56:34.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.300 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.300 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:34.300 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002586+0000", 2024-09-14T23:56:34.301 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1963999999999998e-05 2024-09-14T23:56:34.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.301 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.301 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:34.301 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002606+0000", 2024-09-14T23:56:34.301 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9758e-05 2024-09-14T23:56:34.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.301 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.301 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:34.301 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.004958+0000", 2024-09-14T23:56:34.302 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023515319999999999 2024-09-14T23:56:34.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.302 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.302 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:34.302 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.005004+0000", 2024-09-14T23:56:34.302 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6493999999999999e-05 2024-09-14T23:56:34.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.302 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.302 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:56:34.302 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.005089+0000", 2024-09-14T23:56:34.302 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4577e-05 2024-09-14T23:56:34.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.302 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.302 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:34.302 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.006803+0000", 2024-09-14T23:56:34.303 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017141229999999999 2024-09-14T23:56:34.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.303 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:56:34.303 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.006810+0000", 2024-09-14T23:56:34.303 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1389999999999998e-06 2024-09-14T23:56:34.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.303 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:34.303 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.006835+0000", 2024-09-14T23:56:34.303 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5179999999999999e-05 2024-09-14T23:56:34.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.303 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:34.304 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.006856+0000", 2024-09-14T23:56:34.304 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0531999999999999e-05 2024-09-14T23:56:34.304 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:34.304 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:34.304 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:34.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.304 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.304 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4209.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-14T23:56:34.304 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:16.002615+0000", 2024-09-14T23:56:34.304 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.213546999999998, 2024-09-14T23:56:34.304 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0049197540000000001, 2024-09-14T23:56:34.304 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:34.304 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:34.304 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:34.305 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:34.305 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4209", 2024-09-14T23:56:34.305 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/1617902543", 2024-09-14T23:56:34.305 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 135 2024-09-14T23:56:34.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.305 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:34.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.305 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:34.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002615+0000", 2024-09-14T23:56:34.305 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:34.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.306 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.306 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:34.306 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002615+0000", 2024-09-14T23:56:34.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:34.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.306 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.306 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:34.306 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002615+0000", 2024-09-14T23:56:34.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.58e-07 2024-09-14T23:56:34.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.306 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.306 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:34.306 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002620+0000", 2024-09-14T23:56:34.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3320000000000002e-06 2024-09-14T23:56:34.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.307 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.307 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:34.307 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002625+0000", 2024-09-14T23:56:34.307 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9840000000000001e-06 2024-09-14T23:56:34.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.307 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.307 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:34.307 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002628+0000", 2024-09-14T23:56:34.307 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4620000000000001e-06 2024-09-14T23:56:34.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.307 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.308 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:34.308 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.005509+0000", 2024-09-14T23:56:34.308 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0028811269999999998 2024-09-14T23:56:34.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.308 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.308 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:34.308 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.005617+0000", 2024-09-14T23:56:34.308 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0001078 2024-09-14T23:56:34.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.308 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.308 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:56:34.308 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.005765+0000", 2024-09-14T23:56:34.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000148314 2024-09-14T23:56:34.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.309 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.309 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:34.309 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.006924+0000", 2024-09-14T23:56:34.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0011589630000000001 2024-09-14T23:56:34.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.309 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.309 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:56:34.309 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.007484+0000", 2024-09-14T23:56:34.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00055926900000000004 2024-09-14T23:56:34.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.309 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.310 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:34.310 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.007512+0000", 2024-09-14T23:56:34.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7988999999999999e-05 2024-09-14T23:56:34.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.310 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:34.310 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.007535+0000", 2024-09-14T23:56:34.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3056000000000001e-05 2024-09-14T23:56:34.311 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:34.311 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:34.311 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:34.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.311 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.311 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4209.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-14T23:56:34.312 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:16.002632+0000", 2024-09-14T23:56:34.312 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.213529821000002, 2024-09-14T23:56:34.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0056205409999999997, 2024-09-14T23:56:34.312 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:34.312 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:34.312 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:34.312 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:34.313 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4209", 2024-09-14T23:56:34.313 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/1617902543", 2024-09-14T23:56:34.313 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 136 2024-09-14T23:56:34.313 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.313 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:34.313 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.314 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:34.314 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002632+0000", 2024-09-14T23:56:34.314 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:34.314 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.314 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.314 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:34.314 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002632+0000", 2024-09-14T23:56:34.315 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:34.315 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.315 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.315 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:34.315 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002632+0000", 2024-09-14T23:56:34.315 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4699999999999998e-07 2024-09-14T23:56:34.315 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.316 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.316 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:34.316 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002634+0000", 2024-09-14T23:56:34.316 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.199e-06 2024-09-14T23:56:34.316 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.316 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.317 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:34.317 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002646+0000", 2024-09-14T23:56:34.317 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.111e-05 2024-09-14T23:56:34.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.317 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.317 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:34.317 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002650+0000", 2024-09-14T23:56:34.318 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.938e-06 2024-09-14T23:56:34.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.318 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.318 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:34.318 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.006332+0000", 2024-09-14T23:56:34.318 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003681068 2024-09-14T23:56:34.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.318 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.318 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:34.318 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.006412+0000", 2024-09-14T23:56:34.319 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0127e-05 2024-09-14T23:56:34.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.319 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.319 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:56:34.319 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.006502+0000", 2024-09-14T23:56:34.319 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0064999999999999e-05 2024-09-14T23:56:34.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.319 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.319 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:34.319 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.008170+0000", 2024-09-14T23:56:34.319 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016677580000000001 2024-09-14T23:56:34.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.319 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.320 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:56:34.320 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.008179+0000", 2024-09-14T23:56:34.320 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9669999999999996e-06 2024-09-14T23:56:34.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.320 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.320 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:34.320 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.008214+0000", 2024-09-14T23:56:34.320 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4878e-05 2024-09-14T23:56:34.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.320 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.320 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:34.320 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.008253+0000", 2024-09-14T23:56:34.320 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8183999999999999e-05 2024-09-14T23:56:34.320 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:34.321 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:34.321 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:34.321 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.321 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.321 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4209.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-14T23:56:34.321 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:16.002663+0000", 2024-09-14T23:56:34.321 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.213498557000001, 2024-09-14T23:56:34.321 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0071466480000000002, 2024-09-14T23:56:34.321 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:34.321 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:34.321 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:34.321 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:34.321 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4209", 2024-09-14T23:56:34.322 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/1617902543", 2024-09-14T23:56:34.322 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 137 2024-09-14T23:56:34.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.322 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:34.322 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.322 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:34.322 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002663+0000", 2024-09-14T23:56:34.322 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:34.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.322 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.322 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:34.322 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002663+0000", 2024-09-14T23:56:34.322 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:34.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.323 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.323 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:34.323 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002664+0000", 2024-09-14T23:56:34.323 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6100000000000001e-07 2024-09-14T23:56:34.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.323 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.323 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:34.323 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002786+0000", 2024-09-14T23:56:34.323 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000122664 2024-09-14T23:56:34.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.324 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.324 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:34.324 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002798+0000", 2024-09-14T23:56:34.324 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1671000000000001e-05 2024-09-14T23:56:34.324 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.324 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.325 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:34.325 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002808+0000", 2024-09-14T23:56:34.325 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8330000000000006e-06 2024-09-14T23:56:34.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.325 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.325 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:34.325 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.006952+0000", 2024-09-14T23:56:34.326 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0041437059999999996 2024-09-14T23:56:34.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.326 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.326 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:34.326 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.006998+0000", 2024-09-14T23:56:34.326 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6508999999999998e-05 2024-09-14T23:56:34.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.327 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.327 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:56:34.327 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.007100+0000", 2024-09-14T23:56:34.327 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010176 2024-09-14T23:56:34.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.327 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.327 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:56:34.328 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.009307+0000", 2024-09-14T23:56:34.328 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0022067459999999999 2024-09-14T23:56:34.328 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.328 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.328 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:34.328 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.009757+0000", 2024-09-14T23:56:34.328 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00045041800000000001 2024-09-14T23:56:34.329 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.329 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.329 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:34.329 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.009789+0000", 2024-09-14T23:56:34.329 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1599e-05 2024-09-14T23:56:34.329 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.330 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.330 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:34.330 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.009810+0000", 2024-09-14T23:56:34.330 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1280999999999998e-05 2024-09-14T23:56:34.330 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:34.330 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:34.331 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:34.331 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.331 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.331 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4209.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-14T23:56:34.331 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:16.002824+0000", 2024-09-14T23:56:34.331 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.213338233999998, 2024-09-14T23:56:34.331 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0070812080000000003, 2024-09-14T23:56:34.331 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:34.331 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:34.332 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:34.332 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:34.332 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4209", 2024-09-14T23:56:34.332 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/1617902543", 2024-09-14T23:56:34.332 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 138 2024-09-14T23:56:34.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.332 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:34.332 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.332 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:34.332 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002824+0000", 2024-09-14T23:56:34.332 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:34.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.332 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.333 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:34.333 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002824+0000", 2024-09-14T23:56:34.333 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:34.333 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.333 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.333 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:34.333 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002824+0000", 2024-09-14T23:56:34.333 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7999999999999995e-07 2024-09-14T23:56:34.333 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.333 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.333 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:34.333 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002828+0000", 2024-09-14T23:56:34.333 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.262e-06 2024-09-14T23:56:34.334 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.334 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.334 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:34.334 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002836+0000", 2024-09-14T23:56:34.334 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6769999999999995e-06 2024-09-14T23:56:34.334 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.334 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.334 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:34.334 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002842+0000", 2024-09-14T23:56:34.334 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4809999999999997e-06 2024-09-14T23:56:34.334 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.334 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.335 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:34.335 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.007558+0000", 2024-09-14T23:56:34.335 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0047168720000000004 2024-09-14T23:56:34.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.335 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.335 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:34.335 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.007643+0000", 2024-09-14T23:56:34.335 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4289999999999994e-05 2024-09-14T23:56:34.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.335 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.335 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:56:34.335 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.007783+0000", 2024-09-14T23:56:34.335 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013978800000000001 2024-09-14T23:56:34.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.336 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.336 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:56:34.336 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.009817+0000", 2024-09-14T23:56:34.336 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0020344930000000001 2024-09-14T23:56:34.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.336 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.336 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:34.336 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.009854+0000", 2024-09-14T23:56:34.336 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6891000000000003e-05 2024-09-14T23:56:34.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.336 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.336 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:34.336 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.009882+0000", 2024-09-14T23:56:34.336 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8130999999999999e-05 2024-09-14T23:56:34.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.337 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.337 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:34.337 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.009905+0000", 2024-09-14T23:56:34.337 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2742999999999999e-05 2024-09-14T23:56:34.337 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:34.337 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:34.337 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:34.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.337 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.337 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4209.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-14T23:56:34.337 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:16.002851+0000", 2024-09-14T23:56:34.337 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.213310985, 2024-09-14T23:56:34.337 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0071603530000000004, 2024-09-14T23:56:34.338 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:34.338 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:34.338 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:34.338 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:34.338 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4209", 2024-09-14T23:56:34.338 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/1617902543", 2024-09-14T23:56:34.338 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 139 2024-09-14T23:56:34.338 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.338 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:34.338 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.338 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:34.338 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002851+0000", 2024-09-14T23:56:34.338 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:34.338 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.339 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.339 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:34.339 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002851+0000", 2024-09-14T23:56:34.339 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:34.339 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.339 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.339 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:34.339 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002851+0000", 2024-09-14T23:56:34.339 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0699999999999998e-07 2024-09-14T23:56:34.339 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.339 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.339 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:34.339 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002853+0000", 2024-09-14T23:56:34.339 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2560000000000001e-06 2024-09-14T23:56:34.340 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.340 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.340 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:34.340 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002861+0000", 2024-09-14T23:56:34.340 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7139999999999993e-06 2024-09-14T23:56:34.340 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.340 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.340 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:34.340 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002866+0000", 2024-09-14T23:56:34.340 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1329999999999998e-06 2024-09-14T23:56:34.340 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.340 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.340 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:34.340 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.008306+0000", 2024-09-14T23:56:34.341 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.005439503 2024-09-14T23:56:34.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.341 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.341 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:34.341 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.008387+0000", 2024-09-14T23:56:34.341 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1051999999999996e-05 2024-09-14T23:56:34.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.341 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.341 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:56:34.341 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.008477+0000", 2024-09-14T23:56:34.341 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9944000000000005e-05 2024-09-14T23:56:34.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.342 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.342 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:56:34.342 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.009824+0000", 2024-09-14T23:56:34.342 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0013476390000000001 2024-09-14T23:56:34.342 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.343 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.343 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:34.344 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.009945+0000", 2024-09-14T23:56:34.344 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012059500000000001 2024-09-14T23:56:34.344 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.344 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.344 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:34.344 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.009980+0000", 2024-09-14T23:56:34.344 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5469999999999998e-05 2024-09-14T23:56:34.344 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.344 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.344 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:34.344 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.010011+0000", 2024-09-14T23:56:34.344 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.074e-05 2024-09-14T23:56:34.344 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:34.345 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:34.345 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:34.345 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.345 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.345 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4209.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-14T23:56:34.345 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:16.002875+0000", 2024-09-14T23:56:34.345 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.213286872000001, 2024-09-14T23:56:34.345 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0073135270000000002, 2024-09-14T23:56:34.345 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:34.345 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:34.345 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:34.346 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:34.346 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4209", 2024-09-14T23:56:34.346 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/1617902543", 2024-09-14T23:56:34.346 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 140 2024-09-14T23:56:34.346 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.346 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:34.346 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.346 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:34.346 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002875+0000", 2024-09-14T23:56:34.346 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:34.346 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.346 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.346 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:34.347 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002875+0000", 2024-09-14T23:56:34.347 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:34.347 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.347 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.347 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:34.347 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002875+0000", 2024-09-14T23:56:34.347 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2500000000000001e-07 2024-09-14T23:56:34.347 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.347 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.347 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:34.347 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002936+0000", 2024-09-14T23:56:34.347 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0621000000000001e-05 2024-09-14T23:56:34.347 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.347 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.347 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:34.348 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002948+0000", 2024-09-14T23:56:34.348 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1651000000000001e-05 2024-09-14T23:56:34.348 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.348 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.348 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:34.348 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002954+0000", 2024-09-14T23:56:34.348 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3169999999999998e-06 2024-09-14T23:56:34.348 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.348 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.348 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:34.349 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.008711+0000", 2024-09-14T23:56:34.349 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057567269999999997 2024-09-14T23:56:34.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.349 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.349 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:34.349 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.008782+0000", 2024-09-14T23:56:34.349 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0970999999999995e-05 2024-09-14T23:56:34.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.349 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.349 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:56:34.349 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.008870+0000", 2024-09-14T23:56:34.349 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7849999999999994e-05 2024-09-14T23:56:34.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.349 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.350 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:56:34.350 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.010019+0000", 2024-09-14T23:56:34.350 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0011495979999999999 2024-09-14T23:56:34.350 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.350 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.350 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:34.350 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.010140+0000", 2024-09-14T23:56:34.350 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000120541 2024-09-14T23:56:34.350 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.350 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.350 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:34.350 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.010167+0000", 2024-09-14T23:56:34.350 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7549e-05 2024-09-14T23:56:34.351 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.351 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.351 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:34.351 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.010188+0000", 2024-09-14T23:56:34.351 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1277e-05 2024-09-14T23:56:34.351 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:34.351 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:34.351 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:34.351 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.351 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.351 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4209.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-14T23:56:34.351 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:16.002967+0000", 2024-09-14T23:56:34.351 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.213194395999999, 2024-09-14T23:56:34.351 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0074891690000000004, 2024-09-14T23:56:34.352 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:34.352 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:34.352 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:34.352 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:34.352 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4209", 2024-09-14T23:56:34.352 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/1617902543", 2024-09-14T23:56:34.352 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 141 2024-09-14T23:56:34.352 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.352 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:34.352 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.352 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:34.352 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002967+0000", 2024-09-14T23:56:34.352 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:34.353 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.353 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.353 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:34.353 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002967+0000", 2024-09-14T23:56:34.353 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:34.353 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.353 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.353 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:34.353 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002968+0000", 2024-09-14T23:56:34.353 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0699999999999997e-07 2024-09-14T23:56:34.353 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.353 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.353 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:34.354 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002972+0000", 2024-09-14T23:56:34.354 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.782e-06 2024-09-14T23:56:34.354 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.354 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.354 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:34.354 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002979+0000", 2024-09-14T23:56:34.354 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2089999999999999e-06 2024-09-14T23:56:34.354 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.354 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.354 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:34.354 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002984+0000", 2024-09-14T23:56:34.354 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1399999999999999e-06 2024-09-14T23:56:34.354 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.354 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.355 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:34.355 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.009341+0000", 2024-09-14T23:56:34.355 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0063574060000000003 2024-09-14T23:56:34.355 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.355 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.355 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:34.355 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.009388+0000", 2024-09-14T23:56:34.355 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.651e-05 2024-09-14T23:56:34.355 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.355 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.355 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:56:34.355 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.009464+0000", 2024-09-14T23:56:34.355 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5829999999999997e-05 2024-09-14T23:56:34.355 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.356 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.356 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:56:34.356 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.010024+0000", 2024-09-14T23:56:34.356 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00056062099999999997 2024-09-14T23:56:34.356 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.356 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.356 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:34.356 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.010408+0000", 2024-09-14T23:56:34.356 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00038369899999999999 2024-09-14T23:56:34.356 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.356 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.356 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:34.356 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.010435+0000", 2024-09-14T23:56:34.356 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6648e-05 2024-09-14T23:56:34.357 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:34.357 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:34.357 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:34.357 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.010457+0000", 2024-09-14T23:56:34.357 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1817000000000001e-05 2024-09-14T23:56:34.357 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:34.357 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:34.357 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:34.357 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:34.357 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:34.357 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:56:34.359 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T23:56:34.359 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:56:34.357+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:56:34.398 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is set 2024-09-14T23:56:34.459 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:56:34.458+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:56:34.486 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:56:34.486 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:56:34.486 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:56:34.486 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:56:34.495 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T23:56:34.560 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:56:34.559+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:56:34.643 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:56:34.643 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:56:34.644 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:56:34.644 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:56:34.644 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:56:34.652 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T23:56:34.696 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:56:34.695+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:56:34.763 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:56:34.763 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:56:34.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:56:34.763 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:56:34.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.763 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4209.0:20 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'10, mlcod=16'8)", 2024-09-14T23:56:34.763 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:15.349996+0000", 2024-09-14T23:56:34.763 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.410059051000001, 2024-09-14T23:56:34.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010569310000000001, 2024-09-14T23:56:34.763 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:34.763 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:34.764 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:34.764 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:34.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.764 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:34.764 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.349996+0000", 2024-09-14T23:56:34.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:34.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.764 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:34.764 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.349996+0000", 2024-09-14T23:56:34.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:34.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.764 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:34.765 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.349997+0000", 2024-09-14T23:56:34.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2200000000000002e-07 2024-09-14T23:56:34.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.765 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:34.765 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.350123+0000", 2024-09-14T23:56:34.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000126797 2024-09-14T23:56:34.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.765 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:34.765 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.350141+0000", 2024-09-14T23:56:34.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7110000000000001e-05 2024-09-14T23:56:34.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:34.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.350150+0000", 2024-09-14T23:56:34.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3370000000000006e-06 2024-09-14T23:56:34.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:34.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.350213+0000", 2024-09-14T23:56:34.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2718999999999994e-05 2024-09-14T23:56:34.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:34.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.350235+0000", 2024-09-14T23:56:34.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2055000000000001e-05 2024-09-14T23:56:34.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:34.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.350990+0000", 2024-09-14T23:56:34.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00075517900000000003 2024-09-14T23:56:34.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:34.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.351053+0000", 2024-09-14T23:56:34.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2811999999999994e-05 2024-09-14T23:56:34.767 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:34.767 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:34.767 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:34.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.768 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4209.0:31 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'17, mlcod=16'14)", 2024-09-14T23:56:34.768 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:15.358180+0000", 2024-09-14T23:56:34.768 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.401874685999999, 2024-09-14T23:56:34.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001091533, 2024-09-14T23:56:34.768 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:34.768 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:34.768 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:34.768 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:34.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:34.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.358180+0000", 2024-09-14T23:56:34.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:34.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.769 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:34.769 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.358180+0000", 2024-09-14T23:56:34.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:34.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.769 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:34.769 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.358181+0000", 2024-09-14T23:56:34.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0100000000000004e-07 2024-09-14T23:56:34.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.769 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:34.769 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.358187+0000", 2024-09-14T23:56:34.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6509999999999998e-06 2024-09-14T23:56:34.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.770 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:34.770 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.358195+0000", 2024-09-14T23:56:34.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7590000000000003e-06 2024-09-14T23:56:34.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.770 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:34.770 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.358202+0000", 2024-09-14T23:56:34.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6089999999999997e-06 2024-09-14T23:56:34.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.770 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:34.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.358250+0000", 2024-09-14T23:56:34.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8519000000000001e-05 2024-09-14T23:56:34.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:34.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.358265+0000", 2024-09-14T23:56:34.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4749999999999999e-05 2024-09-14T23:56:34.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:34.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.359220+0000", 2024-09-14T23:56:34.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00095522200000000004 2024-09-14T23:56:34.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:34.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.359272+0000", 2024-09-14T23:56:34.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1221999999999998e-05 2024-09-14T23:56:34.772 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:34.772 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:34.772 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:34.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.772 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4209.0:32 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'18, mlcod=16'14)", 2024-09-14T23:56:34.772 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:15.358686+0000", 2024-09-14T23:56:34.773 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.401369272, 2024-09-14T23:56:34.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00098863800000000002, 2024-09-14T23:56:34.773 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:34.773 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:34.773 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:34.773 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:34.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:34.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.358686+0000", 2024-09-14T23:56:34.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:34.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:34.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.358686+0000", 2024-09-14T23:56:34.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:34.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:34.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.358686+0000", 2024-09-14T23:56:34.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6000000000000002e-07 2024-09-14T23:56:34.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:34.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.358755+0000", 2024-09-14T23:56:34.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8532e-05 2024-09-14T23:56:34.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:34.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.358767+0000", 2024-09-14T23:56:34.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1960999999999999e-05 2024-09-14T23:56:34.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:34.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.358774+0000", 2024-09-14T23:56:34.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5379999999999997e-06 2024-09-14T23:56:34.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:34.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.358819+0000", 2024-09-14T23:56:34.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4601999999999997e-05 2024-09-14T23:56:34.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:34.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.358832+0000", 2024-09-14T23:56:34.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3125000000000001e-05 2024-09-14T23:56:34.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:34.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.359614+0000", 2024-09-14T23:56:34.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00078198899999999997 2024-09-14T23:56:34.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:34.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.359674+0000", 2024-09-14T23:56:34.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0031000000000002e-05 2024-09-14T23:56:34.776 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:34.776 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:34.777 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:34.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.777 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4209.0:33 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'19, mlcod=16'14)", 2024-09-14T23:56:34.777 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:15.359228+0000", 2024-09-14T23:56:34.777 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.400827275000001, 2024-09-14T23:56:34.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00094243599999999995, 2024-09-14T23:56:34.777 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:34.777 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:34.777 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:34.777 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:34.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:34.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.359228+0000", 2024-09-14T23:56:34.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:34.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:34.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.359228+0000", 2024-09-14T23:56:34.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:34.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:34.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.359228+0000", 2024-09-14T23:56:34.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2799999999999995e-07 2024-09-14T23:56:34.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:34.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.359236+0000", 2024-09-14T23:56:34.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3839999999999999e-06 2024-09-14T23:56:34.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:34.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.359247+0000", 2024-09-14T23:56:34.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1353e-05 2024-09-14T23:56:34.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:34.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.359264+0000", 2024-09-14T23:56:34.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6784000000000001e-05 2024-09-14T23:56:34.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:34.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.359294+0000", 2024-09-14T23:56:34.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0372e-05 2024-09-14T23:56:34.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:34.781 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.359309+0000", 2024-09-14T23:56:34.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5243e-05 2024-09-14T23:56:34.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.781 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:34.781 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.360142+0000", 2024-09-14T23:56:34.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00083309900000000001 2024-09-14T23:56:34.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.781 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:34.781 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.360170+0000", 2024-09-14T23:56:34.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7472999999999999e-05 2024-09-14T23:56:34.781 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:34.782 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:34.782 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:34.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.782 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4209.0:35 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'21, mlcod=16'16)", 2024-09-14T23:56:34.782 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:15.360261+0000", 2024-09-14T23:56:34.782 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.399794257, 2024-09-14T23:56:34.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010405530000000001, 2024-09-14T23:56:34.782 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:34.782 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:34.782 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:34.782 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:34.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:34.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.360261+0000", 2024-09-14T23:56:34.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:34.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:34.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.360261+0000", 2024-09-14T23:56:34.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:34.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:34.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.360261+0000", 2024-09-14T23:56:34.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3300000000000001e-07 2024-09-14T23:56:34.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:34.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.360321+0000", 2024-09-14T23:56:34.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9626999999999997e-05 2024-09-14T23:56:34.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:34.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.360334+0000", 2024-09-14T23:56:34.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2686e-05 2024-09-14T23:56:34.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:34.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.360340+0000", 2024-09-14T23:56:34.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8909999999999998e-06 2024-09-14T23:56:34.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:34.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.360382+0000", 2024-09-14T23:56:34.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1896e-05 2024-09-14T23:56:34.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:34.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.360397+0000", 2024-09-14T23:56:34.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4959e-05 2024-09-14T23:56:34.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:34.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.361275+0000", 2024-09-14T23:56:34.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00087766000000000005 2024-09-14T23:56:34.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:34.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.361301+0000", 2024-09-14T23:56:34.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6101e-05 2024-09-14T23:56:34.787 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:34.787 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:34.787 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:34.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.787 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4209.0:54 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'32, mlcod=16'29)", 2024-09-14T23:56:34.787 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:15.375492+0000", 2024-09-14T23:56:34.787 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.384562705, 2024-09-14T23:56:34.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015276700000000001, 2024-09-14T23:56:34.787 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:34.787 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:34.787 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:34.787 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:34.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:34.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.375492+0000", 2024-09-14T23:56:34.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:34.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:34.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.375492+0000", 2024-09-14T23:56:34.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:34.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:34.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.375493+0000", 2024-09-14T23:56:34.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0299999999999999e-07 2024-09-14T23:56:34.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:34.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.375497+0000", 2024-09-14T23:56:34.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.189e-06 2024-09-14T23:56:34.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:34.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.375505+0000", 2024-09-14T23:56:34.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4679999999999992e-06 2024-09-14T23:56:34.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:34.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.375510+0000", 2024-09-14T23:56:34.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8720000000000001e-06 2024-09-14T23:56:34.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:34.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.375558+0000", 2024-09-14T23:56:34.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7590999999999999e-05 2024-09-14T23:56:34.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:34.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.375581+0000", 2024-09-14T23:56:34.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3082999999999999e-05 2024-09-14T23:56:34.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:34.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.376982+0000", 2024-09-14T23:56:34.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0014012670000000001 2024-09-14T23:56:34.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:34.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.377020+0000", 2024-09-14T23:56:34.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7697000000000002e-05 2024-09-14T23:56:34.791 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:34.791 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:34.791 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:34.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.791 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4209.0:55 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'33, mlcod=16'29)", 2024-09-14T23:56:34.792 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:15.376055+0000", 2024-09-14T23:56:34.792 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.383999731999999, 2024-09-14T23:56:34.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00100033, 2024-09-14T23:56:34.792 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:34.792 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:34.792 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:34.792 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:34.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:34.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.376055+0000", 2024-09-14T23:56:34.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:34.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:34.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.376055+0000", 2024-09-14T23:56:34.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:34.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:34.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.376056+0000", 2024-09-14T23:56:34.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8500000000000002e-07 2024-09-14T23:56:34.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:34.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.376124+0000", 2024-09-14T23:56:34.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8608000000000004e-05 2024-09-14T23:56:34.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:34.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.376134+0000", 2024-09-14T23:56:34.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9329999999999996e-06 2024-09-14T23:56:34.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:34.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.376138+0000", 2024-09-14T23:56:34.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3420000000000001e-06 2024-09-14T23:56:34.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:34.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.376184+0000", 2024-09-14T23:56:34.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5666e-05 2024-09-14T23:56:34.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:34.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.376205+0000", 2024-09-14T23:56:34.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0812999999999999e-05 2024-09-14T23:56:34.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:34.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.377024+0000", 2024-09-14T23:56:34.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00081868100000000003 2024-09-14T23:56:34.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.796 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:34.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.377055+0000", 2024-09-14T23:56:34.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1801999999999998e-05 2024-09-14T23:56:34.796 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:34.796 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:34.796 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:34.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.796 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4209.0:68 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'42, mlcod=16'39)", 2024-09-14T23:56:34.796 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:15.387115+0000", 2024-09-14T23:56:34.797 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.372939880000001, 2024-09-14T23:56:34.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011477900000000001, 2024-09-14T23:56:34.797 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:34.797 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:34.797 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:34.797 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:34.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:34.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.387115+0000", 2024-09-14T23:56:34.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:34.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:34.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.387115+0000", 2024-09-14T23:56:34.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:34.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:34.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.387115+0000", 2024-09-14T23:56:34.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1799999999999995e-07 2024-09-14T23:56:34.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:34.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.387119+0000", 2024-09-14T23:56:34.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0459999999999999e-06 2024-09-14T23:56:34.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:34.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.387128+0000", 2024-09-14T23:56:34.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8459999999999997e-06 2024-09-14T23:56:34.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:34.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.387133+0000", 2024-09-14T23:56:34.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6e-06 2024-09-14T23:56:34.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:34.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.387175+0000", 2024-09-14T23:56:34.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1841999999999997e-05 2024-09-14T23:56:34.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:34.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.387187+0000", 2024-09-14T23:56:34.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2476999999999999e-05 2024-09-14T23:56:34.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:34.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.388226+0000", 2024-09-14T23:56:34.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010386060000000001 2024-09-14T23:56:34.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:34.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.388263+0000", 2024-09-14T23:56:34.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6854999999999999e-05 2024-09-14T23:56:34.801 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:34.801 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:34.801 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:34.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.802 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4209.0:69 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'43, mlcod=16'39)", 2024-09-14T23:56:34.802 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:15.387660+0000", 2024-09-14T23:56:34.802 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.372395019999999, 2024-09-14T23:56:34.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001072859, 2024-09-14T23:56:34.802 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:34.802 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:34.802 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:34.802 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:34.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:34.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.387660+0000", 2024-09-14T23:56:34.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:34.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:34.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.387660+0000", 2024-09-14T23:56:34.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:34.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:34.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.387660+0000", 2024-09-14T23:56:34.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0399999999999996e-07 2024-09-14T23:56:34.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:34.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.387732+0000", 2024-09-14T23:56:34.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1585e-05 2024-09-14T23:56:34.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:34.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.387743+0000", 2024-09-14T23:56:34.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1015e-05 2024-09-14T23:56:34.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:34.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.387748+0000", 2024-09-14T23:56:34.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4750000000000001e-06 2024-09-14T23:56:34.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:34.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.387806+0000", 2024-09-14T23:56:34.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7398000000000002e-05 2024-09-14T23:56:34.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:34.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.387825+0000", 2024-09-14T23:56:34.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9369999999999999e-05 2024-09-14T23:56:34.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:34.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.388693+0000", 2024-09-14T23:56:34.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000868106 2024-09-14T23:56:34.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:34.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.388733+0000", 2024-09-14T23:56:34.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9305999999999999e-05 2024-09-14T23:56:34.806 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:34.806 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:34.806 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:34.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.806 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4209.0:72 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'46, mlcod=16'43)", 2024-09-14T23:56:34.806 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:15.390494+0000", 2024-09-14T23:56:34.806 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.369560958000001, 2024-09-14T23:56:34.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00097883900000000001, 2024-09-14T23:56:34.807 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:34.807 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:34.807 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:34.807 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:34.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:34.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.390494+0000", 2024-09-14T23:56:34.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:34.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:34.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.390494+0000", 2024-09-14T23:56:34.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:34.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:34.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.390494+0000", 2024-09-14T23:56:34.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5000000000000002e-07 2024-09-14T23:56:34.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:34.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.390587+0000", 2024-09-14T23:56:34.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2516999999999995e-05 2024-09-14T23:56:34.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:34.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.390607+0000", 2024-09-14T23:56:34.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9667e-05 2024-09-14T23:56:34.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:34.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.390617+0000", 2024-09-14T23:56:34.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8949999999999993e-06 2024-09-14T23:56:34.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:34.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.390690+0000", 2024-09-14T23:56:34.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3946e-05 2024-09-14T23:56:34.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:34.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.390704+0000", 2024-09-14T23:56:34.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3021e-05 2024-09-14T23:56:34.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:34.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.391437+0000", 2024-09-14T23:56:34.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000733404 2024-09-14T23:56:34.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:34.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.391473+0000", 2024-09-14T23:56:34.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5738999999999998e-05 2024-09-14T23:56:34.811 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:34.811 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:34.811 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:34.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.811 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4209.0:83 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'53, mlcod=16'50)", 2024-09-14T23:56:34.811 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:15.398499+0000", 2024-09-14T23:56:34.811 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.361555996, 2024-09-14T23:56:34.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001002333, 2024-09-14T23:56:34.811 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:34.812 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:34.812 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:34.812 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:34.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:34.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.398499+0000", 2024-09-14T23:56:34.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:34.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:34.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.398499+0000", 2024-09-14T23:56:34.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:34.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:34.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.398499+0000", 2024-09-14T23:56:34.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8299999999999997e-07 2024-09-14T23:56:34.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:34.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.398505+0000", 2024-09-14T23:56:34.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2680000000000002e-06 2024-09-14T23:56:34.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:34.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.398513+0000", 2024-09-14T23:56:34.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8970000000000008e-06 2024-09-14T23:56:34.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:34.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.398517+0000", 2024-09-14T23:56:34.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1949999999999996e-06 2024-09-14T23:56:34.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:34.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.398567+0000", 2024-09-14T23:56:34.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9917999999999999e-05 2024-09-14T23:56:34.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:34.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.398590+0000", 2024-09-14T23:56:34.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2952e-05 2024-09-14T23:56:34.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:34.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.399464+0000", 2024-09-14T23:56:34.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00087452199999999997 2024-09-14T23:56:34.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:34.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.399501+0000", 2024-09-14T23:56:34.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6998000000000001e-05 2024-09-14T23:56:34.815 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:34.815 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:34.815 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:34.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.816 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4209.0:84 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'54, mlcod=16'50)", 2024-09-14T23:56:34.816 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:15.399005+0000", 2024-09-14T23:56:34.816 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.361050019, 2024-09-14T23:56:34.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00098027800000000001, 2024-09-14T23:56:34.816 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:34.816 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:34.816 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:34.816 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:34.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:34.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.399005+0000", 2024-09-14T23:56:34.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:34.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:34.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.399005+0000", 2024-09-14T23:56:34.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:34.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:34.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.399005+0000", 2024-09-14T23:56:34.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8800000000000003e-07 2024-09-14T23:56:34.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:34.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.399068+0000", 2024-09-14T23:56:34.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2464000000000004e-05 2024-09-14T23:56:34.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:34.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.399076+0000", 2024-09-14T23:56:34.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7560000000000005e-06 2024-09-14T23:56:34.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:34.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.399081+0000", 2024-09-14T23:56:34.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4399999999999998e-06 2024-09-14T23:56:34.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:34.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.399123+0000", 2024-09-14T23:56:34.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2113000000000002e-05 2024-09-14T23:56:34.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:34.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.399138+0000", 2024-09-14T23:56:34.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5251e-05 2024-09-14T23:56:34.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:34.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.399947+0000", 2024-09-14T23:56:34.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00080886499999999997 2024-09-14T23:56:34.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:34.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.399985+0000", 2024-09-14T23:56:34.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7901000000000002e-05 2024-09-14T23:56:34.820 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:34.820 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:34.820 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:34.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.820 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4209.0:87 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'57, mlcod=16'54)", 2024-09-14T23:56:34.820 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:15.401127+0000", 2024-09-14T23:56:34.820 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.358927333, 2024-09-14T23:56:34.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010080410000000001, 2024-09-14T23:56:34.820 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:34.821 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:34.821 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:34.821 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:34.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:34.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.401127+0000", 2024-09-14T23:56:34.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:34.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:34.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.401127+0000", 2024-09-14T23:56:34.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:34.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:34.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.401128+0000", 2024-09-14T23:56:34.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1200000000000003e-07 2024-09-14T23:56:34.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:34.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.401197+0000", 2024-09-14T23:56:34.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8922000000000002e-05 2024-09-14T23:56:34.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:34.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.401207+0000", 2024-09-14T23:56:34.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6350000000000001e-06 2024-09-14T23:56:34.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:34.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.401211+0000", 2024-09-14T23:56:34.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8910000000000003e-06 2024-09-14T23:56:34.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:34.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.401252+0000", 2024-09-14T23:56:34.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0308000000000001e-05 2024-09-14T23:56:34.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:34.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.401266+0000", 2024-09-14T23:56:34.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3966e-05 2024-09-14T23:56:34.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:34.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.402098+0000", 2024-09-14T23:56:34.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00083255899999999997 2024-09-14T23:56:34.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:34.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.402135+0000", 2024-09-14T23:56:34.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7248e-05 2024-09-14T23:56:34.824 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:34.824 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:34.824 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:34.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.825 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4209.0:89 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'59, mlcod=16'57)", 2024-09-14T23:56:34.825 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:15.403067+0000", 2024-09-14T23:56:34.825 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.356987961000002, 2024-09-14T23:56:34.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001037406, 2024-09-14T23:56:34.825 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:34.825 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:34.825 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:34.825 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:34.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:34.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.403067+0000", 2024-09-14T23:56:34.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:34.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:34.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.403067+0000", 2024-09-14T23:56:34.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:34.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:34.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.403068+0000", 2024-09-14T23:56:34.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9599999999999999e-07 2024-09-14T23:56:34.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:34.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.403113+0000", 2024-09-14T23:56:34.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5893999999999998e-05 2024-09-14T23:56:34.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:34.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.403125+0000", 2024-09-14T23:56:34.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1654999999999999e-05 2024-09-14T23:56:34.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:34.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.403130+0000", 2024-09-14T23:56:34.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2190000000000003e-06 2024-09-14T23:56:34.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:34.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.403178+0000", 2024-09-14T23:56:34.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8183999999999998e-05 2024-09-14T23:56:34.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:34.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.403193+0000", 2024-09-14T23:56:34.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4457000000000001e-05 2024-09-14T23:56:34.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:34.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.404068+0000", 2024-09-14T23:56:34.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00087506499999999996 2024-09-14T23:56:34.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:34.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.404104+0000", 2024-09-14T23:56:34.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6235999999999997e-05 2024-09-14T23:56:34.830 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:34.830 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:34.830 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:34.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.831 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4209.0:105 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'67, mlcod=16'64)", 2024-09-14T23:56:34.831 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:15.573180+0000", 2024-09-14T23:56:34.831 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.186874800999998, 2024-09-14T23:56:34.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001054538, 2024-09-14T23:56:34.831 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:34.831 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:34.831 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:34.831 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:34.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:34.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.573180+0000", 2024-09-14T23:56:34.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:34.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:34.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.573180+0000", 2024-09-14T23:56:34.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:34.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:34.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.573181+0000", 2024-09-14T23:56:34.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8899999999999999e-07 2024-09-14T23:56:34.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:34.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.573188+0000", 2024-09-14T23:56:34.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2880000000000004e-06 2024-09-14T23:56:34.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:34.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.573199+0000", 2024-09-14T23:56:34.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1197e-05 2024-09-14T23:56:34.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:34.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.573209+0000", 2024-09-14T23:56:34.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0006e-05 2024-09-14T23:56:34.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:34.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.573249+0000", 2024-09-14T23:56:34.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0204999999999999e-05 2024-09-14T23:56:34.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:34.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.573267+0000", 2024-09-14T23:56:34.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7484000000000001e-05 2024-09-14T23:56:34.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:34.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.574205+0000", 2024-09-14T23:56:34.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00093827299999999999 2024-09-14T23:56:34.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:34.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.574235+0000", 2024-09-14T23:56:34.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9496e-05 2024-09-14T23:56:34.835 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:34.835 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:34.835 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:34.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.835 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4209.0:115 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'77, mlcod=16'73)", 2024-09-14T23:56:34.835 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:15.580084+0000", 2024-09-14T23:56:34.835 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.179970381, 2024-09-14T23:56:34.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010510949999999999, 2024-09-14T23:56:34.835 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:34.836 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:34.836 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:34.836 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:34.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:34.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.580084+0000", 2024-09-14T23:56:34.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:34.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:34.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.580084+0000", 2024-09-14T23:56:34.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:34.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:34.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.580085+0000", 2024-09-14T23:56:34.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4400000000000001e-07 2024-09-14T23:56:34.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:34.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.580219+0000", 2024-09-14T23:56:34.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000133996 2024-09-14T23:56:34.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:34.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.580227+0000", 2024-09-14T23:56:34.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1210000000000007e-06 2024-09-14T23:56:34.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:34.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.580231+0000", 2024-09-14T23:56:34.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5379999999999996e-06 2024-09-14T23:56:34.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:34.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.580278+0000", 2024-09-14T23:56:34.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6882999999999998e-05 2024-09-14T23:56:34.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:34.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.580296+0000", 2024-09-14T23:56:34.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7771e-05 2024-09-14T23:56:34.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:34.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.581099+0000", 2024-09-14T23:56:34.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00080261199999999997 2024-09-14T23:56:34.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:34.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.581135+0000", 2024-09-14T23:56:34.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6730000000000002e-05 2024-09-14T23:56:34.839 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:34.840 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:34.840 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:34.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.840 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4209.0:117 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'79, mlcod=16'74)", 2024-09-14T23:56:34.840 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:15.580495+0000", 2024-09-14T23:56:34.840 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.179559944000001, 2024-09-14T23:56:34.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00098058900000000003, 2024-09-14T23:56:34.840 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:34.840 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:34.840 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:34.840 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:34.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:34.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.580495+0000", 2024-09-14T23:56:34.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:34.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:34.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.580495+0000", 2024-09-14T23:56:34.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:34.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:34.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.580495+0000", 2024-09-14T23:56:34.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4300000000000003e-07 2024-09-14T23:56:34.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:34.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.580500+0000", 2024-09-14T23:56:34.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.673e-06 2024-09-14T23:56:34.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:34.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.580512+0000", 2024-09-14T23:56:34.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2119999999999999e-05 2024-09-14T23:56:34.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:34.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.580523+0000", 2024-09-14T23:56:34.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.058e-05 2024-09-14T23:56:34.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:34.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.580907+0000", 2024-09-14T23:56:34.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038464499999999997 2024-09-14T23:56:34.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:34.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.580928+0000", 2024-09-14T23:56:34.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0839000000000002e-05 2024-09-14T23:56:34.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:34.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.581441+0000", 2024-09-14T23:56:34.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00051274700000000003 2024-09-14T23:56:34.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:34.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.581475+0000", 2024-09-14T23:56:34.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4442e-05 2024-09-14T23:56:34.844 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:34.844 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:34.844 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:34.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.845 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4209.0:140 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'98, mlcod=16'93)", 2024-09-14T23:56:34.845 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:16.008964+0000", 2024-09-14T23:56:34.845 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.751090543, 2024-09-14T23:56:34.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00109066, 2024-09-14T23:56:34.845 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:34.845 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:34.845 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:34.845 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:34.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:34.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:16.008964+0000", 2024-09-14T23:56:34.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:34.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:34.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:16.008964+0000", 2024-09-14T23:56:34.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:34.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:34.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:16.008965+0000", 2024-09-14T23:56:34.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9500000000000003e-07 2024-09-14T23:56:34.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:34.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:16.009117+0000", 2024-09-14T23:56:34.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015197400000000001 2024-09-14T23:56:34.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:34.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:16.009126+0000", 2024-09-14T23:56:34.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7259999999999997e-06 2024-09-14T23:56:34.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:34.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:16.009133+0000", 2024-09-14T23:56:34.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5699999999999998e-06 2024-09-14T23:56:34.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:34.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:16.009249+0000", 2024-09-14T23:56:34.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000115565 2024-09-14T23:56:34.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:34.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:16.009263+0000", 2024-09-14T23:56:34.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4447000000000001e-05 2024-09-14T23:56:34.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:34.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:16.010027+0000", 2024-09-14T23:56:34.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00076407500000000002 2024-09-14T23:56:34.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:34.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:16.010055+0000", 2024-09-14T23:56:34.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7807999999999999e-05 2024-09-14T23:56:34.849 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:34.849 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:34.850 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:34.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.850 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4209.0:160 1.0 e17/15 1:30306672:devicehealth::main.db.0000000000000000:head v 17'114, mlcod=17'112)", 2024-09-14T23:56:34.850 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:16.613316+0000", 2024-09-14T23:56:34.850 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.146738701, 2024-09-14T23:56:34.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010047190000000001, 2024-09-14T23:56:34.850 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:34.850 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:34.850 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:34.851 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:34.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:34.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:16.613316+0000", 2024-09-14T23:56:34.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:34.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:34.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:16.613316+0000", 2024-09-14T23:56:34.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:34.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:34.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:16.613317+0000", 2024-09-14T23:56:34.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6000000000000004e-07 2024-09-14T23:56:34.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:34.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:16.613367+0000", 2024-09-14T23:56:34.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0430000000000003e-05 2024-09-14T23:56:34.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:34.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:16.613377+0000", 2024-09-14T23:56:34.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6690000000000001e-06 2024-09-14T23:56:34.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:34.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:16.613381+0000", 2024-09-14T23:56:34.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5650000000000003e-06 2024-09-14T23:56:34.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:34.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:16.613439+0000", 2024-09-14T23:56:34.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8173e-05 2024-09-14T23:56:34.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:34.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:16.613451+0000", 2024-09-14T23:56:34.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1977999999999999e-05 2024-09-14T23:56:34.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:34.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:16.614283+0000", 2024-09-14T23:56:34.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000832003 2024-09-14T23:56:34.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:34.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:16.614321+0000", 2024-09-14T23:56:34.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7341e-05 2024-09-14T23:56:34.854 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:34.854 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:34.854 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:34.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.855 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:11 3.1f 3:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T23:56:34.855 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:34.410107+0000", 2024-09-14T23:56:34.855 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.34994774099999998, 2024-09-14T23:56:34.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00151684, 2024-09-14T23:56:34.855 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:34.855 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:34.855 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:34.855 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:34.855 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:34.855 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:34.855 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 11 2024-09-14T23:56:34.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.856 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:34.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:34.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.410107+0000", 2024-09-14T23:56:34.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:34.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:34.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.410107+0000", 2024-09-14T23:56:34.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:34.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:34.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.410109+0000", 2024-09-14T23:56:34.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.973e-06 2024-09-14T23:56:34.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:34.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.410128+0000", 2024-09-14T23:56:34.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8865999999999999e-05 2024-09-14T23:56:34.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:34.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.410147+0000", 2024-09-14T23:56:34.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9572000000000002e-05 2024-09-14T23:56:34.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:34.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.410160+0000", 2024-09-14T23:56:34.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2652e-05 2024-09-14T23:56:34.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:34.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.410231+0000", 2024-09-14T23:56:34.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1093999999999994e-05 2024-09-14T23:56:34.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:34.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.410357+0000", 2024-09-14T23:56:34.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012558 2024-09-14T23:56:34.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-14T23:56:34.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.410515+0000", 2024-09-14T23:56:34.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000158345 2024-09-14T23:56:34.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:34.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.411377+0000", 2024-09-14T23:56:34.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00086149100000000004 2024-09-14T23:56:34.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:56:34.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.411542+0000", 2024-09-14T23:56:34.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000165186 2024-09-14T23:56:34.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:34.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.411584+0000", 2024-09-14T23:56:34.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2358999999999999e-05 2024-09-14T23:56:34.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:34.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:34.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:34.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.411624+0000", 2024-09-14T23:56:34.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9721999999999997e-05 2024-09-14T23:56:34.860 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:34.860 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:34.860 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:34.860 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:34.861 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:34.861 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:56:34.861 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:56:34.760+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:56:34.862 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T23:56:34.864 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:56:34.862+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:56:34.964 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:56:34.963+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:56:35.012 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:56:35.012 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:56:35.012 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:56:35.012 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:56:35.021 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T23:56:35.104 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:56:35.103+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:56:35.167 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:56:35.165+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:56:35.169 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:56:35.169 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:56:35.169 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:56:35.169 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:56:35.169 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:56:35.178 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T23:56:35.267 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:56:35.266+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:56:35.327 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:56:35.327 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:56:35.327 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:56:35.327 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:56:35.328 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:35.328 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4284.0:5 2.2 e20/18 2:5cea7035:::rbd_info:head v 20'1, mlcod=0'0)", 2024-09-14T23:56:35.328 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:23.532954+0000", 2024-09-14T23:56:35.328 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 11.793306482, 2024-09-14T23:56:35.328 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001093295, 2024-09-14T23:56:35.328 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:35.328 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:35.328 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:35.328 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:35.328 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:35.328 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:35.328 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:23.532954+0000", 2024-09-14T23:56:35.328 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:35.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:35.329 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:35.329 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:35.329 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:23.532954+0000", 2024-09-14T23:56:35.329 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:35.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:35.329 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:35.329 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:35.329 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:23.532954+0000", 2024-09-14T23:56:35.329 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9999999999999999e-07 2024-09-14T23:56:35.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:35.329 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:35.329 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:35.329 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:23.532960+0000", 2024-09-14T23:56:35.330 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0519999999999998e-06 2024-09-14T23:56:35.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:35.330 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:35.330 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:35.330 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:23.532979+0000", 2024-09-14T23:56:35.330 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8485999999999999e-05 2024-09-14T23:56:35.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:35.330 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:35.330 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:35.330 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:23.532992+0000", 2024-09-14T23:56:35.330 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3023000000000001e-05 2024-09-14T23:56:35.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:35.330 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:35.331 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:35.331 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:23.533066+0000", 2024-09-14T23:56:35.331 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3935000000000003e-05 2024-09-14T23:56:35.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:35.331 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:35.331 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:35.331 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:23.533093+0000", 2024-09-14T23:56:35.331 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7288999999999999e-05 2024-09-14T23:56:35.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:35.331 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:35.331 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:35.331 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:23.534009+0000", 2024-09-14T23:56:35.331 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000916404 2024-09-14T23:56:35.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:35.332 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:35.332 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:35.332 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:23.534047+0000", 2024-09-14T23:56:35.332 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7905999999999999e-05 2024-09-14T23:56:35.332 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:35.332 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:35.332 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:35.332 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:35.332 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:35.332 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4284.0:7 2.2 e21/18 2:5cea7035:::rbd_info:head v 21'2, mlcod=0'0)", 2024-09-14T23:56:35.332 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:24.537466+0000", 2024-09-14T23:56:35.332 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.788794439, 2024-09-14T23:56:35.332 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00083814299999999996, 2024-09-14T23:56:35.332 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:35.333 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:35.333 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:35.333 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:35.333 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:35.333 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:35.333 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:24.537466+0000", 2024-09-14T23:56:35.333 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:35.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:35.333 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:35.333 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:35.333 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:24.537466+0000", 2024-09-14T23:56:35.333 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:35.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:35.334 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:35.334 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:35.334 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:24.537466+0000", 2024-09-14T23:56:35.334 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.28e-07 2024-09-14T23:56:35.334 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:35.334 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:35.334 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:35.334 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:24.537471+0000", 2024-09-14T23:56:35.334 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5519999999999998e-06 2024-09-14T23:56:35.334 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:35.334 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:35.334 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:35.334 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:24.537487+0000", 2024-09-14T23:56:35.334 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6602000000000001e-05 2024-09-14T23:56:35.334 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:35.335 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:35.335 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:35.335 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:24.537499+0000", 2024-09-14T23:56:35.335 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1250000000000001e-05 2024-09-14T23:56:35.335 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:35.335 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:35.335 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:35.335 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:24.537569+0000", 2024-09-14T23:56:35.335 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9766999999999998e-05 2024-09-14T23:56:35.335 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:35.335 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:35.335 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:35.335 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:24.537588+0000", 2024-09-14T23:56:35.336 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9874999999999999e-05 2024-09-14T23:56:35.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:35.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:35.336 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:35.336 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:24.538234+0000", 2024-09-14T23:56:35.336 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00064588600000000003 2024-09-14T23:56:35.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:35.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:35.336 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:35.336 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:24.538304+0000", 2024-09-14T23:56:35.336 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9982999999999997e-05 2024-09-14T23:56:35.336 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:35.336 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:35.336 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:35.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:35.337 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:35.337 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:7 3.1a e25/23 3:5cea7035:::rbd_info:head v 25'1, mlcod=0'0)", 2024-09-14T23:56:35.337 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:33.396939+0000", 2024-09-14T23:56:35.337 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 1.9293219189999999, 2024-09-14T23:56:35.337 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00063904399999999996, 2024-09-14T23:56:35.337 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:35.337 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:35.337 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:35.337 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:35.337 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:35.337 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:35.337 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:33.396939+0000", 2024-09-14T23:56:35.338 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:35.338 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:35.338 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:35.338 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:35.338 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:33.396939+0000", 2024-09-14T23:56:35.338 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:35.338 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:35.338 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:35.338 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:35.338 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:33.396939+0000", 2024-09-14T23:56:35.338 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.73e-07 2024-09-14T23:56:35.338 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:35.338 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:35.338 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:35.339 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:33.396943+0000", 2024-09-14T23:56:35.339 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6280000000000004e-06 2024-09-14T23:56:35.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:35.339 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:35.339 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:35.339 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:33.396955+0000", 2024-09-14T23:56:35.339 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1800000000000001e-05 2024-09-14T23:56:35.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:35.339 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:35.339 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:35.339 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:33.396962+0000", 2024-09-14T23:56:35.339 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7889999999999997e-06 2024-09-14T23:56:35.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:35.339 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:35.339 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:35.340 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:33.397002+0000", 2024-09-14T23:56:35.340 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0479999999999999e-05 2024-09-14T23:56:35.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:35.340 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:35.340 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:35.340 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:33.397018+0000", 2024-09-14T23:56:35.340 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.543e-05 2024-09-14T23:56:35.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:35.340 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:35.340 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:35.340 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:33.397543+0000", 2024-09-14T23:56:35.340 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00052527499999999996 2024-09-14T23:56:35.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:35.340 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:35.341 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:35.341 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:33.397578+0000", 2024-09-14T23:56:35.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4468999999999998e-05 2024-09-14T23:56:35.341 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:35.341 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:35.341 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:35.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:35.341 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:35.341 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:9 3.1a e26/23 3:5cea7035:::rbd_info:head v 26'2, mlcod=0'0)", 2024-09-14T23:56:35.341 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:34.404833+0000", 2024-09-14T23:56:35.341 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.92142769700000005, 2024-09-14T23:56:35.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011562289999999999, 2024-09-14T23:56:35.341 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:35.342 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:35.342 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:35.342 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:35.342 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:35.342 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:35.342 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.404833+0000", 2024-09-14T23:56:35.342 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:35.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:35.342 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:35.342 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:35.342 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.404833+0000", 2024-09-14T23:56:35.342 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:35.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:35.342 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:35.343 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:35.343 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.404833+0000", 2024-09-14T23:56:35.343 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0900000000000001e-07 2024-09-14T23:56:35.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:35.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:35.343 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:35.343 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.404842+0000", 2024-09-14T23:56:35.343 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2230000000000007e-06 2024-09-14T23:56:35.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:35.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:35.343 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:35.343 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.404852+0000", 2024-09-14T23:56:35.343 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9769999999999995e-06 2024-09-14T23:56:35.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:35.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:35.343 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:35.344 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.404865+0000", 2024-09-14T23:56:35.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3519e-05 2024-09-14T23:56:35.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:35.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:35.344 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:35.344 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.404962+0000", 2024-09-14T23:56:35.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7301999999999998e-05 2024-09-14T23:56:35.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:35.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:35.344 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:35.344 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.404985+0000", 2024-09-14T23:56:35.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2646e-05 2024-09-14T23:56:35.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:35.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:35.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:35.345 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.405953+0000", 2024-09-14T23:56:35.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00096790799999999996 2024-09-14T23:56:35.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:35.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:35.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:35.345 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.405989+0000", 2024-09-14T23:56:35.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6044999999999998e-05 2024-09-14T23:56:35.345 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:35.345 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:35.345 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:35.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:35.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:35.346 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:13 3.8 e26/23 3:1601aefa:::rbd_object_map.1118d8884d62:head v 26'1, mlcod=0'0)", 2024-09-14T23:56:35.346 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:34.417229+0000", 2024-09-14T23:56:35.346 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.90903178799999995, 2024-09-14T23:56:35.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00087394399999999998, 2024-09-14T23:56:35.346 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:35.346 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:35.346 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:35.346 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:35.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:35.346 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:35.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.417229+0000", 2024-09-14T23:56:35.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:35.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:35.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:35.347 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:35.347 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.417229+0000", 2024-09-14T23:56:35.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:35.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:35.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:35.347 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:35.347 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.417229+0000", 2024-09-14T23:56:35.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7700000000000001e-07 2024-09-14T23:56:35.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:35.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:35.347 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:35.347 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.417235+0000", 2024-09-14T23:56:35.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9680000000000003e-06 2024-09-14T23:56:35.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:35.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:35.348 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:35.348 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.417251+0000", 2024-09-14T23:56:35.348 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6109000000000001e-05 2024-09-14T23:56:35.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:35.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:35.348 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:35.348 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.417262+0000", 2024-09-14T23:56:35.348 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.115e-05 2024-09-14T23:56:35.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:35.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:35.348 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:35.348 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.417322+0000", 2024-09-14T23:56:35.349 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9517e-05 2024-09-14T23:56:35.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:35.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:35.349 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:35.349 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.417345+0000", 2024-09-14T23:56:35.349 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3473e-05 2024-09-14T23:56:35.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:35.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:35.349 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:35.349 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.418050+0000", 2024-09-14T23:56:35.349 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00070452199999999996 2024-09-14T23:56:35.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:35.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:35.349 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:35.350 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.418103+0000", 2024-09-14T23:56:35.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2927999999999999e-05 2024-09-14T23:56:35.350 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:35.350 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:35.350 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:35.350 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:35.350 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:35.350 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:56:35.351 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T23:56:35.353 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T23:56:35.393 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:56:35.391+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:56:35.469 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:56:35.468+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:56:35.504 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:56:35.504 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:56:35.504 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:56:35.504 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:56:35.513 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T23:56:35.520 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:56:35.520 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:56:35.537 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:56:35.538 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:56:35.555 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:56:35.555 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:56:35.570 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:56:35.568+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:56:35.572 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:56:35.572 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:56:35.589 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:56:35.589 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:56:35.607 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:56:35.607 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:56:35.624 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:35.624 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:35.624 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:56:35.624 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:56:35.633 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T23:56:35.643 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:56:35.643 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:56:35.695 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:56:35.694+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:56:35.750 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:35.750 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:35.759 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T23:56:35.801 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:56:35.799+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:56:35.872 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:56:35.871+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:56:35.908 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:35.908 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:35.917 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T23:56:36.001 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:56:36.000+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:56:36.066 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:36.066 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:36.073 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:56:36.071+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:56:36.078 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T23:56:36.130 INFO:teuthology.orchestra.run.smithi042.stdout:skipping zero length punch hole 2024-09-14T23:56:36.151 INFO:teuthology.orchestra.run.smithi042.stdout:3 writesame 0xd42fa50 thru 0xd43567e data_size 0x1(0x5c2f bytes) 2024-09-14T23:56:36.151 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:36.151 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:36.151 INFO:teuthology.orchestra.run.smithi042.stdout:6 writesame 0xcb5e01d thru 0xcb64330 data_size 0x22(0x6314 bytes) 2024-09-14T23:56:36.154 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:36.155 INFO:teuthology.orchestra.run.smithi042.stdout:9 read 0xc822c6d thru 0xc82b52e (0x88c2 bytes) 2024-09-14T23:56:36.155 INFO:teuthology.orchestra.run.smithi042.stdout:10 writesame 0xaaa24a7 thru 0xaaa3722 data_size 0xe(0x127c bytes) 2024-09-14T23:56:36.158 INFO:teuthology.orchestra.run.smithi042.stdout:12 read 0x4f61dc7 thru 0x4f6ffa0 (0xe1da bytes) 2024-09-14T23:56:36.158 INFO:teuthology.orchestra.run.smithi042.stdout:13 write 0xc6cddf6 thru 0xc6dadf1 (0xcffc bytes) 2024-09-14T23:56:36.162 INFO:teuthology.orchestra.run.smithi042.stdout:14 read 0x3f479fa thru 0x3f53dca (0xc3d1 bytes) 2024-09-14T23:56:36.162 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:36.162 INFO:teuthology.orchestra.run.smithi042.stdout:16 write 0x1d31120 thru 0x1d371e4 (0x60c5 bytes) 2024-09-14T23:56:36.165 INFO:teuthology.orchestra.run.smithi042.stdout:17 write 0x3ad0a4b thru 0x3ad74d9 (0x6a8f bytes) 2024-09-14T23:56:36.168 INFO:teuthology.orchestra.run.smithi042.stdout:19 writesame 0x9d13590 thru 0x9d2237f data_size 0xbc(0xedf0 bytes) 2024-09-14T23:56:36.172 INFO:teuthology.orchestra.run.smithi042.stdout:20 write 0x2710864 thru 0x271edaf (0xe54c bytes) 2024-09-14T23:56:36.174 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:56:36.172+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:56:36.175 INFO:teuthology.orchestra.run.smithi042.stdout:22 write 0x7736a9b thru 0x77382a8 (0x180e bytes) 2024-09-14T23:56:36.178 INFO:teuthology.orchestra.run.smithi042.stdout:23 punch from 0x22cbbf8 to 0x22d6e9f, (0xb2a7 bytes) 2024-09-14T23:56:36.179 INFO:teuthology.orchestra.run.smithi042.stdout:26 write 0x6c03813 thru 0x6c13127 (0xf915 bytes) 2024-09-14T23:56:36.182 INFO:teuthology.orchestra.run.smithi042.stdout:27 read 0x3a2f1ed thru 0x3a3db97 (0xe9ab bytes) 2024-09-14T23:56:36.182 INFO:teuthology.orchestra.run.smithi042.stdout:28 write 0x80c1c11 thru 0x80c33ed (0x17dd bytes) 2024-09-14T23:56:36.188 INFO:teuthology.orchestra.run.smithi042.stdout:29 read 0x18295a8 thru 0x183399c (0xa3f5 bytes) 2024-09-14T23:56:36.188 INFO:teuthology.orchestra.run.smithi042.stdout:31 write 0xa944b0a thru 0xa948ab5 (0x3fac bytes) 2024-09-14T23:56:36.188 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:36.188 INFO:teuthology.orchestra.run.smithi042.stdout:38 punch from 0x45442d3 to 0x4547925, (0x3652 bytes) 2024-09-14T23:56:36.188 INFO:teuthology.orchestra.run.smithi042.stdout:40 write 0x6a7d8c8 thru 0x6a8048c (0x2bc5 bytes) 2024-09-14T23:56:36.190 INFO:teuthology.orchestra.run.smithi042.stdout:41 clone 1 order 18 su 4096 sc 15 2024-09-14T23:56:36.236 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:36.236 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:36.244 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T23:56:36.285 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:56:36.284+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:56:36.361 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:36.361 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:36.369 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T23:56:36.414 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:56:36.411+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:56:36.419 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T23:56:36.476 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:56:36.474+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:56:36.477 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:36.477 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:36.486 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T23:56:36.577 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:56:36.575+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:56:36.634 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:36.634 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:36.642 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T23:56:36.683 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:56:36.681+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:56:36.779 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:56:36.778+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:56:36.790 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:36.790 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:36.798 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T23:56:36.880 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:56:36.879+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:56:36.905 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:36.906 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:36.914 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T23:56:36.971 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is unset 2024-09-14T23:56:36.980 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:56:36.978+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:56:37.025 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:37.025 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:37.034 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T23:56:37.081 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:56:37.079+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:56:37.087 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T23:56:37.087 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T23:56:37.087 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-14T23:56:37.087 INFO:tasks.thrashosds.thrasher:Setting osd 5 primary_affinity to 0.000000 2024-09-14T23:56:37.088 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 0 2024-09-14T23:56:37.150 INFO:teuthology.orchestra.run.smithi042.stdout:43 read 0xb4dc18e thru 0xb4e7e23 (0xbc96 bytes) 2024-09-14T23:56:37.151 INFO:teuthology.orchestra.run.smithi042.stdout:44 write 0x3bdeb98 thru 0x3be6c82 (0x80eb bytes) 2024-09-14T23:56:37.184 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:56:37.179+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:56:37.185 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:37.185 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:37.192 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T23:56:37.259 INFO:teuthology.orchestra.run.smithi042.stdout:45 read 0x4c7528 thru 0x4cc5f9 (0x50d2 bytes) 2024-09-14T23:56:37.259 INFO:teuthology.orchestra.run.smithi042.stdout:46 write 0x46a3ac4 thru 0x46adc80 (0xa1bd bytes) 2024-09-14T23:56:37.270 INFO:teuthology.orchestra.run.smithi042.stdout:48 writesame 0x2129ee thru 0x218900 data_size 0x3d(0x5f13 bytes) 2024-09-14T23:56:37.279 INFO:teuthology.orchestra.run.smithi042.stdout:49 write 0x6dbb847 thru 0x6dcaee8 (0xf6a2 bytes) 2024-09-14T23:56:37.285 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:56:37.283+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:56:37.354 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:37.355 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:37.364 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T23:56:37.386 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:56:37.385+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:56:37.435 INFO:teuthology.orchestra.run.smithi042.stdout:50 read 0xd336121 thru 0xd33f432 (0x9312 bytes) 2024-09-14T23:56:37.435 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:37.435 INFO:teuthology.orchestra.run.smithi042.stdout:54 write 0x9c72fa8 thru 0x9c73bd7 (0xc30 bytes) 2024-09-14T23:56:37.446 INFO:teuthology.orchestra.run.smithi042.stdout:57 punch from 0x3e6dbe9 to 0x3e6fddf, (0x21f6 bytes) 2024-09-14T23:56:37.446 INFO:teuthology.orchestra.run.smithi042.stdout:58 read 0x609c042 thru 0x609c754 (0x713 bytes) 2024-09-14T23:56:37.446 INFO:teuthology.orchestra.run.smithi042.stdout:59 write 0x5e05f57 thru 0x5e0aa07 (0x4ab1 bytes) 2024-09-14T23:56:37.458 INFO:teuthology.orchestra.run.smithi042.stdout:60 writesame 0xe253461 thru 0xe254324 data_size 0xa(0xec4 bytes) 2024-09-14T23:56:37.458 INFO:teuthology.orchestra.run.smithi042.stdout:63 read 0xdf271ee thru 0xdf3317c (0xbf8f bytes) 2024-09-14T23:56:37.458 INFO:teuthology.orchestra.run.smithi042.stdout:truncating to largest ever: 0x78d722a 2024-09-14T23:56:37.469 INFO:teuthology.orchestra.run.smithi042.stdout:64 trunc from 0xe254325 to 0x78d722a 2024-09-14T23:56:37.488 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:56:37.485+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:56:37.488 INFO:teuthology.orchestra.run.smithi042.stdout:65 write 0x1e68e5f thru 0x1e7426e (0xb410 bytes) 2024-09-14T23:56:37.538 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:37.538 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:37.550 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T23:56:37.551 INFO:teuthology.orchestra.run.smithi042.stdout:66 read 0x21c393a thru 0x21d146d (0xdb34 bytes) 2024-09-14T23:56:37.551 INFO:teuthology.orchestra.run.smithi042.stdout:67 write 0xec7c1 thru 0xf71a1 (0xa9e1 bytes) 2024-09-14T23:56:37.559 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:37.559 INFO:teuthology.orchestra.run.smithi042.stdout:truncating to largest ever: 0xc3f02b1 2024-09-14T23:56:37.564 INFO:teuthology.orchestra.run.smithi042.stdout:69 trunc from 0x78d722a to 0xc3f02b1 2024-09-14T23:56:37.569 INFO:teuthology.orchestra.run.smithi042.stdout:70 write 0xa6c2163 thru 0xa6c5fb4 (0x3e52 bytes) 2024-09-14T23:56:37.573 INFO:teuthology.orchestra.run.smithi042.stdout:72 write 0x8d9d36a thru 0x8da0467 (0x30fe bytes) 2024-09-14T23:56:37.578 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:37.580 INFO:teuthology.orchestra.run.smithi042.stdout:74 trunc from 0xc3f02b1 to 0xa34be18 2024-09-14T23:56:37.591 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:56:37.589+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:56:37.593 INFO:teuthology.orchestra.run.smithi042.stdout:75 trunc from 0xa34be18 to 0x6bc478e 2024-09-14T23:56:37.655 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T23:56:37.690 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:56:37.687+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:56:37.705 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:37.705 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:37.715 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T23:56:37.745 INFO:teuthology.orchestra.run.smithi042.stdout:76 write 0x5e9505c thru 0x5e95883 (0x828 bytes) 2024-09-14T23:56:37.751 INFO:teuthology.orchestra.run.smithi042.stdout:77 trunc from 0x6bc478e to 0x9db379d 2024-09-14T23:56:37.756 INFO:teuthology.orchestra.run.smithi042.stdout:78 writesame 0x9bb61ae thru 0x9bb8e39 data_size 0x2(0x2c8c bytes) 2024-09-14T23:56:37.761 INFO:teuthology.orchestra.run.smithi042.stdout:80 punch from 0xab4889 to 0xabe5cf, (0x9d46 bytes) 2024-09-14T23:56:37.767 INFO:teuthology.orchestra.run.smithi042.stdout:81 writesame 0xafb35d8 thru 0xafb6166 data_size 0x15(0x2b8f bytes) 2024-09-14T23:56:37.768 INFO:teuthology.orchestra.run.smithi042.stdout:83 punch from 0x4e679fd to 0x4e6e1fd, (0x6800 bytes) 2024-09-14T23:56:37.773 INFO:teuthology.orchestra.run.smithi042.stdout:85 trunc from 0xafb6167 to 0x5b71763 2024-09-14T23:56:37.799 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:56:37.797+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:56:37.806 INFO:teuthology.orchestra.run.smithi042.stdout:86 clone 2 order 23 su 131072 sc 6 2024-09-14T23:56:37.814 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:56:37.814 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:56:37.831 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:56:37.831 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:56:37.848 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:56:37.848 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:56:37.864 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:56:37.864 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:56:37.869 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:37.869 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:37.878 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T23:56:37.880 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:56:37.880 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:56:37.891 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:56:37.890+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:56:37.897 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:56:37.897 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:56:37.914 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:56:37.914 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:56:37.930 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:56:37.931 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:56:37.975 INFO:teuthology.orchestra.run.smithi042.stderr:set osd.5 primary-affinity to 0 (802) 2024-09-14T23:56:37.992 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:56:37.989+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:56:38.030 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:56:38.030 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:56:38.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:56:38.031 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:56:38.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.031 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4209.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-14T23:56:38.031 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:15.357728+0000", 2024-09-14T23:56:38.031 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.669808167999999, 2024-09-14T23:56:38.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0043495850000000004, 2024-09-14T23:56:38.031 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:38.031 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:38.031 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:38.031 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:38.031 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4209", 2024-09-14T23:56:38.031 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/1617902543", 2024-09-14T23:56:38.031 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 37 2024-09-14T23:56:38.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.032 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:38.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:38.032 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.357728+0000", 2024-09-14T23:56:38.032 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:38.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:38.032 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.357728+0000", 2024-09-14T23:56:38.032 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:38.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.033 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:38.033 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.357728+0000", 2024-09-14T23:56:38.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1e-07 2024-09-14T23:56:38.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.033 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:38.033 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.357730+0000", 2024-09-14T23:56:38.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1720000000000001e-06 2024-09-14T23:56:38.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.033 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:38.033 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.357733+0000", 2024-09-14T23:56:38.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5300000000000001e-06 2024-09-14T23:56:38.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.034 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:38.034 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.357736+0000", 2024-09-14T23:56:38.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5900000000000002e-06 2024-09-14T23:56:38.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.034 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:38.034 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.361176+0000", 2024-09-14T23:56:38.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0034402899999999999 2024-09-14T23:56:38.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:38.035 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.361228+0000", 2024-09-14T23:56:38.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1561999999999999e-05 2024-09-14T23:56:38.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:56:38.035 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.361326+0000", 2024-09-14T23:56:38.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8369999999999995e-05 2024-09-14T23:56:38.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:56:38.035 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.361744+0000", 2024-09-14T23:56:38.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000417279 2024-09-14T23:56:38.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:38.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.362030+0000", 2024-09-14T23:56:38.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000285965 2024-09-14T23:56:38.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:38.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.362058+0000", 2024-09-14T23:56:38.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8479999999999998e-05 2024-09-14T23:56:38.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:38.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.362077+0000", 2024-09-14T23:56:38.037 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9137e-05 2024-09-14T23:56:38.037 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:38.037 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:38.037 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:38.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.037 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.037 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4209.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-14T23:56:38.037 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:15.397958+0000", 2024-09-14T23:56:38.037 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.629578006999999, 2024-09-14T23:56:38.037 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0043849279999999997, 2024-09-14T23:56:38.037 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:38.037 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:38.037 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:38.038 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:38.038 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4209", 2024-09-14T23:56:38.038 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/1617902543", 2024-09-14T23:56:38.038 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 87 2024-09-14T23:56:38.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.038 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:38.038 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.038 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:38.038 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.397958+0000", 2024-09-14T23:56:38.038 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:38.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.038 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.038 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:38.038 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.397958+0000", 2024-09-14T23:56:38.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:38.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:38.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.397958+0000", 2024-09-14T23:56:38.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7800000000000002e-07 2024-09-14T23:56:38.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:38.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.398008+0000", 2024-09-14T23:56:38.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0309000000000002e-05 2024-09-14T23:56:38.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:38.040 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.398014+0000", 2024-09-14T23:56:38.040 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.823e-06 2024-09-14T23:56:38.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.040 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:38.040 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.398018+0000", 2024-09-14T23:56:38.040 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.206e-06 2024-09-14T23:56:38.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.040 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:38.040 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.400798+0000", 2024-09-14T23:56:38.040 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0027797329999999999 2024-09-14T23:56:38.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.041 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:38.041 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.400877+0000", 2024-09-14T23:56:38.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9295999999999994e-05 2024-09-14T23:56:38.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.041 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.041 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:56:38.041 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.401011+0000", 2024-09-14T23:56:38.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013304699999999999 2024-09-14T23:56:38.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.041 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.041 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:56:38.041 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.401940+0000", 2024-09-14T23:56:38.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00092952400000000004 2024-09-14T23:56:38.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.042 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.042 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:38.042 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.402244+0000", 2024-09-14T23:56:38.042 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00030433700000000001 2024-09-14T23:56:38.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.042 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.042 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:38.042 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.402300+0000", 2024-09-14T23:56:38.042 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6051999999999998e-05 2024-09-14T23:56:38.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.042 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.042 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:38.042 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.402343+0000", 2024-09-14T23:56:38.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2222999999999999e-05 2024-09-14T23:56:38.043 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:38.043 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:38.043 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:38.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.043 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4209.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-14T23:56:38.043 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:15.398027+0000", 2024-09-14T23:56:38.043 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.629508868999999, 2024-09-14T23:56:38.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0045217030000000002, 2024-09-14T23:56:38.043 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:38.043 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:38.043 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:38.043 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:38.044 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4209", 2024-09-14T23:56:38.044 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/1617902543", 2024-09-14T23:56:38.044 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 88 2024-09-14T23:56:38.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.044 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:38.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.044 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:38.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.398027+0000", 2024-09-14T23:56:38.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:38.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.044 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:38.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.398027+0000", 2024-09-14T23:56:38.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:38.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.045 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.045 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:38.045 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.398027+0000", 2024-09-14T23:56:38.045 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9000000000000002e-07 2024-09-14T23:56:38.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.045 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.045 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:38.045 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.398029+0000", 2024-09-14T23:56:38.045 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1600000000000001e-06 2024-09-14T23:56:38.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.045 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.045 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:38.046 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.398049+0000", 2024-09-14T23:56:38.046 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9745000000000002e-05 2024-09-14T23:56:38.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.046 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.046 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:38.046 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.398061+0000", 2024-09-14T23:56:38.046 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1832e-05 2024-09-14T23:56:38.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.046 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.046 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:38.046 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.401556+0000", 2024-09-14T23:56:38.046 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0034946280000000001 2024-09-14T23:56:38.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.047 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:38.047 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.401600+0000", 2024-09-14T23:56:38.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4512999999999999e-05 2024-09-14T23:56:38.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.047 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:56:38.047 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.401690+0000", 2024-09-14T23:56:38.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9420999999999996e-05 2024-09-14T23:56:38.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.048 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.048 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:56:38.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.402355+0000", 2024-09-14T23:56:38.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00066586200000000003 2024-09-14T23:56:38.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.048 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.048 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:38.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.402499+0000", 2024-09-14T23:56:38.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000143098 2024-09-14T23:56:38.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.048 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.048 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:38.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.402527+0000", 2024-09-14T23:56:38.049 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8433e-05 2024-09-14T23:56:38.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.049 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.049 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:38.049 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.402549+0000", 2024-09-14T23:56:38.049 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1620999999999998e-05 2024-09-14T23:56:38.049 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:38.049 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:38.049 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:38.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.049 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.049 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4209.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-14T23:56:38.049 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:15.572684+0000", 2024-09-14T23:56:38.050 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.454851850000001, 2024-09-14T23:56:38.050 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.004979804, 2024-09-14T23:56:38.050 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:38.050 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:38.050 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:38.050 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:38.050 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4209", 2024-09-14T23:56:38.050 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/1617902543", 2024-09-14T23:56:38.050 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 109 2024-09-14T23:56:38.050 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.050 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:38.050 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.050 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:38.051 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.572684+0000", 2024-09-14T23:56:38.051 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:38.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.051 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.051 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:38.051 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.572684+0000", 2024-09-14T23:56:38.051 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:38.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.051 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.051 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:38.051 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.572684+0000", 2024-09-14T23:56:38.051 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8599999999999998e-07 2024-09-14T23:56:38.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.051 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.052 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:38.052 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.572765+0000", 2024-09-14T23:56:38.052 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0195e-05 2024-09-14T23:56:38.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.052 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.052 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:38.052 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.572778+0000", 2024-09-14T23:56:38.052 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3209e-05 2024-09-14T23:56:38.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.052 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.052 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:38.052 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.572793+0000", 2024-09-14T23:56:38.052 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4884e-05 2024-09-14T23:56:38.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.053 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:38.053 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.575300+0000", 2024-09-14T23:56:38.053 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0025069390000000001 2024-09-14T23:56:38.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.053 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:38.053 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.575346+0000", 2024-09-14T23:56:38.053 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6718000000000002e-05 2024-09-14T23:56:38.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.053 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:56:38.053 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.575423+0000", 2024-09-14T23:56:38.054 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7071999999999997e-05 2024-09-14T23:56:38.054 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.054 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.054 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:38.054 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.577057+0000", 2024-09-14T23:56:38.054 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016335259999999999 2024-09-14T23:56:38.054 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.054 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.054 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:56:38.054 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.577584+0000", 2024-09-14T23:56:38.054 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00052741500000000005 2024-09-14T23:56:38.054 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.054 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.054 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:38.055 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.577639+0000", 2024-09-14T23:56:38.055 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4623000000000002e-05 2024-09-14T23:56:38.055 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.055 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.055 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:38.055 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.577664+0000", 2024-09-14T23:56:38.055 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4637000000000001e-05 2024-09-14T23:56:38.055 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:38.055 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:38.055 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:38.055 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.055 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.055 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4209.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-14T23:56:38.055 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:15.572821+0000", 2024-09-14T23:56:38.056 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.454715037, 2024-09-14T23:56:38.056 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0053598150000000004, 2024-09-14T23:56:38.056 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:38.056 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:38.056 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:38.056 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:38.056 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4209", 2024-09-14T23:56:38.056 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/1617902543", 2024-09-14T23:56:38.056 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 110 2024-09-14T23:56:38.056 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.056 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:38.056 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.056 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:38.056 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.572821+0000", 2024-09-14T23:56:38.057 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:38.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.057 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.057 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:38.057 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.572821+0000", 2024-09-14T23:56:38.057 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:38.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.057 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.057 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:38.057 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.572821+0000", 2024-09-14T23:56:38.057 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5000000000000002e-07 2024-09-14T23:56:38.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.057 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.057 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:38.058 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.572826+0000", 2024-09-14T23:56:38.058 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9570000000000001e-06 2024-09-14T23:56:38.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.058 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.058 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:38.058 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.572837+0000", 2024-09-14T23:56:38.058 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0766e-05 2024-09-14T23:56:38.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.058 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.058 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:38.058 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.572844+0000", 2024-09-14T23:56:38.058 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4710000000000004e-06 2024-09-14T23:56:38.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.058 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.059 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:38.059 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.576100+0000", 2024-09-14T23:56:38.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003256292 2024-09-14T23:56:38.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.059 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:38.059 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.576190+0000", 2024-09-14T23:56:38.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0000000000000006e-05 2024-09-14T23:56:38.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.059 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:56:38.059 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.576330+0000", 2024-09-14T23:56:38.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014050400000000001 2024-09-14T23:56:38.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.060 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.060 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:56:38.060 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.577683+0000", 2024-09-14T23:56:38.060 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0013524030000000001 2024-09-14T23:56:38.060 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.060 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.060 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:38.060 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.578126+0000", 2024-09-14T23:56:38.060 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00044277599999999998 2024-09-14T23:56:38.060 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.060 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.060 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:38.060 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.578159+0000", 2024-09-14T23:56:38.061 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3254999999999999e-05 2024-09-14T23:56:38.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.061 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.061 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:38.061 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.578181+0000", 2024-09-14T23:56:38.061 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1741000000000001e-05 2024-09-14T23:56:38.061 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:38.061 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:38.061 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:38.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.061 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.061 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4209.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-14T23:56:38.061 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:15.572859+0000", 2024-09-14T23:56:38.061 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.454676785, 2024-09-14T23:56:38.062 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0054419050000000004, 2024-09-14T23:56:38.062 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:38.062 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:38.062 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:38.062 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:38.062 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4209", 2024-09-14T23:56:38.062 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/1617902543", 2024-09-14T23:56:38.062 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 111 2024-09-14T23:56:38.062 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.062 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:38.062 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.062 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:38.062 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.572859+0000", 2024-09-14T23:56:38.062 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:38.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:38.063 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.572859+0000", 2024-09-14T23:56:38.063 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:38.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:38.063 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.572860+0000", 2024-09-14T23:56:38.063 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9200000000000001e-07 2024-09-14T23:56:38.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:38.063 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.572864+0000", 2024-09-14T23:56:38.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.408e-06 2024-09-14T23:56:38.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:38.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.572876+0000", 2024-09-14T23:56:38.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2378000000000001e-05 2024-09-14T23:56:38.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:38.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.572883+0000", 2024-09-14T23:56:38.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4400000000000002e-06 2024-09-14T23:56:38.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.065 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:38.065 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.576636+0000", 2024-09-14T23:56:38.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003752891 2024-09-14T23:56:38.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.065 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.065 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:38.065 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.576692+0000", 2024-09-14T23:56:38.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6042999999999999e-05 2024-09-14T23:56:38.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.065 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.065 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:56:38.065 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.576765+0000", 2024-09-14T23:56:38.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3731000000000003e-05 2024-09-14T23:56:38.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.066 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.066 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:56:38.066 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.578188+0000", 2024-09-14T23:56:38.066 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001422399 2024-09-14T23:56:38.066 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.066 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.066 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:38.066 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.578219+0000", 2024-09-14T23:56:38.066 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0694000000000001e-05 2024-09-14T23:56:38.066 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.066 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.066 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:38.066 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.578262+0000", 2024-09-14T23:56:38.066 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3511999999999999e-05 2024-09-14T23:56:38.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.067 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.067 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:38.067 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.578301+0000", 2024-09-14T23:56:38.067 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8816999999999998e-05 2024-09-14T23:56:38.067 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:38.067 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:38.067 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:38.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.067 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.067 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4209.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-14T23:56:38.067 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:15.572919+0000", 2024-09-14T23:56:38.067 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.454616681000001, 2024-09-14T23:56:38.067 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0055260450000000003, 2024-09-14T23:56:38.068 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:38.068 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:38.068 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:38.068 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:38.068 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4209", 2024-09-14T23:56:38.068 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/1617902543", 2024-09-14T23:56:38.068 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 112 2024-09-14T23:56:38.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.068 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:38.068 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.068 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:38.068 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.572919+0000", 2024-09-14T23:56:38.068 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:38.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.069 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.069 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:38.069 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.572919+0000", 2024-09-14T23:56:38.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:38.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.069 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.069 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:38.069 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.572920+0000", 2024-09-14T23:56:38.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8999999999999995e-07 2024-09-14T23:56:38.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.069 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.069 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:38.070 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.572981+0000", 2024-09-14T23:56:38.070 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1577999999999999e-05 2024-09-14T23:56:38.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.070 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:38.070 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.572989+0000", 2024-09-14T23:56:38.070 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5970000000000003e-06 2024-09-14T23:56:38.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.070 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:38.070 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.572993+0000", 2024-09-14T23:56:38.070 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3529999999999997e-06 2024-09-14T23:56:38.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.071 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:38.071 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.577086+0000", 2024-09-14T23:56:38.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0040926670000000004 2024-09-14T23:56:38.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.071 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.071 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:38.071 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.577142+0000", 2024-09-14T23:56:38.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5875999999999998e-05 2024-09-14T23:56:38.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.071 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.071 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:56:38.071 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.577219+0000", 2024-09-14T23:56:38.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6652999999999999e-05 2024-09-14T23:56:38.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.072 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.072 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:38.072 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.578356+0000", 2024-09-14T23:56:38.072 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0011372789999999999 2024-09-14T23:56:38.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.072 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.072 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:56:38.072 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.578372+0000", 2024-09-14T23:56:38.072 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6529e-05 2024-09-14T23:56:38.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.072 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.072 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:38.072 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.578422+0000", 2024-09-14T23:56:38.072 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9317999999999998e-05 2024-09-14T23:56:38.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.073 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.073 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:38.073 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.578445+0000", 2024-09-14T23:56:38.073 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3405e-05 2024-09-14T23:56:38.073 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:38.073 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:38.073 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:38.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.073 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.073 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4209.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-14T23:56:38.073 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:15.573005+0000", 2024-09-14T23:56:38.073 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.454530734999999, 2024-09-14T23:56:38.073 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0066227630000000003, 2024-09-14T23:56:38.074 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:38.074 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:38.074 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:38.074 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:38.074 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4209", 2024-09-14T23:56:38.074 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/1617902543", 2024-09-14T23:56:38.074 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 113 2024-09-14T23:56:38.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.074 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:38.074 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.074 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:38.074 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573005+0000", 2024-09-14T23:56:38.074 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:38.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.075 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.075 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:38.075 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573005+0000", 2024-09-14T23:56:38.075 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:38.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.075 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.075 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:38.075 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573005+0000", 2024-09-14T23:56:38.075 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0900000000000002e-07 2024-09-14T23:56:38.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.075 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.075 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:38.075 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573009+0000", 2024-09-14T23:56:38.076 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1080000000000001e-06 2024-09-14T23:56:38.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.076 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.076 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:38.076 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573023+0000", 2024-09-14T23:56:38.076 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4103e-05 2024-09-14T23:56:38.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.076 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.076 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:38.076 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573027+0000", 2024-09-14T23:56:38.076 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.929e-06 2024-09-14T23:56:38.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.076 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.076 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:38.077 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.577716+0000", 2024-09-14T23:56:38.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0046899380000000003 2024-09-14T23:56:38.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.077 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:38.077 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.577763+0000", 2024-09-14T23:56:38.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6261999999999999e-05 2024-09-14T23:56:38.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.077 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:56:38.077 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.577841+0000", 2024-09-14T23:56:38.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7906999999999998e-05 2024-09-14T23:56:38.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.078 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:56:38.078 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.579521+0000", 2024-09-14T23:56:38.078 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016806480000000001 2024-09-14T23:56:38.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.078 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.078 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:38.078 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.579571+0000", 2024-09-14T23:56:38.078 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9700000000000002e-05 2024-09-14T23:56:38.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.078 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.078 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:38.078 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.579601+0000", 2024-09-14T23:56:38.078 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0340000000000001e-05 2024-09-14T23:56:38.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.079 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.079 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:38.079 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.579628+0000", 2024-09-14T23:56:38.079 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6418999999999999e-05 2024-09-14T23:56:38.079 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:38.079 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:38.079 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:38.079 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.079 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.079 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4209.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-14T23:56:38.079 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:15.573032+0000", 2024-09-14T23:56:38.079 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.454503262999999, 2024-09-14T23:56:38.079 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0076935190000000002, 2024-09-14T23:56:38.079 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:38.080 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:38.080 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:38.080 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:38.080 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4209", 2024-09-14T23:56:38.080 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/1617902543", 2024-09-14T23:56:38.080 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 114 2024-09-14T23:56:38.080 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.080 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:38.080 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.080 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:38.080 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573032+0000", 2024-09-14T23:56:38.080 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:38.080 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.081 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.081 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:38.081 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573032+0000", 2024-09-14T23:56:38.081 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:38.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.081 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.081 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:38.081 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573033+0000", 2024-09-14T23:56:38.081 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8000000000000002e-07 2024-09-14T23:56:38.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.081 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.081 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:38.081 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573035+0000", 2024-09-14T23:56:38.081 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.627e-06 2024-09-14T23:56:38.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.082 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.082 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:38.082 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573041+0000", 2024-09-14T23:56:38.082 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.237e-06 2024-09-14T23:56:38.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.082 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.082 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:38.082 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573049+0000", 2024-09-14T23:56:38.082 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3930000000000008e-06 2024-09-14T23:56:38.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.082 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.082 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:38.082 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.578474+0000", 2024-09-14T23:56:38.083 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0054253160000000003 2024-09-14T23:56:38.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.083 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.083 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:38.083 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.578532+0000", 2024-09-14T23:56:38.083 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8146999999999997e-05 2024-09-14T23:56:38.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.083 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.083 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:56:38.083 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.578646+0000", 2024-09-14T23:56:38.083 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000113428 2024-09-14T23:56:38.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.083 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.083 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:56:38.084 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.580086+0000", 2024-09-14T23:56:38.084 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001439957 2024-09-14T23:56:38.084 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.084 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.084 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:38.084 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.580661+0000", 2024-09-14T23:56:38.084 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000575393 2024-09-14T23:56:38.084 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.084 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.084 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:38.084 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.580703+0000", 2024-09-14T23:56:38.084 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1359999999999997e-05 2024-09-14T23:56:38.084 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.084 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.085 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:38.085 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.580726+0000", 2024-09-14T23:56:38.085 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3380999999999999e-05 2024-09-14T23:56:38.085 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:38.085 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:38.085 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:38.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.085 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4209.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-14T23:56:38.085 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:15.573151+0000", 2024-09-14T23:56:38.085 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.454385159000001, 2024-09-14T23:56:38.085 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0081104469999999998, 2024-09-14T23:56:38.085 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:38.085 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:38.086 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:38.086 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:38.086 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4209", 2024-09-14T23:56:38.086 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/1617902543", 2024-09-14T23:56:38.086 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 115 2024-09-14T23:56:38.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.086 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:38.086 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.086 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:38.086 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573151+0000", 2024-09-14T23:56:38.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:38.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.086 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.086 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:38.087 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573151+0000", 2024-09-14T23:56:38.087 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:38.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.087 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:38.087 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573151+0000", 2024-09-14T23:56:38.087 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9599999999999999e-07 2024-09-14T23:56:38.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.087 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:38.087 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573245+0000", 2024-09-14T23:56:38.087 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3423000000000003e-05 2024-09-14T23:56:38.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.088 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:38.088 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573251+0000", 2024-09-14T23:56:38.088 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5819999999999998e-06 2024-09-14T23:56:38.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.088 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.088 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:38.088 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573255+0000", 2024-09-14T23:56:38.088 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6899999999999998e-06 2024-09-14T23:56:38.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.088 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.088 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:38.088 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.578943+0000", 2024-09-14T23:56:38.088 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0056883999999999997 2024-09-14T23:56:38.089 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.089 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.089 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:38.089 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.578991+0000", 2024-09-14T23:56:38.089 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8049e-05 2024-09-14T23:56:38.089 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.089 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.089 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:56:38.089 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.579079+0000", 2024-09-14T23:56:38.089 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7886000000000005e-05 2024-09-14T23:56:38.089 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.089 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.089 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:56:38.090 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.580095+0000", 2024-09-14T23:56:38.090 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001016213 2024-09-14T23:56:38.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.090 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.090 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:38.090 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.581209+0000", 2024-09-14T23:56:38.090 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001113677 2024-09-14T23:56:38.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.090 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.090 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:38.090 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.581238+0000", 2024-09-14T23:56:38.090 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9189000000000001e-05 2024-09-14T23:56:38.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.090 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:38.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.581261+0000", 2024-09-14T23:56:38.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2742e-05 2024-09-14T23:56:38.091 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:38.091 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:38.091 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:38.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.091 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4209.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-14T23:56:38.091 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:15.573263+0000", 2024-09-14T23:56:38.091 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.454272907, 2024-09-14T23:56:38.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0080966040000000003, 2024-09-14T23:56:38.091 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:38.092 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:38.092 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:38.092 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:38.092 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4209", 2024-09-14T23:56:38.092 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/1617902543", 2024-09-14T23:56:38.092 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 116 2024-09-14T23:56:38.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.092 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:38.092 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.092 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:38.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573263+0000", 2024-09-14T23:56:38.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:38.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.093 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.093 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:38.093 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573263+0000", 2024-09-14T23:56:38.093 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:38.093 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.093 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.093 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:38.093 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573263+0000", 2024-09-14T23:56:38.093 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3200000000000001e-07 2024-09-14T23:56:38.093 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.093 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.093 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:38.093 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573272+0000", 2024-09-14T23:56:38.094 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4889999999999995e-06 2024-09-14T23:56:38.094 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.094 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.094 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:38.094 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573279+0000", 2024-09-14T23:56:38.094 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2899999999999997e-06 2024-09-14T23:56:38.094 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.094 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.094 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:38.094 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573288+0000", 2024-09-14T23:56:38.094 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1810000000000007e-06 2024-09-14T23:56:38.094 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.094 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:38.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.579675+0000", 2024-09-14T23:56:38.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0063870799999999998 2024-09-14T23:56:38.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:38.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.579743+0000", 2024-09-14T23:56:38.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8256000000000005e-05 2024-09-14T23:56:38.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:56:38.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.579847+0000", 2024-09-14T23:56:38.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000103805 2024-09-14T23:56:38.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.096 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:56:38.096 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.580743+0000", 2024-09-14T23:56:38.096 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00089532000000000001 2024-09-14T23:56:38.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.096 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:38.096 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.581301+0000", 2024-09-14T23:56:38.096 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00055803999999999999 2024-09-14T23:56:38.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.097 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.097 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:38.097 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.581335+0000", 2024-09-14T23:56:38.097 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4022000000000001e-05 2024-09-14T23:56:38.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.097 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.097 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:38.097 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.581359+0000", 2024-09-14T23:56:38.097 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4788999999999999e-05 2024-09-14T23:56:38.097 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:38.097 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:38.097 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:38.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.098 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.098 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4209.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-14T23:56:38.098 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:15.573293+0000", 2024-09-14T23:56:38.098 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.454242326999999, 2024-09-14T23:56:38.098 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0082988439999999997, 2024-09-14T23:56:38.098 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:38.098 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:38.098 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:38.098 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:38.098 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4209", 2024-09-14T23:56:38.098 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/1617902543", 2024-09-14T23:56:38.098 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 117 2024-09-14T23:56:38.098 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.098 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:38.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.099 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:38.099 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573293+0000", 2024-09-14T23:56:38.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:38.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.099 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:38.099 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573293+0000", 2024-09-14T23:56:38.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:38.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.099 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:38.099 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573294+0000", 2024-09-14T23:56:38.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2999999999999999e-07 2024-09-14T23:56:38.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.100 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.100 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:38.100 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573296+0000", 2024-09-14T23:56:38.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3319999999999999e-06 2024-09-14T23:56:38.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.100 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.100 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:38.100 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573300+0000", 2024-09-14T23:56:38.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7670000000000001e-06 2024-09-14T23:56:38.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.100 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.101 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:38.101 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.573304+0000", 2024-09-14T23:56:38.101 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1620000000000001e-06 2024-09-14T23:56:38.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.101 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:38.101 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.580145+0000", 2024-09-14T23:56:38.101 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0068414859999999999 2024-09-14T23:56:38.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.101 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:38.101 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.580231+0000", 2024-09-14T23:56:38.101 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5997999999999999e-05 2024-09-14T23:56:38.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.102 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.102 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:56:38.102 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.580372+0000", 2024-09-14T23:56:38.102 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000140889 2024-09-14T23:56:38.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.102 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.102 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:56:38.102 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.580946+0000", 2024-09-14T23:56:38.102 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00057388100000000004 2024-09-14T23:56:38.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.102 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.102 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:38.102 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.581545+0000", 2024-09-14T23:56:38.103 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00059883899999999999 2024-09-14T23:56:38.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.103 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.103 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:38.103 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.581573+0000", 2024-09-14T23:56:38.103 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7596000000000001e-05 2024-09-14T23:56:38.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.103 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.103 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:38.103 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:15.581592+0000", 2024-09-14T23:56:38.103 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9664e-05 2024-09-14T23:56:38.103 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:38.103 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:38.103 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:38.104 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.104 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.104 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4209.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-14T23:56:38.104 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:16.002477+0000", 2024-09-14T23:56:38.104 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.025058893000001, 2024-09-14T23:56:38.104 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0043788669999999998, 2024-09-14T23:56:38.104 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:38.104 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:38.104 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:38.104 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:38.104 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4209", 2024-09-14T23:56:38.104 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/1617902543", 2024-09-14T23:56:38.104 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 134 2024-09-14T23:56:38.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.105 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:38.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.105 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:38.105 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002477+0000", 2024-09-14T23:56:38.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:38.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.105 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:38.105 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002477+0000", 2024-09-14T23:56:38.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:38.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.105 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:38.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002477+0000", 2024-09-14T23:56:38.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1699999999999998e-07 2024-09-14T23:56:38.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.106 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:38.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002564+0000", 2024-09-14T23:56:38.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7051000000000004e-05 2024-09-14T23:56:38.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.106 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:38.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002586+0000", 2024-09-14T23:56:38.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1963999999999998e-05 2024-09-14T23:56:38.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.107 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:38.107 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002606+0000", 2024-09-14T23:56:38.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9758e-05 2024-09-14T23:56:38.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.107 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:38.107 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.004958+0000", 2024-09-14T23:56:38.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023515319999999999 2024-09-14T23:56:38.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.107 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:38.107 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.005004+0000", 2024-09-14T23:56:38.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6493999999999999e-05 2024-09-14T23:56:38.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:56:38.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.005089+0000", 2024-09-14T23:56:38.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4577e-05 2024-09-14T23:56:38.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:38.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.006803+0000", 2024-09-14T23:56:38.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017141229999999999 2024-09-14T23:56:38.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:56:38.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.006810+0000", 2024-09-14T23:56:38.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1389999999999998e-06 2024-09-14T23:56:38.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:38.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.006835+0000", 2024-09-14T23:56:38.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5179999999999999e-05 2024-09-14T23:56:38.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:38.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.006856+0000", 2024-09-14T23:56:38.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0531999999999999e-05 2024-09-14T23:56:38.109 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:38.110 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:38.110 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:38.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.110 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.110 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4209.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-14T23:56:38.110 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:16.002615+0000", 2024-09-14T23:56:38.110 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.024920922, 2024-09-14T23:56:38.110 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0049197540000000001, 2024-09-14T23:56:38.110 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:38.110 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:38.110 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:38.110 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:38.110 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4209", 2024-09-14T23:56:38.110 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/1617902543", 2024-09-14T23:56:38.111 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 135 2024-09-14T23:56:38.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.111 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:38.111 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.111 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:38.111 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002615+0000", 2024-09-14T23:56:38.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:38.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.111 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.111 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:38.111 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002615+0000", 2024-09-14T23:56:38.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:38.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:38.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002615+0000", 2024-09-14T23:56:38.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.58e-07 2024-09-14T23:56:38.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:38.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002620+0000", 2024-09-14T23:56:38.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3320000000000002e-06 2024-09-14T23:56:38.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:38.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002625+0000", 2024-09-14T23:56:38.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9840000000000001e-06 2024-09-14T23:56:38.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.113 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:38.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002628+0000", 2024-09-14T23:56:38.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4620000000000001e-06 2024-09-14T23:56:38.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.113 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:38.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.005509+0000", 2024-09-14T23:56:38.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0028811269999999998 2024-09-14T23:56:38.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.113 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:38.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.005617+0000", 2024-09-14T23:56:38.114 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0001078 2024-09-14T23:56:38.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.114 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:56:38.114 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.005765+0000", 2024-09-14T23:56:38.114 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000148314 2024-09-14T23:56:38.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.114 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:38.114 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.006924+0000", 2024-09-14T23:56:38.114 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0011589630000000001 2024-09-14T23:56:38.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.114 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:56:38.115 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.007484+0000", 2024-09-14T23:56:38.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00055926900000000004 2024-09-14T23:56:38.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.115 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:38.115 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.007512+0000", 2024-09-14T23:56:38.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7988999999999999e-05 2024-09-14T23:56:38.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.115 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:38.115 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.007535+0000", 2024-09-14T23:56:38.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3056000000000001e-05 2024-09-14T23:56:38.116 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:38.116 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:38.116 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:38.116 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.116 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.116 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4209.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-14T23:56:38.116 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:16.002632+0000", 2024-09-14T23:56:38.116 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.024903742999999, 2024-09-14T23:56:38.116 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0056205409999999997, 2024-09-14T23:56:38.116 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:38.116 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:38.116 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:38.116 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:38.117 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4209", 2024-09-14T23:56:38.117 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/1617902543", 2024-09-14T23:56:38.117 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 136 2024-09-14T23:56:38.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.117 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:38.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:38.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002632+0000", 2024-09-14T23:56:38.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:38.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:38.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002632+0000", 2024-09-14T23:56:38.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:38.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.118 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.118 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:38.118 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002632+0000", 2024-09-14T23:56:38.118 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4699999999999998e-07 2024-09-14T23:56:38.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.118 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.118 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:38.118 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002634+0000", 2024-09-14T23:56:38.118 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.199e-06 2024-09-14T23:56:38.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.118 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.118 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:38.118 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002646+0000", 2024-09-14T23:56:38.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.111e-05 2024-09-14T23:56:38.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.119 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:38.119 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002650+0000", 2024-09-14T23:56:38.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.938e-06 2024-09-14T23:56:38.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.119 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:38.119 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.006332+0000", 2024-09-14T23:56:38.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003681068 2024-09-14T23:56:38.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.119 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:38.120 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.006412+0000", 2024-09-14T23:56:38.120 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0127e-05 2024-09-14T23:56:38.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.120 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.120 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:56:38.120 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.006502+0000", 2024-09-14T23:56:38.120 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0064999999999999e-05 2024-09-14T23:56:38.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.120 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.120 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:38.120 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.008170+0000", 2024-09-14T23:56:38.120 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016677580000000001 2024-09-14T23:56:38.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.121 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.121 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:56:38.121 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.008179+0000", 2024-09-14T23:56:38.121 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9669999999999996e-06 2024-09-14T23:56:38.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.121 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.121 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:38.121 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.008214+0000", 2024-09-14T23:56:38.121 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4878e-05 2024-09-14T23:56:38.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.121 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.121 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:38.121 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.008253+0000", 2024-09-14T23:56:38.122 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8183999999999999e-05 2024-09-14T23:56:38.122 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:38.122 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:38.122 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:38.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.122 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.122 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4209.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-14T23:56:38.122 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:16.002663+0000", 2024-09-14T23:56:38.122 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.024872478999999, 2024-09-14T23:56:38.122 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0071466480000000002, 2024-09-14T23:56:38.122 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:38.122 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:38.122 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:38.122 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:38.123 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4209", 2024-09-14T23:56:38.123 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/1617902543", 2024-09-14T23:56:38.123 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 137 2024-09-14T23:56:38.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.123 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:38.123 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.123 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:38.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002663+0000", 2024-09-14T23:56:38.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:38.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.123 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.123 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:38.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002663+0000", 2024-09-14T23:56:38.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:38.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.124 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:38.124 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002664+0000", 2024-09-14T23:56:38.124 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6100000000000001e-07 2024-09-14T23:56:38.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.124 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:38.124 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002786+0000", 2024-09-14T23:56:38.124 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000122664 2024-09-14T23:56:38.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.124 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:38.124 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002798+0000", 2024-09-14T23:56:38.125 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1671000000000001e-05 2024-09-14T23:56:38.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.125 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.125 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:38.125 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002808+0000", 2024-09-14T23:56:38.125 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8330000000000006e-06 2024-09-14T23:56:38.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.125 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.125 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:38.125 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.006952+0000", 2024-09-14T23:56:38.125 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0041437059999999996 2024-09-14T23:56:38.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.125 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.126 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:38.126 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.006998+0000", 2024-09-14T23:56:38.126 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6508999999999998e-05 2024-09-14T23:56:38.126 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.126 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.126 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:56:38.126 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.007100+0000", 2024-09-14T23:56:38.126 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010176 2024-09-14T23:56:38.126 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.126 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.126 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:56:38.126 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.009307+0000", 2024-09-14T23:56:38.126 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0022067459999999999 2024-09-14T23:56:38.126 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.126 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.126 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:38.127 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.009757+0000", 2024-09-14T23:56:38.127 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00045041800000000001 2024-09-14T23:56:38.127 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.127 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.127 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:38.127 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.009789+0000", 2024-09-14T23:56:38.127 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1599e-05 2024-09-14T23:56:38.127 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.127 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.127 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:38.127 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.009810+0000", 2024-09-14T23:56:38.127 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1280999999999998e-05 2024-09-14T23:56:38.127 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:38.127 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:38.128 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:38.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.128 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.128 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4209.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-14T23:56:38.128 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:16.002824+0000", 2024-09-14T23:56:38.128 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.024712156, 2024-09-14T23:56:38.128 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0070812080000000003, 2024-09-14T23:56:38.128 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:38.128 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:38.128 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:38.128 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:38.128 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4209", 2024-09-14T23:56:38.128 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/1617902543", 2024-09-14T23:56:38.128 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 138 2024-09-14T23:56:38.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.129 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:38.129 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.129 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:38.129 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002824+0000", 2024-09-14T23:56:38.129 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:38.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.129 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.129 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:38.129 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002824+0000", 2024-09-14T23:56:38.129 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:38.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.129 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.130 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:38.130 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002824+0000", 2024-09-14T23:56:38.130 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7999999999999995e-07 2024-09-14T23:56:38.130 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.130 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.130 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:38.130 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002828+0000", 2024-09-14T23:56:38.130 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.262e-06 2024-09-14T23:56:38.130 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.130 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.130 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:38.130 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002836+0000", 2024-09-14T23:56:38.130 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6769999999999995e-06 2024-09-14T23:56:38.130 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.130 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.131 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:38.131 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002842+0000", 2024-09-14T23:56:38.131 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4809999999999997e-06 2024-09-14T23:56:38.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.131 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.131 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:38.131 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.007558+0000", 2024-09-14T23:56:38.131 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0047168720000000004 2024-09-14T23:56:38.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.131 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.131 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:38.131 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.007643+0000", 2024-09-14T23:56:38.131 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4289999999999994e-05 2024-09-14T23:56:38.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.131 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.132 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:56:38.132 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.007783+0000", 2024-09-14T23:56:38.132 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013978800000000001 2024-09-14T23:56:38.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.132 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.132 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:56:38.132 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.009817+0000", 2024-09-14T23:56:38.132 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0020344930000000001 2024-09-14T23:56:38.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.132 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.132 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:38.132 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.009854+0000", 2024-09-14T23:56:38.132 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6891000000000003e-05 2024-09-14T23:56:38.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.133 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:38.133 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.009882+0000", 2024-09-14T23:56:38.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8130999999999999e-05 2024-09-14T23:56:38.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.133 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:38.133 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.009905+0000", 2024-09-14T23:56:38.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2742999999999999e-05 2024-09-14T23:56:38.133 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:38.133 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:38.133 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:38.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.134 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4209.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-14T23:56:38.134 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:16.002851+0000", 2024-09-14T23:56:38.134 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.024684907000001, 2024-09-14T23:56:38.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0071603530000000004, 2024-09-14T23:56:38.134 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:38.134 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:38.134 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:38.134 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:38.134 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4209", 2024-09-14T23:56:38.134 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/1617902543", 2024-09-14T23:56:38.134 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 139 2024-09-14T23:56:38.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.135 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:38.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.135 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:38.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002851+0000", 2024-09-14T23:56:38.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:38.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.135 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:38.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002851+0000", 2024-09-14T23:56:38.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:38.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.135 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:38.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002851+0000", 2024-09-14T23:56:38.136 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0699999999999998e-07 2024-09-14T23:56:38.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.136 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.136 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:38.136 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002853+0000", 2024-09-14T23:56:38.136 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2560000000000001e-06 2024-09-14T23:56:38.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.136 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.136 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:38.136 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002861+0000", 2024-09-14T23:56:38.136 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7139999999999993e-06 2024-09-14T23:56:38.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.136 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:38.137 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002866+0000", 2024-09-14T23:56:38.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1329999999999998e-06 2024-09-14T23:56:38.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:38.137 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.008306+0000", 2024-09-14T23:56:38.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.005439503 2024-09-14T23:56:38.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:38.137 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.008387+0000", 2024-09-14T23:56:38.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1051999999999996e-05 2024-09-14T23:56:38.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:56:38.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.008477+0000", 2024-09-14T23:56:38.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9944000000000005e-05 2024-09-14T23:56:38.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:56:38.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.009824+0000", 2024-09-14T23:56:38.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0013476390000000001 2024-09-14T23:56:38.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:38.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.009945+0000", 2024-09-14T23:56:38.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012059500000000001 2024-09-14T23:56:38.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.139 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:38.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.009980+0000", 2024-09-14T23:56:38.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5469999999999998e-05 2024-09-14T23:56:38.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.139 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:38.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.010011+0000", 2024-09-14T23:56:38.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.074e-05 2024-09-14T23:56:38.139 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:38.139 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:38.139 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:38.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.140 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.140 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4209.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-14T23:56:38.140 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:16.002875+0000", 2024-09-14T23:56:38.140 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.024660793999999, 2024-09-14T23:56:38.140 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0073135270000000002, 2024-09-14T23:56:38.140 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:38.140 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:38.140 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:38.140 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:38.140 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4209", 2024-09-14T23:56:38.140 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/1617902543", 2024-09-14T23:56:38.140 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 140 2024-09-14T23:56:38.140 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.141 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:38.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.141 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:38.141 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002875+0000", 2024-09-14T23:56:38.141 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:38.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.141 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:38.141 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002875+0000", 2024-09-14T23:56:38.141 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:38.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.141 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:38.141 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002875+0000", 2024-09-14T23:56:38.141 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2500000000000001e-07 2024-09-14T23:56:38.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.142 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:38.142 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002936+0000", 2024-09-14T23:56:38.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0621000000000001e-05 2024-09-14T23:56:38.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.142 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:38.142 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002948+0000", 2024-09-14T23:56:38.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1651000000000001e-05 2024-09-14T23:56:38.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.142 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:38.142 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002954+0000", 2024-09-14T23:56:38.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3169999999999998e-06 2024-09-14T23:56:38.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:38.143 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.008711+0000", 2024-09-14T23:56:38.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057567269999999997 2024-09-14T23:56:38.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:38.143 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.008782+0000", 2024-09-14T23:56:38.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0970999999999995e-05 2024-09-14T23:56:38.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:56:38.143 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.008870+0000", 2024-09-14T23:56:38.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7849999999999994e-05 2024-09-14T23:56:38.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.144 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.144 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:56:38.144 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.010019+0000", 2024-09-14T23:56:38.144 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0011495979999999999 2024-09-14T23:56:38.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.144 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.144 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:38.144 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.010140+0000", 2024-09-14T23:56:38.144 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000120541 2024-09-14T23:56:38.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.144 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.144 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:38.144 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.010167+0000", 2024-09-14T23:56:38.144 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7549e-05 2024-09-14T23:56:38.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.145 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.145 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:38.145 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.010188+0000", 2024-09-14T23:56:38.145 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1277e-05 2024-09-14T23:56:38.145 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:38.145 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:38.145 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:38.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.145 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.145 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4209.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-14T23:56:38.145 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:16.002967+0000", 2024-09-14T23:56:38.145 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.024568318, 2024-09-14T23:56:38.145 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0074891690000000004, 2024-09-14T23:56:38.146 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:38.146 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:38.146 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:38.146 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:38.146 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4209", 2024-09-14T23:56:38.146 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/1617902543", 2024-09-14T23:56:38.146 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 141 2024-09-14T23:56:38.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.146 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:38.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.146 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:38.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002967+0000", 2024-09-14T23:56:38.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:38.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.147 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:38.147 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002967+0000", 2024-09-14T23:56:38.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:38.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.147 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:38.147 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002968+0000", 2024-09-14T23:56:38.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0699999999999997e-07 2024-09-14T23:56:38.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.147 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:38.148 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002972+0000", 2024-09-14T23:56:38.148 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.782e-06 2024-09-14T23:56:38.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.148 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.148 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:38.148 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002979+0000", 2024-09-14T23:56:38.148 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2089999999999999e-06 2024-09-14T23:56:38.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.148 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.148 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:38.148 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.002984+0000", 2024-09-14T23:56:38.148 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1399999999999999e-06 2024-09-14T23:56:38.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.149 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:38.149 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.009341+0000", 2024-09-14T23:56:38.149 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0063574060000000003 2024-09-14T23:56:38.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.149 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:38.149 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.009388+0000", 2024-09-14T23:56:38.149 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.651e-05 2024-09-14T23:56:38.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.149 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:56:38.149 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.009464+0000", 2024-09-14T23:56:38.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5829999999999997e-05 2024-09-14T23:56:38.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:56:38.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.010024+0000", 2024-09-14T23:56:38.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00056062099999999997 2024-09-14T23:56:38.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:38.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.010408+0000", 2024-09-14T23:56:38.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00038369899999999999 2024-09-14T23:56:38.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:38.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.010435+0000", 2024-09-14T23:56:38.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6648e-05 2024-09-14T23:56:38.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:38.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:38.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:38.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:16.010457+0000", 2024-09-14T23:56:38.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1817000000000001e-05 2024-09-14T23:56:38.151 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:38.151 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:38.151 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:38.151 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:38.151 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:38.151 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:56:38.153 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T23:56:38.154 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:56:38.151+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:56:38.254 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:56:38.252+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:56:38.261 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:56:38.261 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:56:38.261 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:56:38.261 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:56:38.270 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T23:56:38.354 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:56:38.353+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:56:38.377 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:56:38.377 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:56:38.377 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:56:38.377 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:56:38.377 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:56:38.385 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T23:56:38.455 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:56:38.453+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:56:38.494 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:56:38.494 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:56:38.494 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:56:38.494 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:56:38.494 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.495 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4209.0:20 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'10, mlcod=16'8)", 2024-09-14T23:56:38.495 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:15.349996+0000", 2024-09-14T23:56:38.495 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.141977774000001, 2024-09-14T23:56:38.495 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010569310000000001, 2024-09-14T23:56:38.495 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:38.495 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:38.495 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:38.495 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:38.495 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.495 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:38.495 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.349996+0000", 2024-09-14T23:56:38.495 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:38.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.495 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.495 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:38.496 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.349996+0000", 2024-09-14T23:56:38.496 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:38.496 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.496 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.496 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:38.496 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.349997+0000", 2024-09-14T23:56:38.496 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2200000000000002e-07 2024-09-14T23:56:38.496 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.497 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.497 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:38.497 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.350123+0000", 2024-09-14T23:56:38.497 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000126797 2024-09-14T23:56:38.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.497 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.497 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:38.497 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.350141+0000", 2024-09-14T23:56:38.497 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7110000000000001e-05 2024-09-14T23:56:38.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.497 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.497 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:38.497 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.350150+0000", 2024-09-14T23:56:38.497 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3370000000000006e-06 2024-09-14T23:56:38.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.498 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:38.498 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.350213+0000", 2024-09-14T23:56:38.498 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2718999999999994e-05 2024-09-14T23:56:38.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.498 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:38.498 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.350235+0000", 2024-09-14T23:56:38.498 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2055000000000001e-05 2024-09-14T23:56:38.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.498 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:38.498 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.350990+0000", 2024-09-14T23:56:38.498 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00075517900000000003 2024-09-14T23:56:38.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.499 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:38.499 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.351053+0000", 2024-09-14T23:56:38.499 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2811999999999994e-05 2024-09-14T23:56:38.499 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:38.499 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:38.499 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:38.499 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.499 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.499 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4209.0:31 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'17, mlcod=16'14)", 2024-09-14T23:56:38.499 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:15.358180+0000", 2024-09-14T23:56:38.499 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.133793408999999, 2024-09-14T23:56:38.499 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001091533, 2024-09-14T23:56:38.499 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:38.499 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:38.499 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:38.500 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:38.500 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.500 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:38.500 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.358180+0000", 2024-09-14T23:56:38.500 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:38.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.500 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.500 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:38.500 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.358180+0000", 2024-09-14T23:56:38.500 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:38.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.500 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.500 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:38.500 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.358181+0000", 2024-09-14T23:56:38.500 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0100000000000004e-07 2024-09-14T23:56:38.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.501 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.501 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:38.501 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.358187+0000", 2024-09-14T23:56:38.501 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6509999999999998e-06 2024-09-14T23:56:38.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.501 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.501 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:38.501 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.358195+0000", 2024-09-14T23:56:38.501 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7590000000000003e-06 2024-09-14T23:56:38.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.501 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.501 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:38.501 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.358202+0000", 2024-09-14T23:56:38.502 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6089999999999997e-06 2024-09-14T23:56:38.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.502 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.502 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:38.502 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.358250+0000", 2024-09-14T23:56:38.502 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8519000000000001e-05 2024-09-14T23:56:38.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.502 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.502 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:38.502 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.358265+0000", 2024-09-14T23:56:38.502 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4749999999999999e-05 2024-09-14T23:56:38.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.502 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.503 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:38.503 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.359220+0000", 2024-09-14T23:56:38.503 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00095522200000000004 2024-09-14T23:56:38.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.503 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.503 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:38.503 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.359272+0000", 2024-09-14T23:56:38.503 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1221999999999998e-05 2024-09-14T23:56:38.503 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:38.503 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:38.503 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:38.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.503 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.503 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4209.0:32 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'18, mlcod=16'14)", 2024-09-14T23:56:38.503 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:15.358686+0000", 2024-09-14T23:56:38.503 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.133287995, 2024-09-14T23:56:38.504 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00098863800000000002, 2024-09-14T23:56:38.504 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:38.504 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:38.504 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:38.504 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:38.504 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.504 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:38.504 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.358686+0000", 2024-09-14T23:56:38.504 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:38.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.504 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.504 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:38.504 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.358686+0000", 2024-09-14T23:56:38.504 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:38.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.505 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.505 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:38.505 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.358686+0000", 2024-09-14T23:56:38.505 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6000000000000002e-07 2024-09-14T23:56:38.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.505 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.505 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:38.505 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.358755+0000", 2024-09-14T23:56:38.505 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8532e-05 2024-09-14T23:56:38.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.505 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.505 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:38.505 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.358767+0000", 2024-09-14T23:56:38.505 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1960999999999999e-05 2024-09-14T23:56:38.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.505 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.506 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:38.506 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.358774+0000", 2024-09-14T23:56:38.506 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5379999999999997e-06 2024-09-14T23:56:38.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.506 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:38.506 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.358819+0000", 2024-09-14T23:56:38.506 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4601999999999997e-05 2024-09-14T23:56:38.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.506 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:38.506 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.358832+0000", 2024-09-14T23:56:38.506 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3125000000000001e-05 2024-09-14T23:56:38.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.506 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:38.507 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.359614+0000", 2024-09-14T23:56:38.507 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00078198899999999997 2024-09-14T23:56:38.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.507 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.507 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:38.507 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.359674+0000", 2024-09-14T23:56:38.507 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0031000000000002e-05 2024-09-14T23:56:38.507 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:38.507 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:38.507 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:38.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.507 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.507 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4209.0:35 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'21, mlcod=16'16)", 2024-09-14T23:56:38.507 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:15.360261+0000", 2024-09-14T23:56:38.507 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.13171298, 2024-09-14T23:56:38.507 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010405530000000001, 2024-09-14T23:56:38.508 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:38.508 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:38.508 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:38.508 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:38.508 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.508 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:38.508 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.360261+0000", 2024-09-14T23:56:38.508 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:38.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.508 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.508 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:38.508 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.360261+0000", 2024-09-14T23:56:38.508 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:38.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.508 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.509 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:38.509 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.360261+0000", 2024-09-14T23:56:38.509 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3300000000000001e-07 2024-09-14T23:56:38.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.509 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.509 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:38.509 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.360321+0000", 2024-09-14T23:56:38.509 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9626999999999997e-05 2024-09-14T23:56:38.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.509 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.509 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:38.509 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.360334+0000", 2024-09-14T23:56:38.509 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2686e-05 2024-09-14T23:56:38.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.509 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.510 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:38.510 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.360340+0000", 2024-09-14T23:56:38.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8909999999999998e-06 2024-09-14T23:56:38.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.510 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:38.510 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.360382+0000", 2024-09-14T23:56:38.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1896e-05 2024-09-14T23:56:38.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.510 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:38.510 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.360397+0000", 2024-09-14T23:56:38.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4959e-05 2024-09-14T23:56:38.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.511 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:38.511 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.361275+0000", 2024-09-14T23:56:38.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00087766000000000005 2024-09-14T23:56:38.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.511 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:38.511 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.361301+0000", 2024-09-14T23:56:38.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6101e-05 2024-09-14T23:56:38.511 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:38.511 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:38.511 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:38.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.511 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4209.0:54 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'32, mlcod=16'29)", 2024-09-14T23:56:38.512 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:15.375492+0000", 2024-09-14T23:56:38.512 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.116481428, 2024-09-14T23:56:38.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015276700000000001, 2024-09-14T23:56:38.512 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:38.512 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:38.512 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:38.512 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:38.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.512 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:38.512 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.375492+0000", 2024-09-14T23:56:38.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:38.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.512 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:38.512 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.375492+0000", 2024-09-14T23:56:38.513 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:38.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.513 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:38.513 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.375493+0000", 2024-09-14T23:56:38.513 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0299999999999999e-07 2024-09-14T23:56:38.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.513 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:38.513 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.375497+0000", 2024-09-14T23:56:38.513 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.189e-06 2024-09-14T23:56:38.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:38.514 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.375505+0000", 2024-09-14T23:56:38.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4679999999999992e-06 2024-09-14T23:56:38.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:38.514 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.375510+0000", 2024-09-14T23:56:38.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8720000000000001e-06 2024-09-14T23:56:38.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:38.514 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.375558+0000", 2024-09-14T23:56:38.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7590999999999999e-05 2024-09-14T23:56:38.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.515 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:38.515 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.375581+0000", 2024-09-14T23:56:38.515 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3082999999999999e-05 2024-09-14T23:56:38.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.515 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:38.515 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.376982+0000", 2024-09-14T23:56:38.515 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0014012670000000001 2024-09-14T23:56:38.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.515 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:38.515 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.377020+0000", 2024-09-14T23:56:38.515 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7697000000000002e-05 2024-09-14T23:56:38.516 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:38.516 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:38.516 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:38.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.516 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4209.0:55 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'33, mlcod=16'29)", 2024-09-14T23:56:38.516 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:15.376055+0000", 2024-09-14T23:56:38.516 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.115918454999999, 2024-09-14T23:56:38.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00100033, 2024-09-14T23:56:38.516 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:38.516 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:38.516 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:38.516 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:38.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.517 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:38.517 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.376055+0000", 2024-09-14T23:56:38.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:38.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.517 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:38.517 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.376055+0000", 2024-09-14T23:56:38.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:38.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.517 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:38.517 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.376056+0000", 2024-09-14T23:56:38.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8500000000000002e-07 2024-09-14T23:56:38.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.518 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:38.518 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.376124+0000", 2024-09-14T23:56:38.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8608000000000004e-05 2024-09-14T23:56:38.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.518 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:38.518 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.376134+0000", 2024-09-14T23:56:38.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9329999999999996e-06 2024-09-14T23:56:38.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.519 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:38.519 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.376138+0000", 2024-09-14T23:56:38.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3420000000000001e-06 2024-09-14T23:56:38.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.519 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:38.519 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.376184+0000", 2024-09-14T23:56:38.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5666e-05 2024-09-14T23:56:38.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.519 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:38.519 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.376205+0000", 2024-09-14T23:56:38.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0812999999999999e-05 2024-09-14T23:56:38.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.520 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:38.520 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.377024+0000", 2024-09-14T23:56:38.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00081868100000000003 2024-09-14T23:56:38.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.520 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:38.520 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.377055+0000", 2024-09-14T23:56:38.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1801999999999998e-05 2024-09-14T23:56:38.520 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:38.520 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:38.520 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:38.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.521 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4209.0:68 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'42, mlcod=16'39)", 2024-09-14T23:56:38.521 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:15.387115+0000", 2024-09-14T23:56:38.521 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.104858603, 2024-09-14T23:56:38.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011477900000000001, 2024-09-14T23:56:38.521 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:38.521 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:38.521 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:38.521 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:38.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.521 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:38.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.387115+0000", 2024-09-14T23:56:38.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:38.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:38.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.387115+0000", 2024-09-14T23:56:38.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:38.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:38.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.387115+0000", 2024-09-14T23:56:38.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1799999999999995e-07 2024-09-14T23:56:38.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:38.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.387119+0000", 2024-09-14T23:56:38.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0459999999999999e-06 2024-09-14T23:56:38.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:38.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.387128+0000", 2024-09-14T23:56:38.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8459999999999997e-06 2024-09-14T23:56:38.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:38.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.387133+0000", 2024-09-14T23:56:38.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6e-06 2024-09-14T23:56:38.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:38.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.387175+0000", 2024-09-14T23:56:38.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1841999999999997e-05 2024-09-14T23:56:38.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:38.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.387187+0000", 2024-09-14T23:56:38.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2476999999999999e-05 2024-09-14T23:56:38.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:38.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.388226+0000", 2024-09-14T23:56:38.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010386060000000001 2024-09-14T23:56:38.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:38.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.388263+0000", 2024-09-14T23:56:38.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6854999999999999e-05 2024-09-14T23:56:38.525 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:38.525 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:38.525 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:38.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.525 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4209.0:69 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'43, mlcod=16'39)", 2024-09-14T23:56:38.525 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:15.387660+0000", 2024-09-14T23:56:38.525 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.104313742999999, 2024-09-14T23:56:38.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001072859, 2024-09-14T23:56:38.525 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:38.525 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:38.525 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:38.526 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:38.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.526 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:38.526 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.387660+0000", 2024-09-14T23:56:38.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:38.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.526 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:38.526 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.387660+0000", 2024-09-14T23:56:38.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:38.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.526 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:38.526 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.387660+0000", 2024-09-14T23:56:38.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0399999999999996e-07 2024-09-14T23:56:38.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:38.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.387732+0000", 2024-09-14T23:56:38.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1585e-05 2024-09-14T23:56:38.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:38.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.387743+0000", 2024-09-14T23:56:38.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1015e-05 2024-09-14T23:56:38.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:38.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.387748+0000", 2024-09-14T23:56:38.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4750000000000001e-06 2024-09-14T23:56:38.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:38.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.387806+0000", 2024-09-14T23:56:38.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7398000000000002e-05 2024-09-14T23:56:38.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:38.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.387825+0000", 2024-09-14T23:56:38.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9369999999999999e-05 2024-09-14T23:56:38.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:38.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.388693+0000", 2024-09-14T23:56:38.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000868106 2024-09-14T23:56:38.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:38.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.388733+0000", 2024-09-14T23:56:38.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9305999999999999e-05 2024-09-14T23:56:38.529 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:38.529 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:38.529 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:38.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.529 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4209.0:83 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'53, mlcod=16'50)", 2024-09-14T23:56:38.530 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:15.398499+0000", 2024-09-14T23:56:38.530 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.093474719, 2024-09-14T23:56:38.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001002333, 2024-09-14T23:56:38.530 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:38.530 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:38.530 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:38.530 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:38.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:38.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.398499+0000", 2024-09-14T23:56:38.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:38.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:38.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.398499+0000", 2024-09-14T23:56:38.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:38.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:38.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.398499+0000", 2024-09-14T23:56:38.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8299999999999997e-07 2024-09-14T23:56:38.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:38.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.398505+0000", 2024-09-14T23:56:38.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2680000000000002e-06 2024-09-14T23:56:38.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:38.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.398513+0000", 2024-09-14T23:56:38.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8970000000000008e-06 2024-09-14T23:56:38.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:38.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.398517+0000", 2024-09-14T23:56:38.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1949999999999996e-06 2024-09-14T23:56:38.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:38.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.398567+0000", 2024-09-14T23:56:38.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9917999999999999e-05 2024-09-14T23:56:38.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:38.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.398590+0000", 2024-09-14T23:56:38.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2952e-05 2024-09-14T23:56:38.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:38.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.399464+0000", 2024-09-14T23:56:38.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00087452199999999997 2024-09-14T23:56:38.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:38.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.399501+0000", 2024-09-14T23:56:38.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6998000000000001e-05 2024-09-14T23:56:38.533 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:38.534 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:38.534 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:38.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.534 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4209.0:87 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'57, mlcod=16'54)", 2024-09-14T23:56:38.534 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:15.401127+0000", 2024-09-14T23:56:38.534 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.090846056, 2024-09-14T23:56:38.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010080410000000001, 2024-09-14T23:56:38.534 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:38.534 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:38.534 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:38.534 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:38.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:38.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.401127+0000", 2024-09-14T23:56:38.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:38.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:38.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.401127+0000", 2024-09-14T23:56:38.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:38.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:38.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.401128+0000", 2024-09-14T23:56:38.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1200000000000003e-07 2024-09-14T23:56:38.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:38.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.401197+0000", 2024-09-14T23:56:38.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8922000000000002e-05 2024-09-14T23:56:38.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:38.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.401207+0000", 2024-09-14T23:56:38.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6350000000000001e-06 2024-09-14T23:56:38.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:38.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.401211+0000", 2024-09-14T23:56:38.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8910000000000003e-06 2024-09-14T23:56:38.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:38.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.401252+0000", 2024-09-14T23:56:38.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0308000000000001e-05 2024-09-14T23:56:38.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:38.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.401266+0000", 2024-09-14T23:56:38.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3966e-05 2024-09-14T23:56:38.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:38.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.402098+0000", 2024-09-14T23:56:38.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00083255899999999997 2024-09-14T23:56:38.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:38.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.402135+0000", 2024-09-14T23:56:38.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7248e-05 2024-09-14T23:56:38.538 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:38.538 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:38.538 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:38.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.538 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4209.0:89 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'59, mlcod=16'57)", 2024-09-14T23:56:38.538 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:15.403067+0000", 2024-09-14T23:56:38.538 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.088906684000001, 2024-09-14T23:56:38.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001037406, 2024-09-14T23:56:38.538 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:38.539 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:38.539 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:38.539 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:38.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:38.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.403067+0000", 2024-09-14T23:56:38.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:38.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:38.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.403067+0000", 2024-09-14T23:56:38.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:38.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:38.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.403068+0000", 2024-09-14T23:56:38.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9599999999999999e-07 2024-09-14T23:56:38.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:38.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.403113+0000", 2024-09-14T23:56:38.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5893999999999998e-05 2024-09-14T23:56:38.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:38.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.403125+0000", 2024-09-14T23:56:38.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1654999999999999e-05 2024-09-14T23:56:38.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:38.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.403130+0000", 2024-09-14T23:56:38.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2190000000000003e-06 2024-09-14T23:56:38.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:38.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.403178+0000", 2024-09-14T23:56:38.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8183999999999998e-05 2024-09-14T23:56:38.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:38.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.403193+0000", 2024-09-14T23:56:38.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4457000000000001e-05 2024-09-14T23:56:38.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:38.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.404068+0000", 2024-09-14T23:56:38.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00087506499999999996 2024-09-14T23:56:38.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:38.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.404104+0000", 2024-09-14T23:56:38.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6235999999999997e-05 2024-09-14T23:56:38.542 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:38.542 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:38.542 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:38.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.543 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4209.0:105 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'67, mlcod=16'64)", 2024-09-14T23:56:38.543 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:15.573180+0000", 2024-09-14T23:56:38.543 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.918793524000002, 2024-09-14T23:56:38.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001054538, 2024-09-14T23:56:38.543 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:38.543 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:38.543 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:38.543 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:38.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:38.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.573180+0000", 2024-09-14T23:56:38.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:38.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:38.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.573180+0000", 2024-09-14T23:56:38.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:38.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:38.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.573181+0000", 2024-09-14T23:56:38.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8899999999999999e-07 2024-09-14T23:56:38.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:38.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.573188+0000", 2024-09-14T23:56:38.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2880000000000004e-06 2024-09-14T23:56:38.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:38.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.573199+0000", 2024-09-14T23:56:38.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1197e-05 2024-09-14T23:56:38.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:38.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.573209+0000", 2024-09-14T23:56:38.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0006e-05 2024-09-14T23:56:38.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:38.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.573249+0000", 2024-09-14T23:56:38.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0204999999999999e-05 2024-09-14T23:56:38.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:38.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.573267+0000", 2024-09-14T23:56:38.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7484000000000001e-05 2024-09-14T23:56:38.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:38.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.574205+0000", 2024-09-14T23:56:38.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00093827299999999999 2024-09-14T23:56:38.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:38.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.574235+0000", 2024-09-14T23:56:38.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9496e-05 2024-09-14T23:56:38.547 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:38.547 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:38.547 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:38.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.547 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4209.0:115 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'77, mlcod=16'73)", 2024-09-14T23:56:38.547 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:15.580084+0000", 2024-09-14T23:56:38.547 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.911889104, 2024-09-14T23:56:38.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010510949999999999, 2024-09-14T23:56:38.547 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:38.547 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:38.547 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:38.547 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:38.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:38.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.580084+0000", 2024-09-14T23:56:38.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:38.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:38.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.580084+0000", 2024-09-14T23:56:38.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:38.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:38.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.580085+0000", 2024-09-14T23:56:38.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4400000000000001e-07 2024-09-14T23:56:38.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:38.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.580219+0000", 2024-09-14T23:56:38.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000133996 2024-09-14T23:56:38.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:38.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.580227+0000", 2024-09-14T23:56:38.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1210000000000007e-06 2024-09-14T23:56:38.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:38.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.580231+0000", 2024-09-14T23:56:38.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5379999999999996e-06 2024-09-14T23:56:38.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:38.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.580278+0000", 2024-09-14T23:56:38.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6882999999999998e-05 2024-09-14T23:56:38.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:38.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.580296+0000", 2024-09-14T23:56:38.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7771e-05 2024-09-14T23:56:38.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:38.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.581099+0000", 2024-09-14T23:56:38.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00080261199999999997 2024-09-14T23:56:38.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:38.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:15.581135+0000", 2024-09-14T23:56:38.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6730000000000002e-05 2024-09-14T23:56:38.551 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:38.551 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:38.551 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:38.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.551 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4209.0:140 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'98, mlcod=16'93)", 2024-09-14T23:56:38.551 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:16.008964+0000", 2024-09-14T23:56:38.551 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.483009266, 2024-09-14T23:56:38.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00109066, 2024-09-14T23:56:38.552 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:38.552 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:38.552 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:38.552 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:38.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:38.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:16.008964+0000", 2024-09-14T23:56:38.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:38.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:38.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:16.008964+0000", 2024-09-14T23:56:38.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:38.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:38.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:16.008965+0000", 2024-09-14T23:56:38.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9500000000000003e-07 2024-09-14T23:56:38.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:38.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:16.009117+0000", 2024-09-14T23:56:38.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015197400000000001 2024-09-14T23:56:38.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:38.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:16.009126+0000", 2024-09-14T23:56:38.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7259999999999997e-06 2024-09-14T23:56:38.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:38.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:16.009133+0000", 2024-09-14T23:56:38.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5699999999999998e-06 2024-09-14T23:56:38.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:38.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:16.009249+0000", 2024-09-14T23:56:38.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000115565 2024-09-14T23:56:38.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:38.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:16.009263+0000", 2024-09-14T23:56:38.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4447000000000001e-05 2024-09-14T23:56:38.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:38.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:16.010027+0000", 2024-09-14T23:56:38.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00076407500000000002 2024-09-14T23:56:38.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:38.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:16.010055+0000", 2024-09-14T23:56:38.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7807999999999999e-05 2024-09-14T23:56:38.555 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:38.555 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:38.555 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:38.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.556 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4209.0:160 1.0 e17/15 1:30306672:devicehealth::main.db.0000000000000000:head v 17'114, mlcod=17'112)", 2024-09-14T23:56:38.556 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:16.613316+0000", 2024-09-14T23:56:38.556 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 21.878657424, 2024-09-14T23:56:38.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010047190000000001, 2024-09-14T23:56:38.556 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:38.556 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:38.556 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:38.556 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:38.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:38.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:16.613316+0000", 2024-09-14T23:56:38.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:38.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:38.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:16.613316+0000", 2024-09-14T23:56:38.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:38.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:38.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:16.613317+0000", 2024-09-14T23:56:38.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6000000000000004e-07 2024-09-14T23:56:38.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:38.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:16.613367+0000", 2024-09-14T23:56:38.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0430000000000003e-05 2024-09-14T23:56:38.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:38.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:16.613377+0000", 2024-09-14T23:56:38.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6690000000000001e-06 2024-09-14T23:56:38.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:38.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:16.613381+0000", 2024-09-14T23:56:38.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5650000000000003e-06 2024-09-14T23:56:38.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:38.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:16.613439+0000", 2024-09-14T23:56:38.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8173e-05 2024-09-14T23:56:38.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:38.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:16.613451+0000", 2024-09-14T23:56:38.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1977999999999999e-05 2024-09-14T23:56:38.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:38.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:16.614283+0000", 2024-09-14T23:56:38.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000832003 2024-09-14T23:56:38.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:38.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:16.614321+0000", 2024-09-14T23:56:38.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7341e-05 2024-09-14T23:56:38.560 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:38.560 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:38.560 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:38.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.560 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:11 3.1f 3:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T23:56:38.560 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:34.410107+0000", 2024-09-14T23:56:38.560 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.081866464, 2024-09-14T23:56:38.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00151684, 2024-09-14T23:56:38.560 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:38.560 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:38.560 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:38.560 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:38.561 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:38.561 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:38.561 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 11 2024-09-14T23:56:38.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.561 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:38.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:38.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.410107+0000", 2024-09-14T23:56:38.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:38.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:38.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.410107+0000", 2024-09-14T23:56:38.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:38.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:38.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.410109+0000", 2024-09-14T23:56:38.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.973e-06 2024-09-14T23:56:38.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:38.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.410128+0000", 2024-09-14T23:56:38.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8865999999999999e-05 2024-09-14T23:56:38.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:38.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.410147+0000", 2024-09-14T23:56:38.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9572000000000002e-05 2024-09-14T23:56:38.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:38.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.410160+0000", 2024-09-14T23:56:38.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2652e-05 2024-09-14T23:56:38.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:38.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.410231+0000", 2024-09-14T23:56:38.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1093999999999994e-05 2024-09-14T23:56:38.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:38.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.410357+0000", 2024-09-14T23:56:38.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012558 2024-09-14T23:56:38.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-14T23:56:38.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.410515+0000", 2024-09-14T23:56:38.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000158345 2024-09-14T23:56:38.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:38.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.411377+0000", 2024-09-14T23:56:38.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00086149100000000004 2024-09-14T23:56:38.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:56:38.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.411542+0000", 2024-09-14T23:56:38.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000165186 2024-09-14T23:56:38.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:38.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.411584+0000", 2024-09-14T23:56:38.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2358999999999999e-05 2024-09-14T23:56:38.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:38.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.411624+0000", 2024-09-14T23:56:38.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9721999999999997e-05 2024-09-14T23:56:38.565 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:38.566 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:38.566 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:38.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.566 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:3483 3.1 3:821e813d:::rbd_header.1118de1168b:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-14T23:56:38.566 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:37.977981+0000", 2024-09-14T23:56:38.566 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.51399281500000005, 2024-09-14T23:56:38.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0017620960000000001, 2024-09-14T23:56:38.566 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:38.566 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:38.566 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:38.566 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:38.566 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:38.566 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:38.566 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3483 2024-09-14T23:56:38.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.567 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:38.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:38.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:37.977981+0000", 2024-09-14T23:56:38.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:38.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:38.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:37.977981+0000", 2024-09-14T23:56:38.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:38.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:38.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:37.977982+0000", 2024-09-14T23:56:38.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.443e-06 2024-09-14T23:56:38.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:38.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:37.977990+0000", 2024-09-14T23:56:38.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5660000000000001e-06 2024-09-14T23:56:38.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:38.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:37.978003+0000", 2024-09-14T23:56:38.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3363000000000001e-05 2024-09-14T23:56:38.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:38.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:37.978011+0000", 2024-09-14T23:56:38.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9160000000000001e-06 2024-09-14T23:56:38.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:38.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:37.978065+0000", 2024-09-14T23:56:38.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4029000000000001e-05 2024-09-14T23:56:38.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:38.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:37.978154+0000", 2024-09-14T23:56:38.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8792e-05 2024-09-14T23:56:38.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 4", 2024-09-14T23:56:38.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:37.978408+0000", 2024-09-14T23:56:38.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025430400000000001 2024-09-14T23:56:38.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:56:38.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:37.979018+0000", 2024-09-14T23:56:38.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00061022699999999997 2024-09-14T23:56:38.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:38.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:37.979623+0000", 2024-09-14T23:56:38.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00060421100000000001 2024-09-14T23:56:38.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:38.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:37.979688+0000", 2024-09-14T23:56:38.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5217999999999998e-05 2024-09-14T23:56:38.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:38.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:37.979743+0000", 2024-09-14T23:56:38.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5027e-05 2024-09-14T23:56:38.571 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:38.571 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:38.571 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:38.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.571 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:3485 3.17 3:e92a1031:::rbd_object_map.1118de1168b.0000000000000004:head [writefull 0~128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-14T23:56:38.571 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:37.980415+0000", 2024-09-14T23:56:38.571 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.51155893900000005, 2024-09-14T23:56:38.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0020681990000000002, 2024-09-14T23:56:38.572 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:38.572 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:38.572 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:38.572 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:38.572 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:38.572 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:38.572 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3485 2024-09-14T23:56:38.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.572 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:38.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:38.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:37.980415+0000", 2024-09-14T23:56:38.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:38.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:38.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:37.980415+0000", 2024-09-14T23:56:38.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:38.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:38.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:37.980415+0000", 2024-09-14T23:56:38.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2600000000000001e-07 2024-09-14T23:56:38.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:38.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:37.980420+0000", 2024-09-14T23:56:38.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9919999999999998e-06 2024-09-14T23:56:38.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:38.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:37.980434+0000", 2024-09-14T23:56:38.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.358e-05 2024-09-14T23:56:38.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:38.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:37.980442+0000", 2024-09-14T23:56:38.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7859999999999997e-06 2024-09-14T23:56:38.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:38.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:37.980563+0000", 2024-09-14T23:56:38.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000120888 2024-09-14T23:56:38.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:38.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:37.980704+0000", 2024-09-14T23:56:38.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014151499999999999 2024-09-14T23:56:38.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-14T23:56:38.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:37.980827+0000", 2024-09-14T23:56:38.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000122951 2024-09-14T23:56:38.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:56:38.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:37.981806+0000", 2024-09-14T23:56:38.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00097907799999999998 2024-09-14T23:56:38.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:38.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:37.982414+0000", 2024-09-14T23:56:38.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00060825400000000002 2024-09-14T23:56:38.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:38.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:37.982450+0000", 2024-09-14T23:56:38.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5187e-05 2024-09-14T23:56:38.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:38.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:37.982483+0000", 2024-09-14T23:56:38.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3142000000000002e-05 2024-09-14T23:56:38.577 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:38.577 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:38.577 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:38.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.577 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:3486 3.1 3:8172dac6:::rbd_object_map.1118de1168b:head [call lock.assert_locked in=17b,call] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-14T23:56:38.577 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:37.982591+0000", 2024-09-14T23:56:38.577 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.50938253, 2024-09-14T23:56:38.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011303039999999999, 2024-09-14T23:56:38.577 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:38.577 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:38.577 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:38.577 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:38.577 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:38.578 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:38.578 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3486 2024-09-14T23:56:38.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.578 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:38.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:38.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:37.982591+0000", 2024-09-14T23:56:38.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:38.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:38.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:37.982591+0000", 2024-09-14T23:56:38.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:38.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:38.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:37.982592+0000", 2024-09-14T23:56:38.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3600000000000002e-07 2024-09-14T23:56:38.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:38.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:37.982596+0000", 2024-09-14T23:56:38.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5279999999999997e-06 2024-09-14T23:56:38.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:38.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:37.982606+0000", 2024-09-14T23:56:38.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7869999999999996e-06 2024-09-14T23:56:38.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:38.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:37.982611+0000", 2024-09-14T23:56:38.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0839999999999999e-06 2024-09-14T23:56:38.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:38.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:37.982660+0000", 2024-09-14T23:56:38.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9184999999999997e-05 2024-09-14T23:56:38.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:38.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:37.982723+0000", 2024-09-14T23:56:38.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2792999999999993e-05 2024-09-14T23:56:38.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 4", 2024-09-14T23:56:38.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:37.982893+0000", 2024-09-14T23:56:38.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000169576 2024-09-14T23:56:38.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:56:38.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:37.983542+0000", 2024-09-14T23:56:38.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00064973400000000001 2024-09-14T23:56:38.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:38.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:37.983667+0000", 2024-09-14T23:56:38.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012413499999999999 2024-09-14T23:56:38.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:38.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:37.983697+0000", 2024-09-14T23:56:38.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0698999999999999e-05 2024-09-14T23:56:38.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:38.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:37.983721+0000", 2024-09-14T23:56:38.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3947e-05 2024-09-14T23:56:38.582 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:38.582 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:38.582 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:38.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.583 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:3497 3.1 3:821e813d:::rbd_header.1118de1168b:head [call rbd.set_protection_status in=24b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-14T23:56:38.583 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:37.988087+0000", 2024-09-14T23:56:38.583 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.50388645600000004, 2024-09-14T23:56:38.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013404459999999999, 2024-09-14T23:56:38.583 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:38.583 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:38.583 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:38.583 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:38.583 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:38.583 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:38.583 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 3497 2024-09-14T23:56:38.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.583 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:38.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:38.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:37.988087+0000", 2024-09-14T23:56:38.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:38.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:38.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:37.988087+0000", 2024-09-14T23:56:38.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:38.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:38.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:37.988088+0000", 2024-09-14T23:56:38.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5000000000000002e-07 2024-09-14T23:56:38.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:38.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:37.988092+0000", 2024-09-14T23:56:38.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0860000000000004e-06 2024-09-14T23:56:38.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:38.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:37.988102+0000", 2024-09-14T23:56:38.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0609e-05 2024-09-14T23:56:38.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:38.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:37.988107+0000", 2024-09-14T23:56:38.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8790000000000002e-06 2024-09-14T23:56:38.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:38.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:37.988157+0000", 2024-09-14T23:56:38.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0219999999999997e-05 2024-09-14T23:56:38.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:38.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:37.988244+0000", 2024-09-14T23:56:38.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6630000000000002e-05 2024-09-14T23:56:38.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 4", 2024-09-14T23:56:38.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:37.988487+0000", 2024-09-14T23:56:38.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000242795 2024-09-14T23:56:38.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:56:38.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:37.989101+0000", 2024-09-14T23:56:38.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00061378500000000005 2024-09-14T23:56:38.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:38.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:37.989352+0000", 2024-09-14T23:56:38.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00025173200000000001 2024-09-14T23:56:38.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:38.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:37.989386+0000", 2024-09-14T23:56:38.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3627999999999997e-05 2024-09-14T23:56:38.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:38.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:37.989427+0000", 2024-09-14T23:56:38.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1431999999999999e-05 2024-09-14T23:56:38.588 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:38.588 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:38.588 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:38.588 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:38.588 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:38.588 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:56:38.590 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T23:56:38.590 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:56:38.588+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:56:38.690 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:56:38.689+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:56:38.698 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:56:38.698 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:56:38.698 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:56:38.698 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:56:38.707 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T23:56:38.791 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:56:38.789+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:56:38.813 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:56:38.814 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:56:38.814 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:56:38.814 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:56:38.814 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:56:38.823 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T23:56:38.892 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:56:38.890+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:56:38.930 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:56:38.931 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:56:38.931 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:56:38.931 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:56:38.931 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.931 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4284.0:5 2.2 e20/18 2:5cea7035:::rbd_info:head v 20'1, mlcod=0'0)", 2024-09-14T23:56:38.931 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:23.532954+0000", 2024-09-14T23:56:38.931 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.396993684, 2024-09-14T23:56:38.931 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001093295, 2024-09-14T23:56:38.931 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:38.931 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:38.931 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:38.931 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:38.932 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.932 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:38.932 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:23.532954+0000", 2024-09-14T23:56:38.932 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:38.932 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.932 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.932 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:38.932 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:23.532954+0000", 2024-09-14T23:56:38.932 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:38.932 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.932 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.932 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:38.932 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:23.532954+0000", 2024-09-14T23:56:38.933 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9999999999999999e-07 2024-09-14T23:56:38.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.933 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.933 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:38.933 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:23.532960+0000", 2024-09-14T23:56:38.933 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0519999999999998e-06 2024-09-14T23:56:38.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.933 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.933 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:38.933 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:23.532979+0000", 2024-09-14T23:56:38.933 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8485999999999999e-05 2024-09-14T23:56:38.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.933 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.933 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:38.934 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:23.532992+0000", 2024-09-14T23:56:38.934 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3023000000000001e-05 2024-09-14T23:56:38.934 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.934 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.934 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:38.934 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:23.533066+0000", 2024-09-14T23:56:38.934 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3935000000000003e-05 2024-09-14T23:56:38.934 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.934 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.934 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:38.934 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:23.533093+0000", 2024-09-14T23:56:38.934 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7288999999999999e-05 2024-09-14T23:56:38.934 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.934 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.934 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:38.934 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:23.534009+0000", 2024-09-14T23:56:38.935 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000916404 2024-09-14T23:56:38.935 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.935 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.935 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:38.935 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:23.534047+0000", 2024-09-14T23:56:38.935 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7905999999999999e-05 2024-09-14T23:56:38.935 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:38.935 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:38.935 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:38.935 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.935 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.935 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4284.0:7 2.2 e21/18 2:5cea7035:::rbd_info:head v 21'2, mlcod=0'0)", 2024-09-14T23:56:38.935 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:24.537466+0000", 2024-09-14T23:56:38.935 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.392481641, 2024-09-14T23:56:38.936 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00083814299999999996, 2024-09-14T23:56:38.936 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:38.936 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:38.936 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:38.936 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:38.936 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.936 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:38.936 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:24.537466+0000", 2024-09-14T23:56:38.936 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:38.936 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.936 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.936 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:38.936 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:24.537466+0000", 2024-09-14T23:56:38.936 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:38.937 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.937 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.937 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:38.937 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:24.537466+0000", 2024-09-14T23:56:38.937 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.28e-07 2024-09-14T23:56:38.937 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.937 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.937 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:38.937 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:24.537471+0000", 2024-09-14T23:56:38.937 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5519999999999998e-06 2024-09-14T23:56:38.937 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.937 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.937 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:38.937 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:24.537487+0000", 2024-09-14T23:56:38.938 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6602000000000001e-05 2024-09-14T23:56:38.938 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.938 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.938 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:38.938 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:24.537499+0000", 2024-09-14T23:56:38.938 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1250000000000001e-05 2024-09-14T23:56:38.938 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.938 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.938 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:38.938 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:24.537569+0000", 2024-09-14T23:56:38.938 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9766999999999998e-05 2024-09-14T23:56:38.938 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.938 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.938 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:38.939 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:24.537588+0000", 2024-09-14T23:56:38.939 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9874999999999999e-05 2024-09-14T23:56:38.939 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.939 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.939 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:38.939 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:24.538234+0000", 2024-09-14T23:56:38.939 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00064588600000000003 2024-09-14T23:56:38.939 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.939 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.939 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:38.939 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:24.538304+0000", 2024-09-14T23:56:38.939 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9982999999999997e-05 2024-09-14T23:56:38.939 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:38.939 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:38.940 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:38.940 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.940 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.940 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:7 3.1a e25/23 3:5cea7035:::rbd_info:head v 25'1, mlcod=0'0)", 2024-09-14T23:56:38.940 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:33.396939+0000", 2024-09-14T23:56:38.940 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.5330091210000001, 2024-09-14T23:56:38.940 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00063904399999999996, 2024-09-14T23:56:38.940 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:38.940 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:38.940 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:38.940 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:38.940 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.940 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:38.940 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:33.396939+0000", 2024-09-14T23:56:38.941 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:38.941 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.941 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.941 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:38.941 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:33.396939+0000", 2024-09-14T23:56:38.941 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:38.941 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.941 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.941 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:38.941 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:33.396939+0000", 2024-09-14T23:56:38.942 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.73e-07 2024-09-14T23:56:38.942 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.942 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.942 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:38.942 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:33.396943+0000", 2024-09-14T23:56:38.942 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6280000000000004e-06 2024-09-14T23:56:38.942 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.942 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.942 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:38.942 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:33.396955+0000", 2024-09-14T23:56:38.942 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1800000000000001e-05 2024-09-14T23:56:38.942 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.942 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.942 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:38.942 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:33.396962+0000", 2024-09-14T23:56:38.943 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7889999999999997e-06 2024-09-14T23:56:38.943 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.943 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.943 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:38.943 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:33.397002+0000", 2024-09-14T23:56:38.943 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0479999999999999e-05 2024-09-14T23:56:38.943 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.943 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.943 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:38.943 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:33.397018+0000", 2024-09-14T23:56:38.943 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.543e-05 2024-09-14T23:56:38.943 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.943 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.943 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:38.944 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:33.397543+0000", 2024-09-14T23:56:38.944 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00052527499999999996 2024-09-14T23:56:38.944 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.944 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.944 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:38.944 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:33.397578+0000", 2024-09-14T23:56:38.944 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4468999999999998e-05 2024-09-14T23:56:38.944 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:38.944 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:38.944 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:38.944 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.944 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.944 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:9 3.1a e26/23 3:5cea7035:::rbd_info:head v 26'2, mlcod=0'0)", 2024-09-14T23:56:38.944 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:34.404833+0000", 2024-09-14T23:56:38.945 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.5251148990000001, 2024-09-14T23:56:38.945 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011562289999999999, 2024-09-14T23:56:38.945 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:38.945 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:38.945 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:38.945 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:38.945 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.945 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:38.945 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.404833+0000", 2024-09-14T23:56:38.945 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:38.945 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.945 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.945 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:38.946 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.404833+0000", 2024-09-14T23:56:38.946 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:38.946 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.946 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.946 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:38.946 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.404833+0000", 2024-09-14T23:56:38.946 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0900000000000001e-07 2024-09-14T23:56:38.946 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.946 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.946 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:38.946 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.404842+0000", 2024-09-14T23:56:38.946 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2230000000000007e-06 2024-09-14T23:56:38.946 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.946 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.946 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:38.946 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.404852+0000", 2024-09-14T23:56:38.947 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9769999999999995e-06 2024-09-14T23:56:38.947 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.947 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.947 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:38.947 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.404865+0000", 2024-09-14T23:56:38.947 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3519e-05 2024-09-14T23:56:38.947 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.947 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.947 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:38.947 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.404962+0000", 2024-09-14T23:56:38.947 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7301999999999998e-05 2024-09-14T23:56:38.947 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.947 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.947 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:38.948 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.404985+0000", 2024-09-14T23:56:38.948 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2646e-05 2024-09-14T23:56:38.948 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.948 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.948 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:38.948 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.405953+0000", 2024-09-14T23:56:38.948 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00096790799999999996 2024-09-14T23:56:38.948 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.948 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.948 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:38.948 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.405989+0000", 2024-09-14T23:56:38.948 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6044999999999998e-05 2024-09-14T23:56:38.948 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:38.948 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:38.948 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:38.949 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.949 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.949 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:13 3.8 e26/23 3:1601aefa:::rbd_object_map.1118d8884d62:head v 26'1, mlcod=0'0)", 2024-09-14T23:56:38.949 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:34.417229+0000", 2024-09-14T23:56:38.949 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.5127189899999998, 2024-09-14T23:56:38.949 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00087394399999999998, 2024-09-14T23:56:38.949 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:38.949 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:38.949 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:38.949 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:38.949 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.949 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:38.950 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.417229+0000", 2024-09-14T23:56:38.950 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:38.950 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.950 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.950 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:38.950 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.417229+0000", 2024-09-14T23:56:38.950 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:38.950 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.950 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.950 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:38.950 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.417229+0000", 2024-09-14T23:56:38.950 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7700000000000001e-07 2024-09-14T23:56:38.950 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.950 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.951 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:38.951 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.417235+0000", 2024-09-14T23:56:38.951 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9680000000000003e-06 2024-09-14T23:56:38.951 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.951 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.951 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:38.951 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.417251+0000", 2024-09-14T23:56:38.951 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6109000000000001e-05 2024-09-14T23:56:38.951 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.951 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.951 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:38.951 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.417262+0000", 2024-09-14T23:56:38.952 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.115e-05 2024-09-14T23:56:38.952 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.952 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.952 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:38.952 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.417322+0000", 2024-09-14T23:56:38.952 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9517e-05 2024-09-14T23:56:38.952 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.952 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.952 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:38.952 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.417345+0000", 2024-09-14T23:56:38.952 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3473e-05 2024-09-14T23:56:38.952 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.952 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.953 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:38.953 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.418050+0000", 2024-09-14T23:56:38.953 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00070452199999999996 2024-09-14T23:56:38.953 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:38.953 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:38.953 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:38.953 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:34.418103+0000", 2024-09-14T23:56:38.953 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2927999999999999e-05 2024-09-14T23:56:38.953 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:38.953 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:38.953 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:38.953 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:38.953 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:38.953 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:56:38.954 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T23:56:38.987 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T23:56:38.996 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:56:38.994+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:56:39.031 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone1 from 0x5b71763 (overlap 0x5b71763) to 0x924f23 2024-09-14T23:56:39.094 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:56:39.091+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:56:39.130 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:56:39.131 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:56:39.131 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:56:39.131 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:56:39.144 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T23:56:39.194 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:56:39.192+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:56:39.292 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:56:39.292 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:56:39.292 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:56:39.292 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:56:39.293 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:56:39.295 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:56:39.293+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:56:39.304 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T23:56:39.395 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:56:39.393+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:56:39.476 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:56:39.476 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:56:39.476 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:56:39.476 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:56:39.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.476 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4284.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-14T23:56:39.476 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:22.527811+0000", 2024-09-14T23:56:39.477 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 16.94549954, 2024-09-14T23:56:39.477 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00113675, 2024-09-14T23:56:39.477 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:39.477 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:39.477 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:39.477 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:39.477 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.477 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:39.477 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:22.527811+0000", 2024-09-14T23:56:39.477 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:39.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.477 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.477 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:39.477 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:22.527811+0000", 2024-09-14T23:56:39.477 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:39.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.478 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:39.478 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:22.527811+0000", 2024-09-14T23:56:39.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4799999999999998e-07 2024-09-14T23:56:39.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.478 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:39.478 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:22.527826+0000", 2024-09-14T23:56:39.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4436999999999999e-05 2024-09-14T23:56:39.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.478 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:39.478 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:22.527844+0000", 2024-09-14T23:56:39.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7878999999999999e-05 2024-09-14T23:56:39.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.479 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:39.479 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:22.527856+0000", 2024-09-14T23:56:39.479 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2089000000000001e-05 2024-09-14T23:56:39.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.479 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:39.479 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:22.527937+0000", 2024-09-14T23:56:39.479 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0852000000000004e-05 2024-09-14T23:56:39.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.479 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:39.479 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:22.527971+0000", 2024-09-14T23:56:39.479 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4069000000000002e-05 2024-09-14T23:56:39.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.480 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.480 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:39.480 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:22.528885+0000", 2024-09-14T23:56:39.480 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00091406300000000001 2024-09-14T23:56:39.480 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.480 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.480 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:39.480 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:22.528948+0000", 2024-09-14T23:56:39.480 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2812999999999996e-05 2024-09-14T23:56:39.480 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:39.480 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:39.480 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:39.480 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.480 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.481 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:3531 3.0 e29/23 3:04479396:::rbd_id.image_client.0-clone2:head v 29'7, mlcod=27'5)", 2024-09-14T23:56:39.481 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.003617+0000", 2024-09-14T23:56:39.481 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.46969349599999999, 2024-09-14T23:56:39.481 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00081455799999999995, 2024-09-14T23:56:39.481 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:39.481 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:39.481 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:39.481 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:39.481 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.481 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:39.481 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.003617+0000", 2024-09-14T23:56:39.481 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:39.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.481 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.482 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:39.482 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.003617+0000", 2024-09-14T23:56:39.482 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:39.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.482 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.482 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:39.482 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.003617+0000", 2024-09-14T23:56:39.482 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7799999999999997e-07 2024-09-14T23:56:39.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.482 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.482 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:39.482 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.003625+0000", 2024-09-14T23:56:39.482 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5009999999999998e-06 2024-09-14T23:56:39.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.483 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.483 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:39.483 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.003643+0000", 2024-09-14T23:56:39.483 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8255999999999999e-05 2024-09-14T23:56:39.483 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.483 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.483 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:39.483 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.003654+0000", 2024-09-14T23:56:39.483 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1388e-05 2024-09-14T23:56:39.483 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.483 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.483 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:39.483 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.003721+0000", 2024-09-14T23:56:39.483 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6703000000000002e-05 2024-09-14T23:56:39.483 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.484 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.484 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:39.484 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.003751+0000", 2024-09-14T23:56:39.484 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.004e-05 2024-09-14T23:56:39.484 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.484 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.484 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:39.484 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.004384+0000", 2024-09-14T23:56:39.484 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00063280200000000002 2024-09-14T23:56:39.484 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.484 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.484 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:39.484 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.004432+0000", 2024-09-14T23:56:39.484 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7589999999999997e-05 2024-09-14T23:56:39.485 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:39.485 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:39.485 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:39.485 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.485 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.485 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "pg_update_log_missing(3.4 epoch 29/23 rep_tid 20 entries 29'12 (0'0) error 3:2298d860:::rbd_data.1118de1168b.0000000000000030:head by client.4376.0:3569 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 27'10)", 2024-09-14T23:56:39.485 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.033832+0000", 2024-09-14T23:56:39.485 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.43947889000000001, 2024-09-14T23:56:39.485 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011863259999999999, 2024-09-14T23:56:39.485 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:39.485 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:39.485 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:56:39.485 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:39.485 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.485 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:39.485 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.033832+0000", 2024-09-14T23:56:39.486 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:39.486 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.486 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.486 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:39.486 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.033832+0000", 2024-09-14T23:56:39.486 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:39.486 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.486 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.486 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:39.486 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.033833+0000", 2024-09-14T23:56:39.486 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.356e-06 2024-09-14T23:56:39.486 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.486 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.486 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:39.487 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.033847+0000", 2024-09-14T23:56:39.487 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4099e-05 2024-09-14T23:56:39.487 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.487 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.487 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:39.487 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.033880+0000", 2024-09-14T23:56:39.487 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2659e-05 2024-09-14T23:56:39.487 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.487 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.487 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:39.487 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.033904+0000", 2024-09-14T23:56:39.487 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4184000000000001e-05 2024-09-14T23:56:39.487 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.487 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.487 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:39.487 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.034005+0000", 2024-09-14T23:56:39.488 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010145900000000001 2024-09-14T23:56:39.488 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.488 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.488 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:39.488 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.035018+0000", 2024-09-14T23:56:39.488 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010125690000000001 2024-09-14T23:56:39.488 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:39.488 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:39.488 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:39.488 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.488 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.488 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "pg_update_log_missing(3.4 epoch 29/23 rep_tid 21 entries 29'13 (0'0) error 3:27290fe6:::rbd_data.1118de1168b.0000000000000032:head by client.4376.0:3571 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 27'10)", 2024-09-14T23:56:39.488 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.034263+0000", 2024-09-14T23:56:39.488 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.43904706999999998, 2024-09-14T23:56:39.488 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00091825499999999998, 2024-09-14T23:56:39.489 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:39.489 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:39.489 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:56:39.489 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:39.489 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.489 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:39.489 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.034263+0000", 2024-09-14T23:56:39.489 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:39.489 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.489 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.489 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:39.489 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.034263+0000", 2024-09-14T23:56:39.489 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:39.489 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.489 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.490 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:39.490 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.034264+0000", 2024-09-14T23:56:39.490 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6499999999999998e-07 2024-09-14T23:56:39.490 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.490 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.490 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:39.490 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.034273+0000", 2024-09-14T23:56:39.490 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4360000000000002e-06 2024-09-14T23:56:39.490 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.490 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.490 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:39.490 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.034291+0000", 2024-09-14T23:56:39.490 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8383e-05 2024-09-14T23:56:39.490 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.490 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.490 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:39.491 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.034306+0000", 2024-09-14T23:56:39.491 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4942e-05 2024-09-14T23:56:39.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.491 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.491 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:39.491 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.034376+0000", 2024-09-14T23:56:39.491 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0096000000000001e-05 2024-09-14T23:56:39.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.491 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.491 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:39.491 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.035182+0000", 2024-09-14T23:56:39.491 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00080563300000000004 2024-09-14T23:56:39.491 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:39.491 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:39.491 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:39.492 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.492 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.492 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "pg_update_log_missing(3.4 epoch 29/23 rep_tid 22 entries 29'14 (0'0) error 3:2779453d:::rbd_data.1118de1168b.0000000000000034:head by client.4376.0:3573 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 27'10)", 2024-09-14T23:56:39.492 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.034720+0000", 2024-09-14T23:56:39.492 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.43859041100000001, 2024-09-14T23:56:39.492 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00083097300000000004, 2024-09-14T23:56:39.492 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:39.492 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:39.492 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:56:39.492 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:39.492 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.492 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:39.492 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.034720+0000", 2024-09-14T23:56:39.492 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:39.493 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.493 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.493 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:39.493 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.034720+0000", 2024-09-14T23:56:39.493 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:39.493 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.493 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.493 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:39.493 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.034721+0000", 2024-09-14T23:56:39.493 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9100000000000003e-07 2024-09-14T23:56:39.493 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.493 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.493 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:39.493 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.034730+0000", 2024-09-14T23:56:39.493 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1290000000000002e-06 2024-09-14T23:56:39.494 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.494 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.494 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:39.494 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.034750+0000", 2024-09-14T23:56:39.494 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0160999999999999e-05 2024-09-14T23:56:39.494 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.494 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.494 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:39.494 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.034769+0000", 2024-09-14T23:56:39.494 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8624e-05 2024-09-14T23:56:39.494 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.494 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.494 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:39.494 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.034867+0000", 2024-09-14T23:56:39.494 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8028000000000004e-05 2024-09-14T23:56:39.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.495 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.495 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:39.495 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.035551+0000", 2024-09-14T23:56:39.495 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00068424000000000002 2024-09-14T23:56:39.495 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:39.495 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:39.495 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:39.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.495 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.495 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:3578 3.4 e29/23 3:2298d860:::rbd_data.1118de1168b.0000000000000030:head v 29'15, mlcod=29'13)", 2024-09-14T23:56:39.495 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.036455+0000", 2024-09-14T23:56:39.495 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.436855308, 2024-09-14T23:56:39.495 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019449440000000001, 2024-09-14T23:56:39.495 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:39.496 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:39.496 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:39.496 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:39.496 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.496 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:39.496 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036455+0000", 2024-09-14T23:56:39.496 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:39.496 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.496 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.496 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:39.496 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036455+0000", 2024-09-14T23:56:39.496 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:39.496 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.496 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.496 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:39.497 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036456+0000", 2024-09-14T23:56:39.497 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0999999999999997e-07 2024-09-14T23:56:39.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.497 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.497 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:39.497 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036463+0000", 2024-09-14T23:56:39.497 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2200000000000003e-06 2024-09-14T23:56:39.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.497 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.497 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:39.497 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036478+0000", 2024-09-14T23:56:39.497 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4977000000000001e-05 2024-09-14T23:56:39.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.497 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.497 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:39.498 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036485+0000", 2024-09-14T23:56:39.498 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0559999999999999e-06 2024-09-14T23:56:39.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.498 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:39.498 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036693+0000", 2024-09-14T23:56:39.498 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000208051 2024-09-14T23:56:39.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.498 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:39.498 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036715+0000", 2024-09-14T23:56:39.498 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1330000000000001e-05 2024-09-14T23:56:39.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.499 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:39.499 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.038356+0000", 2024-09-14T23:56:39.499 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001641586 2024-09-14T23:56:39.499 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.499 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.499 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:39.499 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.038400+0000", 2024-09-14T23:56:39.499 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3914e-05 2024-09-14T23:56:39.499 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:39.499 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:39.499 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:39.499 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.499 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.499 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:3579 3.4 e29/23 3:27290fe6:::rbd_data.1118de1168b.0000000000000032:head v 29'16, mlcod=29'13)", 2024-09-14T23:56:39.500 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.037085+0000", 2024-09-14T23:56:39.500 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.43622571799999998, 2024-09-14T23:56:39.500 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0020019220000000002, 2024-09-14T23:56:39.500 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:39.500 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:39.500 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:39.500 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:39.500 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.500 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:39.500 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037085+0000", 2024-09-14T23:56:39.500 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:39.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.500 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.500 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:39.500 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037085+0000", 2024-09-14T23:56:39.501 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:39.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.501 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.501 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:39.501 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037085+0000", 2024-09-14T23:56:39.501 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5499999999999997e-07 2024-09-14T23:56:39.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.501 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.501 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:39.501 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037091+0000", 2024-09-14T23:56:39.501 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7869999999999998e-06 2024-09-14T23:56:39.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.501 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.501 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:39.501 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037102+0000", 2024-09-14T23:56:39.502 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.031e-05 2024-09-14T23:56:39.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.502 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.502 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:39.502 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037116+0000", 2024-09-14T23:56:39.502 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4119e-05 2024-09-14T23:56:39.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.502 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.502 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:39.502 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037219+0000", 2024-09-14T23:56:39.502 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010372799999999999 2024-09-14T23:56:39.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.502 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.502 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:39.503 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037239+0000", 2024-09-14T23:56:39.503 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9993e-05 2024-09-14T23:56:39.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.503 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.503 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:39.503 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.039046+0000", 2024-09-14T23:56:39.503 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001806666 2024-09-14T23:56:39.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.503 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.503 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:39.503 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.039087+0000", 2024-09-14T23:56:39.503 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0564e-05 2024-09-14T23:56:39.503 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:39.503 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:39.503 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:39.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.504 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.504 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:3584 3.4 e29/23 3:2779453d:::rbd_data.1118de1168b.0000000000000034:head v 29'17, mlcod=29'13)", 2024-09-14T23:56:39.504 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.037588+0000", 2024-09-14T23:56:39.504 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.435722679, 2024-09-14T23:56:39.504 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001526828, 2024-09-14T23:56:39.504 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:39.504 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:39.504 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:39.504 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:39.504 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.504 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:39.504 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037588+0000", 2024-09-14T23:56:39.504 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:39.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.504 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.504 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:39.505 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037588+0000", 2024-09-14T23:56:39.505 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:39.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.505 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.505 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:39.505 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037588+0000", 2024-09-14T23:56:39.505 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1200000000000002e-07 2024-09-14T23:56:39.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.505 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.505 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:39.505 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037595+0000", 2024-09-14T23:56:39.505 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7580000000000003e-06 2024-09-14T23:56:39.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.505 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.505 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:39.505 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037609+0000", 2024-09-14T23:56:39.506 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4037e-05 2024-09-14T23:56:39.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.506 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:39.506 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037616+0000", 2024-09-14T23:56:39.506 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0509999999999999e-06 2024-09-14T23:56:39.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.506 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:39.506 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037787+0000", 2024-09-14T23:56:39.506 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017084899999999999 2024-09-14T23:56:39.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.506 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:39.506 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037806+0000", 2024-09-14T23:56:39.506 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9159000000000001e-05 2024-09-14T23:56:39.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.507 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.507 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:39.507 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.039089+0000", 2024-09-14T23:56:39.507 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012823909999999999 2024-09-14T23:56:39.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.507 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.507 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:39.507 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.039115+0000", 2024-09-14T23:56:39.507 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5871e-05 2024-09-14T23:56:39.507 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:39.507 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:39.507 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:39.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.507 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.507 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "pg_update_log_missing(3.0 epoch 29/23 rep_tid 28 entries 29'8 (0'0) error 3:012207e9:::rbd_data.1118de1168b.000000000000004f:head by client.4376.0:3631 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 27'6)", 2024-09-14T23:56:39.508 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.049933+0000", 2024-09-14T23:56:39.508 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.42337743100000003, 2024-09-14T23:56:39.508 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00075187500000000005, 2024-09-14T23:56:39.508 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:39.508 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:39.508 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:56:39.508 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:39.508 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.508 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:39.508 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.049933+0000", 2024-09-14T23:56:39.508 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:39.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.508 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.508 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:39.508 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.049933+0000", 2024-09-14T23:56:39.509 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:39.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.509 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.509 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:39.509 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.049934+0000", 2024-09-14T23:56:39.509 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1289999999999999e-06 2024-09-14T23:56:39.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.509 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.509 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:39.509 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.049947+0000", 2024-09-14T23:56:39.509 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2979000000000001e-05 2024-09-14T23:56:39.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.509 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.509 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:39.509 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.049976+0000", 2024-09-14T23:56:39.509 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9300000000000001e-05 2024-09-14T23:56:39.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.510 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:39.510 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.049996+0000", 2024-09-14T23:56:39.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9117e-05 2024-09-14T23:56:39.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.510 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:39.510 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.050084+0000", 2024-09-14T23:56:39.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8805000000000001e-05 2024-09-14T23:56:39.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.510 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:39.510 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.050685+0000", 2024-09-14T23:56:39.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00060054499999999996 2024-09-14T23:56:39.510 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:39.511 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:39.511 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:39.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.511 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "pg_update_log_missing(3.4 epoch 29/23 rep_tid 30 entries 29'18 (0'0) error 3:2196168c:::rbd_data.1118de1168b.0000000000000056:head by client.4376.0:3642 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 29'16)", 2024-09-14T23:56:39.511 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.053057+0000", 2024-09-14T23:56:39.511 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.42025386599999998, 2024-09-14T23:56:39.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00080079700000000001, 2024-09-14T23:56:39.511 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:39.511 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:39.511 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:56:39.511 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:39.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.511 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:39.511 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.053057+0000", 2024-09-14T23:56:39.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:39.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.512 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:39.512 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.053057+0000", 2024-09-14T23:56:39.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:39.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.512 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:39.512 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.053057+0000", 2024-09-14T23:56:39.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.92e-07 2024-09-14T23:56:39.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.512 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:39.512 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.053066+0000", 2024-09-14T23:56:39.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3610000000000001e-06 2024-09-14T23:56:39.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.513 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:39.513 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.053089+0000", 2024-09-14T23:56:39.513 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3689999999999998e-05 2024-09-14T23:56:39.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.513 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:39.513 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.053105+0000", 2024-09-14T23:56:39.513 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6021000000000001e-05 2024-09-14T23:56:39.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.513 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:39.513 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.053210+0000", 2024-09-14T23:56:39.513 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010480999999999999 2024-09-14T23:56:39.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:39.514 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.053857+0000", 2024-09-14T23:56:39.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00064722300000000005 2024-09-14T23:56:39.514 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:39.514 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:39.514 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:39.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.514 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:3651 3.4 e29/23 3:2196168c:::rbd_data.1118de1168b.0000000000000056:head v 29'19, mlcod=29'17)", 2024-09-14T23:56:39.514 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.054929+0000", 2024-09-14T23:56:39.514 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.41838112399999999, 2024-09-14T23:56:39.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00093062400000000001, 2024-09-14T23:56:39.514 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:39.514 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:39.515 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:39.515 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:39.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.515 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:39.515 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.054929+0000", 2024-09-14T23:56:39.515 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:39.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.515 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:39.515 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.054929+0000", 2024-09-14T23:56:39.515 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:39.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.515 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:39.515 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.054930+0000", 2024-09-14T23:56:39.515 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2799999999999995e-07 2024-09-14T23:56:39.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.516 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:39.516 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.054950+0000", 2024-09-14T23:56:39.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9670999999999999e-05 2024-09-14T23:56:39.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.516 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:39.516 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.054963+0000", 2024-09-14T23:56:39.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3502e-05 2024-09-14T23:56:39.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.516 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:39.516 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.054970+0000", 2024-09-14T23:56:39.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1040000000000001e-06 2024-09-14T23:56:39.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.517 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:39.517 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.055026+0000", 2024-09-14T23:56:39.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5909000000000003e-05 2024-09-14T23:56:39.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.517 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:39.517 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.055047+0000", 2024-09-14T23:56:39.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0511999999999999e-05 2024-09-14T23:56:39.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.517 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:39.517 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.055819+0000", 2024-09-14T23:56:39.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00077175300000000002 2024-09-14T23:56:39.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.518 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:39.518 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.055860+0000", 2024-09-14T23:56:39.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1344999999999998e-05 2024-09-14T23:56:39.518 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:39.518 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:39.518 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:39.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.518 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:5280 3.0 e29/23 3:0760f535:::rbd_data.1118de1168b.000000000000009a:head v 29'11, mlcod=29'9)", 2024-09-14T23:56:39.518 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.262702+0000", 2024-09-14T23:56:39.518 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.210608299, 2024-09-14T23:56:39.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001433122, 2024-09-14T23:56:39.519 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:39.519 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:39.519 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:39.519 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:39.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.519 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:39.519 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262702+0000", 2024-09-14T23:56:39.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:39.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.519 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:39.519 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262702+0000", 2024-09-14T23:56:39.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:39.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.520 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:39.520 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262704+0000", 2024-09-14T23:56:39.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.384e-06 2024-09-14T23:56:39.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.520 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:39.520 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262791+0000", 2024-09-14T23:56:39.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7264999999999999e-05 2024-09-14T23:56:39.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.520 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:39.520 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262813+0000", 2024-09-14T23:56:39.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.173e-05 2024-09-14T23:56:39.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.521 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:39.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262822+0000", 2024-09-14T23:56:39.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5929999999999995e-06 2024-09-14T23:56:39.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.521 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:39.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262886+0000", 2024-09-14T23:56:39.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4052000000000003e-05 2024-09-14T23:56:39.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.521 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:39.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262920+0000", 2024-09-14T23:56:39.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.366e-05 2024-09-14T23:56:39.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:39.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.264095+0000", 2024-09-14T23:56:39.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001175178 2024-09-14T23:56:39.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:39.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.264135+0000", 2024-09-14T23:56:39.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0259999999999997e-05 2024-09-14T23:56:39.522 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:39.522 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:39.522 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:39.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.522 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:5281 3.0 e29/23 3:0760f535:::rbd_data.1118de1168b.000000000000009a:head v 29'13, mlcod=29'9)", 2024-09-14T23:56:39.523 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.263503+0000", 2024-09-14T23:56:39.523 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.20980743499999999, 2024-09-14T23:56:39.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012834910000000001, 2024-09-14T23:56:39.523 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:39.523 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:39.523 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:39.523 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:39.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:39.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.263503+0000", 2024-09-14T23:56:39.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:39.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:39.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.263503+0000", 2024-09-14T23:56:39.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:39.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:39.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.263504+0000", 2024-09-14T23:56:39.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4600000000000004e-07 2024-09-14T23:56:39.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:39.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.263510+0000", 2024-09-14T23:56:39.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5950000000000004e-06 2024-09-14T23:56:39.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:39.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.263522+0000", 2024-09-14T23:56:39.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1289e-05 2024-09-14T23:56:39.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:39.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.263529+0000", 2024-09-14T23:56:39.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.951e-06 2024-09-14T23:56:39.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:39.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.263656+0000", 2024-09-14T23:56:39.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012731099999999999 2024-09-14T23:56:39.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.526 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:39.526 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.263675+0000", 2024-09-14T23:56:39.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9147000000000002e-05 2024-09-14T23:56:39.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.526 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:39.526 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.264746+0000", 2024-09-14T23:56:39.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001070997 2024-09-14T23:56:39.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.526 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:39.526 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.264787+0000", 2024-09-14T23:56:39.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0555e-05 2024-09-14T23:56:39.526 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:39.526 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:39.526 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:39.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.527 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:5599 3.4 e29/23 3:248aa036:::rbd_data.1118de1168b.00000000000000e2:head v 29'21, mlcod=29'18)", 2024-09-14T23:56:39.527 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.355560+0000", 2024-09-14T23:56:39.527 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.117750333, 2024-09-14T23:56:39.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001151369, 2024-09-14T23:56:39.527 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:39.527 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:39.527 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:39.527 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:39.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:39.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.355560+0000", 2024-09-14T23:56:39.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:39.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:39.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.355560+0000", 2024-09-14T23:56:39.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:39.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:39.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.355562+0000", 2024-09-14T23:56:39.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.409e-06 2024-09-14T23:56:39.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:39.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.355573+0000", 2024-09-14T23:56:39.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1453e-05 2024-09-14T23:56:39.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:39.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.355593+0000", 2024-09-14T23:56:39.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0290999999999999e-05 2024-09-14T23:56:39.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:39.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.355604+0000", 2024-09-14T23:56:39.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0983e-05 2024-09-14T23:56:39.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:39.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.355674+0000", 2024-09-14T23:56:39.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0233000000000003e-05 2024-09-14T23:56:39.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:39.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.355696+0000", 2024-09-14T23:56:39.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.181e-05 2024-09-14T23:56:39.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:39.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.356670+0000", 2024-09-14T23:56:39.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00097364299999999995 2024-09-14T23:56:39.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:39.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.356711+0000", 2024-09-14T23:56:39.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1547e-05 2024-09-14T23:56:39.531 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:39.531 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:39.531 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:39.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.531 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:6082 3.9 e29/23 3:97525414:::rbd_data.1118de1168b.000000000000010c:head v 29'6, mlcod=29'4)", 2024-09-14T23:56:39.531 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.411014+0000", 2024-09-14T23:56:39.531 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.062296307000000002, 2024-09-14T23:56:39.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010181719999999999, 2024-09-14T23:56:39.531 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:39.531 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:39.531 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:39.531 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:39.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:39.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.411014+0000", 2024-09-14T23:56:39.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:39.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:39.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.411014+0000", 2024-09-14T23:56:39.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:39.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:39.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.411015+0000", 2024-09-14T23:56:39.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9699999999999995e-07 2024-09-14T23:56:39.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:39.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.411021+0000", 2024-09-14T23:56:39.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2809999999999997e-06 2024-09-14T23:56:39.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:39.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.411035+0000", 2024-09-14T23:56:39.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3471e-05 2024-09-14T23:56:39.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:39.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.411042+0000", 2024-09-14T23:56:39.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3640000000000001e-06 2024-09-14T23:56:39.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:39.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.411085+0000", 2024-09-14T23:56:39.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3408999999999997e-05 2024-09-14T23:56:39.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:39.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.411104+0000", 2024-09-14T23:56:39.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8139e-05 2024-09-14T23:56:39.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:39.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.411987+0000", 2024-09-14T23:56:39.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00088333400000000005 2024-09-14T23:56:39.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:39.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.412032+0000", 2024-09-14T23:56:39.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5476999999999998e-05 2024-09-14T23:56:39.535 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:39.535 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:39.535 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:39.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.535 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:6097 3.0 e29/23 3:03b86c80:::rbd_data.1118de1168b.0000000000000115:head v 29'15, mlcod=29'13)", 2024-09-14T23:56:39.535 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.419107+0000", 2024-09-14T23:56:39.535 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.054203225000000001, 2024-09-14T23:56:39.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00078201099999999999, 2024-09-14T23:56:39.536 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:39.536 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:39.536 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:39.536 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:39.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:39.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.419107+0000", 2024-09-14T23:56:39.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:39.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:39.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.419107+0000", 2024-09-14T23:56:39.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:39.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:39.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.419108+0000", 2024-09-14T23:56:39.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0299999999999998e-07 2024-09-14T23:56:39.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:39.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.419115+0000", 2024-09-14T23:56:39.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4819999999999999e-06 2024-09-14T23:56:39.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:39.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.419130+0000", 2024-09-14T23:56:39.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5130000000000001e-05 2024-09-14T23:56:39.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:39.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.419137+0000", 2024-09-14T23:56:39.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7309999999999993e-06 2024-09-14T23:56:39.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:39.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.419197+0000", 2024-09-14T23:56:39.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9806999999999998e-05 2024-09-14T23:56:39.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:39.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.419216+0000", 2024-09-14T23:56:39.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8787000000000002e-05 2024-09-14T23:56:39.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:39.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.419848+0000", 2024-09-14T23:56:39.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00063250000000000003 2024-09-14T23:56:39.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:39.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.419889+0000", 2024-09-14T23:56:39.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0771e-05 2024-09-14T23:56:39.539 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:39.539 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:39.539 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:39.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.540 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "pg_update_log_missing(3.0 epoch 29/23 rep_tid 54 entries 29'16 (0'0) error 3:03a09414:::rbd_data.1118de1168b.0000000000000124:head by client.4376.0:6116 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 29'14)", 2024-09-14T23:56:39.540 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.437451+0000", 2024-09-14T23:56:39.540 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.035859672000000002, 2024-09-14T23:56:39.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00080486800000000003, 2024-09-14T23:56:39.540 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:39.540 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:39.540 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:56:39.540 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:39.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:39.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.437451+0000", 2024-09-14T23:56:39.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:39.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:39.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.437451+0000", 2024-09-14T23:56:39.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:39.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:39.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.437452+0000", 2024-09-14T23:56:39.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0139999999999999e-06 2024-09-14T23:56:39.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:39.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.437465+0000", 2024-09-14T23:56:39.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2799000000000001e-05 2024-09-14T23:56:39.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:39.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.437493+0000", 2024-09-14T23:56:39.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7905000000000001e-05 2024-09-14T23:56:39.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:39.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.437509+0000", 2024-09-14T23:56:39.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6891000000000001e-05 2024-09-14T23:56:39.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:39.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.437597+0000", 2024-09-14T23:56:39.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8090000000000005e-05 2024-09-14T23:56:39.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:39.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.438256+0000", 2024-09-14T23:56:39.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00065816900000000001 2024-09-14T23:56:39.543 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:39.543 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:39.543 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:39.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.544 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "pg_update_log_missing(3.9 epoch 29/23 rep_tid 123 entries 29'7 (0'0) error 3:96b6d439:::rbd_data.1118de1168b.0000000000000130:head by client.4376.0:6140 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 29'5)", 2024-09-14T23:56:39.544 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.444016+0000", 2024-09-14T23:56:39.544 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.029294772, 2024-09-14T23:56:39.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00084171599999999995, 2024-09-14T23:56:39.544 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:39.544 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:39.544 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:56:39.544 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:39.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:39.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.444016+0000", 2024-09-14T23:56:39.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:39.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:39.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.444016+0000", 2024-09-14T23:56:39.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:39.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:39.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.444017+0000", 2024-09-14T23:56:39.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2799999999999995e-07 2024-09-14T23:56:39.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:39.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.444025+0000", 2024-09-14T23:56:39.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0509999999999997e-06 2024-09-14T23:56:39.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:39.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.444042+0000", 2024-09-14T23:56:39.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7779999999999999e-05 2024-09-14T23:56:39.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:39.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.444053+0000", 2024-09-14T23:56:39.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0604e-05 2024-09-14T23:56:39.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:39.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.444139+0000", 2024-09-14T23:56:39.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6046999999999998e-05 2024-09-14T23:56:39.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:39.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.444857+0000", 2024-09-14T23:56:39.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00071840599999999999 2024-09-14T23:56:39.547 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:39.548 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:39.548 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:39.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.548 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:6147 3.9 e29/23 3:96b6d439:::rbd_data.1118de1168b.0000000000000130:head v 29'8, mlcod=29'6)", 2024-09-14T23:56:39.548 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.446075+0000", 2024-09-14T23:56:39.548 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.027235031, 2024-09-14T23:56:39.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00081335000000000001, 2024-09-14T23:56:39.548 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:39.548 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:39.548 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:39.548 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:39.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:39.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.446075+0000", 2024-09-14T23:56:39.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:39.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:39.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.446075+0000", 2024-09-14T23:56:39.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:39.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:39.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.446076+0000", 2024-09-14T23:56:39.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7899999999999997e-07 2024-09-14T23:56:39.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:39.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.446083+0000", 2024-09-14T23:56:39.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0480000000000001e-06 2024-09-14T23:56:39.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:39.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.446097+0000", 2024-09-14T23:56:39.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3793e-05 2024-09-14T23:56:39.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:39.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.446104+0000", 2024-09-14T23:56:39.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3590000000000002e-06 2024-09-14T23:56:39.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:39.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.446241+0000", 2024-09-14T23:56:39.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013681699999999999 2024-09-14T23:56:39.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:39.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.446261+0000", 2024-09-14T23:56:39.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9972000000000001e-05 2024-09-14T23:56:39.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:39.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.446846+0000", 2024-09-14T23:56:39.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000585281 2024-09-14T23:56:39.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:39.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.446889+0000", 2024-09-14T23:56:39.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2301e-05 2024-09-14T23:56:39.552 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:39.552 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:39.552 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:39.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.552 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:6166 3.9 e29/23 3:9572db23:::rbd_data.1118de1168b.000000000000013a:head v 29'10, mlcod=29'8)", 2024-09-14T23:56:39.552 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.452465+0000", 2024-09-14T23:56:39.552 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.020845266000000001, 2024-09-14T23:56:39.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000763282, 2024-09-14T23:56:39.552 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:39.552 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:39.553 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:39.553 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:39.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:39.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.452465+0000", 2024-09-14T23:56:39.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:39.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:39.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.452465+0000", 2024-09-14T23:56:39.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:39.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:39.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.452466+0000", 2024-09-14T23:56:39.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8199999999999999e-07 2024-09-14T23:56:39.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:39.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.452472+0000", 2024-09-14T23:56:39.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5769999999999999e-06 2024-09-14T23:56:39.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:39.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.452486+0000", 2024-09-14T23:56:39.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3957e-05 2024-09-14T23:56:39.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:39.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.452497+0000", 2024-09-14T23:56:39.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0699e-05 2024-09-14T23:56:39.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:39.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.452546+0000", 2024-09-14T23:56:39.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8576999999999999e-05 2024-09-14T23:56:39.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:39.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.452566+0000", 2024-09-14T23:56:39.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0225e-05 2024-09-14T23:56:39.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:39.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.453183+0000", 2024-09-14T23:56:39.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00061722800000000005 2024-09-14T23:56:39.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:39.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:39.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:39.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.453228+0000", 2024-09-14T23:56:39.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5337000000000003e-05 2024-09-14T23:56:39.556 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:39.556 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:39.556 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:39.556 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:39.556 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:39.556 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:56:39.558 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T23:56:39.558 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-09-14T23:56:39.559 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:56:39.555+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:56:39.658 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:56:39.656+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:56:39.678 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:56:39.678 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:56:39.678 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:56:39.678 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:56:39.687 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T23:56:39.771 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:56:39.770+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:56:39.783 INFO:teuthology.orchestra.run.smithi042.stdout:87 trunc from 0x5b71763 to 0xdd0d15 2024-09-14T23:56:39.861 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:56:39.859+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:56:39.871 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:56:39.871 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:56:39.871 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:56:39.871 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:56:39.871 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:56:39.883 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T23:56:39.927 INFO:teuthology.orchestra.run.smithi042.stdout:88 trunc from 0xdd0d15 to 0xbe6098b 2024-09-14T23:56:39.933 INFO:teuthology.orchestra.run.smithi042.stdout:89 trunc from 0xbe6098b to 0xacbe6fb 2024-09-14T23:56:39.939 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:39.939 INFO:teuthology.orchestra.run.smithi042.stdout:92 trunc from 0xacbe6fb to 0xb413167 2024-09-14T23:56:39.942 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T23:56:39.956 INFO:teuthology.orchestra.run.smithi042.stdout:93 write 0xdedb07a thru 0xdee9176 (0xe0fd bytes) 2024-09-14T23:56:39.961 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:56:39.960+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:56:39.966 INFO:teuthology.orchestra.run.smithi042.stdout:94 writesame 0xedd1c69 thru 0xedddb24 data_size 0x1a(0xbebc bytes) 2024-09-14T23:56:39.967 INFO:teuthology.orchestra.run.smithi042.stdout:95 write 0x9ea671a thru 0x9eb0a9a (0xa381 bytes) 2024-09-14T23:56:39.970 INFO:teuthology.orchestra.run.smithi042.stdout:truncating to largest ever: 0xd19f2ac 2024-09-14T23:56:39.972 INFO:teuthology.orchestra.run.smithi042.stdout:97 trunc from 0xedddb25 to 0xd19f2ac 2024-09-14T23:56:39.979 INFO:teuthology.orchestra.run.smithi042.stdout:98 punch from 0x69543d2 to 0x696279f, (0xe3cd bytes) 2024-09-14T23:56:39.988 INFO:teuthology.orchestra.run.smithi042.stdout:99 trunc from 0xd19f2ac to 0x6100b6e 2024-09-14T23:56:39.991 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is unset 2024-09-14T23:56:40.003 INFO:teuthology.orchestra.run.smithi042.stdout:102 read 0x2b148d4 thru 0x2b1f77e (0xaeab bytes) 2024-09-14T23:56:40.003 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:40.003 INFO:teuthology.orchestra.run.smithi042.stdout:104 read 0x26e85fd thru 0x26ea8b4 (0x22b8 bytes) 2024-09-14T23:56:40.003 INFO:teuthology.orchestra.run.smithi042.stdout:106 write 0x4006cfa thru 0x400b52e (0x4835 bytes) 2024-09-14T23:56:40.009 INFO:teuthology.orchestra.run.smithi042.stdout:108 trunc from 0x6100b6e to 0x80ae494 2024-09-14T23:56:40.014 INFO:teuthology.orchestra.run.smithi042.stdout:109 punch from 0x6bbcb28 to 0x6bcab67, (0xe03f bytes) 2024-09-14T23:56:40.014 INFO:teuthology.orchestra.run.smithi042.stdout:110 read 0x6330c26 thru 0x6337faa (0x7385 bytes) 2024-09-14T23:56:40.023 INFO:teuthology.orchestra.run.smithi042.stdout:111 writesame 0xbc27c70 thru 0xbc2f3b6 data_size 0x5(0x7747 bytes) 2024-09-14T23:56:40.023 INFO:teuthology.orchestra.run.smithi042.stdout:112 writesame 0x5a10212 thru 0x5a12e60 data_size 0x13(0x2c4f bytes) 2024-09-14T23:56:40.026 INFO:teuthology.orchestra.run.smithi042.stdout:113 write 0x2ed7df8 thru 0x2ee5bc6 (0xddcf bytes) 2024-09-14T23:56:40.034 INFO:teuthology.orchestra.run.smithi042.stdout:114 write 0xcc784c0 thru 0xcc8345e (0xaf9f bytes) 2024-09-14T23:56:40.038 INFO:teuthology.orchestra.run.smithi042.stdout:115 read 0x85dbf9f thru 0x85e3677 (0x76d9 bytes) 2024-09-14T23:56:40.038 INFO:teuthology.orchestra.run.smithi042.stdout:116 write 0xb374044 thru 0xb3741af (0x16c bytes) 2024-09-14T23:56:40.048 INFO:teuthology.orchestra.run.smithi042.stdout:118 writesame 0xdf4ee3d thru 0xdf50620 data_size 0x16(0x17e4 bytes) 2024-09-14T23:56:40.049 INFO:teuthology.orchestra.run.smithi042.stdout:119 write 0xd2999d5 thru 0xd29fcc5 (0x62f1 bytes) 2024-09-14T23:56:40.051 INFO:teuthology.orchestra.run.smithi042.stdout:121 read 0xab19a68 thru 0xab2691f (0xceb8 bytes) 2024-09-14T23:56:40.052 INFO:teuthology.orchestra.run.smithi042.stdout:122 read 0xb40d718 thru 0xb41b4c6 (0xddaf bytes) 2024-09-14T23:56:40.052 INFO:teuthology.orchestra.run.smithi042.stdout:123 read 0xc7b0af1 thru 0xc7bda70 (0xcf80 bytes) 2024-09-14T23:56:40.052 INFO:teuthology.orchestra.run.smithi042.stdout:124 read 0xd5a0828 thru 0xd5a171e (0xef7 bytes) 2024-09-14T23:56:40.053 INFO:teuthology.orchestra.run.smithi042.stdout:126 writesame 0x3eb1226 thru 0x3ebae9d data_size 0xc(0x9c78 bytes) 2024-09-14T23:56:40.056 INFO:teuthology.orchestra.run.smithi042.stdout:127 read 0xcf96795 thru 0xcf9e19e (0x7a0a bytes) 2024-09-14T23:56:40.057 INFO:teuthology.orchestra.run.smithi042.stdout:128 write 0x277a1b2 thru 0x2788f59 (0xeda8 bytes) 2024-09-14T23:56:40.062 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:56:40.062 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:56:40.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:56:40.062 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:56:40.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.062 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5180 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3883008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:40.062 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.250058+0000", 2024-09-14T23:56:40.062 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.80845932300000001, 2024-09-14T23:56:40.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057351471000000001, 2024-09-14T23:56:40.063 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:40.063 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:40.063 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:40.063 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:40.063 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:40.063 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:40.063 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5180 2024-09-14T23:56:40.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.063 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:40.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:40.063 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250058+0000", 2024-09-14T23:56:40.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:40.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.064 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:40.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250058+0000", 2024-09-14T23:56:40.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:40.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.064 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:40.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250786+0000", 2024-09-14T23:56:40.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00072813399999999996 2024-09-14T23:56:40.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.064 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:40.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250789+0000", 2024-09-14T23:56:40.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.647e-06 2024-09-14T23:56:40.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:40.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250798+0000", 2024-09-14T23:56:40.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0000000000000002e-06 2024-09-14T23:56:40.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:40.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250803+0000", 2024-09-14T23:56:40.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.301e-06 2024-09-14T23:56:40.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:40.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.307234+0000", 2024-09-14T23:56:40.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056431165999999998 2024-09-14T23:56:40.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:40.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.307321+0000", 2024-09-14T23:56:40.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6831999999999998e-05 2024-09-14T23:56:40.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:40.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.307409+0000", 2024-09-14T23:56:40.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8391000000000001e-05 2024-09-14T23:56:40.067 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:40.067 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:40.067 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:40.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.067 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5184 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 4128768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:40.067 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.250851+0000", 2024-09-14T23:56:40.067 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.80766585199999996, 2024-09-14T23:56:40.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057319038000000003, 2024-09-14T23:56:40.067 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:40.067 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:40.067 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:40.067 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:40.067 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:40.068 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:40.068 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5184 2024-09-14T23:56:40.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.068 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:40.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:40.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250851+0000", 2024-09-14T23:56:40.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:40.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:40.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250851+0000", 2024-09-14T23:56:40.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:40.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:40.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.251900+0000", 2024-09-14T23:56:40.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001048577 2024-09-14T23:56:40.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:40.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.251904+0000", 2024-09-14T23:56:40.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9580000000000001e-06 2024-09-14T23:56:40.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:40.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.251914+0000", 2024-09-14T23:56:40.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0186999999999999e-05 2024-09-14T23:56:40.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:40.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.251927+0000", 2024-09-14T23:56:40.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3593e-05 2024-09-14T23:56:40.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:40.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.308020+0000", 2024-09-14T23:56:40.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056092867999999997 2024-09-14T23:56:40.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:40.071 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.308094+0000", 2024-09-14T23:56:40.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3533000000000002e-05 2024-09-14T23:56:40.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:40.071 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.308170+0000", 2024-09-14T23:56:40.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6322000000000006e-05 2024-09-14T23:56:40.071 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:40.071 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:40.071 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:40.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.071 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5196 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 1925120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:40.071 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.252157+0000", 2024-09-14T23:56:40.072 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.80635973800000005, 2024-09-14T23:56:40.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057285801999999997, 2024-09-14T23:56:40.072 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:40.072 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:40.072 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:40.072 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:40.072 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:40.072 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:40.072 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5196 2024-09-14T23:56:40.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.073 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:40.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:40.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.252157+0000", 2024-09-14T23:56:40.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:40.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:40.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.252157+0000", 2024-09-14T23:56:40.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:40.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:40.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253243+0000", 2024-09-14T23:56:40.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010861340000000001 2024-09-14T23:56:40.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:40.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253248+0000", 2024-09-14T23:56:40.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8799999999999999e-06 2024-09-14T23:56:40.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:40.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253271+0000", 2024-09-14T23:56:40.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2836e-05 2024-09-14T23:56:40.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:40.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253281+0000", 2024-09-14T23:56:40.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6520000000000001e-06 2024-09-14T23:56:40.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:40.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.309336+0000", 2024-09-14T23:56:40.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056054983000000003 2024-09-14T23:56:40.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:40.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.309389+0000", 2024-09-14T23:56:40.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3298999999999999e-05 2024-09-14T23:56:40.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:40.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.309443+0000", 2024-09-14T23:56:40.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4017999999999997e-05 2024-09-14T23:56:40.076 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:40.076 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:40.076 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:40.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.076 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5200 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2170880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:40.076 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253394+0000", 2024-09-14T23:56:40.076 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.80512296100000003, 2024-09-14T23:56:40.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057428154000000002, 2024-09-14T23:56:40.076 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:40.077 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:40.077 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:40.077 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:40.077 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:40.077 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:40.077 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5200 2024-09-14T23:56:40.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.077 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:40.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:40.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253394+0000", 2024-09-14T23:56:40.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:40.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:40.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253394+0000", 2024-09-14T23:56:40.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:40.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:40.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253395+0000", 2024-09-14T23:56:40.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9100000000000004e-07 2024-09-14T23:56:40.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:40.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253397+0000", 2024-09-14T23:56:40.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2790000000000001e-06 2024-09-14T23:56:40.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:40.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253404+0000", 2024-09-14T23:56:40.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7959999999999998e-06 2024-09-14T23:56:40.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:40.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253409+0000", 2024-09-14T23:56:40.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1460000000000004e-06 2024-09-14T23:56:40.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:40.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310627+0000", 2024-09-14T23:56:40.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057218160999999997 2024-09-14T23:56:40.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:40.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310710+0000", 2024-09-14T23:56:40.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3102999999999994e-05 2024-09-14T23:56:40.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:40.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310822+0000", 2024-09-14T23:56:40.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000112178 2024-09-14T23:56:40.080 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:40.080 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:40.080 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:40.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.081 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5201 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2232320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:40.081 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:56:40.081 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.80509828900000002, 2024-09-14T23:56:40.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057643228999999997, 2024-09-14T23:56:40.081 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:40.081 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:40.081 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:40.081 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:40.081 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:40.081 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:40.081 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5201 2024-09-14T23:56:40.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.081 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:40.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:40.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:56:40.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:40.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:40.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:56:40.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:40.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:40.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:56:40.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0699999999999998e-07 2024-09-14T23:56:40.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:40.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253422+0000", 2024-09-14T23:56:40.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.514e-06 2024-09-14T23:56:40.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:40.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253431+0000", 2024-09-14T23:56:40.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1919999999999994e-06 2024-09-14T23:56:40.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:40.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253436+0000", 2024-09-14T23:56:40.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5350000000000004e-06 2024-09-14T23:56:40.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:40.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310856+0000", 2024-09-14T23:56:40.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057419381999999998 2024-09-14T23:56:40.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:40.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310953+0000", 2024-09-14T23:56:40.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6774000000000006e-05 2024-09-14T23:56:40.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:40.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311062+0000", 2024-09-14T23:56:40.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000109425 2024-09-14T23:56:40.085 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:40.085 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:40.085 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:40.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.085 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5202 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2293760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:40.085 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:56:40.085 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.80507354099999995, 2024-09-14T23:56:40.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057818255999999998, 2024-09-14T23:56:40.085 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:40.085 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:40.086 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:40.086 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:40.086 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:40.086 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:40.086 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5202 2024-09-14T23:56:40.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.086 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:40.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:40.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:56:40.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:40.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:40.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:56:40.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:40.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:40.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253573+0000", 2024-09-14T23:56:40.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012934000000000001 2024-09-14T23:56:40.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:40.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253576+0000", 2024-09-14T23:56:40.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5310000000000002e-06 2024-09-14T23:56:40.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:40.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253587+0000", 2024-09-14T23:56:40.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0522e-05 2024-09-14T23:56:40.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:40.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253595+0000", 2024-09-14T23:56:40.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8290000000000008e-06 2024-09-14T23:56:40.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:40.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311093+0000", 2024-09-14T23:56:40.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057498312000000003 2024-09-14T23:56:40.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:40.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311167+0000", 2024-09-14T23:56:40.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3582000000000001e-05 2024-09-14T23:56:40.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:40.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311262+0000", 2024-09-14T23:56:40.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5140000000000001e-05 2024-09-14T23:56:40.089 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:40.089 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:40.090 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:40.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.090 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5203 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2355200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:40.090 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:56:40.090 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.80491079899999995, 2024-09-14T23:56:40.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057903938000000002, 2024-09-14T23:56:40.090 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:40.090 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:40.090 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:40.090 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:40.090 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:40.090 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:40.090 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5203 2024-09-14T23:56:40.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.091 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:40.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:40.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:56:40.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:40.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:40.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:56:40.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:40.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:40.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253902+0000", 2024-09-14T23:56:40.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029615400000000002 2024-09-14T23:56:40.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:40.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253906+0000", 2024-09-14T23:56:40.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9249999999999997e-06 2024-09-14T23:56:40.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:40.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253913+0000", 2024-09-14T23:56:40.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9099999999999999e-06 2024-09-14T23:56:40.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:40.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253917+0000", 2024-09-14T23:56:40.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9210000000000002e-06 2024-09-14T23:56:40.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:40.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311295+0000", 2024-09-14T23:56:40.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057378268000000003 2024-09-14T23:56:40.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:40.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311402+0000", 2024-09-14T23:56:40.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010644100000000001 2024-09-14T23:56:40.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:40.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311510+0000", 2024-09-14T23:56:40.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108319 2024-09-14T23:56:40.094 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:40.094 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:40.094 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:40.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.094 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5204 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2416640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:40.094 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:56:40.094 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.80459260200000005, 2024-09-14T23:56:40.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057774860999999997, 2024-09-14T23:56:40.095 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:40.095 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:40.095 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:40.095 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:40.095 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:40.095 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:40.095 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5204 2024-09-14T23:56:40.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.095 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:40.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:40.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:56:40.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:40.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:40.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:56:40.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:40.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:40.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255903+0000", 2024-09-14T23:56:40.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001978807 2024-09-14T23:56:40.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:40.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255907+0000", 2024-09-14T23:56:40.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3610000000000001e-06 2024-09-14T23:56:40.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:40.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255914+0000", 2024-09-14T23:56:40.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.357e-06 2024-09-14T23:56:40.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:40.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255918+0000", 2024-09-14T23:56:40.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1779999999999996e-06 2024-09-14T23:56:40.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:40.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311541+0000", 2024-09-14T23:56:40.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055622697999999998 2024-09-14T23:56:40.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:40.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311614+0000", 2024-09-14T23:56:40.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3102999999999994e-05 2024-09-14T23:56:40.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:40.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311699+0000", 2024-09-14T23:56:40.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5357000000000003e-05 2024-09-14T23:56:40.098 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:40.098 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:40.098 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:40.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.098 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5210 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2785280~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:40.099 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256011+0000", 2024-09-14T23:56:40.099 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.80250596500000004, 2024-09-14T23:56:40.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057273662000000003, 2024-09-14T23:56:40.099 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:40.099 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:40.099 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:40.099 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:40.099 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:40.099 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:40.099 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5210 2024-09-14T23:56:40.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.099 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:40.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:40.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256011+0000", 2024-09-14T23:56:40.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:40.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:40.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256011+0000", 2024-09-14T23:56:40.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:40.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:40.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256011+0000", 2024-09-14T23:56:40.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4999999999999999e-07 2024-09-14T23:56:40.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:40.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256013+0000", 2024-09-14T23:56:40.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3969999999999999e-06 2024-09-14T23:56:40.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:40.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256017+0000", 2024-09-14T23:56:40.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1409999999999998e-06 2024-09-14T23:56:40.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:40.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256019+0000", 2024-09-14T23:56:40.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6280000000000001e-06 2024-09-14T23:56:40.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:40.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313116+0000", 2024-09-14T23:56:40.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057096457000000003 2024-09-14T23:56:40.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:40.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313193+0000", 2024-09-14T23:56:40.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7439000000000002e-05 2024-09-14T23:56:40.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:40.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313285+0000", 2024-09-14T23:56:40.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1349999999999998e-05 2024-09-14T23:56:40.102 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:40.103 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:40.103 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:40.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.103 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5211 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2846720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:40.103 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:56:40.103 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.80249438799999995, 2024-09-14T23:56:40.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057528151, 2024-09-14T23:56:40.103 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:40.103 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:40.103 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:40.103 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:40.103 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:40.103 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:40.104 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5211 2024-09-14T23:56:40.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.104 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:40.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:40.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:56:40.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:40.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:40.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:56:40.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:40.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:40.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:56:40.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1399999999999998e-07 2024-09-14T23:56:40.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:40.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256024+0000", 2024-09-14T23:56:40.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.114e-06 2024-09-14T23:56:40.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:40.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256028+0000", 2024-09-14T23:56:40.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0840000000000002e-06 2024-09-14T23:56:40.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:40.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256031+0000", 2024-09-14T23:56:40.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6819999999999999e-06 2024-09-14T23:56:40.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:40.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313317+0000", 2024-09-14T23:56:40.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057286295000000001 2024-09-14T23:56:40.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:40.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313403+0000", 2024-09-14T23:56:40.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6287999999999998e-05 2024-09-14T23:56:40.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:40.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313551+0000", 2024-09-14T23:56:40.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000147374 2024-09-14T23:56:40.107 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:40.107 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:40.107 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:40.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.108 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5212 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2908160~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:40.108 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:56:40.108 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.80248319400000001, 2024-09-14T23:56:40.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057735679999999998, 2024-09-14T23:56:40.108 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:40.108 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:40.108 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:40.108 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:40.108 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:40.108 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:40.108 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5212 2024-09-14T23:56:40.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.108 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:40.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:40.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:56:40.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:40.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:40.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:56:40.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:40.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:40.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:56:40.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0900000000000001e-07 2024-09-14T23:56:40.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:40.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256035+0000", 2024-09-14T23:56:40.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1200000000000001e-06 2024-09-14T23:56:40.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:40.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256038+0000", 2024-09-14T23:56:40.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.05e-06 2024-09-14T23:56:40.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:40.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256041+0000", 2024-09-14T23:56:40.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.667e-06 2024-09-14T23:56:40.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:40.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313584+0000", 2024-09-14T23:56:40.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057543484999999998 2024-09-14T23:56:40.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:40.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313668+0000", 2024-09-14T23:56:40.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3807000000000006e-05 2024-09-14T23:56:40.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:40.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313770+0000", 2024-09-14T23:56:40.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010134200000000001 2024-09-14T23:56:40.112 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:40.112 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:40.112 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:40.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.112 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5213 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2969600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:40.112 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:56:40.112 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.80247310699999996, 2024-09-14T23:56:40.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058145467999999999, 2024-09-14T23:56:40.112 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:40.112 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:40.112 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:40.113 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:40.113 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:40.113 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:40.113 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5213 2024-09-14T23:56:40.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.113 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:40.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:40.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:56:40.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:40.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:40.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:56:40.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:40.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:40.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257901+0000", 2024-09-14T23:56:40.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018567519999999999 2024-09-14T23:56:40.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:40.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257904+0000", 2024-09-14T23:56:40.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4850000000000001e-06 2024-09-14T23:56:40.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:40.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257912+0000", 2024-09-14T23:56:40.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2090000000000005e-06 2024-09-14T23:56:40.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:40.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257918+0000", 2024-09-14T23:56:40.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6899999999999997e-06 2024-09-14T23:56:40.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:40.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313816+0000", 2024-09-14T23:56:40.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055898075999999998 2024-09-14T23:56:40.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:40.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.314082+0000", 2024-09-14T23:56:40.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000265386 2024-09-14T23:56:40.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:40.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.314189+0000", 2024-09-14T23:56:40.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010787 2024-09-14T23:56:40.116 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:40.116 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:40.116 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:40.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.116 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5217 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3215360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:40.117 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257961+0000", 2024-09-14T23:56:40.117 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.80055612799999998, 2024-09-14T23:56:40.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057320969999999999, 2024-09-14T23:56:40.117 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:40.117 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:40.117 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:40.117 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:40.117 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:40.117 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:40.117 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5217 2024-09-14T23:56:40.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.117 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:40.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:40.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257961+0000", 2024-09-14T23:56:40.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:40.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:40.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257961+0000", 2024-09-14T23:56:40.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:40.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:40.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257961+0000", 2024-09-14T23:56:40.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.22e-07 2024-09-14T23:56:40.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:40.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257962+0000", 2024-09-14T23:56:40.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1400000000000001e-06 2024-09-14T23:56:40.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:40.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257965+0000", 2024-09-14T23:56:40.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0929999999999999e-06 2024-09-14T23:56:40.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:40.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257968+0000", 2024-09-14T23:56:40.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5749999999999999e-06 2024-09-14T23:56:40.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:40.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315047+0000", 2024-09-14T23:56:40.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057078730000000001 2024-09-14T23:56:40.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:40.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315087+0000", 2024-09-14T23:56:40.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0806999999999997e-05 2024-09-14T23:56:40.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:40.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315282+0000", 2024-09-14T23:56:40.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000194403 2024-09-14T23:56:40.121 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:40.121 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:40.121 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:40.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.121 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5218 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3276800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:40.121 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:56:40.121 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.80054609799999998, 2024-09-14T23:56:40.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057497557999999997, 2024-09-14T23:56:40.121 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:40.121 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:40.121 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:40.121 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:40.122 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:40.122 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:40.122 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5218 2024-09-14T23:56:40.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.122 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:40.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:40.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:56:40.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:40.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:40.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:56:40.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:40.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:40.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:56:40.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1899999999999999e-07 2024-09-14T23:56:40.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:40.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257972+0000", 2024-09-14T23:56:40.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.217e-06 2024-09-14T23:56:40.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:40.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257975+0000", 2024-09-14T23:56:40.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9359999999999999e-06 2024-09-14T23:56:40.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:40.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257978+0000", 2024-09-14T23:56:40.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1049999999999999e-06 2024-09-14T23:56:40.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:40.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315331+0000", 2024-09-14T23:56:40.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057352093999999999 2024-09-14T23:56:40.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:40.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315384+0000", 2024-09-14T23:56:40.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3780999999999999e-05 2024-09-14T23:56:40.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:40.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315468+0000", 2024-09-14T23:56:40.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4206e-05 2024-09-14T23:56:40.125 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:40.125 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:40.125 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:40.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.125 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5219 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3338240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:40.125 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:56:40.125 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.80053542200000005, 2024-09-14T23:56:40.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057637764000000001, 2024-09-14T23:56:40.126 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:40.126 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:40.126 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:40.126 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:40.126 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:40.126 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:40.126 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5219 2024-09-14T23:56:40.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.126 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:40.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:40.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:56:40.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:40.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:40.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:56:40.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:40.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:40.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:56:40.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.03e-07 2024-09-14T23:56:40.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:40.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257983+0000", 2024-09-14T23:56:40.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9199999999999999e-07 2024-09-14T23:56:40.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:40.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257986+0000", 2024-09-14T23:56:40.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0860000000000002e-06 2024-09-14T23:56:40.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:40.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257989+0000", 2024-09-14T23:56:40.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6589999999999999e-06 2024-09-14T23:56:40.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:40.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315509+0000", 2024-09-14T23:56:40.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057520262000000003 2024-09-14T23:56:40.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:40.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315554+0000", 2024-09-14T23:56:40.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5015999999999997e-05 2024-09-14T23:56:40.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:40.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315619+0000", 2024-09-14T23:56:40.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5545999999999999e-05 2024-09-14T23:56:40.129 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:40.129 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:40.129 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:40.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.130 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5220 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3399680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:40.130 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:56:40.130 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.80052546599999996, 2024-09-14T23:56:40.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057792128999999998, 2024-09-14T23:56:40.130 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:40.130 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:40.130 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:40.130 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:40.130 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:40.130 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:40.130 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5220 2024-09-14T23:56:40.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.130 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:40.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:40.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:56:40.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:40.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:40.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:56:40.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:40.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:40.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:56:40.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1799999999999999e-07 2024-09-14T23:56:40.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:40.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257993+0000", 2024-09-14T23:56:40.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6200000000000006e-07 2024-09-14T23:56:40.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:40.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257996+0000", 2024-09-14T23:56:40.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9050000000000001e-06 2024-09-14T23:56:40.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:40.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257998+0000", 2024-09-14T23:56:40.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5950000000000001e-06 2024-09-14T23:56:40.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:40.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315672+0000", 2024-09-14T23:56:40.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057673637999999999 2024-09-14T23:56:40.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:40.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315734+0000", 2024-09-14T23:56:40.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2216000000000002e-05 2024-09-14T23:56:40.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:40.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315784+0000", 2024-09-14T23:56:40.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9595000000000002e-05 2024-09-14T23:56:40.133 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:40.134 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:40.134 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:40.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.134 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5221 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3461120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:40.134 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:56:40.134 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.80051574800000003, 2024-09-14T23:56:40.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058003557999999997, 2024-09-14T23:56:40.134 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:40.134 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:40.134 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:40.134 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:40.134 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:40.134 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:40.135 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5221 2024-09-14T23:56:40.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.135 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:40.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:40.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:56:40.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:40.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:40.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:56:40.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:40.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:40.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259905+0000", 2024-09-14T23:56:40.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019035759999999999 2024-09-14T23:56:40.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:40.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259915+0000", 2024-09-14T23:56:40.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0518e-05 2024-09-14T23:56:40.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:40.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259925+0000", 2024-09-14T23:56:40.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7180000000000008e-06 2024-09-14T23:56:40.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:40.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259930+0000", 2024-09-14T23:56:40.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4769999999999997e-06 2024-09-14T23:56:40.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:40.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315820+0000", 2024-09-14T23:56:40.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055889960000000002 2024-09-14T23:56:40.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:40.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315914+0000", 2024-09-14T23:56:40.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4625999999999998e-05 2024-09-14T23:56:40.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:40.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.316005+0000", 2024-09-14T23:56:40.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0682999999999999e-05 2024-09-14T23:56:40.138 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:40.138 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:40.138 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:40.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.138 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5230 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 4014080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:40.138 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.260060+0000", 2024-09-14T23:56:40.138 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.79845690999999996, 2024-09-14T23:56:40.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057353556999999999, 2024-09-14T23:56:40.138 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:40.139 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:40.139 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:40.139 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:40.139 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:40.139 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:40.139 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5230 2024-09-14T23:56:40.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.139 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:40.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:40.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260060+0000", 2024-09-14T23:56:40.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:40.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:40.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260060+0000", 2024-09-14T23:56:40.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:40.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:40.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260060+0000", 2024-09-14T23:56:40.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.17e-07 2024-09-14T23:56:40.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:40.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260061+0000", 2024-09-14T23:56:40.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0300000000000001e-06 2024-09-14T23:56:40.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:40.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260064+0000", 2024-09-14T23:56:40.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9119999999999998e-06 2024-09-14T23:56:40.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:40.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260067+0000", 2024-09-14T23:56:40.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.678e-06 2024-09-14T23:56:40.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:40.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317269+0000", 2024-09-14T23:56:40.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057201771999999998 2024-09-14T23:56:40.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:40.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317342+0000", 2024-09-14T23:56:40.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2842000000000005e-05 2024-09-14T23:56:40.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:40.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317414+0000", 2024-09-14T23:56:40.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2106000000000004e-05 2024-09-14T23:56:40.142 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:40.142 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:40.142 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:40.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.142 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5231 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 4075520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:40.143 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:56:40.143 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.79844701399999995, 2024-09-14T23:56:40.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057524733000000002, 2024-09-14T23:56:40.143 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:40.143 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:40.143 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:40.143 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:40.143 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:40.143 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:40.143 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5231 2024-09-14T23:56:40.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.143 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:40.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:40.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:56:40.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:40.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:40.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:56:40.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:40.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:40.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261904+0000", 2024-09-14T23:56:40.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018338790000000001 2024-09-14T23:56:40.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:40.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261910+0000", 2024-09-14T23:56:40.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9689999999999999e-06 2024-09-14T23:56:40.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:40.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261923+0000", 2024-09-14T23:56:40.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3061000000000001e-05 2024-09-14T23:56:40.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:40.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261931+0000", 2024-09-14T23:56:40.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3159999999999997e-06 2024-09-14T23:56:40.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:40.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317456+0000", 2024-09-14T23:56:40.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055525219000000001 2024-09-14T23:56:40.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:40.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317520+0000", 2024-09-14T23:56:40.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4010000000000005e-05 2024-09-14T23:56:40.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:40.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317595+0000", 2024-09-14T23:56:40.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4278999999999998e-05 2024-09-14T23:56:40.147 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:40.147 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:40.147 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:40.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.147 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5236 3.8 3:1071668d:::rbd_data.1118de1168b.0000000000000092:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:40.147 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:56:40.147 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.79643355000000005, 2024-09-14T23:56:40.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057499462000000001, 2024-09-14T23:56:40.147 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:40.147 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:40.147 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:40.147 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:40.147 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:40.147 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:40.148 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5236 2024-09-14T23:56:40.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.148 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:40.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:40.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:56:40.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:40.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:40.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:56:40.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:40.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:40.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262084+0000", 2024-09-14T23:56:40.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3399999999999999e-07 2024-09-14T23:56:40.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:40.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262087+0000", 2024-09-14T23:56:40.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2310000000000001e-06 2024-09-14T23:56:40.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:40.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262095+0000", 2024-09-14T23:56:40.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3799999999999996e-06 2024-09-14T23:56:40.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:40.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262101+0000", 2024-09-14T23:56:40.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0009999999999998e-06 2024-09-14T23:56:40.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:40.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317625+0000", 2024-09-14T23:56:40.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055524111000000001 2024-09-14T23:56:40.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:40.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317702+0000", 2024-09-14T23:56:40.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7533999999999993e-05 2024-09-14T23:56:40.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-14T23:56:40.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317892+0000", 2024-09-14T23:56:40.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018991400000000001 2024-09-14T23:56:40.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:56:40.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319407+0000", 2024-09-14T23:56:40.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015145250000000001 2024-09-14T23:56:40.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:40.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319481+0000", 2024-09-14T23:56:40.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3826999999999996e-05 2024-09-14T23:56:40.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:40.152 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319541+0000", 2024-09-14T23:56:40.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0470999999999998e-05 2024-09-14T23:56:40.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:40.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:40.152 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:40.152 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319583+0000", 2024-09-14T23:56:40.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1934000000000002e-05 2024-09-14T23:56:40.152 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:40.152 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:40.152 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:40.152 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:40.152 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:40.152 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:56:40.153 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:56:40.060+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:56:40.153 INFO:teuthology.orchestra.run.smithi042.stdout:129 trunc from 0xdf50621 to 0x9a25f74 2024-09-14T23:56:40.154 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T23:56:40.155 INFO:teuthology.orchestra.run.smithi042.stdout:130 write 0x56b64f8 thru 0x56ba514 (0x401d bytes) 2024-09-14T23:56:40.155 INFO:teuthology.orchestra.run.smithi042.stdout:132 writesame 0x3f4aefc thru 0x3f4ceea data_size 0x19(0x1fef bytes) 2024-09-14T23:56:40.155 INFO:teuthology.orchestra.run.smithi042.stdout:133 write 0xd194201 thru 0xd197703 (0x3503 bytes) 2024-09-14T23:56:40.155 INFO:teuthology.orchestra.run.smithi042.stdout:134 read 0x96dd128 thru 0x96e08d2 (0x37ab bytes) 2024-09-14T23:56:40.155 INFO:teuthology.orchestra.run.smithi042.stdout:136 write 0x3f26807 thru 0x3f2fc37 (0x9431 bytes) 2024-09-14T23:56:40.155 INFO:teuthology.orchestra.run.smithi042.stdout:138 read 0xa41f9a9 thru 0xa42d7bf (0xde17 bytes) 2024-09-14T23:56:40.155 INFO:teuthology.orchestra.run.smithi042.stdout:139 writesame 0x7ec019e thru 0x7ec5628 data_size 0x1(0x548b bytes) 2024-09-14T23:56:40.156 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:56:40.156 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:56:40.157 INFO:teuthology.orchestra.run.smithi042.stdout:140 write 0x94bb369 thru 0x94bf9d6 (0x466e bytes) 2024-09-14T23:56:40.157 INFO:teuthology.orchestra.run.smithi042.stdout:141 read 0x345b10f thru 0x3468996 (0xd888 bytes) 2024-09-14T23:56:40.157 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:40.157 INFO:teuthology.orchestra.run.smithi042.stdout:143 read 0x3973f70 thru 0x3980b65 (0xcbf6 bytes) 2024-09-14T23:56:40.157 INFO:teuthology.orchestra.run.smithi042.stdout:144 writesame 0x9cd53c7 thru 0x9cd9dec data_size 0x1(0x4a26 bytes) 2024-09-14T23:56:40.157 INFO:teuthology.orchestra.run.smithi042.stdout:145 writesame 0x5c1092d thru 0x5c17c51 data_size 0x1(0x7325 bytes) 2024-09-14T23:56:40.157 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:56:40.158 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:56:40.158 INFO:teuthology.orchestra.run.smithi042.stdout:146 punch from 0x8b9e0d3 to 0x8ba1ad2, (0x39ff bytes) 2024-09-14T23:56:40.158 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:40.158 INFO:teuthology.orchestra.run.smithi042.stdout:149 read 0x716765 thru 0x72430a (0xdba6 bytes) 2024-09-14T23:56:40.158 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:56:40.159 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:56:40.159 INFO:teuthology.orchestra.run.smithi042.stdout:151 trunc from 0xd197704 to 0x117520c 2024-09-14T23:56:40.159 INFO:teuthology.orchestra.run.smithi042.stdout:152 read 0x3e948d thru 0x3f3fb4 (0xab28 bytes) 2024-09-14T23:56:40.159 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:56:40.159 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:56:40.162 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:56:40.161+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:56:40.164 INFO:teuthology.orchestra.run.smithi042.stdout:154 write 0x393e778 thru 0x394726c (0x8af5 bytes) 2024-09-14T23:56:40.186 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:56:40.186 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:56:40.186 INFO:teuthology.orchestra.run.smithi042.stdout:155 writesame 0xcfa866c thru 0xcfaabb8 data_size 0x3(0x254d bytes) 2024-09-14T23:56:40.186 INFO:teuthology.orchestra.run.smithi042.stdout:156 write 0xb1f7bb6 thru 0xb2039ed (0xbe38 bytes) 2024-09-14T23:56:40.190 INFO:teuthology.orchestra.run.smithi042.stdout:157 read 0x90f240b thru 0x90fd352 (0xaf48 bytes) 2024-09-14T23:56:40.190 INFO:teuthology.orchestra.run.smithi042.stdout:159 writesame 0x5f91027 thru 0x5f9cb9e data_size 0x1c(0xbb78 bytes) 2024-09-14T23:56:40.192 INFO:teuthology.orchestra.run.smithi042.stdout:160 write 0x4f0ea0 thru 0x4f36cb (0x282c bytes) 2024-09-14T23:56:40.194 INFO:teuthology.orchestra.run.smithi042.stdout:161 write 0x2fee50b thru 0x2ff0565 (0x205b bytes) 2024-09-14T23:56:40.195 INFO:teuthology.orchestra.run.smithi042.stdout:162 writesame 0x7c04a74 thru 0x7c0af4d data_size 0x27(0x64da bytes) 2024-09-14T23:56:40.205 INFO:teuthology.orchestra.run.smithi042.stdout:165 trunc from 0xcfaabb9 to 0x726b4df 2024-09-14T23:56:40.205 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:56:40.206 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:56:40.219 INFO:teuthology.orchestra.run.smithi042.stdout:168 trunc from 0x726b4df to 0x39143cc 2024-09-14T23:56:40.227 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:56:40.227 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:56:40.228 INFO:teuthology.orchestra.run.smithi042.stdout:169 read 0x1aca40b thru 0x1ad956c (0xf162 bytes) 2024-09-14T23:56:40.229 INFO:teuthology.orchestra.run.smithi042.stdout:170 read 0x1cce0bd thru 0x1cd79c8 (0x990c bytes) 2024-09-14T23:56:40.229 INFO:teuthology.orchestra.run.smithi042.stdout:171 punch from 0x29061db to 0x29158ff, (0xf724 bytes) 2024-09-14T23:56:40.230 INFO:teuthology.orchestra.run.smithi042.stdout:172 read 0x3210cf2 thru 0x321e007 (0xd316 bytes) 2024-09-14T23:56:40.230 INFO:teuthology.orchestra.run.smithi042.stdout:173 read 0x6dc6fa thru 0x6e88dd (0xc1e4 bytes) 2024-09-14T23:56:40.230 INFO:teuthology.orchestra.run.smithi042.stdout:174 read 0x9591a3 thru 0x961d9d (0x8bfb bytes) 2024-09-14T23:56:40.230 INFO:teuthology.orchestra.run.smithi042.stdout:176 clone 3 order 21 su 8192 sc 11 2024-09-14T23:56:40.244 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:56:40.244 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:56:40.268 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:40.268 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:40.277 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T23:56:40.296 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:56:40.295+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:56:40.365 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:56:40.363+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:56:40.383 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:40.383 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:40.392 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T23:56:40.465 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:56:40.464+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:56:40.499 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:40.499 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:40.508 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T23:56:40.566 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:56:40.564+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:56:40.615 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:40.615 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:40.624 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T23:56:40.667 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:56:40.666+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:56:40.768 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:56:40.765+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:56:40.770 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:40.771 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:40.780 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T23:56:40.869 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:56:40.867+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:56:40.926 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:40.926 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:40.935 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T23:56:40.969 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:56:40.968+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:56:41.046 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:41.046 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:41.056 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T23:56:41.061 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone2 from 0x39143cc (overlap 0xdd0d15) to 0xb0d744 2024-09-14T23:56:41.070 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:56:41.068+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:56:41.136 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-09-14T23:56:41.153 INFO:teuthology.orchestra.run.smithi042.stdout:178 trunc from 0x39143cc to 0x4357a3d 2024-09-14T23:56:41.168 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:41.169 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:41.171 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:56:41.169+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:56:41.178 INFO:teuthology.orchestra.run.smithi042.stdout:179 writesame 0xb3e4d4b thru 0xb3eb626 data_size 0x6(0x68dc bytes) 2024-09-14T23:56:41.178 INFO:teuthology.orchestra.run.smithi042.stdout:180 writesame 0x6804827 thru 0x6809817 data_size 0x1(0x4ff1 bytes) 2024-09-14T23:56:41.180 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T23:56:41.184 INFO:teuthology.orchestra.run.smithi042.stdout:182 write 0x2f30fc4 thru 0x2f381df (0x721c bytes) 2024-09-14T23:56:41.212 INFO:teuthology.orchestra.run.smithi042.stdout:185 write 0x2d8e2b9 thru 0x2d9c9b6 (0xe6fe bytes) 2024-09-14T23:56:41.230 INFO:teuthology.orchestra.run.smithi042.stdout:186 punch from 0x1c1aaf to 0x1c2f55, (0x14a6 bytes) 2024-09-14T23:56:41.237 INFO:teuthology.orchestra.run.smithi042.stdout:187 write 0xb6a9e2a thru 0xb6b399f (0x9b76 bytes) 2024-09-14T23:56:41.245 INFO:teuthology.orchestra.run.smithi042.stdout:188 read 0x3ba10c3 thru 0x3ba2d0e (0x1c4c bytes) 2024-09-14T23:56:41.245 INFO:teuthology.orchestra.run.smithi042.stdout:189 read 0xb65a0eb thru 0xb6654d4 (0xb3ea bytes) 2024-09-14T23:56:41.245 INFO:teuthology.orchestra.run.smithi042.stdout:190 read 0x48c1bc3 thru 0x48cc629 (0xaa67 bytes) 2024-09-14T23:56:41.245 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:41.245 INFO:teuthology.orchestra.run.smithi042.stdout:192 read 0x7bfd899 thru 0x7bfee1e (0x1586 bytes) 2024-09-14T23:56:41.245 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:41.245 INFO:teuthology.orchestra.run.smithi042.stdout:196 punch from 0x78725ef to 0x787b392, (0x8da3 bytes) 2024-09-14T23:56:41.246 INFO:teuthology.orchestra.run.smithi042.stdout:197 read 0x6165b82 thru 0x6167dce (0x224d bytes) 2024-09-14T23:56:41.254 INFO:teuthology.orchestra.run.smithi042.stdout:198 writesame 0xccaca2a thru 0xccad83c data_size 0x1(0xe13 bytes) 2024-09-14T23:56:41.254 INFO:teuthology.orchestra.run.smithi042.stdout:199 write 0x8441dca thru 0x8447abb (0x5cf2 bytes) 2024-09-14T23:56:41.259 INFO:teuthology.orchestra.run.smithi042.stdout:200 write 0xcbec518 thru 0xcbfb122 (0xec0b bytes) 2024-09-14T23:56:41.267 INFO:teuthology.orchestra.run.smithi042.stdout:203 write 0x3038650 thru 0x30443d3 (0xbd84 bytes) 2024-09-14T23:56:41.269 INFO:teuthology.orchestra.run.smithi042.stdout:204 punch from 0xbb8640e to 0xbb8b905, (0x54f7 bytes) 2024-09-14T23:56:41.270 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:41.271 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:56:41.269+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:56:41.277 INFO:teuthology.orchestra.run.smithi042.stdout:208 trunc from 0xccad83d to 0x6173855 2024-09-14T23:56:41.292 INFO:teuthology.orchestra.run.smithi042.stdout:209 writesame 0x445c56 thru 0x447690 data_size 0x11(0x1a3b bytes) 2024-09-14T23:56:41.315 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:41.315 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:41.325 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T23:56:41.334 INFO:teuthology.orchestra.run.smithi042.stdout:210 write 0xa26a135 thru 0xa277b1a (0xd9e6 bytes) 2024-09-14T23:56:41.342 INFO:teuthology.orchestra.run.smithi042.stdout:214 clone 4 order 23 su 524288 sc 12 2024-09-14T23:56:41.373 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:56:41.372+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:56:41.472 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:41.473 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:41.473 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:56:41.472+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:56:41.481 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T23:56:41.574 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:56:41.572+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:56:41.588 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:41.588 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:41.597 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T23:56:41.675 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:56:41.674+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:56:41.744 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:41.744 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:41.753 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T23:56:41.798 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:56:41.797+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:56:41.864 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:41.865 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:41.873 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T23:56:41.915 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:56:41.913+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:56:41.978 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:56:41.976+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:56:42.009 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T23:56:42.026 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:42.026 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:42.036 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T23:56:42.065 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone3 from 0xa277b1b (overlap 0x39143cc) to 0x2da9ca3 2024-09-14T23:56:42.108 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:56:42.107+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:56:42.180 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:56:42.178+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:56:42.184 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-09-14T23:56:42.195 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:42.195 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:42.203 INFO:teuthology.orchestra.run.smithi042.stdout:216 write 0x18b892b thru 0x18beeb0 (0x6586 bytes) 2024-09-14T23:56:42.205 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T23:56:42.252 INFO:teuthology.orchestra.run.smithi042.stdout:218 writesame 0x142ad16 thru 0x1438acf data_size 0x65(0xddba bytes) 2024-09-14T23:56:42.256 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T23:56:42.280 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:56:42.279+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:56:42.296 INFO:teuthology.orchestra.run.smithi042.stdout:219 write 0x6e649b5 thru 0x6e72863 (0xdeaf bytes) 2024-09-14T23:56:42.311 INFO:teuthology.orchestra.run.smithi042.stdout:220 trunc from 0xa277b1b to 0x16f5ce3 2024-09-14T23:56:42.383 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:56:42.380+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:56:42.418 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:42.418 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:42.434 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T23:56:42.500 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:56:42.500 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:56:42.505 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:56:42.504+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:56:42.543 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:56:42.543 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:56:42.572 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:56:42.572 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:56:42.583 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:56:42.581+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:56:42.593 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:56:42.594 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:56:42.612 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:56:42.612 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:56:42.629 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:56:42.629 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:56:42.629 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:56:42.629 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:56:42.629 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:56:42.631 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:56:42.631 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:56:42.640 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T23:56:42.651 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:56:42.652 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:56:42.670 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:56:42.670 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:56:42.672 INFO:teuthology.orchestra.run.smithi042.stdout:221 write 0xb9e27f8 thru 0xb9e2eb3 (0x6bc bytes) 2024-09-14T23:56:42.675 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:42.676 INFO:teuthology.orchestra.run.smithi042.stdout:223 read 0x853ba19 thru 0x8546b23 (0xb10b bytes) 2024-09-14T23:56:42.676 INFO:teuthology.orchestra.run.smithi042.stdout:225 writesame 0x18764b9 thru 0x187cc57 data_size 0x1(0x679f bytes) 2024-09-14T23:56:42.690 INFO:teuthology.orchestra.run.smithi042.stdout:226 trunc from 0xb9e2eb4 to 0x6aa0522 2024-09-14T23:56:42.690 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:56:42.689+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:56:42.696 INFO:teuthology.orchestra.run.smithi042.stdout:227 write 0x1e1c11a thru 0x1e231ba (0x70a1 bytes) 2024-09-14T23:56:42.698 INFO:teuthology.orchestra.run.smithi042.stdout:228 read 0x4b1b409 thru 0x4b1ee1d (0x3a15 bytes) 2024-09-14T23:56:42.698 INFO:teuthology.orchestra.run.smithi042.stdout:229 write 0x68d3766 thru 0x68d79cb (0x4266 bytes) 2024-09-14T23:56:42.701 INFO:teuthology.orchestra.run.smithi042.stdout:230 punch from 0x35fc4ce to 0x360ae91, (0xe9c3 bytes) 2024-09-14T23:56:42.701 INFO:teuthology.orchestra.run.smithi042.stdout:232 read 0x5743a0f thru 0x57497aa (0x5d9c bytes) 2024-09-14T23:56:42.701 INFO:teuthology.orchestra.run.smithi042.stdout:233 write 0x3304e3e thru 0x3312f88 (0xe14b bytes) 2024-09-14T23:56:42.703 INFO:teuthology.orchestra.run.smithi042.stdout:234 read 0x343d6bc thru 0x343fd78 (0x26bd bytes) 2024-09-14T23:56:42.703 INFO:teuthology.orchestra.run.smithi042.stdout:truncating to largest ever: 0xd80bd1c 2024-09-14T23:56:42.711 INFO:teuthology.orchestra.run.smithi042.stdout:236 trunc from 0x6aa0522 to 0xd80bd1c 2024-09-14T23:56:42.716 INFO:teuthology.orchestra.run.smithi042.stdout:237 write 0x3d3dbff thru 0x3d4992a (0xbd2c bytes) 2024-09-14T23:56:42.718 INFO:teuthology.orchestra.run.smithi042.stdout:239 writesame 0xc35491b thru 0xc35caaa data_size 0x18(0x8190 bytes) 2024-09-14T23:56:42.721 INFO:teuthology.orchestra.run.smithi042.stdout:240 punch from 0x89fae41 to 0x8a00784, (0x5943 bytes) 2024-09-14T23:56:42.721 INFO:teuthology.orchestra.run.smithi042.stdout:242 read 0x7fc874e thru 0x7fcfebf (0x7772 bytes) 2024-09-14T23:56:42.721 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:42.721 INFO:teuthology.orchestra.run.smithi042.stdout:245 write 0x8d8c3a7 thru 0x8d9b928 (0xf582 bytes) 2024-09-14T23:56:42.724 INFO:teuthology.orchestra.run.smithi042.stdout:247 write 0x680296d thru 0x680fd95 (0xd429 bytes) 2024-09-14T23:56:42.726 INFO:teuthology.orchestra.run.smithi042.stdout:248 punch from 0x6bd9437 to 0x6be79cf, (0xe598 bytes) 2024-09-14T23:56:42.726 INFO:teuthology.orchestra.run.smithi042.stdout:249 writesame 0xcd121db thru 0xcd1b564 data_size 0xf(0x938a bytes) 2024-09-14T23:56:42.729 INFO:teuthology.orchestra.run.smithi042.stdout:251 read 0x88e0130 thru 0x88eb507 (0xb3d8 bytes) 2024-09-14T23:56:42.729 INFO:teuthology.orchestra.run.smithi042.stdout:253 write 0xae9300a thru 0xae9fb77 (0xcb6e bytes) 2024-09-14T23:56:42.733 INFO:teuthology.orchestra.run.smithi042.stdout:254 read 0xc2a4a11 thru 0xc2ab848 (0x6e38 bytes) 2024-09-14T23:56:42.733 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:42.733 INFO:teuthology.orchestra.run.smithi042.stdout:256 read 0x6d39aa5 thru 0x6d4351b (0x9a77 bytes) 2024-09-14T23:56:42.733 INFO:teuthology.orchestra.run.smithi042.stdout:257 punch from 0x612f8ac to 0x613f007, (0xf75b bytes) 2024-09-14T23:56:42.733 INFO:teuthology.orchestra.run.smithi042.stdout:258 writesame 0x107c096 thru 0x1089abf data_size 0x19(0xda2a bytes) 2024-09-14T23:56:42.735 INFO:teuthology.orchestra.run.smithi042.stdout:259 write 0x96ca4c1 thru 0x96cbc0e (0x174e bytes) 2024-09-14T23:56:42.750 INFO:teuthology.orchestra.run.smithi042.stdout:260 trunc from 0xd80bd1c to 0xd8b7d1 2024-09-14T23:56:42.763 INFO:teuthology.orchestra.run.smithi042.stdout:261 read 0xc4641e thru 0xc48b95 (0x2778 bytes) 2024-09-14T23:56:42.769 INFO:teuthology.orchestra.run.smithi042.stdout:263 write 0x14a73dd thru 0x14aac22 (0x3846 bytes) 2024-09-14T23:56:42.782 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:56:42.783 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:56:42.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:56:42.783 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:56:42.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.783 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5880 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:42.783 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375821+0000", 2024-09-14T23:56:42.783 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.4044535389999999, 2024-09-14T23:56:42.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050482096999999997, 2024-09-14T23:56:42.783 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:42.783 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:42.783 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:56:42.783 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:42.783 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:42.784 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:42.784 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5880 2024-09-14T23:56:42.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.784 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:42.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:42.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375821+0000", 2024-09-14T23:56:42.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:42.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:42.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375821+0000", 2024-09-14T23:56:42.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:42.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:42.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375821+0000", 2024-09-14T23:56:42.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.28e-07 2024-09-14T23:56:42.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:42.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375822+0000", 2024-09-14T23:56:42.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7000000000000003e-07 2024-09-14T23:56:42.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:42.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375825+0000", 2024-09-14T23:56:42.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9560000000000002e-06 2024-09-14T23:56:42.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:42.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375827+0000", 2024-09-14T23:56:42.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3259999999999998e-06 2024-09-14T23:56:42.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:42.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426294+0000", 2024-09-14T23:56:42.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050466748999999998 2024-09-14T23:56:42.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:42.787 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426303+0000", 2024-09-14T23:56:42.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8680000000000005e-06 2024-09-14T23:56:42.787 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:42.787 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:42.787 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:42.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.787 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5881 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:42.787 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375830+0000", 2024-09-14T23:56:42.787 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.4044440929999999, 2024-09-14T23:56:42.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050523831999999998, 2024-09-14T23:56:42.787 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:42.787 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:42.787 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:56:42.787 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:42.788 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:42.788 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:42.788 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5881 2024-09-14T23:56:42.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.788 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:42.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:42.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375830+0000", 2024-09-14T23:56:42.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:42.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:42.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375830+0000", 2024-09-14T23:56:42.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:42.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:42.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375830+0000", 2024-09-14T23:56:42.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.34e-07 2024-09-14T23:56:42.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:42.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375831+0000", 2024-09-14T23:56:42.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9599999999999998e-07 2024-09-14T23:56:42.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:42.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375834+0000", 2024-09-14T23:56:42.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0489999999999999e-06 2024-09-14T23:56:42.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:42.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375838+0000", 2024-09-14T23:56:42.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5319999999999998e-06 2024-09-14T23:56:42.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:42.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426342+0000", 2024-09-14T23:56:42.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050504391000000003 2024-09-14T23:56:42.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:42.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426354+0000", 2024-09-14T23:56:42.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.173e-05 2024-09-14T23:56:42.791 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:42.791 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:42.791 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:42.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.791 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5882 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:42.791 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:56:42.791 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.4044334890000001, 2024-09-14T23:56:42.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050549666, 2024-09-14T23:56:42.791 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:42.792 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:42.792 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:56:42.792 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:42.792 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:42.792 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:42.792 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5882 2024-09-14T23:56:42.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.792 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:42.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:42.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:56:42.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:42.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:42.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:56:42.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:42.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:42.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:56:42.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8099999999999998e-07 2024-09-14T23:56:42.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:42.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375842+0000", 2024-09-14T23:56:42.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9900000000000009e-07 2024-09-14T23:56:42.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:42.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375845+0000", 2024-09-14T23:56:42.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.055e-06 2024-09-14T23:56:42.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:42.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375848+0000", 2024-09-14T23:56:42.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7609999999999999e-06 2024-09-14T23:56:42.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:42.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426381+0000", 2024-09-14T23:56:42.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050533392000000003 2024-09-14T23:56:42.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:42.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426390+0000", 2024-09-14T23:56:42.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0780000000000002e-06 2024-09-14T23:56:42.795 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:42.796 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:42.796 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:42.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.796 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5884 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:42.796 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:56:42.796 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.404423349, 2024-09-14T23:56:42.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050717439000000003, 2024-09-14T23:56:42.796 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:42.796 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:42.796 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:56:42.796 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:42.796 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:42.797 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:42.797 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5884 2024-09-14T23:56:42.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.797 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:42.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:42.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:56:42.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:42.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:42.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:56:42.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:42.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:42.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:56:42.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0800000000000001e-07 2024-09-14T23:56:42.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:42.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375852+0000", 2024-09-14T23:56:42.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.02e-06 2024-09-14T23:56:42.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:42.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375855+0000", 2024-09-14T23:56:42.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9900000000000002e-06 2024-09-14T23:56:42.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:42.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375857+0000", 2024-09-14T23:56:42.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3499999999999999e-06 2024-09-14T23:56:42.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:42.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426556+0000", 2024-09-14T23:56:42.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050698139000000003 2024-09-14T23:56:42.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:42.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426568+0000", 2024-09-14T23:56:42.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2731999999999999e-05 2024-09-14T23:56:42.800 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:42.800 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:42.800 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:42.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.800 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5885 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:42.800 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:56:42.800 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.4044043240000001, 2024-09-14T23:56:42.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050737333000000003, 2024-09-14T23:56:42.800 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:42.800 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:42.801 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:56:42.801 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:42.801 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:42.801 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:42.801 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5885 2024-09-14T23:56:42.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.801 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:42.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:42.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:56:42.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:42.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:42.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:56:42.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:42.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:42.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:56:42.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5899999999999998e-07 2024-09-14T23:56:42.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:42.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375872+0000", 2024-09-14T23:56:42.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4219999999999999e-06 2024-09-14T23:56:42.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:42.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375878+0000", 2024-09-14T23:56:42.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4980000000000003e-06 2024-09-14T23:56:42.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:42.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375881+0000", 2024-09-14T23:56:42.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6110000000000001e-06 2024-09-14T23:56:42.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:42.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426596+0000", 2024-09-14T23:56:42.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050715320000000001 2024-09-14T23:56:42.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:42.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426607+0000", 2024-09-14T23:56:42.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1223000000000001e-05 2024-09-14T23:56:42.804 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:42.804 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:42.804 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:42.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.804 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5886 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:42.804 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:56:42.804 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.4043902109999999, 2024-09-14T23:56:42.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050803395000000001, 2024-09-14T23:56:42.805 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:42.805 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:42.805 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:56:42.805 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:42.805 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:42.805 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:42.805 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5886 2024-09-14T23:56:42.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.805 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:42.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:42.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:56:42.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:42.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:42.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:56:42.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:42.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:42.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:56:42.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2700000000000001e-07 2024-09-14T23:56:42.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:42.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375886+0000", 2024-09-14T23:56:42.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7060000000000001e-06 2024-09-14T23:56:42.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:42.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375890+0000", 2024-09-14T23:56:42.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2259999999999999e-06 2024-09-14T23:56:42.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:42.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375893+0000", 2024-09-14T23:56:42.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0819999999999999e-06 2024-09-14T23:56:42.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:42.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426677+0000", 2024-09-14T23:56:42.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050784158000000003 2024-09-14T23:56:42.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:42.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426687+0000", 2024-09-14T23:56:42.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9960000000000005e-06 2024-09-14T23:56:42.808 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:42.808 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:42.808 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:42.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.809 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5887 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:42.809 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:56:42.809 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.404368125, 2024-09-14T23:56:42.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050833070000000001, 2024-09-14T23:56:42.809 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:42.809 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:42.809 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:56:42.809 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:42.809 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:42.809 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:42.809 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5887 2024-09-14T23:56:42.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.809 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:42.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:42.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:56:42.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:42.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:42.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:56:42.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:42.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:42.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:56:42.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3299999999999998e-07 2024-09-14T23:56:42.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:42.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375908+0000", 2024-09-14T23:56:42.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7689999999999999e-06 2024-09-14T23:56:42.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:42.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375912+0000", 2024-09-14T23:56:42.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.535e-06 2024-09-14T23:56:42.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:42.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375916+0000", 2024-09-14T23:56:42.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8979999999999998e-06 2024-09-14T23:56:42.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:42.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426729+0000", 2024-09-14T23:56:42.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050813453000000001 2024-09-14T23:56:42.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:42.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426739+0000", 2024-09-14T23:56:42.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0081999999999999e-05 2024-09-14T23:56:42.812 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:42.812 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:42.812 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:42.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.813 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5888 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:42.813 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:56:42.813 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.4043550480000002, 2024-09-14T23:56:42.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050854652, 2024-09-14T23:56:42.813 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:42.813 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:42.813 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:56:42.813 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:42.813 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:42.813 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:42.813 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5888 2024-09-14T23:56:42.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.814 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:42.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:42.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:56:42.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:42.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:42.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:56:42.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:42.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:42.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:56:42.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1400000000000001e-07 2024-09-14T23:56:42.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:42.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375921+0000", 2024-09-14T23:56:42.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.119e-06 2024-09-14T23:56:42.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:42.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375928+0000", 2024-09-14T23:56:42.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3170000000000004e-06 2024-09-14T23:56:42.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:42.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375931+0000", 2024-09-14T23:56:42.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6970000000000002e-06 2024-09-14T23:56:42.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:42.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426768+0000", 2024-09-14T23:56:42.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050837355000000001 2024-09-14T23:56:42.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:42.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426774+0000", 2024-09-14T23:56:42.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9499999999999998e-06 2024-09-14T23:56:42.816 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:42.817 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:42.817 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:42.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.817 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5889 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:42.817 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:56:42.817 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.4043381240000001, 2024-09-14T23:56:42.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050885446000000001, 2024-09-14T23:56:42.817 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:42.817 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:42.817 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:56:42.818 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:42.818 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:42.818 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:42.818 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5889 2024-09-14T23:56:42.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.818 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:42.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:42.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:56:42.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:42.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:42.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:56:42.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:42.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:42.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:56:42.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.36e-07 2024-09-14T23:56:42.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:42.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375940+0000", 2024-09-14T23:56:42.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.844e-06 2024-09-14T23:56:42.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:42.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375944+0000", 2024-09-14T23:56:42.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6720000000000002e-06 2024-09-14T23:56:42.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:42.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375946+0000", 2024-09-14T23:56:42.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5629999999999999e-06 2024-09-14T23:56:42.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:42.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426810+0000", 2024-09-14T23:56:42.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050863255000000003 2024-09-14T23:56:42.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:42.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426822+0000", 2024-09-14T23:56:42.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1875999999999999e-05 2024-09-14T23:56:42.821 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:42.821 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:42.821 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:42.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.821 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5890 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:42.821 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:56:42.822 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.404324312, 2024-09-14T23:56:42.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050917549999999999, 2024-09-14T23:56:42.822 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:42.822 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:42.822 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:56:42.822 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:42.822 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:42.822 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:42.822 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5890 2024-09-14T23:56:42.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.822 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:42.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:42.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:56:42.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:42.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:42.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:56:42.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:42.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:42.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:56:42.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4299999999999999e-07 2024-09-14T23:56:42.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:42.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375951+0000", 2024-09-14T23:56:42.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1000000000000001e-06 2024-09-14T23:56:42.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:42.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375956+0000", 2024-09-14T23:56:42.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3370000000000001e-06 2024-09-14T23:56:42.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:42.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375958+0000", 2024-09-14T23:56:42.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4820000000000001e-06 2024-09-14T23:56:42.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:42.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426858+0000", 2024-09-14T23:56:42.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050899666000000003 2024-09-14T23:56:42.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:42.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426867+0000", 2024-09-14T23:56:42.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7219999999999994e-06 2024-09-14T23:56:42.825 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:42.825 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:42.825 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:42.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.826 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5891 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:42.826 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:56:42.826 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.4043129900000002, 2024-09-14T23:56:42.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050948067, 2024-09-14T23:56:42.826 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:42.826 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:42.826 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:56:42.826 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:42.826 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:42.826 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:42.826 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5891 2024-09-14T23:56:42.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.827 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:42.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:42.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:56:42.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:42.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:42.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:56:42.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:42.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:42.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:56:42.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0800000000000001e-07 2024-09-14T23:56:42.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:42.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375962+0000", 2024-09-14T23:56:42.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9600000000000008e-07 2024-09-14T23:56:42.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:42.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375970+0000", 2024-09-14T23:56:42.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1600000000000001e-06 2024-09-14T23:56:42.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:42.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375972+0000", 2024-09-14T23:56:42.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4849999999999999e-06 2024-09-14T23:56:42.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:42.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426902+0000", 2024-09-14T23:56:42.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050929813999999997 2024-09-14T23:56:42.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:42.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426909+0000", 2024-09-14T23:56:42.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4039999999999998e-06 2024-09-14T23:56:42.829 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:42.830 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:42.830 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:42.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.830 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5892 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:42.830 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:56:42.830 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.4042981330000002, 2024-09-14T23:56:42.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050965036999999998, 2024-09-14T23:56:42.830 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:42.830 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:42.830 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:56:42.830 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:42.831 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:42.831 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:42.831 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5892 2024-09-14T23:56:42.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.831 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:42.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:42.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:56:42.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:42.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:42.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:56:42.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:42.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:42.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:56:42.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.34e-07 2024-09-14T23:56:42.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:42.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375978+0000", 2024-09-14T23:56:42.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1880000000000001e-06 2024-09-14T23:56:42.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:42.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375981+0000", 2024-09-14T23:56:42.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2399999999999999e-06 2024-09-14T23:56:42.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:42.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375984+0000", 2024-09-14T23:56:42.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9110000000000002e-06 2024-09-14T23:56:42.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:42.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426934+0000", 2024-09-14T23:56:42.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050949912 2024-09-14T23:56:42.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426941+0000", 2024-09-14T23:56:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5519999999999999e-06 2024-09-14T23:56:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5893 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:56:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.404287536, 2024-09-14T23:56:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050975131999999999, 2024-09-14T23:56:42.835 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:42.835 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:42.835 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:56:42.835 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:42.835 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:42.835 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:42.835 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5893 2024-09-14T23:56:42.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.835 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:42.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:42.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:56:42.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:42.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:42.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:56:42.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:42.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:42.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:56:42.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2999999999999999e-07 2024-09-14T23:56:42.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:42.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375988+0000", 2024-09-14T23:56:42.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0410000000000001e-06 2024-09-14T23:56:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375992+0000", 2024-09-14T23:56:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.104e-06 2024-09-14T23:56:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375995+0000", 2024-09-14T23:56:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4310000000000001e-06 2024-09-14T23:56:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426958+0000", 2024-09-14T23:56:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050963642000000003 2024-09-14T23:56:42.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:42.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426962+0000", 2024-09-14T23:56:42.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6839999999999998e-06 2024-09-14T23:56:42.838 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:42.838 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:42.838 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:42.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.838 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5894 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:42.838 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:56:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.404276136, 2024-09-14T23:56:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050978333000000001, 2024-09-14T23:56:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:56:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5894 2024-09-14T23:56:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:56:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:42.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:42.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:56:42.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:42.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:42.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:56:42.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.23e-07 2024-09-14T23:56:42.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:42.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375999+0000", 2024-09-14T23:56:42.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9800000000000002e-07 2024-09-14T23:56:42.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:42.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376007+0000", 2024-09-14T23:56:42.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8539999999999997e-06 2024-09-14T23:56:42.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:42.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376011+0000", 2024-09-14T23:56:42.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.399e-06 2024-09-14T23:56:42.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:42.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426973+0000", 2024-09-14T23:56:42.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050962454999999997 2024-09-14T23:56:42.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:42.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426976+0000", 2024-09-14T23:56:42.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4039999999999999e-06 2024-09-14T23:56:42.842 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:42.842 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:42.842 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:42.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.842 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5895 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:42.842 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376014+0000", 2024-09-14T23:56:42.842 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.404259792, 2024-09-14T23:56:42.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050975951999999998, 2024-09-14T23:56:42.843 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:42.843 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:42.843 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:56:42.843 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:42.843 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:42.843 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:42.843 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5895 2024-09-14T23:56:42.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.843 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:42.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:42.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376014+0000", 2024-09-14T23:56:42.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:42.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:42.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376014+0000", 2024-09-14T23:56:42.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:42.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:42.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376015+0000", 2024-09-14T23:56:42.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2999999999999999e-07 2024-09-14T23:56:42.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:42.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376016+0000", 2024-09-14T23:56:42.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1659999999999999e-06 2024-09-14T23:56:42.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:42.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376024+0000", 2024-09-14T23:56:42.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8930000000000005e-06 2024-09-14T23:56:42.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:42.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376026+0000", 2024-09-14T23:56:42.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6869999999999999e-06 2024-09-14T23:56:42.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:42.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426987+0000", 2024-09-14T23:56:42.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050960601000000001 2024-09-14T23:56:42.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:42.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426990+0000", 2024-09-14T23:56:42.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3749999999999999e-06 2024-09-14T23:56:42.846 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:42.846 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:42.846 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:42.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.846 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5896 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:42.846 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:56:42.846 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.4042440549999999, 2024-09-14T23:56:42.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050975656000000001, 2024-09-14T23:56:42.847 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:42.847 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:42.847 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:56:42.847 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:42.847 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:42.847 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:42.847 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5896 2024-09-14T23:56:42.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.847 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:42.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:42.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:56:42.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:42.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:42.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:56:42.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:42.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:42.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:56:42.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5100000000000001e-07 2024-09-14T23:56:42.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:42.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376032+0000", 2024-09-14T23:56:42.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.141e-06 2024-09-14T23:56:42.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:42.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376036+0000", 2024-09-14T23:56:42.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.25e-06 2024-09-14T23:56:42.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:42.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376038+0000", 2024-09-14T23:56:42.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5469999999999999e-06 2024-09-14T23:56:42.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:42.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427002+0000", 2024-09-14T23:56:42.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050964029000000001 2024-09-14T23:56:42.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:42.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427006+0000", 2024-09-14T23:56:42.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4379999999999999e-06 2024-09-14T23:56:42.850 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:42.850 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:42.850 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:42.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.850 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5897 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:42.851 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:56:42.851 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.4042322550000002, 2024-09-14T23:56:42.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050978001000000002, 2024-09-14T23:56:42.851 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:42.851 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:42.851 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:56:42.851 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:42.851 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:42.851 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:42.851 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5897 2024-09-14T23:56:42.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.851 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:42.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:42.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:56:42.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:42.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:42.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:56:42.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:42.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:42.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:56:42.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2600000000000001e-07 2024-09-14T23:56:42.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:42.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376043+0000", 2024-09-14T23:56:42.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.054e-06 2024-09-14T23:56:42.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:42.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376047+0000", 2024-09-14T23:56:42.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0659999999999997e-06 2024-09-14T23:56:42.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:42.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376050+0000", 2024-09-14T23:56:42.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.88e-06 2024-09-14T23:56:42.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:42.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427017+0000", 2024-09-14T23:56:42.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050966453000000002 2024-09-14T23:56:42.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:42.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427020+0000", 2024-09-14T23:56:42.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3220000000000001e-06 2024-09-14T23:56:42.854 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:42.854 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:42.854 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:42.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.855 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5898 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:42.855 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376053+0000", 2024-09-14T23:56:42.855 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.4042207700000002, 2024-09-14T23:56:42.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050982024000000001, 2024-09-14T23:56:42.855 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:42.855 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:42.855 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:56:42.855 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:42.855 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:42.855 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:42.855 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5898 2024-09-14T23:56:42.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.855 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:42.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:42.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376053+0000", 2024-09-14T23:56:42.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:42.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:42.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376053+0000", 2024-09-14T23:56:42.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:42.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:42.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376278+0000", 2024-09-14T23:56:42.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022426400000000001 2024-09-14T23:56:42.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:42.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376282+0000", 2024-09-14T23:56:42.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4739999999999999e-06 2024-09-14T23:56:42.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:42.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376296+0000", 2024-09-14T23:56:42.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4229999999999999e-05 2024-09-14T23:56:42.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:42.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376303+0000", 2024-09-14T23:56:42.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2199999999999997e-06 2024-09-14T23:56:42.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:42.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427031+0000", 2024-09-14T23:56:42.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050728154999999997 2024-09-14T23:56:42.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:42.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427036+0000", 2024-09-14T23:56:42.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6809999999999997e-06 2024-09-14T23:56:42.858 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:42.858 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:42.858 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:42.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.859 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5899 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:42.859 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376338+0000", 2024-09-14T23:56:42.859 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.4039360460000001, 2024-09-14T23:56:42.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050712100000000003, 2024-09-14T23:56:42.859 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:42.859 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:42.859 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:56:42.859 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:42.859 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:42.859 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:42.859 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5899 2024-09-14T23:56:42.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.859 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:42.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:42.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376338+0000", 2024-09-14T23:56:42.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:42.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:42.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376338+0000", 2024-09-14T23:56:42.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:42.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:42.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376339+0000", 2024-09-14T23:56:42.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9100000000000003e-07 2024-09-14T23:56:42.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:42.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376342+0000", 2024-09-14T23:56:42.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7e-06 2024-09-14T23:56:42.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:42.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376350+0000", 2024-09-14T23:56:42.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5280000000000003e-06 2024-09-14T23:56:42.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:42.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376356+0000", 2024-09-14T23:56:42.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6010000000000002e-06 2024-09-14T23:56:42.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:42.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427047+0000", 2024-09-14T23:56:42.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050690962999999999 2024-09-14T23:56:42.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:42.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427050+0000", 2024-09-14T23:56:42.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.517e-06 2024-09-14T23:56:42.862 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:42.862 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:42.862 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:42.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.863 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5900 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:42.863 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376366+0000", 2024-09-14T23:56:42.863 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.4039077870000001, 2024-09-14T23:56:42.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050697858999999998, 2024-09-14T23:56:42.863 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:42.863 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:42.863 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:56:42.863 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:42.863 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:42.863 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:42.863 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5900 2024-09-14T23:56:42.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.863 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:42.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.864 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:42.864 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376366+0000", 2024-09-14T23:56:42.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:42.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.864 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:42.864 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376366+0000", 2024-09-14T23:56:42.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:42.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.864 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:42.864 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377650+0000", 2024-09-14T23:56:42.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001283834 2024-09-14T23:56:42.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:42.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377655+0000", 2024-09-14T23:56:42.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7230000000000004e-06 2024-09-14T23:56:42.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:42.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377667+0000", 2024-09-14T23:56:42.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1746000000000001e-05 2024-09-14T23:56:42.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:42.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377676+0000", 2024-09-14T23:56:42.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3619999999999995e-06 2024-09-14T23:56:42.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:42.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427061+0000", 2024-09-14T23:56:42.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049384755000000002 2024-09-14T23:56:42.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:42.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:42.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:42.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427064+0000", 2024-09-14T23:56:42.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.439e-06 2024-09-14T23:56:42.866 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:42.867 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:42.867 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:42.867 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:42.867 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:42.867 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:56:42.868 INFO:teuthology.orchestra.run.smithi042.stdout:265 writesame 0xd61df0f thru 0xd61e68c data_size 0x7(0x77e bytes) 2024-09-14T23:56:42.869 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T23:56:42.870 INFO:teuthology.orchestra.run.smithi042.stdout:266 write 0xd9395a8 thru 0xd9464a4 (0xcefd bytes) 2024-09-14T23:56:42.870 INFO:teuthology.orchestra.run.smithi042.stdout:267 trunc from 0xd9464a5 to 0xb1825fe 2024-09-14T23:56:42.870 INFO:teuthology.orchestra.run.smithi042.stdout:269 read 0x47224c2 thru 0x4728198 (0x5cd7 bytes) 2024-09-14T23:56:42.871 INFO:teuthology.orchestra.run.smithi042.stdout:270 write 0xd064b2c thru 0xd072fc2 (0xe497 bytes) 2024-09-14T23:56:42.871 INFO:teuthology.orchestra.run.smithi042.stdout:271 read 0x5b56094 thru 0x5b6351c (0xd489 bytes) 2024-09-14T23:56:42.871 INFO:teuthology.orchestra.run.smithi042.stdout:274 writesame 0xbf8169a thru 0xbf816d6 data_size 0x1(0x3d bytes) 2024-09-14T23:56:42.871 INFO:teuthology.orchestra.run.smithi042.stdout:275 read 0xa86dd83 thru 0xa8768ab (0x8b29 bytes) 2024-09-14T23:56:42.871 INFO:teuthology.orchestra.run.smithi042.stdout:276 write 0x377a51f thru 0x3789e08 (0xf8ea bytes) 2024-09-14T23:56:42.871 INFO:teuthology.orchestra.run.smithi042.stdout:279 punch from 0x9783a96 to 0x9787208, (0x3772 bytes) 2024-09-14T23:56:42.871 INFO:teuthology.orchestra.run.smithi042.stdout:280 trunc from 0xd072fc3 to 0x87e89b9 2024-09-14T23:56:42.871 INFO:teuthology.orchestra.run.smithi042.stdout:281 trunc from 0x87e89b9 to 0x519ce21 2024-09-14T23:56:42.871 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:56:42.866+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:56:42.872 INFO:teuthology.orchestra.run.smithi042.stdout:282 write 0xb6b195f thru 0xb6beb17 (0xd1b9 bytes) 2024-09-14T23:56:42.872 INFO:teuthology.orchestra.run.smithi042.stdout:283 write 0x150cc7f thru 0x1512415 (0x5797 bytes) 2024-09-14T23:56:42.873 INFO:teuthology.orchestra.run.smithi042.stdout:284 write 0xc4f389 thru 0xc4f408 (0x80 bytes) 2024-09-14T23:56:42.882 INFO:teuthology.orchestra.run.smithi042.stdout:285 write 0xdcacef6 thru 0xdcb0291 (0x339c bytes) 2024-09-14T23:56:42.892 INFO:teuthology.orchestra.run.smithi042.stdout:286 write 0xe732241 thru 0xe738b9d (0x695d bytes) 2024-09-14T23:56:42.896 INFO:teuthology.orchestra.run.smithi042.stdout:288 write 0xb6dc68d thru 0xb6e4c57 (0x85cb bytes) 2024-09-14T23:56:42.897 INFO:teuthology.orchestra.run.smithi042.stdout:289 read 0x914434a thru 0x914969c (0x5353 bytes) 2024-09-14T23:56:42.898 INFO:teuthology.orchestra.run.smithi042.stdout:291 write 0xe41d489 thru 0xe41e572 (0x10ea bytes) 2024-09-14T23:56:42.903 INFO:teuthology.orchestra.run.smithi042.stdout:294 trunc from 0xe738b9e to 0xbe4a089 2024-09-14T23:56:42.914 INFO:teuthology.orchestra.run.smithi042.stdout:295 punch from 0x2974694 to 0x29767de, (0x214a bytes) 2024-09-14T23:56:42.914 INFO:teuthology.orchestra.run.smithi042.stdout:297 punch from 0xaf62ea1 to 0xaf6e823, (0xb982 bytes) 2024-09-14T23:56:42.914 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:42.914 INFO:teuthology.orchestra.run.smithi042.stdout:299 read 0x9363a83 thru 0x93692b4 (0x5832 bytes) 2024-09-14T23:56:42.914 INFO:teuthology.orchestra.run.smithi042.stdout:301 read 0x8a37d06 thru 0x8a3da5e (0x5d59 bytes) 2024-09-14T23:56:42.914 INFO:teuthology.orchestra.run.smithi042.stdout:302 write 0x4965146 thru 0x4970e64 (0xbd1f bytes) 2024-09-14T23:56:42.915 INFO:teuthology.orchestra.run.smithi042.stdout:303 read 0x665cbc0 thru 0x6666665 (0x9aa6 bytes) 2024-09-14T23:56:42.915 INFO:teuthology.orchestra.run.smithi042.stdout:304 write 0x1d8ec46 thru 0x1d940cb (0x5486 bytes) 2024-09-14T23:56:42.928 INFO:teuthology.orchestra.run.smithi042.stdout:305 trunc from 0xbe4a089 to 0x10cc9a5 2024-09-14T23:56:42.941 INFO:teuthology.orchestra.run.smithi042.stdout:306 punch from 0xc1336 to 0xc3908, (0x25d2 bytes) 2024-09-14T23:56:42.941 INFO:teuthology.orchestra.run.smithi042.stdout:307 read 0xb4bf69 thru 0xb4e316 (0x23ae bytes) 2024-09-14T23:56:42.947 INFO:teuthology.orchestra.run.smithi042.stdout:308 writesame 0x1f8c1aa thru 0x1f8debd data_size 0x2(0x1d14 bytes) 2024-09-14T23:56:42.961 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is set 2024-09-14T23:56:42.969 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:56:42.968+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:56:42.970 INFO:teuthology.orchestra.run.smithi042.stdout:309 writesame 0xb63774f thru 0xb639699 data_size 0x1(0x1f4b bytes) 2024-09-14T23:56:42.970 INFO:teuthology.orchestra.run.smithi042.stdout:310 read 0x7521fd thru 0x756db5 (0x4bb9 bytes) 2024-09-14T23:56:42.970 INFO:teuthology.orchestra.run.smithi042.stdout:311 writesame 0x20e48f1 thru 0x20e8ddc data_size 0x16(0x44ec bytes) 2024-09-14T23:56:42.972 INFO:teuthology.orchestra.run.smithi042.stdout:313 write 0x45b827a thru 0x45c5f62 (0xdce9 bytes) 2024-09-14T23:56:42.974 INFO:teuthology.orchestra.run.smithi042.stdout:314 read 0x3d197f0 thru 0x3d1dd59 (0x456a bytes) 2024-09-14T23:56:42.974 INFO:teuthology.orchestra.run.smithi042.stdout:316 read 0x188c5b5 thru 0x18913c9 (0x4e15 bytes) 2024-09-14T23:56:42.974 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:42.982 INFO:teuthology.orchestra.run.smithi042.stdout:319 trunc from 0xb63969a to 0x4b9c543 2024-09-14T23:56:42.992 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T23:56:42.992 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T23:56:42.992 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-14T23:56:42.992 INFO:tasks.thrashosds.thrasher:Setting osd 4 primary_affinity to 0.970352 2024-09-14T23:56:42.992 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 0.9703515352778888 2024-09-14T23:56:42.993 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:43.005 INFO:teuthology.orchestra.run.smithi042.stdout:321 write 0xb11c983 thru 0xb127b3b (0xb1b9 bytes) 2024-09-14T23:56:43.008 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:56:43.008 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:56:43.008 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:56:43.008 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:56:43.011 INFO:teuthology.orchestra.run.smithi042.stdout:322 trunc from 0xb127b3c to 0xd2c933a 2024-09-14T23:56:43.017 INFO:teuthology.orchestra.run.smithi042.stdout:323 read 0x752d711 thru 0x753bb84 (0xe474 bytes) 2024-09-14T23:56:43.017 INFO:teuthology.orchestra.run.smithi042.stdout:324 read 0x4537110 thru 0x45459f3 (0xe8e4 bytes) 2024-09-14T23:56:43.018 INFO:teuthology.orchestra.run.smithi042.stdout:325 read 0x1df4b21 thru 0x1e008c7 (0xbda7 bytes) 2024-09-14T23:56:43.018 INFO:teuthology.orchestra.run.smithi042.stdout:326 writesame 0xcd30877 thru 0xcd3ef06 data_size 0x70(0xe690 bytes) 2024-09-14T23:56:43.020 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T23:56:43.022 INFO:teuthology.orchestra.run.smithi042.stdout:327 writesame 0x1d0247a thru 0x1d0fbe6 data_size 0x1f(0xd76d bytes) 2024-09-14T23:56:43.024 INFO:teuthology.orchestra.run.smithi042.stdout:328 punch from 0x5fb5cfb to 0x5fbc271, (0x6576 bytes) 2024-09-14T23:56:43.024 INFO:teuthology.orchestra.run.smithi042.stdout:329 read 0x9f1dec6 thru 0x9f29c3a (0xbd75 bytes) 2024-09-14T23:56:43.024 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:43.024 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:43.024 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:43.024 INFO:teuthology.orchestra.run.smithi042.stdout:334 read 0x5e1ac9f thru 0x5e2326c (0x85ce bytes) 2024-09-14T23:56:43.025 INFO:teuthology.orchestra.run.smithi042.stdout:335 writesame 0x2394bb9 thru 0x23a24a2 data_size 0x2d(0xd8ea bytes) 2024-09-14T23:56:43.026 INFO:teuthology.orchestra.run.smithi042.stdout:337 read 0x6b318ed thru 0x6b3dfdb (0xc6ef bytes) 2024-09-14T23:56:43.026 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:43.026 INFO:teuthology.orchestra.run.smithi042.stdout:343 read 0x1050cb6 thru 0x105c86e (0xbbb9 bytes) 2024-09-14T23:56:43.027 INFO:teuthology.orchestra.run.smithi042.stdout:344 read 0x4616696 thru 0x461d131 (0x6a9c bytes) 2024-09-14T23:56:43.032 INFO:teuthology.orchestra.run.smithi042.stdout:345 trunc from 0xd2c933a to 0x91b1b3a 2024-09-14T23:56:43.043 INFO:teuthology.orchestra.run.smithi042.stdout:346 punch from 0x8a87dd7 to 0x8a94afd, (0xcd26 bytes) 2024-09-14T23:56:43.055 INFO:teuthology.orchestra.run.smithi042.stdout:347 write 0xd5272c1 thru 0xd532998 (0xb6d8 bytes) 2024-09-14T23:56:43.058 INFO:teuthology.orchestra.run.smithi042.stdout:349 read 0x1766067 thru 0x17661ea (0x184 bytes) 2024-09-14T23:56:43.059 INFO:teuthology.orchestra.run.smithi042.stdout:351 read 0x9be717d thru 0x9bf4ebb (0xdd3f bytes) 2024-09-14T23:56:43.059 INFO:teuthology.orchestra.run.smithi042.stdout:352 trunc from 0xd532999 to 0xd16234a 2024-09-14T23:56:43.066 INFO:teuthology.orchestra.run.smithi042.stdout:353 read 0x14a6ac2 thru 0x14abffc (0x553b bytes) 2024-09-14T23:56:43.070 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:56:43.069+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:56:43.071 INFO:teuthology.orchestra.run.smithi042.stdout:354 trunc from 0xd16234a to 0x95aae69 2024-09-14T23:56:43.093 INFO:teuthology.orchestra.run.smithi042.stdout:356 write 0xdb760af thru 0xdb7b632 (0x5584 bytes) 2024-09-14T23:56:43.096 INFO:teuthology.orchestra.run.smithi042.stdout:359 write 0x71271e7 thru 0x712ad97 (0x3bb1 bytes) 2024-09-14T23:56:43.099 INFO:teuthology.orchestra.run.smithi042.stdout:360 write 0xcba0bcc thru 0xcba357d (0x29b2 bytes) 2024-09-14T23:56:43.103 INFO:teuthology.orchestra.run.smithi042.stdout:361 write 0x2894890 thru 0x289e2bb (0x9a2c bytes) 2024-09-14T23:56:43.104 INFO:teuthology.orchestra.run.smithi042.stdout:362 punch from 0xb884a22 to 0xb88e598, (0x9b76 bytes) 2024-09-14T23:56:43.116 INFO:teuthology.orchestra.run.smithi042.stdout:363 trunc from 0xdb7b633 to 0x4971e18 2024-09-14T23:56:43.131 INFO:teuthology.orchestra.run.smithi042.stdout:364 read 0x44f9cef thru 0x44fc4f1 (0x2803 bytes) 2024-09-14T23:56:43.132 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:43.132 INFO:teuthology.orchestra.run.smithi042.stdout:366 punch from 0x2bbbca4 to 0x2bbc0eb, (0x447 bytes) 2024-09-14T23:56:43.132 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:43.141 INFO:teuthology.orchestra.run.smithi042.stdout:368 trunc from 0x4971e18 to 0xc30517d 2024-09-14T23:56:43.148 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:43.148 INFO:teuthology.orchestra.run.smithi042.stdout:371 write 0xc08a8ba thru 0xc0942cc (0x9a13 bytes) 2024-09-14T23:56:43.151 INFO:teuthology.orchestra.run.smithi042.stdout:372 punch from 0x8c81f03 to 0x8c91bb3, (0xfcb0 bytes) 2024-09-14T23:56:43.151 INFO:teuthology.orchestra.run.smithi042.stdout:373 read 0x76039ae thru 0x760c184 (0x87d7 bytes) 2024-09-14T23:56:43.151 INFO:teuthology.orchestra.run.smithi042.stdout:376 read 0x71577cf thru 0x7162f73 (0xb7a5 bytes) 2024-09-14T23:56:43.151 INFO:teuthology.orchestra.run.smithi042.stdout:377 punch from 0x50dacf0 to 0x50e5c47, (0xaf57 bytes) 2024-09-14T23:56:43.157 INFO:teuthology.orchestra.run.smithi042.stdout:378 write 0xc598373 thru 0xc5a6207 (0xde95 bytes) 2024-09-14T23:56:43.160 INFO:teuthology.orchestra.run.smithi042.stdout:379 read 0xd889f4 thru 0xd969dc (0xdfe9 bytes) 2024-09-14T23:56:43.161 INFO:teuthology.orchestra.run.smithi042.stdout:380 trunc from 0xc5a6208 to 0xd45273d 2024-09-14T23:56:43.168 INFO:teuthology.orchestra.run.smithi042.stdout:382 write 0x3f1425d thru 0x3f201c0 (0xbf64 bytes) 2024-09-14T23:56:43.168 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:56:43.168 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:56:43.168 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:56:43.168 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:56:43.168 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:56:43.169 INFO:teuthology.orchestra.run.smithi042.stdout:384 punch from 0xd23d852 to 0xd23fdf0, (0x259e bytes) 2024-09-14T23:56:43.169 INFO:teuthology.orchestra.run.smithi042.stdout:386 writesame 0x501da3b thru 0x501ec01 data_size 0x25(0x11c7 bytes) 2024-09-14T23:56:43.170 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:56:43.169+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:56:43.171 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:43.171 INFO:teuthology.orchestra.run.smithi042.stdout:388 punch from 0x782e4b to 0x783c04, (0xdb9 bytes) 2024-09-14T23:56:43.171 INFO:teuthology.orchestra.run.smithi042.stdout:389 write 0x9d9ebd7 thru 0x9da996a (0xad94 bytes) 2024-09-14T23:56:43.174 INFO:teuthology.orchestra.run.smithi042.stdout:390 writesame 0x782e232 thru 0x783d38b data_size 0x1(0xf15a bytes) 2024-09-14T23:56:43.179 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T23:56:43.200 INFO:teuthology.orchestra.run.smithi042.stdout:391 read 0xc600e13 thru 0xc60aa1e (0x9c0c bytes) 2024-09-14T23:56:43.213 INFO:teuthology.orchestra.run.smithi042.stdout:392 trunc from 0xd45273d to 0x19afc9f 2024-09-14T23:56:43.247 INFO:teuthology.orchestra.run.smithi042.stdout:395 write 0xc4d7550 thru 0xc4e247d (0xaf2e bytes) 2024-09-14T23:56:43.260 INFO:teuthology.orchestra.run.smithi042.stdout:398 trunc from 0xc4e247e to 0x3222839 2024-09-14T23:56:43.271 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:56:43.270+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:56:43.272 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:43.273 INFO:teuthology.orchestra.run.smithi042.stdout:400 trunc from 0x3222839 to 0x3b2bd58 2024-09-14T23:56:43.289 INFO:teuthology.orchestra.run.smithi042.stdout:401 trunc from 0x3b2bd58 to 0xc6f0ae9 2024-09-14T23:56:43.294 INFO:teuthology.orchestra.run.smithi042.stdout:403 punch from 0x1db10c2 to 0x1db5a78, (0x49b6 bytes) 2024-09-14T23:56:43.295 INFO:teuthology.orchestra.run.smithi042.stdout:406 read 0x11e0846 thru 0x11e592e (0x50e9 bytes) 2024-09-14T23:56:43.295 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:43.296 INFO:teuthology.orchestra.run.smithi042.stdout:409 punch from 0x361e015 to 0x36244b4, (0x649f bytes) 2024-09-14T23:56:43.296 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:43.296 INFO:teuthology.orchestra.run.smithi042.stdout:411 read 0xc2e235b thru 0xc2e5da6 (0x3a4c bytes) 2024-09-14T23:56:43.296 INFO:teuthology.orchestra.run.smithi042.stdout:412 write 0x86eb870 thru 0x86ec54f (0xce0 bytes) 2024-09-14T23:56:43.298 INFO:teuthology.orchestra.run.smithi042.stdout:415 writesame 0x17e032a thru 0x17e3565 data_size 0x1(0x323c bytes) 2024-09-14T23:56:43.305 INFO:teuthology.orchestra.run.smithi042.stdout:416 punch from 0x3cf6046 to 0x3cf6d64, (0xd1e bytes) 2024-09-14T23:56:43.316 INFO:teuthology.orchestra.run.smithi042.stdout:418 trunc from 0xc6f0ae9 to 0x244466a 2024-09-14T23:56:43.334 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:56:43.334 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:56:43.334 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:56:43.334 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:56:43.334 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.334 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4284.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-14T23:56:43.335 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:22.527811+0000", 2024-09-14T23:56:43.335 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.804070648, 2024-09-14T23:56:43.335 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00113675, 2024-09-14T23:56:43.335 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:43.335 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:43.335 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:43.335 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:43.335 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.335 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:43.335 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:22.527811+0000", 2024-09-14T23:56:43.335 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.335 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.335 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.336 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:43.336 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:22.527811+0000", 2024-09-14T23:56:43.336 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.336 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:43.336 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:22.527811+0000", 2024-09-14T23:56:43.336 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4799999999999998e-07 2024-09-14T23:56:43.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.336 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:43.336 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:22.527826+0000", 2024-09-14T23:56:43.336 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4436999999999999e-05 2024-09-14T23:56:43.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.337 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.337 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:43.337 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:22.527844+0000", 2024-09-14T23:56:43.337 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7878999999999999e-05 2024-09-14T23:56:43.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.337 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.337 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:43.337 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:22.527856+0000", 2024-09-14T23:56:43.337 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2089000000000001e-05 2024-09-14T23:56:43.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.337 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.337 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:43.337 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:22.527937+0000", 2024-09-14T23:56:43.338 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0852000000000004e-05 2024-09-14T23:56:43.338 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.338 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.338 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:43.338 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:22.527971+0000", 2024-09-14T23:56:43.338 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4069000000000002e-05 2024-09-14T23:56:43.338 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.338 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.338 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:43.338 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:22.528885+0000", 2024-09-14T23:56:43.338 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00091406300000000001 2024-09-14T23:56:43.338 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.338 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.338 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:43.339 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:22.528948+0000", 2024-09-14T23:56:43.339 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2812999999999996e-05 2024-09-14T23:56:43.339 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.339 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:43.339 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.339 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.339 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "pg_update_log_missing(3.4 epoch 29/23 rep_tid 20 entries 29'12 (0'0) error 3:2298d860:::rbd_data.1118de1168b.0000000000000030:head by client.4376.0:3569 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 27'10)", 2024-09-14T23:56:43.339 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.033832+0000", 2024-09-14T23:56:43.339 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.2980499979999998, 2024-09-14T23:56:43.339 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011863259999999999, 2024-09-14T23:56:43.339 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:43.339 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:43.339 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:56:43.340 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:43.340 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.340 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:43.340 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.033832+0000", 2024-09-14T23:56:43.340 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.340 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.340 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:43.340 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.033832+0000", 2024-09-14T23:56:43.340 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.340 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.340 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:43.341 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.033833+0000", 2024-09-14T23:56:43.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.356e-06 2024-09-14T23:56:43.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.341 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.341 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:43.341 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.033847+0000", 2024-09-14T23:56:43.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4099e-05 2024-09-14T23:56:43.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.341 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.341 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:43.341 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.033880+0000", 2024-09-14T23:56:43.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2659e-05 2024-09-14T23:56:43.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.341 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.342 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:43.342 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.033904+0000", 2024-09-14T23:56:43.342 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4184000000000001e-05 2024-09-14T23:56:43.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.342 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.342 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:43.342 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.034005+0000", 2024-09-14T23:56:43.342 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010145900000000001 2024-09-14T23:56:43.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.342 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.342 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:43.342 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.035018+0000", 2024-09-14T23:56:43.342 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010125690000000001 2024-09-14T23:56:43.342 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.343 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:43.343 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.343 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:3578 3.4 e29/23 3:2298d860:::rbd_data.1118de1168b.0000000000000030:head v 29'15, mlcod=29'13)", 2024-09-14T23:56:43.343 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.036455+0000", 2024-09-14T23:56:43.343 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.2954264159999997, 2024-09-14T23:56:43.343 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019449440000000001, 2024-09-14T23:56:43.343 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:43.343 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:43.343 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:43.343 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:43.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.343 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:43.344 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036455+0000", 2024-09-14T23:56:43.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.344 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:43.344 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036455+0000", 2024-09-14T23:56:43.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.344 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:43.344 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036456+0000", 2024-09-14T23:56:43.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0999999999999997e-07 2024-09-14T23:56:43.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:43.345 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036463+0000", 2024-09-14T23:56:43.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2200000000000003e-06 2024-09-14T23:56:43.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:43.345 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036478+0000", 2024-09-14T23:56:43.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4977000000000001e-05 2024-09-14T23:56:43.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:43.345 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036485+0000", 2024-09-14T23:56:43.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0559999999999999e-06 2024-09-14T23:56:43.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.346 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:43.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036693+0000", 2024-09-14T23:56:43.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000208051 2024-09-14T23:56:43.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.346 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:43.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036715+0000", 2024-09-14T23:56:43.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1330000000000001e-05 2024-09-14T23:56:43.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.346 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:43.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.038356+0000", 2024-09-14T23:56:43.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001641586 2024-09-14T23:56:43.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.347 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:43.347 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.038400+0000", 2024-09-14T23:56:43.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3914e-05 2024-09-14T23:56:43.347 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.347 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:43.347 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.347 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:3579 3.4 e29/23 3:27290fe6:::rbd_data.1118de1168b.0000000000000032:head v 29'16, mlcod=29'13)", 2024-09-14T23:56:43.347 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.037085+0000", 2024-09-14T23:56:43.347 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.2947968259999998, 2024-09-14T23:56:43.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0020019220000000002, 2024-09-14T23:56:43.347 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:43.348 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:43.348 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:43.348 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:43.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.348 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:43.348 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037085+0000", 2024-09-14T23:56:43.348 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.348 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:43.348 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037085+0000", 2024-09-14T23:56:43.348 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.349 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:43.349 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037085+0000", 2024-09-14T23:56:43.349 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5499999999999997e-07 2024-09-14T23:56:43.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.349 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:43.349 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037091+0000", 2024-09-14T23:56:43.349 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7869999999999998e-06 2024-09-14T23:56:43.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.349 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:43.349 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037102+0000", 2024-09-14T23:56:43.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.031e-05 2024-09-14T23:56:43.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.350 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:43.350 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037116+0000", 2024-09-14T23:56:43.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4119e-05 2024-09-14T23:56:43.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.350 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:43.350 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037219+0000", 2024-09-14T23:56:43.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010372799999999999 2024-09-14T23:56:43.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.350 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:43.350 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037239+0000", 2024-09-14T23:56:43.351 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9993e-05 2024-09-14T23:56:43.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.351 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:43.351 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.039046+0000", 2024-09-14T23:56:43.351 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001806666 2024-09-14T23:56:43.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.351 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:43.352 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.039087+0000", 2024-09-14T23:56:43.352 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0564e-05 2024-09-14T23:56:43.352 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.352 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:43.352 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.352 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.352 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:3584 3.4 e29/23 3:2779453d:::rbd_data.1118de1168b.0000000000000034:head v 29'17, mlcod=29'13)", 2024-09-14T23:56:43.352 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.037588+0000", 2024-09-14T23:56:43.352 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.294293787, 2024-09-14T23:56:43.352 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001526828, 2024-09-14T23:56:43.352 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:43.352 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:43.353 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:43.353 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:43.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:43.353 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037588+0000", 2024-09-14T23:56:43.353 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:43.353 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037588+0000", 2024-09-14T23:56:43.353 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.354 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:43.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037588+0000", 2024-09-14T23:56:43.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1200000000000002e-07 2024-09-14T23:56:43.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.354 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:43.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037595+0000", 2024-09-14T23:56:43.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7580000000000003e-06 2024-09-14T23:56:43.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.354 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:43.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037609+0000", 2024-09-14T23:56:43.355 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4037e-05 2024-09-14T23:56:43.355 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.355 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.355 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:43.355 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037616+0000", 2024-09-14T23:56:43.355 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0509999999999999e-06 2024-09-14T23:56:43.355 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.355 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.355 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:43.355 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037787+0000", 2024-09-14T23:56:43.355 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017084899999999999 2024-09-14T23:56:43.355 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.355 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.356 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:43.356 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037806+0000", 2024-09-14T23:56:43.356 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9159000000000001e-05 2024-09-14T23:56:43.356 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.356 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.356 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:43.356 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.039089+0000", 2024-09-14T23:56:43.356 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012823909999999999 2024-09-14T23:56:43.356 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.356 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.356 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:43.356 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.039115+0000", 2024-09-14T23:56:43.356 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5871e-05 2024-09-14T23:56:43.357 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.357 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:43.357 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.357 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.357 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.357 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:5280 3.0 e29/23 3:0760f535:::rbd_data.1118de1168b.000000000000009a:head v 29'11, mlcod=29'9)", 2024-09-14T23:56:43.357 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.262702+0000", 2024-09-14T23:56:43.357 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.069179407, 2024-09-14T23:56:43.357 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001433122, 2024-09-14T23:56:43.357 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:43.357 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:43.357 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:43.357 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:43.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.358 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:43.358 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262702+0000", 2024-09-14T23:56:43.358 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.358 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:43.358 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262702+0000", 2024-09-14T23:56:43.358 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.358 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:43.358 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262704+0000", 2024-09-14T23:56:43.358 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.384e-06 2024-09-14T23:56:43.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.359 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.359 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:43.359 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262791+0000", 2024-09-14T23:56:43.359 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7264999999999999e-05 2024-09-14T23:56:43.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.359 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.359 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:43.359 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262813+0000", 2024-09-14T23:56:43.359 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.173e-05 2024-09-14T23:56:43.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.359 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.359 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:43.359 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262822+0000", 2024-09-14T23:56:43.360 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5929999999999995e-06 2024-09-14T23:56:43.360 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.360 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.360 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:43.360 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262886+0000", 2024-09-14T23:56:43.360 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4052000000000003e-05 2024-09-14T23:56:43.360 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.360 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.360 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:43.360 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262920+0000", 2024-09-14T23:56:43.360 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.366e-05 2024-09-14T23:56:43.360 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.360 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.360 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:43.361 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.264095+0000", 2024-09-14T23:56:43.361 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001175178 2024-09-14T23:56:43.361 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.361 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.361 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:43.361 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.264135+0000", 2024-09-14T23:56:43.361 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0259999999999997e-05 2024-09-14T23:56:43.361 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.361 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:43.361 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.361 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.361 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.361 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:5281 3.0 e29/23 3:0760f535:::rbd_data.1118de1168b.000000000000009a:head v 29'13, mlcod=29'9)", 2024-09-14T23:56:43.361 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.263503+0000", 2024-09-14T23:56:43.362 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.0683785429999997, 2024-09-14T23:56:43.362 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012834910000000001, 2024-09-14T23:56:43.362 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:43.362 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:43.362 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:43.362 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:43.362 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.362 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:43.362 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.263503+0000", 2024-09-14T23:56:43.362 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.362 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.362 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.362 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:43.363 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.263503+0000", 2024-09-14T23:56:43.363 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.363 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.363 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:43.363 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.263504+0000", 2024-09-14T23:56:43.363 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4600000000000004e-07 2024-09-14T23:56:43.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.363 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.363 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:43.363 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.263510+0000", 2024-09-14T23:56:43.363 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5950000000000004e-06 2024-09-14T23:56:43.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.364 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.364 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:43.364 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.263522+0000", 2024-09-14T23:56:43.364 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1289e-05 2024-09-14T23:56:43.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.364 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.364 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:43.364 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.263529+0000", 2024-09-14T23:56:43.364 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.951e-06 2024-09-14T23:56:43.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.364 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.364 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:43.364 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.263656+0000", 2024-09-14T23:56:43.364 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012731099999999999 2024-09-14T23:56:43.365 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.365 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.365 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:43.365 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.263675+0000", 2024-09-14T23:56:43.365 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9147000000000002e-05 2024-09-14T23:56:43.365 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.365 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.365 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:43.365 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.264746+0000", 2024-09-14T23:56:43.365 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001070997 2024-09-14T23:56:43.365 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.365 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.365 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:43.365 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.264787+0000", 2024-09-14T23:56:43.366 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0555e-05 2024-09-14T23:56:43.366 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.366 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:43.366 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.366 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.366 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.366 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:5599 3.4 e29/23 3:248aa036:::rbd_data.1118de1168b.00000000000000e2:head v 29'21, mlcod=29'18)", 2024-09-14T23:56:43.366 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.355560+0000", 2024-09-14T23:56:43.366 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.9763214410000001, 2024-09-14T23:56:43.366 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001151369, 2024-09-14T23:56:43.366 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:43.366 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:43.366 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:43.367 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:43.367 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.367 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:43.367 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.355560+0000", 2024-09-14T23:56:43.367 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.367 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.367 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:43.367 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.355560+0000", 2024-09-14T23:56:43.367 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.367 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.367 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:43.367 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.355562+0000", 2024-09-14T23:56:43.368 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.409e-06 2024-09-14T23:56:43.368 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.368 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.368 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:43.368 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.355573+0000", 2024-09-14T23:56:43.368 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1453e-05 2024-09-14T23:56:43.368 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.368 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.368 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:43.368 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.355593+0000", 2024-09-14T23:56:43.368 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0290999999999999e-05 2024-09-14T23:56:43.368 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.368 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.368 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:43.369 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.355604+0000", 2024-09-14T23:56:43.369 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0983e-05 2024-09-14T23:56:43.369 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.369 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.369 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:43.369 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.355674+0000", 2024-09-14T23:56:43.369 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0233000000000003e-05 2024-09-14T23:56:43.369 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.369 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.369 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:43.369 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.355696+0000", 2024-09-14T23:56:43.369 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.181e-05 2024-09-14T23:56:43.369 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.370 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.370 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:43.370 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.356670+0000", 2024-09-14T23:56:43.370 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00097364299999999995 2024-09-14T23:56:43.370 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.370 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.370 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:43.370 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.356711+0000", 2024-09-14T23:56:43.370 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1547e-05 2024-09-14T23:56:43.370 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.370 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:43.370 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.370 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.370 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.370 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:6082 3.9 e29/23 3:97525414:::rbd_data.1118de1168b.000000000000010c:head v 29'6, mlcod=29'4)", 2024-09-14T23:56:43.371 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.411014+0000", 2024-09-14T23:56:43.371 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.920867415, 2024-09-14T23:56:43.371 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010181719999999999, 2024-09-14T23:56:43.371 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:43.371 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:43.371 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:43.371 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:43.371 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.371 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:43.371 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.411014+0000", 2024-09-14T23:56:43.371 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.371 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.371 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.371 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:43.372 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.411014+0000", 2024-09-14T23:56:43.372 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.372 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.372 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.372 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:43.372 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.411015+0000", 2024-09-14T23:56:43.372 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9699999999999995e-07 2024-09-14T23:56:43.372 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.372 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.372 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:43.372 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.411021+0000", 2024-09-14T23:56:43.372 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2809999999999997e-06 2024-09-14T23:56:43.372 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.372 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.373 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:43.373 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.411035+0000", 2024-09-14T23:56:43.373 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3471e-05 2024-09-14T23:56:43.373 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.373 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.373 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:43.373 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.411042+0000", 2024-09-14T23:56:43.373 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3640000000000001e-06 2024-09-14T23:56:43.373 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.373 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.373 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:43.373 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.411085+0000", 2024-09-14T23:56:43.373 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3408999999999997e-05 2024-09-14T23:56:43.374 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.374 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.374 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:43.374 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.411104+0000", 2024-09-14T23:56:43.374 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8139e-05 2024-09-14T23:56:43.374 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.374 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.374 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:43.374 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.411987+0000", 2024-09-14T23:56:43.374 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00088333400000000005 2024-09-14T23:56:43.374 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.374 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.374 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:43.374 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.412032+0000", 2024-09-14T23:56:43.374 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5476999999999998e-05 2024-09-14T23:56:43.375 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.375 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:43.375 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.375 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.375 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.375 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:6210 3.0 e29/23 3:02712f44:::rbd_data.1118de1168b.000000000000014f:head v 29'19, mlcod=29'17)", 2024-09-14T23:56:43.375 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.470852+0000", 2024-09-14T23:56:43.375 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.8610300120000001, 2024-09-14T23:56:43.375 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010735250000000001, 2024-09-14T23:56:43.375 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:43.375 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:43.375 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:43.375 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:43.375 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.376 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:43.376 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.470852+0000", 2024-09-14T23:56:43.376 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.376 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.376 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.376 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:43.376 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.470852+0000", 2024-09-14T23:56:43.376 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.376 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.376 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.376 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:43.376 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.470852+0000", 2024-09-14T23:56:43.376 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1099999999999995e-07 2024-09-14T23:56:43.377 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.377 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.377 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:43.377 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.470869+0000", 2024-09-14T23:56:43.377 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6399e-05 2024-09-14T23:56:43.377 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.377 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.377 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:43.377 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.470879+0000", 2024-09-14T23:56:43.377 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.029e-05 2024-09-14T23:56:43.377 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.377 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.377 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:43.377 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.470886+0000", 2024-09-14T23:56:43.378 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6209999999999998e-06 2024-09-14T23:56:43.378 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.378 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.378 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:43.378 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.470944+0000", 2024-09-14T23:56:43.378 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8403000000000003e-05 2024-09-14T23:56:43.378 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.378 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.378 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:43.378 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.470964+0000", 2024-09-14T23:56:43.378 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9551999999999998e-05 2024-09-14T23:56:43.378 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.378 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.378 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:43.378 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.471876+0000", 2024-09-14T23:56:43.379 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00091266199999999996 2024-09-14T23:56:43.379 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.379 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.379 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:43.379 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.471925+0000", 2024-09-14T23:56:43.379 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8887000000000001e-05 2024-09-14T23:56:43.379 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.379 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:43.379 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.379 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.379 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.379 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "pg_update_log_missing(3.4 epoch 29/23 rep_tid 65 entries 29'23 (0'0) error 3:27c3fe9d:::rbd_data.1118de1168b.000000000000010b:head by client.4376.0:6078 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 29'19)", 2024-09-14T23:56:43.379 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.474315+0000", 2024-09-14T23:56:43.380 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.85756656, 2024-09-14T23:56:43.380 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00122671, 2024-09-14T23:56:43.380 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:43.380 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:43.380 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:56:43.380 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:43.380 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.380 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:43.380 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.474315+0000", 2024-09-14T23:56:43.380 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.380 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.380 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.381 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:43.381 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.474315+0000", 2024-09-14T23:56:43.381 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.381 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.381 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.381 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:43.381 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.474316+0000", 2024-09-14T23:56:43.381 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8800000000000002e-07 2024-09-14T23:56:43.381 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.381 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.381 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:43.382 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.474324+0000", 2024-09-14T23:56:43.382 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5790000000000004e-06 2024-09-14T23:56:43.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.382 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.382 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:43.382 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.474352+0000", 2024-09-14T23:56:43.382 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8246999999999999e-05 2024-09-14T23:56:43.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.382 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.382 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:43.382 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.474367+0000", 2024-09-14T23:56:43.382 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4096e-05 2024-09-14T23:56:43.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.382 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.383 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:43.383 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.474431+0000", 2024-09-14T23:56:43.383 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4947e-05 2024-09-14T23:56:43.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.383 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.383 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:43.383 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.475542+0000", 2024-09-14T23:56:43.383 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011102530000000001 2024-09-14T23:56:43.383 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.383 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:43.383 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.383 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.383 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "pg_update_log_missing(3.4 epoch 29/23 rep_tid 66 entries 29'24 (0'0) error 3:2514893c:::rbd_data.1118de1168b.0000000000000123:head by client.4376.0:6113 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 29'19)", 2024-09-14T23:56:43.384 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.474842+0000", 2024-09-14T23:56:43.384 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.8570400029999998, 2024-09-14T23:56:43.384 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00099879799999999996, 2024-09-14T23:56:43.384 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:43.384 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:43.384 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:56:43.384 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:43.384 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.384 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:43.384 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.474842+0000", 2024-09-14T23:56:43.384 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.384 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.384 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.384 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:43.385 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.474842+0000", 2024-09-14T23:56:43.385 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.385 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.385 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.385 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:43.385 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.474842+0000", 2024-09-14T23:56:43.385 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7800000000000001e-07 2024-09-14T23:56:43.385 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.385 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.386 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:43.386 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.474851+0000", 2024-09-14T23:56:43.386 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1669999999999999e-06 2024-09-14T23:56:43.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.386 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.386 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:43.386 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.474873+0000", 2024-09-14T23:56:43.386 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2639000000000001e-05 2024-09-14T23:56:43.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.386 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.386 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:43.386 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.474887+0000", 2024-09-14T23:56:43.387 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4242e-05 2024-09-14T23:56:43.387 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.387 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.387 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:43.387 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.474962+0000", 2024-09-14T23:56:43.387 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4559000000000002e-05 2024-09-14T23:56:43.387 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.387 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.387 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:43.387 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.475840+0000", 2024-09-14T23:56:43.387 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00087841299999999998 2024-09-14T23:56:43.387 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.387 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:43.387 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.388 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.388 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.388 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:6229 3.4 e29/23 3:27c3fe9d:::rbd_data.1118de1168b.000000000000010b:head v 29'28, mlcod=29'25)", 2024-09-14T23:56:43.388 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.478739+0000", 2024-09-14T23:56:43.388 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.8531425349999999, 2024-09-14T23:56:43.388 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010777060000000001, 2024-09-14T23:56:43.388 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:43.388 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:43.388 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:43.388 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:43.388 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.388 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:43.388 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.478739+0000", 2024-09-14T23:56:43.389 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.389 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.389 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.389 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:43.389 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.478739+0000", 2024-09-14T23:56:43.389 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.389 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.389 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.389 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:43.389 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.478740+0000", 2024-09-14T23:56:43.389 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8000000000000004e-07 2024-09-14T23:56:43.390 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.390 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.390 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:43.390 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.478746+0000", 2024-09-14T23:56:43.390 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9349999999999999e-06 2024-09-14T23:56:43.390 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.390 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.390 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:43.390 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.478760+0000", 2024-09-14T23:56:43.390 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4028e-05 2024-09-14T23:56:43.390 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.390 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.390 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:43.391 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.478769+0000", 2024-09-14T23:56:43.391 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6600000000000001e-06 2024-09-14T23:56:43.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.391 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.391 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:43.391 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.478828+0000", 2024-09-14T23:56:43.391 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9233e-05 2024-09-14T23:56:43.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.391 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.391 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:43.391 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.478851+0000", 2024-09-14T23:56:43.391 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3354e-05 2024-09-14T23:56:43.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.391 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.392 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:43.392 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.479775+0000", 2024-09-14T23:56:43.392 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00092360800000000002 2024-09-14T23:56:43.392 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.392 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.392 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:43.392 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.479817+0000", 2024-09-14T23:56:43.392 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2008000000000002e-05 2024-09-14T23:56:43.392 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.392 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:43.392 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.392 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.392 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.392 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:8784 3.0 e30/23 3:04302cfd:::rbd_data.1118987b60ce.0000000000000002:head v 30'21, mlcod=29'19)", 2024-09-14T23:56:43.393 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:40.026663+0000", 2024-09-14T23:56:43.393 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.3052180980000001, 2024-09-14T23:56:43.393 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001065391, 2024-09-14T23:56:43.393 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:43.393 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:43.393 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:43.393 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:43.393 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.393 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:43.393 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:40.026663+0000", 2024-09-14T23:56:43.393 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.393 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.393 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.394 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:43.394 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:40.026663+0000", 2024-09-14T23:56:43.394 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.394 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.394 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.394 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:43.394 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:40.026664+0000", 2024-09-14T23:56:43.394 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4700000000000002e-07 2024-09-14T23:56:43.394 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.394 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.394 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:43.394 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:40.026701+0000", 2024-09-14T23:56:43.394 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6944999999999999e-05 2024-09-14T23:56:43.394 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.395 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:43.395 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:40.026715+0000", 2024-09-14T23:56:43.395 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4025999999999999e-05 2024-09-14T23:56:43.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.395 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:43.395 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:40.026727+0000", 2024-09-14T23:56:43.395 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1725e-05 2024-09-14T23:56:43.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.395 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:43.395 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:40.026800+0000", 2024-09-14T23:56:43.396 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3214999999999996e-05 2024-09-14T23:56:43.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.396 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.396 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:43.396 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:40.026823+0000", 2024-09-14T23:56:43.396 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3621e-05 2024-09-14T23:56:43.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.396 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.396 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:43.396 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:40.027684+0000", 2024-09-14T23:56:43.396 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00086105399999999999 2024-09-14T23:56:43.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.396 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.396 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:43.397 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:40.027729+0000", 2024-09-14T23:56:43.397 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4458e-05 2024-09-14T23:56:43.397 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.397 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:43.397 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.397 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.397 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.397 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:17095 3.0 e32/23 3:05bad47a:::rbd_data.11187be34cc8.0000000000000003:head v 32'31, mlcod=32'29)", 2024-09-14T23:56:43.397 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:42.848795+0000", 2024-09-14T23:56:43.397 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.48308623699999997, 2024-09-14T23:56:43.397 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011548610000000001, 2024-09-14T23:56:43.397 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:43.397 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:43.397 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:43.398 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:43.398 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.398 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:43.398 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.848795+0000", 2024-09-14T23:56:43.398 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.398 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.398 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.398 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:43.398 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.848795+0000", 2024-09-14T23:56:43.398 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.398 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.398 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.398 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:43.398 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.848796+0000", 2024-09-14T23:56:43.399 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.046e-06 2024-09-14T23:56:43.399 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.399 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.399 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:43.399 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.848808+0000", 2024-09-14T23:56:43.399 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1151e-05 2024-09-14T23:56:43.399 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.399 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.399 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:43.399 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.848826+0000", 2024-09-14T23:56:43.399 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8597000000000002e-05 2024-09-14T23:56:43.399 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.399 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.399 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:43.400 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.848836+0000", 2024-09-14T23:56:43.400 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5279999999999992e-06 2024-09-14T23:56:43.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.400 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:43.400 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.848928+0000", 2024-09-14T23:56:43.400 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2750999999999993e-05 2024-09-14T23:56:43.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.400 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:43.400 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.848949+0000", 2024-09-14T23:56:43.400 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1039999999999998e-05 2024-09-14T23:56:43.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.401 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:43.401 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.849856+0000", 2024-09-14T23:56:43.401 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00090634299999999999 2024-09-14T23:56:43.401 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.401 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.401 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:43.401 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.849950+0000", 2024-09-14T23:56:43.401 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4405000000000002e-05 2024-09-14T23:56:43.401 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.401 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:43.401 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.401 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.401 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.402 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:17093 3.4 e32/23 3:225c4d41:::rbd_data.11187be34cc8.0000000000000001:head v 32'48, mlcod=32'46)", 2024-09-14T23:56:43.402 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:42.849142+0000", 2024-09-14T23:56:43.402 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.48273969700000002, 2024-09-14T23:56:43.402 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001210076, 2024-09-14T23:56:43.402 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:43.402 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:43.402 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:43.402 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:43.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.402 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:43.402 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.849142+0000", 2024-09-14T23:56:43.402 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.403 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:43.403 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.849142+0000", 2024-09-14T23:56:43.403 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.403 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.403 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:43.403 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.849143+0000", 2024-09-14T23:56:43.403 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.06e-07 2024-09-14T23:56:43.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.403 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.403 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:43.403 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.849149+0000", 2024-09-14T23:56:43.403 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4150000000000004e-06 2024-09-14T23:56:43.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.404 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:43.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.849160+0000", 2024-09-14T23:56:43.404 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1041e-05 2024-09-14T23:56:43.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.404 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:43.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.849170+0000", 2024-09-14T23:56:43.404 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0071000000000001e-05 2024-09-14T23:56:43.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.404 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:43.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.849233+0000", 2024-09-14T23:56:43.404 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3227999999999998e-05 2024-09-14T23:56:43.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.405 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:43.405 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.849255+0000", 2024-09-14T23:56:43.405 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1895e-05 2024-09-14T23:56:43.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.405 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:43.405 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.850310+0000", 2024-09-14T23:56:43.405 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001054768 2024-09-14T23:56:43.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.405 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:43.405 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.850352+0000", 2024-09-14T23:56:43.406 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1952000000000001e-05 2024-09-14T23:56:43.406 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.406 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:43.406 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.406 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.406 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:17174 3.4 e32/23 3:225c4d41:::rbd_data.11187be34cc8.0000000000000001:head v 32'49, mlcod=32'47)", 2024-09-14T23:56:43.406 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:42.933131+0000", 2024-09-14T23:56:43.406 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.39875018200000001, 2024-09-14T23:56:43.406 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001172736, 2024-09-14T23:56:43.406 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:43.406 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:43.406 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:43.406 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:43.407 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.407 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:43.407 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.933131+0000", 2024-09-14T23:56:43.407 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.407 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.407 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:43.407 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.933131+0000", 2024-09-14T23:56:43.407 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.407 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.407 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:43.407 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.933133+0000", 2024-09-14T23:56:43.407 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2559999999999999e-06 2024-09-14T23:56:43.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.408 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:43.408 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.933142+0000", 2024-09-14T23:56:43.408 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2259999999999994e-06 2024-09-14T23:56:43.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.408 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:43.408 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.933163+0000", 2024-09-14T23:56:43.408 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1458999999999999e-05 2024-09-14T23:56:43.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.408 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:43.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.933173+0000", 2024-09-14T23:56:43.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8199999999999992e-06 2024-09-14T23:56:43.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:43.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.933242+0000", 2024-09-14T23:56:43.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8732000000000005e-05 2024-09-14T23:56:43.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:43.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.933263+0000", 2024-09-14T23:56:43.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0639e-05 2024-09-14T23:56:43.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:43.410 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.934260+0000", 2024-09-14T23:56:43.410 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00099757600000000002 2024-09-14T23:56:43.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.410 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.410 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:43.410 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.934304+0000", 2024-09-14T23:56:43.410 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4027999999999999e-05 2024-09-14T23:56:43.410 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.410 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:43.410 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.410 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.410 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:17176 3.0 e32/23 3:05bad47a:::rbd_data.11187be34cc8.0000000000000003:head v 32'33, mlcod=32'31)", 2024-09-14T23:56:43.410 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:42.933475+0000", 2024-09-14T23:56:43.411 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.398406502, 2024-09-14T23:56:43.411 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011338839999999999, 2024-09-14T23:56:43.411 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:43.411 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:43.411 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:43.411 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:43.411 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.411 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:43.411 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.933475+0000", 2024-09-14T23:56:43.411 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.411 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.411 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:43.411 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.933475+0000", 2024-09-14T23:56:43.412 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.412 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.412 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:43.412 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.933476+0000", 2024-09-14T23:56:43.412 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.58e-07 2024-09-14T23:56:43.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.412 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.412 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:43.413 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.933482+0000", 2024-09-14T23:56:43.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9029999999999998e-06 2024-09-14T23:56:43.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.413 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:43.413 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.933491+0000", 2024-09-14T23:56:43.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7669999999999992e-06 2024-09-14T23:56:43.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.413 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:43.413 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.933498+0000", 2024-09-14T23:56:43.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3149999999999997e-06 2024-09-14T23:56:43.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.414 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:43.414 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.933683+0000", 2024-09-14T23:56:43.414 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000185916 2024-09-14T23:56:43.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.414 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.414 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:43.414 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.933704+0000", 2024-09-14T23:56:43.414 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.103e-05 2024-09-14T23:56:43.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.414 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.414 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:43.414 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.934565+0000", 2024-09-14T23:56:43.414 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00086067299999999995 2024-09-14T23:56:43.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.415 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.415 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:43.415 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.934609+0000", 2024-09-14T23:56:43.415 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3822000000000002e-05 2024-09-14T23:56:43.415 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.415 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:43.415 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.415 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.415 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:17221 3.4 e33/23 3:225c4d41:::rbd_data.11187be34cc8.0000000000000001:head v 33'50, mlcod=32'48)", 2024-09-14T23:56:43.415 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:42.983242+0000", 2024-09-14T23:56:43.415 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.34864002700000002, 2024-09-14T23:56:43.415 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001313032, 2024-09-14T23:56:43.415 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:43.415 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:43.416 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:43.416 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:43.416 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.416 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:43.416 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.983242+0000", 2024-09-14T23:56:43.416 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.416 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.416 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:43.416 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.983242+0000", 2024-09-14T23:56:43.416 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.416 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.416 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:43.416 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.983243+0000", 2024-09-14T23:56:43.417 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.102e-06 2024-09-14T23:56:43.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.417 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.417 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:43.417 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.983256+0000", 2024-09-14T23:56:43.417 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2897e-05 2024-09-14T23:56:43.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.417 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.417 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:43.417 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.983277+0000", 2024-09-14T23:56:43.417 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1566999999999999e-05 2024-09-14T23:56:43.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.417 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.418 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:43.418 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.983288+0000", 2024-09-14T23:56:43.418 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0457e-05 2024-09-14T23:56:43.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.418 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:43.418 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.983355+0000", 2024-09-14T23:56:43.418 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7337999999999998e-05 2024-09-14T23:56:43.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.418 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:43.418 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.983422+0000", 2024-09-14T23:56:43.418 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6716000000000003e-05 2024-09-14T23:56:43.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.419 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.419 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:43.419 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.984515+0000", 2024-09-14T23:56:43.419 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001093761 2024-09-14T23:56:43.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.419 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.419 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:43.419 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.984555+0000", 2024-09-14T23:56:43.419 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9193999999999998e-05 2024-09-14T23:56:43.419 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.419 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:43.419 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.419 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.419 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:17225 3.0 e33/23 3:020fb8a1:::rbd_data.11187be34cc8.0000000000000005:head v 33'38, mlcod=32'35)", 2024-09-14T23:56:43.420 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:42.983959+0000", 2024-09-14T23:56:43.420 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.34792287199999999, 2024-09-14T23:56:43.420 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010373159999999999, 2024-09-14T23:56:43.420 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:43.420 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:43.420 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:43.420 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:43.420 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.420 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:43.420 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.983959+0000", 2024-09-14T23:56:43.420 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.420 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.420 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.420 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:43.421 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.983959+0000", 2024-09-14T23:56:43.421 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.421 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.421 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:43.421 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.983959+0000", 2024-09-14T23:56:43.421 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6700000000000003e-07 2024-09-14T23:56:43.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.421 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.421 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:43.421 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.983965+0000", 2024-09-14T23:56:43.421 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8150000000000002e-06 2024-09-14T23:56:43.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.421 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.422 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:43.422 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.983977+0000", 2024-09-14T23:56:43.422 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2286000000000001e-05 2024-09-14T23:56:43.422 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.422 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.422 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:43.422 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.983984+0000", 2024-09-14T23:56:43.422 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9179999999999997e-06 2024-09-14T23:56:43.422 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.422 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.422 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:43.422 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.984045+0000", 2024-09-14T23:56:43.422 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0671000000000002e-05 2024-09-14T23:56:43.422 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.422 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.422 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:43.423 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.984064+0000", 2024-09-14T23:56:43.423 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8870000000000001e-05 2024-09-14T23:56:43.423 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.423 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.423 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:43.423 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.984952+0000", 2024-09-14T23:56:43.423 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00088836399999999999 2024-09-14T23:56:43.423 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.423 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.423 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:43.423 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.984996+0000", 2024-09-14T23:56:43.423 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3724999999999999e-05 2024-09-14T23:56:43.423 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.423 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:43.423 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.423 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.424 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:43.424 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:56:43.425 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T23:56:43.427 INFO:teuthology.orchestra.run.smithi042.stdout:419 write 0xdfeaf4f thru 0xdff9ca7 (0xed59 bytes) 2024-09-14T23:56:43.427 INFO:teuthology.orchestra.run.smithi042.stdout:420 trunc from 0xdff9ca8 to 0xc23511b 2024-09-14T23:56:43.427 INFO:teuthology.orchestra.run.smithi042.stdout:421 trunc from 0xc23511b to 0x9398158 2024-09-14T23:56:43.427 INFO:teuthology.orchestra.run.smithi042.stdout:truncating to largest ever: 0xea14799 2024-09-14T23:56:43.427 INFO:teuthology.orchestra.run.smithi042.stdout:422 trunc from 0x9398158 to 0xea14799 2024-09-14T23:56:43.427 INFO:teuthology.orchestra.run.smithi042.stdout:425 writesame 0x9eb21d9 thru 0x9ec0348 data_size 0x8(0xe170 bytes) 2024-09-14T23:56:43.427 INFO:teuthology.orchestra.run.smithi042.stdout:426 read 0xae7c557 thru 0xae8be99 (0xf943 bytes) 2024-09-14T23:56:43.427 INFO:teuthology.orchestra.run.smithi042.stdout:427 read 0x3f001ae thru 0x3f09f07 (0x9d5a bytes) 2024-09-14T23:56:43.427 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:43.427 INFO:teuthology.orchestra.run.smithi042.stdout:429 trunc from 0xea14799 to 0x53bf99c 2024-09-14T23:56:43.427 INFO:teuthology.orchestra.run.smithi042.stdout:430 punch from 0x369a496 to 0x36a2963, (0x84cd bytes) 2024-09-14T23:56:43.427 INFO:teuthology.orchestra.run.smithi042.stdout:432 read 0x405690e thru 0x4066229 (0xf91c bytes) 2024-09-14T23:56:43.427 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:43.428 INFO:teuthology.orchestra.run.smithi042.stdout:436 read 0x34b24bb thru 0x34c09e7 (0xe52d bytes) 2024-09-14T23:56:43.428 INFO:teuthology.orchestra.run.smithi042.stdout:438 trunc from 0x53bf99c to 0x2f329f8 2024-09-14T23:56:43.428 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:56:43.424+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:56:43.441 INFO:teuthology.orchestra.run.smithi042.stdout:439 write 0xdf13b99 thru 0xdf19a7f (0x5ee7 bytes) 2024-09-14T23:56:43.444 INFO:teuthology.orchestra.run.smithi042.stdout:440 write 0x95d525f thru 0x95dfc42 (0xa9e4 bytes) 2024-09-14T23:56:43.447 INFO:teuthology.orchestra.run.smithi042.stdout:441 write 0x1753059 thru 0x17588b3 (0x585b bytes) 2024-09-14T23:56:43.449 INFO:teuthology.orchestra.run.smithi042.stdout:442 punch from 0x1dc6725 to 0x1dc9e02, (0x36dd bytes) 2024-09-14T23:56:43.449 INFO:teuthology.orchestra.run.smithi042.stdout:443 write 0x70fe193 thru 0x70ff2cf (0x113d bytes) 2024-09-14T23:56:43.452 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:43.457 INFO:teuthology.orchestra.run.smithi042.stdout:445 trunc from 0xdf19a80 to 0x9360320 2024-09-14T23:56:43.468 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:43.468 INFO:teuthology.orchestra.run.smithi042.stdout:447 read 0x74a6e71 thru 0x74ac27f (0x540f bytes) 2024-09-14T23:56:43.469 INFO:teuthology.orchestra.run.smithi042.stdout:448 read 0x1ebf710 thru 0x1ecb665 (0xbf56 bytes) 2024-09-14T23:56:43.469 INFO:teuthology.orchestra.run.smithi042.stdout:449 writesame 0x530278d thru 0x530de3c data_size 0x4f(0xb6b0 bytes) 2024-09-14T23:56:43.470 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:43.470 INFO:teuthology.orchestra.run.smithi042.stdout:453 write 0x627e840 thru 0x628c554 (0xdd15 bytes) 2024-09-14T23:56:43.474 INFO:teuthology.orchestra.run.smithi042.stdout:454 write 0xbfe2b7 thru 0xc08b03 (0xa84d bytes) 2024-09-14T23:56:43.476 INFO:teuthology.orchestra.run.smithi042.stdout:455 read 0x2e972a9 thru 0x2e9c34a (0x50a2 bytes) 2024-09-14T23:56:43.476 INFO:teuthology.orchestra.run.smithi042.stdout:456 read 0x6e4e3c thru 0x6f2221 (0xd3e6 bytes) 2024-09-14T23:56:43.477 INFO:teuthology.orchestra.run.smithi042.stdout:458 write 0x548d8fb thru 0x5494267 (0x696d bytes) 2024-09-14T23:56:43.488 INFO:teuthology.orchestra.run.smithi042.stdout:459 trunc from 0x9360320 to 0xc3a486 2024-09-14T23:56:43.509 INFO:teuthology.orchestra.run.smithi042.stdout:460 write 0x34eadd7 thru 0x34f0ad3 (0x5cfd bytes) 2024-09-14T23:56:43.510 INFO:teuthology.orchestra.run.smithi042.stdout:461 read 0x3109475 thru 0x310c3b3 (0x2f3f bytes) 2024-09-14T23:56:43.526 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:56:43.524+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:56:43.528 INFO:teuthology.orchestra.run.smithi042.stdout:462 write 0xdef0d05 thru 0xdefb28f (0xa58b bytes) 2024-09-14T23:56:43.532 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:43.532 INFO:teuthology.orchestra.run.smithi042.stdout:465 read 0x5878f45 thru 0x58874ae (0xe56a bytes) 2024-09-14T23:56:43.532 INFO:teuthology.orchestra.run.smithi042.stdout:467 write 0x863f35e thru 0x8647c75 (0x8918 bytes) 2024-09-14T23:56:43.535 INFO:teuthology.orchestra.run.smithi042.stdout:468 write 0x1225559 thru 0x122fa1e (0xa4c6 bytes) 2024-09-14T23:56:43.537 INFO:teuthology.orchestra.run.smithi042.stdout:469 trunc from 0xdefb290 to 0xe0c0388 2024-09-14T23:56:43.556 INFO:teuthology.orchestra.run.smithi042.stdout:471 trunc from 0xe0c0388 to 0x21ecd44 2024-09-14T23:56:43.569 INFO:teuthology.orchestra.run.smithi042.stdout:472 read 0x1b51219 thru 0x1b5ce6f (0xbc57 bytes) 2024-09-14T23:56:43.570 INFO:teuthology.orchestra.run.smithi042.stdout:473 punch from 0xc6f97 to 0xca445, (0x34ae bytes) 2024-09-14T23:56:43.570 INFO:teuthology.orchestra.run.smithi042.stdout:475 read 0x806afb thru 0x8109d8 (0x9ede bytes) 2024-09-14T23:56:43.570 INFO:teuthology.orchestra.run.smithi042.stdout:476 punch from 0x1e7677b to 0x1e7fc0c, (0x9491 bytes) 2024-09-14T23:56:43.571 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:56:43.571 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:56:43.571 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:56:43.571 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:56:43.572 INFO:teuthology.orchestra.run.smithi042.stdout:477 trunc from 0x21ecd44 to 0x427816e 2024-09-14T23:56:43.577 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:43.583 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T23:56:43.584 INFO:teuthology.orchestra.run.smithi042.stdout:479 trunc from 0x427816e to 0x8ef165e 2024-09-14T23:56:43.588 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:43.588 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:43.593 INFO:teuthology.orchestra.run.smithi042.stdout:482 trunc from 0x8ef165e to 0xda4e3e7 2024-09-14T23:56:43.599 INFO:teuthology.orchestra.run.smithi042.stdout:483 read 0xbc6e1a6 thru 0xbc73cb5 (0x5b10 bytes) 2024-09-14T23:56:43.600 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:43.600 INFO:teuthology.orchestra.run.smithi042.stdout:485 read 0x5deb3fb thru 0x5df8323 (0xcf29 bytes) 2024-09-14T23:56:43.600 INFO:teuthology.orchestra.run.smithi042.stdout:486 read 0x103ee31 thru 0x10418a1 (0x2a71 bytes) 2024-09-14T23:56:43.600 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:43.600 INFO:teuthology.orchestra.run.smithi042.stdout:493 read 0x68e99ec thru 0x68f4989 (0xaf9e bytes) 2024-09-14T23:56:43.600 INFO:teuthology.orchestra.run.smithi042.stdout:494 write 0xb8c4852 thru 0xb8c6cb9 (0x2468 bytes) 2024-09-14T23:56:43.602 INFO:teuthology.orchestra.run.smithi042.stdout:496 writesame 0xc574969 thru 0xc581fa0 data_size 0x3c(0xd638 bytes) 2024-09-14T23:56:43.606 INFO:teuthology.orchestra.run.smithi042.stdout:499 write 0xcf5476f thru 0xcf597b8 (0x504a bytes) 2024-09-14T23:56:43.623 INFO:teuthology.orchestra.run.smithi042.stdout:500 trunc from 0xda4e3e7 to 0x1fe3579 2024-09-14T23:56:43.627 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:56:43.625+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:56:43.635 INFO:teuthology.orchestra.run.smithi042.stdout:501 writesame 0x149c8e7 thru 0x14ab895 data_size 0x71(0xefaf bytes) 2024-09-14T23:56:43.642 INFO:teuthology.orchestra.run.smithi042.stdout:502 write 0x236ced1 thru 0x2377681 (0xa7b1 bytes) 2024-09-14T23:56:43.643 INFO:teuthology.orchestra.run.smithi042.stdout:503 writesame 0x1fd201 thru 0x1ff193 data_size 0x1(0x1f93 bytes) 2024-09-14T23:56:43.654 INFO:teuthology.orchestra.run.smithi042.stdout:504 trunc from 0x2377682 to 0x89971ed 2024-09-14T23:56:43.666 INFO:teuthology.orchestra.run.smithi042.stdout:505 write 0xa2d3f16 thru 0xa2de27d (0xa368 bytes) 2024-09-14T23:56:43.670 INFO:teuthology.orchestra.run.smithi042.stdout:506 write 0x1684370 thru 0x168ea5e (0xa6ef bytes) 2024-09-14T23:56:43.673 INFO:teuthology.orchestra.run.smithi042.stdout:507 trunc from 0xa2de27e to 0xb787b58 2024-09-14T23:56:43.678 INFO:teuthology.orchestra.run.smithi042.stdout:509 read 0xa59203 thru 0xa69172 (0xff70 bytes) 2024-09-14T23:56:43.679 INFO:teuthology.orchestra.run.smithi042.stdout:510 punch from 0x3874478 to 0x387fc88, (0xb810 bytes) 2024-09-14T23:56:43.679 INFO:teuthology.orchestra.run.smithi042.stdout:511 punch from 0xa73128c to 0xa7347c3, (0x3537 bytes) 2024-09-14T23:56:43.680 INFO:teuthology.orchestra.run.smithi042.stdout:512 trunc from 0xb787b58 to 0xb76a0f9 2024-09-14T23:56:43.684 INFO:teuthology.orchestra.run.smithi042.stdout:513 write 0xb6a4cf6 thru 0xb6b0a2c (0xbd37 bytes) 2024-09-14T23:56:43.687 INFO:teuthology.orchestra.run.smithi042.stdout:514 read 0x3d90edb thru 0x3d96a09 (0x5b2f bytes) 2024-09-14T23:56:43.688 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:43.689 INFO:teuthology.orchestra.run.smithi042.stdout:518 trunc from 0xb76a0f9 to 0xc6e1d89 2024-09-14T23:56:43.701 INFO:teuthology.orchestra.run.smithi042.stdout:519 write 0xd70d551 thru 0xd71a1ce (0xcc7e bytes) 2024-09-14T23:56:43.704 INFO:teuthology.orchestra.run.smithi042.stdout:521 write 0x3426655 thru 0x3426bb2 (0x55e bytes) 2024-09-14T23:56:43.705 INFO:teuthology.orchestra.run.smithi042.stdout:522 writesame 0xd42cba5 thru 0xd43aecf data_size 0x5(0xe32b bytes) 2024-09-14T23:56:43.712 INFO:teuthology.orchestra.run.smithi042.stdout:523 write 0xce4ff19 thru 0xce52674 (0x275c bytes) 2024-09-14T23:56:43.713 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:56:43.713 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:56:43.713 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:56:43.713 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:56:43.713 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:56:43.714 INFO:teuthology.orchestra.run.smithi042.stdout:524 read 0x361d4b5 thru 0x362a510 (0xd05c bytes) 2024-09-14T23:56:43.715 INFO:teuthology.orchestra.run.smithi042.stdout:525 write 0xbe0b846 thru 0xbe10f19 (0x56d4 bytes) 2024-09-14T23:56:43.717 INFO:teuthology.orchestra.run.smithi042.stdout:526 read 0x30a3e48 thru 0x30adace (0x9c87 bytes) 2024-09-14T23:56:43.717 INFO:teuthology.orchestra.run.smithi042.stdout:527 write 0x540cfc1 thru 0x5418e5b (0xbe9b bytes) 2024-09-14T23:56:43.719 INFO:teuthology.orchestra.run.smithi042.stdout:528 read 0x664e47a thru 0x6655e38 (0x79bf bytes) 2024-09-14T23:56:43.719 INFO:teuthology.orchestra.run.smithi042.stdout:529 read 0x78166fc thru 0x7823fc1 (0xd8c6 bytes) 2024-09-14T23:56:43.720 INFO:teuthology.orchestra.run.smithi042.stdout:531 write 0x8083a94 thru 0x8087d7c (0x42e9 bytes) 2024-09-14T23:56:43.720 INFO:teuthology.orchestra.run.smithi042.stdout:532 write 0x1af649f thru 0x1af682f (0x391 bytes) 2024-09-14T23:56:43.722 INFO:teuthology.orchestra.run.smithi042.stdout:533 read 0x6c75b1 thru 0x6ceecb (0x791b bytes) 2024-09-14T23:56:43.722 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:43.723 INFO:teuthology.orchestra.run.smithi042.stdout:535 read 0xa57727a thru 0xa5814ed (0xa274 bytes) 2024-09-14T23:56:43.723 INFO:teuthology.orchestra.run.smithi042.stdout:536 read 0x33e0d59 thru 0x33e7d17 (0x6fbf bytes) 2024-09-14T23:56:43.724 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T23:56:43.725 INFO:teuthology.orchestra.run.smithi042.stdout:537 punch from 0xc48558 to 0xc49e17, (0x18bf bytes) 2024-09-14T23:56:43.725 INFO:teuthology.orchestra.run.smithi042.stdout:538 write 0xbc09d5a thru 0xbc15c54 (0xbefb bytes) 2024-09-14T23:56:43.727 INFO:teuthology.orchestra.run.smithi042.stdout:539 read 0x4a4257c thru 0x4a4dd66 (0xb7eb bytes) 2024-09-14T23:56:43.727 INFO:teuthology.orchestra.run.smithi042.stdout:540 punch from 0x185033d to 0x18585f1, (0x82b4 bytes) 2024-09-14T23:56:43.727 INFO:teuthology.orchestra.run.smithi042.stdout:542 read 0x6117199 thru 0x611be77 (0x4cdf bytes) 2024-09-14T23:56:43.728 INFO:teuthology.orchestra.run.smithi042.stdout:543 writesame 0x7b733e thru 0x7bb3d6 data_size 0x1(0x4099 bytes) 2024-09-14T23:56:43.729 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:56:43.728+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:56:43.733 INFO:teuthology.orchestra.run.smithi042.stdout:545 write 0xbde94cf thru 0xbdf7378 (0xdeaa bytes) 2024-09-14T23:56:43.736 INFO:teuthology.orchestra.run.smithi042.stdout:546 read 0x4fcfb96 thru 0x4fd64ca (0x6935 bytes) 2024-09-14T23:56:43.737 INFO:teuthology.orchestra.run.smithi042.stdout:547 write 0x915ab65 thru 0x916377c (0x8c18 bytes) 2024-09-14T23:56:43.739 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:43.740 INFO:teuthology.orchestra.run.smithi042.stdout:549 writesame 0x6d580d7 thru 0x6d5b760 data_size 0x27(0x368a bytes) 2024-09-14T23:56:43.741 INFO:teuthology.orchestra.run.smithi042.stdout:551 punch from 0x181a505 to 0x18278bb, (0xd3b6 bytes) 2024-09-14T23:56:43.741 INFO:teuthology.orchestra.run.smithi042.stdout:553 write 0x19da917 thru 0x19dfe44 (0x552e bytes) 2024-09-14T23:56:43.742 INFO:teuthology.orchestra.run.smithi042.stdout:554 write 0x60490d8 thru 0x6058242 (0xf16b bytes) 2024-09-14T23:56:43.754 INFO:teuthology.orchestra.run.smithi042.stdout:555 writesame 0xec89ddf thru 0xec94996 data_size 0x8c(0xabb8 bytes) 2024-09-14T23:56:43.754 INFO:teuthology.orchestra.run.smithi042.stdout:556 punch from 0xd6524df to 0xd65f09a, (0xcbbb bytes) 2024-09-14T23:56:43.754 INFO:teuthology.orchestra.run.smithi042.stdout:557 read 0xcb8beab thru 0xcb99c42 (0xdd98 bytes) 2024-09-14T23:56:43.754 INFO:teuthology.orchestra.run.smithi042.stdout:558 punch from 0xdf63812 to 0xdf711dc, (0xd9ca bytes) 2024-09-14T23:56:43.759 INFO:teuthology.orchestra.run.smithi042.stdout:560 trunc from 0xec94997 to 0x917c06a 2024-09-14T23:56:43.771 INFO:teuthology.orchestra.run.smithi042.stdout:566 read 0x4fc21c5 thru 0x4fc2d7a (0xbb6 bytes) 2024-09-14T23:56:43.771 INFO:teuthology.orchestra.run.smithi042.stdout:567 write 0x4c0dfd1 thru 0x4c15ff4 (0x8024 bytes) 2024-09-14T23:56:43.780 INFO:teuthology.orchestra.run.smithi042.stdout:568 writesame 0xac4118c thru 0xac4e9ea data_size 0x29(0xd85f bytes) 2024-09-14T23:56:43.781 INFO:teuthology.orchestra.run.smithi042.stdout:569 write 0xa396c02 thru 0xa3998c1 (0x2cc0 bytes) 2024-09-14T23:56:43.784 INFO:teuthology.orchestra.run.smithi042.stdout:570 punch from 0x3793d2a to 0x379793d, (0x3c13 bytes) 2024-09-14T23:56:43.784 INFO:teuthology.orchestra.run.smithi042.stdout:571 writesame 0x6b7f8b4 thru 0x6b853d3 data_size 0x60(0x5b20 bytes) 2024-09-14T23:56:43.788 INFO:teuthology.orchestra.run.smithi042.stdout:572 punch from 0x8394015 to 0x8398ad6, (0x4ac1 bytes) 2024-09-14T23:56:43.788 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:43.788 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:43.789 INFO:teuthology.orchestra.run.smithi042.stdout:575 punch from 0x11466f8 to 0x1149e30, (0x3738 bytes) 2024-09-14T23:56:43.789 INFO:teuthology.orchestra.run.smithi042.stdout:576 write 0x71b6a01 thru 0x71c3c6e (0xd26e bytes) 2024-09-14T23:56:43.789 INFO:teuthology.orchestra.run.smithi042.stdout:577 punch from 0x8a8c300 to 0x8a9922c, (0xcf2c bytes) 2024-09-14T23:56:43.796 INFO:teuthology.orchestra.run.smithi042.stdout:578 write 0xba88420 thru 0xba917f1 (0x93d2 bytes) 2024-09-14T23:56:43.806 INFO:teuthology.orchestra.run.smithi042.stdout:579 trunc from 0xba917f2 to 0x3e38fda 2024-09-14T23:56:43.819 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:43.822 INFO:teuthology.orchestra.run.smithi042.stdout:581 trunc from 0x3e38fda to 0x11e38c6 2024-09-14T23:56:43.830 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:56:43.828+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:56:43.843 INFO:teuthology.orchestra.run.smithi042.stdout:582 write 0x725059d thru 0x725a584 (0x9fe8 bytes) 2024-09-14T23:56:43.845 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:43.848 INFO:teuthology.orchestra.run.smithi042.stdout:586 trunc from 0x725a585 to 0x4f8c540 2024-09-14T23:56:43.858 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:56:43.859 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:56:43.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:56:43.859 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:56:43.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.859 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5180 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3883008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:43.859 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.250058+0000", 2024-09-14T23:56:43.860 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.6056220029999997, 2024-09-14T23:56:43.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057351471000000001, 2024-09-14T23:56:43.860 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:43.861 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:43.861 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:43.861 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:43.861 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:43.861 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:43.861 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5180 2024-09-14T23:56:43.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.862 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:43.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:43.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250058+0000", 2024-09-14T23:56:43.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:43.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250058+0000", 2024-09-14T23:56:43.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.864 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:43.864 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250786+0000", 2024-09-14T23:56:43.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00072813399999999996 2024-09-14T23:56:43.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:43.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250789+0000", 2024-09-14T23:56:43.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.647e-06 2024-09-14T23:56:43.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:43.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250798+0000", 2024-09-14T23:56:43.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0000000000000002e-06 2024-09-14T23:56:43.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:43.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250803+0000", 2024-09-14T23:56:43.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.301e-06 2024-09-14T23:56:43.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.868 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:43.868 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.307234+0000", 2024-09-14T23:56:43.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056431165999999998 2024-09-14T23:56:43.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.869 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:43.869 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.307321+0000", 2024-09-14T23:56:43.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6831999999999998e-05 2024-09-14T23:56:43.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.869 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:43.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.307409+0000", 2024-09-14T23:56:43.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8391000000000001e-05 2024-09-14T23:56:43.870 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.870 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:43.870 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.871 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5184 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 4128768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:43.871 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.250851+0000", 2024-09-14T23:56:43.871 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.604828532, 2024-09-14T23:56:43.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057319038000000003, 2024-09-14T23:56:43.871 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:43.872 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:43.872 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:43.872 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:43.872 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:43.872 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:43.872 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5184 2024-09-14T23:56:43.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.873 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:43.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.873 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:43.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250851+0000", 2024-09-14T23:56:43.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:43.874 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250851+0000", 2024-09-14T23:56:43.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:43.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.251900+0000", 2024-09-14T23:56:43.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001048577 2024-09-14T23:56:43.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:43.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.251904+0000", 2024-09-14T23:56:43.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9580000000000001e-06 2024-09-14T23:56:43.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:43.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.251914+0000", 2024-09-14T23:56:43.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0186999999999999e-05 2024-09-14T23:56:43.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:43.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.251927+0000", 2024-09-14T23:56:43.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3593e-05 2024-09-14T23:56:43.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:43.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.308020+0000", 2024-09-14T23:56:43.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056092867999999997 2024-09-14T23:56:43.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:43.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.308094+0000", 2024-09-14T23:56:43.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3533000000000002e-05 2024-09-14T23:56:43.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:43.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.308170+0000", 2024-09-14T23:56:43.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6322000000000006e-05 2024-09-14T23:56:43.880 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.880 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:43.880 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.880 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5196 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 1925120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:43.880 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.252157+0000", 2024-09-14T23:56:43.880 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.6035224179999998, 2024-09-14T23:56:43.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057285801999999997, 2024-09-14T23:56:43.880 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:43.880 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:43.880 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:43.880 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:43.881 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:43.881 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:43.881 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5196 2024-09-14T23:56:43.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.881 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:43.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:43.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.252157+0000", 2024-09-14T23:56:43.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:43.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.252157+0000", 2024-09-14T23:56:43.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.882 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:43.882 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253243+0000", 2024-09-14T23:56:43.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010861340000000001 2024-09-14T23:56:43.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.882 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:43.882 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253248+0000", 2024-09-14T23:56:43.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8799999999999999e-06 2024-09-14T23:56:43.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.883 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:43.883 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253271+0000", 2024-09-14T23:56:43.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2836e-05 2024-09-14T23:56:43.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.883 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:43.883 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253281+0000", 2024-09-14T23:56:43.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6520000000000001e-06 2024-09-14T23:56:43.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.883 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:43.883 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.309336+0000", 2024-09-14T23:56:43.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056054983000000003 2024-09-14T23:56:43.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:43.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.309389+0000", 2024-09-14T23:56:43.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3298999999999999e-05 2024-09-14T23:56:43.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:43.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.309443+0000", 2024-09-14T23:56:43.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4017999999999997e-05 2024-09-14T23:56:43.885 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.885 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:43.885 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.885 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5200 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2170880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:43.885 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253394+0000", 2024-09-14T23:56:43.885 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.6022856409999999, 2024-09-14T23:56:43.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057428154000000002, 2024-09-14T23:56:43.885 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:43.885 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:43.885 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:43.885 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:43.886 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:43.886 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:43.886 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5200 2024-09-14T23:56:43.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.886 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:43.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:43.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253394+0000", 2024-09-14T23:56:43.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:43.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253394+0000", 2024-09-14T23:56:43.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:43.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253395+0000", 2024-09-14T23:56:43.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9100000000000004e-07 2024-09-14T23:56:43.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:43.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253397+0000", 2024-09-14T23:56:43.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2790000000000001e-06 2024-09-14T23:56:43.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.888 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:43.888 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253404+0000", 2024-09-14T23:56:43.888 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7959999999999998e-06 2024-09-14T23:56:43.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.888 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:43.888 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253409+0000", 2024-09-14T23:56:43.888 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1460000000000004e-06 2024-09-14T23:56:43.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.888 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:43.888 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310627+0000", 2024-09-14T23:56:43.888 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057218160999999997 2024-09-14T23:56:43.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:43.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310710+0000", 2024-09-14T23:56:43.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3102999999999994e-05 2024-09-14T23:56:43.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:43.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310822+0000", 2024-09-14T23:56:43.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000112178 2024-09-14T23:56:43.890 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.890 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:43.891 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.891 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5201 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2232320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:43.891 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:56:43.891 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.6022609689999996, 2024-09-14T23:56:43.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057643228999999997, 2024-09-14T23:56:43.892 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:43.892 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:43.892 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:43.892 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:43.892 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:43.892 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:43.893 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5201 2024-09-14T23:56:43.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.893 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:43.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:43.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:56:43.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.894 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:43.894 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:56:43.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:43.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:56:43.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0699999999999998e-07 2024-09-14T23:56:43.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.896 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:43.896 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253422+0000", 2024-09-14T23:56:43.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.514e-06 2024-09-14T23:56:43.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.897 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:43.897 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253431+0000", 2024-09-14T23:56:43.897 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1919999999999994e-06 2024-09-14T23:56:43.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.897 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:43.898 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253436+0000", 2024-09-14T23:56:43.898 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5350000000000004e-06 2024-09-14T23:56:43.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.898 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.898 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:43.898 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310856+0000", 2024-09-14T23:56:43.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057419381999999998 2024-09-14T23:56:43.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.899 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:43.899 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310953+0000", 2024-09-14T23:56:43.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6774000000000006e-05 2024-09-14T23:56:43.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:43.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311062+0000", 2024-09-14T23:56:43.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000109425 2024-09-14T23:56:43.901 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.901 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:43.901 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.901 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5202 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2293760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:43.902 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:56:43.902 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.6022362210000001, 2024-09-14T23:56:43.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057818255999999998, 2024-09-14T23:56:43.902 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:43.902 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:43.902 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:43.903 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:43.903 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:43.903 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:43.903 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5202 2024-09-14T23:56:43.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.903 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:43.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.904 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:43.904 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:56:43.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.905 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:43.905 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:56:43.905 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.906 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:43.906 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253573+0000", 2024-09-14T23:56:43.906 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012934000000000001 2024-09-14T23:56:43.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.906 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.906 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:43.907 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253576+0000", 2024-09-14T23:56:43.907 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5310000000000002e-06 2024-09-14T23:56:43.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.907 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:43.907 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253587+0000", 2024-09-14T23:56:43.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0522e-05 2024-09-14T23:56:43.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.908 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:43.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253595+0000", 2024-09-14T23:56:43.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8290000000000008e-06 2024-09-14T23:56:43.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:43.909 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311093+0000", 2024-09-14T23:56:43.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057498312000000003 2024-09-14T23:56:43.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.910 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.910 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:43.910 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311167+0000", 2024-09-14T23:56:43.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3582000000000001e-05 2024-09-14T23:56:43.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.910 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.911 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:43.911 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311262+0000", 2024-09-14T23:56:43.911 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5140000000000001e-05 2024-09-14T23:56:43.911 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.911 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:43.911 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.912 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.912 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5203 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2355200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:43.912 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:56:43.912 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.6020734789999995, 2024-09-14T23:56:43.912 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057903938000000002, 2024-09-14T23:56:43.913 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:43.913 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:43.913 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:43.913 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:43.913 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:43.913 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:43.914 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5203 2024-09-14T23:56:43.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.914 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:43.914 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.914 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:43.915 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:56:43.915 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.915 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.915 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:43.915 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:56:43.916 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.916 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.916 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.916 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:43.916 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253902+0000", 2024-09-14T23:56:43.917 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029615400000000002 2024-09-14T23:56:43.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.917 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.917 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:43.917 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253906+0000", 2024-09-14T23:56:43.917 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9249999999999997e-06 2024-09-14T23:56:43.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.918 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.918 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:43.918 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253913+0000", 2024-09-14T23:56:43.918 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9099999999999999e-06 2024-09-14T23:56:43.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.919 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.919 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:43.919 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253917+0000", 2024-09-14T23:56:43.919 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9210000000000002e-06 2024-09-14T23:56:43.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.919 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.920 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:43.920 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311295+0000", 2024-09-14T23:56:43.920 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057378268000000003 2024-09-14T23:56:43.920 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.920 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.921 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:43.921 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311402+0000", 2024-09-14T23:56:43.921 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010644100000000001 2024-09-14T23:56:43.921 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.921 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.921 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:43.922 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311510+0000", 2024-09-14T23:56:43.922 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108319 2024-09-14T23:56:43.922 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.922 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:43.922 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.923 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.923 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.923 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5204 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2416640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:43.923 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:56:43.923 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.6017552820000001, 2024-09-14T23:56:43.923 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057774860999999997, 2024-09-14T23:56:43.924 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:43.924 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:43.924 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:43.924 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:43.924 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:43.924 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:43.925 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5204 2024-09-14T23:56:43.925 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.925 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:43.925 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.925 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:43.925 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:56:43.926 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.926 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.926 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.926 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:43.926 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:56:43.927 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.927 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.927 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.927 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:43.927 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255903+0000", 2024-09-14T23:56:43.927 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001978807 2024-09-14T23:56:43.928 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.928 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.928 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:43.928 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255907+0000", 2024-09-14T23:56:43.928 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3610000000000001e-06 2024-09-14T23:56:43.928 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.929 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.929 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:43.929 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255914+0000", 2024-09-14T23:56:43.929 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.357e-06 2024-09-14T23:56:43.929 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.929 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.930 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:43.930 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255918+0000", 2024-09-14T23:56:43.930 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1779999999999996e-06 2024-09-14T23:56:43.930 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.930 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.930 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:43.931 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311541+0000", 2024-09-14T23:56:43.931 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055622697999999998 2024-09-14T23:56:43.931 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.931 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.931 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:43.932 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311614+0000", 2024-09-14T23:56:43.932 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3102999999999994e-05 2024-09-14T23:56:43.932 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.932 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.932 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:43.932 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311699+0000", 2024-09-14T23:56:43.933 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5357000000000003e-05 2024-09-14T23:56:43.933 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.933 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:43.933 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.933 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.934 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5210 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2785280~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:43.934 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256011+0000", 2024-09-14T23:56:43.934 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.5996686449999995, 2024-09-14T23:56:43.934 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057273662000000003, 2024-09-14T23:56:43.934 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:43.935 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:43.935 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:43.935 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:43.935 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:43.935 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:43.935 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5210 2024-09-14T23:56:43.936 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.936 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:43.936 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.936 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:43.936 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256011+0000", 2024-09-14T23:56:43.936 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.937 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.937 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.937 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:43.937 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256011+0000", 2024-09-14T23:56:43.937 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.938 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.938 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.938 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:43.938 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256011+0000", 2024-09-14T23:56:43.938 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4999999999999999e-07 2024-09-14T23:56:43.938 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.939 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.939 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:43.939 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256013+0000", 2024-09-14T23:56:43.939 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3969999999999999e-06 2024-09-14T23:56:43.939 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.939 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.940 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:43.940 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256017+0000", 2024-09-14T23:56:43.940 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1409999999999998e-06 2024-09-14T23:56:43.940 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.940 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.941 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:43.941 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256019+0000", 2024-09-14T23:56:43.941 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6280000000000001e-06 2024-09-14T23:56:43.941 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.941 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.941 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:43.942 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313116+0000", 2024-09-14T23:56:43.942 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057096457000000003 2024-09-14T23:56:43.942 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.942 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.942 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:43.942 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313193+0000", 2024-09-14T23:56:43.943 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7439000000000002e-05 2024-09-14T23:56:43.943 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.943 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.943 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:43.943 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313285+0000", 2024-09-14T23:56:43.944 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1349999999999998e-05 2024-09-14T23:56:43.944 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.944 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:43.944 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.944 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.944 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.945 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5211 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2846720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:43.945 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:56:43.945 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.599657068, 2024-09-14T23:56:43.945 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057528151, 2024-09-14T23:56:43.945 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:43.946 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:43.946 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:43.946 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:43.946 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:43.946 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:43.946 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5211 2024-09-14T23:56:43.947 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.947 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:43.947 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.947 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:43.947 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:56:43.947 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.948 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.948 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.948 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:43.948 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:56:43.948 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.948 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.949 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.949 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:43.949 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:56:43.949 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1399999999999998e-07 2024-09-14T23:56:43.949 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.950 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.950 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:43.950 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256024+0000", 2024-09-14T23:56:43.950 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.114e-06 2024-09-14T23:56:43.950 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.950 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.951 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:43.951 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256028+0000", 2024-09-14T23:56:43.951 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0840000000000002e-06 2024-09-14T23:56:43.951 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.951 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.951 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:43.952 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256031+0000", 2024-09-14T23:56:43.952 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6819999999999999e-06 2024-09-14T23:56:43.952 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.952 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.952 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:43.953 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313317+0000", 2024-09-14T23:56:43.953 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057286295000000001 2024-09-14T23:56:43.953 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.953 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.953 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:43.954 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313403+0000", 2024-09-14T23:56:43.954 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6287999999999998e-05 2024-09-14T23:56:43.954 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.954 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.954 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:43.954 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313551+0000", 2024-09-14T23:56:43.955 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000147374 2024-09-14T23:56:43.955 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.955 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:43.955 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.955 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.955 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.956 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5212 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2908160~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:43.956 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:56:43.956 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.5996458740000001, 2024-09-14T23:56:43.956 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057735679999999998, 2024-09-14T23:56:43.956 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:43.957 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:43.957 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:43.957 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:43.957 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:43.957 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:43.957 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5212 2024-09-14T23:56:43.958 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.958 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:43.958 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.958 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:43.958 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:56:43.958 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.959 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.959 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.959 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:43.959 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:56:43.959 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.959 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.959 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.959 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:43.959 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:56:43.960 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0900000000000001e-07 2024-09-14T23:56:43.960 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.960 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.960 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:43.960 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256035+0000", 2024-09-14T23:56:43.960 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1200000000000001e-06 2024-09-14T23:56:43.960 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.960 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.960 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:43.960 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256038+0000", 2024-09-14T23:56:43.960 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.05e-06 2024-09-14T23:56:43.960 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.960 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.961 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:43.961 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256041+0000", 2024-09-14T23:56:43.961 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.667e-06 2024-09-14T23:56:43.961 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.961 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.961 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:43.961 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313584+0000", 2024-09-14T23:56:43.961 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057543484999999998 2024-09-14T23:56:43.961 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.961 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.961 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:43.961 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313668+0000", 2024-09-14T23:56:43.961 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3807000000000006e-05 2024-09-14T23:56:43.961 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.962 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.962 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:43.962 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313770+0000", 2024-09-14T23:56:43.962 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010134200000000001 2024-09-14T23:56:43.962 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.962 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:43.962 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.962 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.962 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.962 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5213 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2969600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:43.963 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:56:43.964 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.5996357870000004, 2024-09-14T23:56:43.964 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058145467999999999, 2024-09-14T23:56:43.964 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:43.964 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:43.964 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:43.964 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:43.964 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:43.964 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:43.964 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5213 2024-09-14T23:56:43.964 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.964 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:43.964 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.964 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:43.964 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:56:43.965 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.965 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:43.965 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:56:43.965 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.965 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:43.965 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257901+0000", 2024-09-14T23:56:43.965 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018567519999999999 2024-09-14T23:56:43.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.965 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:43.966 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257904+0000", 2024-09-14T23:56:43.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4850000000000001e-06 2024-09-14T23:56:43.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.966 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:43.966 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257912+0000", 2024-09-14T23:56:43.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2090000000000005e-06 2024-09-14T23:56:43.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.966 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:43.966 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257918+0000", 2024-09-14T23:56:43.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6899999999999997e-06 2024-09-14T23:56:43.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.967 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:43.967 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313816+0000", 2024-09-14T23:56:43.967 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055898075999999998 2024-09-14T23:56:43.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.967 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:43.967 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.314082+0000", 2024-09-14T23:56:43.967 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000265386 2024-09-14T23:56:43.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.967 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:43.967 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.314189+0000", 2024-09-14T23:56:43.967 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010787 2024-09-14T23:56:43.967 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.968 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:43.968 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.968 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.968 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.968 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5217 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3215360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:43.968 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257961+0000", 2024-09-14T23:56:43.968 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.5977188079999998, 2024-09-14T23:56:43.968 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057320969999999999, 2024-09-14T23:56:43.968 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:43.968 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:43.968 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:43.968 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:43.968 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:43.968 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:43.969 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5217 2024-09-14T23:56:43.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.969 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:43.969 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.969 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:43.969 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257961+0000", 2024-09-14T23:56:43.969 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.969 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.969 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:43.969 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257961+0000", 2024-09-14T23:56:43.969 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.969 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.970 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:43.970 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257961+0000", 2024-09-14T23:56:43.970 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.22e-07 2024-09-14T23:56:43.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.970 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.970 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:43.970 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257962+0000", 2024-09-14T23:56:43.970 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1400000000000001e-06 2024-09-14T23:56:43.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.970 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.970 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:43.970 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257965+0000", 2024-09-14T23:56:43.970 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0929999999999999e-06 2024-09-14T23:56:43.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.971 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:43.971 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257968+0000", 2024-09-14T23:56:43.971 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5749999999999999e-06 2024-09-14T23:56:43.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.971 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:43.971 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315047+0000", 2024-09-14T23:56:43.971 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057078730000000001 2024-09-14T23:56:43.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.971 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:43.971 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315087+0000", 2024-09-14T23:56:43.971 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0806999999999997e-05 2024-09-14T23:56:43.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.972 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.972 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:43.972 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315282+0000", 2024-09-14T23:56:43.972 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000194403 2024-09-14T23:56:43.972 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.972 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:43.972 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.972 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.972 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5218 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3276800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:43.972 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:56:43.972 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.5977087780000003, 2024-09-14T23:56:43.973 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057497557999999997, 2024-09-14T23:56:43.973 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:43.973 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:43.973 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:43.973 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:43.973 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:43.973 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:43.973 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5218 2024-09-14T23:56:43.973 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.973 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:43.973 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.973 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:43.973 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:56:43.973 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.974 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.974 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:43.974 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:56:43.974 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.974 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.974 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:43.974 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:56:43.974 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1899999999999999e-07 2024-09-14T23:56:43.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.974 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.974 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:43.974 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257972+0000", 2024-09-14T23:56:43.974 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.217e-06 2024-09-14T23:56:43.975 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.975 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.975 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:43.975 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257975+0000", 2024-09-14T23:56:43.975 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9359999999999999e-06 2024-09-14T23:56:43.975 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.975 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.975 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:43.975 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257978+0000", 2024-09-14T23:56:43.975 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1049999999999999e-06 2024-09-14T23:56:43.975 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.975 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.975 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:43.976 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315331+0000", 2024-09-14T23:56:43.976 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057352093999999999 2024-09-14T23:56:43.976 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.976 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.976 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:43.976 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315384+0000", 2024-09-14T23:56:43.976 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3780999999999999e-05 2024-09-14T23:56:43.976 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.976 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.976 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:43.976 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315468+0000", 2024-09-14T23:56:43.976 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4206e-05 2024-09-14T23:56:43.976 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.976 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:43.977 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.977 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.977 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.977 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5219 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3338240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:43.977 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:56:43.977 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.5976981019999998, 2024-09-14T23:56:43.977 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057637764000000001, 2024-09-14T23:56:43.977 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:43.977 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:43.977 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:43.977 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:43.977 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:43.977 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:43.977 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5219 2024-09-14T23:56:43.978 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.978 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:43.978 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.978 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:43.978 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:56:43.978 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.978 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.978 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.978 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:43.978 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:56:43.978 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.978 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.978 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.979 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:43.979 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:56:43.979 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.03e-07 2024-09-14T23:56:43.979 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.979 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.979 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:43.979 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257983+0000", 2024-09-14T23:56:43.979 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9199999999999999e-07 2024-09-14T23:56:43.979 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.979 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.979 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:43.979 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257986+0000", 2024-09-14T23:56:43.979 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0860000000000002e-06 2024-09-14T23:56:43.979 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.980 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.980 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:43.980 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257989+0000", 2024-09-14T23:56:43.980 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6589999999999999e-06 2024-09-14T23:56:43.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.980 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.980 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:43.980 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315509+0000", 2024-09-14T23:56:43.980 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057520262000000003 2024-09-14T23:56:43.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.980 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.980 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:43.980 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315554+0000", 2024-09-14T23:56:43.980 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5015999999999997e-05 2024-09-14T23:56:43.981 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.981 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.981 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:43.981 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315619+0000", 2024-09-14T23:56:43.981 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5545999999999999e-05 2024-09-14T23:56:43.981 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.981 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:43.981 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.981 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.981 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.981 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5220 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3399680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:43.981 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:56:43.981 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.5976881460000003, 2024-09-14T23:56:43.981 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057792128999999998, 2024-09-14T23:56:43.982 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:43.982 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:43.982 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:43.982 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:43.982 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:43.982 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:43.982 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5220 2024-09-14T23:56:43.982 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.982 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:43.982 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.982 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:43.982 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:56:43.982 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.982 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.982 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.983 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:43.983 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:56:43.983 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.983 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.983 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:43.983 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:56:43.983 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1799999999999999e-07 2024-09-14T23:56:43.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.983 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.983 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:43.983 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257993+0000", 2024-09-14T23:56:43.983 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6200000000000006e-07 2024-09-14T23:56:43.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.984 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.984 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:43.984 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257996+0000", 2024-09-14T23:56:43.984 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9050000000000001e-06 2024-09-14T23:56:43.984 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.984 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.984 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:43.984 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257998+0000", 2024-09-14T23:56:43.984 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5950000000000001e-06 2024-09-14T23:56:43.984 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.984 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.984 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:43.984 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315672+0000", 2024-09-14T23:56:43.985 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057673637999999999 2024-09-14T23:56:43.985 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.985 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.985 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:43.985 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315734+0000", 2024-09-14T23:56:43.985 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2216000000000002e-05 2024-09-14T23:56:43.985 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.985 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.985 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:43.985 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315784+0000", 2024-09-14T23:56:43.985 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9595000000000002e-05 2024-09-14T23:56:43.985 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.985 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:43.985 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.986 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.986 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5221 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3461120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:43.986 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:56:43.986 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.597678428, 2024-09-14T23:56:43.986 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058003557999999997, 2024-09-14T23:56:43.986 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:43.986 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:43.986 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:43.986 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:43.986 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:43.986 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:43.986 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5221 2024-09-14T23:56:43.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.986 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:43.987 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.987 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:43.987 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:56:43.987 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.987 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.987 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:43.987 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:56:43.987 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.987 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.987 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:43.987 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259905+0000", 2024-09-14T23:56:43.987 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019035759999999999 2024-09-14T23:56:43.988 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.988 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.988 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:43.988 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259915+0000", 2024-09-14T23:56:43.988 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0518e-05 2024-09-14T23:56:43.988 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.988 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.988 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:43.988 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259925+0000", 2024-09-14T23:56:43.988 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7180000000000008e-06 2024-09-14T23:56:43.988 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.988 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.988 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:43.988 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259930+0000", 2024-09-14T23:56:43.989 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4769999999999997e-06 2024-09-14T23:56:43.989 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.989 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.989 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:43.989 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315820+0000", 2024-09-14T23:56:43.989 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055889960000000002 2024-09-14T23:56:43.989 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.989 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.989 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:43.989 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315914+0000", 2024-09-14T23:56:43.989 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4625999999999998e-05 2024-09-14T23:56:43.989 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.989 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.989 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:43.990 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.316005+0000", 2024-09-14T23:56:43.990 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0682999999999999e-05 2024-09-14T23:56:43.990 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.990 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:43.990 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.990 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.990 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.990 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5230 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 4014080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:43.990 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.260060+0000", 2024-09-14T23:56:43.990 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.5956195900000001, 2024-09-14T23:56:43.990 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057353556999999999, 2024-09-14T23:56:43.990 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:43.990 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:43.990 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:43.991 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:43.991 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:43.991 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:43.991 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5230 2024-09-14T23:56:43.991 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.991 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:43.991 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.991 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:43.991 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260060+0000", 2024-09-14T23:56:43.991 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.991 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.991 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.991 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:43.991 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260060+0000", 2024-09-14T23:56:43.992 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.992 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.992 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.992 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:43.992 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260060+0000", 2024-09-14T23:56:43.992 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.17e-07 2024-09-14T23:56:43.992 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.992 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.992 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:43.992 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260061+0000", 2024-09-14T23:56:43.992 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0300000000000001e-06 2024-09-14T23:56:43.992 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.992 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.992 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:43.993 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260064+0000", 2024-09-14T23:56:43.993 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9119999999999998e-06 2024-09-14T23:56:43.993 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.993 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.993 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:43.993 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260067+0000", 2024-09-14T23:56:43.993 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.678e-06 2024-09-14T23:56:43.993 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.993 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.993 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:43.993 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317269+0000", 2024-09-14T23:56:43.993 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057201771999999998 2024-09-14T23:56:43.993 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.993 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.994 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:43.994 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317342+0000", 2024-09-14T23:56:43.994 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2842000000000005e-05 2024-09-14T23:56:43.994 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.994 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.994 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:43.994 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317414+0000", 2024-09-14T23:56:43.994 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2106000000000004e-05 2024-09-14T23:56:43.994 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.994 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:43.994 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.994 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.994 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.994 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5231 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 4075520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:43.995 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:56:43.995 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.5956096940000002, 2024-09-14T23:56:43.995 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057524733000000002, 2024-09-14T23:56:43.995 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:43.995 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:43.995 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:43.995 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:43.995 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:43.995 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:43.995 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5231 2024-09-14T23:56:43.995 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.995 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:43.995 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.995 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:43.996 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:56:43.996 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.996 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.996 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.996 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:43.996 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:56:43.996 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:43.996 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.996 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.996 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:43.996 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261904+0000", 2024-09-14T23:56:43.996 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018338790000000001 2024-09-14T23:56:43.996 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.996 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.996 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:43.997 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261910+0000", 2024-09-14T23:56:43.997 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9689999999999999e-06 2024-09-14T23:56:43.997 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.997 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.997 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:43.997 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261923+0000", 2024-09-14T23:56:43.997 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3061000000000001e-05 2024-09-14T23:56:43.997 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.997 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.997 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:43.997 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261931+0000", 2024-09-14T23:56:43.997 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3159999999999997e-06 2024-09-14T23:56:43.997 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.997 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.998 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:43.998 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317456+0000", 2024-09-14T23:56:43.998 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055525219000000001 2024-09-14T23:56:43.998 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.998 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.998 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:43.998 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317520+0000", 2024-09-14T23:56:43.998 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4010000000000005e-05 2024-09-14T23:56:43.998 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.998 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.998 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:43.998 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317595+0000", 2024-09-14T23:56:43.998 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4278999999999998e-05 2024-09-14T23:56:43.998 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.999 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:43.999 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:43.999 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:43.999 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:43.999 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5236 3.8 3:1071668d:::rbd_data.1118de1168b.0000000000000092:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:43.999 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:56:43.999 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.5935962300000002, 2024-09-14T23:56:43.999 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057499462000000001, 2024-09-14T23:56:43.999 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:43.999 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:43.999 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:43.999 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:43.999 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:43.999 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:44.000 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5236 2024-09-14T23:56:44.000 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:44.000 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:44.000 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:44.000 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:44.000 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:56:44.000 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:44.000 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:44.000 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:44.000 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:44.000 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:56:44.000 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:44.000 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:44.000 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:44.000 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:44.001 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262084+0000", 2024-09-14T23:56:44.001 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3399999999999999e-07 2024-09-14T23:56:44.001 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:44.001 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:44.001 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:44.001 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262087+0000", 2024-09-14T23:56:44.001 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2310000000000001e-06 2024-09-14T23:56:44.001 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:44.001 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:44.001 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:44.001 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262095+0000", 2024-09-14T23:56:44.001 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3799999999999996e-06 2024-09-14T23:56:44.001 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:44.001 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:44.002 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:44.002 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262101+0000", 2024-09-14T23:56:44.002 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0009999999999998e-06 2024-09-14T23:56:44.002 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:44.002 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:44.002 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:44.002 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317625+0000", 2024-09-14T23:56:44.002 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055524111000000001 2024-09-14T23:56:44.002 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:44.002 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:44.002 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:44.002 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317702+0000", 2024-09-14T23:56:44.002 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7533999999999993e-05 2024-09-14T23:56:44.003 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:44.003 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:44.003 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-14T23:56:44.003 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317892+0000", 2024-09-14T23:56:44.003 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018991400000000001 2024-09-14T23:56:44.003 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:44.003 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:44.003 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:56:44.003 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319407+0000", 2024-09-14T23:56:44.003 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015145250000000001 2024-09-14T23:56:44.003 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:44.003 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:44.003 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:44.003 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319481+0000", 2024-09-14T23:56:44.004 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3826999999999996e-05 2024-09-14T23:56:44.004 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:44.004 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:44.004 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:44.004 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319541+0000", 2024-09-14T23:56:44.004 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0470999999999998e-05 2024-09-14T23:56:44.004 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:44.004 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:44.004 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:44.004 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319583+0000", 2024-09-14T23:56:44.004 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1934000000000002e-05 2024-09-14T23:56:44.004 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:44.004 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:44.004 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:44.005 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:44.005 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:44.005 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:56:44.006 INFO:teuthology.orchestra.run.smithi042.stdout:587 trunc from 0x4f8c540 to 0x7e0f7ba 2024-09-14T23:56:44.007 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T23:56:44.010 INFO:teuthology.orchestra.run.smithi042.stdout:588 punch from 0x3a5b403 to 0x3a67fb3, (0xcbb0 bytes) 2024-09-14T23:56:44.010 INFO:teuthology.orchestra.run.smithi042.stdout:589 read 0x28d7320 thru 0x28e2a33 (0xb714 bytes) 2024-09-14T23:56:44.010 INFO:teuthology.orchestra.run.smithi042.stdout:590 punch from 0x179e51d to 0x17a1e0c, (0x38ef bytes) 2024-09-14T23:56:44.010 INFO:teuthology.orchestra.run.smithi042.stdout:591 writesame 0x5075994 thru 0x507da80 data_size 0x29(0x80ed bytes) 2024-09-14T23:56:44.010 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:44.010 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:44.010 INFO:teuthology.orchestra.run.smithi042.stdout:596 trunc from 0x7e0f7ba to 0xb5880c7 2024-09-14T23:56:44.010 INFO:teuthology.orchestra.run.smithi042.stdout:598 punch from 0xa2bbe72 to 0xa2c1720, (0x58ae bytes) 2024-09-14T23:56:44.010 INFO:teuthology.orchestra.run.smithi042.stdout:599 write 0xe7b58d8 thru 0xe7b60d7 (0x800 bytes) 2024-09-14T23:56:44.010 INFO:teuthology.orchestra.run.smithi042.stdout:600 read 0x28f4b44 thru 0x2903aa4 (0xef61 bytes) 2024-09-14T23:56:44.010 INFO:teuthology.orchestra.run.smithi042.stdout:602 writesame 0xcccc102 thru 0xcccca86 data_size 0x1(0x985 bytes) 2024-09-14T23:56:44.010 INFO:teuthology.orchestra.run.smithi042.stdout:603 writesame 0x9539a82 thru 0x95490cf data_size 0x12(0xf64e bytes) 2024-09-14T23:56:44.011 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:44.011 INFO:teuthology.orchestra.run.smithi042.stdout:606 punch from 0x649ead1 to 0x64a35e3, (0x4b12 bytes) 2024-09-14T23:56:44.011 INFO:teuthology.orchestra.run.smithi042.stdout:607 read 0xbdce57c thru 0xbdcf9f6 (0x147b bytes) 2024-09-14T23:56:44.011 INFO:teuthology.orchestra.run.smithi042.stdout:610 write 0x722c70c thru 0x72305c7 (0x3ebc bytes) 2024-09-14T23:56:44.011 INFO:teuthology.orchestra.run.smithi042.stdout:613 read 0x3ff6f54 thru 0x4004001 (0xd0ae bytes) 2024-09-14T23:56:44.011 INFO:teuthology.orchestra.run.smithi042.stdout:614 read 0xc01468c thru 0xc01f7e6 (0xb15b bytes) 2024-09-14T23:56:44.011 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:44.011 INFO:teuthology.orchestra.run.smithi042.stdout:618 trunc from 0xe7b60d8 to 0xda969f1 2024-09-14T23:56:44.011 INFO:teuthology.orchestra.run.smithi042.stdout:619 write 0x47e893e thru 0x47f7222 (0xe8e5 bytes) 2024-09-14T23:56:44.011 INFO:teuthology.orchestra.run.smithi042.stdout:620 trunc from 0xda969f1 to 0xa7d7cc6 2024-09-14T23:56:44.011 INFO:teuthology.orchestra.run.smithi042.stdout:621 writesame 0x772ea03 thru 0x772f3c0 data_size 0x1d(0x9be bytes) 2024-09-14T23:56:44.011 INFO:teuthology.orchestra.run.smithi042.stdout:622 read 0x432353b thru 0x4331a13 (0xe4d9 bytes) 2024-09-14T23:56:44.011 INFO:teuthology.orchestra.run.smithi042.stdout:623 write 0xa668fca thru 0xa67657b (0xd5b2 bytes) 2024-09-14T23:56:44.011 INFO:teuthology.orchestra.run.smithi042.stdout:624 writesame 0x1be59aa thru 0x1be962e data_size 0x1(0x3c85 bytes) 2024-09-14T23:56:44.012 INFO:teuthology.orchestra.run.smithi042.stdout:626 trunc from 0xa7d7cc6 to 0xcb7e5a5 2024-09-14T23:56:44.012 INFO:teuthology.orchestra.run.smithi042.stdout:628 trunc from 0xcb7e5a5 to 0xd54d895 2024-09-14T23:56:44.012 INFO:teuthology.orchestra.run.smithi042.stdout:629 write 0xb6ac855 thru 0xb6b0fe0 (0x478c bytes) 2024-09-14T23:56:44.012 INFO:teuthology.orchestra.run.smithi042.stdout:630 trunc from 0xd54d895 to 0x3814a65 2024-09-14T23:56:44.012 INFO:teuthology.orchestra.run.smithi042.stdout:631 read 0x2ed8308 thru 0x2ede3a7 (0x60a0 bytes) 2024-09-14T23:56:44.012 INFO:teuthology.orchestra.run.smithi042.stdout:633 punch from 0x35b84d8 to 0x35ba454, (0x1f7c bytes) 2024-09-14T23:56:44.012 INFO:teuthology.orchestra.run.smithi042.stdout:634 write 0xb59894a thru 0xb5a260d (0x9cc4 bytes) 2024-09-14T23:56:44.012 INFO:teuthology.orchestra.run.smithi042.stdout:635 punch from 0x2688db9 to 0x2697976, (0xebbd bytes) 2024-09-14T23:56:44.012 INFO:teuthology.orchestra.run.smithi042.stdout:637 writesame 0x5bc9642 thru 0x5bc9c3e data_size 0x7(0x5fd bytes) 2024-09-14T23:56:44.012 INFO:teuthology.orchestra.run.smithi042.stderr:set osd.4 primary-affinity to 0.970352 (8635922) 2024-09-14T23:56:44.013 INFO:teuthology.orchestra.run.smithi042.stdout:638 punch from 0x9cb3a49 to 0x9cbe617, (0xabce bytes) 2024-09-14T23:56:44.013 INFO:teuthology.orchestra.run.smithi042.stdout:639 punch from 0x5eb7f3f to 0x5ec600f, (0xe0d0 bytes) 2024-09-14T23:56:44.013 INFO:teuthology.orchestra.run.smithi042.stdout:640 trunc from 0xb5a260e to 0x5bc3475 2024-09-14T23:56:44.013 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:56:44.004+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:56:44.019 INFO:teuthology.orchestra.run.smithi042.stdout:641 read 0x271c0ac thru 0x272a936 (0xe88b bytes) 2024-09-14T23:56:44.019 INFO:teuthology.orchestra.run.smithi042.stdout:643 read 0x1e99060 thru 0x1ea74ad (0xe44e bytes) 2024-09-14T23:56:44.029 INFO:teuthology.orchestra.run.smithi042.stdout:644 write 0x9070dcc thru 0x9070e13 (0x48 bytes) 2024-09-14T23:56:44.031 INFO:teuthology.orchestra.run.smithi042.stdout:645 writesame 0x77c70de thru 0x77d5aee data_size 0x1(0xea11 bytes) 2024-09-14T23:56:44.062 INFO:teuthology.orchestra.run.smithi042.stdout:646 read 0x578260d thru 0x578ecdb (0xc6cf bytes) 2024-09-14T23:56:44.062 INFO:teuthology.orchestra.run.smithi042.stdout:649 write 0x34d1363 thru 0x34e0075 (0xed13 bytes) 2024-09-14T23:56:44.064 INFO:teuthology.orchestra.run.smithi042.stdout:650 read 0x4b659e7 thru 0x4b6e2e1 (0x88fb bytes) 2024-09-14T23:56:44.064 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:44.064 INFO:teuthology.orchestra.run.smithi042.stdout:653 punch from 0x6f6e9c8 to 0x6f6f8a0, (0xed8 bytes) 2024-09-14T23:56:44.064 INFO:teuthology.orchestra.run.smithi042.stdout:654 read 0x6b9fbe8 thru 0x6ba8000 (0x8419 bytes) 2024-09-14T23:56:44.065 INFO:teuthology.orchestra.run.smithi042.stdout:655 punch from 0xfa12e3 to 0xfaa8a5, (0x95c2 bytes) 2024-09-14T23:56:44.065 INFO:teuthology.orchestra.run.smithi042.stdout:656 write 0x7e6a9d5 thru 0x7e6d5a4 (0x2bd0 bytes) 2024-09-14T23:56:44.076 INFO:teuthology.orchestra.run.smithi042.stdout:657 write 0xd033ffc thru 0xd043fb0 (0xffb5 bytes) 2024-09-14T23:56:44.080 INFO:teuthology.orchestra.run.smithi042.stdout:658 write 0x459759 thru 0x4610c5 (0x796d bytes) 2024-09-14T23:56:44.081 INFO:teuthology.orchestra.run.smithi042.stdout:660 read 0x3cad9e1 thru 0x3cb7e9c (0xa4bc bytes) 2024-09-14T23:56:44.082 INFO:teuthology.orchestra.run.smithi042.stdout:661 writesame 0x4a597d7 thru 0x4a600ca data_size 0x6(0x68f4 bytes) 2024-09-14T23:56:44.084 INFO:teuthology.orchestra.run.smithi042.stdout:662 read 0x292017e thru 0x292fdc2 (0xfc45 bytes) 2024-09-14T23:56:44.085 INFO:teuthology.orchestra.run.smithi042.stdout:663 punch from 0x63d5514 to 0x63d5b44, (0x630 bytes) 2024-09-14T23:56:44.085 INFO:teuthology.orchestra.run.smithi042.stdout:664 punch from 0x9a09eac to 0x9a11811, (0x7965 bytes) 2024-09-14T23:56:44.091 INFO:teuthology.orchestra.run.smithi042.stdout:665 write 0xd6ddc25 thru 0xd6ecdac (0xf188 bytes) 2024-09-14T23:56:44.093 INFO:teuthology.orchestra.run.smithi042.stdout:666 write 0xc1153b0 thru 0xc119f21 (0x4b72 bytes) 2024-09-14T23:56:44.106 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:56:44.105+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:56:44.108 INFO:teuthology.orchestra.run.smithi042.stdout:667 trunc from 0xd6ecdad to 0x26e402b 2024-09-14T23:56:44.125 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:56:44.125 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:56:44.125 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:56:44.125 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:56:44.131 INFO:teuthology.orchestra.run.smithi042.stdout:670 trunc from 0x26e402b to 0xc002b6f 2024-09-14T23:56:44.134 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T23:56:44.136 INFO:teuthology.orchestra.run.smithi042.stdout:671 writesame 0xfc46df thru 0xfcc8e5 data_size 0x1(0x8207 bytes) 2024-09-14T23:56:44.152 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:44.152 INFO:teuthology.orchestra.run.smithi042.stdout:674 read 0xbcdfed3 thru 0xbce3332 (0x3460 bytes) 2024-09-14T23:56:44.152 INFO:teuthology.orchestra.run.smithi042.stdout:675 write 0x7243f6a thru 0x724bbff (0x7c96 bytes) 2024-09-14T23:56:44.155 INFO:teuthology.orchestra.run.smithi042.stdout:676 read 0x3d2b4ca thru 0x3d34b59 (0x9690 bytes) 2024-09-14T23:56:44.155 INFO:teuthology.orchestra.run.smithi042.stdout:677 writesame 0x228ed62 thru 0x22946d1 data_size 0x36(0x5970 bytes) 2024-09-14T23:56:44.157 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:44.157 INFO:teuthology.orchestra.run.smithi042.stdout:679 write 0x6ffd698 thru 0x7001223 (0x3b8c bytes) 2024-09-14T23:56:44.160 INFO:teuthology.orchestra.run.smithi042.stdout:681 read 0x8a347c thru 0x8a3feb (0xb70 bytes) 2024-09-14T23:56:44.161 INFO:teuthology.orchestra.run.smithi042.stdout:682 read 0x56f45a thru 0x576b3b (0x76e2 bytes) 2024-09-14T23:56:44.161 INFO:teuthology.orchestra.run.smithi042.stdout:684 read 0x41d30c thru 0x42d064 (0xfd59 bytes) 2024-09-14T23:56:44.162 INFO:teuthology.orchestra.run.smithi042.stdout:685 punch from 0x873ce16 to 0x874ad10, (0xdefa bytes) 2024-09-14T23:56:44.162 INFO:teuthology.orchestra.run.smithi042.stdout:686 read 0x1a3e68b thru 0x1a49be8 (0xb55e bytes) 2024-09-14T23:56:44.162 INFO:teuthology.orchestra.run.smithi042.stdout:688 flatten 2024-09-14T23:56:44.169 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:44.169 INFO:teuthology.orchestra.run.smithi042.stdout:693 write 0x4d10c66 thru 0x4d19775 (0x8b10 bytes) 2024-09-14T23:56:44.172 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:44.180 INFO:teuthology.orchestra.run.smithi042.stdout:695 trunc from 0xc002b6f to 0x2ebe010 2024-09-14T23:56:44.199 INFO:teuthology.orchestra.run.smithi042.stdout:696 write 0x4a8a857 thru 0x4a8f7a7 (0x4f51 bytes) 2024-09-14T23:56:44.200 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:44.200 INFO:teuthology.orchestra.run.smithi042.stdout:699 write 0x3aa774e thru 0x3ab7155 (0xfa08 bytes) 2024-09-14T23:56:44.202 INFO:teuthology.orchestra.run.smithi042.stdout:700 read 0x127e56f thru 0x1287455 (0x8ee7 bytes) 2024-09-14T23:56:44.202 INFO:teuthology.orchestra.run.smithi042.stdout:701 read 0x4a87bbf thru 0x4a8f7a7 (0x7be9 bytes) 2024-09-14T23:56:44.206 INFO:teuthology.orchestra.run.smithi042.stdout:702 trunc from 0x4a8f7a8 to 0x1b56e90 2024-09-14T23:56:44.207 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:56:44.206+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:56:44.219 INFO:teuthology.orchestra.run.smithi042.stdout:703 write 0x316ae8e thru 0x3175fec (0xb15f bytes) 2024-09-14T23:56:44.221 INFO:teuthology.orchestra.run.smithi042.stdout:704 clone 5 order 24 su 131072 sc 2 2024-09-14T23:56:44.244 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:56:44.244 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:56:44.244 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:56:44.244 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:56:44.244 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:56:44.253 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T23:56:44.308 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:56:44.307+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:56:44.402 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:56:44.403 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:56:44.403 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:56:44.403 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:56:44.403 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.403 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:3500 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T23:56:44.403 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:37.990193+0000", 2024-09-14T23:56:44.403 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 6.4099021240000003, 2024-09-14T23:56:44.403 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99440750899999997, 2024-09-14T23:56:44.403 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:44.403 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:44.403 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:44.403 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:44.404 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:44.404 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:44.404 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3500 2024-09-14T23:56:44.404 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.404 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:44.404 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.404 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:44.404 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990193+0000", 2024-09-14T23:56:44.404 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:44.404 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.404 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.404 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:44.404 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990193+0000", 2024-09-14T23:56:44.404 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:44.404 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.405 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.405 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:44.405 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990194+0000", 2024-09-14T23:56:44.405 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5000000000000001e-07 2024-09-14T23:56:44.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.405 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.405 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:44.405 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990200+0000", 2024-09-14T23:56:44.405 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4389999999999997e-06 2024-09-14T23:56:44.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.405 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.405 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:44.405 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990215+0000", 2024-09-14T23:56:44.405 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4613999999999999e-05 2024-09-14T23:56:44.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.406 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:44.406 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990224+0000", 2024-09-14T23:56:44.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1740000000000006e-06 2024-09-14T23:56:44.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.406 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:44.406 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990283+0000", 2024-09-14T23:56:44.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9231999999999998e-05 2024-09-14T23:56:44.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.406 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:56:44.406 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990291+0000", 2024-09-14T23:56:44.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5900000000000002e-06 2024-09-14T23:56:44.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.407 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:44.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:38.984249+0000", 2024-09-14T23:56:44.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99395822700000003 2024-09-14T23:56:44.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.407 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:44.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:38.984448+0000", 2024-09-14T23:56:44.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00019855900000000001 2024-09-14T23:56:44.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.407 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:44.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:38.984601+0000", 2024-09-14T23:56:44.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000152824 2024-09-14T23:56:44.408 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:44.408 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:44.408 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:44.408 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.408 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.408 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4054 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1032192~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:44.408 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099053+0000", 2024-09-14T23:56:44.408 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.3010421399999998, 2024-09-14T23:56:44.408 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044065339000000002, 2024-09-14T23:56:44.408 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:44.408 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:44.408 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:44.408 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:44.409 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:44.409 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:44.409 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4054 2024-09-14T23:56:44.409 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.409 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:44.409 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.409 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:44.409 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099053+0000", 2024-09-14T23:56:44.409 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:44.409 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.409 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.409 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:44.409 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099053+0000", 2024-09-14T23:56:44.410 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:44.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.410 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.410 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:44.410 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099053+0000", 2024-09-14T23:56:44.410 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1100000000000002e-07 2024-09-14T23:56:44.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.410 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.410 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:44.410 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099055+0000", 2024-09-14T23:56:44.410 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.404e-06 2024-09-14T23:56:44.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.410 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.411 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:44.411 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099059+0000", 2024-09-14T23:56:44.411 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.749e-06 2024-09-14T23:56:44.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.411 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.411 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:44.411 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099062+0000", 2024-09-14T23:56:44.411 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7809999999999999e-06 2024-09-14T23:56:44.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.411 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.411 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:44.411 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143012+0000", 2024-09-14T23:56:44.412 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043949929999999998 2024-09-14T23:56:44.412 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.412 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.412 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:44.412 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143058+0000", 2024-09-14T23:56:44.412 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5260999999999999e-05 2024-09-14T23:56:44.412 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.412 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.412 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:44.412 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143118+0000", 2024-09-14T23:56:44.412 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0903000000000003e-05 2024-09-14T23:56:44.412 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:44.413 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:44.413 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:44.413 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.413 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.413 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4055 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1093632~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:44.413 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099066+0000", 2024-09-14T23:56:44.413 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.3010292359999998, 2024-09-14T23:56:44.413 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044174782000000003, 2024-09-14T23:56:44.413 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:44.413 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:44.413 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:44.413 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:44.413 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:44.414 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:44.414 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4055 2024-09-14T23:56:44.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.414 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:44.414 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.414 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:44.414 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099066+0000", 2024-09-14T23:56:44.414 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:44.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.414 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.414 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:44.414 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099066+0000", 2024-09-14T23:56:44.414 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:44.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.415 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.415 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:44.415 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099066+0000", 2024-09-14T23:56:44.415 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3700000000000001e-07 2024-09-14T23:56:44.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.415 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.415 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:44.415 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099067+0000", 2024-09-14T23:56:44.415 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1379999999999999e-06 2024-09-14T23:56:44.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.415 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.415 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:44.415 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099073+0000", 2024-09-14T23:56:44.415 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4790000000000004e-06 2024-09-14T23:56:44.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.416 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.416 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:44.416 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099075+0000", 2024-09-14T23:56:44.416 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5320000000000001e-06 2024-09-14T23:56:44.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.416 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.416 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:44.416 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143147+0000", 2024-09-14T23:56:44.416 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044071676999999997 2024-09-14T23:56:44.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.416 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.416 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:44.416 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143186+0000", 2024-09-14T23:56:44.417 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8821e-05 2024-09-14T23:56:44.417 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.417 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.417 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:44.417 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143241+0000", 2024-09-14T23:56:44.417 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4797999999999999e-05 2024-09-14T23:56:44.417 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:44.417 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:44.417 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:44.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.418 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4056 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1155072~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:44.418 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099079+0000", 2024-09-14T23:56:44.418 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.3010165210000002, 2024-09-14T23:56:44.418 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044295020999999997, 2024-09-14T23:56:44.418 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:44.418 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:44.418 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:44.418 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:44.418 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:44.418 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:44.418 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4056 2024-09-14T23:56:44.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.419 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:44.419 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.419 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:44.419 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099079+0000", 2024-09-14T23:56:44.419 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:44.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.419 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.419 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:44.419 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099079+0000", 2024-09-14T23:56:44.419 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:44.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.419 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.419 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:44.419 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099079+0000", 2024-09-14T23:56:44.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1500000000000001e-07 2024-09-14T23:56:44.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.420 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.420 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:44.420 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099080+0000", 2024-09-14T23:56:44.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0210000000000001e-06 2024-09-14T23:56:44.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.420 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.420 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:44.420 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099083+0000", 2024-09-14T23:56:44.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.174e-06 2024-09-14T23:56:44.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.420 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.420 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:44.420 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099086+0000", 2024-09-14T23:56:44.421 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.446e-06 2024-09-14T23:56:44.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.421 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.421 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:44.421 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143269+0000", 2024-09-14T23:56:44.421 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044183489999999999 2024-09-14T23:56:44.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.421 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.421 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:44.421 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143314+0000", 2024-09-14T23:56:44.421 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5135000000000001e-05 2024-09-14T23:56:44.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.421 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.421 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:44.422 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143374+0000", 2024-09-14T23:56:44.422 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9540000000000003e-05 2024-09-14T23:56:44.422 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:44.422 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:44.422 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:44.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.422 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.422 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4057 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1216512~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:44.422 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099088+0000", 2024-09-14T23:56:44.422 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.3010067339999996, 2024-09-14T23:56:44.422 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044406119000000001, 2024-09-14T23:56:44.422 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:44.422 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:44.423 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:44.423 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:44.423 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:44.423 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:44.423 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4057 2024-09-14T23:56:44.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.423 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:44.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.423 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:44.423 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099088+0000", 2024-09-14T23:56:44.423 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:44.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.423 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:44.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099088+0000", 2024-09-14T23:56:44.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:44.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:44.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099089+0000", 2024-09-14T23:56:44.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.04e-07 2024-09-14T23:56:44.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:44.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099090+0000", 2024-09-14T23:56:44.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.026e-06 2024-09-14T23:56:44.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:44.425 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099093+0000", 2024-09-14T23:56:44.425 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0819999999999999e-06 2024-09-14T23:56:44.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.425 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:44.425 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099095+0000", 2024-09-14T23:56:44.425 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4930000000000001e-06 2024-09-14T23:56:44.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.425 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:44.425 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143400+0000", 2024-09-14T23:56:44.425 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044304698000000003 2024-09-14T23:56:44.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.426 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:44.426 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143441+0000", 2024-09-14T23:56:44.426 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1100000000000003e-05 2024-09-14T23:56:44.426 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.426 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.426 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:44.426 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143495+0000", 2024-09-14T23:56:44.426 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3516000000000001e-05 2024-09-14T23:56:44.426 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:44.426 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:44.426 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:44.426 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.427 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.427 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4058 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1277952~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:44.427 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099099+0000", 2024-09-14T23:56:44.427 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.3009962430000002, 2024-09-14T23:56:44.427 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044514583000000003, 2024-09-14T23:56:44.427 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:44.427 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:44.427 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:44.427 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:44.427 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:44.427 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:44.427 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4058 2024-09-14T23:56:44.427 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.427 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:44.427 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.428 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:44.428 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099099+0000", 2024-09-14T23:56:44.428 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:44.428 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.428 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.428 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:44.428 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099099+0000", 2024-09-14T23:56:44.428 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:44.428 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.428 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.428 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:44.428 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099099+0000", 2024-09-14T23:56:44.428 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9999999999999999e-07 2024-09-14T23:56:44.428 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.429 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:44.429 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099100+0000", 2024-09-14T23:56:44.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7199999999999997e-07 2024-09-14T23:56:44.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.429 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:44.429 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099103+0000", 2024-09-14T23:56:44.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.095e-06 2024-09-14T23:56:44.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.429 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:44.429 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099106+0000", 2024-09-14T23:56:44.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4229999999999999e-06 2024-09-14T23:56:44.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.430 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:44.430 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143524+0000", 2024-09-14T23:56:44.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044417891000000001 2024-09-14T23:56:44.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.430 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:44.430 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143568+0000", 2024-09-14T23:56:44.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4060000000000002e-05 2024-09-14T23:56:44.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.430 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:44.430 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143614+0000", 2024-09-14T23:56:44.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5942000000000002e-05 2024-09-14T23:56:44.430 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:44.431 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:44.431 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:44.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.431 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.431 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4059 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1339392~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:44.431 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099114+0000", 2024-09-14T23:56:44.431 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.3009817039999998, 2024-09-14T23:56:44.431 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044617194999999998, 2024-09-14T23:56:44.431 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:44.431 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:44.431 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:44.431 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:44.431 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:44.431 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:44.432 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4059 2024-09-14T23:56:44.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.432 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:44.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.432 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:44.432 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099114+0000", 2024-09-14T23:56:44.432 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:44.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.432 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:44.432 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099114+0000", 2024-09-14T23:56:44.432 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:44.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.433 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:44.433 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099114+0000", 2024-09-14T23:56:44.433 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5899999999999998e-07 2024-09-14T23:56:44.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.433 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:44.433 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099115+0000", 2024-09-14T23:56:44.433 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3510000000000001e-06 2024-09-14T23:56:44.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.433 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:44.433 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099119+0000", 2024-09-14T23:56:44.433 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.16e-06 2024-09-14T23:56:44.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.434 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:44.434 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099123+0000", 2024-09-14T23:56:44.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1339999999999997e-06 2024-09-14T23:56:44.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.434 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:44.434 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143643+0000", 2024-09-14T23:56:44.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044519232999999998 2024-09-14T23:56:44.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.434 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:44.434 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143686+0000", 2024-09-14T23:56:44.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3349000000000001e-05 2024-09-14T23:56:44.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.435 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:44.435 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143731+0000", 2024-09-14T23:56:44.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4709000000000002e-05 2024-09-14T23:56:44.435 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:44.435 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:44.435 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:44.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.435 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4060 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1400832~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:44.435 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099129+0000", 2024-09-14T23:56:44.435 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.3009664809999997, 2024-09-14T23:56:44.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044721271999999999, 2024-09-14T23:56:44.436 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:44.436 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:44.436 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:44.436 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:44.436 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:44.436 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:44.436 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4060 2024-09-14T23:56:44.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.436 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:44.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.436 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:44.436 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099129+0000", 2024-09-14T23:56:44.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:44.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.437 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:44.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099129+0000", 2024-09-14T23:56:44.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:44.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.437 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:44.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099191+0000", 2024-09-14T23:56:44.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1805999999999997e-05 2024-09-14T23:56:44.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.437 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:44.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099193+0000", 2024-09-14T23:56:44.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2330000000000001e-06 2024-09-14T23:56:44.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:44.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099200+0000", 2024-09-14T23:56:44.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4200000000000001e-06 2024-09-14T23:56:44.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:44.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099204+0000", 2024-09-14T23:56:44.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7500000000000001e-06 2024-09-14T23:56:44.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.439 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:44.439 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143760+0000", 2024-09-14T23:56:44.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044556182 2024-09-14T23:56:44.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.439 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:44.439 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143804+0000", 2024-09-14T23:56:44.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4239000000000001e-05 2024-09-14T23:56:44.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.439 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:44.439 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143850+0000", 2024-09-14T23:56:44.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5642000000000001e-05 2024-09-14T23:56:44.440 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:44.440 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:44.440 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:44.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.440 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4061 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1462272~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:44.440 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099215+0000", 2024-09-14T23:56:44.440 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.3008803560000004, 2024-09-14T23:56:44.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044745659, 2024-09-14T23:56:44.440 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:44.440 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:44.440 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:44.440 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:44.440 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4061 2024-09-14T23:56:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099215+0000", 2024-09-14T23:56:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099215+0000", 2024-09-14T23:56:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100672+0000", 2024-09-14T23:56:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00145689 2024-09-14T23:56:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100677+0000", 2024-09-14T23:56:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0139999999999998e-06 2024-09-14T23:56:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100689+0000", 2024-09-14T23:56:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.183e-05 2024-09-14T23:56:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100696+0000", 2024-09-14T23:56:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2599999999999999e-06 2024-09-14T23:56:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143877+0000", 2024-09-14T23:56:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043181279000000003 2024-09-14T23:56:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143919+0000", 2024-09-14T23:56:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1783000000000003e-05 2024-09-14T23:56:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143961+0000", 2024-09-14T23:56:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1603000000000001e-05 2024-09-14T23:56:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4070 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 2015232~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.100903+0000", 2024-09-14T23:56:44.445 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.299192648, 2024-09-14T23:56:44.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044103815999999997, 2024-09-14T23:56:44.445 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:44.445 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:44.445 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:44.445 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:44.445 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:44.445 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:44.445 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4070 2024-09-14T23:56:44.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.445 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:44.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:44.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100903+0000", 2024-09-14T23:56:44.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:44.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.446 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:44.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100903+0000", 2024-09-14T23:56:44.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:44.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.446 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:44.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.102671+0000", 2024-09-14T23:56:44.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001768279 2024-09-14T23:56:44.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.102675+0000", 2024-09-14T23:56:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9609999999999999e-06 2024-09-14T23:56:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.102684+0000", 2024-09-14T23:56:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.329e-06 2024-09-14T23:56:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.102689+0000", 2024-09-14T23:56:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7079999999999996e-06 2024-09-14T23:56:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.144912+0000", 2024-09-14T23:56:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.042223305000000003 2024-09-14T23:56:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.144960+0000", 2024-09-14T23:56:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7670000000000003e-05 2024-09-14T23:56:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.145006+0000", 2024-09-14T23:56:44.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6563999999999997e-05 2024-09-14T23:56:44.449 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:44.449 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:44.449 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:44.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.449 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4096 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1282048~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:44.449 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.104957+0000", 2024-09-14T23:56:44.449 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.2951384749999999, 2024-09-14T23:56:44.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044093016999999998, 2024-09-14T23:56:44.449 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:44.449 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:44.449 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:44.449 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:44.450 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:44.450 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:44.450 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4096 2024-09-14T23:56:44.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.450 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:44.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.450 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:44.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104957+0000", 2024-09-14T23:56:44.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:44.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.450 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:44.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104957+0000", 2024-09-14T23:56:44.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104957+0000", 2024-09-14T23:56:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7400000000000003e-07 2024-09-14T23:56:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104960+0000", 2024-09-14T23:56:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3800000000000001e-06 2024-09-14T23:56:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104967+0000", 2024-09-14T23:56:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4030000000000001e-06 2024-09-14T23:56:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104973+0000", 2024-09-14T23:56:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7370000000000003e-06 2024-09-14T23:56:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.148883+0000", 2024-09-14T23:56:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043910124000000002 2024-09-14T23:56:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:44.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.148966+0000", 2024-09-14T23:56:44.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2868999999999996e-05 2024-09-14T23:56:44.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:44.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149050+0000", 2024-09-14T23:56:44.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3930000000000004e-05 2024-09-14T23:56:44.453 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:44.453 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:44.453 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:44.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.454 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4097 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1343488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:44.454 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.104983+0000", 2024-09-14T23:56:44.454 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.2951126860000004, 2024-09-14T23:56:44.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044182104999999999, 2024-09-14T23:56:44.454 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:44.454 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:44.454 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:44.454 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:44.454 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:44.454 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:44.454 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4097 2024-09-14T23:56:44.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.454 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:44.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:44.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104983+0000", 2024-09-14T23:56:44.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:44.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:44.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104983+0000", 2024-09-14T23:56:44.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:44.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105207+0000", 2024-09-14T23:56:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000224687 2024-09-14T23:56:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105211+0000", 2024-09-14T23:56:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3050000000000001e-06 2024-09-14T23:56:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105217+0000", 2024-09-14T23:56:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8170000000000001e-06 2024-09-14T23:56:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105222+0000", 2024-09-14T23:56:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.267e-06 2024-09-14T23:56:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149079+0000", 2024-09-14T23:56:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043857581999999999 2024-09-14T23:56:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149122+0000", 2024-09-14T23:56:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2899999999999999e-05 2024-09-14T23:56:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:44.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149165+0000", 2024-09-14T23:56:44.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2546999999999998e-05 2024-09-14T23:56:44.458 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:44.458 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:44.458 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:44.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.458 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4098 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1404928~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:44.458 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.105242+0000", 2024-09-14T23:56:44.458 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.294852874, 2024-09-14T23:56:44.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044121104000000001, 2024-09-14T23:56:44.458 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:44.459 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:44.459 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:44.459 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:44.459 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:44.459 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:44.459 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4098 2024-09-14T23:56:44.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.459 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:44.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:44.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105242+0000", 2024-09-14T23:56:44.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105242+0000", 2024-09-14T23:56:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105243+0000", 2024-09-14T23:56:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0399999999999995e-07 2024-09-14T23:56:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105246+0000", 2024-09-14T23:56:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6309999999999999e-06 2024-09-14T23:56:44.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:44.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105253+0000", 2024-09-14T23:56:44.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0940000000000002e-06 2024-09-14T23:56:44.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:44.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105260+0000", 2024-09-14T23:56:44.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1799999999999999e-06 2024-09-14T23:56:44.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:44.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149201+0000", 2024-09-14T23:56:44.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043940629000000002 2024-09-14T23:56:44.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:44.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149283+0000", 2024-09-14T23:56:44.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2360000000000004e-05 2024-09-14T23:56:44.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:44.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149363+0000", 2024-09-14T23:56:44.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0506000000000005e-05 2024-09-14T23:56:44.463 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:44.463 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:44.463 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:44.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.463 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4099 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1466368~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:44.463 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.105270+0000", 2024-09-14T23:56:44.463 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.2948252790000003, 2024-09-14T23:56:44.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044195433999999999, 2024-09-14T23:56:44.463 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:44.463 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:44.463 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:44.464 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:44.464 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:44.464 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:44.464 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4099 2024-09-14T23:56:44.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.464 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:44.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:44.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105270+0000", 2024-09-14T23:56:44.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:44.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:44.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105270+0000", 2024-09-14T23:56:44.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105429+0000", 2024-09-14T23:56:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015940299999999999 2024-09-14T23:56:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105433+0000", 2024-09-14T23:56:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3579999999999999e-06 2024-09-14T23:56:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105439+0000", 2024-09-14T23:56:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1759999999999998e-06 2024-09-14T23:56:44.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:44.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105445+0000", 2024-09-14T23:56:44.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9689999999999999e-06 2024-09-14T23:56:44.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:44.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149381+0000", 2024-09-14T23:56:44.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043936389999999999 2024-09-14T23:56:44.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:44.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149422+0000", 2024-09-14T23:56:44.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0865000000000003e-05 2024-09-14T23:56:44.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:44.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149465+0000", 2024-09-14T23:56:44.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3272999999999997e-05 2024-09-14T23:56:44.467 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:44.467 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:44.467 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:44.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.467 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4100 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1527808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:44.467 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.105454+0000", 2024-09-14T23:56:44.467 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.2946408079999996, 2024-09-14T23:56:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044217411999999998, 2024-09-14T23:56:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4100 2024-09-14T23:56:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105454+0000", 2024-09-14T23:56:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105454+0000", 2024-09-14T23:56:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105578+0000", 2024-09-14T23:56:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012401799999999999 2024-09-14T23:56:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105583+0000", 2024-09-14T23:56:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1779999999999996e-06 2024-09-14T23:56:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105590+0000", 2024-09-14T23:56:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4279999999999999e-06 2024-09-14T23:56:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105596+0000", 2024-09-14T23:56:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0120000000000002e-06 2024-09-14T23:56:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149503+0000", 2024-09-14T23:56:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043906580000000001 2024-09-14T23:56:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:44.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149585+0000", 2024-09-14T23:56:44.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2356999999999998e-05 2024-09-14T23:56:44.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:44.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149672+0000", 2024-09-14T23:56:44.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6839e-05 2024-09-14T23:56:44.471 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:44.471 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:44.471 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:44.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4101 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1589248~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.105609+0000", 2024-09-14T23:56:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.2944862690000001, 2024-09-14T23:56:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044175987999999999, 2024-09-14T23:56:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4101 2024-09-14T23:56:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:44.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105609+0000", 2024-09-14T23:56:44.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:44.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:44.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105609+0000", 2024-09-14T23:56:44.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:44.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:44.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.106671+0000", 2024-09-14T23:56:44.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010619270000000001 2024-09-14T23:56:44.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:44.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.106675+0000", 2024-09-14T23:56:44.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4429999999999996e-06 2024-09-14T23:56:44.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:44.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.106685+0000", 2024-09-14T23:56:44.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7429999999999997e-06 2024-09-14T23:56:44.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:44.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.106690+0000", 2024-09-14T23:56:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.293e-06 2024-09-14T23:56:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149700+0000", 2024-09-14T23:56:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043009512 2024-09-14T23:56:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149742+0000", 2024-09-14T23:56:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2083999999999999e-05 2024-09-14T23:56:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149785+0000", 2024-09-14T23:56:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2985999999999998e-05 2024-09-14T23:56:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4116 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 180224~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.107575+0000", 2024-09-14T23:56:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.2925198010000001, 2024-09-14T23:56:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044157522999999997, 2024-09-14T23:56:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4116 2024-09-14T23:56:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.107575+0000", 2024-09-14T23:56:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.107575+0000", 2024-09-14T23:56:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108671+0000", 2024-09-14T23:56:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010954510000000001 2024-09-14T23:56:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108676+0000", 2024-09-14T23:56:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6639999999999997e-06 2024-09-14T23:56:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108685+0000", 2024-09-14T23:56:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1130000000000007e-06 2024-09-14T23:56:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108691+0000", 2024-09-14T23:56:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6019999999999996e-06 2024-09-14T23:56:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.151530+0000", 2024-09-14T23:56:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.042838345999999999 2024-09-14T23:56:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.151619+0000", 2024-09-14T23:56:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9302999999999995e-05 2024-09-14T23:56:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.151733+0000", 2024-09-14T23:56:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011404399999999999 2024-09-14T23:56:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4122 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 548864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.108842+0000", 2024-09-14T23:56:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.2912529629999998, 2024-09-14T23:56:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044103631999999997, 2024-09-14T23:56:44.481 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:44.481 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:44.481 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:44.481 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:44.481 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:44.481 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:44.481 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4122 2024-09-14T23:56:44.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.481 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:44.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:44.481 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108842+0000", 2024-09-14T23:56:44.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:44.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:44.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108842+0000", 2024-09-14T23:56:44.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:44.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:44.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108843+0000", 2024-09-14T23:56:44.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7000000000000001e-07 2024-09-14T23:56:44.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108844+0000", 2024-09-14T23:56:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6479999999999999e-06 2024-09-14T23:56:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108848+0000", 2024-09-14T23:56:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8029999999999998e-06 2024-09-14T23:56:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108851+0000", 2024-09-14T23:56:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7939999999999998e-06 2024-09-14T23:56:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.152838+0000", 2024-09-14T23:56:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043987351000000001 2024-09-14T23:56:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.152885+0000", 2024-09-14T23:56:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7237000000000002e-05 2024-09-14T23:56:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.152946+0000", 2024-09-14T23:56:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0529000000000003e-05 2024-09-14T23:56:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4123 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 610304~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.108855+0000", 2024-09-14T23:56:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.2912406340000002, 2024-09-14T23:56:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044321610999999997, 2024-09-14T23:56:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4123 2024-09-14T23:56:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108855+0000", 2024-09-14T23:56:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108855+0000", 2024-09-14T23:56:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:44.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.110671+0000", 2024-09-14T23:56:44.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018159719999999999 2024-09-14T23:56:44.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:44.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.110676+0000", 2024-09-14T23:56:44.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9450000000000001e-06 2024-09-14T23:56:44.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:44.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.110685+0000", 2024-09-14T23:56:44.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5359999999999998e-06 2024-09-14T23:56:44.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:44.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.110692+0000", 2024-09-14T23:56:44.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9280000000000004e-06 2024-09-14T23:56:44.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:44.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.152984+0000", 2024-09-14T23:56:44.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.042291677999999999 2024-09-14T23:56:44.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:44.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.153066+0000", 2024-09-14T23:56:44.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2657999999999994e-05 2024-09-14T23:56:44.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:44.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.153176+0000", 2024-09-14T23:56:44.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000109894 2024-09-14T23:56:44.489 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:44.489 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:44.489 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:44.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.489 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4136 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:44.489 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.111056+0000", 2024-09-14T23:56:44.490 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.2890394860000001, 2024-09-14T23:56:44.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044097428000000001, 2024-09-14T23:56:44.490 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:44.490 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:44.490 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:44.490 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:44.490 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:44.490 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:44.490 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4136 2024-09-14T23:56:44.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.490 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:44.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:44.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.111056+0000", 2024-09-14T23:56:44.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:44.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:44.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.111056+0000", 2024-09-14T23:56:44.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:44.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:44.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.112671+0000", 2024-09-14T23:56:44.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016153999999999999 2024-09-14T23:56:44.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:44.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.112675+0000", 2024-09-14T23:56:44.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2830000000000003e-06 2024-09-14T23:56:44.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:44.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.112684+0000", 2024-09-14T23:56:44.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7800000000000006e-06 2024-09-14T23:56:44.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:44.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.112689+0000", 2024-09-14T23:56:44.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0000000000000004e-06 2024-09-14T23:56:44.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:44.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.155064+0000", 2024-09-14T23:56:44.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.042374423000000001 2024-09-14T23:56:44.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:44.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.155111+0000", 2024-09-14T23:56:44.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7367000000000002e-05 2024-09-14T23:56:44.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:44.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:44.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:44.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.155153+0000", 2024-09-14T23:56:44.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2175000000000002e-05 2024-09-14T23:56:44.494 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:44.494 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:44.494 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:44.494 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:44.494 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:44.494 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:56:44.496 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T23:56:44.496 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:56:44.494+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:56:44.596 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:56:44.595+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:56:44.644 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:56:44.644 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:56:44.644 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:56:44.644 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:56:44.653 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T23:56:44.683 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T23:56:44.727 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:56:44.726+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:56:44.798 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:56:44.796+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:56:44.801 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:56:44.801 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:56:44.801 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:56:44.801 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:56:44.801 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:56:44.809 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T23:56:44.814 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:56:44.814 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:56:44.831 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:56:44.831 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:56:44.846 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:56:44.847 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:56:44.863 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:56:44.863 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:56:44.879 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:56:44.879 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:56:44.896 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:56:44.896 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:56:44.899 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:56:44.897+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:56:44.912 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:56:44.913 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:56:44.916 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:44.916 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:44.925 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T23:56:44.929 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:56:44.929 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:56:44.981 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T23:56:45.024 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:56:45.023+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:56:45.035 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:45.035 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:45.045 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T23:56:45.100 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:56:45.099+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:56:45.124 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-09-14T23:56:45.162 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:45.163 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:45.175 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T23:56:45.220 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:56:45.219+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:56:45.303 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:56:45.301+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:56:45.314 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:45.314 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:45.324 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T23:56:45.408 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:56:45.407+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:56:45.454 INFO:teuthology.orchestra.run.smithi042.stdout:707 punch from 0x2472a27 to 0x2479b47, (0x7120 bytes) 2024-09-14T23:56:45.462 INFO:teuthology.orchestra.run.smithi042.stdout:708 trunc from 0x3175fed to 0x62be641 2024-09-14T23:56:45.467 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:45.467 INFO:teuthology.orchestra.run.smithi042.stdout:711 write 0x42f454a thru 0x42fc8f6 (0x83ad bytes) 2024-09-14T23:56:45.474 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:45.474 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:45.474 INFO:teuthology.orchestra.run.smithi042.stdout:712 trunc from 0x62be641 to 0x9a1aac6 2024-09-14T23:56:45.481 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:45.482 INFO:teuthology.orchestra.run.smithi042.stdout:714 trunc from 0x9a1aac6 to 0xab3d171 2024-09-14T23:56:45.484 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T23:56:45.487 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:45.487 INFO:teuthology.orchestra.run.smithi042.stdout:716 read 0x9e462f3 thru 0x9e4818a (0x1e98 bytes) 2024-09-14T23:56:45.488 INFO:teuthology.orchestra.run.smithi042.stdout:718 trunc from 0xab3d171 to 0xb9f75f7 2024-09-14T23:56:45.493 INFO:teuthology.orchestra.run.smithi042.stdout:719 write 0x6332647 thru 0x63390ec (0x6aa6 bytes) 2024-09-14T23:56:45.496 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:45.496 INFO:teuthology.orchestra.run.smithi042.stdout:721 write 0xb1735a9 thru 0xb17941f (0x5e77 bytes) 2024-09-14T23:56:45.499 INFO:teuthology.orchestra.run.smithi042.stdout:723 read 0x2af8331 thru 0x2afdc52 (0x5922 bytes) 2024-09-14T23:56:45.499 INFO:teuthology.orchestra.run.smithi042.stdout:724 read 0xd5bb84 thru 0xd5c034 (0x4b1 bytes) 2024-09-14T23:56:45.500 INFO:teuthology.orchestra.run.smithi042.stdout:725 write 0xb4e0a9c thru 0xb4ec251 (0xb7b6 bytes) 2024-09-14T23:56:45.501 INFO:teuthology.orchestra.run.smithi042.stdout:726 trunc from 0xb9f75f7 to 0xb9c83b0 2024-09-14T23:56:45.505 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:56:45.504+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:56:45.507 INFO:teuthology.orchestra.run.smithi042.stdout:728 read 0xa290600 thru 0xa29b3ff (0xae00 bytes) 2024-09-14T23:56:45.507 INFO:teuthology.orchestra.run.smithi042.stdout:729 read 0x9810d3 thru 0x9889fe (0x792c bytes) 2024-09-14T23:56:45.507 INFO:teuthology.orchestra.run.smithi042.stdout:730 write 0x9363786 thru 0x9372cb0 (0xf52b bytes) 2024-09-14T23:56:45.510 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:45.510 INFO:teuthology.orchestra.run.smithi042.stdout:732 read 0x6b22f13 thru 0x6b24243 (0x1331 bytes) 2024-09-14T23:56:45.510 INFO:teuthology.orchestra.run.smithi042.stdout:733 read 0x7ab52a thru 0x7b900d (0xdae4 bytes) 2024-09-14T23:56:45.511 INFO:teuthology.orchestra.run.smithi042.stdout:734 write 0xa82ea06 thru 0xa834f4e (0x6549 bytes) 2024-09-14T23:56:45.512 INFO:teuthology.orchestra.run.smithi042.stdout:736 clone 6 order 20 su 65536 sc 13 2024-09-14T23:56:45.606 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:56:45.605+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:56:45.631 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:45.631 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:45.640 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T23:56:45.707 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:56:45.706+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:56:45.788 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:45.788 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:45.796 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T23:56:45.841 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:56:45.839+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:56:45.905 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:45.906 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:45.908 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:56:45.907+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:56:45.914 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T23:56:46.006 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is set 2024-09-14T23:56:46.009 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:56:46.007+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:56:46.065 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone5 from 0xb9c83b0 (overlap 0x3175fed) to 0x0 2024-09-14T23:56:46.071 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:46.071 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:46.086 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T23:56:46.129 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:56:46.128+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:56:46.139 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-09-14T23:56:46.197 INFO:teuthology.orchestra.run.smithi042.stdout:737 read 0xb404835 thru 0xb40b5e8 (0x6db4 bytes) 2024-09-14T23:56:46.198 INFO:teuthology.orchestra.run.smithi042.stdout:738 write 0xb6d17bc thru 0xb6e10f3 (0xf938 bytes) 2024-09-14T23:56:46.203 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:46.204 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:46.206 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:46.206 INFO:teuthology.orchestra.run.smithi042.stdout:743 read 0x530a387 thru 0x530aa5b (0x6d5 bytes) 2024-09-14T23:56:46.210 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:56:46.209+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:56:46.214 INFO:teuthology.orchestra.run.smithi042.stdout:744 write 0xe68c1c8 thru 0xe68d89d (0x16d6 bytes) 2024-09-14T23:56:46.215 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T23:56:46.217 INFO:teuthology.orchestra.run.smithi042.stdout:745 write 0x85accdd thru 0x85b65e9 (0x990d bytes) 2024-09-14T23:56:46.223 INFO:teuthology.orchestra.run.smithi042.stdout:747 writesame 0x6218058 thru 0x6225b78 data_size 0x45(0xdb21 bytes) 2024-09-14T23:56:46.229 INFO:teuthology.orchestra.run.smithi042.stdout:748 read 0x3605304 thru 0x3614c6a (0xf967 bytes) 2024-09-14T23:56:46.229 INFO:teuthology.orchestra.run.smithi042.stdout:749 writesame 0xc57f995 thru 0xc582b1c data_size 0x14(0x3188 bytes) 2024-09-14T23:56:46.233 INFO:teuthology.orchestra.run.smithi042.stdout:751 write 0x29e12cd thru 0x29e2455 (0x1189 bytes) 2024-09-14T23:56:46.237 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:46.237 INFO:teuthology.orchestra.run.smithi042.stdout:753 punch from 0x669f431 to 0x66a075d, (0x132c bytes) 2024-09-14T23:56:46.237 INFO:teuthology.orchestra.run.smithi042.stdout:755 write 0x99b011 thru 0x99f497 (0x4487 bytes) 2024-09-14T23:56:46.242 INFO:teuthology.orchestra.run.smithi042.stdout:757 write 0x27dda5e thru 0x27de6f4 (0xc97 bytes) 2024-09-14T23:56:46.246 INFO:teuthology.orchestra.run.smithi042.stdout:759 write 0xe4800b2 thru 0xe486768 (0x66b7 bytes) 2024-09-14T23:56:46.254 INFO:teuthology.orchestra.run.smithi042.stdout:760 writesame 0xedfa96d thru 0xee0a51a data_size 0x11(0xfbae bytes) 2024-09-14T23:56:46.255 INFO:teuthology.orchestra.run.smithi042.stdout:761 read 0x9e19299 thru 0x9e23591 (0xa2f9 bytes) 2024-09-14T23:56:46.255 INFO:teuthology.orchestra.run.smithi042.stdout:762 write 0x46ac17 thru 0x47a743 (0xfb2d bytes) 2024-09-14T23:56:46.263 INFO:teuthology.orchestra.run.smithi042.stdout:763 writesame 0x42804fe thru 0x428252c data_size 0xb(0x202f bytes) 2024-09-14T23:56:46.267 INFO:teuthology.orchestra.run.smithi042.stdout:764 read 0xabbb851 thru 0xabbcb59 (0x1309 bytes) 2024-09-14T23:56:46.267 INFO:teuthology.orchestra.run.smithi042.stdout:765 writesame 0xd8d1370 thru 0xd8d28f2 data_size 0x1(0x1583 bytes) 2024-09-14T23:56:46.271 INFO:teuthology.orchestra.run.smithi042.stdout:767 write 0x3585aa thru 0x360485 (0x7edc bytes) 2024-09-14T23:56:46.280 INFO:teuthology.orchestra.run.smithi042.stdout:768 read 0x1818f98 thru 0x181c36b (0x33d4 bytes) 2024-09-14T23:56:46.280 INFO:teuthology.orchestra.run.smithi042.stdout:769 writesame 0xdba1657 thru 0xdba3aea data_size 0x2(0x2494 bytes) 2024-09-14T23:56:46.285 INFO:teuthology.orchestra.run.smithi042.stdout:770 punch from 0x39bc8ff to 0x39bd536, (0xc37 bytes) 2024-09-14T23:56:46.285 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:46.285 INFO:teuthology.orchestra.run.smithi042.stdout:774 read 0xab88b62 thru 0xab8faf3 (0x6f92 bytes) 2024-09-14T23:56:46.285 INFO:teuthology.orchestra.run.smithi042.stdout:775 read 0x10e1bd7 thru 0x10e2859 (0xc83 bytes) 2024-09-14T23:56:46.287 INFO:teuthology.orchestra.run.smithi042.stdout:776 trunc from 0xee0a51b to 0xd4eff5a 2024-09-14T23:56:46.298 INFO:teuthology.orchestra.run.smithi042.stdout:777 writesame 0x268b208 thru 0x2690abf data_size 0x8(0x58b8 bytes) 2024-09-14T23:56:46.303 INFO:teuthology.orchestra.run.smithi042.stdout:778 write 0x1629dc5 thru 0x162f27d (0x54b9 bytes) 2024-09-14T23:56:46.308 INFO:teuthology.orchestra.run.smithi042.stdout:779 punch from 0x4fb6ed6 to 0x4fbf9ff, (0x8b29 bytes) 2024-09-14T23:56:46.308 INFO:teuthology.orchestra.run.smithi042.stdout:780 write 0x89dd578 thru 0x89e03ea (0x2e73 bytes) 2024-09-14T23:56:46.311 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:56:46.310+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:56:46.313 INFO:teuthology.orchestra.run.smithi042.stdout:781 punch from 0x96ceb6 to 0x975036, (0x8180 bytes) 2024-09-14T23:56:46.314 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:46.314 INFO:teuthology.orchestra.run.smithi042.stdout:784 punch from 0xb5e7294 to 0xb5f570a, (0xe476 bytes) 2024-09-14T23:56:46.314 INFO:teuthology.orchestra.run.smithi042.stdout:785 read 0xad1cba7 thru 0xad271ac (0xa606 bytes) 2024-09-14T23:56:46.314 INFO:teuthology.orchestra.run.smithi042.stdout:787 punch from 0x83487b5 to 0x83522e8, (0x9b33 bytes) 2024-09-14T23:56:46.319 INFO:teuthology.orchestra.run.smithi042.stdout:788 write 0xdf15b7c thru 0xdf198e5 (0x3d6a bytes) 2024-09-14T23:56:46.322 INFO:teuthology.orchestra.run.smithi042.stdout:789 read 0x9065e38 thru 0x9070e20 (0xafe9 bytes) 2024-09-14T23:56:46.322 INFO:teuthology.orchestra.run.smithi042.stdout:790 read 0x2e0a264 thru 0x2e16b8e (0xc92b bytes) 2024-09-14T23:56:46.323 INFO:teuthology.orchestra.run.smithi042.stdout:791 punch from 0xb53d7cb to 0xb5405d7, (0x2e0c bytes) 2024-09-14T23:56:46.323 INFO:teuthology.orchestra.run.smithi042.stdout:794 writesame 0x7b4e87c thru 0x7b53ca9 data_size 0x19(0x542e bytes) 2024-09-14T23:56:46.329 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:46.329 INFO:teuthology.orchestra.run.smithi042.stdout:798 read 0x384f72d thru 0x3857659 (0x7f2d bytes) 2024-09-14T23:56:46.329 INFO:teuthology.orchestra.run.smithi042.stdout:800 read 0x5df55f5 thru 0x5df9457 (0x3e63 bytes) 2024-09-14T23:56:46.330 INFO:teuthology.orchestra.run.smithi042.stdout:801 writesame 0x757ae98 thru 0x758683c data_size 0x5(0xb9a5 bytes) 2024-09-14T23:56:46.335 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:46.335 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:46.336 INFO:teuthology.orchestra.run.smithi042.stdout:802 write 0x44e499f thru 0x44e72f3 (0x2955 bytes) 2024-09-14T23:56:46.341 INFO:teuthology.orchestra.run.smithi042.stdout:803 read 0xb7dd9e thru 0xb7f183 (0x13e6 bytes) 2024-09-14T23:56:46.341 INFO:teuthology.orchestra.run.smithi042.stdout:806 read 0x3c20530 thru 0x3c282e8 (0x7db9 bytes) 2024-09-14T23:56:46.342 INFO:teuthology.orchestra.run.smithi042.stdout:807 write 0x8cc690f thru 0x8ccdb77 (0x7269 bytes) 2024-09-14T23:56:46.345 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T23:56:46.346 INFO:teuthology.orchestra.run.smithi042.stdout:808 read 0x1ebf8f thru 0x1f1303 (0x5375 bytes) 2024-09-14T23:56:46.347 INFO:teuthology.orchestra.run.smithi042.stdout:810 writesame 0x8491be5 thru 0x84928a5 data_size 0x1(0xcc1 bytes) 2024-09-14T23:56:46.351 INFO:teuthology.orchestra.run.smithi042.stdout:812 write 0x98a89d thru 0x98f770 (0x4ed4 bytes) 2024-09-14T23:56:46.357 INFO:teuthology.orchestra.run.smithi042.stdout:813 write 0x58f1fdd thru 0x58fe41f (0xc443 bytes) 2024-09-14T23:56:46.362 INFO:teuthology.orchestra.run.smithi042.stdout:814 writesame 0x85296ac thru 0x852bdc7 data_size 0x2(0x271c bytes) 2024-09-14T23:56:46.368 INFO:teuthology.orchestra.run.smithi042.stdout:818 read 0x70ee88f thru 0x70f33ae (0x4b20 bytes) 2024-09-14T23:56:46.369 INFO:teuthology.orchestra.run.smithi042.stdout:819 read 0xc834509 thru 0xc836937 (0x242f bytes) 2024-09-14T23:56:46.369 INFO:teuthology.orchestra.run.smithi042.stdout:821 write 0xb103597 thru 0xb10a59a (0x7004 bytes) 2024-09-14T23:56:46.373 INFO:teuthology.orchestra.run.smithi042.stdout:822 writesame 0x1a3bcec thru 0x1a3c1c1 data_size 0x1(0x4d6 bytes) 2024-09-14T23:56:46.376 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:46.387 INFO:teuthology.orchestra.run.smithi042.stdout:827 trunc from 0xdf198e6 to 0x45fc141 2024-09-14T23:56:46.412 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:56:46.410+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:56:46.421 INFO:teuthology.orchestra.run.smithi042.stdout:828 punch from 0x1a8a4d6 to 0x1a97b6a, (0xd694 bytes) 2024-09-14T23:56:46.421 INFO:teuthology.orchestra.run.smithi042.stdout:829 read 0x1fa907b thru 0x1fae69d (0x5623 bytes) 2024-09-14T23:56:46.421 INFO:teuthology.orchestra.run.smithi042.stdout:831 read 0x3bf8e35 thru 0x3bfb911 (0x2add bytes) 2024-09-14T23:56:46.421 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:46.421 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:46.421 INFO:teuthology.orchestra.run.smithi042.stdout:834 read 0x2faaf26 thru 0x2fabe60 (0xf3b bytes) 2024-09-14T23:56:46.435 INFO:teuthology.orchestra.run.smithi042.stdout:835 write 0xc64f731 thru 0xc65efd9 (0xf8a9 bytes) 2024-09-14T23:56:46.438 INFO:teuthology.orchestra.run.smithi042.stdout:837 write 0x3328991 thru 0x332a3bf (0x1a2f bytes) 2024-09-14T23:56:46.442 INFO:teuthology.orchestra.run.smithi042.stdout:838 punch from 0x80ef6ab to 0x80fa4b9, (0xae0e bytes) 2024-09-14T23:56:46.442 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:46.442 INFO:teuthology.orchestra.run.smithi042.stdout:840 write 0x6e0ab16 thru 0x6e0abad (0x98 bytes) 2024-09-14T23:56:46.445 INFO:teuthology.orchestra.run.smithi042.stdout:truncating to largest ever: 0xed3a201 2024-09-14T23:56:46.447 INFO:teuthology.orchestra.run.smithi042.stdout:841 trunc from 0xc65efda to 0xed3a201 2024-09-14T23:56:46.453 INFO:teuthology.orchestra.run.smithi042.stdout:842 write 0x8bac506 thru 0x8bb8ffc (0xcaf7 bytes) 2024-09-14T23:56:46.457 INFO:teuthology.orchestra.run.smithi042.stdout:843 read 0x144e9b0 thru 0x145da28 (0xf079 bytes) 2024-09-14T23:56:46.458 INFO:teuthology.orchestra.run.smithi042.stdout:844 read 0x5c712da thru 0x5c75db5 (0x4adc bytes) 2024-09-14T23:56:46.458 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:46.458 INFO:teuthology.orchestra.run.smithi042.stdout:846 write 0x33343e1 thru 0x3338797 (0x43b7 bytes) 2024-09-14T23:56:46.459 INFO:teuthology.orchestra.run.smithi042.stdout:847 read 0xe72f759 thru 0xe73267d (0x2f25 bytes) 2024-09-14T23:56:46.459 INFO:teuthology.orchestra.run.smithi042.stdout:848 write 0xaf97ab0 thru 0xaf986cd (0xc1e bytes) 2024-09-14T23:56:46.462 INFO:teuthology.orchestra.run.smithi042.stdout:849 write 0xea8d176 thru 0xea9be29 (0xecb4 bytes) 2024-09-14T23:56:46.466 INFO:teuthology.orchestra.run.smithi042.stdout:852 write 0xf611b8 thru 0xf6dc69 (0xcab2 bytes) 2024-09-14T23:56:46.468 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:46.468 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:46.471 INFO:teuthology.orchestra.run.smithi042.stdout:853 read 0x3049ea thru 0x30dbd1 (0x91e8 bytes) 2024-09-14T23:56:46.475 INFO:teuthology.orchestra.run.smithi042.stdout:854 trunc from 0xed3a201 to 0xb22c96e 2024-09-14T23:56:46.480 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T23:56:46.487 INFO:teuthology.orchestra.run.smithi042.stdout:855 write 0x318d8ae thru 0x31959bd (0x8110 bytes) 2024-09-14T23:56:46.489 INFO:teuthology.orchestra.run.smithi042.stdout:856 read 0x24efae1 thru 0x24fa54a (0xaa6a bytes) 2024-09-14T23:56:46.489 INFO:teuthology.orchestra.run.smithi042.stdout:857 punch from 0xb959e to 0xbcc46, (0x36a8 bytes) 2024-09-14T23:56:46.489 INFO:teuthology.orchestra.run.smithi042.stdout:859 read 0x44f1182 thru 0x44f6075 (0x4ef4 bytes) 2024-09-14T23:56:46.489 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:46.489 INFO:teuthology.orchestra.run.smithi042.stdout:861 writesame 0xa0a1822 thru 0xa0aaacf data_size 0x19(0x92ae bytes) 2024-09-14T23:56:46.492 INFO:teuthology.orchestra.run.smithi042.stdout:862 read 0xa8bda4d thru 0xa8c2c49 (0x51fd bytes) 2024-09-14T23:56:46.492 INFO:teuthology.orchestra.run.smithi042.stdout:864 punch from 0x4c8f574 to 0x4c9a120, (0xabac bytes) 2024-09-14T23:56:46.493 INFO:teuthology.orchestra.run.smithi042.stdout:865 write 0xa783d17 thru 0xa7863f8 (0x26e2 bytes) 2024-09-14T23:56:46.495 INFO:teuthology.orchestra.run.smithi042.stdout:867 write 0x2738dcd thru 0x2746512 (0xd746 bytes) 2024-09-14T23:56:46.506 INFO:teuthology.orchestra.run.smithi042.stdout:868 writesame 0xbd94eb7 thru 0xbd9f146 data_size 0x6(0xa290 bytes) 2024-09-14T23:56:46.507 INFO:teuthology.orchestra.run.smithi042.stdout:869 trunc from 0xbd9f147 to 0xa88d719 2024-09-14T23:56:46.513 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:56:46.511+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:56:46.518 INFO:teuthology.orchestra.run.smithi042.stdout:870 punch from 0x7e4d3ce to 0x7e4fe11, (0x2a43 bytes) 2024-09-14T23:56:46.518 INFO:teuthology.orchestra.run.smithi042.stdout:873 write 0xc0c716 thru 0xc0f9c4 (0x32af bytes) 2024-09-14T23:56:46.519 INFO:teuthology.orchestra.run.smithi042.stdout:875 punch from 0x3aaabd8 to 0x3aaf2ef, (0x4717 bytes) 2024-09-14T23:56:46.519 INFO:teuthology.orchestra.run.smithi042.stdout:877 read 0x6659919 thru 0x665e921 (0x5009 bytes) 2024-09-14T23:56:46.521 INFO:teuthology.orchestra.run.smithi042.stdout:878 trunc from 0xa88d719 to 0xbb733a0 2024-09-14T23:56:46.525 INFO:teuthology.orchestra.run.smithi042.stdout:880 read 0x8442aaa thru 0x8451d67 (0xf2be bytes) 2024-09-14T23:56:46.526 INFO:teuthology.orchestra.run.smithi042.stdout:881 read 0x161b0b5 thru 0x161d6d8 (0x2624 bytes) 2024-09-14T23:56:46.526 INFO:teuthology.orchestra.run.smithi042.stdout:882 read 0x5044b73 thru 0x5049705 (0x4b93 bytes) 2024-09-14T23:56:46.526 INFO:teuthology.orchestra.run.smithi042.stdout:883 writesame 0x4f85cab thru 0x4f93edc data_size 0x9(0xe232 bytes) 2024-09-14T23:56:46.532 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:46.539 INFO:teuthology.orchestra.run.smithi042.stdout:885 writesame 0xde9975c thru 0xdea325b data_size 0x50(0x9b00 bytes) 2024-09-14T23:56:46.546 INFO:teuthology.orchestra.run.smithi042.stdout:886 trunc from 0xdea325c to 0x801d2b4 2024-09-14T23:56:46.565 INFO:teuthology.orchestra.run.smithi042.stdout:888 writesame 0xb2f2a9e thru 0xb300d3f data_size 0x1(0xe2a2 bytes) 2024-09-14T23:56:46.565 INFO:teuthology.orchestra.run.smithi042.stdout:889 write 0x3fe55fa thru 0x3fe8d98 (0x379f bytes) 2024-09-14T23:56:46.568 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:46.577 INFO:teuthology.orchestra.run.smithi042.stdout:893 write 0xe9ee993 thru 0xe9f6825 (0x7e93 bytes) 2024-09-14T23:56:46.581 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:46.581 INFO:teuthology.orchestra.run.smithi042.stdout:895 writesame 0x966ca3c thru 0x9670b20 data_size 0x1(0x40e5 bytes) 2024-09-14T23:56:46.591 INFO:teuthology.orchestra.run.smithi042.stdout:896 trunc from 0xe9f6826 to 0xcd23125 2024-09-14T23:56:46.605 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:46.605 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:46.611 INFO:teuthology.orchestra.run.smithi042.stdout:898 trunc from 0xcd23125 to 0x2d55b85 2024-09-14T23:56:46.613 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:56:46.612+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:56:46.619 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T23:56:46.645 INFO:teuthology.orchestra.run.smithi042.stdout:899 trunc from 0x2d55b85 to 0xba538d4 2024-09-14T23:56:46.650 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:46.650 INFO:teuthology.orchestra.run.smithi042.stdout:901 write 0x78b1a49 thru 0x78c190d (0xfec5 bytes) 2024-09-14T23:56:46.654 INFO:teuthology.orchestra.run.smithi042.stdout:902 read 0xa3de6d3 thru 0xa3e56b2 (0x6fe0 bytes) 2024-09-14T23:56:46.654 INFO:teuthology.orchestra.run.smithi042.stdout:904 read 0x706c159 thru 0x707b215 (0xf0bd bytes) 2024-09-14T23:56:46.654 INFO:teuthology.orchestra.run.smithi042.stdout:906 punch from 0xa30acbc to 0xa3191ae, (0xe4f2 bytes) 2024-09-14T23:56:46.659 INFO:teuthology.orchestra.run.smithi042.stdout:907 trunc from 0xba538d4 to 0x73d0f94 2024-09-14T23:56:46.668 INFO:teuthology.orchestra.run.smithi042.stdout:908 read 0x6fdadc1 thru 0x6fdb3da (0x61a bytes) 2024-09-14T23:56:46.668 INFO:teuthology.orchestra.run.smithi042.stdout:909 read 0x59bebe3 thru 0x59c8f48 (0xa366 bytes) 2024-09-14T23:56:46.671 INFO:teuthology.orchestra.run.smithi042.stdout:910 trunc from 0x73d0f94 to 0x9e74f9c 2024-09-14T23:56:46.676 INFO:teuthology.orchestra.run.smithi042.stdout:911 write 0x217505e thru 0x2184962 (0xf905 bytes) 2024-09-14T23:56:46.690 INFO:teuthology.orchestra.run.smithi042.stdout:912 write 0xda0fa3d thru 0xda1c972 (0xcf36 bytes) 2024-09-14T23:56:46.693 INFO:teuthology.orchestra.run.smithi042.stdout:913 punch from 0x2e2d04f to 0x2e39316, (0xc2c7 bytes) 2024-09-14T23:56:46.693 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:46.693 INFO:teuthology.orchestra.run.smithi042.stdout:916 read 0x7f09708 thru 0x7f0ce2d (0x3726 bytes) 2024-09-14T23:56:46.694 INFO:teuthology.orchestra.run.smithi042.stdout:918 write 0x9565c2d thru 0x956b2d6 (0x56aa bytes) 2024-09-14T23:56:46.702 INFO:teuthology.orchestra.run.smithi042.stdout:919 trunc from 0xda1c973 to 0x8a14be2 2024-09-14T23:56:46.714 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:56:46.713+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:56:46.720 INFO:teuthology.orchestra.run.smithi042.stdout:920 write 0xb101ddb thru 0xb103777 (0x199d bytes) 2024-09-14T23:56:46.722 INFO:teuthology.orchestra.run.smithi042.stdout:921 write 0x713bcf8 thru 0x714064f (0x4958 bytes) 2024-09-14T23:56:46.726 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:46.726 INFO:teuthology.orchestra.run.smithi042.stdout:925 clone 7 order 24 su 65536 sc 15 2024-09-14T23:56:46.760 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:46.760 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:46.769 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T23:56:46.852 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:56:46.851+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:56:46.916 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:46.916 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:46.916 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:56:46.914+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:56:46.924 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T23:56:46.941 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T23:56:47.017 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:56:47.015+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:56:47.074 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone6 from 0xb103778 (overlap 0x2d55b85) to 0xd99b74 2024-09-14T23:56:47.088 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:47.089 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:47.101 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T23:56:47.113 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:56:47.113 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:56:47.117 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:56:47.116+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:56:47.131 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:56:47.131 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:56:47.149 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:56:47.150 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:56:47.168 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:56:47.169 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:56:47.170 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-09-14T23:56:47.172 INFO:teuthology.orchestra.run.smithi042.stdout:926 writesame 0x8af94cf thru 0x8afb439 data_size 0x7(0x1f6b bytes) 2024-09-14T23:56:47.186 INFO:teuthology.orchestra.run.smithi042.stdout:927 write 0x541ba88 thru 0x54288a4 (0xce1d bytes) 2024-09-14T23:56:47.192 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:56:47.193 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:56:47.203 INFO:teuthology.orchestra.run.smithi042.stdout:928 clone 8 order 18 su 4096 sc 15 2024-09-14T23:56:47.211 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:56:47.211 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:56:47.218 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:56:47.217+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:56:47.228 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:56:47.228 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:56:47.231 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:56:47.232 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:56:47.232 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:56:47.232 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:56:47.241 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T23:56:47.246 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:56:47.246 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:56:47.319 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:56:47.318+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:56:47.353 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:56:47.353 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:56:47.353 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:56:47.353 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:56:47.353 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:56:47.361 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T23:56:47.420 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:56:47.419+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:56:47.511 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:56:47.511 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:56:47.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:56:47.511 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:56:47.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.511 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5180 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3883008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:47.511 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.250058+0000", 2024-09-14T23:56:47.511 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.2588545440000001, 2024-09-14T23:56:47.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057351471000000001, 2024-09-14T23:56:47.512 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:47.512 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:47.512 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:47.512 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:47.512 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:47.512 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:47.512 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5180 2024-09-14T23:56:47.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.512 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:47.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.512 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:47.513 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250058+0000", 2024-09-14T23:56:47.513 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:47.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.513 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:47.513 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250058+0000", 2024-09-14T23:56:47.513 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:47.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:47.514 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250786+0000", 2024-09-14T23:56:47.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00072813399999999996 2024-09-14T23:56:47.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:47.514 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250789+0000", 2024-09-14T23:56:47.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.647e-06 2024-09-14T23:56:47.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:47.514 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250798+0000", 2024-09-14T23:56:47.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0000000000000002e-06 2024-09-14T23:56:47.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.515 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:47.515 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250803+0000", 2024-09-14T23:56:47.515 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.301e-06 2024-09-14T23:56:47.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.515 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:47.515 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.307234+0000", 2024-09-14T23:56:47.515 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056431165999999998 2024-09-14T23:56:47.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.515 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:47.515 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.307321+0000", 2024-09-14T23:56:47.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6831999999999998e-05 2024-09-14T23:56:47.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.516 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:47.516 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.307409+0000", 2024-09-14T23:56:47.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8391000000000001e-05 2024-09-14T23:56:47.516 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:47.516 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:47.516 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:47.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.516 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5184 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 4128768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:47.516 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.250851+0000", 2024-09-14T23:56:47.516 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.2580610730000004, 2024-09-14T23:56:47.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057319038000000003, 2024-09-14T23:56:47.517 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:47.517 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:47.517 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:47.517 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:47.517 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:47.517 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:47.517 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5184 2024-09-14T23:56:47.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.517 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:47.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.517 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:47.517 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250851+0000", 2024-09-14T23:56:47.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:47.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.518 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:47.518 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250851+0000", 2024-09-14T23:56:47.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:47.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.518 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:47.518 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.251900+0000", 2024-09-14T23:56:47.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001048577 2024-09-14T23:56:47.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.518 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:47.519 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.251904+0000", 2024-09-14T23:56:47.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9580000000000001e-06 2024-09-14T23:56:47.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.519 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:47.519 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.251914+0000", 2024-09-14T23:56:47.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0186999999999999e-05 2024-09-14T23:56:47.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.519 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:47.519 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.251927+0000", 2024-09-14T23:56:47.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3593e-05 2024-09-14T23:56:47.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.519 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:47.520 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.308020+0000", 2024-09-14T23:56:47.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056092867999999997 2024-09-14T23:56:47.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.520 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:47.520 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.308094+0000", 2024-09-14T23:56:47.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3533000000000002e-05 2024-09-14T23:56:47.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.520 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:47.520 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.308170+0000", 2024-09-14T23:56:47.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6322000000000006e-05 2024-09-14T23:56:47.520 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:47.520 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:47.521 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:47.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.521 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5196 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 1925120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:47.521 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.252157+0000", 2024-09-14T23:56:47.521 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.2567549590000002, 2024-09-14T23:56:47.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057285801999999997, 2024-09-14T23:56:47.521 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:47.521 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:47.521 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:47.521 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:47.521 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:47.521 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:47.522 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5196 2024-09-14T23:56:47.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.522 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:47.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:47.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.252157+0000", 2024-09-14T23:56:47.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:47.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:47.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.252157+0000", 2024-09-14T23:56:47.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:47.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:47.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253243+0000", 2024-09-14T23:56:47.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010861340000000001 2024-09-14T23:56:47.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:47.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253248+0000", 2024-09-14T23:56:47.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8799999999999999e-06 2024-09-14T23:56:47.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:47.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253271+0000", 2024-09-14T23:56:47.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2836e-05 2024-09-14T23:56:47.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:47.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253281+0000", 2024-09-14T23:56:47.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6520000000000001e-06 2024-09-14T23:56:47.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:47.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.309336+0000", 2024-09-14T23:56:47.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056054983000000003 2024-09-14T23:56:47.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:47.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.309389+0000", 2024-09-14T23:56:47.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3298999999999999e-05 2024-09-14T23:56:47.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:47.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.309443+0000", 2024-09-14T23:56:47.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4017999999999997e-05 2024-09-14T23:56:47.525 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:47.525 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:47.525 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:47.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.525 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5200 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2170880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:47.525 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253394+0000", 2024-09-14T23:56:47.525 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.2555181819999994, 2024-09-14T23:56:47.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057428154000000002, 2024-09-14T23:56:47.525 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:47.525 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:47.526 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:47.526 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:47.526 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:47.526 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:47.526 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5200 2024-09-14T23:56:47.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.526 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:47.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.526 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:47.526 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253394+0000", 2024-09-14T23:56:47.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:47.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.526 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:47.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253394+0000", 2024-09-14T23:56:47.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:47.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:47.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253395+0000", 2024-09-14T23:56:47.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9100000000000004e-07 2024-09-14T23:56:47.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:47.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253397+0000", 2024-09-14T23:56:47.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2790000000000001e-06 2024-09-14T23:56:47.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:47.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253404+0000", 2024-09-14T23:56:47.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7959999999999998e-06 2024-09-14T23:56:47.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:47.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253409+0000", 2024-09-14T23:56:47.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1460000000000004e-06 2024-09-14T23:56:47.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:47.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310627+0000", 2024-09-14T23:56:47.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057218160999999997 2024-09-14T23:56:47.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:47.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310710+0000", 2024-09-14T23:56:47.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3102999999999994e-05 2024-09-14T23:56:47.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:47.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310822+0000", 2024-09-14T23:56:47.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000112178 2024-09-14T23:56:47.529 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:47.529 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:47.529 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:47.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.530 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5201 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2232320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:47.530 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:56:47.530 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.2554935100000009, 2024-09-14T23:56:47.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057643228999999997, 2024-09-14T23:56:47.530 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:47.530 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:47.530 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:47.530 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:47.530 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:47.530 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:47.530 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5201 2024-09-14T23:56:47.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.531 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:47.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:47.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:56:47.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:47.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:47.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:56:47.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:47.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:47.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:56:47.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0699999999999998e-07 2024-09-14T23:56:47.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:47.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253422+0000", 2024-09-14T23:56:47.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.514e-06 2024-09-14T23:56:47.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:47.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253431+0000", 2024-09-14T23:56:47.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1919999999999994e-06 2024-09-14T23:56:47.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:47.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253436+0000", 2024-09-14T23:56:47.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5350000000000004e-06 2024-09-14T23:56:47.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:47.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310856+0000", 2024-09-14T23:56:47.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057419381999999998 2024-09-14T23:56:47.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:47.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310953+0000", 2024-09-14T23:56:47.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6774000000000006e-05 2024-09-14T23:56:47.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:47.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311062+0000", 2024-09-14T23:56:47.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000109425 2024-09-14T23:56:47.534 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:47.534 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:47.534 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:47.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.534 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5202 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2293760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:47.534 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:56:47.534 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.2554687619999996, 2024-09-14T23:56:47.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057818255999999998, 2024-09-14T23:56:47.534 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:47.535 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:47.535 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:47.535 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:47.535 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:47.535 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:47.535 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5202 2024-09-14T23:56:47.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.535 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:47.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:47.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:56:47.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:47.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:47.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:56:47.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:47.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:47.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253573+0000", 2024-09-14T23:56:47.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012934000000000001 2024-09-14T23:56:47.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:47.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253576+0000", 2024-09-14T23:56:47.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5310000000000002e-06 2024-09-14T23:56:47.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:47.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253587+0000", 2024-09-14T23:56:47.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0522e-05 2024-09-14T23:56:47.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:47.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253595+0000", 2024-09-14T23:56:47.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8290000000000008e-06 2024-09-14T23:56:47.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:47.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311093+0000", 2024-09-14T23:56:47.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057498312000000003 2024-09-14T23:56:47.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:47.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311167+0000", 2024-09-14T23:56:47.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3582000000000001e-05 2024-09-14T23:56:47.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:47.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311262+0000", 2024-09-14T23:56:47.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5140000000000001e-05 2024-09-14T23:56:47.541 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:47.541 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:47.541 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:47.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.542 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5203 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2355200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:47.542 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:56:47.542 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.2553060200000008, 2024-09-14T23:56:47.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057903938000000002, 2024-09-14T23:56:47.542 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:47.542 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:47.542 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:47.543 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:47.543 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:47.543 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:47.543 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5203 2024-09-14T23:56:47.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.543 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:47.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:47.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:56:47.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:47.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:47.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:56:47.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:47.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:47.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253902+0000", 2024-09-14T23:56:47.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029615400000000002 2024-09-14T23:56:47.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:47.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253906+0000", 2024-09-14T23:56:47.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9249999999999997e-06 2024-09-14T23:56:47.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:47.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253913+0000", 2024-09-14T23:56:47.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9099999999999999e-06 2024-09-14T23:56:47.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:47.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253917+0000", 2024-09-14T23:56:47.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9210000000000002e-06 2024-09-14T23:56:47.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:47.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311295+0000", 2024-09-14T23:56:47.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057378268000000003 2024-09-14T23:56:47.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:47.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311402+0000", 2024-09-14T23:56:47.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010644100000000001 2024-09-14T23:56:47.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:47.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311510+0000", 2024-09-14T23:56:47.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108319 2024-09-14T23:56:47.547 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:47.547 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:47.547 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:47.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.547 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5204 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2416640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:47.547 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:56:47.547 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.2549878230000004, 2024-09-14T23:56:47.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057774860999999997, 2024-09-14T23:56:47.547 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:47.547 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:47.547 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:47.547 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:47.547 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:47.548 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:47.548 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5204 2024-09-14T23:56:47.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.548 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:47.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:47.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:56:47.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:47.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:47.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:56:47.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:47.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:47.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255903+0000", 2024-09-14T23:56:47.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001978807 2024-09-14T23:56:47.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:47.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255907+0000", 2024-09-14T23:56:47.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3610000000000001e-06 2024-09-14T23:56:47.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:47.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255914+0000", 2024-09-14T23:56:47.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.357e-06 2024-09-14T23:56:47.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:47.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255918+0000", 2024-09-14T23:56:47.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1779999999999996e-06 2024-09-14T23:56:47.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:47.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311541+0000", 2024-09-14T23:56:47.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055622697999999998 2024-09-14T23:56:47.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:47.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311614+0000", 2024-09-14T23:56:47.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3102999999999994e-05 2024-09-14T23:56:47.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:47.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311699+0000", 2024-09-14T23:56:47.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5357000000000003e-05 2024-09-14T23:56:47.552 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:47.552 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:47.552 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:47.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.552 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5210 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2785280~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:47.552 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256011+0000", 2024-09-14T23:56:47.552 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.2529011860000008, 2024-09-14T23:56:47.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057273662000000003, 2024-09-14T23:56:47.552 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:47.552 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:47.552 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:47.553 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:47.553 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:47.553 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:47.553 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5210 2024-09-14T23:56:47.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.553 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:47.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:47.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256011+0000", 2024-09-14T23:56:47.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:47.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:47.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256011+0000", 2024-09-14T23:56:47.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:47.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:47.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256011+0000", 2024-09-14T23:56:47.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4999999999999999e-07 2024-09-14T23:56:47.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:47.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256013+0000", 2024-09-14T23:56:47.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3969999999999999e-06 2024-09-14T23:56:47.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:47.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256017+0000", 2024-09-14T23:56:47.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1409999999999998e-06 2024-09-14T23:56:47.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:47.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256019+0000", 2024-09-14T23:56:47.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6280000000000001e-06 2024-09-14T23:56:47.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:47.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313116+0000", 2024-09-14T23:56:47.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057096457000000003 2024-09-14T23:56:47.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:47.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313193+0000", 2024-09-14T23:56:47.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7439000000000002e-05 2024-09-14T23:56:47.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:47.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313285+0000", 2024-09-14T23:56:47.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1349999999999998e-05 2024-09-14T23:56:47.556 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:47.556 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:47.556 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:47.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.556 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5211 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2846720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:47.557 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:56:47.557 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.2528896090000003, 2024-09-14T23:56:47.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057528151, 2024-09-14T23:56:47.557 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:47.557 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:47.557 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:47.557 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:47.557 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:47.557 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:47.557 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5211 2024-09-14T23:56:47.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.557 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:47.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:47.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:56:47.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:47.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:47.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:56:47.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:47.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:47.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:56:47.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1399999999999998e-07 2024-09-14T23:56:47.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:47.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256024+0000", 2024-09-14T23:56:47.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.114e-06 2024-09-14T23:56:47.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:47.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256028+0000", 2024-09-14T23:56:47.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0840000000000002e-06 2024-09-14T23:56:47.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:47.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256031+0000", 2024-09-14T23:56:47.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6819999999999999e-06 2024-09-14T23:56:47.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:47.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313317+0000", 2024-09-14T23:56:47.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057286295000000001 2024-09-14T23:56:47.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:47.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313403+0000", 2024-09-14T23:56:47.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6287999999999998e-05 2024-09-14T23:56:47.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:47.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313551+0000", 2024-09-14T23:56:47.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000147374 2024-09-14T23:56:47.561 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:47.561 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:47.561 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:47.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.561 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5212 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2908160~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:47.561 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:56:47.561 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.2528784149999996, 2024-09-14T23:56:47.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057735679999999998, 2024-09-14T23:56:47.562 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:47.562 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:47.562 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:47.562 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:47.562 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:47.562 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:47.562 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5212 2024-09-14T23:56:47.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.562 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:47.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:47.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:56:47.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:47.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:47.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:56:47.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:47.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:47.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:56:47.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0900000000000001e-07 2024-09-14T23:56:47.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:47.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256035+0000", 2024-09-14T23:56:47.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1200000000000001e-06 2024-09-14T23:56:47.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:47.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256038+0000", 2024-09-14T23:56:47.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.05e-06 2024-09-14T23:56:47.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:47.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256041+0000", 2024-09-14T23:56:47.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.667e-06 2024-09-14T23:56:47.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:47.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313584+0000", 2024-09-14T23:56:47.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057543484999999998 2024-09-14T23:56:47.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:47.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313668+0000", 2024-09-14T23:56:47.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3807000000000006e-05 2024-09-14T23:56:47.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:47.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313770+0000", 2024-09-14T23:56:47.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010134200000000001 2024-09-14T23:56:47.565 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:47.565 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:47.566 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:47.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.566 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5213 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2969600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:47.566 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:56:47.566 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.2528683279999999, 2024-09-14T23:56:47.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058145467999999999, 2024-09-14T23:56:47.566 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:47.566 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:47.566 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:47.566 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:47.567 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:47.567 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:47.567 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5213 2024-09-14T23:56:47.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.567 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:47.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:47.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:56:47.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:47.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:47.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:56:47.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:47.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:47.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257901+0000", 2024-09-14T23:56:47.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018567519999999999 2024-09-14T23:56:47.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:47.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257904+0000", 2024-09-14T23:56:47.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4850000000000001e-06 2024-09-14T23:56:47.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:47.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257912+0000", 2024-09-14T23:56:47.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2090000000000005e-06 2024-09-14T23:56:47.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:47.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257918+0000", 2024-09-14T23:56:47.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6899999999999997e-06 2024-09-14T23:56:47.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:47.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313816+0000", 2024-09-14T23:56:47.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055898075999999998 2024-09-14T23:56:47.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:47.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.314082+0000", 2024-09-14T23:56:47.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000265386 2024-09-14T23:56:47.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:47.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.314189+0000", 2024-09-14T23:56:47.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010787 2024-09-14T23:56:47.570 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:47.570 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:47.570 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:47.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.570 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5217 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3215360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:47.571 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257961+0000", 2024-09-14T23:56:47.571 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.2509513489999993, 2024-09-14T23:56:47.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057320969999999999, 2024-09-14T23:56:47.571 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:47.571 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:47.571 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:47.571 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:47.571 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:47.571 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:47.571 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5217 2024-09-14T23:56:47.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.571 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:47.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:47.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257961+0000", 2024-09-14T23:56:47.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:47.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:47.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257961+0000", 2024-09-14T23:56:47.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:47.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:47.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257961+0000", 2024-09-14T23:56:47.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.22e-07 2024-09-14T23:56:47.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:47.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257962+0000", 2024-09-14T23:56:47.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1400000000000001e-06 2024-09-14T23:56:47.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:47.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257965+0000", 2024-09-14T23:56:47.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0929999999999999e-06 2024-09-14T23:56:47.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:47.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257968+0000", 2024-09-14T23:56:47.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5749999999999999e-06 2024-09-14T23:56:47.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:47.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315047+0000", 2024-09-14T23:56:47.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057078730000000001 2024-09-14T23:56:47.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:47.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315087+0000", 2024-09-14T23:56:47.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0806999999999997e-05 2024-09-14T23:56:47.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:47.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315282+0000", 2024-09-14T23:56:47.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000194403 2024-09-14T23:56:47.576 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:47.576 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:47.577 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:47.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.577 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5218 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3276800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:47.577 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:56:47.577 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.2509413190000007, 2024-09-14T23:56:47.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057497557999999997, 2024-09-14T23:56:47.577 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:47.577 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:47.578 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:47.578 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:47.578 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:47.578 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:47.578 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5218 2024-09-14T23:56:47.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.579 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:47.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:47.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:56:47.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:47.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:47.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:56:47.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:47.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:47.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:56:47.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1899999999999999e-07 2024-09-14T23:56:47.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:47.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257972+0000", 2024-09-14T23:56:47.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.217e-06 2024-09-14T23:56:47.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:47.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257975+0000", 2024-09-14T23:56:47.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9359999999999999e-06 2024-09-14T23:56:47.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:47.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257978+0000", 2024-09-14T23:56:47.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1049999999999999e-06 2024-09-14T23:56:47.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:47.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315331+0000", 2024-09-14T23:56:47.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057352093999999999 2024-09-14T23:56:47.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:47.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315384+0000", 2024-09-14T23:56:47.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3780999999999999e-05 2024-09-14T23:56:47.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:47.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315468+0000", 2024-09-14T23:56:47.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4206e-05 2024-09-14T23:56:47.583 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:47.583 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:47.583 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:47.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.583 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5219 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3338240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:47.583 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:56:47.583 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.2509306430000002, 2024-09-14T23:56:47.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057637764000000001, 2024-09-14T23:56:47.583 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:47.583 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:47.583 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:47.583 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:47.583 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:47.583 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:47.584 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5219 2024-09-14T23:56:47.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.584 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:47.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:47.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:56:47.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:47.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:47.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:56:47.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:47.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:47.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:56:47.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.03e-07 2024-09-14T23:56:47.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:47.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257983+0000", 2024-09-14T23:56:47.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9199999999999999e-07 2024-09-14T23:56:47.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:47.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257986+0000", 2024-09-14T23:56:47.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0860000000000002e-06 2024-09-14T23:56:47.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:47.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257989+0000", 2024-09-14T23:56:47.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6589999999999999e-06 2024-09-14T23:56:47.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:47.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315509+0000", 2024-09-14T23:56:47.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057520262000000003 2024-09-14T23:56:47.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:47.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315554+0000", 2024-09-14T23:56:47.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5015999999999997e-05 2024-09-14T23:56:47.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:47.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315619+0000", 2024-09-14T23:56:47.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5545999999999999e-05 2024-09-14T23:56:47.589 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:47.589 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:47.589 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:47.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.590 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5220 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3399680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:47.590 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:56:47.590 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.2509206870000007, 2024-09-14T23:56:47.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057792128999999998, 2024-09-14T23:56:47.590 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:47.590 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:47.590 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:47.590 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:47.590 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:47.590 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:47.590 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5220 2024-09-14T23:56:47.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.590 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:47.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:47.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:56:47.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:47.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:47.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:56:47.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:47.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:47.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:56:47.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1799999999999999e-07 2024-09-14T23:56:47.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:47.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257993+0000", 2024-09-14T23:56:47.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6200000000000006e-07 2024-09-14T23:56:47.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:47.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257996+0000", 2024-09-14T23:56:47.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9050000000000001e-06 2024-09-14T23:56:47.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:47.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257998+0000", 2024-09-14T23:56:47.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5950000000000001e-06 2024-09-14T23:56:47.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:47.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315672+0000", 2024-09-14T23:56:47.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057673637999999999 2024-09-14T23:56:47.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:47.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315734+0000", 2024-09-14T23:56:47.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2216000000000002e-05 2024-09-14T23:56:47.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:47.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315784+0000", 2024-09-14T23:56:47.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9595000000000002e-05 2024-09-14T23:56:47.594 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:47.594 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:47.594 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:47.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.594 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5221 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3461120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:47.594 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:56:47.594 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.2509109689999995, 2024-09-14T23:56:47.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058003557999999997, 2024-09-14T23:56:47.595 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:47.595 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:47.595 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:47.595 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:47.595 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:47.595 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:47.595 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5221 2024-09-14T23:56:47.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.595 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:47.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:47.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:56:47.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:47.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:47.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:56:47.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:47.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:47.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259905+0000", 2024-09-14T23:56:47.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019035759999999999 2024-09-14T23:56:47.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:47.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259915+0000", 2024-09-14T23:56:47.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0518e-05 2024-09-14T23:56:47.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:47.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259925+0000", 2024-09-14T23:56:47.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7180000000000008e-06 2024-09-14T23:56:47.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:47.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259930+0000", 2024-09-14T23:56:47.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4769999999999997e-06 2024-09-14T23:56:47.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:47.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315820+0000", 2024-09-14T23:56:47.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055889960000000002 2024-09-14T23:56:47.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:47.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315914+0000", 2024-09-14T23:56:47.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4625999999999998e-05 2024-09-14T23:56:47.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:47.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.316005+0000", 2024-09-14T23:56:47.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0682999999999999e-05 2024-09-14T23:56:47.599 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:47.599 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:47.599 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:47.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.599 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5230 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 4014080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:47.599 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.260060+0000", 2024-09-14T23:56:47.599 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.2488521309999996, 2024-09-14T23:56:47.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057353556999999999, 2024-09-14T23:56:47.599 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:47.599 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:47.600 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:47.600 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:47.600 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:47.600 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:47.600 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5230 2024-09-14T23:56:47.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.600 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:47.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:47.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260060+0000", 2024-09-14T23:56:47.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:47.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:47.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260060+0000", 2024-09-14T23:56:47.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:47.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:47.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260060+0000", 2024-09-14T23:56:47.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.17e-07 2024-09-14T23:56:47.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:47.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260061+0000", 2024-09-14T23:56:47.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0300000000000001e-06 2024-09-14T23:56:47.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:47.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260064+0000", 2024-09-14T23:56:47.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9119999999999998e-06 2024-09-14T23:56:47.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:47.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260067+0000", 2024-09-14T23:56:47.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.678e-06 2024-09-14T23:56:47.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:47.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317269+0000", 2024-09-14T23:56:47.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057201771999999998 2024-09-14T23:56:47.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:47.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317342+0000", 2024-09-14T23:56:47.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2842000000000005e-05 2024-09-14T23:56:47.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:47.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317414+0000", 2024-09-14T23:56:47.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2106000000000004e-05 2024-09-14T23:56:47.604 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:47.604 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:47.604 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:47.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.604 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5231 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 4075520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:47.604 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:56:47.604 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.2488422349999997, 2024-09-14T23:56:47.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057524733000000002, 2024-09-14T23:56:47.604 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:47.604 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:47.604 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:47.604 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:47.604 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:47.605 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:47.605 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5231 2024-09-14T23:56:47.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.605 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:47.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:47.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:56:47.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:47.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:47.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:56:47.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:47.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:47.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261904+0000", 2024-09-14T23:56:47.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018338790000000001 2024-09-14T23:56:47.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:47.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261910+0000", 2024-09-14T23:56:47.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9689999999999999e-06 2024-09-14T23:56:47.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:47.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261923+0000", 2024-09-14T23:56:47.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3061000000000001e-05 2024-09-14T23:56:47.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:47.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261931+0000", 2024-09-14T23:56:47.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3159999999999997e-06 2024-09-14T23:56:47.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:47.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317456+0000", 2024-09-14T23:56:47.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055525219000000001 2024-09-14T23:56:47.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:47.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317520+0000", 2024-09-14T23:56:47.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4010000000000005e-05 2024-09-14T23:56:47.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:47.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317595+0000", 2024-09-14T23:56:47.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4278999999999998e-05 2024-09-14T23:56:47.608 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:47.608 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:47.608 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:47.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.608 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5236 3.8 3:1071668d:::rbd_data.1118de1168b.0000000000000092:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:47.608 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:56:47.609 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.2468287710000006, 2024-09-14T23:56:47.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057499462000000001, 2024-09-14T23:56:47.609 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:47.609 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:47.609 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:47.609 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:47.609 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:47.609 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:47.609 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5236 2024-09-14T23:56:47.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.609 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:47.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:47.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:56:47.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:47.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:47.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:56:47.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:47.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:47.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262084+0000", 2024-09-14T23:56:47.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3399999999999999e-07 2024-09-14T23:56:47.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:47.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262087+0000", 2024-09-14T23:56:47.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2310000000000001e-06 2024-09-14T23:56:47.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:47.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262095+0000", 2024-09-14T23:56:47.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3799999999999996e-06 2024-09-14T23:56:47.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:47.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262101+0000", 2024-09-14T23:56:47.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0009999999999998e-06 2024-09-14T23:56:47.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:47.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317625+0000", 2024-09-14T23:56:47.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055524111000000001 2024-09-14T23:56:47.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:47.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317702+0000", 2024-09-14T23:56:47.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7533999999999993e-05 2024-09-14T23:56:47.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-14T23:56:47.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317892+0000", 2024-09-14T23:56:47.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018991400000000001 2024-09-14T23:56:47.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:56:47.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319407+0000", 2024-09-14T23:56:47.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015145250000000001 2024-09-14T23:56:47.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:47.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319481+0000", 2024-09-14T23:56:47.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3826999999999996e-05 2024-09-14T23:56:47.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:47.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319541+0000", 2024-09-14T23:56:47.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0470999999999998e-05 2024-09-14T23:56:47.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:47.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:47.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:47.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319583+0000", 2024-09-14T23:56:47.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1934000000000002e-05 2024-09-14T23:56:47.614 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:47.614 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:47.614 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:47.614 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:47.614 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:47.615 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:56:47.616 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T23:56:47.617 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:56:47.614+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:56:47.717 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:56:47.715+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:56:47.726 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:56:47.726 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:56:47.726 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:56:47.726 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:56:47.734 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T23:56:47.817 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:56:47.816+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:56:47.841 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:56:47.841 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:56:47.841 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:56:47.841 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:56:47.842 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:56:47.850 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T23:56:47.918 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:56:47.917+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:56:47.999 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:56:47.999 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:56:47.999 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:56:48.000 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:56:48.000 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.000 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:3500 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T23:56:48.000 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:37.990193+0000", 2024-09-14T23:56:48.000 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 10.006834488999999, 2024-09-14T23:56:48.000 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99440750899999997, 2024-09-14T23:56:48.000 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:48.000 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:48.000 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:48.001 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:48.001 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:48.001 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:48.001 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3500 2024-09-14T23:56:48.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.001 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:48.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:48.001 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990193+0000", 2024-09-14T23:56:48.001 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:48.002 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990193+0000", 2024-09-14T23:56:48.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.002 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:48.002 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990194+0000", 2024-09-14T23:56:48.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5000000000000001e-07 2024-09-14T23:56:48.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.002 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:48.003 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990200+0000", 2024-09-14T23:56:48.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4389999999999997e-06 2024-09-14T23:56:48.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.003 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:48.003 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990215+0000", 2024-09-14T23:56:48.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4613999999999999e-05 2024-09-14T23:56:48.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.003 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:48.003 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990224+0000", 2024-09-14T23:56:48.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1740000000000006e-06 2024-09-14T23:56:48.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:48.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990283+0000", 2024-09-14T23:56:48.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9231999999999998e-05 2024-09-14T23:56:48.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.004 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:56:48.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990291+0000", 2024-09-14T23:56:48.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5900000000000002e-06 2024-09-14T23:56:48.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.004 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:48.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:38.984249+0000", 2024-09-14T23:56:48.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99395822700000003 2024-09-14T23:56:48.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.005 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.005 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:48.005 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:38.984448+0000", 2024-09-14T23:56:48.005 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00019855900000000001 2024-09-14T23:56:48.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.005 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.005 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:48.005 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:38.984601+0000", 2024-09-14T23:56:48.005 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000152824 2024-09-14T23:56:48.005 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.005 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:48.005 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.006 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4054 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1032192~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:48.006 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099053+0000", 2024-09-14T23:56:48.006 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.8979745050000005, 2024-09-14T23:56:48.006 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044065339000000002, 2024-09-14T23:56:48.006 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:48.006 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:48.006 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:48.006 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:48.006 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:48.006 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:48.006 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4054 2024-09-14T23:56:48.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.007 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:48.007 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.007 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:48.007 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099053+0000", 2024-09-14T23:56:48.007 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.007 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.007 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:48.007 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099053+0000", 2024-09-14T23:56:48.007 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.007 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.007 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:48.007 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099053+0000", 2024-09-14T23:56:48.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1100000000000002e-07 2024-09-14T23:56:48.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:48.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099055+0000", 2024-09-14T23:56:48.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.404e-06 2024-09-14T23:56:48.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:48.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099059+0000", 2024-09-14T23:56:48.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.749e-06 2024-09-14T23:56:48.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:48.009 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099062+0000", 2024-09-14T23:56:48.009 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7809999999999999e-06 2024-09-14T23:56:48.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.009 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.009 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:48.009 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143012+0000", 2024-09-14T23:56:48.009 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043949929999999998 2024-09-14T23:56:48.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.009 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.009 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:48.009 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143058+0000", 2024-09-14T23:56:48.009 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5260999999999999e-05 2024-09-14T23:56:48.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.010 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.010 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:48.010 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143118+0000", 2024-09-14T23:56:48.010 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0903000000000003e-05 2024-09-14T23:56:48.010 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.010 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:48.010 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.010 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.010 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4055 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1093632~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:48.010 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099066+0000", 2024-09-14T23:56:48.010 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.8979616010000004, 2024-09-14T23:56:48.010 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044174782000000003, 2024-09-14T23:56:48.011 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:48.011 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:48.011 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:48.011 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:48.011 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:48.011 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:48.011 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4055 2024-09-14T23:56:48.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.011 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:48.011 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.011 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:48.011 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099066+0000", 2024-09-14T23:56:48.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.012 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:48.012 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099066+0000", 2024-09-14T23:56:48.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.012 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:48.012 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099066+0000", 2024-09-14T23:56:48.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3700000000000001e-07 2024-09-14T23:56:48.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.012 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:48.013 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099067+0000", 2024-09-14T23:56:48.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1379999999999999e-06 2024-09-14T23:56:48.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.013 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:48.013 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099073+0000", 2024-09-14T23:56:48.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4790000000000004e-06 2024-09-14T23:56:48.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.013 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:48.013 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099075+0000", 2024-09-14T23:56:48.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5320000000000001e-06 2024-09-14T23:56:48.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.014 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:48.014 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143147+0000", 2024-09-14T23:56:48.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044071676999999997 2024-09-14T23:56:48.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.014 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:48.014 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143186+0000", 2024-09-14T23:56:48.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8821e-05 2024-09-14T23:56:48.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.014 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:48.014 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143241+0000", 2024-09-14T23:56:48.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4797999999999999e-05 2024-09-14T23:56:48.015 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.015 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:48.015 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.015 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4056 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1155072~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:48.015 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099079+0000", 2024-09-14T23:56:48.015 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.897948886, 2024-09-14T23:56:48.015 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044295020999999997, 2024-09-14T23:56:48.015 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:48.015 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:48.015 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:48.015 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:48.015 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:48.016 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:48.016 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4056 2024-09-14T23:56:48.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.016 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:48.016 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.016 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:48.016 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099079+0000", 2024-09-14T23:56:48.016 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.016 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.016 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:48.016 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099079+0000", 2024-09-14T23:56:48.017 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.017 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.017 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:48.017 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099079+0000", 2024-09-14T23:56:48.017 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1500000000000001e-07 2024-09-14T23:56:48.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.017 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.017 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:48.017 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099080+0000", 2024-09-14T23:56:48.017 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0210000000000001e-06 2024-09-14T23:56:48.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.018 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:48.018 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099083+0000", 2024-09-14T23:56:48.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.174e-06 2024-09-14T23:56:48.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.018 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:48.018 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099086+0000", 2024-09-14T23:56:48.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.446e-06 2024-09-14T23:56:48.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.018 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:48.018 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143269+0000", 2024-09-14T23:56:48.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044183489999999999 2024-09-14T23:56:48.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:48.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143314+0000", 2024-09-14T23:56:48.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5135000000000001e-05 2024-09-14T23:56:48.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:48.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143374+0000", 2024-09-14T23:56:48.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9540000000000003e-05 2024-09-14T23:56:48.019 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.019 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:48.019 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.019 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4057 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1216512~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:48.020 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099088+0000", 2024-09-14T23:56:48.020 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.8979390990000002, 2024-09-14T23:56:48.020 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044406119000000001, 2024-09-14T23:56:48.020 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:48.020 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:48.020 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:48.020 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:48.020 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:48.020 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:48.020 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4057 2024-09-14T23:56:48.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.020 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:48.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.020 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:48.021 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099088+0000", 2024-09-14T23:56:48.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.021 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:48.021 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099088+0000", 2024-09-14T23:56:48.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.021 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:48.021 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099089+0000", 2024-09-14T23:56:48.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.04e-07 2024-09-14T23:56:48.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.022 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:48.022 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099090+0000", 2024-09-14T23:56:48.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.026e-06 2024-09-14T23:56:48.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.022 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:48.022 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099093+0000", 2024-09-14T23:56:48.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0819999999999999e-06 2024-09-14T23:56:48.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.022 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:48.022 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099095+0000", 2024-09-14T23:56:48.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4930000000000001e-06 2024-09-14T23:56:48.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.023 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:48.023 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143400+0000", 2024-09-14T23:56:48.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044304698000000003 2024-09-14T23:56:48.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.023 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:48.023 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143441+0000", 2024-09-14T23:56:48.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1100000000000003e-05 2024-09-14T23:56:48.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.023 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:48.023 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143495+0000", 2024-09-14T23:56:48.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3516000000000001e-05 2024-09-14T23:56:48.023 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.023 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:48.024 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.024 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4058 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1277952~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:48.024 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099099+0000", 2024-09-14T23:56:48.024 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.8979286080000008, 2024-09-14T23:56:48.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044514583000000003, 2024-09-14T23:56:48.024 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:48.024 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:48.024 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:48.024 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:48.024 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:48.024 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:48.024 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4058 2024-09-14T23:56:48.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.025 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:48.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:48.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099099+0000", 2024-09-14T23:56:48.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:48.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099099+0000", 2024-09-14T23:56:48.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:48.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099099+0000", 2024-09-14T23:56:48.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9999999999999999e-07 2024-09-14T23:56:48.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:48.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099100+0000", 2024-09-14T23:56:48.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7199999999999997e-07 2024-09-14T23:56:48.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:48.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099103+0000", 2024-09-14T23:56:48.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.095e-06 2024-09-14T23:56:48.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:48.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099106+0000", 2024-09-14T23:56:48.027 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4229999999999999e-06 2024-09-14T23:56:48.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.027 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.027 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:48.027 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143524+0000", 2024-09-14T23:56:48.027 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044417891000000001 2024-09-14T23:56:48.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.027 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.027 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:48.027 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143568+0000", 2024-09-14T23:56:48.027 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4060000000000002e-05 2024-09-14T23:56:48.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.027 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.027 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:48.027 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143614+0000", 2024-09-14T23:56:48.027 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5942000000000002e-05 2024-09-14T23:56:48.028 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.028 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:48.028 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.028 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4059 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1339392~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:48.028 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099114+0000", 2024-09-14T23:56:48.028 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.8979140690000005, 2024-09-14T23:56:48.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044617194999999998, 2024-09-14T23:56:48.028 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:48.028 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:48.028 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:48.028 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:48.028 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:48.029 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:48.029 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4059 2024-09-14T23:56:48.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.029 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:48.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:48.029 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099114+0000", 2024-09-14T23:56:48.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:48.029 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099114+0000", 2024-09-14T23:56:48.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:48.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099114+0000", 2024-09-14T23:56:48.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5899999999999998e-07 2024-09-14T23:56:48.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:48.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099115+0000", 2024-09-14T23:56:48.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3510000000000001e-06 2024-09-14T23:56:48.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:48.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099119+0000", 2024-09-14T23:56:48.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.16e-06 2024-09-14T23:56:48.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.031 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:48.031 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099123+0000", 2024-09-14T23:56:48.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1339999999999997e-06 2024-09-14T23:56:48.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.031 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:48.031 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143643+0000", 2024-09-14T23:56:48.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044519232999999998 2024-09-14T23:56:48.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.031 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:48.032 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143686+0000", 2024-09-14T23:56:48.032 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3349000000000001e-05 2024-09-14T23:56:48.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:48.032 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143731+0000", 2024-09-14T23:56:48.032 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4709000000000002e-05 2024-09-14T23:56:48.032 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.032 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:48.032 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.032 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4060 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1400832~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:48.032 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099129+0000", 2024-09-14T23:56:48.032 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.8978988460000004, 2024-09-14T23:56:48.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044721271999999999, 2024-09-14T23:56:48.033 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:48.033 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:48.033 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:48.033 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:48.033 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:48.033 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:48.033 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4060 2024-09-14T23:56:48.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.033 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:48.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.033 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:48.033 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099129+0000", 2024-09-14T23:56:48.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.034 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:48.034 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099129+0000", 2024-09-14T23:56:48.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.034 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:48.034 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099191+0000", 2024-09-14T23:56:48.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1805999999999997e-05 2024-09-14T23:56:48.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.034 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:48.034 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099193+0000", 2024-09-14T23:56:48.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2330000000000001e-06 2024-09-14T23:56:48.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:48.035 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099200+0000", 2024-09-14T23:56:48.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4200000000000001e-06 2024-09-14T23:56:48.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:48.035 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099204+0000", 2024-09-14T23:56:48.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7500000000000001e-06 2024-09-14T23:56:48.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:48.035 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143760+0000", 2024-09-14T23:56:48.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044556182 2024-09-14T23:56:48.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:48.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143804+0000", 2024-09-14T23:56:48.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4239000000000001e-05 2024-09-14T23:56:48.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:48.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143850+0000", 2024-09-14T23:56:48.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5642000000000001e-05 2024-09-14T23:56:48.036 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.036 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:48.037 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.037 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.037 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4061 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1462272~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:48.037 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099215+0000", 2024-09-14T23:56:48.037 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.8978127209999993, 2024-09-14T23:56:48.037 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044745659, 2024-09-14T23:56:48.037 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:48.037 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:48.037 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:48.037 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:48.037 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:48.037 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:48.037 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4061 2024-09-14T23:56:48.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.038 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:48.038 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.038 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:48.038 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099215+0000", 2024-09-14T23:56:48.038 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.038 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.038 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:48.038 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099215+0000", 2024-09-14T23:56:48.038 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.038 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.038 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:48.038 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100672+0000", 2024-09-14T23:56:48.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00145689 2024-09-14T23:56:48.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:48.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100677+0000", 2024-09-14T23:56:48.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0139999999999998e-06 2024-09-14T23:56:48.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:48.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100689+0000", 2024-09-14T23:56:48.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.183e-05 2024-09-14T23:56:48.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:48.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100696+0000", 2024-09-14T23:56:48.040 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2599999999999999e-06 2024-09-14T23:56:48.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.040 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:48.040 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143877+0000", 2024-09-14T23:56:48.040 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043181279000000003 2024-09-14T23:56:48.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.040 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:48.040 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143919+0000", 2024-09-14T23:56:48.040 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1783000000000003e-05 2024-09-14T23:56:48.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.041 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:48.041 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143961+0000", 2024-09-14T23:56:48.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1603000000000001e-05 2024-09-14T23:56:48.041 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.041 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:48.041 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.041 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.041 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4070 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 2015232~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:48.041 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.100903+0000", 2024-09-14T23:56:48.041 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.8961250130000007, 2024-09-14T23:56:48.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044103815999999997, 2024-09-14T23:56:48.041 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:48.041 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:48.041 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:48.042 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:48.042 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:48.042 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:48.042 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4070 2024-09-14T23:56:48.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.042 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:48.042 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.042 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:48.042 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100903+0000", 2024-09-14T23:56:48.042 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.042 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.042 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:48.042 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100903+0000", 2024-09-14T23:56:48.042 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.043 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:48.043 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.102671+0000", 2024-09-14T23:56:48.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001768279 2024-09-14T23:56:48.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.043 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:48.043 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.102675+0000", 2024-09-14T23:56:48.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9609999999999999e-06 2024-09-14T23:56:48.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.043 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:48.043 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.102684+0000", 2024-09-14T23:56:48.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.329e-06 2024-09-14T23:56:48.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.044 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:48.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.102689+0000", 2024-09-14T23:56:48.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7079999999999996e-06 2024-09-14T23:56:48.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.044 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:48.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.144912+0000", 2024-09-14T23:56:48.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.042223305000000003 2024-09-14T23:56:48.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.044 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:48.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.144960+0000", 2024-09-14T23:56:48.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7670000000000003e-05 2024-09-14T23:56:48.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.045 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.045 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:48.045 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.145006+0000", 2024-09-14T23:56:48.045 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6563999999999997e-05 2024-09-14T23:56:48.045 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.045 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:48.045 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.045 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.045 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4096 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1282048~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:48.045 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.104957+0000", 2024-09-14T23:56:48.046 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.8920708400000006, 2024-09-14T23:56:48.046 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044093016999999998, 2024-09-14T23:56:48.046 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:48.046 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:48.046 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:48.046 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:48.046 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:48.046 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:48.046 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4096 2024-09-14T23:56:48.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.046 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:48.046 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.046 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:48.046 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104957+0000", 2024-09-14T23:56:48.046 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.047 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:48.047 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104957+0000", 2024-09-14T23:56:48.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.047 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:48.047 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104957+0000", 2024-09-14T23:56:48.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7400000000000003e-07 2024-09-14T23:56:48.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.047 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:48.047 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104960+0000", 2024-09-14T23:56:48.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3800000000000001e-06 2024-09-14T23:56:48.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.048 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.048 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:48.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104967+0000", 2024-09-14T23:56:48.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4030000000000001e-06 2024-09-14T23:56:48.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.048 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.048 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:48.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104973+0000", 2024-09-14T23:56:48.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7370000000000003e-06 2024-09-14T23:56:48.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.048 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.049 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:48.049 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.148883+0000", 2024-09-14T23:56:48.049 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043910124000000002 2024-09-14T23:56:48.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.049 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.049 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:48.049 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.148966+0000", 2024-09-14T23:56:48.049 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2868999999999996e-05 2024-09-14T23:56:48.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.049 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.049 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:48.049 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149050+0000", 2024-09-14T23:56:48.049 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3930000000000004e-05 2024-09-14T23:56:48.049 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.049 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:48.050 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.050 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.050 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.050 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4097 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1343488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:48.050 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.104983+0000", 2024-09-14T23:56:48.050 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.8920450510000002, 2024-09-14T23:56:48.050 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044182104999999999, 2024-09-14T23:56:48.050 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:48.050 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:48.050 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:48.050 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:48.050 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:48.050 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:48.050 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4097 2024-09-14T23:56:48.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.051 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:48.051 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.051 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:48.051 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104983+0000", 2024-09-14T23:56:48.051 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.051 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.051 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:48.051 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104983+0000", 2024-09-14T23:56:48.051 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.051 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.051 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:48.052 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105207+0000", 2024-09-14T23:56:48.052 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000224687 2024-09-14T23:56:48.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.052 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.052 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:48.052 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105211+0000", 2024-09-14T23:56:48.052 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3050000000000001e-06 2024-09-14T23:56:48.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.052 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.052 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:48.052 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105217+0000", 2024-09-14T23:56:48.052 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8170000000000001e-06 2024-09-14T23:56:48.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.053 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:48.053 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105222+0000", 2024-09-14T23:56:48.053 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.267e-06 2024-09-14T23:56:48.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.053 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:48.053 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149079+0000", 2024-09-14T23:56:48.053 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043857581999999999 2024-09-14T23:56:48.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.053 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:48.053 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149122+0000", 2024-09-14T23:56:48.053 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2899999999999999e-05 2024-09-14T23:56:48.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.054 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.054 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:48.054 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149165+0000", 2024-09-14T23:56:48.054 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2546999999999998e-05 2024-09-14T23:56:48.054 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.054 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:48.056 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.056 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.056 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.056 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4098 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1404928~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:48.056 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.105242+0000", 2024-09-14T23:56:48.056 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.8917852390000007, 2024-09-14T23:56:48.056 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044121104000000001, 2024-09-14T23:56:48.056 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:48.056 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:48.056 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:48.056 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:48.056 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:48.056 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:48.057 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4098 2024-09-14T23:56:48.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.057 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:48.057 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.057 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:48.057 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105242+0000", 2024-09-14T23:56:48.057 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.057 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.057 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:48.057 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105242+0000", 2024-09-14T23:56:48.058 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.058 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.058 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:48.058 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105243+0000", 2024-09-14T23:56:48.058 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0399999999999995e-07 2024-09-14T23:56:48.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.058 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.058 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:48.058 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105246+0000", 2024-09-14T23:56:48.058 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6309999999999999e-06 2024-09-14T23:56:48.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.058 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.058 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:48.059 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105253+0000", 2024-09-14T23:56:48.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0940000000000002e-06 2024-09-14T23:56:48.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.059 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:48.059 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105260+0000", 2024-09-14T23:56:48.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1799999999999999e-06 2024-09-14T23:56:48.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.059 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:48.059 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149201+0000", 2024-09-14T23:56:48.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043940629000000002 2024-09-14T23:56:48.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.060 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:48.060 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149283+0000", 2024-09-14T23:56:48.060 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2360000000000004e-05 2024-09-14T23:56:48.060 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.060 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.060 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:48.060 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149363+0000", 2024-09-14T23:56:48.060 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0506000000000005e-05 2024-09-14T23:56:48.061 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.061 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:48.061 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.061 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.061 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4099 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1466368~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:48.061 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.105270+0000", 2024-09-14T23:56:48.061 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.8917576440000001, 2024-09-14T23:56:48.061 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044195433999999999, 2024-09-14T23:56:48.061 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:48.061 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:48.061 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:48.061 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:48.062 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:48.062 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:48.062 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4099 2024-09-14T23:56:48.062 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.062 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:48.062 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.062 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:48.062 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105270+0000", 2024-09-14T23:56:48.062 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.062 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.062 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.062 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:48.062 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105270+0000", 2024-09-14T23:56:48.062 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:48.063 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105429+0000", 2024-09-14T23:56:48.063 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015940299999999999 2024-09-14T23:56:48.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:48.063 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105433+0000", 2024-09-14T23:56:48.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3579999999999999e-06 2024-09-14T23:56:48.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:48.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105439+0000", 2024-09-14T23:56:48.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1759999999999998e-06 2024-09-14T23:56:48.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:48.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105445+0000", 2024-09-14T23:56:48.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9689999999999999e-06 2024-09-14T23:56:48.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.065 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.065 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:48.065 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149381+0000", 2024-09-14T23:56:48.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043936389999999999 2024-09-14T23:56:48.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.065 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.065 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:48.065 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149422+0000", 2024-09-14T23:56:48.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0865000000000003e-05 2024-09-14T23:56:48.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.065 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.065 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:48.065 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149465+0000", 2024-09-14T23:56:48.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3272999999999997e-05 2024-09-14T23:56:48.066 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.066 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:48.066 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.066 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.066 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.066 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4100 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1527808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:48.066 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.105454+0000", 2024-09-14T23:56:48.066 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.8915731729999994, 2024-09-14T23:56:48.066 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044217411999999998, 2024-09-14T23:56:48.066 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:48.066 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:48.066 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:48.066 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:48.066 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:48.067 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:48.067 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4100 2024-09-14T23:56:48.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.067 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:48.067 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.067 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:48.067 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105454+0000", 2024-09-14T23:56:48.067 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.067 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.067 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:48.067 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105454+0000", 2024-09-14T23:56:48.067 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.067 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.068 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:48.068 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105578+0000", 2024-09-14T23:56:48.068 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012401799999999999 2024-09-14T23:56:48.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.068 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.068 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:48.068 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105583+0000", 2024-09-14T23:56:48.068 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1779999999999996e-06 2024-09-14T23:56:48.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.068 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.068 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:48.068 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105590+0000", 2024-09-14T23:56:48.068 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4279999999999999e-06 2024-09-14T23:56:48.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.069 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.069 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:48.069 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105596+0000", 2024-09-14T23:56:48.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0120000000000002e-06 2024-09-14T23:56:48.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.069 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.069 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:48.069 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149503+0000", 2024-09-14T23:56:48.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043906580000000001 2024-09-14T23:56:48.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.069 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.069 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:48.069 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149585+0000", 2024-09-14T23:56:48.070 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2356999999999998e-05 2024-09-14T23:56:48.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.070 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:48.070 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149672+0000", 2024-09-14T23:56:48.070 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6839e-05 2024-09-14T23:56:48.070 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.070 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:48.070 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.070 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4101 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1589248~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:48.070 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.105609+0000", 2024-09-14T23:56:48.070 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.8914186340000008, 2024-09-14T23:56:48.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044175987999999999, 2024-09-14T23:56:48.071 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:48.071 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:48.071 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:48.071 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:48.071 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:48.071 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:48.071 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4101 2024-09-14T23:56:48.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.071 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:48.071 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.071 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:48.071 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105609+0000", 2024-09-14T23:56:48.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.072 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.072 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:48.072 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105609+0000", 2024-09-14T23:56:48.072 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.072 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.072 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:48.072 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.106671+0000", 2024-09-14T23:56:48.072 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010619270000000001 2024-09-14T23:56:48.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.072 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.072 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:48.072 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.106675+0000", 2024-09-14T23:56:48.073 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4429999999999996e-06 2024-09-14T23:56:48.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.073 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.073 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:48.073 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.106685+0000", 2024-09-14T23:56:48.073 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7429999999999997e-06 2024-09-14T23:56:48.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.073 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.073 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:48.073 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.106690+0000", 2024-09-14T23:56:48.073 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.293e-06 2024-09-14T23:56:48.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.073 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.074 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:48.074 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149700+0000", 2024-09-14T23:56:48.074 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043009512 2024-09-14T23:56:48.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.074 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.074 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:48.074 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149742+0000", 2024-09-14T23:56:48.074 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2083999999999999e-05 2024-09-14T23:56:48.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.074 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.074 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:48.075 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149785+0000", 2024-09-14T23:56:48.075 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2985999999999998e-05 2024-09-14T23:56:48.075 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.075 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:48.075 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.075 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.075 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4116 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 180224~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:48.075 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.107575+0000", 2024-09-14T23:56:48.075 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.8894521659999999, 2024-09-14T23:56:48.075 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044157522999999997, 2024-09-14T23:56:48.075 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:48.075 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:48.076 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:48.076 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:48.076 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:48.076 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:48.076 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4116 2024-09-14T23:56:48.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.076 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:48.076 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.076 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:48.076 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.107575+0000", 2024-09-14T23:56:48.076 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.076 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.076 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:48.077 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.107575+0000", 2024-09-14T23:56:48.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.077 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:48.077 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108671+0000", 2024-09-14T23:56:48.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010954510000000001 2024-09-14T23:56:48.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.077 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:48.077 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108676+0000", 2024-09-14T23:56:48.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6639999999999997e-06 2024-09-14T23:56:48.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.078 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:48.078 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108685+0000", 2024-09-14T23:56:48.078 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1130000000000007e-06 2024-09-14T23:56:48.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.078 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.078 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:48.078 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108691+0000", 2024-09-14T23:56:48.078 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6019999999999996e-06 2024-09-14T23:56:48.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.078 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.078 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:48.078 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.151530+0000", 2024-09-14T23:56:48.078 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.042838345999999999 2024-09-14T23:56:48.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.079 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.079 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:48.079 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.151619+0000", 2024-09-14T23:56:48.079 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9302999999999995e-05 2024-09-14T23:56:48.079 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.079 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.079 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:48.079 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.151733+0000", 2024-09-14T23:56:48.079 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011404399999999999 2024-09-14T23:56:48.079 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.079 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:48.080 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.080 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.080 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.080 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4122 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 548864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:48.080 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.108842+0000", 2024-09-14T23:56:48.080 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.8881853280000005, 2024-09-14T23:56:48.080 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044103631999999997, 2024-09-14T23:56:48.080 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:48.080 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:48.080 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:48.080 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:48.080 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:48.081 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:48.081 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4122 2024-09-14T23:56:48.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.081 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:48.081 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.081 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:48.081 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108842+0000", 2024-09-14T23:56:48.081 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.081 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.081 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:48.081 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108842+0000", 2024-09-14T23:56:48.081 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.081 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.082 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:48.082 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108843+0000", 2024-09-14T23:56:48.082 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7000000000000001e-07 2024-09-14T23:56:48.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.082 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.082 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:48.082 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108844+0000", 2024-09-14T23:56:48.082 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6479999999999999e-06 2024-09-14T23:56:48.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.082 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.082 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:48.083 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108848+0000", 2024-09-14T23:56:48.083 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8029999999999998e-06 2024-09-14T23:56:48.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.083 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.083 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:48.083 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108851+0000", 2024-09-14T23:56:48.083 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7939999999999998e-06 2024-09-14T23:56:48.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.083 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.083 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:48.083 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.152838+0000", 2024-09-14T23:56:48.083 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043987351000000001 2024-09-14T23:56:48.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.083 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.084 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:48.084 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.152885+0000", 2024-09-14T23:56:48.084 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7237000000000002e-05 2024-09-14T23:56:48.084 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.084 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.084 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:48.084 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.152946+0000", 2024-09-14T23:56:48.084 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0529000000000003e-05 2024-09-14T23:56:48.084 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.084 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:48.084 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.084 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.084 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.084 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4123 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 610304~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:48.085 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.108855+0000", 2024-09-14T23:56:48.085 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.888172999, 2024-09-14T23:56:48.085 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044321610999999997, 2024-09-14T23:56:48.085 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:48.085 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:48.085 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:48.085 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:48.085 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:48.085 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:48.085 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4123 2024-09-14T23:56:48.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.085 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:48.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.086 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:48.086 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108855+0000", 2024-09-14T23:56:48.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.086 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.086 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:48.086 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108855+0000", 2024-09-14T23:56:48.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.086 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.086 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:48.087 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.110671+0000", 2024-09-14T23:56:48.087 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018159719999999999 2024-09-14T23:56:48.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.087 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:48.087 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.110676+0000", 2024-09-14T23:56:48.087 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9450000000000001e-06 2024-09-14T23:56:48.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.087 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:48.087 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.110685+0000", 2024-09-14T23:56:48.087 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5359999999999998e-06 2024-09-14T23:56:48.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.088 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.088 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:48.088 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.110692+0000", 2024-09-14T23:56:48.088 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9280000000000004e-06 2024-09-14T23:56:48.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.088 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.088 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:48.088 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.152984+0000", 2024-09-14T23:56:48.088 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.042291677999999999 2024-09-14T23:56:48.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.088 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.088 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:48.089 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.153066+0000", 2024-09-14T23:56:48.089 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2657999999999994e-05 2024-09-14T23:56:48.089 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.089 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.089 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:48.089 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.153176+0000", 2024-09-14T23:56:48.089 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000109894 2024-09-14T23:56:48.089 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.089 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:48.089 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.089 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.089 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.090 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4136 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:48.090 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.111056+0000", 2024-09-14T23:56:48.090 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.8859718510000008, 2024-09-14T23:56:48.090 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044097428000000001, 2024-09-14T23:56:48.090 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:48.090 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:48.090 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:48.090 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:48.090 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:48.090 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:48.090 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4136 2024-09-14T23:56:48.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.090 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:48.090 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.090 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:48.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.111056+0000", 2024-09-14T23:56:48.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:48.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.111056+0000", 2024-09-14T23:56:48.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:48.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.112671+0000", 2024-09-14T23:56:48.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016153999999999999 2024-09-14T23:56:48.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:48.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.112675+0000", 2024-09-14T23:56:48.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2830000000000003e-06 2024-09-14T23:56:48.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.092 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.092 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:48.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.112684+0000", 2024-09-14T23:56:48.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7800000000000006e-06 2024-09-14T23:56:48.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.092 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.092 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:48.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.112689+0000", 2024-09-14T23:56:48.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0000000000000004e-06 2024-09-14T23:56:48.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.093 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.093 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:48.093 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.155064+0000", 2024-09-14T23:56:48.093 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.042374423000000001 2024-09-14T23:56:48.093 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.093 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.093 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:48.093 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.155111+0000", 2024-09-14T23:56:48.093 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7367000000000002e-05 2024-09-14T23:56:48.093 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.093 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.093 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:48.093 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.155153+0000", 2024-09-14T23:56:48.093 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2175000000000002e-05 2024-09-14T23:56:48.093 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.094 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:48.094 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.094 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.094 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:48.094 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:56:48.094 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T23:56:48.096 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:56:48.094+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:56:48.096 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T23:56:48.098 INFO:teuthology.orchestra.run.smithi042.stdout:leaving image image_client.0-clone7 intact 2024-09-14T23:56:48.196 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:56:48.194+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:56:48.205 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:56:48.206 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:56:48.206 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:56:48.206 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:56:48.215 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T23:56:48.257 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-09-14T23:56:48.289 INFO:teuthology.orchestra.run.smithi042.stdout:929 punch from 0x2e5f3bc to 0x2e6504a, (0x5c8e bytes) 2024-09-14T23:56:48.293 INFO:teuthology.orchestra.run.smithi042.stdout:930 punch from 0x47256f2 to 0x4728545, (0x2e53 bytes) 2024-09-14T23:56:48.293 INFO:teuthology.orchestra.run.smithi042.stdout:931 writesame 0x273e904 thru 0x2745822 data_size 0x1(0x6f1f bytes) 2024-09-14T23:56:48.296 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:56:48.295+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:56:48.331 INFO:teuthology.orchestra.run.smithi042.stdout:932 trunc from 0xb103778 to 0x7761dc5 2024-09-14T23:56:48.335 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:56:48.335 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:56:48.335 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:56:48.335 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:56:48.335 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:56:48.348 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T23:56:48.371 INFO:teuthology.orchestra.run.smithi042.stdout:933 write 0xe99cc6e thru 0xe9a78b7 (0xac4a bytes) 2024-09-14T23:56:48.384 INFO:teuthology.orchestra.run.smithi042.stdout:934 punch from 0x639adf1 to 0x63a2f83, (0x8192 bytes) 2024-09-14T23:56:48.384 INFO:teuthology.orchestra.run.smithi042.stdout:935 writesame 0x9451707 thru 0x945bc0d data_size 0x1(0xa507 bytes) 2024-09-14T23:56:48.397 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:56:48.396+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:56:48.401 INFO:teuthology.orchestra.run.smithi042.stdout:936 read 0xaaa8dbf thru 0xaab215b (0x939d bytes) 2024-09-14T23:56:48.401 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:48.401 INFO:teuthology.orchestra.run.smithi042.stdout:938 read 0x42e66d thru 0x42f743 (0x10d7 bytes) 2024-09-14T23:56:48.401 INFO:teuthology.orchestra.run.smithi042.stdout:940 write 0xa2dd83a thru 0xa2eb961 (0xe128 bytes) 2024-09-14T23:56:48.420 INFO:teuthology.orchestra.run.smithi042.stdout:941 writesame 0x213efc2 thru 0x21490c0 data_size 0x1(0xa0ff bytes) 2024-09-14T23:56:48.474 INFO:teuthology.orchestra.run.smithi042.stdout:942 trunc from 0xe9a78b8 to 0x3c991c8 2024-09-14T23:56:48.498 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:56:48.496+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:56:48.515 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:48.515 INFO:teuthology.orchestra.run.smithi042.stdout:945 read 0x211c9c5 thru 0x212bcec (0xf328 bytes) 2024-09-14T23:56:48.516 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:48.516 INFO:teuthology.orchestra.run.smithi042.stdout:947 punch from 0x3bdef48 to 0x3be8bb6, (0x9c6e bytes) 2024-09-14T23:56:48.516 INFO:teuthology.orchestra.run.smithi042.stdout:949 read 0x11d2938 thru 0x11e064f (0xdd18 bytes) 2024-09-14T23:56:48.519 INFO:teuthology.orchestra.run.smithi042.stdout:950 punch from 0x1bbe8dd to 0x1bcd82b, (0xef4e bytes) 2024-09-14T23:56:48.519 INFO:teuthology.orchestra.run.smithi042.stdout:951 read 0x2e29dec thru 0x2e398e3 (0xfaf8 bytes) 2024-09-14T23:56:48.521 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:48.522 INFO:teuthology.orchestra.run.smithi042.stdout:955 trunc from 0x3c991c8 to 0x4cc8c25 2024-09-14T23:56:48.534 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:56:48.534 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:56:48.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:56:48.534 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:56:48.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.534 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4284.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-14T23:56:48.534 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:56:48.535 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 26.007601018999999, 2024-09-14T23:56:48.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.004942086, 2024-09-14T23:56:48.535 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:48.535 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:48.535 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:48.535 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:48.535 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4284", 2024-09-14T23:56:48.535 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/4278347926", 2024-09-14T23:56:48.535 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1 2024-09-14T23:56:48.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.535 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:48.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:48.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:56:48.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:48.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:56:48.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:48.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524331+0000", 2024-09-14T23:56:48.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.383e-05 2024-09-14T23:56:48.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:48.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524339+0000", 2024-09-14T23:56:48.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8290000000000008e-06 2024-09-14T23:56:48.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:48.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524359+0000", 2024-09-14T23:56:48.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9998000000000001e-05 2024-09-14T23:56:48.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.538 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:48.538 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524373+0000", 2024-09-14T23:56:48.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4304000000000001e-05 2024-09-14T23:56:48.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.538 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:48.538 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524488+0000", 2024-09-14T23:56:48.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114663 2024-09-14T23:56:48.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.538 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:56:48.539 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524501+0000", 2024-09-14T23:56:48.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3349999999999999e-05 2024-09-14T23:56:48.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.539 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:48.539 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.526868+0000", 2024-09-14T23:56:48.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023670090000000002 2024-09-14T23:56:48.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.539 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:48.539 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.527041+0000", 2024-09-14T23:56:48.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000172597 2024-09-14T23:56:48.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-14T23:56:48.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.527384+0000", 2024-09-14T23:56:48.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00034309500000000002 2024-09-14T23:56:48.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:48.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529103+0000", 2024-09-14T23:56:48.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017185220000000001 2024-09-14T23:56:48.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:56:48.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529188+0000", 2024-09-14T23:56:48.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5591000000000001e-05 2024-09-14T23:56:48.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:48.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529227+0000", 2024-09-14T23:56:48.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8563000000000003e-05 2024-09-14T23:56:48.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:48.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529259+0000", 2024-09-14T23:56:48.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2734999999999997e-05 2024-09-14T23:56:48.542 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.542 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:48.542 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.542 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4284.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-14T23:56:48.542 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:23.530683+0000", 2024-09-14T23:56:48.542 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 25.001235353999999, 2024-09-14T23:56:48.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0036133110000000001, 2024-09-14T23:56:48.542 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:48.542 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:48.542 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:48.542 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:48.542 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4284", 2024-09-14T23:56:48.543 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/4278347926", 2024-09-14T23:56:48.543 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5 2024-09-14T23:56:48.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.543 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:48.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.543 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:48.543 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530683+0000", 2024-09-14T23:56:48.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.543 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:48.543 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530683+0000", 2024-09-14T23:56:48.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:48.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530685+0000", 2024-09-14T23:56:48.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.075e-06 2024-09-14T23:56:48.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:48.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530695+0000", 2024-09-14T23:56:48.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0261999999999999e-05 2024-09-14T23:56:48.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:48.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530715+0000", 2024-09-14T23:56:48.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9930000000000001e-05 2024-09-14T23:56:48.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:48.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530724+0000", 2024-09-14T23:56:48.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.901e-06 2024-09-14T23:56:48.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:48.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530800+0000", 2024-09-14T23:56:48.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5873999999999999e-05 2024-09-14T23:56:48.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-14T23:56:48.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530807+0000", 2024-09-14T23:56:48.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2389999999999996e-06 2024-09-14T23:56:48.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:48.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.532494+0000", 2024-09-14T23:56:48.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016871830000000001 2024-09-14T23:56:48.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:48.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.532599+0000", 2024-09-14T23:56:48.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010472 2024-09-14T23:56:48.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1", 2024-09-14T23:56:48.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.532715+0000", 2024-09-14T23:56:48.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011568199999999999 2024-09-14T23:56:48.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:56:48.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.533460+0000", 2024-09-14T23:56:48.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00074494999999999997 2024-09-14T23:56:48.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:48.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534230+0000", 2024-09-14T23:56:48.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00077036599999999998 2024-09-14T23:56:48.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.548 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.548 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:48.548 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534266+0000", 2024-09-14T23:56:48.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5725000000000001e-05 2024-09-14T23:56:48.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.548 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.548 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:48.548 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534296+0000", 2024-09-14T23:56:48.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0403999999999999e-05 2024-09-14T23:56:48.548 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.548 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:48.548 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.549 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4284.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-14T23:56:48.549 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:24.535160+0000", 2024-09-14T23:56:48.549 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.996758712999998, 2024-09-14T23:56:48.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033473650000000001, 2024-09-14T23:56:48.549 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:48.549 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:48.549 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:48.549 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:48.549 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4284", 2024-09-14T23:56:48.549 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/4278347926", 2024-09-14T23:56:48.549 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 7 2024-09-14T23:56:48.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.550 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:48.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:48.550 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535160+0000", 2024-09-14T23:56:48.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:48.550 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535160+0000", 2024-09-14T23:56:48.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:48.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535162+0000", 2024-09-14T23:56:48.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3369999999999998e-06 2024-09-14T23:56:48.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:48.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535171+0000", 2024-09-14T23:56:48.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7700000000000007e-06 2024-09-14T23:56:48.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:48.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535189+0000", 2024-09-14T23:56:48.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7938e-05 2024-09-14T23:56:48.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.552 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.552 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:48.552 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535197+0000", 2024-09-14T23:56:48.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5639999999999997e-06 2024-09-14T23:56:48.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.552 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.552 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:48.552 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535266+0000", 2024-09-14T23:56:48.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8926999999999999e-05 2024-09-14T23:56:48.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.552 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.552 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-14T23:56:48.553 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535271+0000", 2024-09-14T23:56:48.553 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0329999999999999e-06 2024-09-14T23:56:48.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.553 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:48.553 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.537064+0000", 2024-09-14T23:56:48.553 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001792929 2024-09-14T23:56:48.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.553 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:48.553 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.537131+0000", 2024-09-14T23:56:48.553 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6605000000000004e-05 2024-09-14T23:56:48.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1", 2024-09-14T23:56:48.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.537271+0000", 2024-09-14T23:56:48.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013992400000000001 2024-09-14T23:56:48.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:56:48.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.538093+0000", 2024-09-14T23:56:48.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00082210200000000001 2024-09-14T23:56:48.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:48.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.538439+0000", 2024-09-14T23:56:48.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00034595999999999997 2024-09-14T23:56:48.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:48.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.538476+0000", 2024-09-14T23:56:48.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6770000000000002e-05 2024-09-14T23:56:48.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:48.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.538507+0000", 2024-09-14T23:56:48.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1506e-05 2024-09-14T23:56:48.555 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.555 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:48.555 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.556 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T23:56:48.556 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:34.406877+0000", 2024-09-14T23:56:48.556 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 14.125041707999999, 2024-09-14T23:56:48.556 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030351169999999999, 2024-09-14T23:56:48.556 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:48.556 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:48.556 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:48.556 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:48.556 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:48.556 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:48.556 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 10 2024-09-14T23:56:48.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.556 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:48.557 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.557 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:48.557 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406877+0000", 2024-09-14T23:56:48.557 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.557 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.557 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.557 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:48.557 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406877+0000", 2024-09-14T23:56:48.557 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.557 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.557 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.557 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:48.557 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406878+0000", 2024-09-14T23:56:48.558 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.291e-06 2024-09-14T23:56:48.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.558 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.558 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:48.558 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406883+0000", 2024-09-14T23:56:48.558 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2399999999999998e-06 2024-09-14T23:56:48.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.558 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.558 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:48.558 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406891+0000", 2024-09-14T23:56:48.558 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5889999999999997e-06 2024-09-14T23:56:48.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.558 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:48.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406898+0000", 2024-09-14T23:56:48.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3599999999999998e-06 2024-09-14T23:56:48.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:48.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406948+0000", 2024-09-14T23:56:48.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9435999999999999e-05 2024-09-14T23:56:48.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-14T23:56:48.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406953+0000", 2024-09-14T23:56:48.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2809999999999999e-06 2024-09-14T23:56:48.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:48.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.408559+0000", 2024-09-14T23:56:48.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00160607 2024-09-14T23:56:48.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:48.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.408637+0000", 2024-09-14T23:56:48.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8504000000000006e-05 2024-09-14T23:56:48.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 4", 2024-09-14T23:56:48.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.408781+0000", 2024-09-14T23:56:48.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014374799999999999 2024-09-14T23:56:48.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.561 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:56:48.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.409659+0000", 2024-09-14T23:56:48.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000877781 2024-09-14T23:56:48.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.561 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:48.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.409822+0000", 2024-09-14T23:56:48.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016295400000000001 2024-09-14T23:56:48.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.561 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:48.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.409872+0000", 2024-09-14T23:56:48.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0022000000000003e-05 2024-09-14T23:56:48.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:48.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.409912+0000", 2024-09-14T23:56:48.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9841e-05 2024-09-14T23:56:48.562 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.562 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:48.562 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.562 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:3583 3.5 e29/28 3:a23f6ed6:::rbd_data.1118de1168b.0000000000000035:head v 29'16, mlcod=29'13)", 2024-09-14T23:56:48.562 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.040000+0000", 2024-09-14T23:56:48.562 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.4919183379999996, 2024-09-14T23:56:48.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0013206349999999999, 2024-09-14T23:56:48.563 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:48.563 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:48.563 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:48.563 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:48.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.563 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:48.563 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.040000+0000", 2024-09-14T23:56:48.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.563 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:48.563 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.040000+0000", 2024-09-14T23:56:48.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:48.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.040000+0000", 2024-09-14T23:56:48.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0400000000000002e-07 2024-09-14T23:56:48.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:48.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.040006+0000", 2024-09-14T23:56:48.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2700000000000004e-06 2024-09-14T23:56:48.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:48.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.040014+0000", 2024-09-14T23:56:48.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2660000000000001e-06 2024-09-14T23:56:48.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.565 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:48.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.040020+0000", 2024-09-14T23:56:48.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7239999999999997e-06 2024-09-14T23:56:48.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.565 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:48.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.040324+0000", 2024-09-14T23:56:48.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00030448600000000003 2024-09-14T23:56:48.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.565 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:48.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.040367+0000", 2024-09-14T23:56:48.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2749e-05 2024-09-14T23:56:48.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:48.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.041275+0000", 2024-09-14T23:56:48.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00090818799999999999 2024-09-14T23:56:48.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:48.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.041321+0000", 2024-09-14T23:56:48.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5547999999999998e-05 2024-09-14T23:56:48.566 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.566 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:48.566 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.567 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:3621 3.1f e29/23 3:fdd18962:::rbd_data.1118de1168b.0000000000000045:head v 29'12, mlcod=27'9)", 2024-09-14T23:56:48.567 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.047969+0000", 2024-09-14T23:56:48.567 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.4839489409999995, 2024-09-14T23:56:48.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001436122, 2024-09-14T23:56:48.567 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:48.567 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:48.567 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:48.567 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:48.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.567 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:48.567 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.047969+0000", 2024-09-14T23:56:48.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.568 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:48.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.047969+0000", 2024-09-14T23:56:48.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.568 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:48.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.047970+0000", 2024-09-14T23:56:48.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1199999999999998e-07 2024-09-14T23:56:48.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.568 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:48.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.047974+0000", 2024-09-14T23:56:48.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.549e-06 2024-09-14T23:56:48.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.569 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:48.569 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.047983+0000", 2024-09-14T23:56:48.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9509999999999995e-06 2024-09-14T23:56:48.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.569 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:48.569 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.047989+0000", 2024-09-14T23:56:48.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0939999999999996e-06 2024-09-14T23:56:48.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.569 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:48.569 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.048062+0000", 2024-09-14T23:56:48.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2274000000000006e-05 2024-09-14T23:56:48.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.570 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:48.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.048086+0000", 2024-09-14T23:56:48.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3875000000000001e-05 2024-09-14T23:56:48.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.570 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:48.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.049367+0000", 2024-09-14T23:56:48.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0012811719999999999 2024-09-14T23:56:48.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.570 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:48.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.049406+0000", 2024-09-14T23:56:48.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8794999999999997e-05 2024-09-14T23:56:48.571 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.571 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:48.571 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.571 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "pg_update_log_missing(3.1d epoch 29/28 rep_tid 39 entries 29'11 (0'0) error 3:b98f80e3:::rbd_data.1118de1168b.0000000000000067:head by client.4376.0:3676 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-14T23:56:48.571 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.061591+0000", 2024-09-14T23:56:48.571 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.4703272819999995, 2024-09-14T23:56:48.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001346643, 2024-09-14T23:56:48.571 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:48.571 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:48.571 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:56:48.572 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:48.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:48.572 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.061591+0000", 2024-09-14T23:56:48.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:48.572 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.061591+0000", 2024-09-14T23:56:48.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:48.572 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.061592+0000", 2024-09-14T23:56:48.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7700000000000004e-07 2024-09-14T23:56:48.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.573 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:48.573 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.061603+0000", 2024-09-14T23:56:48.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1148000000000001e-05 2024-09-14T23:56:48.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.573 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:48.573 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.061624+0000", 2024-09-14T23:56:48.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0930000000000001e-05 2024-09-14T23:56:48.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.574 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:48.574 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.061639+0000", 2024-09-14T23:56:48.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5194000000000001e-05 2024-09-14T23:56:48.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.574 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:48.574 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.061758+0000", 2024-09-14T23:56:48.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000119164 2024-09-14T23:56:48.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.574 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:48.574 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.062938+0000", 2024-09-14T23:56:48.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00117953 2024-09-14T23:56:48.574 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.574 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:48.575 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.575 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:3678 3.c e29/28 3:376a6b1b:::rbd_data.1118de1168b.0000000000000061:head v 29'9, mlcod=29'7)", 2024-09-14T23:56:48.575 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.061695+0000", 2024-09-14T23:56:48.575 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.4702233830000004, 2024-09-14T23:56:48.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001292236, 2024-09-14T23:56:48.575 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:48.575 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:48.575 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:48.575 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:48.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:48.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.061695+0000", 2024-09-14T23:56:48.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:48.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.061695+0000", 2024-09-14T23:56:48.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:48.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.061696+0000", 2024-09-14T23:56:48.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3799999999999996e-07 2024-09-14T23:56:48.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.577 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:48.577 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.061703+0000", 2024-09-14T23:56:48.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8229999999999997e-06 2024-09-14T23:56:48.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.577 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:48.577 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.061713+0000", 2024-09-14T23:56:48.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9979999999999998e-06 2024-09-14T23:56:48.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.577 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:48.577 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.061719+0000", 2024-09-14T23:56:48.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4479999999999999e-06 2024-09-14T23:56:48.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:48.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.061800+0000", 2024-09-14T23:56:48.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1107999999999997e-05 2024-09-14T23:56:48.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:48.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.061821+0000", 2024-09-14T23:56:48.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1104999999999999e-05 2024-09-14T23:56:48.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:48.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.062930+0000", 2024-09-14T23:56:48.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0011091149999999999 2024-09-14T23:56:48.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:48.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.062987+0000", 2024-09-14T23:56:48.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6901000000000003e-05 2024-09-14T23:56:48.579 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.579 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:48.579 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.579 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:4073 3.16 e29/28 3:6a953de9:::rbd_data.1118de1168b.000000000000006d:head v 29'7, mlcod=27'5)", 2024-09-14T23:56:48.579 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.080983+0000", 2024-09-14T23:56:48.579 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.450935458, 2024-09-14T23:56:48.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001444802, 2024-09-14T23:56:48.580 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:48.580 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:48.580 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:48.580 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:48.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:48.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.080983+0000", 2024-09-14T23:56:48.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:48.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.080983+0000", 2024-09-14T23:56:48.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:48.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.080984+0000", 2024-09-14T23:56:48.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4e-07 2024-09-14T23:56:48.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:48.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.081057+0000", 2024-09-14T23:56:48.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3271999999999999e-05 2024-09-14T23:56:48.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:48.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.081069+0000", 2024-09-14T23:56:48.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2319e-05 2024-09-14T23:56:48.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:48.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.081077+0000", 2024-09-14T23:56:48.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1910000000000003e-06 2024-09-14T23:56:48.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:48.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.081135+0000", 2024-09-14T23:56:48.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8686e-05 2024-09-14T23:56:48.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:48.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.081158+0000", 2024-09-14T23:56:48.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2600999999999999e-05 2024-09-14T23:56:48.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:48.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.082374+0000", 2024-09-14T23:56:48.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001216373 2024-09-14T23:56:48.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:48.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.082428+0000", 2024-09-14T23:56:48.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3520000000000003e-05 2024-09-14T23:56:48.583 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.583 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:48.584 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.584 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:5575 3.17 e29/23 3:ee5ec205:::rbd_data.1118de1168b.00000000000000a2:head v 29'36, mlcod=29'32)", 2024-09-14T23:56:48.584 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.349366+0000", 2024-09-14T23:56:48.584 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.1825525409999997, 2024-09-14T23:56:48.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0013367349999999999, 2024-09-14T23:56:48.584 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:48.584 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:48.584 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:48.585 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:48.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:48.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.349366+0000", 2024-09-14T23:56:48.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:48.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.349366+0000", 2024-09-14T23:56:48.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:48.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.349366+0000", 2024-09-14T23:56:48.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5699999999999998e-07 2024-09-14T23:56:48.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:48.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.349376+0000", 2024-09-14T23:56:48.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9389999999999992e-06 2024-09-14T23:56:48.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:48.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.349387+0000", 2024-09-14T23:56:48.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0417999999999999e-05 2024-09-14T23:56:48.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:48.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.349393+0000", 2024-09-14T23:56:48.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0789999999999997e-06 2024-09-14T23:56:48.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:48.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.349815+0000", 2024-09-14T23:56:48.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00042267999999999998 2024-09-14T23:56:48.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:48.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.349838+0000", 2024-09-14T23:56:48.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2846000000000001e-05 2024-09-14T23:56:48.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:48.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.350654+0000", 2024-09-14T23:56:48.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00081540099999999999 2024-09-14T23:56:48.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:48.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.350703+0000", 2024-09-14T23:56:48.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8915000000000002e-05 2024-09-14T23:56:48.589 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.589 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:48.589 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.589 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:6256 3.1d e29/28 3:bc58c580:::rbd_data.1118de1168b.0000000000000169:head v 29'35, mlcod=29'32)", 2024-09-14T23:56:48.589 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.483347+0000", 2024-09-14T23:56:48.589 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.0485718310000003, 2024-09-14T23:56:48.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001330398, 2024-09-14T23:56:48.589 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:48.589 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:48.589 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:48.589 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:48.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:48.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.483347+0000", 2024-09-14T23:56:48.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:48.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.483347+0000", 2024-09-14T23:56:48.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:48.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.483347+0000", 2024-09-14T23:56:48.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1399999999999996e-07 2024-09-14T23:56:48.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:48.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.483353+0000", 2024-09-14T23:56:48.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5679999999999999e-06 2024-09-14T23:56:48.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:48.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.483363+0000", 2024-09-14T23:56:48.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0285999999999999e-05 2024-09-14T23:56:48.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:48.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.483370+0000", 2024-09-14T23:56:48.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4899999999999997e-06 2024-09-14T23:56:48.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:48.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.483429+0000", 2024-09-14T23:56:48.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9441999999999998e-05 2024-09-14T23:56:48.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:48.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.483448+0000", 2024-09-14T23:56:48.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9428000000000001e-05 2024-09-14T23:56:48.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:48.593 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.484632+0000", 2024-09-14T23:56:48.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0011830510000000001 2024-09-14T23:56:48.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:48.593 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.484677+0000", 2024-09-14T23:56:48.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5419e-05 2024-09-14T23:56:48.593 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.593 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:48.593 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.594 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:17405 3.2 e33/23 3:450461f9:::rbd_data.11187be34cc8.000000000000000c:head v 33'56, mlcod=32'53)", 2024-09-14T23:56:48.594 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:43.185550+0000", 2024-09-14T23:56:48.594 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.3463683819999996, 2024-09-14T23:56:48.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0020455920000000002, 2024-09-14T23:56:48.594 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:48.594 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:48.594 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:48.594 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:48.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.594 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:48.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:43.185550+0000", 2024-09-14T23:56:48.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:48.595 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:43.185550+0000", 2024-09-14T23:56:48.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:48.595 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:43.185551+0000", 2024-09-14T23:56:48.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4000000000000001e-07 2024-09-14T23:56:48.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:48.595 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:43.186697+0000", 2024-09-14T23:56:48.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001146701 2024-09-14T23:56:48.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.596 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:48.596 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:43.186708+0000", 2024-09-14T23:56:48.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0261999999999999e-05 2024-09-14T23:56:48.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.596 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:48.596 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:43.186716+0000", 2024-09-14T23:56:48.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1289999999999996e-06 2024-09-14T23:56:48.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.596 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:48.596 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:43.186767+0000", 2024-09-14T23:56:48.597 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1477999999999997e-05 2024-09-14T23:56:48.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.597 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:48.597 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:43.186787+0000", 2024-09-14T23:56:48.597 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9850000000000001e-05 2024-09-14T23:56:48.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.597 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:48.597 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:43.187556+0000", 2024-09-14T23:56:48.597 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00076922299999999998 2024-09-14T23:56:48.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.598 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:48.598 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:43.187596+0000", 2024-09-14T23:56:48.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9308999999999999e-05 2024-09-14T23:56:48.598 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.598 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:48.598 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.598 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:18193 3.17 e34/23 3:e951801c:::rbd_data.11187be34cc8.000000000000000a:head v 34'69, mlcod=33'67)", 2024-09-14T23:56:48.598 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:44.011272+0000", 2024-09-14T23:56:48.598 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.5206466030000003, 2024-09-14T23:56:48.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0014004499999999999, 2024-09-14T23:56:48.598 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:48.598 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:48.599 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:48.599 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:48.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.599 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:48.599 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.011272+0000", 2024-09-14T23:56:48.599 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.599 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:48.599 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.011272+0000", 2024-09-14T23:56:48.599 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.599 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:48.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.011272+0000", 2024-09-14T23:56:48.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9600000000000001e-07 2024-09-14T23:56:48.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.600 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:48.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.011277+0000", 2024-09-14T23:56:48.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1610000000000002e-06 2024-09-14T23:56:48.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.600 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:48.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.011293+0000", 2024-09-14T23:56:48.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6268000000000001e-05 2024-09-14T23:56:48.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:48.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.011300+0000", 2024-09-14T23:56:48.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5420000000000002e-06 2024-09-14T23:56:48.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:48.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.011356+0000", 2024-09-14T23:56:48.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6159999999999998e-05 2024-09-14T23:56:48.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:48.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.011375+0000", 2024-09-14T23:56:48.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.893e-05 2024-09-14T23:56:48.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.602 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:48.602 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.012631+0000", 2024-09-14T23:56:48.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0012563839999999999 2024-09-14T23:56:48.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.602 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:48.602 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.012672+0000", 2024-09-14T23:56:48.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0809000000000002e-05 2024-09-14T23:56:48.602 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.602 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:48.602 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.603 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:18190 3.1f e34/23 3:f852abec:::rbd_data.11187be34cc8.0000000000000007:head v 34'75, mlcod=33'73)", 2024-09-14T23:56:48.603 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:44.011316+0000", 2024-09-14T23:56:48.603 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.5206025150000002, 2024-09-14T23:56:48.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0013126030000000001, 2024-09-14T23:56:48.603 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:48.603 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:48.603 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:48.603 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:48.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:48.603 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.011316+0000", 2024-09-14T23:56:48.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:48.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.011316+0000", 2024-09-14T23:56:48.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:48.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.011316+0000", 2024-09-14T23:56:48.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.72e-07 2024-09-14T23:56:48.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:48.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.011324+0000", 2024-09-14T23:56:48.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0660000000000004e-06 2024-09-14T23:56:48.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:48.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.011329+0000", 2024-09-14T23:56:48.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.853e-06 2024-09-14T23:56:48.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:48.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.011336+0000", 2024-09-14T23:56:48.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8260000000000004e-06 2024-09-14T23:56:48.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:48.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.011384+0000", 2024-09-14T23:56:48.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8233999999999999e-05 2024-09-14T23:56:48.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.606 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:48.606 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.011398+0000", 2024-09-14T23:56:48.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3699999999999999e-05 2024-09-14T23:56:48.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.606 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:48.606 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.012588+0000", 2024-09-14T23:56:48.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0011898639999999999 2024-09-14T23:56:48.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.606 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:48.607 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.012628+0000", 2024-09-14T23:56:48.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0788000000000003e-05 2024-09-14T23:56:48.607 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.607 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:48.607 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.607 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:18263 3.5 e34/28 3:a40e472c:::rbd_data.11187be34cc8.0000000000000004:head v 34'65, mlcod=34'63)", 2024-09-14T23:56:48.607 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:44.113544+0000", 2024-09-14T23:56:48.607 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.4183741850000002, 2024-09-14T23:56:48.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0014541949999999999, 2024-09-14T23:56:48.607 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:48.607 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:48.608 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:48.608 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:48.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:48.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.113544+0000", 2024-09-14T23:56:48.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:48.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.113544+0000", 2024-09-14T23:56:48.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:48.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.113545+0000", 2024-09-14T23:56:48.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6000000000000003e-07 2024-09-14T23:56:48.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:48.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.113551+0000", 2024-09-14T23:56:48.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1140000000000002e-06 2024-09-14T23:56:48.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:48.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.113565+0000", 2024-09-14T23:56:48.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4165000000000001e-05 2024-09-14T23:56:48.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:48.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.113572+0000", 2024-09-14T23:56:48.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1269999999999997e-06 2024-09-14T23:56:48.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:48.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.113635+0000", 2024-09-14T23:56:48.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2545999999999994e-05 2024-09-14T23:56:48.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:48.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.113652+0000", 2024-09-14T23:56:48.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7577000000000001e-05 2024-09-14T23:56:48.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.611 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:48.611 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.114957+0000", 2024-09-14T23:56:48.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001304204 2024-09-14T23:56:48.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.611 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:48.611 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.114998+0000", 2024-09-14T23:56:48.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1702000000000002e-05 2024-09-14T23:56:48.611 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.611 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:48.611 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.611 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:18261 3.1f e34/23 3:f953ed77:::rbd_data.11187be34cc8.0000000000000002:head v 34'76, mlcod=33'74)", 2024-09-14T23:56:48.612 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:44.113618+0000", 2024-09-14T23:56:48.612 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.4183002309999999, 2024-09-14T23:56:48.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001402845, 2024-09-14T23:56:48.612 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:48.612 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:48.612 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:48.612 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:48.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.612 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:48.612 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.113618+0000", 2024-09-14T23:56:48.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:48.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.113618+0000", 2024-09-14T23:56:48.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:48.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.113619+0000", 2024-09-14T23:56:48.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3900000000000005e-07 2024-09-14T23:56:48.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:48.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.113623+0000", 2024-09-14T23:56:48.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3839999999999999e-06 2024-09-14T23:56:48.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.614 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:48.614 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.113633+0000", 2024-09-14T23:56:48.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4890000000000001e-06 2024-09-14T23:56:48.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.614 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:48.614 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.113639+0000", 2024-09-14T23:56:48.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0780000000000001e-06 2024-09-14T23:56:48.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.614 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:48.614 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.113698+0000", 2024-09-14T23:56:48.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9131e-05 2024-09-14T23:56:48.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.614 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:48.615 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.113716+0000", 2024-09-14T23:56:48.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8328000000000001e-05 2024-09-14T23:56:48.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.615 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:48.615 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.114954+0000", 2024-09-14T23:56:48.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001237812 2024-09-14T23:56:48.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.615 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:48.615 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.115021+0000", 2024-09-14T23:56:48.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7083999999999997e-05 2024-09-14T23:56:48.615 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.615 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:48.616 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.616 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:27076 3.1d e38/28 3:bd4c1528:::rbd_data.111814b482f8.0000000000000096:head v 38'89, mlcod=38'87)", 2024-09-14T23:56:48.616 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.317778+0000", 2024-09-14T23:56:48.616 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.21414054799999999, 2024-09-14T23:56:48.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001300638, 2024-09-14T23:56:48.616 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:48.616 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:48.616 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:48.616 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:48.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:48.616 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.317778+0000", 2024-09-14T23:56:48.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:48.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.317778+0000", 2024-09-14T23:56:48.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:48.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.317778+0000", 2024-09-14T23:56:48.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3200000000000005e-07 2024-09-14T23:56:48.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:48.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.317960+0000", 2024-09-14T23:56:48.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018179000000000001 2024-09-14T23:56:48.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:48.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.317973+0000", 2024-09-14T23:56:48.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2796000000000001e-05 2024-09-14T23:56:48.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:48.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.317981+0000", 2024-09-14T23:56:48.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2150000000000001e-06 2024-09-14T23:56:48.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:48.619 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.318036+0000", 2024-09-14T23:56:48.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4975000000000001e-05 2024-09-14T23:56:48.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.619 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:48.619 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.318056+0000", 2024-09-14T23:56:48.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9429999999999999e-05 2024-09-14T23:56:48.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.619 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:48.619 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.319033+0000", 2024-09-14T23:56:48.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00097723199999999997 2024-09-14T23:56:48.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:48.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.319078+0000", 2024-09-14T23:56:48.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5568000000000001e-05 2024-09-14T23:56:48.620 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.620 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:48.620 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.620 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:27077 3.16 e38/28 3:6be8b54e:::rbd_data.111814b482f8.0000000000000098:head v 38'107, mlcod=37'104)", 2024-09-14T23:56:48.620 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.317964+0000", 2024-09-14T23:56:48.620 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.213954226, 2024-09-14T23:56:48.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016474440000000001, 2024-09-14T23:56:48.620 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:48.621 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:48.621 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:48.621 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:48.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:48.621 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.317964+0000", 2024-09-14T23:56:48.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:48.621 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.317964+0000", 2024-09-14T23:56:48.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:48.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.317965+0000", 2024-09-14T23:56:48.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7899999999999998e-07 2024-09-14T23:56:48.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:48.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.318170+0000", 2024-09-14T23:56:48.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000205582 2024-09-14T23:56:48.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:48.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.318183+0000", 2024-09-14T23:56:48.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2286999999999999e-05 2024-09-14T23:56:48.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:48.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.318190+0000", 2024-09-14T23:56:48.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3590000000000002e-06 2024-09-14T23:56:48.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:48.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.318270+0000", 2024-09-14T23:56:48.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0136999999999995e-05 2024-09-14T23:56:48.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:48.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.318290+0000", 2024-09-14T23:56:48.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9933e-05 2024-09-14T23:56:48.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.624 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:48.624 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.319575+0000", 2024-09-14T23:56:48.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001285097 2024-09-14T23:56:48.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.624 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:48.624 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.319612+0000", 2024-09-14T23:56:48.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6369999999999999e-05 2024-09-14T23:56:48.624 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.624 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:48.624 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.625 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:28182 3.1f e38/23 3:fd5b853f:::rbd_data.111814b482f8.000000000000007e:head v 38'98, mlcod=38'96)", 2024-09-14T23:56:48.625 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.449767+0000", 2024-09-14T23:56:48.625 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.082151102000000004, 2024-09-14T23:56:48.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001458155, 2024-09-14T23:56:48.625 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:48.625 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:48.625 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:48.625 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:48.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.625 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:48.625 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.449767+0000", 2024-09-14T23:56:48.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:48.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.449767+0000", 2024-09-14T23:56:48.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:48.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.449768+0000", 2024-09-14T23:56:48.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.082e-06 2024-09-14T23:56:48.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:48.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.450032+0000", 2024-09-14T23:56:48.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00026392700000000001 2024-09-14T23:56:48.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:48.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.450043+0000", 2024-09-14T23:56:48.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1153e-05 2024-09-14T23:56:48.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:48.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.450050+0000", 2024-09-14T23:56:48.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7660000000000001e-06 2024-09-14T23:56:48.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:48.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.450104+0000", 2024-09-14T23:56:48.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3443999999999999e-05 2024-09-14T23:56:48.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:48.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.450120+0000", 2024-09-14T23:56:48.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6177000000000001e-05 2024-09-14T23:56:48.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:48.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.451162+0000", 2024-09-14T23:56:48.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001042081 2024-09-14T23:56:48.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:48.629 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.451225+0000", 2024-09-14T23:56:48.629 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3524999999999999e-05 2024-09-14T23:56:48.629 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.629 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:48.629 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.629 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:28230 3.2 e38/23 3:461ef2b6:::rbd_data.111814b482f8.000000000000028c:head v 38'86, mlcod=38'84)", 2024-09-14T23:56:48.629 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.482806+0000", 2024-09-14T23:56:48.629 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.049112308, 2024-09-14T23:56:48.629 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0014455169999999999, 2024-09-14T23:56:48.629 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:48.629 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:48.629 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:48.630 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:48.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:48.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.482806+0000", 2024-09-14T23:56:48.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:48.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.482806+0000", 2024-09-14T23:56:48.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:48.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:48.631 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.482806+0000", 2024-09-14T23:56:48.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9900000000000001e-07 2024-09-14T23:56:48.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.631 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:48.631 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.482819+0000", 2024-09-14T23:56:48.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3019e-05 2024-09-14T23:56:48.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.631 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:48.631 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.482827+0000", 2024-09-14T23:56:48.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9540000000000004e-06 2024-09-14T23:56:48.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.632 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:48.632 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.482833+0000", 2024-09-14T23:56:48.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8579999999999996e-06 2024-09-14T23:56:48.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.632 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:48.632 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.482885+0000", 2024-09-14T23:56:48.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1549999999999999e-05 2024-09-14T23:56:48.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.632 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:48.632 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.482905+0000", 2024-09-14T23:56:48.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0319999999999999e-05 2024-09-14T23:56:48.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.633 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:48.633 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.484209+0000", 2024-09-14T23:56:48.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001304073 2024-09-14T23:56:48.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:48.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:48.633 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:48.633 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.484252+0000", 2024-09-14T23:56:48.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2344e-05 2024-09-14T23:56:48.633 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.633 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:48.633 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.633 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:48.634 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:48.634 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:56:48.636 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T23:56:48.637 INFO:teuthology.orchestra.run.smithi042.stdout:956 writesame 0xc7f1545 thru 0xc800889 data_size 0x1(0xf345 bytes) 2024-09-14T23:56:48.638 INFO:teuthology.orchestra.run.smithi042.stdout:957 punch from 0x7cca26 to 0x7d10bb, (0x4695 bytes) 2024-09-14T23:56:48.638 INFO:teuthology.orchestra.run.smithi042.stdout:959 writesame 0x4832089 thru 0x484206e data_size 0x5(0xffe6 bytes) 2024-09-14T23:56:48.638 INFO:teuthology.orchestra.run.smithi042.stdout:960 read 0x8e933a6 thru 0x8e966ee (0x3349 bytes) 2024-09-14T23:56:48.638 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:48.638 INFO:teuthology.orchestra.run.smithi042.stdout:962 read 0x70c55c5 thru 0x70d2690 (0xd0cc bytes) 2024-09-14T23:56:48.638 INFO:teuthology.orchestra.run.smithi042.stdout:963 write 0x8aa180c thru 0x8aac643 (0xae38 bytes) 2024-09-14T23:56:48.638 INFO:teuthology.orchestra.run.smithi042.stdout:964 punch from 0xce8f2 to 0xd9b3e, (0xb24c bytes) 2024-09-14T23:56:48.638 INFO:teuthology.orchestra.run.smithi042.stdout:966 write 0x69de2c4 thru 0x69e7b54 (0x9891 bytes) 2024-09-14T23:56:48.638 INFO:teuthology.orchestra.run.smithi042.stdout:968 read 0xc668ec2 thru 0xc66e716 (0x5855 bytes) 2024-09-14T23:56:48.638 INFO:teuthology.orchestra.run.smithi042.stdout:969 write 0x4228833 thru 0x42374ed (0xecbb bytes) 2024-09-14T23:56:48.638 INFO:teuthology.orchestra.run.smithi042.stdout:971 writesame 0x4314daf thru 0x4321d56 data_size 0x3c(0xcfa8 bytes) 2024-09-14T23:56:48.638 INFO:teuthology.orchestra.run.smithi042.stdout:972 read 0xbe285fe thru 0xbe2aacf (0x24d2 bytes) 2024-09-14T23:56:48.638 INFO:teuthology.orchestra.run.smithi042.stdout:974 punch from 0xc19121 to 0xc1f09b, (0x5f7a bytes) 2024-09-14T23:56:48.638 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:48.639 INFO:teuthology.orchestra.run.smithi042.stdout:979 read 0xa9479de thru 0xa953532 (0xbb55 bytes) 2024-09-14T23:56:48.639 INFO:teuthology.orchestra.run.smithi042.stdout:980 read 0xb65058d thru 0xb659969 (0x93dd bytes) 2024-09-14T23:56:48.639 INFO:teuthology.orchestra.run.smithi042.stdout:981 read 0x893e07e thru 0x8942e49 (0x4dcc bytes) 2024-09-14T23:56:48.639 INFO:teuthology.orchestra.run.smithi042.stdout:983 trunc from 0xc80088a to 0x86b4911 2024-09-14T23:56:48.639 INFO:teuthology.orchestra.run.smithi042.stdout:984 read 0x6b6720a thru 0x6b68c2c (0x1a23 bytes) 2024-09-14T23:56:48.639 INFO:teuthology.orchestra.run.smithi042.stdout:985 writesame 0xdbf1fcf thru 0xdbfbd72 data_size 0x4c(0x9da4 bytes) 2024-09-14T23:56:48.639 INFO:teuthology.orchestra.run.smithi042.stdout:986 writesame 0x22a0660 thru 0x22a3038 data_size 0x3(0x29d9 bytes) 2024-09-14T23:56:48.639 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:56:48.636+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:56:48.646 INFO:teuthology.orchestra.run.smithi042.stdout:987 punch from 0xd5dd3dd to 0xd5e6b7c, (0x979f bytes) 2024-09-14T23:56:48.657 INFO:teuthology.orchestra.run.smithi042.stdout:988 trunc from 0xdbfbd73 to 0x3c2597e 2024-09-14T23:56:48.679 INFO:teuthology.orchestra.run.smithi042.stdout:989 read 0x32fa10a thru 0x3304c70 (0xab67 bytes) 2024-09-14T23:56:48.695 INFO:teuthology.orchestra.run.smithi042.stdout:990 write 0xbe4ce70 thru 0xbe587cf (0xb960 bytes) 2024-09-14T23:56:48.710 INFO:teuthology.orchestra.run.smithi042.stdout:991 read 0x170459c thru 0x17141ab (0xfc10 bytes) 2024-09-14T23:56:48.713 INFO:teuthology.orchestra.run.smithi042.stdout:992 write 0x6be6681 thru 0x6be94c5 (0x2e45 bytes) 2024-09-14T23:56:48.719 INFO:teuthology.orchestra.run.smithi042.stdout:995 write 0xa19b423 thru 0xa1a195c (0x653a bytes) 2024-09-14T23:56:48.727 INFO:teuthology.orchestra.run.smithi042.stdout:996 write 0x569980f thru 0x569e2ef (0x4ae1 bytes) 2024-09-14T23:56:48.734 INFO:teuthology.orchestra.run.smithi042.stdout:997 read 0x5a3b37 thru 0x5b129b (0xd765 bytes) 2024-09-14T23:56:48.735 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:56:48.734+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:56:48.742 INFO:teuthology.orchestra.run.smithi042.stdout:998 trunc from 0xbe587d0 to 0x792bcc5 2024-09-14T23:56:48.755 INFO:teuthology.orchestra.run.smithi042.stdout:1000 write 0x6df4bce thru 0x6dfe72d (0x9b60 bytes) 2024-09-14T23:56:48.768 INFO:teuthology.orchestra.run.smithi042.stdout:1002 read 0x8b8fa7 thru 0x8bda05 (0x4a5f bytes) 2024-09-14T23:56:48.774 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:56:48.774 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:56:48.774 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:56:48.774 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:56:48.783 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T23:56:48.784 INFO:teuthology.orchestra.run.smithi042.stdout:1005 write 0xe7dbd3f thru 0xe7e41f2 (0x84b4 bytes) 2024-09-14T23:56:48.795 INFO:teuthology.orchestra.run.smithi042.stdout:1006 read 0x4a55a90 thru 0x4a5eb12 (0x9083 bytes) 2024-09-14T23:56:48.795 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:48.796 INFO:teuthology.orchestra.run.smithi042.stdout:1009 write 0x28823e5 thru 0x288ae6b (0x8a87 bytes) 2024-09-14T23:56:48.817 INFO:teuthology.orchestra.run.smithi042.stdout:1010 punch from 0xd7b64c to 0xd85839, (0xa1ed bytes) 2024-09-14T23:56:48.817 INFO:teuthology.orchestra.run.smithi042.stdout:1012 punch from 0xb8f995d to 0xb900328, (0x69cb bytes) 2024-09-14T23:56:48.817 INFO:teuthology.orchestra.run.smithi042.stdout:1013 read 0x3aafe21 thru 0x3ab1748 (0x1928 bytes) 2024-09-14T23:56:48.817 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:48.823 INFO:teuthology.orchestra.run.smithi042.stdout:1015 trunc from 0xe7e41f3 to 0x9232769 2024-09-14T23:56:48.836 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:56:48.834+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:56:48.844 INFO:teuthology.orchestra.run.smithi042.stdout:1017 write 0xc4f0fe6 thru 0xc4f378c (0x27a7 bytes) 2024-09-14T23:56:48.849 INFO:teuthology.orchestra.run.smithi042.stdout:1019 write 0x1cd6662 thru 0x1cd7f6b (0x190a bytes) 2024-09-14T23:56:48.860 INFO:teuthology.orchestra.run.smithi042.stdout:1020 write 0x5fa199 thru 0x6061dc (0xc044 bytes) 2024-09-14T23:56:48.904 INFO:teuthology.orchestra.run.smithi042.stdout:1021 read 0x1c978e2 thru 0x1ca54ea (0xdc09 bytes) 2024-09-14T23:56:48.906 INFO:teuthology.orchestra.run.smithi042.stdout:1022 write 0x7153d98 thru 0x715c740 (0x89a9 bytes) 2024-09-14T23:56:48.920 INFO:teuthology.orchestra.run.smithi042.stdout:1023 trunc from 0xc4f378d to 0xa9c306f 2024-09-14T23:56:48.927 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:56:48.928 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:56:48.928 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:56:48.928 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:56:48.928 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:56:48.937 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T23:56:48.938 INFO:teuthology.orchestra.run.smithi042.stdout:1026 trunc from 0xa9c306f to 0x6143db9 2024-09-14T23:56:48.938 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:56:48.935+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:56:48.950 INFO:teuthology.orchestra.run.smithi042.stdout:1029 read 0x4ade08c thru 0x4aedde5 (0xfd5a bytes) 2024-09-14T23:56:48.965 INFO:teuthology.orchestra.run.smithi042.stdout:1030 write 0xdd83362 thru 0xdd87e2c (0x4acb bytes) 2024-09-14T23:56:48.973 INFO:teuthology.orchestra.run.smithi042.stdout:1031 write 0xa34263f thru 0xa3433dc (0xd9e bytes) 2024-09-14T23:56:48.976 INFO:teuthology.orchestra.run.smithi042.stdout:1032 read 0xbb952f thru 0xbc5a38 (0xc50a bytes) 2024-09-14T23:56:48.978 INFO:teuthology.orchestra.run.smithi042.stdout:1033 read 0x8acb6ae thru 0x8ad3173 (0x7ac6 bytes) 2024-09-14T23:56:48.978 INFO:teuthology.orchestra.run.smithi042.stdout:1034 read 0xc5d1ad7 thru 0xc5d336c (0x1896 bytes) 2024-09-14T23:56:48.978 INFO:teuthology.orchestra.run.smithi042.stdout:1035 read 0x556b5b4 thru 0x5570f86 (0x59d3 bytes) 2024-09-14T23:56:48.978 INFO:teuthology.orchestra.run.smithi042.stdout:1036 read 0x3dcc770 thru 0x3dcff0d (0x379e bytes) 2024-09-14T23:56:48.987 INFO:teuthology.orchestra.run.smithi042.stdout:1038 trunc from 0xdd87e2d to 0x6529419 2024-09-14T23:56:48.999 INFO:teuthology.orchestra.run.smithi042.stdout:1039 read 0x2de7906 thru 0x2df35c7 (0xbcc2 bytes) 2024-09-14T23:56:49.001 INFO:teuthology.orchestra.run.smithi042.stdout:1040 read 0x37e2f51 thru 0x37e74c2 (0x4572 bytes) 2024-09-14T23:56:49.001 INFO:teuthology.orchestra.run.smithi042.stdout:1042 flatten 2024-09-14T23:56:49.015 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T23:56:49.015 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T23:56:49.016 INFO:tasks.thrashosds.thrasher:Removing osd 5, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T23:56:49.016 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 5 2024-09-14T23:56:49.022 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is unset 2024-09-14T23:56:49.037 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:56:49.036+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:56:49.086 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:56:49.086 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:56:49.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:56:49.087 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:56:49.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.087 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:18218 3.10 3:0c729a68:::rbd_data.11187be34cc8.0000000000000017:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-14T23:56:49.087 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:44.032604+0000", 2024-09-14T23:56:49.087 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.0508861600000001, 2024-09-14T23:56:49.087 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.028735283, 2024-09-14T23:56:49.087 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:49.087 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:49.087 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:49.087 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:49.087 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:49.087 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:49.087 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 18218 2024-09-14T23:56:49.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.088 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:49.088 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.088 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:49.088 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.032604+0000", 2024-09-14T23:56:49.088 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:49.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.088 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.088 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:49.088 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.032604+0000", 2024-09-14T23:56:49.089 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:49.089 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.089 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.089 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:49.089 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.032606+0000", 2024-09-14T23:56:49.089 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7999999999999999e-06 2024-09-14T23:56:49.089 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.089 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.089 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:49.089 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.032614+0000", 2024-09-14T23:56:49.089 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1810000000000001e-06 2024-09-14T23:56:49.089 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.089 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.090 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:49.090 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.032631+0000", 2024-09-14T23:56:49.090 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7363e-05 2024-09-14T23:56:49.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.090 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.090 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:49.090 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.032638+0000", 2024-09-14T23:56:49.090 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0430000000000002e-06 2024-09-14T23:56:49.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.090 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.090 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:49.090 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.032708+0000", 2024-09-14T23:56:49.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0257000000000002e-05 2024-09-14T23:56:49.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-14T23:56:49.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.032713+0000", 2024-09-14T23:56:49.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4190000000000004e-06 2024-09-14T23:56:49.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:49.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.034010+0000", 2024-09-14T23:56:49.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001297407 2024-09-14T23:56:49.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.092 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:49.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.034059+0000", 2024-09-14T23:56:49.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8912999999999997e-05 2024-09-14T23:56:49.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.092 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.092 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:56:49.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.038995+0000", 2024-09-14T23:56:49.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0049356010000000004 2024-09-14T23:56:49.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.092 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.092 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:49.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.061258+0000", 2024-09-14T23:56:49.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.022262963 2024-09-14T23:56:49.093 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.093 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.093 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:56:49.093 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.061269+0000", 2024-09-14T23:56:49.093 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1671000000000001e-05 2024-09-14T23:56:49.093 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.093 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.093 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:49.093 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.061306+0000", 2024-09-14T23:56:49.093 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6474999999999999e-05 2024-09-14T23:56:49.093 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.093 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.093 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:49.093 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.061339+0000", 2024-09-14T23:56:49.094 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3189999999999999e-05 2024-09-14T23:56:49.094 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:49.094 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:49.094 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:49.094 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.094 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.094 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30289 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 585728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:49.094 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.865354+0000", 2024-09-14T23:56:49.094 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.21813633299999999, 2024-09-14T23:56:49.094 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015192881, 2024-09-14T23:56:49.094 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:49.094 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:49.094 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:49.094 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:49.094 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:49.095 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:49.095 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30289 2024-09-14T23:56:49.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.095 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:49.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:49.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865354+0000", 2024-09-14T23:56:49.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:49.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:49.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865354+0000", 2024-09-14T23:56:49.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:49.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.096 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:49.096 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865354+0000", 2024-09-14T23:56:49.096 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8500000000000001e-07 2024-09-14T23:56:49.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.096 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:49.096 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865357+0000", 2024-09-14T23:56:49.096 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6199999999999999e-06 2024-09-14T23:56:49.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.096 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:49.096 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865364+0000", 2024-09-14T23:56:49.096 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0840000000000003e-06 2024-09-14T23:56:49.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.097 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.097 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:49.097 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865369+0000", 2024-09-14T23:56:49.097 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3709999999999999e-06 2024-09-14T23:56:49.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.097 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.097 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:49.097 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.880378+0000", 2024-09-14T23:56:49.097 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015009043 2024-09-14T23:56:49.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.097 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.097 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:49.097 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.880451+0000", 2024-09-14T23:56:49.098 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2747e-05 2024-09-14T23:56:49.098 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.098 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.098 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:49.098 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.880546+0000", 2024-09-14T23:56:49.098 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5431000000000002e-05 2024-09-14T23:56:49.098 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:49.098 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:49.099 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:49.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.099 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30298 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 438272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:49.099 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.865377+0000", 2024-09-14T23:56:49.099 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.21811286499999999, 2024-09-14T23:56:49.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015369668, 2024-09-14T23:56:49.099 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:49.099 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:49.099 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:49.100 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:49.100 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:49.100 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:49.100 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30298 2024-09-14T23:56:49.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.100 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:49.100 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.100 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:49.100 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865377+0000", 2024-09-14T23:56:49.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:49.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.100 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.100 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:49.101 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865377+0000", 2024-09-14T23:56:49.101 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:49.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.101 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:49.101 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865377+0000", 2024-09-14T23:56:49.101 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9499999999999998e-07 2024-09-14T23:56:49.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.101 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:49.101 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865381+0000", 2024-09-14T23:56:49.101 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.371e-06 2024-09-14T23:56:49.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.102 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:49.102 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865389+0000", 2024-09-14T23:56:49.102 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3469999999999999e-06 2024-09-14T23:56:49.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.102 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.102 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:49.103 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865394+0000", 2024-09-14T23:56:49.103 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9869999999999999e-06 2024-09-14T23:56:49.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.103 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.103 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:49.103 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.880577+0000", 2024-09-14T23:56:49.104 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015183103 2024-09-14T23:56:49.104 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.104 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.104 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:49.104 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.880649+0000", 2024-09-14T23:56:49.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1989999999999993e-05 2024-09-14T23:56:49.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.105 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:49.105 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.880747+0000", 2024-09-14T23:56:49.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7474999999999998e-05 2024-09-14T23:56:49.105 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:49.105 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:49.105 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:49.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.106 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30302 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 364544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:49.106 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.865401+0000", 2024-09-14T23:56:49.106 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.218088961, 2024-09-14T23:56:49.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015547874999999999, 2024-09-14T23:56:49.106 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:49.106 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:49.106 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:49.106 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:49.106 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:49.106 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:49.106 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30302 2024-09-14T23:56:49.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.106 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:49.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.107 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:49.107 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865401+0000", 2024-09-14T23:56:49.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:49.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.107 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:49.107 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865401+0000", 2024-09-14T23:56:49.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:49.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.107 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:49.107 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865402+0000", 2024-09-14T23:56:49.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4499999999999997e-07 2024-09-14T23:56:49.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:49.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865404+0000", 2024-09-14T23:56:49.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9240000000000001e-06 2024-09-14T23:56:49.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:49.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865410+0000", 2024-09-14T23:56:49.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3570000000000003e-06 2024-09-14T23:56:49.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:49.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865415+0000", 2024-09-14T23:56:49.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.553e-06 2024-09-14T23:56:49.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:49.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.880779+0000", 2024-09-14T23:56:49.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015363411 2024-09-14T23:56:49.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:49.110 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.880852+0000", 2024-09-14T23:56:49.110 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2916000000000005e-05 2024-09-14T23:56:49.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.110 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.110 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:49.110 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.880949+0000", 2024-09-14T23:56:49.110 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7069000000000002e-05 2024-09-14T23:56:49.110 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:49.110 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:49.110 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:49.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.110 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.110 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30308 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 294912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:49.110 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.865537+0000", 2024-09-14T23:56:49.111 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.21795334999999999, 2024-09-14T23:56:49.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015615162, 2024-09-14T23:56:49.111 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:49.111 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:49.111 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:49.111 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:49.111 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:49.111 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:49.111 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30308 2024-09-14T23:56:49.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.111 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:49.111 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:49.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865537+0000", 2024-09-14T23:56:49.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:49.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:49.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865537+0000", 2024-09-14T23:56:49.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:49.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:49.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865537+0000", 2024-09-14T23:56:49.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0299999999999999e-07 2024-09-14T23:56:49.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.113 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:49.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865540+0000", 2024-09-14T23:56:49.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8420000000000001e-06 2024-09-14T23:56:49.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.113 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:49.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865548+0000", 2024-09-14T23:56:49.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2230000000000007e-06 2024-09-14T23:56:49.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.113 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:49.114 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865555+0000", 2024-09-14T23:56:49.114 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7780000000000002e-06 2024-09-14T23:56:49.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.114 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:49.114 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.880982+0000", 2024-09-14T23:56:49.114 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015426832 2024-09-14T23:56:49.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.114 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:49.114 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.881059+0000", 2024-09-14T23:56:49.114 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7095999999999995e-05 2024-09-14T23:56:49.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.115 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:49.115 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.881152+0000", 2024-09-14T23:56:49.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2788000000000007e-05 2024-09-14T23:56:49.115 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:49.115 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:49.115 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:49.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.115 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30371 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 442368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:49.115 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.866343+0000", 2024-09-14T23:56:49.115 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.21714687199999999, 2024-09-14T23:56:49.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015209663, 2024-09-14T23:56:49.116 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:49.116 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:49.116 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:49.116 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:49.116 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:49.116 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:49.116 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30371 2024-09-14T23:56:49.116 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.116 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:49.116 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.116 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:49.116 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866343+0000", 2024-09-14T23:56:49.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:49.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:49.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866343+0000", 2024-09-14T23:56:49.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:49.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:49.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866344+0000", 2024-09-14T23:56:49.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8199999999999999e-07 2024-09-14T23:56:49.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:49.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866347+0000", 2024-09-14T23:56:49.118 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4759999999999998e-06 2024-09-14T23:56:49.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.118 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.118 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:49.118 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866358+0000", 2024-09-14T23:56:49.118 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0353999999999999e-05 2024-09-14T23:56:49.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.118 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.118 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:49.118 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866364+0000", 2024-09-14T23:56:49.118 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3260000000000001e-06 2024-09-14T23:56:49.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.118 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.118 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:49.119 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.881388+0000", 2024-09-14T23:56:49.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015023791999999999 2024-09-14T23:56:49.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.119 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:49.119 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.881460+0000", 2024-09-14T23:56:49.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2509999999999995e-05 2024-09-14T23:56:49.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.120 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.120 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:49.120 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.881553+0000", 2024-09-14T23:56:49.120 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2423000000000006e-05 2024-09-14T23:56:49.120 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:49.120 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:49.120 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:49.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.120 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.120 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30374 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 368640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:49.120 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.866381+0000", 2024-09-14T23:56:49.121 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.21710845100000001, 2024-09-14T23:56:49.121 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015382209000000001, 2024-09-14T23:56:49.121 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:49.121 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:49.121 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:49.121 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:49.121 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:49.121 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:49.121 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30374 2024-09-14T23:56:49.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.121 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:49.121 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.121 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:49.121 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866381+0000", 2024-09-14T23:56:49.122 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:49.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.122 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.122 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:49.122 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866381+0000", 2024-09-14T23:56:49.122 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:49.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.122 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.122 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:49.122 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866382+0000", 2024-09-14T23:56:49.122 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9800000000000003e-07 2024-09-14T23:56:49.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.122 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.122 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:49.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866385+0000", 2024-09-14T23:56:49.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1599999999999998e-06 2024-09-14T23:56:49.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.123 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.123 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:49.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866393+0000", 2024-09-14T23:56:49.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7929999999999998e-06 2024-09-14T23:56:49.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.123 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.123 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:49.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866399+0000", 2024-09-14T23:56:49.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5260000000000001e-06 2024-09-14T23:56:49.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.123 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.124 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:49.124 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.881585+0000", 2024-09-14T23:56:49.124 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.01518624 2024-09-14T23:56:49.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.124 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:49.124 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.881661+0000", 2024-09-14T23:56:49.124 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5779999999999996e-05 2024-09-14T23:56:49.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.124 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:49.124 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.881764+0000", 2024-09-14T23:56:49.124 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000103012 2024-09-14T23:56:49.124 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:49.125 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:49.125 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:49.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.125 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.125 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30379 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 299008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:49.125 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.866461+0000", 2024-09-14T23:56:49.125 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.21702894, 2024-09-14T23:56:49.125 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015514637, 2024-09-14T23:56:49.125 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:49.125 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:49.126 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:49.126 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:49.126 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:49.126 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:49.126 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30379 2024-09-14T23:56:49.126 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.126 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:49.126 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.126 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:49.126 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866461+0000", 2024-09-14T23:56:49.126 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:49.126 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.126 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.127 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:49.127 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866461+0000", 2024-09-14T23:56:49.127 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:49.127 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.127 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.127 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:49.127 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866462+0000", 2024-09-14T23:56:49.127 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5700000000000002e-07 2024-09-14T23:56:49.127 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.127 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.127 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:49.127 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866466+0000", 2024-09-14T23:56:49.128 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6759999999999998e-06 2024-09-14T23:56:49.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.128 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.128 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:49.128 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866477+0000", 2024-09-14T23:56:49.128 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0094e-05 2024-09-14T23:56:49.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.128 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.128 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:49.128 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866483+0000", 2024-09-14T23:56:49.128 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1460000000000001e-06 2024-09-14T23:56:49.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.128 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.128 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:49.129 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.881800+0000", 2024-09-14T23:56:49.129 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015317769 2024-09-14T23:56:49.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.129 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.129 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:49.129 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.881875+0000", 2024-09-14T23:56:49.129 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4726000000000002e-05 2024-09-14T23:56:49.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.129 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.129 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:49.129 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.881976+0000", 2024-09-14T23:56:49.129 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100469 2024-09-14T23:56:49.129 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:49.130 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:49.130 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:49.130 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.130 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.130 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30396 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 532480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:49.130 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.866569+0000", 2024-09-14T23:56:49.130 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.21692051700000001, 2024-09-14T23:56:49.130 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015617786, 2024-09-14T23:56:49.130 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:49.131 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:49.131 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:49.131 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:49.131 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:49.131 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:49.131 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30396 2024-09-14T23:56:49.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.131 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:49.131 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.131 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:49.131 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866569+0000", 2024-09-14T23:56:49.131 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:49.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.131 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.132 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:49.132 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866569+0000", 2024-09-14T23:56:49.132 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:49.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.132 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.132 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:49.132 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866570+0000", 2024-09-14T23:56:49.132 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4000000000000001e-07 2024-09-14T23:56:49.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.132 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.132 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:49.132 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866574+0000", 2024-09-14T23:56:49.132 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7330000000000001e-06 2024-09-14T23:56:49.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.133 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:49.133 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866584+0000", 2024-09-14T23:56:49.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0052e-05 2024-09-14T23:56:49.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.133 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:49.133 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866591+0000", 2024-09-14T23:56:49.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4359999999999996e-06 2024-09-14T23:56:49.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:49.134 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.882011+0000", 2024-09-14T23:56:49.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015419871999999999 2024-09-14T23:56:49.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:49.134 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.882088+0000", 2024-09-14T23:56:49.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7118000000000003e-05 2024-09-14T23:56:49.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:49.134 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.882187+0000", 2024-09-14T23:56:49.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8834999999999998e-05 2024-09-14T23:56:49.135 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:49.135 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:49.135 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:49.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.135 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30405 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 446464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:49.135 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.866670+0000", 2024-09-14T23:56:49.135 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.21681974500000001, 2024-09-14T23:56:49.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015777653999999999, 2024-09-14T23:56:49.135 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:49.135 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:49.135 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:49.136 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:49.136 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:49.136 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:49.136 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30405 2024-09-14T23:56:49.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.136 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:49.136 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.136 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:49.136 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866670+0000", 2024-09-14T23:56:49.136 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:49.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.136 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.136 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:49.137 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866670+0000", 2024-09-14T23:56:49.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:49.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:49.137 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866671+0000", 2024-09-14T23:56:49.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3900000000000004e-07 2024-09-14T23:56:49.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:49.137 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866675+0000", 2024-09-14T23:56:49.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8569999999999997e-06 2024-09-14T23:56:49.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:49.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866685+0000", 2024-09-14T23:56:49.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0002e-05 2024-09-14T23:56:49.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:49.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866692+0000", 2024-09-14T23:56:49.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.691e-06 2024-09-14T23:56:49.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:49.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.882259+0000", 2024-09-14T23:56:49.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015567399000000001 2024-09-14T23:56:49.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.139 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:49.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.882342+0000", 2024-09-14T23:56:49.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2919999999999999e-05 2024-09-14T23:56:49.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.139 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:49.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.882448+0000", 2024-09-14T23:56:49.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010594600000000001 2024-09-14T23:56:49.139 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:49.139 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:49.140 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:49.140 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.140 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.140 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30407 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 372736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:49.140 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.866702+0000", 2024-09-14T23:56:49.140 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.21678752600000001, 2024-09-14T23:56:49.140 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015922993, 2024-09-14T23:56:49.140 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:49.140 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:49.141 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:49.141 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:49.141 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:49.141 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:49.141 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30407 2024-09-14T23:56:49.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.141 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:49.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.141 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:49.142 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866702+0000", 2024-09-14T23:56:49.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:49.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.142 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:49.142 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866702+0000", 2024-09-14T23:56:49.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:49.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.142 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:49.142 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866703+0000", 2024-09-14T23:56:49.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5799999999999999e-07 2024-09-14T23:56:49.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:49.143 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866706+0000", 2024-09-14T23:56:49.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8890000000000002e-06 2024-09-14T23:56:49.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:49.143 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866713+0000", 2024-09-14T23:56:49.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.99e-06 2024-09-14T23:56:49.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:49.143 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866718+0000", 2024-09-14T23:56:49.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.254e-06 2024-09-14T23:56:49.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.144 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.144 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:49.144 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.882497+0000", 2024-09-14T23:56:49.144 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015778664000000001 2024-09-14T23:56:49.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.144 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.144 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:49.144 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.882557+0000", 2024-09-14T23:56:49.144 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0408999999999997e-05 2024-09-14T23:56:49.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.144 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.144 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:49.145 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.882625+0000", 2024-09-14T23:56:49.145 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8229e-05 2024-09-14T23:56:49.145 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:49.145 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:49.145 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:49.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.145 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.145 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30415 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 303104~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:49.145 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.866775+0000", 2024-09-14T23:56:49.145 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.21671510799999999, 2024-09-14T23:56:49.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.016119712000000001, 2024-09-14T23:56:49.146 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:49.146 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:49.146 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:49.146 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:49.146 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:49.146 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:49.146 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30415 2024-09-14T23:56:49.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.146 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:49.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.146 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:49.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866775+0000", 2024-09-14T23:56:49.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:49.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.147 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:49.147 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866775+0000", 2024-09-14T23:56:49.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:49.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.147 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:49.147 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866776+0000", 2024-09-14T23:56:49.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5499999999999997e-07 2024-09-14T23:56:49.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.147 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:49.147 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866779+0000", 2024-09-14T23:56:49.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.613e-06 2024-09-14T23:56:49.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.148 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.148 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:49.148 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866789+0000", 2024-09-14T23:56:49.148 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5030000000000003e-06 2024-09-14T23:56:49.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.148 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.148 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:49.148 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866799+0000", 2024-09-14T23:56:49.148 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9170000000000001e-06 2024-09-14T23:56:49.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.148 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.148 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:49.148 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.882704+0000", 2024-09-14T23:56:49.149 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015905333000000001 2024-09-14T23:56:49.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.149 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:49.149 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.882787+0000", 2024-09-14T23:56:49.149 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3264999999999996e-05 2024-09-14T23:56:49.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.149 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:49.149 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.882895+0000", 2024-09-14T23:56:49.149 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010722600000000001 2024-09-14T23:56:49.149 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:49.149 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:49.149 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:49.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.150 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30430 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 536576~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:49.150 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.867043+0000", 2024-09-14T23:56:49.150 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.21644714200000001, 2024-09-14T23:56:49.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.016054556000000001, 2024-09-14T23:56:49.150 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:49.150 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:49.150 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:49.150 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:49.150 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:49.150 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:49.150 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30430 2024-09-14T23:56:49.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.150 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:49.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:49.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867043+0000", 2024-09-14T23:56:49.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:49.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:49.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867043+0000", 2024-09-14T23:56:49.152 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:49.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.152 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:49.152 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867044+0000", 2024-09-14T23:56:49.152 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.23e-07 2024-09-14T23:56:49.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.152 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:49.152 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867053+0000", 2024-09-14T23:56:49.152 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7759999999999992e-06 2024-09-14T23:56:49.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.153 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:49.153 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867066+0000", 2024-09-14T23:56:49.153 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2376e-05 2024-09-14T23:56:49.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.153 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.153 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:49.153 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867075+0000", 2024-09-14T23:56:49.153 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1349999999999998e-06 2024-09-14T23:56:49.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.153 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.153 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:49.153 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.882925+0000", 2024-09-14T23:56:49.153 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015850566 2024-09-14T23:56:49.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.154 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.154 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:49.154 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.883000+0000", 2024-09-14T23:56:49.154 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4746999999999994e-05 2024-09-14T23:56:49.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.154 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.154 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:49.154 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.883097+0000", 2024-09-14T23:56:49.154 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7133000000000007e-05 2024-09-14T23:56:49.154 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:49.154 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:49.154 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:49.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.154 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.155 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30440 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 450560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:49.155 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.867174+0000", 2024-09-14T23:56:49.155 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.216315699, 2024-09-14T23:56:49.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.016169586999999999, 2024-09-14T23:56:49.155 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:49.155 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:49.155 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:49.155 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:49.155 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:49.155 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:49.155 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30440 2024-09-14T23:56:49.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.155 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:49.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.156 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:49.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867174+0000", 2024-09-14T23:56:49.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:49.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.156 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:49.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867174+0000", 2024-09-14T23:56:49.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:49.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.156 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:49.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867175+0000", 2024-09-14T23:56:49.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1100000000000005e-07 2024-09-14T23:56:49.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.157 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.157 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:49.157 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867178+0000", 2024-09-14T23:56:49.157 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.994e-06 2024-09-14T23:56:49.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.157 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.157 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:49.157 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867190+0000", 2024-09-14T23:56:49.157 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2072000000000001e-05 2024-09-14T23:56:49.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.157 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.157 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:49.157 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867196+0000", 2024-09-14T23:56:49.158 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1340000000000001e-06 2024-09-14T23:56:49.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.158 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.158 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:49.158 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.883165+0000", 2024-09-14T23:56:49.158 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015969061 2024-09-14T23:56:49.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.158 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.158 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:49.158 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.883240+0000", 2024-09-14T23:56:49.158 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4616999999999994e-05 2024-09-14T23:56:49.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.158 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.158 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:49.159 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.883344+0000", 2024-09-14T23:56:49.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000103898 2024-09-14T23:56:49.159 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:49.159 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:49.159 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:49.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.159 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30444 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 376832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:49.159 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.867242+0000", 2024-09-14T23:56:49.159 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.216248263, 2024-09-14T23:56:49.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.016313708999999999, 2024-09-14T23:56:49.159 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:49.159 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:49.160 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:49.160 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:49.160 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:49.160 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:49.160 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30444 2024-09-14T23:56:49.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.160 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:49.160 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.160 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:49.160 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867242+0000", 2024-09-14T23:56:49.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:49.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.160 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.161 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:49.161 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867242+0000", 2024-09-14T23:56:49.161 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:49.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.161 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.161 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:49.161 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867242+0000", 2024-09-14T23:56:49.161 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0100000000000005e-07 2024-09-14T23:56:49.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.162 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.162 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:49.162 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867248+0000", 2024-09-14T23:56:49.162 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0100000000000001e-06 2024-09-14T23:56:49.162 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.162 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.162 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:49.162 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867260+0000", 2024-09-14T23:56:49.162 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2179e-05 2024-09-14T23:56:49.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.163 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:49.163 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867267+0000", 2024-09-14T23:56:49.163 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7780000000000002e-06 2024-09-14T23:56:49.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.163 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:49.163 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.883374+0000", 2024-09-14T23:56:49.163 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.016106466999999999 2024-09-14T23:56:49.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.163 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:49.163 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.883449+0000", 2024-09-14T23:56:49.164 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5475000000000005e-05 2024-09-14T23:56:49.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.164 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.164 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:49.164 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.883555+0000", 2024-09-14T23:56:49.164 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000106199 2024-09-14T23:56:49.164 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:49.164 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:49.165 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:49.165 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.165 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.165 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30451 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 307200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:49.165 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.867309+0000", 2024-09-14T23:56:49.165 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.21618053300000001, 2024-09-14T23:56:49.166 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.016427048, 2024-09-14T23:56:49.166 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:49.166 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:49.166 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:49.166 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:49.166 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:49.166 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:49.166 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30451 2024-09-14T23:56:49.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.167 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:49.167 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.167 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:49.167 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867309+0000", 2024-09-14T23:56:49.167 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:49.167 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.167 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.167 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:49.167 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867309+0000", 2024-09-14T23:56:49.167 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:49.167 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.167 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.167 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:49.167 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867310+0000", 2024-09-14T23:56:49.168 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5799999999999999e-07 2024-09-14T23:56:49.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.168 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.168 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:49.168 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867316+0000", 2024-09-14T23:56:49.168 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4650000000000002e-06 2024-09-14T23:56:49.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.168 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.168 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:49.168 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867325+0000", 2024-09-14T23:56:49.168 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0659999999999993e-06 2024-09-14T23:56:49.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.168 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.169 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:49.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867331+0000", 2024-09-14T23:56:49.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2419999999999997e-06 2024-09-14T23:56:49.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.169 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:49.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.883585+0000", 2024-09-14T23:56:49.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.016254599000000002 2024-09-14T23:56:49.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.169 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:49.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.883659+0000", 2024-09-14T23:56:49.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3697999999999998e-05 2024-09-14T23:56:49.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.170 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.170 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:49.170 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.883736+0000", 2024-09-14T23:56:49.170 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7319999999999998e-05 2024-09-14T23:56:49.170 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:49.170 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:49.170 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:49.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.170 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.170 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30463 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 540672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:49.170 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.867871+0000", 2024-09-14T23:56:49.170 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.21561919800000001, 2024-09-14T23:56:49.171 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.016001520000000002, 2024-09-14T23:56:49.171 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:49.171 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:49.171 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:49.171 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:49.171 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:49.171 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:49.171 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30463 2024-09-14T23:56:49.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.171 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:49.171 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.171 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:49.172 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867871+0000", 2024-09-14T23:56:49.172 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:49.172 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.172 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.172 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:49.172 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867871+0000", 2024-09-14T23:56:49.172 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:49.172 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.173 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.173 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:49.173 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867872+0000", 2024-09-14T23:56:49.173 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9800000000000002e-07 2024-09-14T23:56:49.173 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.173 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.173 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:49.173 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867876+0000", 2024-09-14T23:56:49.173 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.318e-06 2024-09-14T23:56:49.173 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.174 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.174 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:49.174 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867887+0000", 2024-09-14T23:56:49.174 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1055e-05 2024-09-14T23:56:49.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.174 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.174 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:49.174 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868679+0000", 2024-09-14T23:56:49.174 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00079199000000000001 2024-09-14T23:56:49.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.174 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.174 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:49.174 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.883763+0000", 2024-09-14T23:56:49.175 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015083675 2024-09-14T23:56:49.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.175 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.175 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:49.175 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.883816+0000", 2024-09-14T23:56:49.175 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3757e-05 2024-09-14T23:56:49.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.175 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.175 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:49.175 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.883872+0000", 2024-09-14T23:56:49.175 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5826999999999999e-05 2024-09-14T23:56:49.175 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:49.175 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:49.175 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:49.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.176 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.176 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30479 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:49.176 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.868703+0000", 2024-09-14T23:56:49.176 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.214787279, 2024-09-14T23:56:49.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015351293, 2024-09-14T23:56:49.176 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:49.176 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:49.176 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:49.176 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:49.176 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:49.176 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:49.176 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30479 2024-09-14T23:56:49.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.176 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:49.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.177 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:49.177 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868703+0000", 2024-09-14T23:56:49.177 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:49.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.177 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:49.177 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868703+0000", 2024-09-14T23:56:49.177 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:49.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.177 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:49.177 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868703+0000", 2024-09-14T23:56:49.177 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3300000000000001e-07 2024-09-14T23:56:49.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.178 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:49.178 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868708+0000", 2024-09-14T23:56:49.178 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.279e-06 2024-09-14T23:56:49.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.178 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:49.178 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868717+0000", 2024-09-14T23:56:49.178 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0469999999999999e-06 2024-09-14T23:56:49.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.178 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:49.178 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868723+0000", 2024-09-14T23:56:49.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.64e-06 2024-09-14T23:56:49.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.179 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.179 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:49.179 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.883898+0000", 2024-09-14T23:56:49.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015175016 2024-09-14T23:56:49.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.179 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.179 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:49.179 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.883959+0000", 2024-09-14T23:56:49.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0729000000000001e-05 2024-09-14T23:56:49.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.180 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.180 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:49.180 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.884054+0000", 2024-09-14T23:56:49.180 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4848999999999999e-05 2024-09-14T23:56:49.180 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:49.180 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:49.180 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:49.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.180 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.180 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30480 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 454656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:49.180 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.868733+0000", 2024-09-14T23:56:49.180 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.214756696, 2024-09-14T23:56:49.181 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015505417000000001, 2024-09-14T23:56:49.181 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:49.181 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:49.181 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:49.181 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:49.181 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:49.181 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:49.181 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30480 2024-09-14T23:56:49.181 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.181 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:49.181 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.181 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:49.181 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868733+0000", 2024-09-14T23:56:49.182 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:49.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.182 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.182 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:49.182 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868733+0000", 2024-09-14T23:56:49.182 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:49.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.182 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.182 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:49.182 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868734+0000", 2024-09-14T23:56:49.182 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3099999999999998e-07 2024-09-14T23:56:49.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.182 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.182 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:49.183 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868744+0000", 2024-09-14T23:56:49.183 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0429e-05 2024-09-14T23:56:49.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.183 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.183 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:49.183 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868754+0000", 2024-09-14T23:56:49.183 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5400000000000001e-06 2024-09-14T23:56:49.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.183 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.184 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:49.184 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868760+0000", 2024-09-14T23:56:49.184 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0170000000000002e-06 2024-09-14T23:56:49.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.184 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:49.184 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.884084+0000", 2024-09-14T23:56:49.184 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015324071 2024-09-14T23:56:49.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.184 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:49.184 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.884151+0000", 2024-09-14T23:56:49.184 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7531000000000001e-05 2024-09-14T23:56:49.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.185 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.185 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:49.185 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.884239+0000", 2024-09-14T23:56:49.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7398000000000006e-05 2024-09-14T23:56:49.185 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:49.185 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:49.185 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:49.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.185 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.185 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30483 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 380928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:49.185 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.868769+0000", 2024-09-14T23:56:49.185 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.214721148, 2024-09-14T23:56:49.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015654879, 2024-09-14T23:56:49.186 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:49.186 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:49.186 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:49.186 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:49.186 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:49.186 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:49.186 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30483 2024-09-14T23:56:49.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.186 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:49.186 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.186 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:49.186 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868769+0000", 2024-09-14T23:56:49.186 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:49.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.187 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.187 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:49.187 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868769+0000", 2024-09-14T23:56:49.187 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:49.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.187 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.187 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:49.187 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868769+0000", 2024-09-14T23:56:49.187 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8299999999999997e-07 2024-09-14T23:56:49.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.187 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.187 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:49.187 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868772+0000", 2024-09-14T23:56:49.187 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.497e-06 2024-09-14T23:56:49.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.188 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.188 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:49.188 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868778+0000", 2024-09-14T23:56:49.188 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6919999999999996e-06 2024-09-14T23:56:49.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.188 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.188 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:49.188 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868784+0000", 2024-09-14T23:56:49.188 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5280000000000003e-06 2024-09-14T23:56:49.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.188 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.188 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:49.188 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.884264+0000", 2024-09-14T23:56:49.189 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015479629 2024-09-14T23:56:49.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.189 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.189 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:49.189 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.884332+0000", 2024-09-14T23:56:49.189 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8617999999999999e-05 2024-09-14T23:56:49.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:49.189 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:49.189 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:49.189 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.884424+0000", 2024-09-14T23:56:49.189 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1432000000000001e-05 2024-09-14T23:56:49.189 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:49.189 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:49.190 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:49.190 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:49.190 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:49.190 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:56:49.191 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T23:56:49.191 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:56:49.190+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:56:49.258 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T23:56:49.302 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:56:49.301+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:56:49.377 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:56:49.377 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:56:49.377 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:56:49.377 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:56:49.386 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T23:56:49.392 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:56:49.391+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:56:49.449 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:56:49.449 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:56:49.493 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:56:49.492+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:56:49.500 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:56:49.500 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:56:49.521 INFO:tasks.ceph.mon.a.smithi042.stderr:2024-09-14T23:56:49.519+0000 7f1713676640 -1 mon.a@0(leader).osd e39 definitely_dead 0 2024-09-14T23:56:49.532 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:56:49.533 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:56:49.536 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:56:49.536 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:56:49.536 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:56:49.536 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:56:49.536 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:56:49.549 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T23:56:49.564 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:56:49.564 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:56:49.591 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:56:49.591 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:56:49.593 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:56:49.591+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:56:49.612 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:56:49.612 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:56:49.622 INFO:teuthology.orchestra.run.smithi042.stdout:1044 write 0xad4ec57 thru 0xad539bb (0x4d65 bytes) 2024-09-14T23:56:49.630 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:56:49.630 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:56:49.630 INFO:teuthology.orchestra.run.smithi042.stdout:1046 read 0x18fe4ba thru 0x1907ab5 (0x95fc bytes) 2024-09-14T23:56:49.631 INFO:teuthology.orchestra.run.smithi042.stdout:1049 writesame 0x4338bf6 thru 0x433ce88 data_size 0x13(0x4293 bytes) 2024-09-14T23:56:49.639 INFO:teuthology.orchestra.run.smithi042.stdout:1051 read 0x453c79f thru 0x454b7a6 (0xf008 bytes) 2024-09-14T23:56:49.639 INFO:teuthology.orchestra.run.smithi042.stdout:1052 punch from 0x9441d66 to 0x944fccc, (0xdf66 bytes) 2024-09-14T23:56:49.639 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:49.639 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:49.639 INFO:teuthology.orchestra.run.smithi042.stdout:1056 write 0x7afee1b thru 0x7b0a3d1 (0xb5b7 bytes) 2024-09-14T23:56:49.651 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:56:49.651 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:56:49.655 INFO:teuthology.orchestra.run.smithi042.stdout:1057 read 0x2f1effa thru 0x2f20197 (0x119e bytes) 2024-09-14T23:56:49.655 INFO:teuthology.orchestra.run.smithi042.stdout:1058 write 0x9d5bb2d thru 0x9d6333a (0x780e bytes) 2024-09-14T23:56:49.666 INFO:teuthology.orchestra.run.smithi042.stdout:1061 write 0x3c41f4 thru 0x3cd533 (0x9340 bytes) 2024-09-14T23:56:49.669 INFO:teuthology.orchestra.run.smithi042.stdout:1063 read 0x15f29e8 thru 0x15fc3a6 (0x99bf bytes) 2024-09-14T23:56:49.669 INFO:teuthology.orchestra.run.smithi042.stdout:1064 clone 9 order 18 su 65536 sc 3 2024-09-14T23:56:49.694 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:56:49.692+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:56:49.715 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:49.715 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:49.723 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T23:56:49.795 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:56:49.794+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:56:49.831 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:49.831 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:49.840 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T23:56:49.896 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:56:49.894+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:56:49.947 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:49.947 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:49.956 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T23:56:49.997 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:56:49.996+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:56:50.022 INFO:teuthology.orchestra.run.smithi042.stderr:marked out osd.5. 2024-09-14T23:56:50.097 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:56:50.096+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:56:50.108 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:50.108 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:50.117 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T23:56:50.198 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:56:50.197+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:56:50.225 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:50.225 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:50.234 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T23:56:50.300 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:56:50.299+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:56:50.382 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:50.382 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:50.390 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T23:56:50.434 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:56:50.433+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:56:50.498 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:50.498 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:50.502 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:56:50.500+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:56:50.507 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T23:56:50.603 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:56:50.602+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:56:50.654 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:50.654 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:50.663 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T23:56:50.704 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:56:50.702+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:56:50.771 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:50.771 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:50.780 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T23:56:50.804 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:56:50.803+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:56:50.887 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:50.887 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:50.896 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T23:56:50.906 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:56:50.904+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:56:51.003 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:51.003 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:51.006 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:56:51.005+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:56:51.012 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T23:56:51.043 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T23:56:51.107 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:56:51.106+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:56:51.144 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:51.144 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:51.153 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T23:56:51.208 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:56:51.207+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:56:51.262 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:51.263 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:51.272 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T23:56:51.316 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:56:51.315+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:56:51.386 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:51.387 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:51.395 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T23:56:51.436 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:56:51.435+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:56:51.510 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:56:51.509+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:56:51.544 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:51.545 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:51.552 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T23:56:51.612 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:56:51.611+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:56:51.663 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T23:56:51.700 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:51.700 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:51.709 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T23:56:51.712 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:56:51.711+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:56:51.789 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:56:51.790 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:56:51.806 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:56:51.806 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:56:51.813 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:56:51.811+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:56:51.817 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:51.817 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:51.823 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:56:51.823 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:56:51.826 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T23:56:51.839 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:56:51.839 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:56:51.856 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:56:51.856 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:56:51.873 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:56:51.873 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:56:51.889 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:56:51.890 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:56:51.907 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:56:51.907 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:56:51.913 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:56:51.912+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:56:51.933 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:56:51.933 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:56:51.933 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:56:51.933 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:56:51.933 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:56:51.942 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T23:56:52.014 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:56:52.013+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:56:52.024 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is unset 2024-09-14T23:56:52.053 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:56:52.053 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:56:52.053 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:56:52.053 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:56:52.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.053 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4284.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-14T23:56:52.053 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:56:52.053 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 29.526544214000001, 2024-09-14T23:56:52.054 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.004942086, 2024-09-14T23:56:52.054 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:52.054 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:52.054 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:52.054 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:52.054 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4284", 2024-09-14T23:56:52.054 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/4278347926", 2024-09-14T23:56:52.054 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1 2024-09-14T23:56:52.054 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.054 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:52.054 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.054 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:52.054 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:56:52.055 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.055 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.055 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.055 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:52.055 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:56:52.055 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.055 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.055 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.055 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:52.055 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524331+0000", 2024-09-14T23:56:52.055 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.383e-05 2024-09-14T23:56:52.055 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.055 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.055 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:52.056 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524339+0000", 2024-09-14T23:56:52.056 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8290000000000008e-06 2024-09-14T23:56:52.056 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.056 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.056 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:52.056 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524359+0000", 2024-09-14T23:56:52.056 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9998000000000001e-05 2024-09-14T23:56:52.056 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.056 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.056 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:52.056 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524373+0000", 2024-09-14T23:56:52.056 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4304000000000001e-05 2024-09-14T23:56:52.056 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.056 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.057 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:52.057 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524488+0000", 2024-09-14T23:56:52.057 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114663 2024-09-14T23:56:52.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.057 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.057 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:56:52.057 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524501+0000", 2024-09-14T23:56:52.057 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3349999999999999e-05 2024-09-14T23:56:52.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.057 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.057 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:52.057 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.526868+0000", 2024-09-14T23:56:52.057 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023670090000000002 2024-09-14T23:56:52.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.058 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.058 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:52.058 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.527041+0000", 2024-09-14T23:56:52.058 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000172597 2024-09-14T23:56:52.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.058 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.058 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-14T23:56:52.058 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.527384+0000", 2024-09-14T23:56:52.058 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00034309500000000002 2024-09-14T23:56:52.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.058 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.058 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:52.058 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529103+0000", 2024-09-14T23:56:52.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017185220000000001 2024-09-14T23:56:52.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.059 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:56:52.059 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529188+0000", 2024-09-14T23:56:52.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5591000000000001e-05 2024-09-14T23:56:52.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.059 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:52.059 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529227+0000", 2024-09-14T23:56:52.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8563000000000003e-05 2024-09-14T23:56:52.060 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.060 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.060 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:52.060 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529259+0000", 2024-09-14T23:56:52.060 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2734999999999997e-05 2024-09-14T23:56:52.060 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.060 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:52.060 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.060 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.060 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.060 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4284.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-14T23:56:52.060 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:23.530683+0000", 2024-09-14T23:56:52.061 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 28.520178549000001, 2024-09-14T23:56:52.061 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0036133110000000001, 2024-09-14T23:56:52.061 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:52.061 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:52.061 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:52.061 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:52.061 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4284", 2024-09-14T23:56:52.061 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/4278347926", 2024-09-14T23:56:52.061 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5 2024-09-14T23:56:52.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.061 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:52.061 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.061 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:52.061 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530683+0000", 2024-09-14T23:56:52.062 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.062 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.062 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.062 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:52.062 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530683+0000", 2024-09-14T23:56:52.062 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.062 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.062 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.062 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:52.062 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530685+0000", 2024-09-14T23:56:52.062 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.075e-06 2024-09-14T23:56:52.062 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.062 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.062 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:52.062 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530695+0000", 2024-09-14T23:56:52.063 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0261999999999999e-05 2024-09-14T23:56:52.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:52.063 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530715+0000", 2024-09-14T23:56:52.063 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9930000000000001e-05 2024-09-14T23:56:52.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:52.063 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530724+0000", 2024-09-14T23:56:52.063 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.901e-06 2024-09-14T23:56:52.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:52.063 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530800+0000", 2024-09-14T23:56:52.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5873999999999999e-05 2024-09-14T23:56:52.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-14T23:56:52.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530807+0000", 2024-09-14T23:56:52.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2389999999999996e-06 2024-09-14T23:56:52.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:52.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.532494+0000", 2024-09-14T23:56:52.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016871830000000001 2024-09-14T23:56:52.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:52.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.532599+0000", 2024-09-14T23:56:52.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010472 2024-09-14T23:56:52.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.065 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.065 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1", 2024-09-14T23:56:52.065 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.532715+0000", 2024-09-14T23:56:52.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011568199999999999 2024-09-14T23:56:52.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.065 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.065 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:56:52.065 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.533460+0000", 2024-09-14T23:56:52.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00074494999999999997 2024-09-14T23:56:52.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.065 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.066 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:52.066 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534230+0000", 2024-09-14T23:56:52.066 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00077036599999999998 2024-09-14T23:56:52.066 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.066 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.066 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:52.066 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534266+0000", 2024-09-14T23:56:52.066 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5725000000000001e-05 2024-09-14T23:56:52.066 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.066 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.066 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:52.066 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534296+0000", 2024-09-14T23:56:52.066 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0403999999999999e-05 2024-09-14T23:56:52.066 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.066 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:52.067 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.067 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.067 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4284.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-14T23:56:52.067 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:24.535160+0000", 2024-09-14T23:56:52.067 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.515701908, 2024-09-14T23:56:52.067 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033473650000000001, 2024-09-14T23:56:52.067 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:52.067 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:52.067 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:52.067 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:52.067 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4284", 2024-09-14T23:56:52.067 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/4278347926", 2024-09-14T23:56:52.067 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 7 2024-09-14T23:56:52.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.068 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:52.068 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.068 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:52.068 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535160+0000", 2024-09-14T23:56:52.068 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.068 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.068 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:52.068 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535160+0000", 2024-09-14T23:56:52.068 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.068 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.068 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:52.068 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535162+0000", 2024-09-14T23:56:52.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3369999999999998e-06 2024-09-14T23:56:52.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.069 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.069 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:52.069 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535171+0000", 2024-09-14T23:56:52.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7700000000000007e-06 2024-09-14T23:56:52.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.069 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.069 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:52.069 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535189+0000", 2024-09-14T23:56:52.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7938e-05 2024-09-14T23:56:52.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.069 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.069 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:52.069 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535197+0000", 2024-09-14T23:56:52.070 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5639999999999997e-06 2024-09-14T23:56:52.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.070 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:52.070 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535266+0000", 2024-09-14T23:56:52.070 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8926999999999999e-05 2024-09-14T23:56:52.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.070 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-14T23:56:52.070 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535271+0000", 2024-09-14T23:56:52.070 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0329999999999999e-06 2024-09-14T23:56:52.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.070 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:52.071 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.537064+0000", 2024-09-14T23:56:52.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001792929 2024-09-14T23:56:52.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.071 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.071 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:52.071 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.537131+0000", 2024-09-14T23:56:52.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6605000000000004e-05 2024-09-14T23:56:52.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.071 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.071 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1", 2024-09-14T23:56:52.071 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.537271+0000", 2024-09-14T23:56:52.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013992400000000001 2024-09-14T23:56:52.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.072 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.072 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:56:52.072 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.538093+0000", 2024-09-14T23:56:52.072 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00082210200000000001 2024-09-14T23:56:52.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.072 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.072 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:52.072 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.538439+0000", 2024-09-14T23:56:52.072 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00034595999999999997 2024-09-14T23:56:52.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.072 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.072 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:52.072 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.538476+0000", 2024-09-14T23:56:52.072 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6770000000000002e-05 2024-09-14T23:56:52.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.073 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.073 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:52.073 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.538507+0000", 2024-09-14T23:56:52.073 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1506e-05 2024-09-14T23:56:52.073 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.073 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:52.073 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.073 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.073 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T23:56:52.073 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:34.406877+0000", 2024-09-14T23:56:52.073 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 17.643984903, 2024-09-14T23:56:52.073 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030351169999999999, 2024-09-14T23:56:52.074 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:52.074 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:52.074 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:52.074 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:52.074 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:52.074 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:52.074 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 10 2024-09-14T23:56:52.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.074 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:52.074 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.074 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:52.074 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406877+0000", 2024-09-14T23:56:52.074 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.074 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.075 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:52.075 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406877+0000", 2024-09-14T23:56:52.075 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.075 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.075 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:52.075 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406878+0000", 2024-09-14T23:56:52.075 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.291e-06 2024-09-14T23:56:52.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.075 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.075 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:52.075 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406883+0000", 2024-09-14T23:56:52.075 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2399999999999998e-06 2024-09-14T23:56:52.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.075 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.076 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:52.076 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406891+0000", 2024-09-14T23:56:52.076 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5889999999999997e-06 2024-09-14T23:56:52.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.076 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.076 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:52.076 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406898+0000", 2024-09-14T23:56:52.076 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3599999999999998e-06 2024-09-14T23:56:52.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.076 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.076 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:52.076 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406948+0000", 2024-09-14T23:56:52.076 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9435999999999999e-05 2024-09-14T23:56:52.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.077 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-14T23:56:52.077 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406953+0000", 2024-09-14T23:56:52.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2809999999999999e-06 2024-09-14T23:56:52.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.077 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:52.077 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.408559+0000", 2024-09-14T23:56:52.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00160607 2024-09-14T23:56:52.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.077 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:52.077 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.408637+0000", 2024-09-14T23:56:52.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8504000000000006e-05 2024-09-14T23:56:52.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.078 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.078 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 4", 2024-09-14T23:56:52.078 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.408781+0000", 2024-09-14T23:56:52.078 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014374799999999999 2024-09-14T23:56:52.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.078 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.078 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:56:52.078 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.409659+0000", 2024-09-14T23:56:52.078 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000877781 2024-09-14T23:56:52.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.078 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.078 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:52.078 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.409822+0000", 2024-09-14T23:56:52.078 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016295400000000001 2024-09-14T23:56:52.079 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.079 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.079 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:52.079 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.409872+0000", 2024-09-14T23:56:52.079 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0022000000000003e-05 2024-09-14T23:56:52.079 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.079 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.079 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:52.079 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.409912+0000", 2024-09-14T23:56:52.079 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9841e-05 2024-09-14T23:56:52.079 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.079 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:52.079 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.079 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.079 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.080 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:3621 3.1f e29/23 3:fdd18962:::rbd_data.1118de1168b.0000000000000045:head v 29'12, mlcod=27'9)", 2024-09-14T23:56:52.080 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.047969+0000", 2024-09-14T23:56:52.080 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.002892136, 2024-09-14T23:56:52.080 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001436122, 2024-09-14T23:56:52.080 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:52.080 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:52.080 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:52.080 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:52.080 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.080 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:52.080 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.047969+0000", 2024-09-14T23:56:52.080 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.080 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.080 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.081 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:52.081 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.047969+0000", 2024-09-14T23:56:52.081 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.081 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.081 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:52.081 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.047970+0000", 2024-09-14T23:56:52.081 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1199999999999998e-07 2024-09-14T23:56:52.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.081 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.081 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:52.081 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.047974+0000", 2024-09-14T23:56:52.082 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.549e-06 2024-09-14T23:56:52.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.082 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.082 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:52.082 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.047983+0000", 2024-09-14T23:56:52.082 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9509999999999995e-06 2024-09-14T23:56:52.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.082 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.082 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:52.082 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.047989+0000", 2024-09-14T23:56:52.082 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0939999999999996e-06 2024-09-14T23:56:52.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.082 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.082 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:52.083 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.048062+0000", 2024-09-14T23:56:52.083 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2274000000000006e-05 2024-09-14T23:56:52.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.083 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.083 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:52.083 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.048086+0000", 2024-09-14T23:56:52.083 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3875000000000001e-05 2024-09-14T23:56:52.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.083 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.083 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:52.083 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.049367+0000", 2024-09-14T23:56:52.083 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0012811719999999999 2024-09-14T23:56:52.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.083 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.084 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:52.084 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.049406+0000", 2024-09-14T23:56:52.084 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8794999999999997e-05 2024-09-14T23:56:52.084 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.084 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:52.084 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.084 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.084 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.084 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "pg_update_log_missing(3.1d epoch 29/28 rep_tid 39 entries 29'11 (0'0) error 3:b98f80e3:::rbd_data.1118de1168b.0000000000000067:head by client.4376.0:3676 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-14T23:56:52.084 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.061591+0000", 2024-09-14T23:56:52.084 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 12.989270477, 2024-09-14T23:56:52.084 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001346643, 2024-09-14T23:56:52.084 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:52.084 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:52.085 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:56:52.085 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:52.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.085 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:52.085 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.061591+0000", 2024-09-14T23:56:52.085 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.085 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:52.085 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.061591+0000", 2024-09-14T23:56:52.085 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.086 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:52.086 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.061592+0000", 2024-09-14T23:56:52.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7700000000000004e-07 2024-09-14T23:56:52.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.086 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.086 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:52.086 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.061603+0000", 2024-09-14T23:56:52.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1148000000000001e-05 2024-09-14T23:56:52.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.086 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.086 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:52.086 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.061624+0000", 2024-09-14T23:56:52.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0930000000000001e-05 2024-09-14T23:56:52.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.087 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:52.087 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.061639+0000", 2024-09-14T23:56:52.087 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5194000000000001e-05 2024-09-14T23:56:52.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.087 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:52.087 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.061758+0000", 2024-09-14T23:56:52.087 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000119164 2024-09-14T23:56:52.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.087 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:52.087 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.062938+0000", 2024-09-14T23:56:52.087 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00117953 2024-09-14T23:56:52.088 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.088 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:52.088 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.088 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.088 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:4073 3.16 e29/28 3:6a953de9:::rbd_data.1118de1168b.000000000000006d:head v 29'7, mlcod=27'5)", 2024-09-14T23:56:52.088 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.080983+0000", 2024-09-14T23:56:52.088 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 12.969878653, 2024-09-14T23:56:52.088 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001444802, 2024-09-14T23:56:52.088 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:52.088 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:52.088 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:52.089 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:52.089 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.089 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:52.089 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.080983+0000", 2024-09-14T23:56:52.089 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.089 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.089 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.090 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:52.090 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.080983+0000", 2024-09-14T23:56:52.090 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.090 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.090 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:52.090 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.080984+0000", 2024-09-14T23:56:52.090 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4e-07 2024-09-14T23:56:52.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.090 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.090 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:52.090 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.081057+0000", 2024-09-14T23:56:52.090 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3271999999999999e-05 2024-09-14T23:56:52.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:52.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.081069+0000", 2024-09-14T23:56:52.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2319e-05 2024-09-14T23:56:52.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:52.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.081077+0000", 2024-09-14T23:56:52.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1910000000000003e-06 2024-09-14T23:56:52.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:52.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.081135+0000", 2024-09-14T23:56:52.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8686e-05 2024-09-14T23:56:52.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.092 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.092 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:52.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.081158+0000", 2024-09-14T23:56:52.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2600999999999999e-05 2024-09-14T23:56:52.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.092 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.092 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:52.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.082374+0000", 2024-09-14T23:56:52.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001216373 2024-09-14T23:56:52.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.092 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.092 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:52.093 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.082428+0000", 2024-09-14T23:56:52.093 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3520000000000003e-05 2024-09-14T23:56:52.093 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.093 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:52.093 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.093 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.093 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.093 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:5575 3.17 e29/23 3:ee5ec205:::rbd_data.1118de1168b.00000000000000a2:head v 29'36, mlcod=29'32)", 2024-09-14T23:56:52.093 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.349366+0000", 2024-09-14T23:56:52.093 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 12.701495736, 2024-09-14T23:56:52.093 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0013367349999999999, 2024-09-14T23:56:52.093 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:52.093 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:52.094 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:52.094 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:52.094 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.094 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:52.094 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.349366+0000", 2024-09-14T23:56:52.094 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.094 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.094 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.094 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:52.094 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.349366+0000", 2024-09-14T23:56:52.094 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.094 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.094 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.094 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:52.094 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.349366+0000", 2024-09-14T23:56:52.094 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5699999999999998e-07 2024-09-14T23:56:52.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:52.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.349376+0000", 2024-09-14T23:56:52.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9389999999999992e-06 2024-09-14T23:56:52.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:52.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.349387+0000", 2024-09-14T23:56:52.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0417999999999999e-05 2024-09-14T23:56:52.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.096 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:52.096 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.349393+0000", 2024-09-14T23:56:52.096 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0789999999999997e-06 2024-09-14T23:56:52.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.096 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:52.096 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.349815+0000", 2024-09-14T23:56:52.096 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00042267999999999998 2024-09-14T23:56:52.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.096 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:52.096 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.349838+0000", 2024-09-14T23:56:52.096 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2846000000000001e-05 2024-09-14T23:56:52.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.097 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:52.097 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.350654+0000", 2024-09-14T23:56:52.097 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00081540099999999999 2024-09-14T23:56:52.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.097 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.097 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:52.097 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.350703+0000", 2024-09-14T23:56:52.097 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8915000000000002e-05 2024-09-14T23:56:52.097 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.097 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:52.097 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.097 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.097 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:17405 3.2 e33/23 3:450461f9:::rbd_data.11187be34cc8.000000000000000c:head v 33'56, mlcod=32'53)", 2024-09-14T23:56:52.098 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:43.185550+0000", 2024-09-14T23:56:52.098 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.8653115769999999, 2024-09-14T23:56:52.098 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0020455920000000002, 2024-09-14T23:56:52.098 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:52.098 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:52.098 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:52.098 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:52.098 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.098 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:52.098 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:43.185550+0000", 2024-09-14T23:56:52.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.099 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:52.099 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:43.185550+0000", 2024-09-14T23:56:52.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.099 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:52.099 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:43.185551+0000", 2024-09-14T23:56:52.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4000000000000001e-07 2024-09-14T23:56:52.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.099 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:52.100 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:43.186697+0000", 2024-09-14T23:56:52.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001146701 2024-09-14T23:56:52.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.100 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.100 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:52.100 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:43.186708+0000", 2024-09-14T23:56:52.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0261999999999999e-05 2024-09-14T23:56:52.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.100 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.100 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:52.100 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:43.186716+0000", 2024-09-14T23:56:52.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1289999999999996e-06 2024-09-14T23:56:52.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.101 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:52.101 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:43.186767+0000", 2024-09-14T23:56:52.101 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1477999999999997e-05 2024-09-14T23:56:52.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.101 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:52.101 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:43.186787+0000", 2024-09-14T23:56:52.101 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9850000000000001e-05 2024-09-14T23:56:52.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.101 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:52.101 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:43.187556+0000", 2024-09-14T23:56:52.101 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00076922299999999998 2024-09-14T23:56:52.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.102 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.102 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:52.102 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:43.187596+0000", 2024-09-14T23:56:52.102 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9308999999999999e-05 2024-09-14T23:56:52.102 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.102 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:52.102 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.102 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.102 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:18193 3.17 e34/23 3:e951801c:::rbd_data.11187be34cc8.000000000000000a:head v 34'69, mlcod=33'67)", 2024-09-14T23:56:52.102 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:44.011272+0000", 2024-09-14T23:56:52.102 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.0395897979999997, 2024-09-14T23:56:52.102 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0014004499999999999, 2024-09-14T23:56:52.102 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:52.103 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:52.103 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:52.103 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:52.103 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.103 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:52.103 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.011272+0000", 2024-09-14T23:56:52.103 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.103 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.103 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:52.103 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.011272+0000", 2024-09-14T23:56:52.103 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.103 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.104 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:52.104 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.011272+0000", 2024-09-14T23:56:52.104 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9600000000000001e-07 2024-09-14T23:56:52.104 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.104 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.104 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:52.104 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.011277+0000", 2024-09-14T23:56:52.104 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1610000000000002e-06 2024-09-14T23:56:52.104 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.104 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.104 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:52.104 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.011293+0000", 2024-09-14T23:56:52.104 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6268000000000001e-05 2024-09-14T23:56:52.104 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.105 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:52.105 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.011300+0000", 2024-09-14T23:56:52.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5420000000000002e-06 2024-09-14T23:56:52.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.105 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:52.105 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.011356+0000", 2024-09-14T23:56:52.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6159999999999998e-05 2024-09-14T23:56:52.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.105 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:52.105 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.011375+0000", 2024-09-14T23:56:52.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.893e-05 2024-09-14T23:56:52.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.106 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:52.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.012631+0000", 2024-09-14T23:56:52.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0012563839999999999 2024-09-14T23:56:52.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.106 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:52.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.012672+0000", 2024-09-14T23:56:52.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0809000000000002e-05 2024-09-14T23:56:52.106 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.106 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:52.106 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.107 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:18263 3.5 e34/28 3:a40e472c:::rbd_data.11187be34cc8.0000000000000004:head v 34'65, mlcod=34'63)", 2024-09-14T23:56:52.107 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:44.113544+0000", 2024-09-14T23:56:52.107 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.9373173799999996, 2024-09-14T23:56:52.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0014541949999999999, 2024-09-14T23:56:52.107 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:52.107 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:52.107 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:52.107 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:52.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.107 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:52.107 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.113544+0000", 2024-09-14T23:56:52.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:52.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.113544+0000", 2024-09-14T23:56:52.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:52.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.113545+0000", 2024-09-14T23:56:52.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6000000000000003e-07 2024-09-14T23:56:52.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:52.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.113551+0000", 2024-09-14T23:56:52.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1140000000000002e-06 2024-09-14T23:56:52.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:52.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.113565+0000", 2024-09-14T23:56:52.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4165000000000001e-05 2024-09-14T23:56:52.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:52.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.113572+0000", 2024-09-14T23:56:52.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1269999999999997e-06 2024-09-14T23:56:52.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:52.110 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.113635+0000", 2024-09-14T23:56:52.110 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2545999999999994e-05 2024-09-14T23:56:52.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.110 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.110 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:52.110 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.113652+0000", 2024-09-14T23:56:52.110 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7577000000000001e-05 2024-09-14T23:56:52.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.110 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.110 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:52.110 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.114957+0000", 2024-09-14T23:56:52.110 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001304204 2024-09-14T23:56:52.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.110 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.111 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:52.111 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.114998+0000", 2024-09-14T23:56:52.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1702000000000002e-05 2024-09-14T23:56:52.111 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.111 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:52.111 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.111 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.111 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:18261 3.1f e34/23 3:f953ed77:::rbd_data.11187be34cc8.0000000000000002:head v 34'76, mlcod=33'74)", 2024-09-14T23:56:52.111 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:44.113618+0000", 2024-09-14T23:56:52.111 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.9372434260000002, 2024-09-14T23:56:52.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001402845, 2024-09-14T23:56:52.111 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:52.111 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:52.112 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:52.112 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:52.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:52.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.113618+0000", 2024-09-14T23:56:52.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:52.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.113618+0000", 2024-09-14T23:56:52.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:52.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.113619+0000", 2024-09-14T23:56:52.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3900000000000005e-07 2024-09-14T23:56:52.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.113 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:52.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.113623+0000", 2024-09-14T23:56:52.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3839999999999999e-06 2024-09-14T23:56:52.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.113 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:52.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.113633+0000", 2024-09-14T23:56:52.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4890000000000001e-06 2024-09-14T23:56:52.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.114 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:52.114 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.113639+0000", 2024-09-14T23:56:52.114 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0780000000000001e-06 2024-09-14T23:56:52.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.114 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:52.114 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.113698+0000", 2024-09-14T23:56:52.114 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9131e-05 2024-09-14T23:56:52.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.115 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:52.115 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.113716+0000", 2024-09-14T23:56:52.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8328000000000001e-05 2024-09-14T23:56:52.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.115 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:52.115 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.114954+0000", 2024-09-14T23:56:52.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001237812 2024-09-14T23:56:52.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.115 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:52.115 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.115021+0000", 2024-09-14T23:56:52.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7083999999999997e-05 2024-09-14T23:56:52.115 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.116 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:52.116 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.116 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.116 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.116 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:27077 3.16 e38/28 3:6be8b54e:::rbd_data.111814b482f8.0000000000000098:head v 38'107, mlcod=37'104)", 2024-09-14T23:56:52.116 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.317964+0000", 2024-09-14T23:56:52.116 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.7328974210000001, 2024-09-14T23:56:52.116 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016474440000000001, 2024-09-14T23:56:52.116 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:52.116 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:52.116 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:52.116 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:52.116 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.116 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:52.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.317964+0000", 2024-09-14T23:56:52.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:52.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.317964+0000", 2024-09-14T23:56:52.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:52.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.317965+0000", 2024-09-14T23:56:52.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7899999999999998e-07 2024-09-14T23:56:52.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.118 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:52.118 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.318170+0000", 2024-09-14T23:56:52.118 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000205582 2024-09-14T23:56:52.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.118 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.118 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:52.118 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.318183+0000", 2024-09-14T23:56:52.118 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2286999999999999e-05 2024-09-14T23:56:52.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.118 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.118 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:52.118 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.318190+0000", 2024-09-14T23:56:52.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3590000000000002e-06 2024-09-14T23:56:52.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.119 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:52.119 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.318270+0000", 2024-09-14T23:56:52.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0136999999999995e-05 2024-09-14T23:56:52.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.119 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:52.119 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.318290+0000", 2024-09-14T23:56:52.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9933e-05 2024-09-14T23:56:52.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.119 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:52.119 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.319575+0000", 2024-09-14T23:56:52.120 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001285097 2024-09-14T23:56:52.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.120 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.120 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:52.120 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.319612+0000", 2024-09-14T23:56:52.120 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6369999999999999e-05 2024-09-14T23:56:52.120 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.120 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:52.120 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.120 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.120 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:28182 3.1f e38/23 3:fd5b853f:::rbd_data.111814b482f8.000000000000007e:head v 38'98, mlcod=38'96)", 2024-09-14T23:56:52.120 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.449767+0000", 2024-09-14T23:56:52.120 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.6010942969999999, 2024-09-14T23:56:52.121 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001458155, 2024-09-14T23:56:52.121 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:52.121 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:52.121 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:52.121 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:52.121 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.121 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:52.121 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.449767+0000", 2024-09-14T23:56:52.121 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.121 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.121 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:52.121 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.449767+0000", 2024-09-14T23:56:52.122 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.122 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.122 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:52.122 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.449768+0000", 2024-09-14T23:56:52.122 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.082e-06 2024-09-14T23:56:52.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.122 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.122 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:52.122 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.450032+0000", 2024-09-14T23:56:52.122 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00026392700000000001 2024-09-14T23:56:52.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.122 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.122 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:52.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.450043+0000", 2024-09-14T23:56:52.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1153e-05 2024-09-14T23:56:52.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.123 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.123 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:52.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.450050+0000", 2024-09-14T23:56:52.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7660000000000001e-06 2024-09-14T23:56:52.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.123 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.123 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:52.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.450104+0000", 2024-09-14T23:56:52.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3443999999999999e-05 2024-09-14T23:56:52.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.123 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.123 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:52.124 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.450120+0000", 2024-09-14T23:56:52.124 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6177000000000001e-05 2024-09-14T23:56:52.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.124 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:52.124 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.451162+0000", 2024-09-14T23:56:52.124 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001042081 2024-09-14T23:56:52.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.124 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:52.125 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.451225+0000", 2024-09-14T23:56:52.125 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3524999999999999e-05 2024-09-14T23:56:52.125 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.125 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:52.125 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.125 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.125 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:28230 3.2 e38/23 3:461ef2b6:::rbd_data.111814b482f8.000000000000028c:head v 38'86, mlcod=38'84)", 2024-09-14T23:56:52.125 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.482806+0000", 2024-09-14T23:56:52.125 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.5680555030000001, 2024-09-14T23:56:52.125 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0014455169999999999, 2024-09-14T23:56:52.125 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:52.126 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:52.126 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:52.126 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:52.126 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.126 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:52.126 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.482806+0000", 2024-09-14T23:56:52.126 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.126 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.126 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.126 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:52.126 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.482806+0000", 2024-09-14T23:56:52.126 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.126 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.127 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.127 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:52.127 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.482806+0000", 2024-09-14T23:56:52.127 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9900000000000001e-07 2024-09-14T23:56:52.127 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.127 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.127 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:52.127 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.482819+0000", 2024-09-14T23:56:52.127 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3019e-05 2024-09-14T23:56:52.127 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.127 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.127 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:52.127 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.482827+0000", 2024-09-14T23:56:52.128 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9540000000000004e-06 2024-09-14T23:56:52.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.128 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.128 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:52.128 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.482833+0000", 2024-09-14T23:56:52.128 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8579999999999996e-06 2024-09-14T23:56:52.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.128 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.128 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:52.128 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.482885+0000", 2024-09-14T23:56:52.128 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1549999999999999e-05 2024-09-14T23:56:52.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.128 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.128 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:52.129 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.482905+0000", 2024-09-14T23:56:52.129 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0319999999999999e-05 2024-09-14T23:56:52.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.129 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.129 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:52.129 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.484209+0000", 2024-09-14T23:56:52.129 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001304073 2024-09-14T23:56:52.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.129 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.129 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:52.129 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.484252+0000", 2024-09-14T23:56:52.129 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2344e-05 2024-09-14T23:56:52.129 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.130 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:52.130 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.130 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.130 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.130 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:29382 3.1f e38/23 3:f8f8f64b:::rbd_data.111814b482f8.00000000000000a3:head v 38'105, mlcod=38'103)", 2024-09-14T23:56:52.130 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.799875+0000", 2024-09-14T23:56:52.130 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.2509863029999999, 2024-09-14T23:56:52.130 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018309019999999999, 2024-09-14T23:56:52.130 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:52.130 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:52.130 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:52.130 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:52.130 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.130 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:52.131 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.799875+0000", 2024-09-14T23:56:52.131 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.131 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.131 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:52.131 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.799875+0000", 2024-09-14T23:56:52.131 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.131 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.131 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:52.131 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.799876+0000", 2024-09-14T23:56:52.131 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1200000000000001e-06 2024-09-14T23:56:52.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.132 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.132 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:52.132 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.799887+0000", 2024-09-14T23:56:52.132 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0981e-05 2024-09-14T23:56:52.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.132 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.132 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:52.132 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.799908+0000", 2024-09-14T23:56:52.132 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1101999999999999e-05 2024-09-14T23:56:52.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.132 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.132 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:52.132 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.799917+0000", 2024-09-14T23:56:52.132 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9269999999999994e-06 2024-09-14T23:56:52.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.133 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:52.133 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.799981+0000", 2024-09-14T23:56:52.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3781000000000005e-05 2024-09-14T23:56:52.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.133 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:52.133 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.800005+0000", 2024-09-14T23:56:52.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4048000000000001e-05 2024-09-14T23:56:52.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.133 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:52.133 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.801655+0000", 2024-09-14T23:56:52.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016501770000000001 2024-09-14T23:56:52.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:52.134 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.801706+0000", 2024-09-14T23:56:52.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0766000000000001e-05 2024-09-14T23:56:52.134 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.134 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:52.134 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.134 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:30622 3.2 e38/23 3:46f95097:::rbd_data.111814b482f8.0000000000000012:head v 38'102, mlcod=38'100)", 2024-09-14T23:56:52.134 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.886635+0000", 2024-09-14T23:56:52.134 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.1642270680000002, 2024-09-14T23:56:52.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001534728, 2024-09-14T23:56:52.135 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:52.135 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:52.135 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:52.135 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:52.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.135 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:52.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.886635+0000", 2024-09-14T23:56:52.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.135 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:52.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.886635+0000", 2024-09-14T23:56:52.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.136 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.136 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:52.136 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.886636+0000", 2024-09-14T23:56:52.136 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.378e-06 2024-09-14T23:56:52.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.136 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.136 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:52.136 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.886926+0000", 2024-09-14T23:56:52.136 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000289883 2024-09-14T23:56:52.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.136 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.136 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:52.136 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.886945+0000", 2024-09-14T23:56:52.136 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9453000000000002e-05 2024-09-14T23:56:52.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:52.137 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.886955+0000", 2024-09-14T23:56:52.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0000000000000001e-05 2024-09-14T23:56:52.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:52.137 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.887037+0000", 2024-09-14T23:56:52.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1725999999999996e-05 2024-09-14T23:56:52.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:52.137 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.887061+0000", 2024-09-14T23:56:52.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4090000000000001e-05 2024-09-14T23:56:52.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:52.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.888122+0000", 2024-09-14T23:56:52.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010607800000000001 2024-09-14T23:56:52.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:52.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.888169+0000", 2024-09-14T23:56:52.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7417999999999999e-05 2024-09-14T23:56:52.138 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.138 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:52.138 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.139 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:30697 3.1d e38/28 3:bf147eca:::rbd_data.111814b482f8.00000000000001bb:head v 38'116, mlcod=38'114)", 2024-09-14T23:56:52.139 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.938459+0000", 2024-09-14T23:56:52.139 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.112402232, 2024-09-14T23:56:52.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0015404150000000001, 2024-09-14T23:56:52.139 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:52.139 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:52.139 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:52.139 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:52.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.139 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:52.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.938459+0000", 2024-09-14T23:56:52.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.140 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.140 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.140 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:52.140 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.938459+0000", 2024-09-14T23:56:52.140 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.140 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.140 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.140 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:52.140 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.938461+0000", 2024-09-14T23:56:52.140 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3680000000000001e-06 2024-09-14T23:56:52.140 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.140 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.140 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:52.140 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.938470+0000", 2024-09-14T23:56:52.141 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3530000000000001e-06 2024-09-14T23:56:52.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.141 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:52.141 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.938493+0000", 2024-09-14T23:56:52.141 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3186000000000001e-05 2024-09-14T23:56:52.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.141 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:52.141 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.938507+0000", 2024-09-14T23:56:52.141 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3307e-05 2024-09-14T23:56:52.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.141 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:52.142 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.939053+0000", 2024-09-14T23:56:52.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00054603099999999999 2024-09-14T23:56:52.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.142 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:52.142 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.939070+0000", 2024-09-14T23:56:52.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7365999999999999e-05 2024-09-14T23:56:52.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.142 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:52.142 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.939958+0000", 2024-09-14T23:56:52.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00088815300000000003 2024-09-14T23:56:52.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:52.143 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.940000+0000", 2024-09-14T23:56:52.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1650999999999998e-05 2024-09-14T23:56:52.143 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.143 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:52.143 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.143 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:35143 3.2 e39/23 3:45453b3e:::rbd_data.111814b482f8.0000000000000046:head v 39'107, mlcod=38'105)", 2024-09-14T23:56:52.143 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:49.246523+0000", 2024-09-14T23:56:52.143 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 2.8043384840000001, 2024-09-14T23:56:52.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001355453, 2024-09-14T23:56:52.143 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:52.143 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:52.143 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:52.144 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:52.144 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.144 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:52.144 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:49.246523+0000", 2024-09-14T23:56:52.144 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.144 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.144 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:52.144 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:49.246523+0000", 2024-09-14T23:56:52.144 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.144 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.144 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:52.144 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:49.246524+0000", 2024-09-14T23:56:52.145 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5300000000000003e-07 2024-09-14T23:56:52.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.145 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.145 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:52.145 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:49.246846+0000", 2024-09-14T23:56:52.145 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00032207999999999998 2024-09-14T23:56:52.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.145 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.145 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:52.145 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:49.246856+0000", 2024-09-14T23:56:52.145 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0339e-05 2024-09-14T23:56:52.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.145 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.145 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:52.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:49.246862+0000", 2024-09-14T23:56:52.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5809999999999996e-06 2024-09-14T23:56:52.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.146 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:52.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:49.246919+0000", 2024-09-14T23:56:52.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7537000000000002e-05 2024-09-14T23:56:52.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.146 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:52.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:49.246944+0000", 2024-09-14T23:56:52.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4711000000000001e-05 2024-09-14T23:56:52.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.146 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:52.147 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:49.247829+0000", 2024-09-14T23:56:52.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00088515499999999999 2024-09-14T23:56:52.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.147 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:52.147 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:49.247879+0000", 2024-09-14T23:56:52.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9296999999999999e-05 2024-09-14T23:56:52.147 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.147 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:52.147 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.147 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:35278 3.16 e39/28 3:68fa735c:::rbd_data.111814b482f8.0000000000000049:head v 39'117, mlcod=38'115)", 2024-09-14T23:56:52.147 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:49.256324+0000", 2024-09-14T23:56:52.147 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 2.7945371300000001, 2024-09-14T23:56:52.148 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001368815, 2024-09-14T23:56:52.148 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:52.148 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:52.148 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:52.148 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:52.148 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.148 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:52.148 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:49.256324+0000", 2024-09-14T23:56:52.148 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.148 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.148 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:52.148 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:49.256324+0000", 2024-09-14T23:56:52.148 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.149 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:52.149 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:49.256325+0000", 2024-09-14T23:56:52.149 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.09e-07 2024-09-14T23:56:52.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.149 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:52.149 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:49.256474+0000", 2024-09-14T23:56:52.149 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014861599999999999 2024-09-14T23:56:52.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.149 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:52.149 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:49.256489+0000", 2024-09-14T23:56:52.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5113000000000001e-05 2024-09-14T23:56:52.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:52.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:49.256496+0000", 2024-09-14T23:56:52.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4959999999999999e-06 2024-09-14T23:56:52.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:52.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:49.256575+0000", 2024-09-14T23:56:52.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8145999999999993e-05 2024-09-14T23:56:52.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:52.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:49.256595+0000", 2024-09-14T23:56:52.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0409999999999999e-05 2024-09-14T23:56:52.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:52.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:49.257653+0000", 2024-09-14T23:56:52.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001057788 2024-09-14T23:56:52.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:52.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:49.257693+0000", 2024-09-14T23:56:52.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0436999999999999e-05 2024-09-14T23:56:52.151 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.151 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:52.151 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.151 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.152 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:52.152 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:56:52.156 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T23:56:52.156 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:56:52.152+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:56:52.254 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:56:52.253+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:56:52.309 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:56:52.309 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:56:52.309 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.309 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "MOSDPGPush(3.2 42/41 [PushOp(3:40362cd0:::rbd_data.1118de1168b.000000000000010f:4, version: 29'31, data_included: [221184~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:40362cd0:::rbd_data.1118de1168b.000000000000010f:4@29'31, size: 225280, copy_subset: [(0, 225280)], clone_subset: {}, snapset: 4=[]:{4: [4]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 225280, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T23:56:52.310 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:52.308187+0000", 2024-09-14T23:56:52.310 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.00023343999999999999, 2024-09-14T23:56:52.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00028051500000000001, 2024-09-14T23:56:52.310 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:52.310 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:52.310 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:52.310 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:52.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.310 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:52.310 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:52.308187+0000", 2024-09-14T23:56:52.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.311 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:52.311 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:52.308187+0000", 2024-09-14T23:56:52.311 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.311 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.311 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:52.311 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:52.308188+0000", 2024-09-14T23:56:52.311 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7899999999999998e-07 2024-09-14T23:56:52.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.312 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:52.312 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:52.308204+0000", 2024-09-14T23:56:52.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6552999999999999e-05 2024-09-14T23:56:52.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.312 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:52.312 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:52.308220+0000", 2024-09-14T23:56:52.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5435999999999999e-05 2024-09-14T23:56:52.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.313 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:52.313 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:52.308231+0000", 2024-09-14T23:56:52.313 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1861999999999999e-05 2024-09-14T23:56:52.313 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.313 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.313 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:52.313 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:52.308299+0000", 2024-09-14T23:56:52.313 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7595999999999995e-05 2024-09-14T23:56:52.314 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.314 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.314 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:52.314 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:52.308306+0000", 2024-09-14T23:56:52.314 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9299999999999997e-06 2024-09-14T23:56:52.314 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.314 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:52.314 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.314 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.314 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-14T23:56:52.314 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 1 2024-09-14T23:56:52.314 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:56:52.321 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T23:56:52.355 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:56:52.354+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:56:52.439 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:56:52.439 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:56:52.439 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:56:52.439 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:56:52.439 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:56:52.449 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T23:56:52.456 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:56:52.455+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:56:52.478 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-09-14T23:56:52.558 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:56:52.556+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:56:52.570 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:56:52.570 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:56:52.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:56:52.570 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:56:52.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.570 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:18218 3.10 3:0c729a68:::rbd_data.11187be34cc8.0000000000000017:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-14T23:56:52.570 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:44.032604+0000", 2024-09-14T23:56:52.570 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.5353796660000008, 2024-09-14T23:56:52.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.028735283, 2024-09-14T23:56:52.571 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:52.571 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:52.571 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:52.571 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:52.571 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:52.571 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:52.571 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 18218 2024-09-14T23:56:52.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.571 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:52.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:52.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.032604+0000", 2024-09-14T23:56:52.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:52.572 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.032604+0000", 2024-09-14T23:56:52.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:52.572 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.032606+0000", 2024-09-14T23:56:52.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7999999999999999e-06 2024-09-14T23:56:52.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:52.572 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.032614+0000", 2024-09-14T23:56:52.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1810000000000001e-06 2024-09-14T23:56:52.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.573 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:52.573 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.032631+0000", 2024-09-14T23:56:52.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7363e-05 2024-09-14T23:56:52.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.573 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:52.573 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.032638+0000", 2024-09-14T23:56:52.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0430000000000002e-06 2024-09-14T23:56:52.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.573 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:52.574 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.032708+0000", 2024-09-14T23:56:52.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0257000000000002e-05 2024-09-14T23:56:52.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.574 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-14T23:56:52.574 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.032713+0000", 2024-09-14T23:56:52.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4190000000000004e-06 2024-09-14T23:56:52.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.574 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:52.574 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.034010+0000", 2024-09-14T23:56:52.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001297407 2024-09-14T23:56:52.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.574 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:52.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.034059+0000", 2024-09-14T23:56:52.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8912999999999997e-05 2024-09-14T23:56:52.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:56:52.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.038995+0000", 2024-09-14T23:56:52.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0049356010000000004 2024-09-14T23:56:52.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:52.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.061258+0000", 2024-09-14T23:56:52.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.022262963 2024-09-14T23:56:52.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:56:52.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.061269+0000", 2024-09-14T23:56:52.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1671000000000001e-05 2024-09-14T23:56:52.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:56:52.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.061306+0000", 2024-09-14T23:56:52.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6474999999999999e-05 2024-09-14T23:56:52.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:52.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.061339+0000", 2024-09-14T23:56:52.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3189999999999999e-05 2024-09-14T23:56:52.576 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.577 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:52.577 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.577 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30289 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 585728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:52.577 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.865354+0000", 2024-09-14T23:56:52.577 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.7026298390000001, 2024-09-14T23:56:52.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015192881, 2024-09-14T23:56:52.577 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:52.577 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:52.577 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:52.577 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:52.577 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:52.578 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:52.578 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30289 2024-09-14T23:56:52.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.578 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:52.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:52.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865354+0000", 2024-09-14T23:56:52.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:52.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865354+0000", 2024-09-14T23:56:52.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:52.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865354+0000", 2024-09-14T23:56:52.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8500000000000001e-07 2024-09-14T23:56:52.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:52.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865357+0000", 2024-09-14T23:56:52.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6199999999999999e-06 2024-09-14T23:56:52.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:52.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865364+0000", 2024-09-14T23:56:52.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0840000000000003e-06 2024-09-14T23:56:52.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:52.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865369+0000", 2024-09-14T23:56:52.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3709999999999999e-06 2024-09-14T23:56:52.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:52.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.880378+0000", 2024-09-14T23:56:52.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015009043 2024-09-14T23:56:52.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:52.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.880451+0000", 2024-09-14T23:56:52.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2747e-05 2024-09-14T23:56:52.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:52.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.880546+0000", 2024-09-14T23:56:52.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5431000000000002e-05 2024-09-14T23:56:52.581 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.581 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:52.581 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.581 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30298 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 438272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:52.581 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.865377+0000", 2024-09-14T23:56:52.581 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.7026063709999999, 2024-09-14T23:56:52.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015369668, 2024-09-14T23:56:52.582 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:52.582 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:52.582 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:52.582 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:52.582 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:52.582 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:52.582 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30298 2024-09-14T23:56:52.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.582 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:52.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:52.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865377+0000", 2024-09-14T23:56:52.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:52.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865377+0000", 2024-09-14T23:56:52.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:52.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865377+0000", 2024-09-14T23:56:52.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9499999999999998e-07 2024-09-14T23:56:52.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:52.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865381+0000", 2024-09-14T23:56:52.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.371e-06 2024-09-14T23:56:52.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.584 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:52.584 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865389+0000", 2024-09-14T23:56:52.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3469999999999999e-06 2024-09-14T23:56:52.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.584 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:52.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865394+0000", 2024-09-14T23:56:52.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9869999999999999e-06 2024-09-14T23:56:52.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:52.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.880577+0000", 2024-09-14T23:56:52.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015183103 2024-09-14T23:56:52.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:52.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.880649+0000", 2024-09-14T23:56:52.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1989999999999993e-05 2024-09-14T23:56:52.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:52.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.880747+0000", 2024-09-14T23:56:52.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7474999999999998e-05 2024-09-14T23:56:52.586 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.586 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:52.586 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.587 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30302 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 364544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:52.587 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.865401+0000", 2024-09-14T23:56:52.587 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.702582467, 2024-09-14T23:56:52.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015547874999999999, 2024-09-14T23:56:52.587 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:52.587 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:52.587 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:52.587 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:52.587 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:52.587 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:52.587 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30302 2024-09-14T23:56:52.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.588 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:52.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:52.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865401+0000", 2024-09-14T23:56:52.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:52.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865401+0000", 2024-09-14T23:56:52.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:52.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865402+0000", 2024-09-14T23:56:52.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4499999999999997e-07 2024-09-14T23:56:52.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:52.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865404+0000", 2024-09-14T23:56:52.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9240000000000001e-06 2024-09-14T23:56:52.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:52.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865410+0000", 2024-09-14T23:56:52.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3570000000000003e-06 2024-09-14T23:56:52.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:52.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865415+0000", 2024-09-14T23:56:52.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.553e-06 2024-09-14T23:56:52.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:52.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.880779+0000", 2024-09-14T23:56:52.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015363411 2024-09-14T23:56:52.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:52.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.880852+0000", 2024-09-14T23:56:52.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2916000000000005e-05 2024-09-14T23:56:52.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:52.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.880949+0000", 2024-09-14T23:56:52.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7069000000000002e-05 2024-09-14T23:56:52.592 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.592 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:52.592 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.592 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30308 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 294912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:52.593 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.865537+0000", 2024-09-14T23:56:52.593 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.7024468559999999, 2024-09-14T23:56:52.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015615162, 2024-09-14T23:56:52.593 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:52.593 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:52.593 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:52.593 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:52.593 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:52.593 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:52.593 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30308 2024-09-14T23:56:52.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.593 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:52.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.594 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:52.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865537+0000", 2024-09-14T23:56:52.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.594 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:52.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865537+0000", 2024-09-14T23:56:52.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:52.595 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865537+0000", 2024-09-14T23:56:52.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0299999999999999e-07 2024-09-14T23:56:52.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:52.595 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865540+0000", 2024-09-14T23:56:52.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8420000000000001e-06 2024-09-14T23:56:52.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:52.595 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865548+0000", 2024-09-14T23:56:52.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2230000000000007e-06 2024-09-14T23:56:52.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.596 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:52.596 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865555+0000", 2024-09-14T23:56:52.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7780000000000002e-06 2024-09-14T23:56:52.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.597 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:52.597 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.880982+0000", 2024-09-14T23:56:52.597 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015426832 2024-09-14T23:56:52.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.597 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:52.597 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.881059+0000", 2024-09-14T23:56:52.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7095999999999995e-05 2024-09-14T23:56:52.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.598 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:52.598 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.881152+0000", 2024-09-14T23:56:52.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2788000000000007e-05 2024-09-14T23:56:52.598 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.599 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:52.599 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.599 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30371 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 442368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:52.599 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.866343+0000", 2024-09-14T23:56:52.599 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.701640378, 2024-09-14T23:56:52.599 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015209663, 2024-09-14T23:56:52.599 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:52.599 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:52.599 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:52.600 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:52.600 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:52.600 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:52.600 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30371 2024-09-14T23:56:52.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.600 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:52.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.600 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:52.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866343+0000", 2024-09-14T23:56:52.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:52.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866343+0000", 2024-09-14T23:56:52.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:52.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866344+0000", 2024-09-14T23:56:52.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8199999999999999e-07 2024-09-14T23:56:52.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.602 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:52.602 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866347+0000", 2024-09-14T23:56:52.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4759999999999998e-06 2024-09-14T23:56:52.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.602 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:52.602 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866358+0000", 2024-09-14T23:56:52.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0353999999999999e-05 2024-09-14T23:56:52.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:52.603 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866364+0000", 2024-09-14T23:56:52.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3260000000000001e-06 2024-09-14T23:56:52.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:52.603 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.881388+0000", 2024-09-14T23:56:52.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015023791999999999 2024-09-14T23:56:52.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:52.603 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.881460+0000", 2024-09-14T23:56:52.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2509999999999995e-05 2024-09-14T23:56:52.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:52.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.881553+0000", 2024-09-14T23:56:52.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2423000000000006e-05 2024-09-14T23:56:52.604 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.604 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:52.604 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.604 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30374 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 368640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:52.605 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.866381+0000", 2024-09-14T23:56:52.605 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.7016019570000003, 2024-09-14T23:56:52.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015382209000000001, 2024-09-14T23:56:52.605 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:52.605 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:52.605 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:52.605 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:52.605 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:52.605 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:52.605 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30374 2024-09-14T23:56:52.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.606 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:52.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.606 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:52.606 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866381+0000", 2024-09-14T23:56:52.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.606 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:52.606 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866381+0000", 2024-09-14T23:56:52.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.606 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:52.606 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866382+0000", 2024-09-14T23:56:52.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9800000000000003e-07 2024-09-14T23:56:52.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.607 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:52.607 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866385+0000", 2024-09-14T23:56:52.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1599999999999998e-06 2024-09-14T23:56:52.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.607 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:52.607 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866393+0000", 2024-09-14T23:56:52.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7929999999999998e-06 2024-09-14T23:56:52.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:52.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866399+0000", 2024-09-14T23:56:52.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5260000000000001e-06 2024-09-14T23:56:52.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:52.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.881585+0000", 2024-09-14T23:56:52.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.01518624 2024-09-14T23:56:52.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:52.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.881661+0000", 2024-09-14T23:56:52.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5779999999999996e-05 2024-09-14T23:56:52.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:52.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.881764+0000", 2024-09-14T23:56:52.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000103012 2024-09-14T23:56:52.610 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.610 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:52.610 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.610 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30379 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 299008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:52.610 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.866461+0000", 2024-09-14T23:56:52.610 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.7015224460000002, 2024-09-14T23:56:52.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015514637, 2024-09-14T23:56:52.610 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:52.610 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:52.610 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:52.611 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:52.611 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:52.611 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:52.611 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30379 2024-09-14T23:56:52.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.611 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:52.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.611 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:52.611 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866461+0000", 2024-09-14T23:56:52.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.612 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:52.612 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866461+0000", 2024-09-14T23:56:52.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.612 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:52.612 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866462+0000", 2024-09-14T23:56:52.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5700000000000002e-07 2024-09-14T23:56:52.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:52.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866466+0000", 2024-09-14T23:56:52.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6759999999999998e-06 2024-09-14T23:56:52.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:52.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866477+0000", 2024-09-14T23:56:52.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0094e-05 2024-09-14T23:56:52.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:52.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866483+0000", 2024-09-14T23:56:52.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1460000000000001e-06 2024-09-14T23:56:52.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.614 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:52.614 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.881800+0000", 2024-09-14T23:56:52.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015317769 2024-09-14T23:56:52.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.614 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:52.614 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.881875+0000", 2024-09-14T23:56:52.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4726000000000002e-05 2024-09-14T23:56:52.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.614 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:52.614 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.881976+0000", 2024-09-14T23:56:52.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100469 2024-09-14T23:56:52.615 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.615 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:52.615 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.615 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30396 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 532480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:52.615 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.866569+0000", 2024-09-14T23:56:52.615 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.7014140229999999, 2024-09-14T23:56:52.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015617786, 2024-09-14T23:56:52.615 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:52.615 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:52.615 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:52.615 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:52.616 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:52.616 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:52.616 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30396 2024-09-14T23:56:52.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.616 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:52.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:52.616 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866569+0000", 2024-09-14T23:56:52.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:52.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866569+0000", 2024-09-14T23:56:52.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:52.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866570+0000", 2024-09-14T23:56:52.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4000000000000001e-07 2024-09-14T23:56:52.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:52.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866574+0000", 2024-09-14T23:56:52.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7330000000000001e-06 2024-09-14T23:56:52.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:52.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866584+0000", 2024-09-14T23:56:52.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0052e-05 2024-09-14T23:56:52.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:52.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866591+0000", 2024-09-14T23:56:52.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4359999999999996e-06 2024-09-14T23:56:52.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.619 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:52.619 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.882011+0000", 2024-09-14T23:56:52.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015419871999999999 2024-09-14T23:56:52.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.619 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:52.619 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.882088+0000", 2024-09-14T23:56:52.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7118000000000003e-05 2024-09-14T23:56:52.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.619 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:52.619 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.882187+0000", 2024-09-14T23:56:52.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8834999999999998e-05 2024-09-14T23:56:52.619 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.620 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:52.620 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.620 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30405 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 446464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:52.620 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.866670+0000", 2024-09-14T23:56:52.620 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.7013132510000002, 2024-09-14T23:56:52.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015777653999999999, 2024-09-14T23:56:52.620 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:52.620 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:52.620 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:52.621 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:52.621 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:52.621 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:52.621 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30405 2024-09-14T23:56:52.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.621 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:52.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:52.621 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866670+0000", 2024-09-14T23:56:52.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:52.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866670+0000", 2024-09-14T23:56:52.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:52.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866671+0000", 2024-09-14T23:56:52.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3900000000000004e-07 2024-09-14T23:56:52.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:52.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866675+0000", 2024-09-14T23:56:52.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8569999999999997e-06 2024-09-14T23:56:52.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:52.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866685+0000", 2024-09-14T23:56:52.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0002e-05 2024-09-14T23:56:52.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:52.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866692+0000", 2024-09-14T23:56:52.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.691e-06 2024-09-14T23:56:52.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.624 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:52.624 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.882259+0000", 2024-09-14T23:56:52.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015567399000000001 2024-09-14T23:56:52.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.624 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:52.624 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.882342+0000", 2024-09-14T23:56:52.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2919999999999999e-05 2024-09-14T23:56:52.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.625 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:52.625 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.882448+0000", 2024-09-14T23:56:52.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010594600000000001 2024-09-14T23:56:52.625 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.625 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:52.625 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.625 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30407 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 372736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:52.626 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.866702+0000", 2024-09-14T23:56:52.626 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.7012810319999998, 2024-09-14T23:56:52.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015922993, 2024-09-14T23:56:52.626 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:52.626 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:52.626 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:52.626 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:52.626 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:52.626 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:52.626 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30407 2024-09-14T23:56:52.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.626 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:52.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:52.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866702+0000", 2024-09-14T23:56:52.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:52.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866702+0000", 2024-09-14T23:56:52.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:52.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866703+0000", 2024-09-14T23:56:52.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5799999999999999e-07 2024-09-14T23:56:52.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:52.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866706+0000", 2024-09-14T23:56:52.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8890000000000002e-06 2024-09-14T23:56:52.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:52.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866713+0000", 2024-09-14T23:56:52.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.99e-06 2024-09-14T23:56:52.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.629 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:52.629 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866718+0000", 2024-09-14T23:56:52.629 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.254e-06 2024-09-14T23:56:52.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.629 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:52.629 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.882497+0000", 2024-09-14T23:56:52.629 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015778664000000001 2024-09-14T23:56:52.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.629 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:52.629 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.882557+0000", 2024-09-14T23:56:52.629 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0408999999999997e-05 2024-09-14T23:56:52.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:52.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.882625+0000", 2024-09-14T23:56:52.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8229e-05 2024-09-14T23:56:52.630 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.630 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:52.630 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.630 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30415 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 303104~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:52.630 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.866775+0000", 2024-09-14T23:56:52.630 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.701208614, 2024-09-14T23:56:52.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.016119712000000001, 2024-09-14T23:56:52.631 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:52.631 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:52.631 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:52.631 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:52.631 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:52.631 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:52.631 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30415 2024-09-14T23:56:52.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.631 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:52.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.631 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:52.631 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866775+0000", 2024-09-14T23:56:52.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.632 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:52.632 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866775+0000", 2024-09-14T23:56:52.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.633 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:52.633 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866776+0000", 2024-09-14T23:56:52.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5499999999999997e-07 2024-09-14T23:56:52.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.633 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:52.633 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866779+0000", 2024-09-14T23:56:52.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.613e-06 2024-09-14T23:56:52.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.633 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:52.633 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866789+0000", 2024-09-14T23:56:52.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5030000000000003e-06 2024-09-14T23:56:52.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:52.634 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866799+0000", 2024-09-14T23:56:52.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9170000000000001e-06 2024-09-14T23:56:52.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:52.634 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.882704+0000", 2024-09-14T23:56:52.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015905333000000001 2024-09-14T23:56:52.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:52.634 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.882787+0000", 2024-09-14T23:56:52.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3264999999999996e-05 2024-09-14T23:56:52.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:52.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.882895+0000", 2024-09-14T23:56:52.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010722600000000001 2024-09-14T23:56:52.635 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.635 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:52.635 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.635 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30430 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 536576~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:52.635 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.867043+0000", 2024-09-14T23:56:52.635 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.700940648, 2024-09-14T23:56:52.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.016054556000000001, 2024-09-14T23:56:52.636 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:52.636 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:52.636 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:52.636 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:52.636 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:52.636 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:52.636 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30430 2024-09-14T23:56:52.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.636 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:52.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:52.637 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867043+0000", 2024-09-14T23:56:52.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.637 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:52.637 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867043+0000", 2024-09-14T23:56:52.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.637 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:52.637 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867044+0000", 2024-09-14T23:56:52.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.23e-07 2024-09-14T23:56:52.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.638 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:52.638 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867053+0000", 2024-09-14T23:56:52.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7759999999999992e-06 2024-09-14T23:56:52.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.638 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:52.638 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867066+0000", 2024-09-14T23:56:52.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2376e-05 2024-09-14T23:56:52.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.638 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:52.638 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867075+0000", 2024-09-14T23:56:52.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1349999999999998e-06 2024-09-14T23:56:52.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:52.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.882925+0000", 2024-09-14T23:56:52.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015850566 2024-09-14T23:56:52.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:52.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.883000+0000", 2024-09-14T23:56:52.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4746999999999994e-05 2024-09-14T23:56:52.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:52.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.883097+0000", 2024-09-14T23:56:52.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7133000000000007e-05 2024-09-14T23:56:52.639 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.640 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:52.640 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.640 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30440 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 450560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:52.640 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.867174+0000", 2024-09-14T23:56:52.640 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.7008092050000001, 2024-09-14T23:56:52.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.016169586999999999, 2024-09-14T23:56:52.640 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:52.640 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:52.640 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:52.640 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:52.640 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:52.641 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:52.641 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30440 2024-09-14T23:56:52.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.641 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:52.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:52.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867174+0000", 2024-09-14T23:56:52.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:52.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867174+0000", 2024-09-14T23:56:52.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:52.642 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867175+0000", 2024-09-14T23:56:52.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1100000000000005e-07 2024-09-14T23:56:52.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:52.642 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867178+0000", 2024-09-14T23:56:52.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.994e-06 2024-09-14T23:56:52.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:52.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867190+0000", 2024-09-14T23:56:52.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2072000000000001e-05 2024-09-14T23:56:52.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:52.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867196+0000", 2024-09-14T23:56:52.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1340000000000001e-06 2024-09-14T23:56:52.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:52.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.883165+0000", 2024-09-14T23:56:52.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015969061 2024-09-14T23:56:52.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:52.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.883240+0000", 2024-09-14T23:56:52.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4616999999999994e-05 2024-09-14T23:56:52.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:52.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.883344+0000", 2024-09-14T23:56:52.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000103898 2024-09-14T23:56:52.644 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.644 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:52.645 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.645 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30444 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 376832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:52.645 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.867242+0000", 2024-09-14T23:56:52.645 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.700741769, 2024-09-14T23:56:52.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.016313708999999999, 2024-09-14T23:56:52.645 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:52.645 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:52.645 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:52.645 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:52.646 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:52.646 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:52.646 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30444 2024-09-14T23:56:52.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.646 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:52.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:52.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867242+0000", 2024-09-14T23:56:52.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:52.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867242+0000", 2024-09-14T23:56:52.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:52.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867242+0000", 2024-09-14T23:56:52.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0100000000000005e-07 2024-09-14T23:56:52.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:52.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867248+0000", 2024-09-14T23:56:52.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0100000000000001e-06 2024-09-14T23:56:52.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:52.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867260+0000", 2024-09-14T23:56:52.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2179e-05 2024-09-14T23:56:52.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:52.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867267+0000", 2024-09-14T23:56:52.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7780000000000002e-06 2024-09-14T23:56:52.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:52.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.883374+0000", 2024-09-14T23:56:52.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.016106466999999999 2024-09-14T23:56:52.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:52.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.883449+0000", 2024-09-14T23:56:52.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5475000000000005e-05 2024-09-14T23:56:52.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:52.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.883555+0000", 2024-09-14T23:56:52.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000106199 2024-09-14T23:56:52.649 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.649 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:52.650 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.650 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30451 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 307200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:52.650 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.867309+0000", 2024-09-14T23:56:52.650 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.7006740389999999, 2024-09-14T23:56:52.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.016427048, 2024-09-14T23:56:52.650 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:52.650 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:52.650 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:52.650 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:52.650 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:52.650 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:52.650 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30451 2024-09-14T23:56:52.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.651 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:52.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:52.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867309+0000", 2024-09-14T23:56:52.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:52.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867309+0000", 2024-09-14T23:56:52.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.652 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:52.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867310+0000", 2024-09-14T23:56:52.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5799999999999999e-07 2024-09-14T23:56:52.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.652 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:52.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867316+0000", 2024-09-14T23:56:52.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4650000000000002e-06 2024-09-14T23:56:52.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:52.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867325+0000", 2024-09-14T23:56:52.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0659999999999993e-06 2024-09-14T23:56:52.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:52.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867331+0000", 2024-09-14T23:56:52.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2419999999999997e-06 2024-09-14T23:56:52.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:52.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.883585+0000", 2024-09-14T23:56:52.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.016254599000000002 2024-09-14T23:56:52.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:52.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.883659+0000", 2024-09-14T23:56:52.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3697999999999998e-05 2024-09-14T23:56:52.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:52.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.883736+0000", 2024-09-14T23:56:52.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7319999999999998e-05 2024-09-14T23:56:52.655 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.655 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:52.655 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.655 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30463 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 540672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:52.655 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.867871+0000", 2024-09-14T23:56:52.655 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.7001127039999999, 2024-09-14T23:56:52.655 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.016001520000000002, 2024-09-14T23:56:52.655 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:52.655 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:52.655 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:52.655 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:52.655 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:52.655 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:52.656 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30463 2024-09-14T23:56:52.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.656 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:52.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:52.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867871+0000", 2024-09-14T23:56:52.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:52.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867871+0000", 2024-09-14T23:56:52.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:52.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867872+0000", 2024-09-14T23:56:52.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9800000000000002e-07 2024-09-14T23:56:52.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:52.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867876+0000", 2024-09-14T23:56:52.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.318e-06 2024-09-14T23:56:52.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:52.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867887+0000", 2024-09-14T23:56:52.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1055e-05 2024-09-14T23:56:52.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:52.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868679+0000", 2024-09-14T23:56:52.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00079199000000000001 2024-09-14T23:56:52.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:52.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.883763+0000", 2024-09-14T23:56:52.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015083675 2024-09-14T23:56:52.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:52.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.883816+0000", 2024-09-14T23:56:52.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3757e-05 2024-09-14T23:56:52.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.659 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:52.659 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.883872+0000", 2024-09-14T23:56:52.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5826999999999999e-05 2024-09-14T23:56:52.659 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.659 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:52.659 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.659 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30479 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:52.659 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.868703+0000", 2024-09-14T23:56:52.659 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.699280785, 2024-09-14T23:56:52.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015351293, 2024-09-14T23:56:52.660 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:52.660 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:52.660 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:52.660 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:52.660 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:52.660 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:52.660 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30479 2024-09-14T23:56:52.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.660 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:52.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:52.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868703+0000", 2024-09-14T23:56:52.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:52.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868703+0000", 2024-09-14T23:56:52.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:52.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868703+0000", 2024-09-14T23:56:52.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3300000000000001e-07 2024-09-14T23:56:52.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:52.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868708+0000", 2024-09-14T23:56:52.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.279e-06 2024-09-14T23:56:52.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:52.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868717+0000", 2024-09-14T23:56:52.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0469999999999999e-06 2024-09-14T23:56:52.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:52.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868723+0000", 2024-09-14T23:56:52.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.64e-06 2024-09-14T23:56:52.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:52.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.883898+0000", 2024-09-14T23:56:52.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015175016 2024-09-14T23:56:52.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.663 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:52.663 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.883959+0000", 2024-09-14T23:56:52.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0729000000000001e-05 2024-09-14T23:56:52.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.663 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:52.663 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.884054+0000", 2024-09-14T23:56:52.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4848999999999999e-05 2024-09-14T23:56:52.664 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.664 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:52.664 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.664 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.664 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30480 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 454656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:52.664 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.868733+0000", 2024-09-14T23:56:52.664 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.699250202, 2024-09-14T23:56:52.664 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015505417000000001, 2024-09-14T23:56:52.664 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:52.664 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:52.664 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:52.664 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:52.664 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:52.665 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:52.665 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30480 2024-09-14T23:56:52.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.665 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:52.665 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.665 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:52.665 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868733+0000", 2024-09-14T23:56:52.665 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.665 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.665 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:52.665 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868733+0000", 2024-09-14T23:56:52.665 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:52.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868734+0000", 2024-09-14T23:56:52.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3099999999999998e-07 2024-09-14T23:56:52.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:52.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868744+0000", 2024-09-14T23:56:52.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0429e-05 2024-09-14T23:56:52.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:52.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868754+0000", 2024-09-14T23:56:52.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5400000000000001e-06 2024-09-14T23:56:52.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:52.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868760+0000", 2024-09-14T23:56:52.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0170000000000002e-06 2024-09-14T23:56:52.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:52.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.884084+0000", 2024-09-14T23:56:52.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015324071 2024-09-14T23:56:52.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:52.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.884151+0000", 2024-09-14T23:56:52.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7531000000000001e-05 2024-09-14T23:56:52.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.668 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.668 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:52.668 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.884239+0000", 2024-09-14T23:56:52.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7398000000000006e-05 2024-09-14T23:56:52.668 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.668 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:52.668 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.668 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.668 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30483 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 380928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:52.668 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.868769+0000", 2024-09-14T23:56:52.668 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.6992146539999999, 2024-09-14T23:56:52.669 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015654879, 2024-09-14T23:56:52.669 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:52.669 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:52.669 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:52.669 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:52.669 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:52.669 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:52.669 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30483 2024-09-14T23:56:52.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.669 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:52.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.669 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:52.669 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868769+0000", 2024-09-14T23:56:52.669 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:52.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868769+0000", 2024-09-14T23:56:52.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:52.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:52.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868769+0000", 2024-09-14T23:56:52.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8299999999999997e-07 2024-09-14T23:56:52.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:52.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868772+0000", 2024-09-14T23:56:52.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.497e-06 2024-09-14T23:56:52.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.671 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.671 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:52.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868778+0000", 2024-09-14T23:56:52.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6919999999999996e-06 2024-09-14T23:56:52.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.671 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.671 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:52.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868784+0000", 2024-09-14T23:56:52.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5280000000000003e-06 2024-09-14T23:56:52.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.671 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.671 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:52.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.884264+0000", 2024-09-14T23:56:52.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015479629 2024-09-14T23:56:52.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.672 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.672 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:52.672 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.884332+0000", 2024-09-14T23:56:52.672 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8617999999999999e-05 2024-09-14T23:56:52.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:52.672 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:52.672 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:52.672 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.884424+0000", 2024-09-14T23:56:52.672 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1432000000000001e-05 2024-09-14T23:56:52.672 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.672 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:52.672 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.673 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:52.673 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:52.673 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:56:52.674 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T23:56:52.675 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:56:52.672+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:56:52.731 INFO:teuthology.orchestra.run.smithi042.stdout:1065 trunc from 0xad539bc to 0xc0ddb7 2024-09-14T23:56:52.746 INFO:teuthology.orchestra.run.smithi042.stdout:1067 read 0x470f39 thru 0x47c35c (0xb424 bytes) 2024-09-14T23:56:52.748 INFO:teuthology.orchestra.run.smithi042.stdout:1068 punch from 0xb7f90d to 0xb8e42d, (0xeb20 bytes) 2024-09-14T23:56:52.749 INFO:teuthology.orchestra.run.smithi042.stdout:1069 trunc from 0xc0ddb7 to 0x13629ac 2024-09-14T23:56:52.753 INFO:teuthology.orchestra.run.smithi042.stdout:1070 read 0x7a8dfd thru 0x7afc17 (0x6e1b bytes) 2024-09-14T23:56:52.754 INFO:teuthology.orchestra.run.smithi042.stdout:1071 write 0x121da5b thru 0x12258d8 (0x7e7e bytes) 2024-09-14T23:56:52.758 INFO:teuthology.orchestra.run.smithi042.stdout:1073 punch from 0xe5036c to 0xe5a3c2, (0xa056 bytes) 2024-09-14T23:56:52.758 INFO:teuthology.orchestra.run.smithi042.stdout:1075 read 0x12be6ee thru 0x12ce28d (0xfba0 bytes) 2024-09-14T23:56:52.768 INFO:teuthology.orchestra.run.smithi042.stdout:1076 write 0x4502193 thru 0x450e5bc (0xc42a bytes) 2024-09-14T23:56:52.774 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:56:52.773+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:56:52.778 INFO:teuthology.orchestra.run.smithi042.stdout:1080 writesame 0x723f8e2 thru 0x7247a00 data_size 0xb(0x811f bytes) 2024-09-14T23:56:52.779 INFO:teuthology.orchestra.run.smithi042.stdout:1081 write 0x1136362 thru 0x1139b8e (0x382d bytes) 2024-09-14T23:56:52.781 INFO:teuthology.orchestra.run.smithi042.stdout:1082 punch from 0x4d81ba7 to 0x4d81df6, (0x24f bytes) 2024-09-14T23:56:52.794 INFO:teuthology.orchestra.run.smithi042.stdout:1083 write 0xd4a21cd thru 0xd4ab4ce (0x9302 bytes) 2024-09-14T23:56:52.797 INFO:teuthology.orchestra.run.smithi042.stdout:1084 read 0x4c265f thru 0x4c9e98 (0x783a bytes) 2024-09-14T23:56:52.798 INFO:teuthology.orchestra.run.smithi042.stdout:1086 writesame 0xc861e3a thru 0xc8689b5 data_size 0x6(0x6b7c bytes) 2024-09-14T23:56:52.802 INFO:teuthology.orchestra.run.smithi042.stdout:1087 write 0x5ad71b thru 0x5b2c6c (0x5552 bytes) 2024-09-14T23:56:52.811 INFO:teuthology.orchestra.run.smithi042.stdout:1088 read 0x5434dbd thru 0x5439f42 (0x5186 bytes) 2024-09-14T23:56:52.811 INFO:teuthology.orchestra.run.smithi042.stdout:1091 read 0x8730db4 thru 0x873e9a0 (0xdbed bytes) 2024-09-14T23:56:52.812 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:52.816 INFO:teuthology.orchestra.run.smithi042.stdout:1094 trunc from 0xd4ab4cf to 0x8cb8374 2024-09-14T23:56:52.828 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:56:52.828 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:56:52.828 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:56:52.828 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:56:52.832 INFO:teuthology.orchestra.run.smithi042.stdout:1096 trunc from 0x8cb8374 to 0xeb8e6b2 2024-09-14T23:56:52.837 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T23:56:52.838 INFO:teuthology.orchestra.run.smithi042.stdout:1098 read 0x5bb93e9 thru 0x5bbbd75 (0x298d bytes) 2024-09-14T23:56:52.838 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:52.838 INFO:teuthology.orchestra.run.smithi042.stdout:1100 write 0x73c2b99 thru 0x73d0d0e (0xe176 bytes) 2024-09-14T23:56:52.841 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:52.842 INFO:teuthology.orchestra.run.smithi042.stdout:1102 punch from 0x12c25fc to 0x12c69d1, (0x43d5 bytes) 2024-09-14T23:56:52.842 INFO:teuthology.orchestra.run.smithi042.stdout:1104 read 0xc61cae9 thru 0xc6272cd (0xa7e5 bytes) 2024-09-14T23:56:52.842 INFO:teuthology.orchestra.run.smithi042.stdout:1105 write 0x147f4d4 thru 0x14849a9 (0x54d6 bytes) 2024-09-14T23:56:52.845 INFO:teuthology.orchestra.run.smithi042.stdout:1106 writesame 0xaa2c04e thru 0xaa35e25 data_size 0x4(0x9dd8 bytes) 2024-09-14T23:56:52.851 INFO:teuthology.orchestra.run.smithi042.stdout:1109 write 0xc446907 thru 0xc4499fb (0x30f5 bytes) 2024-09-14T23:56:52.854 INFO:teuthology.orchestra.run.smithi042.stdout:1111 read 0x19f2871 thru 0x19fc193 (0x9923 bytes) 2024-09-14T23:56:52.854 INFO:teuthology.orchestra.run.smithi042.stdout:1113 read 0x298516b thru 0x298852c (0x33c2 bytes) 2024-09-14T23:56:52.854 INFO:teuthology.orchestra.run.smithi042.stdout:1114 write 0xe352006 thru 0xe3615c7 (0xf5c2 bytes) 2024-09-14T23:56:52.857 INFO:teuthology.orchestra.run.smithi042.stdout:1115 write 0x3fde8ea thru 0x3fdfa3c (0x1153 bytes) 2024-09-14T23:56:52.860 INFO:teuthology.orchestra.run.smithi042.stdout:1116 read 0x824614a thru 0x8252358 (0xc20f bytes) 2024-09-14T23:56:52.860 INFO:teuthology.orchestra.run.smithi042.stdout:1117 read 0xe693e6d thru 0xe69e404 (0xa598 bytes) 2024-09-14T23:56:52.860 INFO:teuthology.orchestra.run.smithi042.stdout:1118 write 0xc7d340d thru 0xc7e1a3e (0xe632 bytes) 2024-09-14T23:56:52.864 INFO:teuthology.orchestra.run.smithi042.stdout:1119 read 0xe2a19da thru 0xe2b0212 (0xe839 bytes) 2024-09-14T23:56:52.864 INFO:teuthology.orchestra.run.smithi042.stdout:1120 write 0x7c01b09 thru 0x7c086d6 (0x6bce bytes) 2024-09-14T23:56:52.867 INFO:teuthology.orchestra.run.smithi042.stdout:1121 read 0x43be9b9 thru 0x43c3752 (0x4d9a bytes) 2024-09-14T23:56:52.867 INFO:teuthology.orchestra.run.smithi042.stdout:1122 punch from 0xaee1c54 to 0xaee682b, (0x4bd7 bytes) 2024-09-14T23:56:52.867 INFO:teuthology.orchestra.run.smithi042.stdout:1123 writesame 0xcd7546 thru 0xcd8465 data_size 0x20(0xf20 bytes) 2024-09-14T23:56:52.869 INFO:teuthology.orchestra.run.smithi042.stdout:1124 read 0x9e923d thru 0x9ee289 (0x504d bytes) 2024-09-14T23:56:52.873 INFO:teuthology.orchestra.run.smithi042.stdout:1125 trunc from 0xeb8e6b2 to 0xc1f98ac 2024-09-14T23:56:52.881 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:56:52.880+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:56:52.882 INFO:teuthology.orchestra.run.smithi042.stdout:1126 punch from 0xb94ac0d to 0xb951058, (0x644b bytes) 2024-09-14T23:56:52.883 INFO:teuthology.orchestra.run.smithi042.stdout:1128 read 0x35f194f thru 0x35f2915 (0xfc7 bytes) 2024-09-14T23:56:52.883 INFO:teuthology.orchestra.run.smithi042.stdout:1130 punch from 0x4f9905 to 0x501284, (0x797f bytes) 2024-09-14T23:56:52.883 INFO:teuthology.orchestra.run.smithi042.stdout:1131 write 0x9fbe9be thru 0x9fcb66b (0xccae bytes) 2024-09-14T23:56:52.894 INFO:teuthology.orchestra.run.smithi042.stdout:1132 trunc from 0xc1f98ac to 0x5a610c4 2024-09-14T23:56:52.904 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:52.904 INFO:teuthology.orchestra.run.smithi042.stdout:1136 read 0xab0abc thru 0xab965c (0x8ba1 bytes) 2024-09-14T23:56:52.911 INFO:teuthology.orchestra.run.smithi042.stdout:1137 trunc from 0x5a610c4 to 0xb447e43 2024-09-14T23:56:52.916 INFO:teuthology.orchestra.run.smithi042.stdout:1138 read 0x1774431 thru 0x177f89e (0xb46e bytes) 2024-09-14T23:56:52.916 INFO:teuthology.orchestra.run.smithi042.stdout:1139 punch from 0x3357899 to 0x335ef71, (0x76d8 bytes) 2024-09-14T23:56:52.927 INFO:teuthology.orchestra.run.smithi042.stdout:1140 trunc from 0xb447e43 to 0x17ac4ac 2024-09-14T23:56:52.937 INFO:teuthology.orchestra.run.smithi042.stdout:1141 read 0x142d639 thru 0x1434a64 (0x742c bytes) 2024-09-14T23:56:52.937 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:52.946 INFO:teuthology.orchestra.run.smithi042.stdout:1145 writesame 0x40a3483 thru 0x40a9b68 data_size 0x1(0x66e6 bytes) 2024-09-14T23:56:52.946 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:52.946 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:52.946 INFO:teuthology.orchestra.run.smithi042.stdout:1149 punch from 0x4bf6c to 0x57419, (0xb4ad bytes) 2024-09-14T23:56:52.946 INFO:teuthology.orchestra.run.smithi042.stdout:1150 read 0x1b8f1b2 thru 0x1b9a863 (0xb6b2 bytes) 2024-09-14T23:56:52.946 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:52.946 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:52.951 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:56:52.951 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:56:52.951 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:56:52.951 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:56:52.951 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:56:52.960 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T23:56:52.965 INFO:teuthology.orchestra.run.smithi042.stdout:1155 write 0xbb4e774 thru 0xbb558f1 (0x717e bytes) 2024-09-14T23:56:53.001 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:56:53.000+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:56:53.076 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:56:53.075+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:56:53.110 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:56:53.110 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:56:53.110 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:56:53.110 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:56:53.110 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.110 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30565 3.15 3:acd4c478:::rbd_data.1118fff5d5e0.0000000000000001:8 [read 3043328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:53.110 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.872553+0000", 2024-09-14T23:56:53.111 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.2354969740000001, 2024-09-14T23:56:53.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.020786221000000001, 2024-09-14T23:56:53.111 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:53.111 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:53.111 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:53.111 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:53.111 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:53.111 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:53.111 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30565 2024-09-14T23:56:53.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.111 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:53.111 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:53.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872553+0000", 2024-09-14T23:56:53.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:53.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:53.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872553+0000", 2024-09-14T23:56:53.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:53.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:53.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872553+0000", 2024-09-14T23:56:53.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2000000000000001e-07 2024-09-14T23:56:53.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.113 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:53.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872555+0000", 2024-09-14T23:56:53.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8059999999999999e-06 2024-09-14T23:56:53.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.113 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:53.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872561+0000", 2024-09-14T23:56:53.114 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6110000000000001e-06 2024-09-14T23:56:53.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.114 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:53.114 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872565+0000", 2024-09-14T23:56:53.114 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3919999999999996e-06 2024-09-14T23:56:53.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.114 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:53.114 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.893155+0000", 2024-09-14T23:56:53.114 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.020590286999999999 2024-09-14T23:56:53.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.114 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:53.115 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.893242+0000", 2024-09-14T23:56:53.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6644999999999995e-05 2024-09-14T23:56:53.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.115 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:53.115 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.893339+0000", 2024-09-14T23:56:53.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7159999999999998e-05 2024-09-14T23:56:53.115 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:53.115 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:53.115 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:53.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.116 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.116 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30567 3.15 3:acd4c478:::rbd_data.1118fff5d5e0.0000000000000001:8 [read 3104768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:53.116 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.872570+0000", 2024-09-14T23:56:53.116 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.2354797450000001, 2024-09-14T23:56:53.116 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.020969402000000002, 2024-09-14T23:56:53.116 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:53.116 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:53.116 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:53.116 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:53.116 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:53.116 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:53.116 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30567 2024-09-14T23:56:53.116 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.117 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:53.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:53.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872570+0000", 2024-09-14T23:56:53.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:53.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:53.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872570+0000", 2024-09-14T23:56:53.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:53.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:53.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872571+0000", 2024-09-14T23:56:53.118 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7599999999999997e-07 2024-09-14T23:56:53.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.118 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.118 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:53.118 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872573+0000", 2024-09-14T23:56:53.118 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.742e-06 2024-09-14T23:56:53.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.118 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.118 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:53.118 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872578+0000", 2024-09-14T23:56:53.118 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5330000000000002e-06 2024-09-14T23:56:53.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.119 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:53.119 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872584+0000", 2024-09-14T23:56:53.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1890000000000004e-06 2024-09-14T23:56:53.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.119 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:53.119 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.893368+0000", 2024-09-14T23:56:53.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.020783414 2024-09-14T23:56:53.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.119 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:53.119 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.893447+0000", 2024-09-14T23:56:53.120 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9153000000000006e-05 2024-09-14T23:56:53.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.120 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.120 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:53.120 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.893540+0000", 2024-09-14T23:56:53.120 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2894999999999997e-05 2024-09-14T23:56:53.120 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:53.120 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:53.120 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:53.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.121 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.121 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30569 3.15 3:acd4c478:::rbd_data.1118fff5d5e0.0000000000000001:8 [read 3624960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:53.121 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.872869+0000", 2024-09-14T23:56:53.121 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.2351814269999997, 2024-09-14T23:56:53.121 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.020954618000000001, 2024-09-14T23:56:53.121 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:53.121 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:53.121 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:53.121 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:53.121 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:53.122 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:53.122 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30569 2024-09-14T23:56:53.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.122 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:53.122 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.122 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:53.122 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872869+0000", 2024-09-14T23:56:53.122 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:53.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.122 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.122 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:53.122 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872869+0000", 2024-09-14T23:56:53.122 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:53.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.123 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.123 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:53.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872869+0000", 2024-09-14T23:56:53.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7999999999999996e-07 2024-09-14T23:56:53.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.123 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.123 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:53.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872872+0000", 2024-09-14T23:56:53.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3400000000000002e-06 2024-09-14T23:56:53.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.123 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.123 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:53.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872882+0000", 2024-09-14T23:56:53.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5060000000000001e-06 2024-09-14T23:56:53.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.124 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:53.124 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872888+0000", 2024-09-14T23:56:53.124 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2249999999999997e-06 2024-09-14T23:56:53.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.124 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:53.124 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.893581+0000", 2024-09-14T23:56:53.124 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.020692933 2024-09-14T23:56:53.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.124 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:53.124 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.893685+0000", 2024-09-14T23:56:53.125 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000103936 2024-09-14T23:56:53.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.125 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.125 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:53.125 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.893823+0000", 2024-09-14T23:56:53.125 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013819799999999999 2024-09-14T23:56:53.125 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:53.125 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:53.125 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:53.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.126 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.126 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30570 3.15 3:acd4c478:::rbd_data.1118fff5d5e0.0000000000000001:8 [read 2637824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:53.126 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.872897+0000", 2024-09-14T23:56:53.126 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.235152888, 2024-09-14T23:56:53.126 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021159374000000002, 2024-09-14T23:56:53.126 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:53.126 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:53.126 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:53.127 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:53.127 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:53.127 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:53.127 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30570 2024-09-14T23:56:53.127 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.127 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:53.127 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.127 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:53.127 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872897+0000", 2024-09-14T23:56:53.127 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:53.127 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.127 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.127 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:53.127 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872897+0000", 2024-09-14T23:56:53.128 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:53.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.128 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.128 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:53.128 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872898+0000", 2024-09-14T23:56:53.128 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8000000000000001e-07 2024-09-14T23:56:53.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.128 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.128 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:53.128 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872905+0000", 2024-09-14T23:56:53.128 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6820000000000003e-06 2024-09-14T23:56:53.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.128 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.129 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:53.129 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872912+0000", 2024-09-14T23:56:53.129 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8770000000000004e-06 2024-09-14T23:56:53.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.129 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.129 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:53.129 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872918+0000", 2024-09-14T23:56:53.129 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7620000000000001e-06 2024-09-14T23:56:53.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.129 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.129 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:53.129 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.893868+0000", 2024-09-14T23:56:53.129 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.020949857999999998 2024-09-14T23:56:53.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.130 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.130 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:53.130 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.893950+0000", 2024-09-14T23:56:53.130 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2177999999999998e-05 2024-09-14T23:56:53.130 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.130 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.130 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:53.130 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.894057+0000", 2024-09-14T23:56:53.130 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000106637 2024-09-14T23:56:53.130 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:53.130 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:53.130 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:53.130 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.130 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.131 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30573 3.15 3:acd4c478:::rbd_data.1118fff5d5e0.0000000000000001:8 [read 3158016~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:53.131 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.872924+0000", 2024-09-14T23:56:53.131 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.2351256319999999, 2024-09-14T23:56:53.131 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021329519000000002, 2024-09-14T23:56:53.131 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:53.131 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:53.131 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:53.131 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:53.131 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:53.131 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:53.131 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30573 2024-09-14T23:56:53.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.132 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:53.132 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.132 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:53.132 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872924+0000", 2024-09-14T23:56:53.132 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:53.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.132 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.132 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:53.132 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872924+0000", 2024-09-14T23:56:53.132 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:53.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.133 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:53.133 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872925+0000", 2024-09-14T23:56:53.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9999999999999998e-07 2024-09-14T23:56:53.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.133 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:53.133 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872927+0000", 2024-09-14T23:56:53.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.765e-06 2024-09-14T23:56:53.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.133 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:53.133 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872933+0000", 2024-09-14T23:56:53.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8610000000000003e-06 2024-09-14T23:56:53.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:53.134 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872937+0000", 2024-09-14T23:56:53.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5839999999999996e-06 2024-09-14T23:56:53.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:53.134 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.894117+0000", 2024-09-14T23:56:53.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021179429 2024-09-14T23:56:53.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:53.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.894170+0000", 2024-09-14T23:56:53.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3122999999999999e-05 2024-09-14T23:56:53.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.135 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:53.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.894254+0000", 2024-09-14T23:56:53.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4257000000000003e-05 2024-09-14T23:56:53.135 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:53.135 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:53.135 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:53.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.135 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30575 3.15 3:acd4c478:::rbd_data.1118fff5d5e0.0000000000000001:8 [read 3219456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:53.136 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.872943+0000", 2024-09-14T23:56:53.136 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.2351069519999998, 2024-09-14T23:56:53.136 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021535109, 2024-09-14T23:56:53.136 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:53.136 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:53.136 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:53.136 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:53.136 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:53.136 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:53.136 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30575 2024-09-14T23:56:53.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.136 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:53.136 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:53.137 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872943+0000", 2024-09-14T23:56:53.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:53.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:53.137 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872943+0000", 2024-09-14T23:56:53.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:53.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:53.137 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872943+0000", 2024-09-14T23:56:53.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.72e-07 2024-09-14T23:56:53.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:53.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872945+0000", 2024-09-14T23:56:53.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.951e-06 2024-09-14T23:56:53.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:53.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872952+0000", 2024-09-14T23:56:53.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.268e-06 2024-09-14T23:56:53.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.139 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:53.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872956+0000", 2024-09-14T23:56:53.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8010000000000003e-06 2024-09-14T23:56:53.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.139 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:53.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.894295+0000", 2024-09-14T23:56:53.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021338475999999999 2024-09-14T23:56:53.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.139 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:53.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.894380+0000", 2024-09-14T23:56:53.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5222999999999994e-05 2024-09-14T23:56:53.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.140 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.140 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:53.140 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.894478+0000", 2024-09-14T23:56:53.140 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8017999999999996e-05 2024-09-14T23:56:53.140 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:53.140 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:53.140 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:53.140 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.140 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.140 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30577 3.15 3:acd4c478:::rbd_data.1118fff5d5e0.0000000000000001:8 [read 3334144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:53.140 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.872963+0000", 2024-09-14T23:56:53.140 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.2350871489999999, 2024-09-14T23:56:53.140 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021659955000000002, 2024-09-14T23:56:53.141 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:53.141 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:53.141 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:53.141 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:53.141 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:53.141 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:53.141 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30577 2024-09-14T23:56:53.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.141 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:53.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.141 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:53.141 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872963+0000", 2024-09-14T23:56:53.141 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:53.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.142 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:53.142 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872963+0000", 2024-09-14T23:56:53.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:53.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.142 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:53.142 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872963+0000", 2024-09-14T23:56:53.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1899999999999998e-07 2024-09-14T23:56:53.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.142 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:53.142 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872966+0000", 2024-09-14T23:56:53.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.317e-06 2024-09-14T23:56:53.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:53.143 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872972+0000", 2024-09-14T23:56:53.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1859999999999997e-06 2024-09-14T23:56:53.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:53.143 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872977+0000", 2024-09-14T23:56:53.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9259999999999999e-06 2024-09-14T23:56:53.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:53.143 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.894517+0000", 2024-09-14T23:56:53.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021539776 2024-09-14T23:56:53.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.144 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.144 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:53.144 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.894565+0000", 2024-09-14T23:56:53.144 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8968999999999998e-05 2024-09-14T23:56:53.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.144 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.144 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:53.144 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.894623+0000", 2024-09-14T23:56:53.144 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7361999999999997e-05 2024-09-14T23:56:53.144 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:53.144 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:53.144 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:53.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.145 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.145 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30580 3.15 3:acd4c478:::rbd_data.1118fff5d5e0.0000000000000001:8 [read 3395584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:53.145 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.872983+0000", 2024-09-14T23:56:53.145 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.2350672300000003, 2024-09-14T23:56:53.145 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021878231000000001, 2024-09-14T23:56:53.145 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:53.145 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:53.145 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:53.145 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:53.145 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:53.145 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:53.146 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30580 2024-09-14T23:56:53.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.146 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:53.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.146 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:53.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872983+0000", 2024-09-14T23:56:53.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:53.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.146 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:53.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872983+0000", 2024-09-14T23:56:53.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:53.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.147 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:53.147 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872983+0000", 2024-09-14T23:56:53.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2800000000000003e-07 2024-09-14T23:56:53.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.147 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:53.147 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872985+0000", 2024-09-14T23:56:53.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.804e-06 2024-09-14T23:56:53.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.147 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:53.147 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872992+0000", 2024-09-14T23:56:53.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3950000000000003e-06 2024-09-14T23:56:53.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.148 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.148 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:53.148 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872998+0000", 2024-09-14T23:56:53.148 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1109999999999996e-06 2024-09-14T23:56:53.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.148 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.148 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:53.148 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.894679+0000", 2024-09-14T23:56:53.148 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021680095999999999 2024-09-14T23:56:53.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.148 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.148 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:53.148 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.894763+0000", 2024-09-14T23:56:53.149 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.475e-05 2024-09-14T23:56:53.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.149 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:53.149 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.894861+0000", 2024-09-14T23:56:53.149 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7746999999999998e-05 2024-09-14T23:56:53.149 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:53.149 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:53.149 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:53.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.149 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30584 3.15 3:acd4c478:::rbd_data.1118fff5d5e0.0000000000000001:8 [read 2932736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:53.149 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.873005+0000", 2024-09-14T23:56:53.150 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.2350450909999999, 2024-09-14T23:56:53.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021964951, 2024-09-14T23:56:53.150 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:53.150 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:53.150 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:53.150 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:53.150 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:53.150 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:53.150 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30584 2024-09-14T23:56:53.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.150 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:53.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:53.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873005+0000", 2024-09-14T23:56:53.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:53.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:53.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873005+0000", 2024-09-14T23:56:53.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:53.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:53.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873005+0000", 2024-09-14T23:56:53.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6300000000000001e-07 2024-09-14T23:56:53.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:53.152 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873007+0000", 2024-09-14T23:56:53.152 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8670000000000001e-06 2024-09-14T23:56:53.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.152 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:53.152 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873013+0000", 2024-09-14T23:56:53.152 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5829999999999997e-06 2024-09-14T23:56:53.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.152 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:53.152 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873017+0000", 2024-09-14T23:56:53.152 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3669999999999999e-06 2024-09-14T23:56:53.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.153 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.153 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:53.153 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.894876+0000", 2024-09-14T23:56:53.153 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021858532 2024-09-14T23:56:53.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.153 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.153 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:53.153 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.894917+0000", 2024-09-14T23:56:53.154 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1485e-05 2024-09-14T23:56:53.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.154 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.154 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:53.154 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.894970+0000", 2024-09-14T23:56:53.154 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2753999999999997e-05 2024-09-14T23:56:53.154 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:53.154 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:53.154 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:53.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.154 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.154 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30586 3.15 3:acd4c478:::rbd_data.1118fff5d5e0.0000000000000001:8 [read 2994176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:53.155 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.873023+0000", 2024-09-14T23:56:53.155 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.2350271179999996, 2024-09-14T23:56:53.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.022266837000000001, 2024-09-14T23:56:53.155 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:53.155 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:53.155 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:53.155 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:53.155 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:53.155 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:53.155 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30586 2024-09-14T23:56:53.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.156 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:53.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.156 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:53.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873023+0000", 2024-09-14T23:56:53.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:53.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.156 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:53.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873023+0000", 2024-09-14T23:56:53.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:53.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.156 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:53.157 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873023+0000", 2024-09-14T23:56:53.157 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2500000000000001e-07 2024-09-14T23:56:53.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.157 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.157 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:53.157 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873026+0000", 2024-09-14T23:56:53.157 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7199999999999998e-06 2024-09-14T23:56:53.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.157 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.157 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:53.157 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873032+0000", 2024-09-14T23:56:53.158 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.494e-06 2024-09-14T23:56:53.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.158 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.158 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:53.158 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873037+0000", 2024-09-14T23:56:53.158 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4880000000000001e-06 2024-09-14T23:56:53.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.158 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.158 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:53.158 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.895006+0000", 2024-09-14T23:56:53.158 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021968713000000001 2024-09-14T23:56:53.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.159 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:53.159 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.895142+0000", 2024-09-14T23:56:53.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013594900000000001 2024-09-14T23:56:53.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.159 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:53.159 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.895290+0000", 2024-09-14T23:56:53.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014814799999999999 2024-09-14T23:56:53.159 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:53.159 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:53.159 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:53.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.160 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.160 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30588 3.15 3:acd4c478:::rbd_data.1118fff5d5e0.0000000000000001:8 [read 3514368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:53.160 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.873042+0000", 2024-09-14T23:56:53.160 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.2350078120000001, 2024-09-14T23:56:53.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.022463265, 2024-09-14T23:56:53.160 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:53.160 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:53.160 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:53.160 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:53.160 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:53.160 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:53.160 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30588 2024-09-14T23:56:53.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.161 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:53.161 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.161 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:53.161 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873042+0000", 2024-09-14T23:56:53.161 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:53.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.161 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.161 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:53.161 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873042+0000", 2024-09-14T23:56:53.161 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:53.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.161 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.161 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:53.161 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873043+0000", 2024-09-14T23:56:53.161 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4299999999999999e-07 2024-09-14T23:56:53.162 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.162 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.162 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:53.162 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873044+0000", 2024-09-14T23:56:53.162 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.68e-06 2024-09-14T23:56:53.162 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.162 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.162 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:53.162 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873050+0000", 2024-09-14T23:56:53.162 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4979999999999997e-06 2024-09-14T23:56:53.162 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.162 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.162 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:53.163 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873054+0000", 2024-09-14T23:56:53.163 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5510000000000001e-06 2024-09-14T23:56:53.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.163 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:53.163 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.895322+0000", 2024-09-14T23:56:53.163 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.022267797999999998 2024-09-14T23:56:53.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.163 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:53.163 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.895404+0000", 2024-09-14T23:56:53.163 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1803000000000003e-05 2024-09-14T23:56:53.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.164 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:53.164 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.895506+0000", 2024-09-14T23:56:53.164 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101592 2024-09-14T23:56:53.164 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:53.164 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:53.164 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:53.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.164 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.164 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30590 3.15 3:acd4c478:::rbd_data.1118fff5d5e0.0000000000000001:8 [read 3047424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:53.164 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.873059+0000", 2024-09-14T23:56:53.164 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.2349908510000001, 2024-09-14T23:56:53.164 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.022706343, 2024-09-14T23:56:53.165 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:53.165 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:53.165 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:53.165 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:53.165 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:53.165 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:53.165 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30590 2024-09-14T23:56:53.165 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.165 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:53.165 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.165 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:53.165 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873059+0000", 2024-09-14T23:56:53.165 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:53.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.166 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.166 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:53.166 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873059+0000", 2024-09-14T23:56:53.166 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:53.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.166 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.166 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:53.166 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873060+0000", 2024-09-14T23:56:53.166 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7e-07 2024-09-14T23:56:53.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.166 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.166 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:53.166 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873061+0000", 2024-09-14T23:56:53.167 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.652e-06 2024-09-14T23:56:53.167 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.167 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.167 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:53.167 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873067+0000", 2024-09-14T23:56:53.167 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5570000000000003e-06 2024-09-14T23:56:53.167 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.167 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.167 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:53.167 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873071+0000", 2024-09-14T23:56:53.167 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.33e-06 2024-09-14T23:56:53.167 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.167 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.168 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:53.168 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.895536+0000", 2024-09-14T23:56:53.168 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.022464631999999998 2024-09-14T23:56:53.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.168 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.168 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:53.168 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.895617+0000", 2024-09-14T23:56:53.168 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1316000000000005e-05 2024-09-14T23:56:53.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.168 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.168 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:53.168 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.895766+0000", 2024-09-14T23:56:53.168 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000148386 2024-09-14T23:56:53.168 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:53.169 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:53.169 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:53.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.169 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30593 3.15 3:acd4c478:::rbd_data.1118fff5d5e0.0000000000000001:8 [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:53.169 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.873076+0000", 2024-09-14T23:56:53.169 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.2349737730000001, 2024-09-14T23:56:53.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.02293823, 2024-09-14T23:56:53.169 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:53.169 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:53.169 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:53.169 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:53.169 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:53.170 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:53.170 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30593 2024-09-14T23:56:53.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.170 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:53.170 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.170 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:53.170 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873076+0000", 2024-09-14T23:56:53.170 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:53.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.170 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.170 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:53.170 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873076+0000", 2024-09-14T23:56:53.170 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:53.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.171 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.171 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:53.171 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873077+0000", 2024-09-14T23:56:53.171 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2399999999999999e-07 2024-09-14T23:56:53.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.171 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.171 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:53.171 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873078+0000", 2024-09-14T23:56:53.171 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6279999999999999e-06 2024-09-14T23:56:53.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.171 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.171 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:53.171 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873084+0000", 2024-09-14T23:56:53.172 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4940000000000003e-06 2024-09-14T23:56:53.172 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.172 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.172 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:53.172 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873089+0000", 2024-09-14T23:56:53.172 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8189999999999999e-06 2024-09-14T23:56:53.172 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.172 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.172 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:53.172 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.895827+0000", 2024-09-14T23:56:53.172 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.022738441000000002 2024-09-14T23:56:53.172 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.172 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.173 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:53.173 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.895911+0000", 2024-09-14T23:56:53.173 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4128999999999994e-05 2024-09-14T23:56:53.173 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.173 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.173 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:53.173 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.896015+0000", 2024-09-14T23:56:53.174 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000103395 2024-09-14T23:56:53.174 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:53.174 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:53.174 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:53.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.174 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.174 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30595 3.15 3:acd4c478:::rbd_data.1118fff5d5e0.0000000000000001:8 [read 3629056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:53.174 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.873244+0000", 2024-09-14T23:56:53.174 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.2348065029999997, 2024-09-14T23:56:53.174 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.022932491999999999, 2024-09-14T23:56:53.174 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:53.174 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:53.174 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:53.174 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:53.175 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:53.175 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:53.175 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30595 2024-09-14T23:56:53.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.175 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:53.175 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.175 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:53.175 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873244+0000", 2024-09-14T23:56:53.175 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:53.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.175 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.175 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:53.175 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873244+0000", 2024-09-14T23:56:53.175 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:53.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.176 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.176 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:53.176 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873244+0000", 2024-09-14T23:56:53.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7199999999999999e-07 2024-09-14T23:56:53.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.176 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.176 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:53.176 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873247+0000", 2024-09-14T23:56:53.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.208e-06 2024-09-14T23:56:53.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.176 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.176 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:53.176 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873266+0000", 2024-09-14T23:56:53.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8689999999999999e-05 2024-09-14T23:56:53.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.177 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:53.177 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873274+0000", 2024-09-14T23:56:53.177 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.089e-06 2024-09-14T23:56:53.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.177 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:53.178 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.896055+0000", 2024-09-14T23:56:53.178 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.02278059 2024-09-14T23:56:53.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.178 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:53.181 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.896107+0000", 2024-09-14T23:56:53.181 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2645000000000002e-05 2024-09-14T23:56:53.181 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.181 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.181 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:53.181 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.896176+0000", 2024-09-14T23:56:53.181 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8697999999999999e-05 2024-09-14T23:56:53.181 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:53.181 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:53.181 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:53.181 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.182 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.182 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30597 3.15 3:acd4c478:::rbd_data.1118fff5d5e0.0000000000000001:8 [read 2641920~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:53.182 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.873284+0000", 2024-09-14T23:56:53.182 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.2347662699999997, 2024-09-14T23:56:53.182 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.023132685, 2024-09-14T23:56:53.182 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:53.182 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:53.182 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:53.182 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:53.182 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:53.182 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:53.182 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30597 2024-09-14T23:56:53.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.182 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:53.183 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.183 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:53.183 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873284+0000", 2024-09-14T23:56:53.183 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:53.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.183 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.183 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:53.183 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873284+0000", 2024-09-14T23:56:53.183 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:53.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.183 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.183 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:53.183 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873284+0000", 2024-09-14T23:56:53.183 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.08e-07 2024-09-14T23:56:53.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.184 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:53.184 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873287+0000", 2024-09-14T23:56:53.184 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4269999999999998e-06 2024-09-14T23:56:53.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.184 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:53.184 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873293+0000", 2024-09-14T23:56:53.184 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3489999999999997e-06 2024-09-14T23:56:53.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.184 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:53.185 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873298+0000", 2024-09-14T23:56:53.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8559999999999998e-06 2024-09-14T23:56:53.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.185 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.185 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:53.185 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.896220+0000", 2024-09-14T23:56:53.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.022921725 2024-09-14T23:56:53.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.185 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.185 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:53.185 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.896313+0000", 2024-09-14T23:56:53.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3697999999999996e-05 2024-09-14T23:56:53.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.186 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.186 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:53.186 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.896416+0000", 2024-09-14T23:56:53.186 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000103222 2024-09-14T23:56:53.186 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:53.186 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:53.186 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:53.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.186 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.186 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30599 3.15 3:acd4c478:::rbd_data.1118fff5d5e0.0000000000000001:8 [read 3162112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:53.186 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.873304+0000", 2024-09-14T23:56:53.186 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.2347459799999996, 2024-09-14T23:56:53.187 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.023260915, 2024-09-14T23:56:53.187 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:53.187 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:53.187 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:53.187 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:53.187 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:53.187 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:53.187 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30599 2024-09-14T23:56:53.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.187 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:53.187 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.187 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:53.187 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873304+0000", 2024-09-14T23:56:53.188 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:53.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.188 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.188 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:53.188 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873304+0000", 2024-09-14T23:56:53.188 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:53.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.188 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.188 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:53.188 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873304+0000", 2024-09-14T23:56:53.188 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6899999999999998e-07 2024-09-14T23:56:53.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.188 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.189 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:53.189 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873313+0000", 2024-09-14T23:56:53.189 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3450000000000006e-06 2024-09-14T23:56:53.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.189 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.189 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:53.189 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873319+0000", 2024-09-14T23:56:53.189 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3609999999999997e-06 2024-09-14T23:56:53.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.189 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.189 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:53.189 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873324+0000", 2024-09-14T23:56:53.189 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8300000000000003e-06 2024-09-14T23:56:53.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.190 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.190 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:53.190 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.896454+0000", 2024-09-14T23:56:53.190 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.023130064999999998 2024-09-14T23:56:53.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.190 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.190 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:53.190 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.896503+0000", 2024-09-14T23:56:53.190 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8900000000000003e-05 2024-09-14T23:56:53.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.190 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.190 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:53.191 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.896565+0000", 2024-09-14T23:56:53.191 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2045000000000007e-05 2024-09-14T23:56:53.191 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:53.191 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:53.191 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:53.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.191 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.191 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30601 3.15 3:acd4c478:::rbd_data.1118fff5d5e0.0000000000000001:8 [read 3223552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:53.191 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.873359+0000", 2024-09-14T23:56:53.191 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.2346914660000001, 2024-09-14T23:56:53.191 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.023428353999999998, 2024-09-14T23:56:53.191 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:53.191 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:53.191 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:53.192 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:53.192 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:53.192 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:53.192 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30601 2024-09-14T23:56:53.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.192 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:53.192 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.192 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:53.192 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873359+0000", 2024-09-14T23:56:53.192 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:53.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.192 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.192 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:53.192 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873359+0000", 2024-09-14T23:56:53.193 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:53.193 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.193 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.193 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:53.193 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873359+0000", 2024-09-14T23:56:53.193 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7199999999999998e-07 2024-09-14T23:56:53.193 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.193 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.193 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:53.194 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873363+0000", 2024-09-14T23:56:53.194 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9219999999999999e-06 2024-09-14T23:56:53.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.194 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.194 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:53.194 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873375+0000", 2024-09-14T23:56:53.194 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2078e-05 2024-09-14T23:56:53.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.195 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.195 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:53.195 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873383+0000", 2024-09-14T23:56:53.195 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7069999999999992e-06 2024-09-14T23:56:53.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.195 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.195 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:53.195 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.896603+0000", 2024-09-14T23:56:53.195 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.023219758 2024-09-14T23:56:53.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.195 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.195 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:53.196 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.896693+0000", 2024-09-14T23:56:53.196 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0600999999999995e-05 2024-09-14T23:56:53.196 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.196 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.196 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:53.196 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.896787+0000", 2024-09-14T23:56:53.196 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3516000000000004e-05 2024-09-14T23:56:53.196 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:53.196 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:53.196 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:53.196 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.196 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.197 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30603 3.15 3:acd4c478:::rbd_data.1118fff5d5e0.0000000000000001:8 [read 3276800~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:53.197 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.873395+0000", 2024-09-14T23:56:53.197 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.2346554550000004, 2024-09-14T23:56:53.197 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.023533291000000001, 2024-09-14T23:56:53.197 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:53.197 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:53.197 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:53.197 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:53.197 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:53.197 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:53.197 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30603 2024-09-14T23:56:53.197 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.197 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:53.198 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.198 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:53.198 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873395+0000", 2024-09-14T23:56:53.198 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:53.198 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.198 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.198 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:53.198 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873395+0000", 2024-09-14T23:56:53.198 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:53.198 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.198 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.198 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:53.198 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873395+0000", 2024-09-14T23:56:53.198 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9299999999999998e-07 2024-09-14T23:56:53.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.199 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.199 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:53.199 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873398+0000", 2024-09-14T23:56:53.199 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.481e-06 2024-09-14T23:56:53.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.199 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.199 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:53.199 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873405+0000", 2024-09-14T23:56:53.199 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.25e-06 2024-09-14T23:56:53.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.199 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.199 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:53.200 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873411+0000", 2024-09-14T23:56:53.200 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9359999999999996e-06 2024-09-14T23:56:53.200 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.200 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.200 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:53.200 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.896817+0000", 2024-09-14T23:56:53.200 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.023406347000000001 2024-09-14T23:56:53.200 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.200 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.200 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:53.200 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.896868+0000", 2024-09-14T23:56:53.201 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1258000000000003e-05 2024-09-14T23:56:53.201 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.201 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.201 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:53.201 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.896928+0000", 2024-09-14T23:56:53.201 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9426000000000003e-05 2024-09-14T23:56:53.201 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:53.201 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:53.201 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:53.201 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.201 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.201 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30605 3.15 3:acd4c478:::rbd_data.1118fff5d5e0.0000000000000001:8 [read 3338240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:53.201 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.873418+0000", 2024-09-14T23:56:53.202 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.2346321729999996, 2024-09-14T23:56:53.202 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.023645488999999999, 2024-09-14T23:56:53.202 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:53.202 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:53.202 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:53.202 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:53.202 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:53.202 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:53.202 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30605 2024-09-14T23:56:53.202 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.202 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:53.202 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.202 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:53.202 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873418+0000", 2024-09-14T23:56:53.203 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:53.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.203 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.203 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:53.203 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873418+0000", 2024-09-14T23:56:53.203 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:53.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.203 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.203 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:53.203 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873418+0000", 2024-09-14T23:56:53.203 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5200000000000002e-07 2024-09-14T23:56:53.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.203 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.203 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:53.204 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873422+0000", 2024-09-14T23:56:53.204 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0450000000000002e-06 2024-09-14T23:56:53.204 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.204 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.204 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:53.204 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873429+0000", 2024-09-14T23:56:53.204 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5950000000000004e-06 2024-09-14T23:56:53.204 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.204 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.204 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:53.204 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873434+0000", 2024-09-14T23:56:53.205 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4389999999999999e-06 2024-09-14T23:56:53.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.205 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.205 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:53.205 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.896957+0000", 2024-09-14T23:56:53.205 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.023522568000000001 2024-09-14T23:56:53.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.205 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.205 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:53.205 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.897006+0000", 2024-09-14T23:56:53.205 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8925000000000003e-05 2024-09-14T23:56:53.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.206 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.206 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:53.206 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.897063+0000", 2024-09-14T23:56:53.206 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7465e-05 2024-09-14T23:56:53.206 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:53.206 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:53.206 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:53.206 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.206 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.206 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30608 3.15 3:acd4c478:::rbd_data.1118fff5d5e0.0000000000000001:8 [read 3399680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:53.206 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.873441+0000", 2024-09-14T23:56:53.206 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.2346089610000002, 2024-09-14T23:56:53.206 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.023846589000000001, 2024-09-14T23:56:53.206 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:53.207 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:53.207 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:53.207 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:53.207 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:53.207 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:53.207 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30608 2024-09-14T23:56:53.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.207 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:53.207 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.207 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:53.207 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873441+0000", 2024-09-14T23:56:53.207 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:53.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.207 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.207 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:53.208 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873441+0000", 2024-09-14T23:56:53.208 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:53.208 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.208 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.208 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:53.208 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873442+0000", 2024-09-14T23:56:53.208 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2600000000000002e-07 2024-09-14T23:56:53.208 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.208 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.208 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:53.208 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873444+0000", 2024-09-14T23:56:53.208 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.238e-06 2024-09-14T23:56:53.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.209 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.209 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:53.209 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873451+0000", 2024-09-14T23:56:53.209 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.861e-06 2024-09-14T23:56:53.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.209 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.209 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:53.209 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873456+0000", 2024-09-14T23:56:53.209 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1490000000000002e-06 2024-09-14T23:56:53.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.209 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.210 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:53.210 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.897094+0000", 2024-09-14T23:56:53.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.023638461 2024-09-14T23:56:53.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.210 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:53.210 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.897187+0000", 2024-09-14T23:56:53.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2763000000000006e-05 2024-09-14T23:56:53.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:53.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:53.210 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:53.210 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.897288+0000", 2024-09-14T23:56:53.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100591 2024-09-14T23:56:53.210 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:53.211 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:53.211 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:53.211 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:53.211 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:53.211 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:56:53.212 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:56:53.211+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:56:53.213 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T23:56:53.314 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:56:53.312+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:56:53.323 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:56:53.323 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:56:53.323 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.323 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:43054 3.1f 3.69a7fdff (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T23:56:53.323 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:56:53.323 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.355605422, 2024-09-14T23:56:53.323 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.35562635999999997, 2024-09-14T23:56:53.323 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:53.323 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:53.323 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "waiting for peered", 2024-09-14T23:56:53.323 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:53.323 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:53.324 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:53.324 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 43054 2024-09-14T23:56:53.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.324 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:53.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.324 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:53.324 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:56:53.324 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:53.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.324 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:53.324 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:56:53.324 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:53.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.325 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:53.325 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966552+0000", 2024-09-14T23:56:53.325 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3790000000000001e-06 2024-09-14T23:56:53.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.325 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:53.325 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966565+0000", 2024-09-14T23:56:53.325 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2459999999999999e-05 2024-09-14T23:56:53.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.325 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:53.325 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966576+0000", 2024-09-14T23:56:53.325 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1039000000000001e-05 2024-09-14T23:56:53.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.326 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.326 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:53.326 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966582+0000", 2024-09-14T23:56:53.326 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9660000000000001e-06 2024-09-14T23:56:53.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.326 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.326 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:53.326 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966625+0000", 2024-09-14T23:56:53.326 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3529999999999998e-05 2024-09-14T23:56:53.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.326 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.326 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:56:53.326 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966632+0000", 2024-09-14T23:56:53.326 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9809999999999997e-06 2024-09-14T23:56:53.327 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:53.327 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:53.327 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:53.327 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:53.327 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-14T23:56:53.327 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 1 2024-09-14T23:56:53.327 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:56:53.331 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T23:56:53.414 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:56:53.412+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:56:53.438 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:56:53.439 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:56:53.439 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:56:53.439 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:56:53.439 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:56:53.447 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T23:56:53.515 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:56:53.513+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:56:53.597 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:56:53.597 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:56:53.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:56:53.598 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:56:53.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.598 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:6284 3.1 3:8172dac6:::rbd_object_map.1118de1168b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:53.598 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.492324+0000", 2024-09-14T23:56:53.598 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.102783833, 2024-09-14T23:56:53.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.008193456, 2024-09-14T23:56:53.598 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:53.598 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:53.598 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:53.598 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:53.599 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:53.599 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:53.599 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 6284 2024-09-14T23:56:53.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.599 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:53.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:53.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492324+0000", 2024-09-14T23:56:53.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:53.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:53.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492324+0000", 2024-09-14T23:56:53.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:53.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:53.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492325+0000", 2024-09-14T23:56:53.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4700000000000001e-07 2024-09-14T23:56:53.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:53.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492329+0000", 2024-09-14T23:56:53.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7339999999999999e-06 2024-09-14T23:56:53.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:53.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492340+0000", 2024-09-14T23:56:53.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0210000000000001e-05 2024-09-14T23:56:53.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:53.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492346+0000", 2024-09-14T23:56:53.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5259999999999999e-06 2024-09-14T23:56:53.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:53.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.497971+0000", 2024-09-14T23:56:53.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0056243040000000001 2024-09-14T23:56:53.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:53.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.498021+0000", 2024-09-14T23:56:53.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0222000000000001e-05 2024-09-14T23:56:53.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 4", 2024-09-14T23:56:53.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.498225+0000", 2024-09-14T23:56:53.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020451900000000001 2024-09-14T23:56:53.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:56:53.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.500419+0000", 2024-09-14T23:56:53.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002193469 2024-09-14T23:56:53.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:53.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.500445+0000", 2024-09-14T23:56:53.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6302e-05 2024-09-14T23:56:53.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:53.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.500475+0000", 2024-09-14T23:56:53.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9791999999999999e-05 2024-09-14T23:56:53.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:53.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.500517+0000", 2024-09-14T23:56:53.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2630999999999999e-05 2024-09-14T23:56:53.604 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:53.604 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:53.604 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:53.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.604 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:6285 3.1 3:8172dac6:::rbd_object_map.1118de1168b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:53.604 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.492456+0000", 2024-09-14T23:56:53.604 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.102652275000001, 2024-09-14T23:56:53.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0082137960000000006, 2024-09-14T23:56:53.604 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:53.604 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:53.604 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:53.604 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:53.605 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:53.605 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:53.605 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 6285 2024-09-14T23:56:53.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.605 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:53.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:53.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492456+0000", 2024-09-14T23:56:53.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:53.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:53.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492456+0000", 2024-09-14T23:56:53.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:53.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:53.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492456+0000", 2024-09-14T23:56:53.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2799999999999995e-07 2024-09-14T23:56:53.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:53.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492461+0000", 2024-09-14T23:56:53.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0050000000000004e-06 2024-09-14T23:56:53.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:53.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492471+0000", 2024-09-14T23:56:53.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5489999999999995e-06 2024-09-14T23:56:53.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:53.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492478+0000", 2024-09-14T23:56:53.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7549999999999997e-06 2024-09-14T23:56:53.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:53.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.498565+0000", 2024-09-14T23:56:53.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0060878629999999998 2024-09-14T23:56:53.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:53.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.498637+0000", 2024-09-14T23:56:53.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1585e-05 2024-09-14T23:56:53.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 4", 2024-09-14T23:56:53.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.498843+0000", 2024-09-14T23:56:53.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020629399999999999 2024-09-14T23:56:53.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:56:53.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.500532+0000", 2024-09-14T23:56:53.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016882150000000001 2024-09-14T23:56:53.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:53.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.500596+0000", 2024-09-14T23:56:53.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4534000000000002e-05 2024-09-14T23:56:53.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:53.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.500632+0000", 2024-09-14T23:56:53.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5654000000000001e-05 2024-09-14T23:56:53.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:53.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.500669+0000", 2024-09-14T23:56:53.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7614000000000003e-05 2024-09-14T23:56:53.610 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:53.610 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:53.610 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:53.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.610 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:6286 3.1 3:8172dac6:::rbd_object_map.1118de1168b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:53.610 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.492592+0000", 2024-09-14T23:56:53.610 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.102515650000001, 2024-09-14T23:56:53.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0091976439999999996, 2024-09-14T23:56:53.611 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:53.611 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:53.611 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:53.611 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:53.611 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:53.611 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:53.611 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 6286 2024-09-14T23:56:53.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.611 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:53.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:53.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492592+0000", 2024-09-14T23:56:53.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:53.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:53.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492592+0000", 2024-09-14T23:56:53.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:53.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:53.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492593+0000", 2024-09-14T23:56:53.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2699999999999998e-07 2024-09-14T23:56:53.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:53.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492598+0000", 2024-09-14T23:56:53.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8690000000000003e-06 2024-09-14T23:56:53.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:53.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492615+0000", 2024-09-14T23:56:53.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6977e-05 2024-09-14T23:56:53.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:53.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492622+0000", 2024-09-14T23:56:53.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8410000000000002e-06 2024-09-14T23:56:53.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:53.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.499278+0000", 2024-09-14T23:56:53.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0066565419999999997 2024-09-14T23:56:53.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:53.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.499335+0000", 2024-09-14T23:56:53.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6598000000000003e-05 2024-09-14T23:56:53.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 4", 2024-09-14T23:56:53.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.499543+0000", 2024-09-14T23:56:53.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020844099999999999 2024-09-14T23:56:53.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:56:53.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.500537+0000", 2024-09-14T23:56:53.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00099392600000000001 2024-09-14T23:56:53.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:53.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.501691+0000", 2024-09-14T23:56:53.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011535409999999999 2024-09-14T23:56:53.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:53.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.501749+0000", 2024-09-14T23:56:53.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8233000000000003e-05 2024-09-14T23:56:53.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:53.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.501790+0000", 2024-09-14T23:56:53.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0849000000000001e-05 2024-09-14T23:56:53.617 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:53.617 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:53.617 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:53.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.617 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:6287 3.1 3:8172dac6:::rbd_object_map.1118de1168b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:53.617 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.492747+0000", 2024-09-14T23:56:53.617 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.102360794000001, 2024-09-14T23:56:53.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0091967720000000006, 2024-09-14T23:56:53.617 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:53.617 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:53.617 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:53.617 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:53.618 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:53.618 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:53.618 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 6287 2024-09-14T23:56:53.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.618 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:53.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:53.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492747+0000", 2024-09-14T23:56:53.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:53.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:53.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492747+0000", 2024-09-14T23:56:53.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:53.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:53.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492748+0000", 2024-09-14T23:56:53.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3399999999999998e-07 2024-09-14T23:56:53.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:53.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492753+0000", 2024-09-14T23:56:53.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6829999999999999e-06 2024-09-14T23:56:53.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:53.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492760+0000", 2024-09-14T23:56:53.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8930000000000005e-06 2024-09-14T23:56:53.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:53.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492765+0000", 2024-09-14T23:56:53.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5569999999999997e-06 2024-09-14T23:56:53.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:53.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.499828+0000", 2024-09-14T23:56:53.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0070631440000000004 2024-09-14T23:56:53.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:53.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.499878+0000", 2024-09-14T23:56:53.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0117999999999997e-05 2024-09-14T23:56:53.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 4", 2024-09-14T23:56:53.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.500157+0000", 2024-09-14T23:56:53.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00027882699999999999 2024-09-14T23:56:53.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:53.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.501864+0000", 2024-09-14T23:56:53.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001706656 2024-09-14T23:56:53.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:56:53.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.501879+0000", 2024-09-14T23:56:53.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5502000000000002e-05 2024-09-14T23:56:53.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:53.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.501920+0000", 2024-09-14T23:56:53.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0482000000000003e-05 2024-09-14T23:56:53.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:53.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.501944+0000", 2024-09-14T23:56:53.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4076000000000001e-05 2024-09-14T23:56:53.623 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:53.623 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:53.623 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:53.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.623 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:6288 3.1 3:8172dac6:::rbd_object_map.1118de1168b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:53.624 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.492849+0000", 2024-09-14T23:56:53.624 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.102258497999999, 2024-09-14T23:56:53.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0098451100000000007, 2024-09-14T23:56:53.624 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:53.624 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:53.624 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:53.624 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:53.624 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:53.624 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:53.624 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 6288 2024-09-14T23:56:53.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.625 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:53.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:53.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492849+0000", 2024-09-14T23:56:53.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:53.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:53.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492849+0000", 2024-09-14T23:56:53.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:53.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:53.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492850+0000", 2024-09-14T23:56:53.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5200000000000002e-07 2024-09-14T23:56:53.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:53.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492853+0000", 2024-09-14T23:56:53.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0189999999999998e-06 2024-09-14T23:56:53.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:53.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492859+0000", 2024-09-14T23:56:53.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0850000000000002e-06 2024-09-14T23:56:53.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:53.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492862+0000", 2024-09-14T23:56:53.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5599999999999998e-06 2024-09-14T23:56:53.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:53.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.500719+0000", 2024-09-14T23:56:53.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0078569499999999997 2024-09-14T23:56:53.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:53.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.500798+0000", 2024-09-14T23:56:53.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9012999999999997e-05 2024-09-14T23:56:53.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 4", 2024-09-14T23:56:53.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.501194+0000", 2024-09-14T23:56:53.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00039602099999999998 2024-09-14T23:56:53.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:56:53.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.501956+0000", 2024-09-14T23:56:53.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00076112900000000004 2024-09-14T23:56:53.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:53.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.502639+0000", 2024-09-14T23:56:53.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00068322800000000003 2024-09-14T23:56:53.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:53.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.502671+0000", 2024-09-14T23:56:53.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1798000000000003e-05 2024-09-14T23:56:53.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:53.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.502694+0000", 2024-09-14T23:56:53.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3855000000000001e-05 2024-09-14T23:56:53.630 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:53.630 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:53.630 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:53.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.630 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:6289 3.1 3:8172dac6:::rbd_object_map.1118de1168b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:53.630 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.492884+0000", 2024-09-14T23:56:53.630 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.102223648000001, 2024-09-14T23:56:53.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010525079, 2024-09-14T23:56:53.630 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:53.630 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:53.630 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:53.631 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:53.631 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:53.631 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:53.631 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 6289 2024-09-14T23:56:53.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.631 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:53.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:53.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492884+0000", 2024-09-14T23:56:53.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:53.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:53.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492884+0000", 2024-09-14T23:56:53.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:53.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:53.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492885+0000", 2024-09-14T23:56:53.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1699999999999999e-07 2024-09-14T23:56:53.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:53.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492887+0000", 2024-09-14T23:56:53.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.193e-06 2024-09-14T23:56:53.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:53.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492897+0000", 2024-09-14T23:56:53.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0703000000000001e-05 2024-09-14T23:56:53.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:53.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492902+0000", 2024-09-14T23:56:53.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4360000000000004e-06 2024-09-14T23:56:53.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:53.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.501981+0000", 2024-09-14T23:56:53.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0090787290000000007 2024-09-14T23:56:53.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:53.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.502045+0000", 2024-09-14T23:56:53.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4787000000000002e-05 2024-09-14T23:56:53.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 4", 2024-09-14T23:56:53.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.502280+0000", 2024-09-14T23:56:53.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000234663 2024-09-14T23:56:53.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:56:53.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.502927+0000", 2024-09-14T23:56:53.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00064658499999999998 2024-09-14T23:56:53.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:53.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.503332+0000", 2024-09-14T23:56:53.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000405139 2024-09-14T23:56:53.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:53.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.503361+0000", 2024-09-14T23:56:53.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9626999999999999e-05 2024-09-14T23:56:53.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:53.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.503409+0000", 2024-09-14T23:56:53.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7899999999999999e-05 2024-09-14T23:56:53.636 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:53.636 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:53.636 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:53.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.636 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:16573 3.1 3:862ed24f:::rbd_data.1118987b60ce.0000000000000003:5 [read 2072576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T23:56:53.636 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:42.519802+0000", 2024-09-14T23:56:53.636 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 11.075305716000001, 2024-09-14T23:56:53.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0085017849999999996, 2024-09-14T23:56:53.636 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:53.636 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:53.637 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:53.637 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:53.637 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:53.637 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:53.637 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 16573 2024-09-14T23:56:53.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.637 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:53.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:53.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519802+0000", 2024-09-14T23:56:53.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:53.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:53.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519802+0000", 2024-09-14T23:56:53.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:53.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:53.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519803+0000", 2024-09-14T23:56:53.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9599999999999999e-07 2024-09-14T23:56:53.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:53.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519805+0000", 2024-09-14T23:56:53.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5840000000000002e-06 2024-09-14T23:56:53.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:53.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519812+0000", 2024-09-14T23:56:53.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4420000000000003e-06 2024-09-14T23:56:53.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:53.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519817+0000", 2024-09-14T23:56:53.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2989999999999996e-06 2024-09-14T23:56:53.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:53.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.527021+0000", 2024-09-14T23:56:53.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0072037669999999998 2024-09-14T23:56:53.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:53.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.527105+0000", 2024-09-14T23:56:53.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4498000000000003e-05 2024-09-14T23:56:53.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:53.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.528304+0000", 2024-09-14T23:56:53.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011986989999999999 2024-09-14T23:56:53.640 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:53.640 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:53.640 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:53.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.641 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:16585 3.1 3:862ed24f:::rbd_data.1118987b60ce.0000000000000003:5 [read 1015808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T23:56:53.641 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:42.519907+0000", 2024-09-14T23:56:53.641 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 11.075201272999999, 2024-09-14T23:56:53.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0083133819999999994, 2024-09-14T23:56:53.641 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:53.641 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:53.641 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:53.641 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:53.641 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:53.641 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:53.641 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 16585 2024-09-14T23:56:53.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.641 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:53.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:53.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519907+0000", 2024-09-14T23:56:53.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:53.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:53.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519907+0000", 2024-09-14T23:56:53.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:53.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:53.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519907+0000", 2024-09-14T23:56:53.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0999999999999999e-07 2024-09-14T23:56:53.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:53.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519910+0000", 2024-09-14T23:56:53.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6369999999999999e-06 2024-09-14T23:56:53.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:53.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519917+0000", 2024-09-14T23:56:53.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8329999999999996e-06 2024-09-14T23:56:53.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:53.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519922+0000", 2024-09-14T23:56:53.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1909999999999999e-06 2024-09-14T23:56:53.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:53.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.527996+0000", 2024-09-14T23:56:53.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0080740629999999994 2024-09-14T23:56:53.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:53.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.528124+0000", 2024-09-14T23:56:53.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012850200000000001 2024-09-14T23:56:53.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:53.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.528220+0000", 2024-09-14T23:56:53.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5645999999999999e-05 2024-09-14T23:56:53.645 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:53.645 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:53.645 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:53.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.645 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:16588 3.1 3:862ed24f:::rbd_data.1118987b60ce.0000000000000003:5 [read 1990656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T23:56:53.646 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:42.519930+0000", 2024-09-14T23:56:53.646 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 11.075178165000001, 2024-09-14T23:56:53.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0085309359999999994, 2024-09-14T23:56:53.646 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:53.646 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:53.646 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:53.646 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:53.646 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:53.646 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:53.646 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 16588 2024-09-14T23:56:53.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.646 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:53.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:53.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519930+0000", 2024-09-14T23:56:53.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:53.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:53.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519930+0000", 2024-09-14T23:56:53.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:53.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:53.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519930+0000", 2024-09-14T23:56:53.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3000000000000001e-07 2024-09-14T23:56:53.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.648 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:53.648 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519933+0000", 2024-09-14T23:56:53.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4049999999999998e-06 2024-09-14T23:56:53.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.648 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:53.648 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519942+0000", 2024-09-14T23:56:53.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6129999999999993e-06 2024-09-14T23:56:53.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.648 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:53.648 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519947+0000", 2024-09-14T23:56:53.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3260000000000003e-06 2024-09-14T23:56:53.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:53.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.528245+0000", 2024-09-14T23:56:53.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0082979950000000007 2024-09-14T23:56:53.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:53.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.528328+0000", 2024-09-14T23:56:53.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2195999999999997e-05 2024-09-14T23:56:53.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:53.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.528461+0000", 2024-09-14T23:56:53.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013297100000000001 2024-09-14T23:56:53.650 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:53.650 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:53.650 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:53.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.650 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:16590 3.1 3:862ed24f:::rbd_data.1118987b60ce.0000000000000003:5 [read 2080768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T23:56:53.650 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:42.519956+0000", 2024-09-14T23:56:53.650 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 11.075151922, 2024-09-14T23:56:53.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0087515460000000007, 2024-09-14T23:56:53.650 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:53.650 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:53.650 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:53.651 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:53.651 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:53.651 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:53.651 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 16590 2024-09-14T23:56:53.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.651 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:53.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.651 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:53.651 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519956+0000", 2024-09-14T23:56:53.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:53.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.651 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:53.651 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519956+0000", 2024-09-14T23:56:53.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:53.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.652 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:53.652 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519956+0000", 2024-09-14T23:56:53.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7599999999999997e-07 2024-09-14T23:56:53.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.652 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:53.652 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519959+0000", 2024-09-14T23:56:53.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3920000000000001e-06 2024-09-14T23:56:53.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.653 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:53.653 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519965+0000", 2024-09-14T23:56:53.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3230000000000003e-06 2024-09-14T23:56:53.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.653 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:53.653 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519970+0000", 2024-09-14T23:56:53.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0259999999999998e-06 2024-09-14T23:56:53.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.653 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:53.653 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.528507+0000", 2024-09-14T23:56:53.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0085364159999999998 2024-09-14T23:56:53.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.654 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:53.654 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.528601+0000", 2024-09-14T23:56:53.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3988999999999997e-05 2024-09-14T23:56:53.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.654 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:53.654 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.528707+0000", 2024-09-14T23:56:53.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010692399999999999 2024-09-14T23:56:53.654 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:53.654 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:53.654 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:53.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.655 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:16595 3.1 3:862ed24f:::rbd_data.1118987b60ce.0000000000000003:5 [read 3055616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T23:56:53.655 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:42.520231+0000", 2024-09-14T23:56:53.655 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 11.074876733, 2024-09-14T23:56:53.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0086842910000000002, 2024-09-14T23:56:53.655 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:53.655 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:53.655 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:53.655 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:53.655 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:53.655 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:53.655 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 16595 2024-09-14T23:56:53.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.656 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:53.656 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.656 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:53.656 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520231+0000", 2024-09-14T23:56:53.656 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:53.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.656 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.656 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:53.656 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520231+0000", 2024-09-14T23:56:53.656 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:53.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.656 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.657 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:53.657 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520232+0000", 2024-09-14T23:56:53.657 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3699999999999999e-07 2024-09-14T23:56:53.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.657 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.657 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:53.657 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520236+0000", 2024-09-14T23:56:53.657 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1320000000000004e-06 2024-09-14T23:56:53.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.657 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.657 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:53.657 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520247+0000", 2024-09-14T23:56:53.657 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1306e-05 2024-09-14T23:56:53.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:53.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520255+0000", 2024-09-14T23:56:53.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1110000000000002e-06 2024-09-14T23:56:53.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:53.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.528742+0000", 2024-09-14T23:56:53.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0084870009999999992 2024-09-14T23:56:53.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:53.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.528830+0000", 2024-09-14T23:56:53.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8226000000000005e-05 2024-09-14T23:56:53.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:53.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.528915+0000", 2024-09-14T23:56:53.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5578e-05 2024-09-14T23:56:53.659 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:53.659 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:53.659 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:53.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.659 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:16600 3.1 3:862ed24f:::rbd_data.1118987b60ce.0000000000000003:5 [read 950272~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T23:56:53.659 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:42.520281+0000", 2024-09-14T23:56:53.660 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 11.074826507999999, 2024-09-14T23:56:53.660 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0088528040000000006, 2024-09-14T23:56:53.660 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:53.660 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:53.660 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:53.660 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:53.660 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:53.660 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:53.660 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 16600 2024-09-14T23:56:53.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.660 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:53.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.660 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:53.661 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520281+0000", 2024-09-14T23:56:53.661 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:53.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.661 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:53.661 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520281+0000", 2024-09-14T23:56:53.661 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:53.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.661 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:53.661 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520282+0000", 2024-09-14T23:56:53.661 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1999999999999999e-07 2024-09-14T23:56:53.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.662 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:53.662 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520285+0000", 2024-09-14T23:56:53.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.162e-06 2024-09-14T23:56:53.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.662 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:53.662 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520295+0000", 2024-09-14T23:56:53.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5510000000000005e-06 2024-09-14T23:56:53.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.662 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:53.662 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520301+0000", 2024-09-14T23:56:53.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0290000000000002e-06 2024-09-14T23:56:53.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:53.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.528947+0000", 2024-09-14T23:56:53.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.008646763 2024-09-14T23:56:53.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:53.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.529031+0000", 2024-09-14T23:56:53.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3669000000000001e-05 2024-09-14T23:56:53.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:53.664 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.529134+0000", 2024-09-14T23:56:53.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010301 2024-09-14T23:56:53.664 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:53.664 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:53.664 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:53.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.664 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:16602 3.1 3:862ed24f:::rbd_data.1118987b60ce.0000000000000003:5 [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T23:56:53.664 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:42.520352+0000", 2024-09-14T23:56:53.664 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 11.074756005999999, 2024-09-14T23:56:53.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0090844380000000002, 2024-09-14T23:56:53.664 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:53.664 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:53.664 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:53.665 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:53.665 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:53.665 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:53.665 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 16602 2024-09-14T23:56:53.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.665 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:53.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.665 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:53.665 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520352+0000", 2024-09-14T23:56:53.665 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:53.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.665 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:53.666 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520352+0000", 2024-09-14T23:56:53.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:53.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.666 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:53.666 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520353+0000", 2024-09-14T23:56:53.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3099999999999997e-07 2024-09-14T23:56:53.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.666 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:53.666 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520356+0000", 2024-09-14T23:56:53.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6950000000000002e-06 2024-09-14T23:56:53.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.666 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:53.667 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520368+0000", 2024-09-14T23:56:53.667 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1701e-05 2024-09-14T23:56:53.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.667 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:53.667 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520375+0000", 2024-09-14T23:56:53.667 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5459999999999997e-06 2024-09-14T23:56:53.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.667 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:53.667 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.529212+0000", 2024-09-14T23:56:53.667 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0088376240000000005 2024-09-14T23:56:53.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.668 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:53.668 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.529297+0000", 2024-09-14T23:56:53.668 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5222000000000005e-05 2024-09-14T23:56:53.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.668 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:53.668 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.529436+0000", 2024-09-14T23:56:53.668 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013891899999999999 2024-09-14T23:56:53.668 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:53.668 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:53.668 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:53.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.668 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:16607 3.1 3:862ed24f:::rbd_data.1118987b60ce.0000000000000003:5 [read 2015232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T23:56:53.669 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:42.520699+0000", 2024-09-14T23:56:53.669 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 11.074408557, 2024-09-14T23:56:53.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0089861669999999998, 2024-09-14T23:56:53.669 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:53.669 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:53.669 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:53.669 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:53.669 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:53.669 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:53.669 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 16607 2024-09-14T23:56:53.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.669 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:53.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.670 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:53.670 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520699+0000", 2024-09-14T23:56:53.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:53.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.670 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:53.670 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520699+0000", 2024-09-14T23:56:53.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:53.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.670 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:53.670 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520700+0000", 2024-09-14T23:56:53.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4099999999999998e-07 2024-09-14T23:56:53.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.671 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:53.671 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520704+0000", 2024-09-14T23:56:53.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.369e-06 2024-09-14T23:56:53.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.671 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:53.671 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520715+0000", 2024-09-14T23:56:53.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1165000000000001e-05 2024-09-14T23:56:53.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.671 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:53.671 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520723+0000", 2024-09-14T23:56:53.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3860000000000001e-06 2024-09-14T23:56:53.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:53.672 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.529468+0000", 2024-09-14T23:56:53.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0087455539999999991 2024-09-14T23:56:53.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:53.672 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.529551+0000", 2024-09-14T23:56:53.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2671999999999997e-05 2024-09-14T23:56:53.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:53.673 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.529685+0000", 2024-09-14T23:56:53.673 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013438 2024-09-14T23:56:53.673 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:53.673 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:53.673 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:53.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.673 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:16612 3.1 3:862ed24f:::rbd_data.1118987b60ce.0000000000000003:5 [read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T23:56:53.673 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:42.521352+0000", 2024-09-14T23:56:53.673 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 11.073755493, 2024-09-14T23:56:53.673 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0085567620000000007, 2024-09-14T23:56:53.673 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:53.674 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:53.674 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:53.674 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:53.674 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:53.674 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:53.674 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 16612 2024-09-14T23:56:53.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.674 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:53.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.674 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:53.674 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.521352+0000", 2024-09-14T23:56:53.674 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:53.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:53.675 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.521352+0000", 2024-09-14T23:56:53.675 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:53.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:53.675 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.521353+0000", 2024-09-14T23:56:53.675 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9100000000000002e-07 2024-09-14T23:56:53.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:53.675 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.521358+0000", 2024-09-14T23:56:53.675 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4549999999999997e-06 2024-09-14T23:56:53.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.676 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:53.676 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.521369+0000", 2024-09-14T23:56:53.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1601e-05 2024-09-14T23:56:53.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.676 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:53.676 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.521377+0000", 2024-09-14T23:56:53.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0520000000000002e-06 2024-09-14T23:56:53.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.676 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:53.677 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.529728+0000", 2024-09-14T23:56:53.677 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0083509770000000007 2024-09-14T23:56:53.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.677 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.677 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:53.677 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.529818+0000", 2024-09-14T23:56:53.677 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9629999999999994e-05 2024-09-14T23:56:53.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.677 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.677 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:53.677 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.529909+0000", 2024-09-14T23:56:53.677 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1156000000000006e-05 2024-09-14T23:56:53.677 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:53.677 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:53.678 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:53.678 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.678 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.678 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:16617 3.1 3:862ed24f:::rbd_data.1118987b60ce.0000000000000003:5 [read 958464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T23:56:53.678 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:42.521952+0000", 2024-09-14T23:56:53.678 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 11.073155905, 2024-09-14T23:56:53.678 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0081838499999999995, 2024-09-14T23:56:53.678 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:53.678 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:53.678 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:53.678 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:53.678 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:53.678 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:53.679 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 16617 2024-09-14T23:56:53.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.679 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:53.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.679 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:53.679 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.521952+0000", 2024-09-14T23:56:53.679 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:53.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.679 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:53.679 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.521952+0000", 2024-09-14T23:56:53.679 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:53.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.680 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:53.680 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.521953+0000", 2024-09-14T23:56:53.680 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7700000000000003e-07 2024-09-14T23:56:53.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.680 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:53.680 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.521957+0000", 2024-09-14T23:56:53.680 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2169999999999996e-06 2024-09-14T23:56:53.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.680 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:53.680 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.521967+0000", 2024-09-14T23:56:53.680 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.696e-06 2024-09-14T23:56:53.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.681 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:53.681 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.521974+0000", 2024-09-14T23:56:53.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8170000000000001e-06 2024-09-14T23:56:53.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.681 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:53.681 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.529941+0000", 2024-09-14T23:56:53.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0079670660000000001 2024-09-14T23:56:53.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.681 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:53.681 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.530030+0000", 2024-09-14T23:56:53.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9477999999999999e-05 2024-09-14T23:56:53.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.682 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:53.682 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.530136+0000", 2024-09-14T23:56:53.682 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010569900000000001 2024-09-14T23:56:53.682 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:53.682 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:53.682 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:53.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.682 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:18289 3.1f 3:f852abec:::rbd_data.11187be34cc8.0000000000000007:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-14T23:56:53.682 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:44.136538+0000", 2024-09-14T23:56:53.682 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.4585696989999999, 2024-09-14T23:56:53.682 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015276301000000001, 2024-09-14T23:56:53.683 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:53.683 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:53.683 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:53.683 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:53.683 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:53.683 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:53.683 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 18289 2024-09-14T23:56:53.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.683 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:53.683 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.683 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:53.683 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.136538+0000", 2024-09-14T23:56:53.683 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:53.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.684 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:53.684 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.136538+0000", 2024-09-14T23:56:53.684 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:53.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.684 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:53.684 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.136539+0000", 2024-09-14T23:56:53.684 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2200000000000003e-07 2024-09-14T23:56:53.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.684 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:53.684 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.136543+0000", 2024-09-14T23:56:53.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9689999999999996e-06 2024-09-14T23:56:53.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.685 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:53.685 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.136552+0000", 2024-09-14T23:56:53.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7919999999999998e-06 2024-09-14T23:56:53.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.685 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:53.685 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.136556+0000", 2024-09-14T23:56:53.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5210000000000004e-06 2024-09-14T23:56:53.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.685 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:53.686 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.136606+0000", 2024-09-14T23:56:53.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0231000000000001e-05 2024-09-14T23:56:53.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.686 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:53.686 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.136682+0000", 2024-09-14T23:56:53.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5576999999999998e-05 2024-09-14T23:56:53.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.686 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-14T23:56:53.686 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.139652+0000", 2024-09-14T23:56:53.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0029698709999999998 2024-09-14T23:56:53.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.687 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:53.687 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.151725+0000", 2024-09-14T23:56:53.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012073607 2024-09-14T23:56:53.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.687 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:56:53.687 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.151739+0000", 2024-09-14T23:56:53.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3287e-05 2024-09-14T23:56:53.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.687 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:53.687 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.151770+0000", 2024-09-14T23:56:53.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0913e-05 2024-09-14T23:56:53.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.688 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:53.688 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.151814+0000", 2024-09-14T23:56:53.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4811000000000002e-05 2024-09-14T23:56:53.688 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:53.688 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:53.688 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:53.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.688 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:28434 3.6 3:617cff09:::rbd_data.111888d8b54.000000000000000b:a [read 4337664~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:53.688 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:48.493748+0000", 2024-09-14T23:56:53.688 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.1013598199999999, 2024-09-14T23:56:53.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0082073379999999998, 2024-09-14T23:56:53.689 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:53.689 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:53.689 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:53.689 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:53.689 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:53.689 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:53.689 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 28434 2024-09-14T23:56:53.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.689 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:53.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.689 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:53.689 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.493748+0000", 2024-09-14T23:56:53.690 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:53.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.690 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:53.690 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.493748+0000", 2024-09-14T23:56:53.690 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:53.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.690 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:53.690 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.493749+0000", 2024-09-14T23:56:53.690 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7800000000000001e-07 2024-09-14T23:56:53.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.690 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:53.691 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.493752+0000", 2024-09-14T23:56:53.691 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2289999999999999e-06 2024-09-14T23:56:53.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.691 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:53.691 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.493767+0000", 2024-09-14T23:56:53.691 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5597e-05 2024-09-14T23:56:53.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.691 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:53.691 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.493782+0000", 2024-09-14T23:56:53.691 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4333e-05 2024-09-14T23:56:53.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.692 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:53.692 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.501821+0000", 2024-09-14T23:56:53.692 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.008038913 2024-09-14T23:56:53.692 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.692 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.692 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:53.692 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.501874+0000", 2024-09-14T23:56:53.692 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3601999999999999e-05 2024-09-14T23:56:53.692 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.692 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.692 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:53.692 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.501955+0000", 2024-09-14T23:56:53.692 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1086000000000003e-05 2024-09-14T23:56:53.693 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:53.693 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:53.693 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:53.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.693 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.693 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:28435 3.6 3:617cff09:::rbd_data.111888d8b54.000000000000000b:a [read 4403200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:53.693 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:48.493793+0000", 2024-09-14T23:56:53.693 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.101314382, 2024-09-14T23:56:53.693 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0083982390000000001, 2024-09-14T23:56:53.693 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:53.693 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:53.693 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:53.693 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:53.693 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:53.694 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:53.694 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 28435 2024-09-14T23:56:53.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.694 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:53.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:53.694 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.493793+0000", 2024-09-14T23:56:53.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:53.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:53.694 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.493793+0000", 2024-09-14T23:56:53.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:53.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.695 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:53.695 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.493794+0000", 2024-09-14T23:56:53.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2499999999999995e-07 2024-09-14T23:56:53.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.695 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:53.695 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.493797+0000", 2024-09-14T23:56:53.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8609999999999998e-06 2024-09-14T23:56:53.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.695 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:53.696 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.493804+0000", 2024-09-14T23:56:53.696 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2189999999999998e-06 2024-09-14T23:56:53.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.696 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:53.696 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.493810+0000", 2024-09-14T23:56:53.696 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.463e-06 2024-09-14T23:56:53.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.696 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:53.697 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.502025+0000", 2024-09-14T23:56:53.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.008215478 2024-09-14T23:56:53.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.697 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:53.697 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.502104+0000", 2024-09-14T23:56:53.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9107e-05 2024-09-14T23:56:53.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.697 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:53.697 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.502192+0000", 2024-09-14T23:56:53.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7585999999999998e-05 2024-09-14T23:56:53.697 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:53.698 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:53.698 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:53.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.698 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:28436 3.6 3:668d2474:::rbd_data.111814b482f8.00000000000000f0:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,truncate 45056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:53.698 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:48.494404+0000", 2024-09-14T23:56:53.698 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.1007040039999998, 2024-09-14T23:56:53.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0089320140000000003, 2024-09-14T23:56:53.698 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:53.698 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:53.698 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:53.698 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:53.698 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:53.698 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:53.699 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 28436 2024-09-14T23:56:53.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.699 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:53.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.699 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:53.699 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.494404+0000", 2024-09-14T23:56:53.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:53.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.699 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:53.699 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.494404+0000", 2024-09-14T23:56:53.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:53.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.700 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:53.700 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.494405+0000", 2024-09-14T23:56:53.700 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1900000000000001e-07 2024-09-14T23:56:53.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.700 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:53.700 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.494410+0000", 2024-09-14T23:56:53.700 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1739999999999999e-06 2024-09-14T23:56:53.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.700 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:53.700 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.494420+0000", 2024-09-14T23:56:53.700 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7929999999999992e-06 2024-09-14T23:56:53.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.701 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:53.701 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.494427+0000", 2024-09-14T23:56:53.701 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1709999999999996e-06 2024-09-14T23:56:53.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.701 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:53.701 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.502220+0000", 2024-09-14T23:56:53.701 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0077929979999999998 2024-09-14T23:56:53.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.701 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:53.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.502342+0000", 2024-09-14T23:56:53.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012175700000000001 2024-09-14T23:56:53.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-14T23:56:53.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.502500+0000", 2024-09-14T23:56:53.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015829099999999999 2024-09-14T23:56:53.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:56:53.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.502976+0000", 2024-09-14T23:56:53.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00047596599999999998 2024-09-14T23:56:53.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:53.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.503282+0000", 2024-09-14T23:56:53.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030643400000000001 2024-09-14T23:56:53.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.703 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:53.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.503313+0000", 2024-09-14T23:56:53.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0994000000000002e-05 2024-09-14T23:56:53.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:53.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:53.703 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:53.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.503336+0000", 2024-09-14T23:56:53.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2517000000000001e-05 2024-09-14T23:56:53.703 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:53.704 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:53.704 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:53.704 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:53.704 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:53.704 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:56:53.705 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T23:56:53.706 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:56:53.704+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:56:53.806 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:56:53.805+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:56:53.853 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:56:53.853 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:56:53.853 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:56:53.853 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:56:53.861 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T23:56:53.907 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:56:53.906+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:56:53.918 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T23:56:53.967 INFO:teuthology.orchestra.run.smithi042.stdout:1156 read 0x2a2994 thru 0x2a83c4 (0x5a31 bytes) 2024-09-14T23:56:53.969 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:53.975 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:56:53.975 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:56:53.975 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:56:53.975 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:56:53.976 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:56:53.976 INFO:teuthology.orchestra.run.smithi042.stdout:1158 write 0xcaf73ec thru 0xcb055b5 (0xe1ca bytes) 2024-09-14T23:56:53.980 INFO:teuthology.orchestra.run.smithi042.stdout:1160 punch from 0xbbcbf6b to 0xbbd64bb, (0xa550 bytes) 2024-09-14T23:56:53.980 INFO:teuthology.orchestra.run.smithi042.stdout:1163 write 0x2efcd54 thru 0x2f05637 (0x88e4 bytes) 2024-09-14T23:56:53.984 INFO:teuthology.orchestra.run.smithi042.stdout:1165 punch from 0x311e60a to 0x31266db, (0x80d1 bytes) 2024-09-14T23:56:53.984 INFO:teuthology.orchestra.run.smithi042.stdout:1166 read 0x9ff8d16 thru 0x9ffb739 (0x2a24 bytes) 2024-09-14T23:56:53.985 INFO:teuthology.orchestra.run.smithi042.stdout:1168 write 0x586bd69 thru 0x587a1a3 (0xe43b bytes) 2024-09-14T23:56:53.986 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T23:56:53.989 INFO:teuthology.orchestra.run.smithi042.stdout:1169 write 0x484bc8c thru 0x485327c (0x75f1 bytes) 2024-09-14T23:56:53.992 INFO:teuthology.orchestra.run.smithi042.stdout:1170 read 0x2581ba6 thru 0x258b23e (0x9699 bytes) 2024-09-14T23:56:54.001 INFO:teuthology.orchestra.run.smithi042.stdout:1171 writesame 0xe9cc2ba thru 0xe9cfc6e data_size 0x11(0x39b5 bytes) 2024-09-14T23:56:54.001 INFO:teuthology.orchestra.run.smithi042.stdout:1172 read 0xa52686e thru 0xa52d627 (0x6dba bytes) 2024-09-14T23:56:54.008 INFO:teuthology.orchestra.run.smithi042.stdout:1173 trunc from 0xe9cfc6f to 0x950ac8a 2024-09-14T23:56:54.009 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:56:54.007+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:56:54.026 INFO:teuthology.orchestra.run.smithi042.stdout:1174 write 0xb1e46d7 thru 0xb1e5faa (0x18d4 bytes) 2024-09-14T23:56:54.029 INFO:teuthology.orchestra.run.smithi042.stdout:1176 write 0x138ea4d thru 0x139bb8d (0xd141 bytes) 2024-09-14T23:56:54.042 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T23:56:54.043 INFO:teuthology.orchestra.run.smithi042.stdout:1177 trunc from 0xb1e5fab to 0x23cf353 2024-09-14T23:56:54.054 INFO:teuthology.orchestra.run.smithi042.stdout:1178 flatten 2024-09-14T23:56:54.079 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:56:54.081 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:56:54.116 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:56:54.111+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:56:54.118 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:56:54.118 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:56:54.150 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:54.150 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:54.163 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:56:54.163 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:56:54.165 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T23:56:54.205 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:56:54.205 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:56:54.210 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:56:54.209+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:56:54.229 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:56:54.229 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:56:54.232 INFO:teuthology.orchestra.run.smithi042.stdout:1179 write 0x694751c thru 0x695408d (0xcb72 bytes) 2024-09-14T23:56:54.246 INFO:teuthology.orchestra.run.smithi042.stdout:1180 writesame 0x8f4edce thru 0x8f55d77 data_size 0x1(0x6faa bytes) 2024-09-14T23:56:54.246 INFO:teuthology.orchestra.run.smithi042.stdout:1181 read 0x39476b3 thru 0x394b8b4 (0x4202 bytes) 2024-09-14T23:56:54.246 INFO:teuthology.orchestra.run.smithi042.stdout:1182 punch from 0x1bc0837 to 0x1bce714, (0xdedd bytes) 2024-09-14T23:56:54.246 INFO:teuthology.orchestra.run.smithi042.stdout:1183 writesame 0x31e0af7 thru 0x31e4e37 data_size 0x3(0x4341 bytes) 2024-09-14T23:56:54.250 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:56:54.250 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:56:54.252 INFO:teuthology.orchestra.run.smithi042.stdout:1185 read 0x1b894dc thru 0x1b96d1a (0xd83f bytes) 2024-09-14T23:56:54.252 INFO:teuthology.orchestra.run.smithi042.stdout:1186 punch from 0x4e781f3 to 0x4e84198, (0xbfa5 bytes) 2024-09-14T23:56:54.252 INFO:teuthology.orchestra.run.smithi042.stdout:1187 read 0x20d7f33 thru 0x20dbb62 (0x3c30 bytes) 2024-09-14T23:56:54.252 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:54.252 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:54.266 INFO:teuthology.orchestra.run.smithi042.stdout:1194 writesame 0xe6f3863 thru 0xe6fa335 data_size 0x1d(0x6ad3 bytes) 2024-09-14T23:56:54.266 INFO:teuthology.orchestra.run.smithi042.stdout:1195 read 0x50d472f thru 0x50d8cf4 (0x45c6 bytes) 2024-09-14T23:56:54.266 INFO:teuthology.orchestra.run.smithi042.stdout:1196 punch from 0x2636fe8 to 0x263bd76, (0x4d8e bytes) 2024-09-14T23:56:54.267 INFO:teuthology.orchestra.run.smithi042.stdout:1197 punch from 0x195dcff to 0x1963758, (0x5a59 bytes) 2024-09-14T23:56:54.267 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:56:54.267 INFO:teuthology.orchestra.run.smithi042.stdout:1201 read 0xd552ac2 thru 0xd55e374 (0xb8b3 bytes) 2024-09-14T23:56:54.267 INFO:teuthology.orchestra.run.smithi042.stdout:1202 read 0x7c079e8 thru 0x7c0a74d (0x2d66 bytes) 2024-09-14T23:56:54.267 INFO:teuthology.orchestra.run.smithi042.stdout:1203 writesame 0x2b72ff2 thru 0x2b74bb0 data_size 0x1(0x1bbf bytes) 2024-09-14T23:56:54.272 INFO:teuthology.orchestra.run.smithi042.stdout:1204 writesame 0x64d2445 thru 0x64dbe59 data_size 0x73(0x9a15 bytes) 2024-09-14T23:56:54.275 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:56:54.275 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:56:54.276 INFO:teuthology.orchestra.run.smithi042.stdout:1205 writesame 0x619c639 thru 0x619f18b data_size 0x1(0x2b53 bytes) 2024-09-14T23:56:54.285 INFO:teuthology.orchestra.run.smithi042.stdout:1206 clone 10 order 23 su 262144 sc 12 2024-09-14T23:56:54.293 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:56:54.293 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:56:54.329 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:54.329 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:54.330 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:56:54.328+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:56:54.338 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T23:56:54.422 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:56:54.421+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:56:54.485 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:54.486 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:54.494 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T23:56:54.536 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:56:54.534+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:56:54.614 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:56:54.613+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:56:54.642 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:54.642 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:54.651 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T23:56:54.715 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:56:54.714+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:56:54.757 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:54.758 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:54.766 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T23:56:54.816 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:56:54.814+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:56:54.873 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:54.874 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:54.882 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T23:56:54.924 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:56:54.922+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:56:55.018 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:56:55.016+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:56:55.030 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:55.030 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:55.036 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is set 2024-09-14T23:56:55.039 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T23:56:55.039 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T23:56:55.039 INFO:tasks.thrashosds.thrasher:Adding osd 5 2024-09-14T23:56:55.040 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 5 2024-09-14T23:56:55.041 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T23:56:55.119 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:56:55.118+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:56:55.164 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:55.164 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:55.173 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T23:56:55.219 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:56:55.218+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:56:55.237 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-09-14T23:56:55.320 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:56:55.319+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:56:55.332 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:55.332 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:55.339 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T23:56:55.372 INFO:tasks.ceph.mon.a.smithi042.stderr:2024-09-14T23:56:55.370+0000 7f1713676640 -1 mon.a@0(leader).osd e45 definitely_dead 0 2024-09-14T23:56:55.423 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:56:55.422+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:56:55.460 INFO:teuthology.orchestra.run.smithi042.stdout:1207 writesame 0x932f07b thru 0x933583a data_size 0x53(0x67c0 bytes) 2024-09-14T23:56:55.469 INFO:teuthology.orchestra.run.smithi042.stdout:1209 read 0xb39e955 thru 0xb3a9494 (0xab40 bytes) 2024-09-14T23:56:55.469 INFO:teuthology.orchestra.run.smithi042.stdout:1210 write 0x916d355 thru 0x916e8ef (0x159b bytes) 2024-09-14T23:56:55.472 INFO:teuthology.orchestra.run.smithi042.stdout:1211 punch from 0x254d061 to 0x25590ed, (0xc08c bytes) 2024-09-14T23:56:55.472 INFO:teuthology.orchestra.run.smithi042.stdout:1212 writesame 0x7648505 thru 0x7651dbf data_size 0x1(0x98bb bytes) 2024-09-14T23:56:55.487 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:55.487 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:55.490 INFO:teuthology.orchestra.run.smithi042.stdout:1213 write 0xab67bfe thru 0xab760c6 (0xe4c9 bytes) 2024-09-14T23:56:55.494 INFO:teuthology.orchestra.run.smithi042.stdout:1215 writesame 0xc34961d thru 0xc350083 data_size 0x1(0x6a67 bytes) 2024-09-14T23:56:55.496 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T23:56:55.509 INFO:teuthology.orchestra.run.smithi042.stdout:1216 write 0x5e8d56c thru 0x5e92374 (0x4e09 bytes) 2024-09-14T23:56:55.521 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:56:55.520+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:56:55.530 INFO:teuthology.orchestra.run.smithi042.stdout:1217 trunc from 0xe6fa336 to 0x180b433 2024-09-14T23:56:55.557 INFO:teuthology.orchestra.run.smithi042.stdout:1219 read 0x9844a1 thru 0x991738 (0xd298 bytes) 2024-09-14T23:56:55.557 INFO:teuthology.orchestra.run.smithi042.stdout:1220 clone 11 order 18 su 4096 sc 7 2024-09-14T23:56:55.622 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:56:55.621+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:56:55.648 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:55.648 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:55.656 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T23:56:55.723 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:56:55.722+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:56:55.763 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:55.763 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:55.772 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T23:56:55.856 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:56:55.855+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:56:55.920 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:55.920 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:55.925 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:56:55.924+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:56:55.929 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T23:56:56.026 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:56:56.025+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:56:56.042 INFO:teuthology.orchestra.run.smithi042.stderr:marked in osd.5. 2024-09-14T23:56:56.060 INFO:tasks.thrashosds.thrasher:Added osd 5 2024-09-14T23:56:56.079 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:56.080 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:56.089 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T23:56:56.127 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:56:56.126+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:56:56.229 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:56:56.228+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:56:56.236 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:56.236 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:56.245 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T23:56:56.306 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T23:56:56.349 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:56:56.348+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:56:56.352 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:56.353 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:56.361 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T23:56:56.431 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:56:56.430+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:56:56.435 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:56:56.436 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:56:56.452 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:56:56.453 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:56:56.468 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:56:56.468 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:56:56.470 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:56.471 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:56.479 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T23:56:56.485 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:56:56.486 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:56:56.501 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:56:56.501 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:56:56.517 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:56:56.518 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:56:56.532 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:56:56.530+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:56:56.534 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:56:56.534 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:56:56.550 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:56:56.551 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:56:56.587 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:56:56.587 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:56:56.587 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:56:56.587 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:56:56.587 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:56:56.596 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T23:56:56.633 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:56:56.631+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:56:56.705 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:56:56.706 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:56:56.706 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:56:56.706 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:56:56.706 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.706 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30565 3.15 3:acd4c478:::rbd_data.1118fff5d5e0.0000000000000001:8 [read 3043328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:56.706 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.872553+0000", 2024-09-14T23:56:56.706 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.8307460070000001, 2024-09-14T23:56:56.706 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.020786221000000001, 2024-09-14T23:56:56.706 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:56.706 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:56.706 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:56.707 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:56.707 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:56.707 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:56.707 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30565 2024-09-14T23:56:56.707 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.707 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:56.707 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.707 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:56.707 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872553+0000", 2024-09-14T23:56:56.707 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:56.707 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.707 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.707 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:56.707 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872553+0000", 2024-09-14T23:56:56.708 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:56.708 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.708 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.708 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:56.708 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872553+0000", 2024-09-14T23:56:56.708 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2000000000000001e-07 2024-09-14T23:56:56.708 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.708 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.708 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:56.708 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872555+0000", 2024-09-14T23:56:56.708 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8059999999999999e-06 2024-09-14T23:56:56.708 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.709 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.709 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:56.709 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872561+0000", 2024-09-14T23:56:56.709 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6110000000000001e-06 2024-09-14T23:56:56.709 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.709 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.709 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:56.709 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872565+0000", 2024-09-14T23:56:56.709 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3919999999999996e-06 2024-09-14T23:56:56.709 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.709 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.710 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:56.710 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.893155+0000", 2024-09-14T23:56:56.710 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.020590286999999999 2024-09-14T23:56:56.710 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.710 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.710 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:56.710 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.893242+0000", 2024-09-14T23:56:56.710 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6644999999999995e-05 2024-09-14T23:56:56.710 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.710 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.710 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:56.711 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.893339+0000", 2024-09-14T23:56:56.711 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7159999999999998e-05 2024-09-14T23:56:56.711 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:56.711 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:56.711 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:56.711 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.711 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.711 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30567 3.15 3:acd4c478:::rbd_data.1118fff5d5e0.0000000000000001:8 [read 3104768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:56.711 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.872570+0000", 2024-09-14T23:56:56.711 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.8307287780000001, 2024-09-14T23:56:56.711 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.020969402000000002, 2024-09-14T23:56:56.711 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:56.712 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:56.712 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:56.712 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:56.712 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:56.712 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:56.712 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30567 2024-09-14T23:56:56.712 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.712 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:56.712 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.712 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:56.712 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872570+0000", 2024-09-14T23:56:56.713 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:56.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.713 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.713 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:56.713 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872570+0000", 2024-09-14T23:56:56.713 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:56.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.713 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.713 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:56.713 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872571+0000", 2024-09-14T23:56:56.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7599999999999997e-07 2024-09-14T23:56:56.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.714 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.714 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:56.714 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872573+0000", 2024-09-14T23:56:56.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.742e-06 2024-09-14T23:56:56.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.714 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.714 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:56.714 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872578+0000", 2024-09-14T23:56:56.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5330000000000002e-06 2024-09-14T23:56:56.715 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.715 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.715 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:56.715 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872584+0000", 2024-09-14T23:56:56.715 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1890000000000004e-06 2024-09-14T23:56:56.715 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.715 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.715 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:56.715 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.893368+0000", 2024-09-14T23:56:56.715 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.020783414 2024-09-14T23:56:56.716 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.716 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.716 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:56.716 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.893447+0000", 2024-09-14T23:56:56.716 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9153000000000006e-05 2024-09-14T23:56:56.716 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.716 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.716 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:56.716 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.893540+0000", 2024-09-14T23:56:56.716 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2894999999999997e-05 2024-09-14T23:56:56.717 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:56.717 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:56.717 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:56.717 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.717 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.717 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30569 3.15 3:acd4c478:::rbd_data.1118fff5d5e0.0000000000000001:8 [read 3624960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:56.717 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.872869+0000", 2024-09-14T23:56:56.717 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.8304304599999996, 2024-09-14T23:56:56.718 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.020954618000000001, 2024-09-14T23:56:56.718 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:56.718 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:56.718 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:56.718 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:56.718 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:56.718 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:56.719 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30569 2024-09-14T23:56:56.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.719 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:56.719 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.719 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:56.719 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872869+0000", 2024-09-14T23:56:56.719 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:56.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.720 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.720 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:56.720 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872869+0000", 2024-09-14T23:56:56.720 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:56.720 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.720 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.720 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:56.720 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872869+0000", 2024-09-14T23:56:56.720 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7999999999999996e-07 2024-09-14T23:56:56.721 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.721 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.721 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:56.721 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872872+0000", 2024-09-14T23:56:56.721 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3400000000000002e-06 2024-09-14T23:56:56.721 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.722 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.722 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:56.722 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872882+0000", 2024-09-14T23:56:56.722 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5060000000000001e-06 2024-09-14T23:56:56.722 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.722 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.723 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:56.723 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872888+0000", 2024-09-14T23:56:56.723 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2249999999999997e-06 2024-09-14T23:56:56.723 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.723 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.723 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:56.723 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.893581+0000", 2024-09-14T23:56:56.723 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.020692933 2024-09-14T23:56:56.724 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.724 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.724 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:56.724 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.893685+0000", 2024-09-14T23:56:56.724 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000103936 2024-09-14T23:56:56.724 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.725 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.725 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:56.725 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.893823+0000", 2024-09-14T23:56:56.725 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013819799999999999 2024-09-14T23:56:56.725 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:56.725 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:56.726 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:56.726 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.726 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.726 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30570 3.15 3:acd4c478:::rbd_data.1118fff5d5e0.0000000000000001:8 [read 2637824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:56.726 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.872897+0000", 2024-09-14T23:56:56.727 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.830401921, 2024-09-14T23:56:56.727 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021159374000000002, 2024-09-14T23:56:56.727 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:56.727 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:56.727 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:56.727 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:56.727 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:56.727 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:56.727 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30570 2024-09-14T23:56:56.728 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.728 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:56.728 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.728 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:56.728 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872897+0000", 2024-09-14T23:56:56.728 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:56.728 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.729 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.729 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:56.729 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872897+0000", 2024-09-14T23:56:56.729 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:56.729 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.729 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.729 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:56.729 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872898+0000", 2024-09-14T23:56:56.730 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8000000000000001e-07 2024-09-14T23:56:56.730 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.730 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.730 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:56.730 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872905+0000", 2024-09-14T23:56:56.730 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6820000000000003e-06 2024-09-14T23:56:56.730 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.730 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.731 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:56.731 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872912+0000", 2024-09-14T23:56:56.731 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8770000000000004e-06 2024-09-14T23:56:56.731 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.731 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.731 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:56.731 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872918+0000", 2024-09-14T23:56:56.732 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7620000000000001e-06 2024-09-14T23:56:56.732 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.732 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.732 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:56.732 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.893868+0000", 2024-09-14T23:56:56.732 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.020949857999999998 2024-09-14T23:56:56.732 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.732 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.733 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:56.733 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.893950+0000", 2024-09-14T23:56:56.733 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2177999999999998e-05 2024-09-14T23:56:56.733 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.733 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.733 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:56.733 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.894057+0000", 2024-09-14T23:56:56.734 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000106637 2024-09-14T23:56:56.734 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:56.734 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:56.734 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:56.734 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.734 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.734 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30573 3.15 3:acd4c478:::rbd_data.1118fff5d5e0.0000000000000001:8 [read 3158016~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:56.734 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.872924+0000", 2024-09-14T23:56:56.735 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.8303746649999999, 2024-09-14T23:56:56.735 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021329519000000002, 2024-09-14T23:56:56.735 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:56.735 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:56.735 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:56.735 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:56.735 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:56.736 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:56.736 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30573 2024-09-14T23:56:56.736 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.736 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:56.736 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.736 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:56.736 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872924+0000", 2024-09-14T23:56:56.736 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:56.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.737 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.737 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:56.737 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872924+0000", 2024-09-14T23:56:56.737 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:56.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.737 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.738 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:56.738 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872925+0000", 2024-09-14T23:56:56.738 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9999999999999998e-07 2024-09-14T23:56:56.738 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.738 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.738 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:56.738 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872927+0000", 2024-09-14T23:56:56.738 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.765e-06 2024-09-14T23:56:56.739 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.739 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.739 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:56.739 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872933+0000", 2024-09-14T23:56:56.739 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8610000000000003e-06 2024-09-14T23:56:56.739 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.739 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.740 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:56.740 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872937+0000", 2024-09-14T23:56:56.740 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5839999999999996e-06 2024-09-14T23:56:56.740 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.740 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.740 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:56.740 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.894117+0000", 2024-09-14T23:56:56.741 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021179429 2024-09-14T23:56:56.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.741 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.741 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:56.741 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.894170+0000", 2024-09-14T23:56:56.741 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3122999999999999e-05 2024-09-14T23:56:56.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.741 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.742 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:56.742 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.894254+0000", 2024-09-14T23:56:56.742 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4257000000000003e-05 2024-09-14T23:56:56.742 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:56.742 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:56.742 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:56.742 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.743 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.743 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30575 3.15 3:acd4c478:::rbd_data.1118fff5d5e0.0000000000000001:8 [read 3219456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:56.743 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.872943+0000", 2024-09-14T23:56:56.743 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.8303559849999997, 2024-09-14T23:56:56.743 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021535109, 2024-09-14T23:56:56.743 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:56.743 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:56.743 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:56.744 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:56.744 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:56.744 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:56.744 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30575 2024-09-14T23:56:56.744 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.744 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:56.744 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.744 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:56.745 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872943+0000", 2024-09-14T23:56:56.745 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:56.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.745 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.745 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:56.745 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872943+0000", 2024-09-14T23:56:56.745 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:56.746 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.746 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.746 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:56.746 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872943+0000", 2024-09-14T23:56:56.746 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.72e-07 2024-09-14T23:56:56.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.747 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.747 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:56.747 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872945+0000", 2024-09-14T23:56:56.747 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.951e-06 2024-09-14T23:56:56.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.747 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.747 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:56.748 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872952+0000", 2024-09-14T23:56:56.748 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.268e-06 2024-09-14T23:56:56.748 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.748 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.748 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:56.748 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872956+0000", 2024-09-14T23:56:56.748 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8010000000000003e-06 2024-09-14T23:56:56.748 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.749 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.749 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:56.749 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.894295+0000", 2024-09-14T23:56:56.749 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021338475999999999 2024-09-14T23:56:56.749 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.749 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.749 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:56.749 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.894380+0000", 2024-09-14T23:56:56.749 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5222999999999994e-05 2024-09-14T23:56:56.749 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.749 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.750 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:56.750 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.894478+0000", 2024-09-14T23:56:56.750 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8017999999999996e-05 2024-09-14T23:56:56.750 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:56.750 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:56.750 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:56.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.750 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.750 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30577 3.15 3:acd4c478:::rbd_data.1118fff5d5e0.0000000000000001:8 [read 3334144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:56.750 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.872963+0000", 2024-09-14T23:56:56.750 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.8303361819999999, 2024-09-14T23:56:56.750 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021659955000000002, 2024-09-14T23:56:56.750 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:56.751 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:56.751 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:56.751 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:56.751 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:56.751 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:56.751 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30577 2024-09-14T23:56:56.751 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.751 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:56.751 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.751 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:56.752 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872963+0000", 2024-09-14T23:56:56.752 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:56.752 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.752 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.752 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:56.752 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872963+0000", 2024-09-14T23:56:56.752 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:56.752 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.752 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.753 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:56.753 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872963+0000", 2024-09-14T23:56:56.753 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1899999999999998e-07 2024-09-14T23:56:56.753 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.753 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.753 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:56.753 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872966+0000", 2024-09-14T23:56:56.753 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.317e-06 2024-09-14T23:56:56.753 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.753 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.754 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:56.754 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872972+0000", 2024-09-14T23:56:56.754 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1859999999999997e-06 2024-09-14T23:56:56.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.754 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.754 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:56.754 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872977+0000", 2024-09-14T23:56:56.754 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9259999999999999e-06 2024-09-14T23:56:56.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.754 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.754 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:56.754 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.894517+0000", 2024-09-14T23:56:56.755 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021539776 2024-09-14T23:56:56.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.755 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:56.755 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.894565+0000", 2024-09-14T23:56:56.755 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8968999999999998e-05 2024-09-14T23:56:56.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.755 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:56.755 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.894623+0000", 2024-09-14T23:56:56.755 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7361999999999997e-05 2024-09-14T23:56:56.756 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:56.756 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:56.756 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:56.756 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.756 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.756 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30580 3.15 3:acd4c478:::rbd_data.1118fff5d5e0.0000000000000001:8 [read 3395584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:56.756 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.872983+0000", 2024-09-14T23:56:56.756 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.8303162630000003, 2024-09-14T23:56:56.756 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021878231000000001, 2024-09-14T23:56:56.756 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:56.757 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:56.757 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:56.757 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:56.757 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:56.757 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:56.757 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30580 2024-09-14T23:56:56.757 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.757 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:56.757 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.758 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:56.758 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872983+0000", 2024-09-14T23:56:56.758 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:56.758 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.758 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.758 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:56.758 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872983+0000", 2024-09-14T23:56:56.758 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:56.758 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.758 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.758 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:56.759 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872983+0000", 2024-09-14T23:56:56.759 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2800000000000003e-07 2024-09-14T23:56:56.759 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.759 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.759 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:56.759 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872985+0000", 2024-09-14T23:56:56.759 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.804e-06 2024-09-14T23:56:56.759 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.759 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.759 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:56.759 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872992+0000", 2024-09-14T23:56:56.759 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3950000000000003e-06 2024-09-14T23:56:56.759 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.759 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.759 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:56.760 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872998+0000", 2024-09-14T23:56:56.760 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1109999999999996e-06 2024-09-14T23:56:56.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.760 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.760 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:56.760 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.894679+0000", 2024-09-14T23:56:56.760 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021680095999999999 2024-09-14T23:56:56.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.760 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.761 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:56.761 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.894763+0000", 2024-09-14T23:56:56.761 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.475e-05 2024-09-14T23:56:56.761 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.761 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.761 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:56.761 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.894861+0000", 2024-09-14T23:56:56.761 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7746999999999998e-05 2024-09-14T23:56:56.761 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:56.761 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:56.761 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:56.761 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.761 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.761 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30584 3.15 3:acd4c478:::rbd_data.1118fff5d5e0.0000000000000001:8 [read 2932736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:56.762 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.873005+0000", 2024-09-14T23:56:56.762 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.8302941239999999, 2024-09-14T23:56:56.762 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021964951, 2024-09-14T23:56:56.762 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:56.762 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:56.762 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:56.762 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:56.762 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:56.762 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:56.762 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30584 2024-09-14T23:56:56.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.762 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:56.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.763 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:56.763 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873005+0000", 2024-09-14T23:56:56.763 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:56.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.763 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:56.763 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873005+0000", 2024-09-14T23:56:56.764 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:56.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.764 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.764 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:56.764 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873005+0000", 2024-09-14T23:56:56.764 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6300000000000001e-07 2024-09-14T23:56:56.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.764 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.764 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:56.764 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873007+0000", 2024-09-14T23:56:56.764 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8670000000000001e-06 2024-09-14T23:56:56.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.765 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.765 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:56.765 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873013+0000", 2024-09-14T23:56:56.765 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5829999999999997e-06 2024-09-14T23:56:56.765 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.765 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.765 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:56.765 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873017+0000", 2024-09-14T23:56:56.765 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3669999999999999e-06 2024-09-14T23:56:56.765 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.765 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.765 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:56.765 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.894876+0000", 2024-09-14T23:56:56.765 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021858532 2024-09-14T23:56:56.766 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.766 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.766 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:56.766 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.894917+0000", 2024-09-14T23:56:56.766 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1485e-05 2024-09-14T23:56:56.766 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.766 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.766 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:56.766 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.894970+0000", 2024-09-14T23:56:56.766 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2753999999999997e-05 2024-09-14T23:56:56.766 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:56.766 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:56.766 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:56.766 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.767 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.767 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30586 3.15 3:acd4c478:::rbd_data.1118fff5d5e0.0000000000000001:8 [read 2994176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:56.767 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.873023+0000", 2024-09-14T23:56:56.767 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.8302761509999996, 2024-09-14T23:56:56.767 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.022266837000000001, 2024-09-14T23:56:56.767 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:56.767 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:56.767 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:56.767 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:56.767 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:56.767 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:56.767 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30586 2024-09-14T23:56:56.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.767 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:56.768 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.768 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:56.768 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873023+0000", 2024-09-14T23:56:56.768 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:56.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.768 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.768 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:56.768 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873023+0000", 2024-09-14T23:56:56.768 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:56.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.769 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.769 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:56.769 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873023+0000", 2024-09-14T23:56:56.769 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2500000000000001e-07 2024-09-14T23:56:56.769 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.769 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.769 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:56.769 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873026+0000", 2024-09-14T23:56:56.769 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7199999999999998e-06 2024-09-14T23:56:56.769 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.769 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.770 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:56.770 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873032+0000", 2024-09-14T23:56:56.770 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.494e-06 2024-09-14T23:56:56.770 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.770 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.770 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:56.770 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873037+0000", 2024-09-14T23:56:56.770 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4880000000000001e-06 2024-09-14T23:56:56.770 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.770 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.770 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:56.770 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.895006+0000", 2024-09-14T23:56:56.771 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021968713000000001 2024-09-14T23:56:56.771 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.771 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.771 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:56.771 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.895142+0000", 2024-09-14T23:56:56.771 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013594900000000001 2024-09-14T23:56:56.771 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.771 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.771 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:56.771 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.895290+0000", 2024-09-14T23:56:56.771 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014814799999999999 2024-09-14T23:56:56.771 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:56.771 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:56.771 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:56.772 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.772 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.772 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30588 3.15 3:acd4c478:::rbd_data.1118fff5d5e0.0000000000000001:8 [read 3514368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:56.772 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.873042+0000", 2024-09-14T23:56:56.772 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.8302568450000001, 2024-09-14T23:56:56.772 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.022463265, 2024-09-14T23:56:56.772 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:56.772 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:56.773 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:56.773 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:56.773 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:56.773 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:56.773 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30588 2024-09-14T23:56:56.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.773 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:56.773 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.773 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:56.773 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873042+0000", 2024-09-14T23:56:56.773 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:56.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.774 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.774 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:56.774 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873042+0000", 2024-09-14T23:56:56.774 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:56.774 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.774 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.774 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:56.774 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873043+0000", 2024-09-14T23:56:56.774 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4299999999999999e-07 2024-09-14T23:56:56.774 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.774 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.774 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:56.774 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873044+0000", 2024-09-14T23:56:56.774 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.68e-06 2024-09-14T23:56:56.775 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.775 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.775 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:56.775 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873050+0000", 2024-09-14T23:56:56.775 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4979999999999997e-06 2024-09-14T23:56:56.775 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.775 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.775 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:56.775 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873054+0000", 2024-09-14T23:56:56.775 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5510000000000001e-06 2024-09-14T23:56:56.775 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.775 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.775 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:56.775 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.895322+0000", 2024-09-14T23:56:56.776 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.022267797999999998 2024-09-14T23:56:56.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.776 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.776 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:56.776 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.895404+0000", 2024-09-14T23:56:56.776 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1803000000000003e-05 2024-09-14T23:56:56.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.776 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.776 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:56.776 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.895506+0000", 2024-09-14T23:56:56.776 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101592 2024-09-14T23:56:56.776 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:56.776 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:56.776 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:56.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.777 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.777 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30590 3.15 3:acd4c478:::rbd_data.1118fff5d5e0.0000000000000001:8 [read 3047424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:56.777 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.873059+0000", 2024-09-14T23:56:56.777 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.830239884, 2024-09-14T23:56:56.777 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.022706343, 2024-09-14T23:56:56.777 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:56.777 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:56.777 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:56.777 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:56.777 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:56.777 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:56.777 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30590 2024-09-14T23:56:56.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.778 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:56.778 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.778 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:56.778 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873059+0000", 2024-09-14T23:56:56.778 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:56.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.778 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.778 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:56.778 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873059+0000", 2024-09-14T23:56:56.778 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:56.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.778 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.778 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:56.778 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873060+0000", 2024-09-14T23:56:56.779 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7e-07 2024-09-14T23:56:56.779 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.779 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.779 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:56.779 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873061+0000", 2024-09-14T23:56:56.779 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.652e-06 2024-09-14T23:56:56.779 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.779 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.779 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:56.779 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873067+0000", 2024-09-14T23:56:56.779 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5570000000000003e-06 2024-09-14T23:56:56.779 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.779 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.779 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:56.780 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873071+0000", 2024-09-14T23:56:56.780 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.33e-06 2024-09-14T23:56:56.780 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.780 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.780 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:56.780 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.895536+0000", 2024-09-14T23:56:56.780 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.022464631999999998 2024-09-14T23:56:56.780 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.780 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.780 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:56.780 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.895617+0000", 2024-09-14T23:56:56.780 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1316000000000005e-05 2024-09-14T23:56:56.780 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.781 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:56.781 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.895766+0000", 2024-09-14T23:56:56.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000148386 2024-09-14T23:56:56.781 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:56.781 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:56.781 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:56.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.781 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30593 3.15 3:acd4c478:::rbd_data.1118fff5d5e0.0000000000000001:8 [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:56.781 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.873076+0000", 2024-09-14T23:56:56.781 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.8302228060000001, 2024-09-14T23:56:56.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.02293823, 2024-09-14T23:56:56.782 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:56.782 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:56.782 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:56.782 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:56.782 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:56.782 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:56.782 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30593 2024-09-14T23:56:56.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.782 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:56.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.782 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:56.782 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873076+0000", 2024-09-14T23:56:56.782 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:56.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.783 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.783 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:56.783 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873076+0000", 2024-09-14T23:56:56.783 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:56.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.783 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.783 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:56.783 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873077+0000", 2024-09-14T23:56:56.783 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2399999999999999e-07 2024-09-14T23:56:56.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.783 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.783 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:56.783 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873078+0000", 2024-09-14T23:56:56.783 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6279999999999999e-06 2024-09-14T23:56:56.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.784 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.784 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:56.784 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873084+0000", 2024-09-14T23:56:56.784 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4940000000000003e-06 2024-09-14T23:56:56.784 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.784 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.784 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:56.784 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873089+0000", 2024-09-14T23:56:56.784 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8189999999999999e-06 2024-09-14T23:56:56.784 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.784 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.784 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:56.784 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.895827+0000", 2024-09-14T23:56:56.784 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.022738441000000002 2024-09-14T23:56:56.785 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.785 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.785 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:56.785 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.895911+0000", 2024-09-14T23:56:56.785 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4128999999999994e-05 2024-09-14T23:56:56.785 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.785 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.785 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:56.785 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.896015+0000", 2024-09-14T23:56:56.785 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000103395 2024-09-14T23:56:56.785 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:56.785 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:56.785 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:56.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.786 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30595 3.15 3:acd4c478:::rbd_data.1118fff5d5e0.0000000000000001:8 [read 3629056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:56.786 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.873244+0000", 2024-09-14T23:56:56.786 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.8300555359999997, 2024-09-14T23:56:56.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.022932491999999999, 2024-09-14T23:56:56.786 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:56.786 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:56.786 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:56.786 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:56.786 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:56.786 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:56.787 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30595 2024-09-14T23:56:56.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.787 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:56.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.787 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:56.787 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873244+0000", 2024-09-14T23:56:56.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:56.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.787 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:56.787 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873244+0000", 2024-09-14T23:56:56.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:56.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.787 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:56.788 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873244+0000", 2024-09-14T23:56:56.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7199999999999999e-07 2024-09-14T23:56:56.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.788 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:56.788 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873247+0000", 2024-09-14T23:56:56.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.208e-06 2024-09-14T23:56:56.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.788 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:56.788 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873266+0000", 2024-09-14T23:56:56.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8689999999999999e-05 2024-09-14T23:56:56.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.788 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:56.789 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873274+0000", 2024-09-14T23:56:56.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.089e-06 2024-09-14T23:56:56.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.789 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:56.789 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.896055+0000", 2024-09-14T23:56:56.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.02278059 2024-09-14T23:56:56.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.789 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:56.789 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.896107+0000", 2024-09-14T23:56:56.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2645000000000002e-05 2024-09-14T23:56:56.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:56.790 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.896176+0000", 2024-09-14T23:56:56.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8697999999999999e-05 2024-09-14T23:56:56.790 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:56.790 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:56.790 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:56.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.790 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30597 3.15 3:acd4c478:::rbd_data.1118fff5d5e0.0000000000000001:8 [read 2641920~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:56.790 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.873284+0000", 2024-09-14T23:56:56.790 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.8300153029999997, 2024-09-14T23:56:56.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.023132685, 2024-09-14T23:56:56.790 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:56.790 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:56.791 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:56.791 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:56.791 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:56.791 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:56.791 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30597 2024-09-14T23:56:56.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.791 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:56.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.791 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:56.791 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873284+0000", 2024-09-14T23:56:56.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:56.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.791 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:56.791 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873284+0000", 2024-09-14T23:56:56.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:56.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.792 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:56.792 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873284+0000", 2024-09-14T23:56:56.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.08e-07 2024-09-14T23:56:56.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.792 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:56.792 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873287+0000", 2024-09-14T23:56:56.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4269999999999998e-06 2024-09-14T23:56:56.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.792 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:56.792 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873293+0000", 2024-09-14T23:56:56.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3489999999999997e-06 2024-09-14T23:56:56.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.793 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:56.793 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873298+0000", 2024-09-14T23:56:56.793 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8559999999999998e-06 2024-09-14T23:56:56.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.793 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:56.793 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.896220+0000", 2024-09-14T23:56:56.793 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.022921725 2024-09-14T23:56:56.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.793 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:56.793 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.896313+0000", 2024-09-14T23:56:56.793 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3697999999999996e-05 2024-09-14T23:56:56.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.794 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:56.794 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.896416+0000", 2024-09-14T23:56:56.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000103222 2024-09-14T23:56:56.794 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:56.794 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:56.794 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:56.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.794 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30599 3.15 3:acd4c478:::rbd_data.1118fff5d5e0.0000000000000001:8 [read 3162112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:56.794 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.873304+0000", 2024-09-14T23:56:56.794 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.8299950129999996, 2024-09-14T23:56:56.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.023260915, 2024-09-14T23:56:56.794 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:56.795 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:56.795 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:56.795 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:56.795 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:56.795 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:56.795 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30599 2024-09-14T23:56:56.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.795 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:56.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:56.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873304+0000", 2024-09-14T23:56:56.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:56.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:56.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873304+0000", 2024-09-14T23:56:56.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:56.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.796 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:56.796 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873304+0000", 2024-09-14T23:56:56.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6899999999999998e-07 2024-09-14T23:56:56.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.796 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:56.796 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873313+0000", 2024-09-14T23:56:56.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3450000000000006e-06 2024-09-14T23:56:56.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.796 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:56.797 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873319+0000", 2024-09-14T23:56:56.797 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3609999999999997e-06 2024-09-14T23:56:56.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.797 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:56.797 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873324+0000", 2024-09-14T23:56:56.797 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8300000000000003e-06 2024-09-14T23:56:56.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.797 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:56.797 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.896454+0000", 2024-09-14T23:56:56.797 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.023130064999999998 2024-09-14T23:56:56.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.798 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:56.798 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.896503+0000", 2024-09-14T23:56:56.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8900000000000003e-05 2024-09-14T23:56:56.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.798 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:56.798 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.896565+0000", 2024-09-14T23:56:56.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2045000000000007e-05 2024-09-14T23:56:56.798 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:56.798 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:56.798 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:56.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.798 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30601 3.15 3:acd4c478:::rbd_data.1118fff5d5e0.0000000000000001:8 [read 3223552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:56.798 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.873359+0000", 2024-09-14T23:56:56.799 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.8299404990000001, 2024-09-14T23:56:56.799 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.023428353999999998, 2024-09-14T23:56:56.799 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:56.799 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:56.799 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:56.799 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:56.799 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:56.799 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:56.799 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30601 2024-09-14T23:56:56.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.799 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:56.799 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.799 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:56.799 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873359+0000", 2024-09-14T23:56:56.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:56.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:56.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873359+0000", 2024-09-14T23:56:56.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:56.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:56.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873359+0000", 2024-09-14T23:56:56.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7199999999999998e-07 2024-09-14T23:56:56.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:56.801 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873363+0000", 2024-09-14T23:56:56.801 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9219999999999999e-06 2024-09-14T23:56:56.801 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.801 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.801 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:56.801 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873375+0000", 2024-09-14T23:56:56.801 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2078e-05 2024-09-14T23:56:56.801 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.801 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.801 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:56.801 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873383+0000", 2024-09-14T23:56:56.801 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7069999999999992e-06 2024-09-14T23:56:56.801 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.801 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.802 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:56.802 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.896603+0000", 2024-09-14T23:56:56.802 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.023219758 2024-09-14T23:56:56.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.802 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.802 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:56.802 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.896693+0000", 2024-09-14T23:56:56.802 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0600999999999995e-05 2024-09-14T23:56:56.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.802 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.802 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:56.802 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.896787+0000", 2024-09-14T23:56:56.802 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3516000000000004e-05 2024-09-14T23:56:56.802 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:56.803 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:56.803 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:56.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.803 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30603 3.15 3:acd4c478:::rbd_data.1118fff5d5e0.0000000000000001:8 [read 3276800~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:56.803 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.873395+0000", 2024-09-14T23:56:56.803 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.8299044880000004, 2024-09-14T23:56:56.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.023533291000000001, 2024-09-14T23:56:56.803 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:56.803 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:56.803 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:56.803 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:56.803 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:56.803 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:56.804 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30603 2024-09-14T23:56:56.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.804 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:56.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:56.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873395+0000", 2024-09-14T23:56:56.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:56.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:56.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873395+0000", 2024-09-14T23:56:56.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:56.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.805 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:56.805 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873395+0000", 2024-09-14T23:56:56.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9299999999999998e-07 2024-09-14T23:56:56.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.805 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.805 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:56.805 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873398+0000", 2024-09-14T23:56:56.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.481e-06 2024-09-14T23:56:56.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.805 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.805 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:56.805 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873405+0000", 2024-09-14T23:56:56.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.25e-06 2024-09-14T23:56:56.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.806 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.806 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:56.806 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873411+0000", 2024-09-14T23:56:56.806 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9359999999999996e-06 2024-09-14T23:56:56.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.806 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.806 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:56.806 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.896817+0000", 2024-09-14T23:56:56.806 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.023406347000000001 2024-09-14T23:56:56.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.806 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.806 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:56.806 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.896868+0000", 2024-09-14T23:56:56.807 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1258000000000003e-05 2024-09-14T23:56:56.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.807 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:56.807 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.896928+0000", 2024-09-14T23:56:56.807 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9426000000000003e-05 2024-09-14T23:56:56.807 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:56.807 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:56.807 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:56.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.807 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30605 3.15 3:acd4c478:::rbd_data.1118fff5d5e0.0000000000000001:8 [read 3338240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:56.807 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.873418+0000", 2024-09-14T23:56:56.807 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.8298812059999996, 2024-09-14T23:56:56.808 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.023645488999999999, 2024-09-14T23:56:56.808 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:56.808 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:56.808 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:56.808 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:56.808 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:56.808 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:56.808 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30605 2024-09-14T23:56:56.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.808 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:56.808 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.808 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:56.808 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873418+0000", 2024-09-14T23:56:56.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:56.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.809 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:56.809 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873418+0000", 2024-09-14T23:56:56.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:56.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.809 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:56.809 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873418+0000", 2024-09-14T23:56:56.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5200000000000002e-07 2024-09-14T23:56:56.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.809 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:56.810 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873422+0000", 2024-09-14T23:56:56.810 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0450000000000002e-06 2024-09-14T23:56:56.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.810 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.810 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:56.810 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873429+0000", 2024-09-14T23:56:56.810 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5950000000000004e-06 2024-09-14T23:56:56.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.810 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.810 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:56.810 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873434+0000", 2024-09-14T23:56:56.810 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4389999999999999e-06 2024-09-14T23:56:56.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.810 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.811 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:56.811 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.896957+0000", 2024-09-14T23:56:56.811 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.023522568000000001 2024-09-14T23:56:56.811 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.811 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.811 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:56.811 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.897006+0000", 2024-09-14T23:56:56.811 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8925000000000003e-05 2024-09-14T23:56:56.811 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.811 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.811 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:56.811 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.897063+0000", 2024-09-14T23:56:56.811 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7465e-05 2024-09-14T23:56:56.811 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:56.812 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:56.812 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:56.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.812 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.812 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30608 3.15 3:acd4c478:::rbd_data.1118fff5d5e0.0000000000000001:8 [read 3399680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:56.812 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.873441+0000", 2024-09-14T23:56:56.812 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.8298579940000002, 2024-09-14T23:56:56.812 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.023846589000000001, 2024-09-14T23:56:56.812 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:56:56.812 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:56:56.812 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:56:56.812 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:56:56.812 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:56:56.813 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:56.813 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30608 2024-09-14T23:56:56.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.813 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:56:56.813 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.813 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:56:56.813 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873441+0000", 2024-09-14T23:56:56.813 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:56.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.813 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.813 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:56:56.813 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873441+0000", 2024-09-14T23:56:56.813 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:56:56.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.814 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.814 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:56:56.814 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873442+0000", 2024-09-14T23:56:56.814 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2600000000000002e-07 2024-09-14T23:56:56.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.814 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.814 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:56:56.814 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873444+0000", 2024-09-14T23:56:56.814 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.238e-06 2024-09-14T23:56:56.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.814 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.814 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:56:56.814 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873451+0000", 2024-09-14T23:56:56.815 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.861e-06 2024-09-14T23:56:56.815 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.815 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.815 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:56:56.815 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873456+0000", 2024-09-14T23:56:56.815 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1490000000000002e-06 2024-09-14T23:56:56.815 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.815 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.815 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:56:56.815 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.897094+0000", 2024-09-14T23:56:56.815 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.023638461 2024-09-14T23:56:56.815 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.815 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.815 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:56:56.816 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.897187+0000", 2024-09-14T23:56:56.816 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2763000000000006e-05 2024-09-14T23:56:56.816 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:56:56.816 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:56:56.816 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:56:56.816 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.897288+0000", 2024-09-14T23:56:56.816 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100591 2024-09-14T23:56:56.816 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:56.816 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:56.816 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:56.816 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:56:56.816 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:56:56.816 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:56:56.818 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:56:56.816+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:56:56.818 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T23:56:56.919 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:56:56.917+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:56:56.926 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:56:56.926 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:56:56.926 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:56:56.926 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:56:56.935 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T23:56:57.020 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:56:57.019+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:56:57.052 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T23:56:57.088 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:56:57.088 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:56:57.088 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:56:57.088 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:56:57.088 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:56:57.097 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T23:56:57.138 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:56:57.137+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:56:57.222 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:56:57.221+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:56:57.248 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:56:57.248 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:56:57.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:56:57.248 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:56:57.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.248 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:6284 3.1 3:8172dac6:::rbd_object_map.1118de1168b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:57.248 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.492324+0000", 2024-09-14T23:56:57.249 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 17.753430551000001, 2024-09-14T23:56:57.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.008193456, 2024-09-14T23:56:57.249 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:57.249 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:57.249 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:57.249 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:57.249 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:57.249 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:57.249 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 6284 2024-09-14T23:56:57.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.249 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:57.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.249 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:57.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492324+0000", 2024-09-14T23:56:57.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.250 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:57.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492324+0000", 2024-09-14T23:56:57.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.250 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:57.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492325+0000", 2024-09-14T23:56:57.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4700000000000001e-07 2024-09-14T23:56:57.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:57.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492329+0000", 2024-09-14T23:56:57.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7339999999999999e-06 2024-09-14T23:56:57.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:57.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492340+0000", 2024-09-14T23:56:57.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0210000000000001e-05 2024-09-14T23:56:57.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:57.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492346+0000", 2024-09-14T23:56:57.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5259999999999999e-06 2024-09-14T23:56:57.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:57.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.497971+0000", 2024-09-14T23:56:57.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0056243040000000001 2024-09-14T23:56:57.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:57.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.498021+0000", 2024-09-14T23:56:57.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0222000000000001e-05 2024-09-14T23:56:57.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 4", 2024-09-14T23:56:57.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.498225+0000", 2024-09-14T23:56:57.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020451900000000001 2024-09-14T23:56:57.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.253 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.253 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:56:57.253 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.500419+0000", 2024-09-14T23:56:57.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002193469 2024-09-14T23:56:57.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.253 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.253 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:57.253 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.500445+0000", 2024-09-14T23:56:57.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6302e-05 2024-09-14T23:56:57.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.253 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.253 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:57.254 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.500475+0000", 2024-09-14T23:56:57.254 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9791999999999999e-05 2024-09-14T23:56:57.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.254 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:57.254 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.500517+0000", 2024-09-14T23:56:57.254 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2630999999999999e-05 2024-09-14T23:56:57.254 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.254 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:57.254 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.254 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:6285 3.1 3:8172dac6:::rbd_object_map.1118de1168b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:57.254 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.492456+0000", 2024-09-14T23:56:57.255 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 17.753298993000001, 2024-09-14T23:56:57.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0082137960000000006, 2024-09-14T23:56:57.255 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:57.255 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:57.255 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:57.255 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:57.255 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:57.255 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:57.255 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 6285 2024-09-14T23:56:57.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.255 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:57.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:57.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492456+0000", 2024-09-14T23:56:57.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:57.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492456+0000", 2024-09-14T23:56:57.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:57.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492456+0000", 2024-09-14T23:56:57.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2799999999999995e-07 2024-09-14T23:56:57.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:57.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492461+0000", 2024-09-14T23:56:57.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0050000000000004e-06 2024-09-14T23:56:57.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:57.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492471+0000", 2024-09-14T23:56:57.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5489999999999995e-06 2024-09-14T23:56:57.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:57.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492478+0000", 2024-09-14T23:56:57.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7549999999999997e-06 2024-09-14T23:56:57.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:57.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.498565+0000", 2024-09-14T23:56:57.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0060878629999999998 2024-09-14T23:56:57.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:57.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.498637+0000", 2024-09-14T23:56:57.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1585e-05 2024-09-14T23:56:57.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 4", 2024-09-14T23:56:57.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.498843+0000", 2024-09-14T23:56:57.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020629399999999999 2024-09-14T23:56:57.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.259 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:56:57.259 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.500532+0000", 2024-09-14T23:56:57.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016882150000000001 2024-09-14T23:56:57.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.259 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:57.259 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.500596+0000", 2024-09-14T23:56:57.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4534000000000002e-05 2024-09-14T23:56:57.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:57.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.500632+0000", 2024-09-14T23:56:57.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5654000000000001e-05 2024-09-14T23:56:57.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:57.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.500669+0000", 2024-09-14T23:56:57.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7614000000000003e-05 2024-09-14T23:56:57.260 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.260 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:57.260 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.261 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:6286 3.1 3:8172dac6:::rbd_object_map.1118de1168b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:57.261 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.492592+0000", 2024-09-14T23:56:57.261 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 17.753162368000002, 2024-09-14T23:56:57.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0091976439999999996, 2024-09-14T23:56:57.261 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:57.261 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:57.261 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:57.261 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:57.261 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:57.261 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:57.261 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 6286 2024-09-14T23:56:57.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.261 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:57.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:57.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492592+0000", 2024-09-14T23:56:57.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:57.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492592+0000", 2024-09-14T23:56:57.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:57.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492593+0000", 2024-09-14T23:56:57.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2699999999999998e-07 2024-09-14T23:56:57.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.263 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:57.263 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492598+0000", 2024-09-14T23:56:57.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8690000000000003e-06 2024-09-14T23:56:57.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.263 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:57.263 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492615+0000", 2024-09-14T23:56:57.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6977e-05 2024-09-14T23:56:57.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.263 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:57.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492622+0000", 2024-09-14T23:56:57.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8410000000000002e-06 2024-09-14T23:56:57.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:57.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.499278+0000", 2024-09-14T23:56:57.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0066565419999999997 2024-09-14T23:56:57.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:57.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.499335+0000", 2024-09-14T23:56:57.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6598000000000003e-05 2024-09-14T23:56:57.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.265 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 4", 2024-09-14T23:56:57.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.499543+0000", 2024-09-14T23:56:57.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020844099999999999 2024-09-14T23:56:57.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.265 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:56:57.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.500537+0000", 2024-09-14T23:56:57.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00099392600000000001 2024-09-14T23:56:57.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.265 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:57.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.501691+0000", 2024-09-14T23:56:57.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011535409999999999 2024-09-14T23:56:57.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:57.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.501749+0000", 2024-09-14T23:56:57.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8233000000000003e-05 2024-09-14T23:56:57.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:57.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.501790+0000", 2024-09-14T23:56:57.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0849000000000001e-05 2024-09-14T23:56:57.266 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.266 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:57.266 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.267 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:6287 3.1 3:8172dac6:::rbd_object_map.1118de1168b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:57.267 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.492747+0000", 2024-09-14T23:56:57.267 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 17.753007512, 2024-09-14T23:56:57.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0091967720000000006, 2024-09-14T23:56:57.267 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:57.267 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:57.267 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:57.267 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:57.267 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:57.267 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:57.267 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 6287 2024-09-14T23:56:57.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.267 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:57.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:57.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492747+0000", 2024-09-14T23:56:57.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:57.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492747+0000", 2024-09-14T23:56:57.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:57.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492748+0000", 2024-09-14T23:56:57.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3399999999999998e-07 2024-09-14T23:56:57.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:57.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492753+0000", 2024-09-14T23:56:57.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6829999999999999e-06 2024-09-14T23:56:57.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:57.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492760+0000", 2024-09-14T23:56:57.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8930000000000005e-06 2024-09-14T23:56:57.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:57.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492765+0000", 2024-09-14T23:56:57.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5569999999999997e-06 2024-09-14T23:56:57.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:57.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.499828+0000", 2024-09-14T23:56:57.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0070631440000000004 2024-09-14T23:56:57.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:57.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.499878+0000", 2024-09-14T23:56:57.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0117999999999997e-05 2024-09-14T23:56:57.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 4", 2024-09-14T23:56:57.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.500157+0000", 2024-09-14T23:56:57.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00027882699999999999 2024-09-14T23:56:57.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:57.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.501864+0000", 2024-09-14T23:56:57.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001706656 2024-09-14T23:56:57.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:56:57.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.501879+0000", 2024-09-14T23:56:57.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5502000000000002e-05 2024-09-14T23:56:57.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:57.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.501920+0000", 2024-09-14T23:56:57.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0482000000000003e-05 2024-09-14T23:56:57.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:57.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.501944+0000", 2024-09-14T23:56:57.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4076000000000001e-05 2024-09-14T23:56:57.272 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.272 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:57.272 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.272 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:6288 3.1 3:8172dac6:::rbd_object_map.1118de1168b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:57.273 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.492849+0000", 2024-09-14T23:56:57.273 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 17.752905215999998, 2024-09-14T23:56:57.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0098451100000000007, 2024-09-14T23:56:57.273 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:57.273 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:57.273 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:57.273 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:57.273 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:57.273 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:57.273 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 6288 2024-09-14T23:56:57.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.273 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:57.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:57.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492849+0000", 2024-09-14T23:56:57.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:57.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492849+0000", 2024-09-14T23:56:57.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:57.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492850+0000", 2024-09-14T23:56:57.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5200000000000002e-07 2024-09-14T23:56:57.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:57.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492853+0000", 2024-09-14T23:56:57.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0189999999999998e-06 2024-09-14T23:56:57.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:57.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492859+0000", 2024-09-14T23:56:57.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0850000000000002e-06 2024-09-14T23:56:57.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:57.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492862+0000", 2024-09-14T23:56:57.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5599999999999998e-06 2024-09-14T23:56:57.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.276 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:57.276 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.500719+0000", 2024-09-14T23:56:57.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0078569499999999997 2024-09-14T23:56:57.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.276 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:57.276 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.500798+0000", 2024-09-14T23:56:57.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9012999999999997e-05 2024-09-14T23:56:57.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.276 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 4", 2024-09-14T23:56:57.276 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.501194+0000", 2024-09-14T23:56:57.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00039602099999999998 2024-09-14T23:56:57.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:56:57.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.501956+0000", 2024-09-14T23:56:57.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00076112900000000004 2024-09-14T23:56:57.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:57.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.502639+0000", 2024-09-14T23:56:57.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00068322800000000003 2024-09-14T23:56:57.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:57.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.502671+0000", 2024-09-14T23:56:57.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1798000000000003e-05 2024-09-14T23:56:57.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:57.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.502694+0000", 2024-09-14T23:56:57.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3855000000000001e-05 2024-09-14T23:56:57.280 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.280 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:57.280 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.280 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:6289 3.1 3:8172dac6:::rbd_object_map.1118de1168b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:57.280 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.492884+0000", 2024-09-14T23:56:57.280 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 17.752870366, 2024-09-14T23:56:57.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010525079, 2024-09-14T23:56:57.280 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:57.280 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:57.280 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:57.281 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:57.281 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:57.281 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:57.281 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 6289 2024-09-14T23:56:57.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.281 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:57.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:57.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492884+0000", 2024-09-14T23:56:57.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:57.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492884+0000", 2024-09-14T23:56:57.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:57.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492885+0000", 2024-09-14T23:56:57.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1699999999999999e-07 2024-09-14T23:56:57.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:57.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492887+0000", 2024-09-14T23:56:57.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.193e-06 2024-09-14T23:56:57.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:57.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492897+0000", 2024-09-14T23:56:57.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0703000000000001e-05 2024-09-14T23:56:57.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:57.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492902+0000", 2024-09-14T23:56:57.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4360000000000004e-06 2024-09-14T23:56:57.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:57.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.501981+0000", 2024-09-14T23:56:57.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0090787290000000007 2024-09-14T23:56:57.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:57.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.502045+0000", 2024-09-14T23:56:57.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4787000000000002e-05 2024-09-14T23:56:57.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 4", 2024-09-14T23:56:57.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.502280+0000", 2024-09-14T23:56:57.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000234663 2024-09-14T23:56:57.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:56:57.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.502927+0000", 2024-09-14T23:56:57.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00064658499999999998 2024-09-14T23:56:57.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.285 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:57.285 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.503332+0000", 2024-09-14T23:56:57.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000405139 2024-09-14T23:56:57.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.285 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:57.285 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.503361+0000", 2024-09-14T23:56:57.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9626999999999999e-05 2024-09-14T23:56:57.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.285 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:57.285 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.503409+0000", 2024-09-14T23:56:57.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7899999999999999e-05 2024-09-14T23:56:57.286 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.286 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:57.286 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.286 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:16573 3.1 3:862ed24f:::rbd_data.1118987b60ce.0000000000000003:5 [read 2072576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T23:56:57.286 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:42.519802+0000", 2024-09-14T23:56:57.286 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.725952434, 2024-09-14T23:56:57.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0085017849999999996, 2024-09-14T23:56:57.286 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:57.286 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:57.286 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:57.286 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:57.287 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:57.287 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:57.287 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 16573 2024-09-14T23:56:57.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.287 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:57.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.287 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:57.287 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519802+0000", 2024-09-14T23:56:57.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.287 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:57.287 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519802+0000", 2024-09-14T23:56:57.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:57.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519803+0000", 2024-09-14T23:56:57.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9599999999999999e-07 2024-09-14T23:56:57.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:57.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519805+0000", 2024-09-14T23:56:57.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5840000000000002e-06 2024-09-14T23:56:57.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:57.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519812+0000", 2024-09-14T23:56:57.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4420000000000003e-06 2024-09-14T23:56:57.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:57.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519817+0000", 2024-09-14T23:56:57.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2989999999999996e-06 2024-09-14T23:56:57.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:57.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.527021+0000", 2024-09-14T23:56:57.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0072037669999999998 2024-09-14T23:56:57.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:57.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.527105+0000", 2024-09-14T23:56:57.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4498000000000003e-05 2024-09-14T23:56:57.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:57.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.528304+0000", 2024-09-14T23:56:57.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011986989999999999 2024-09-14T23:56:57.290 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.290 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:57.290 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.290 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:16585 3.1 3:862ed24f:::rbd_data.1118987b60ce.0000000000000003:5 [read 1015808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T23:56:57.290 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:42.519907+0000", 2024-09-14T23:56:57.291 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.725847991, 2024-09-14T23:56:57.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0083133819999999994, 2024-09-14T23:56:57.291 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:57.291 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:57.291 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:57.291 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:57.291 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:57.291 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:57.291 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 16585 2024-09-14T23:56:57.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.291 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:57.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:57.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519907+0000", 2024-09-14T23:56:57.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.292 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:57.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519907+0000", 2024-09-14T23:56:57.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.292 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:57.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519907+0000", 2024-09-14T23:56:57.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0999999999999999e-07 2024-09-14T23:56:57.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.293 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:57.293 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519910+0000", 2024-09-14T23:56:57.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6369999999999999e-06 2024-09-14T23:56:57.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.293 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:57.293 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519917+0000", 2024-09-14T23:56:57.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8329999999999996e-06 2024-09-14T23:56:57.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.293 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:57.293 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519922+0000", 2024-09-14T23:56:57.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1909999999999999e-06 2024-09-14T23:56:57.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:57.294 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.527996+0000", 2024-09-14T23:56:57.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0080740629999999994 2024-09-14T23:56:57.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:57.294 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.528124+0000", 2024-09-14T23:56:57.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012850200000000001 2024-09-14T23:56:57.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:57.294 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.528220+0000", 2024-09-14T23:56:57.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5645999999999999e-05 2024-09-14T23:56:57.295 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.295 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:57.295 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.295 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:16588 3.1 3:862ed24f:::rbd_data.1118987b60ce.0000000000000003:5 [read 1990656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T23:56:57.295 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:42.519930+0000", 2024-09-14T23:56:57.295 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.725824883, 2024-09-14T23:56:57.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0085309359999999994, 2024-09-14T23:56:57.295 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:57.295 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:57.295 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:57.296 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:57.296 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:57.296 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:57.296 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 16588 2024-09-14T23:56:57.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.296 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:57.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.296 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:57.296 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519930+0000", 2024-09-14T23:56:57.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.296 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:57.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519930+0000", 2024-09-14T23:56:57.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:57.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519930+0000", 2024-09-14T23:56:57.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3000000000000001e-07 2024-09-14T23:56:57.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:57.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519933+0000", 2024-09-14T23:56:57.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4049999999999998e-06 2024-09-14T23:56:57.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.298 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:57.298 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519942+0000", 2024-09-14T23:56:57.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6129999999999993e-06 2024-09-14T23:56:57.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.298 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.298 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:57.298 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519947+0000", 2024-09-14T23:56:57.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3260000000000003e-06 2024-09-14T23:56:57.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.298 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.298 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:57.298 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.528245+0000", 2024-09-14T23:56:57.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0082979950000000007 2024-09-14T23:56:57.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.299 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:57.299 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.528328+0000", 2024-09-14T23:56:57.299 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2195999999999997e-05 2024-09-14T23:56:57.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.299 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:57.299 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.528461+0000", 2024-09-14T23:56:57.299 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013297100000000001 2024-09-14T23:56:57.299 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.299 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:57.299 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.300 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.300 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:16590 3.1 3:862ed24f:::rbd_data.1118987b60ce.0000000000000003:5 [read 2080768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T23:56:57.300 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:42.519956+0000", 2024-09-14T23:56:57.300 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.725798640000001, 2024-09-14T23:56:57.300 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0087515460000000007, 2024-09-14T23:56:57.300 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:57.300 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:57.300 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:57.300 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:57.300 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:57.300 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:57.300 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 16590 2024-09-14T23:56:57.300 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.300 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:57.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.301 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:57.301 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519956+0000", 2024-09-14T23:56:57.301 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.301 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:57.301 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519956+0000", 2024-09-14T23:56:57.301 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.301 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:57.301 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519956+0000", 2024-09-14T23:56:57.301 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7599999999999997e-07 2024-09-14T23:56:57.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.302 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.302 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:57.302 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519959+0000", 2024-09-14T23:56:57.302 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3920000000000001e-06 2024-09-14T23:56:57.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.302 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.302 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:57.302 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519965+0000", 2024-09-14T23:56:57.302 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3230000000000003e-06 2024-09-14T23:56:57.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.302 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.302 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:57.302 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519970+0000", 2024-09-14T23:56:57.302 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0259999999999998e-06 2024-09-14T23:56:57.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.303 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.303 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:57.303 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.528507+0000", 2024-09-14T23:56:57.303 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0085364159999999998 2024-09-14T23:56:57.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.303 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.303 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:57.303 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.528601+0000", 2024-09-14T23:56:57.303 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3988999999999997e-05 2024-09-14T23:56:57.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.303 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.303 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:57.304 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.528707+0000", 2024-09-14T23:56:57.304 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010692399999999999 2024-09-14T23:56:57.304 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.304 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:57.304 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.304 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.304 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:16595 3.1 3:862ed24f:::rbd_data.1118987b60ce.0000000000000003:5 [read 3055616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T23:56:57.304 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:42.520231+0000", 2024-09-14T23:56:57.304 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.725523451000001, 2024-09-14T23:56:57.304 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0086842910000000002, 2024-09-14T23:56:57.304 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:57.304 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:57.305 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:57.305 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:57.305 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:57.305 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:57.305 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 16595 2024-09-14T23:56:57.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.305 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:57.305 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.305 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:57.305 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520231+0000", 2024-09-14T23:56:57.305 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.305 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.305 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:57.306 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520231+0000", 2024-09-14T23:56:57.306 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.306 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.306 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:57.306 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520232+0000", 2024-09-14T23:56:57.306 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3699999999999999e-07 2024-09-14T23:56:57.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.306 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.306 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:57.306 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520236+0000", 2024-09-14T23:56:57.306 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1320000000000004e-06 2024-09-14T23:56:57.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.307 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.307 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:57.307 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520247+0000", 2024-09-14T23:56:57.307 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1306e-05 2024-09-14T23:56:57.307 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.307 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.307 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:57.307 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520255+0000", 2024-09-14T23:56:57.307 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1110000000000002e-06 2024-09-14T23:56:57.307 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.307 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.307 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:57.308 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.528742+0000", 2024-09-14T23:56:57.308 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0084870009999999992 2024-09-14T23:56:57.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.308 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.308 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:57.308 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.528830+0000", 2024-09-14T23:56:57.308 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8226000000000005e-05 2024-09-14T23:56:57.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.308 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.308 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:57.308 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.528915+0000", 2024-09-14T23:56:57.309 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5578e-05 2024-09-14T23:56:57.309 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.309 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:57.309 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.309 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.309 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.309 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:16600 3.1 3:862ed24f:::rbd_data.1118987b60ce.0000000000000003:5 [read 950272~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T23:56:57.309 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:42.520281+0000", 2024-09-14T23:56:57.309 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.725473226, 2024-09-14T23:56:57.309 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0088528040000000006, 2024-09-14T23:56:57.309 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:57.309 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:57.309 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:57.310 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:57.310 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:57.310 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:57.310 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 16600 2024-09-14T23:56:57.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.310 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:57.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.310 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:57.310 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520281+0000", 2024-09-14T23:56:57.310 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.310 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:57.310 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520281+0000", 2024-09-14T23:56:57.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.311 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:57.311 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520282+0000", 2024-09-14T23:56:57.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1999999999999999e-07 2024-09-14T23:56:57.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.311 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:57.311 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520285+0000", 2024-09-14T23:56:57.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.162e-06 2024-09-14T23:56:57.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.312 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.312 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:57.312 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520295+0000", 2024-09-14T23:56:57.312 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5510000000000005e-06 2024-09-14T23:56:57.312 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.312 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.312 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:57.312 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520301+0000", 2024-09-14T23:56:57.312 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0290000000000002e-06 2024-09-14T23:56:57.312 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.312 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.312 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:57.312 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.528947+0000", 2024-09-14T23:56:57.313 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.008646763 2024-09-14T23:56:57.313 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.313 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.313 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:57.313 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.529031+0000", 2024-09-14T23:56:57.313 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3669000000000001e-05 2024-09-14T23:56:57.313 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.313 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.313 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:57.313 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.529134+0000", 2024-09-14T23:56:57.313 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010301 2024-09-14T23:56:57.313 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.313 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:57.313 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.314 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.314 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.314 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:16602 3.1 3:862ed24f:::rbd_data.1118987b60ce.0000000000000003:5 [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T23:56:57.314 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:42.520352+0000", 2024-09-14T23:56:57.314 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.725402724, 2024-09-14T23:56:57.314 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0090844380000000002, 2024-09-14T23:56:57.314 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:57.314 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:57.314 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:57.314 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:57.314 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:57.314 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:57.314 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 16602 2024-09-14T23:56:57.314 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.314 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:57.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.315 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:57.315 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520352+0000", 2024-09-14T23:56:57.315 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.315 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:57.315 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520352+0000", 2024-09-14T23:56:57.315 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.315 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:57.315 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520353+0000", 2024-09-14T23:56:57.315 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3099999999999997e-07 2024-09-14T23:56:57.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.316 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.316 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:57.316 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520356+0000", 2024-09-14T23:56:57.316 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6950000000000002e-06 2024-09-14T23:56:57.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.316 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.316 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:57.316 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520368+0000", 2024-09-14T23:56:57.316 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1701e-05 2024-09-14T23:56:57.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.316 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.316 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:57.316 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520375+0000", 2024-09-14T23:56:57.317 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5459999999999997e-06 2024-09-14T23:56:57.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.317 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.317 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:57.317 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.529212+0000", 2024-09-14T23:56:57.317 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0088376240000000005 2024-09-14T23:56:57.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.317 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.317 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:57.317 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.529297+0000", 2024-09-14T23:56:57.317 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5222000000000005e-05 2024-09-14T23:56:57.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.317 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.318 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:57.318 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.529436+0000", 2024-09-14T23:56:57.318 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013891899999999999 2024-09-14T23:56:57.318 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.318 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:57.318 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.318 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.318 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:16607 3.1 3:862ed24f:::rbd_data.1118987b60ce.0000000000000003:5 [read 2015232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T23:56:57.318 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:42.520699+0000", 2024-09-14T23:56:57.318 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.725055275000001, 2024-09-14T23:56:57.318 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0089861669999999998, 2024-09-14T23:56:57.318 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:57.318 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:57.318 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:57.319 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:57.319 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:57.319 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:57.319 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 16607 2024-09-14T23:56:57.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.319 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:57.319 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.319 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:57.319 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520699+0000", 2024-09-14T23:56:57.319 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.319 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.319 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:57.319 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520699+0000", 2024-09-14T23:56:57.320 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.320 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.320 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:57.320 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520700+0000", 2024-09-14T23:56:57.320 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4099999999999998e-07 2024-09-14T23:56:57.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.320 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.320 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:57.320 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520704+0000", 2024-09-14T23:56:57.320 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.369e-06 2024-09-14T23:56:57.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.320 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.320 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:57.321 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520715+0000", 2024-09-14T23:56:57.321 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1165000000000001e-05 2024-09-14T23:56:57.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.321 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.321 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:57.321 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520723+0000", 2024-09-14T23:56:57.321 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3860000000000001e-06 2024-09-14T23:56:57.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.321 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.321 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:57.321 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.529468+0000", 2024-09-14T23:56:57.321 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0087455539999999991 2024-09-14T23:56:57.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.321 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.322 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:57.322 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.529551+0000", 2024-09-14T23:56:57.322 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2671999999999997e-05 2024-09-14T23:56:57.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.322 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.322 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:57.322 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.529685+0000", 2024-09-14T23:56:57.322 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013438 2024-09-14T23:56:57.322 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.322 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:57.322 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.322 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.322 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:16612 3.1 3:862ed24f:::rbd_data.1118987b60ce.0000000000000003:5 [read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T23:56:57.323 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:42.521352+0000", 2024-09-14T23:56:57.323 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.724402210999999, 2024-09-14T23:56:57.323 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0085567620000000007, 2024-09-14T23:56:57.323 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:57.323 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:57.323 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:57.323 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:57.323 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:57.323 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:57.323 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 16612 2024-09-14T23:56:57.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.323 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:57.323 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.324 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:57.324 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.521352+0000", 2024-09-14T23:56:57.324 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.324 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:57.324 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.521352+0000", 2024-09-14T23:56:57.324 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.324 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:57.324 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.521353+0000", 2024-09-14T23:56:57.324 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9100000000000002e-07 2024-09-14T23:56:57.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.325 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:57.325 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.521358+0000", 2024-09-14T23:56:57.325 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4549999999999997e-06 2024-09-14T23:56:57.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.325 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:57.325 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.521369+0000", 2024-09-14T23:56:57.325 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1601e-05 2024-09-14T23:56:57.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.325 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:57.325 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.521377+0000", 2024-09-14T23:56:57.325 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0520000000000002e-06 2024-09-14T23:56:57.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.326 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.326 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:57.326 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.529728+0000", 2024-09-14T23:56:57.326 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0083509770000000007 2024-09-14T23:56:57.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.326 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.326 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:57.326 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.529818+0000", 2024-09-14T23:56:57.326 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9629999999999994e-05 2024-09-14T23:56:57.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.326 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.326 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:57.326 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.529909+0000", 2024-09-14T23:56:57.327 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1156000000000006e-05 2024-09-14T23:56:57.327 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.327 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:57.327 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.327 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.327 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:18289 3.1f 3:f852abec:::rbd_data.11187be34cc8.0000000000000007:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-14T23:56:57.327 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:44.136538+0000", 2024-09-14T23:56:57.327 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.109216417000001, 2024-09-14T23:56:57.327 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015276301000000001, 2024-09-14T23:56:57.327 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:57.327 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:57.327 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:57.328 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:57.328 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:57.328 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:57.328 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 18289 2024-09-14T23:56:57.328 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.328 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:57.328 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.328 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:57.328 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.136538+0000", 2024-09-14T23:56:57.328 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.328 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.328 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.328 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:57.329 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.136538+0000", 2024-09-14T23:56:57.329 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.329 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.329 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:57.329 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.136539+0000", 2024-09-14T23:56:57.329 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2200000000000003e-07 2024-09-14T23:56:57.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.329 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.329 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:57.329 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.136543+0000", 2024-09-14T23:56:57.329 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9689999999999996e-06 2024-09-14T23:56:57.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.330 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.330 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:57.330 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.136552+0000", 2024-09-14T23:56:57.330 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7919999999999998e-06 2024-09-14T23:56:57.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.330 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.330 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:57.330 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.136556+0000", 2024-09-14T23:56:57.330 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5210000000000004e-06 2024-09-14T23:56:57.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.330 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.330 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:57.330 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.136606+0000", 2024-09-14T23:56:57.330 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0231000000000001e-05 2024-09-14T23:56:57.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.331 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.331 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:57.331 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.136682+0000", 2024-09-14T23:56:57.331 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5576999999999998e-05 2024-09-14T23:56:57.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.331 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.331 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-14T23:56:57.331 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.139652+0000", 2024-09-14T23:56:57.331 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0029698709999999998 2024-09-14T23:56:57.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.331 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.331 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:57.331 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.151725+0000", 2024-09-14T23:56:57.332 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012073607 2024-09-14T23:56:57.332 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.332 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.332 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:56:57.332 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.151739+0000", 2024-09-14T23:56:57.332 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3287e-05 2024-09-14T23:56:57.332 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.332 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.332 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:57.332 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.151770+0000", 2024-09-14T23:56:57.332 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0913e-05 2024-09-14T23:56:57.332 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.332 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.333 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:57.333 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.151814+0000", 2024-09-14T23:56:57.333 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4811000000000002e-05 2024-09-14T23:56:57.333 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.333 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:57.333 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.333 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.333 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:28434 3.6 3:617cff09:::rbd_data.111888d8b54.000000000000000b:a [read 4337664~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:57.333 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:48.493748+0000", 2024-09-14T23:56:57.333 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.7520065379999998, 2024-09-14T23:56:57.333 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0082073379999999998, 2024-09-14T23:56:57.333 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:57.333 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:57.333 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:57.334 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:57.334 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:57.334 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:57.334 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 28434 2024-09-14T23:56:57.334 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.334 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:57.334 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.334 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:57.334 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.493748+0000", 2024-09-14T23:56:57.334 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.334 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.334 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.334 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:57.335 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.493748+0000", 2024-09-14T23:56:57.335 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.335 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.335 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.335 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:57.335 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.493749+0000", 2024-09-14T23:56:57.335 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7800000000000001e-07 2024-09-14T23:56:57.335 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.335 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.335 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:57.335 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.493752+0000", 2024-09-14T23:56:57.335 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2289999999999999e-06 2024-09-14T23:56:57.335 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.336 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:57.336 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.493767+0000", 2024-09-14T23:56:57.336 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5597e-05 2024-09-14T23:56:57.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.336 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:57.336 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.493782+0000", 2024-09-14T23:56:57.336 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4333e-05 2024-09-14T23:56:57.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.336 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:57.336 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.501821+0000", 2024-09-14T23:56:57.337 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.008038913 2024-09-14T23:56:57.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.337 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.337 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:57.337 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.501874+0000", 2024-09-14T23:56:57.337 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3601999999999999e-05 2024-09-14T23:56:57.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.337 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.337 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:57.337 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.501955+0000", 2024-09-14T23:56:57.337 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1086000000000003e-05 2024-09-14T23:56:57.338 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.338 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:57.338 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.338 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.338 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.338 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:28435 3.6 3:617cff09:::rbd_data.111888d8b54.000000000000000b:a [read 4403200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:57.338 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:48.493793+0000", 2024-09-14T23:56:57.338 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.7519611000000008, 2024-09-14T23:56:57.338 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0083982390000000001, 2024-09-14T23:56:57.338 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:57.338 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:57.338 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:56:57.338 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:57.339 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:57.339 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:57.339 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 28435 2024-09-14T23:56:57.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.339 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:57.339 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.339 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:57.339 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.493793+0000", 2024-09-14T23:56:57.339 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.339 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.339 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:57.339 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.493793+0000", 2024-09-14T23:56:57.340 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.340 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.340 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:57.340 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.493794+0000", 2024-09-14T23:56:57.340 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2499999999999995e-07 2024-09-14T23:56:57.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.340 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.340 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:57.340 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.493797+0000", 2024-09-14T23:56:57.340 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8609999999999998e-06 2024-09-14T23:56:57.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.340 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.341 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:57.341 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.493804+0000", 2024-09-14T23:56:57.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2189999999999998e-06 2024-09-14T23:56:57.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.341 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.341 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:57.341 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.493810+0000", 2024-09-14T23:56:57.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.463e-06 2024-09-14T23:56:57.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.341 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.341 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:57.341 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.502025+0000", 2024-09-14T23:56:57.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.008215478 2024-09-14T23:56:57.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.342 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.342 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:57.342 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.502104+0000", 2024-09-14T23:56:57.342 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9107e-05 2024-09-14T23:56:57.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.342 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.342 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:57.342 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.502192+0000", 2024-09-14T23:56:57.342 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7585999999999998e-05 2024-09-14T23:56:57.342 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.342 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:57.342 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.343 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:28436 3.6 3:668d2474:::rbd_data.111814b482f8.00000000000000f0:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,truncate 45056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:56:57.343 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:48.494404+0000", 2024-09-14T23:56:57.343 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.7513507219999997, 2024-09-14T23:56:57.343 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0089320140000000003, 2024-09-14T23:56:57.343 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:57.343 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:57.343 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:57.343 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:57.343 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:57.343 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:57.343 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 28436 2024-09-14T23:56:57.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.344 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:57.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.344 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:57.344 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.494404+0000", 2024-09-14T23:56:57.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.344 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:57.344 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.494404+0000", 2024-09-14T23:56:57.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.344 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:57.344 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.494405+0000", 2024-09-14T23:56:57.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1900000000000001e-07 2024-09-14T23:56:57.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:57.345 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.494410+0000", 2024-09-14T23:56:57.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1739999999999999e-06 2024-09-14T23:56:57.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:57.345 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.494420+0000", 2024-09-14T23:56:57.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7929999999999992e-06 2024-09-14T23:56:57.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:57.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.494427+0000", 2024-09-14T23:56:57.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1709999999999996e-06 2024-09-14T23:56:57.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.346 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:57.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.502220+0000", 2024-09-14T23:56:57.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0077929979999999998 2024-09-14T23:56:57.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.346 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:57.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.502342+0000", 2024-09-14T23:56:57.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012175700000000001 2024-09-14T23:56:57.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.347 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-14T23:56:57.347 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.502500+0000", 2024-09-14T23:56:57.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015829099999999999 2024-09-14T23:56:57.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.347 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:56:57.347 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.502976+0000", 2024-09-14T23:56:57.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00047596599999999998 2024-09-14T23:56:57.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.347 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:57.347 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.503282+0000", 2024-09-14T23:56:57.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030643400000000001 2024-09-14T23:56:57.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.348 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:57.348 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.503313+0000", 2024-09-14T23:56:57.348 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0994000000000002e-05 2024-09-14T23:56:57.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.348 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:57.348 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.503336+0000", 2024-09-14T23:56:57.348 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2517000000000001e-05 2024-09-14T23:56:57.348 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.348 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:57.348 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.349 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:43054 3.1f 3:ffbfe596:::rbd_data.1118350d5ae5.00000000000002eb:head [set-alloc-hint object_size 262144 write_size 262144,write 196608~22770] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T23:56:57.349 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:56:57.349 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.2792038740000002, 2024-09-14T23:56:57.349 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.99973334400000002, 2024-09-14T23:56:57.349 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:57.349 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:57.349 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:57.349 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:57.349 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:57.349 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:57.349 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 43054 2024-09-14T23:56:57.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.349 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:57.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.350 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:57.350 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:56:57.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.350 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:57.350 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:56:57.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.350 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:57.350 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966552+0000", 2024-09-14T23:56:57.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3790000000000001e-06 2024-09-14T23:56:57.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.351 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:57.351 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966565+0000", 2024-09-14T23:56:57.351 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2459999999999999e-05 2024-09-14T23:56:57.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.351 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:57.351 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966576+0000", 2024-09-14T23:56:57.351 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1039000000000001e-05 2024-09-14T23:56:57.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.351 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:57.352 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966582+0000", 2024-09-14T23:56:57.352 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9660000000000001e-06 2024-09-14T23:56:57.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.352 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.352 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:57.352 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966625+0000", 2024-09-14T23:56:57.352 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3529999999999998e-05 2024-09-14T23:56:57.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.352 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.352 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:56:57.352 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966632+0000", 2024-09-14T23:56:57.352 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9809999999999997e-06 2024-09-14T23:56:57.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.352 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:57.353 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.964460+0000", 2024-09-14T23:56:57.353 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.99782768300000002 2024-09-14T23:56:57.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:57.353 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.964641+0000", 2024-09-14T23:56:57.353 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000181089 2024-09-14T23:56:57.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-14T23:56:57.353 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.964809+0000", 2024-09-14T23:56:57.353 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000168505 2024-09-14T23:56:57.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.354 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:56:57.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966030+0000", 2024-09-14T23:56:57.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012206929999999999 2024-09-14T23:56:57.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.354 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:56:57.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966181+0000", 2024-09-14T23:56:57.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000150569 2024-09-14T23:56:57.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.354 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:57.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966243+0000", 2024-09-14T23:56:57.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2331999999999999e-05 2024-09-14T23:56:57.355 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.355 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.355 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:57.355 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966284+0000", 2024-09-14T23:56:57.355 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1118000000000002e-05 2024-09-14T23:56:57.355 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.355 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:57.355 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.355 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.355 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:57.355 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:56:57.357 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T23:56:57.357 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:56:57.355+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:56:57.458 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:56:57.456+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:56:57.464 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:56:57.464 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:56:57.464 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:56:57.465 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:56:57.473 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T23:56:57.559 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:56:57.557+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:56:57.580 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:56:57.581 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:56:57.581 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:56:57.581 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:56:57.582 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:56:57.590 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T23:56:57.675 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:56:57.673+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:56:57.738 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:56:57.738 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:56:57.739 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:56:57.739 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:56:57.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.739 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5880 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:57.739 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375821+0000", 2024-09-14T23:56:57.739 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.360541967, 2024-09-14T23:56:57.739 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050482096999999997, 2024-09-14T23:56:57.739 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:57.739 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:57.739 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:56:57.739 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:57.740 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:57.740 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:57.740 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5880 2024-09-14T23:56:57.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.740 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:57.740 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.740 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:57.740 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375821+0000", 2024-09-14T23:56:57.740 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.740 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.740 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:57.741 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375821+0000", 2024-09-14T23:56:57.741 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.741 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:57.741 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375821+0000", 2024-09-14T23:56:57.741 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.28e-07 2024-09-14T23:56:57.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.741 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:57.741 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375822+0000", 2024-09-14T23:56:57.742 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7000000000000003e-07 2024-09-14T23:56:57.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.742 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:57.742 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375825+0000", 2024-09-14T23:56:57.742 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9560000000000002e-06 2024-09-14T23:56:57.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.742 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:57.742 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375827+0000", 2024-09-14T23:56:57.742 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3259999999999998e-06 2024-09-14T23:56:57.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:57.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426294+0000", 2024-09-14T23:56:57.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050466748999999998 2024-09-14T23:56:57.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:57.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426303+0000", 2024-09-14T23:56:57.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8680000000000005e-06 2024-09-14T23:56:57.744 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.744 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:57.744 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.744 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5881 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:57.744 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375830+0000", 2024-09-14T23:56:57.744 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.360532521, 2024-09-14T23:56:57.744 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050523831999999998, 2024-09-14T23:56:57.744 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:57.745 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:57.745 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:56:57.745 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:57.745 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:57.745 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:57.745 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5881 2024-09-14T23:56:57.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.746 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:57.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.746 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:57.746 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375830+0000", 2024-09-14T23:56:57.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.746 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:57.746 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375830+0000", 2024-09-14T23:56:57.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.747 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:57.747 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375830+0000", 2024-09-14T23:56:57.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.34e-07 2024-09-14T23:56:57.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.747 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:57.747 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375831+0000", 2024-09-14T23:56:57.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9599999999999998e-07 2024-09-14T23:56:57.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.747 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:57.748 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375834+0000", 2024-09-14T23:56:57.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0489999999999999e-06 2024-09-14T23:56:57.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.748 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:57.748 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375838+0000", 2024-09-14T23:56:57.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5319999999999998e-06 2024-09-14T23:56:57.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.748 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:57.748 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426342+0000", 2024-09-14T23:56:57.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050504391000000003 2024-09-14T23:56:57.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.749 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.749 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:57.749 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426354+0000", 2024-09-14T23:56:57.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.173e-05 2024-09-14T23:56:57.749 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.749 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:57.749 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.749 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.749 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5882 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:57.750 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:56:57.750 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.360521917, 2024-09-14T23:56:57.750 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050549666, 2024-09-14T23:56:57.750 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:57.750 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:57.750 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:56:57.750 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:57.750 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:57.750 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:57.750 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5882 2024-09-14T23:56:57.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.751 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:57.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.751 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:57.751 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:56:57.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.751 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:57.751 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:56:57.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.752 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.752 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:57.752 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:56:57.752 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8099999999999998e-07 2024-09-14T23:56:57.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.752 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.752 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:57.752 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375842+0000", 2024-09-14T23:56:57.752 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9900000000000009e-07 2024-09-14T23:56:57.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.752 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.752 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:57.753 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375845+0000", 2024-09-14T23:56:57.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.055e-06 2024-09-14T23:56:57.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.753 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:57.753 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375848+0000", 2024-09-14T23:56:57.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7609999999999999e-06 2024-09-14T23:56:57.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.753 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:57.753 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426381+0000", 2024-09-14T23:56:57.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050533392000000003 2024-09-14T23:56:57.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.754 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:57.754 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426390+0000", 2024-09-14T23:56:57.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0780000000000002e-06 2024-09-14T23:56:57.754 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.754 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:57.754 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.754 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5884 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:57.755 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:56:57.755 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.360511776999999, 2024-09-14T23:56:57.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050717439000000003, 2024-09-14T23:56:57.755 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:57.755 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:57.755 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:56:57.755 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:57.755 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:57.755 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:57.755 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5884 2024-09-14T23:56:57.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.756 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:57.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.756 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:57.756 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:56:57.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.756 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:57.756 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:56:57.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.757 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:57.757 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:56:57.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0800000000000001e-07 2024-09-14T23:56:57.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.757 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:57.757 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375852+0000", 2024-09-14T23:56:57.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.02e-06 2024-09-14T23:56:57.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.758 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:57.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375855+0000", 2024-09-14T23:56:57.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9900000000000002e-06 2024-09-14T23:56:57.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.758 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:57.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375857+0000", 2024-09-14T23:56:57.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3499999999999999e-06 2024-09-14T23:56:57.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.758 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:57.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426556+0000", 2024-09-14T23:56:57.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050698139000000003 2024-09-14T23:56:57.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:57.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426568+0000", 2024-09-14T23:56:57.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2731999999999999e-05 2024-09-14T23:56:57.759 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.759 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:57.759 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.760 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5885 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:57.760 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:56:57.760 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.360492751999999, 2024-09-14T23:56:57.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050737333000000003, 2024-09-14T23:56:57.760 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:57.760 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:57.760 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:56:57.760 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:57.760 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:57.760 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:57.760 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5885 2024-09-14T23:56:57.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.761 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:57.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.761 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:57.761 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:56:57.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.761 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:57.761 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:56:57.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:57.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:56:57.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5899999999999998e-07 2024-09-14T23:56:57.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:57.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375872+0000", 2024-09-14T23:56:57.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4219999999999999e-06 2024-09-14T23:56:57.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:57.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375878+0000", 2024-09-14T23:56:57.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4980000000000003e-06 2024-09-14T23:56:57.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:57.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375881+0000", 2024-09-14T23:56:57.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6110000000000001e-06 2024-09-14T23:56:57.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:57.764 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426596+0000", 2024-09-14T23:56:57.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050715320000000001 2024-09-14T23:56:57.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.764 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:57.764 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426607+0000", 2024-09-14T23:56:57.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1223000000000001e-05 2024-09-14T23:56:57.764 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.764 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:57.764 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.765 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5886 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:57.765 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:56:57.765 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.360478639, 2024-09-14T23:56:57.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050803395000000001, 2024-09-14T23:56:57.765 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:57.765 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:57.765 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:56:57.765 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:57.765 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:57.765 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:57.765 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5886 2024-09-14T23:56:57.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.766 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:57.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:57.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:56:57.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:57.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:56:57.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:57.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:56:57.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2700000000000001e-07 2024-09-14T23:56:57.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:57.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375886+0000", 2024-09-14T23:56:57.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7060000000000001e-06 2024-09-14T23:56:57.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:57.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375890+0000", 2024-09-14T23:56:57.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2259999999999999e-06 2024-09-14T23:56:57.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:57.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375893+0000", 2024-09-14T23:56:57.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0819999999999999e-06 2024-09-14T23:56:57.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.769 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:57.769 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426677+0000", 2024-09-14T23:56:57.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050784158000000003 2024-09-14T23:56:57.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.769 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:57.769 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426687+0000", 2024-09-14T23:56:57.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9960000000000005e-06 2024-09-14T23:56:57.769 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.769 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:57.770 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.770 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5887 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:57.770 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:56:57.770 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.360456552999999, 2024-09-14T23:56:57.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050833070000000001, 2024-09-14T23:56:57.770 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:57.771 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:57.771 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:56:57.771 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:57.771 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:57.771 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:57.771 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5887 2024-09-14T23:56:57.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.771 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:57.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:57.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:56:57.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:57.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:56:57.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:57.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:56:57.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3299999999999998e-07 2024-09-14T23:56:57.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:57.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375908+0000", 2024-09-14T23:56:57.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7689999999999999e-06 2024-09-14T23:56:57.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:57.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375912+0000", 2024-09-14T23:56:57.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.535e-06 2024-09-14T23:56:57.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:57.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375916+0000", 2024-09-14T23:56:57.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8979999999999998e-06 2024-09-14T23:56:57.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:57.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426729+0000", 2024-09-14T23:56:57.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050813453000000001 2024-09-14T23:56:57.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:57.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426739+0000", 2024-09-14T23:56:57.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0081999999999999e-05 2024-09-14T23:56:57.775 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.775 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:57.775 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.775 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5888 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:57.775 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:56:57.775 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.360443476, 2024-09-14T23:56:57.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050854652, 2024-09-14T23:56:57.776 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:57.776 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:57.776 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:56:57.776 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:57.776 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:57.776 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:57.776 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5888 2024-09-14T23:56:57.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.776 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:57.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:57.777 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:56:57.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.777 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:57.777 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:56:57.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.777 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:57.777 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:56:57.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1400000000000001e-07 2024-09-14T23:56:57.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:57.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375921+0000", 2024-09-14T23:56:57.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.119e-06 2024-09-14T23:56:57.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:57.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375928+0000", 2024-09-14T23:56:57.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3170000000000004e-06 2024-09-14T23:56:57.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:57.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375931+0000", 2024-09-14T23:56:57.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6970000000000002e-06 2024-09-14T23:56:57.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:57.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426768+0000", 2024-09-14T23:56:57.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050837355000000001 2024-09-14T23:56:57.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:57.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426774+0000", 2024-09-14T23:56:57.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9499999999999998e-06 2024-09-14T23:56:57.780 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.780 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:57.780 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.780 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5889 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:57.780 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:56:57.781 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.360426552, 2024-09-14T23:56:57.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050885446000000001, 2024-09-14T23:56:57.781 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:57.781 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:57.781 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:56:57.781 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:57.781 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:57.781 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:57.781 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5889 2024-09-14T23:56:57.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.782 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:57.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:57.782 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:56:57.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:57.782 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:56:57.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:57.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:56:57.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.36e-07 2024-09-14T23:56:57.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:57.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375940+0000", 2024-09-14T23:56:57.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.844e-06 2024-09-14T23:56:57.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:57.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375944+0000", 2024-09-14T23:56:57.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6720000000000002e-06 2024-09-14T23:56:57.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:57.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375946+0000", 2024-09-14T23:56:57.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5629999999999999e-06 2024-09-14T23:56:57.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:57.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426810+0000", 2024-09-14T23:56:57.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050863255000000003 2024-09-14T23:56:57.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:57.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426822+0000", 2024-09-14T23:56:57.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1875999999999999e-05 2024-09-14T23:56:57.785 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.785 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:57.785 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.785 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5890 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:57.785 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:56:57.786 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.360412740000001, 2024-09-14T23:56:57.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050917549999999999, 2024-09-14T23:56:57.786 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:57.786 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:57.786 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:56:57.786 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:57.786 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:57.786 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:57.786 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5890 2024-09-14T23:56:57.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.786 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:57.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:57.787 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:56:57.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:57.787 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:56:57.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:57.787 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:56:57.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4299999999999999e-07 2024-09-14T23:56:57.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:57.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375951+0000", 2024-09-14T23:56:57.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1000000000000001e-06 2024-09-14T23:56:57.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:57.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375956+0000", 2024-09-14T23:56:57.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3370000000000001e-06 2024-09-14T23:56:57.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:57.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375958+0000", 2024-09-14T23:56:57.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4820000000000001e-06 2024-09-14T23:56:57.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:57.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426858+0000", 2024-09-14T23:56:57.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050899666000000003 2024-09-14T23:56:57.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:57.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426867+0000", 2024-09-14T23:56:57.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7219999999999994e-06 2024-09-14T23:56:57.790 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.790 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:57.790 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.790 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5891 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:57.790 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:56:57.791 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.360401417999999, 2024-09-14T23:56:57.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050948067, 2024-09-14T23:56:57.791 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:57.791 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:57.791 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:56:57.791 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:57.791 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:57.791 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:57.791 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5891 2024-09-14T23:56:57.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.791 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:57.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:57.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:56:57.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:57.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:56:57.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:57.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:56:57.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0800000000000001e-07 2024-09-14T23:56:57.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:57.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375962+0000", 2024-09-14T23:56:57.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9600000000000008e-07 2024-09-14T23:56:57.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:57.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375970+0000", 2024-09-14T23:56:57.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1600000000000001e-06 2024-09-14T23:56:57.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:57.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375972+0000", 2024-09-14T23:56:57.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4849999999999999e-06 2024-09-14T23:56:57.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:57.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426902+0000", 2024-09-14T23:56:57.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050929813999999997 2024-09-14T23:56:57.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:57.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426909+0000", 2024-09-14T23:56:57.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4039999999999998e-06 2024-09-14T23:56:57.795 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.795 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:57.795 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.795 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5892 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:57.795 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:56:57.795 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.360386560999999, 2024-09-14T23:56:57.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050965036999999998, 2024-09-14T23:56:57.796 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:57.796 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:57.796 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:56:57.796 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:57.796 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:57.796 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:57.796 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5892 2024-09-14T23:56:57.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.796 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:57.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:57.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:56:57.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:57.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:56:57.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:57.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:56:57.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.34e-07 2024-09-14T23:56:57.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:57.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375978+0000", 2024-09-14T23:56:57.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1880000000000001e-06 2024-09-14T23:56:57.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:57.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375981+0000", 2024-09-14T23:56:57.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2399999999999999e-06 2024-09-14T23:56:57.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:57.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375984+0000", 2024-09-14T23:56:57.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9110000000000002e-06 2024-09-14T23:56:57.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:57.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426934+0000", 2024-09-14T23:56:57.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050949912 2024-09-14T23:56:57.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:57.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426941+0000", 2024-09-14T23:56:57.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5519999999999999e-06 2024-09-14T23:56:57.800 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.800 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:57.800 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.800 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5893 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:57.800 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:56:57.801 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.360375963999999, 2024-09-14T23:56:57.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050975131999999999, 2024-09-14T23:56:57.801 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:57.801 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:57.801 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:56:57.801 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:57.801 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:57.801 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:57.801 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5893 2024-09-14T23:56:57.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.802 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:57.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:57.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:56:57.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:57.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:56:57.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:57.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:56:57.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2999999999999999e-07 2024-09-14T23:56:57.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:57.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375988+0000", 2024-09-14T23:56:57.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0410000000000001e-06 2024-09-14T23:56:57.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:57.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375992+0000", 2024-09-14T23:56:57.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.104e-06 2024-09-14T23:56:57.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:57.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375995+0000", 2024-09-14T23:56:57.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4310000000000001e-06 2024-09-14T23:56:57.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:57.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426958+0000", 2024-09-14T23:56:57.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050963642000000003 2024-09-14T23:56:57.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:57.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426962+0000", 2024-09-14T23:56:57.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6839999999999998e-06 2024-09-14T23:56:57.805 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.806 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:57.806 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.806 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5894 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:57.806 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:56:57.806 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.360364564000001, 2024-09-14T23:56:57.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050978333000000001, 2024-09-14T23:56:57.806 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:57.806 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:57.806 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:56:57.806 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:57.807 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:57.807 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:57.807 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5894 2024-09-14T23:56:57.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.807 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:57.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:57.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:56:57.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:57.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:56:57.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:57.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:56:57.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.23e-07 2024-09-14T23:56:57.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:57.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375999+0000", 2024-09-14T23:56:57.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9800000000000002e-07 2024-09-14T23:56:57.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:57.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376007+0000", 2024-09-14T23:56:57.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8539999999999997e-06 2024-09-14T23:56:57.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:57.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376011+0000", 2024-09-14T23:56:57.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.399e-06 2024-09-14T23:56:57.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:57.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426973+0000", 2024-09-14T23:56:57.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050962454999999997 2024-09-14T23:56:57.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:57.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426976+0000", 2024-09-14T23:56:57.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4039999999999999e-06 2024-09-14T23:56:57.810 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.810 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:57.811 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.811 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5895 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:57.811 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376014+0000", 2024-09-14T23:56:57.811 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.360348219999999, 2024-09-14T23:56:57.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050975951999999998, 2024-09-14T23:56:57.811 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:57.811 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:57.811 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:56:57.812 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:57.812 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:57.812 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:57.812 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5895 2024-09-14T23:56:57.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.812 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:57.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:57.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376014+0000", 2024-09-14T23:56:57.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:57.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376014+0000", 2024-09-14T23:56:57.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:57.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376015+0000", 2024-09-14T23:56:57.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2999999999999999e-07 2024-09-14T23:56:57.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:57.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376016+0000", 2024-09-14T23:56:57.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1659999999999999e-06 2024-09-14T23:56:57.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:57.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376024+0000", 2024-09-14T23:56:57.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8930000000000005e-06 2024-09-14T23:56:57.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:57.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376026+0000", 2024-09-14T23:56:57.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6869999999999999e-06 2024-09-14T23:56:57.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:57.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426987+0000", 2024-09-14T23:56:57.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050960601000000001 2024-09-14T23:56:57.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:57.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426990+0000", 2024-09-14T23:56:57.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3749999999999999e-06 2024-09-14T23:56:57.816 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.816 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:57.816 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.817 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5896 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:57.817 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:56:57.817 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.360332483000001, 2024-09-14T23:56:57.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050975656000000001, 2024-09-14T23:56:57.817 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:57.817 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:57.817 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:56:57.817 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:57.817 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:57.818 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:57.818 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5896 2024-09-14T23:56:57.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.818 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:57.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:57.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:56:57.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:57.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:56:57.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:57.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:56:57.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5100000000000001e-07 2024-09-14T23:56:57.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:57.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376032+0000", 2024-09-14T23:56:57.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.141e-06 2024-09-14T23:56:57.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:57.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376036+0000", 2024-09-14T23:56:57.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.25e-06 2024-09-14T23:56:57.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:57.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376038+0000", 2024-09-14T23:56:57.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5469999999999999e-06 2024-09-14T23:56:57.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:57.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427002+0000", 2024-09-14T23:56:57.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050964029000000001 2024-09-14T23:56:57.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:57.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427006+0000", 2024-09-14T23:56:57.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4379999999999999e-06 2024-09-14T23:56:57.821 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.822 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:57.822 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.822 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5897 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:57.822 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:56:57.822 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.360320683000001, 2024-09-14T23:56:57.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050978001000000002, 2024-09-14T23:56:57.822 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:57.822 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:57.822 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:56:57.822 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:57.823 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:57.823 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:57.823 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5897 2024-09-14T23:56:57.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.823 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:57.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:57.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:56:57.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:57.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:56:57.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:57.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:56:57.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2600000000000001e-07 2024-09-14T23:56:57.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:57.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376043+0000", 2024-09-14T23:56:57.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.054e-06 2024-09-14T23:56:57.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:57.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376047+0000", 2024-09-14T23:56:57.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0659999999999997e-06 2024-09-14T23:56:57.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:57.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376050+0000", 2024-09-14T23:56:57.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.88e-06 2024-09-14T23:56:57.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:57.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427017+0000", 2024-09-14T23:56:57.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050966453000000002 2024-09-14T23:56:57.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:57.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427020+0000", 2024-09-14T23:56:57.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3220000000000001e-06 2024-09-14T23:56:57.826 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.826 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:57.827 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.827 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5898 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:57.827 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376053+0000", 2024-09-14T23:56:57.827 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.360309197999999, 2024-09-14T23:56:57.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050982024000000001, 2024-09-14T23:56:57.827 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:57.827 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:57.827 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:56:57.827 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:57.827 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:57.827 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:57.828 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5898 2024-09-14T23:56:57.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.828 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:57.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:57.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376053+0000", 2024-09-14T23:56:57.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:57.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376053+0000", 2024-09-14T23:56:57.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:57.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376278+0000", 2024-09-14T23:56:57.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022426400000000001 2024-09-14T23:56:57.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:57.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376282+0000", 2024-09-14T23:56:57.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4739999999999999e-06 2024-09-14T23:56:57.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:57.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376296+0000", 2024-09-14T23:56:57.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4229999999999999e-05 2024-09-14T23:56:57.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:57.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376303+0000", 2024-09-14T23:56:57.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2199999999999997e-06 2024-09-14T23:56:57.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:57.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427031+0000", 2024-09-14T23:56:57.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050728154999999997 2024-09-14T23:56:57.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:57.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427036+0000", 2024-09-14T23:56:57.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6809999999999997e-06 2024-09-14T23:56:57.831 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.831 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:57.831 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.832 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5899 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:57.832 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376338+0000", 2024-09-14T23:56:57.832 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.360024473999999, 2024-09-14T23:56:57.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050712100000000003, 2024-09-14T23:56:57.832 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:57.832 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:57.832 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:56:57.832 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:57.832 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:57.832 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:57.832 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5899 2024-09-14T23:56:57.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.833 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:57.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:57.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376338+0000", 2024-09-14T23:56:57.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:57.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376338+0000", 2024-09-14T23:56:57.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:57.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376339+0000", 2024-09-14T23:56:57.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9100000000000003e-07 2024-09-14T23:56:57.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:57.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376342+0000", 2024-09-14T23:56:57.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7e-06 2024-09-14T23:56:57.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:57.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376350+0000", 2024-09-14T23:56:57.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5280000000000003e-06 2024-09-14T23:56:57.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:57.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376356+0000", 2024-09-14T23:56:57.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6010000000000002e-06 2024-09-14T23:56:57.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:57.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427047+0000", 2024-09-14T23:56:57.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050690962999999999 2024-09-14T23:56:57.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:57.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427050+0000", 2024-09-14T23:56:57.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.517e-06 2024-09-14T23:56:57.837 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.837 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:57.837 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.837 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5900 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:56:57.837 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376366+0000", 2024-09-14T23:56:57.837 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.359996214999999, 2024-09-14T23:56:57.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050697858999999998, 2024-09-14T23:56:57.837 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:57.837 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:57.838 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:56:57.838 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:56:57.838 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:56:57.838 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:56:57.838 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5900 2024-09-14T23:56:57.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.838 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:57.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:57.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376366+0000", 2024-09-14T23:56:57.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:57.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376366+0000", 2024-09-14T23:56:57.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:57.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:57.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377650+0000", 2024-09-14T23:56:57.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001283834 2024-09-14T23:56:57.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:57.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377655+0000", 2024-09-14T23:56:57.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7230000000000004e-06 2024-09-14T23:56:57.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:57.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377667+0000", 2024-09-14T23:56:57.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1746000000000001e-05 2024-09-14T23:56:57.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:57.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377676+0000", 2024-09-14T23:56:57.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3619999999999995e-06 2024-09-14T23:56:57.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:57.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427061+0000", 2024-09-14T23:56:57.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049384755000000002 2024-09-14T23:56:57.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:57.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:57.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:57.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427064+0000", 2024-09-14T23:56:57.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.439e-06 2024-09-14T23:56:57.842 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.842 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:57.842 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.842 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:57.842 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:57.842 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:56:57.844 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T23:56:57.844 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:56:57.842+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:56:57.944 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:56:57.943+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:56:57.991 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:56:57.992 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:56:57.992 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:56:57.992 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:56:58.000 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T23:56:58.045 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:56:58.044+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:56:58.053 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is set 2024-09-14T23:56:58.147 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:56:58.145+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:56:58.158 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:56:58.158 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:56:58.158 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:56:58.158 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:56:58.159 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:56:58.168 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T23:56:58.247 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:56:58.246+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:56:58.322 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:56:58.322 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:56:58.323 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:56:58.323 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:56:58.323 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.323 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:3578 3.4 e29/23 3:2298d860:::rbd_data.1118de1168b.0000000000000030:head v 29'15, mlcod=29'13)", 2024-09-14T23:56:58.323 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.036455+0000", 2024-09-14T23:56:58.323 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.283500741000001, 2024-09-14T23:56:58.323 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019449440000000001, 2024-09-14T23:56:58.323 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:58.323 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:58.323 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:58.323 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:58.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.324 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:58.324 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036455+0000", 2024-09-14T23:56:58.324 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:58.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.324 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:58.324 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036455+0000", 2024-09-14T23:56:58.324 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:58.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.324 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:58.324 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036456+0000", 2024-09-14T23:56:58.325 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0999999999999997e-07 2024-09-14T23:56:58.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.325 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:58.325 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036463+0000", 2024-09-14T23:56:58.325 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2200000000000003e-06 2024-09-14T23:56:58.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.325 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:58.325 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036478+0000", 2024-09-14T23:56:58.325 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4977000000000001e-05 2024-09-14T23:56:58.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.326 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.326 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:58.326 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036485+0000", 2024-09-14T23:56:58.326 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0559999999999999e-06 2024-09-14T23:56:58.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.326 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.326 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:58.326 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036693+0000", 2024-09-14T23:56:58.326 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000208051 2024-09-14T23:56:58.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.326 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.326 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:58.327 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036715+0000", 2024-09-14T23:56:58.327 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1330000000000001e-05 2024-09-14T23:56:58.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.327 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.327 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:58.327 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.038356+0000", 2024-09-14T23:56:58.327 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001641586 2024-09-14T23:56:58.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.327 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.327 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:58.327 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.038400+0000", 2024-09-14T23:56:58.327 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3914e-05 2024-09-14T23:56:58.327 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:58.327 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:58.328 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:58.328 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.328 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.328 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:3579 3.4 e29/23 3:27290fe6:::rbd_data.1118de1168b.0000000000000032:head v 29'16, mlcod=29'13)", 2024-09-14T23:56:58.328 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.037085+0000", 2024-09-14T23:56:58.328 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.282871150999998, 2024-09-14T23:56:58.328 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0020019220000000002, 2024-09-14T23:56:58.328 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:58.328 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:58.328 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:58.328 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:58.328 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.328 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:58.329 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037085+0000", 2024-09-14T23:56:58.329 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:58.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.329 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.329 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:58.329 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037085+0000", 2024-09-14T23:56:58.329 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:58.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.329 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.329 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:58.329 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037085+0000", 2024-09-14T23:56:58.329 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5499999999999997e-07 2024-09-14T23:56:58.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.330 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.330 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:58.330 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037091+0000", 2024-09-14T23:56:58.330 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7869999999999998e-06 2024-09-14T23:56:58.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.330 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.330 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:58.330 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037102+0000", 2024-09-14T23:56:58.330 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.031e-05 2024-09-14T23:56:58.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.330 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.331 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:58.331 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037116+0000", 2024-09-14T23:56:58.331 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4119e-05 2024-09-14T23:56:58.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.331 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.331 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:58.331 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037219+0000", 2024-09-14T23:56:58.331 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010372799999999999 2024-09-14T23:56:58.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.331 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.331 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:58.331 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037239+0000", 2024-09-14T23:56:58.331 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9993e-05 2024-09-14T23:56:58.332 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.332 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.332 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:58.332 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.039046+0000", 2024-09-14T23:56:58.332 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001806666 2024-09-14T23:56:58.332 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.332 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.332 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:58.332 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.039087+0000", 2024-09-14T23:56:58.332 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0564e-05 2024-09-14T23:56:58.332 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:58.332 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:58.332 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:58.332 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.333 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.333 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:3584 3.4 e29/23 3:2779453d:::rbd_data.1118de1168b.0000000000000034:head v 29'17, mlcod=29'13)", 2024-09-14T23:56:58.333 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.037588+0000", 2024-09-14T23:56:58.333 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.282368112, 2024-09-14T23:56:58.333 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001526828, 2024-09-14T23:56:58.333 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:58.333 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:58.333 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:58.333 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:58.333 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.333 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:58.333 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037588+0000", 2024-09-14T23:56:58.333 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:58.334 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.334 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.334 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:58.334 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037588+0000", 2024-09-14T23:56:58.334 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:58.334 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.334 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.334 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:58.334 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037588+0000", 2024-09-14T23:56:58.334 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1200000000000002e-07 2024-09-14T23:56:58.334 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.334 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.334 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:58.334 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037595+0000", 2024-09-14T23:56:58.335 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7580000000000003e-06 2024-09-14T23:56:58.335 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.335 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.335 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:58.335 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037609+0000", 2024-09-14T23:56:58.335 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4037e-05 2024-09-14T23:56:58.335 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.335 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.335 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:58.335 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037616+0000", 2024-09-14T23:56:58.335 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0509999999999999e-06 2024-09-14T23:56:58.335 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.335 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.336 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:58.336 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037787+0000", 2024-09-14T23:56:58.336 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017084899999999999 2024-09-14T23:56:58.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.336 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:58.336 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037806+0000", 2024-09-14T23:56:58.336 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9159000000000001e-05 2024-09-14T23:56:58.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.336 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:58.336 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.039089+0000", 2024-09-14T23:56:58.336 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012823909999999999 2024-09-14T23:56:58.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.337 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.337 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:58.337 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.039115+0000", 2024-09-14T23:56:58.337 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5871e-05 2024-09-14T23:56:58.337 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:58.337 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:58.337 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:58.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.337 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.337 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:5280 3.0 e29/23 3:0760f535:::rbd_data.1118de1168b.000000000000009a:head v 29'11, mlcod=29'9)", 2024-09-14T23:56:58.337 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.262702+0000", 2024-09-14T23:56:58.337 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.057253732, 2024-09-14T23:56:58.338 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001433122, 2024-09-14T23:56:58.338 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:58.338 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:58.338 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:58.338 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:58.338 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.338 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:58.338 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262702+0000", 2024-09-14T23:56:58.338 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:58.338 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.338 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.338 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:58.338 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262702+0000", 2024-09-14T23:56:58.339 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:58.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.339 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.339 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:58.339 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262704+0000", 2024-09-14T23:56:58.339 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.384e-06 2024-09-14T23:56:58.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.339 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.339 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:58.339 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262791+0000", 2024-09-14T23:56:58.339 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7264999999999999e-05 2024-09-14T23:56:58.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.339 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.340 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:58.340 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262813+0000", 2024-09-14T23:56:58.340 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.173e-05 2024-09-14T23:56:58.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.340 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.340 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:58.340 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262822+0000", 2024-09-14T23:56:58.340 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5929999999999995e-06 2024-09-14T23:56:58.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.340 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.340 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:58.340 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262886+0000", 2024-09-14T23:56:58.340 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4052000000000003e-05 2024-09-14T23:56:58.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.341 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.341 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:58.341 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262920+0000", 2024-09-14T23:56:58.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.366e-05 2024-09-14T23:56:58.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.341 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.341 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:58.341 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.264095+0000", 2024-09-14T23:56:58.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001175178 2024-09-14T23:56:58.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.341 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.341 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:58.342 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.264135+0000", 2024-09-14T23:56:58.342 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0259999999999997e-05 2024-09-14T23:56:58.342 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:58.342 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:58.342 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:58.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.342 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.342 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:5281 3.0 e29/23 3:0760f535:::rbd_data.1118de1168b.000000000000009a:head v 29'13, mlcod=29'9)", 2024-09-14T23:56:58.342 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.263503+0000", 2024-09-14T23:56:58.342 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.056452868000001, 2024-09-14T23:56:58.342 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012834910000000001, 2024-09-14T23:56:58.342 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:58.342 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:58.342 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:58.343 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:58.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.343 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:58.343 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.263503+0000", 2024-09-14T23:56:58.343 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:58.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.343 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:58.343 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.263503+0000", 2024-09-14T23:56:58.343 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:58.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.343 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:58.343 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.263504+0000", 2024-09-14T23:56:58.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4600000000000004e-07 2024-09-14T23:56:58.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.344 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:58.344 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.263510+0000", 2024-09-14T23:56:58.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5950000000000004e-06 2024-09-14T23:56:58.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.344 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:58.344 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.263522+0000", 2024-09-14T23:56:58.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1289e-05 2024-09-14T23:56:58.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.344 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:58.345 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.263529+0000", 2024-09-14T23:56:58.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.951e-06 2024-09-14T23:56:58.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:58.345 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.263656+0000", 2024-09-14T23:56:58.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012731099999999999 2024-09-14T23:56:58.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:58.345 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.263675+0000", 2024-09-14T23:56:58.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9147000000000002e-05 2024-09-14T23:56:58.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.346 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:58.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.264746+0000", 2024-09-14T23:56:58.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001070997 2024-09-14T23:56:58.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.346 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:58.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.264787+0000", 2024-09-14T23:56:58.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0555e-05 2024-09-14T23:56:58.346 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:58.346 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:58.346 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:58.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.347 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:17221 3.4 e33/23 3:225c4d41:::rbd_data.11187be34cc8.0000000000000001:head v 33'50, mlcod=32'48)", 2024-09-14T23:56:58.347 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:42.983242+0000", 2024-09-14T23:56:58.347 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.336714352, 2024-09-14T23:56:58.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001313032, 2024-09-14T23:56:58.347 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:58.347 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:58.347 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:58.347 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:58.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.347 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:58.347 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.983242+0000", 2024-09-14T23:56:58.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:58.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.348 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:58.348 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.983242+0000", 2024-09-14T23:56:58.348 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:58.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.348 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:58.348 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.983243+0000", 2024-09-14T23:56:58.348 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.102e-06 2024-09-14T23:56:58.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.348 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:58.349 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.983256+0000", 2024-09-14T23:56:58.349 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2897e-05 2024-09-14T23:56:58.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.349 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:58.349 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.983277+0000", 2024-09-14T23:56:58.349 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1566999999999999e-05 2024-09-14T23:56:58.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.349 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:58.349 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.983288+0000", 2024-09-14T23:56:58.349 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0457e-05 2024-09-14T23:56:58.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.350 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:58.350 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.983355+0000", 2024-09-14T23:56:58.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7337999999999998e-05 2024-09-14T23:56:58.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.350 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:58.350 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.983422+0000", 2024-09-14T23:56:58.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6716000000000003e-05 2024-09-14T23:56:58.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.350 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:58.350 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.984515+0000", 2024-09-14T23:56:58.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001093761 2024-09-14T23:56:58.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.351 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:58.351 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.984555+0000", 2024-09-14T23:56:58.351 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9193999999999998e-05 2024-09-14T23:56:58.351 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:58.351 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:58.351 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:58.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.351 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:17453 3.0 e33/23 3:020fb8a1:::rbd_data.11187be34cc8.0000000000000005:head v 33'46, mlcod=33'43)", 2024-09-14T23:56:58.351 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:43.265507+0000", 2024-09-14T23:56:58.351 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.054448504, 2024-09-14T23:56:58.351 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013288880000000001, 2024-09-14T23:56:58.352 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:58.352 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:58.352 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:58.352 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:58.352 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.352 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:58.352 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.265507+0000", 2024-09-14T23:56:58.352 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:58.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.352 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.352 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:58.352 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.265507+0000", 2024-09-14T23:56:58.352 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:58.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:58.353 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.265508+0000", 2024-09-14T23:56:58.353 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4400000000000001e-07 2024-09-14T23:56:58.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:58.353 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.265513+0000", 2024-09-14T23:56:58.353 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.887e-06 2024-09-14T23:56:58.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:58.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.265520+0000", 2024-09-14T23:56:58.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8249999999999999e-06 2024-09-14T23:56:58.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.354 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:58.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.265524+0000", 2024-09-14T23:56:58.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8130000000000003e-06 2024-09-14T23:56:58.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.354 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:58.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.265727+0000", 2024-09-14T23:56:58.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000202518 2024-09-14T23:56:58.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.355 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.355 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:58.355 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.265744+0000", 2024-09-14T23:56:58.355 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.753e-05 2024-09-14T23:56:58.355 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.355 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.355 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:58.355 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.266802+0000", 2024-09-14T23:56:58.355 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001057992 2024-09-14T23:56:58.355 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.355 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.355 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:58.355 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.266836+0000", 2024-09-14T23:56:58.355 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3878999999999999e-05 2024-09-14T23:56:58.356 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:58.356 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:58.356 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:58.356 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.356 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.356 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:17495 3.4 e33/23 3:225c4d41:::rbd_data.11187be34cc8.0000000000000001:head v 33'54, mlcod=33'52)", 2024-09-14T23:56:58.356 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:43.321582+0000", 2024-09-14T23:56:58.356 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.998374102, 2024-09-14T23:56:58.356 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012315080000000001, 2024-09-14T23:56:58.356 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:58.356 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:58.356 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:58.357 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:58.357 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.357 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:58.357 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.321582+0000", 2024-09-14T23:56:58.357 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:58.357 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.357 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.357 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:58.357 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.321582+0000", 2024-09-14T23:56:58.357 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:58.357 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.357 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.357 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:58.357 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.321582+0000", 2024-09-14T23:56:58.358 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1800000000000002e-07 2024-09-14T23:56:58.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.358 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:58.358 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.321587+0000", 2024-09-14T23:56:58.358 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5009999999999998e-06 2024-09-14T23:56:58.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.358 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:58.358 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.321594+0000", 2024-09-14T23:56:58.358 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.379e-06 2024-09-14T23:56:58.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.359 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:58.359 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.321609+0000", 2024-09-14T23:56:58.359 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5316999999999999e-05 2024-09-14T23:56:58.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.359 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.359 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:58.359 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.321662+0000", 2024-09-14T23:56:58.359 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2754999999999999e-05 2024-09-14T23:56:58.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.360 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.360 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:58.360 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.321682+0000", 2024-09-14T23:56:58.360 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0064999999999998e-05 2024-09-14T23:56:58.360 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.360 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.360 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:58.360 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.322776+0000", 2024-09-14T23:56:58.360 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010939369999999999 2024-09-14T23:56:58.360 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.360 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.361 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:58.361 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.322813+0000", 2024-09-14T23:56:58.361 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7236000000000001e-05 2024-09-14T23:56:58.361 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:58.361 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:58.361 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:58.361 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.361 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.361 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:17655 3.4 e33/23 3:225c4d41:::rbd_data.11187be34cc8.0000000000000001:head v 33'58, mlcod=33'56)", 2024-09-14T23:56:58.361 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:43.492726+0000", 2024-09-14T23:56:58.361 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.827229929, 2024-09-14T23:56:58.361 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012321809999999999, 2024-09-14T23:56:58.361 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:58.362 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:58.362 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:58.362 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:58.362 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.362 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:58.362 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.492726+0000", 2024-09-14T23:56:58.362 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:58.362 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.362 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.362 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:58.362 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.492726+0000", 2024-09-14T23:56:58.362 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:58.362 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.363 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.363 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:58.363 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.492727+0000", 2024-09-14T23:56:58.363 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2810000000000001e-06 2024-09-14T23:56:58.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.363 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.363 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:58.363 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.492736+0000", 2024-09-14T23:56:58.363 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9299999999999992e-06 2024-09-14T23:56:58.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.363 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.363 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:58.363 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.492756+0000", 2024-09-14T23:56:58.363 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0180999999999999e-05 2024-09-14T23:56:58.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.364 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.364 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:58.364 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.492771+0000", 2024-09-14T23:56:58.364 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4265e-05 2024-09-14T23:56:58.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.364 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.364 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:58.364 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.492833+0000", 2024-09-14T23:56:58.364 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2434999999999994e-05 2024-09-14T23:56:58.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.364 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.364 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:58.364 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.492854+0000", 2024-09-14T23:56:58.365 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0457000000000001e-05 2024-09-14T23:56:58.365 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.365 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.365 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:58.365 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.493920+0000", 2024-09-14T23:56:58.365 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001066142 2024-09-14T23:56:58.365 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.365 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.365 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:58.365 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.493958+0000", 2024-09-14T23:56:58.365 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8489999999999999e-05 2024-09-14T23:56:58.365 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:58.365 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:58.366 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:58.366 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.366 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.366 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:17657 3.0 e33/23 3:05bad47a:::rbd_data.11187be34cc8.0000000000000003:head v 33'53, mlcod=33'51)", 2024-09-14T23:56:58.366 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:43.493115+0000", 2024-09-14T23:56:58.366 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.826840419, 2024-09-14T23:56:58.366 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0014551950000000001, 2024-09-14T23:56:58.366 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:58.366 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:58.366 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:58.366 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:58.366 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.366 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:58.366 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.493115+0000", 2024-09-14T23:56:58.367 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:58.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.367 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.367 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:58.367 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.493115+0000", 2024-09-14T23:56:58.367 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:58.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.367 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.367 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:58.367 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.493116+0000", 2024-09-14T23:56:58.367 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7899999999999998e-07 2024-09-14T23:56:58.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.367 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.368 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:58.368 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.493121+0000", 2024-09-14T23:56:58.368 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3959999999999997e-06 2024-09-14T23:56:58.368 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.368 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.368 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:58.368 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.493131+0000", 2024-09-14T23:56:58.368 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4900000000000006e-06 2024-09-14T23:56:58.368 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.368 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.368 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:58.368 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.493138+0000", 2024-09-14T23:56:58.368 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6730000000000003e-06 2024-09-14T23:56:58.369 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.369 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.369 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:58.369 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.493263+0000", 2024-09-14T23:56:58.369 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000125468 2024-09-14T23:56:58.369 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.369 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.369 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:58.369 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.493282+0000", 2024-09-14T23:56:58.369 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8831999999999999e-05 2024-09-14T23:56:58.369 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.369 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.369 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:58.370 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.494528+0000", 2024-09-14T23:56:58.370 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012456990000000001 2024-09-14T23:56:58.370 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.370 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.370 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:58.370 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.494571+0000", 2024-09-14T23:56:58.370 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3058e-05 2024-09-14T23:56:58.370 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:58.370 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:58.370 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:58.370 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.370 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.370 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:17997 3.0 e33/23 3:05bad47a:::rbd_data.11187be34cc8.0000000000000003:head v 33'68, mlcod=33'66)", 2024-09-14T23:56:58.371 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:43.822910+0000", 2024-09-14T23:56:58.371 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.497045877, 2024-09-14T23:56:58.371 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001512535, 2024-09-14T23:56:58.371 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:58.371 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:58.371 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:58.371 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:58.371 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.371 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:58.371 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.822910+0000", 2024-09-14T23:56:58.371 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:58.371 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.371 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.372 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:58.372 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.822910+0000", 2024-09-14T23:56:58.372 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:58.372 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.372 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.372 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:58.372 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.822910+0000", 2024-09-14T23:56:58.372 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9999999999999998e-07 2024-09-14T23:56:58.372 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.372 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.372 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:58.372 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.822915+0000", 2024-09-14T23:56:58.373 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7790000000000003e-06 2024-09-14T23:56:58.373 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.373 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.373 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:58.373 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.822923+0000", 2024-09-14T23:56:58.373 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2300000000000008e-06 2024-09-14T23:56:58.373 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.373 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.373 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:58.373 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.822929+0000", 2024-09-14T23:56:58.373 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5219999999999998e-06 2024-09-14T23:56:58.373 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.373 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.374 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:58.374 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.823001+0000", 2024-09-14T23:56:58.374 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1903000000000006e-05 2024-09-14T23:56:58.374 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.374 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.374 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:58.374 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.823021+0000", 2024-09-14T23:56:58.374 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0129e-05 2024-09-14T23:56:58.374 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.374 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.374 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:58.374 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.824376+0000", 2024-09-14T23:56:58.374 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013548550000000001 2024-09-14T23:56:58.375 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.375 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.375 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:58.375 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.824423+0000", 2024-09-14T23:56:58.375 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6717e-05 2024-09-14T23:56:58.375 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:58.375 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:58.375 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:58.375 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.375 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.375 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:17999 3.0 e33/23 3:020fb8a1:::rbd_data.11187be34cc8.0000000000000005:head v 33'69, mlcod=33'66)", 2024-09-14T23:56:58.375 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:43.823649+0000", 2024-09-14T23:56:58.375 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.496306613, 2024-09-14T23:56:58.375 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013396320000000001, 2024-09-14T23:56:58.376 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:58.376 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:58.376 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:58.376 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:58.376 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.376 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:58.376 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.823649+0000", 2024-09-14T23:56:58.376 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:58.376 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.376 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.376 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:58.376 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.823649+0000", 2024-09-14T23:56:58.376 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:58.377 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.377 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.377 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:58.377 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.823650+0000", 2024-09-14T23:56:58.377 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7499999999999999e-07 2024-09-14T23:56:58.377 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.377 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.377 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:58.377 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.823656+0000", 2024-09-14T23:56:58.377 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1500000000000004e-06 2024-09-14T23:56:58.377 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.377 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.377 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:58.378 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.823667+0000", 2024-09-14T23:56:58.378 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0788000000000001e-05 2024-09-14T23:56:58.378 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.378 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.378 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:58.378 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.823673+0000", 2024-09-14T23:56:58.378 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4099999999999996e-06 2024-09-14T23:56:58.378 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.378 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.378 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:58.378 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.823722+0000", 2024-09-14T23:56:58.378 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8733000000000002e-05 2024-09-14T23:56:58.378 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.378 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.379 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:58.379 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.823741+0000", 2024-09-14T23:56:58.379 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9156000000000001e-05 2024-09-14T23:56:58.379 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.379 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.379 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:58.379 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.824961+0000", 2024-09-14T23:56:58.379 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001219901 2024-09-14T23:56:58.379 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.379 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.379 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:58.379 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.824989+0000", 2024-09-14T23:56:58.379 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7619000000000001e-05 2024-09-14T23:56:58.380 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:58.380 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:58.380 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:58.380 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.380 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.380 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:18030 3.4 e33/23 3:225c4d41:::rbd_data.11187be34cc8.0000000000000001:head v 33'63, mlcod=33'61)", 2024-09-14T23:56:58.380 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:43.852748+0000", 2024-09-14T23:56:58.380 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.467208053, 2024-09-14T23:56:58.380 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001530696, 2024-09-14T23:56:58.380 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:58.380 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:58.380 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:58.380 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:58.380 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.381 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:58.381 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.852748+0000", 2024-09-14T23:56:58.381 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:58.381 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.381 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.381 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:58.381 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.852748+0000", 2024-09-14T23:56:58.381 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:58.381 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.381 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.381 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:58.381 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.852749+0000", 2024-09-14T23:56:58.381 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.164e-06 2024-09-14T23:56:58.381 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.382 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.382 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:58.382 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.852759+0000", 2024-09-14T23:56:58.382 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0265000000000001e-05 2024-09-14T23:56:58.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.382 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.382 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:58.382 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.852779+0000", 2024-09-14T23:56:58.382 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9247999999999999e-05 2024-09-14T23:56:58.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.382 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.382 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:58.382 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.852788+0000", 2024-09-14T23:56:58.382 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3260000000000002e-06 2024-09-14T23:56:58.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.383 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.383 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:58.383 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.853121+0000", 2024-09-14T23:56:58.383 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00033336699999999999 2024-09-14T23:56:58.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.383 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.383 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:58.383 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.853142+0000", 2024-09-14T23:56:58.383 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0500999999999999e-05 2024-09-14T23:56:58.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.383 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.383 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:58.384 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.854240+0000", 2024-09-14T23:56:58.384 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001098665 2024-09-14T23:56:58.384 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.384 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.384 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:58.384 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.854279+0000", 2024-09-14T23:56:58.384 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8160000000000001e-05 2024-09-14T23:56:58.384 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:58.384 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:58.384 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:58.384 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.384 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.384 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:18260 3.4 e34/23 3:225c4d41:::rbd_data.11187be34cc8.0000000000000001:head v 34'65, mlcod=33'63)", 2024-09-14T23:56:58.384 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:44.113710+0000", 2024-09-14T23:56:58.385 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.206245868, 2024-09-14T23:56:58.385 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001257174, 2024-09-14T23:56:58.385 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:58.385 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:58.385 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:58.385 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:58.385 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.385 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:58.385 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.113710+0000", 2024-09-14T23:56:58.385 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:58.385 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.385 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.385 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:58.386 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.113710+0000", 2024-09-14T23:56:58.386 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:58.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.386 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.386 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:58.386 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.113710+0000", 2024-09-14T23:56:58.386 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4900000000000001e-07 2024-09-14T23:56:58.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.386 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.386 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:58.386 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.113716+0000", 2024-09-14T23:56:58.386 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5110000000000003e-06 2024-09-14T23:56:58.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.387 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.387 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:58.387 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.113724+0000", 2024-09-14T23:56:58.387 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1499999999999999e-06 2024-09-14T23:56:58.387 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.387 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.387 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:58.387 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.113730+0000", 2024-09-14T23:56:58.387 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6919999999999999e-06 2024-09-14T23:56:58.387 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.387 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.387 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:58.387 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.113960+0000", 2024-09-14T23:56:58.388 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00023030699999999999 2024-09-14T23:56:58.388 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.388 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.388 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:58.388 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.113986+0000", 2024-09-14T23:56:58.388 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6330999999999999e-05 2024-09-14T23:56:58.388 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.388 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.388 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:58.388 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.114923+0000", 2024-09-14T23:56:58.388 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00093685100000000005 2024-09-14T23:56:58.388 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.388 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.388 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:58.389 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.114967+0000", 2024-09-14T23:56:58.389 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3883e-05 2024-09-14T23:56:58.389 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:58.389 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:58.389 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:58.389 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.389 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.389 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:18356 3.0 e34/23 3:05bad47a:::rbd_data.11187be34cc8.0000000000000003:head v 34'84, mlcod=34'82)", 2024-09-14T23:56:58.389 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:44.206422+0000", 2024-09-14T23:56:58.389 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.113533459999999, 2024-09-14T23:56:58.389 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001310603, 2024-09-14T23:56:58.389 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:58.389 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:58.390 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:58.390 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:58.390 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.390 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:58.390 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.206422+0000", 2024-09-14T23:56:58.390 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:58.390 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.390 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.390 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:58.390 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.206422+0000", 2024-09-14T23:56:58.390 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:58.390 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.390 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.390 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:58.391 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.206424+0000", 2024-09-14T23:56:58.391 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3e-06 2024-09-14T23:56:58.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.391 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.391 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:58.391 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.206437+0000", 2024-09-14T23:56:58.391 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3040999999999999e-05 2024-09-14T23:56:58.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.391 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.391 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:58.391 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.206455+0000", 2024-09-14T23:56:58.391 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8369e-05 2024-09-14T23:56:58.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.391 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.392 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:58.392 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.206466+0000", 2024-09-14T23:56:58.392 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0767999999999999e-05 2024-09-14T23:56:58.392 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.392 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.392 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:58.392 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.206551+0000", 2024-09-14T23:56:58.392 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4858999999999995e-05 2024-09-14T23:56:58.392 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.392 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.392 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:58.392 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.206574+0000", 2024-09-14T23:56:58.392 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2741000000000001e-05 2024-09-14T23:56:58.393 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.393 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.393 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:58.393 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.207690+0000", 2024-09-14T23:56:58.393 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011167880000000001 2024-09-14T23:56:58.393 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.393 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.393 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:58.393 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.207733+0000", 2024-09-14T23:56:58.393 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2737000000000001e-05 2024-09-14T23:56:58.393 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:58.393 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:58.393 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:58.393 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.394 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.394 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:18354 3.4 e34/23 3:225c4d41:::rbd_data.11187be34cc8.0000000000000001:head v 34'67, mlcod=34'65)", 2024-09-14T23:56:58.394 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:44.206794+0000", 2024-09-14T23:56:58.394 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.113161475, 2024-09-14T23:56:58.394 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013689819999999999, 2024-09-14T23:56:58.394 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:58.394 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:58.394 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:58.394 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:58.394 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.394 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:58.394 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.206794+0000", 2024-09-14T23:56:58.394 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:58.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.395 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:58.395 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.206794+0000", 2024-09-14T23:56:58.395 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:58.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.395 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:58.395 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.206795+0000", 2024-09-14T23:56:58.395 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5400000000000002e-07 2024-09-14T23:56:58.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.395 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:58.395 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.206800+0000", 2024-09-14T23:56:58.396 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0479999999999998e-06 2024-09-14T23:56:58.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.396 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.396 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:58.396 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.206808+0000", 2024-09-14T23:56:58.396 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8420000000000005e-06 2024-09-14T23:56:58.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.396 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.396 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:58.396 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.206814+0000", 2024-09-14T23:56:58.396 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8640000000000001e-06 2024-09-14T23:56:58.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.396 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.397 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:58.397 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.206988+0000", 2024-09-14T23:56:58.397 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000174599 2024-09-14T23:56:58.397 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.397 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.397 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:58.397 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.207007+0000", 2024-09-14T23:56:58.397 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8695e-05 2024-09-14T23:56:58.397 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.397 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.397 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:58.397 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.208109+0000", 2024-09-14T23:56:58.397 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011016310000000001 2024-09-14T23:56:58.398 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.398 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.398 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:58.398 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.208163+0000", 2024-09-14T23:56:58.398 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4849000000000003e-05 2024-09-14T23:56:58.398 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:58.398 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:58.398 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:58.398 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.398 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.398 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:32024 3.9 e39/23 3:94dc260b:::rbd_data.111814b482f8.0000000000000009:head v 39'45, mlcod=38'43)", 2024-09-14T23:56:58.398 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:49.061018+0000", 2024-09-14T23:56:58.398 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.2589378119999992, 2024-09-14T23:56:58.399 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018482240000000001, 2024-09-14T23:56:58.399 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:58.399 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:58.399 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:58.399 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:58.399 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.399 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:58.399 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061018+0000", 2024-09-14T23:56:58.399 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:58.399 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.399 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.399 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:58.399 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061018+0000", 2024-09-14T23:56:58.399 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:58.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.400 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:58.400 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061018+0000", 2024-09-14T23:56:58.400 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9999999999999998e-07 2024-09-14T23:56:58.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.400 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:58.400 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061211+0000", 2024-09-14T23:56:58.400 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019206 2024-09-14T23:56:58.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.400 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:58.401 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061221+0000", 2024-09-14T23:56:58.401 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0659000000000001e-05 2024-09-14T23:56:58.401 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.401 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.401 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:58.401 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061230+0000", 2024-09-14T23:56:58.401 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4610000000000008e-06 2024-09-14T23:56:58.401 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.401 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.401 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:58.401 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061317+0000", 2024-09-14T23:56:58.401 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7514000000000003e-05 2024-09-14T23:56:58.401 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.401 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.402 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:58.402 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061338+0000", 2024-09-14T23:56:58.402 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0395999999999999e-05 2024-09-14T23:56:58.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.402 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:58.402 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.062830+0000", 2024-09-14T23:56:58.402 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0014921940000000001 2024-09-14T23:56:58.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.402 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:58.402 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.062866+0000", 2024-09-14T23:56:58.402 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6539999999999999e-05 2024-09-14T23:56:58.402 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:58.403 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:58.403 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:58.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.403 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.403 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:32420 3.9 e39/23 3:91dc34e7:::rbd_data.111814b482f8.000000000000001e:head v 39'46, mlcod=38'44)", 2024-09-14T23:56:58.403 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:49.086124+0000", 2024-09-14T23:56:58.403 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.2338323130000006, 2024-09-14T23:56:58.403 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001686819, 2024-09-14T23:56:58.403 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:58.403 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:58.403 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:58.403 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:58.403 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.403 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:58.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086124+0000", 2024-09-14T23:56:58.404 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:58.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.404 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:58.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086124+0000", 2024-09-14T23:56:58.404 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:58.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.404 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:58.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086125+0000", 2024-09-14T23:56:58.404 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.361e-06 2024-09-14T23:56:58.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.405 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:58.405 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086605+0000", 2024-09-14T23:56:58.405 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00048026600000000003 2024-09-14T23:56:58.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.405 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:58.405 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086626+0000", 2024-09-14T23:56:58.405 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.037e-05 2024-09-14T23:56:58.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.405 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:58.406 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086635+0000", 2024-09-14T23:56:58.406 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5880000000000004e-06 2024-09-14T23:56:58.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.406 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.406 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:58.406 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086701+0000", 2024-09-14T23:56:58.406 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5905e-05 2024-09-14T23:56:58.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.406 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.406 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:58.406 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086727+0000", 2024-09-14T23:56:58.406 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5573000000000001e-05 2024-09-14T23:56:58.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.406 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.407 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:58.407 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.087767+0000", 2024-09-14T23:56:58.407 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001040537 2024-09-14T23:56:58.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.407 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.407 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:58.407 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.087810+0000", 2024-09-14T23:56:58.407 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3219000000000001e-05 2024-09-14T23:56:58.407 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:58.407 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:58.407 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:58.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.407 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.407 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:34296 3.9 e39/23 3:974a38c7:::rbd_data.111814b482f8.000000000000003a:head v 39'47, mlcod=39'45)", 2024-09-14T23:56:58.408 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:49.192105+0000", 2024-09-14T23:56:58.408 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.1278504300000005, 2024-09-14T23:56:58.408 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001607516, 2024-09-14T23:56:58.408 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:58.408 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:58.408 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:58.408 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:58.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.408 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:58.408 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.192105+0000", 2024-09-14T23:56:58.408 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:58.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:58.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.192105+0000", 2024-09-14T23:56:58.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:58.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:58.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.192107+0000", 2024-09-14T23:56:58.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3549999999999999e-06 2024-09-14T23:56:58.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:58.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.192332+0000", 2024-09-14T23:56:58.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022486300000000001 2024-09-14T23:56:58.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.410 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.410 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:58.410 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.192349+0000", 2024-09-14T23:56:58.410 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6966e-05 2024-09-14T23:56:58.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.410 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.410 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:58.410 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.192358+0000", 2024-09-14T23:56:58.410 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8309999999999996e-06 2024-09-14T23:56:58.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.410 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.410 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:58.410 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.192422+0000", 2024-09-14T23:56:58.410 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3915e-05 2024-09-14T23:56:58.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.411 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.411 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:58.411 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.192450+0000", 2024-09-14T23:56:58.411 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7957e-05 2024-09-14T23:56:58.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.411 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.411 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:58.411 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.193662+0000", 2024-09-14T23:56:58.411 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012113270000000001 2024-09-14T23:56:58.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.411 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.411 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:58.411 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.193713+0000", 2024-09-14T23:56:58.412 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1301999999999998e-05 2024-09-14T23:56:58.412 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:58.412 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:58.412 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:58.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.412 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.412 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:35145 3.4 e39/23 3:22e94ee2:::rbd_data.111814b482f8.0000000000000047:head v 39'92, mlcod=39'90)", 2024-09-14T23:56:58.412 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:49.248843+0000", 2024-09-14T23:56:58.412 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.0711130119999996, 2024-09-14T23:56:58.412 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001261206, 2024-09-14T23:56:58.412 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:56:58.412 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:56:58.413 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:56:58.413 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:56:58.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.413 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:56:58.413 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.248843+0000", 2024-09-14T23:56:58.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:58.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.413 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:56:58.413 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.248843+0000", 2024-09-14T23:56:58.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:56:58.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.414 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:56:58.414 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.248844+0000", 2024-09-14T23:56:58.414 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.271e-06 2024-09-14T23:56:58.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.414 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.414 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:56:58.414 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.248934+0000", 2024-09-14T23:56:58.414 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0109000000000001e-05 2024-09-14T23:56:58.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.414 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.414 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:56:58.414 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.248955+0000", 2024-09-14T23:56:58.414 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0877999999999999e-05 2024-09-14T23:56:58.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.415 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.415 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:56:58.415 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.248965+0000", 2024-09-14T23:56:58.415 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0113e-05 2024-09-14T23:56:58.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.415 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.415 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:56:58.415 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.249035+0000", 2024-09-14T23:56:58.415 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0038999999999998e-05 2024-09-14T23:56:58.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.415 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.415 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:56:58.415 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.249060+0000", 2024-09-14T23:56:58.415 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4722000000000001e-05 2024-09-14T23:56:58.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.416 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.416 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:56:58.416 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.250058+0000", 2024-09-14T23:56:58.416 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00099809599999999997 2024-09-14T23:56:58.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:56:58.416 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:56:58.416 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:56:58.416 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.250104+0000", 2024-09-14T23:56:58.416 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5977999999999999e-05 2024-09-14T23:56:58.416 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:58.416 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:58.416 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:58.416 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:56:58.416 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:56:58.416 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:56:58.417 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone10 from 0x180b433 (overlap 0x180b433) to 0xe6d285 2024-09-14T23:56:58.418 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T23:56:58.419 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-09-14T23:56:58.419 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:56:58.416+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:56:58.518 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:56:58.517+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:56:58.562 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T23:56:58.563 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:56:58.563 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:56:58.563 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:56:58.563 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:56:58.572 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T23:56:58.657 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:56:58.655+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:56:58.689 INFO:teuthology.orchestra.run.smithi042.stdout:1221 read 0x283d04 thru 0x28b032 (0x732f bytes) 2024-09-14T23:56:58.694 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:56:58.694 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:56:58.712 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:56:58.713 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:56:58.721 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:56:58.719+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:56:58.728 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:56:58.728 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:56:58.729 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:56:58.730 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:56:58.730 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:56:58.730 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:56:58.730 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:56:58.739 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T23:56:58.745 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:56:58.745 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:56:58.762 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:56:58.762 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:56:58.779 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:56:58.779 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:56:58.796 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:56:58.796 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:56:58.813 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:56:58.813 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:56:58.822 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:56:58.820+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:56:58.852 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:58.853 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:58.861 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T23:56:58.894 INFO:teuthology.orchestra.run.smithi042.stdout:1224 trunc from 0x180b433 to 0x8840bc 2024-09-14T23:56:58.922 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:56:58.921+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:56:58.932 INFO:teuthology.orchestra.run.smithi042.stdout:1225 clone 12 order 24 su 262144 sc 15 2024-09-14T23:56:58.980 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:58.980 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:58.988 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T23:56:59.032 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:56:59.031+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:56:59.098 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:59.098 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:59.106 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T23:56:59.125 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:56:59.124+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:56:59.226 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:56:59.225+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:56:59.254 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:59.254 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:59.264 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T23:56:59.327 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:56:59.326+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:56:59.412 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:59.412 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:59.421 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T23:56:59.428 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:56:59.427+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:56:59.528 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:59.529 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:59.529 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:56:59.527+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:56:59.537 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T23:56:59.630 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:56:59.629+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:56:59.684 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:59.684 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:59.693 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T23:56:59.731 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:56:59.730+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:56:59.832 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:56:59.830+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:56:59.841 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:56:59.879 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:56:59.880 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T23:56:59.932 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:56:59.931+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:00.027 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:00.027 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:00.033 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:00.032+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:00.036 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T23:57:00.070 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T23:57:00.133 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:00.132+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:00.136 INFO:teuthology.orchestra.run.smithi042.stdout:flattening image image_client.0-clone11 2024-09-14T23:57:00.162 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:00.162 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:00.175 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T23:57:00.261 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:00.260+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:00.278 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-09-14T23:57:00.312 INFO:teuthology.orchestra.run.smithi042.stdout:1226 write 0x3eba954 thru 0x3ec8ac0 (0xe16d bytes) 2024-09-14T23:57:00.317 INFO:teuthology.orchestra.run.smithi042.stdout:1227 write 0x3adabd4 thru 0x3ae169a (0x6ac7 bytes) 2024-09-14T23:57:00.319 INFO:teuthology.orchestra.run.smithi042.stdout:1228 write 0x11a16a7 thru 0x11b003e (0xe998 bytes) 2024-09-14T23:57:00.320 INFO:teuthology.orchestra.run.smithi042.stdout:1229 clone 13 order 19 su 131072 sc 13 2024-09-14T23:57:00.335 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:00.334+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:00.336 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:00.336 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:00.344 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T23:57:00.437 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:00.436+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:00.492 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:00.492 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:00.501 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T23:57:00.541 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:00.540+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:00.639 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:00.638+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:00.648 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:00.648 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:00.657 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T23:57:00.740 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:00.739+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:00.766 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:00.767 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:00.776 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T23:57:00.824 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T23:57:00.841 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:00.840+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:00.924 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:00.925 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:00.934 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T23:57:00.977 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:00.976+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:00.987 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:57:00.987 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:57:01.004 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:57:01.005 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:57:01.021 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:57:01.021 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:57:01.038 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:57:01.038 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:57:01.042 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:01.041+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:01.045 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:01.045 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:57:01.045 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:57:01.045 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:57:01.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.046 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:6285 3.1 3:8172dac6:::rbd_object_map.1118de1168b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:01.046 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.492456+0000", 2024-09-14T23:57:01.046 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 21.550689679000001, 2024-09-14T23:57:01.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0082137960000000006, 2024-09-14T23:57:01.046 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:01.046 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:01.046 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:01.046 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:01.046 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:01.046 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:01.046 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 6285 2024-09-14T23:57:01.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.047 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:01.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:01.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492456+0000", 2024-09-14T23:57:01.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:01.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492456+0000", 2024-09-14T23:57:01.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:01.048 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492456+0000", 2024-09-14T23:57:01.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2799999999999995e-07 2024-09-14T23:57:01.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.048 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:01.048 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492461+0000", 2024-09-14T23:57:01.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0050000000000004e-06 2024-09-14T23:57:01.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.048 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:01.048 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492471+0000", 2024-09-14T23:57:01.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5489999999999995e-06 2024-09-14T23:57:01.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.049 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:01.049 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492478+0000", 2024-09-14T23:57:01.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7549999999999997e-06 2024-09-14T23:57:01.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.049 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:01.049 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.498565+0000", 2024-09-14T23:57:01.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0060878629999999998 2024-09-14T23:57:01.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.049 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:01.049 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.498637+0000", 2024-09-14T23:57:01.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1585e-05 2024-09-14T23:57:01.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.050 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 4", 2024-09-14T23:57:01.050 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.498843+0000", 2024-09-14T23:57:01.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020629399999999999 2024-09-14T23:57:01.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.050 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:57:01.050 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.500532+0000", 2024-09-14T23:57:01.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016882150000000001 2024-09-14T23:57:01.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.050 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:01.050 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.500596+0000", 2024-09-14T23:57:01.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4534000000000002e-05 2024-09-14T23:57:01.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:01.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.500632+0000", 2024-09-14T23:57:01.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5654000000000001e-05 2024-09-14T23:57:01.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:01.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.500669+0000", 2024-09-14T23:57:01.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7614000000000003e-05 2024-09-14T23:57:01.051 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.051 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.051 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.052 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.052 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:6286 3.1 3:8172dac6:::rbd_object_map.1118de1168b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:01.052 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.492592+0000", 2024-09-14T23:57:01.052 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 21.550553054000002, 2024-09-14T23:57:01.052 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0091976439999999996, 2024-09-14T23:57:01.052 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:01.052 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:01.052 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:01.052 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:01.052 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:01.052 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:01.052 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 6286 2024-09-14T23:57:01.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.053 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:01.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.053 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:01.053 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492592+0000", 2024-09-14T23:57:01.053 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.053 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:01.053 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492592+0000", 2024-09-14T23:57:01.053 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.053 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:01.053 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492593+0000", 2024-09-14T23:57:01.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2699999999999998e-07 2024-09-14T23:57:01.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.054 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:01.054 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492598+0000", 2024-09-14T23:57:01.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8690000000000003e-06 2024-09-14T23:57:01.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.054 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:01.054 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492615+0000", 2024-09-14T23:57:01.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6977e-05 2024-09-14T23:57:01.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.054 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:01.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492622+0000", 2024-09-14T23:57:01.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8410000000000002e-06 2024-09-14T23:57:01.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:01.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.499278+0000", 2024-09-14T23:57:01.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0066565419999999997 2024-09-14T23:57:01.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:01.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.499335+0000", 2024-09-14T23:57:01.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6598000000000003e-05 2024-09-14T23:57:01.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 4", 2024-09-14T23:57:01.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.499543+0000", 2024-09-14T23:57:01.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00020844099999999999 2024-09-14T23:57:01.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:57:01.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.500537+0000", 2024-09-14T23:57:01.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00099392600000000001 2024-09-14T23:57:01.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:01.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.501691+0000", 2024-09-14T23:57:01.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011535409999999999 2024-09-14T23:57:01.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.057 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:01.057 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.501749+0000", 2024-09-14T23:57:01.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8233000000000003e-05 2024-09-14T23:57:01.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.057 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:01.057 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.501790+0000", 2024-09-14T23:57:01.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0849000000000001e-05 2024-09-14T23:57:01.057 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.057 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.057 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.058 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:6287 3.1 3:8172dac6:::rbd_object_map.1118de1168b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:01.058 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.492747+0000", 2024-09-14T23:57:01.058 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 21.550398198, 2024-09-14T23:57:01.058 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0091967720000000006, 2024-09-14T23:57:01.058 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:01.058 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:01.058 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:01.058 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:01.058 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:01.058 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:01.058 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 6287 2024-09-14T23:57:01.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.058 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:01.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:01.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492747+0000", 2024-09-14T23:57:01.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:01.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492747+0000", 2024-09-14T23:57:01.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:01.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492748+0000", 2024-09-14T23:57:01.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3399999999999998e-07 2024-09-14T23:57:01.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:01.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492753+0000", 2024-09-14T23:57:01.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6829999999999999e-06 2024-09-14T23:57:01.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:01.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492760+0000", 2024-09-14T23:57:01.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8930000000000005e-06 2024-09-14T23:57:01.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:01.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492765+0000", 2024-09-14T23:57:01.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5569999999999997e-06 2024-09-14T23:57:01.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.061 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:01.061 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.499828+0000", 2024-09-14T23:57:01.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0070631440000000004 2024-09-14T23:57:01.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.061 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:01.061 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.499878+0000", 2024-09-14T23:57:01.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0117999999999997e-05 2024-09-14T23:57:01.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.061 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 4", 2024-09-14T23:57:01.062 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.500157+0000", 2024-09-14T23:57:01.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00027882699999999999 2024-09-14T23:57:01.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.062 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:01.062 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.501864+0000", 2024-09-14T23:57:01.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001706656 2024-09-14T23:57:01.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.062 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:57:01.062 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.501879+0000", 2024-09-14T23:57:01.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5502000000000002e-05 2024-09-14T23:57:01.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:01.063 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.501920+0000", 2024-09-14T23:57:01.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0482000000000003e-05 2024-09-14T23:57:01.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:01.063 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.501944+0000", 2024-09-14T23:57:01.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4076000000000001e-05 2024-09-14T23:57:01.063 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.063 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.063 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.064 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:6288 3.1 3:8172dac6:::rbd_object_map.1118de1168b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:01.064 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.492849+0000", 2024-09-14T23:57:01.064 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 21.550295901999998, 2024-09-14T23:57:01.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0098451100000000007, 2024-09-14T23:57:01.064 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:01.064 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:01.064 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:01.064 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:01.064 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:01.064 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:01.064 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 6288 2024-09-14T23:57:01.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.064 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:01.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.064 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:01.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492849+0000", 2024-09-14T23:57:01.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:01.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492849+0000", 2024-09-14T23:57:01.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:01.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492850+0000", 2024-09-14T23:57:01.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5200000000000002e-07 2024-09-14T23:57:01.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:01.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492853+0000", 2024-09-14T23:57:01.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0189999999999998e-06 2024-09-14T23:57:01.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:01.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492859+0000", 2024-09-14T23:57:01.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0850000000000002e-06 2024-09-14T23:57:01.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:01.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492862+0000", 2024-09-14T23:57:01.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5599999999999998e-06 2024-09-14T23:57:01.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:01.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.500719+0000", 2024-09-14T23:57:01.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0078569499999999997 2024-09-14T23:57:01.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:01.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.500798+0000", 2024-09-14T23:57:01.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9012999999999997e-05 2024-09-14T23:57:01.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 4", 2024-09-14T23:57:01.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.501194+0000", 2024-09-14T23:57:01.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00039602099999999998 2024-09-14T23:57:01.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:57:01.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.501956+0000", 2024-09-14T23:57:01.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00076112900000000004 2024-09-14T23:57:01.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:01.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.502639+0000", 2024-09-14T23:57:01.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00068322800000000003 2024-09-14T23:57:01.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:01.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.502671+0000", 2024-09-14T23:57:01.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1798000000000003e-05 2024-09-14T23:57:01.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:01.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.502694+0000", 2024-09-14T23:57:01.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3855000000000001e-05 2024-09-14T23:57:01.069 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.070 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.070 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.070 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:6289 3.1 3:8172dac6:::rbd_object_map.1118de1168b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:01.070 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.492884+0000", 2024-09-14T23:57:01.070 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 21.550261052, 2024-09-14T23:57:01.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010525079, 2024-09-14T23:57:01.070 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:01.070 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:01.070 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:01.070 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:01.071 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:01.071 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:01.071 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 6289 2024-09-14T23:57:01.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.071 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:01.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:01.071 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492884+0000", 2024-09-14T23:57:01.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:01.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492884+0000", 2024-09-14T23:57:01.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:01.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492885+0000", 2024-09-14T23:57:01.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1699999999999999e-07 2024-09-14T23:57:01.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:01.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492887+0000", 2024-09-14T23:57:01.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.193e-06 2024-09-14T23:57:01.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:01.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492897+0000", 2024-09-14T23:57:01.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0703000000000001e-05 2024-09-14T23:57:01.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:01.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.492902+0000", 2024-09-14T23:57:01.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4360000000000004e-06 2024-09-14T23:57:01.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:01.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.501981+0000", 2024-09-14T23:57:01.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0090787290000000007 2024-09-14T23:57:01.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:01.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.502045+0000", 2024-09-14T23:57:01.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4787000000000002e-05 2024-09-14T23:57:01.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 4", 2024-09-14T23:57:01.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.502280+0000", 2024-09-14T23:57:01.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000234663 2024-09-14T23:57:01.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:57:01.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.502927+0000", 2024-09-14T23:57:01.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00064658499999999998 2024-09-14T23:57:01.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:01.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.503332+0000", 2024-09-14T23:57:01.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000405139 2024-09-14T23:57:01.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:01.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.503361+0000", 2024-09-14T23:57:01.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9626999999999999e-05 2024-09-14T23:57:01.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:01.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.503409+0000", 2024-09-14T23:57:01.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7899999999999999e-05 2024-09-14T23:57:01.076 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.076 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.076 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.076 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:16573 3.1 3:862ed24f:::rbd_data.1118987b60ce.0000000000000003:5 [read 2072576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T23:57:01.076 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:42.519802+0000", 2024-09-14T23:57:01.076 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.52334312, 2024-09-14T23:57:01.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0085017849999999996, 2024-09-14T23:57:01.076 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:01.076 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:01.076 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:01.076 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:01.076 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:01.076 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:01.077 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 16573 2024-09-14T23:57:01.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.077 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:01.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:01.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519802+0000", 2024-09-14T23:57:01.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:01.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519802+0000", 2024-09-14T23:57:01.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:01.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519803+0000", 2024-09-14T23:57:01.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9599999999999999e-07 2024-09-14T23:57:01.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:01.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519805+0000", 2024-09-14T23:57:01.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5840000000000002e-06 2024-09-14T23:57:01.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:01.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519812+0000", 2024-09-14T23:57:01.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4420000000000003e-06 2024-09-14T23:57:01.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:01.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519817+0000", 2024-09-14T23:57:01.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2989999999999996e-06 2024-09-14T23:57:01.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:01.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.527021+0000", 2024-09-14T23:57:01.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0072037669999999998 2024-09-14T23:57:01.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:01.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.527105+0000", 2024-09-14T23:57:01.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4498000000000003e-05 2024-09-14T23:57:01.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:01.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.528304+0000", 2024-09-14T23:57:01.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011986989999999999 2024-09-14T23:57:01.080 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.080 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.080 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.080 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:16585 3.1 3:862ed24f:::rbd_data.1118987b60ce.0000000000000003:5 [read 1015808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T23:57:01.080 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:42.519907+0000", 2024-09-14T23:57:01.081 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.523238676999998, 2024-09-14T23:57:01.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0083133819999999994, 2024-09-14T23:57:01.081 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:01.081 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:01.081 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:01.081 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:01.081 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:01.081 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:01.081 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 16585 2024-09-14T23:57:01.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.081 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:01.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:01.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519907+0000", 2024-09-14T23:57:01.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:01.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519907+0000", 2024-09-14T23:57:01.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:01.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519907+0000", 2024-09-14T23:57:01.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0999999999999999e-07 2024-09-14T23:57:01.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:01.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519910+0000", 2024-09-14T23:57:01.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6369999999999999e-06 2024-09-14T23:57:01.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:01.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519917+0000", 2024-09-14T23:57:01.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8329999999999996e-06 2024-09-14T23:57:01.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:01.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519922+0000", 2024-09-14T23:57:01.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1909999999999999e-06 2024-09-14T23:57:01.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:01.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.527996+0000", 2024-09-14T23:57:01.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0080740629999999994 2024-09-14T23:57:01.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:01.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.528124+0000", 2024-09-14T23:57:01.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012850200000000001 2024-09-14T23:57:01.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:01.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.528220+0000", 2024-09-14T23:57:01.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5645999999999999e-05 2024-09-14T23:57:01.085 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.085 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.085 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.085 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:16588 3.1 3:862ed24f:::rbd_data.1118987b60ce.0000000000000003:5 [read 1990656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T23:57:01.085 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:42.519930+0000", 2024-09-14T23:57:01.085 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.523215569000001, 2024-09-14T23:57:01.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0085309359999999994, 2024-09-14T23:57:01.085 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:01.086 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:01.086 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:01.086 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:01.086 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:01.086 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:01.086 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 16588 2024-09-14T23:57:01.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.086 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:01.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:01.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519930+0000", 2024-09-14T23:57:01.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:01.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519930+0000", 2024-09-14T23:57:01.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:01.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519930+0000", 2024-09-14T23:57:01.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3000000000000001e-07 2024-09-14T23:57:01.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:01.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519933+0000", 2024-09-14T23:57:01.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4049999999999998e-06 2024-09-14T23:57:01.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:01.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519942+0000", 2024-09-14T23:57:01.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6129999999999993e-06 2024-09-14T23:57:01.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:01.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519947+0000", 2024-09-14T23:57:01.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3260000000000003e-06 2024-09-14T23:57:01.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:01.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.528245+0000", 2024-09-14T23:57:01.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0082979950000000007 2024-09-14T23:57:01.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:01.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.528328+0000", 2024-09-14T23:57:01.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2195999999999997e-05 2024-09-14T23:57:01.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:01.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.528461+0000", 2024-09-14T23:57:01.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013297100000000001 2024-09-14T23:57:01.090 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.090 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.090 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.090 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:16590 3.1 3:862ed24f:::rbd_data.1118987b60ce.0000000000000003:5 [read 2080768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T23:57:01.090 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:42.519956+0000", 2024-09-14T23:57:01.091 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.523189326000001, 2024-09-14T23:57:01.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0087515460000000007, 2024-09-14T23:57:01.091 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:01.091 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:01.091 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:01.091 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:01.091 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:01.091 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:01.091 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 16590 2024-09-14T23:57:01.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.092 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:01.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:01.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519956+0000", 2024-09-14T23:57:01.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:01.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519956+0000", 2024-09-14T23:57:01.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:01.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519956+0000", 2024-09-14T23:57:01.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7599999999999997e-07 2024-09-14T23:57:01.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:01.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519959+0000", 2024-09-14T23:57:01.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3920000000000001e-06 2024-09-14T23:57:01.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:01.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519965+0000", 2024-09-14T23:57:01.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3230000000000003e-06 2024-09-14T23:57:01.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:01.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.519970+0000", 2024-09-14T23:57:01.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0259999999999998e-06 2024-09-14T23:57:01.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:01.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.528507+0000", 2024-09-14T23:57:01.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0085364159999999998 2024-09-14T23:57:01.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:01.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.528601+0000", 2024-09-14T23:57:01.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3988999999999997e-05 2024-09-14T23:57:01.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:01.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.528707+0000", 2024-09-14T23:57:01.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010692399999999999 2024-09-14T23:57:01.095 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.095 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.096 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.096 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:16595 3.1 3:862ed24f:::rbd_data.1118987b60ce.0000000000000003:5 [read 3055616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T23:57:01.096 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:42.520231+0000", 2024-09-14T23:57:01.096 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.522914137000001, 2024-09-14T23:57:01.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0086842910000000002, 2024-09-14T23:57:01.096 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:01.096 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:01.096 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:01.096 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:01.097 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:01.097 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:01.097 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 16595 2024-09-14T23:57:01.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.097 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:01.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:01.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520231+0000", 2024-09-14T23:57:01.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:01.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520231+0000", 2024-09-14T23:57:01.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:01.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520232+0000", 2024-09-14T23:57:01.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3699999999999999e-07 2024-09-14T23:57:01.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:01.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520236+0000", 2024-09-14T23:57:01.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1320000000000004e-06 2024-09-14T23:57:01.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:01.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520247+0000", 2024-09-14T23:57:01.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1306e-05 2024-09-14T23:57:01.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:01.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520255+0000", 2024-09-14T23:57:01.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1110000000000002e-06 2024-09-14T23:57:01.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:01.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.528742+0000", 2024-09-14T23:57:01.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0084870009999999992 2024-09-14T23:57:01.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:01.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.528830+0000", 2024-09-14T23:57:01.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8226000000000005e-05 2024-09-14T23:57:01.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:01.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.528915+0000", 2024-09-14T23:57:01.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5578e-05 2024-09-14T23:57:01.100 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.100 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.100 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.101 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:16600 3.1 3:862ed24f:::rbd_data.1118987b60ce.0000000000000003:5 [read 950272~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T23:57:01.101 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:42.520281+0000", 2024-09-14T23:57:01.101 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.522863911999998, 2024-09-14T23:57:01.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0088528040000000006, 2024-09-14T23:57:01.101 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:01.101 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:01.101 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:01.101 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:01.101 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:01.101 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:01.101 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 16600 2024-09-14T23:57:01.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.102 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:01.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:01.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520281+0000", 2024-09-14T23:57:01.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:01.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520281+0000", 2024-09-14T23:57:01.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:01.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520282+0000", 2024-09-14T23:57:01.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1999999999999999e-07 2024-09-14T23:57:01.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:01.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520285+0000", 2024-09-14T23:57:01.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.162e-06 2024-09-14T23:57:01.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:01.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520295+0000", 2024-09-14T23:57:01.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5510000000000005e-06 2024-09-14T23:57:01.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:01.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520301+0000", 2024-09-14T23:57:01.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0290000000000002e-06 2024-09-14T23:57:01.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:01.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.528947+0000", 2024-09-14T23:57:01.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.008646763 2024-09-14T23:57:01.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:01.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.529031+0000", 2024-09-14T23:57:01.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3669000000000001e-05 2024-09-14T23:57:01.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:01.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.529134+0000", 2024-09-14T23:57:01.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010301 2024-09-14T23:57:01.105 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.105 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.105 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.106 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:16602 3.1 3:862ed24f:::rbd_data.1118987b60ce.0000000000000003:5 [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T23:57:01.106 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:42.520352+0000", 2024-09-14T23:57:01.106 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.522793409999998, 2024-09-14T23:57:01.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0090844380000000002, 2024-09-14T23:57:01.106 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:01.106 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:01.106 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:01.106 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:01.106 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:01.106 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:01.106 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 16602 2024-09-14T23:57:01.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.106 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:01.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:01.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520352+0000", 2024-09-14T23:57:01.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:01.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520352+0000", 2024-09-14T23:57:01.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:01.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520353+0000", 2024-09-14T23:57:01.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3099999999999997e-07 2024-09-14T23:57:01.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:01.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520356+0000", 2024-09-14T23:57:01.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6950000000000002e-06 2024-09-14T23:57:01.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:01.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520368+0000", 2024-09-14T23:57:01.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1701e-05 2024-09-14T23:57:01.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:01.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520375+0000", 2024-09-14T23:57:01.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5459999999999997e-06 2024-09-14T23:57:01.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:01.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.529212+0000", 2024-09-14T23:57:01.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0088376240000000005 2024-09-14T23:57:01.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:01.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.529297+0000", 2024-09-14T23:57:01.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5222000000000005e-05 2024-09-14T23:57:01.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:01.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.529436+0000", 2024-09-14T23:57:01.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013891899999999999 2024-09-14T23:57:01.110 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.110 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.110 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.110 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:16607 3.1 3:862ed24f:::rbd_data.1118987b60ce.0000000000000003:5 [read 2015232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T23:57:01.110 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:42.520699+0000", 2024-09-14T23:57:01.110 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.522445960999999, 2024-09-14T23:57:01.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0089861669999999998, 2024-09-14T23:57:01.110 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:01.110 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:01.110 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:01.110 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:01.110 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:01.110 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:01.110 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 16607 2024-09-14T23:57:01.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.111 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:01.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:01.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520699+0000", 2024-09-14T23:57:01.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:01.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520699+0000", 2024-09-14T23:57:01.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:01.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520700+0000", 2024-09-14T23:57:01.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4099999999999998e-07 2024-09-14T23:57:01.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:01.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520704+0000", 2024-09-14T23:57:01.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.369e-06 2024-09-14T23:57:01.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:01.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520715+0000", 2024-09-14T23:57:01.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1165000000000001e-05 2024-09-14T23:57:01.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:01.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.520723+0000", 2024-09-14T23:57:01.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3860000000000001e-06 2024-09-14T23:57:01.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:01.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.529468+0000", 2024-09-14T23:57:01.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0087455539999999991 2024-09-14T23:57:01.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:01.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.529551+0000", 2024-09-14T23:57:01.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2671999999999997e-05 2024-09-14T23:57:01.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:01.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.529685+0000", 2024-09-14T23:57:01.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013438 2024-09-14T23:57:01.114 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.114 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.114 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.114 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:16612 3.1 3:862ed24f:::rbd_data.1118987b60ce.0000000000000003:5 [read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T23:57:01.114 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:42.521352+0000", 2024-09-14T23:57:01.114 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.521792897000001, 2024-09-14T23:57:01.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0085567620000000007, 2024-09-14T23:57:01.114 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:01.114 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:01.114 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:01.114 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:01.115 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:01.115 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:01.115 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 16612 2024-09-14T23:57:01.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.115 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:01.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:01.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.521352+0000", 2024-09-14T23:57:01.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:01.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.521352+0000", 2024-09-14T23:57:01.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:01.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.521353+0000", 2024-09-14T23:57:01.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9100000000000002e-07 2024-09-14T23:57:01.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:01.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.521358+0000", 2024-09-14T23:57:01.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4549999999999997e-06 2024-09-14T23:57:01.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:01.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.521369+0000", 2024-09-14T23:57:01.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1601e-05 2024-09-14T23:57:01.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:01.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.521377+0000", 2024-09-14T23:57:01.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0520000000000002e-06 2024-09-14T23:57:01.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:01.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.529728+0000", 2024-09-14T23:57:01.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0083509770000000007 2024-09-14T23:57:01.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:01.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.529818+0000", 2024-09-14T23:57:01.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9629999999999994e-05 2024-09-14T23:57:01.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:01.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.529909+0000", 2024-09-14T23:57:01.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1156000000000006e-05 2024-09-14T23:57:01.118 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.118 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.118 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.118 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:18289 3.1f 3:f852abec:::rbd_data.11187be34cc8.0000000000000007:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-14T23:57:01.118 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:44.136538+0000", 2024-09-14T23:57:01.119 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 16.906607102999999, 2024-09-14T23:57:01.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.015276301000000001, 2024-09-14T23:57:01.119 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:01.119 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:01.119 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:01.119 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:01.119 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:01.119 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:01.119 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 18289 2024-09-14T23:57:01.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.119 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:01.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:01.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.136538+0000", 2024-09-14T23:57:01.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:01.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.136538+0000", 2024-09-14T23:57:01.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:01.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.136539+0000", 2024-09-14T23:57:01.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2200000000000003e-07 2024-09-14T23:57:01.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:01.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.136543+0000", 2024-09-14T23:57:01.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9689999999999996e-06 2024-09-14T23:57:01.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:01.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.136552+0000", 2024-09-14T23:57:01.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7919999999999998e-06 2024-09-14T23:57:01.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:01.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.136556+0000", 2024-09-14T23:57:01.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5210000000000004e-06 2024-09-14T23:57:01.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:01.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.136606+0000", 2024-09-14T23:57:01.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0231000000000001e-05 2024-09-14T23:57:01.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:01.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.136682+0000", 2024-09-14T23:57:01.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5576999999999998e-05 2024-09-14T23:57:01.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-14T23:57:01.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.139652+0000", 2024-09-14T23:57:01.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0029698709999999998 2024-09-14T23:57:01.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:01.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.151725+0000", 2024-09-14T23:57:01.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012073607 2024-09-14T23:57:01.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:57:01.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.151739+0000", 2024-09-14T23:57:01.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3287e-05 2024-09-14T23:57:01.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:01.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.151770+0000", 2024-09-14T23:57:01.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0913e-05 2024-09-14T23:57:01.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:01.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.151814+0000", 2024-09-14T23:57:01.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4811000000000002e-05 2024-09-14T23:57:01.125 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.125 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.125 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.126 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:28434 3.6 3:617cff09:::rbd_data.111888d8b54.000000000000000b:a [read 4337664~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:57:01.126 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:48.493748+0000", 2024-09-14T23:57:01.126 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 12.549397224, 2024-09-14T23:57:01.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0082073379999999998, 2024-09-14T23:57:01.126 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:01.126 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:01.126 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:01.126 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:01.126 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:01.126 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:01.127 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 28434 2024-09-14T23:57:01.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.127 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:01.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:01.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.493748+0000", 2024-09-14T23:57:01.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:01.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.493748+0000", 2024-09-14T23:57:01.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:01.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.493749+0000", 2024-09-14T23:57:01.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7800000000000001e-07 2024-09-14T23:57:01.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:01.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.493752+0000", 2024-09-14T23:57:01.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2289999999999999e-06 2024-09-14T23:57:01.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:01.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.493767+0000", 2024-09-14T23:57:01.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5597e-05 2024-09-14T23:57:01.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:01.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.493782+0000", 2024-09-14T23:57:01.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4333e-05 2024-09-14T23:57:01.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:01.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.501821+0000", 2024-09-14T23:57:01.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.008038913 2024-09-14T23:57:01.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:01.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.501874+0000", 2024-09-14T23:57:01.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3601999999999999e-05 2024-09-14T23:57:01.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:01.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.501955+0000", 2024-09-14T23:57:01.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1086000000000003e-05 2024-09-14T23:57:01.131 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.131 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.131 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.131 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:28435 3.6 3:617cff09:::rbd_data.111888d8b54.000000000000000b:a [read 4403200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:57:01.131 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:48.493793+0000", 2024-09-14T23:57:01.131 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 12.549351786000001, 2024-09-14T23:57:01.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0083982390000000001, 2024-09-14T23:57:01.132 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:01.132 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:01.132 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:01.132 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:01.132 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:01.132 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:01.132 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 28435 2024-09-14T23:57:01.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.132 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:01.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:01.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.493793+0000", 2024-09-14T23:57:01.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:01.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.493793+0000", 2024-09-14T23:57:01.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:01.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.493794+0000", 2024-09-14T23:57:01.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2499999999999995e-07 2024-09-14T23:57:01.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:01.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.493797+0000", 2024-09-14T23:57:01.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8609999999999998e-06 2024-09-14T23:57:01.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:01.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.493804+0000", 2024-09-14T23:57:01.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2189999999999998e-06 2024-09-14T23:57:01.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:01.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.493810+0000", 2024-09-14T23:57:01.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.463e-06 2024-09-14T23:57:01.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:01.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.502025+0000", 2024-09-14T23:57:01.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.008215478 2024-09-14T23:57:01.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:01.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.502104+0000", 2024-09-14T23:57:01.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9107e-05 2024-09-14T23:57:01.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:01.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.502192+0000", 2024-09-14T23:57:01.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7585999999999998e-05 2024-09-14T23:57:01.136 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.136 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.136 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.136 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:28436 3.6 3:668d2474:::rbd_data.111814b482f8.00000000000000f0:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,truncate 45056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:57:01.136 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:48.494404+0000", 2024-09-14T23:57:01.136 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 12.548741408, 2024-09-14T23:57:01.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0089320140000000003, 2024-09-14T23:57:01.137 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:01.137 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:01.137 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:01.137 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:01.137 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:01.137 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:01.137 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 28436 2024-09-14T23:57:01.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.137 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:01.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:01.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.494404+0000", 2024-09-14T23:57:01.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:01.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.494404+0000", 2024-09-14T23:57:01.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:01.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.494405+0000", 2024-09-14T23:57:01.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1900000000000001e-07 2024-09-14T23:57:01.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:01.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.494410+0000", 2024-09-14T23:57:01.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1739999999999999e-06 2024-09-14T23:57:01.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:01.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.494420+0000", 2024-09-14T23:57:01.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7929999999999992e-06 2024-09-14T23:57:01.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:01.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.494427+0000", 2024-09-14T23:57:01.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1709999999999996e-06 2024-09-14T23:57:01.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:01.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.502220+0000", 2024-09-14T23:57:01.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0077929979999999998 2024-09-14T23:57:01.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:01.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.502342+0000", 2024-09-14T23:57:01.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012175700000000001 2024-09-14T23:57:01.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-14T23:57:01.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.502500+0000", 2024-09-14T23:57:01.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015829099999999999 2024-09-14T23:57:01.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:57:01.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.502976+0000", 2024-09-14T23:57:01.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00047596599999999998 2024-09-14T23:57:01.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:01.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.503282+0000", 2024-09-14T23:57:01.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030643400000000001 2024-09-14T23:57:01.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:01.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.503313+0000", 2024-09-14T23:57:01.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0994000000000002e-05 2024-09-14T23:57:01.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:01.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:48.503336+0000", 2024-09-14T23:57:01.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2517000000000001e-05 2024-09-14T23:57:01.142 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.142 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.142 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.142 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:43054 3.1f 3:ffbfe596:::rbd_data.1118350d5ae5.00000000000002eb:head [set-alloc-hint object_size 262144 write_size 262144,write 196608~22770] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T23:57:01.142 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:57:01.142 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.0765945600000002, 2024-09-14T23:57:01.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.99973334400000002, 2024-09-14T23:57:01.142 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:01.142 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:01.143 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:01.143 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:01.143 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:01.143 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:01.143 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 43054 2024-09-14T23:57:01.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.143 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:01.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:01.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:57:01.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:01.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:57:01.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:01.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966552+0000", 2024-09-14T23:57:01.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3790000000000001e-06 2024-09-14T23:57:01.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:01.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966565+0000", 2024-09-14T23:57:01.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2459999999999999e-05 2024-09-14T23:57:01.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:01.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966576+0000", 2024-09-14T23:57:01.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1039000000000001e-05 2024-09-14T23:57:01.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:01.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966582+0000", 2024-09-14T23:57:01.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9660000000000001e-06 2024-09-14T23:57:01.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:01.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966625+0000", 2024-09-14T23:57:01.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3529999999999998e-05 2024-09-14T23:57:01.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:57:01.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966632+0000", 2024-09-14T23:57:01.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9809999999999997e-06 2024-09-14T23:57:01.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:01.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.964460+0000", 2024-09-14T23:57:01.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.99782768300000002 2024-09-14T23:57:01.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:01.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.964641+0000", 2024-09-14T23:57:01.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000181089 2024-09-14T23:57:01.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-14T23:57:01.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.964809+0000", 2024-09-14T23:57:01.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000168505 2024-09-14T23:57:01.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:57:01.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966030+0000", 2024-09-14T23:57:01.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012206929999999999 2024-09-14T23:57:01.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:01.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966181+0000", 2024-09-14T23:57:01.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000150569 2024-09-14T23:57:01.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:01.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966243+0000", 2024-09-14T23:57:01.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2331999999999999e-05 2024-09-14T23:57:01.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:01.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966284+0000", 2024-09-14T23:57:01.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1118000000000002e-05 2024-09-14T23:57:01.149 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.149 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.149 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.149 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:46942 3.1f 3:fbe169d3:::rbd_header.11188bb0c80a:head [watch reconnect cookie 140670715433840 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-14T23:57:01.149 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:58.450175+0000", 2024-09-14T23:57:01.149 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 2.592970115, 2024-09-14T23:57:01.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.012166938, 2024-09-14T23:57:01.150 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:01.150 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:01.150 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:01.150 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:01.150 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:01.150 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:01.150 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 46942 2024-09-14T23:57:01.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.150 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:01.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:01.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:58.450175+0000", 2024-09-14T23:57:01.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:01.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:58.450175+0000", 2024-09-14T23:57:01.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:01.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:58.450176+0000", 2024-09-14T23:57:01.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4799999999999999e-07 2024-09-14T23:57:01.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.152 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:01.152 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:58.450178+0000", 2024-09-14T23:57:01.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.103e-06 2024-09-14T23:57:01.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.152 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:01.152 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:58.450184+0000", 2024-09-14T23:57:01.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6950000000000003e-06 2024-09-14T23:57:01.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:01.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:58.450189+0000", 2024-09-14T23:57:01.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0950000000000003e-06 2024-09-14T23:57:01.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:01.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:58.451663+0000", 2024-09-14T23:57:01.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001473607 2024-09-14T23:57:01.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:01.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:58.451757+0000", 2024-09-14T23:57:01.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3443999999999995e-05 2024-09-14T23:57:01.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:01.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:58.462342+0000", 2024-09-14T23:57:01.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010585545999999999 2024-09-14T23:57:01.154 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.154 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.154 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.154 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.154 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.154 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:01.155 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T23:57:01.155 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T23:57:01.155 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-14T23:57:01.155 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T23:57:01.157 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T23:57:01.158 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:57:01.158 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:57:01.159 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is unset 2024-09-14T23:57:01.159 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:57:01.159 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:57:01.160 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:57:01.160 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:57:01.160 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:57:01.161 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:57:01.161 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone12 from 0x3ec8ac1 (overlap 0x8840bc) to 0x0 2024-09-14T23:57:01.162 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:01.155+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:01.187 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-09-14T23:57:01.221 INFO:teuthology.orchestra.run.smithi042.stdout:1231 write 0xa06b1db thru 0xa06ec08 (0x3a2e bytes) 2024-09-14T23:57:01.224 INFO:teuthology.orchestra.run.smithi042.stdout:1232 read 0xff1045 thru 0xffc442 (0xb3fe bytes) 2024-09-14T23:57:01.224 INFO:teuthology.orchestra.run.smithi042.stdout:1233 read 0x3c3d7e1 thru 0x3c4c354 (0xeb74 bytes) 2024-09-14T23:57:01.232 INFO:teuthology.orchestra.run.smithi042.stdout:1234 write 0xcce6b2d thru 0xccebb84 (0x5058 bytes) 2024-09-14T23:57:01.236 INFO:teuthology.orchestra.run.smithi042.stdout:1235 writesame 0x96a5305 thru 0x96b0516 data_size 0x17(0xb212 bytes) 2024-09-14T23:57:01.239 INFO:teuthology.orchestra.run.smithi042.stdout:1236 writesame 0x615fa3 thru 0x623275 data_size 0x1(0xd2d3 bytes) 2024-09-14T23:57:01.257 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:01.256+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:01.272 INFO:teuthology.orchestra.run.smithi042.stdout:1237 read 0x9583154 thru 0x958e412 (0xb2bf bytes) 2024-09-14T23:57:01.272 INFO:teuthology.orchestra.run.smithi042.stdout:1238 write 0xa2ce23c thru 0xa2d79ef (0x97b4 bytes) 2024-09-14T23:57:01.276 INFO:teuthology.orchestra.run.smithi042.stdout:1239 read 0x40d5d8c thru 0x40ddff8 (0x826d bytes) 2024-09-14T23:57:01.276 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:01.276 INFO:teuthology.orchestra.run.smithi042.stdout:1241 write 0xaf101ec thru 0xaf11ce9 (0x1afe bytes) 2024-09-14T23:57:01.278 INFO:teuthology.orchestra.run.smithi042.stdout:1242 write 0x931b508 thru 0x931f663 (0x415c bytes) 2024-09-14T23:57:01.281 INFO:teuthology.orchestra.run.smithi042.stdout:1243 punch from 0x32af236 to 0x32bd28c, (0xe056 bytes) 2024-09-14T23:57:01.291 INFO:teuthology.orchestra.run.smithi042.stdout:1244 trunc from 0xccebb85 to 0x3f63ffd 2024-09-14T23:57:01.318 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:01.319 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:57:01.319 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:57:01.319 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:01.322 INFO:teuthology.orchestra.run.smithi042.stdout:1245 write 0x5631b9e thru 0x5633cf0 (0x2153 bytes) 2024-09-14T23:57:01.329 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T23:57:01.331 INFO:teuthology.orchestra.run.smithi042.stdout:1246 trunc from 0x5633cf1 to 0x57db5f 2024-09-14T23:57:01.358 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:01.357+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:01.376 INFO:teuthology.orchestra.run.smithi042.stdout:1249 write 0xe9d06de thru 0xe9d677c (0x609f bytes) 2024-09-14T23:57:01.378 INFO:teuthology.orchestra.run.smithi042.stdout:1253 writesame 0xd7e1bc4 thru 0xd7e7022 data_size 0x1(0x545f bytes) 2024-09-14T23:57:01.393 INFO:teuthology.orchestra.run.smithi042.stdout:1254 trunc from 0xe9d677d to 0xec17744 2024-09-14T23:57:01.398 INFO:teuthology.orchestra.run.smithi042.stdout:1255 write 0x3939717 thru 0x3947fa0 (0xe88a bytes) 2024-09-14T23:57:01.402 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:01.402 INFO:teuthology.orchestra.run.smithi042.stdout:1258 writesame 0x2e9a4e8 thru 0x2ea3f55 data_size 0x21(0x9a6e bytes) 2024-09-14T23:57:01.406 INFO:teuthology.orchestra.run.smithi042.stdout:1259 punch from 0xb6ed900 to 0xb6f64aa, (0x8baa bytes) 2024-09-14T23:57:01.410 INFO:teuthology.orchestra.run.smithi042.stdout:1260 write 0xecff5b3 thru 0xed02498 (0x2ee6 bytes) 2024-09-14T23:57:01.413 INFO:teuthology.orchestra.run.smithi042.stdout:1261 writesame 0xd2fb265 thru 0xd2ff5e2 data_size 0x35(0x437e bytes) 2024-09-14T23:57:01.416 INFO:teuthology.orchestra.run.smithi042.stdout:1262 writesame 0xcb5097c thru 0xcb5ce5b data_size 0xd2(0xc4e0 bytes) 2024-09-14T23:57:01.420 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:01.420 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:01.420 INFO:teuthology.orchestra.run.smithi042.stdout:1265 write 0x3b075c6 thru 0x3b14f5c (0xd997 bytes) 2024-09-14T23:57:01.423 INFO:teuthology.orchestra.run.smithi042.stdout:1266 read 0x354e1b1 thru 0x355296d (0x47bd bytes) 2024-09-14T23:57:01.423 INFO:teuthology.orchestra.run.smithi042.stdout:1267 writesame 0x3b3ea30 thru 0x3b42bec data_size 0x1(0x41bd bytes) 2024-09-14T23:57:01.430 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:01.434 INFO:teuthology.orchestra.run.smithi042.stdout:1271 trunc from 0xed02499 to 0xadd5b75 2024-09-14T23:57:01.445 INFO:teuthology.orchestra.run.smithi042.stdout:1272 writesame 0x6c1a995 thru 0x6c1d297 data_size 0x1(0x2903 bytes) 2024-09-14T23:57:01.452 INFO:teuthology.orchestra.run.smithi042.stdout:1273 punch from 0xa941e6 to 0xa9c02b, (0x7e45 bytes) 2024-09-14T23:57:01.452 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:01.452 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:01.452 INFO:teuthology.orchestra.run.smithi042.stdout:1276 clone 14 order 21 su 1048576 sc 8 2024-09-14T23:57:01.453 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:01.454 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:57:01.454 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:57:01.454 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:57:01.454 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:01.459 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:01.458+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:01.463 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T23:57:01.513 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:57:01.514 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:01.514 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 51, 2024-09-14T23:57:01.514 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "21916fcc-f95c-4bfa-8b8b-11ca45845613", 2024-09-14T23:57:01.514 INFO:teuthology.orchestra.run.smithi042.stdout: "created": "2024-09-14T23:56:03.167948+0000", 2024-09-14T23:57:01.514 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-09-14T23:57:01.056942+0000", 2024-09-14T23:57:01.514 INFO:teuthology.orchestra.run.smithi042.stdout: "last_up_change": "2024-09-14T23:56:14.322882+0000", 2024-09-14T23:57:01.514 INFO:teuthology.orchestra.run.smithi042.stdout: "last_in_change": "2024-09-14T23:56:56.035972+0000", 2024-09-14T23:57:01.514 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T23:57:01.514 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_num": 5804032, 2024-09-14T23:57:01.514 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_set": [ 2024-09-14T23:57:01.514 INFO:teuthology.orchestra.run.smithi042.stdout: "nodeep-scrub", 2024-09-14T23:57:01.515 INFO:teuthology.orchestra.run.smithi042.stdout: "pglog_hardlimit", 2024-09-14T23:57:01.515 INFO:teuthology.orchestra.run.smithi042.stdout: "purged_snapdirs", 2024-09-14T23:57:01.515 INFO:teuthology.orchestra.run.smithi042.stdout: "recovery_deletes", 2024-09-14T23:57:01.515 INFO:teuthology.orchestra.run.smithi042.stdout: "sortbitwise" 2024-09-14T23:57:01.515 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-14T23:57:01.515 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_version": 6, 2024-09-14T23:57:01.515 INFO:teuthology.orchestra.run.smithi042.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T23:57:01.515 INFO:teuthology.orchestra.run.smithi042.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T23:57:01.515 INFO:teuthology.orchestra.run.smithi042.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T23:57:01.515 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_snapshot": "", 2024-09-14T23:57:01.515 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_max": 3, 2024-09-14T23:57:01.515 INFO:teuthology.orchestra.run.smithi042.stdout: "max_osd": 8, 2024-09-14T23:57:01.515 INFO:teuthology.orchestra.run.smithi042.stdout: "require_min_compat_client": "luminous", 2024-09-14T23:57:01.515 INFO:teuthology.orchestra.run.smithi042.stdout: "min_compat_client": "jewel", 2024-09-14T23:57:01.516 INFO:teuthology.orchestra.run.smithi042.stdout: "require_osd_release": "squid", 2024-09-14T23:57:01.516 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_crimson": false, 2024-09-14T23:57:01.516 INFO:teuthology.orchestra.run.smithi042.stdout: "pools": [ 2024-09-14T23:57:01.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.516 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 1, 2024-09-14T23:57:01.516 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": ".mgr", 2024-09-14T23:57:01.516 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-14T23:56:13.500197+0000", 2024-09-14T23:57:01.516 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-09-14T23:57:01.516 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool", 2024-09-14T23:57:01.516 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-14T23:57:01.516 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-14T23:57:01.516 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-14T23:57:01.516 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-14T23:57:01.517 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-14T23:57:01.517 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-14T23:57:01.517 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T23:57:01.517 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T23:57:01.517 INFO:teuthology.orchestra.run.smithi042.stdout: "is_stretch_pool": false, 2024-09-14T23:57:01.517 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-14T23:57:01.517 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-14T23:57:01.517 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 1, 2024-09-14T23:57:01.517 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 1, 2024-09-14T23:57:01.517 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 1, 2024-09-14T23:57:01.517 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 1, 2024-09-14T23:57:01.517 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 1, 2024-09-14T23:57:01.517 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-14T23:57:01.517 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-14T23:57:01.518 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-14T23:57:01.518 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-14T23:57:01.518 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-14T23:57:01.518 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-14T23:57:01.518 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-14T23:57:01.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.518 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "17", 2024-09-14T23:57:01.518 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "0", 2024-09-14T23:57:01.518 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T23:57:01.518 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T23:57:01.518 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-14T23:57:01.518 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-14T23:57:01.518 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 0, 2024-09-14T23:57:01.518 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 0, 2024-09-14T23:57:01.519 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-14T23:57:01.519 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-14T23:57:01.519 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-14T23:57:01.519 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-14T23:57:01.519 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-14T23:57:01.519 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-14T23:57:01.519 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-14T23:57:01.519 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-14T23:57:01.519 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-14T23:57:01.519 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-14T23:57:01.519 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-14T23:57:01.519 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T23:57:01.519 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T23:57:01.520 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T23:57:01.520 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-14T23:57:01.520 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-14T23:57:01.520 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-14T23:57:01.520 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-14T23:57:01.520 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-14T23:57:01.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.520 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-14T23:57:01.520 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-14T23:57:01.520 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-14T23:57:01.520 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-14T23:57:01.520 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-14T23:57:01.520 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T23:57:01.520 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-14T23:57:01.521 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-14T23:57:01.521 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-14T23:57:01.521 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-14T23:57:01.521 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-14T23:57:01.521 INFO:teuthology.orchestra.run.smithi042.stdout: "options": { 2024-09-14T23:57:01.521 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_max": 32, 2024-09-14T23:57:01.521 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_min": 1 2024-09-14T23:57:01.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.521 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-14T23:57:01.521 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": {} 2024-09-14T23:57:01.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.521 INFO:teuthology.orchestra.run.smithi042.stdout: "read_balance": { 2024-09-14T23:57:01.521 INFO:teuthology.orchestra.run.smithi042.stdout: "score_type": "Fair distribution", 2024-09-14T23:57:01.521 INFO:teuthology.orchestra.run.smithi042.stdout: "score_acting": 6.9000000953674316, 2024-09-14T23:57:01.522 INFO:teuthology.orchestra.run.smithi042.stdout: "score_stable": 6.9000000953674316, 2024-09-14T23:57:01.522 INFO:teuthology.orchestra.run.smithi042.stdout: "optimal_score": 0.28999999165534973, 2024-09-14T23:57:01.522 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_acting": 2, 2024-09-14T23:57:01.522 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_stable": 2, 2024-09-14T23:57:01.522 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity_weighted": 0.8712921142578125, 2024-09-14T23:57:01.522 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity": 1, 2024-09-14T23:57:01.522 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity_weighted": 0.87000000476837158 2024-09-14T23:57:01.522 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.522 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 2, 2024-09-14T23:57:01.522 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": "rbd", 2024-09-14T23:57:01.522 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-14T23:56:21.284705+0000", 2024-09-14T23:57:01.522 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 8193, 2024-09-14T23:57:01.523 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T23:57:01.523 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-14T23:57:01.523 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-14T23:57:01.523 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-14T23:57:01.523 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-14T23:57:01.523 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-14T23:57:01.523 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-14T23:57:01.523 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T23:57:01.523 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T23:57:01.523 INFO:teuthology.orchestra.run.smithi042.stdout: "is_stretch_pool": false, 2024-09-14T23:57:01.523 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-14T23:57:01.523 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-14T23:57:01.524 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 8, 2024-09-14T23:57:01.524 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 8, 2024-09-14T23:57:01.524 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 8, 2024-09-14T23:57:01.524 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 8, 2024-09-14T23:57:01.524 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 8, 2024-09-14T23:57:01.524 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-14T23:57:01.524 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-14T23:57:01.524 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-14T23:57:01.524 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-14T23:57:01.524 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-14T23:57:01.524 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-14T23:57:01.524 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-14T23:57:01.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.524 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "21", 2024-09-14T23:57:01.525 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "0", 2024-09-14T23:57:01.525 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T23:57:01.525 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T23:57:01.525 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-14T23:57:01.525 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-14T23:57:01.525 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 2, 2024-09-14T23:57:01.525 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 21, 2024-09-14T23:57:01.525 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-14T23:57:01.525 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-14T23:57:01.525 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-14T23:57:01.525 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-14T23:57:01.525 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-14T23:57:01.525 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-14T23:57:01.525 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-14T23:57:01.526 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-14T23:57:01.526 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-14T23:57:01.526 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-14T23:57:01.526 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-14T23:57:01.526 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T23:57:01.526 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T23:57:01.526 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T23:57:01.526 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-14T23:57:01.526 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-14T23:57:01.526 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-14T23:57:01.526 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-14T23:57:01.526 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-14T23:57:01.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.526 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-14T23:57:01.526 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-14T23:57:01.527 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-14T23:57:01.527 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-14T23:57:01.527 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-14T23:57:01.527 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T23:57:01.527 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-14T23:57:01.527 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-14T23:57:01.527 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-14T23:57:01.527 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-14T23:57:01.527 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-14T23:57:01.527 INFO:teuthology.orchestra.run.smithi042.stdout: "options": {}, 2024-09-14T23:57:01.527 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-14T23:57:01.527 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd": {} 2024-09-14T23:57:01.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.527 INFO:teuthology.orchestra.run.smithi042.stdout: "read_balance": { 2024-09-14T23:57:01.527 INFO:teuthology.orchestra.run.smithi042.stdout: "score_type": "Fair distribution", 2024-09-14T23:57:01.528 INFO:teuthology.orchestra.run.smithi042.stdout: "score_acting": 2.630000114440918, 2024-09-14T23:57:01.528 INFO:teuthology.orchestra.run.smithi042.stdout: "score_stable": 2.630000114440918, 2024-09-14T23:57:01.528 INFO:teuthology.orchestra.run.smithi042.stdout: "optimal_score": 1, 2024-09-14T23:57:01.528 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_acting": 2.630000114440918, 2024-09-14T23:57:01.528 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_stable": 2.630000114440918, 2024-09-14T23:57:01.528 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity_weighted": 0.8712921142578125, 2024-09-14T23:57:01.528 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity": 1, 2024-09-14T23:57:01.528 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity_weighted": 0.87000000476837158 2024-09-14T23:57:01.528 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.528 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 3, 2024-09-14T23:57:01.528 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": "pool_client.0", 2024-09-14T23:57:01.528 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-14T23:56:30.861504+0000", 2024-09-14T23:57:01.529 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 8193, 2024-09-14T23:57:01.529 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T23:57:01.529 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-14T23:57:01.529 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-14T23:57:01.529 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-14T23:57:01.529 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-14T23:57:01.529 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-14T23:57:01.529 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-14T23:57:01.529 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T23:57:01.529 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T23:57:01.529 INFO:teuthology.orchestra.run.smithi042.stdout: "is_stretch_pool": false, 2024-09-14T23:57:01.529 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-14T23:57:01.529 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-14T23:57:01.529 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 32, 2024-09-14T23:57:01.529 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 32, 2024-09-14T23:57:01.530 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 32, 2024-09-14T23:57:01.530 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 32, 2024-09-14T23:57:01.530 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 32, 2024-09-14T23:57:01.530 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-14T23:57:01.530 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-14T23:57:01.530 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-14T23:57:01.530 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-14T23:57:01.530 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-14T23:57:01.530 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-14T23:57:01.530 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-14T23:57:01.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.530 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "51", 2024-09-14T23:57:01.530 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "0", 2024-09-14T23:57:01.530 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T23:57:01.531 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T23:57:01.531 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-14T23:57:01.531 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-14T23:57:01.531 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 15, 2024-09-14T23:57:01.531 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 51, 2024-09-14T23:57:01.531 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-14T23:57:01.531 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-14T23:57:01.531 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-14T23:57:01.531 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-14T23:57:01.531 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-14T23:57:01.531 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-14T23:57:01.531 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-14T23:57:01.531 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-14T23:57:01.531 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-14T23:57:01.531 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-14T23:57:01.532 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-14T23:57:01.532 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T23:57:01.532 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T23:57:01.532 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T23:57:01.532 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-14T23:57:01.532 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-14T23:57:01.532 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-14T23:57:01.532 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-14T23:57:01.532 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-14T23:57:01.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.532 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-14T23:57:01.532 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-14T23:57:01.532 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-14T23:57:01.533 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-14T23:57:01.533 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-14T23:57:01.533 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T23:57:01.533 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-14T23:57:01.533 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-14T23:57:01.533 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-14T23:57:01.533 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-14T23:57:01.533 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-14T23:57:01.533 INFO:teuthology.orchestra.run.smithi042.stdout: "options": {}, 2024-09-14T23:57:01.533 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-14T23:57:01.533 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd": {} 2024-09-14T23:57:01.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.534 INFO:teuthology.orchestra.run.smithi042.stdout: "read_balance": { 2024-09-14T23:57:01.534 INFO:teuthology.orchestra.run.smithi042.stdout: "score_type": "Fair distribution", 2024-09-14T23:57:01.534 INFO:teuthology.orchestra.run.smithi042.stdout: "score_acting": 1.75, 2024-09-14T23:57:01.534 INFO:teuthology.orchestra.run.smithi042.stdout: "score_stable": 1.75, 2024-09-14T23:57:01.534 INFO:teuthology.orchestra.run.smithi042.stdout: "optimal_score": 1.1399999856948853, 2024-09-14T23:57:01.534 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_acting": 2, 2024-09-14T23:57:01.534 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_stable": 2, 2024-09-14T23:57:01.534 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity_weighted": 0.8712921142578125, 2024-09-14T23:57:01.534 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity": 1, 2024-09-14T23:57:01.534 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity_weighted": 0.87000000476837158 2024-09-14T23:57:01.534 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.535 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.535 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-14T23:57:01.535 INFO:teuthology.orchestra.run.smithi042.stdout: "osds": [ 2024-09-14T23:57:01.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.535 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 0, 2024-09-14T23:57:01.535 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "db9d0c67-2d60-4592-9ad3-f99e5294f93f", 2024-09-14T23:57:01.535 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-14T23:57:01.535 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-14T23:57:01.535 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-14T23:57:01.535 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-14T23:57:01.535 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-14T23:57:01.535 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-14T23:57:01.535 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 14, 2024-09-14T23:57:01.536 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 49, 2024-09-14T23:57:01.536 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-14T23:57:01.536 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-14T23:57:01.536 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-14T23:57:01.536 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:01.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.536 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:01.536 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6816", 2024-09-14T23:57:01.536 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 314865173 2024-09-14T23:57:01.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.537 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:01.537 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6817", 2024-09-14T23:57:01.537 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 314865173 2024-09-14T23:57:01.537 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.537 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.537 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-14T23:57:01.537 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:01.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.537 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:01.537 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6818", 2024-09-14T23:57:01.537 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 314865173 2024-09-14T23:57:01.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.538 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:01.538 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6819", 2024-09-14T23:57:01.538 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 314865173 2024-09-14T23:57:01.538 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.538 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.538 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-14T23:57:01.538 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:01.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.538 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:01.538 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6822", 2024-09-14T23:57:01.538 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 314865173 2024-09-14T23:57:01.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.539 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:01.539 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6823", 2024-09-14T23:57:01.539 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 314865173 2024-09-14T23:57:01.539 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.539 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.539 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-14T23:57:01.539 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:01.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.539 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:01.539 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6820", 2024-09-14T23:57:01.539 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 314865173 2024-09-14T23:57:01.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.540 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:01.540 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6821", 2024-09-14T23:57:01.540 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 314865173 2024-09-14T23:57:01.540 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.540 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.540 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6817/314865173", 2024-09-14T23:57:01.540 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6819/314865173", 2024-09-14T23:57:01.540 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6823/314865173", 2024-09-14T23:57:01.540 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6821/314865173", 2024-09-14T23:57:01.540 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-14T23:57:01.541 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-14T23:57:01.541 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-14T23:57:01.541 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.541 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 1, 2024-09-14T23:57:01.541 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "6c1a2765-c591-4c57-8ab3-ee6645970aa5", 2024-09-14T23:57:01.541 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-14T23:57:01.541 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-14T23:57:01.541 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-14T23:57:01.541 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-14T23:57:01.541 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-14T23:57:01.541 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-14T23:57:01.542 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 14, 2024-09-14T23:57:01.542 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 46, 2024-09-14T23:57:01.542 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-14T23:57:01.542 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-14T23:57:01.542 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-14T23:57:01.542 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:01.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.542 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:01.542 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6800", 2024-09-14T23:57:01.542 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3197912310 2024-09-14T23:57:01.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.542 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:01.542 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6801", 2024-09-14T23:57:01.543 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3197912310 2024-09-14T23:57:01.543 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.543 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.543 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-14T23:57:01.543 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:01.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.543 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:01.543 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6802", 2024-09-14T23:57:01.543 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3197912310 2024-09-14T23:57:01.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.543 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:01.543 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6803", 2024-09-14T23:57:01.544 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3197912310 2024-09-14T23:57:01.544 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.544 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.544 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-14T23:57:01.544 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:01.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.544 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:01.544 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6806", 2024-09-14T23:57:01.544 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3197912310 2024-09-14T23:57:01.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.544 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:01.544 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6807", 2024-09-14T23:57:01.545 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3197912310 2024-09-14T23:57:01.545 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.545 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.545 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-14T23:57:01.545 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:01.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.545 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:01.545 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6804", 2024-09-14T23:57:01.545 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3197912310 2024-09-14T23:57:01.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.545 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:01.546 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6805", 2024-09-14T23:57:01.546 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3197912310 2024-09-14T23:57:01.546 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.546 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.546 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6801/3197912310", 2024-09-14T23:57:01.546 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6803/3197912310", 2024-09-14T23:57:01.546 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6807/3197912310", 2024-09-14T23:57:01.546 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6805/3197912310", 2024-09-14T23:57:01.546 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-14T23:57:01.546 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-14T23:57:01.546 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-14T23:57:01.546 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.547 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 2, 2024-09-14T23:57:01.547 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "930db893-7be6-4645-a7f1-e043c5ec95eb", 2024-09-14T23:57:01.547 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-14T23:57:01.547 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-14T23:57:01.547 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-14T23:57:01.547 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-14T23:57:01.547 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-14T23:57:01.547 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-14T23:57:01.547 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 13, 2024-09-14T23:57:01.547 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 46, 2024-09-14T23:57:01.547 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-14T23:57:01.547 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-14T23:57:01.547 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-14T23:57:01.548 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:01.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.548 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:01.548 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6808", 2024-09-14T23:57:01.548 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3241153660 2024-09-14T23:57:01.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.548 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:01.548 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6809", 2024-09-14T23:57:01.548 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3241153660 2024-09-14T23:57:01.548 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.548 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.548 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-14T23:57:01.549 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:01.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.549 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:01.549 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6810", 2024-09-14T23:57:01.549 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3241153660 2024-09-14T23:57:01.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.549 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:01.549 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6811", 2024-09-14T23:57:01.549 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3241153660 2024-09-14T23:57:01.549 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.549 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.549 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-14T23:57:01.550 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:01.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.550 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:01.550 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6814", 2024-09-14T23:57:01.550 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3241153660 2024-09-14T23:57:01.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.550 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:01.550 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6815", 2024-09-14T23:57:01.550 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3241153660 2024-09-14T23:57:01.550 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.550 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.551 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-14T23:57:01.551 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:01.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.551 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:01.551 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6812", 2024-09-14T23:57:01.551 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3241153660 2024-09-14T23:57:01.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.551 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:01.551 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6813", 2024-09-14T23:57:01.551 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3241153660 2024-09-14T23:57:01.551 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.551 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.552 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6809/3241153660", 2024-09-14T23:57:01.552 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6811/3241153660", 2024-09-14T23:57:01.552 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6815/3241153660", 2024-09-14T23:57:01.552 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6813/3241153660", 2024-09-14T23:57:01.552 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-14T23:57:01.552 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-14T23:57:01.552 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-14T23:57:01.552 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.552 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 3, 2024-09-14T23:57:01.552 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "45bb3b84-b1c4-4f87-a7d1-0eb77b81741b", 2024-09-14T23:57:01.552 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-14T23:57:01.552 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-14T23:57:01.553 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-14T23:57:01.553 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-14T23:57:01.553 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-14T23:57:01.553 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-14T23:57:01.553 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-14T23:57:01.553 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 49, 2024-09-14T23:57:01.553 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-14T23:57:01.553 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-14T23:57:01.553 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-14T23:57:01.553 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:01.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.553 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:01.553 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6824", 2024-09-14T23:57:01.553 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1736416326 2024-09-14T23:57:01.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.554 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:01.554 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6825", 2024-09-14T23:57:01.554 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1736416326 2024-09-14T23:57:01.554 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.554 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.554 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-14T23:57:01.554 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:01.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.554 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:01.554 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6826", 2024-09-14T23:57:01.555 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1736416326 2024-09-14T23:57:01.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.555 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:01.555 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6827", 2024-09-14T23:57:01.555 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1736416326 2024-09-14T23:57:01.555 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.555 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.555 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-14T23:57:01.555 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:01.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.555 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:01.555 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6830", 2024-09-14T23:57:01.556 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1736416326 2024-09-14T23:57:01.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.556 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:01.556 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6831", 2024-09-14T23:57:01.556 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1736416326 2024-09-14T23:57:01.556 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.556 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.556 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-14T23:57:01.556 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:01.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.556 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:01.556 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6828", 2024-09-14T23:57:01.557 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1736416326 2024-09-14T23:57:01.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.557 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:01.557 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6829", 2024-09-14T23:57:01.557 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1736416326 2024-09-14T23:57:01.557 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.557 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.557 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6825/1736416326", 2024-09-14T23:57:01.557 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6827/1736416326", 2024-09-14T23:57:01.557 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6831/1736416326", 2024-09-14T23:57:01.557 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6829/1736416326", 2024-09-14T23:57:01.558 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-14T23:57:01.558 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-14T23:57:01.558 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-14T23:57:01.558 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.558 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 4, 2024-09-14T23:57:01.558 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "a812e38f-c0f9-4d33-9373-1910fb09103c", 2024-09-14T23:57:01.558 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-14T23:57:01.558 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-14T23:57:01.558 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-14T23:57:01.559 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 0.9703369140625, 2024-09-14T23:57:01.559 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-14T23:57:01.559 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-14T23:57:01.559 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-14T23:57:01.559 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 49, 2024-09-14T23:57:01.559 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-14T23:57:01.559 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-14T23:57:01.559 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-14T23:57:01.559 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:01.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.559 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:01.559 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6808", 2024-09-14T23:57:01.559 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1755154427 2024-09-14T23:57:01.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.560 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:01.560 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6809", 2024-09-14T23:57:01.560 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1755154427 2024-09-14T23:57:01.560 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.560 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.560 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-14T23:57:01.560 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:01.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.560 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:01.560 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6810", 2024-09-14T23:57:01.560 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1755154427 2024-09-14T23:57:01.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.561 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:01.561 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6811", 2024-09-14T23:57:01.561 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1755154427 2024-09-14T23:57:01.561 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.561 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.561 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-14T23:57:01.561 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:01.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.561 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:01.561 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6814", 2024-09-14T23:57:01.561 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1755154427 2024-09-14T23:57:01.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.562 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:01.562 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6815", 2024-09-14T23:57:01.562 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1755154427 2024-09-14T23:57:01.562 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.562 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.562 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-14T23:57:01.562 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:01.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.562 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:01.562 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6812", 2024-09-14T23:57:01.562 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1755154427 2024-09-14T23:57:01.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.563 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:01.563 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6813", 2024-09-14T23:57:01.563 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1755154427 2024-09-14T23:57:01.563 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.563 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.563 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.43:6809/1755154427", 2024-09-14T23:57:01.563 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.43:6811/1755154427", 2024-09-14T23:57:01.563 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.43:6815/1755154427", 2024-09-14T23:57:01.563 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.43:6813/1755154427", 2024-09-14T23:57:01.563 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-14T23:57:01.564 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-14T23:57:01.564 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-14T23:57:01.564 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.564 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 5, 2024-09-14T23:57:01.564 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "0de87e5b-7a21-4fb4-9543-d96c065786f8", 2024-09-14T23:57:01.564 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-14T23:57:01.564 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-14T23:57:01.564 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-14T23:57:01.564 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 0, 2024-09-14T23:57:01.564 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-14T23:57:01.564 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-14T23:57:01.564 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-14T23:57:01.565 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 23, 2024-09-14T23:57:01.565 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-14T23:57:01.565 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-14T23:57:01.565 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-14T23:57:01.565 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:01.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.565 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:01.565 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6816", 2024-09-14T23:57:01.565 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3033769028 2024-09-14T23:57:01.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.565 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:01.565 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6817", 2024-09-14T23:57:01.565 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3033769028 2024-09-14T23:57:01.566 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.566 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.566 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-14T23:57:01.566 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:01.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.566 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:01.566 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6818", 2024-09-14T23:57:01.566 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3033769028 2024-09-14T23:57:01.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.566 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:01.566 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6819", 2024-09-14T23:57:01.566 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3033769028 2024-09-14T23:57:01.567 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.567 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.567 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-14T23:57:01.567 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:01.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.567 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:01.567 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6822", 2024-09-14T23:57:01.567 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3033769028 2024-09-14T23:57:01.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.567 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:01.567 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6823", 2024-09-14T23:57:01.568 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3033769028 2024-09-14T23:57:01.568 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.568 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.568 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-14T23:57:01.568 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:01.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.568 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:01.568 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6820", 2024-09-14T23:57:01.568 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3033769028 2024-09-14T23:57:01.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.568 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:01.568 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6821", 2024-09-14T23:57:01.569 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3033769028 2024-09-14T23:57:01.569 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.569 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.569 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.43:6817/3033769028", 2024-09-14T23:57:01.569 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.43:6819/3033769028", 2024-09-14T23:57:01.569 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.43:6823/3033769028", 2024-09-14T23:57:01.569 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.43:6821/3033769028", 2024-09-14T23:57:01.569 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-14T23:57:01.569 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-14T23:57:01.569 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-14T23:57:01.569 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.570 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 6, 2024-09-14T23:57:01.570 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "c58c1d59-ad02-40f0-98e3-9078ef7e622b", 2024-09-14T23:57:01.570 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-14T23:57:01.570 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-14T23:57:01.570 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-14T23:57:01.570 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-14T23:57:01.570 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-14T23:57:01.570 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-14T23:57:01.570 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-14T23:57:01.570 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 23, 2024-09-14T23:57:01.570 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-14T23:57:01.570 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-14T23:57:01.570 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-14T23:57:01.570 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:01.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.571 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:01.571 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6800", 2024-09-14T23:57:01.571 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3314282416 2024-09-14T23:57:01.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.571 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:01.571 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6801", 2024-09-14T23:57:01.571 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3314282416 2024-09-14T23:57:01.571 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.571 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.571 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-14T23:57:01.571 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:01.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.572 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:01.572 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6802", 2024-09-14T23:57:01.572 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3314282416 2024-09-14T23:57:01.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.572 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:01.572 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6803", 2024-09-14T23:57:01.572 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3314282416 2024-09-14T23:57:01.572 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.572 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.573 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-14T23:57:01.573 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:01.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.573 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:01.573 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6806", 2024-09-14T23:57:01.573 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3314282416 2024-09-14T23:57:01.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.573 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:01.574 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6807", 2024-09-14T23:57:01.574 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3314282416 2024-09-14T23:57:01.574 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.574 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.574 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-14T23:57:01.574 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:01.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.574 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:01.574 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6804", 2024-09-14T23:57:01.574 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3314282416 2024-09-14T23:57:01.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.575 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:01.575 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6805", 2024-09-14T23:57:01.575 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3314282416 2024-09-14T23:57:01.575 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.575 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.575 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.43:6801/3314282416", 2024-09-14T23:57:01.575 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.43:6803/3314282416", 2024-09-14T23:57:01.575 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.43:6807/3314282416", 2024-09-14T23:57:01.575 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.43:6805/3314282416", 2024-09-14T23:57:01.575 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-14T23:57:01.575 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-14T23:57:01.576 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-14T23:57:01.576 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.576 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 7, 2024-09-14T23:57:01.576 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "389bdb04-20e3-4a7a-8504-687e034bd7dc", 2024-09-14T23:57:01.576 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-14T23:57:01.576 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-14T23:57:01.576 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-14T23:57:01.576 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-14T23:57:01.576 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-14T23:57:01.576 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-14T23:57:01.576 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-14T23:57:01.577 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 49, 2024-09-14T23:57:01.577 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-14T23:57:01.577 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-14T23:57:01.577 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-14T23:57:01.577 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:01.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.577 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:01.577 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6824", 2024-09-14T23:57:01.577 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1481350118 2024-09-14T23:57:01.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.578 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:01.578 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6825", 2024-09-14T23:57:01.578 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1481350118 2024-09-14T23:57:01.578 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.578 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.578 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-14T23:57:01.578 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:01.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.578 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:01.578 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6826", 2024-09-14T23:57:01.578 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1481350118 2024-09-14T23:57:01.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.579 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:01.579 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6827", 2024-09-14T23:57:01.579 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1481350118 2024-09-14T23:57:01.579 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.579 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.579 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-14T23:57:01.579 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:01.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.579 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:01.579 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6830", 2024-09-14T23:57:01.579 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1481350118 2024-09-14T23:57:01.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.580 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:01.580 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6831", 2024-09-14T23:57:01.580 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1481350118 2024-09-14T23:57:01.580 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.580 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.580 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-14T23:57:01.580 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:01.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.580 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:01.580 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6828", 2024-09-14T23:57:01.580 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1481350118 2024-09-14T23:57:01.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.581 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:01.581 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6829", 2024-09-14T23:57:01.581 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1481350118 2024-09-14T23:57:01.581 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.581 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.581 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.43:6825/1481350118", 2024-09-14T23:57:01.581 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.43:6827/1481350118", 2024-09-14T23:57:01.581 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.43:6831/1481350118", 2024-09-14T23:57:01.581 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.43:6829/1481350118", 2024-09-14T23:57:01.581 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-14T23:57:01.581 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-14T23:57:01.581 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-14T23:57:01.582 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.582 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.582 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-14T23:57:01.582 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_xinfo": [ 2024-09-14T23:57:01.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.582 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 0, 2024-09-14T23:57:01.582 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-14T23:57:01.582 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-14T23:57:01.582 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-14T23:57:01.582 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-14T23:57:01.582 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-14T23:57:01.582 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-14T23:56:11.077810+0000", 2024-09-14T23:57:01.582 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-14T23:57:01.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.583 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 1, 2024-09-14T23:57:01.583 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-14T23:57:01.583 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-14T23:57:01.583 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-14T23:57:01.583 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-14T23:57:01.583 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-14T23:57:01.583 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-14T23:56:11.578838+0000", 2024-09-14T23:57:01.583 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-14T23:57:01.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.583 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 2, 2024-09-14T23:57:01.583 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-14T23:57:01.583 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-14T23:57:01.584 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-14T23:57:01.584 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-14T23:57:01.584 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-14T23:57:01.584 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-14T23:56:10.895490+0000", 2024-09-14T23:57:01.584 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-14T23:57:01.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.584 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 3, 2024-09-14T23:57:01.584 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-14T23:57:01.584 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-14T23:57:01.584 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-14T23:57:01.584 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-14T23:57:01.584 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-14T23:57:01.584 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-14T23:56:12.290821+0000", 2024-09-14T23:57:01.585 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-14T23:57:01.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.585 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 4, 2024-09-14T23:57:01.585 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-14T23:57:01.585 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-14T23:57:01.585 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-14T23:57:01.585 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-14T23:57:01.585 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-14T23:57:01.585 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-14T23:56:11.800581+0000", 2024-09-14T23:57:01.585 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-14T23:57:01.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.585 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 5, 2024-09-14T23:57:01.586 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-14T23:57:01.586 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-14T23:57:01.586 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-14T23:57:01.586 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-14T23:57:01.586 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-14T23:57:01.586 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-14T23:56:11.883155+0000", 2024-09-14T23:57:01.586 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-14T23:57:01.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.586 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 6, 2024-09-14T23:57:01.586 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-14T23:57:01.586 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-14T23:57:01.586 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-14T23:57:01.587 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-14T23:57:01.587 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-14T23:57:01.587 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-14T23:56:11.604790+0000", 2024-09-14T23:57:01.587 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-14T23:57:01.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.587 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 7, 2024-09-14T23:57:01.587 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-14T23:57:01.587 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-14T23:57:01.587 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-14T23:57:01.587 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-14T23:57:01.587 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-14T23:57:01.587 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-14T23:56:12.067759+0000", 2024-09-14T23:57:01.588 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-14T23:57:01.588 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.588 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-14T23:57:01.588 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_upmap": [], 2024-09-14T23:57:01.588 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_upmap_items": [], 2024-09-14T23:57:01.588 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_upmap_primaries": [], 2024-09-14T23:57:01.588 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_temp": [], 2024-09-14T23:57:01.588 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_temp": [], 2024-09-14T23:57:01.588 INFO:teuthology.orchestra.run.smithi042.stdout: "blocklist": {}, 2024-09-14T23:57:01.588 INFO:teuthology.orchestra.run.smithi042.stdout: "range_blocklist": {}, 2024-09-14T23:57:01.588 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profiles": { 2024-09-14T23:57:01.588 INFO:teuthology.orchestra.run.smithi042.stdout: "default": { 2024-09-14T23:57:01.588 INFO:teuthology.orchestra.run.smithi042.stdout: "crush-failure-domain": "osd", 2024-09-14T23:57:01.588 INFO:teuthology.orchestra.run.smithi042.stdout: "k": "2", 2024-09-14T23:57:01.589 INFO:teuthology.orchestra.run.smithi042.stdout: "m": "1", 2024-09-14T23:57:01.589 INFO:teuthology.orchestra.run.smithi042.stdout: "plugin": "jerasure", 2024-09-14T23:57:01.589 INFO:teuthology.orchestra.run.smithi042.stdout: "technique": "reed_sol_van" 2024-09-14T23:57:01.589 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.589 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps_queue": [], 2024-09-14T23:57:01.589 INFO:teuthology.orchestra.run.smithi042.stdout: "new_removed_snaps": [], 2024-09-14T23:57:01.589 INFO:teuthology.orchestra.run.smithi042.stdout: "new_purged_snaps": [], 2024-09-14T23:57:01.589 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_node_flags": {}, 2024-09-14T23:57:01.589 INFO:teuthology.orchestra.run.smithi042.stdout: "device_class_flags": {}, 2024-09-14T23:57:01.589 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode": { 2024-09-14T23:57:01.589 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode_enabled": false, 2024-09-14T23:57:01.589 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_bucket_count": 0, 2024-09-14T23:57:01.589 INFO:teuthology.orchestra.run.smithi042.stdout: "degraded_stretch_mode": 0, 2024-09-14T23:57:01.589 INFO:teuthology.orchestra.run.smithi042.stdout: "recovering_stretch_mode": 0, 2024-09-14T23:57:01.590 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode_bucket": 0 2024-09-14T23:57:01.590 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.590 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:01.592 INFO:tasks.thrashosds.thrasher:j is {'epoch': 51, 'fsid': '21916fcc-f95c-4bfa-8b8b-11ca45845613', 'created': '2024-09-14T23:56:03.167948+0000', 'modified': '2024-09-14T23:57:01.056942+0000', 'last_up_change': '2024-09-14T23:56:14.322882+0000', 'last_in_change': '2024-09-14T23:56:56.035972+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T23:56:13.500197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8712921142578125, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8700000047683716}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T23:56:21.284705+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.630000114440918, 'score_stable': 2.630000114440918, 'optimal_score': 1, 'raw_score_acting': 2.630000114440918, 'raw_score_stable': 2.630000114440918, 'primary_affinity_weighted': 0.8712921142578125, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8700000047683716}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T23:56:30.861504+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 51, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8712921142578125, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8700000047683716}}], 'osds': [{'osd': 0, 'uuid': 'db9d0c67-2d60-4592-9ad3-f99e5294f93f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 49, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6816', 'nonce': 314865173}, {'type': 'v1', 'addr': '172.21.15.42:6817', 'nonce': 314865173}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6818', 'nonce': 314865173}, {'type': 'v1', 'addr': '172.21.15.42:6819', 'nonce': 314865173}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6822', 'nonce': 314865173}, {'type': 'v1', 'addr': '172.21.15.42:6823', 'nonce': 314865173}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6820', 'nonce': 314865173}, {'type': 'v1', 'addr': '172.21.15.42:6821', 'nonce': 314865173}]}, 'public_addr': '172.21.15.42:6817/314865173', 'cluster_addr': '172.21.15.42:6819/314865173', 'heartbeat_back_addr': '172.21.15.42:6823/314865173', 'heartbeat_front_addr': '172.21.15.42:6821/314865173', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '6c1a2765-c591-4c57-8ab3-ee6645970aa5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 46, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6800', 'nonce': 3197912310}, {'type': 'v1', 'addr': '172.21.15.42:6801', 'nonce': 3197912310}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6802', 'nonce': 3197912310}, {'type': 'v1', 'addr': '172.21.15.42:6803', 'nonce': 3197912310}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6806', 'nonce': 3197912310}, {'type': 'v1', 'addr': '172.21.15.42:6807', 'nonce': 3197912310}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6804', 'nonce': 3197912310}, {'type': 'v1', 'addr': '172.21.15.42:6805', 'nonce': 3197912310}]}, 'public_addr': '172.21.15.42:6801/3197912310', 'cluster_addr': '172.21.15.42:6803/3197912310', 'heartbeat_back_addr': '172.21.15.42:6807/3197912310', 'heartbeat_front_addr': '172.21.15.42:6805/3197912310', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '930db893-7be6-4645-a7f1-e043c5ec95eb', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 46, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6808', 'nonce': 3241153660}, {'type': 'v1', 'addr': '172.21.15.42:6809', 'nonce': 3241153660}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6810', 'nonce': 3241153660}, {'type': 'v1', 'addr': '172.21.15.42:6811', 'nonce': 3241153660}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6814', 'nonce': 3241153660}, {'type': 'v1', 'addr': '172.21.15.42:6815', 'nonce': 3241153660}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6812', 'nonce': 3241153660}, {'type': 'v1', 'addr': '172.21.15.42:6813', 'nonce': 3241153660}]}, 'public_addr': '172.21.15.42:6809/3241153660', 'cluster_addr': '172.21.15.42:6811/3241153660', 'heartbeat_back_addr': '172.21.15.42:6815/3241153660', 'heartbeat_front_addr': '172.21.15.42:6813/3241153660', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '45bb3b84-b1c4-4f87-a7d1-0eb77b81741b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 49, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6824', 'nonce': 1736416326}, {'type': 'v1', 'addr': '172.21.15.42:6825', 'nonce': 1736416326}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6826', 'nonce': 1736416326}, {'type': 'v1', 'addr': '172.21.15.42:6827', 'nonce': 1736416326}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6830', 'nonce': 1736416326}, {'type': 'v1', 'addr': '172.21.15.42:6831', 'nonce': 1736416326}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6828', 'nonce': 1736416326}, {'type': 'v1', 'addr': '172.21.15.42:6829', 'nonce': 1736416326}]}, 'public_addr': '172.21.15.42:6825/1736416326', 'cluster_addr': '172.21.15.42:6827/1736416326', 'heartbeat_back_addr': '172.21.15.42:6831/1736416326', 'heartbeat_front_addr': '172.21.15.42:6829/1736416326', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'a812e38f-c0f9-4d33-9373-1910fb09103c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.9703369140625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 49, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6808', 'nonce': 1755154427}, {'type': 'v1', 'addr': '172.21.15.43:6809', 'nonce': 1755154427}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6810', 'nonce': 1755154427}, {'type': 'v1', 'addr': '172.21.15.43:6811', 'nonce': 1755154427}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6814', 'nonce': 1755154427}, {'type': 'v1', 'addr': '172.21.15.43:6815', 'nonce': 1755154427}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6812', 'nonce': 1755154427}, {'type': 'v1', 'addr': '172.21.15.43:6813', 'nonce': 1755154427}]}, 'public_addr': '172.21.15.43:6809/1755154427', 'cluster_addr': '172.21.15.43:6811/1755154427', 'heartbeat_back_addr': '172.21.15.43:6815/1755154427', 'heartbeat_front_addr': '172.21.15.43:6813/1755154427', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '0de87e5b-7a21-4fb4-9543-d96c065786f8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6816', 'nonce': 3033769028}, {'type': 'v1', 'addr': '172.21.15.43:6817', 'nonce': 3033769028}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6818', 'nonce': 3033769028}, {'type': 'v1', 'addr': '172.21.15.43:6819', 'nonce': 3033769028}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6822', 'nonce': 3033769028}, {'type': 'v1', 'addr': '172.21.15.43:6823', 'nonce': 3033769028}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6820', 'nonce': 3033769028}, {'type': 'v1', 'addr': '172.21.15.43:6821', 'nonce': 3033769028}]}, 'public_addr': '172.21.15.43:6817/3033769028', 'cluster_addr': '172.21.15.43:6819/3033769028', 'heartbeat_back_addr': '172.21.15.43:6823/3033769028', 'heartbeat_front_addr': '172.21.15.43:6821/3033769028', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'c58c1d59-ad02-40f0-98e3-9078ef7e622b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6800', 'nonce': 3314282416}, {'type': 'v1', 'addr': '172.21.15.43:6801', 'nonce': 3314282416}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6802', 'nonce': 3314282416}, {'type': 'v1', 'addr': '172.21.15.43:6803', 'nonce': 3314282416}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6806', 'nonce': 3314282416}, {'type': 'v1', 'addr': '172.21.15.43:6807', 'nonce': 3314282416}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6804', 'nonce': 3314282416}, {'type': 'v1', 'addr': '172.21.15.43:6805', 'nonce': 3314282416}]}, 'public_addr': '172.21.15.43:6801/3314282416', 'cluster_addr': '172.21.15.43:6803/3314282416', 'heartbeat_back_addr': '172.21.15.43:6807/3314282416', 'heartbeat_front_addr': '172.21.15.43:6805/3314282416', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '389bdb04-20e3-4a7a-8504-687e034bd7dc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 49, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6824', 'nonce': 1481350118}, {'type': 'v1', 'addr': '172.21.15.43:6825', 'nonce': 1481350118}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6826', 'nonce': 1481350118}, {'type': 'v1', 'addr': '172.21.15.43:6827', 'nonce': 1481350118}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6830', 'nonce': 1481350118}, {'type': 'v1', 'addr': '172.21.15.43:6831', 'nonce': 1481350118}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6828', 'nonce': 1481350118}, {'type': 'v1', 'addr': '172.21.15.43:6829', 'nonce': 1481350118}]}, 'public_addr': '172.21.15.43:6825/1481350118', 'cluster_addr': '172.21.15.43:6827/1481350118', 'heartbeat_back_addr': '172.21.15.43:6831/1481350118', 'heartbeat_front_addr': '172.21.15.43:6829/1481350118', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T23:56:11.077810+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T23:56:11.578838+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T23:56:10.895490+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T23:56:12.290821+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T23:56:11.800581+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T23:56:11.883155+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T23:56:11.604790+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T23:56:12.067759+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T23:57:01.593 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T23:57:01.594 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:01.594 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:57:01.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:57:01.594 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:57:01.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.594 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5880 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:01.594 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375821+0000", 2024-09-14T23:57:01.594 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.194490603999999, 2024-09-14T23:57:01.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050482096999999997, 2024-09-14T23:57:01.595 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:01.595 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:01.595 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:01.595 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:01.595 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:01.595 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:01.595 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5880 2024-09-14T23:57:01.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.595 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:01.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:01.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375821+0000", 2024-09-14T23:57:01.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:01.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375821+0000", 2024-09-14T23:57:01.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:01.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375821+0000", 2024-09-14T23:57:01.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.28e-07 2024-09-14T23:57:01.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:01.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375822+0000", 2024-09-14T23:57:01.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7000000000000003e-07 2024-09-14T23:57:01.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:01.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375825+0000", 2024-09-14T23:57:01.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9560000000000002e-06 2024-09-14T23:57:01.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:01.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375827+0000", 2024-09-14T23:57:01.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3259999999999998e-06 2024-09-14T23:57:01.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:01.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426294+0000", 2024-09-14T23:57:01.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050466748999999998 2024-09-14T23:57:01.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:01.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426303+0000", 2024-09-14T23:57:01.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8680000000000005e-06 2024-09-14T23:57:01.598 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.598 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.598 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.598 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5881 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:01.599 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375830+0000", 2024-09-14T23:57:01.599 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.194481157999999, 2024-09-14T23:57:01.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050523831999999998, 2024-09-14T23:57:01.599 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:01.599 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:01.599 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:01.599 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:01.599 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:01.599 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:01.599 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5881 2024-09-14T23:57:01.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.599 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:01.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:01.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375830+0000", 2024-09-14T23:57:01.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:01.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375830+0000", 2024-09-14T23:57:01.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:01.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375830+0000", 2024-09-14T23:57:01.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.34e-07 2024-09-14T23:57:01.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:01.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375831+0000", 2024-09-14T23:57:01.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9599999999999998e-07 2024-09-14T23:57:01.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:01.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375834+0000", 2024-09-14T23:57:01.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0489999999999999e-06 2024-09-14T23:57:01.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:01.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375838+0000", 2024-09-14T23:57:01.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5319999999999998e-06 2024-09-14T23:57:01.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:01.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426342+0000", 2024-09-14T23:57:01.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050504391000000003 2024-09-14T23:57:01.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:01.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426354+0000", 2024-09-14T23:57:01.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.173e-05 2024-09-14T23:57:01.602 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.602 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.602 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.603 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5882 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:01.603 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:57:01.603 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.194470553999999, 2024-09-14T23:57:01.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050549666, 2024-09-14T23:57:01.603 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:01.603 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:01.603 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:01.603 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:01.603 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:01.603 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:01.603 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5882 2024-09-14T23:57:01.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.603 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:01.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:01.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:57:01.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:01.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:57:01.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:01.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:57:01.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8099999999999998e-07 2024-09-14T23:57:01.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:01.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375842+0000", 2024-09-14T23:57:01.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9900000000000009e-07 2024-09-14T23:57:01.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:01.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375845+0000", 2024-09-14T23:57:01.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.055e-06 2024-09-14T23:57:01.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:01.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375848+0000", 2024-09-14T23:57:01.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7609999999999999e-06 2024-09-14T23:57:01.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:01.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426381+0000", 2024-09-14T23:57:01.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050533392000000003 2024-09-14T23:57:01.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:01.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426390+0000", 2024-09-14T23:57:01.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0780000000000002e-06 2024-09-14T23:57:01.606 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.606 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.606 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.607 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5884 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:01.607 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:57:01.607 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.194460414000002, 2024-09-14T23:57:01.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050717439000000003, 2024-09-14T23:57:01.607 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:01.607 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:01.607 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:01.607 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:01.607 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:01.607 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:01.607 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5884 2024-09-14T23:57:01.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.608 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:01.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:01.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:57:01.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:01.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:57:01.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:01.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:57:01.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0800000000000001e-07 2024-09-14T23:57:01.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:01.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375852+0000", 2024-09-14T23:57:01.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.02e-06 2024-09-14T23:57:01.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:01.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375855+0000", 2024-09-14T23:57:01.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9900000000000002e-06 2024-09-14T23:57:01.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:01.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375857+0000", 2024-09-14T23:57:01.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3499999999999999e-06 2024-09-14T23:57:01.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:01.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426556+0000", 2024-09-14T23:57:01.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050698139000000003 2024-09-14T23:57:01.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:01.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426568+0000", 2024-09-14T23:57:01.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2731999999999999e-05 2024-09-14T23:57:01.610 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.610 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.611 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.611 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5885 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:01.611 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:57:01.611 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.194441389000001, 2024-09-14T23:57:01.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050737333000000003, 2024-09-14T23:57:01.611 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:01.611 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:01.611 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:01.611 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:01.611 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:01.611 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:01.611 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5885 2024-09-14T23:57:01.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.612 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:01.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:01.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:57:01.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:01.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:57:01.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:01.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:57:01.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5899999999999998e-07 2024-09-14T23:57:01.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:01.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375872+0000", 2024-09-14T23:57:01.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4219999999999999e-06 2024-09-14T23:57:01.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:01.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375878+0000", 2024-09-14T23:57:01.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4980000000000003e-06 2024-09-14T23:57:01.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:01.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375881+0000", 2024-09-14T23:57:01.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6110000000000001e-06 2024-09-14T23:57:01.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:01.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426596+0000", 2024-09-14T23:57:01.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050715320000000001 2024-09-14T23:57:01.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:01.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426607+0000", 2024-09-14T23:57:01.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1223000000000001e-05 2024-09-14T23:57:01.614 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.614 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.615 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.615 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5886 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:01.615 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:57:01.615 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.194427275999999, 2024-09-14T23:57:01.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050803395000000001, 2024-09-14T23:57:01.615 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:01.615 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:01.615 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:01.615 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:01.615 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:01.615 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:01.615 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5886 2024-09-14T23:57:01.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.616 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:01.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:01.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:57:01.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:01.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:57:01.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:01.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:57:01.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2700000000000001e-07 2024-09-14T23:57:01.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:01.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375886+0000", 2024-09-14T23:57:01.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7060000000000001e-06 2024-09-14T23:57:01.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:01.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375890+0000", 2024-09-14T23:57:01.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2259999999999999e-06 2024-09-14T23:57:01.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:01.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375893+0000", 2024-09-14T23:57:01.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0819999999999999e-06 2024-09-14T23:57:01.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:01.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426677+0000", 2024-09-14T23:57:01.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050784158000000003 2024-09-14T23:57:01.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:01.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426687+0000", 2024-09-14T23:57:01.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9960000000000005e-06 2024-09-14T23:57:01.619 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.619 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.619 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.619 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5887 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:01.619 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:57:01.619 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.194405190000001, 2024-09-14T23:57:01.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050833070000000001, 2024-09-14T23:57:01.619 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:01.619 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:01.619 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:01.619 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:01.620 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:01.620 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:01.620 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5887 2024-09-14T23:57:01.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.620 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:01.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:01.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:57:01.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:01.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:57:01.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:01.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:57:01.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3299999999999998e-07 2024-09-14T23:57:01.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:01.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375908+0000", 2024-09-14T23:57:01.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7689999999999999e-06 2024-09-14T23:57:01.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:01.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375912+0000", 2024-09-14T23:57:01.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.535e-06 2024-09-14T23:57:01.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:01.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375916+0000", 2024-09-14T23:57:01.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8979999999999998e-06 2024-09-14T23:57:01.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:01.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426729+0000", 2024-09-14T23:57:01.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050813453000000001 2024-09-14T23:57:01.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:01.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426739+0000", 2024-09-14T23:57:01.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0081999999999999e-05 2024-09-14T23:57:01.623 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.623 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.623 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.623 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5888 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:01.623 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:57:01.624 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.194392112999999, 2024-09-14T23:57:01.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050854652, 2024-09-14T23:57:01.624 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:01.624 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:01.624 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:01.624 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:01.624 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:01.624 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:01.624 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5888 2024-09-14T23:57:01.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.624 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:01.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:01.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:57:01.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:01.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:57:01.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:01.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:57:01.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1400000000000001e-07 2024-09-14T23:57:01.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:01.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375921+0000", 2024-09-14T23:57:01.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.119e-06 2024-09-14T23:57:01.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:01.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375928+0000", 2024-09-14T23:57:01.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3170000000000004e-06 2024-09-14T23:57:01.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:01.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375931+0000", 2024-09-14T23:57:01.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6970000000000002e-06 2024-09-14T23:57:01.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:01.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426768+0000", 2024-09-14T23:57:01.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050837355000000001 2024-09-14T23:57:01.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:01.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426774+0000", 2024-09-14T23:57:01.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9499999999999998e-06 2024-09-14T23:57:01.627 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.627 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.627 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.628 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5889 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:01.628 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:57:01.628 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.194375188999999, 2024-09-14T23:57:01.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050885446000000001, 2024-09-14T23:57:01.628 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:01.628 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:01.628 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:01.628 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:01.628 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:01.628 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:01.628 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5889 2024-09-14T23:57:01.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.629 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:01.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:01.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:57:01.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:01.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:57:01.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:01.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:57:01.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.36e-07 2024-09-14T23:57:01.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:01.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375940+0000", 2024-09-14T23:57:01.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.844e-06 2024-09-14T23:57:01.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:01.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375944+0000", 2024-09-14T23:57:01.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6720000000000002e-06 2024-09-14T23:57:01.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:01.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375946+0000", 2024-09-14T23:57:01.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5629999999999999e-06 2024-09-14T23:57:01.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:01.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426810+0000", 2024-09-14T23:57:01.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050863255000000003 2024-09-14T23:57:01.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:01.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426822+0000", 2024-09-14T23:57:01.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1875999999999999e-05 2024-09-14T23:57:01.631 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.631 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.632 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.632 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5890 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:01.632 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:57:01.632 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.194361377, 2024-09-14T23:57:01.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050917549999999999, 2024-09-14T23:57:01.632 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:01.632 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:01.632 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:01.632 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:01.632 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:01.632 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:01.632 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5890 2024-09-14T23:57:01.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.633 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:01.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:01.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:57:01.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:01.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:57:01.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:01.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:57:01.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4299999999999999e-07 2024-09-14T23:57:01.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:01.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375951+0000", 2024-09-14T23:57:01.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1000000000000001e-06 2024-09-14T23:57:01.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:01.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375956+0000", 2024-09-14T23:57:01.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3370000000000001e-06 2024-09-14T23:57:01.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:01.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375958+0000", 2024-09-14T23:57:01.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4820000000000001e-06 2024-09-14T23:57:01.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:01.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426858+0000", 2024-09-14T23:57:01.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050899666000000003 2024-09-14T23:57:01.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:01.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426867+0000", 2024-09-14T23:57:01.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7219999999999994e-06 2024-09-14T23:57:01.636 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.636 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.636 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.636 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5891 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:01.636 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:57:01.636 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.194350055000001, 2024-09-14T23:57:01.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050948067, 2024-09-14T23:57:01.636 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:01.636 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:01.636 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:01.636 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:01.636 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:01.637 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:01.637 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5891 2024-09-14T23:57:01.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.637 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:01.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:01.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:57:01.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:01.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:57:01.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:01.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:57:01.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0800000000000001e-07 2024-09-14T23:57:01.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:01.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375962+0000", 2024-09-14T23:57:01.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9600000000000008e-07 2024-09-14T23:57:01.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:01.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375970+0000", 2024-09-14T23:57:01.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1600000000000001e-06 2024-09-14T23:57:01.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:01.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375972+0000", 2024-09-14T23:57:01.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4849999999999999e-06 2024-09-14T23:57:01.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:01.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426902+0000", 2024-09-14T23:57:01.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050929813999999997 2024-09-14T23:57:01.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:01.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426909+0000", 2024-09-14T23:57:01.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4039999999999998e-06 2024-09-14T23:57:01.640 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.640 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.640 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.640 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5892 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:01.640 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:57:01.640 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.194335198000001, 2024-09-14T23:57:01.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050965036999999998, 2024-09-14T23:57:01.640 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:01.640 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:01.640 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:01.641 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:01.641 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:01.641 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:01.641 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5892 2024-09-14T23:57:01.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.641 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:01.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:01.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:57:01.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:01.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:57:01.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:01.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:57:01.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.34e-07 2024-09-14T23:57:01.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:01.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375978+0000", 2024-09-14T23:57:01.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1880000000000001e-06 2024-09-14T23:57:01.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:01.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375981+0000", 2024-09-14T23:57:01.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2399999999999999e-06 2024-09-14T23:57:01.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:01.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375984+0000", 2024-09-14T23:57:01.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9110000000000002e-06 2024-09-14T23:57:01.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:01.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426934+0000", 2024-09-14T23:57:01.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050949912 2024-09-14T23:57:01.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:01.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426941+0000", 2024-09-14T23:57:01.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5519999999999999e-06 2024-09-14T23:57:01.644 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.644 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.644 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.644 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5893 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:01.644 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:57:01.644 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.194324601000002, 2024-09-14T23:57:01.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050975131999999999, 2024-09-14T23:57:01.644 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:01.645 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:01.645 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:01.645 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:01.645 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:01.645 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:01.645 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5893 2024-09-14T23:57:01.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.645 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:01.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:01.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:57:01.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:01.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:57:01.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:01.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:57:01.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2999999999999999e-07 2024-09-14T23:57:01.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:01.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375988+0000", 2024-09-14T23:57:01.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0410000000000001e-06 2024-09-14T23:57:01.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:01.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375992+0000", 2024-09-14T23:57:01.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.104e-06 2024-09-14T23:57:01.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:01.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375995+0000", 2024-09-14T23:57:01.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4310000000000001e-06 2024-09-14T23:57:01.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:01.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426958+0000", 2024-09-14T23:57:01.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050963642000000003 2024-09-14T23:57:01.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.648 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:01.648 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426962+0000", 2024-09-14T23:57:01.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6839999999999998e-06 2024-09-14T23:57:01.648 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.648 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.648 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.648 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5894 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:01.648 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:57:01.648 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.194313201, 2024-09-14T23:57:01.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050978333000000001, 2024-09-14T23:57:01.648 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:01.649 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:01.649 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:01.649 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:01.649 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:01.649 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:01.649 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5894 2024-09-14T23:57:01.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.649 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:01.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:01.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:57:01.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.650 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:01.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:57:01.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.650 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:01.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:57:01.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.23e-07 2024-09-14T23:57:01.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.650 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:01.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375999+0000", 2024-09-14T23:57:01.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9800000000000002e-07 2024-09-14T23:57:01.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.651 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:01.651 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376007+0000", 2024-09-14T23:57:01.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8539999999999997e-06 2024-09-14T23:57:01.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.651 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:01.651 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376011+0000", 2024-09-14T23:57:01.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.399e-06 2024-09-14T23:57:01.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.651 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:01.652 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426973+0000", 2024-09-14T23:57:01.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050962454999999997 2024-09-14T23:57:01.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.652 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:01.652 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426976+0000", 2024-09-14T23:57:01.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4039999999999999e-06 2024-09-14T23:57:01.652 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.652 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.652 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.652 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5895 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:01.652 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376014+0000", 2024-09-14T23:57:01.653 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.194296857000001, 2024-09-14T23:57:01.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050975951999999998, 2024-09-14T23:57:01.653 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:01.653 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:01.653 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:01.653 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:01.653 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:01.653 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:01.653 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5895 2024-09-14T23:57:01.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.653 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:01.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.653 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:01.654 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376014+0000", 2024-09-14T23:57:01.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.654 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:01.654 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376014+0000", 2024-09-14T23:57:01.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.654 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:01.654 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376015+0000", 2024-09-14T23:57:01.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2999999999999999e-07 2024-09-14T23:57:01.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.655 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:01.655 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376016+0000", 2024-09-14T23:57:01.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1659999999999999e-06 2024-09-14T23:57:01.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.655 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:01.655 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376024+0000", 2024-09-14T23:57:01.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8930000000000005e-06 2024-09-14T23:57:01.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.655 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:01.655 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376026+0000", 2024-09-14T23:57:01.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6869999999999999e-06 2024-09-14T23:57:01.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.656 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.656 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:01.656 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426987+0000", 2024-09-14T23:57:01.656 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050960601000000001 2024-09-14T23:57:01.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.656 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.656 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:01.656 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426990+0000", 2024-09-14T23:57:01.656 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3749999999999999e-06 2024-09-14T23:57:01.656 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.656 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.656 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.657 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.657 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5896 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:01.657 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:57:01.657 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.194281119999999, 2024-09-14T23:57:01.657 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050975656000000001, 2024-09-14T23:57:01.657 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:01.657 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:01.657 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:01.657 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:01.657 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:01.657 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:01.657 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5896 2024-09-14T23:57:01.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.657 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:01.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:01.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:57:01.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:01.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:57:01.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:01.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:57:01.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5100000000000001e-07 2024-09-14T23:57:01.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:01.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376032+0000", 2024-09-14T23:57:01.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.141e-06 2024-09-14T23:57:01.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:01.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376036+0000", 2024-09-14T23:57:01.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.25e-06 2024-09-14T23:57:01.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:01.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376038+0000", 2024-09-14T23:57:01.660 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5469999999999999e-06 2024-09-14T23:57:01.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.660 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:01.660 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427002+0000", 2024-09-14T23:57:01.660 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050964029000000001 2024-09-14T23:57:01.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.660 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:01.660 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427006+0000", 2024-09-14T23:57:01.660 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4379999999999999e-06 2024-09-14T23:57:01.660 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.660 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.661 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.661 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5897 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:01.661 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:57:01.661 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.19426932, 2024-09-14T23:57:01.661 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050978001000000002, 2024-09-14T23:57:01.661 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:01.661 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:01.661 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:01.661 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:01.661 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:01.661 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:01.661 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5897 2024-09-14T23:57:01.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.662 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:01.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.662 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:01.662 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:57:01.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.662 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:01.662 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:57:01.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.662 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:01.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:57:01.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2600000000000001e-07 2024-09-14T23:57:01.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:01.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376043+0000", 2024-09-14T23:57:01.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.054e-06 2024-09-14T23:57:01.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:01.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376047+0000", 2024-09-14T23:57:01.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0659999999999997e-06 2024-09-14T23:57:01.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.664 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:01.664 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376050+0000", 2024-09-14T23:57:01.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.88e-06 2024-09-14T23:57:01.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.664 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:01.664 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427017+0000", 2024-09-14T23:57:01.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050966453000000002 2024-09-14T23:57:01.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.664 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:01.664 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427020+0000", 2024-09-14T23:57:01.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3220000000000001e-06 2024-09-14T23:57:01.664 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.665 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.665 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.665 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5898 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:01.665 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376053+0000", 2024-09-14T23:57:01.665 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.194257834999998, 2024-09-14T23:57:01.665 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050982024000000001, 2024-09-14T23:57:01.665 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:01.665 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:01.665 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:01.665 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:01.665 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:01.665 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:01.666 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5898 2024-09-14T23:57:01.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.666 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:01.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.666 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:01.666 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376053+0000", 2024-09-14T23:57:01.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.666 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:01.666 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376053+0000", 2024-09-14T23:57:01.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.667 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:01.667 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376278+0000", 2024-09-14T23:57:01.667 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022426400000000001 2024-09-14T23:57:01.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.667 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:01.667 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376282+0000", 2024-09-14T23:57:01.667 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4739999999999999e-06 2024-09-14T23:57:01.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.667 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:01.667 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376296+0000", 2024-09-14T23:57:01.667 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4229999999999999e-05 2024-09-14T23:57:01.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.668 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:01.668 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376303+0000", 2024-09-14T23:57:01.668 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2199999999999997e-06 2024-09-14T23:57:01.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.668 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:01.668 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427031+0000", 2024-09-14T23:57:01.668 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050728154999999997 2024-09-14T23:57:01.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.668 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:01.668 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427036+0000", 2024-09-14T23:57:01.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6809999999999997e-06 2024-09-14T23:57:01.669 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.669 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.669 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.669 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5899 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:01.669 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376338+0000", 2024-09-14T23:57:01.669 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.193973110999998, 2024-09-14T23:57:01.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050712100000000003, 2024-09-14T23:57:01.669 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:01.669 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:01.669 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:01.669 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:01.670 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:01.670 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:01.670 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5899 2024-09-14T23:57:01.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.670 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:01.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.670 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:01.670 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376338+0000", 2024-09-14T23:57:01.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.670 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:01.670 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376338+0000", 2024-09-14T23:57:01.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.671 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:01.671 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376339+0000", 2024-09-14T23:57:01.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9100000000000003e-07 2024-09-14T23:57:01.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.671 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:01.671 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376342+0000", 2024-09-14T23:57:01.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7e-06 2024-09-14T23:57:01.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.671 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:01.671 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376350+0000", 2024-09-14T23:57:01.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5280000000000003e-06 2024-09-14T23:57:01.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:01.672 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376356+0000", 2024-09-14T23:57:01.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6010000000000002e-06 2024-09-14T23:57:01.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:01.672 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427047+0000", 2024-09-14T23:57:01.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050690962999999999 2024-09-14T23:57:01.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:01.673 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427050+0000", 2024-09-14T23:57:01.673 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.517e-06 2024-09-14T23:57:01.673 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.673 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.673 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.673 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5900 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:01.673 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376366+0000", 2024-09-14T23:57:01.673 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.193944852000001, 2024-09-14T23:57:01.673 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050697858999999998, 2024-09-14T23:57:01.674 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:01.674 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:01.674 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:01.674 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:01.674 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:01.674 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:01.674 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5900 2024-09-14T23:57:01.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.674 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:01.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.674 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:01.674 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376366+0000", 2024-09-14T23:57:01.674 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:01.675 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376366+0000", 2024-09-14T23:57:01.675 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:01.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:01.675 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377650+0000", 2024-09-14T23:57:01.675 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001283834 2024-09-14T23:57:01.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:01.675 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377655+0000", 2024-09-14T23:57:01.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7230000000000004e-06 2024-09-14T23:57:01.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.676 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:01.676 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377667+0000", 2024-09-14T23:57:01.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1746000000000001e-05 2024-09-14T23:57:01.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.676 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:01.676 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377676+0000", 2024-09-14T23:57:01.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3619999999999995e-06 2024-09-14T23:57:01.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.677 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:01.677 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427061+0000", 2024-09-14T23:57:01.677 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049384755000000002 2024-09-14T23:57:01.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:01.677 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:01.677 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:01.677 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427064+0000", 2024-09-14T23:57:01.677 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.439e-06 2024-09-14T23:57:01.677 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.677 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.677 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.677 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:01.677 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:01.677 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:01.678 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T23:57:01.679 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:01.590+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:01.721 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:01.720+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:01.793 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:01.792+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:01.795 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:01.795 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:57:01.795 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:57:01.795 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:01.805 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T23:57:01.894 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:01.893+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:01.952 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:01.952 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:57:01.953 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:57:01.953 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:57:01.953 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:01.961 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T23:57:01.979 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:57:01.980 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-14T23:57:01.991 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2024-09-14T23:57:01.459762+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":265828798,"num_objects":905,"num_object_clones":147,"num_object_copies":1810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":905,"num_whiteouts":382,"num_read":55036,"num_read_kb":197351,"num_write":5566,"num_write_kb":88071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":631,"num_bytes_recovered":25850194,"num_keys_recovered":109,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4442,"ondisk_log_size":4442,"up":82,"acting":80,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":314916,"kb_used_data":23796,"kb_used_omap":12,"kb_used_meta":290931,"kb_avail":754659804,"statfs":{"total":773094113280,"available":772771639296,"internally_reserved":0,"allocated":24367104,"data_stored":157396562,"data_compressed":11471132,"data_compressed_allocated":13983744,"data_compressed_original":150509250,"omap_allocated":12752,"internal_metadata":297913904},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":31158948,"num_objects":63,"num_object_clones":21,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":3740,"num_read_kb":42394,"num_write":332,"num_write_kb":9157,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":312,"num_bytes_recovered":13182925,"num_keys_recovered":68,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000739"},"pg_stats":[{"pgid":"3.1f","version":"45'118","reported_seq":753,"reported_epoch":48,"state":"active+undersized+remapped","last_fresh":"2024-09-14T23:56:58.118700+0000","last_change":"2024-09-14T23:56:58.118885+0000","last_active":"2024-09-14T23:56:58.118700+0000","last_peered":"2024-09-14T23:56:58.118700+0000","last_clean":"2024-09-14T23:56:55.537684+0000","last_became_active":"2024-09-14T23:56:58.058006+0000","last_became_peered":"2024-09-14T23:56:58.058006+0000","last_unstale":"2024-09-14T23:56:58.118700+0000","last_undegraded":"2024-09-14T23:56:58.118700+0000","last_fullsized":"2024-09-14T23:56:58.051401+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015692659999999999,"stat_sum":{"num_bytes":8597501,"num_objects":26,"num_object_clones":4,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":13,"num_read":462,"num_read_kb":1817,"num_write":118,"num_write_kb":1557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":2327658,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0],"avail_no_missing":["0","2","5"],"object_location_counts":[{"shards":"0","objects":26}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"50'53","reported_seq":486,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-14T23:57:00.163431+0000","last_change":"2024-09-14T23:56:34.415748+0000","last_active":"2024-09-14T23:57:00.163431+0000","last_peered":"2024-09-14T23:57:00.163431+0000","last_clean":"2024-09-14T23:57:00.163431+0000","last_became_active":"2024-09-14T23:56:32.404385+0000","last_became_peered":"2024-09-14T23:56:32.404385+0000","last_unstale":"2024-09-14T23:57:00.163431+0000","last_undegraded":"2024-09-14T23:57:00.163431+0000","last_fullsized":"2024-09-14T23:57:00.163431+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:02:34.249520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010144430000000001,"stat_sum":{"num_bytes":3333418,"num_objects":22,"num_object_clones":2,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":11,"num_read":378,"num_read_kb":2023,"num_write":45,"num_write_kb":1303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"50'157","reported_seq":1465,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-14T23:57:00.189549+0000","last_change":"2024-09-14T23:57:00.062960+0000","last_active":"2024-09-14T23:57:00.189549+0000","last_peered":"2024-09-14T23:57:00.189549+0000","last_clean":"2024-09-14T23:57:00.189549+0000","last_became_active":"2024-09-14T23:57:00.062242+0000","last_became_peered":"2024-09-14T23:57:00.062242+0000","last_unstale":"2024-09-14T23:57:00.189549+0000","last_undegraded":"2024-09-14T23:57:00.189549+0000","last_fullsized":"2024-09-14T23:57:00.189549+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:14:46.589423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001148175,"stat_sum":{"num_bytes":15792520,"num_objects":34,"num_object_clones":7,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":18,"num_read":1536,"num_read_kb":4626,"num_write":128,"num_write_kb":1415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":1249963,"num_keys_recovered":34,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"50'174","reported_seq":1698,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-14T23:57:00.197921+0000","last_change":"2024-09-14T23:57:00.062694+0000","last_active":"2024-09-14T23:57:00.197921+0000","last_peered":"2024-09-14T23:57:00.197921+0000","last_clean":"2024-09-14T23:57:00.197921+0000","last_became_active":"2024-09-14T23:57:00.062126+0000","last_became_peered":"2024-09-14T23:57:00.062126+0000","last_unstale":"2024-09-14T23:57:00.197921+0000","last_undegraded":"2024-09-14T23:57:00.197921+0000","last_fullsized":"2024-09-14T23:57:00.197921+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:27:21.531851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095260199999999998,"stat_sum":{"num_bytes":5817366,"num_objects":27,"num_object_clones":4,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":10,"num_read":1853,"num_read_kb":5595,"num_write":248,"num_write_kb":1949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":2675491,"num_keys_recovered":40,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"42'75","reported_seq":416,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-14T23:56:58.051226+0000","last_change":"2024-09-14T23:56:34.401802+0000","last_active":"2024-09-14T23:56:58.051226+0000","last_peered":"2024-09-14T23:56:58.051226+0000","last_clean":"2024-09-14T23:56:58.051226+0000","last_became_active":"2024-09-14T23:56:32.398329+0000","last_became_peered":"2024-09-14T23:56:32.398329+0000","last_unstale":"2024-09-14T23:56:58.051226+0000","last_undegraded":"2024-09-14T23:56:58.051226+0000","last_fullsized":"2024-09-14T23:56:58.051226+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:46:37.232445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018313229999999999,"stat_sum":{"num_bytes":2527232,"num_objects":24,"num_object_clones":2,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":12,"num_read":293,"num_read_kb":2714,"num_write":70,"num_write_kb":1818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"48'107","reported_seq":1303,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-14T23:57:00.128098+0000","last_change":"2024-09-14T23:56:34.405093+0000","last_active":"2024-09-14T23:57:00.128098+0000","last_peered":"2024-09-14T23:57:00.128098+0000","last_clean":"2024-09-14T23:57:00.128098+0000","last_became_active":"2024-09-14T23:56:32.399655+0000","last_became_peered":"2024-09-14T23:56:32.399655+0000","last_unstale":"2024-09-14T23:57:00.128098+0000","last_undegraded":"2024-09-14T23:57:00.128098+0000","last_fullsized":"2024-09-14T23:57:00.128098+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:54:14.928888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096230199999999995,"stat_sum":{"num_bytes":8170902,"num_objects":28,"num_object_clones":5,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":14,"num_read":1174,"num_read_kb":5275,"num_write":121,"num_write_kb":3375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"42'88","reported_seq":747,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-14T23:56:58.052055+0000","last_change":"2024-09-14T23:56:34.400617+0000","last_active":"2024-09-14T23:56:58.052055+0000","last_peered":"2024-09-14T23:56:58.052055+0000","last_clean":"2024-09-14T23:56:58.052055+0000","last_became_active":"2024-09-14T23:56:32.404364+0000","last_became_peered":"2024-09-14T23:56:32.404364+0000","last_unstale":"2024-09-14T23:56:58.052055+0000","last_undegraded":"2024-09-14T23:56:58.052055+0000","last_fullsized":"2024-09-14T23:56:58.052055+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:37:19.786534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001003264,"stat_sum":{"num_bytes":7007965,"num_objects":30,"num_object_clones":4,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":14,"num_read":616,"num_read_kb":3644,"num_write":81,"num_write_kb":2368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"50'146","reported_seq":2591,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-14T23:57:00.214510+0000","last_change":"2024-09-14T23:56:34.415021+0000","last_active":"2024-09-14T23:57:00.214510+0000","last_peered":"2024-09-14T23:57:00.214510+0000","last_clean":"2024-09-14T23:57:00.214510+0000","last_became_active":"2024-09-14T23:56:32.401799+0000","last_became_peered":"2024-09-14T23:56:32.401799+0000","last_unstale":"2024-09-14T23:57:00.214510+0000","last_undegraded":"2024-09-14T23:57:00.214510+0000","last_fullsized":"2024-09-14T23:57:00.214510+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:04.390496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062985000000000005,"stat_sum":{"num_bytes":8130538,"num_objects":32,"num_object_clones":7,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":12,"num_read":3155,"num_read_kb":10004,"num_write":146,"num_write_kb":1045,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"45'102","reported_seq":588,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-14T23:56:58.061441+0000","last_change":"2024-09-14T23:56:58.061584+0000","last_active":"2024-09-14T23:56:58.061441+0000","last_peered":"2024-09-14T23:56:58.061441+0000","last_clean":"2024-09-14T23:56:55.544344+0000","last_became_active":"2024-09-14T23:56:58.061284+0000","last_became_peered":"2024-09-14T23:56:58.061284+0000","last_unstale":"2024-09-14T23:56:58.061441+0000","last_undegraded":"2024-09-14T23:56:58.052662+0000","last_fullsized":"2024-09-14T23:56:58.052067+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014783859999999999,"stat_sum":{"num_bytes":5818751,"num_objects":21,"num_object_clones":2,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":6,"num_read":510,"num_read_kb":2347,"num_write":126,"num_write_kb":2360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":1430087,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":21}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"50'122","reported_seq":1091,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-14T23:57:00.175565+0000","last_change":"2024-09-14T23:57:00.063163+0000","last_active":"2024-09-14T23:57:00.175565+0000","last_peered":"2024-09-14T23:57:00.175565+0000","last_clean":"2024-09-14T23:57:00.175565+0000","last_became_active":"2024-09-14T23:57:00.062655+0000","last_became_peered":"2024-09-14T23:57:00.062655+0000","last_unstale":"2024-09-14T23:57:00.175565+0000","last_undegraded":"2024-09-14T23:57:00.175565+0000","last_fullsized":"2024-09-14T23:57:00.175565+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:57:57.949582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039416400000000002,"stat_sum":{"num_bytes":10313640,"num_objects":25,"num_object_clones":4,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":9,"num_read":804,"num_read_kb":1090,"num_write":112,"num_write_kb":1524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":1625809,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.15","version":"48'156","reported_seq":5315,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-14T23:57:00.209473+0000","last_change":"2024-09-14T23:56:34.399382+0000","last_active":"2024-09-14T23:57:00.209473+0000","last_peered":"2024-09-14T23:57:00.209473+0000","last_clean":"2024-09-14T23:57:00.209473+0000","last_became_active":"2024-09-14T23:56:32.405387+0000","last_became_peered":"2024-09-14T23:56:32.405387+0000","last_unstale":"2024-09-14T23:57:00.209473+0000","last_undegraded":"2024-09-14T23:57:00.209473+0000","last_fullsized":"2024-09-14T23:57:00.209473+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:13:58.373078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046328199999999997,"stat_sum":{"num_bytes":19301398,"num_objects":21,"num_object_clones":5,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":6,"num_read":5345,"num_read_kb":16396,"num_write":227,"num_write_kb":8196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"48'600","reported_seq":2948,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-14T23:57:00.058854+0000","last_change":"2024-09-14T23:56:34.404536+0000","last_active":"2024-09-14T23:57:00.058854+0000","last_peered":"2024-09-14T23:57:00.058854+0000","last_clean":"2024-09-14T23:57:00.058854+0000","last_became_active":"2024-09-14T23:56:32.403019+0000","last_became_peered":"2024-09-14T23:56:32.403019+0000","last_unstale":"2024-09-14T23:57:00.058854+0000","last_undegraded":"2024-09-14T23:57:00.058854+0000","last_fullsized":"2024-09-14T23:57:00.058854+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":600,"log_dups_size":0,"ondisk_log_size":600,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:59:00.434152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037505299999999997,"stat_sum":{"num_bytes":6351596,"num_objects":27,"num_object_clones":5,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":9,"num_read":5283,"num_read_kb":10838,"num_write":1078,"num_write_kb":1924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":77,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-14T23:57:00.122477+0000","last_change":"2024-09-14T23:56:24.727061+0000","last_active":"2024-09-14T23:57:00.122477+0000","last_peered":"2024-09-14T23:57:00.122477+0000","last_clean":"2024-09-14T23:57:00.122477+0000","last_became_active":"2024-09-14T23:56:22.526915+0000","last_became_peered":"2024-09-14T23:56:22.526915+0000","last_unstale":"2024-09-14T23:57:00.122477+0000","last_undegraded":"2024-09-14T23:57:00.122477+0000","last_fullsized":"2024-09-14T23:57:00.122477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:43:15.359266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045105900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"45'177","reported_seq":2190,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-14T23:56:58.051326+0000","last_change":"2024-09-14T23:56:34.402234+0000","last_active":"2024-09-14T23:56:58.051326+0000","last_peered":"2024-09-14T23:56:58.051326+0000","last_clean":"2024-09-14T23:56:58.051326+0000","last_became_active":"2024-09-14T23:56:32.398664+0000","last_became_peered":"2024-09-14T23:56:32.398664+0000","last_unstale":"2024-09-14T23:56:58.051326+0000","last_undegraded":"2024-09-14T23:56:58.051326+0000","last_fullsized":"2024-09-14T23:56:58.051326+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:45:01.079820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012179999999999999,"stat_sum":{"num_bytes":12038497,"num_objects":25,"num_object_clones":1,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":8,"num_read":3288,"num_read_kb":10213,"num_write":186,"num_write_kb":4104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-14T23:56:58.051458+0000","last_change":"2024-09-14T23:56:24.534632+0000","last_active":"2024-09-14T23:56:58.051458+0000","last_peered":"2024-09-14T23:56:58.051458+0000","last_clean":"2024-09-14T23:56:58.051458+0000","last_became_active":"2024-09-14T23:56:22.526355+0000","last_became_peered":"2024-09-14T23:56:22.526355+0000","last_unstale":"2024-09-14T23:56:58.051458+0000","last_undegraded":"2024-09-14T23:56:58.051458+0000","last_fullsized":"2024-09-14T23:56:58.051458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:02:16.059627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049910000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"44'92","reported_seq":3612,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-14T23:57:00.062269+0000","last_change":"2024-09-14T23:56:34.416134+0000","last_active":"2024-09-14T23:57:00.062269+0000","last_peered":"2024-09-14T23:57:00.062269+0000","last_clean":"2024-09-14T23:57:00.062269+0000","last_became_active":"2024-09-14T23:56:32.404281+0000","last_became_peered":"2024-09-14T23:56:32.404281+0000","last_unstale":"2024-09-14T23:57:00.062269+0000","last_undegraded":"2024-09-14T23:57:00.062269+0000","last_fullsized":"2024-09-14T23:57:00.062269+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:43:32.182713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013337620000000001,"stat_sum":{"num_bytes":14552536,"num_objects":27,"num_object_clones":4,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":12,"num_read":3852,"num_read_kb":22244,"num_write":91,"num_write_kb":2229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":77,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-14T23:57:00.060978+0000","last_change":"2024-09-14T23:56:25.093112+0000","last_active":"2024-09-14T23:57:00.060978+0000","last_peered":"2024-09-14T23:57:00.060978+0000","last_clean":"2024-09-14T23:57:00.060978+0000","last_became_active":"2024-09-14T23:56:22.528211+0000","last_became_peered":"2024-09-14T23:56:22.528211+0000","last_unstale":"2024-09-14T23:57:00.060978+0000","last_undegraded":"2024-09-14T23:57:00.060978+0000","last_fullsized":"2024-09-14T23:57:00.060978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:14:11.495286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000459686,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"45'99","reported_seq":1722,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-14T23:56:58.052293+0000","last_change":"2024-09-14T23:56:34.401111+0000","last_active":"2024-09-14T23:56:58.052293+0000","last_peered":"2024-09-14T23:56:58.052293+0000","last_clean":"2024-09-14T23:56:58.052293+0000","last_became_active":"2024-09-14T23:56:32.401089+0000","last_became_peered":"2024-09-14T23:56:32.401089+0000","last_unstale":"2024-09-14T23:56:58.052293+0000","last_undegraded":"2024-09-14T23:56:58.052293+0000","last_fullsized":"2024-09-14T23:56:58.052293+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:28:22.884819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013033610000000001,"stat_sum":{"num_bytes":11740036,"num_objects":30,"num_object_clones":4,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":12,"num_read":1613,"num_read_kb":8104,"num_write":138,"num_write_kb":6612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-14T23:56:58.052383+0000","last_change":"2024-09-14T23:56:24.534510+0000","last_active":"2024-09-14T23:56:58.052383+0000","last_peered":"2024-09-14T23:56:58.052383+0000","last_clean":"2024-09-14T23:56:58.052383+0000","last_became_active":"2024-09-14T23:56:22.527611+0000","last_became_peered":"2024-09-14T23:56:22.527611+0000","last_unstale":"2024-09-14T23:56:58.052383+0000","last_undegraded":"2024-09-14T23:56:58.052383+0000","last_fullsized":"2024-09-14T23:56:58.052383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:15:45.606026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049916299999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"50'147","reported_seq":1596,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-14T23:57:00.169860+0000","last_change":"2024-09-14T23:57:00.065551+0000","last_active":"2024-09-14T23:57:00.169860+0000","last_peered":"2024-09-14T23:57:00.169860+0000","last_clean":"2024-09-14T23:57:00.169860+0000","last_became_active":"2024-09-14T23:57:00.064678+0000","last_became_peered":"2024-09-14T23:57:00.064678+0000","last_unstale":"2024-09-14T23:57:00.169860+0000","last_undegraded":"2024-09-14T23:57:00.169860+0000","last_fullsized":"2024-09-14T23:57:00.169860+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:08:17.341323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001272947,"stat_sum":{"num_bytes":6414955,"num_objects":40,"num_object_clones":8,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":17,"num_read":1708,"num_read_kb":4812,"num_write":154,"num_write_kb":5566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":3349587,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"50'97","reported_seq":835,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-14T23:57:00.201344+0000","last_change":"2024-09-14T23:56:34.404751+0000","last_active":"2024-09-14T23:57:00.201344+0000","last_peered":"2024-09-14T23:57:00.201344+0000","last_clean":"2024-09-14T23:57:00.201344+0000","last_became_active":"2024-09-14T23:56:32.397883+0000","last_became_peered":"2024-09-14T23:56:32.397883+0000","last_unstale":"2024-09-14T23:57:00.201344+0000","last_undegraded":"2024-09-14T23:57:00.201344+0000","last_fullsized":"2024-09-14T23:57:00.201344+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:25:37.352459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048955799999999996,"stat_sum":{"num_bytes":4034437,"num_objects":38,"num_object_clones":7,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":16,"num_read":728,"num_read_kb":2853,"num_write":85,"num_write_kb":2419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-14T23:56:58.051633+0000","last_change":"2024-09-14T23:56:57.052969+0000","last_active":"2024-09-14T23:56:58.051633+0000","last_peered":"2024-09-14T23:56:58.051633+0000","last_clean":"2024-09-14T23:56:58.051633+0000","last_became_active":"2024-09-14T23:56:57.050456+0000","last_became_peered":"2024-09-14T23:56:57.050456+0000","last_unstale":"2024-09-14T23:56:58.051633+0000","last_undegraded":"2024-09-14T23:56:58.051633+0000","last_fullsized":"2024-09-14T23:56:58.051633+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:57:07.644011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"44'112","reported_seq":2511,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-14T23:56:58.051795+0000","last_change":"2024-09-14T23:56:34.401159+0000","last_active":"2024-09-14T23:56:58.051795+0000","last_peered":"2024-09-14T23:56:58.051795+0000","last_clean":"2024-09-14T23:56:58.051795+0000","last_became_active":"2024-09-14T23:56:32.400278+0000","last_became_peered":"2024-09-14T23:56:32.400278+0000","last_unstale":"2024-09-14T23:56:58.051795+0000","last_undegraded":"2024-09-14T23:56:58.051795+0000","last_fullsized":"2024-09-14T23:56:58.051795+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:34:28.416821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082507200000000004,"stat_sum":{"num_bytes":10438397,"num_objects":21,"num_object_clones":2,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":5,"num_read":2402,"num_read_kb":6812,"num_write":108,"num_write_kb":1334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-14T23:56:57.044467+0000","last_change":"2024-09-14T23:56:24.909260+0000","last_active":"2024-09-14T23:56:57.044467+0000","last_peered":"2024-09-14T23:56:57.044467+0000","last_clean":"2024-09-14T23:56:57.044467+0000","last_became_active":"2024-09-14T23:56:22.822598+0000","last_became_peered":"2024-09-14T23:56:22.822598+0000","last_unstale":"2024-09-14T23:56:57.044467+0000","last_undegraded":"2024-09-14T23:56:57.044467+0000","last_fullsized":"2024-09-14T23:56:57.044467+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:32:32.568028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026863400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"44'273","reported_seq":2523,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-14T23:56:58.051094+0000","last_change":"2024-09-14T23:56:34.401366+0000","last_active":"2024-09-14T23:56:58.051094+0000","last_peered":"2024-09-14T23:56:58.051094+0000","last_clean":"2024-09-14T23:56:58.051094+0000","last_became_active":"2024-09-14T23:56:32.397551+0000","last_became_peered":"2024-09-14T23:56:32.397551+0000","last_unstale":"2024-09-14T23:56:58.051094+0000","last_undegraded":"2024-09-14T23:56:58.051094+0000","last_fullsized":"2024-09-14T23:56:58.051094+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":273,"log_dups_size":0,"ondisk_log_size":273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:13:25.661385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001418816,"stat_sum":{"num_bytes":5745114,"num_objects":33,"num_object_clones":4,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":18,"num_read":3579,"num_read_kb":8253,"num_write":418,"num_write_kb":1846,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":77,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-14T23:57:00.060919+0000","last_change":"2024-09-14T23:56:25.093090+0000","last_active":"2024-09-14T23:57:00.060919+0000","last_peered":"2024-09-14T23:57:00.060919+0000","last_clean":"2024-09-14T23:57:00.060919+0000","last_became_active":"2024-09-14T23:56:22.528115+0000","last_became_peered":"2024-09-14T23:56:22.528115+0000","last_unstale":"2024-09-14T23:57:00.060919+0000","last_undegraded":"2024-09-14T23:57:00.060919+0000","last_fullsized":"2024-09-14T23:57:00.060919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:57:36.610730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036520999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"39'109","reported_seq":1225,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-14T23:57:00.065382+0000","last_change":"2024-09-14T23:57:00.065382+0000","last_active":"2024-09-14T23:57:00.065382+0000","last_peered":"2024-09-14T23:57:00.065382+0000","last_clean":"2024-09-14T23:57:00.065382+0000","last_became_active":"2024-09-14T23:57:00.064640+0000","last_became_peered":"2024-09-14T23:57:00.064640+0000","last_unstale":"2024-09-14T23:57:00.065382+0000","last_undegraded":"2024-09-14T23:57:00.065382+0000","last_fullsized":"2024-09-14T23:57:00.065382+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:44:39.412610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073009599999999996,"stat_sum":{"num_bytes":5469545,"num_objects":38,"num_object_clones":9,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":16,"num_read":830,"num_read_kb":3417,"num_write":105,"num_write_kb":2128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":3781706,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-14T23:56:57.054999+0000","last_change":"2024-09-14T23:56:57.054999+0000","last_active":"2024-09-14T23:56:57.054999+0000","last_peered":"2024-09-14T23:56:57.054999+0000","last_clean":"2024-09-14T23:56:57.054999+0000","last_became_active":"2024-09-14T23:56:57.050729+0000","last_became_peered":"2024-09-14T23:56:57.050729+0000","last_unstale":"2024-09-14T23:56:57.054999+0000","last_undegraded":"2024-09-14T23:56:57.054999+0000","last_fullsized":"2024-09-14T23:56:57.054999+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:23:11.727903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":228,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-14T23:57:00.061023+0000","last_change":"2024-09-14T23:56:15.332936+0000","last_active":"2024-09-14T23:57:00.061023+0000","last_peered":"2024-09-14T23:57:00.061023+0000","last_clean":"2024-09-14T23:57:00.061023+0000","last_became_active":"2024-09-14T23:56:15.331994+0000","last_became_peered":"2024-09-14T23:56:15.331994+0000","last_unstale":"2024-09-14T23:57:00.061023+0000","last_undegraded":"2024-09-14T23:57:00.061023+0000","last_fullsized":"2024-09-14T23:57:00.061023+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:14.322882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:14.322882+0000","last_clean_scrub_stamp":"2024-09-14T23:56:14.322882+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:36:12.639000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"50'88","reported_seq":2353,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-14T23:57:00.164496+0000","last_change":"2024-09-14T23:56:34.415037+0000","last_active":"2024-09-14T23:57:00.164496+0000","last_peered":"2024-09-14T23:57:00.164496+0000","last_clean":"2024-09-14T23:57:00.164496+0000","last_became_active":"2024-09-14T23:56:32.403572+0000","last_became_peered":"2024-09-14T23:56:32.403572+0000","last_unstale":"2024-09-14T23:57:00.164496+0000","last_undegraded":"2024-09-14T23:57:00.164496+0000","last_fullsized":"2024-09-14T23:57:00.164496+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:57:08.597280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052643700000000004,"stat_sum":{"num_bytes":4480383,"num_objects":25,"num_object_clones":5,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":11,"num_read":2305,"num_read_kb":8025,"num_write":94,"num_write_kb":3047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"48'55","reported_seq":453,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-14T23:57:00.197722+0000","last_change":"2024-09-14T23:56:34.404977+0000","last_active":"2024-09-14T23:57:00.197722+0000","last_peered":"2024-09-14T23:57:00.197722+0000","last_clean":"2024-09-14T23:57:00.197722+0000","last_became_active":"2024-09-14T23:56:32.400017+0000","last_became_peered":"2024-09-14T23:56:32.400017+0000","last_unstale":"2024-09-14T23:57:00.197722+0000","last_undegraded":"2024-09-14T23:57:00.197722+0000","last_fullsized":"2024-09-14T23:57:00.197722+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:16:34.856558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068884800000000004,"stat_sum":{"num_bytes":5140302,"num_objects":23,"num_object_clones":3,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":8,"num_read":344,"num_read_kb":3046,"num_write":62,"num_write_kb":2274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"50'64","reported_seq":1595,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-14T23:57:00.122656+0000","last_change":"2024-09-14T23:56:34.498020+0000","last_active":"2024-09-14T23:57:00.122656+0000","last_peered":"2024-09-14T23:57:00.122656+0000","last_clean":"2024-09-14T23:57:00.122656+0000","last_became_active":"2024-09-14T23:56:32.402769+0000","last_became_peered":"2024-09-14T23:56:32.402769+0000","last_unstale":"2024-09-14T23:57:00.122656+0000","last_undegraded":"2024-09-14T23:57:00.122656+0000","last_fullsized":"2024-09-14T23:57:00.122656+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:12:30.741482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035483999999999998,"stat_sum":{"num_bytes":8893168,"num_objects":24,"num_object_clones":2,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":10,"num_read":1494,"num_read_kb":9719,"num_write":72,"num_write_kb":3110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"50'304","reported_seq":3642,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-14T23:57:00.214807+0000","last_change":"2024-09-14T23:56:34.415414+0000","last_active":"2024-09-14T23:57:00.214807+0000","last_peered":"2024-09-14T23:57:00.214807+0000","last_clean":"2024-09-14T23:57:00.214807+0000","last_became_active":"2024-09-14T23:56:32.403231+0000","last_became_peered":"2024-09-14T23:56:32.403231+0000","last_unstale":"2024-09-14T23:57:00.214807+0000","last_undegraded":"2024-09-14T23:57:00.214807+0000","last_fullsized":"2024-09-14T23:57:00.214807+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":304,"log_dups_size":0,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:29:28.841482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072791000000000002,"stat_sum":{"num_bytes":13050411,"num_objects":30,"num_object_clones":6,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":10,"num_read":4373,"num_read_kb":11820,"num_write":460,"num_write_kb":4992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"50'160","reported_seq":2031,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-14T23:57:00.205210+0000","last_change":"2024-09-14T23:57:00.066551+0000","last_active":"2024-09-14T23:57:00.205210+0000","last_peered":"2024-09-14T23:57:00.205210+0000","last_clean":"2024-09-14T23:57:00.205210+0000","last_became_active":"2024-09-14T23:57:00.065790+0000","last_became_peered":"2024-09-14T23:57:00.065790+0000","last_unstale":"2024-09-14T23:57:00.205210+0000","last_undegraded":"2024-09-14T23:57:00.205210+0000","last_fullsized":"2024-09-14T23:57:00.205210+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:19:52.414751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092638099999999999,"stat_sum":{"num_bytes":15463591,"num_objects":25,"num_object_clones":5,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":8,"num_read":1287,"num_read_kb":7323,"num_write":255,"num_write_kb":4717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":7912941,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"45'61","reported_seq":325,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-14T23:57:00.060944+0000","last_change":"2024-09-14T23:56:57.101334+0000","last_active":"2024-09-14T23:57:00.060944+0000","last_peered":"2024-09-14T23:57:00.060944+0000","last_clean":"2024-09-14T23:57:00.060944+0000","last_became_active":"2024-09-14T23:56:57.052072+0000","last_became_peered":"2024-09-14T23:56:57.052072+0000","last_unstale":"2024-09-14T23:57:00.060944+0000","last_undegraded":"2024-09-14T23:57:00.060944+0000","last_fullsized":"2024-09-14T23:57:00.060944+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:04:36.710572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057774300000000003,"stat_sum":{"num_bytes":2154533,"num_objects":19,"num_object_clones":2,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":9,"num_read":71,"num_read_kb":854,"num_write":57,"num_write_kb":1026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":1496914,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"48'87","reported_seq":1236,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-14T23:57:00.208086+0000","last_change":"2024-09-14T23:56:34.399894+0000","last_active":"2024-09-14T23:57:00.208086+0000","last_peered":"2024-09-14T23:57:00.208086+0000","last_clean":"2024-09-14T23:57:00.208086+0000","last_became_active":"2024-09-14T23:56:32.406251+0000","last_became_peered":"2024-09-14T23:56:32.406251+0000","last_unstale":"2024-09-14T23:57:00.208086+0000","last_undegraded":"2024-09-14T23:57:00.208086+0000","last_fullsized":"2024-09-14T23:57:00.208086+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:09:22.410305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073115799999999998,"stat_sum":{"num_bytes":11554097,"num_objects":40,"num_object_clones":7,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":20,"num_read":1126,"num_read_kb":6237,"num_write":89,"num_write_kb":2568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"48'143","reported_seq":1878,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-14T23:57:00.203239+0000","last_change":"2024-09-14T23:56:34.399966+0000","last_active":"2024-09-14T23:57:00.203239+0000","last_peered":"2024-09-14T23:57:00.203239+0000","last_clean":"2024-09-14T23:57:00.203239+0000","last_became_active":"2024-09-14T23:56:32.407109+0000","last_became_peered":"2024-09-14T23:56:32.407109+0000","last_unstale":"2024-09-14T23:57:00.203239+0000","last_undegraded":"2024-09-14T23:57:00.203239+0000","last_fullsized":"2024-09-14T23:57:00.203239+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:06:53.607337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072766900000000002,"stat_sum":{"num_bytes":16803862,"num_objects":42,"num_object_clones":8,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":19,"num_read":2582,"num_read_kb":6075,"num_write":124,"num_write_kb":2344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"42'146","reported_seq":414,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-14T23:57:00.122414+0000","last_change":"2024-09-14T23:56:34.498033+0000","last_active":"2024-09-14T23:57:00.122414+0000","last_peered":"2024-09-14T23:57:00.122414+0000","last_clean":"2024-09-14T23:57:00.122414+0000","last_became_active":"2024-09-14T23:56:32.402973+0000","last_became_peered":"2024-09-14T23:56:32.402973+0000","last_unstale":"2024-09-14T23:57:00.122414+0000","last_undegraded":"2024-09-14T23:57:00.122414+0000","last_fullsized":"2024-09-14T23:57:00.122414+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:12:39.512158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035072199999999998,"stat_sum":{"num_bytes":2744363,"num_objects":30,"num_object_clones":5,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":15,"num_read":572,"num_read_kb":2578,"num_write":205,"num_write_kb":1612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"50'91","reported_seq":514,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-14T23:57:00.178069+0000","last_change":"2024-09-14T23:56:34.399387+0000","last_active":"2024-09-14T23:57:00.178069+0000","last_peered":"2024-09-14T23:57:00.178069+0000","last_clean":"2024-09-14T23:57:00.178069+0000","last_became_active":"2024-09-14T23:56:32.405457+0000","last_became_peered":"2024-09-14T23:56:32.405457+0000","last_unstale":"2024-09-14T23:57:00.178069+0000","last_undegraded":"2024-09-14T23:57:00.178069+0000","last_fullsized":"2024-09-14T23:57:00.178069+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:35:45.282459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048125700000000002,"stat_sum":{"num_bytes":6368142,"num_objects":33,"num_object_clones":6,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":13,"num_read":377,"num_read_kb":4350,"num_write":94,"num_write_kb":2874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"44'57","reported_seq":718,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-14T23:56:58.051161+0000","last_change":"2024-09-14T23:56:34.402592+0000","last_active":"2024-09-14T23:56:58.051161+0000","last_peered":"2024-09-14T23:56:58.051161+0000","last_clean":"2024-09-14T23:56:58.051161+0000","last_became_active":"2024-09-14T23:56:32.402782+0000","last_became_peered":"2024-09-14T23:56:32.402782+0000","last_unstale":"2024-09-14T23:56:58.051161+0000","last_undegraded":"2024-09-14T23:56:58.051161+0000","last_fullsized":"2024-09-14T23:56:58.051161+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:40:00.601906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001446697,"stat_sum":{"num_bytes":3851398,"num_objects":21,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":11,"num_read":613,"num_read_kb":2595,"num_write":51,"num_write_kb":1196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"48'62","reported_seq":470,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-14T23:57:00.205680+0000","last_change":"2024-09-14T23:56:34.399629+0000","last_active":"2024-09-14T23:57:00.205680+0000","last_peered":"2024-09-14T23:57:00.205680+0000","last_clean":"2024-09-14T23:57:00.205680+0000","last_became_active":"2024-09-14T23:56:32.407220+0000","last_became_peered":"2024-09-14T23:56:32.407220+0000","last_unstale":"2024-09-14T23:57:00.205680+0000","last_undegraded":"2024-09-14T23:57:00.205680+0000","last_fullsized":"2024-09-14T23:57:00.205680+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:28:21.243339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062462200000000002,"stat_sum":{"num_bytes":3137817,"num_objects":20,"num_object_clones":4,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":10,"num_read":367,"num_read_kb":1552,"num_write":57,"num_write_kb":821,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":265238411,"num_objects":901,"num_object_clones":147,"num_object_copies":1802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":901,"num_whiteouts":382,"num_read":54920,"num_read_kb":197251,"num_write":5405,"num_write_kb":85653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":627,"num_bytes_recovered":25850156,"num_keys_recovered":109,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":19161088,"data_stored":155036389,"data_compressed":11513331,"data_compressed_allocated":13574144,"data_compressed_original":150410946,"omap_allocated":0,"internal_metadata":0},"log_size":4322,"ondisk_log_size":4322,"up":64,"acting":62,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64594,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509454,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38784,"kb_used_data":3436,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333056,"statfs":{"total":96636764160,"available":96597049344,"internally_reserved":0,"allocated":3518464,"data_stored":23790268,"data_compressed":1831638,"data_compressed_allocated":2088960,"data_compressed_original":22825095,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509453,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33316,"kb_used_data":2124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338524,"statfs":{"total":96636764160,"available":96602648576,"internally_reserved":0,"allocated":2174976,"data_stored":12315326,"data_compressed":752400,"data_compressed_allocated":1040384,"data_compressed_original":11575296,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509452,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38052,"kb_used_data":2704,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333788,"statfs":{"total":96636764160,"available":96597798912,"internally_reserved":0,"allocated":2768896,"data_stored":16665942,"data_compressed":1218978,"data_compressed_allocated":1425408,"data_compressed_original":15790080,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509454,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43348,"kb_used_data":3840,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94328492,"statfs":{"total":96636764160,"available":96592375808,"internally_reserved":0,"allocated":3932160,"data_stored":24010913,"data_compressed":1880704,"data_compressed_allocated":2318336,"data_compressed_original":22867136,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509452,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43580,"kb_used_data":4072,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94328260,"statfs":{"total":96636764160,"available":96592138240,"internally_reserved":0,"allocated":4169728,"data_stored":33515535,"data_compressed":2392583,"data_compressed_allocated":2674688,"data_compressed_original":32525783,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542156,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38228,"kb_used_data":2872,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333612,"statfs":{"total":96636764160,"available":96597618688,"internally_reserved":0,"allocated":2940928,"data_stored":18453243,"data_compressed":1361104,"data_compressed_allocated":1716224,"data_compressed_original":17635140,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542155,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42448,"kb_used_data":2940,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329392,"statfs":{"total":96636764160,"available":96593297408,"internally_reserved":0,"allocated":3010560,"data_stored":19369747,"data_compressed":1303425,"data_compressed_allocated":1765376,"data_compressed_original":18529376,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37160,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334680,"statfs":{"total":96636764160,"available":96598712320,"internally_reserved":0,"allocated":1851392,"data_stored":9275588,"data_compressed":730300,"data_compressed_allocated":954368,"data_compressed_original":8761344,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32297,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32297,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2416640,"data_stored":18550424,"data_compressed":1271128,"data_compressed_allocated":1720320,"data_compressed_original":17955936,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2408448,"data_stored":18224269,"data_compressed":1361104,"data_compressed_allocated":1716224,"data_compressed_original":17635140,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1335296,"data_stored":9052886,"data_compressed":730300,"data_compressed_allocated":954368,"data_compressed_original":8761344,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3489792,"data_stored":34504531,"data_compressed":2423152,"data_compressed_allocated":2592768,"data_compressed_original":33791447,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":3489792,"data_stored":26391056,"data_compressed":2049744,"data_compressed_allocated":2424832,"data_compressed_original":25488576,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2301952,"data_stored":18521740,"data_compressed":1360628,"data_compressed_allocated":1515520,"data_compressed_original":17887232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":770048,"data_stored":5784342,"data_compressed":439706,"data_compressed_allocated":557056,"data_compressed_original":5591040,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2949120,"data_stored":24007141,"data_compressed":1877569,"data_compressed_allocated":2093056,"data_compressed_original":23300231,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T23:57:01.994 INFO:tasks.thrashosds.thrasher:Setting 3.7 to [0, 3, 1, 5] 2024-09-14T23:57:01.994 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.7', '0', '3', '1', '5'] 2024-09-14T23:57:01.994 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.7 0 3 1 5 2024-09-14T23:57:01.996 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:01.994+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:02.092 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:02.093 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:57:02.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:57:02.093 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:57:02.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.093 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:3578 3.4 e29/23 3:2298d860:::rbd_data.1118de1168b.0000000000000030:head v 29'15, mlcod=29'13)", 2024-09-14T23:57:02.093 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.036455+0000", 2024-09-14T23:57:02.093 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.053707073999998, 2024-09-14T23:57:02.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019449440000000001, 2024-09-14T23:57:02.093 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:02.093 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:02.093 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:02.094 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:02.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:02.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036455+0000", 2024-09-14T23:57:02.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:02.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036455+0000", 2024-09-14T23:57:02.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:02.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036456+0000", 2024-09-14T23:57:02.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0999999999999997e-07 2024-09-14T23:57:02.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:02.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036463+0000", 2024-09-14T23:57:02.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2200000000000003e-06 2024-09-14T23:57:02.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:02.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036478+0000", 2024-09-14T23:57:02.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4977000000000001e-05 2024-09-14T23:57:02.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:02.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036485+0000", 2024-09-14T23:57:02.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0559999999999999e-06 2024-09-14T23:57:02.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:02.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036693+0000", 2024-09-14T23:57:02.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000208051 2024-09-14T23:57:02.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:02.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036715+0000", 2024-09-14T23:57:02.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1330000000000001e-05 2024-09-14T23:57:02.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:02.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.038356+0000", 2024-09-14T23:57:02.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001641586 2024-09-14T23:57:02.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:02.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.038400+0000", 2024-09-14T23:57:02.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3914e-05 2024-09-14T23:57:02.098 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.098 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:02.098 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.098 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:3579 3.4 e29/23 3:27290fe6:::rbd_data.1118de1168b.0000000000000032:head v 29'16, mlcod=29'13)", 2024-09-14T23:57:02.098 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.037085+0000", 2024-09-14T23:57:02.098 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.053077483999999, 2024-09-14T23:57:02.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0020019220000000002, 2024-09-14T23:57:02.098 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:02.098 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:02.098 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:02.098 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:02.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:02.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037085+0000", 2024-09-14T23:57:02.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:02.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037085+0000", 2024-09-14T23:57:02.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:02.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037085+0000", 2024-09-14T23:57:02.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5499999999999997e-07 2024-09-14T23:57:02.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:02.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037091+0000", 2024-09-14T23:57:02.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7869999999999998e-06 2024-09-14T23:57:02.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:02.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037102+0000", 2024-09-14T23:57:02.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.031e-05 2024-09-14T23:57:02.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:02.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037116+0000", 2024-09-14T23:57:02.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4119e-05 2024-09-14T23:57:02.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:02.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037219+0000", 2024-09-14T23:57:02.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010372799999999999 2024-09-14T23:57:02.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:02.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037239+0000", 2024-09-14T23:57:02.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9993e-05 2024-09-14T23:57:02.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:02.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.039046+0000", 2024-09-14T23:57:02.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001806666 2024-09-14T23:57:02.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:02.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.039087+0000", 2024-09-14T23:57:02.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0564e-05 2024-09-14T23:57:02.103 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.103 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:02.103 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.103 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:3584 3.4 e29/23 3:2779453d:::rbd_data.1118de1168b.0000000000000034:head v 29'17, mlcod=29'13)", 2024-09-14T23:57:02.103 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.037588+0000", 2024-09-14T23:57:02.103 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.052574445000001, 2024-09-14T23:57:02.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001526828, 2024-09-14T23:57:02.103 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:02.103 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:02.103 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:02.104 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:02.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:02.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037588+0000", 2024-09-14T23:57:02.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:02.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037588+0000", 2024-09-14T23:57:02.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:02.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037588+0000", 2024-09-14T23:57:02.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1200000000000002e-07 2024-09-14T23:57:02.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:02.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037595+0000", 2024-09-14T23:57:02.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7580000000000003e-06 2024-09-14T23:57:02.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:02.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037609+0000", 2024-09-14T23:57:02.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4037e-05 2024-09-14T23:57:02.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:02.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037616+0000", 2024-09-14T23:57:02.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0509999999999999e-06 2024-09-14T23:57:02.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:02.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037787+0000", 2024-09-14T23:57:02.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017084899999999999 2024-09-14T23:57:02.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:02.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037806+0000", 2024-09-14T23:57:02.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9159000000000001e-05 2024-09-14T23:57:02.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:02.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.039089+0000", 2024-09-14T23:57:02.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012823909999999999 2024-09-14T23:57:02.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:02.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.039115+0000", 2024-09-14T23:57:02.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5871e-05 2024-09-14T23:57:02.108 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.108 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:02.109 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.109 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:5280 3.0 e29/23 3:0760f535:::rbd_data.1118de1168b.000000000000009a:head v 29'11, mlcod=29'9)", 2024-09-14T23:57:02.109 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.262702+0000", 2024-09-14T23:57:02.109 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.827460065, 2024-09-14T23:57:02.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001433122, 2024-09-14T23:57:02.109 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:02.109 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:02.109 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:02.109 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:02.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:02.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262702+0000", 2024-09-14T23:57:02.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:02.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262702+0000", 2024-09-14T23:57:02.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:02.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262704+0000", 2024-09-14T23:57:02.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.384e-06 2024-09-14T23:57:02.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:02.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262791+0000", 2024-09-14T23:57:02.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7264999999999999e-05 2024-09-14T23:57:02.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:02.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262813+0000", 2024-09-14T23:57:02.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.173e-05 2024-09-14T23:57:02.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:02.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262822+0000", 2024-09-14T23:57:02.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5929999999999995e-06 2024-09-14T23:57:02.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:02.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262886+0000", 2024-09-14T23:57:02.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4052000000000003e-05 2024-09-14T23:57:02.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:02.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262920+0000", 2024-09-14T23:57:02.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.366e-05 2024-09-14T23:57:02.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:02.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.264095+0000", 2024-09-14T23:57:02.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001175178 2024-09-14T23:57:02.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:02.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.264135+0000", 2024-09-14T23:57:02.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0259999999999997e-05 2024-09-14T23:57:02.114 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.114 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:02.114 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.115 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:5281 3.0 e29/23 3:0760f535:::rbd_data.1118de1168b.000000000000009a:head v 29'13, mlcod=29'9)", 2024-09-14T23:57:02.115 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.263503+0000", 2024-09-14T23:57:02.115 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.826659200999998, 2024-09-14T23:57:02.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012834910000000001, 2024-09-14T23:57:02.115 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:02.115 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:02.115 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:02.115 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:02.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:02.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.263503+0000", 2024-09-14T23:57:02.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:02.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.263503+0000", 2024-09-14T23:57:02.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:02.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.263504+0000", 2024-09-14T23:57:02.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4600000000000004e-07 2024-09-14T23:57:02.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:02.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.263510+0000", 2024-09-14T23:57:02.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5950000000000004e-06 2024-09-14T23:57:02.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:02.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.263522+0000", 2024-09-14T23:57:02.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1289e-05 2024-09-14T23:57:02.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:02.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.263529+0000", 2024-09-14T23:57:02.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.951e-06 2024-09-14T23:57:02.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:02.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.263656+0000", 2024-09-14T23:57:02.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012731099999999999 2024-09-14T23:57:02.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:02.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.263675+0000", 2024-09-14T23:57:02.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9147000000000002e-05 2024-09-14T23:57:02.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:02.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.264746+0000", 2024-09-14T23:57:02.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001070997 2024-09-14T23:57:02.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:02.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.264787+0000", 2024-09-14T23:57:02.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0555e-05 2024-09-14T23:57:02.120 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.120 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:02.120 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.120 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:17221 3.4 e33/23 3:225c4d41:::rbd_data.11187be34cc8.0000000000000001:head v 33'50, mlcod=32'48)", 2024-09-14T23:57:02.120 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:42.983242+0000", 2024-09-14T23:57:02.121 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.106920684999999, 2024-09-14T23:57:02.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001313032, 2024-09-14T23:57:02.121 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:02.121 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:02.121 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:02.121 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:02.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:02.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.983242+0000", 2024-09-14T23:57:02.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:02.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.983242+0000", 2024-09-14T23:57:02.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:02.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.983243+0000", 2024-09-14T23:57:02.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.102e-06 2024-09-14T23:57:02.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:02.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.983256+0000", 2024-09-14T23:57:02.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2897e-05 2024-09-14T23:57:02.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:02.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.983277+0000", 2024-09-14T23:57:02.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1566999999999999e-05 2024-09-14T23:57:02.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:02.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.983288+0000", 2024-09-14T23:57:02.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0457e-05 2024-09-14T23:57:02.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:02.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.983355+0000", 2024-09-14T23:57:02.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7337999999999998e-05 2024-09-14T23:57:02.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:02.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.983422+0000", 2024-09-14T23:57:02.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6716000000000003e-05 2024-09-14T23:57:02.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:02.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.984515+0000", 2024-09-14T23:57:02.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001093761 2024-09-14T23:57:02.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:02.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.984555+0000", 2024-09-14T23:57:02.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9193999999999998e-05 2024-09-14T23:57:02.126 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.126 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:02.126 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.127 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:17453 3.0 e33/23 3:020fb8a1:::rbd_data.11187be34cc8.0000000000000005:head v 33'46, mlcod=33'43)", 2024-09-14T23:57:02.127 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:43.265507+0000", 2024-09-14T23:57:02.127 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.824654837000001, 2024-09-14T23:57:02.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013288880000000001, 2024-09-14T23:57:02.127 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:02.127 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:02.127 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:02.127 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:02.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:02.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.265507+0000", 2024-09-14T23:57:02.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:02.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.265507+0000", 2024-09-14T23:57:02.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:02.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.265508+0000", 2024-09-14T23:57:02.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4400000000000001e-07 2024-09-14T23:57:02.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:02.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.265513+0000", 2024-09-14T23:57:02.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.887e-06 2024-09-14T23:57:02.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:02.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.265520+0000", 2024-09-14T23:57:02.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8249999999999999e-06 2024-09-14T23:57:02.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:02.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.265524+0000", 2024-09-14T23:57:02.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8130000000000003e-06 2024-09-14T23:57:02.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:02.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.265727+0000", 2024-09-14T23:57:02.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000202518 2024-09-14T23:57:02.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:02.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.265744+0000", 2024-09-14T23:57:02.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.753e-05 2024-09-14T23:57:02.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:02.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.266802+0000", 2024-09-14T23:57:02.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001057992 2024-09-14T23:57:02.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:02.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.266836+0000", 2024-09-14T23:57:02.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3878999999999999e-05 2024-09-14T23:57:02.133 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.133 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:02.133 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.133 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:17657 3.0 e33/23 3:05bad47a:::rbd_data.11187be34cc8.0000000000000003:head v 33'53, mlcod=33'51)", 2024-09-14T23:57:02.133 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:43.493115+0000", 2024-09-14T23:57:02.133 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.597046752000001, 2024-09-14T23:57:02.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0014551950000000001, 2024-09-14T23:57:02.134 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:02.134 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:02.134 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:02.134 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:02.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:02.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.493115+0000", 2024-09-14T23:57:02.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:02.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.493115+0000", 2024-09-14T23:57:02.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:02.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.493116+0000", 2024-09-14T23:57:02.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7899999999999998e-07 2024-09-14T23:57:02.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:02.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.493121+0000", 2024-09-14T23:57:02.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3959999999999997e-06 2024-09-14T23:57:02.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:02.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.493131+0000", 2024-09-14T23:57:02.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4900000000000006e-06 2024-09-14T23:57:02.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:02.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.493138+0000", 2024-09-14T23:57:02.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6730000000000003e-06 2024-09-14T23:57:02.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:02.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.493263+0000", 2024-09-14T23:57:02.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000125468 2024-09-14T23:57:02.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:02.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.493282+0000", 2024-09-14T23:57:02.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8831999999999999e-05 2024-09-14T23:57:02.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:02.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.494528+0000", 2024-09-14T23:57:02.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012456990000000001 2024-09-14T23:57:02.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:02.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.494571+0000", 2024-09-14T23:57:02.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3058e-05 2024-09-14T23:57:02.139 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.139 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:02.139 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.139 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:17997 3.0 e33/23 3:05bad47a:::rbd_data.11187be34cc8.0000000000000003:head v 33'68, mlcod=33'66)", 2024-09-14T23:57:02.139 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:43.822910+0000", 2024-09-14T23:57:02.139 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.267252209999999, 2024-09-14T23:57:02.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001512535, 2024-09-14T23:57:02.140 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:02.140 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:02.140 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:02.140 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:02.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:02.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.822910+0000", 2024-09-14T23:57:02.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:02.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.822910+0000", 2024-09-14T23:57:02.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:02.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.822910+0000", 2024-09-14T23:57:02.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9999999999999998e-07 2024-09-14T23:57:02.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:02.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.822915+0000", 2024-09-14T23:57:02.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7790000000000003e-06 2024-09-14T23:57:02.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:02.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.822923+0000", 2024-09-14T23:57:02.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2300000000000008e-06 2024-09-14T23:57:02.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:02.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.822929+0000", 2024-09-14T23:57:02.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5219999999999998e-06 2024-09-14T23:57:02.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:02.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.823001+0000", 2024-09-14T23:57:02.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1903000000000006e-05 2024-09-14T23:57:02.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:02.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.823021+0000", 2024-09-14T23:57:02.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0129e-05 2024-09-14T23:57:02.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:02.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.824376+0000", 2024-09-14T23:57:02.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013548550000000001 2024-09-14T23:57:02.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:02.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.824423+0000", 2024-09-14T23:57:02.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6717e-05 2024-09-14T23:57:02.145 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.145 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:02.145 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.146 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:17999 3.0 e33/23 3:020fb8a1:::rbd_data.11187be34cc8.0000000000000005:head v 33'69, mlcod=33'66)", 2024-09-14T23:57:02.146 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:43.823649+0000", 2024-09-14T23:57:02.146 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.266512945999999, 2024-09-14T23:57:02.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013396320000000001, 2024-09-14T23:57:02.146 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:02.146 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:02.146 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:02.146 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:02.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:02.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.823649+0000", 2024-09-14T23:57:02.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:02.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.823649+0000", 2024-09-14T23:57:02.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:02.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.823650+0000", 2024-09-14T23:57:02.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7499999999999999e-07 2024-09-14T23:57:02.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:02.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.823656+0000", 2024-09-14T23:57:02.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1500000000000004e-06 2024-09-14T23:57:02.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:02.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.823667+0000", 2024-09-14T23:57:02.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0788000000000001e-05 2024-09-14T23:57:02.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:02.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.823673+0000", 2024-09-14T23:57:02.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4099999999999996e-06 2024-09-14T23:57:02.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:02.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.823722+0000", 2024-09-14T23:57:02.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8733000000000002e-05 2024-09-14T23:57:02.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:02.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.823741+0000", 2024-09-14T23:57:02.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9156000000000001e-05 2024-09-14T23:57:02.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:02.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.824961+0000", 2024-09-14T23:57:02.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001219901 2024-09-14T23:57:02.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:02.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.824989+0000", 2024-09-14T23:57:02.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7619000000000001e-05 2024-09-14T23:57:02.151 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.151 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:02.151 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.152 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:18030 3.4 e33/23 3:225c4d41:::rbd_data.11187be34cc8.0000000000000001:head v 33'63, mlcod=33'61)", 2024-09-14T23:57:02.152 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:43.852748+0000", 2024-09-14T23:57:02.152 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.237414386000001, 2024-09-14T23:57:02.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001530696, 2024-09-14T23:57:02.152 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:02.152 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:02.152 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:02.152 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:02.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:02.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.852748+0000", 2024-09-14T23:57:02.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:02.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.852748+0000", 2024-09-14T23:57:02.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:02.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.852749+0000", 2024-09-14T23:57:02.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.164e-06 2024-09-14T23:57:02.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:02.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.852759+0000", 2024-09-14T23:57:02.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0265000000000001e-05 2024-09-14T23:57:02.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:02.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.852779+0000", 2024-09-14T23:57:02.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9247999999999999e-05 2024-09-14T23:57:02.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:02.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.852788+0000", 2024-09-14T23:57:02.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3260000000000002e-06 2024-09-14T23:57:02.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:02.156 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.853121+0000", 2024-09-14T23:57:02.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00033336699999999999 2024-09-14T23:57:02.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.156 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:02.156 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.853142+0000", 2024-09-14T23:57:02.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0500999999999999e-05 2024-09-14T23:57:02.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.156 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:02.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.854240+0000", 2024-09-14T23:57:02.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001098665 2024-09-14T23:57:02.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.157 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:02.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.854279+0000", 2024-09-14T23:57:02.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8160000000000001e-05 2024-09-14T23:57:02.157 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.157 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:02.157 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.158 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:18260 3.4 e34/23 3:225c4d41:::rbd_data.11187be34cc8.0000000000000001:head v 34'65, mlcod=33'63)", 2024-09-14T23:57:02.158 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:44.113710+0000", 2024-09-14T23:57:02.158 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 17.976452201000001, 2024-09-14T23:57:02.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001257174, 2024-09-14T23:57:02.158 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:02.158 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:02.158 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:02.158 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:02.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:02.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.113710+0000", 2024-09-14T23:57:02.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:02.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.113710+0000", 2024-09-14T23:57:02.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:02.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.113710+0000", 2024-09-14T23:57:02.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4900000000000001e-07 2024-09-14T23:57:02.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:02.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.113716+0000", 2024-09-14T23:57:02.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5110000000000003e-06 2024-09-14T23:57:02.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.161 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:02.161 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.113724+0000", 2024-09-14T23:57:02.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1499999999999999e-06 2024-09-14T23:57:02.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.161 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:02.161 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.113730+0000", 2024-09-14T23:57:02.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6919999999999999e-06 2024-09-14T23:57:02.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:02.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.113960+0000", 2024-09-14T23:57:02.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00023030699999999999 2024-09-14T23:57:02.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:02.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.113986+0000", 2024-09-14T23:57:02.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6330999999999999e-05 2024-09-14T23:57:02.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:02.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.114923+0000", 2024-09-14T23:57:02.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00093685100000000005 2024-09-14T23:57:02.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:02.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.114967+0000", 2024-09-14T23:57:02.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3883e-05 2024-09-14T23:57:02.164 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.164 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:02.164 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.164 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:18356 3.0 e34/23 3:05bad47a:::rbd_data.11187be34cc8.0000000000000003:head v 34'84, mlcod=34'82)", 2024-09-14T23:57:02.164 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:44.206422+0000", 2024-09-14T23:57:02.164 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 17.883739793, 2024-09-14T23:57:02.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001310603, 2024-09-14T23:57:02.164 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:02.164 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:02.165 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:02.165 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:02.165 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.165 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:02.165 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.206422+0000", 2024-09-14T23:57:02.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.165 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.165 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:02.165 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.206422+0000", 2024-09-14T23:57:02.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:02.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.206424+0000", 2024-09-14T23:57:02.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3e-06 2024-09-14T23:57:02.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:02.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.206437+0000", 2024-09-14T23:57:02.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3040999999999999e-05 2024-09-14T23:57:02.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:02.167 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.206455+0000", 2024-09-14T23:57:02.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8369e-05 2024-09-14T23:57:02.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:02.167 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.206466+0000", 2024-09-14T23:57:02.168 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0767999999999999e-05 2024-09-14T23:57:02.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.168 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:02.168 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.206551+0000", 2024-09-14T23:57:02.168 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4858999999999995e-05 2024-09-14T23:57:02.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.168 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:02.168 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.206574+0000", 2024-09-14T23:57:02.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2741000000000001e-05 2024-09-14T23:57:02.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.169 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:02.169 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.207690+0000", 2024-09-14T23:57:02.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011167880000000001 2024-09-14T23:57:02.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.169 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:02.169 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.207733+0000", 2024-09-14T23:57:02.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2737000000000001e-05 2024-09-14T23:57:02.170 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.170 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:02.170 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.170 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.170 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:18354 3.4 e34/23 3:225c4d41:::rbd_data.11187be34cc8.0000000000000001:head v 34'67, mlcod=34'65)", 2024-09-14T23:57:02.170 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:44.206794+0000", 2024-09-14T23:57:02.170 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 17.883367807999999, 2024-09-14T23:57:02.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013689819999999999, 2024-09-14T23:57:02.170 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:02.171 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:02.171 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:02.171 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:02.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:02.171 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.206794+0000", 2024-09-14T23:57:02.171 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:02.172 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.206794+0000", 2024-09-14T23:57:02.172 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.172 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:02.172 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.206795+0000", 2024-09-14T23:57:02.172 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5400000000000002e-07 2024-09-14T23:57:02.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.172 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:02.172 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.206800+0000", 2024-09-14T23:57:02.173 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0479999999999998e-06 2024-09-14T23:57:02.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.173 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.173 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:02.173 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.206808+0000", 2024-09-14T23:57:02.173 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8420000000000005e-06 2024-09-14T23:57:02.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.173 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.173 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:02.173 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.206814+0000", 2024-09-14T23:57:02.174 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8640000000000001e-06 2024-09-14T23:57:02.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.174 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.174 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:02.174 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.206988+0000", 2024-09-14T23:57:02.174 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000174599 2024-09-14T23:57:02.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.174 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.174 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:02.174 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.207007+0000", 2024-09-14T23:57:02.174 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8695e-05 2024-09-14T23:57:02.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.175 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:02.175 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.208109+0000", 2024-09-14T23:57:02.175 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011016310000000001 2024-09-14T23:57:02.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.175 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:02.175 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.208163+0000", 2024-09-14T23:57:02.175 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4849000000000003e-05 2024-09-14T23:57:02.176 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.176 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:02.176 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.176 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:32024 3.9 e39/23 3:94dc260b:::rbd_data.111814b482f8.0000000000000009:head v 39'45, mlcod=38'43)", 2024-09-14T23:57:02.176 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:49.061018+0000", 2024-09-14T23:57:02.176 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.029144145, 2024-09-14T23:57:02.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018482240000000001, 2024-09-14T23:57:02.176 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:02.176 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:02.177 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:02.177 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:02.177 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.177 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:02.177 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061018+0000", 2024-09-14T23:57:02.177 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.177 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.177 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.177 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:02.177 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061018+0000", 2024-09-14T23:57:02.177 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.178 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.178 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:02.178 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061018+0000", 2024-09-14T23:57:02.178 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9999999999999998e-07 2024-09-14T23:57:02.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.178 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.178 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:02.178 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061211+0000", 2024-09-14T23:57:02.178 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019206 2024-09-14T23:57:02.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.179 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.179 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:02.179 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061221+0000", 2024-09-14T23:57:02.179 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0659000000000001e-05 2024-09-14T23:57:02.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.179 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.179 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:02.179 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061230+0000", 2024-09-14T23:57:02.179 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4610000000000008e-06 2024-09-14T23:57:02.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:02.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061317+0000", 2024-09-14T23:57:02.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7514000000000003e-05 2024-09-14T23:57:02.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:02.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061338+0000", 2024-09-14T23:57:02.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0395999999999999e-05 2024-09-14T23:57:02.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.181 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.181 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:02.181 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.062830+0000", 2024-09-14T23:57:02.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0014921940000000001 2024-09-14T23:57:02.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.181 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.181 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:02.181 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.062866+0000", 2024-09-14T23:57:02.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6539999999999999e-05 2024-09-14T23:57:02.181 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.182 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:02.182 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.182 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:32420 3.9 e39/23 3:91dc34e7:::rbd_data.111814b482f8.000000000000001e:head v 39'46, mlcod=38'44)", 2024-09-14T23:57:02.182 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:49.086124+0000", 2024-09-14T23:57:02.182 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.004038646, 2024-09-14T23:57:02.182 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001686819, 2024-09-14T23:57:02.182 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:02.182 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:02.182 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:02.183 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:02.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:02.183 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086124+0000", 2024-09-14T23:57:02.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:02.183 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086124+0000", 2024-09-14T23:57:02.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.184 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.184 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:02.184 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086125+0000", 2024-09-14T23:57:02.184 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.361e-06 2024-09-14T23:57:02.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.184 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.184 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:02.184 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086605+0000", 2024-09-14T23:57:02.184 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00048026600000000003 2024-09-14T23:57:02.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.185 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:02.185 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086626+0000", 2024-09-14T23:57:02.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.037e-05 2024-09-14T23:57:02.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.185 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:02.185 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086635+0000", 2024-09-14T23:57:02.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5880000000000004e-06 2024-09-14T23:57:02.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.186 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:02.186 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086701+0000", 2024-09-14T23:57:02.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5905e-05 2024-09-14T23:57:02.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.186 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:02.186 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086727+0000", 2024-09-14T23:57:02.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5573000000000001e-05 2024-09-14T23:57:02.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:02.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.087767+0000", 2024-09-14T23:57:02.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001040537 2024-09-14T23:57:02.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:02.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.087810+0000", 2024-09-14T23:57:02.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3219000000000001e-05 2024-09-14T23:57:02.188 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.188 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:02.188 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.188 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:34296 3.9 e39/23 3:974a38c7:::rbd_data.111814b482f8.000000000000003a:head v 39'47, mlcod=39'45)", 2024-09-14T23:57:02.188 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:49.192105+0000", 2024-09-14T23:57:02.188 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 12.898056763, 2024-09-14T23:57:02.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001607516, 2024-09-14T23:57:02.189 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:02.189 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:02.189 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:02.189 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:02.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:02.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.192105+0000", 2024-09-14T23:57:02.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:02.190 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.192105+0000", 2024-09-14T23:57:02.190 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.190 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.190 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:02.190 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.192107+0000", 2024-09-14T23:57:02.190 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3549999999999999e-06 2024-09-14T23:57:02.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.190 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.190 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:02.190 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.192332+0000", 2024-09-14T23:57:02.190 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022486300000000001 2024-09-14T23:57:02.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.190 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.192 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:02.192 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.192349+0000", 2024-09-14T23:57:02.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6966e-05 2024-09-14T23:57:02.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.193 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:02.193 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.192358+0000", 2024-09-14T23:57:02.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8309999999999996e-06 2024-09-14T23:57:02.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.193 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.193 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:02.193 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.192422+0000", 2024-09-14T23:57:02.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3915e-05 2024-09-14T23:57:02.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.194 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:02.194 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.192450+0000", 2024-09-14T23:57:02.194 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7957e-05 2024-09-14T23:57:02.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.194 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:02.194 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.193662+0000", 2024-09-14T23:57:02.194 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012113270000000001 2024-09-14T23:57:02.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.195 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:02.195 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.193713+0000", 2024-09-14T23:57:02.195 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1301999999999998e-05 2024-09-14T23:57:02.195 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.195 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:02.195 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.195 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.195 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:35145 3.4 e39/23 3:22e94ee2:::rbd_data.111814b482f8.0000000000000047:head v 39'92, mlcod=39'90)", 2024-09-14T23:57:02.195 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:49.248843+0000", 2024-09-14T23:57:02.195 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 12.841319345, 2024-09-14T23:57:02.196 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001261206, 2024-09-14T23:57:02.196 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:02.196 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:02.196 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:02.196 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:02.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.196 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:02.196 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.248843+0000", 2024-09-14T23:57:02.196 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.197 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:02.197 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.248843+0000", 2024-09-14T23:57:02.197 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.197 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:02.197 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.248844+0000", 2024-09-14T23:57:02.197 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.271e-06 2024-09-14T23:57:02.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.197 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:02.198 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.248934+0000", 2024-09-14T23:57:02.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0109000000000001e-05 2024-09-14T23:57:02.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.198 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:02.198 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.248955+0000", 2024-09-14T23:57:02.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0877999999999999e-05 2024-09-14T23:57:02.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.198 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:02.198 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.248965+0000", 2024-09-14T23:57:02.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0113e-05 2024-09-14T23:57:02.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.199 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:02.199 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.249035+0000", 2024-09-14T23:57:02.199 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0038999999999998e-05 2024-09-14T23:57:02.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.199 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:02.199 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.249060+0000", 2024-09-14T23:57:02.199 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4722000000000001e-05 2024-09-14T23:57:02.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.200 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:02.200 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.250058+0000", 2024-09-14T23:57:02.200 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00099809599999999997 2024-09-14T23:57:02.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.200 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:02.200 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.250104+0000", 2024-09-14T23:57:02.200 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5977999999999999e-05 2024-09-14T23:57:02.201 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.201 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:02.201 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.201 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:46667 3.9 e48/23 3:9377641f:::rbd_data.1118cee06433.0000000000000000:head v 48'55, mlcod=44'52)", 2024-09-14T23:57:02.201 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:58.332832+0000", 2024-09-14T23:57:02.201 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.7573297810000001, 2024-09-14T23:57:02.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013548060000000001, 2024-09-14T23:57:02.201 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:02.201 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:02.202 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:02.202 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:02.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.202 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:02.202 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:58.332832+0000", 2024-09-14T23:57:02.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.202 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:02.202 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:58.332832+0000", 2024-09-14T23:57:02.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.203 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:02.203 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:58.332833+0000", 2024-09-14T23:57:02.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5999999999999999e-07 2024-09-14T23:57:02.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.203 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:02.203 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:58.332838+0000", 2024-09-14T23:57:02.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1039999999999998e-06 2024-09-14T23:57:02.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.204 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:02.204 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:58.332847+0000", 2024-09-14T23:57:02.204 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2429999999999994e-06 2024-09-14T23:57:02.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.204 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:02.204 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:58.332854+0000", 2024-09-14T23:57:02.204 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7789999999999998e-06 2024-09-14T23:57:02.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:02.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:58.332918+0000", 2024-09-14T23:57:02.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3943999999999996e-05 2024-09-14T23:57:02.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:02.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:58.332942+0000", 2024-09-14T23:57:02.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4091999999999999e-05 2024-09-14T23:57:02.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:02.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:58.334144+0000", 2024-09-14T23:57:02.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001201765 2024-09-14T23:57:02.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:02.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:58.334187+0000", 2024-09-14T23:57:02.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3519000000000001e-05 2024-09-14T23:57:02.206 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.207 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:02.207 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.207 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:50582 3.9 e51/23 3:92198e7a:::rbd_data.11183b9aac34.000000000000000a:head v 51'63, mlcod=50'60)", 2024-09-14T23:57:02.207 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:01.131541+0000", 2024-09-14T23:57:02.207 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.95862145499999996, 2024-09-14T23:57:02.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001795547, 2024-09-14T23:57:02.207 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:02.207 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:02.208 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:02.208 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:02.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:02.208 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131541+0000", 2024-09-14T23:57:02.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:02.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131541+0000", 2024-09-14T23:57:02.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.209 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:02.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131541+0000", 2024-09-14T23:57:02.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7099999999999997e-07 2024-09-14T23:57:02.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.209 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:02.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131548+0000", 2024-09-14T23:57:02.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.708e-06 2024-09-14T23:57:02.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:02.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131570+0000", 2024-09-14T23:57:02.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1815999999999999e-05 2024-09-14T23:57:02.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:02.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131581+0000", 2024-09-14T23:57:02.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0933e-05 2024-09-14T23:57:02.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:02.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131660+0000", 2024-09-14T23:57:02.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9071000000000002e-05 2024-09-14T23:57:02.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:02.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131682+0000", 2024-09-14T23:57:02.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2534000000000001e-05 2024-09-14T23:57:02.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.212 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:02.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.133287+0000", 2024-09-14T23:57:02.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016045510000000001 2024-09-14T23:57:02.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.212 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:02.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.133336+0000", 2024-09-14T23:57:02.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9563e-05 2024-09-14T23:57:02.213 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.213 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:02.213 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.213 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.213 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:02.213 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:02.215 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T23:57:02.216 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone13 from 0xadd5b75 (overlap 0x57db5f) to 0x3d7ff5 2024-09-14T23:57:02.216 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-09-14T23:57:02.217 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:02.213+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:02.217 INFO:teuthology.orchestra.run.smithi042.stdout:1277 writesame 0x5635238 thru 0x5640d4d data_size 0x4d(0xbb16 bytes) 2024-09-14T23:57:02.221 INFO:teuthology.orchestra.run.smithi042.stdout:1278 read 0x39316d7 thru 0x3935b27 (0x4451 bytes) 2024-09-14T23:57:02.222 INFO:teuthology.orchestra.run.smithi042.stdout:1279 write 0x7b716af thru 0x7b7b3da (0x9d2c bytes) 2024-09-14T23:57:02.226 INFO:teuthology.orchestra.run.smithi042.stdout:1280 read 0x88e5641 thru 0x88e5d4e (0x70e bytes) 2024-09-14T23:57:02.226 INFO:teuthology.orchestra.run.smithi042.stdout:1281 read 0x793beae thru 0x793dae5 (0x1c38 bytes) 2024-09-14T23:57:02.226 INFO:teuthology.orchestra.run.smithi042.stdout:1283 write 0x8673f88 thru 0x8680ee7 (0xcf60 bytes) 2024-09-14T23:57:02.229 INFO:teuthology.orchestra.run.smithi042.stdout:1284 read 0x8791a54 thru 0x87a0224 (0xe7d1 bytes) 2024-09-14T23:57:02.229 INFO:teuthology.orchestra.run.smithi042.stdout:1285 read 0x58a6754 thru 0x58af51e (0x8dcb bytes) 2024-09-14T23:57:02.235 INFO:teuthology.orchestra.run.smithi042.stdout:1288 write 0xb4f1ebb thru 0xb4f8ce4 (0x6e2a bytes) 2024-09-14T23:57:02.238 INFO:teuthology.orchestra.run.smithi042.stdout:1289 read 0x5f22b85 thru 0x5f2f7a7 (0xcc23 bytes) 2024-09-14T23:57:02.238 INFO:teuthology.orchestra.run.smithi042.stdout:1290 read 0x64e524a thru 0x64eed28 (0x9adf bytes) 2024-09-14T23:57:02.239 INFO:teuthology.orchestra.run.smithi042.stdout:1291 punch from 0x8088683 to 0x808d4e7, (0x4e64 bytes) 2024-09-14T23:57:02.239 INFO:teuthology.orchestra.run.smithi042.stdout:1295 punch from 0x854d1b1 to 0x8555b28, (0x8977 bytes) 2024-09-14T23:57:02.239 INFO:teuthology.orchestra.run.smithi042.stdout:1297 read 0x7b82ffc thru 0x7b8cf24 (0x9f29 bytes) 2024-09-14T23:57:02.249 INFO:teuthology.orchestra.run.smithi042.stdout:1298 writesame 0xe8b8bbc thru 0xe8bdf48 data_size 0x1(0x538d bytes) 2024-09-14T23:57:02.249 INFO:teuthology.orchestra.run.smithi042.stdout:1300 read 0x44900 thru 0x4976d (0x4e6e bytes) 2024-09-14T23:57:02.249 INFO:teuthology.orchestra.run.smithi042.stdout:1301 punch from 0x9de874c to 0x9df2be5, (0xa499 bytes) 2024-09-14T23:57:02.249 INFO:teuthology.orchestra.run.smithi042.stdout:1302 punch from 0x82fda0b to 0x830c685, (0xec7a bytes) 2024-09-14T23:57:02.249 INFO:teuthology.orchestra.run.smithi042.stdout:1303 read 0x49a32bb thru 0x49a3a88 (0x7ce bytes) 2024-09-14T23:57:02.249 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:02.249 INFO:teuthology.orchestra.run.smithi042.stdout:1308 read 0xc2a5e48 thru 0xc2aa449 (0x4602 bytes) 2024-09-14T23:57:02.250 INFO:teuthology.orchestra.run.smithi042.stdout:1309 write 0xac81221 thru 0xac8aedd (0x9cbd bytes) 2024-09-14T23:57:02.252 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:02.253 INFO:teuthology.orchestra.run.smithi042.stdout:1312 read 0x4cdfc03 thru 0x4ce5943 (0x5d41 bytes) 2024-09-14T23:57:02.253 INFO:teuthology.orchestra.run.smithi042.stdout:1313 write 0xe23b6c thru 0xe2be83 (0x8318 bytes) 2024-09-14T23:57:02.261 INFO:teuthology.orchestra.run.smithi042.stdout:1314 writesame 0xe92b795 thru 0xe92b90a data_size 0x2(0x176 bytes) 2024-09-14T23:57:02.261 INFO:teuthology.orchestra.run.smithi042.stdout:1315 read 0x4124747 thru 0x413056a (0xbe24 bytes) 2024-09-14T23:57:02.261 INFO:teuthology.orchestra.run.smithi042.stdout:1317 read 0x6db8f10 thru 0x6dc164b (0x873c bytes) 2024-09-14T23:57:02.261 INFO:teuthology.orchestra.run.smithi042.stdout:1319 write 0x3fba4b6 thru 0x3fc9e34 (0xf97f bytes) 2024-09-14T23:57:02.265 INFO:teuthology.orchestra.run.smithi042.stdout:1320 write 0x2a7f223 thru 0x2a8c2ca (0xd0a8 bytes) 2024-09-14T23:57:02.280 INFO:teuthology.orchestra.run.smithi042.stdout:1322 trunc from 0xe92b90b to 0x5866e59 2024-09-14T23:57:02.299 INFO:teuthology.orchestra.run.smithi042.stdout:1323 punch from 0x49ea0c7 to 0x49f4c59, (0xab92 bytes) 2024-09-14T23:57:02.299 INFO:teuthology.orchestra.run.smithi042.stdout:1324 read 0xd3b2d9 thru 0xd41efa (0x6c22 bytes) 2024-09-14T23:57:02.300 INFO:teuthology.orchestra.run.smithi042.stdout:1325 trunc from 0x5866e59 to 0x64f8ca7 2024-09-14T23:57:02.305 INFO:teuthology.orchestra.run.smithi042.stdout:1326 trunc from 0x64f8ca7 to 0x687eeec 2024-09-14T23:57:02.315 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:02.314+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:02.316 INFO:teuthology.orchestra.run.smithi042.stdout:1327 write 0x730e126 thru 0x731285a (0x4735 bytes) 2024-09-14T23:57:02.319 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:02.319 INFO:teuthology.orchestra.run.smithi042.stdout:1330 write 0x47039c6 thru 0x4713181 (0xf7bc bytes) 2024-09-14T23:57:02.322 INFO:teuthology.orchestra.run.smithi042.stdout:1331 read 0x36ae546 thru 0x36bcfd7 (0xea92 bytes) 2024-09-14T23:57:02.331 INFO:teuthology.orchestra.run.smithi042.stdout:1332 writesame 0x9dacbda thru 0x9db9740 data_size 0x11(0xcb67 bytes) 2024-09-14T23:57:02.337 INFO:teuthology.orchestra.run.smithi042.stdout:1333 writesame 0xadb7720 thru 0xadc3832 data_size 0x17(0xc113 bytes) 2024-09-14T23:57:02.337 INFO:teuthology.orchestra.run.smithi042.stdout:1334 read 0x70979e7 thru 0x70a708f (0xf6a9 bytes) 2024-09-14T23:57:02.337 INFO:teuthology.orchestra.run.smithi042.stdout:1335 punch from 0x2e065cb to 0x2e13c72, (0xd6a7 bytes) 2024-09-14T23:57:02.337 INFO:teuthology.orchestra.run.smithi042.stdout:1336 trunc from 0xadc3833 to 0xaf21379 2024-09-14T23:57:02.341 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:02.342 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:57:02.342 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:57:02.342 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:02.350 INFO:teuthology.orchestra.run.smithi042.stdout:1337 trunc from 0xaf21379 to 0x42976f8 2024-09-14T23:57:02.355 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T23:57:02.367 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:02.367 INFO:teuthology.orchestra.run.smithi042.stdout:1339 read 0x139959e thru 0x139cc00 (0x3663 bytes) 2024-09-14T23:57:02.372 INFO:teuthology.orchestra.run.smithi042.stdout:1342 trunc from 0x42976f8 to 0x19d19b 2024-09-14T23:57:02.392 INFO:teuthology.orchestra.run.smithi042.stdout:1343 punch from 0x28366 to 0x30f09, (0x8ba3 bytes) 2024-09-14T23:57:02.393 INFO:teuthology.orchestra.run.smithi042.stdout:1344 read 0x168869 thru 0x17879e (0xff36 bytes) 2024-09-14T23:57:02.393 INFO:teuthology.orchestra.run.smithi042.stdout:1345 read 0xcd18b thru 0xd7dd9 (0xac4f bytes) 2024-09-14T23:57:02.393 INFO:teuthology.orchestra.run.smithi042.stdout:1346 punch from 0x1b3de to 0x25f97, (0xabb9 bytes) 2024-09-14T23:57:02.393 INFO:teuthology.orchestra.run.smithi042.stdout:1347 punch from 0x198beb to 0x1992a7, (0x6bc bytes) 2024-09-14T23:57:02.403 INFO:teuthology.orchestra.run.smithi042.stdout:1350 write 0x3a5606b thru 0x3a5ddff (0x7d95 bytes) 2024-09-14T23:57:02.406 INFO:teuthology.orchestra.run.smithi042.stdout:1352 read 0x2940c34 thru 0x294121c (0x5e9 bytes) 2024-09-14T23:57:02.406 INFO:teuthology.orchestra.run.smithi042.stdout:1353 punch from 0x38596ce to 0x38630c3, (0x99f5 bytes) 2024-09-14T23:57:02.406 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:02.406 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:02.406 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:02.406 INFO:teuthology.orchestra.run.smithi042.stdout:1357 punch from 0x1b3762f to 0x1b45ef7, (0xe8c8 bytes) 2024-09-14T23:57:02.417 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:02.415+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:02.417 INFO:teuthology.orchestra.run.smithi042.stdout:1358 write 0x88d5057 thru 0x88dc796 (0x7740 bytes) 2024-09-14T23:57:02.420 INFO:teuthology.orchestra.run.smithi042.stdout:1359 punch from 0x3e6697c to 0x3e6a0ed, (0x3771 bytes) 2024-09-14T23:57:02.420 INFO:teuthology.orchestra.run.smithi042.stdout:1360 read 0x5e300ec thru 0x5e39ab3 (0x99c8 bytes) 2024-09-14T23:57:02.420 INFO:teuthology.orchestra.run.smithi042.stdout:1361 read 0x45a16bd thru 0x45aaf5e (0x98a2 bytes) 2024-09-14T23:57:02.429 INFO:teuthology.orchestra.run.smithi042.stdout:1362 trunc from 0x88dc797 to 0x1050bc 2024-09-14T23:57:02.461 INFO:teuthology.orchestra.run.smithi042.stdout:1363 write 0xdc57c4d thru 0xdc65678 (0xda2c bytes) 2024-09-14T23:57:02.463 INFO:teuthology.orchestra.run.smithi042.stdout:1364 write 0x7240958 thru 0x7248d5d (0x8406 bytes) 2024-09-14T23:57:02.466 INFO:teuthology.orchestra.run.smithi042.stdout:1365 punch from 0x550b8ca to 0x55194e1, (0xdc17 bytes) 2024-09-14T23:57:02.466 INFO:teuthology.orchestra.run.smithi042.stdout:1366 write 0x1fc7371 thru 0x1fd6788 (0xf418 bytes) 2024-09-14T23:57:02.469 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:02.469 INFO:teuthology.orchestra.run.smithi042.stdout:1369 read 0x1d6e07b thru 0x1d76fed (0x8f73 bytes) 2024-09-14T23:57:02.469 INFO:teuthology.orchestra.run.smithi042.stdout:1371 read 0x2078016 thru 0x207ad0d (0x2cf8 bytes) 2024-09-14T23:57:02.469 INFO:teuthology.orchestra.run.smithi042.stdout:1372 read 0xd515bca thru 0xd52579e (0xfbd5 bytes) 2024-09-14T23:57:02.469 INFO:teuthology.orchestra.run.smithi042.stdout:1375 write 0x3004a4c thru 0x3007ab1 (0x3066 bytes) 2024-09-14T23:57:02.472 INFO:teuthology.orchestra.run.smithi042.stdout:1376 write 0x5ed51d6 thru 0x5edf859 (0xa684 bytes) 2024-09-14T23:57:02.475 INFO:teuthology.orchestra.run.smithi042.stdout:1377 punch from 0xc0b57ef to 0xc0ba793, (0x4fa4 bytes) 2024-09-14T23:57:02.475 INFO:teuthology.orchestra.run.smithi042.stdout:1378 write 0x673edb9 thru 0x67424f7 (0x373f bytes) 2024-09-14T23:57:02.478 INFO:teuthology.orchestra.run.smithi042.stdout:1379 writesame 0x45bd250 thru 0x45c7d0c data_size 0x1(0xaabd bytes) 2024-09-14T23:57:02.491 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:02.491 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:57:02.491 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:57:02.491 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:57:02.491 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:02.497 INFO:teuthology.orchestra.run.smithi042.stdout:1380 write 0x1ebefd1 thru 0x1ec34e0 (0x4510 bytes) 2024-09-14T23:57:02.501 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T23:57:02.501 INFO:teuthology.orchestra.run.smithi042.stdout:1381 read 0xcc14d08 thru 0xcc153b0 (0x6a9 bytes) 2024-09-14T23:57:02.502 INFO:teuthology.orchestra.run.smithi042.stdout:1382 writesame 0xc377c4a thru 0xc37da01 data_size 0x4(0x5db8 bytes) 2024-09-14T23:57:02.505 INFO:teuthology.orchestra.run.smithi042.stdout:1383 writesame 0x11585c4 thru 0x116175c data_size 0x1(0x9199 bytes) 2024-09-14T23:57:02.521 INFO:teuthology.orchestra.run.smithi042.stdout:1384 write 0x35ef253 thru 0x35fc538 (0xd2e6 bytes) 2024-09-14T23:57:02.521 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:02.520+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:02.524 INFO:teuthology.orchestra.run.smithi042.stdout:1385 punch from 0x953c447 to 0x953d344, (0xefd bytes) 2024-09-14T23:57:02.525 INFO:teuthology.orchestra.run.smithi042.stdout:1386 read 0xc186155 thru 0xc191959 (0xb805 bytes) 2024-09-14T23:57:02.525 INFO:teuthology.orchestra.run.smithi042.stdout:1387 writesame 0xbd26eef thru 0xbd286c2 data_size 0x5(0x17d4 bytes) 2024-09-14T23:57:02.528 INFO:teuthology.orchestra.run.smithi042.stdout:1388 punch from 0x432dce1 to 0x433898d, (0xacac bytes) 2024-09-14T23:57:02.531 INFO:teuthology.orchestra.run.smithi042.stdout:1389 trunc from 0xdc65679 to 0xb1cbffd 2024-09-14T23:57:02.544 INFO:teuthology.orchestra.run.smithi042.stdout:1390 write 0x3474ed2 thru 0x3475167 (0x296 bytes) 2024-09-14T23:57:02.546 INFO:teuthology.orchestra.run.smithi042.stdout:1391 write 0x65672f7 thru 0x65697b1 (0x24bb bytes) 2024-09-14T23:57:02.549 INFO:teuthology.orchestra.run.smithi042.stdout:1392 read 0x32328ec thru 0x323f240 (0xc955 bytes) 2024-09-14T23:57:02.558 INFO:teuthology.orchestra.run.smithi042.stdout:1393 writesame 0xee58749 thru 0xee644e2 data_size 0x7(0xbd9a bytes) 2024-09-14T23:57:02.558 INFO:teuthology.orchestra.run.smithi042.stdout:1394 punch from 0x3c90373 to 0x3c930f5, (0x2d82 bytes) 2024-09-14T23:57:02.558 INFO:teuthology.orchestra.run.smithi042.stdout:1395 punch from 0xa89028c to 0xa891e53, (0x1bc7 bytes) 2024-09-14T23:57:02.558 INFO:teuthology.orchestra.run.smithi042.stdout:1396 read 0x87624df thru 0x876609c (0x3bbe bytes) 2024-09-14T23:57:02.559 INFO:teuthology.orchestra.run.smithi042.stdout:1398 writesame 0x8860ccd thru 0x886547f data_size 0x1(0x47b3 bytes) 2024-09-14T23:57:02.571 INFO:teuthology.orchestra.run.smithi042.stdout:1399 trunc from 0xee644e3 to 0xcfcedac 2024-09-14T23:57:02.576 INFO:teuthology.orchestra.run.smithi042.stdout:1400 punch from 0x4312a2b to 0x431f50e, (0xcae3 bytes) 2024-09-14T23:57:02.576 INFO:teuthology.orchestra.run.smithi042.stdout:1401 punch from 0x171fb8a to 0x172639d, (0x6813 bytes) 2024-09-14T23:57:02.576 INFO:teuthology.orchestra.run.smithi042.stdout:1402 write 0xed3c54 thru 0xee3754 (0xfb01 bytes) 2024-09-14T23:57:02.579 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:02.579 INFO:teuthology.orchestra.run.smithi042.stdout:1405 write 0x3f91aaa thru 0x3f9b580 (0x9ad7 bytes) 2024-09-14T23:57:02.588 INFO:teuthology.orchestra.run.smithi042.stdout:1406 write 0xd9e0e77 thru 0xd9e12f0 (0x47a bytes) 2024-09-14T23:57:02.591 INFO:teuthology.orchestra.run.smithi042.stdout:1408 read 0x95dae85 thru 0x95dfb83 (0x4cff bytes) 2024-09-14T23:57:02.591 INFO:teuthology.orchestra.run.smithi042.stdout:1409 writesame 0x92bbefa thru 0x92c259f data_size 0x7(0x66a6 bytes) 2024-09-14T23:57:02.595 INFO:teuthology.orchestra.run.smithi042.stdout:1410 punch from 0x800d7e8 to 0x800db73, (0x38b bytes) 2024-09-14T23:57:02.595 INFO:teuthology.orchestra.run.smithi042.stdout:1411 write 0xcbe56ff thru 0xcbf52ee (0xfbf0 bytes) 2024-09-14T23:57:02.598 INFO:teuthology.orchestra.run.smithi042.stdout:1413 write 0xb47b831 thru 0xb48177d (0x5f4d bytes) 2024-09-14T23:57:02.601 INFO:teuthology.orchestra.run.smithi042.stdout:1414 punch from 0x62b4e9c to 0x62b848f, (0x35f3 bytes) 2024-09-14T23:57:02.601 INFO:teuthology.orchestra.run.smithi042.stdout:1415 read 0x434904 thru 0x43b47c (0x6b79 bytes) 2024-09-14T23:57:02.601 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:02.601 INFO:teuthology.orchestra.run.smithi042.stdout:1417 read 0xb7704a3 thru 0xb77bc45 (0xb7a3 bytes) 2024-09-14T23:57:02.601 INFO:teuthology.orchestra.run.smithi042.stdout:1418 write 0xcfe837e thru 0xcff7a96 (0xf719 bytes) 2024-09-14T23:57:02.604 INFO:teuthology.orchestra.run.smithi042.stdout:1419 writesame 0x7465cc3 thru 0x746f5d8 data_size 0x5(0x9916 bytes) 2024-09-14T23:57:02.609 INFO:teuthology.orchestra.run.smithi042.stdout:1421 write 0x487835e thru 0x488607f (0xdd22 bytes) 2024-09-14T23:57:02.612 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:02.613 INFO:teuthology.orchestra.run.smithi042.stdout:1423 write 0x4fcf60e thru 0x4fd3318 (0x3d0b bytes) 2024-09-14T23:57:02.615 INFO:teuthology.orchestra.run.smithi042.stdout:1424 writesame 0x8caca6c thru 0x8cb8f44 data_size 0x17(0xc4d9 bytes) 2024-09-14T23:57:02.618 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:02.617+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:02.619 INFO:teuthology.orchestra.run.smithi042.stdout:1425 punch from 0xa6a38e2 to 0xa6ad406, (0x9b24 bytes) 2024-09-14T23:57:02.619 INFO:teuthology.orchestra.run.smithi042.stdout:1426 writesame 0x700a133 thru 0x70159e7 data_size 0x31(0xb8b5 bytes) 2024-09-14T23:57:02.626 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:02.626 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:57:02.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:57:02.626 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:57:02.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.626 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5180 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3883008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:02.626 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.250058+0000", 2024-09-14T23:57:02.626 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.373384003000002, 2024-09-14T23:57:02.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057351471000000001, 2024-09-14T23:57:02.627 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:02.627 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:02.627 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:02.627 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:02.627 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:02.627 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:02.627 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5180 2024-09-14T23:57:02.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.627 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:02.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:02.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250058+0000", 2024-09-14T23:57:02.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:02.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250058+0000", 2024-09-14T23:57:02.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:02.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250786+0000", 2024-09-14T23:57:02.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00072813399999999996 2024-09-14T23:57:02.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:02.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250789+0000", 2024-09-14T23:57:02.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.647e-06 2024-09-14T23:57:02.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:02.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250798+0000", 2024-09-14T23:57:02.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0000000000000002e-06 2024-09-14T23:57:02.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:02.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250803+0000", 2024-09-14T23:57:02.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.301e-06 2024-09-14T23:57:02.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:02.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.307234+0000", 2024-09-14T23:57:02.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056431165999999998 2024-09-14T23:57:02.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:02.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.307321+0000", 2024-09-14T23:57:02.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6831999999999998e-05 2024-09-14T23:57:02.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:02.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.307409+0000", 2024-09-14T23:57:02.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8391000000000001e-05 2024-09-14T23:57:02.631 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.631 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:02.631 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.631 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5184 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 4128768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:02.631 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.250851+0000", 2024-09-14T23:57:02.631 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.372590532, 2024-09-14T23:57:02.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057319038000000003, 2024-09-14T23:57:02.631 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:02.631 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:02.631 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:02.631 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:02.632 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:02.632 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:02.632 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5184 2024-09-14T23:57:02.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.632 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:02.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:02.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250851+0000", 2024-09-14T23:57:02.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:02.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250851+0000", 2024-09-14T23:57:02.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:02.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.251900+0000", 2024-09-14T23:57:02.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001048577 2024-09-14T23:57:02.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:02.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.251904+0000", 2024-09-14T23:57:02.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9580000000000001e-06 2024-09-14T23:57:02.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:02.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.251914+0000", 2024-09-14T23:57:02.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0186999999999999e-05 2024-09-14T23:57:02.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:02.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.251927+0000", 2024-09-14T23:57:02.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3593e-05 2024-09-14T23:57:02.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:02.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.308020+0000", 2024-09-14T23:57:02.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056092867999999997 2024-09-14T23:57:02.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:02.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.308094+0000", 2024-09-14T23:57:02.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3533000000000002e-05 2024-09-14T23:57:02.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:02.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.308170+0000", 2024-09-14T23:57:02.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6322000000000006e-05 2024-09-14T23:57:02.635 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.636 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:02.636 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.636 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5196 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 1925120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:02.636 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.252157+0000", 2024-09-14T23:57:02.636 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.371284417999998, 2024-09-14T23:57:02.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057285801999999997, 2024-09-14T23:57:02.636 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:02.636 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:02.636 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:02.637 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:02.637 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:02.637 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:02.637 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5196 2024-09-14T23:57:02.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.637 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:02.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:02.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.252157+0000", 2024-09-14T23:57:02.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:02.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.252157+0000", 2024-09-14T23:57:02.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:02.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253243+0000", 2024-09-14T23:57:02.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010861340000000001 2024-09-14T23:57:02.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:02.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253248+0000", 2024-09-14T23:57:02.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8799999999999999e-06 2024-09-14T23:57:02.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:02.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253271+0000", 2024-09-14T23:57:02.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2836e-05 2024-09-14T23:57:02.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:02.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253281+0000", 2024-09-14T23:57:02.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6520000000000001e-06 2024-09-14T23:57:02.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:02.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.309336+0000", 2024-09-14T23:57:02.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056054983000000003 2024-09-14T23:57:02.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:02.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.309389+0000", 2024-09-14T23:57:02.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3298999999999999e-05 2024-09-14T23:57:02.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:02.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.309443+0000", 2024-09-14T23:57:02.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4017999999999997e-05 2024-09-14T23:57:02.641 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.641 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:02.641 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.642 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5200 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2170880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:02.642 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253394+0000", 2024-09-14T23:57:02.642 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.370047640999999, 2024-09-14T23:57:02.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057428154000000002, 2024-09-14T23:57:02.642 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:02.642 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:02.642 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:02.642 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:02.642 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:02.642 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:02.642 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5200 2024-09-14T23:57:02.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.643 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:02.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:02.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253394+0000", 2024-09-14T23:57:02.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:02.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253394+0000", 2024-09-14T23:57:02.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:02.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253395+0000", 2024-09-14T23:57:02.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9100000000000004e-07 2024-09-14T23:57:02.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:02.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253397+0000", 2024-09-14T23:57:02.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2790000000000001e-06 2024-09-14T23:57:02.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:02.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253404+0000", 2024-09-14T23:57:02.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7959999999999998e-06 2024-09-14T23:57:02.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:02.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253409+0000", 2024-09-14T23:57:02.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1460000000000004e-06 2024-09-14T23:57:02.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:02.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310627+0000", 2024-09-14T23:57:02.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057218160999999997 2024-09-14T23:57:02.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:02.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310710+0000", 2024-09-14T23:57:02.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3102999999999994e-05 2024-09-14T23:57:02.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:02.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310822+0000", 2024-09-14T23:57:02.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000112178 2024-09-14T23:57:02.646 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.646 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:02.646 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.646 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5201 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2232320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:02.646 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:57:02.646 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.370022969000001, 2024-09-14T23:57:02.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057643228999999997, 2024-09-14T23:57:02.647 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:02.647 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:02.647 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:02.647 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:02.647 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:02.647 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:02.647 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5201 2024-09-14T23:57:02.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.647 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:02.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:02.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:57:02.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.648 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:02.648 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:57:02.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.648 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:02.648 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:57:02.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0699999999999998e-07 2024-09-14T23:57:02.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.648 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:02.648 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253422+0000", 2024-09-14T23:57:02.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.514e-06 2024-09-14T23:57:02.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:02.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253431+0000", 2024-09-14T23:57:02.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1919999999999994e-06 2024-09-14T23:57:02.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:02.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253436+0000", 2024-09-14T23:57:02.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5350000000000004e-06 2024-09-14T23:57:02.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:02.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310856+0000", 2024-09-14T23:57:02.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057419381999999998 2024-09-14T23:57:02.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.650 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:02.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310953+0000", 2024-09-14T23:57:02.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6774000000000006e-05 2024-09-14T23:57:02.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.650 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:02.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311062+0000", 2024-09-14T23:57:02.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000109425 2024-09-14T23:57:02.650 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.650 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:02.650 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.651 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5202 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2293760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:02.651 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:57:02.651 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.369998220999999, 2024-09-14T23:57:02.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057818255999999998, 2024-09-14T23:57:02.651 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:02.651 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:02.651 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:02.651 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:02.651 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:02.651 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:02.651 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5202 2024-09-14T23:57:02.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.652 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:02.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.652 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:02.652 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:57:02.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.652 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:02.652 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:57:02.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.652 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:02.653 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253573+0000", 2024-09-14T23:57:02.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012934000000000001 2024-09-14T23:57:02.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.653 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:02.653 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253576+0000", 2024-09-14T23:57:02.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5310000000000002e-06 2024-09-14T23:57:02.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.653 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:02.653 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253587+0000", 2024-09-14T23:57:02.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0522e-05 2024-09-14T23:57:02.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.654 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:02.654 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253595+0000", 2024-09-14T23:57:02.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8290000000000008e-06 2024-09-14T23:57:02.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.654 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:02.654 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311093+0000", 2024-09-14T23:57:02.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057498312000000003 2024-09-14T23:57:02.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.654 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:02.654 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311167+0000", 2024-09-14T23:57:02.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3582000000000001e-05 2024-09-14T23:57:02.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.655 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:02.655 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311262+0000", 2024-09-14T23:57:02.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5140000000000001e-05 2024-09-14T23:57:02.655 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.655 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:02.655 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.655 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5203 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2355200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:02.655 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:57:02.655 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.369835478999999, 2024-09-14T23:57:02.656 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057903938000000002, 2024-09-14T23:57:02.656 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:02.656 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:02.656 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:02.656 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:02.656 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:02.656 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:02.656 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5203 2024-09-14T23:57:02.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.656 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:02.656 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.656 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:02.656 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:57:02.657 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.657 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.657 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:02.657 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:57:02.657 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.657 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.657 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:02.657 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253902+0000", 2024-09-14T23:57:02.657 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029615400000000002 2024-09-14T23:57:02.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.657 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.657 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:02.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253906+0000", 2024-09-14T23:57:02.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9249999999999997e-06 2024-09-14T23:57:02.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:02.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253913+0000", 2024-09-14T23:57:02.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9099999999999999e-06 2024-09-14T23:57:02.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:02.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253917+0000", 2024-09-14T23:57:02.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9210000000000002e-06 2024-09-14T23:57:02.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:02.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311295+0000", 2024-09-14T23:57:02.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057378268000000003 2024-09-14T23:57:02.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:02.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311402+0000", 2024-09-14T23:57:02.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010644100000000001 2024-09-14T23:57:02.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:02.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311510+0000", 2024-09-14T23:57:02.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108319 2024-09-14T23:57:02.659 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.660 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:02.660 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.660 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5204 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2416640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:02.660 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:57:02.660 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.369517282, 2024-09-14T23:57:02.660 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057774860999999997, 2024-09-14T23:57:02.660 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:02.660 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:02.660 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:02.660 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:02.660 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:02.660 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:02.661 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5204 2024-09-14T23:57:02.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.661 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:02.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.661 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:02.661 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:57:02.661 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.661 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:02.661 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:57:02.661 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.662 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:02.662 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255903+0000", 2024-09-14T23:57:02.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001978807 2024-09-14T23:57:02.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.662 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:02.662 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255907+0000", 2024-09-14T23:57:02.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3610000000000001e-06 2024-09-14T23:57:02.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.662 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:02.662 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255914+0000", 2024-09-14T23:57:02.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.357e-06 2024-09-14T23:57:02.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:02.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255918+0000", 2024-09-14T23:57:02.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1779999999999996e-06 2024-09-14T23:57:02.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:02.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311541+0000", 2024-09-14T23:57:02.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055622697999999998 2024-09-14T23:57:02.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:02.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311614+0000", 2024-09-14T23:57:02.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3102999999999994e-05 2024-09-14T23:57:02.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.664 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:02.664 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311699+0000", 2024-09-14T23:57:02.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5357000000000003e-05 2024-09-14T23:57:02.664 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.664 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:02.664 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.664 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5210 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2785280~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:02.664 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256011+0000", 2024-09-14T23:57:02.664 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.367430644999999, 2024-09-14T23:57:02.665 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057273662000000003, 2024-09-14T23:57:02.665 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:02.665 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:02.665 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:02.665 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:02.665 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:02.665 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:02.665 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5210 2024-09-14T23:57:02.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.665 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:02.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.665 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:02.665 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256011+0000", 2024-09-14T23:57:02.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.666 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:02.666 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256011+0000", 2024-09-14T23:57:02.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.666 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:02.666 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256011+0000", 2024-09-14T23:57:02.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4999999999999999e-07 2024-09-14T23:57:02.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.666 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:02.666 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256013+0000", 2024-09-14T23:57:02.667 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3969999999999999e-06 2024-09-14T23:57:02.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.667 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:02.667 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256017+0000", 2024-09-14T23:57:02.667 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1409999999999998e-06 2024-09-14T23:57:02.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.667 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:02.667 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256019+0000", 2024-09-14T23:57:02.667 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6280000000000001e-06 2024-09-14T23:57:02.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.668 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:02.668 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313116+0000", 2024-09-14T23:57:02.668 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057096457000000003 2024-09-14T23:57:02.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.668 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:02.668 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313193+0000", 2024-09-14T23:57:02.668 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7439000000000002e-05 2024-09-14T23:57:02.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.668 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:02.668 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313285+0000", 2024-09-14T23:57:02.668 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1349999999999998e-05 2024-09-14T23:57:02.668 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.669 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:02.669 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.669 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5211 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2846720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:02.669 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:57:02.669 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.367419068, 2024-09-14T23:57:02.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057528151, 2024-09-14T23:57:02.669 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:02.669 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:02.669 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:02.669 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:02.669 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:02.669 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:02.670 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5211 2024-09-14T23:57:02.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.670 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:02.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.670 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:02.670 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:57:02.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.670 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:02.670 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:57:02.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.671 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:02.671 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:57:02.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1399999999999998e-07 2024-09-14T23:57:02.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.671 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:02.671 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256024+0000", 2024-09-14T23:57:02.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.114e-06 2024-09-14T23:57:02.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.671 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:02.671 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256028+0000", 2024-09-14T23:57:02.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0840000000000002e-06 2024-09-14T23:57:02.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:02.672 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256031+0000", 2024-09-14T23:57:02.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6819999999999999e-06 2024-09-14T23:57:02.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:02.672 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313317+0000", 2024-09-14T23:57:02.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057286295000000001 2024-09-14T23:57:02.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:02.673 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313403+0000", 2024-09-14T23:57:02.673 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6287999999999998e-05 2024-09-14T23:57:02.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.673 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:02.673 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313551+0000", 2024-09-14T23:57:02.673 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000147374 2024-09-14T23:57:02.673 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.673 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:02.673 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.673 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5212 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2908160~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:02.673 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:57:02.674 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.367407874000001, 2024-09-14T23:57:02.674 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057735679999999998, 2024-09-14T23:57:02.674 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:02.674 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:02.674 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:02.674 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:02.674 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:02.674 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:02.674 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5212 2024-09-14T23:57:02.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.674 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:02.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.674 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:02.675 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:57:02.675 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:02.675 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:57:02.675 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:02.675 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:57:02.675 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0900000000000001e-07 2024-09-14T23:57:02.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.676 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:02.676 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256035+0000", 2024-09-14T23:57:02.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1200000000000001e-06 2024-09-14T23:57:02.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.676 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:02.676 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256038+0000", 2024-09-14T23:57:02.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.05e-06 2024-09-14T23:57:02.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.676 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:02.676 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256041+0000", 2024-09-14T23:57:02.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.667e-06 2024-09-14T23:57:02.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.677 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.677 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:02.677 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313584+0000", 2024-09-14T23:57:02.677 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057543484999999998 2024-09-14T23:57:02.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.677 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.677 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:02.677 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313668+0000", 2024-09-14T23:57:02.677 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3807000000000006e-05 2024-09-14T23:57:02.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.677 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.677 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:02.677 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313770+0000", 2024-09-14T23:57:02.678 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010134200000000001 2024-09-14T23:57:02.678 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.678 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:02.678 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.678 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.678 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.678 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5213 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2969600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:02.678 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:57:02.678 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.367397787000002, 2024-09-14T23:57:02.678 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058145467999999999, 2024-09-14T23:57:02.678 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:02.678 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:02.678 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:02.679 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:02.679 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:02.679 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:02.679 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5213 2024-09-14T23:57:02.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.679 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:02.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.679 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:02.679 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:57:02.679 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.679 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:02.679 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:57:02.679 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.680 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:02.680 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257901+0000", 2024-09-14T23:57:02.680 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018567519999999999 2024-09-14T23:57:02.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.680 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:02.680 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257904+0000", 2024-09-14T23:57:02.680 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4850000000000001e-06 2024-09-14T23:57:02.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.680 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:02.680 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257912+0000", 2024-09-14T23:57:02.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2090000000000005e-06 2024-09-14T23:57:02.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.681 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:02.681 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257918+0000", 2024-09-14T23:57:02.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6899999999999997e-06 2024-09-14T23:57:02.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.681 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:02.681 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313816+0000", 2024-09-14T23:57:02.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055898075999999998 2024-09-14T23:57:02.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.682 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:02.682 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.314082+0000", 2024-09-14T23:57:02.682 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000265386 2024-09-14T23:57:02.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.682 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:02.682 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.314189+0000", 2024-09-14T23:57:02.682 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010787 2024-09-14T23:57:02.682 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.682 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:02.682 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.682 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5217 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3215360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:02.683 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257961+0000", 2024-09-14T23:57:02.683 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.365480808000001, 2024-09-14T23:57:02.683 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057320969999999999, 2024-09-14T23:57:02.683 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:02.683 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:02.683 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:02.683 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:02.683 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:02.683 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:02.683 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5217 2024-09-14T23:57:02.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.683 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:02.683 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.683 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:02.684 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257961+0000", 2024-09-14T23:57:02.684 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.684 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:02.684 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257961+0000", 2024-09-14T23:57:02.684 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.684 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:02.684 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257961+0000", 2024-09-14T23:57:02.684 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.22e-07 2024-09-14T23:57:02.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.685 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:02.685 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257962+0000", 2024-09-14T23:57:02.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1400000000000001e-06 2024-09-14T23:57:02.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.685 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:02.685 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257965+0000", 2024-09-14T23:57:02.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0929999999999999e-06 2024-09-14T23:57:02.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.685 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:02.686 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257968+0000", 2024-09-14T23:57:02.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5749999999999999e-06 2024-09-14T23:57:02.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.686 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:02.686 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315047+0000", 2024-09-14T23:57:02.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057078730000000001 2024-09-14T23:57:02.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.686 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:02.686 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315087+0000", 2024-09-14T23:57:02.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0806999999999997e-05 2024-09-14T23:57:02.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.687 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:02.687 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315282+0000", 2024-09-14T23:57:02.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000194403 2024-09-14T23:57:02.687 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.687 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:02.687 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.687 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5218 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3276800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:02.688 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:57:02.688 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.365470777999999, 2024-09-14T23:57:02.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057497557999999997, 2024-09-14T23:57:02.688 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:02.688 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:02.688 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:02.688 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:02.688 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:02.688 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:02.688 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5218 2024-09-14T23:57:02.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.688 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:02.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.689 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:02.689 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:57:02.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.689 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:02.689 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:57:02.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.689 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:02.690 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:57:02.690 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1899999999999999e-07 2024-09-14T23:57:02.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.690 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:02.690 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257972+0000", 2024-09-14T23:57:02.690 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.217e-06 2024-09-14T23:57:02.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.690 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:02.690 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257975+0000", 2024-09-14T23:57:02.690 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9359999999999999e-06 2024-09-14T23:57:02.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.691 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:02.691 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257978+0000", 2024-09-14T23:57:02.691 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1049999999999999e-06 2024-09-14T23:57:02.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.691 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:02.691 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315331+0000", 2024-09-14T23:57:02.691 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057352093999999999 2024-09-14T23:57:02.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.691 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:02.692 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315384+0000", 2024-09-14T23:57:02.692 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3780999999999999e-05 2024-09-14T23:57:02.692 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.692 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.692 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:02.692 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315468+0000", 2024-09-14T23:57:02.692 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4206e-05 2024-09-14T23:57:02.692 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.692 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:02.692 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.692 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.692 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.693 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5219 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3338240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:02.693 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:57:02.693 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.365460102, 2024-09-14T23:57:02.693 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057637764000000001, 2024-09-14T23:57:02.693 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:02.693 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:02.693 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:02.693 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:02.693 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:02.693 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:02.693 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5219 2024-09-14T23:57:02.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.693 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:02.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:02.694 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:57:02.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:02.694 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:57:02.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:02.694 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:57:02.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.03e-07 2024-09-14T23:57:02.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.695 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:02.695 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257983+0000", 2024-09-14T23:57:02.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9199999999999999e-07 2024-09-14T23:57:02.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.695 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:02.695 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257986+0000", 2024-09-14T23:57:02.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0860000000000002e-06 2024-09-14T23:57:02.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.696 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:02.696 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257989+0000", 2024-09-14T23:57:02.696 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6589999999999999e-06 2024-09-14T23:57:02.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.696 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:02.696 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315509+0000", 2024-09-14T23:57:02.696 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057520262000000003 2024-09-14T23:57:02.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.696 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:02.696 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315554+0000", 2024-09-14T23:57:02.696 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5015999999999997e-05 2024-09-14T23:57:02.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.697 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:02.697 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315619+0000", 2024-09-14T23:57:02.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5545999999999999e-05 2024-09-14T23:57:02.697 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.697 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:02.697 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.697 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5220 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3399680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:02.697 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:57:02.697 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.365450146000001, 2024-09-14T23:57:02.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057792128999999998, 2024-09-14T23:57:02.698 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:02.698 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:02.698 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:02.698 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:02.698 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:02.698 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:02.698 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5220 2024-09-14T23:57:02.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.698 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:02.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.698 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:02.698 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:57:02.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.699 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:02.699 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:57:02.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.699 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:02.699 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:57:02.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1799999999999999e-07 2024-09-14T23:57:02.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.700 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:02.700 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257993+0000", 2024-09-14T23:57:02.700 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6200000000000006e-07 2024-09-14T23:57:02.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.700 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:02.700 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257996+0000", 2024-09-14T23:57:02.700 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9050000000000001e-06 2024-09-14T23:57:02.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.700 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:02.701 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257998+0000", 2024-09-14T23:57:02.701 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5950000000000001e-06 2024-09-14T23:57:02.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.701 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:02.701 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315672+0000", 2024-09-14T23:57:02.701 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057673637999999999 2024-09-14T23:57:02.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.701 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:02.701 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315734+0000", 2024-09-14T23:57:02.701 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2216000000000002e-05 2024-09-14T23:57:02.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:02.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315784+0000", 2024-09-14T23:57:02.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9595000000000002e-05 2024-09-14T23:57:02.702 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.702 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:02.702 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.702 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5221 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3461120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:02.702 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:57:02.702 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.365440427999999, 2024-09-14T23:57:02.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058003557999999997, 2024-09-14T23:57:02.703 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:02.703 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:02.703 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:02.703 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:02.703 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:02.703 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:02.703 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5221 2024-09-14T23:57:02.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.703 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:02.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.703 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:02.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:57:02.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.704 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:02.704 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:57:02.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.704 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:02.704 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259905+0000", 2024-09-14T23:57:02.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019035759999999999 2024-09-14T23:57:02.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.704 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:02.704 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259915+0000", 2024-09-14T23:57:02.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0518e-05 2024-09-14T23:57:02.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.705 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:02.705 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259925+0000", 2024-09-14T23:57:02.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7180000000000008e-06 2024-09-14T23:57:02.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.705 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:02.705 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259930+0000", 2024-09-14T23:57:02.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4769999999999997e-06 2024-09-14T23:57:02.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.705 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:02.705 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315820+0000", 2024-09-14T23:57:02.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055889960000000002 2024-09-14T23:57:02.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:02.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315914+0000", 2024-09-14T23:57:02.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4625999999999998e-05 2024-09-14T23:57:02.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:02.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.316005+0000", 2024-09-14T23:57:02.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0682999999999999e-05 2024-09-14T23:57:02.706 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.706 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:02.706 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.707 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5230 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 4014080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:02.707 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.260060+0000", 2024-09-14T23:57:02.707 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.363381589999999, 2024-09-14T23:57:02.707 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057353556999999999, 2024-09-14T23:57:02.707 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:02.707 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:02.707 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:02.707 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:02.707 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:02.707 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:02.707 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5230 2024-09-14T23:57:02.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.708 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:02.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:02.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260060+0000", 2024-09-14T23:57:02.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:02.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260060+0000", 2024-09-14T23:57:02.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:02.709 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260060+0000", 2024-09-14T23:57:02.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.17e-07 2024-09-14T23:57:02.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:02.709 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260061+0000", 2024-09-14T23:57:02.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0300000000000001e-06 2024-09-14T23:57:02.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.710 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:02.710 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260064+0000", 2024-09-14T23:57:02.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9119999999999998e-06 2024-09-14T23:57:02.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.710 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:02.710 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260067+0000", 2024-09-14T23:57:02.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.678e-06 2024-09-14T23:57:02.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.710 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:02.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317269+0000", 2024-09-14T23:57:02.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057201771999999998 2024-09-14T23:57:02.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:02.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317342+0000", 2024-09-14T23:57:02.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2842000000000005e-05 2024-09-14T23:57:02.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:02.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317414+0000", 2024-09-14T23:57:02.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2106000000000004e-05 2024-09-14T23:57:02.711 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.711 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:02.712 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.712 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.712 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5231 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 4075520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:02.712 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:57:02.712 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.363371694000001, 2024-09-14T23:57:02.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057524733000000002, 2024-09-14T23:57:02.712 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:02.712 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:02.712 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:02.712 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:02.712 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:02.712 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:02.712 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5231 2024-09-14T23:57:02.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.713 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:02.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.713 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:02.713 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:57:02.713 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.713 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:02.713 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:57:02.713 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.713 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:02.714 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261904+0000", 2024-09-14T23:57:02.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018338790000000001 2024-09-14T23:57:02.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.714 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:02.714 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261910+0000", 2024-09-14T23:57:02.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9689999999999999e-06 2024-09-14T23:57:02.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.714 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:02.714 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261923+0000", 2024-09-14T23:57:02.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3061000000000001e-05 2024-09-14T23:57:02.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:02.715 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261931+0000", 2024-09-14T23:57:02.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3159999999999997e-06 2024-09-14T23:57:02.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:02.715 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317456+0000", 2024-09-14T23:57:02.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055525219000000001 2024-09-14T23:57:02.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:02.715 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317520+0000", 2024-09-14T23:57:02.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4010000000000005e-05 2024-09-14T23:57:02.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.716 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:02.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317595+0000", 2024-09-14T23:57:02.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4278999999999998e-05 2024-09-14T23:57:02.716 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.716 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:02.716 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.716 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5236 3.8 3:1071668d:::rbd_data.1118de1168b.0000000000000092:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:02.716 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:57:02.716 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.36135823, 2024-09-14T23:57:02.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057499462000000001, 2024-09-14T23:57:02.716 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:02.716 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:02.717 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:02.717 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:02.717 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:02.717 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:02.717 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5236 2024-09-14T23:57:02.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.717 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:02.717 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.717 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:02.717 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:57:02.717 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.717 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.717 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:02.718 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:57:02.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:02.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:02.718 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262084+0000", 2024-09-14T23:57:02.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3399999999999999e-07 2024-09-14T23:57:02.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:02.718 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262087+0000", 2024-09-14T23:57:02.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2310000000000001e-06 2024-09-14T23:57:02.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:02.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262095+0000", 2024-09-14T23:57:02.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3799999999999996e-06 2024-09-14T23:57:02.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:02.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262101+0000", 2024-09-14T23:57:02.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0009999999999998e-06 2024-09-14T23:57:02.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:02.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317625+0000", 2024-09-14T23:57:02.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055524111000000001 2024-09-14T23:57:02.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.720 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:02.720 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317702+0000", 2024-09-14T23:57:02.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7533999999999993e-05 2024-09-14T23:57:02.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.720 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-14T23:57:02.720 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317892+0000", 2024-09-14T23:57:02.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018991400000000001 2024-09-14T23:57:02.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.720 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:57:02.720 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319407+0000", 2024-09-14T23:57:02.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015145250000000001 2024-09-14T23:57:02.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.721 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:02.721 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319481+0000", 2024-09-14T23:57:02.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3826999999999996e-05 2024-09-14T23:57:02.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.721 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:02.721 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319541+0000", 2024-09-14T23:57:02.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0470999999999998e-05 2024-09-14T23:57:02.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:02.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:02.721 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:02.721 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319583+0000", 2024-09-14T23:57:02.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1934000000000002e-05 2024-09-14T23:57:02.722 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.722 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:02.722 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.722 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:02.722 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:02.722 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:02.723 INFO:teuthology.orchestra.run.smithi042.stdout:1427 trunc from 0xd9e12f1 to 0x1141f50 2024-09-14T23:57:02.724 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T23:57:02.725 INFO:teuthology.orchestra.run.smithi042.stdout:1428 write 0xbf75580 thru 0xbf7d187 (0x7c08 bytes) 2024-09-14T23:57:02.725 INFO:teuthology.orchestra.run.smithi042.stdout:1431 writesame 0x5f944e4 thru 0x5f9e9df data_size 0x2(0xa4fc bytes) 2024-09-14T23:57:02.725 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:02.726 INFO:teuthology.orchestra.run.smithi042.stdout:1433 write 0xedb2597 thru 0xedbcf3b (0xa9a5 bytes) 2024-09-14T23:57:02.726 INFO:teuthology.orchestra.run.smithi042.stdout:1434 writesame 0xbebb3e7 thru 0xbebca93 data_size 0x1b(0x16ad bytes) 2024-09-14T23:57:02.726 INFO:teuthology.orchestra.run.smithi042.stdout:1436 write 0x4879c2 thru 0x491b58 (0xa197 bytes) 2024-09-14T23:57:02.726 INFO:teuthology.orchestra.run.smithi042.stdout:1437 trunc from 0xedbcf3c to 0x457691d 2024-09-14T23:57:02.726 INFO:teuthology.orchestra.run.smithi042.stdout:1438 punch from 0x329fa9c to 0x32ad31d, (0xd881 bytes) 2024-09-14T23:57:02.726 INFO:teuthology.orchestra.run.smithi042.stdout:1439 write 0x1bac70 thru 0x1c082c (0x5bbd bytes) 2024-09-14T23:57:02.726 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:02.722+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:02.729 INFO:teuthology.orchestra.run.smithi042.stdout:1440 trunc from 0x457691d to 0x830826 2024-09-14T23:57:02.739 INFO:teuthology.orchestra.run.smithi042.stdout:1442 punch from 0x7cc77d to 0x7cdf47, (0x17ca bytes) 2024-09-14T23:57:02.740 INFO:teuthology.orchestra.run.smithi042.stdout:1444 punch from 0x42d32c to 0x43a44b, (0xd11f bytes) 2024-09-14T23:57:02.740 INFO:teuthology.orchestra.run.smithi042.stdout:1445 read 0x53bb69 thru 0x547725 (0xbbbd bytes) 2024-09-14T23:57:02.740 INFO:teuthology.orchestra.run.smithi042.stdout:1446 punch from 0x2c5dcd to 0x2cb42e, (0x5661 bytes) 2024-09-14T23:57:02.742 INFO:teuthology.orchestra.run.smithi042.stdout:1447 trunc from 0x830826 to 0x2fa687f 2024-09-14T23:57:02.747 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:02.766 INFO:teuthology.orchestra.run.smithi042.stdout:1449 writesame 0xdd341e4 thru 0xdd37bcd data_size 0x15(0x39ea bytes) 2024-09-14T23:57:02.767 INFO:teuthology.orchestra.run.smithi042.stdout:1450 read 0x54a952 thru 0x55a86d (0xff1c bytes) 2024-09-14T23:57:02.767 INFO:teuthology.orchestra.run.smithi042.stdout:1451 punch from 0x5dafba9 to 0x5db6cb3, (0x710a bytes) 2024-09-14T23:57:02.767 INFO:teuthology.orchestra.run.smithi042.stdout:1452 write 0xacbde10 thru 0xacbf7d2 (0x19c3 bytes) 2024-09-14T23:57:02.769 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:02.769 INFO:teuthology.orchestra.run.smithi042.stdout:1455 write 0x6de8778 thru 0x6de975f (0xfe8 bytes) 2024-09-14T23:57:02.772 INFO:teuthology.orchestra.run.smithi042.stdout:1456 punch from 0x646d168 to 0x64721bd, (0x5055 bytes) 2024-09-14T23:57:02.772 INFO:teuthology.orchestra.run.smithi042.stdout:1457 punch from 0x9be15b9 to 0x9bea7c2, (0x9209 bytes) 2024-09-14T23:57:02.772 INFO:teuthology.orchestra.run.smithi042.stdout:1458 write 0x4f34b2c thru 0x4f3f15a (0xa62f bytes) 2024-09-14T23:57:02.774 INFO:teuthology.orchestra.run.smithi042.stdout:1459 write 0xddfe43 thru 0xde77cf (0x798d bytes) 2024-09-14T23:57:02.778 INFO:teuthology.orchestra.run.smithi042.stdout:1462 trunc from 0xdd37bce to 0xd690eb7 2024-09-14T23:57:02.783 INFO:teuthology.orchestra.run.smithi042.stdout:1463 read 0xab1de43 thru 0xab1f271 (0x142f bytes) 2024-09-14T23:57:02.783 INFO:teuthology.orchestra.run.smithi042.stdout:1464 read 0xb7620e7 thru 0xb7649bd (0x28d7 bytes) 2024-09-14T23:57:02.783 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:02.783 INFO:teuthology.orchestra.run.smithi042.stdout:1467 writesame 0xb891649 thru 0xb89c54f data_size 0x7(0xaf07 bytes) 2024-09-14T23:57:02.789 INFO:teuthology.orchestra.run.smithi042.stdout:1468 read 0x6c5e3f8 thru 0x6c6b224 (0xce2d bytes) 2024-09-14T23:57:02.789 INFO:teuthology.orchestra.run.smithi042.stdout:1469 read 0x46f1ef3 thru 0x46ff98f (0xda9d bytes) 2024-09-14T23:57:02.789 INFO:teuthology.orchestra.run.smithi042.stdout:1470 read 0x6c94780 thru 0x6ca3bbc (0xf43d bytes) 2024-09-14T23:57:02.789 INFO:teuthology.orchestra.run.smithi042.stdout:1471 punch from 0xbaac039 to 0xbab8351, (0xc318 bytes) 2024-09-14T23:57:02.790 INFO:teuthology.orchestra.run.smithi042.stdout:1472 punch from 0x71ff370 to 0x720f152, (0xfde2 bytes) 2024-09-14T23:57:02.790 INFO:teuthology.orchestra.run.smithi042.stdout:1473 writesame 0x1461d3c thru 0x146cc32 data_size 0x1(0xaef7 bytes) 2024-09-14T23:57:02.824 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:02.823+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:02.824 INFO:teuthology.orchestra.run.smithi042.stdout:1474 trunc from 0xd690eb7 to 0xc70ec3 2024-09-14T23:57:02.836 INFO:teuthology.orchestra.run.smithi042.stdout:1475 read 0x30b7dd thru 0x314232 (0x8a56 bytes) 2024-09-14T23:57:02.841 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:02.841 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:02.841 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:57:02.841 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:02.847 INFO:teuthology.orchestra.run.smithi042.stdout:1476 write 0x52aa1b1 thru 0x52aa20a (0x5a bytes) 2024-09-14T23:57:02.851 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T23:57:02.851 INFO:teuthology.orchestra.run.smithi042.stdout:1477 punch from 0x4f7a15e to 0x4f7cd32, (0x2bd4 bytes) 2024-09-14T23:57:02.857 INFO:teuthology.orchestra.run.smithi042.stdout:1478 writesame 0x5ea61b2 thru 0x5eac975 data_size 0x3a(0x67c4 bytes) 2024-09-14T23:57:02.857 INFO:teuthology.orchestra.run.smithi042.stdout:1479 punch from 0x5a53cee to 0x5a5c6f8, (0x8a0a bytes) 2024-09-14T23:57:02.857 INFO:teuthology.orchestra.run.smithi042.stdout:1481 punch from 0xa79204 to 0xa7f11e, (0x5f1a bytes) 2024-09-14T23:57:02.857 INFO:teuthology.orchestra.run.smithi042.stdout:1482 read 0x1d59164 thru 0x1d6401f (0xaebc bytes) 2024-09-14T23:57:02.862 INFO:teuthology.orchestra.run.smithi042.stdout:1483 trunc from 0x5eac976 to 0xae2211b 2024-09-14T23:57:02.876 INFO:teuthology.orchestra.run.smithi042.stdout:1485 write 0xd6bc0da thru 0xd6c1a27 (0x594e bytes) 2024-09-14T23:57:02.879 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:02.879 INFO:teuthology.orchestra.run.smithi042.stdout:1488 read 0x82a3a33 thru 0x82afb4d (0xc11b bytes) 2024-09-14T23:57:02.879 INFO:teuthology.orchestra.run.smithi042.stdout:1489 write 0x70f0f5a thru 0x70f48bd (0x3964 bytes) 2024-09-14T23:57:02.882 INFO:teuthology.orchestra.run.smithi042.stdout:1490 read 0x3ebb877 thru 0x3ec972b (0xdeb5 bytes) 2024-09-14T23:57:02.882 INFO:teuthology.orchestra.run.smithi042.stdout:1494 punch from 0xccc4f84 to 0xccd4445, (0xf4c1 bytes) 2024-09-14T23:57:02.882 INFO:teuthology.orchestra.run.smithi042.stdout:1495 read 0xae29d8e thru 0xae34fb3 (0xb226 bytes) 2024-09-14T23:57:02.882 INFO:teuthology.orchestra.run.smithi042.stdout:1497 punch from 0x21fb257 to 0x2203053, (0x7dfc bytes) 2024-09-14T23:57:02.882 INFO:teuthology.orchestra.run.smithi042.stdout:1502 writesame 0x1a12325 thru 0x1a1df04 data_size 0x24(0xbbe0 bytes) 2024-09-14T23:57:02.885 INFO:teuthology.orchestra.run.smithi042.stdout:1503 punch from 0x9969bf9 to 0x996e42f, (0x4836 bytes) 2024-09-14T23:57:02.885 INFO:teuthology.orchestra.run.smithi042.stdout:1504 read 0xbf3eae9 thru 0xbf3f0dd (0x5f5 bytes) 2024-09-14T23:57:02.885 INFO:teuthology.orchestra.run.smithi042.stdout:1505 read 0x7ebdb6f thru 0x7ecc4de (0xe970 bytes) 2024-09-14T23:57:02.885 INFO:teuthology.orchestra.run.smithi042.stdout:1506 write 0xb6eb89 thru 0xb7afa4 (0xc41c bytes) 2024-09-14T23:57:02.887 INFO:teuthology.orchestra.run.smithi042.stdout:1507 read 0x24f5365 thru 0x24f564d (0x2e9 bytes) 2024-09-14T23:57:02.887 INFO:teuthology.orchestra.run.smithi042.stdout:1510 read 0xb6215c8 thru 0xb62ca4c (0xb485 bytes) 2024-09-14T23:57:02.887 INFO:teuthology.orchestra.run.smithi042.stdout:1512 writesame 0x88ac76f thru 0x88b8b45 data_size 0x25(0xc3d7 bytes) 2024-09-14T23:57:02.890 INFO:teuthology.orchestra.run.smithi042.stdout:1514 writesame 0x1b38d64 thru 0x1b429cd data_size 0x1(0x9c6a bytes) 2024-09-14T23:57:02.906 INFO:teuthology.orchestra.run.smithi042.stdout:1515 write 0xa6c3156 thru 0xa6c4393 (0x123e bytes) 2024-09-14T23:57:02.909 INFO:teuthology.orchestra.run.smithi042.stdout:1516 writesame 0xaa42fa4 thru 0xaa465e3 data_size 0x20(0x3640 bytes) 2024-09-14T23:57:02.912 INFO:teuthology.orchestra.run.smithi042.stdout:1517 punch from 0x758134f to 0x75845c6, (0x3277 bytes) 2024-09-14T23:57:02.912 INFO:teuthology.orchestra.run.smithi042.stdout:1519 clone 15 order 23 su 2097152 sc 10 2024-09-14T23:57:02.934 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:02.933+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:02.999 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:02.999 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:02.999 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:57:02.999 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:57:02.999 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:03.007 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T23:57:03.049 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:03.047+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:03.070 INFO:teuthology.orchestra.run.smithi042.stderr:set 3.7 pg_upmap_items mapping to [0->3,1->5] 2024-09-14T23:57:03.127 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:03.125+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:03.141 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone14 from 0xd6c1a28 (overlap 0x1050bc) to 0xd0d63 2024-09-14T23:57:03.160 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T23:57:03.162 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T23:57:03.166 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:03.166 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:57:03.166 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:57:03.166 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:57:03.167 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.167 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:3500 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T23:57:03.167 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:37.990193+0000", 2024-09-14T23:57:03.167 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 25.174077321999999, 2024-09-14T23:57:03.167 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99440750899999997, 2024-09-14T23:57:03.167 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:03.167 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:03.167 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:03.167 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:03.167 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:03.167 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:03.167 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3500 2024-09-14T23:57:03.167 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.168 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:03.168 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.168 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:03.168 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990193+0000", 2024-09-14T23:57:03.168 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:03.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.168 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.169 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:03.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990193+0000", 2024-09-14T23:57:03.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:03.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.169 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:03.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990194+0000", 2024-09-14T23:57:03.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5000000000000001e-07 2024-09-14T23:57:03.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.169 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:03.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990200+0000", 2024-09-14T23:57:03.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4389999999999997e-06 2024-09-14T23:57:03.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.170 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.170 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:03.170 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990215+0000", 2024-09-14T23:57:03.170 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4613999999999999e-05 2024-09-14T23:57:03.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.170 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.170 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:03.170 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990224+0000", 2024-09-14T23:57:03.170 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1740000000000006e-06 2024-09-14T23:57:03.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.170 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.170 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:03.170 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990283+0000", 2024-09-14T23:57:03.171 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9231999999999998e-05 2024-09-14T23:57:03.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.171 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.171 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:57:03.171 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990291+0000", 2024-09-14T23:57:03.171 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5900000000000002e-06 2024-09-14T23:57:03.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.171 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.171 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:03.171 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:38.984249+0000", 2024-09-14T23:57:03.171 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99395822700000003 2024-09-14T23:57:03.172 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.172 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.172 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:03.172 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:38.984448+0000", 2024-09-14T23:57:03.172 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00019855900000000001 2024-09-14T23:57:03.172 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.172 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.172 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:03.172 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:38.984601+0000", 2024-09-14T23:57:03.172 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000152824 2024-09-14T23:57:03.172 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:03.172 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:03.172 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:03.173 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.173 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.173 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4055 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1093632~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:03.173 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099066+0000", 2024-09-14T23:57:03.173 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.065204434000002, 2024-09-14T23:57:03.173 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044174782000000003, 2024-09-14T23:57:03.173 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:03.173 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:03.173 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:03.173 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:03.173 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:03.173 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:03.173 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4055 2024-09-14T23:57:03.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.174 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:03.174 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.174 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:03.174 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099066+0000", 2024-09-14T23:57:03.174 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:03.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.174 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.174 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:03.174 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099066+0000", 2024-09-14T23:57:03.174 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:03.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.174 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.174 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:03.174 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099066+0000", 2024-09-14T23:57:03.175 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3700000000000001e-07 2024-09-14T23:57:03.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.175 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.175 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:03.175 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099067+0000", 2024-09-14T23:57:03.175 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1379999999999999e-06 2024-09-14T23:57:03.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.175 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.175 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:03.175 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099073+0000", 2024-09-14T23:57:03.175 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4790000000000004e-06 2024-09-14T23:57:03.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.175 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.176 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:03.176 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099075+0000", 2024-09-14T23:57:03.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5320000000000001e-06 2024-09-14T23:57:03.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.176 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.176 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:03.176 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143147+0000", 2024-09-14T23:57:03.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044071676999999997 2024-09-14T23:57:03.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.176 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.176 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:03.176 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143186+0000", 2024-09-14T23:57:03.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8821e-05 2024-09-14T23:57:03.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.177 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:03.177 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143241+0000", 2024-09-14T23:57:03.177 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4797999999999999e-05 2024-09-14T23:57:03.177 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:03.177 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:03.177 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:03.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.177 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4056 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1155072~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:03.177 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099079+0000", 2024-09-14T23:57:03.177 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.065191719000001, 2024-09-14T23:57:03.178 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044295020999999997, 2024-09-14T23:57:03.178 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:03.178 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:03.178 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:03.178 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:03.178 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:03.178 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:03.178 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4056 2024-09-14T23:57:03.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.178 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:03.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.178 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:03.179 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099079+0000", 2024-09-14T23:57:03.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:03.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.179 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.179 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:03.179 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099079+0000", 2024-09-14T23:57:03.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:03.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.179 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.179 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:03.179 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099079+0000", 2024-09-14T23:57:03.180 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1500000000000001e-07 2024-09-14T23:57:03.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.180 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.180 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:03.180 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099080+0000", 2024-09-14T23:57:03.180 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0210000000000001e-06 2024-09-14T23:57:03.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.180 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.180 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:03.180 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099083+0000", 2024-09-14T23:57:03.180 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.174e-06 2024-09-14T23:57:03.181 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.181 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.181 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:03.181 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099086+0000", 2024-09-14T23:57:03.181 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.446e-06 2024-09-14T23:57:03.181 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.181 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.181 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:03.181 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143269+0000", 2024-09-14T23:57:03.181 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044183489999999999 2024-09-14T23:57:03.181 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.181 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.181 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:03.182 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143314+0000", 2024-09-14T23:57:03.182 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5135000000000001e-05 2024-09-14T23:57:03.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.182 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.182 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:03.182 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143374+0000", 2024-09-14T23:57:03.182 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9540000000000003e-05 2024-09-14T23:57:03.182 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:03.182 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:03.182 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:03.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.182 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.182 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4057 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1216512~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:03.183 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099088+0000", 2024-09-14T23:57:03.183 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.065181932000002, 2024-09-14T23:57:03.183 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044406119000000001, 2024-09-14T23:57:03.183 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:03.183 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:03.183 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:03.183 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:03.183 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:03.183 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:03.183 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4057 2024-09-14T23:57:03.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.183 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:03.183 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.183 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:03.183 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099088+0000", 2024-09-14T23:57:03.184 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:03.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.184 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:03.184 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099088+0000", 2024-09-14T23:57:03.184 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:03.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.184 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:03.184 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099089+0000", 2024-09-14T23:57:03.184 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.04e-07 2024-09-14T23:57:03.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.185 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:03.185 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099090+0000", 2024-09-14T23:57:03.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.026e-06 2024-09-14T23:57:03.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.185 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.185 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:03.185 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099093+0000", 2024-09-14T23:57:03.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0819999999999999e-06 2024-09-14T23:57:03.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.185 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.185 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:03.185 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099095+0000", 2024-09-14T23:57:03.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4930000000000001e-06 2024-09-14T23:57:03.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.186 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.186 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:03.186 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143400+0000", 2024-09-14T23:57:03.186 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044304698000000003 2024-09-14T23:57:03.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.186 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.186 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:03.186 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143441+0000", 2024-09-14T23:57:03.186 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1100000000000003e-05 2024-09-14T23:57:03.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.186 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.186 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:03.186 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143495+0000", 2024-09-14T23:57:03.186 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3516000000000001e-05 2024-09-14T23:57:03.187 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:03.187 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:03.187 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:03.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.187 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.187 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4058 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1277952~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:03.187 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099099+0000", 2024-09-14T23:57:03.187 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.065171441, 2024-09-14T23:57:03.187 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044514583000000003, 2024-09-14T23:57:03.187 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:03.187 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:03.187 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:03.187 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:03.188 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:03.188 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:03.188 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4058 2024-09-14T23:57:03.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.188 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:03.188 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.188 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:03.188 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099099+0000", 2024-09-14T23:57:03.188 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:03.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.188 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.188 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:03.188 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099099+0000", 2024-09-14T23:57:03.188 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:03.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.189 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.189 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:03.189 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099099+0000", 2024-09-14T23:57:03.189 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9999999999999999e-07 2024-09-14T23:57:03.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.189 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.189 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:03.189 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099100+0000", 2024-09-14T23:57:03.189 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7199999999999997e-07 2024-09-14T23:57:03.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.189 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.189 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:03.189 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099103+0000", 2024-09-14T23:57:03.190 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.095e-06 2024-09-14T23:57:03.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.190 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.190 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:03.190 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099106+0000", 2024-09-14T23:57:03.190 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4229999999999999e-06 2024-09-14T23:57:03.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.190 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.190 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:03.190 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143524+0000", 2024-09-14T23:57:03.190 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044417891000000001 2024-09-14T23:57:03.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.190 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.190 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:03.191 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143568+0000", 2024-09-14T23:57:03.191 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4060000000000002e-05 2024-09-14T23:57:03.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.191 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.191 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:03.191 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143614+0000", 2024-09-14T23:57:03.191 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5942000000000002e-05 2024-09-14T23:57:03.191 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:03.191 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:03.191 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:03.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.191 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.191 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4059 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1339392~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:03.192 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099114+0000", 2024-09-14T23:57:03.192 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.065156901999998, 2024-09-14T23:57:03.192 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044617194999999998, 2024-09-14T23:57:03.192 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:03.192 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:03.192 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:03.192 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:03.192 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:03.192 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:03.192 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4059 2024-09-14T23:57:03.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.192 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:03.192 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.193 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:03.193 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099114+0000", 2024-09-14T23:57:03.193 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:03.193 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.193 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.193 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:03.193 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099114+0000", 2024-09-14T23:57:03.193 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:03.193 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.193 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.193 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:03.193 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099114+0000", 2024-09-14T23:57:03.193 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5899999999999998e-07 2024-09-14T23:57:03.193 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.194 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.194 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:03.194 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099115+0000", 2024-09-14T23:57:03.194 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3510000000000001e-06 2024-09-14T23:57:03.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.194 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.194 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:03.194 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099119+0000", 2024-09-14T23:57:03.194 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.16e-06 2024-09-14T23:57:03.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.194 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.194 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:03.194 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099123+0000", 2024-09-14T23:57:03.195 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1339999999999997e-06 2024-09-14T23:57:03.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.195 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.195 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:03.195 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143643+0000", 2024-09-14T23:57:03.195 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044519232999999998 2024-09-14T23:57:03.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.195 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.195 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:03.195 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143686+0000", 2024-09-14T23:57:03.195 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3349000000000001e-05 2024-09-14T23:57:03.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.195 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.195 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:03.195 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143731+0000", 2024-09-14T23:57:03.196 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4709000000000002e-05 2024-09-14T23:57:03.196 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:03.196 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:03.196 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:03.196 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.196 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.196 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4060 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1400832~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:03.196 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099129+0000", 2024-09-14T23:57:03.196 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.065141679, 2024-09-14T23:57:03.196 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044721271999999999, 2024-09-14T23:57:03.196 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:03.196 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:03.197 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:03.197 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:03.197 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:03.197 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:03.197 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4060 2024-09-14T23:57:03.197 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.197 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:03.197 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.197 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:03.197 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099129+0000", 2024-09-14T23:57:03.197 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:03.197 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.197 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.197 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:03.198 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099129+0000", 2024-09-14T23:57:03.198 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:03.198 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.198 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.198 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:03.198 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099191+0000", 2024-09-14T23:57:03.198 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1805999999999997e-05 2024-09-14T23:57:03.198 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.198 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.198 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:03.198 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099193+0000", 2024-09-14T23:57:03.198 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2330000000000001e-06 2024-09-14T23:57:03.198 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.199 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.199 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:03.199 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099200+0000", 2024-09-14T23:57:03.199 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4200000000000001e-06 2024-09-14T23:57:03.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.199 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.199 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:03.199 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099204+0000", 2024-09-14T23:57:03.199 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7500000000000001e-06 2024-09-14T23:57:03.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.199 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.199 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:03.199 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143760+0000", 2024-09-14T23:57:03.199 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044556182 2024-09-14T23:57:03.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.200 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.200 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:03.200 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143804+0000", 2024-09-14T23:57:03.200 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4239000000000001e-05 2024-09-14T23:57:03.200 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.200 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.200 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:03.200 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143850+0000", 2024-09-14T23:57:03.200 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5642000000000001e-05 2024-09-14T23:57:03.200 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:03.200 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:03.200 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:03.200 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.200 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.201 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4061 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1462272~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:03.201 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099215+0000", 2024-09-14T23:57:03.201 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.065055554000001, 2024-09-14T23:57:03.201 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044745659, 2024-09-14T23:57:03.201 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:03.201 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:03.201 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:03.201 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:03.201 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:03.201 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:03.201 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4061 2024-09-14T23:57:03.202 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.202 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:03.202 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.202 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:03.202 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099215+0000", 2024-09-14T23:57:03.202 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:03.202 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.202 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.202 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:03.202 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099215+0000", 2024-09-14T23:57:03.202 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:03.202 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.202 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.203 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:03.203 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100672+0000", 2024-09-14T23:57:03.203 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00145689 2024-09-14T23:57:03.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.203 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.203 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:03.203 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100677+0000", 2024-09-14T23:57:03.203 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0139999999999998e-06 2024-09-14T23:57:03.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.203 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.203 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:03.203 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100689+0000", 2024-09-14T23:57:03.203 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.183e-05 2024-09-14T23:57:03.204 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.204 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.204 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:03.204 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100696+0000", 2024-09-14T23:57:03.204 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2599999999999999e-06 2024-09-14T23:57:03.204 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.204 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.204 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:03.204 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143877+0000", 2024-09-14T23:57:03.204 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043181279000000003 2024-09-14T23:57:03.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.205 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.205 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:03.205 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143919+0000", 2024-09-14T23:57:03.205 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1783000000000003e-05 2024-09-14T23:57:03.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.205 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.205 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:03.205 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143961+0000", 2024-09-14T23:57:03.205 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1603000000000001e-05 2024-09-14T23:57:03.205 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:03.206 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:03.206 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:03.206 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.206 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.206 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4070 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 2015232~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:03.206 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.100903+0000", 2024-09-14T23:57:03.206 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.063367845999998, 2024-09-14T23:57:03.206 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044103815999999997, 2024-09-14T23:57:03.206 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:03.206 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:03.206 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:03.207 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:03.207 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:03.207 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:03.207 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4070 2024-09-14T23:57:03.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.207 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:03.207 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.207 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:03.207 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100903+0000", 2024-09-14T23:57:03.207 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:03.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.207 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.207 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:03.207 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100903+0000", 2024-09-14T23:57:03.207 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:03.208 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.208 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.208 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:03.208 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.102671+0000", 2024-09-14T23:57:03.208 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001768279 2024-09-14T23:57:03.208 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.208 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.208 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:03.208 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.102675+0000", 2024-09-14T23:57:03.208 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9609999999999999e-06 2024-09-14T23:57:03.208 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.208 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.208 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:03.208 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.102684+0000", 2024-09-14T23:57:03.209 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.329e-06 2024-09-14T23:57:03.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.209 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.209 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:03.209 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.102689+0000", 2024-09-14T23:57:03.209 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7079999999999996e-06 2024-09-14T23:57:03.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.209 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.209 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:03.209 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.144912+0000", 2024-09-14T23:57:03.209 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.042223305000000003 2024-09-14T23:57:03.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.209 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.210 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:03.210 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.144960+0000", 2024-09-14T23:57:03.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7670000000000003e-05 2024-09-14T23:57:03.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.210 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:03.210 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.145006+0000", 2024-09-14T23:57:03.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6563999999999997e-05 2024-09-14T23:57:03.210 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:03.210 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:03.210 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:03.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.210 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4096 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1282048~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:03.211 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.104957+0000", 2024-09-14T23:57:03.211 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.059313672999998, 2024-09-14T23:57:03.211 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044093016999999998, 2024-09-14T23:57:03.211 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:03.211 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:03.211 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:03.211 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:03.211 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:03.211 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:03.211 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4096 2024-09-14T23:57:03.212 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.212 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:03.212 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.212 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:03.212 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104957+0000", 2024-09-14T23:57:03.212 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:03.212 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.212 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.212 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:03.212 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104957+0000", 2024-09-14T23:57:03.212 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:03.212 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.212 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.212 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:03.213 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104957+0000", 2024-09-14T23:57:03.213 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7400000000000003e-07 2024-09-14T23:57:03.213 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.213 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.213 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:03.213 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104960+0000", 2024-09-14T23:57:03.213 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3800000000000001e-06 2024-09-14T23:57:03.213 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.213 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.213 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:03.214 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104967+0000", 2024-09-14T23:57:03.214 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4030000000000001e-06 2024-09-14T23:57:03.214 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.214 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.214 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:03.214 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104973+0000", 2024-09-14T23:57:03.214 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7370000000000003e-06 2024-09-14T23:57:03.214 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.214 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.214 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:03.214 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.148883+0000", 2024-09-14T23:57:03.214 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043910124000000002 2024-09-14T23:57:03.214 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.214 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.214 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:03.215 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.148966+0000", 2024-09-14T23:57:03.215 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2868999999999996e-05 2024-09-14T23:57:03.215 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.215 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.215 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:03.215 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149050+0000", 2024-09-14T23:57:03.215 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3930000000000004e-05 2024-09-14T23:57:03.215 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:03.215 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:03.215 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:03.215 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.215 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.215 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4097 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1343488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:03.215 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.104983+0000", 2024-09-14T23:57:03.216 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.059287884, 2024-09-14T23:57:03.216 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044182104999999999, 2024-09-14T23:57:03.216 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:03.216 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:03.216 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:03.216 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:03.216 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:03.216 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:03.216 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4097 2024-09-14T23:57:03.216 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.216 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:03.216 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.216 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:03.217 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104983+0000", 2024-09-14T23:57:03.217 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:03.217 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.217 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.217 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:03.217 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104983+0000", 2024-09-14T23:57:03.217 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:03.217 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.217 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.217 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:03.218 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105207+0000", 2024-09-14T23:57:03.218 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000224687 2024-09-14T23:57:03.218 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.218 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.218 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:03.218 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105211+0000", 2024-09-14T23:57:03.218 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3050000000000001e-06 2024-09-14T23:57:03.218 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.219 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.219 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:03.219 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105217+0000", 2024-09-14T23:57:03.219 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8170000000000001e-06 2024-09-14T23:57:03.219 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.219 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.219 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:03.219 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105222+0000", 2024-09-14T23:57:03.219 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.267e-06 2024-09-14T23:57:03.219 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.219 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.219 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:03.220 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149079+0000", 2024-09-14T23:57:03.220 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043857581999999999 2024-09-14T23:57:03.220 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.220 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.220 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:03.220 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149122+0000", 2024-09-14T23:57:03.220 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2899999999999999e-05 2024-09-14T23:57:03.220 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.220 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.220 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:03.220 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149165+0000", 2024-09-14T23:57:03.220 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2546999999999998e-05 2024-09-14T23:57:03.220 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:03.220 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:03.221 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:03.221 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.221 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.221 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4098 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1404928~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:03.221 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.105242+0000", 2024-09-14T23:57:03.221 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.059028072, 2024-09-14T23:57:03.221 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044121104000000001, 2024-09-14T23:57:03.221 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:03.221 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:03.221 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:03.221 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:03.221 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:03.221 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:03.222 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4098 2024-09-14T23:57:03.222 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.222 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:03.222 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.222 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:03.222 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105242+0000", 2024-09-14T23:57:03.222 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:03.222 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.222 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.223 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:03.223 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105242+0000", 2024-09-14T23:57:03.223 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:03.223 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.223 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.223 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:03.223 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105243+0000", 2024-09-14T23:57:03.224 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0399999999999995e-07 2024-09-14T23:57:03.224 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.224 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.224 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:03.224 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105246+0000", 2024-09-14T23:57:03.224 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6309999999999999e-06 2024-09-14T23:57:03.225 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.225 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.225 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:03.225 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105253+0000", 2024-09-14T23:57:03.225 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0940000000000002e-06 2024-09-14T23:57:03.225 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.226 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.226 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:03.226 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105260+0000", 2024-09-14T23:57:03.226 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1799999999999999e-06 2024-09-14T23:57:03.226 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.226 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.227 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:03.227 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149201+0000", 2024-09-14T23:57:03.227 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043940629000000002 2024-09-14T23:57:03.227 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.227 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.227 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:03.227 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149283+0000", 2024-09-14T23:57:03.228 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2360000000000004e-05 2024-09-14T23:57:03.228 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.228 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.228 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:03.228 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149363+0000", 2024-09-14T23:57:03.228 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0506000000000005e-05 2024-09-14T23:57:03.228 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:03.228 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:03.228 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:03.228 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.228 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.228 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4099 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1466368~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:03.228 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.105270+0000", 2024-09-14T23:57:03.229 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.059000477000001, 2024-09-14T23:57:03.229 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044195433999999999, 2024-09-14T23:57:03.229 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:03.229 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:03.229 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:03.229 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:03.229 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:03.229 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:03.229 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4099 2024-09-14T23:57:03.230 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.230 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:03.230 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.230 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:03.230 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105270+0000", 2024-09-14T23:57:03.230 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:03.231 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.231 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.231 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:03.231 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105270+0000", 2024-09-14T23:57:03.231 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:03.231 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.231 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.231 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:03.231 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105429+0000", 2024-09-14T23:57:03.231 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015940299999999999 2024-09-14T23:57:03.231 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.232 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.232 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:03.232 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105433+0000", 2024-09-14T23:57:03.232 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3579999999999999e-06 2024-09-14T23:57:03.232 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.232 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.232 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:03.232 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105439+0000", 2024-09-14T23:57:03.232 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1759999999999998e-06 2024-09-14T23:57:03.232 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.232 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.232 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:03.232 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105445+0000", 2024-09-14T23:57:03.232 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9689999999999999e-06 2024-09-14T23:57:03.233 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.233 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.233 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:03.233 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149381+0000", 2024-09-14T23:57:03.233 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043936389999999999 2024-09-14T23:57:03.233 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.233 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.233 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:03.233 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149422+0000", 2024-09-14T23:57:03.233 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0865000000000003e-05 2024-09-14T23:57:03.233 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.233 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.233 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:03.233 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149465+0000", 2024-09-14T23:57:03.234 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3272999999999997e-05 2024-09-14T23:57:03.234 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:03.234 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:03.234 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:03.234 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.234 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.234 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4100 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1527808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:03.234 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.105454+0000", 2024-09-14T23:57:03.234 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.058816006000001, 2024-09-14T23:57:03.234 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044217411999999998, 2024-09-14T23:57:03.234 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:03.234 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:03.235 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:03.235 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:03.235 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:03.235 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:03.235 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4100 2024-09-14T23:57:03.235 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.235 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:03.235 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.235 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:03.235 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105454+0000", 2024-09-14T23:57:03.235 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:03.235 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.235 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.236 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:03.236 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105454+0000", 2024-09-14T23:57:03.236 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:03.236 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.236 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.236 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:03.236 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105578+0000", 2024-09-14T23:57:03.236 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012401799999999999 2024-09-14T23:57:03.236 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.236 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.236 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:03.236 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105583+0000", 2024-09-14T23:57:03.236 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1779999999999996e-06 2024-09-14T23:57:03.236 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.237 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.237 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:03.237 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105590+0000", 2024-09-14T23:57:03.237 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4279999999999999e-06 2024-09-14T23:57:03.237 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.237 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.237 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:03.237 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105596+0000", 2024-09-14T23:57:03.237 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0120000000000002e-06 2024-09-14T23:57:03.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.238 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.238 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:03.238 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149503+0000", 2024-09-14T23:57:03.238 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043906580000000001 2024-09-14T23:57:03.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.238 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.239 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:03.239 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149585+0000", 2024-09-14T23:57:03.239 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2356999999999998e-05 2024-09-14T23:57:03.239 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.239 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.239 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:03.240 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149672+0000", 2024-09-14T23:57:03.240 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6839e-05 2024-09-14T23:57:03.240 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:03.240 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:03.240 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:03.240 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.240 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.241 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4101 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1589248~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:03.241 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.105609+0000", 2024-09-14T23:57:03.241 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.058661467, 2024-09-14T23:57:03.241 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044175987999999999, 2024-09-14T23:57:03.241 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:03.241 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:03.241 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:03.241 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:03.242 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:03.242 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:03.242 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4101 2024-09-14T23:57:03.242 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.242 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:03.242 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.242 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:03.242 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105609+0000", 2024-09-14T23:57:03.243 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:03.243 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.243 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.243 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:03.243 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105609+0000", 2024-09-14T23:57:03.243 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:03.244 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.244 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.244 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:03.244 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.106671+0000", 2024-09-14T23:57:03.244 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010619270000000001 2024-09-14T23:57:03.244 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.245 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.245 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:03.245 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.106675+0000", 2024-09-14T23:57:03.245 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4429999999999996e-06 2024-09-14T23:57:03.245 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.245 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.245 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:03.246 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.106685+0000", 2024-09-14T23:57:03.246 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7429999999999997e-06 2024-09-14T23:57:03.246 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.246 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.246 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:03.246 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.106690+0000", 2024-09-14T23:57:03.246 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.293e-06 2024-09-14T23:57:03.246 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.246 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.246 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:03.246 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149700+0000", 2024-09-14T23:57:03.246 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043009512 2024-09-14T23:57:03.247 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.247 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.247 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:03.247 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149742+0000", 2024-09-14T23:57:03.247 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2083999999999999e-05 2024-09-14T23:57:03.247 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.247 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.247 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:03.247 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149785+0000", 2024-09-14T23:57:03.247 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2985999999999998e-05 2024-09-14T23:57:03.247 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:03.247 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:03.247 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:03.247 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.248 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.248 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4116 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 180224~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:03.248 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.107575+0000", 2024-09-14T23:57:03.248 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.056694999000001, 2024-09-14T23:57:03.248 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044157522999999997, 2024-09-14T23:57:03.248 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:03.248 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:03.248 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:03.248 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:03.248 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:03.248 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:03.248 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4116 2024-09-14T23:57:03.248 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.248 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:03.249 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.249 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:03.249 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.107575+0000", 2024-09-14T23:57:03.249 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:03.249 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.249 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.249 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:03.249 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.107575+0000", 2024-09-14T23:57:03.249 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:03.249 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.249 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.250 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:03.250 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108671+0000", 2024-09-14T23:57:03.250 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010954510000000001 2024-09-14T23:57:03.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.250 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.250 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:03.250 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108676+0000", 2024-09-14T23:57:03.250 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6639999999999997e-06 2024-09-14T23:57:03.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.250 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.250 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:03.250 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108685+0000", 2024-09-14T23:57:03.250 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1130000000000007e-06 2024-09-14T23:57:03.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.251 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.251 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:03.251 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108691+0000", 2024-09-14T23:57:03.251 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6019999999999996e-06 2024-09-14T23:57:03.251 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.251 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.251 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:03.251 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.151530+0000", 2024-09-14T23:57:03.251 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.042838345999999999 2024-09-14T23:57:03.251 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.251 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.251 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:03.251 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.151619+0000", 2024-09-14T23:57:03.251 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9302999999999995e-05 2024-09-14T23:57:03.252 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.252 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.252 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:03.252 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.151733+0000", 2024-09-14T23:57:03.252 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011404399999999999 2024-09-14T23:57:03.252 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:03.252 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:03.252 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:03.252 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.252 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.252 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4122 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 548864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:03.252 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.108842+0000", 2024-09-14T23:57:03.252 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.055428160999998, 2024-09-14T23:57:03.252 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044103631999999997, 2024-09-14T23:57:03.253 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:03.253 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:03.253 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:03.253 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:03.253 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:03.253 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:03.253 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4122 2024-09-14T23:57:03.253 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.253 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:03.253 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.253 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:03.253 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108842+0000", 2024-09-14T23:57:03.253 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:03.254 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.254 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.254 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:03.254 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108842+0000", 2024-09-14T23:57:03.254 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:03.254 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.254 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.254 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:03.254 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108843+0000", 2024-09-14T23:57:03.254 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7000000000000001e-07 2024-09-14T23:57:03.254 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.254 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.254 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:03.254 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108844+0000", 2024-09-14T23:57:03.254 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6479999999999999e-06 2024-09-14T23:57:03.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.255 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:03.255 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108848+0000", 2024-09-14T23:57:03.255 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8029999999999998e-06 2024-09-14T23:57:03.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.255 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:03.255 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108851+0000", 2024-09-14T23:57:03.255 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7939999999999998e-06 2024-09-14T23:57:03.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.255 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:03.255 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.152838+0000", 2024-09-14T23:57:03.256 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043987351000000001 2024-09-14T23:57:03.256 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.256 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.256 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:03.256 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.152885+0000", 2024-09-14T23:57:03.256 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7237000000000002e-05 2024-09-14T23:57:03.256 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.256 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.256 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:03.256 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.152946+0000", 2024-09-14T23:57:03.256 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0529000000000003e-05 2024-09-14T23:57:03.256 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:03.256 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:03.256 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:03.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.257 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.257 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4123 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 610304~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:03.257 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.108855+0000", 2024-09-14T23:57:03.257 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.055415832000001, 2024-09-14T23:57:03.257 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044321610999999997, 2024-09-14T23:57:03.257 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:03.257 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:03.257 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:03.257 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:03.257 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:03.257 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:03.257 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4123 2024-09-14T23:57:03.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.258 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:03.258 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.258 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:03.258 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108855+0000", 2024-09-14T23:57:03.258 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:03.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.258 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.258 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:03.258 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108855+0000", 2024-09-14T23:57:03.258 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:03.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.258 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.258 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:03.258 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.110671+0000", 2024-09-14T23:57:03.259 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018159719999999999 2024-09-14T23:57:03.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.259 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.259 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:03.259 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.110676+0000", 2024-09-14T23:57:03.259 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9450000000000001e-06 2024-09-14T23:57:03.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.259 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.259 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:03.259 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.110685+0000", 2024-09-14T23:57:03.259 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5359999999999998e-06 2024-09-14T23:57:03.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.259 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.259 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:03.260 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.110692+0000", 2024-09-14T23:57:03.260 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9280000000000004e-06 2024-09-14T23:57:03.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.260 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.260 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:03.260 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.152984+0000", 2024-09-14T23:57:03.260 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.042291677999999999 2024-09-14T23:57:03.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.260 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.260 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:03.260 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.153066+0000", 2024-09-14T23:57:03.260 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2657999999999994e-05 2024-09-14T23:57:03.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.260 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.153176+0000", 2024-09-14T23:57:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000109894 2024-09-14T23:57:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4136 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.111056+0000", 2024-09-14T23:57:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.053214684, 2024-09-14T23:57:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044097428000000001, 2024-09-14T23:57:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:03.262 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:03.262 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:03.262 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:03.262 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:03.262 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:03.262 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4136 2024-09-14T23:57:03.262 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.262 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:03.262 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.262 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:03.262 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.111056+0000", 2024-09-14T23:57:03.262 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:03.262 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.262 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.262 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:03.263 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.111056+0000", 2024-09-14T23:57:03.263 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:03.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.263 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:03.263 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.112671+0000", 2024-09-14T23:57:03.263 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016153999999999999 2024-09-14T23:57:03.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.264 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.264 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:03.264 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.112675+0000", 2024-09-14T23:57:03.264 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2830000000000003e-06 2024-09-14T23:57:03.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.264 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.264 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:03.264 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.112684+0000", 2024-09-14T23:57:03.264 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7800000000000006e-06 2024-09-14T23:57:03.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.264 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.264 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.112689+0000", 2024-09-14T23:57:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0000000000000004e-06 2024-09-14T23:57:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.155064+0000", 2024-09-14T23:57:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.042374423000000001 2024-09-14T23:57:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.155111+0000", 2024-09-14T23:57:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7367000000000002e-05 2024-09-14T23:57:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.266 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:03.266 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.155153+0000", 2024-09-14T23:57:03.266 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2175000000000002e-05 2024-09-14T23:57:03.266 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:03.266 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:03.266 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:03.266 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.266 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.266 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:46572 3.1d 3:b9e1f524:::rbd_header.1118cee06433:head [watch reconnect cookie 140670782499824 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-14T23:57:03.266 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:57.046868+0000", 2024-09-14T23:57:03.266 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 6.117402878, 2024-09-14T23:57:03.266 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0243412620000001, 2024-09-14T23:57:03.266 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:03.266 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:03.267 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:03.267 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:03.267 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:03.267 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:03.267 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 46572 2024-09-14T23:57:03.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.267 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:03.267 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.267 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:03.267 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046868+0000", 2024-09-14T23:57:03.267 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:03.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.267 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.267 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:03.268 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046868+0000", 2024-09-14T23:57:03.268 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:03.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.268 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.268 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:03.268 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046870+0000", 2024-09-14T23:57:03.268 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.204e-06 2024-09-14T23:57:03.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.269 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.269 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:03.269 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046877+0000", 2024-09-14T23:57:03.269 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5100000000000001e-06 2024-09-14T23:57:03.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.269 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.269 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:03.269 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046898+0000", 2024-09-14T23:57:03.269 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1149999999999999e-05 2024-09-14T23:57:03.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.269 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.269 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:03.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046911+0000", 2024-09-14T23:57:03.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2234e-05 2024-09-14T23:57:03.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.270 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.270 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:03.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046988+0000", 2024-09-14T23:57:03.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7445000000000001e-05 2024-09-14T23:57:03.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.270 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.270 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:57:03.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.047001+0000", 2024-09-14T23:57:03.271 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2951e-05 2024-09-14T23:57:03.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.271 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.271 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:03.271 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.069287+0000", 2024-09-14T23:57:03.271 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.022285748 2024-09-14T23:57:03.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.271 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.271 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for rw locks", 2024-09-14T23:57:03.271 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.069397+0000", 2024-09-14T23:57:03.271 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000110326 2024-09-14T23:57:03.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.271 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.271 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:03.272 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.069997+0000", 2024-09-14T23:57:03.272 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00059984399999999998 2024-09-14T23:57:03.272 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.272 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.272 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:03.272 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.070109+0000", 2024-09-14T23:57:03.273 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000112367 2024-09-14T23:57:03.273 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:03.273 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:03.273 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:03.273 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.071209+0000", 2024-09-14T23:57:03.273 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010994830000000001 2024-09-14T23:57:03.274 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:03.274 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:03.274 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:03.274 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:03.274 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:03.274 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:03.278 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T23:57:03.279 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:03.275+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:03.279 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-09-14T23:57:03.280 INFO:teuthology.orchestra.run.smithi042.stdout:1520 read 0x8dd61dc thru 0x8ddebe0 (0x8a05 bytes) 2024-09-14T23:57:03.280 INFO:teuthology.orchestra.run.smithi042.stdout:1521 write 0x8471ebc thru 0x8479e0d (0x7f52 bytes) 2024-09-14T23:57:03.280 INFO:teuthology.orchestra.run.smithi042.stdout:1522 read 0x97566a3 thru 0x975763d (0xf9b bytes) 2024-09-14T23:57:03.280 INFO:teuthology.orchestra.run.smithi042.stdout:1523 writesame 0xd4aaebc thru 0xd4b7b31 data_size 0x1(0xcc76 bytes) 2024-09-14T23:57:03.280 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:03.280 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:03.280 INFO:teuthology.orchestra.run.smithi042.stdout:1527 write 0x8b607dd thru 0x8b69ecf (0x96f3 bytes) 2024-09-14T23:57:03.281 INFO:teuthology.orchestra.run.smithi042.stdout:1529 read 0x72880e thru 0x72c8ef (0x40e2 bytes) 2024-09-14T23:57:03.281 INFO:teuthology.orchestra.run.smithi042.stdout:1530 write 0x7e126b0 thru 0x7e148b1 (0x2202 bytes) 2024-09-14T23:57:03.287 INFO:teuthology.orchestra.run.smithi042.stdout:1532 write 0xe62622a thru 0xe633ad1 (0xd8a8 bytes) 2024-09-14T23:57:03.292 INFO:teuthology.orchestra.run.smithi042.stdout:1533 trunc from 0xe633ad2 to 0xd7c7b2b 2024-09-14T23:57:03.299 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:57:03.300 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:57:03.306 INFO:teuthology.orchestra.run.smithi042.stdout:1534 write 0x5482e75 thru 0x54860e7 (0x3273 bytes) 2024-09-14T23:57:03.309 INFO:teuthology.orchestra.run.smithi042.stdout:1535 write 0x188de9 thru 0x194464 (0xb67c bytes) 2024-09-14T23:57:03.315 INFO:teuthology.orchestra.run.smithi042.stdout:1536 read 0x6d5740d thru 0x6d62703 (0xb2f7 bytes) 2024-09-14T23:57:03.315 INFO:teuthology.orchestra.run.smithi042.stdout:1537 read 0xd32893e thru 0xd32a976 (0x2039 bytes) 2024-09-14T23:57:03.316 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:03.316 INFO:teuthology.orchestra.run.smithi042.stdout:1539 write 0x83617a9 thru 0x836c328 (0xab80 bytes) 2024-09-14T23:57:03.318 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:57:03.319 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:57:03.319 INFO:teuthology.orchestra.run.smithi042.stdout:1540 write 0x4efe34e thru 0x4f0c55c (0xe20f bytes) 2024-09-14T23:57:03.322 INFO:teuthology.orchestra.run.smithi042.stdout:1542 read 0x3cc58ec thru 0x3ccc118 (0x682d bytes) 2024-09-14T23:57:03.333 INFO:teuthology.orchestra.run.smithi042.stdout:1543 trunc from 0xd7c7b2b to 0x5d11c73 2024-09-14T23:57:03.348 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:57:03.348 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:57:03.351 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:03.365 INFO:teuthology.orchestra.run.smithi042.stdout:1547 write 0xdbd396e thru 0xdbe33d4 (0xfa67 bytes) 2024-09-14T23:57:03.366 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:57:03.366 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:57:03.368 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:03.368 INFO:teuthology.orchestra.run.smithi042.stdout:1549 writesame 0x79e7a25 thru 0x79ea2cc data_size 0x4(0x28a8 bytes) 2024-09-14T23:57:03.370 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:03.370 INFO:teuthology.orchestra.run.smithi042.stdout:1551 read 0x187bada thru 0x1885e08 (0xa32f bytes) 2024-09-14T23:57:03.370 INFO:teuthology.orchestra.run.smithi042.stdout:1552 read 0xc809d0a thru 0xc80f1c8 (0x54bf bytes) 2024-09-14T23:57:03.377 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:03.376+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:03.383 INFO:teuthology.orchestra.run.smithi042.stdout:1553 trunc from 0xdbe33d5 to 0x3de6877 2024-09-14T23:57:03.398 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:57:03.399 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:57:03.404 INFO:teuthology.orchestra.run.smithi042.stdout:1554 punch from 0x200407d to 0x2010dbe, (0xcd41 bytes) 2024-09-14T23:57:03.414 INFO:teuthology.orchestra.run.smithi042.stdout:1557 write 0x7f0b6a1 thru 0x7f17ee1 (0xc841 bytes) 2024-09-14T23:57:03.416 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:03.417 INFO:teuthology.orchestra.run.smithi042.stdout:1559 read 0x63faf65 thru 0x64041fb (0x9297 bytes) 2024-09-14T23:57:03.417 INFO:teuthology.orchestra.run.smithi042.stdout:1560 read 0x47af912 thru 0x47b4324 (0x4a13 bytes) 2024-09-14T23:57:03.417 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:57:03.418 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:57:03.418 INFO:teuthology.orchestra.run.smithi042.stdout:1561 write 0x26f4b9 thru 0x27e4c3 (0xf00b bytes) 2024-09-14T23:57:03.419 INFO:teuthology.orchestra.run.smithi042.stdout:1563 write 0x65877b2 thru 0x658f483 (0x7cd2 bytes) 2024-09-14T23:57:03.421 INFO:teuthology.orchestra.run.smithi042.stdout:1565 read 0x318fa4c thru 0x31980f0 (0x86a5 bytes) 2024-09-14T23:57:03.427 INFO:teuthology.orchestra.run.smithi042.stdout:1566 write 0x8239ea7 thru 0x8245a81 (0xbbdb bytes) 2024-09-14T23:57:03.429 INFO:teuthology.orchestra.run.smithi042.stdout:1569 write 0x11b4f30 thru 0x11b9702 (0x47d3 bytes) 2024-09-14T23:57:03.433 INFO:teuthology.orchestra.run.smithi042.stdout:1570 trunc from 0x8245a82 to 0xa3ebff7 2024-09-14T23:57:03.434 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:57:03.434 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:57:03.438 INFO:teuthology.orchestra.run.smithi042.stdout:1571 punch from 0x8cfe4c1 to 0x8d0e2e2, (0xfe21 bytes) 2024-09-14T23:57:03.439 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:03.439 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:03.446 INFO:teuthology.orchestra.run.smithi042.stdout:1572 write 0xc25f879 thru 0xc266dcd (0x7555 bytes) 2024-09-14T23:57:03.448 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T23:57:03.449 INFO:teuthology.orchestra.run.smithi042.stdout:1573 read 0x4d140c8 thru 0x4d1c616 (0x854f bytes) 2024-09-14T23:57:03.455 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:57:03.455 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:57:03.457 INFO:teuthology.orchestra.run.smithi042.stdout:1575 writesame 0xeb867b7 thru 0xeb87740 data_size 0x27(0xf8a bytes) 2024-09-14T23:57:03.459 INFO:teuthology.orchestra.run.smithi042.stdout:1576 read 0x84519d6 thru 0x84533c3 (0x19ee bytes) 2024-09-14T23:57:03.459 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:03.459 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:03.459 INFO:teuthology.orchestra.run.smithi042.stdout:1579 read 0x343796b thru 0x34428b6 (0xaf4c bytes) 2024-09-14T23:57:03.459 INFO:teuthology.orchestra.run.smithi042.stdout:1580 write 0xb54fbcd thru 0xb55096f (0xda3 bytes) 2024-09-14T23:57:03.463 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:03.463 INFO:teuthology.orchestra.run.smithi042.stdout:1583 read 0x206f992 thru 0x2076b8e (0x71fd bytes) 2024-09-14T23:57:03.463 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:03.471 INFO:teuthology.orchestra.run.smithi042.stdout:1586 trunc from 0xeb87741 to 0x7688356 2024-09-14T23:57:03.477 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:03.476+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:03.490 INFO:teuthology.orchestra.run.smithi042.stdout:1587 write 0xadabcbb thru 0xadb7b6f (0xbeb5 bytes) 2024-09-14T23:57:03.494 INFO:teuthology.orchestra.run.smithi042.stdout:1588 read 0x9eac6e thru 0x9eaea4 (0x237 bytes) 2024-09-14T23:57:03.494 INFO:teuthology.orchestra.run.smithi042.stdout:1589 write 0x13f70c3 thru 0x13fa23d (0x317b bytes) 2024-09-14T23:57:03.506 INFO:teuthology.orchestra.run.smithi042.stdout:1591 trunc from 0xadb7b70 to 0x1bebe15 2024-09-14T23:57:03.526 INFO:teuthology.orchestra.run.smithi042.stdout:1592 trunc from 0x1bebe15 to 0x7885007 2024-09-14T23:57:03.532 INFO:teuthology.orchestra.run.smithi042.stdout:1593 writesame 0x163d642 thru 0x164b8a2 data_size 0x11(0xe261 bytes) 2024-09-14T23:57:03.535 INFO:teuthology.orchestra.run.smithi042.stdout:1594 writesame 0x4ff4a79 thru 0x4ff5904 data_size 0x4(0xe8c bytes) 2024-09-14T23:57:03.536 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:03.546 INFO:teuthology.orchestra.run.smithi042.stdout:1596 write 0x9d23a31 thru 0x9d27e5b (0x442b bytes) 2024-09-14T23:57:03.554 INFO:teuthology.orchestra.run.smithi042.stdout:1597 trunc from 0x9d27e5c to 0xec3d0df 2024-09-14T23:57:03.560 INFO:teuthology.orchestra.run.smithi042.stdout:1598 read 0xea51f3d thru 0xea5676f (0x4833 bytes) 2024-09-14T23:57:03.560 INFO:teuthology.orchestra.run.smithi042.stdout:1599 write 0x80fc08a thru 0x810b1c7 (0xf13e bytes) 2024-09-14T23:57:03.563 INFO:teuthology.orchestra.run.smithi042.stdout:1601 read 0x9fd6bb1 thru 0x9fe17cf (0xac1f bytes) 2024-09-14T23:57:03.563 INFO:teuthology.orchestra.run.smithi042.stdout:1602 read 0xda4918 thru 0xda8fc1 (0x46aa bytes) 2024-09-14T23:57:03.563 INFO:teuthology.orchestra.run.smithi042.stdout:1604 write 0x1f23cd7 thru 0x1f33623 (0xf94d bytes) 2024-09-14T23:57:03.565 INFO:teuthology.orchestra.run.smithi042.stdout:1605 read 0xb5b3e42 thru 0xb5c1c30 (0xddef bytes) 2024-09-14T23:57:03.578 INFO:teuthology.orchestra.run.smithi042.stdout:1606 trunc from 0xec3d0df to 0x2cbe630 2024-09-14T23:57:03.579 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:03.577+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:03.591 INFO:teuthology.orchestra.run.smithi042.stdout:1607 punch from 0x6ead39 to 0x6f7d60, (0xd027 bytes) 2024-09-14T23:57:03.591 INFO:teuthology.orchestra.run.smithi042.stdout:1609 read 0x702809 thru 0x7033af (0xba7 bytes) 2024-09-14T23:57:03.591 INFO:teuthology.orchestra.run.smithi042.stdout:1611 read 0x122f8d0 thru 0x123b91c (0xc04d bytes) 2024-09-14T23:57:03.600 INFO:teuthology.orchestra.run.smithi042.stdout:1612 trunc from 0x2cbe630 to 0xb38a0c4 2024-09-14T23:57:03.602 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:03.602 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:03.606 INFO:teuthology.orchestra.run.smithi042.stdout:1614 read 0x63f45e0 thru 0x63f9c98 (0x56b9 bytes) 2024-09-14T23:57:03.606 INFO:teuthology.orchestra.run.smithi042.stdout:1616 write 0x5addd68 thru 0x5ae0b51 (0x2dea bytes) 2024-09-14T23:57:03.608 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:03.608 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:03.608 INFO:teuthology.orchestra.run.smithi042.stdout:1621 write 0x1f7e970 thru 0x1f8370a (0x4d9b bytes) 2024-09-14T23:57:03.610 INFO:teuthology.orchestra.run.smithi042.stdout:1623 read 0xbc9395 thru 0xbd5525 (0xc191 bytes) 2024-09-14T23:57:03.612 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T23:57:03.616 INFO:teuthology.orchestra.run.smithi042.stdout:1624 writesame 0xb4a792d thru 0xb4b5fc8 data_size 0x2(0xe69c bytes) 2024-09-14T23:57:03.625 INFO:teuthology.orchestra.run.smithi042.stdout:1625 write 0xbdb8b0f thru 0xbdbf8bb (0x6dad bytes) 2024-09-14T23:57:03.625 INFO:teuthology.orchestra.run.smithi042.stdout:1626 read 0x1a92349 thru 0x1a9df5d (0xbc15 bytes) 2024-09-14T23:57:03.625 INFO:teuthology.orchestra.run.smithi042.stdout:1628 writesame 0x8841e1a thru 0x8843656 data_size 0x11(0x183d bytes) 2024-09-14T23:57:03.638 INFO:teuthology.orchestra.run.smithi042.stdout:1629 trunc from 0xbdbf8bc to 0x2683509 2024-09-14T23:57:03.662 INFO:teuthology.orchestra.run.smithi042.stdout:1630 writesame 0x86014ef thru 0x860ca84 data_size 0xb(0xb596 bytes) 2024-09-14T23:57:03.662 INFO:teuthology.orchestra.run.smithi042.stdout:1631 read 0x18ca425 thru 0x18cd50e (0x30ea bytes) 2024-09-14T23:57:03.662 INFO:teuthology.orchestra.run.smithi042.stdout:1633 read 0x20e67ae thru 0x20eaa74 (0x42c7 bytes) 2024-09-14T23:57:03.666 INFO:teuthology.orchestra.run.smithi042.stdout:1634 trunc from 0x860ca85 to 0xb6c8c45 2024-09-14T23:57:03.671 INFO:teuthology.orchestra.run.smithi042.stdout:1635 punch from 0x2c84851 to 0x2c932ad, (0xea5c bytes) 2024-09-14T23:57:03.671 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:03.679 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:03.678+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:03.683 INFO:teuthology.orchestra.run.smithi042.stdout:1637 trunc from 0xb6c8c45 to 0x81a5fd 2024-09-14T23:57:03.704 INFO:teuthology.orchestra.run.smithi042.stdout:1638 trunc from 0x81a5fd to 0x62da607 2024-09-14T23:57:03.709 INFO:teuthology.orchestra.run.smithi042.stdout:1639 read 0x1ad7e24 thru 0x1ada530 (0x270d bytes) 2024-09-14T23:57:03.710 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:03.710 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:03.719 INFO:teuthology.orchestra.run.smithi042.stdout:1642 trunc from 0x62da607 to 0xdf55adb 2024-09-14T23:57:03.724 INFO:teuthology.orchestra.run.smithi042.stdout:1643 punch from 0x688525c to 0x6889339, (0x40dd bytes) 2024-09-14T23:57:03.725 INFO:teuthology.orchestra.run.smithi042.stdout:1644 trunc from 0xdf55adb to 0xe960d25 2024-09-14T23:57:03.728 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:03.729 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:03.730 INFO:teuthology.orchestra.run.smithi042.stdout:1645 read 0x1705d78 thru 0x1711116 (0xb39f bytes) 2024-09-14T23:57:03.730 INFO:teuthology.orchestra.run.smithi042.stdout:1646 clone 16 order 22 su 2097152 sc 11 2024-09-14T23:57:03.738 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T23:57:03.781 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:03.780+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:03.881 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:03.880+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:03.885 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:03.886 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:03.894 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T23:57:03.982 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:03.981+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:04.042 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:04.042 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:04.050 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T23:57:04.078 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is unset 2024-09-14T23:57:04.094 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:04.093+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:04.162 INFO:teuthology.orchestra.run.smithi042.stdout:leaving image image_client.0-clone15 intact 2024-09-14T23:57:04.165 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:04.165 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:04.174 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T23:57:04.184 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:04.183+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:04.285 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:04.284+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:04.323 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:04.323 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:04.332 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T23:57:04.364 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-09-14T23:57:04.368 INFO:teuthology.orchestra.run.smithi042.stdout:1647 read 0x2130a95 thru 0x213c5b6 (0xbb22 bytes) 2024-09-14T23:57:04.369 INFO:teuthology.orchestra.run.smithi042.stdout:1649 read 0x33a5a47 thru 0x33a602a (0x5e4 bytes) 2024-09-14T23:57:04.369 INFO:teuthology.orchestra.run.smithi042.stdout:1650 read 0x45370db thru 0x4543ee9 (0xce0f bytes) 2024-09-14T23:57:04.369 INFO:teuthology.orchestra.run.smithi042.stdout:1651 read 0xab8b15c thru 0xab95d29 (0xabce bytes) 2024-09-14T23:57:04.370 INFO:teuthology.orchestra.run.smithi042.stdout:1652 read 0xd10098c thru 0xd10bc0f (0xb284 bytes) 2024-09-14T23:57:04.379 INFO:teuthology.orchestra.run.smithi042.stdout:1653 trunc from 0xe960d25 to 0x6584efb 2024-09-14T23:57:04.386 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:04.385+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:04.408 INFO:teuthology.orchestra.run.smithi042.stdout:1654 write 0xb729e72 thru 0xb7326c4 (0x8853 bytes) 2024-09-14T23:57:04.419 INFO:teuthology.orchestra.run.smithi042.stdout:1655 write 0xceb0e9f thru 0xceb8229 (0x738b bytes) 2024-09-14T23:57:04.421 INFO:teuthology.orchestra.run.smithi042.stdout:1657 write 0x5621074 thru 0x5623329 (0x22b6 bytes) 2024-09-14T23:57:04.437 INFO:teuthology.orchestra.run.smithi042.stdout:1658 trunc from 0xceb822a to 0x1a848af 2024-09-14T23:57:04.447 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:04.447 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:04.457 INFO:teuthology.orchestra.run.smithi042.stdout:1659 read 0x1a378f2 thru 0x1a45612 (0xdd21 bytes) 2024-09-14T23:57:04.458 INFO:teuthology.orchestra.run.smithi042.stdout:1660 write 0xee4d7e thru 0xeef0e2 (0xa365 bytes) 2024-09-14T23:57:04.458 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T23:57:04.459 INFO:teuthology.orchestra.run.smithi042.stdout:1661 flatten 2024-09-14T23:57:04.474 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:04.474 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:04.474 INFO:teuthology.orchestra.run.smithi042.stdout:1664 read 0x1059a96 thru 0x1065e13 (0xc37e bytes) 2024-09-14T23:57:04.475 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:04.475 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:04.475 INFO:teuthology.orchestra.run.smithi042.stdout:1669 read 0x1532859 thru 0x153fbf6 (0xd39e bytes) 2024-09-14T23:57:04.475 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:04.487 INFO:teuthology.orchestra.run.smithi042.stdout:1672 write 0x79224ca thru 0x792c383 (0x9eba bytes) 2024-09-14T23:57:04.487 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:04.485+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:04.489 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:04.495 INFO:teuthology.orchestra.run.smithi042.stdout:1674 trunc from 0x792c384 to 0xcb725e2 2024-09-14T23:57:04.500 INFO:teuthology.orchestra.run.smithi042.stdout:1675 punch from 0xbb0da01 to 0xbb0e728, (0xd27 bytes) 2024-09-14T23:57:04.500 INFO:teuthology.orchestra.run.smithi042.stdout:1676 writesame 0x5d75526 thru 0x5d7eb2c data_size 0x1(0x9607 bytes) 2024-09-14T23:57:04.519 INFO:teuthology.orchestra.run.smithi042.stdout:1677 read 0xd5d286 thru 0xd5d80d (0x588 bytes) 2024-09-14T23:57:04.520 INFO:teuthology.orchestra.run.smithi042.stdout:1678 writesame 0x7b724e0 thru 0x7b76a19 data_size 0x1(0x453a bytes) 2024-09-14T23:57:04.536 INFO:teuthology.orchestra.run.smithi042.stdout:1680 write 0xe955cc6 thru 0xe962f77 (0xd2b2 bytes) 2024-09-14T23:57:04.539 INFO:teuthology.orchestra.run.smithi042.stdout:1681 read 0x6b48851 thru 0x6b54536 (0xbce6 bytes) 2024-09-14T23:57:04.539 INFO:teuthology.orchestra.run.smithi042.stdout:1682 writesame 0xb74a64f thru 0xb74d500 data_size 0x2b(0x2eb2 bytes) 2024-09-14T23:57:04.541 INFO:teuthology.orchestra.run.smithi042.stdout:1683 read 0x7029c3a thru 0x703662b (0xc9f2 bytes) 2024-09-14T23:57:04.556 INFO:teuthology.orchestra.run.smithi042.stdout:1684 trunc from 0xe962f78 to 0x194afab 2024-09-14T23:57:04.580 INFO:teuthology.orchestra.run.smithi042.stdout:1686 trunc from 0x194afab to 0xbdc2c73 2024-09-14T23:57:04.585 INFO:teuthology.orchestra.run.smithi042.stdout:1687 read 0xae3e86c thru 0xae45ea4 (0x7639 bytes) 2024-09-14T23:57:04.586 INFO:teuthology.orchestra.run.smithi042.stdout:1689 read 0x204ee02 thru 0x2054d15 (0x5f14 bytes) 2024-09-14T23:57:04.586 INFO:teuthology.orchestra.run.smithi042.stdout:1691 read 0x486d11b thru 0x486edbf (0x1ca5 bytes) 2024-09-14T23:57:04.586 INFO:teuthology.orchestra.run.smithi042.stdout:1692 writesame 0x7521a38 thru 0x753064a data_size 0x5(0xec13 bytes) 2024-09-14T23:57:04.588 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:04.586+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:04.593 INFO:teuthology.orchestra.run.smithi042.stdout:1693 punch from 0x3f97641 to 0x3fa533d, (0xdcfc bytes) 2024-09-14T23:57:04.593 INFO:teuthology.orchestra.run.smithi042.stdout:1694 read 0x2d4c26a thru 0x2d53170 (0x6f07 bytes) 2024-09-14T23:57:04.593 INFO:teuthology.orchestra.run.smithi042.stdout:1696 read 0x673564d thru 0x673b16a (0x5b1e bytes) 2024-09-14T23:57:04.594 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:04.594 INFO:teuthology.orchestra.run.smithi042.stdout:1698 read 0xc0d8e2 thru 0xc0f48b (0x1baa bytes) 2024-09-14T23:57:04.594 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:04.594 INFO:teuthology.orchestra.run.smithi042.stdout:1701 read 0xb00ec97 thru 0xb016d2f (0x8099 bytes) 2024-09-14T23:57:04.594 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:04.595 INFO:teuthology.orchestra.run.smithi042.stdout:1703 read 0x6c04d1a thru 0x6c0da2a (0x8d11 bytes) 2024-09-14T23:57:04.595 INFO:teuthology.orchestra.run.smithi042.stdout:1704 read 0x47f27b2 thru 0x47f7de9 (0x5638 bytes) 2024-09-14T23:57:04.595 INFO:teuthology.orchestra.run.smithi042.stdout:1705 writesame 0xb935415 thru 0xb935993 data_size 0x3(0x57f bytes) 2024-09-14T23:57:04.597 INFO:teuthology.orchestra.run.smithi042.stdout:1707 write 0x91cf241 thru 0x91d0f4b (0x1d0b bytes) 2024-09-14T23:57:04.600 INFO:teuthology.orchestra.run.smithi042.stdout:1708 write 0x58a899e thru 0x58aac55 (0x22b8 bytes) 2024-09-14T23:57:04.602 INFO:teuthology.orchestra.run.smithi042.stdout:1709 writesame 0xbce04e7 thru 0xbce77fd data_size 0x3d(0x7317 bytes) 2024-09-14T23:57:04.605 INFO:teuthology.orchestra.run.smithi042.stdout:1710 punch from 0x7db16f3 to 0x7db88e5, (0x71f2 bytes) 2024-09-14T23:57:04.605 INFO:teuthology.orchestra.run.smithi042.stdout:1711 write 0xa601fd2 thru 0xa60c553 (0xa582 bytes) 2024-09-14T23:57:04.607 INFO:teuthology.orchestra.run.smithi042.stdout:1713 writesame 0xbbcffba thru 0xbbd193d data_size 0x17(0x1984 bytes) 2024-09-14T23:57:04.609 INFO:teuthology.orchestra.run.smithi042.stdout:1714 read 0x7540046 thru 0x754f011 (0xefcc bytes) 2024-09-14T23:57:04.610 INFO:teuthology.orchestra.run.smithi042.stdout:1716 read 0x4f9fe1c thru 0x4fa0d3d (0xf22 bytes) 2024-09-14T23:57:04.610 INFO:teuthology.orchestra.run.smithi042.stdout:1720 read 0x7cc4e49 thru 0x7ccf4b7 (0xa66f bytes) 2024-09-14T23:57:04.611 INFO:teuthology.orchestra.run.smithi042.stdout:1721 read 0x636c5e8 thru 0x63756a1 (0x90ba bytes) 2024-09-14T23:57:04.611 INFO:teuthology.orchestra.run.smithi042.stdout:1722 punch from 0x7b2201d to 0x7b22b85, (0xb68 bytes) 2024-09-14T23:57:04.611 INFO:teuthology.orchestra.run.smithi042.stdout:1723 read 0x126044e thru 0x1260cd8 (0x88b bytes) 2024-09-14T23:57:04.611 INFO:teuthology.orchestra.run.smithi042.stdout:1724 read 0x2762157 thru 0x276da37 (0xb8e1 bytes) 2024-09-14T23:57:04.611 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:04.611 INFO:teuthology.orchestra.run.smithi042.stdout:1726 writesame 0x2028e40 thru 0x20388f7 data_size 0x8e(0xfab8 bytes) 2024-09-14T23:57:04.612 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:04.613 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:04.613 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:04.613 INFO:teuthology.orchestra.run.smithi042.stdout:1729 read 0x391f216 thru 0x392a4b5 (0xb2a0 bytes) 2024-09-14T23:57:04.613 INFO:teuthology.orchestra.run.smithi042.stdout:1730 writesame 0x721d4f3 thru 0x722cd2b data_size 0x5(0xf839 bytes) 2024-09-14T23:57:04.621 INFO:teuthology.orchestra.run.smithi042.stdout:1731 read 0xa801f7f thru 0xa80760f (0x5691 bytes) 2024-09-14T23:57:04.621 INFO:teuthology.orchestra.run.smithi042.stdout:1732 punch from 0xb346f55 to 0xb34f000, (0x80ab bytes) 2024-09-14T23:57:04.622 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T23:57:04.622 INFO:teuthology.orchestra.run.smithi042.stdout:1734 trunc from 0xbdc2c73 to 0xb1a86bb 2024-09-14T23:57:04.633 INFO:teuthology.orchestra.run.smithi042.stdout:1735 read 0x1394138 thru 0x139cb10 (0x89d9 bytes) 2024-09-14T23:57:04.633 INFO:teuthology.orchestra.run.smithi042.stdout:1736 write 0x9419dcb thru 0x9419ee6 (0x11c bytes) 2024-09-14T23:57:04.636 INFO:teuthology.orchestra.run.smithi042.stdout:1737 write 0xa27db7e thru 0xa285b7a (0x7ffd bytes) 2024-09-14T23:57:04.648 INFO:teuthology.orchestra.run.smithi042.stdout:1739 trunc from 0xb1a86bb to 0x37b847c 2024-09-14T23:57:04.657 INFO:teuthology.orchestra.run.smithi042.stdout:1741 clone 17 order 22 su 16384 sc 11 2024-09-14T23:57:04.689 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:04.688+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:04.732 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:04.733 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:04.741 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T23:57:04.825 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:04.824+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:04.889 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:04.889 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:04.891 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:04.890+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:04.897 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T23:57:04.992 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:04.990+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:05.045 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:05.046 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:05.055 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T23:57:05.093 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:05.091+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:05.193 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:05.192+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:05.207 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:05.207 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:05.209 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-09-14T23:57:05.218 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T23:57:05.303 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:05.302+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:05.373 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:05.373 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:05.382 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T23:57:05.395 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:05.394+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:05.461 INFO:teuthology.orchestra.run.smithi042.stdout:1742 clone 18 order 18 su 8192 sc 9 2024-09-14T23:57:05.470 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T23:57:05.514 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:05.513+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:05.533 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:05.533 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:05.543 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T23:57:05.596 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:57:05.597 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:57:05.597 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:05.595+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:05.613 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:57:05.614 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:57:05.630 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:57:05.630 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:57:05.646 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:57:05.646 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:57:05.664 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:57:05.664 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:57:05.681 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:57:05.681 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:57:05.690 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:05.691 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:57:05.691 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:57:05.691 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:05.698 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:05.696+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:05.698 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:57:05.698 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:57:05.700 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T23:57:05.715 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:57:05.716 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:57:05.799 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:05.797+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:05.823 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:05.823 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:57:05.823 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:57:05.823 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:57:05.823 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:05.832 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T23:57:05.916 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:05.915+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:05.981 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:05.982 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:57:05.982 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:57:05.982 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:57:05.982 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:05.982 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:3578 3.4 e29/23 3:2298d860:::rbd_data.1118de1168b.0000000000000030:head v 29'15, mlcod=29'13)", 2024-09-14T23:57:05.982 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.036455+0000", 2024-09-14T23:57:05.982 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.942967543000002, 2024-09-14T23:57:05.982 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019449440000000001, 2024-09-14T23:57:05.982 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:05.982 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:05.982 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:05.982 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:05.982 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:05.983 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:05.983 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036455+0000", 2024-09-14T23:57:05.983 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:05.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:05.983 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:05.983 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:05.983 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036455+0000", 2024-09-14T23:57:05.983 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:05.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:05.983 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:05.983 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:05.983 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036456+0000", 2024-09-14T23:57:05.983 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0999999999999997e-07 2024-09-14T23:57:05.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:05.984 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:05.984 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:05.984 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036463+0000", 2024-09-14T23:57:05.984 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2200000000000003e-06 2024-09-14T23:57:05.984 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:05.984 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:05.984 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:05.984 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036478+0000", 2024-09-14T23:57:05.984 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4977000000000001e-05 2024-09-14T23:57:05.984 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:05.984 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:05.984 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:05.984 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036485+0000", 2024-09-14T23:57:05.985 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0559999999999999e-06 2024-09-14T23:57:05.985 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:05.985 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:05.985 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:05.985 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036693+0000", 2024-09-14T23:57:05.985 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000208051 2024-09-14T23:57:05.985 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:05.985 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:05.985 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:05.985 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036715+0000", 2024-09-14T23:57:05.985 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1330000000000001e-05 2024-09-14T23:57:05.985 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:05.985 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:05.986 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:05.986 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.038356+0000", 2024-09-14T23:57:05.986 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001641586 2024-09-14T23:57:05.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:05.986 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:05.986 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:05.986 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.038400+0000", 2024-09-14T23:57:05.986 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3914e-05 2024-09-14T23:57:05.986 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:05.986 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:05.986 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:05.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:05.986 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:05.986 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:3579 3.4 e29/23 3:27290fe6:::rbd_data.1118de1168b.0000000000000032:head v 29'16, mlcod=29'13)", 2024-09-14T23:57:05.986 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.037085+0000", 2024-09-14T23:57:05.987 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.942337952999999, 2024-09-14T23:57:05.987 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0020019220000000002, 2024-09-14T23:57:05.987 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:05.987 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:05.987 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:05.987 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:05.987 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:05.987 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:05.987 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037085+0000", 2024-09-14T23:57:05.987 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:05.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:05.987 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:05.987 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:05.987 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037085+0000", 2024-09-14T23:57:05.988 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:05.988 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:05.988 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:05.988 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:05.988 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037085+0000", 2024-09-14T23:57:05.988 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5499999999999997e-07 2024-09-14T23:57:05.988 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:05.988 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:05.988 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:05.988 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037091+0000", 2024-09-14T23:57:05.988 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7869999999999998e-06 2024-09-14T23:57:05.988 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:05.988 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:05.988 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:05.989 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037102+0000", 2024-09-14T23:57:05.989 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.031e-05 2024-09-14T23:57:05.989 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:05.989 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:05.989 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:05.989 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037116+0000", 2024-09-14T23:57:05.989 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4119e-05 2024-09-14T23:57:05.989 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:05.989 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:05.989 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:05.989 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037219+0000", 2024-09-14T23:57:05.989 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010372799999999999 2024-09-14T23:57:05.989 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:05.989 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:05.990 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:05.990 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037239+0000", 2024-09-14T23:57:05.990 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9993e-05 2024-09-14T23:57:05.990 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:05.990 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:05.990 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:05.990 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.039046+0000", 2024-09-14T23:57:05.990 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001806666 2024-09-14T23:57:05.990 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:05.990 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:05.990 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:05.990 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.039087+0000", 2024-09-14T23:57:05.990 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0564e-05 2024-09-14T23:57:05.990 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:05.991 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:05.991 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:05.991 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:05.991 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:05.991 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:3584 3.4 e29/23 3:2779453d:::rbd_data.1118de1168b.0000000000000034:head v 29'17, mlcod=29'13)", 2024-09-14T23:57:05.991 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.037588+0000", 2024-09-14T23:57:05.991 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.941834914000001, 2024-09-14T23:57:05.991 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001526828, 2024-09-14T23:57:05.991 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:05.991 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:05.991 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:05.991 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:05.991 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:05.991 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:05.991 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037588+0000", 2024-09-14T23:57:05.992 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:05.992 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:05.992 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:05.992 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:05.992 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037588+0000", 2024-09-14T23:57:05.992 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:05.992 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:05.992 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:05.992 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:05.992 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037588+0000", 2024-09-14T23:57:05.992 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1200000000000002e-07 2024-09-14T23:57:05.992 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:05.992 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:05.992 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:05.993 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037595+0000", 2024-09-14T23:57:05.993 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7580000000000003e-06 2024-09-14T23:57:05.993 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:05.993 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:05.993 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:05.993 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037609+0000", 2024-09-14T23:57:05.993 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4037e-05 2024-09-14T23:57:05.993 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:05.994 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:05.994 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:05.994 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037616+0000", 2024-09-14T23:57:05.994 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0509999999999999e-06 2024-09-14T23:57:05.994 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:05.994 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:05.994 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:05.994 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037787+0000", 2024-09-14T23:57:05.994 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017084899999999999 2024-09-14T23:57:05.994 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:05.994 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:05.994 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:05.994 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037806+0000", 2024-09-14T23:57:05.994 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9159000000000001e-05 2024-09-14T23:57:05.995 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:05.995 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:05.995 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:05.995 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.039089+0000", 2024-09-14T23:57:05.995 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012823909999999999 2024-09-14T23:57:05.995 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:05.995 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:05.995 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:05.995 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.039115+0000", 2024-09-14T23:57:05.995 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5871e-05 2024-09-14T23:57:05.995 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:05.995 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:05.995 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:05.996 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:05.996 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:05.996 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:5280 3.0 e29/23 3:0760f535:::rbd_data.1118de1168b.000000000000009a:head v 29'11, mlcod=29'9)", 2024-09-14T23:57:05.996 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.262702+0000", 2024-09-14T23:57:05.996 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.716720534, 2024-09-14T23:57:05.996 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001433122, 2024-09-14T23:57:05.996 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:05.996 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:05.996 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:05.996 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:05.996 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:05.997 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:05.997 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262702+0000", 2024-09-14T23:57:05.997 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:05.997 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:05.997 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:05.997 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:05.997 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262702+0000", 2024-09-14T23:57:05.997 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:05.997 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:05.997 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:05.997 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:05.997 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262704+0000", 2024-09-14T23:57:05.997 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.384e-06 2024-09-14T23:57:05.998 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:05.998 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:05.998 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:05.998 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262791+0000", 2024-09-14T23:57:05.998 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7264999999999999e-05 2024-09-14T23:57:05.998 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:05.998 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:05.998 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:05.998 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262813+0000", 2024-09-14T23:57:05.998 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.173e-05 2024-09-14T23:57:05.998 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:05.998 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:05.998 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:05.999 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262822+0000", 2024-09-14T23:57:05.999 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5929999999999995e-06 2024-09-14T23:57:05.999 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:05.999 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:05.999 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:05.999 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262886+0000", 2024-09-14T23:57:05.999 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4052000000000003e-05 2024-09-14T23:57:05.999 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:05.999 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:05.999 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:05.999 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262920+0000", 2024-09-14T23:57:05.999 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.366e-05 2024-09-14T23:57:05.999 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:05.999 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.000 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:06.000 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.264095+0000", 2024-09-14T23:57:06.000 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001175178 2024-09-14T23:57:06.000 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.000 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.000 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:06.000 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.264135+0000", 2024-09-14T23:57:06.000 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0259999999999997e-05 2024-09-14T23:57:06.000 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.000 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:06.000 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.000 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.000 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.000 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:5281 3.0 e29/23 3:0760f535:::rbd_data.1118de1168b.000000000000009a:head v 29'13, mlcod=29'9)", 2024-09-14T23:57:06.001 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.263503+0000", 2024-09-14T23:57:06.001 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.715919670000002, 2024-09-14T23:57:06.001 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012834910000000001, 2024-09-14T23:57:06.001 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:06.001 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:06.001 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:06.001 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:06.001 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.001 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:06.001 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.263503+0000", 2024-09-14T23:57:06.001 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.001 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.001 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.002 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:06.002 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.263503+0000", 2024-09-14T23:57:06.002 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.002 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.002 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.002 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:06.002 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.263504+0000", 2024-09-14T23:57:06.002 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4600000000000004e-07 2024-09-14T23:57:06.002 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.002 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.003 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:06.003 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.263510+0000", 2024-09-14T23:57:06.003 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5950000000000004e-06 2024-09-14T23:57:06.003 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.003 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.003 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:06.003 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.263522+0000", 2024-09-14T23:57:06.003 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1289e-05 2024-09-14T23:57:06.003 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.003 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.003 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:06.003 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.263529+0000", 2024-09-14T23:57:06.004 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.951e-06 2024-09-14T23:57:06.004 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.004 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.004 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:06.004 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.263656+0000", 2024-09-14T23:57:06.004 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012731099999999999 2024-09-14T23:57:06.004 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.004 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.004 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:06.004 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.263675+0000", 2024-09-14T23:57:06.004 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9147000000000002e-05 2024-09-14T23:57:06.004 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.004 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.005 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:06.005 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.264746+0000", 2024-09-14T23:57:06.005 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001070997 2024-09-14T23:57:06.005 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.005 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.005 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:06.005 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.264787+0000", 2024-09-14T23:57:06.005 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0555e-05 2024-09-14T23:57:06.005 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.005 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:06.005 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.005 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.005 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.005 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:17221 3.4 e33/23 3:225c4d41:::rbd_data.11187be34cc8.0000000000000001:head v 33'50, mlcod=32'48)", 2024-09-14T23:57:06.005 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:42.983242+0000", 2024-09-14T23:57:06.006 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.996181153999999, 2024-09-14T23:57:06.006 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001313032, 2024-09-14T23:57:06.006 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:06.006 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:06.006 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:06.006 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:06.006 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.006 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:06.006 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.983242+0000", 2024-09-14T23:57:06.006 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.006 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.006 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.006 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:06.006 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.983242+0000", 2024-09-14T23:57:06.007 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.007 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.007 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.007 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:06.007 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.983243+0000", 2024-09-14T23:57:06.007 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.102e-06 2024-09-14T23:57:06.007 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.007 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.007 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:06.007 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.983256+0000", 2024-09-14T23:57:06.007 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2897e-05 2024-09-14T23:57:06.007 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.007 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.007 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:06.008 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.983277+0000", 2024-09-14T23:57:06.008 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1566999999999999e-05 2024-09-14T23:57:06.008 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.008 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.008 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:06.008 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.983288+0000", 2024-09-14T23:57:06.008 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0457e-05 2024-09-14T23:57:06.008 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.008 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.008 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:06.008 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.983355+0000", 2024-09-14T23:57:06.008 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7337999999999998e-05 2024-09-14T23:57:06.008 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.008 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.009 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:06.009 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.983422+0000", 2024-09-14T23:57:06.009 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6716000000000003e-05 2024-09-14T23:57:06.009 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.009 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.009 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:06.009 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.984515+0000", 2024-09-14T23:57:06.009 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001093761 2024-09-14T23:57:06.009 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.009 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.009 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:06.009 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:42.984555+0000", 2024-09-14T23:57:06.009 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9193999999999998e-05 2024-09-14T23:57:06.009 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.010 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:06.010 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.010 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.010 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.010 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:17453 3.0 e33/23 3:020fb8a1:::rbd_data.11187be34cc8.0000000000000005:head v 33'46, mlcod=33'43)", 2024-09-14T23:57:06.010 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:43.265507+0000", 2024-09-14T23:57:06.010 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.713915306000001, 2024-09-14T23:57:06.010 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013288880000000001, 2024-09-14T23:57:06.010 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:06.010 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:06.010 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:06.010 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:06.010 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.010 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:06.011 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.265507+0000", 2024-09-14T23:57:06.011 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.011 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.011 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.011 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:06.011 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.265507+0000", 2024-09-14T23:57:06.011 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.011 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.011 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.011 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:06.011 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.265508+0000", 2024-09-14T23:57:06.011 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4400000000000001e-07 2024-09-14T23:57:06.011 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.011 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.012 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:06.012 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.265513+0000", 2024-09-14T23:57:06.012 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.887e-06 2024-09-14T23:57:06.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.012 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.012 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:06.012 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.265520+0000", 2024-09-14T23:57:06.012 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8249999999999999e-06 2024-09-14T23:57:06.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.012 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.012 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:06.012 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.265524+0000", 2024-09-14T23:57:06.012 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8130000000000003e-06 2024-09-14T23:57:06.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.013 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.013 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:06.013 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.265727+0000", 2024-09-14T23:57:06.013 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000202518 2024-09-14T23:57:06.013 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.013 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.013 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:06.013 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.265744+0000", 2024-09-14T23:57:06.013 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.753e-05 2024-09-14T23:57:06.013 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.013 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.013 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:06.013 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.266802+0000", 2024-09-14T23:57:06.013 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001057992 2024-09-14T23:57:06.014 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.014 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.014 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:06.014 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.266836+0000", 2024-09-14T23:57:06.014 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3878999999999999e-05 2024-09-14T23:57:06.014 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.014 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:06.014 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.014 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.014 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.014 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:17657 3.0 e33/23 3:05bad47a:::rbd_data.11187be34cc8.0000000000000003:head v 33'53, mlcod=33'51)", 2024-09-14T23:57:06.014 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:43.493115+0000", 2024-09-14T23:57:06.014 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.486307221000001, 2024-09-14T23:57:06.014 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0014551950000000001, 2024-09-14T23:57:06.015 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:06.015 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:06.015 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:06.015 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:06.015 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.015 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:06.015 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.493115+0000", 2024-09-14T23:57:06.015 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.015 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.015 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:06.015 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.493115+0000", 2024-09-14T23:57:06.015 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.016 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.016 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.016 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:06.016 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.493116+0000", 2024-09-14T23:57:06.016 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7899999999999998e-07 2024-09-14T23:57:06.016 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.016 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.016 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:06.016 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.493121+0000", 2024-09-14T23:57:06.016 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3959999999999997e-06 2024-09-14T23:57:06.016 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.016 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.016 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:06.016 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.493131+0000", 2024-09-14T23:57:06.017 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4900000000000006e-06 2024-09-14T23:57:06.017 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.017 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.017 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:06.017 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.493138+0000", 2024-09-14T23:57:06.017 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6730000000000003e-06 2024-09-14T23:57:06.017 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.017 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.017 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:06.017 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.493263+0000", 2024-09-14T23:57:06.017 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000125468 2024-09-14T23:57:06.017 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.017 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.018 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:06.018 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.493282+0000", 2024-09-14T23:57:06.018 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8831999999999999e-05 2024-09-14T23:57:06.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.018 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.018 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:06.018 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.494528+0000", 2024-09-14T23:57:06.018 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012456990000000001 2024-09-14T23:57:06.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.018 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.018 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:06.018 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.494571+0000", 2024-09-14T23:57:06.018 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3058e-05 2024-09-14T23:57:06.018 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.019 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:06.019 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.019 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.019 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.019 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:17997 3.0 e33/23 3:05bad47a:::rbd_data.11187be34cc8.0000000000000003:head v 33'68, mlcod=33'66)", 2024-09-14T23:57:06.019 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:43.822910+0000", 2024-09-14T23:57:06.019 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.156512678999999, 2024-09-14T23:57:06.019 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001512535, 2024-09-14T23:57:06.019 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:06.019 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:06.019 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:06.019 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:06.019 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.019 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:06.020 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.822910+0000", 2024-09-14T23:57:06.020 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.020 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.020 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.020 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:06.020 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.822910+0000", 2024-09-14T23:57:06.020 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.020 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.020 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.020 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:06.020 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.822910+0000", 2024-09-14T23:57:06.020 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9999999999999998e-07 2024-09-14T23:57:06.020 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.020 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.021 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:06.021 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.822915+0000", 2024-09-14T23:57:06.021 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7790000000000003e-06 2024-09-14T23:57:06.021 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.021 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.021 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:06.021 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.822923+0000", 2024-09-14T23:57:06.021 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2300000000000008e-06 2024-09-14T23:57:06.021 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.021 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.021 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:06.021 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.822929+0000", 2024-09-14T23:57:06.021 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5219999999999998e-06 2024-09-14T23:57:06.021 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.022 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.022 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:06.022 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.823001+0000", 2024-09-14T23:57:06.022 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1903000000000006e-05 2024-09-14T23:57:06.022 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.022 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.022 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:06.022 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.823021+0000", 2024-09-14T23:57:06.022 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0129e-05 2024-09-14T23:57:06.022 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.022 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.022 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:06.022 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.824376+0000", 2024-09-14T23:57:06.023 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013548550000000001 2024-09-14T23:57:06.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.023 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.023 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:06.023 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.824423+0000", 2024-09-14T23:57:06.023 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6717e-05 2024-09-14T23:57:06.023 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.023 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:06.023 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.023 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.023 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:17999 3.0 e33/23 3:020fb8a1:::rbd_data.11187be34cc8.0000000000000005:head v 33'69, mlcod=33'66)", 2024-09-14T23:57:06.023 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:43.823649+0000", 2024-09-14T23:57:06.023 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.155773414999999, 2024-09-14T23:57:06.023 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013396320000000001, 2024-09-14T23:57:06.024 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:06.024 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:06.024 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:06.024 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:06.024 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.024 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:06.024 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.823649+0000", 2024-09-14T23:57:06.024 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.024 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.024 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.024 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:06.024 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.823649+0000", 2024-09-14T23:57:06.024 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.025 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.025 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.025 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:06.025 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.823650+0000", 2024-09-14T23:57:06.025 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7499999999999999e-07 2024-09-14T23:57:06.025 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.025 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.025 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:06.025 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.823656+0000", 2024-09-14T23:57:06.025 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1500000000000004e-06 2024-09-14T23:57:06.025 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.025 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.025 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:06.025 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.823667+0000", 2024-09-14T23:57:06.026 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0788000000000001e-05 2024-09-14T23:57:06.026 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.026 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.026 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:06.026 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.823673+0000", 2024-09-14T23:57:06.026 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4099999999999996e-06 2024-09-14T23:57:06.026 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.026 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.026 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:06.026 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.823722+0000", 2024-09-14T23:57:06.026 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8733000000000002e-05 2024-09-14T23:57:06.026 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.026 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.026 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:06.027 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.823741+0000", 2024-09-14T23:57:06.027 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9156000000000001e-05 2024-09-14T23:57:06.027 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.027 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.027 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:06.027 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.824961+0000", 2024-09-14T23:57:06.027 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001219901 2024-09-14T23:57:06.027 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.027 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.027 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:06.027 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.824989+0000", 2024-09-14T23:57:06.027 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7619000000000001e-05 2024-09-14T23:57:06.027 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.028 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:06.028 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.028 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.028 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:18030 3.4 e33/23 3:225c4d41:::rbd_data.11187be34cc8.0000000000000001:head v 33'63, mlcod=33'61)", 2024-09-14T23:57:06.028 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:43.852748+0000", 2024-09-14T23:57:06.028 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.126674855000001, 2024-09-14T23:57:06.028 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001530696, 2024-09-14T23:57:06.028 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:06.028 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:06.028 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:06.028 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:06.028 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.028 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:06.029 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.852748+0000", 2024-09-14T23:57:06.029 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.029 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.029 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.029 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:06.029 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.852748+0000", 2024-09-14T23:57:06.029 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.029 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.029 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.029 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:06.029 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.852749+0000", 2024-09-14T23:57:06.029 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.164e-06 2024-09-14T23:57:06.029 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.029 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.030 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:06.030 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.852759+0000", 2024-09-14T23:57:06.030 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0265000000000001e-05 2024-09-14T23:57:06.030 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.030 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.030 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:06.030 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.852779+0000", 2024-09-14T23:57:06.030 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9247999999999999e-05 2024-09-14T23:57:06.030 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.030 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.030 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:06.030 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.852788+0000", 2024-09-14T23:57:06.030 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3260000000000002e-06 2024-09-14T23:57:06.030 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.031 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.031 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:06.031 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.853121+0000", 2024-09-14T23:57:06.031 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00033336699999999999 2024-09-14T23:57:06.031 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.031 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.031 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:06.031 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.853142+0000", 2024-09-14T23:57:06.031 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0500999999999999e-05 2024-09-14T23:57:06.031 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.031 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.031 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:06.031 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.854240+0000", 2024-09-14T23:57:06.031 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001098665 2024-09-14T23:57:06.032 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.032 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.032 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:06.032 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.854279+0000", 2024-09-14T23:57:06.032 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8160000000000001e-05 2024-09-14T23:57:06.032 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.032 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:06.032 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.032 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.032 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.032 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:18356 3.0 e34/23 3:05bad47a:::rbd_data.11187be34cc8.0000000000000003:head v 34'84, mlcod=34'82)", 2024-09-14T23:57:06.032 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:44.206422+0000", 2024-09-14T23:57:06.032 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 21.773000262, 2024-09-14T23:57:06.032 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001310603, 2024-09-14T23:57:06.033 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:06.033 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:06.033 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:06.033 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:06.033 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.033 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:06.033 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.206422+0000", 2024-09-14T23:57:06.033 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.033 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.033 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.033 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:06.033 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.206422+0000", 2024-09-14T23:57:06.033 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.033 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.034 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.034 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:06.034 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.206424+0000", 2024-09-14T23:57:06.034 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3e-06 2024-09-14T23:57:06.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.034 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.034 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:06.034 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.206437+0000", 2024-09-14T23:57:06.034 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3040999999999999e-05 2024-09-14T23:57:06.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.034 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.034 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:06.034 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.206455+0000", 2024-09-14T23:57:06.034 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8369e-05 2024-09-14T23:57:06.035 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.035 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.035 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:06.035 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.206466+0000", 2024-09-14T23:57:06.035 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0767999999999999e-05 2024-09-14T23:57:06.035 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.035 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.035 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:06.035 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.206551+0000", 2024-09-14T23:57:06.035 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4858999999999995e-05 2024-09-14T23:57:06.035 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.035 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.035 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:06.035 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.206574+0000", 2024-09-14T23:57:06.036 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2741000000000001e-05 2024-09-14T23:57:06.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.036 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.036 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:06.036 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.207690+0000", 2024-09-14T23:57:06.036 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011167880000000001 2024-09-14T23:57:06.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.036 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.036 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:06.036 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.207733+0000", 2024-09-14T23:57:06.036 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2737000000000001e-05 2024-09-14T23:57:06.036 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.036 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:06.036 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.037 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.037 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:18354 3.4 e34/23 3:225c4d41:::rbd_data.11187be34cc8.0000000000000001:head v 34'67, mlcod=34'65)", 2024-09-14T23:57:06.037 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:44.206794+0000", 2024-09-14T23:57:06.037 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 21.772628276999999, 2024-09-14T23:57:06.037 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013689819999999999, 2024-09-14T23:57:06.037 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:06.037 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:06.037 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:06.037 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:06.037 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.037 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:06.037 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.206794+0000", 2024-09-14T23:57:06.037 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.037 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.038 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.038 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:06.038 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.206794+0000", 2024-09-14T23:57:06.038 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.038 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.038 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.038 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:06.038 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.206795+0000", 2024-09-14T23:57:06.038 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5400000000000002e-07 2024-09-14T23:57:06.038 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.038 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.038 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:06.038 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.206800+0000", 2024-09-14T23:57:06.039 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0479999999999998e-06 2024-09-14T23:57:06.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.039 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.039 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:06.039 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.206808+0000", 2024-09-14T23:57:06.039 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8420000000000005e-06 2024-09-14T23:57:06.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.039 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.039 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:06.039 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.206814+0000", 2024-09-14T23:57:06.039 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8640000000000001e-06 2024-09-14T23:57:06.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.039 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.039 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:06.040 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.206988+0000", 2024-09-14T23:57:06.040 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000174599 2024-09-14T23:57:06.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.040 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.040 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:06.040 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.207007+0000", 2024-09-14T23:57:06.040 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8695e-05 2024-09-14T23:57:06.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.040 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.040 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:06.040 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.208109+0000", 2024-09-14T23:57:06.040 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011016310000000001 2024-09-14T23:57:06.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.040 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.041 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:06.041 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:44.208163+0000", 2024-09-14T23:57:06.041 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4849000000000003e-05 2024-09-14T23:57:06.041 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.041 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:06.041 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.041 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.041 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:32024 3.9 e39/23 3:94dc260b:::rbd_data.111814b482f8.0000000000000009:head v 39'45, mlcod=38'43)", 2024-09-14T23:57:06.041 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:49.061018+0000", 2024-09-14T23:57:06.041 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 16.918404614, 2024-09-14T23:57:06.041 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018482240000000001, 2024-09-14T23:57:06.041 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:06.041 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:06.042 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:06.042 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:06.042 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.042 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:06.042 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061018+0000", 2024-09-14T23:57:06.042 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.042 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.042 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:06.042 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061018+0000", 2024-09-14T23:57:06.042 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.042 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.043 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:06.043 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061018+0000", 2024-09-14T23:57:06.043 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9999999999999998e-07 2024-09-14T23:57:06.043 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.043 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.043 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:06.043 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061211+0000", 2024-09-14T23:57:06.043 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019206 2024-09-14T23:57:06.043 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.043 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.043 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:06.043 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061221+0000", 2024-09-14T23:57:06.043 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0659000000000001e-05 2024-09-14T23:57:06.043 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.044 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.044 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:06.044 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061230+0000", 2024-09-14T23:57:06.044 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4610000000000008e-06 2024-09-14T23:57:06.044 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.044 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.044 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:06.044 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061317+0000", 2024-09-14T23:57:06.044 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7514000000000003e-05 2024-09-14T23:57:06.044 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.044 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.044 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:06.044 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061338+0000", 2024-09-14T23:57:06.044 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0395999999999999e-05 2024-09-14T23:57:06.044 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.045 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.045 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:06.045 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.062830+0000", 2024-09-14T23:57:06.045 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0014921940000000001 2024-09-14T23:57:06.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.045 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.045 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:06.045 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.062866+0000", 2024-09-14T23:57:06.045 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6539999999999999e-05 2024-09-14T23:57:06.045 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.045 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:06.045 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.046 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:32420 3.9 e39/23 3:91dc34e7:::rbd_data.111814b482f8.000000000000001e:head v 39'46, mlcod=38'44)", 2024-09-14T23:57:06.046 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:49.086124+0000", 2024-09-14T23:57:06.046 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 16.893299115000001, 2024-09-14T23:57:06.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001686819, 2024-09-14T23:57:06.046 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:06.046 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:06.046 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:06.046 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:06.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.046 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:06.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086124+0000", 2024-09-14T23:57:06.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:06.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086124+0000", 2024-09-14T23:57:06.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:06.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086125+0000", 2024-09-14T23:57:06.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.361e-06 2024-09-14T23:57:06.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:06.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086605+0000", 2024-09-14T23:57:06.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00048026600000000003 2024-09-14T23:57:06.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.048 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:06.048 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086626+0000", 2024-09-14T23:57:06.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.037e-05 2024-09-14T23:57:06.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.048 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:06.048 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086635+0000", 2024-09-14T23:57:06.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5880000000000004e-06 2024-09-14T23:57:06.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.048 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:06.049 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086701+0000", 2024-09-14T23:57:06.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5905e-05 2024-09-14T23:57:06.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.049 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:06.049 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086727+0000", 2024-09-14T23:57:06.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5573000000000001e-05 2024-09-14T23:57:06.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.049 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:06.049 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.087767+0000", 2024-09-14T23:57:06.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001040537 2024-09-14T23:57:06.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.050 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:06.050 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.087810+0000", 2024-09-14T23:57:06.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3219000000000001e-05 2024-09-14T23:57:06.050 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.050 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:06.050 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.050 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:34296 3.9 e39/23 3:974a38c7:::rbd_data.111814b482f8.000000000000003a:head v 39'47, mlcod=39'45)", 2024-09-14T23:57:06.050 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:49.192105+0000", 2024-09-14T23:57:06.050 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 16.787317231999999, 2024-09-14T23:57:06.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001607516, 2024-09-14T23:57:06.050 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:06.050 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:06.051 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:06.051 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:06.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:06.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.192105+0000", 2024-09-14T23:57:06.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:06.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.192105+0000", 2024-09-14T23:57:06.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:06.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.192107+0000", 2024-09-14T23:57:06.052 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3549999999999999e-06 2024-09-14T23:57:06.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.052 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.052 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:06.052 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.192332+0000", 2024-09-14T23:57:06.052 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022486300000000001 2024-09-14T23:57:06.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.052 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.052 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:06.052 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.192349+0000", 2024-09-14T23:57:06.052 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6966e-05 2024-09-14T23:57:06.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.052 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.052 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:06.053 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.192358+0000", 2024-09-14T23:57:06.053 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8309999999999996e-06 2024-09-14T23:57:06.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.053 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:06.053 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.192422+0000", 2024-09-14T23:57:06.053 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3915e-05 2024-09-14T23:57:06.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.053 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:06.053 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.192450+0000", 2024-09-14T23:57:06.053 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7957e-05 2024-09-14T23:57:06.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.054 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:06.054 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.193662+0000", 2024-09-14T23:57:06.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012113270000000001 2024-09-14T23:57:06.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.054 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:06.054 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.193713+0000", 2024-09-14T23:57:06.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1301999999999998e-05 2024-09-14T23:57:06.054 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.054 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:06.054 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.054 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:46667 3.9 e48/23 3:9377641f:::rbd_data.1118cee06433.0000000000000000:head v 48'55, mlcod=44'52)", 2024-09-14T23:57:06.055 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:58.332832+0000", 2024-09-14T23:57:06.055 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 7.64659025, 2024-09-14T23:57:06.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013548060000000001, 2024-09-14T23:57:06.055 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:06.055 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:06.055 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:06.055 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:06.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:06.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:58.332832+0000", 2024-09-14T23:57:06.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:06.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:58.332832+0000", 2024-09-14T23:57:06.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:06.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:58.332833+0000", 2024-09-14T23:57:06.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5999999999999999e-07 2024-09-14T23:57:06.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:06.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:58.332838+0000", 2024-09-14T23:57:06.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1039999999999998e-06 2024-09-14T23:57:06.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.057 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:06.057 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:58.332847+0000", 2024-09-14T23:57:06.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2429999999999994e-06 2024-09-14T23:57:06.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.057 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:06.057 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:58.332854+0000", 2024-09-14T23:57:06.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7789999999999998e-06 2024-09-14T23:57:06.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.057 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:06.057 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:58.332918+0000", 2024-09-14T23:57:06.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3943999999999996e-05 2024-09-14T23:57:06.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.058 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:06.058 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:58.332942+0000", 2024-09-14T23:57:06.058 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4091999999999999e-05 2024-09-14T23:57:06.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.058 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:06.058 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:58.334144+0000", 2024-09-14T23:57:06.058 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001201765 2024-09-14T23:57:06.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.058 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:06.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:58.334187+0000", 2024-09-14T23:57:06.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3519000000000001e-05 2024-09-14T23:57:06.059 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.059 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:06.059 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.059 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:50582 3.9 e51/23 3:92198e7a:::rbd_data.11183b9aac34.000000000000000a:head v 51'63, mlcod=50'60)", 2024-09-14T23:57:06.059 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:01.131541+0000", 2024-09-14T23:57:06.059 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.8478819240000002, 2024-09-14T23:57:06.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001795547, 2024-09-14T23:57:06.059 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:06.059 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:06.059 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:06.060 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:06.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:06.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131541+0000", 2024-09-14T23:57:06.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:06.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131541+0000", 2024-09-14T23:57:06.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:06.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131541+0000", 2024-09-14T23:57:06.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7099999999999997e-07 2024-09-14T23:57:06.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.061 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:06.061 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131548+0000", 2024-09-14T23:57:06.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.708e-06 2024-09-14T23:57:06.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.061 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:06.061 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131570+0000", 2024-09-14T23:57:06.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1815999999999999e-05 2024-09-14T23:57:06.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.062 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:06.062 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131581+0000", 2024-09-14T23:57:06.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0933e-05 2024-09-14T23:57:06.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.062 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:06.062 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131660+0000", 2024-09-14T23:57:06.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9071000000000002e-05 2024-09-14T23:57:06.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.062 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:06.062 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131682+0000", 2024-09-14T23:57:06.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2534000000000001e-05 2024-09-14T23:57:06.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:06.063 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.133287+0000", 2024-09-14T23:57:06.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016045510000000001 2024-09-14T23:57:06.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:06.063 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.133336+0000", 2024-09-14T23:57:06.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9563e-05 2024-09-14T23:57:06.063 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.063 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:06.063 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.064 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:51182 3.9 e52/23 3:918e4fa0:::rbd_data.11181975a1de.000000000000007b:head v 52'74, mlcod=51'71)", 2024-09-14T23:57:06.064 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:02.138725+0000", 2024-09-14T23:57:06.064 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.8406981870000001, 2024-09-14T23:57:06.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015321009999999999, 2024-09-14T23:57:06.064 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:06.064 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:06.064 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:06.064 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:06.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.064 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:06.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.138725+0000", 2024-09-14T23:57:06.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:06.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.138725+0000", 2024-09-14T23:57:06.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:06.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.138725+0000", 2024-09-14T23:57:06.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.22e-07 2024-09-14T23:57:06.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:06.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.138732+0000", 2024-09-14T23:57:06.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1779999999999997e-06 2024-09-14T23:57:06.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:06.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.138749+0000", 2024-09-14T23:57:06.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6500000000000001e-05 2024-09-14T23:57:06.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:06.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.138758+0000", 2024-09-14T23:57:06.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6239999999999997e-06 2024-09-14T23:57:06.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:06.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.138816+0000", 2024-09-14T23:57:06.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7939000000000002e-05 2024-09-14T23:57:06.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:06.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.138840+0000", 2024-09-14T23:57:06.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3856e-05 2024-09-14T23:57:06.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:06.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.140204+0000", 2024-09-14T23:57:06.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013641339999999999 2024-09-14T23:57:06.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:06.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.140257+0000", 2024-09-14T23:57:06.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2547999999999999e-05 2024-09-14T23:57:06.068 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.068 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:06.068 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.068 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:51281 3.0 e52/23 3:04e60ec2:::rbd_data.11181975a1de.0000000000000003:head v 52'113, mlcod=44'111)", 2024-09-14T23:57:06.068 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:02.158822+0000", 2024-09-14T23:57:06.068 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.8206004280000001, 2024-09-14T23:57:06.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019860609999999999, 2024-09-14T23:57:06.068 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:06.068 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:06.068 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:06.069 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:06.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:06.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.158822+0000", 2024-09-14T23:57:06.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:06.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.158822+0000", 2024-09-14T23:57:06.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:06.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.158823+0000", 2024-09-14T23:57:06.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6800000000000001e-07 2024-09-14T23:57:06.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:06.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.159132+0000", 2024-09-14T23:57:06.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030974900000000001 2024-09-14T23:57:06.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:06.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.159146+0000", 2024-09-14T23:57:06.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3261000000000001e-05 2024-09-14T23:57:06.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:06.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.159155+0000", 2024-09-14T23:57:06.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7410000000000004e-06 2024-09-14T23:57:06.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:06.071 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.159221+0000", 2024-09-14T23:57:06.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5698999999999996e-05 2024-09-14T23:57:06.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:06.071 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.159244+0000", 2024-09-14T23:57:06.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3108e-05 2024-09-14T23:57:06.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:06.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.160767+0000", 2024-09-14T23:57:06.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015231120000000001 2024-09-14T23:57:06.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:06.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.160808+0000", 2024-09-14T23:57:06.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1022999999999997e-05 2024-09-14T23:57:06.072 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.072 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:06.072 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.072 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.072 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:06.072 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:06.074 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T23:57:06.074 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:06.072+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:06.096 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T23:57:06.144 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone17 from 0x37b847c (overlap 0x37b847c) to 0x0 2024-09-14T23:57:06.175 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:06.173+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:06.275 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:06.274+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:06.289 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:06.289 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:57:06.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.289 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:56403 3.7 3:e7f3b7b3:::rbd_data.1118c4b3aebe.0000000000000011:head [stat,delete] snapc 14=[14] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:06.289 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.286794+0000", 2024-09-14T23:57:06.289 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0010494300000000001, 2024-09-14T23:57:06.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001078215, 2024-09-14T23:57:06.289 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:06.289 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:06.289 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:06.290 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:06.290 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:06.290 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:06.290 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 56403 2024-09-14T23:57:06.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.290 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:06.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:06.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286794+0000", 2024-09-14T23:57:06.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:06.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286794+0000", 2024-09-14T23:57:06.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:06.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286795+0000", 2024-09-14T23:57:06.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2860000000000001e-06 2024-09-14T23:57:06.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:06.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286801+0000", 2024-09-14T23:57:06.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.614e-06 2024-09-14T23:57:06.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:06.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286815+0000", 2024-09-14T23:57:06.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4367e-05 2024-09-14T23:57:06.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.292 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:06.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286821+0000", 2024-09-14T23:57:06.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1210000000000003e-06 2024-09-14T23:57:06.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.292 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:06.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286862+0000", 2024-09-14T23:57:06.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0651000000000001e-05 2024-09-14T23:57:06.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.293 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:06.293 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286958+0000", 2024-09-14T23:57:06.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5731999999999998e-05 2024-09-14T23:57:06.293 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.293 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:06.293 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.293 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:56404 3.c 3:370203a3:::rbd_data.1118c4b3aebe.000000000000000d:head [call rbd.sparse_copyup in=24b,set-alloc-hint object_size 4194304 write_size 4194304,delete] snapc 14=[14] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:06.293 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.287253+0000", 2024-09-14T23:57:06.293 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.00058988200000000001, 2024-09-14T23:57:06.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00068819699999999998, 2024-09-14T23:57:06.293 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:06.294 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:06.294 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "waiting for sub ops", 2024-09-14T23:57:06.294 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:06.294 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:06.294 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:06.294 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 56404 2024-09-14T23:57:06.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.294 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:06.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:06.294 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287253+0000", 2024-09-14T23:57:06.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.295 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:06.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287253+0000", 2024-09-14T23:57:06.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.295 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:06.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287254+0000", 2024-09-14T23:57:06.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5700000000000001e-07 2024-09-14T23:57:06.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.296 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:06.296 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287259+0000", 2024-09-14T23:57:06.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2240000000000003e-06 2024-09-14T23:57:06.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.296 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:06.296 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287269+0000", 2024-09-14T23:57:06.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.38e-06 2024-09-14T23:57:06.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:06.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287276+0000", 2024-09-14T23:57:06.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8469999999999998e-06 2024-09-14T23:57:06.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:06.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287331+0000", 2024-09-14T23:57:06.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5866999999999999e-05 2024-09-14T23:57:06.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:06.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287459+0000", 2024-09-14T23:57:06.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012788099999999999 2024-09-14T23:57:06.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.298 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.298 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-14T23:57:06.298 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287654+0000", 2024-09-14T23:57:06.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019457799999999999 2024-09-14T23:57:06.298 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.298 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:06.298 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.298 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.298 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-14T23:57:06.299 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 2 2024-09-14T23:57:06.299 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:06.303 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T23:57:06.344 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-09-14T23:57:06.376 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:06.375+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:06.442 INFO:teuthology.orchestra.run.smithi042.stdout:1743 writesame 0x7acccc2 thru 0x7adb1cb data_size 0x13(0xe50a bytes) 2024-09-14T23:57:06.448 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:06.448 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:57:06.448 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:57:06.449 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:57:06.449 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:06.453 INFO:teuthology.orchestra.run.smithi042.stdout:1744 write 0xcfeb39b thru 0xcff6567 (0xb1cd bytes) 2024-09-14T23:57:06.460 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T23:57:06.461 INFO:teuthology.orchestra.run.smithi042.stdout:1745 write 0xa7b7651 thru 0xa7c238d (0xad3d bytes) 2024-09-14T23:57:06.471 INFO:teuthology.orchestra.run.smithi042.stdout:1746 write 0x5064b51 thru 0x506d922 (0x8dd2 bytes) 2024-09-14T23:57:06.477 INFO:teuthology.orchestra.run.smithi042.stdout:1747 writesame 0x29a5b47 thru 0x29acc17 data_size 0x3(0x70d1 bytes) 2024-09-14T23:57:06.477 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:06.476+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:06.494 INFO:teuthology.orchestra.run.smithi042.stdout:1748 write 0xd3a4503 thru 0xd3ace35 (0x8933 bytes) 2024-09-14T23:57:06.502 INFO:teuthology.orchestra.run.smithi042.stdout:1749 read 0x72a151b thru 0x72a9877 (0x835d bytes) 2024-09-14T23:57:06.502 INFO:teuthology.orchestra.run.smithi042.stdout:1750 read 0x3fafdf6 thru 0x3fbe2f4 (0xe4ff bytes) 2024-09-14T23:57:06.502 INFO:teuthology.orchestra.run.smithi042.stdout:1751 read 0x49eb3b thru 0x4ae05d (0xf523 bytes) 2024-09-14T23:57:06.503 INFO:teuthology.orchestra.run.smithi042.stdout:1752 punch from 0x656d4b4 to 0x6572f54, (0x5aa0 bytes) 2024-09-14T23:57:06.508 INFO:teuthology.orchestra.run.smithi042.stdout:1753 write 0xd64a016 thru 0xd654de3 (0xadce bytes) 2024-09-14T23:57:06.516 INFO:teuthology.orchestra.run.smithi042.stdout:1754 writesame 0x31fd8f thru 0x32991d data_size 0x1(0x9b8f bytes) 2024-09-14T23:57:06.549 INFO:teuthology.orchestra.run.smithi042.stdout:1755 trunc from 0xd654de4 to 0x40d47b4 2024-09-14T23:57:06.564 INFO:teuthology.orchestra.run.smithi042.stdout:1756 read 0x6b3c7f thru 0x6ba40d (0x678f bytes) 2024-09-14T23:57:06.575 INFO:teuthology.orchestra.run.smithi042.stdout:1757 write 0x8defa4d thru 0x8df94b1 (0x9a65 bytes) 2024-09-14T23:57:06.577 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:06.576+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:06.582 INFO:teuthology.orchestra.run.smithi042.stdout:1760 punch from 0x65fdcaf to 0x660529e, (0x75ef bytes) 2024-09-14T23:57:06.582 INFO:teuthology.orchestra.run.smithi042.stdout:1761 punch from 0x7d832f3 to 0x7d8b2ec, (0x7ff9 bytes) 2024-09-14T23:57:06.582 INFO:teuthology.orchestra.run.smithi042.stdout:1763 read 0x6453857 thru 0x6455ca4 (0x244e bytes) 2024-09-14T23:57:06.583 INFO:teuthology.orchestra.run.smithi042.stdout:1764 punch from 0x8d71b37 to 0x8d7a52b, (0x89f4 bytes) 2024-09-14T23:57:06.583 INFO:teuthology.orchestra.run.smithi042.stdout:1765 trunc from 0x8df94b2 to 0x81fc7f8 2024-09-14T23:57:06.597 INFO:teuthology.orchestra.run.smithi042.stdout:1766 write 0x862d629 thru 0x863b5b0 (0xdf88 bytes) 2024-09-14T23:57:06.599 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:06.599 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:57:06.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:57:06.599 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:57:06.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.599 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5180 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3883008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:06.599 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.250058+0000", 2024-09-14T23:57:06.599 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.346749212999999, 2024-09-14T23:57:06.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057351471000000001, 2024-09-14T23:57:06.600 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:06.600 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:06.600 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:06.600 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:06.600 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:06.600 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:06.600 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5180 2024-09-14T23:57:06.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.600 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:06.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:06.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250058+0000", 2024-09-14T23:57:06.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:06.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250058+0000", 2024-09-14T23:57:06.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:06.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250786+0000", 2024-09-14T23:57:06.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00072813399999999996 2024-09-14T23:57:06.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:06.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250789+0000", 2024-09-14T23:57:06.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.647e-06 2024-09-14T23:57:06.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:06.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250798+0000", 2024-09-14T23:57:06.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0000000000000002e-06 2024-09-14T23:57:06.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:06.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250803+0000", 2024-09-14T23:57:06.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.301e-06 2024-09-14T23:57:06.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:06.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.307234+0000", 2024-09-14T23:57:06.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056431165999999998 2024-09-14T23:57:06.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:06.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.307321+0000", 2024-09-14T23:57:06.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6831999999999998e-05 2024-09-14T23:57:06.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:06.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.307409+0000", 2024-09-14T23:57:06.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8391000000000001e-05 2024-09-14T23:57:06.604 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.604 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:06.604 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.604 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5184 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 4128768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:06.604 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.250851+0000", 2024-09-14T23:57:06.604 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.345955742000001, 2024-09-14T23:57:06.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057319038000000003, 2024-09-14T23:57:06.604 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:06.605 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:06.605 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:06.605 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:06.605 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:06.605 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:06.605 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5184 2024-09-14T23:57:06.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.605 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:06.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:06.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250851+0000", 2024-09-14T23:57:06.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:06.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250851+0000", 2024-09-14T23:57:06.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:06.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.251900+0000", 2024-09-14T23:57:06.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001048577 2024-09-14T23:57:06.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:06.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.251904+0000", 2024-09-14T23:57:06.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9580000000000001e-06 2024-09-14T23:57:06.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:06.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.251914+0000", 2024-09-14T23:57:06.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0186999999999999e-05 2024-09-14T23:57:06.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:06.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.251927+0000", 2024-09-14T23:57:06.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3593e-05 2024-09-14T23:57:06.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:06.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.308020+0000", 2024-09-14T23:57:06.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056092867999999997 2024-09-14T23:57:06.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:06.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.308094+0000", 2024-09-14T23:57:06.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3533000000000002e-05 2024-09-14T23:57:06.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:06.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.308170+0000", 2024-09-14T23:57:06.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6322000000000006e-05 2024-09-14T23:57:06.608 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.609 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:06.609 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.609 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5196 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 1925120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:06.609 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.252157+0000", 2024-09-14T23:57:06.609 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.344649627999999, 2024-09-14T23:57:06.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057285801999999997, 2024-09-14T23:57:06.609 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:06.609 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:06.609 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:06.609 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:06.609 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:06.610 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:06.610 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5196 2024-09-14T23:57:06.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.610 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:06.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:06.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.252157+0000", 2024-09-14T23:57:06.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:06.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.252157+0000", 2024-09-14T23:57:06.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:06.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253243+0000", 2024-09-14T23:57:06.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010861340000000001 2024-09-14T23:57:06.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:06.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253248+0000", 2024-09-14T23:57:06.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8799999999999999e-06 2024-09-14T23:57:06.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:06.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253271+0000", 2024-09-14T23:57:06.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2836e-05 2024-09-14T23:57:06.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:06.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253281+0000", 2024-09-14T23:57:06.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6520000000000001e-06 2024-09-14T23:57:06.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:06.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.309336+0000", 2024-09-14T23:57:06.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056054983000000003 2024-09-14T23:57:06.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:06.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.309389+0000", 2024-09-14T23:57:06.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3298999999999999e-05 2024-09-14T23:57:06.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:06.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.309443+0000", 2024-09-14T23:57:06.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4017999999999997e-05 2024-09-14T23:57:06.613 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.613 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:06.613 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.613 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5200 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2170880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:06.613 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253394+0000", 2024-09-14T23:57:06.613 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.343412851, 2024-09-14T23:57:06.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057428154000000002, 2024-09-14T23:57:06.614 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:06.614 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:06.614 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:06.614 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:06.614 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:06.614 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:06.614 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5200 2024-09-14T23:57:06.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.614 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:06.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:06.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253394+0000", 2024-09-14T23:57:06.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:06.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253394+0000", 2024-09-14T23:57:06.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:06.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253395+0000", 2024-09-14T23:57:06.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9100000000000004e-07 2024-09-14T23:57:06.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:06.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253397+0000", 2024-09-14T23:57:06.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2790000000000001e-06 2024-09-14T23:57:06.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:06.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253404+0000", 2024-09-14T23:57:06.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7959999999999998e-06 2024-09-14T23:57:06.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:06.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253409+0000", 2024-09-14T23:57:06.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1460000000000004e-06 2024-09-14T23:57:06.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:06.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310627+0000", 2024-09-14T23:57:06.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057218160999999997 2024-09-14T23:57:06.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:06.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310710+0000", 2024-09-14T23:57:06.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3102999999999994e-05 2024-09-14T23:57:06.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:06.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310822+0000", 2024-09-14T23:57:06.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000112178 2024-09-14T23:57:06.618 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.618 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:06.618 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.618 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5201 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2232320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:06.618 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:57:06.618 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.343388179000002, 2024-09-14T23:57:06.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057643228999999997, 2024-09-14T23:57:06.618 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:06.618 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:06.618 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:06.618 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:06.618 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:06.618 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:06.619 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5201 2024-09-14T23:57:06.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.619 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:06.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:06.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:57:06.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:06.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:57:06.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:06.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:57:06.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0699999999999998e-07 2024-09-14T23:57:06.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:06.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253422+0000", 2024-09-14T23:57:06.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.514e-06 2024-09-14T23:57:06.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:06.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253431+0000", 2024-09-14T23:57:06.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1919999999999994e-06 2024-09-14T23:57:06.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:06.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253436+0000", 2024-09-14T23:57:06.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5350000000000004e-06 2024-09-14T23:57:06.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:06.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310856+0000", 2024-09-14T23:57:06.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057419381999999998 2024-09-14T23:57:06.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:06.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310953+0000", 2024-09-14T23:57:06.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6774000000000006e-05 2024-09-14T23:57:06.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:06.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311062+0000", 2024-09-14T23:57:06.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000109425 2024-09-14T23:57:06.622 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.622 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:06.622 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.622 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5202 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2293760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:06.622 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:57:06.622 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.343363431, 2024-09-14T23:57:06.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057818255999999998, 2024-09-14T23:57:06.623 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:06.623 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:06.623 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:06.623 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:06.623 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:06.623 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:06.623 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5202 2024-09-14T23:57:06.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.623 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:06.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:06.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:57:06.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:06.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:57:06.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:06.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253573+0000", 2024-09-14T23:57:06.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012934000000000001 2024-09-14T23:57:06.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:06.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253576+0000", 2024-09-14T23:57:06.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5310000000000002e-06 2024-09-14T23:57:06.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:06.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253587+0000", 2024-09-14T23:57:06.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0522e-05 2024-09-14T23:57:06.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:06.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253595+0000", 2024-09-14T23:57:06.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8290000000000008e-06 2024-09-14T23:57:06.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:06.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311093+0000", 2024-09-14T23:57:06.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057498312000000003 2024-09-14T23:57:06.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:06.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311167+0000", 2024-09-14T23:57:06.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3582000000000001e-05 2024-09-14T23:57:06.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:06.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311262+0000", 2024-09-14T23:57:06.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5140000000000001e-05 2024-09-14T23:57:06.626 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.626 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:06.626 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.627 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5203 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2355200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:06.627 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:57:06.627 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.343200689, 2024-09-14T23:57:06.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057903938000000002, 2024-09-14T23:57:06.627 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:06.627 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:06.627 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:06.627 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:06.627 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:06.627 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:06.627 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5203 2024-09-14T23:57:06.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.628 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:06.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:06.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:57:06.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:06.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:57:06.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:06.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253902+0000", 2024-09-14T23:57:06.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029615400000000002 2024-09-14T23:57:06.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:06.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253906+0000", 2024-09-14T23:57:06.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9249999999999997e-06 2024-09-14T23:57:06.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:06.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253913+0000", 2024-09-14T23:57:06.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9099999999999999e-06 2024-09-14T23:57:06.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:06.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253917+0000", 2024-09-14T23:57:06.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9210000000000002e-06 2024-09-14T23:57:06.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:06.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311295+0000", 2024-09-14T23:57:06.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057378268000000003 2024-09-14T23:57:06.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:06.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311402+0000", 2024-09-14T23:57:06.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010644100000000001 2024-09-14T23:57:06.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:06.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311510+0000", 2024-09-14T23:57:06.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108319 2024-09-14T23:57:06.631 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.631 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:06.631 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.631 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5204 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2416640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:06.631 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:57:06.631 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.342882492000001, 2024-09-14T23:57:06.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057774860999999997, 2024-09-14T23:57:06.631 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:06.631 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:06.632 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:06.632 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:06.632 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:06.632 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:06.632 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5204 2024-09-14T23:57:06.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.632 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:06.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:06.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:57:06.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:06.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:57:06.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:06.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255903+0000", 2024-09-14T23:57:06.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001978807 2024-09-14T23:57:06.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:06.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255907+0000", 2024-09-14T23:57:06.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3610000000000001e-06 2024-09-14T23:57:06.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:06.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255914+0000", 2024-09-14T23:57:06.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.357e-06 2024-09-14T23:57:06.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:06.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255918+0000", 2024-09-14T23:57:06.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1779999999999996e-06 2024-09-14T23:57:06.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:06.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311541+0000", 2024-09-14T23:57:06.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055622697999999998 2024-09-14T23:57:06.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:06.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311614+0000", 2024-09-14T23:57:06.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3102999999999994e-05 2024-09-14T23:57:06.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:06.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311699+0000", 2024-09-14T23:57:06.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5357000000000003e-05 2024-09-14T23:57:06.635 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.635 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:06.635 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.636 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5210 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2785280~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:06.636 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256011+0000", 2024-09-14T23:57:06.636 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.340795855, 2024-09-14T23:57:06.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057273662000000003, 2024-09-14T23:57:06.636 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:06.636 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:06.636 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:06.636 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:06.636 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:06.636 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:06.636 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5210 2024-09-14T23:57:06.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.636 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:06.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:06.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256011+0000", 2024-09-14T23:57:06.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:06.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256011+0000", 2024-09-14T23:57:06.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:06.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256011+0000", 2024-09-14T23:57:06.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4999999999999999e-07 2024-09-14T23:57:06.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:06.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256013+0000", 2024-09-14T23:57:06.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3969999999999999e-06 2024-09-14T23:57:06.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:06.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256017+0000", 2024-09-14T23:57:06.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1409999999999998e-06 2024-09-14T23:57:06.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:06.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256019+0000", 2024-09-14T23:57:06.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6280000000000001e-06 2024-09-14T23:57:06.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:06.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313116+0000", 2024-09-14T23:57:06.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057096457000000003 2024-09-14T23:57:06.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:06.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313193+0000", 2024-09-14T23:57:06.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7439000000000002e-05 2024-09-14T23:57:06.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:06.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313285+0000", 2024-09-14T23:57:06.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1349999999999998e-05 2024-09-14T23:57:06.640 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.640 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:06.640 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.640 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5211 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2846720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:06.640 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:57:06.640 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.340784278000001, 2024-09-14T23:57:06.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057528151, 2024-09-14T23:57:06.640 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:06.640 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:06.641 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:06.641 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:06.641 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:06.641 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:06.641 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5211 2024-09-14T23:57:06.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.641 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:06.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:06.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:57:06.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:06.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:57:06.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:06.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:57:06.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1399999999999998e-07 2024-09-14T23:57:06.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:06.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256024+0000", 2024-09-14T23:57:06.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.114e-06 2024-09-14T23:57:06.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:06.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256028+0000", 2024-09-14T23:57:06.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0840000000000002e-06 2024-09-14T23:57:06.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:06.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256031+0000", 2024-09-14T23:57:06.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6819999999999999e-06 2024-09-14T23:57:06.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:06.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313317+0000", 2024-09-14T23:57:06.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057286295000000001 2024-09-14T23:57:06.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:06.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313403+0000", 2024-09-14T23:57:06.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6287999999999998e-05 2024-09-14T23:57:06.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:06.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313551+0000", 2024-09-14T23:57:06.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000147374 2024-09-14T23:57:06.644 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.644 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:06.644 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.645 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5212 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2908160~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:06.645 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:57:06.645 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.340773083999999, 2024-09-14T23:57:06.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057735679999999998, 2024-09-14T23:57:06.645 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:06.645 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:06.645 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:06.645 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:06.645 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:06.645 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:06.645 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5212 2024-09-14T23:57:06.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.645 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:06.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:06.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:57:06.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:06.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:57:06.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:06.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:57:06.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0900000000000001e-07 2024-09-14T23:57:06.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:06.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256035+0000", 2024-09-14T23:57:06.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1200000000000001e-06 2024-09-14T23:57:06.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:06.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256038+0000", 2024-09-14T23:57:06.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.05e-06 2024-09-14T23:57:06.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:06.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256041+0000", 2024-09-14T23:57:06.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.667e-06 2024-09-14T23:57:06.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.648 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:06.648 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313584+0000", 2024-09-14T23:57:06.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057543484999999998 2024-09-14T23:57:06.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.648 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:06.648 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313668+0000", 2024-09-14T23:57:06.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3807000000000006e-05 2024-09-14T23:57:06.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.648 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:06.648 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313770+0000", 2024-09-14T23:57:06.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010134200000000001 2024-09-14T23:57:06.649 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.649 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:06.649 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.649 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5213 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2969600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:06.649 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:57:06.649 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.340762996999999, 2024-09-14T23:57:06.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058145467999999999, 2024-09-14T23:57:06.649 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:06.650 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:06.650 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:06.650 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:06.650 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:06.650 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:06.650 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5213 2024-09-14T23:57:06.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.650 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:06.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.650 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:06.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:57:06.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.651 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:06.651 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:57:06.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.651 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:06.651 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257901+0000", 2024-09-14T23:57:06.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018567519999999999 2024-09-14T23:57:06.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.651 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:06.651 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257904+0000", 2024-09-14T23:57:06.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4850000000000001e-06 2024-09-14T23:57:06.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.652 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:06.652 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257912+0000", 2024-09-14T23:57:06.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2090000000000005e-06 2024-09-14T23:57:06.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.652 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:06.652 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257918+0000", 2024-09-14T23:57:06.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6899999999999997e-06 2024-09-14T23:57:06.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.652 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:06.652 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313816+0000", 2024-09-14T23:57:06.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055898075999999998 2024-09-14T23:57:06.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.653 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:06.653 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.314082+0000", 2024-09-14T23:57:06.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000265386 2024-09-14T23:57:06.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.653 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:06.653 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.314189+0000", 2024-09-14T23:57:06.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010787 2024-09-14T23:57:06.653 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.653 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:06.653 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.654 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5217 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3215360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:06.654 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257961+0000", 2024-09-14T23:57:06.654 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.338846018000002, 2024-09-14T23:57:06.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057320969999999999, 2024-09-14T23:57:06.654 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:06.654 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:06.654 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:06.654 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:06.654 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:06.654 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:06.654 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5217 2024-09-14T23:57:06.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.654 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:06.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.655 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:06.655 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257961+0000", 2024-09-14T23:57:06.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.655 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:06.655 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257961+0000", 2024-09-14T23:57:06.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.655 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:06.655 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257961+0000", 2024-09-14T23:57:06.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.22e-07 2024-09-14T23:57:06.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.656 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.656 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:06.656 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257962+0000", 2024-09-14T23:57:06.656 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1400000000000001e-06 2024-09-14T23:57:06.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.656 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.656 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:06.656 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257965+0000", 2024-09-14T23:57:06.656 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0929999999999999e-06 2024-09-14T23:57:06.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.656 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.656 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:06.656 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257968+0000", 2024-09-14T23:57:06.657 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5749999999999999e-06 2024-09-14T23:57:06.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.657 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.657 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:06.657 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315047+0000", 2024-09-14T23:57:06.657 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057078730000000001 2024-09-14T23:57:06.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.657 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.657 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:06.657 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315087+0000", 2024-09-14T23:57:06.657 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0806999999999997e-05 2024-09-14T23:57:06.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.657 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.657 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:06.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315282+0000", 2024-09-14T23:57:06.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000194403 2024-09-14T23:57:06.658 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.658 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:06.658 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.658 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5218 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3276800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:06.658 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:57:06.658 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.338835988, 2024-09-14T23:57:06.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057497557999999997, 2024-09-14T23:57:06.658 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:06.658 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:06.658 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:06.659 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:06.659 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:06.659 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:06.659 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5218 2024-09-14T23:57:06.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.659 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:06.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:06.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:57:06.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:06.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:57:06.660 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.660 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:06.660 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:57:06.660 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1899999999999999e-07 2024-09-14T23:57:06.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.660 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:06.660 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257972+0000", 2024-09-14T23:57:06.660 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.217e-06 2024-09-14T23:57:06.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.660 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:06.661 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257975+0000", 2024-09-14T23:57:06.661 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9359999999999999e-06 2024-09-14T23:57:06.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.661 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:06.661 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257978+0000", 2024-09-14T23:57:06.661 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1049999999999999e-06 2024-09-14T23:57:06.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.661 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:06.661 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315331+0000", 2024-09-14T23:57:06.661 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057352093999999999 2024-09-14T23:57:06.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.662 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:06.662 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315384+0000", 2024-09-14T23:57:06.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3780999999999999e-05 2024-09-14T23:57:06.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.662 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:06.662 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315468+0000", 2024-09-14T23:57:06.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4206e-05 2024-09-14T23:57:06.662 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.662 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:06.662 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.662 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5219 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3338240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:06.662 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:57:06.663 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.338825312000001, 2024-09-14T23:57:06.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057637764000000001, 2024-09-14T23:57:06.663 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:06.663 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:06.663 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:06.663 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:06.663 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:06.663 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:06.663 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5219 2024-09-14T23:57:06.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.663 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:06.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:06.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:57:06.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.664 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:06.664 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:57:06.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.664 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:06.664 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:57:06.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.03e-07 2024-09-14T23:57:06.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.664 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:06.665 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257983+0000", 2024-09-14T23:57:06.665 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9199999999999999e-07 2024-09-14T23:57:06.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.665 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:06.665 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257986+0000", 2024-09-14T23:57:06.665 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0860000000000002e-06 2024-09-14T23:57:06.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.665 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:06.665 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257989+0000", 2024-09-14T23:57:06.665 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6589999999999999e-06 2024-09-14T23:57:06.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.666 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:06.666 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315509+0000", 2024-09-14T23:57:06.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057520262000000003 2024-09-14T23:57:06.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.666 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:06.666 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315554+0000", 2024-09-14T23:57:06.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5015999999999997e-05 2024-09-14T23:57:06.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.666 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:06.666 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315619+0000", 2024-09-14T23:57:06.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5545999999999999e-05 2024-09-14T23:57:06.666 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.667 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:06.667 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.667 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5220 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3399680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:06.667 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:57:06.667 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.338815356000001, 2024-09-14T23:57:06.667 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057792128999999998, 2024-09-14T23:57:06.667 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:06.667 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:06.667 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:06.667 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:06.667 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:06.667 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:06.668 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5220 2024-09-14T23:57:06.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.668 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:06.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.668 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:06.668 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:57:06.668 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.668 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:06.668 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:57:06.668 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.669 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:06.669 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:57:06.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1799999999999999e-07 2024-09-14T23:57:06.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.669 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:06.669 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257993+0000", 2024-09-14T23:57:06.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6200000000000006e-07 2024-09-14T23:57:06.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.670 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:06.670 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257996+0000", 2024-09-14T23:57:06.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9050000000000001e-06 2024-09-14T23:57:06.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.670 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:06.670 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257998+0000", 2024-09-14T23:57:06.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5950000000000001e-06 2024-09-14T23:57:06.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.671 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:06.671 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315672+0000", 2024-09-14T23:57:06.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057673637999999999 2024-09-14T23:57:06.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.671 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:06.671 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315734+0000", 2024-09-14T23:57:06.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2216000000000002e-05 2024-09-14T23:57:06.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.671 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:06.671 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315784+0000", 2024-09-14T23:57:06.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9595000000000002e-05 2024-09-14T23:57:06.672 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.672 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:06.672 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.672 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5221 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3461120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:06.672 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:57:06.672 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.338805638, 2024-09-14T23:57:06.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058003557999999997, 2024-09-14T23:57:06.672 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:06.672 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:06.672 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:06.672 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:06.673 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:06.673 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:06.673 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5221 2024-09-14T23:57:06.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.673 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:06.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.673 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:06.673 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:57:06.673 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.673 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:06.673 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:57:06.673 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.674 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:06.674 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259905+0000", 2024-09-14T23:57:06.674 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019035759999999999 2024-09-14T23:57:06.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.674 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:06.674 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259915+0000", 2024-09-14T23:57:06.674 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0518e-05 2024-09-14T23:57:06.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.674 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:06.674 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259925+0000", 2024-09-14T23:57:06.675 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7180000000000008e-06 2024-09-14T23:57:06.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:06.675 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259930+0000", 2024-09-14T23:57:06.675 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4769999999999997e-06 2024-09-14T23:57:06.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:06.675 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315820+0000", 2024-09-14T23:57:06.675 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055889960000000002 2024-09-14T23:57:06.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:06.676 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315914+0000", 2024-09-14T23:57:06.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4625999999999998e-05 2024-09-14T23:57:06.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.676 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:06.676 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.316005+0000", 2024-09-14T23:57:06.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0682999999999999e-05 2024-09-14T23:57:06.676 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.676 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:06.676 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.676 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5230 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 4014080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:06.677 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.260060+0000", 2024-09-14T23:57:06.677 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.3367468, 2024-09-14T23:57:06.677 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057353556999999999, 2024-09-14T23:57:06.677 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:06.677 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:06.677 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:06.677 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:06.677 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:06.677 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:06.677 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5230 2024-09-14T23:57:06.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.677 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:06.677 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.678 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:06.678 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260060+0000", 2024-09-14T23:57:06.678 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.678 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.678 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.678 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:06.678 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260060+0000", 2024-09-14T23:57:06.678 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.678 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.678 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.678 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:06.678 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260060+0000", 2024-09-14T23:57:06.678 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.17e-07 2024-09-14T23:57:06.678 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.679 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:06.679 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260061+0000", 2024-09-14T23:57:06.679 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0300000000000001e-06 2024-09-14T23:57:06.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.679 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:06.679 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260064+0000", 2024-09-14T23:57:06.679 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9119999999999998e-06 2024-09-14T23:57:06.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.679 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:06.679 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260067+0000", 2024-09-14T23:57:06.679 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.678e-06 2024-09-14T23:57:06.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.680 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:06.680 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317269+0000", 2024-09-14T23:57:06.680 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057201771999999998 2024-09-14T23:57:06.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.680 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:06.680 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317342+0000", 2024-09-14T23:57:06.680 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2842000000000005e-05 2024-09-14T23:57:06.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.680 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:06.680 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317414+0000", 2024-09-14T23:57:06.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2106000000000004e-05 2024-09-14T23:57:06.681 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.681 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:06.681 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.681 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5231 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 4075520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:06.681 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:57:06.681 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.336736903999999, 2024-09-14T23:57:06.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057524733000000002, 2024-09-14T23:57:06.681 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:06.681 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:06.681 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:06.681 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:06.682 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:06.682 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:06.682 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5231 2024-09-14T23:57:06.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.682 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:06.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.682 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:06.682 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:57:06.682 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.682 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:06.682 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:57:06.682 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.683 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.683 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:06.683 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261904+0000", 2024-09-14T23:57:06.683 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018338790000000001 2024-09-14T23:57:06.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.683 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.683 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:06.683 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261910+0000", 2024-09-14T23:57:06.683 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9689999999999999e-06 2024-09-14T23:57:06.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.683 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.683 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:06.683 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261923+0000", 2024-09-14T23:57:06.684 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3061000000000001e-05 2024-09-14T23:57:06.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.684 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:06.684 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261931+0000", 2024-09-14T23:57:06.684 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3159999999999997e-06 2024-09-14T23:57:06.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.684 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:06.684 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317456+0000", 2024-09-14T23:57:06.684 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055525219000000001 2024-09-14T23:57:06.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.685 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:06.685 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317520+0000", 2024-09-14T23:57:06.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4010000000000005e-05 2024-09-14T23:57:06.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.685 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:06.685 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317595+0000", 2024-09-14T23:57:06.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4278999999999998e-05 2024-09-14T23:57:06.685 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.685 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:06.685 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.685 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5236 3.8 3:1071668d:::rbd_data.1118de1168b.0000000000000092:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:06.686 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:57:06.686 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.334723440000001, 2024-09-14T23:57:06.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057499462000000001, 2024-09-14T23:57:06.686 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:06.686 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:06.686 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:06.686 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:06.686 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:06.686 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:06.686 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5236 2024-09-14T23:57:06.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.686 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:06.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.686 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:06.687 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:57:06.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.687 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:06.687 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:57:06.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:06.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.687 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:06.687 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262084+0000", 2024-09-14T23:57:06.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3399999999999999e-07 2024-09-14T23:57:06.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.687 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:06.688 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262087+0000", 2024-09-14T23:57:06.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2310000000000001e-06 2024-09-14T23:57:06.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.688 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:06.688 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262095+0000", 2024-09-14T23:57:06.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3799999999999996e-06 2024-09-14T23:57:06.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.688 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:06.688 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262101+0000", 2024-09-14T23:57:06.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0009999999999998e-06 2024-09-14T23:57:06.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.689 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:06.689 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317625+0000", 2024-09-14T23:57:06.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055524111000000001 2024-09-14T23:57:06.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.689 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:06.689 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317702+0000", 2024-09-14T23:57:06.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7533999999999993e-05 2024-09-14T23:57:06.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.689 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-14T23:57:06.689 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317892+0000", 2024-09-14T23:57:06.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018991400000000001 2024-09-14T23:57:06.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.690 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:57:06.690 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319407+0000", 2024-09-14T23:57:06.690 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015145250000000001 2024-09-14T23:57:06.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.690 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:06.690 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319481+0000", 2024-09-14T23:57:06.690 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3826999999999996e-05 2024-09-14T23:57:06.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.690 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:06.690 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319541+0000", 2024-09-14T23:57:06.690 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0470999999999998e-05 2024-09-14T23:57:06.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:06.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:06.691 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:06.691 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319583+0000", 2024-09-14T23:57:06.691 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1934000000000002e-05 2024-09-14T23:57:06.691 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.691 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:06.691 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.691 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:06.691 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:06.691 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:06.692 INFO:teuthology.orchestra.run.smithi042.stdout:1767 read 0x67db382 thru 0x67e42b9 (0x8f38 bytes) 2024-09-14T23:57:06.693 INFO:teuthology.orchestra.run.smithi042.stdout:1768 read 0x1f8b236 thru 0x1f8c126 (0xef1 bytes) 2024-09-14T23:57:06.693 INFO:teuthology.orchestra.run.smithi042.stdout:1770 read 0x45796d4 thru 0x45852ca (0xbbf7 bytes) 2024-09-14T23:57:06.693 INFO:teuthology.orchestra.run.smithi042.stdout:1771 read 0x71ea16d thru 0x71f2ec0 (0x8d54 bytes) 2024-09-14T23:57:06.693 INFO:teuthology.orchestra.run.smithi042.stdout:1772 read 0x138d042 thru 0x1391418 (0x43d7 bytes) 2024-09-14T23:57:06.694 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T23:57:06.695 INFO:teuthology.orchestra.run.smithi042.stdout:1773 write 0xc19d270 thru 0xc1a95fa (0xc38b bytes) 2024-09-14T23:57:06.695 INFO:teuthology.orchestra.run.smithi042.stdout:1774 writesame 0xa853f33 thru 0xa85b7e2 data_size 0x8(0x78b0 bytes) 2024-09-14T23:57:06.696 INFO:teuthology.orchestra.run.smithi042.stdout:1776 read 0x9145af0 thru 0x9147933 (0x1e44 bytes) 2024-09-14T23:57:06.696 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:06.696 INFO:teuthology.orchestra.run.smithi042.stdout:1778 write 0x98a43b3 thru 0x98b2922 (0xe570 bytes) 2024-09-14T23:57:06.696 INFO:teuthology.orchestra.run.smithi042.stdout:1779 write 0x6042196 thru 0x604d8a8 (0xb713 bytes) 2024-09-14T23:57:06.696 INFO:teuthology.orchestra.run.smithi042.stdout:1780 read 0xbeaae81 thru 0xbeb9b56 (0xecd6 bytes) 2024-09-14T23:57:06.696 INFO:teuthology.orchestra.run.smithi042.stdout:1781 punch from 0xac792fc to 0xac7e525, (0x5229 bytes) 2024-09-14T23:57:06.696 INFO:teuthology.orchestra.run.smithi042.stdout:1782 read 0x485db58 thru 0x485f8cf (0x1d78 bytes) 2024-09-14T23:57:06.696 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:06.696 INFO:teuthology.orchestra.run.smithi042.stdout:1784 writesame 0x7fa5426 thru 0x7fb26f5 data_size 0x8(0xd2d0 bytes) 2024-09-14T23:57:06.696 INFO:teuthology.orchestra.run.smithi042.stdout:1785 punch from 0x5c28471 to 0x5c2b478, (0x3007 bytes) 2024-09-14T23:57:06.696 INFO:teuthology.orchestra.run.smithi042.stdout:1786 read 0xa0f73d thru 0xa16e23 (0x76e7 bytes) 2024-09-14T23:57:06.696 INFO:teuthology.orchestra.run.smithi042.stdout:1788 writesame 0x91e343d thru 0x91ea402 data_size 0x39(0x6fc6 bytes) 2024-09-14T23:57:06.696 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:06.697 INFO:teuthology.orchestra.run.smithi042.stdout:1790 read 0xb469534 thru 0xb46e155 (0x4c22 bytes) 2024-09-14T23:57:06.697 INFO:teuthology.orchestra.run.smithi042.stdout:1791 writesame 0x22a8ad4 thru 0x22b248c data_size 0x17(0x99b9 bytes) 2024-09-14T23:57:06.697 INFO:teuthology.orchestra.run.smithi042.stdout:1792 trunc from 0xc1a95fb to 0x11f4311 2024-09-14T23:57:06.697 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:06.691+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:06.720 INFO:teuthology.orchestra.run.smithi042.stdout:1794 read 0x40234a thru 0x4095e4 (0x729b bytes) 2024-09-14T23:57:06.725 INFO:teuthology.orchestra.run.smithi042.stdout:1795 trunc from 0x11f4311 to 0x48f7cef 2024-09-14T23:57:06.740 INFO:teuthology.orchestra.run.smithi042.stdout:1796 trunc from 0x48f7cef to 0xd5e1aa3 2024-09-14T23:57:06.745 INFO:teuthology.orchestra.run.smithi042.stdout:1802 punch from 0x5a99fd7 to 0x5aa1e69, (0x7e92 bytes) 2024-09-14T23:57:06.745 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:06.754 INFO:teuthology.orchestra.run.smithi042.stdout:1804 trunc from 0xd5e1aa3 to 0x547b713 2024-09-14T23:57:06.760 INFO:teuthology.orchestra.run.smithi042.stdout:1806 clone 19 order 20 su 4096 sc 8 2024-09-14T23:57:06.793 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:06.792+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:06.810 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:06.810 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:06.810 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:57:06.811 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:06.819 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T23:57:06.903 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:06.902+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:06.966 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:06.967 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:06.967 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:57:06.967 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:57:06.967 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:06.976 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T23:57:06.995 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:06.994+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:07.084 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is set 2024-09-14T23:57:07.096 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:07.095+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:07.130 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:07.130 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:57:07.130 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:57:07.130 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:57:07.130 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.130 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:3500 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T23:57:07.130 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:37.990193+0000", 2024-09-14T23:57:07.130 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 29.137312955999999, 2024-09-14T23:57:07.130 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99440750899999997, 2024-09-14T23:57:07.130 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:07.131 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:07.131 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:07.131 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:07.131 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:07.131 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:07.131 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3500 2024-09-14T23:57:07.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.131 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:07.131 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.131 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:07.131 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990193+0000", 2024-09-14T23:57:07.131 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.131 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.132 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:07.132 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990193+0000", 2024-09-14T23:57:07.132 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.132 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.132 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:07.132 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990194+0000", 2024-09-14T23:57:07.132 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5000000000000001e-07 2024-09-14T23:57:07.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.132 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.133 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:07.133 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990200+0000", 2024-09-14T23:57:07.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4389999999999997e-06 2024-09-14T23:57:07.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.133 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:07.133 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990215+0000", 2024-09-14T23:57:07.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4613999999999999e-05 2024-09-14T23:57:07.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.133 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:07.134 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990224+0000", 2024-09-14T23:57:07.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1740000000000006e-06 2024-09-14T23:57:07.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:07.134 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990283+0000", 2024-09-14T23:57:07.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9231999999999998e-05 2024-09-14T23:57:07.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:57:07.134 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990291+0000", 2024-09-14T23:57:07.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5900000000000002e-06 2024-09-14T23:57:07.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.135 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:07.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:38.984249+0000", 2024-09-14T23:57:07.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99395822700000003 2024-09-14T23:57:07.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.135 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:07.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:38.984448+0000", 2024-09-14T23:57:07.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00019855900000000001 2024-09-14T23:57:07.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.136 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.136 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:07.136 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:38.984601+0000", 2024-09-14T23:57:07.136 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000152824 2024-09-14T23:57:07.136 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.136 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:07.136 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.136 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.136 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4055 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1093632~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:07.136 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099066+0000", 2024-09-14T23:57:07.136 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 28.028440067999998, 2024-09-14T23:57:07.136 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044174782000000003, 2024-09-14T23:57:07.137 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:07.137 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:07.137 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:07.137 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:07.137 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:07.137 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:07.137 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4055 2024-09-14T23:57:07.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.138 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:07.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:07.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099066+0000", 2024-09-14T23:57:07.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:07.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099066+0000", 2024-09-14T23:57:07.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.139 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:07.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099066+0000", 2024-09-14T23:57:07.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3700000000000001e-07 2024-09-14T23:57:07.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.139 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:07.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099067+0000", 2024-09-14T23:57:07.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1379999999999999e-06 2024-09-14T23:57:07.140 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.140 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.140 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:07.140 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099073+0000", 2024-09-14T23:57:07.140 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4790000000000004e-06 2024-09-14T23:57:07.140 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.140 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.140 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:07.141 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099075+0000", 2024-09-14T23:57:07.141 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5320000000000001e-06 2024-09-14T23:57:07.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.141 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:07.141 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143147+0000", 2024-09-14T23:57:07.141 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044071676999999997 2024-09-14T23:57:07.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.141 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:07.141 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143186+0000", 2024-09-14T23:57:07.141 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8821e-05 2024-09-14T23:57:07.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.142 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:07.142 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143241+0000", 2024-09-14T23:57:07.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4797999999999999e-05 2024-09-14T23:57:07.142 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.142 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:07.142 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.142 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4056 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1155072~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:07.142 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099079+0000", 2024-09-14T23:57:07.142 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 28.028427353000001, 2024-09-14T23:57:07.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044295020999999997, 2024-09-14T23:57:07.142 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:07.143 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:07.143 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:07.143 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:07.143 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:07.143 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:07.143 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4056 2024-09-14T23:57:07.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.143 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:07.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:07.144 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099079+0000", 2024-09-14T23:57:07.144 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.144 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.144 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:07.144 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099079+0000", 2024-09-14T23:57:07.144 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.144 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.145 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:07.145 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099079+0000", 2024-09-14T23:57:07.145 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1500000000000001e-07 2024-09-14T23:57:07.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.145 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.145 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:07.145 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099080+0000", 2024-09-14T23:57:07.145 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0210000000000001e-06 2024-09-14T23:57:07.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.145 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.145 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:07.145 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099083+0000", 2024-09-14T23:57:07.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.174e-06 2024-09-14T23:57:07.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.146 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:07.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099086+0000", 2024-09-14T23:57:07.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.446e-06 2024-09-14T23:57:07.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.146 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:07.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143269+0000", 2024-09-14T23:57:07.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044183489999999999 2024-09-14T23:57:07.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.146 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:07.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143314+0000", 2024-09-14T23:57:07.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5135000000000001e-05 2024-09-14T23:57:07.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.147 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:07.147 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143374+0000", 2024-09-14T23:57:07.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9540000000000003e-05 2024-09-14T23:57:07.147 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.147 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:07.147 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.147 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4057 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1216512~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:07.147 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099088+0000", 2024-09-14T23:57:07.148 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 28.028417566000002, 2024-09-14T23:57:07.148 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044406119000000001, 2024-09-14T23:57:07.148 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:07.148 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:07.148 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:07.148 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:07.148 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:07.148 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:07.148 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4057 2024-09-14T23:57:07.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.148 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:07.148 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.148 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:07.149 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099088+0000", 2024-09-14T23:57:07.149 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.149 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:07.149 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099088+0000", 2024-09-14T23:57:07.149 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.149 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:07.149 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099089+0000", 2024-09-14T23:57:07.149 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.04e-07 2024-09-14T23:57:07.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:07.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099090+0000", 2024-09-14T23:57:07.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.026e-06 2024-09-14T23:57:07.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:07.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099093+0000", 2024-09-14T23:57:07.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0819999999999999e-06 2024-09-14T23:57:07.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:07.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099095+0000", 2024-09-14T23:57:07.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4930000000000001e-06 2024-09-14T23:57:07.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:07.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143400+0000", 2024-09-14T23:57:07.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044304698000000003 2024-09-14T23:57:07.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.152 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:07.152 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143441+0000", 2024-09-14T23:57:07.152 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1100000000000003e-05 2024-09-14T23:57:07.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.152 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:07.153 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143495+0000", 2024-09-14T23:57:07.153 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3516000000000001e-05 2024-09-14T23:57:07.153 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.153 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:07.153 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.153 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.153 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4058 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1277952~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:07.153 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099099+0000", 2024-09-14T23:57:07.153 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 28.028407075000001, 2024-09-14T23:57:07.153 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044514583000000003, 2024-09-14T23:57:07.153 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:07.153 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:07.153 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:07.154 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:07.154 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:07.154 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:07.154 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4058 2024-09-14T23:57:07.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.154 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:07.154 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.154 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:07.154 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099099+0000", 2024-09-14T23:57:07.154 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.154 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.154 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:07.154 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099099+0000", 2024-09-14T23:57:07.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:07.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099099+0000", 2024-09-14T23:57:07.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9999999999999999e-07 2024-09-14T23:57:07.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:07.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099100+0000", 2024-09-14T23:57:07.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7199999999999997e-07 2024-09-14T23:57:07.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:07.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099103+0000", 2024-09-14T23:57:07.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.095e-06 2024-09-14T23:57:07.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.156 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:07.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099106+0000", 2024-09-14T23:57:07.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4229999999999999e-06 2024-09-14T23:57:07.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.156 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:07.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143524+0000", 2024-09-14T23:57:07.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044417891000000001 2024-09-14T23:57:07.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.157 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.157 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:07.157 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143568+0000", 2024-09-14T23:57:07.157 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4060000000000002e-05 2024-09-14T23:57:07.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.157 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.157 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:07.157 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143614+0000", 2024-09-14T23:57:07.157 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5942000000000002e-05 2024-09-14T23:57:07.157 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.157 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:07.157 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.158 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.158 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4059 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1339392~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:07.158 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099114+0000", 2024-09-14T23:57:07.158 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 28.028392535999998, 2024-09-14T23:57:07.158 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044617194999999998, 2024-09-14T23:57:07.158 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:07.158 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:07.158 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:07.158 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:07.158 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:07.158 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:07.158 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4059 2024-09-14T23:57:07.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.159 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:07.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.159 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:07.159 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099114+0000", 2024-09-14T23:57:07.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.159 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:07.159 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099114+0000", 2024-09-14T23:57:07.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.159 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:07.159 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099114+0000", 2024-09-14T23:57:07.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5899999999999998e-07 2024-09-14T23:57:07.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.160 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.160 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:07.160 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099115+0000", 2024-09-14T23:57:07.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3510000000000001e-06 2024-09-14T23:57:07.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.160 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.160 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:07.160 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099119+0000", 2024-09-14T23:57:07.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.16e-06 2024-09-14T23:57:07.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.160 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.160 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:07.160 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099123+0000", 2024-09-14T23:57:07.161 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1339999999999997e-06 2024-09-14T23:57:07.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.161 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.161 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:07.161 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143643+0000", 2024-09-14T23:57:07.161 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044519232999999998 2024-09-14T23:57:07.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.161 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.161 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:07.161 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143686+0000", 2024-09-14T23:57:07.161 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3349000000000001e-05 2024-09-14T23:57:07.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.161 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.161 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:07.162 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143731+0000", 2024-09-14T23:57:07.162 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4709000000000002e-05 2024-09-14T23:57:07.162 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.162 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:07.162 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.162 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.162 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.162 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4060 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1400832~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:07.162 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099129+0000", 2024-09-14T23:57:07.162 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 28.028377313, 2024-09-14T23:57:07.162 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044721271999999999, 2024-09-14T23:57:07.162 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:07.162 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:07.162 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:07.162 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:07.163 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:07.163 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:07.163 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4060 2024-09-14T23:57:07.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.163 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:07.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.163 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:07.163 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099129+0000", 2024-09-14T23:57:07.163 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.163 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:07.163 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099129+0000", 2024-09-14T23:57:07.163 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.164 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.164 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:07.164 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099191+0000", 2024-09-14T23:57:07.164 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1805999999999997e-05 2024-09-14T23:57:07.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.164 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.164 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:07.164 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099193+0000", 2024-09-14T23:57:07.164 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2330000000000001e-06 2024-09-14T23:57:07.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.164 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.164 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:07.164 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099200+0000", 2024-09-14T23:57:07.165 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4200000000000001e-06 2024-09-14T23:57:07.165 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.165 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.165 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:07.165 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099204+0000", 2024-09-14T23:57:07.165 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7500000000000001e-06 2024-09-14T23:57:07.165 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.165 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.165 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:07.165 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143760+0000", 2024-09-14T23:57:07.165 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044556182 2024-09-14T23:57:07.165 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.166 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.166 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:07.166 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143804+0000", 2024-09-14T23:57:07.166 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4239000000000001e-05 2024-09-14T23:57:07.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.166 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.166 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:07.166 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143850+0000", 2024-09-14T23:57:07.166 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5642000000000001e-05 2024-09-14T23:57:07.166 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.166 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:07.166 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.166 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.167 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4061 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1462272~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:07.167 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099215+0000", 2024-09-14T23:57:07.167 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 28.028291188000001, 2024-09-14T23:57:07.167 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044745659, 2024-09-14T23:57:07.167 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:07.167 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:07.167 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:07.167 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:07.167 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:07.167 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:07.167 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4061 2024-09-14T23:57:07.167 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.167 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:07.167 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.168 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:07.168 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099215+0000", 2024-09-14T23:57:07.168 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.168 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.168 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:07.168 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099215+0000", 2024-09-14T23:57:07.168 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.168 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.168 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:07.168 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100672+0000", 2024-09-14T23:57:07.168 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00145689 2024-09-14T23:57:07.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.169 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:07.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100677+0000", 2024-09-14T23:57:07.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0139999999999998e-06 2024-09-14T23:57:07.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.169 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:07.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100689+0000", 2024-09-14T23:57:07.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.183e-05 2024-09-14T23:57:07.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.169 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:07.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100696+0000", 2024-09-14T23:57:07.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2599999999999999e-06 2024-09-14T23:57:07.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.170 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.170 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:07.170 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143877+0000", 2024-09-14T23:57:07.170 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043181279000000003 2024-09-14T23:57:07.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.170 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.170 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:07.170 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143919+0000", 2024-09-14T23:57:07.170 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1783000000000003e-05 2024-09-14T23:57:07.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.170 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.170 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:07.170 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143961+0000", 2024-09-14T23:57:07.170 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1603000000000001e-05 2024-09-14T23:57:07.171 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.171 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:07.171 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.171 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.171 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4070 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 2015232~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:07.171 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.100903+0000", 2024-09-14T23:57:07.171 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 28.026603479999999, 2024-09-14T23:57:07.171 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044103815999999997, 2024-09-14T23:57:07.171 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:07.171 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:07.171 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:07.171 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:07.172 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:07.172 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:07.172 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4070 2024-09-14T23:57:07.172 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.172 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:07.172 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.172 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:07.172 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100903+0000", 2024-09-14T23:57:07.172 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.172 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.172 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.173 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:07.173 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100903+0000", 2024-09-14T23:57:07.173 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.173 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.173 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.173 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:07.173 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.102671+0000", 2024-09-14T23:57:07.173 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001768279 2024-09-14T23:57:07.173 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.173 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.173 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:07.173 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.102675+0000", 2024-09-14T23:57:07.173 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9609999999999999e-06 2024-09-14T23:57:07.173 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.174 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.174 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:07.174 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.102684+0000", 2024-09-14T23:57:07.174 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.329e-06 2024-09-14T23:57:07.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.174 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.174 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:07.174 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.102689+0000", 2024-09-14T23:57:07.174 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7079999999999996e-06 2024-09-14T23:57:07.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.174 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.174 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:07.174 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.144912+0000", 2024-09-14T23:57:07.174 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.042223305000000003 2024-09-14T23:57:07.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.175 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.175 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:07.175 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.144960+0000", 2024-09-14T23:57:07.175 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7670000000000003e-05 2024-09-14T23:57:07.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.175 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.175 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:07.175 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.145006+0000", 2024-09-14T23:57:07.175 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6563999999999997e-05 2024-09-14T23:57:07.175 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.175 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:07.175 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.175 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.176 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4096 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1282048~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:07.176 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.104957+0000", 2024-09-14T23:57:07.176 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 28.022549306999998, 2024-09-14T23:57:07.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044093016999999998, 2024-09-14T23:57:07.176 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:07.176 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:07.176 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:07.176 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:07.176 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:07.176 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:07.176 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4096 2024-09-14T23:57:07.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.176 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:07.176 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.177 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:07.177 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104957+0000", 2024-09-14T23:57:07.177 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.177 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:07.177 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104957+0000", 2024-09-14T23:57:07.177 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.177 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:07.177 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104957+0000", 2024-09-14T23:57:07.177 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7400000000000003e-07 2024-09-14T23:57:07.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.178 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:07.178 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104960+0000", 2024-09-14T23:57:07.178 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3800000000000001e-06 2024-09-14T23:57:07.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.178 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:07.178 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104967+0000", 2024-09-14T23:57:07.178 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4030000000000001e-06 2024-09-14T23:57:07.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.178 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:07.178 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104973+0000", 2024-09-14T23:57:07.178 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7370000000000003e-06 2024-09-14T23:57:07.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.179 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.179 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:07.179 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.148883+0000", 2024-09-14T23:57:07.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043910124000000002 2024-09-14T23:57:07.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.179 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.179 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:07.179 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.148966+0000", 2024-09-14T23:57:07.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2868999999999996e-05 2024-09-14T23:57:07.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.179 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.179 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:07.179 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149050+0000", 2024-09-14T23:57:07.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3930000000000004e-05 2024-09-14T23:57:07.180 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.180 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:07.180 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.180 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.180 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4097 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1343488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:07.180 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.104983+0000", 2024-09-14T23:57:07.180 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 28.022523518, 2024-09-14T23:57:07.180 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044182104999999999, 2024-09-14T23:57:07.180 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:07.180 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:07.181 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:07.181 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:07.181 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:07.181 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:07.181 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4097 2024-09-14T23:57:07.181 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.181 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:07.181 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.182 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:07.182 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104983+0000", 2024-09-14T23:57:07.182 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.182 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.182 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:07.182 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104983+0000", 2024-09-14T23:57:07.182 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.182 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.182 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:07.182 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105207+0000", 2024-09-14T23:57:07.182 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000224687 2024-09-14T23:57:07.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.183 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.183 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:07.183 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105211+0000", 2024-09-14T23:57:07.183 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3050000000000001e-06 2024-09-14T23:57:07.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.183 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.183 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:07.183 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105217+0000", 2024-09-14T23:57:07.183 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8170000000000001e-06 2024-09-14T23:57:07.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.183 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.183 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:07.183 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105222+0000", 2024-09-14T23:57:07.183 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.267e-06 2024-09-14T23:57:07.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.184 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:07.184 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149079+0000", 2024-09-14T23:57:07.184 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043857581999999999 2024-09-14T23:57:07.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.184 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:07.184 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149122+0000", 2024-09-14T23:57:07.184 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2899999999999999e-05 2024-09-14T23:57:07.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.184 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:07.184 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149165+0000", 2024-09-14T23:57:07.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2546999999999998e-05 2024-09-14T23:57:07.185 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.185 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:07.185 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.185 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.185 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4098 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1404928~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:07.185 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.105242+0000", 2024-09-14T23:57:07.185 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 28.022263706, 2024-09-14T23:57:07.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044121104000000001, 2024-09-14T23:57:07.185 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:07.185 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:07.185 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:07.185 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:07.186 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:07.186 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:07.186 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4098 2024-09-14T23:57:07.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.186 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:07.186 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.186 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:07.186 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105242+0000", 2024-09-14T23:57:07.186 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.186 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.186 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:07.186 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105242+0000", 2024-09-14T23:57:07.186 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.187 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.187 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:07.187 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105243+0000", 2024-09-14T23:57:07.187 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0399999999999995e-07 2024-09-14T23:57:07.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.187 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.187 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:07.187 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105246+0000", 2024-09-14T23:57:07.187 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6309999999999999e-06 2024-09-14T23:57:07.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.187 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.187 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:07.187 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105253+0000", 2024-09-14T23:57:07.187 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0940000000000002e-06 2024-09-14T23:57:07.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.188 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.188 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:07.188 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105260+0000", 2024-09-14T23:57:07.188 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1799999999999999e-06 2024-09-14T23:57:07.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.188 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.188 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:07.188 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149201+0000", 2024-09-14T23:57:07.188 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043940629000000002 2024-09-14T23:57:07.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.188 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.188 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:07.188 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149283+0000", 2024-09-14T23:57:07.188 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2360000000000004e-05 2024-09-14T23:57:07.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.189 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.189 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:07.189 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149363+0000", 2024-09-14T23:57:07.189 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0506000000000005e-05 2024-09-14T23:57:07.189 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.189 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:07.189 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.189 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.189 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4099 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1466368~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:07.189 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.105270+0000", 2024-09-14T23:57:07.189 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 28.022236111000002, 2024-09-14T23:57:07.189 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044195433999999999, 2024-09-14T23:57:07.189 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:07.190 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:07.190 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:07.190 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:07.190 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:07.190 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:07.190 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4099 2024-09-14T23:57:07.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.190 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:07.190 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.190 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:07.190 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105270+0000", 2024-09-14T23:57:07.190 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.190 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.190 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:07.191 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105270+0000", 2024-09-14T23:57:07.191 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.191 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.191 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:07.191 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105429+0000", 2024-09-14T23:57:07.191 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015940299999999999 2024-09-14T23:57:07.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.191 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.191 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:07.191 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105433+0000", 2024-09-14T23:57:07.191 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3579999999999999e-06 2024-09-14T23:57:07.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.191 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.192 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:07.192 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105439+0000", 2024-09-14T23:57:07.192 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1759999999999998e-06 2024-09-14T23:57:07.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.192 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.192 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:07.192 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105445+0000", 2024-09-14T23:57:07.192 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9689999999999999e-06 2024-09-14T23:57:07.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.192 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.192 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:07.192 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149381+0000", 2024-09-14T23:57:07.192 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043936389999999999 2024-09-14T23:57:07.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.193 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.193 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:07.193 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149422+0000", 2024-09-14T23:57:07.193 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0865000000000003e-05 2024-09-14T23:57:07.193 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.193 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.193 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:07.193 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149465+0000", 2024-09-14T23:57:07.193 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3272999999999997e-05 2024-09-14T23:57:07.193 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.193 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:07.193 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.193 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.193 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.193 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4100 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1527808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:07.194 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.105454+0000", 2024-09-14T23:57:07.194 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 28.022051640000001, 2024-09-14T23:57:07.194 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044217411999999998, 2024-09-14T23:57:07.194 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:07.194 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:07.194 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:07.194 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:07.194 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:07.194 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:07.194 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4100 2024-09-14T23:57:07.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.194 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:07.194 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.194 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:07.195 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105454+0000", 2024-09-14T23:57:07.195 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.195 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.195 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:07.195 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105454+0000", 2024-09-14T23:57:07.196 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.196 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.196 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.196 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:07.196 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105578+0000", 2024-09-14T23:57:07.196 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012401799999999999 2024-09-14T23:57:07.197 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.197 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.197 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:07.197 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105583+0000", 2024-09-14T23:57:07.197 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1779999999999996e-06 2024-09-14T23:57:07.197 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.198 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.198 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:07.198 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105590+0000", 2024-09-14T23:57:07.198 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4279999999999999e-06 2024-09-14T23:57:07.198 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.198 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.199 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:07.199 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105596+0000", 2024-09-14T23:57:07.199 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0120000000000002e-06 2024-09-14T23:57:07.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.199 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.200 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:07.200 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149503+0000", 2024-09-14T23:57:07.200 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043906580000000001 2024-09-14T23:57:07.200 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.200 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.200 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:07.200 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149585+0000", 2024-09-14T23:57:07.200 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2356999999999998e-05 2024-09-14T23:57:07.200 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.200 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.200 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:07.200 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149672+0000", 2024-09-14T23:57:07.200 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6839e-05 2024-09-14T23:57:07.200 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.201 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:07.201 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.201 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.201 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.201 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4101 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1589248~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:07.201 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.105609+0000", 2024-09-14T23:57:07.201 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 28.021897101, 2024-09-14T23:57:07.201 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044175987999999999, 2024-09-14T23:57:07.201 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:07.201 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:07.202 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:07.202 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:07.202 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:07.202 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:07.202 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4101 2024-09-14T23:57:07.202 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.202 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:07.202 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.202 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:07.202 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105609+0000", 2024-09-14T23:57:07.203 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.203 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.203 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:07.203 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105609+0000", 2024-09-14T23:57:07.203 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.203 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.203 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:07.203 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.106671+0000", 2024-09-14T23:57:07.203 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010619270000000001 2024-09-14T23:57:07.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.203 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.203 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:07.203 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.106675+0000", 2024-09-14T23:57:07.204 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4429999999999996e-06 2024-09-14T23:57:07.204 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.204 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.204 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:07.204 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.106685+0000", 2024-09-14T23:57:07.204 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7429999999999997e-06 2024-09-14T23:57:07.204 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.204 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.205 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:07.205 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.106690+0000", 2024-09-14T23:57:07.205 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.293e-06 2024-09-14T23:57:07.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.205 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.205 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:07.205 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149700+0000", 2024-09-14T23:57:07.206 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043009512 2024-09-14T23:57:07.206 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.206 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.206 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:07.206 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149742+0000", 2024-09-14T23:57:07.206 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2083999999999999e-05 2024-09-14T23:57:07.206 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.206 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.206 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:07.206 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149785+0000", 2024-09-14T23:57:07.206 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2985999999999998e-05 2024-09-14T23:57:07.206 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.207 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:07.207 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.207 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.207 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4116 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 180224~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:07.207 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.107575+0000", 2024-09-14T23:57:07.207 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 28.019930633000001, 2024-09-14T23:57:07.207 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044157522999999997, 2024-09-14T23:57:07.207 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:07.207 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:07.207 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:07.207 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:07.207 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:07.207 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:07.208 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4116 2024-09-14T23:57:07.208 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.208 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:07.208 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.208 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:07.208 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.107575+0000", 2024-09-14T23:57:07.208 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.208 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.208 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.208 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:07.208 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.107575+0000", 2024-09-14T23:57:07.208 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.208 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.208 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.209 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:07.209 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108671+0000", 2024-09-14T23:57:07.209 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010954510000000001 2024-09-14T23:57:07.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.209 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.209 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:07.209 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108676+0000", 2024-09-14T23:57:07.209 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6639999999999997e-06 2024-09-14T23:57:07.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.209 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.209 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:07.209 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108685+0000", 2024-09-14T23:57:07.209 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1130000000000007e-06 2024-09-14T23:57:07.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.210 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:07.210 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108691+0000", 2024-09-14T23:57:07.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6019999999999996e-06 2024-09-14T23:57:07.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.210 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:07.210 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.151530+0000", 2024-09-14T23:57:07.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.042838345999999999 2024-09-14T23:57:07.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.210 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:07.210 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.151619+0000", 2024-09-14T23:57:07.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9302999999999995e-05 2024-09-14T23:57:07.211 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.211 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.211 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:07.211 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.151733+0000", 2024-09-14T23:57:07.211 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011404399999999999 2024-09-14T23:57:07.211 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.211 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:07.211 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.211 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.211 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.211 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4122 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 548864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:07.211 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.108842+0000", 2024-09-14T23:57:07.211 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 28.018663794999998, 2024-09-14T23:57:07.211 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044103631999999997, 2024-09-14T23:57:07.212 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:07.212 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:07.212 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:07.212 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:07.212 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:07.212 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:07.212 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4122 2024-09-14T23:57:07.212 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.212 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:07.212 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.212 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:07.212 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108842+0000", 2024-09-14T23:57:07.212 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.212 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.213 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.213 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:07.213 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108842+0000", 2024-09-14T23:57:07.213 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.213 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.213 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.213 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:07.213 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108843+0000", 2024-09-14T23:57:07.213 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7000000000000001e-07 2024-09-14T23:57:07.213 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.213 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.213 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:07.213 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108844+0000", 2024-09-14T23:57:07.213 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6479999999999999e-06 2024-09-14T23:57:07.213 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.214 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.214 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:07.214 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108848+0000", 2024-09-14T23:57:07.214 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8029999999999998e-06 2024-09-14T23:57:07.214 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.214 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.214 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:07.214 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108851+0000", 2024-09-14T23:57:07.214 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7939999999999998e-06 2024-09-14T23:57:07.214 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.214 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.214 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:07.214 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.152838+0000", 2024-09-14T23:57:07.214 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043987351000000001 2024-09-14T23:57:07.215 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.215 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.215 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:07.215 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.152885+0000", 2024-09-14T23:57:07.215 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7237000000000002e-05 2024-09-14T23:57:07.215 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.215 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.215 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:07.215 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.152946+0000", 2024-09-14T23:57:07.216 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0529000000000003e-05 2024-09-14T23:57:07.216 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.216 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:07.216 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.216 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.216 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.216 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4123 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 610304~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:07.216 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.108855+0000", 2024-09-14T23:57:07.216 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 28.018651466000001, 2024-09-14T23:57:07.216 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044321610999999997, 2024-09-14T23:57:07.216 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:07.216 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:07.216 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:07.216 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:07.217 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:07.217 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:07.217 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4123 2024-09-14T23:57:07.217 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.217 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:07.217 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.217 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:07.217 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108855+0000", 2024-09-14T23:57:07.217 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.217 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.217 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.217 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:07.217 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108855+0000", 2024-09-14T23:57:07.217 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.218 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.218 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.218 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:07.218 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.110671+0000", 2024-09-14T23:57:07.218 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018159719999999999 2024-09-14T23:57:07.218 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.218 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.218 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:07.218 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.110676+0000", 2024-09-14T23:57:07.218 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9450000000000001e-06 2024-09-14T23:57:07.218 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.218 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.218 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:07.218 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.110685+0000", 2024-09-14T23:57:07.218 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5359999999999998e-06 2024-09-14T23:57:07.219 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.219 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.219 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:07.219 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.110692+0000", 2024-09-14T23:57:07.219 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9280000000000004e-06 2024-09-14T23:57:07.219 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.219 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.219 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:07.219 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.152984+0000", 2024-09-14T23:57:07.219 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.042291677999999999 2024-09-14T23:57:07.219 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.219 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.219 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:07.219 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.153066+0000", 2024-09-14T23:57:07.219 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2657999999999994e-05 2024-09-14T23:57:07.220 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.220 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.220 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:07.220 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.153176+0000", 2024-09-14T23:57:07.220 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000109894 2024-09-14T23:57:07.220 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.220 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:07.220 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.220 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.220 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.220 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4136 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:07.220 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.111056+0000", 2024-09-14T23:57:07.220 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 28.016450318, 2024-09-14T23:57:07.220 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044097428000000001, 2024-09-14T23:57:07.221 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:07.221 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:07.221 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:07.221 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:07.221 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:07.221 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:07.221 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4136 2024-09-14T23:57:07.221 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.221 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:07.221 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.221 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:07.221 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.111056+0000", 2024-09-14T23:57:07.221 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.221 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.222 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.222 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:07.222 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.111056+0000", 2024-09-14T23:57:07.222 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.222 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.222 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.222 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:07.222 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.112671+0000", 2024-09-14T23:57:07.222 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016153999999999999 2024-09-14T23:57:07.222 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.222 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.222 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:07.222 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.112675+0000", 2024-09-14T23:57:07.222 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2830000000000003e-06 2024-09-14T23:57:07.222 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.223 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.223 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:07.223 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.112684+0000", 2024-09-14T23:57:07.223 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7800000000000006e-06 2024-09-14T23:57:07.223 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.223 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.223 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:07.223 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.112689+0000", 2024-09-14T23:57:07.223 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0000000000000004e-06 2024-09-14T23:57:07.223 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.223 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.223 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:07.223 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.155064+0000", 2024-09-14T23:57:07.223 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.042374423000000001 2024-09-14T23:57:07.223 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.224 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.224 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:07.224 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.155111+0000", 2024-09-14T23:57:07.224 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7367000000000002e-05 2024-09-14T23:57:07.224 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.224 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.224 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:07.224 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.155153+0000", 2024-09-14T23:57:07.224 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2175000000000002e-05 2024-09-14T23:57:07.224 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.224 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:07.224 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.224 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.224 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.225 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:46572 3.1d 3:b9e1f524:::rbd_header.1118cee06433:head [watch reconnect cookie 140670782499824 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-14T23:57:07.225 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:57.046868+0000", 2024-09-14T23:57:07.225 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 10.080638512, 2024-09-14T23:57:07.225 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0243412620000001, 2024-09-14T23:57:07.225 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:07.225 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:07.225 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:07.225 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:07.225 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:07.225 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:07.225 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 46572 2024-09-14T23:57:07.225 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.225 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:07.225 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.226 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:07.226 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046868+0000", 2024-09-14T23:57:07.226 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.226 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.226 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.226 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:07.226 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046868+0000", 2024-09-14T23:57:07.226 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.226 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.226 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.226 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:07.226 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046870+0000", 2024-09-14T23:57:07.226 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.204e-06 2024-09-14T23:57:07.226 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.226 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.227 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:07.227 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046877+0000", 2024-09-14T23:57:07.227 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5100000000000001e-06 2024-09-14T23:57:07.227 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.227 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.227 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:07.227 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046898+0000", 2024-09-14T23:57:07.227 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1149999999999999e-05 2024-09-14T23:57:07.227 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.227 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.228 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:07.228 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046911+0000", 2024-09-14T23:57:07.228 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2234e-05 2024-09-14T23:57:07.228 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.228 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.228 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:07.228 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046988+0000", 2024-09-14T23:57:07.228 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7445000000000001e-05 2024-09-14T23:57:07.228 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.228 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.228 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:57:07.229 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.047001+0000", 2024-09-14T23:57:07.229 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2951e-05 2024-09-14T23:57:07.229 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.229 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.229 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:07.229 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.069287+0000", 2024-09-14T23:57:07.229 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.022285748 2024-09-14T23:57:07.229 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.229 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.229 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for rw locks", 2024-09-14T23:57:07.229 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.069397+0000", 2024-09-14T23:57:07.229 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000110326 2024-09-14T23:57:07.229 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.229 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.230 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:07.230 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.069997+0000", 2024-09-14T23:57:07.230 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00059984399999999998 2024-09-14T23:57:07.230 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.230 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.230 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:07.230 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.070109+0000", 2024-09-14T23:57:07.230 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000112367 2024-09-14T23:57:07.230 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.230 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.230 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:07.230 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.071209+0000", 2024-09-14T23:57:07.231 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010994830000000001 2024-09-14T23:57:07.231 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.231 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:07.231 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.231 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.231 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:07.231 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:07.232 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone18 from 0x547b713 (overlap 0x11f4311) to 0x3973d0 2024-09-14T23:57:07.233 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:07.231+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:07.233 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T23:57:07.308 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-09-14T23:57:07.309 INFO:teuthology.orchestra.run.smithi042.stdout:1808 read 0x34e4475 thru 0x34e7b3a (0x36c6 bytes) 2024-09-14T23:57:07.310 INFO:teuthology.orchestra.run.smithi042.stdout:1809 writesame 0x1929f0d thru 0x1938724 data_size 0x1c(0xe818 bytes) 2024-09-14T23:57:07.333 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:07.332+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:07.334 INFO:teuthology.orchestra.run.smithi042.stdout:1810 writesame 0xadb4893 thru 0xadc2711 data_size 0x4f(0xde7f bytes) 2024-09-14T23:57:07.341 INFO:teuthology.orchestra.run.smithi042.stdout:1811 trunc from 0xadc2712 to 0x4e5a931 2024-09-14T23:57:07.356 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:07.357 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:07.357 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:57:07.357 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:07.365 INFO:teuthology.orchestra.run.smithi042.stdout:1812 writesame 0x7be36f8 thru 0x7be5bb7 data_size 0x54(0x24c0 bytes) 2024-09-14T23:57:07.365 INFO:teuthology.orchestra.run.smithi042.stdout:1813 read 0x23128ff thru 0x2315c4f (0x3351 bytes) 2024-09-14T23:57:07.365 INFO:teuthology.orchestra.run.smithi042.stdout:1814 read 0x729ac39 thru 0x72a32ae (0x8676 bytes) 2024-09-14T23:57:07.365 INFO:teuthology.orchestra.run.smithi042.stdout:1815 read 0x300372f thru 0x3007d6c (0x463e bytes) 2024-09-14T23:57:07.365 INFO:teuthology.orchestra.run.smithi042.stdout:1817 punch from 0x3db47c2 to 0x3dc37ae, (0xefec bytes) 2024-09-14T23:57:07.366 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T23:57:07.375 INFO:teuthology.orchestra.run.smithi042.stdout:1819 write 0xc767b51 thru 0xc77655f (0xea0f bytes) 2024-09-14T23:57:07.385 INFO:teuthology.orchestra.run.smithi042.stdout:1822 write 0x88787ee thru 0x887d970 (0x5183 bytes) 2024-09-14T23:57:07.393 INFO:teuthology.orchestra.run.smithi042.stdout:1823 writesame 0x4e371a0 thru 0x4e40052 data_size 0x29(0x8eb3 bytes) 2024-09-14T23:57:07.396 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:07.396 INFO:teuthology.orchestra.run.smithi042.stdout:1826 writesame 0xc40122 thru 0xc41930 data_size 0x1(0x180f bytes) 2024-09-14T23:57:07.425 INFO:teuthology.orchestra.run.smithi042.stdout:1827 read 0xb962bb thru 0xb978e2 (0x1628 bytes) 2024-09-14T23:57:07.425 INFO:teuthology.orchestra.run.smithi042.stdout:1828 writesame 0x2af7008 thru 0x2afd972 data_size 0x1(0x696b bytes) 2024-09-14T23:57:07.432 INFO:teuthology.orchestra.run.smithi042.stdout:1831 punch from 0x676ab42 to 0x6773ba0, (0x905e bytes) 2024-09-14T23:57:07.432 INFO:teuthology.orchestra.run.smithi042.stdout:1834 write 0x70ada52 thru 0x70bd50e (0xfabd bytes) 2024-09-14T23:57:07.434 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:07.433+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:07.442 INFO:teuthology.orchestra.run.smithi042.stdout:1836 write 0x5fd9d4b thru 0x5fe3352 (0x9608 bytes) 2024-09-14T23:57:07.451 INFO:teuthology.orchestra.run.smithi042.stdout:1837 punch from 0x6a90b1f to 0x6a9e465, (0xd946 bytes) 2024-09-14T23:57:07.451 INFO:teuthology.orchestra.run.smithi042.stdout:1839 read 0x335bf thru 0x3ea6e (0xb4b0 bytes) 2024-09-14T23:57:07.453 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:07.453 INFO:teuthology.orchestra.run.smithi042.stdout:1842 write 0xafb1282 thru 0xafbdeb0 (0xcc2f bytes) 2024-09-14T23:57:07.463 INFO:teuthology.orchestra.run.smithi042.stdout:1843 write 0xabd7fb thru 0xacd638 (0xfe3e bytes) 2024-09-14T23:57:07.524 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:07.524 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:07.524 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:57:07.524 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:57:07.525 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:07.535 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:07.533+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:07.536 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T23:57:07.568 INFO:teuthology.orchestra.run.smithi042.stdout:1844 write 0xaf9ff6a thru 0xafa061b (0x6b2 bytes) 2024-09-14T23:57:07.570 INFO:teuthology.orchestra.run.smithi042.stdout:1845 write 0x8ac3d15 thru 0x8ac4dca (0x10b6 bytes) 2024-09-14T23:57:07.571 INFO:teuthology.orchestra.run.smithi042.stdout:1846 writesame 0x90ada52 thru 0x90b587d data_size 0x64(0x7e2c bytes) 2024-09-14T23:57:07.580 INFO:teuthology.orchestra.run.smithi042.stdout:1847 write 0x1be4854 thru 0x1bebe97 (0x7644 bytes) 2024-09-14T23:57:07.588 INFO:teuthology.orchestra.run.smithi042.stdout:1848 write 0xce37b9a thru 0xce38328 (0x78f bytes) 2024-09-14T23:57:07.592 INFO:teuthology.orchestra.run.smithi042.stdout:1849 punch from 0x45a4b to 0x4d5ca, (0x7b7f bytes) 2024-09-14T23:57:07.592 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:07.592 INFO:teuthology.orchestra.run.smithi042.stdout:1851 read 0x5d47daf thru 0x5d53c57 (0xbea9 bytes) 2024-09-14T23:57:07.593 INFO:teuthology.orchestra.run.smithi042.stdout:1852 read 0xc8b60e4 thru 0xc8bdfe6 (0x7f03 bytes) 2024-09-14T23:57:07.604 INFO:teuthology.orchestra.run.smithi042.stdout:1853 trunc from 0xce38329 to 0x31abb85 2024-09-14T23:57:07.633 INFO:teuthology.orchestra.run.smithi042.stdout:1854 punch from 0x285fe20 to 0x2865a89, (0x5c69 bytes) 2024-09-14T23:57:07.633 INFO:teuthology.orchestra.run.smithi042.stdout:1856 punch from 0x8dcc66 to 0x8de814, (0x1bae bytes) 2024-09-14T23:57:07.633 INFO:teuthology.orchestra.run.smithi042.stdout:1857 writesame 0x164c95 thru 0x165601 data_size 0x1(0x96d bytes) 2024-09-14T23:57:07.636 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:07.635+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:07.658 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:07.658 INFO:teuthology.orchestra.run.smithi042.stdout:1861 write 0xee9332 thru 0xee93e1 (0xb0 bytes) 2024-09-14T23:57:07.659 INFO:teuthology.orchestra.run.smithi042.stdout:1863 read 0xccaf1e thru 0xccd2c1 (0x23a4 bytes) 2024-09-14T23:57:07.669 INFO:teuthology.orchestra.run.smithi042.stdout:1864 write 0x63cde0d thru 0x63dd396 (0xf58a bytes) 2024-09-14T23:57:07.692 INFO:teuthology.orchestra.run.smithi042.stdout:1865 write 0xd8480f6 thru 0xd84a14e (0x2059 bytes) 2024-09-14T23:57:07.697 INFO:teuthology.orchestra.run.smithi042.stdout:1866 punch from 0x51b771b to 0x51c35c2, (0xbea7 bytes) 2024-09-14T23:57:07.709 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:07.709 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:57:07.709 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:57:07.709 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:57:07.709 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.709 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4284.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-14T23:57:07.709 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:57:07.709 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 45.182278955000001, 2024-09-14T23:57:07.709 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.004942086, 2024-09-14T23:57:07.709 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:07.710 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:07.710 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:07.710 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:07.710 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4284", 2024-09-14T23:57:07.710 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/4278347926", 2024-09-14T23:57:07.710 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1 2024-09-14T23:57:07.710 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.710 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:07.710 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.710 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:07.710 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:57:07.710 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.710 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.710 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.711 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:07.711 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:57:07.711 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.711 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.711 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.711 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:07.711 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524331+0000", 2024-09-14T23:57:07.711 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.383e-05 2024-09-14T23:57:07.711 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.711 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.711 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:07.711 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524339+0000", 2024-09-14T23:57:07.711 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8290000000000008e-06 2024-09-14T23:57:07.711 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.711 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.711 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:07.712 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524359+0000", 2024-09-14T23:57:07.712 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9998000000000001e-05 2024-09-14T23:57:07.712 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.712 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.712 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:07.712 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524373+0000", 2024-09-14T23:57:07.712 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4304000000000001e-05 2024-09-14T23:57:07.712 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.712 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.712 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:07.712 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524488+0000", 2024-09-14T23:57:07.712 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114663 2024-09-14T23:57:07.712 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.712 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.713 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:57:07.713 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524501+0000", 2024-09-14T23:57:07.713 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3349999999999999e-05 2024-09-14T23:57:07.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.713 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.713 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:07.713 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.526868+0000", 2024-09-14T23:57:07.713 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023670090000000002 2024-09-14T23:57:07.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.713 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.713 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:07.714 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.527041+0000", 2024-09-14T23:57:07.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000172597 2024-09-14T23:57:07.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.714 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.714 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-14T23:57:07.714 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.527384+0000", 2024-09-14T23:57:07.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00034309500000000002 2024-09-14T23:57:07.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.714 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.714 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:07.714 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529103+0000", 2024-09-14T23:57:07.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017185220000000001 2024-09-14T23:57:07.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.714 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.715 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:57:07.715 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529188+0000", 2024-09-14T23:57:07.715 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5591000000000001e-05 2024-09-14T23:57:07.715 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.715 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.715 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:07.715 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529227+0000", 2024-09-14T23:57:07.715 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8563000000000003e-05 2024-09-14T23:57:07.715 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.715 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.715 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:07.715 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529259+0000", 2024-09-14T23:57:07.715 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2734999999999997e-05 2024-09-14T23:57:07.715 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.716 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:07.716 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.716 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.716 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.716 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4284.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-14T23:57:07.716 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:23.530683+0000", 2024-09-14T23:57:07.716 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 44.175913289999997, 2024-09-14T23:57:07.716 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0036133110000000001, 2024-09-14T23:57:07.716 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:07.716 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:07.716 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:07.716 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:07.716 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4284", 2024-09-14T23:57:07.717 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/4278347926", 2024-09-14T23:57:07.717 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5 2024-09-14T23:57:07.717 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.717 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:07.717 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.717 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:07.717 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530683+0000", 2024-09-14T23:57:07.717 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.717 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.717 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.717 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:07.717 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530683+0000", 2024-09-14T23:57:07.717 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.718 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.718 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.718 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:07.718 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530685+0000", 2024-09-14T23:57:07.718 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.075e-06 2024-09-14T23:57:07.718 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.718 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.718 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:07.718 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530695+0000", 2024-09-14T23:57:07.718 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0261999999999999e-05 2024-09-14T23:57:07.718 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.718 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.718 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:07.718 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530715+0000", 2024-09-14T23:57:07.719 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9930000000000001e-05 2024-09-14T23:57:07.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.719 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.719 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:07.719 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530724+0000", 2024-09-14T23:57:07.719 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.901e-06 2024-09-14T23:57:07.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.719 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.719 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:07.719 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530800+0000", 2024-09-14T23:57:07.719 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5873999999999999e-05 2024-09-14T23:57:07.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.719 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.719 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-14T23:57:07.719 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530807+0000", 2024-09-14T23:57:07.720 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2389999999999996e-06 2024-09-14T23:57:07.720 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.720 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.720 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:07.720 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.532494+0000", 2024-09-14T23:57:07.720 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016871830000000001 2024-09-14T23:57:07.720 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.720 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.720 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:07.720 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.532599+0000", 2024-09-14T23:57:07.720 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010472 2024-09-14T23:57:07.720 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.720 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.720 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1", 2024-09-14T23:57:07.721 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.532715+0000", 2024-09-14T23:57:07.721 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011568199999999999 2024-09-14T23:57:07.721 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.721 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.721 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:57:07.721 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.533460+0000", 2024-09-14T23:57:07.721 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00074494999999999997 2024-09-14T23:57:07.721 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.721 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.721 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:07.721 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534230+0000", 2024-09-14T23:57:07.721 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00077036599999999998 2024-09-14T23:57:07.721 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.721 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.722 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:07.722 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534266+0000", 2024-09-14T23:57:07.722 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5725000000000001e-05 2024-09-14T23:57:07.722 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.722 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.722 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:07.722 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534296+0000", 2024-09-14T23:57:07.722 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0403999999999999e-05 2024-09-14T23:57:07.722 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.722 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:07.722 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.722 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.722 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.722 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4284.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-14T23:57:07.722 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:24.535160+0000", 2024-09-14T23:57:07.722 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 43.171436649, 2024-09-14T23:57:07.723 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033473650000000001, 2024-09-14T23:57:07.723 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:07.723 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:07.723 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:07.723 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:07.723 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4284", 2024-09-14T23:57:07.723 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/4278347926", 2024-09-14T23:57:07.723 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 7 2024-09-14T23:57:07.723 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.723 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:07.723 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.723 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:07.723 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535160+0000", 2024-09-14T23:57:07.723 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.723 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.724 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.724 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:07.724 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535160+0000", 2024-09-14T23:57:07.724 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.724 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.724 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.724 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:07.724 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535162+0000", 2024-09-14T23:57:07.724 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3369999999999998e-06 2024-09-14T23:57:07.724 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.724 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.724 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:07.724 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535171+0000", 2024-09-14T23:57:07.724 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7700000000000007e-06 2024-09-14T23:57:07.724 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.725 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.725 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:07.725 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535189+0000", 2024-09-14T23:57:07.725 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7938e-05 2024-09-14T23:57:07.725 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.725 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.725 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:07.725 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535197+0000", 2024-09-14T23:57:07.725 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5639999999999997e-06 2024-09-14T23:57:07.725 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.725 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.725 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:07.725 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535266+0000", 2024-09-14T23:57:07.726 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8926999999999999e-05 2024-09-14T23:57:07.726 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.726 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.726 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-14T23:57:07.726 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535271+0000", 2024-09-14T23:57:07.726 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0329999999999999e-06 2024-09-14T23:57:07.726 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.726 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.726 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:07.726 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.537064+0000", 2024-09-14T23:57:07.726 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001792929 2024-09-14T23:57:07.726 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.726 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.726 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:07.727 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.537131+0000", 2024-09-14T23:57:07.727 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6605000000000004e-05 2024-09-14T23:57:07.727 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.727 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.727 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1", 2024-09-14T23:57:07.727 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.537271+0000", 2024-09-14T23:57:07.727 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013992400000000001 2024-09-14T23:57:07.727 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.727 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.727 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:57:07.727 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.538093+0000", 2024-09-14T23:57:07.727 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00082210200000000001 2024-09-14T23:57:07.727 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.727 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.728 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:07.728 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.538439+0000", 2024-09-14T23:57:07.728 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00034595999999999997 2024-09-14T23:57:07.728 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.728 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.728 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:07.728 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.538476+0000", 2024-09-14T23:57:07.728 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6770000000000002e-05 2024-09-14T23:57:07.728 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.728 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.728 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:07.728 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.538507+0000", 2024-09-14T23:57:07.728 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1506e-05 2024-09-14T23:57:07.729 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.729 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:07.729 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.729 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.729 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.729 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T23:57:07.729 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:34.406877+0000", 2024-09-14T23:57:07.729 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 33.299719644, 2024-09-14T23:57:07.729 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030351169999999999, 2024-09-14T23:57:07.729 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:07.730 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:07.730 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:07.730 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:07.730 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:07.730 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:07.730 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 10 2024-09-14T23:57:07.730 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.730 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:07.730 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.730 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:07.730 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406877+0000", 2024-09-14T23:57:07.730 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.730 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.730 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.731 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:07.731 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406877+0000", 2024-09-14T23:57:07.731 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.731 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.731 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.731 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:07.731 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406878+0000", 2024-09-14T23:57:07.731 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.291e-06 2024-09-14T23:57:07.731 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.731 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.731 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:07.731 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406883+0000", 2024-09-14T23:57:07.731 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2399999999999998e-06 2024-09-14T23:57:07.731 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.731 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.731 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:07.732 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406891+0000", 2024-09-14T23:57:07.732 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5889999999999997e-06 2024-09-14T23:57:07.732 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.732 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.732 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:07.732 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406898+0000", 2024-09-14T23:57:07.732 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3599999999999998e-06 2024-09-14T23:57:07.732 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.732 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.732 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:07.732 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406948+0000", 2024-09-14T23:57:07.732 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9435999999999999e-05 2024-09-14T23:57:07.732 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.732 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.733 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-14T23:57:07.733 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406953+0000", 2024-09-14T23:57:07.733 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2809999999999999e-06 2024-09-14T23:57:07.733 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.733 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.733 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:07.733 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.408559+0000", 2024-09-14T23:57:07.733 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00160607 2024-09-14T23:57:07.733 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.733 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.733 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:07.733 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.408637+0000", 2024-09-14T23:57:07.733 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8504000000000006e-05 2024-09-14T23:57:07.733 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.734 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.734 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 4", 2024-09-14T23:57:07.734 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.408781+0000", 2024-09-14T23:57:07.734 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014374799999999999 2024-09-14T23:57:07.734 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.734 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.734 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:57:07.734 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.409659+0000", 2024-09-14T23:57:07.734 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000877781 2024-09-14T23:57:07.734 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.734 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.734 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:07.734 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.409822+0000", 2024-09-14T23:57:07.734 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016295400000000001 2024-09-14T23:57:07.734 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.735 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.735 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:07.735 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.409872+0000", 2024-09-14T23:57:07.735 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0022000000000003e-05 2024-09-14T23:57:07.735 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.735 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.735 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:07.735 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.409912+0000", 2024-09-14T23:57:07.735 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9841e-05 2024-09-14T23:57:07.735 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.735 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:07.735 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.735 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.735 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.736 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "MOSDPGPush(3.c 48/47 [PushOp(3:34183e86:::rbd_data.111888d8b54.0000000000000003:head, version: 37'112, data_included: [0~65536,131072~65536,262144~131072,458752~65536,553790~16537,589824~65536,700433~17543,720896~65536,851968~65536,983040~490208], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:34183e86:::rbd_data.111888d8b54.0000000000000003:head@37'112, size: 9745466, copy_subset: [(0, 9745466)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1473248, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T23:57:07.736 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:58.328790+0000", 2024-09-14T23:57:07.736 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.3778058410000007, 2024-09-14T23:57:07.736 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0024861290000000001, 2024-09-14T23:57:07.736 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:07.736 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:07.736 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:07.736 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:07.736 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.736 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:07.736 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.328790+0000", 2024-09-14T23:57:07.737 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.737 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.737 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:07.737 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.328790+0000", 2024-09-14T23:57:07.737 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.737 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.737 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:07.737 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.328791+0000", 2024-09-14T23:57:07.737 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3900000000000004e-07 2024-09-14T23:57:07.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.737 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.737 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:07.737 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.329762+0000", 2024-09-14T23:57:07.738 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000970362 2024-09-14T23:57:07.738 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.738 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.738 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:07.738 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.329779+0000", 2024-09-14T23:57:07.738 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7655999999999998e-05 2024-09-14T23:57:07.738 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.738 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.738 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:07.738 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.329791+0000", 2024-09-14T23:57:07.738 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2131e-05 2024-09-14T23:57:07.738 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.738 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.738 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:07.739 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.329886+0000", 2024-09-14T23:57:07.739 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4439999999999997e-05 2024-09-14T23:57:07.739 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.739 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.739 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:07.739 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.329902+0000", 2024-09-14T23:57:07.739 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6651e-05 2024-09-14T23:57:07.739 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.739 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.739 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:07.739 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.331277+0000", 2024-09-14T23:57:07.739 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00137415 2024-09-14T23:57:07.739 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.739 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:07.739 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.740 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.740 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.740 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "MOSDPGPush(3.c 48/47 [PushOp(3:34183e86:::rbd_data.111888d8b54.0000000000000003:head, version: 37'112, data_included: [1473248~99616,1638400~65536,1835008~262144,2162688~65536,2293760~262144,2686976~65536,9737423~8043], data_size: 828555, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:34183e86:::rbd_data.111888d8b54.0000000000000003:head@37'112, size: 9745466, copy_subset: [(0, 9745466)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 9745466, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1473248, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-14T23:57:07.740 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:58.332069+0000", 2024-09-14T23:57:07.740 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.3745270410000003, 2024-09-14T23:57:07.740 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0025177979999999999, 2024-09-14T23:57:07.740 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:07.740 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:07.740 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:07.740 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:07.740 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.740 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:07.740 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.332069+0000", 2024-09-14T23:57:07.740 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.740 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.741 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.741 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:07.741 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.332069+0000", 2024-09-14T23:57:07.741 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.741 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.741 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:07.741 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.332070+0000", 2024-09-14T23:57:07.741 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1200000000000003e-07 2024-09-14T23:57:07.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.741 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.741 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:07.741 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.332822+0000", 2024-09-14T23:57:07.741 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00075239299999999996 2024-09-14T23:57:07.742 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.742 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.742 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:07.742 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.332845+0000", 2024-09-14T23:57:07.742 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2436999999999999e-05 2024-09-14T23:57:07.742 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.742 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.742 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:07.742 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.332866+0000", 2024-09-14T23:57:07.742 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0874e-05 2024-09-14T23:57:07.742 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.742 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.742 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:07.743 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.332970+0000", 2024-09-14T23:57:07.743 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000104528 2024-09-14T23:57:07.743 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.743 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.743 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:07.743 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.332987+0000", 2024-09-14T23:57:07.743 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7073000000000001e-05 2024-09-14T23:57:07.743 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.743 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.743 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:07.743 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.334587+0000", 2024-09-14T23:57:07.743 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001599881 2024-09-14T23:57:07.743 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.743 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:07.743 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.744 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.744 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.744 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:50626 3.d e51/46 3:b5154cfe:::rbd_data.11183b9aac34.0000000000000004:head v 51'63, mlcod=45'61)", 2024-09-14T23:57:07.744 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:01.140129+0000", 2024-09-14T23:57:07.744 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 6.5664670440000004, 2024-09-14T23:57:07.744 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0029883850000000001, 2024-09-14T23:57:07.744 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:07.744 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:07.744 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:07.744 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:07.744 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.744 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:07.744 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.140129+0000", 2024-09-14T23:57:07.744 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.744 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.745 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.745 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:07.745 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.140129+0000", 2024-09-14T23:57:07.745 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.745 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.745 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:07.745 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.140130+0000", 2024-09-14T23:57:07.745 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0529999999999999e-06 2024-09-14T23:57:07.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.745 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.745 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:07.745 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.140311+0000", 2024-09-14T23:57:07.746 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018054200000000001 2024-09-14T23:57:07.746 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.746 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.746 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:07.746 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.140325+0000", 2024-09-14T23:57:07.746 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.382e-05 2024-09-14T23:57:07.746 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.746 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.746 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:07.746 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.140332+0000", 2024-09-14T23:57:07.746 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5700000000000004e-06 2024-09-14T23:57:07.746 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.746 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.747 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:07.747 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.140392+0000", 2024-09-14T23:57:07.747 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9296000000000003e-05 2024-09-14T23:57:07.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.747 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.747 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:07.747 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.140436+0000", 2024-09-14T23:57:07.747 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4515999999999999e-05 2024-09-14T23:57:07.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.747 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.747 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:07.747 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.143075+0000", 2024-09-14T23:57:07.747 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0026385369999999998 2024-09-14T23:57:07.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.747 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.748 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:07.748 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.143118+0000", 2024-09-14T23:57:07.748 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3050999999999998e-05 2024-09-14T23:57:07.748 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.748 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:07.748 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.748 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.748 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.748 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:51277 3.d e52/46 3:b2645877:::rbd_data.11181975a1de.0000000000000001:head v 52'69, mlcod=51'66)", 2024-09-14T23:57:07.748 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:02.157938+0000", 2024-09-14T23:57:07.748 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.5486580459999999, 2024-09-14T23:57:07.748 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0027517549999999998, 2024-09-14T23:57:07.749 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:07.749 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:07.749 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:07.749 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:07.749 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.749 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:07.749 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.157938+0000", 2024-09-14T23:57:07.749 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.749 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.749 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.749 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:07.749 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.157938+0000", 2024-09-14T23:57:07.749 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.750 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.750 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:07.750 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.157939+0000", 2024-09-14T23:57:07.750 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9199999999999999e-07 2024-09-14T23:57:07.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.750 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.750 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:07.750 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158543+0000", 2024-09-14T23:57:07.750 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00060333299999999995 2024-09-14T23:57:07.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.750 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.750 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:07.750 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158575+0000", 2024-09-14T23:57:07.750 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2801e-05 2024-09-14T23:57:07.751 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.751 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.751 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:07.751 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158586+0000", 2024-09-14T23:57:07.751 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.102e-05 2024-09-14T23:57:07.751 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.751 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.751 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:07.751 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158643+0000", 2024-09-14T23:57:07.751 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6320999999999998e-05 2024-09-14T23:57:07.751 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.751 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.751 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:07.751 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158693+0000", 2024-09-14T23:57:07.752 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0037000000000002e-05 2024-09-14T23:57:07.752 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.752 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.752 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:07.752 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.160641+0000", 2024-09-14T23:57:07.752 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0019485629999999999 2024-09-14T23:57:07.752 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.752 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.752 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:07.752 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.160690+0000", 2024-09-14T23:57:07.752 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8687999999999999e-05 2024-09-14T23:57:07.752 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.752 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:07.752 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.752 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.753 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.753 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:51275 3.2 e52/49 3:47a71653:::rbd_data.11181975a1de.0000000000000000:head v 52'118, mlcod=52'116)", 2024-09-14T23:57:07.753 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:02.158044+0000", 2024-09-14T23:57:07.753 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.5485524740000001, 2024-09-14T23:57:07.753 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0028337200000000001, 2024-09-14T23:57:07.753 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:07.753 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:07.753 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:07.753 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:07.753 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.753 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:07.753 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158044+0000", 2024-09-14T23:57:07.753 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.753 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.753 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.754 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:07.754 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158044+0000", 2024-09-14T23:57:07.754 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.754 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.754 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:07.754 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158045+0000", 2024-09-14T23:57:07.754 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5700000000000002e-07 2024-09-14T23:57:07.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.754 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.754 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:07.754 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158637+0000", 2024-09-14T23:57:07.754 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00059278499999999997 2024-09-14T23:57:07.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.755 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:07.755 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158657+0000", 2024-09-14T23:57:07.755 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9422000000000001e-05 2024-09-14T23:57:07.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.755 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:07.755 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158675+0000", 2024-09-14T23:57:07.755 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8564999999999999e-05 2024-09-14T23:57:07.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.755 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:07.755 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158735+0000", 2024-09-14T23:57:07.755 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9904000000000001e-05 2024-09-14T23:57:07.756 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.756 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.756 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:07.756 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158757+0000", 2024-09-14T23:57:07.756 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1277e-05 2024-09-14T23:57:07.756 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.756 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.756 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:07.756 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.160845+0000", 2024-09-14T23:57:07.756 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0020881419999999999 2024-09-14T23:57:07.756 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.756 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.756 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:07.756 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.160878+0000", 2024-09-14T23:57:07.757 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2867999999999998e-05 2024-09-14T23:57:07.757 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.757 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:07.757 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.757 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.757 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.757 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:51279 3.1f e52/49 3:fd03f784:::rbd_data.11181975a1de.0000000000000002:head v 52'152, mlcod=51'149)", 2024-09-14T23:57:07.757 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:02.158683+0000", 2024-09-14T23:57:07.757 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.5479135020000001, 2024-09-14T23:57:07.757 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0022939549999999999, 2024-09-14T23:57:07.757 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:07.757 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:07.757 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:07.757 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:07.757 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.758 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:07.758 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158683+0000", 2024-09-14T23:57:07.758 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.758 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.758 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.758 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:07.758 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158683+0000", 2024-09-14T23:57:07.758 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.758 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.758 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.758 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:07.758 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158684+0000", 2024-09-14T23:57:07.758 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9800000000000002e-07 2024-09-14T23:57:07.758 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.759 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.759 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:07.759 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.159362+0000", 2024-09-14T23:57:07.759 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00067790199999999998 2024-09-14T23:57:07.759 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.759 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.759 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:07.759 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.159372+0000", 2024-09-14T23:57:07.759 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0869999999999999e-05 2024-09-14T23:57:07.759 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.759 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.759 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:07.759 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.159381+0000", 2024-09-14T23:57:07.759 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4970000000000001e-06 2024-09-14T23:57:07.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.760 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.760 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:07.760 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.159440+0000", 2024-09-14T23:57:07.760 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9268000000000002e-05 2024-09-14T23:57:07.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.760 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.760 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:07.760 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.159462+0000", 2024-09-14T23:57:07.760 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1776e-05 2024-09-14T23:57:07.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.760 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.760 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:07.760 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.160943+0000", 2024-09-14T23:57:07.760 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0014812569999999999 2024-09-14T23:57:07.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.761 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.761 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:07.761 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.160977+0000", 2024-09-14T23:57:07.761 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3487e-05 2024-09-14T23:57:07.761 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.761 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:07.761 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.761 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.761 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.761 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:51580 3.2 e52/49 3:43825c2f:::rbd_data.1118c507b169.0000000000000000:head v 52'168, mlcod=52'165)", 2024-09-14T23:57:07.761 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:02.384221+0000", 2024-09-14T23:57:07.761 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.3223754090000002, 2024-09-14T23:57:07.761 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0022607640000000002, 2024-09-14T23:57:07.761 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:07.761 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:07.762 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:07.762 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:07.762 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.762 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:07.762 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.384221+0000", 2024-09-14T23:57:07.762 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.762 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.762 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:07.762 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.384221+0000", 2024-09-14T23:57:07.762 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.762 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.762 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:07.762 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.384221+0000", 2024-09-14T23:57:07.763 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2199999999999999e-07 2024-09-14T23:57:07.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.763 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:07.763 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.385059+0000", 2024-09-14T23:57:07.763 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00083741800000000004 2024-09-14T23:57:07.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.763 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:07.763 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.385069+0000", 2024-09-14T23:57:07.763 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9799999999999993e-06 2024-09-14T23:57:07.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.763 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:07.763 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.385073+0000", 2024-09-14T23:57:07.764 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7620000000000003e-06 2024-09-14T23:57:07.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.764 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.764 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:07.764 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.385124+0000", 2024-09-14T23:57:07.764 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0591999999999999e-05 2024-09-14T23:57:07.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.764 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.764 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:07.764 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.385143+0000", 2024-09-14T23:57:07.764 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9216999999999999e-05 2024-09-14T23:57:07.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.764 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.764 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:07.765 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.386457+0000", 2024-09-14T23:57:07.765 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001313577 2024-09-14T23:57:07.765 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.765 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.765 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:07.765 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.386482+0000", 2024-09-14T23:57:07.765 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4796000000000001e-05 2024-09-14T23:57:07.765 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.765 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:07.765 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.765 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.765 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.765 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56383 3.c e56/49 3:36b698b0:::rbd_data.1118c4b3aebe.0000000000000000:head v 56'194, mlcod=56'192)", 2024-09-14T23:57:07.765 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.270483+0000", 2024-09-14T23:57:07.766 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 1.4361134899999999, 2024-09-14T23:57:07.766 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0079685880000000004, 2024-09-14T23:57:07.766 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:07.766 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:07.766 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:07.766 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:07.766 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.766 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:07.766 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.270483+0000", 2024-09-14T23:57:07.766 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.766 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.766 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.766 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:07.766 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.270483+0000", 2024-09-14T23:57:07.767 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.767 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.767 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:07.767 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.270484+0000", 2024-09-14T23:57:07.767 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7400000000000002e-07 2024-09-14T23:57:07.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.767 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.767 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:07.767 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273680+0000", 2024-09-14T23:57:07.767 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031966680000000002 2024-09-14T23:57:07.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.767 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.767 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:07.768 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273697+0000", 2024-09-14T23:57:07.768 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6595e-05 2024-09-14T23:57:07.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.768 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.768 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:07.768 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273705+0000", 2024-09-14T23:57:07.768 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7530000000000001e-06 2024-09-14T23:57:07.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.768 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.768 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:07.768 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273763+0000", 2024-09-14T23:57:07.768 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8238e-05 2024-09-14T23:57:07.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.769 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.769 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:07.769 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273781+0000", 2024-09-14T23:57:07.769 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8281e-05 2024-09-14T23:57:07.769 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.769 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.769 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:07.769 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.278408+0000", 2024-09-14T23:57:07.769 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00462716 2024-09-14T23:57:07.769 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.769 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.769 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:07.769 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.278451+0000", 2024-09-14T23:57:07.769 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3019000000000003e-05 2024-09-14T23:57:07.770 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.770 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:07.770 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.770 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.770 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.770 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56384 3.c e56/49 3:36b698b0:::rbd_data.1118c4b3aebe.0000000000000000:head v 56'196, mlcod=56'192)", 2024-09-14T23:57:07.770 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:57:07.770 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 1.432080818, 2024-09-14T23:57:07.770 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0046707529999999997, 2024-09-14T23:57:07.770 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:07.770 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:07.770 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:07.770 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:07.770 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.771 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:07.771 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:57:07.771 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.771 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.771 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.771 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:07.771 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:57:07.771 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.771 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.771 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.771 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:07.771 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:57:07.771 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1699999999999997e-07 2024-09-14T23:57:07.771 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.772 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.772 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:07.772 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274523+0000", 2024-09-14T23:57:07.772 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9129999999999997e-06 2024-09-14T23:57:07.772 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.772 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.772 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:07.772 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274535+0000", 2024-09-14T23:57:07.772 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1788e-05 2024-09-14T23:57:07.772 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.772 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.772 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:07.772 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274542+0000", 2024-09-14T23:57:07.772 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0539999999999997e-06 2024-09-14T23:57:07.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.773 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.773 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:07.773 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275889+0000", 2024-09-14T23:57:07.773 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00134669 2024-09-14T23:57:07.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.773 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.773 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:07.773 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275917+0000", 2024-09-14T23:57:07.773 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7664000000000001e-05 2024-09-14T23:57:07.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.773 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.773 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:07.773 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279134+0000", 2024-09-14T23:57:07.774 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0032178459999999999 2024-09-14T23:57:07.774 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.774 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.774 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:07.774 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279186+0000", 2024-09-14T23:57:07.774 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1881e-05 2024-09-14T23:57:07.774 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.774 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:07.774 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.774 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.775 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.775 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56381 3.1d e56/49 3:bcf9a8cf:::rbd_data.1118c4b3aebe.0000000000000006:head v 56'210, mlcod=56'208)", 2024-09-14T23:57:07.775 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.275462+0000", 2024-09-14T23:57:07.775 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 1.4311344880000001, 2024-09-14T23:57:07.775 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0071310710000000001, 2024-09-14T23:57:07.775 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:07.775 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:07.775 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:07.775 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:07.775 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.775 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:07.775 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275462+0000", 2024-09-14T23:57:07.775 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.775 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.776 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.776 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:07.776 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275462+0000", 2024-09-14T23:57:07.776 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.776 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.776 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:07.776 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275463+0000", 2024-09-14T23:57:07.776 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2699999999999999e-07 2024-09-14T23:57:07.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.776 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.776 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:07.776 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276779+0000", 2024-09-14T23:57:07.777 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0013163649999999999 2024-09-14T23:57:07.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.777 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.777 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:07.777 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276795+0000", 2024-09-14T23:57:07.777 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.59e-05 2024-09-14T23:57:07.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.777 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.777 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:07.777 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276806+0000", 2024-09-14T23:57:07.777 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1106e-05 2024-09-14T23:57:07.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.777 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.777 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:07.778 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276941+0000", 2024-09-14T23:57:07.778 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013477500000000001 2024-09-14T23:57:07.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.778 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.778 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:07.778 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276963+0000", 2024-09-14T23:57:07.778 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2419999999999999e-05 2024-09-14T23:57:07.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.778 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.778 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:07.778 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282546+0000", 2024-09-14T23:57:07.778 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0055827100000000003 2024-09-14T23:57:07.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.778 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.779 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:07.779 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282593+0000", 2024-09-14T23:57:07.779 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7067999999999997e-05 2024-09-14T23:57:07.779 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.779 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:07.779 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.779 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.779 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.779 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56387 3.c e56/49 3:30ee4ba0:::rbd_data.1118c4b3aebe.0000000000000004:head v 56'197, mlcod=56'192)", 2024-09-14T23:57:07.779 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.276498+0000", 2024-09-14T23:57:07.779 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 1.430098393, 2024-09-14T23:57:07.779 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0092192639999999996, 2024-09-14T23:57:07.779 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:07.780 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:07.780 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:07.780 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:07.780 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.780 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:07.780 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276498+0000", 2024-09-14T23:57:07.780 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.780 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.780 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.780 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:07.780 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276498+0000", 2024-09-14T23:57:07.780 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.780 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.780 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.781 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:07.781 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276499+0000", 2024-09-14T23:57:07.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1100000000000004e-07 2024-09-14T23:57:07.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.781 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:07.781 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279646+0000", 2024-09-14T23:57:07.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031473220000000001 2024-09-14T23:57:07.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.781 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:07.781 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279672+0000", 2024-09-14T23:57:07.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6129999999999999e-05 2024-09-14T23:57:07.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.782 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:07.782 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279682+0000", 2024-09-14T23:57:07.782 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7170000000000003e-06 2024-09-14T23:57:07.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.782 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:07.782 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279742+0000", 2024-09-14T23:57:07.782 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9926999999999997e-05 2024-09-14T23:57:07.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.782 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:07.782 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279761+0000", 2024-09-14T23:57:07.782 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9402000000000002e-05 2024-09-14T23:57:07.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.783 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.783 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:07.783 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.285680+0000", 2024-09-14T23:57:07.783 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.005918881 2024-09-14T23:57:07.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.783 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.783 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:07.783 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.285717+0000", 2024-09-14T23:57:07.783 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6974000000000002e-05 2024-09-14T23:57:07.783 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.783 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:07.783 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.784 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.784 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.784 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56388 3.c e56/49 3:30ee4ba0:::rbd_data.1118c4b3aebe.0000000000000004:head v 56'199, mlcod=56'192)", 2024-09-14T23:57:07.784 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:57:07.784 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 1.426606805, 2024-09-14T23:57:07.784 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0060596740000000001, 2024-09-14T23:57:07.784 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:07.784 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:07.784 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:07.784 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:07.784 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.784 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:07.784 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:57:07.784 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.785 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.785 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.785 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:07.785 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:57:07.785 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.785 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.785 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.785 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:07.785 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:57:07.785 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2200000000000003e-07 2024-09-14T23:57:07.785 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:07.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279995+0000", 2024-09-14T23:57:07.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5569999999999997e-06 2024-09-14T23:57:07.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:07.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280004+0000", 2024-09-14T23:57:07.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0080000000000008e-06 2024-09-14T23:57:07.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:07.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280010+0000", 2024-09-14T23:57:07.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5749999999999997e-06 2024-09-14T23:57:07.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.787 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:07.787 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282968+0000", 2024-09-14T23:57:07.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0029579390000000001 2024-09-14T23:57:07.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.787 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:07.787 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282998+0000", 2024-09-14T23:57:07.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9947999999999999e-05 2024-09-14T23:57:07.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.787 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:07.788 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.286012+0000", 2024-09-14T23:57:07.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030132900000000001 2024-09-14T23:57:07.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.788 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:07.788 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.286049+0000", 2024-09-14T23:57:07.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7534999999999999e-05 2024-09-14T23:57:07.788 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.788 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:07.788 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.788 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56382 3.1d e56/49 3:bcf9a8cf:::rbd_data.1118c4b3aebe.0000000000000006:head v 56'212, mlcod=56'208)", 2024-09-14T23:57:07.788 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.280285+0000", 2024-09-14T23:57:07.789 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 1.426311122, 2024-09-14T23:57:07.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035393489999999998, 2024-09-14T23:57:07.789 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:07.789 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:07.789 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:07.789 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:07.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.789 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:07.789 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280285+0000", 2024-09-14T23:57:07.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.789 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:07.789 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280285+0000", 2024-09-14T23:57:07.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:07.790 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280286+0000", 2024-09-14T23:57:07.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5000000000000001e-07 2024-09-14T23:57:07.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:07.790 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280299+0000", 2024-09-14T23:57:07.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3312000000000001e-05 2024-09-14T23:57:07.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:07.791 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280311+0000", 2024-09-14T23:57:07.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1677e-05 2024-09-14T23:57:07.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.791 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:07.791 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280318+0000", 2024-09-14T23:57:07.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.742e-06 2024-09-14T23:57:07.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.791 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:07.791 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280374+0000", 2024-09-14T23:57:07.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.592e-05 2024-09-14T23:57:07.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.792 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:07.792 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280392+0000", 2024-09-14T23:57:07.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8490000000000001e-05 2024-09-14T23:57:07.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.792 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:07.792 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.283693+0000", 2024-09-14T23:57:07.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033012050000000002 2024-09-14T23:57:07.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.792 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:07.792 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.283825+0000", 2024-09-14T23:57:07.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013105300000000001 2024-09-14T23:57:07.792 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.793 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:07.793 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.793 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "pg_update_log_missing(3.c epoch 56/49 rep_tid 1493 entries 56'200 (0'0) error 3:370203a3:::rbd_data.1118c4b3aebe.000000000000000d:head by client.4376.0:56396 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 56'192)", 2024-09-14T23:57:07.793 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:57:07.793 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 1.4260866649999999, 2024-09-14T23:57:07.793 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057370099999999999, 2024-09-14T23:57:07.793 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:07.793 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:07.793 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:57:07.793 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:07.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.794 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:07.794 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:57:07.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.794 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:07.794 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:57:07.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.794 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:07.794 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:57:07.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0699999999999997e-07 2024-09-14T23:57:07.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:07.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280519+0000", 2024-09-14T23:57:07.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6260000000000001e-06 2024-09-14T23:57:07.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:07.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280541+0000", 2024-09-14T23:57:07.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1976000000000001e-05 2024-09-14T23:57:07.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:07.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280556+0000", 2024-09-14T23:57:07.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5075000000000001e-05 2024-09-14T23:57:07.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.796 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:07.796 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.285494+0000", 2024-09-14T23:57:07.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0049378690000000001 2024-09-14T23:57:07.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.796 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:07.796 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.286247+0000", 2024-09-14T23:57:07.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00075285700000000001 2024-09-14T23:57:07.796 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.796 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:07.796 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.796 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56400 3.1c e56/49 3:3f5b81ba:::rbd_data.1118c4b3aebe.000000000000000a:head v 56'236, mlcod=56'234)", 2024-09-14T23:57:07.797 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.295064+0000", 2024-09-14T23:57:07.797 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 1.4115318729999999, 2024-09-14T23:57:07.797 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0054331589999999999, 2024-09-14T23:57:07.797 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:07.797 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:07.797 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:07.797 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:07.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.797 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:07.797 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.295064+0000", 2024-09-14T23:57:07.797 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.797 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:07.798 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.295064+0000", 2024-09-14T23:57:07.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.798 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:07.798 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.295066+0000", 2024-09-14T23:57:07.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.502e-06 2024-09-14T23:57:07.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.798 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:07.798 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296403+0000", 2024-09-14T23:57:07.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001336554 2024-09-14T23:57:07.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.799 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:07.799 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296428+0000", 2024-09-14T23:57:07.799 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5179000000000001e-05 2024-09-14T23:57:07.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.799 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.799 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:07.799 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296440+0000", 2024-09-14T23:57:07.799 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2387e-05 2024-09-14T23:57:07.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.799 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.799 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:07.799 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296554+0000", 2024-09-14T23:57:07.799 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114269 2024-09-14T23:57:07.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:07.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296581+0000", 2024-09-14T23:57:07.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6673999999999999e-05 2024-09-14T23:57:07.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:07.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.300418+0000", 2024-09-14T23:57:07.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0038371249999999998 2024-09-14T23:57:07.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:07.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.300498+0000", 2024-09-14T23:57:07.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9468999999999994e-05 2024-09-14T23:57:07.801 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.801 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:07.801 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.801 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.801 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.801 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56401 3.1c e56/49 3:3f5b81ba:::rbd_data.1118c4b3aebe.000000000000000a:head v 56'238, mlcod=56'234)", 2024-09-14T23:57:07.801 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.297640+0000", 2024-09-14T23:57:07.801 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 1.408956034, 2024-09-14T23:57:07.801 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031491919999999999, 2024-09-14T23:57:07.801 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:07.801 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:07.801 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:07.801 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:07.801 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.801 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:07.802 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.297640+0000", 2024-09-14T23:57:07.802 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.802 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.802 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:07.802 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.297640+0000", 2024-09-14T23:57:07.802 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:07.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.802 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.803 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:07.803 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.297641+0000", 2024-09-14T23:57:07.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9000000000000005e-07 2024-09-14T23:57:07.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.803 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:07.803 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.297649+0000", 2024-09-14T23:57:07.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4510000000000003e-06 2024-09-14T23:57:07.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.803 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:07.803 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.297663+0000", 2024-09-14T23:57:07.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4262e-05 2024-09-14T23:57:07.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:07.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.297679+0000", 2024-09-14T23:57:07.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5798e-05 2024-09-14T23:57:07.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:07.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.298408+0000", 2024-09-14T23:57:07.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00072923699999999998 2024-09-14T23:57:07.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:07.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.298430+0000", 2024-09-14T23:57:07.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1753999999999999e-05 2024-09-14T23:57:07.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.805 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.805 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:07.805 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.300749+0000", 2024-09-14T23:57:07.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023190590000000001 2024-09-14T23:57:07.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:07.805 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:07.805 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:07.805 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.300789+0000", 2024-09-14T23:57:07.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0640999999999999e-05 2024-09-14T23:57:07.805 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.805 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:07.805 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.806 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:07.806 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:07.806 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:07.806 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T23:57:07.808 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T23:57:07.809 INFO:teuthology.orchestra.run.smithi042.stdout:1867 trunc from 0xd84a14f to 0x29b74de 2024-09-14T23:57:07.809 INFO:teuthology.orchestra.run.smithi042.stdout:1868 write 0x18f0187 thru 0x18f7398 (0x7212 bytes) 2024-09-14T23:57:07.809 INFO:teuthology.orchestra.run.smithi042.stdout:1869 write 0xd8c99f0 thru 0xd8d53d0 (0xb9e1 bytes) 2024-09-14T23:57:07.809 INFO:teuthology.orchestra.run.smithi042.stdout:1871 punch from 0xb0eafa3 to 0xb0f7c03, (0xcc60 bytes) 2024-09-14T23:57:07.809 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:07.810 INFO:teuthology.orchestra.run.smithi042.stdout:1873 write 0x4d687fd thru 0x4d75b06 (0xd30a bytes) 2024-09-14T23:57:07.810 INFO:teuthology.orchestra.run.smithi042.stdout:1874 write 0x4bfc949 thru 0x4c00720 (0x3dd8 bytes) 2024-09-14T23:57:07.810 INFO:teuthology.orchestra.run.smithi042.stdout:1875 trunc from 0xd8d53d1 to 0x5b91471 2024-09-14T23:57:07.810 INFO:teuthology.orchestra.run.smithi042.stdout:1876 write 0xa807355 thru 0xa809628 (0x22d4 bytes) 2024-09-14T23:57:07.810 INFO:teuthology.orchestra.run.smithi042.stdout:1877 write 0xbdb41ec thru 0xbdba385 (0x619a bytes) 2024-09-14T23:57:07.810 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:07.806+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:07.814 INFO:teuthology.orchestra.run.smithi042.stdout:1879 punch from 0x899bb0d to 0x89a8339, (0xc82c bytes) 2024-09-14T23:57:07.814 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:07.815 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:07.815 INFO:teuthology.orchestra.run.smithi042.stdout:1882 write 0x61e161f thru 0x61f0a91 (0xf473 bytes) 2024-09-14T23:57:07.824 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:07.824 INFO:teuthology.orchestra.run.smithi042.stdout:1885 write 0x4585c31 thru 0x4587094 (0x1464 bytes) 2024-09-14T23:57:07.829 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:07.829 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:07.838 INFO:teuthology.orchestra.run.smithi042.stdout:1888 writesame 0xebbb708 thru 0xebc7e79 data_size 0x31(0xc772 bytes) 2024-09-14T23:57:07.838 INFO:teuthology.orchestra.run.smithi042.stdout:1889 write 0x234248e thru 0x235023c (0xddaf bytes) 2024-09-14T23:57:07.849 INFO:teuthology.orchestra.run.smithi042.stdout:1891 read 0xdb47e17 thru 0xdb47f14 (0xfe bytes) 2024-09-14T23:57:07.849 INFO:teuthology.orchestra.run.smithi042.stdout:1892 write 0x5d7a4b1 thru 0x5d859de (0xb52e bytes) 2024-09-14T23:57:07.858 INFO:teuthology.orchestra.run.smithi042.stdout:1893 write 0x5d394a thru 0x5d5f9d (0x2654 bytes) 2024-09-14T23:57:07.874 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:07.874 INFO:teuthology.orchestra.run.smithi042.stdout:1895 write 0x7860ecc thru 0x786fcf8 (0xee2d bytes) 2024-09-14T23:57:07.888 INFO:teuthology.orchestra.run.smithi042.stdout:1896 trunc from 0xebc7e7a to 0xa670565 2024-09-14T23:57:07.899 INFO:teuthology.orchestra.run.smithi042.stdout:1897 read 0x395c779 thru 0x395d222 (0xaaa bytes) 2024-09-14T23:57:07.899 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:07.899 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:07.900 INFO:teuthology.orchestra.run.smithi042.stdout:1901 trunc from 0xa670565 to 0xb554316 2024-09-14T23:57:07.905 INFO:teuthology.orchestra.run.smithi042.stdout:1902 read 0x324f50f thru 0x324fbe1 (0x6d3 bytes) 2024-09-14T23:57:07.905 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:07.906 INFO:teuthology.orchestra.run.smithi042.stdout:1906 punch from 0x8a8225d to 0x8a8cd1f, (0xaac2 bytes) 2024-09-14T23:57:07.906 INFO:teuthology.orchestra.run.smithi042.stdout:1907 read 0xb2a890c thru 0xb2b8403 (0xfaf8 bytes) 2024-09-14T23:57:07.908 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:07.907+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:07.912 INFO:teuthology.orchestra.run.smithi042.stdout:1909 write 0xd3962f6 thru 0xd3965b3 (0x2be bytes) 2024-09-14T23:57:07.920 INFO:teuthology.orchestra.run.smithi042.stdout:1910 write 0xd3cc30f thru 0xd3d92c9 (0xcfbb bytes) 2024-09-14T23:57:07.929 INFO:teuthology.orchestra.run.smithi042.stdout:1911 punch from 0xa0cf5c6 to 0xa0dd9e9, (0xe423 bytes) 2024-09-14T23:57:07.929 INFO:teuthology.orchestra.run.smithi042.stdout:1912 write 0x6241fe1 thru 0x624c71b (0xa73b bytes) 2024-09-14T23:57:07.931 INFO:teuthology.orchestra.run.smithi042.stdout:1914 writesame 0xc86fb7b thru 0xc8746d2 data_size 0x4(0x4b58 bytes) 2024-09-14T23:57:07.939 INFO:teuthology.orchestra.run.smithi042.stdout:1915 write 0x419553b thru 0x41a3c75 (0xe73b bytes) 2024-09-14T23:57:07.945 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:07.946 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:07.946 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:57:07.946 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:07.946 INFO:teuthology.orchestra.run.smithi042.stdout:1917 read 0x436b96 thru 0x43fca8 (0x9113 bytes) 2024-09-14T23:57:07.947 INFO:teuthology.orchestra.run.smithi042.stdout:1919 write 0x93519c5 thru 0x936031a (0xe956 bytes) 2024-09-14T23:57:07.953 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:57:07.953 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:57:07.957 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T23:57:07.959 INFO:teuthology.orchestra.run.smithi042.stdout:1921 punch from 0x5293180 to 0x529fcbf, (0xcb3f bytes) 2024-09-14T23:57:07.959 INFO:teuthology.orchestra.run.smithi042.stdout:1922 write 0x16ff4e thru 0x174487 (0x453a bytes) 2024-09-14T23:57:07.986 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:57:07.986 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:57:08.008 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:57:08.008 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:57:08.018 INFO:teuthology.orchestra.run.smithi042.stdout:1923 trunc from 0xd3d92ca to 0x97b3a43 2024-09-14T23:57:08.032 INFO:teuthology.orchestra.run.smithi042.stdout:1924 write 0x7e0e22 thru 0x7ee10b (0xd2ea bytes) 2024-09-14T23:57:08.035 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:57:08.035 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:57:08.042 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:08.041+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:08.049 INFO:teuthology.orchestra.run.smithi042.stdout:1925 read 0x5c05c08 thru 0x5c0a77b (0x4b74 bytes) 2024-09-14T23:57:08.049 INFO:teuthology.orchestra.run.smithi042.stdout:1927 writesame 0x1a90990 thru 0x1a94d17 data_size 0x4(0x4388 bytes) 2024-09-14T23:57:08.052 INFO:teuthology.orchestra.run.smithi042.stdout:1928 write 0x93e1791 thru 0x93e9138 (0x79a8 bytes) 2024-09-14T23:57:08.055 INFO:teuthology.orchestra.run.smithi042.stdout:1929 punch from 0x16cae0f to 0x16dac32, (0xfe23 bytes) 2024-09-14T23:57:08.055 INFO:teuthology.orchestra.run.smithi042.stdout:1930 read 0x3420608 thru 0x3428451 (0x7e4a bytes) 2024-09-14T23:57:08.055 INFO:teuthology.orchestra.run.smithi042.stdout:1931 read 0x8d34e5b thru 0x8d3f5c7 (0xa76d bytes) 2024-09-14T23:57:08.056 INFO:teuthology.orchestra.run.smithi042.stdout:1932 trunc from 0x97b3a43 to 0x888c685 2024-09-14T23:57:08.061 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:57:08.061 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:57:08.067 INFO:teuthology.orchestra.run.smithi042.stdout:1936 write 0x39a2e05 thru 0x39a57e5 (0x29e1 bytes) 2024-09-14T23:57:08.072 INFO:teuthology.orchestra.run.smithi042.stdout:1937 punch from 0x1fef729 to 0x1ffd9e7, (0xe2be bytes) 2024-09-14T23:57:08.072 INFO:teuthology.orchestra.run.smithi042.stdout:1938 write 0x5b87a6a thru 0x5b8f6b0 (0x7c47 bytes) 2024-09-14T23:57:08.074 INFO:teuthology.orchestra.run.smithi042.stdout:1939 punch from 0x3029556 to 0x3037ac7, (0xe571 bytes) 2024-09-14T23:57:08.075 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:08.081 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:57:08.081 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:57:08.081 INFO:teuthology.orchestra.run.smithi042.stdout:1942 trunc from 0x888c685 to 0x2b12318 2024-09-14T23:57:08.087 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T23:57:08.087 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T23:57:08.087 INFO:tasks.thrashosds.thrasher:Reweighting osd 5 to 0.6474473156734907 2024-09-14T23:57:08.087 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 0.6474473156734907 2024-09-14T23:57:08.107 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:57:08.107 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:57:08.110 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:08.109+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:08.115 INFO:teuthology.orchestra.run.smithi042.stdout:1943 writesame 0x5d35b69 thru 0x5d3e5a0 data_size 0x4(0x8a38 bytes) 2024-09-14T23:57:08.115 INFO:teuthology.orchestra.run.smithi042.stdout:1945 punch from 0x151f7d0 to 0x15205f8, (0xe28 bytes) 2024-09-14T23:57:08.126 INFO:teuthology.orchestra.run.smithi042.stdout:1946 writesame 0x92add6d thru 0x92b7737 data_size 0x1(0x99cb bytes) 2024-09-14T23:57:08.126 INFO:teuthology.orchestra.run.smithi042.stdout:1948 punch from 0x5e5affe to 0x5e66a3b, (0xba3d bytes) 2024-09-14T23:57:08.126 INFO:teuthology.orchestra.run.smithi042.stdout:1949 read 0x5026a11 thru 0x5026c09 (0x1f9 bytes) 2024-09-14T23:57:08.126 INFO:teuthology.orchestra.run.smithi042.stdout:1950 read 0x2a7281c thru 0x2a7b7e2 (0x8fc7 bytes) 2024-09-14T23:57:08.126 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:57:08.126 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:57:08.127 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:08.127 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:08.130 INFO:teuthology.orchestra.run.smithi042.stdout:1951 trunc from 0x92b7738 to 0x601137c 2024-09-14T23:57:08.136 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T23:57:08.150 INFO:teuthology.orchestra.run.smithi042.stdout:1952 write 0xd865dd2 thru 0xd87486f (0xea9e bytes) 2024-09-14T23:57:08.158 INFO:teuthology.orchestra.run.smithi042.stdout:1954 clone 20 order 23 su 2097152 sc 15 2024-09-14T23:57:08.211 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:08.210+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:08.283 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:08.284 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:08.292 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T23:57:08.312 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:08.311+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:08.400 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:08.400 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:08.408 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T23:57:08.412 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:08.411+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:08.514 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:08.513+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:08.515 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:08.515 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:08.523 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T23:57:08.614 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:08.613+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:08.631 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:08.631 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:08.639 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T23:57:08.716 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:08.714+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:08.747 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:08.747 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:08.756 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T23:57:08.817 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:08.816+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:08.902 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:08.902 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:08.911 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T23:57:08.918 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:08.917+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:09.018 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:09.018 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:09.019 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:09.018+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:09.026 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T23:57:09.087 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.5 to 0.647447 (a5bf) 2024-09-14T23:57:09.101 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T23:57:09.120 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:09.118+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:09.143 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:09.143 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:09.153 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T23:57:09.165 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone19 from 0xd874870 (overlap 0x29b74de) to 0x0 2024-09-14T23:57:09.221 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:09.220+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:09.290 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:09.290 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:09.299 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T23:57:09.342 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:09.341+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:09.407 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:09.407 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:09.416 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T23:57:09.457 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:09.455+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:09.523 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:09.522+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:09.563 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:09.564 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:09.572 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T23:57:09.625 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:09.623+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:09.719 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:09.719 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:09.725 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:09.723+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:09.728 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T23:57:09.826 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:09.824+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:09.876 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:09.877 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:09.886 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T23:57:09.929 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:09.928+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:09.993 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:09.993 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:10.002 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T23:57:10.043 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:10.042+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:10.088 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is set 2024-09-14T23:57:10.128 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:10.126+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:10.141 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T23:57:10.156 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:10.156 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:10.165 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T23:57:10.249 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:10.248+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:10.302 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:57:10.303 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:57:10.328 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:10.328 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:10.330 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:10.329+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:10.337 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:57:10.338 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:57:10.339 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T23:57:10.354 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:57:10.354 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:57:10.371 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:57:10.371 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:57:10.387 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:57:10.388 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:57:10.404 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:57:10.405 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:57:10.421 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:57:10.422 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:57:10.431 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:10.429+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:10.438 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:57:10.438 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:57:10.457 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:10.457 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:10.457 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:57:10.457 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:10.466 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T23:57:10.532 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:10.531+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:10.573 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:10.573 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:10.573 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:57:10.574 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:57:10.574 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:10.582 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T23:57:10.633 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:10.631+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:10.732 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:10.732 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:57:10.732 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:57:10.732 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:57:10.732 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.732 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:3500 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T23:57:10.732 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:37.990193+0000", 2024-09-14T23:57:10.732 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 32.739098167999998, 2024-09-14T23:57:10.732 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99440750899999997, 2024-09-14T23:57:10.732 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:10.732 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:10.732 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:10.733 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:10.733 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:10.733 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:10.733 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3500 2024-09-14T23:57:10.733 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.733 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:10.733 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.733 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:10.733 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990193+0000", 2024-09-14T23:57:10.733 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:10.733 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.733 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.733 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:10.734 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990193+0000", 2024-09-14T23:57:10.734 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:10.734 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.734 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.734 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:10.734 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990194+0000", 2024-09-14T23:57:10.734 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5000000000000001e-07 2024-09-14T23:57:10.734 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.734 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.734 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:10.734 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990200+0000", 2024-09-14T23:57:10.735 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4389999999999997e-06 2024-09-14T23:57:10.735 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.735 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.735 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:10.735 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990215+0000", 2024-09-14T23:57:10.735 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4613999999999999e-05 2024-09-14T23:57:10.735 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.735 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.735 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:10.735 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990224+0000", 2024-09-14T23:57:10.735 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1740000000000006e-06 2024-09-14T23:57:10.735 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.735 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.736 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:10.736 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990283+0000", 2024-09-14T23:57:10.736 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9231999999999998e-05 2024-09-14T23:57:10.736 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.736 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.736 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:57:10.736 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990291+0000", 2024-09-14T23:57:10.736 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5900000000000002e-06 2024-09-14T23:57:10.736 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.736 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.736 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:10.736 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:38.984249+0000", 2024-09-14T23:57:10.736 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99395822700000003 2024-09-14T23:57:10.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.737 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.737 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:10.737 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:38.984448+0000", 2024-09-14T23:57:10.737 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00019855900000000001 2024-09-14T23:57:10.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.737 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.737 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:10.737 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:38.984601+0000", 2024-09-14T23:57:10.737 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000152824 2024-09-14T23:57:10.737 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:10.737 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:10.737 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:10.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.738 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.738 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4055 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1093632~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:10.738 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099066+0000", 2024-09-14T23:57:10.738 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 31.630225280000001, 2024-09-14T23:57:10.738 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044174782000000003, 2024-09-14T23:57:10.738 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:10.738 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:10.738 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:10.738 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:10.738 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:10.738 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:10.739 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4055 2024-09-14T23:57:10.739 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.739 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:10.739 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.739 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:10.739 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099066+0000", 2024-09-14T23:57:10.739 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:10.739 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.739 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.739 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:10.740 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099066+0000", 2024-09-14T23:57:10.740 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:10.740 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.740 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.740 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:10.740 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099066+0000", 2024-09-14T23:57:10.740 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3700000000000001e-07 2024-09-14T23:57:10.740 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.740 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.740 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:10.740 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099067+0000", 2024-09-14T23:57:10.740 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1379999999999999e-06 2024-09-14T23:57:10.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.741 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.741 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:10.741 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099073+0000", 2024-09-14T23:57:10.741 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4790000000000004e-06 2024-09-14T23:57:10.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.741 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.741 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:10.741 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099075+0000", 2024-09-14T23:57:10.741 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5320000000000001e-06 2024-09-14T23:57:10.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.741 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.741 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:10.742 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143147+0000", 2024-09-14T23:57:10.742 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044071676999999997 2024-09-14T23:57:10.742 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.742 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.742 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:10.742 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143186+0000", 2024-09-14T23:57:10.742 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8821e-05 2024-09-14T23:57:10.742 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.742 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.742 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:10.742 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143241+0000", 2024-09-14T23:57:10.742 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4797999999999999e-05 2024-09-14T23:57:10.742 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:10.742 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:10.743 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:10.743 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.743 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.743 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4056 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1155072~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:10.743 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099079+0000", 2024-09-14T23:57:10.743 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 31.630212565000001, 2024-09-14T23:57:10.743 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044295020999999997, 2024-09-14T23:57:10.743 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:10.743 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:10.743 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:10.743 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:10.743 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:10.743 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:10.743 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4056 2024-09-14T23:57:10.744 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.744 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:10.744 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.744 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:10.744 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099079+0000", 2024-09-14T23:57:10.744 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:10.744 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.744 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.744 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:10.744 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099079+0000", 2024-09-14T23:57:10.744 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:10.744 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.744 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.744 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:10.745 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099079+0000", 2024-09-14T23:57:10.745 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1500000000000001e-07 2024-09-14T23:57:10.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.745 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.745 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:10.745 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099080+0000", 2024-09-14T23:57:10.745 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0210000000000001e-06 2024-09-14T23:57:10.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.745 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.745 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:10.745 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099083+0000", 2024-09-14T23:57:10.745 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.174e-06 2024-09-14T23:57:10.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.746 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.746 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:10.746 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099086+0000", 2024-09-14T23:57:10.746 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.446e-06 2024-09-14T23:57:10.746 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.746 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.746 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:10.746 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143269+0000", 2024-09-14T23:57:10.746 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044183489999999999 2024-09-14T23:57:10.746 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.746 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.746 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:10.746 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143314+0000", 2024-09-14T23:57:10.747 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5135000000000001e-05 2024-09-14T23:57:10.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.747 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.747 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:10.747 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143374+0000", 2024-09-14T23:57:10.747 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9540000000000003e-05 2024-09-14T23:57:10.747 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:10.747 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:10.747 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:10.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.747 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.747 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4057 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1216512~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:10.747 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099088+0000", 2024-09-14T23:57:10.748 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 31.630202778000001, 2024-09-14T23:57:10.748 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044406119000000001, 2024-09-14T23:57:10.748 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:10.748 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:10.748 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:10.748 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:10.748 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:10.748 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:10.748 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4057 2024-09-14T23:57:10.748 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.748 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:10.748 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.748 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:10.749 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099088+0000", 2024-09-14T23:57:10.749 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:10.749 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.749 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.749 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:10.749 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099088+0000", 2024-09-14T23:57:10.749 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:10.749 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.749 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.749 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:10.749 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099089+0000", 2024-09-14T23:57:10.749 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.04e-07 2024-09-14T23:57:10.749 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.749 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.750 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:10.750 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099090+0000", 2024-09-14T23:57:10.750 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.026e-06 2024-09-14T23:57:10.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.750 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.750 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:10.750 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099093+0000", 2024-09-14T23:57:10.750 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0819999999999999e-06 2024-09-14T23:57:10.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.750 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.750 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:10.750 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099095+0000", 2024-09-14T23:57:10.750 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4930000000000001e-06 2024-09-14T23:57:10.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.751 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.751 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:10.751 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143400+0000", 2024-09-14T23:57:10.751 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044304698000000003 2024-09-14T23:57:10.751 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.751 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.751 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:10.751 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143441+0000", 2024-09-14T23:57:10.751 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1100000000000003e-05 2024-09-14T23:57:10.751 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.751 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.751 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:10.752 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143495+0000", 2024-09-14T23:57:10.752 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3516000000000001e-05 2024-09-14T23:57:10.752 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:10.752 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:10.752 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:10.752 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.752 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.752 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4058 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1277952~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:10.752 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099099+0000", 2024-09-14T23:57:10.752 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 31.630192287, 2024-09-14T23:57:10.752 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044514583000000003, 2024-09-14T23:57:10.752 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:10.752 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:10.752 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:10.753 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:10.753 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:10.753 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:10.753 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4058 2024-09-14T23:57:10.753 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.753 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:10.753 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.753 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:10.753 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099099+0000", 2024-09-14T23:57:10.753 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:10.753 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.753 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.753 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:10.753 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099099+0000", 2024-09-14T23:57:10.754 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:10.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.754 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.754 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:10.754 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099099+0000", 2024-09-14T23:57:10.754 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9999999999999999e-07 2024-09-14T23:57:10.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.754 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.754 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:10.754 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099100+0000", 2024-09-14T23:57:10.754 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7199999999999997e-07 2024-09-14T23:57:10.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.754 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.754 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:10.754 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099103+0000", 2024-09-14T23:57:10.755 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.095e-06 2024-09-14T23:57:10.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.755 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:10.755 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099106+0000", 2024-09-14T23:57:10.755 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4229999999999999e-06 2024-09-14T23:57:10.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.755 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:10.755 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143524+0000", 2024-09-14T23:57:10.755 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044417891000000001 2024-09-14T23:57:10.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.755 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:10.755 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143568+0000", 2024-09-14T23:57:10.756 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4060000000000002e-05 2024-09-14T23:57:10.756 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.756 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.756 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:10.756 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143614+0000", 2024-09-14T23:57:10.756 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5942000000000002e-05 2024-09-14T23:57:10.756 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:10.756 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:10.756 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:10.756 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.756 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.756 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4059 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1339392~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:10.756 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099114+0000", 2024-09-14T23:57:10.756 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 31.630177748000001, 2024-09-14T23:57:10.757 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044617194999999998, 2024-09-14T23:57:10.757 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:10.757 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:10.757 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:10.757 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:10.757 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:10.757 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:10.757 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4059 2024-09-14T23:57:10.757 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.758 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:10.758 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.758 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:10.758 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099114+0000", 2024-09-14T23:57:10.758 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:10.758 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.758 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.758 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:10.758 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099114+0000", 2024-09-14T23:57:10.758 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:10.758 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.758 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.758 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:10.758 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099114+0000", 2024-09-14T23:57:10.758 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5899999999999998e-07 2024-09-14T23:57:10.759 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.759 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.759 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:10.759 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099115+0000", 2024-09-14T23:57:10.759 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3510000000000001e-06 2024-09-14T23:57:10.759 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.759 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.759 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:10.759 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099119+0000", 2024-09-14T23:57:10.759 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.16e-06 2024-09-14T23:57:10.759 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.759 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.759 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:10.759 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099123+0000", 2024-09-14T23:57:10.760 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1339999999999997e-06 2024-09-14T23:57:10.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.760 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.760 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:10.760 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143643+0000", 2024-09-14T23:57:10.760 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044519232999999998 2024-09-14T23:57:10.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.760 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.760 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:10.760 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143686+0000", 2024-09-14T23:57:10.760 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3349000000000001e-05 2024-09-14T23:57:10.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.760 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.760 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:10.761 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143731+0000", 2024-09-14T23:57:10.761 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4709000000000002e-05 2024-09-14T23:57:10.761 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:10.761 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:10.761 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:10.761 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.761 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.761 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4060 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1400832~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:10.761 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099129+0000", 2024-09-14T23:57:10.761 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 31.630162524999999, 2024-09-14T23:57:10.761 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044721271999999999, 2024-09-14T23:57:10.762 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:10.762 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:10.762 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:10.762 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:10.762 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:10.762 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:10.762 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4060 2024-09-14T23:57:10.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.762 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:10.762 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.762 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:10.762 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099129+0000", 2024-09-14T23:57:10.762 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:10.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.762 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.763 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:10.763 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099129+0000", 2024-09-14T23:57:10.763 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:10.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.763 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:10.763 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099191+0000", 2024-09-14T23:57:10.763 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1805999999999997e-05 2024-09-14T23:57:10.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.763 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:10.763 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099193+0000", 2024-09-14T23:57:10.764 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2330000000000001e-06 2024-09-14T23:57:10.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.764 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.764 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:10.764 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099200+0000", 2024-09-14T23:57:10.764 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4200000000000001e-06 2024-09-14T23:57:10.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.764 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.764 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:10.764 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099204+0000", 2024-09-14T23:57:10.764 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7500000000000001e-06 2024-09-14T23:57:10.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.764 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.765 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:10.765 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143760+0000", 2024-09-14T23:57:10.765 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044556182 2024-09-14T23:57:10.765 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.765 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.765 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:10.765 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143804+0000", 2024-09-14T23:57:10.765 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4239000000000001e-05 2024-09-14T23:57:10.765 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.765 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.765 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:10.765 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143850+0000", 2024-09-14T23:57:10.765 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5642000000000001e-05 2024-09-14T23:57:10.765 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:10.766 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:10.766 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:10.766 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.766 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.766 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4061 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1462272~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:10.766 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099215+0000", 2024-09-14T23:57:10.766 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 31.6300764, 2024-09-14T23:57:10.766 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044745659, 2024-09-14T23:57:10.766 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:10.766 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:10.766 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:10.766 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:10.766 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:10.766 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:10.766 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4061 2024-09-14T23:57:10.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.767 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:10.767 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.767 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:10.767 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099215+0000", 2024-09-14T23:57:10.767 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:10.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.767 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.767 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:10.767 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099215+0000", 2024-09-14T23:57:10.767 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:10.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.767 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.767 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:10.767 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100672+0000", 2024-09-14T23:57:10.767 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00145689 2024-09-14T23:57:10.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.768 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.768 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:10.768 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100677+0000", 2024-09-14T23:57:10.768 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0139999999999998e-06 2024-09-14T23:57:10.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.768 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.768 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:10.768 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100689+0000", 2024-09-14T23:57:10.768 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.183e-05 2024-09-14T23:57:10.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.768 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.769 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:10.769 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100696+0000", 2024-09-14T23:57:10.769 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2599999999999999e-06 2024-09-14T23:57:10.769 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.769 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.769 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:10.769 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143877+0000", 2024-09-14T23:57:10.769 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043181279000000003 2024-09-14T23:57:10.769 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.769 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.769 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:10.769 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143919+0000", 2024-09-14T23:57:10.769 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1783000000000003e-05 2024-09-14T23:57:10.770 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.770 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.770 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:10.770 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143961+0000", 2024-09-14T23:57:10.770 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1603000000000001e-05 2024-09-14T23:57:10.770 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:10.770 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:10.770 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:10.770 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.770 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.770 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4070 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 2015232~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:10.770 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.100903+0000", 2024-09-14T23:57:10.770 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 31.628388692000001, 2024-09-14T23:57:10.770 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044103815999999997, 2024-09-14T23:57:10.771 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:10.771 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:10.771 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:10.771 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:10.771 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:10.771 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:10.771 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4070 2024-09-14T23:57:10.771 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.771 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:10.772 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.772 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:10.772 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100903+0000", 2024-09-14T23:57:10.772 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:10.772 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.772 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.772 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:10.772 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100903+0000", 2024-09-14T23:57:10.772 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:10.772 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.772 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.772 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:10.772 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.102671+0000", 2024-09-14T23:57:10.773 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001768279 2024-09-14T23:57:10.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.773 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.773 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:10.773 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.102675+0000", 2024-09-14T23:57:10.773 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9609999999999999e-06 2024-09-14T23:57:10.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.773 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.773 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:10.773 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.102684+0000", 2024-09-14T23:57:10.773 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.329e-06 2024-09-14T23:57:10.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.773 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.773 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:10.773 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.102689+0000", 2024-09-14T23:57:10.774 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7079999999999996e-06 2024-09-14T23:57:10.774 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.774 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.774 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:10.774 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.144912+0000", 2024-09-14T23:57:10.774 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.042223305000000003 2024-09-14T23:57:10.774 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.774 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.774 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:10.774 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.144960+0000", 2024-09-14T23:57:10.774 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7670000000000003e-05 2024-09-14T23:57:10.774 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.774 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.774 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:10.774 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.145006+0000", 2024-09-14T23:57:10.775 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6563999999999997e-05 2024-09-14T23:57:10.775 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:10.775 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:10.775 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:10.775 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.775 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.775 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4096 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1282048~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:10.775 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.104957+0000", 2024-09-14T23:57:10.775 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 31.624334519000001, 2024-09-14T23:57:10.775 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044093016999999998, 2024-09-14T23:57:10.775 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:10.775 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:10.775 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:10.775 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:10.776 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:10.776 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:10.776 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4096 2024-09-14T23:57:10.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.776 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:10.776 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.776 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:10.776 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104957+0000", 2024-09-14T23:57:10.776 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:10.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.776 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.776 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:10.776 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104957+0000", 2024-09-14T23:57:10.777 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:10.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.777 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.777 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:10.777 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104957+0000", 2024-09-14T23:57:10.777 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7400000000000003e-07 2024-09-14T23:57:10.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.777 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.777 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:10.777 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104960+0000", 2024-09-14T23:57:10.777 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3800000000000001e-06 2024-09-14T23:57:10.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.777 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.777 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:10.778 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104967+0000", 2024-09-14T23:57:10.778 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4030000000000001e-06 2024-09-14T23:57:10.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.778 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.778 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:10.778 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104973+0000", 2024-09-14T23:57:10.778 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7370000000000003e-06 2024-09-14T23:57:10.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.778 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.778 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:10.778 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.148883+0000", 2024-09-14T23:57:10.778 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043910124000000002 2024-09-14T23:57:10.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.779 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.779 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:10.779 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.148966+0000", 2024-09-14T23:57:10.779 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2868999999999996e-05 2024-09-14T23:57:10.779 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.779 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.779 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:10.779 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149050+0000", 2024-09-14T23:57:10.779 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3930000000000004e-05 2024-09-14T23:57:10.779 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:10.779 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:10.779 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:10.779 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.779 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.779 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4097 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1343488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:10.780 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.104983+0000", 2024-09-14T23:57:10.780 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 31.624308729999999, 2024-09-14T23:57:10.780 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044182104999999999, 2024-09-14T23:57:10.780 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:10.780 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:10.780 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:10.780 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:10.780 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:10.780 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:10.780 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4097 2024-09-14T23:57:10.780 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.780 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:10.780 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.780 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:10.781 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104983+0000", 2024-09-14T23:57:10.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:10.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.781 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:10.781 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104983+0000", 2024-09-14T23:57:10.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:10.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.781 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:10.781 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105207+0000", 2024-09-14T23:57:10.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000224687 2024-09-14T23:57:10.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.782 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:10.782 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105211+0000", 2024-09-14T23:57:10.782 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3050000000000001e-06 2024-09-14T23:57:10.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.783 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:10.783 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105217+0000", 2024-09-14T23:57:10.783 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8170000000000001e-06 2024-09-14T23:57:10.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.783 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.783 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:10.783 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105222+0000", 2024-09-14T23:57:10.783 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.267e-06 2024-09-14T23:57:10.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.783 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.783 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:10.783 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149079+0000", 2024-09-14T23:57:10.783 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043857581999999999 2024-09-14T23:57:10.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.784 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.784 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:10.784 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149122+0000", 2024-09-14T23:57:10.784 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2899999999999999e-05 2024-09-14T23:57:10.784 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.784 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.784 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:10.784 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149165+0000", 2024-09-14T23:57:10.784 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2546999999999998e-05 2024-09-14T23:57:10.784 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:10.784 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:10.784 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:10.784 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.784 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.785 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4098 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1404928~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:10.785 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.105242+0000", 2024-09-14T23:57:10.785 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 31.624048918, 2024-09-14T23:57:10.785 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044121104000000001, 2024-09-14T23:57:10.785 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:10.785 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:10.785 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:10.785 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:10.785 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:10.785 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:10.785 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4098 2024-09-14T23:57:10.785 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.785 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:10.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:10.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105242+0000", 2024-09-14T23:57:10.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:10.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:10.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105242+0000", 2024-09-14T23:57:10.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:10.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:10.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105243+0000", 2024-09-14T23:57:10.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0399999999999995e-07 2024-09-14T23:57:10.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.787 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:10.787 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105246+0000", 2024-09-14T23:57:10.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6309999999999999e-06 2024-09-14T23:57:10.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.787 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:10.787 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105253+0000", 2024-09-14T23:57:10.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0940000000000002e-06 2024-09-14T23:57:10.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.788 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:10.788 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105260+0000", 2024-09-14T23:57:10.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1799999999999999e-06 2024-09-14T23:57:10.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.788 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:10.788 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149201+0000", 2024-09-14T23:57:10.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043940629000000002 2024-09-14T23:57:10.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.788 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:10.788 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149283+0000", 2024-09-14T23:57:10.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2360000000000004e-05 2024-09-14T23:57:10.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.789 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:10.789 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149363+0000", 2024-09-14T23:57:10.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0506000000000005e-05 2024-09-14T23:57:10.789 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:10.789 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:10.789 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:10.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.789 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4099 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1466368~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:10.789 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.105270+0000", 2024-09-14T23:57:10.790 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 31.624021323000001, 2024-09-14T23:57:10.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044195433999999999, 2024-09-14T23:57:10.790 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:10.790 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:10.790 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:10.790 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:10.790 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:10.790 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:10.790 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4099 2024-09-14T23:57:10.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.790 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:10.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:10.790 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105270+0000", 2024-09-14T23:57:10.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:10.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.791 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:10.791 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105270+0000", 2024-09-14T23:57:10.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:10.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.791 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:10.791 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105429+0000", 2024-09-14T23:57:10.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015940299999999999 2024-09-14T23:57:10.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.791 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:10.791 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105433+0000", 2024-09-14T23:57:10.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3579999999999999e-06 2024-09-14T23:57:10.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.792 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:10.792 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105439+0000", 2024-09-14T23:57:10.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1759999999999998e-06 2024-09-14T23:57:10.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.792 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:10.792 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105445+0000", 2024-09-14T23:57:10.793 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9689999999999999e-06 2024-09-14T23:57:10.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.793 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:10.796 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149381+0000", 2024-09-14T23:57:10.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043936389999999999 2024-09-14T23:57:10.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.796 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:10.796 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149422+0000", 2024-09-14T23:57:10.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0865000000000003e-05 2024-09-14T23:57:10.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.797 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:10.797 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149465+0000", 2024-09-14T23:57:10.797 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3272999999999997e-05 2024-09-14T23:57:10.797 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:10.797 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:10.797 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:10.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.797 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4100 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1527808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:10.798 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.105454+0000", 2024-09-14T23:57:10.798 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 31.623836852, 2024-09-14T23:57:10.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044217411999999998, 2024-09-14T23:57:10.798 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:10.798 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:10.798 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:10.798 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:10.798 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:10.798 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:10.798 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4100 2024-09-14T23:57:10.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.798 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:10.799 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.799 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:10.799 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105454+0000", 2024-09-14T23:57:10.799 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:10.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.799 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.799 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:10.799 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105454+0000", 2024-09-14T23:57:10.799 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:10.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.799 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:10.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105578+0000", 2024-09-14T23:57:10.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012401799999999999 2024-09-14T23:57:10.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:10.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105583+0000", 2024-09-14T23:57:10.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1779999999999996e-06 2024-09-14T23:57:10.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:10.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105590+0000", 2024-09-14T23:57:10.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4279999999999999e-06 2024-09-14T23:57:10.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.801 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:10.801 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105596+0000", 2024-09-14T23:57:10.801 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0120000000000002e-06 2024-09-14T23:57:10.801 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.801 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.801 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:10.801 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149503+0000", 2024-09-14T23:57:10.801 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043906580000000001 2024-09-14T23:57:10.801 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.801 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.801 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:10.801 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149585+0000", 2024-09-14T23:57:10.801 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2356999999999998e-05 2024-09-14T23:57:10.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.802 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.802 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:10.802 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149672+0000", 2024-09-14T23:57:10.802 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6839e-05 2024-09-14T23:57:10.802 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:10.802 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:10.802 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:10.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.802 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.802 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4101 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1589248~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:10.802 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.105609+0000", 2024-09-14T23:57:10.803 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 31.623682313, 2024-09-14T23:57:10.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044175987999999999, 2024-09-14T23:57:10.803 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:10.803 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:10.803 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:10.803 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:10.803 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:10.803 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:10.803 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4101 2024-09-14T23:57:10.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.803 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:10.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:10.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105609+0000", 2024-09-14T23:57:10.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:10.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:10.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105609+0000", 2024-09-14T23:57:10.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:10.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:10.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.106671+0000", 2024-09-14T23:57:10.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010619270000000001 2024-09-14T23:57:10.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.805 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.805 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:10.805 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.106675+0000", 2024-09-14T23:57:10.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4429999999999996e-06 2024-09-14T23:57:10.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.805 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.805 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:10.805 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.106685+0000", 2024-09-14T23:57:10.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7429999999999997e-06 2024-09-14T23:57:10.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.805 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.805 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:10.806 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.106690+0000", 2024-09-14T23:57:10.806 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.293e-06 2024-09-14T23:57:10.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.806 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.806 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:10.806 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149700+0000", 2024-09-14T23:57:10.806 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043009512 2024-09-14T23:57:10.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.806 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.806 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:10.806 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149742+0000", 2024-09-14T23:57:10.806 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2083999999999999e-05 2024-09-14T23:57:10.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.806 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.807 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:10.807 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149785+0000", 2024-09-14T23:57:10.807 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2985999999999998e-05 2024-09-14T23:57:10.807 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:10.807 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:10.807 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:10.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.807 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4116 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 180224~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:10.807 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.107575+0000", 2024-09-14T23:57:10.807 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 31.621715845000001, 2024-09-14T23:57:10.807 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044157522999999997, 2024-09-14T23:57:10.807 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:10.807 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:10.807 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:10.808 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:10.808 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:10.808 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:10.808 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4116 2024-09-14T23:57:10.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.808 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:10.808 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.808 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:10.808 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.107575+0000", 2024-09-14T23:57:10.808 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:10.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.808 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.808 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:10.808 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.107575+0000", 2024-09-14T23:57:10.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:10.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.809 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:10.809 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108671+0000", 2024-09-14T23:57:10.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010954510000000001 2024-09-14T23:57:10.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.809 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:10.809 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108676+0000", 2024-09-14T23:57:10.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6639999999999997e-06 2024-09-14T23:57:10.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.810 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.810 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:10.810 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108685+0000", 2024-09-14T23:57:10.810 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1130000000000007e-06 2024-09-14T23:57:10.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.810 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.810 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:10.810 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108691+0000", 2024-09-14T23:57:10.810 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6019999999999996e-06 2024-09-14T23:57:10.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.810 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.810 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:10.810 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.151530+0000", 2024-09-14T23:57:10.810 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.042838345999999999 2024-09-14T23:57:10.811 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.811 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.811 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:10.811 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.151619+0000", 2024-09-14T23:57:10.811 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9302999999999995e-05 2024-09-14T23:57:10.811 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.811 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.811 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:10.811 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.151733+0000", 2024-09-14T23:57:10.811 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011404399999999999 2024-09-14T23:57:10.811 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:10.811 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:10.811 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:10.811 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.811 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.812 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4122 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 548864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:10.812 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.108842+0000", 2024-09-14T23:57:10.812 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 31.620449007000001, 2024-09-14T23:57:10.812 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044103631999999997, 2024-09-14T23:57:10.812 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:10.812 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:10.812 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:10.812 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:10.812 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:10.812 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:10.812 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4122 2024-09-14T23:57:10.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.813 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:10.813 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.813 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:10.813 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108842+0000", 2024-09-14T23:57:10.813 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:10.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.813 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.813 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:10.813 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108842+0000", 2024-09-14T23:57:10.814 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:10.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.814 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.814 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:10.814 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108843+0000", 2024-09-14T23:57:10.814 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7000000000000001e-07 2024-09-14T23:57:10.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.814 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.815 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:10.815 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108844+0000", 2024-09-14T23:57:10.815 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6479999999999999e-06 2024-09-14T23:57:10.815 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.815 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.815 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:10.815 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108848+0000", 2024-09-14T23:57:10.815 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8029999999999998e-06 2024-09-14T23:57:10.815 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.815 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.815 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:10.815 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108851+0000", 2024-09-14T23:57:10.815 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7939999999999998e-06 2024-09-14T23:57:10.816 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.816 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.816 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:10.816 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.152838+0000", 2024-09-14T23:57:10.816 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043987351000000001 2024-09-14T23:57:10.816 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.816 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.816 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:10.816 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.152885+0000", 2024-09-14T23:57:10.816 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7237000000000002e-05 2024-09-14T23:57:10.816 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.816 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.817 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:10.817 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.152946+0000", 2024-09-14T23:57:10.817 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0529000000000003e-05 2024-09-14T23:57:10.817 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:10.817 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:10.817 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:10.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.817 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.817 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4123 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 610304~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:10.817 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.108855+0000", 2024-09-14T23:57:10.817 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 31.620436678000001, 2024-09-14T23:57:10.817 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044321610999999997, 2024-09-14T23:57:10.817 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:10.818 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:10.818 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:10.818 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:10.818 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:10.818 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:10.818 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4123 2024-09-14T23:57:10.818 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.818 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:10.818 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.818 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:10.818 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108855+0000", 2024-09-14T23:57:10.819 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:10.819 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.819 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.819 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:10.819 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108855+0000", 2024-09-14T23:57:10.819 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:10.819 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.819 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.819 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:10.819 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.110671+0000", 2024-09-14T23:57:10.819 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018159719999999999 2024-09-14T23:57:10.819 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.820 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.820 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:10.820 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.110676+0000", 2024-09-14T23:57:10.820 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9450000000000001e-06 2024-09-14T23:57:10.820 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.820 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.820 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:10.820 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.110685+0000", 2024-09-14T23:57:10.820 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5359999999999998e-06 2024-09-14T23:57:10.820 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.820 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.820 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:10.820 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.110692+0000", 2024-09-14T23:57:10.820 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9280000000000004e-06 2024-09-14T23:57:10.821 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.821 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.821 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:10.821 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.152984+0000", 2024-09-14T23:57:10.821 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.042291677999999999 2024-09-14T23:57:10.821 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.821 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.821 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:10.821 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.153066+0000", 2024-09-14T23:57:10.821 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2657999999999994e-05 2024-09-14T23:57:10.821 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.821 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.821 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:10.821 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.153176+0000", 2024-09-14T23:57:10.822 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000109894 2024-09-14T23:57:10.822 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:10.822 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:10.822 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:10.822 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.822 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.822 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4136 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:10.822 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.111056+0000", 2024-09-14T23:57:10.822 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 31.61823553, 2024-09-14T23:57:10.822 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044097428000000001, 2024-09-14T23:57:10.822 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:10.822 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:10.822 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:10.822 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:10.823 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:10.823 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:10.823 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4136 2024-09-14T23:57:10.823 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.823 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:10.823 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.823 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:10.823 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.111056+0000", 2024-09-14T23:57:10.823 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:10.823 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.823 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.823 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:10.824 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.111056+0000", 2024-09-14T23:57:10.824 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:10.824 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.824 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.824 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:10.824 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.112671+0000", 2024-09-14T23:57:10.824 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016153999999999999 2024-09-14T23:57:10.824 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.824 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.824 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:10.824 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.112675+0000", 2024-09-14T23:57:10.825 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2830000000000003e-06 2024-09-14T23:57:10.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.825 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.825 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:10.825 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.112684+0000", 2024-09-14T23:57:10.825 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7800000000000006e-06 2024-09-14T23:57:10.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.825 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.826 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:10.826 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.112689+0000", 2024-09-14T23:57:10.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0000000000000004e-06 2024-09-14T23:57:10.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.826 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.826 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:10.826 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.155064+0000", 2024-09-14T23:57:10.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.042374423000000001 2024-09-14T23:57:10.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.826 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.826 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:10.826 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.155111+0000", 2024-09-14T23:57:10.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7367000000000002e-05 2024-09-14T23:57:10.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.827 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.827 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:10.827 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.155153+0000", 2024-09-14T23:57:10.827 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2175000000000002e-05 2024-09-14T23:57:10.827 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:10.827 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:10.827 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:10.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.827 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.827 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:46572 3.1d 3:b9e1f524:::rbd_header.1118cee06433:head [watch reconnect cookie 140670782499824 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-14T23:57:10.827 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:57.046868+0000", 2024-09-14T23:57:10.827 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.682423724, 2024-09-14T23:57:10.827 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0243412620000001, 2024-09-14T23:57:10.827 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:10.828 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:10.828 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:10.828 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:10.828 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:10.828 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:10.828 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 46572 2024-09-14T23:57:10.828 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.828 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:10.828 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.828 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:10.828 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046868+0000", 2024-09-14T23:57:10.828 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:10.828 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.828 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.829 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:10.829 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046868+0000", 2024-09-14T23:57:10.829 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:10.829 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.829 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.829 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:10.829 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046870+0000", 2024-09-14T23:57:10.829 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.204e-06 2024-09-14T23:57:10.829 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.829 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.829 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:10.829 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046877+0000", 2024-09-14T23:57:10.830 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5100000000000001e-06 2024-09-14T23:57:10.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.830 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.830 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:10.830 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046898+0000", 2024-09-14T23:57:10.830 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1149999999999999e-05 2024-09-14T23:57:10.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.830 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.830 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:10.830 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046911+0000", 2024-09-14T23:57:10.830 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2234e-05 2024-09-14T23:57:10.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.830 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.830 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:10.831 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046988+0000", 2024-09-14T23:57:10.831 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7445000000000001e-05 2024-09-14T23:57:10.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.831 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.831 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:57:10.831 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.047001+0000", 2024-09-14T23:57:10.831 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2951e-05 2024-09-14T23:57:10.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.831 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.831 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:10.831 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.069287+0000", 2024-09-14T23:57:10.831 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.022285748 2024-09-14T23:57:10.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.831 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.831 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for rw locks", 2024-09-14T23:57:10.831 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.069397+0000", 2024-09-14T23:57:10.832 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000110326 2024-09-14T23:57:10.832 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.832 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.832 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:10.832 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.069997+0000", 2024-09-14T23:57:10.832 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00059984399999999998 2024-09-14T23:57:10.832 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.832 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.832 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:10.832 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.070109+0000", 2024-09-14T23:57:10.832 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000112367 2024-09-14T23:57:10.832 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:10.832 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:10.832 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:10.833 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.071209+0000", 2024-09-14T23:57:10.833 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010994830000000001 2024-09-14T23:57:10.833 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:10.833 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:10.833 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:10.833 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:10.833 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:10.833 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:10.835 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T23:57:10.835 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:10.833+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:10.935 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:10.934+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:10.943 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:10.943 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:10.943 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:57:10.944 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:10.952 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T23:57:11.036 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:11.035+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:11.058 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:11.058 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:11.059 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:57:11.059 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:57:11.059 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:11.068 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T23:57:11.152 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:11.151+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:11.186 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-09-14T23:57:11.212 INFO:teuthology.orchestra.run.smithi042.stdout:1955 writesame 0xd6b4bd1 thru 0xd6ba326 data_size 0x7(0x5756 bytes) 2024-09-14T23:57:11.231 INFO:teuthology.orchestra.run.smithi042.stdout:1956 trunc from 0xd874870 to 0x55ad70c 2024-09-14T23:57:11.238 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:11.236+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:11.244 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:11.245 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:57:11.245 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:57:11.245 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:57:11.245 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.245 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4284.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-14T23:57:11.245 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:57:11.245 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 48.716205932000001, 2024-09-14T23:57:11.245 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.004942086, 2024-09-14T23:57:11.246 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:11.246 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:11.246 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:11.246 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:11.246 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4284", 2024-09-14T23:57:11.246 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/4278347926", 2024-09-14T23:57:11.246 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1 2024-09-14T23:57:11.246 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.246 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:11.246 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.246 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:11.247 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:57:11.247 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.247 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.247 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.247 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:11.247 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:57:11.247 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.247 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.247 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.247 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:11.247 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524331+0000", 2024-09-14T23:57:11.248 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.383e-05 2024-09-14T23:57:11.248 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.248 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.248 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:11.248 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524339+0000", 2024-09-14T23:57:11.248 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8290000000000008e-06 2024-09-14T23:57:11.248 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.248 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.248 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:11.249 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524359+0000", 2024-09-14T23:57:11.249 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9998000000000001e-05 2024-09-14T23:57:11.249 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.249 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.249 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:11.249 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524373+0000", 2024-09-14T23:57:11.249 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4304000000000001e-05 2024-09-14T23:57:11.249 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.249 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.249 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:11.250 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524488+0000", 2024-09-14T23:57:11.250 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114663 2024-09-14T23:57:11.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.250 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.250 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:57:11.250 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524501+0000", 2024-09-14T23:57:11.250 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3349999999999999e-05 2024-09-14T23:57:11.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.250 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.250 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:11.250 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.526868+0000", 2024-09-14T23:57:11.251 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023670090000000002 2024-09-14T23:57:11.251 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.251 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.251 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:11.251 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.527041+0000", 2024-09-14T23:57:11.251 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000172597 2024-09-14T23:57:11.251 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.251 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.251 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-14T23:57:11.251 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.527384+0000", 2024-09-14T23:57:11.251 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00034309500000000002 2024-09-14T23:57:11.251 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.252 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.252 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:11.252 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529103+0000", 2024-09-14T23:57:11.252 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017185220000000001 2024-09-14T23:57:11.252 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.252 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.252 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:57:11.252 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529188+0000", 2024-09-14T23:57:11.252 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5591000000000001e-05 2024-09-14T23:57:11.252 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.253 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.253 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:11.253 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529227+0000", 2024-09-14T23:57:11.253 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8563000000000003e-05 2024-09-14T23:57:11.253 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.253 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.253 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:11.253 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529259+0000", 2024-09-14T23:57:11.253 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2734999999999997e-05 2024-09-14T23:57:11.253 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.253 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:11.253 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.254 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.254 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.254 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4284.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-14T23:57:11.254 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:23.530683+0000", 2024-09-14T23:57:11.254 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.709840266999997, 2024-09-14T23:57:11.254 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0036133110000000001, 2024-09-14T23:57:11.254 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:11.254 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:11.254 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:11.254 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:11.254 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4284", 2024-09-14T23:57:11.255 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/4278347926", 2024-09-14T23:57:11.255 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5 2024-09-14T23:57:11.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.255 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:11.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.255 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:11.255 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530683+0000", 2024-09-14T23:57:11.255 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.256 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.256 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.256 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:11.256 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530683+0000", 2024-09-14T23:57:11.256 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.256 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.256 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.256 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:11.257 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530685+0000", 2024-09-14T23:57:11.257 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.075e-06 2024-09-14T23:57:11.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.257 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.257 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:11.257 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530695+0000", 2024-09-14T23:57:11.257 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0261999999999999e-05 2024-09-14T23:57:11.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.257 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.257 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:11.258 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530715+0000", 2024-09-14T23:57:11.258 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9930000000000001e-05 2024-09-14T23:57:11.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.258 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.258 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:11.258 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530724+0000", 2024-09-14T23:57:11.258 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.901e-06 2024-09-14T23:57:11.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.258 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.258 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:11.259 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530800+0000", 2024-09-14T23:57:11.259 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5873999999999999e-05 2024-09-14T23:57:11.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.259 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.259 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-14T23:57:11.259 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530807+0000", 2024-09-14T23:57:11.259 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2389999999999996e-06 2024-09-14T23:57:11.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.259 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.260 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:11.260 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.532494+0000", 2024-09-14T23:57:11.260 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016871830000000001 2024-09-14T23:57:11.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.260 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.260 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:11.260 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.532599+0000", 2024-09-14T23:57:11.260 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010472 2024-09-14T23:57:11.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.261 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.261 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1", 2024-09-14T23:57:11.261 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.532715+0000", 2024-09-14T23:57:11.261 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011568199999999999 2024-09-14T23:57:11.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.261 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.261 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:57:11.261 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.533460+0000", 2024-09-14T23:57:11.261 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00074494999999999997 2024-09-14T23:57:11.262 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.262 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.262 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:11.262 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534230+0000", 2024-09-14T23:57:11.262 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00077036599999999998 2024-09-14T23:57:11.262 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.262 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.262 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:11.262 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534266+0000", 2024-09-14T23:57:11.262 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5725000000000001e-05 2024-09-14T23:57:11.262 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.263 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:11.263 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534296+0000", 2024-09-14T23:57:11.263 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0403999999999999e-05 2024-09-14T23:57:11.263 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.263 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:11.263 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.263 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4284.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-14T23:57:11.263 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:24.535160+0000", 2024-09-14T23:57:11.264 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 46.705363626, 2024-09-14T23:57:11.264 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033473650000000001, 2024-09-14T23:57:11.264 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:11.264 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:11.264 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:11.264 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:11.264 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4284", 2024-09-14T23:57:11.264 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/4278347926", 2024-09-14T23:57:11.265 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 7 2024-09-14T23:57:11.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.265 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:11.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.265 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:11.265 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535160+0000", 2024-09-14T23:57:11.265 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.265 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:11.265 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535160+0000", 2024-09-14T23:57:11.266 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.266 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.266 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.266 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:11.266 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535162+0000", 2024-09-14T23:57:11.266 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3369999999999998e-06 2024-09-14T23:57:11.266 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.266 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.267 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:11.267 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535171+0000", 2024-09-14T23:57:11.267 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7700000000000007e-06 2024-09-14T23:57:11.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.267 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.267 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:11.267 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535189+0000", 2024-09-14T23:57:11.267 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7938e-05 2024-09-14T23:57:11.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.267 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.267 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:11.268 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535197+0000", 2024-09-14T23:57:11.268 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5639999999999997e-06 2024-09-14T23:57:11.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.268 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.268 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:11.268 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535266+0000", 2024-09-14T23:57:11.268 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8926999999999999e-05 2024-09-14T23:57:11.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.268 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.268 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-14T23:57:11.268 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535271+0000", 2024-09-14T23:57:11.269 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0329999999999999e-06 2024-09-14T23:57:11.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.269 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.269 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:11.269 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.537064+0000", 2024-09-14T23:57:11.269 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001792929 2024-09-14T23:57:11.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.269 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.269 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:11.269 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.537131+0000", 2024-09-14T23:57:11.269 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6605000000000004e-05 2024-09-14T23:57:11.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.270 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.270 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1", 2024-09-14T23:57:11.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.537271+0000", 2024-09-14T23:57:11.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013992400000000001 2024-09-14T23:57:11.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.270 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.270 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:57:11.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.538093+0000", 2024-09-14T23:57:11.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00082210200000000001 2024-09-14T23:57:11.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.271 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.271 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:11.271 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.538439+0000", 2024-09-14T23:57:11.271 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00034595999999999997 2024-09-14T23:57:11.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.271 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.271 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:11.271 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.538476+0000", 2024-09-14T23:57:11.271 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6770000000000002e-05 2024-09-14T23:57:11.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.271 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.271 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:11.272 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.538507+0000", 2024-09-14T23:57:11.272 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1506e-05 2024-09-14T23:57:11.272 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.272 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:11.272 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.272 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.272 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.272 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T23:57:11.272 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:34.406877+0000", 2024-09-14T23:57:11.272 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.833646621, 2024-09-14T23:57:11.272 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030351169999999999, 2024-09-14T23:57:11.272 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:11.273 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:11.273 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:11.273 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:11.273 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:11.273 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:11.273 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 10 2024-09-14T23:57:11.273 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.273 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:11.273 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.273 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:11.273 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406877+0000", 2024-09-14T23:57:11.274 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.274 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:11.274 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406877+0000", 2024-09-14T23:57:11.274 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.274 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:11.274 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406878+0000", 2024-09-14T23:57:11.274 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.291e-06 2024-09-14T23:57:11.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.275 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.275 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:11.275 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406883+0000", 2024-09-14T23:57:11.275 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2399999999999998e-06 2024-09-14T23:57:11.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.275 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.275 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:11.275 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406891+0000", 2024-09-14T23:57:11.275 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5889999999999997e-06 2024-09-14T23:57:11.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.275 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.275 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:11.276 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406898+0000", 2024-09-14T23:57:11.276 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3599999999999998e-06 2024-09-14T23:57:11.276 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.276 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.276 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:11.276 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406948+0000", 2024-09-14T23:57:11.276 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9435999999999999e-05 2024-09-14T23:57:11.276 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.276 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.276 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-14T23:57:11.276 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406953+0000", 2024-09-14T23:57:11.277 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2809999999999999e-06 2024-09-14T23:57:11.277 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.277 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.277 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:11.277 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.408559+0000", 2024-09-14T23:57:11.277 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00160607 2024-09-14T23:57:11.277 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.277 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.277 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:11.277 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.408637+0000", 2024-09-14T23:57:11.277 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8504000000000006e-05 2024-09-14T23:57:11.278 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.278 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.278 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 4", 2024-09-14T23:57:11.278 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.408781+0000", 2024-09-14T23:57:11.278 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014374799999999999 2024-09-14T23:57:11.278 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.278 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.278 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:57:11.278 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.409659+0000", 2024-09-14T23:57:11.278 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000877781 2024-09-14T23:57:11.278 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.279 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.279 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:11.279 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.409822+0000", 2024-09-14T23:57:11.279 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016295400000000001 2024-09-14T23:57:11.279 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.279 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.279 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:11.279 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.409872+0000", 2024-09-14T23:57:11.279 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0022000000000003e-05 2024-09-14T23:57:11.279 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.279 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.280 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:11.280 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.409912+0000", 2024-09-14T23:57:11.280 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9841e-05 2024-09-14T23:57:11.280 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.280 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:11.280 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.280 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.280 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.280 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "MOSDPGPush(3.c 48/47 [PushOp(3:34183e86:::rbd_data.111888d8b54.0000000000000003:head, version: 37'112, data_included: [0~65536,131072~65536,262144~131072,458752~65536,553790~16537,589824~65536,700433~17543,720896~65536,851968~65536,983040~490208], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:34183e86:::rbd_data.111888d8b54.0000000000000003:head@37'112, size: 9745466, copy_subset: [(0, 9745466)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1473248, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T23:57:11.280 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:58.328790+0000", 2024-09-14T23:57:11.280 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 12.911732818000001, 2024-09-14T23:57:11.281 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0024861290000000001, 2024-09-14T23:57:11.281 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:11.281 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:11.281 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:11.281 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:11.281 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.281 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:11.281 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.328790+0000", 2024-09-14T23:57:11.281 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.281 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.281 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.281 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:11.282 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.328790+0000", 2024-09-14T23:57:11.282 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.282 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.282 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:11.282 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.328791+0000", 2024-09-14T23:57:11.282 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3900000000000004e-07 2024-09-14T23:57:11.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.282 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.282 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:11.282 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.329762+0000", 2024-09-14T23:57:11.283 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000970362 2024-09-14T23:57:11.283 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.283 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.283 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:11.283 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.329779+0000", 2024-09-14T23:57:11.283 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7655999999999998e-05 2024-09-14T23:57:11.283 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.283 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.283 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:11.284 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.329791+0000", 2024-09-14T23:57:11.284 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2131e-05 2024-09-14T23:57:11.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.284 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.284 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:11.284 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.329886+0000", 2024-09-14T23:57:11.284 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4439999999999997e-05 2024-09-14T23:57:11.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.284 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.284 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:11.285 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.329902+0000", 2024-09-14T23:57:11.285 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6651e-05 2024-09-14T23:57:11.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.285 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.285 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:11.285 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.331277+0000", 2024-09-14T23:57:11.285 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00137415 2024-09-14T23:57:11.285 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.285 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:11.285 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.286 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.286 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.286 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "MOSDPGPush(3.c 48/47 [PushOp(3:34183e86:::rbd_data.111888d8b54.0000000000000003:head, version: 37'112, data_included: [1473248~99616,1638400~65536,1835008~262144,2162688~65536,2293760~262144,2686976~65536,9737423~8043], data_size: 828555, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:34183e86:::rbd_data.111888d8b54.0000000000000003:head@37'112, size: 9745466, copy_subset: [(0, 9745466)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 9745466, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1473248, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-14T23:57:11.286 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:58.332069+0000", 2024-09-14T23:57:11.286 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 12.908454018, 2024-09-14T23:57:11.286 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0025177979999999999, 2024-09-14T23:57:11.286 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:11.286 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:11.286 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:11.286 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:11.286 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.287 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:11.287 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.332069+0000", 2024-09-14T23:57:11.287 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.287 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.287 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:11.287 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.332069+0000", 2024-09-14T23:57:11.287 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.287 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.287 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:11.288 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.332070+0000", 2024-09-14T23:57:11.288 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1200000000000003e-07 2024-09-14T23:57:11.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.288 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.288 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:11.288 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.332822+0000", 2024-09-14T23:57:11.288 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00075239299999999996 2024-09-14T23:57:11.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.289 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.289 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:11.289 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.332845+0000", 2024-09-14T23:57:11.289 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2436999999999999e-05 2024-09-14T23:57:11.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.289 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.289 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:11.289 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.332866+0000", 2024-09-14T23:57:11.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0874e-05 2024-09-14T23:57:11.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.290 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.290 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:11.290 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.332970+0000", 2024-09-14T23:57:11.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000104528 2024-09-14T23:57:11.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.290 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.290 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:11.290 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.332987+0000", 2024-09-14T23:57:11.291 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7073000000000001e-05 2024-09-14T23:57:11.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.291 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.291 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:11.291 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.334587+0000", 2024-09-14T23:57:11.291 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001599881 2024-09-14T23:57:11.291 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.291 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:11.292 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.292 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.292 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:50626 3.d e51/46 3:b5154cfe:::rbd_data.11183b9aac34.0000000000000004:head v 51'63, mlcod=45'61)", 2024-09-14T23:57:11.292 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:01.140129+0000", 2024-09-14T23:57:11.292 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 10.100394021, 2024-09-14T23:57:11.292 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0029883850000000001, 2024-09-14T23:57:11.292 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:11.292 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:11.292 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:11.293 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:11.293 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.293 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:11.293 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.140129+0000", 2024-09-14T23:57:11.293 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.293 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.293 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:11.293 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.140129+0000", 2024-09-14T23:57:11.293 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.294 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.294 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.294 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:11.294 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.140130+0000", 2024-09-14T23:57:11.294 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0529999999999999e-06 2024-09-14T23:57:11.294 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.294 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.295 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:11.295 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.140311+0000", 2024-09-14T23:57:11.295 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018054200000000001 2024-09-14T23:57:11.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.295 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.295 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:11.295 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.140325+0000", 2024-09-14T23:57:11.295 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.382e-05 2024-09-14T23:57:11.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.295 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.295 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:11.295 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.140332+0000", 2024-09-14T23:57:11.296 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5700000000000004e-06 2024-09-14T23:57:11.296 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.296 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.296 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:11.296 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.140392+0000", 2024-09-14T23:57:11.296 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9296000000000003e-05 2024-09-14T23:57:11.296 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.296 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.296 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:11.296 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.140436+0000", 2024-09-14T23:57:11.296 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4515999999999999e-05 2024-09-14T23:57:11.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.297 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.297 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:11.297 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.143075+0000", 2024-09-14T23:57:11.297 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0026385369999999998 2024-09-14T23:57:11.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.297 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.297 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:11.297 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.143118+0000", 2024-09-14T23:57:11.297 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3050999999999998e-05 2024-09-14T23:57:11.297 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.297 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:11.298 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.298 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.298 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:51277 3.d e52/46 3:b2645877:::rbd_data.11181975a1de.0000000000000001:head v 52'69, mlcod=51'66)", 2024-09-14T23:57:11.298 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:02.157938+0000", 2024-09-14T23:57:11.298 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.082585023, 2024-09-14T23:57:11.298 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0027517549999999998, 2024-09-14T23:57:11.298 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:11.298 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:11.298 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:11.298 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:11.298 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.299 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:11.299 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.157938+0000", 2024-09-14T23:57:11.299 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.299 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.299 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.299 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:11.299 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.157938+0000", 2024-09-14T23:57:11.299 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.299 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.299 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.299 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:11.299 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.157939+0000", 2024-09-14T23:57:11.300 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9199999999999999e-07 2024-09-14T23:57:11.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.300 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.300 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:11.300 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158543+0000", 2024-09-14T23:57:11.300 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00060333299999999995 2024-09-14T23:57:11.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.300 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.300 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:11.300 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158575+0000", 2024-09-14T23:57:11.300 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2801e-05 2024-09-14T23:57:11.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.301 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.301 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:11.301 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158586+0000", 2024-09-14T23:57:11.301 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.102e-05 2024-09-14T23:57:11.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.301 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.301 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:11.301 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158643+0000", 2024-09-14T23:57:11.301 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6320999999999998e-05 2024-09-14T23:57:11.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.302 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.302 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:11.302 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158693+0000", 2024-09-14T23:57:11.302 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0037000000000002e-05 2024-09-14T23:57:11.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.302 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.302 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:11.302 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.160641+0000", 2024-09-14T23:57:11.302 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0019485629999999999 2024-09-14T23:57:11.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.302 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.303 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:11.303 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.160690+0000", 2024-09-14T23:57:11.303 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8687999999999999e-05 2024-09-14T23:57:11.303 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.303 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:11.303 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.303 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:51275 3.2 e52/49 3:47a71653:::rbd_data.11181975a1de.0000000000000000:head v 52'118, mlcod=52'116)", 2024-09-14T23:57:11.304 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:02.158044+0000", 2024-09-14T23:57:11.304 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.0824794509999993, 2024-09-14T23:57:11.304 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0028337200000000001, 2024-09-14T23:57:11.304 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:11.304 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:11.304 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:11.304 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:11.304 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.304 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:11.304 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158044+0000", 2024-09-14T23:57:11.304 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.305 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:11.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158044+0000", 2024-09-14T23:57:11.305 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.305 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:11.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158045+0000", 2024-09-14T23:57:11.305 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5700000000000002e-07 2024-09-14T23:57:11.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.306 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.306 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:11.306 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158637+0000", 2024-09-14T23:57:11.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00059278499999999997 2024-09-14T23:57:11.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.306 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.306 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:11.307 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158657+0000", 2024-09-14T23:57:11.307 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9422000000000001e-05 2024-09-14T23:57:11.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.307 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.307 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:11.307 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158675+0000", 2024-09-14T23:57:11.307 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8564999999999999e-05 2024-09-14T23:57:11.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.307 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.307 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:11.307 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158735+0000", 2024-09-14T23:57:11.308 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9904000000000001e-05 2024-09-14T23:57:11.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.308 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.308 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:11.308 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158757+0000", 2024-09-14T23:57:11.308 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1277e-05 2024-09-14T23:57:11.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.308 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.309 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:11.309 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.160845+0000", 2024-09-14T23:57:11.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0020881419999999999 2024-09-14T23:57:11.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.309 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.309 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:11.309 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.160878+0000", 2024-09-14T23:57:11.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2867999999999998e-05 2024-09-14T23:57:11.309 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.309 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:11.309 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.310 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56383 3.c e56/49 3:36b698b0:::rbd_data.1118c4b3aebe.0000000000000000:head v 56'194, mlcod=56'192)", 2024-09-14T23:57:11.310 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.270483+0000", 2024-09-14T23:57:11.310 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.9700404670000005, 2024-09-14T23:57:11.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0079685880000000004, 2024-09-14T23:57:11.310 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:11.310 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:11.310 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:11.310 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:11.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.310 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:11.311 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.270483+0000", 2024-09-14T23:57:11.311 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.311 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.311 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:11.311 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.270483+0000", 2024-09-14T23:57:11.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.312 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:11.312 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.270484+0000", 2024-09-14T23:57:11.313 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7400000000000002e-07 2024-09-14T23:57:11.313 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.313 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.313 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:11.313 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273680+0000", 2024-09-14T23:57:11.314 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031966680000000002 2024-09-14T23:57:11.314 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.314 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.314 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:11.314 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273697+0000", 2024-09-14T23:57:11.314 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6595e-05 2024-09-14T23:57:11.314 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.314 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.315 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:11.315 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273705+0000", 2024-09-14T23:57:11.315 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7530000000000001e-06 2024-09-14T23:57:11.315 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.315 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.315 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:11.315 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273763+0000", 2024-09-14T23:57:11.316 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8238e-05 2024-09-14T23:57:11.316 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.316 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.316 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:11.316 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273781+0000", 2024-09-14T23:57:11.316 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8281e-05 2024-09-14T23:57:11.316 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.316 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.316 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:11.316 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.278408+0000", 2024-09-14T23:57:11.317 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00462716 2024-09-14T23:57:11.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.317 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.317 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:11.317 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.278451+0000", 2024-09-14T23:57:11.317 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3019000000000003e-05 2024-09-14T23:57:11.317 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.317 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:11.317 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.318 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.318 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56384 3.c e56/49 3:36b698b0:::rbd_data.1118c4b3aebe.0000000000000000:head v 56'196, mlcod=56'192)", 2024-09-14T23:57:11.318 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:57:11.318 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.9660077950000003, 2024-09-14T23:57:11.318 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0046707529999999997, 2024-09-14T23:57:11.318 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:11.318 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:11.318 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:11.318 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:11.318 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.318 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:11.318 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:57:11.319 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.319 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.319 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:11.319 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:57:11.319 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.319 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.319 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:11.319 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:57:11.319 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1699999999999997e-07 2024-09-14T23:57:11.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.320 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.320 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:11.320 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274523+0000", 2024-09-14T23:57:11.320 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9129999999999997e-06 2024-09-14T23:57:11.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.320 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.320 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:11.320 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274535+0000", 2024-09-14T23:57:11.320 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1788e-05 2024-09-14T23:57:11.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.320 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.321 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:11.321 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274542+0000", 2024-09-14T23:57:11.321 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0539999999999997e-06 2024-09-14T23:57:11.321 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.321 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.321 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:11.321 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275889+0000", 2024-09-14T23:57:11.322 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00134669 2024-09-14T23:57:11.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.322 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.322 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:11.322 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275917+0000", 2024-09-14T23:57:11.322 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7664000000000001e-05 2024-09-14T23:57:11.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.323 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.323 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:11.323 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279134+0000", 2024-09-14T23:57:11.323 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0032178459999999999 2024-09-14T23:57:11.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.323 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.323 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:11.323 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279186+0000", 2024-09-14T23:57:11.323 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1881e-05 2024-09-14T23:57:11.323 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.324 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:11.324 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.324 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.324 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.324 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56381 3.1d e56/49 3:bcf9a8cf:::rbd_data.1118c4b3aebe.0000000000000006:head v 56'210, mlcod=56'208)", 2024-09-14T23:57:11.324 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.275462+0000", 2024-09-14T23:57:11.324 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.9650614649999998, 2024-09-14T23:57:11.324 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0071310710000000001, 2024-09-14T23:57:11.324 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:11.324 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:11.324 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:11.324 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:11.325 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.325 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:11.325 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275462+0000", 2024-09-14T23:57:11.325 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.325 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.325 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:11.325 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275462+0000", 2024-09-14T23:57:11.325 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.325 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.326 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:11.326 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275463+0000", 2024-09-14T23:57:11.326 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2699999999999999e-07 2024-09-14T23:57:11.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.326 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.326 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:11.326 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276779+0000", 2024-09-14T23:57:11.326 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0013163649999999999 2024-09-14T23:57:11.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.327 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.327 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:11.327 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276795+0000", 2024-09-14T23:57:11.327 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.59e-05 2024-09-14T23:57:11.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.327 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.327 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:11.327 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276806+0000", 2024-09-14T23:57:11.327 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1106e-05 2024-09-14T23:57:11.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.327 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.327 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:11.328 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276941+0000", 2024-09-14T23:57:11.328 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013477500000000001 2024-09-14T23:57:11.328 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.328 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.328 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:11.328 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276963+0000", 2024-09-14T23:57:11.328 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2419999999999999e-05 2024-09-14T23:57:11.328 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.328 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.328 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:11.329 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282546+0000", 2024-09-14T23:57:11.329 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0055827100000000003 2024-09-14T23:57:11.329 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.329 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.329 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:11.329 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282593+0000", 2024-09-14T23:57:11.329 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7067999999999997e-05 2024-09-14T23:57:11.329 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.329 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:11.330 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.330 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.330 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.330 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56387 3.c e56/49 3:30ee4ba0:::rbd_data.1118c4b3aebe.0000000000000004:head v 56'197, mlcod=56'192)", 2024-09-14T23:57:11.330 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.276498+0000", 2024-09-14T23:57:11.330 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.9640253699999999, 2024-09-14T23:57:11.330 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0092192639999999996, 2024-09-14T23:57:11.331 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:11.331 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:11.331 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:11.331 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:11.331 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.331 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:11.331 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276498+0000", 2024-09-14T23:57:11.331 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.331 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.332 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.332 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:11.332 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276498+0000", 2024-09-14T23:57:11.332 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.332 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.332 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:11.332 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276499+0000", 2024-09-14T23:57:11.332 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1100000000000004e-07 2024-09-14T23:57:11.333 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.333 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.333 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:11.333 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279646+0000", 2024-09-14T23:57:11.333 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031473220000000001 2024-09-14T23:57:11.333 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.333 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.333 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:11.333 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279672+0000", 2024-09-14T23:57:11.333 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6129999999999999e-05 2024-09-14T23:57:11.333 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.333 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.334 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:11.334 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279682+0000", 2024-09-14T23:57:11.334 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7170000000000003e-06 2024-09-14T23:57:11.334 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.334 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.334 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:11.334 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279742+0000", 2024-09-14T23:57:11.334 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9926999999999997e-05 2024-09-14T23:57:11.334 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.335 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.335 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:11.335 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279761+0000", 2024-09-14T23:57:11.335 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9402000000000002e-05 2024-09-14T23:57:11.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.335 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.335 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:11.335 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.285680+0000", 2024-09-14T23:57:11.335 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.005918881 2024-09-14T23:57:11.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.335 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.335 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:11.336 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.285717+0000", 2024-09-14T23:57:11.336 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6974000000000002e-05 2024-09-14T23:57:11.336 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.336 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:11.336 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.336 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.336 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56388 3.c e56/49 3:30ee4ba0:::rbd_data.1118c4b3aebe.0000000000000004:head v 56'199, mlcod=56'192)", 2024-09-14T23:57:11.336 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:57:11.336 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.9605337819999997, 2024-09-14T23:57:11.336 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0060596740000000001, 2024-09-14T23:57:11.337 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:11.337 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:11.337 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:11.337 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:11.337 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.337 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:11.337 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:57:11.337 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.338 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.338 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:11.338 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:57:11.338 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.338 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.338 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.338 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:11.338 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:57:11.339 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2200000000000003e-07 2024-09-14T23:57:11.339 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.339 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.339 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:11.339 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279995+0000", 2024-09-14T23:57:11.339 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5569999999999997e-06 2024-09-14T23:57:11.339 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.339 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.339 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:11.339 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280004+0000", 2024-09-14T23:57:11.339 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0080000000000008e-06 2024-09-14T23:57:11.339 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.340 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.340 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:11.340 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280010+0000", 2024-09-14T23:57:11.340 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5749999999999997e-06 2024-09-14T23:57:11.340 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.340 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.340 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:11.340 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282968+0000", 2024-09-14T23:57:11.340 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0029579390000000001 2024-09-14T23:57:11.340 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.340 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.340 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:11.341 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282998+0000", 2024-09-14T23:57:11.341 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9947999999999999e-05 2024-09-14T23:57:11.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.341 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.341 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:11.341 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.286012+0000", 2024-09-14T23:57:11.341 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030132900000000001 2024-09-14T23:57:11.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.341 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.341 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:11.341 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.286049+0000", 2024-09-14T23:57:11.342 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7534999999999999e-05 2024-09-14T23:57:11.342 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.342 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:11.342 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.342 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.342 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.342 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56382 3.1d e56/49 3:bcf9a8cf:::rbd_data.1118c4b3aebe.0000000000000006:head v 56'212, mlcod=56'208)", 2024-09-14T23:57:11.342 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.280285+0000", 2024-09-14T23:57:11.342 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.9602380989999997, 2024-09-14T23:57:11.342 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035393489999999998, 2024-09-14T23:57:11.342 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:11.343 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:11.343 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:11.343 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:11.343 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.343 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:11.343 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280285+0000", 2024-09-14T23:57:11.343 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.343 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.343 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.343 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:11.343 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280285+0000", 2024-09-14T23:57:11.344 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.344 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.344 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.344 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:11.344 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280286+0000", 2024-09-14T23:57:11.344 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5000000000000001e-07 2024-09-14T23:57:11.344 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.344 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.344 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:11.344 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280299+0000", 2024-09-14T23:57:11.344 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3312000000000001e-05 2024-09-14T23:57:11.345 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.345 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.345 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:11.345 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280311+0000", 2024-09-14T23:57:11.345 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1677e-05 2024-09-14T23:57:11.345 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.345 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.345 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:11.345 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280318+0000", 2024-09-14T23:57:11.345 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.742e-06 2024-09-14T23:57:11.345 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.345 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.346 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:11.346 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280374+0000", 2024-09-14T23:57:11.346 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.592e-05 2024-09-14T23:57:11.346 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.346 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.346 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:11.346 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280392+0000", 2024-09-14T23:57:11.346 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8490000000000001e-05 2024-09-14T23:57:11.346 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.346 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.347 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:11.347 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.283693+0000", 2024-09-14T23:57:11.347 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033012050000000002 2024-09-14T23:57:11.347 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.347 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.347 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:11.347 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.283825+0000", 2024-09-14T23:57:11.347 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013105300000000001 2024-09-14T23:57:11.347 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.347 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:11.347 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.347 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.348 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.348 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "pg_update_log_missing(3.c epoch 56/49 rep_tid 1493 entries 56'200 (0'0) error 3:370203a3:::rbd_data.1118c4b3aebe.000000000000000d:head by client.4376.0:56396 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 56'192)", 2024-09-14T23:57:11.348 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:57:11.348 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.9600136419999998, 2024-09-14T23:57:11.348 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057370099999999999, 2024-09-14T23:57:11.348 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:11.348 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:11.348 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:57:11.348 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:11.348 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.348 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:11.349 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:57:11.349 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.349 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.349 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:11.349 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:57:11.349 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.349 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.349 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:11.350 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:57:11.350 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0699999999999997e-07 2024-09-14T23:57:11.350 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.350 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.350 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:11.350 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280519+0000", 2024-09-14T23:57:11.350 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6260000000000001e-06 2024-09-14T23:57:11.350 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.350 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.350 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:11.350 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280541+0000", 2024-09-14T23:57:11.351 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1976000000000001e-05 2024-09-14T23:57:11.351 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.351 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.351 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:11.351 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280556+0000", 2024-09-14T23:57:11.351 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5075000000000001e-05 2024-09-14T23:57:11.351 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.351 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.351 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:11.351 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.285494+0000", 2024-09-14T23:57:11.351 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0049378690000000001 2024-09-14T23:57:11.351 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.352 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.352 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:11.352 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.286247+0000", 2024-09-14T23:57:11.352 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00075285700000000001 2024-09-14T23:57:11.352 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.352 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:11.352 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.352 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.352 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.352 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56400 3.1c e56/49 3:3f5b81ba:::rbd_data.1118c4b3aebe.000000000000000a:head v 56'236, mlcod=56'234)", 2024-09-14T23:57:11.352 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.295064+0000", 2024-09-14T23:57:11.353 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.9454588499999996, 2024-09-14T23:57:11.353 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0054331589999999999, 2024-09-14T23:57:11.353 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:11.353 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:11.353 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:11.353 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:11.353 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.353 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:11.353 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.295064+0000", 2024-09-14T23:57:11.353 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.353 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.353 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.354 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:11.354 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.295064+0000", 2024-09-14T23:57:11.354 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.354 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.354 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.354 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:11.354 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.295066+0000", 2024-09-14T23:57:11.354 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.502e-06 2024-09-14T23:57:11.354 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.354 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.354 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:11.354 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296403+0000", 2024-09-14T23:57:11.355 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001336554 2024-09-14T23:57:11.355 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.355 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.355 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:11.355 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296428+0000", 2024-09-14T23:57:11.355 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5179000000000001e-05 2024-09-14T23:57:11.355 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.355 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.355 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:11.355 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296440+0000", 2024-09-14T23:57:11.355 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2387e-05 2024-09-14T23:57:11.355 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.356 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.356 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:11.356 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296554+0000", 2024-09-14T23:57:11.356 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114269 2024-09-14T23:57:11.356 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.356 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.356 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:11.356 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296581+0000", 2024-09-14T23:57:11.356 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6673999999999999e-05 2024-09-14T23:57:11.356 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.356 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.357 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:11.357 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.300418+0000", 2024-09-14T23:57:11.357 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0038371249999999998 2024-09-14T23:57:11.357 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.357 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.357 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:11.357 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.300498+0000", 2024-09-14T23:57:11.357 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9468999999999994e-05 2024-09-14T23:57:11.357 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.358 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:11.358 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.358 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.358 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.358 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56401 3.1c e56/49 3:3f5b81ba:::rbd_data.1118c4b3aebe.000000000000000a:head v 56'238, mlcod=56'234)", 2024-09-14T23:57:11.358 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.297640+0000", 2024-09-14T23:57:11.358 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.9428830110000002, 2024-09-14T23:57:11.358 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031491919999999999, 2024-09-14T23:57:11.358 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:11.358 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:11.358 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:11.359 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:11.359 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.359 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:11.359 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.297640+0000", 2024-09-14T23:57:11.359 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.359 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.359 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.359 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:11.359 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.297640+0000", 2024-09-14T23:57:11.360 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.360 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.360 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.360 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:11.360 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.297641+0000", 2024-09-14T23:57:11.360 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9000000000000005e-07 2024-09-14T23:57:11.360 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.360 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.360 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:11.360 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.297649+0000", 2024-09-14T23:57:11.361 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4510000000000003e-06 2024-09-14T23:57:11.361 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.361 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.361 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:11.361 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.297663+0000", 2024-09-14T23:57:11.361 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4262e-05 2024-09-14T23:57:11.361 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.361 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.361 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:11.361 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.297679+0000", 2024-09-14T23:57:11.362 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5798e-05 2024-09-14T23:57:11.362 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.362 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.362 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:11.362 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.298408+0000", 2024-09-14T23:57:11.362 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00072923699999999998 2024-09-14T23:57:11.362 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.362 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.362 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:11.362 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.298430+0000", 2024-09-14T23:57:11.362 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1753999999999999e-05 2024-09-14T23:57:11.363 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.363 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.363 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:11.363 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.300749+0000", 2024-09-14T23:57:11.363 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023190590000000001 2024-09-14T23:57:11.363 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.363 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.363 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:11.363 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.300789+0000", 2024-09-14T23:57:11.363 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0640999999999999e-05 2024-09-14T23:57:11.363 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.364 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:11.364 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.364 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.364 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.364 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:61921 3.5 e57/49 3:a05d004f:::rbd_data.11188d356229.000000000000000b:head v 57'279, mlcod=57'277)", 2024-09-14T23:57:11.364 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:07.560389+0000", 2024-09-14T23:57:11.364 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.6801342520000002, 2024-09-14T23:57:11.364 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0034956520000000001, 2024-09-14T23:57:11.364 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:11.364 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:11.365 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:11.365 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:11.365 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.365 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:11.365 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.560389+0000", 2024-09-14T23:57:11.365 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.365 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.365 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.365 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:11.365 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.560389+0000", 2024-09-14T23:57:11.365 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.365 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.366 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.366 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:11.366 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.560390+0000", 2024-09-14T23:57:11.366 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4109999999999999e-06 2024-09-14T23:57:11.366 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.366 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.366 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:11.366 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.561839+0000", 2024-09-14T23:57:11.366 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001448406 2024-09-14T23:57:11.366 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.367 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.367 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:11.367 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.561854+0000", 2024-09-14T23:57:11.367 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5426000000000001e-05 2024-09-14T23:57:11.367 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.367 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.367 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:11.367 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.561863+0000", 2024-09-14T23:57:11.368 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7509999999999997e-06 2024-09-14T23:57:11.368 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.368 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.368 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:11.368 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.561917+0000", 2024-09-14T23:57:11.368 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3531e-05 2024-09-14T23:57:11.368 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.368 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.368 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:11.368 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.561944+0000", 2024-09-14T23:57:11.368 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7528000000000001e-05 2024-09-14T23:57:11.368 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.369 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.369 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:11.369 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563843+0000", 2024-09-14T23:57:11.369 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001899361 2024-09-14T23:57:11.369 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.369 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.369 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:11.369 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563885+0000", 2024-09-14T23:57:11.369 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1238000000000001e-05 2024-09-14T23:57:11.369 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.369 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:11.369 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.370 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.370 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.370 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:61922 3.5 e57/49 3:a11ebf8e:::rbd_data.11188d356229.000000000000000c:head v 57'280, mlcod=57'277)", 2024-09-14T23:57:11.370 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:07.563144+0000", 2024-09-14T23:57:11.370 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.6773791980000001, 2024-09-14T23:57:11.370 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003977848, 2024-09-14T23:57:11.370 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:11.370 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:11.370 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:11.370 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:11.370 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.370 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:11.370 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563144+0000", 2024-09-14T23:57:11.371 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.371 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.371 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.371 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:11.371 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563144+0000", 2024-09-14T23:57:11.371 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.371 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.371 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.371 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:11.371 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563145+0000", 2024-09-14T23:57:11.371 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5400000000000001e-07 2024-09-14T23:57:11.371 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.371 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.371 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:11.372 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565056+0000", 2024-09-14T23:57:11.372 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0019108160000000001 2024-09-14T23:57:11.372 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.372 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.372 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:11.372 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565088+0000", 2024-09-14T23:57:11.372 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2586999999999998e-05 2024-09-14T23:57:11.372 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.372 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.372 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:11.372 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565100+0000", 2024-09-14T23:57:11.373 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1917e-05 2024-09-14T23:57:11.373 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.373 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.373 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:11.373 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565166+0000", 2024-09-14T23:57:11.373 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6245e-05 2024-09-14T23:57:11.373 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.373 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.373 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:11.373 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565190+0000", 2024-09-14T23:57:11.373 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3825999999999999e-05 2024-09-14T23:57:11.374 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.374 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.374 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:11.374 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.567080+0000", 2024-09-14T23:57:11.374 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001889541 2024-09-14T23:57:11.374 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.374 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.374 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:11.374 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.567122+0000", 2024-09-14T23:57:11.374 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2262000000000003e-05 2024-09-14T23:57:11.374 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.374 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:11.375 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.375 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.375 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:11.375 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:11.376 INFO:teuthology.orchestra.run.smithi042.stdout:1957 write 0x4e1a0c9 thru 0x4e1ff59 (0x5e91 bytes) 2024-09-14T23:57:11.376 INFO:teuthology.orchestra.run.smithi042.stdout:1958 clone 21 order 24 su 8388608 sc 14 2024-09-14T23:57:11.377 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T23:57:11.377 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:11.375+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:11.478 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:11.476+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:11.525 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:11.525 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:11.525 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:57:11.525 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:11.533 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T23:57:11.617 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:11.616+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:11.680 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:11.678+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:11.680 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:11.680 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:11.680 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:57:11.680 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:57:11.680 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:11.689 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T23:57:11.781 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:11.779+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:11.838 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:11.838 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:57:11.838 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:57:11.839 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:57:11.839 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.839 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:18218 3.10 3:0c729a68:::rbd_data.11187be34cc8.0000000000000017:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-14T23:57:11.839 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:44.032604+0000", 2024-09-14T23:57:11.839 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.803644324, 2024-09-14T23:57:11.839 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.028735283, 2024-09-14T23:57:11.839 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:11.839 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:11.839 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:11.839 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:11.839 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:11.840 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:11.840 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 18218 2024-09-14T23:57:11.840 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.840 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:11.840 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.840 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:11.840 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.032604+0000", 2024-09-14T23:57:11.840 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.840 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.840 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.840 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:11.840 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.032604+0000", 2024-09-14T23:57:11.841 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.841 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.841 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.841 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:11.841 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.032606+0000", 2024-09-14T23:57:11.841 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7999999999999999e-06 2024-09-14T23:57:11.841 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.841 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.841 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:11.841 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.032614+0000", 2024-09-14T23:57:11.841 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1810000000000001e-06 2024-09-14T23:57:11.841 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.841 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.842 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:11.842 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.032631+0000", 2024-09-14T23:57:11.842 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7363e-05 2024-09-14T23:57:11.842 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.842 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.842 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:11.842 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.032638+0000", 2024-09-14T23:57:11.842 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0430000000000002e-06 2024-09-14T23:57:11.842 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.842 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.842 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:11.842 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.032708+0000", 2024-09-14T23:57:11.842 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0257000000000002e-05 2024-09-14T23:57:11.842 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.843 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.843 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-14T23:57:11.843 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.032713+0000", 2024-09-14T23:57:11.843 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4190000000000004e-06 2024-09-14T23:57:11.843 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.843 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.843 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:11.843 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.034010+0000", 2024-09-14T23:57:11.844 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001297407 2024-09-14T23:57:11.844 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.844 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.844 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:11.844 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.034059+0000", 2024-09-14T23:57:11.844 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8912999999999997e-05 2024-09-14T23:57:11.844 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.844 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.844 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:57:11.844 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.038995+0000", 2024-09-14T23:57:11.844 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0049356010000000004 2024-09-14T23:57:11.845 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.845 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.845 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:11.845 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.061258+0000", 2024-09-14T23:57:11.845 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.022262963 2024-09-14T23:57:11.845 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.846 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.846 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:57:11.846 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.061269+0000", 2024-09-14T23:57:11.846 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1671000000000001e-05 2024-09-14T23:57:11.846 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.846 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.846 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:11.846 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.061306+0000", 2024-09-14T23:57:11.846 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6474999999999999e-05 2024-09-14T23:57:11.846 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.846 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.846 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:11.846 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:44.061339+0000", 2024-09-14T23:57:11.847 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3189999999999999e-05 2024-09-14T23:57:11.847 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.847 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:11.847 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.847 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.847 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.847 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30289 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 585728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:57:11.847 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.865354+0000", 2024-09-14T23:57:11.847 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.970894497, 2024-09-14T23:57:11.847 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015192881, 2024-09-14T23:57:11.847 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:11.847 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:11.847 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:11.847 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:11.848 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:11.848 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:11.848 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30289 2024-09-14T23:57:11.848 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.848 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:11.848 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.848 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:11.848 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865354+0000", 2024-09-14T23:57:11.848 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.848 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.849 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.849 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:11.849 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865354+0000", 2024-09-14T23:57:11.849 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.849 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.849 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.849 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:11.849 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865354+0000", 2024-09-14T23:57:11.849 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8500000000000001e-07 2024-09-14T23:57:11.849 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.849 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.849 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:11.850 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865357+0000", 2024-09-14T23:57:11.850 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6199999999999999e-06 2024-09-14T23:57:11.850 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.850 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.850 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:11.850 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865364+0000", 2024-09-14T23:57:11.850 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0840000000000003e-06 2024-09-14T23:57:11.850 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.850 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.850 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:11.850 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865369+0000", 2024-09-14T23:57:11.850 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3709999999999999e-06 2024-09-14T23:57:11.850 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.850 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.851 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:11.851 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.880378+0000", 2024-09-14T23:57:11.851 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015009043 2024-09-14T23:57:11.851 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.851 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.851 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:11.851 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.880451+0000", 2024-09-14T23:57:11.851 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2747e-05 2024-09-14T23:57:11.851 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.851 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.851 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:11.851 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.880546+0000", 2024-09-14T23:57:11.851 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5431000000000002e-05 2024-09-14T23:57:11.851 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.852 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:11.852 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.852 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.852 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.852 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30298 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 438272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:57:11.852 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.865377+0000", 2024-09-14T23:57:11.852 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.970871029000001, 2024-09-14T23:57:11.852 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015369668, 2024-09-14T23:57:11.852 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:11.852 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:11.852 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:11.852 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:11.852 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:11.852 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:11.853 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30298 2024-09-14T23:57:11.853 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.853 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:11.853 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.853 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:11.853 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865377+0000", 2024-09-14T23:57:11.853 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.853 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.853 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.853 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:11.853 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865377+0000", 2024-09-14T23:57:11.853 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.853 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.853 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.854 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:11.854 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865377+0000", 2024-09-14T23:57:11.854 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9499999999999998e-07 2024-09-14T23:57:11.854 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.854 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.854 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:11.854 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865381+0000", 2024-09-14T23:57:11.854 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.371e-06 2024-09-14T23:57:11.854 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.854 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.854 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:11.854 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865389+0000", 2024-09-14T23:57:11.854 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3469999999999999e-06 2024-09-14T23:57:11.854 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.854 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.855 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:11.855 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865394+0000", 2024-09-14T23:57:11.855 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9869999999999999e-06 2024-09-14T23:57:11.855 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.855 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.855 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:11.855 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.880577+0000", 2024-09-14T23:57:11.855 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015183103 2024-09-14T23:57:11.855 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.855 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.855 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:11.856 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.880649+0000", 2024-09-14T23:57:11.856 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1989999999999993e-05 2024-09-14T23:57:11.856 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.856 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.856 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:11.856 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.880747+0000", 2024-09-14T23:57:11.856 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7474999999999998e-05 2024-09-14T23:57:11.856 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.856 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:11.856 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.856 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.856 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.856 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30302 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 364544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:57:11.856 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.865401+0000", 2024-09-14T23:57:11.857 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.970847124999999, 2024-09-14T23:57:11.857 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015547874999999999, 2024-09-14T23:57:11.857 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:11.857 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:11.857 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:11.857 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:11.857 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:11.857 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:11.857 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30302 2024-09-14T23:57:11.857 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.858 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:11.858 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.858 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:11.858 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865401+0000", 2024-09-14T23:57:11.858 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.858 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.858 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.858 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:11.858 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865401+0000", 2024-09-14T23:57:11.858 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.858 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.858 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.858 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:11.858 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865402+0000", 2024-09-14T23:57:11.858 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4499999999999997e-07 2024-09-14T23:57:11.859 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.859 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.859 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:11.859 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865404+0000", 2024-09-14T23:57:11.859 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9240000000000001e-06 2024-09-14T23:57:11.859 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.859 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.859 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:11.859 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865410+0000", 2024-09-14T23:57:11.859 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3570000000000003e-06 2024-09-14T23:57:11.859 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.859 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.859 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:11.859 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865415+0000", 2024-09-14T23:57:11.860 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.553e-06 2024-09-14T23:57:11.860 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.860 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.860 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:11.860 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.880779+0000", 2024-09-14T23:57:11.860 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015363411 2024-09-14T23:57:11.860 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.860 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.860 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:11.860 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.880852+0000", 2024-09-14T23:57:11.860 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2916000000000005e-05 2024-09-14T23:57:11.861 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.861 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.861 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:11.861 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.880949+0000", 2024-09-14T23:57:11.861 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7069000000000002e-05 2024-09-14T23:57:11.861 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.861 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:11.861 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.861 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.861 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.862 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30308 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 294912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:57:11.862 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.865537+0000", 2024-09-14T23:57:11.862 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.970711514000001, 2024-09-14T23:57:11.862 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015615162, 2024-09-14T23:57:11.862 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:11.862 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:11.862 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:11.862 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:11.862 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:11.862 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:11.863 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30308 2024-09-14T23:57:11.863 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.863 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:11.863 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.863 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:11.863 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865537+0000", 2024-09-14T23:57:11.863 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.863 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.863 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.863 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:11.864 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865537+0000", 2024-09-14T23:57:11.864 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.864 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.864 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.864 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:11.864 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865537+0000", 2024-09-14T23:57:11.864 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0299999999999999e-07 2024-09-14T23:57:11.864 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.864 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.864 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:11.864 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865540+0000", 2024-09-14T23:57:11.864 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8420000000000001e-06 2024-09-14T23:57:11.865 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.865 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.865 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:11.865 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865548+0000", 2024-09-14T23:57:11.865 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2230000000000007e-06 2024-09-14T23:57:11.865 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.865 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.865 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:11.865 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.865555+0000", 2024-09-14T23:57:11.865 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7780000000000002e-06 2024-09-14T23:57:11.865 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.866 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.866 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:11.866 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.880982+0000", 2024-09-14T23:57:11.866 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015426832 2024-09-14T23:57:11.866 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.866 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.866 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:11.867 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.881059+0000", 2024-09-14T23:57:11.867 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7095999999999995e-05 2024-09-14T23:57:11.867 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.867 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.867 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:11.867 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.881152+0000", 2024-09-14T23:57:11.867 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2788000000000007e-05 2024-09-14T23:57:11.868 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.868 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:11.868 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.868 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.868 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.868 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30371 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 442368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:57:11.868 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.866343+0000", 2024-09-14T23:57:11.869 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.969905036, 2024-09-14T23:57:11.869 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015209663, 2024-09-14T23:57:11.869 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:11.869 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:11.869 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:11.869 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:11.870 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:11.870 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:11.870 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30371 2024-09-14T23:57:11.870 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.870 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:11.870 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.871 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:11.871 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866343+0000", 2024-09-14T23:57:11.871 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.871 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.871 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.871 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:11.872 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866343+0000", 2024-09-14T23:57:11.872 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.872 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.872 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.872 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:11.872 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866344+0000", 2024-09-14T23:57:11.873 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8199999999999999e-07 2024-09-14T23:57:11.873 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.873 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.873 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:11.873 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866347+0000", 2024-09-14T23:57:11.873 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4759999999999998e-06 2024-09-14T23:57:11.874 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.874 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.874 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:11.874 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866358+0000", 2024-09-14T23:57:11.874 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0353999999999999e-05 2024-09-14T23:57:11.874 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.874 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.875 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:11.875 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866364+0000", 2024-09-14T23:57:11.875 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3260000000000001e-06 2024-09-14T23:57:11.875 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.875 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.875 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:11.876 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.881388+0000", 2024-09-14T23:57:11.876 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015023791999999999 2024-09-14T23:57:11.876 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.876 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.876 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:11.876 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.881460+0000", 2024-09-14T23:57:11.877 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2509999999999995e-05 2024-09-14T23:57:11.877 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.877 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.877 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:11.877 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.881553+0000", 2024-09-14T23:57:11.877 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2423000000000006e-05 2024-09-14T23:57:11.877 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.878 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:11.878 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.878 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.878 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.878 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30374 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 368640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:57:11.878 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.866381+0000", 2024-09-14T23:57:11.878 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.969866615000001, 2024-09-14T23:57:11.878 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015382209000000001, 2024-09-14T23:57:11.879 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:11.879 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:11.879 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:11.879 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:11.879 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:11.880 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:11.880 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30374 2024-09-14T23:57:11.880 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.880 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:11.880 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.880 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:11.880 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866381+0000", 2024-09-14T23:57:11.881 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.881 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.881 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.881 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:11.881 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866381+0000", 2024-09-14T23:57:11.881 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.881 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.882 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.882 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:11.882 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866382+0000", 2024-09-14T23:57:11.882 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9800000000000003e-07 2024-09-14T23:57:11.882 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.882 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.882 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:11.882 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866385+0000", 2024-09-14T23:57:11.883 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1599999999999998e-06 2024-09-14T23:57:11.883 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.883 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.883 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:11.883 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866393+0000", 2024-09-14T23:57:11.883 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7929999999999998e-06 2024-09-14T23:57:11.884 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.884 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.884 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:11.884 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866399+0000", 2024-09-14T23:57:11.884 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5260000000000001e-06 2024-09-14T23:57:11.884 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.884 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.884 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:11.884 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.881585+0000", 2024-09-14T23:57:11.884 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.01518624 2024-09-14T23:57:11.885 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.885 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.885 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:11.885 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.881661+0000", 2024-09-14T23:57:11.885 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5779999999999996e-05 2024-09-14T23:57:11.885 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.885 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.886 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:11.886 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.881764+0000", 2024-09-14T23:57:11.886 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000103012 2024-09-14T23:57:11.886 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.886 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:11.886 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.886 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.887 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30379 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 299008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:57:11.887 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.866461+0000", 2024-09-14T23:57:11.887 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.969787104000002, 2024-09-14T23:57:11.887 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015514637, 2024-09-14T23:57:11.887 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:11.887 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:11.887 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:11.887 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:11.888 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:11.888 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:11.888 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30379 2024-09-14T23:57:11.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.888 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:11.888 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.888 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:11.888 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866461+0000", 2024-09-14T23:57:11.889 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.889 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.889 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:11.889 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866461+0000", 2024-09-14T23:57:11.889 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.890 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.890 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:11.890 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866462+0000", 2024-09-14T23:57:11.890 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5700000000000002e-07 2024-09-14T23:57:11.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.890 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.890 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:11.891 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866466+0000", 2024-09-14T23:57:11.891 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6759999999999998e-06 2024-09-14T23:57:11.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.891 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.891 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:11.891 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866477+0000", 2024-09-14T23:57:11.891 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0094e-05 2024-09-14T23:57:11.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.892 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.892 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:11.892 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866483+0000", 2024-09-14T23:57:11.892 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1460000000000001e-06 2024-09-14T23:57:11.892 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.892 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.892 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:11.893 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.881800+0000", 2024-09-14T23:57:11.893 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015317769 2024-09-14T23:57:11.893 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.893 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.893 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:11.893 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.881875+0000", 2024-09-14T23:57:11.893 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4726000000000002e-05 2024-09-14T23:57:11.893 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.894 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.894 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:11.894 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.881976+0000", 2024-09-14T23:57:11.894 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100469 2024-09-14T23:57:11.894 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.894 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:11.894 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.894 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.895 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.895 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30396 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 532480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:57:11.895 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.866569+0000", 2024-09-14T23:57:11.895 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.969678681000001, 2024-09-14T23:57:11.895 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015617786, 2024-09-14T23:57:11.895 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:11.895 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:11.895 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:11.895 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:11.896 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:11.896 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:11.896 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30396 2024-09-14T23:57:11.896 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.896 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:11.896 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.896 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:11.897 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866569+0000", 2024-09-14T23:57:11.897 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.897 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.897 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:11.897 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866569+0000", 2024-09-14T23:57:11.897 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.897 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.897 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:11.897 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866570+0000", 2024-09-14T23:57:11.897 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4000000000000001e-07 2024-09-14T23:57:11.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.898 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.898 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:11.898 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866574+0000", 2024-09-14T23:57:11.898 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7330000000000001e-06 2024-09-14T23:57:11.898 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.898 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.898 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:11.898 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866584+0000", 2024-09-14T23:57:11.898 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0052e-05 2024-09-14T23:57:11.898 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.898 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.898 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:11.898 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866591+0000", 2024-09-14T23:57:11.898 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4359999999999996e-06 2024-09-14T23:57:11.899 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.899 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.899 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:11.899 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.882011+0000", 2024-09-14T23:57:11.899 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015419871999999999 2024-09-14T23:57:11.899 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.899 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.899 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:11.899 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.882088+0000", 2024-09-14T23:57:11.899 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7118000000000003e-05 2024-09-14T23:57:11.899 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.899 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.900 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:11.900 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.882187+0000", 2024-09-14T23:57:11.900 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8834999999999998e-05 2024-09-14T23:57:11.900 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.900 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:11.900 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.900 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.900 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.900 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30405 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 446464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:57:11.900 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.866670+0000", 2024-09-14T23:57:11.900 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.969577909000002, 2024-09-14T23:57:11.900 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015777653999999999, 2024-09-14T23:57:11.900 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:11.901 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:11.901 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:11.901 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:11.901 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:11.901 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:11.901 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30405 2024-09-14T23:57:11.901 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.901 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:11.901 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.901 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:11.901 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866670+0000", 2024-09-14T23:57:11.901 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.901 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.901 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.902 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:11.902 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866670+0000", 2024-09-14T23:57:11.902 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.902 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.902 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:11.902 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866671+0000", 2024-09-14T23:57:11.902 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3900000000000004e-07 2024-09-14T23:57:11.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.902 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.902 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:11.902 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866675+0000", 2024-09-14T23:57:11.902 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8569999999999997e-06 2024-09-14T23:57:11.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.902 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.903 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:11.903 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866685+0000", 2024-09-14T23:57:11.903 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0002e-05 2024-09-14T23:57:11.903 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.903 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.903 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:11.903 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866692+0000", 2024-09-14T23:57:11.903 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.691e-06 2024-09-14T23:57:11.903 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.903 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.903 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:11.903 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.882259+0000", 2024-09-14T23:57:11.903 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015567399000000001 2024-09-14T23:57:11.903 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.903 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.904 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:11.904 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.882342+0000", 2024-09-14T23:57:11.904 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2919999999999999e-05 2024-09-14T23:57:11.904 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.904 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.904 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:11.904 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.882448+0000", 2024-09-14T23:57:11.904 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010594600000000001 2024-09-14T23:57:11.904 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.904 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:11.904 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.904 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.904 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.904 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30407 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 372736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:57:11.905 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.866702+0000", 2024-09-14T23:57:11.905 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.96954569, 2024-09-14T23:57:11.905 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015922993, 2024-09-14T23:57:11.905 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:11.905 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:11.905 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:11.905 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:11.905 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:11.905 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:11.905 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30407 2024-09-14T23:57:11.905 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.905 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:11.905 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.905 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:11.905 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866702+0000", 2024-09-14T23:57:11.906 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.906 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.906 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:11.906 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866702+0000", 2024-09-14T23:57:11.906 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.906 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.906 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:11.906 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866703+0000", 2024-09-14T23:57:11.906 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5799999999999999e-07 2024-09-14T23:57:11.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.906 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.907 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:11.907 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866706+0000", 2024-09-14T23:57:11.907 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8890000000000002e-06 2024-09-14T23:57:11.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.907 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.907 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:11.907 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866713+0000", 2024-09-14T23:57:11.907 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.99e-06 2024-09-14T23:57:11.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.907 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.907 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:11.907 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866718+0000", 2024-09-14T23:57:11.907 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.254e-06 2024-09-14T23:57:11.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.907 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.908 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:11.908 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.882497+0000", 2024-09-14T23:57:11.908 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015778664000000001 2024-09-14T23:57:11.908 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.908 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.908 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:11.908 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.882557+0000", 2024-09-14T23:57:11.908 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0408999999999997e-05 2024-09-14T23:57:11.908 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.909 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.909 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:11.909 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.882625+0000", 2024-09-14T23:57:11.909 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8229e-05 2024-09-14T23:57:11.909 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.909 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:11.909 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.909 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.909 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.909 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30415 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 303104~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:57:11.909 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.866775+0000", 2024-09-14T23:57:11.909 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.969473271999998, 2024-09-14T23:57:11.910 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.016119712000000001, 2024-09-14T23:57:11.910 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:11.910 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:11.910 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:11.910 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:11.910 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:11.910 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:11.910 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30415 2024-09-14T23:57:11.910 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.910 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:11.910 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.910 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:11.910 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866775+0000", 2024-09-14T23:57:11.910 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.911 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.911 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.911 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:11.911 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866775+0000", 2024-09-14T23:57:11.911 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.911 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.911 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.911 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:11.911 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866776+0000", 2024-09-14T23:57:11.911 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5499999999999997e-07 2024-09-14T23:57:11.911 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.911 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.911 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:11.911 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866779+0000", 2024-09-14T23:57:11.911 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.613e-06 2024-09-14T23:57:11.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.912 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.912 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:11.912 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866789+0000", 2024-09-14T23:57:11.912 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5030000000000003e-06 2024-09-14T23:57:11.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.912 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.912 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:11.912 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.866799+0000", 2024-09-14T23:57:11.912 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9170000000000001e-06 2024-09-14T23:57:11.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.912 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.912 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:11.912 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.882704+0000", 2024-09-14T23:57:11.913 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015905333000000001 2024-09-14T23:57:11.913 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.913 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.913 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:11.913 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.882787+0000", 2024-09-14T23:57:11.913 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3264999999999996e-05 2024-09-14T23:57:11.913 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.913 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.913 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:11.913 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.882895+0000", 2024-09-14T23:57:11.913 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010722600000000001 2024-09-14T23:57:11.913 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.913 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:11.913 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.914 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.914 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.914 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30430 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 536576~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:57:11.914 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.867043+0000", 2024-09-14T23:57:11.914 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.969205305999999, 2024-09-14T23:57:11.914 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.016054556000000001, 2024-09-14T23:57:11.914 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:11.914 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:11.914 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:11.914 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:11.914 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:11.914 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:11.915 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30430 2024-09-14T23:57:11.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.915 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:11.915 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.915 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:11.915 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867043+0000", 2024-09-14T23:57:11.915 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.915 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.915 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:11.915 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867043+0000", 2024-09-14T23:57:11.915 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.916 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.916 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:11.916 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867044+0000", 2024-09-14T23:57:11.916 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.23e-07 2024-09-14T23:57:11.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.916 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.916 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:11.916 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867053+0000", 2024-09-14T23:57:11.916 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7759999999999992e-06 2024-09-14T23:57:11.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.916 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.916 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:11.916 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867066+0000", 2024-09-14T23:57:11.916 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2376e-05 2024-09-14T23:57:11.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.917 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.917 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:11.917 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867075+0000", 2024-09-14T23:57:11.917 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1349999999999998e-06 2024-09-14T23:57:11.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.917 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.918 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:11.918 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.882925+0000", 2024-09-14T23:57:11.918 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015850566 2024-09-14T23:57:11.918 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.918 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.918 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:11.918 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.883000+0000", 2024-09-14T23:57:11.918 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4746999999999994e-05 2024-09-14T23:57:11.918 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.919 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.919 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:11.919 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.883097+0000", 2024-09-14T23:57:11.919 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7133000000000007e-05 2024-09-14T23:57:11.919 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.919 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:11.919 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.919 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.919 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30440 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 450560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:57:11.919 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.867174+0000", 2024-09-14T23:57:11.920 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.969073862999998, 2024-09-14T23:57:11.920 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.016169586999999999, 2024-09-14T23:57:11.920 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:11.920 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:11.920 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:11.920 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:11.920 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:11.920 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:11.920 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30440 2024-09-14T23:57:11.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.920 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:11.920 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.920 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:11.921 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867174+0000", 2024-09-14T23:57:11.921 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.921 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.921 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:11.921 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867174+0000", 2024-09-14T23:57:11.921 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.921 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.921 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:11.921 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867175+0000", 2024-09-14T23:57:11.921 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1100000000000005e-07 2024-09-14T23:57:11.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.921 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.922 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:11.922 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867178+0000", 2024-09-14T23:57:11.922 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.994e-06 2024-09-14T23:57:11.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.922 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.922 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:11.922 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867190+0000", 2024-09-14T23:57:11.922 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2072000000000001e-05 2024-09-14T23:57:11.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.922 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.922 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:11.922 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867196+0000", 2024-09-14T23:57:11.922 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1340000000000001e-06 2024-09-14T23:57:11.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.922 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.923 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:11.923 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.883165+0000", 2024-09-14T23:57:11.923 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015969061 2024-09-14T23:57:11.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.923 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.923 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:11.923 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.883240+0000", 2024-09-14T23:57:11.923 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4616999999999994e-05 2024-09-14T23:57:11.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.923 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.923 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:11.924 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.883344+0000", 2024-09-14T23:57:11.924 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000103898 2024-09-14T23:57:11.924 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.924 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:11.924 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.924 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.924 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30444 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 376832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:57:11.924 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.867242+0000", 2024-09-14T23:57:11.924 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.969006427, 2024-09-14T23:57:11.925 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.016313708999999999, 2024-09-14T23:57:11.925 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:11.925 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:11.925 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:11.925 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:11.925 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:11.925 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:11.925 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30444 2024-09-14T23:57:11.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.925 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:11.925 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.925 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:11.925 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867242+0000", 2024-09-14T23:57:11.925 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.926 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.926 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.926 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:11.926 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867242+0000", 2024-09-14T23:57:11.926 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.926 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.926 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.926 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:11.926 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867242+0000", 2024-09-14T23:57:11.926 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0100000000000005e-07 2024-09-14T23:57:11.926 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.927 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.927 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:11.927 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867248+0000", 2024-09-14T23:57:11.927 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0100000000000001e-06 2024-09-14T23:57:11.927 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.927 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.927 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:11.927 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867260+0000", 2024-09-14T23:57:11.927 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2179e-05 2024-09-14T23:57:11.927 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.927 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.927 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:11.927 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867267+0000", 2024-09-14T23:57:11.927 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7780000000000002e-06 2024-09-14T23:57:11.927 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.927 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.928 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:11.928 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.883374+0000", 2024-09-14T23:57:11.928 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.016106466999999999 2024-09-14T23:57:11.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.928 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.928 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:11.928 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.883449+0000", 2024-09-14T23:57:11.928 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5475000000000005e-05 2024-09-14T23:57:11.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.928 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.928 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:11.928 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.883555+0000", 2024-09-14T23:57:11.928 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000106199 2024-09-14T23:57:11.928 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.929 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:11.929 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.929 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.929 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.929 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30451 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 307200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:57:11.929 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.867309+0000", 2024-09-14T23:57:11.929 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.968938696999999, 2024-09-14T23:57:11.929 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.016427048, 2024-09-14T23:57:11.929 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:11.929 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:11.929 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:11.929 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:11.929 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:11.929 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:11.930 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30451 2024-09-14T23:57:11.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.930 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:11.930 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.930 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:11.930 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867309+0000", 2024-09-14T23:57:11.930 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.930 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.930 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:11.930 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867309+0000", 2024-09-14T23:57:11.930 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.930 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.930 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:11.930 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867310+0000", 2024-09-14T23:57:11.931 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5799999999999999e-07 2024-09-14T23:57:11.931 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.931 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.931 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:11.931 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867316+0000", 2024-09-14T23:57:11.931 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4650000000000002e-06 2024-09-14T23:57:11.931 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.931 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.931 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:11.931 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867325+0000", 2024-09-14T23:57:11.931 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0659999999999993e-06 2024-09-14T23:57:11.931 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.931 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.931 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:11.932 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867331+0000", 2024-09-14T23:57:11.932 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2419999999999997e-06 2024-09-14T23:57:11.932 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.932 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.932 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:11.932 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.883585+0000", 2024-09-14T23:57:11.932 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.016254599000000002 2024-09-14T23:57:11.932 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.932 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.932 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:11.932 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.883659+0000", 2024-09-14T23:57:11.932 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3697999999999998e-05 2024-09-14T23:57:11.932 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.933 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.933 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:11.933 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.883736+0000", 2024-09-14T23:57:11.933 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7319999999999998e-05 2024-09-14T23:57:11.933 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.933 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:11.933 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.933 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.933 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30463 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 540672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:57:11.933 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.867871+0000", 2024-09-14T23:57:11.933 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.968377361999998, 2024-09-14T23:57:11.933 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.016001520000000002, 2024-09-14T23:57:11.933 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:11.934 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:11.934 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:11.934 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:11.934 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:11.934 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:11.934 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30463 2024-09-14T23:57:11.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.934 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:11.934 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.934 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:11.934 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867871+0000", 2024-09-14T23:57:11.934 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.934 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:11.935 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867871+0000", 2024-09-14T23:57:11.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.935 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:11.935 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867872+0000", 2024-09-14T23:57:11.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9800000000000002e-07 2024-09-14T23:57:11.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.935 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:11.935 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867876+0000", 2024-09-14T23:57:11.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.318e-06 2024-09-14T23:57:11.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.935 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.936 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:11.936 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.867887+0000", 2024-09-14T23:57:11.936 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1055e-05 2024-09-14T23:57:11.936 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.936 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.936 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:11.936 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868679+0000", 2024-09-14T23:57:11.936 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00079199000000000001 2024-09-14T23:57:11.936 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.936 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.936 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:11.936 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.883763+0000", 2024-09-14T23:57:11.936 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015083675 2024-09-14T23:57:11.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.937 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.937 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:11.937 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.883816+0000", 2024-09-14T23:57:11.937 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3757e-05 2024-09-14T23:57:11.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.937 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.937 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:11.937 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.883872+0000", 2024-09-14T23:57:11.937 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5826999999999999e-05 2024-09-14T23:57:11.937 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.937 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:11.937 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.938 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.938 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.938 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30479 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 393216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:57:11.938 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.868703+0000", 2024-09-14T23:57:11.938 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.967545442999999, 2024-09-14T23:57:11.938 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015351293, 2024-09-14T23:57:11.938 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:11.938 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:11.938 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:11.938 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:11.938 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:11.938 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:11.939 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30479 2024-09-14T23:57:11.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.939 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:11.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.939 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:11.939 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868703+0000", 2024-09-14T23:57:11.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.939 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:11.939 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868703+0000", 2024-09-14T23:57:11.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:11.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868703+0000", 2024-09-14T23:57:11.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3300000000000001e-07 2024-09-14T23:57:11.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:11.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868708+0000", 2024-09-14T23:57:11.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.279e-06 2024-09-14T23:57:11.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:11.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868717+0000", 2024-09-14T23:57:11.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0469999999999999e-06 2024-09-14T23:57:11.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.941 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:11.941 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868723+0000", 2024-09-14T23:57:11.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.64e-06 2024-09-14T23:57:11.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.941 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:11.941 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.883898+0000", 2024-09-14T23:57:11.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015175016 2024-09-14T23:57:11.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.941 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:11.942 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.883959+0000", 2024-09-14T23:57:11.942 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0729000000000001e-05 2024-09-14T23:57:11.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.942 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.942 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:11.942 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.884054+0000", 2024-09-14T23:57:11.942 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4848999999999999e-05 2024-09-14T23:57:11.942 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.942 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:11.942 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.942 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.942 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30480 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 454656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:57:11.942 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.868733+0000", 2024-09-14T23:57:11.943 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.967514860000001, 2024-09-14T23:57:11.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015505417000000001, 2024-09-14T23:57:11.943 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:11.943 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:11.943 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:11.943 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:11.943 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:11.943 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:11.943 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30480 2024-09-14T23:57:11.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.943 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:11.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.943 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:11.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868733+0000", 2024-09-14T23:57:11.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:11.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868733+0000", 2024-09-14T23:57:11.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:11.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868734+0000", 2024-09-14T23:57:11.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3099999999999998e-07 2024-09-14T23:57:11.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:11.945 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868744+0000", 2024-09-14T23:57:11.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0429e-05 2024-09-14T23:57:11.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:11.945 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868754+0000", 2024-09-14T23:57:11.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5400000000000001e-06 2024-09-14T23:57:11.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:11.945 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868760+0000", 2024-09-14T23:57:11.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0170000000000002e-06 2024-09-14T23:57:11.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.946 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:11.946 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.884084+0000", 2024-09-14T23:57:11.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015324071 2024-09-14T23:57:11.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.946 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:11.946 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.884151+0000", 2024-09-14T23:57:11.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7531000000000001e-05 2024-09-14T23:57:11.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.946 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:11.946 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.884239+0000", 2024-09-14T23:57:11.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7398000000000006e-05 2024-09-14T23:57:11.947 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.947 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:11.947 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.947 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30483 3.a 3:55321864:::rbd_data.1118dd687b9b.0000000000000009:9 [read 380928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:57:11.947 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.868769+0000", 2024-09-14T23:57:11.947 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.967479312000002, 2024-09-14T23:57:11.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015654879, 2024-09-14T23:57:11.947 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:11.947 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:11.947 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:11.947 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:11.947 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:11.947 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:11.948 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30483 2024-09-14T23:57:11.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.948 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:11.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.948 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:11.948 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868769+0000", 2024-09-14T23:57:11.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.948 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:11.948 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868769+0000", 2024-09-14T23:57:11.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:11.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:11.949 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868769+0000", 2024-09-14T23:57:11.949 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8299999999999997e-07 2024-09-14T23:57:11.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:11.949 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868772+0000", 2024-09-14T23:57:11.949 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.497e-06 2024-09-14T23:57:11.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:11.949 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868778+0000", 2024-09-14T23:57:11.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6919999999999996e-06 2024-09-14T23:57:11.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.950 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:11.950 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.868784+0000", 2024-09-14T23:57:11.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5280000000000003e-06 2024-09-14T23:57:11.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.950 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:11.950 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.884264+0000", 2024-09-14T23:57:11.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015479629 2024-09-14T23:57:11.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.951 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:11.951 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.884332+0000", 2024-09-14T23:57:11.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8617999999999999e-05 2024-09-14T23:57:11.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:11.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:11.951 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:11.951 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.884424+0000", 2024-09-14T23:57:11.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1432000000000001e-05 2024-09-14T23:57:11.951 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.951 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:11.951 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.951 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:11.951 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:11.951 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:11.953 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T23:57:11.953 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:11.952+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:12.053 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:12.052+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:12.061 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:12.061 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:12.061 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:57:12.061 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:12.069 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T23:57:12.106 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T23:57:12.155 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:12.153+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:12.185 INFO:teuthology.orchestra.run.smithi042.stdout:leaving image image_client.0-clone20 intact 2024-09-14T23:57:12.225 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:12.225 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:12.225 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:57:12.225 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:57:12.225 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:12.231 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T23:57:12.256 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:12.254+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:12.287 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-09-14T23:57:12.288 INFO:teuthology.orchestra.run.smithi042.stdout:1960 punch from 0x54ba915 to 0x54c4a3f, (0xa12a bytes) 2024-09-14T23:57:12.291 INFO:teuthology.orchestra.run.smithi042.stdout:1962 read 0x33e2f1a thru 0x33eec49 (0xbd30 bytes) 2024-09-14T23:57:12.344 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:12.344 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:57:12.344 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:57:12.345 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:57:12.345 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.345 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30573 3.15 3:acd4c478:::rbd_data.1118fff5d5e0.0000000000000001:8 [read 3158016~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:57:12.345 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.872924+0000", 2024-09-14T23:57:12.345 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.468160710999999, 2024-09-14T23:57:12.346 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021329519000000002, 2024-09-14T23:57:12.346 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:12.346 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:12.346 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:12.346 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:12.346 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:12.346 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:12.346 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30573 2024-09-14T23:57:12.347 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.347 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:12.347 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.347 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:12.347 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872924+0000", 2024-09-14T23:57:12.347 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:12.347 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.347 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.347 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:12.348 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872924+0000", 2024-09-14T23:57:12.348 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:12.348 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.348 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.348 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:12.348 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872925+0000", 2024-09-14T23:57:12.348 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9999999999999998e-07 2024-09-14T23:57:12.348 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.348 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.348 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:12.349 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872927+0000", 2024-09-14T23:57:12.349 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.765e-06 2024-09-14T23:57:12.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.349 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.349 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:12.349 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872933+0000", 2024-09-14T23:57:12.349 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8610000000000003e-06 2024-09-14T23:57:12.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.349 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.349 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:12.350 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872937+0000", 2024-09-14T23:57:12.350 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5839999999999996e-06 2024-09-14T23:57:12.350 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.350 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.350 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:12.350 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.894117+0000", 2024-09-14T23:57:12.350 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021179429 2024-09-14T23:57:12.350 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.350 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.350 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:12.351 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.894170+0000", 2024-09-14T23:57:12.351 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3122999999999999e-05 2024-09-14T23:57:12.351 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.351 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.351 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:12.351 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.894254+0000", 2024-09-14T23:57:12.351 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4257000000000003e-05 2024-09-14T23:57:12.351 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:12.351 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:12.352 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:12.352 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.352 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.352 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30575 3.15 3:acd4c478:::rbd_data.1118fff5d5e0.0000000000000001:8 [read 3219456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:57:12.352 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.872943+0000", 2024-09-14T23:57:12.352 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.468142030999999, 2024-09-14T23:57:12.352 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021535109, 2024-09-14T23:57:12.352 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:12.352 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:12.353 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:12.353 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:12.353 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:12.353 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:12.353 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30575 2024-09-14T23:57:12.353 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.353 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:12.353 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.353 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:12.353 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872943+0000", 2024-09-14T23:57:12.354 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:12.354 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.354 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.354 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:12.354 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872943+0000", 2024-09-14T23:57:12.354 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:12.354 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.354 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.354 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:12.354 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872943+0000", 2024-09-14T23:57:12.355 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.72e-07 2024-09-14T23:57:12.355 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.355 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.355 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:12.355 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872945+0000", 2024-09-14T23:57:12.355 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.951e-06 2024-09-14T23:57:12.355 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.355 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.355 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:12.356 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872952+0000", 2024-09-14T23:57:12.356 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.268e-06 2024-09-14T23:57:12.356 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.356 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.356 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:12.356 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872956+0000", 2024-09-14T23:57:12.356 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8010000000000003e-06 2024-09-14T23:57:12.356 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.356 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.356 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:12.357 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.894295+0000", 2024-09-14T23:57:12.357 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021338475999999999 2024-09-14T23:57:12.357 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.357 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.357 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:12.357 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.894380+0000", 2024-09-14T23:57:12.357 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5222999999999994e-05 2024-09-14T23:57:12.357 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.358 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.358 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:12.358 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.894478+0000", 2024-09-14T23:57:12.358 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8017999999999996e-05 2024-09-14T23:57:12.358 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:12.358 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:12.359 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:12.359 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.359 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.359 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30577 3.15 3:acd4c478:::rbd_data.1118fff5d5e0.0000000000000001:8 [read 3334144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:57:12.359 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.872963+0000", 2024-09-14T23:57:12.359 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.468122227999999, 2024-09-14T23:57:12.360 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021659955000000002, 2024-09-14T23:57:12.360 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:12.360 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:12.360 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:12.360 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:12.361 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:12.361 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:12.361 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30577 2024-09-14T23:57:12.361 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.361 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:12.361 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.362 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:12.362 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872963+0000", 2024-09-14T23:57:12.362 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:12.362 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.362 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.362 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:12.363 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872963+0000", 2024-09-14T23:57:12.363 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:12.363 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.363 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.363 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:12.364 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872963+0000", 2024-09-14T23:57:12.364 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1899999999999998e-07 2024-09-14T23:57:12.364 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.364 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.364 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:12.365 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872966+0000", 2024-09-14T23:57:12.365 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.317e-06 2024-09-14T23:57:12.365 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.365 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.365 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:12.365 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872972+0000", 2024-09-14T23:57:12.366 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1859999999999997e-06 2024-09-14T23:57:12.366 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.366 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.366 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:12.366 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872977+0000", 2024-09-14T23:57:12.366 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9259999999999999e-06 2024-09-14T23:57:12.367 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.367 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.367 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:12.367 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.894517+0000", 2024-09-14T23:57:12.367 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021539776 2024-09-14T23:57:12.367 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.368 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.368 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:12.368 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.894565+0000", 2024-09-14T23:57:12.368 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8968999999999998e-05 2024-09-14T23:57:12.368 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.368 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.369 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:12.369 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.894623+0000", 2024-09-14T23:57:12.369 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7361999999999997e-05 2024-09-14T23:57:12.369 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:12.369 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:12.369 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:12.370 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.370 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.370 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30580 3.15 3:acd4c478:::rbd_data.1118fff5d5e0.0000000000000001:8 [read 3395584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:57:12.370 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.872983+0000", 2024-09-14T23:57:12.370 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.468102308999999, 2024-09-14T23:57:12.371 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021878231000000001, 2024-09-14T23:57:12.371 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:12.371 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:12.371 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:12.371 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:12.371 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:12.372 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:12.372 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30580 2024-09-14T23:57:12.372 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.372 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:12.372 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.372 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:12.373 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872983+0000", 2024-09-14T23:57:12.373 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:12.373 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.373 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.373 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:12.374 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872983+0000", 2024-09-14T23:57:12.374 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:12.374 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.374 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.374 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:12.374 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872983+0000", 2024-09-14T23:57:12.375 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2800000000000003e-07 2024-09-14T23:57:12.375 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.375 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.375 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:12.375 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872985+0000", 2024-09-14T23:57:12.375 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.804e-06 2024-09-14T23:57:12.375 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.376 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.376 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:12.376 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872992+0000", 2024-09-14T23:57:12.376 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3950000000000003e-06 2024-09-14T23:57:12.376 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.376 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.377 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:12.377 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.872998+0000", 2024-09-14T23:57:12.377 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1109999999999996e-06 2024-09-14T23:57:12.377 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.377 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.378 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:12.378 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.894679+0000", 2024-09-14T23:57:12.378 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021680095999999999 2024-09-14T23:57:12.378 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.378 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.378 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:12.379 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.894763+0000", 2024-09-14T23:57:12.379 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.475e-05 2024-09-14T23:57:12.379 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.379 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.379 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:12.379 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.894861+0000", 2024-09-14T23:57:12.380 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7746999999999998e-05 2024-09-14T23:57:12.380 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:12.380 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:12.380 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:12.380 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.381 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.381 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30584 3.15 3:acd4c478:::rbd_data.1118fff5d5e0.0000000000000001:8 [read 2932736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:57:12.381 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.873005+0000", 2024-09-14T23:57:12.381 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.46808017, 2024-09-14T23:57:12.381 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021964951, 2024-09-14T23:57:12.381 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:12.382 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:12.382 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:12.382 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:12.382 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:12.382 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:12.382 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30584 2024-09-14T23:57:12.383 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.383 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:12.383 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.383 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:12.383 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873005+0000", 2024-09-14T23:57:12.383 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:12.384 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.384 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.384 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:12.384 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873005+0000", 2024-09-14T23:57:12.384 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:12.384 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.384 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.384 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:12.385 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873005+0000", 2024-09-14T23:57:12.385 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6300000000000001e-07 2024-09-14T23:57:12.385 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.385 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.385 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:12.385 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873007+0000", 2024-09-14T23:57:12.385 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8670000000000001e-06 2024-09-14T23:57:12.385 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.386 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.386 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:12.386 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873013+0000", 2024-09-14T23:57:12.386 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5829999999999997e-06 2024-09-14T23:57:12.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.386 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.386 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:12.386 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873017+0000", 2024-09-14T23:57:12.387 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3669999999999999e-06 2024-09-14T23:57:12.387 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.387 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.387 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:12.387 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.894876+0000", 2024-09-14T23:57:12.387 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021858532 2024-09-14T23:57:12.387 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.387 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.388 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:12.388 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.894917+0000", 2024-09-14T23:57:12.388 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1485e-05 2024-09-14T23:57:12.388 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.388 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.388 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:12.388 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.894970+0000", 2024-09-14T23:57:12.388 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2753999999999997e-05 2024-09-14T23:57:12.389 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:12.389 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:12.389 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:12.389 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.389 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.389 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30586 3.15 3:acd4c478:::rbd_data.1118fff5d5e0.0000000000000001:8 [read 2994176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:57:12.389 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.873023+0000", 2024-09-14T23:57:12.389 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.468062196999998, 2024-09-14T23:57:12.390 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.022266837000000001, 2024-09-14T23:57:12.390 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:12.390 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:12.390 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:12.390 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:12.390 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:12.390 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:12.390 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30586 2024-09-14T23:57:12.390 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.391 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:12.391 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.391 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:12.391 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873023+0000", 2024-09-14T23:57:12.391 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:12.391 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.391 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.391 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:12.391 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873023+0000", 2024-09-14T23:57:12.392 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:12.392 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.392 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.392 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:12.392 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873023+0000", 2024-09-14T23:57:12.392 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2500000000000001e-07 2024-09-14T23:57:12.392 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.392 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.393 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:12.393 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873026+0000", 2024-09-14T23:57:12.393 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7199999999999998e-06 2024-09-14T23:57:12.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.393 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.393 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:12.393 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873032+0000", 2024-09-14T23:57:12.393 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.494e-06 2024-09-14T23:57:12.394 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.394 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.394 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:12.394 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873037+0000", 2024-09-14T23:57:12.394 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4880000000000001e-06 2024-09-14T23:57:12.394 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.394 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.394 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:12.395 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.895006+0000", 2024-09-14T23:57:12.395 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021968713000000001 2024-09-14T23:57:12.395 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.395 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.395 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:12.395 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.895142+0000", 2024-09-14T23:57:12.395 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013594900000000001 2024-09-14T23:57:12.395 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.396 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.396 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:12.396 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.895290+0000", 2024-09-14T23:57:12.396 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014814799999999999 2024-09-14T23:57:12.396 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:12.396 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:12.396 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:12.396 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.397 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.397 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30588 3.15 3:acd4c478:::rbd_data.1118fff5d5e0.0000000000000001:8 [read 3514368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:57:12.397 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.873042+0000", 2024-09-14T23:57:12.397 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.468042891, 2024-09-14T23:57:12.397 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.022463265, 2024-09-14T23:57:12.397 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:12.397 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:12.397 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:12.398 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:12.398 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:12.398 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:12.398 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30588 2024-09-14T23:57:12.398 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.398 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:12.398 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.398 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:12.399 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873042+0000", 2024-09-14T23:57:12.399 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:12.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.399 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.399 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:12.399 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873042+0000", 2024-09-14T23:57:12.399 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:12.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.399 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.400 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:12.400 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873043+0000", 2024-09-14T23:57:12.400 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4299999999999999e-07 2024-09-14T23:57:12.400 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.400 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.400 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:12.400 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873044+0000", 2024-09-14T23:57:12.400 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.68e-06 2024-09-14T23:57:12.401 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.401 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.401 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:12.401 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873050+0000", 2024-09-14T23:57:12.401 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4979999999999997e-06 2024-09-14T23:57:12.401 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.401 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.401 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:12.401 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873054+0000", 2024-09-14T23:57:12.401 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5510000000000001e-06 2024-09-14T23:57:12.402 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.402 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.402 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:12.402 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.895322+0000", 2024-09-14T23:57:12.402 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.022267797999999998 2024-09-14T23:57:12.402 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.402 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.402 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:12.402 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.895404+0000", 2024-09-14T23:57:12.402 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1803000000000003e-05 2024-09-14T23:57:12.402 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.402 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.402 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:12.403 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.895506+0000", 2024-09-14T23:57:12.403 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101592 2024-09-14T23:57:12.403 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:12.403 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:12.403 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:12.403 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.403 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.403 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30590 3.15 3:acd4c478:::rbd_data.1118fff5d5e0.0000000000000001:8 [read 3047424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:57:12.403 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.873059+0000", 2024-09-14T23:57:12.403 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.46802593, 2024-09-14T23:57:12.403 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.022706343, 2024-09-14T23:57:12.403 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:12.403 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:12.403 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:12.403 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:12.404 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:12.404 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:12.404 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30590 2024-09-14T23:57:12.404 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.404 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:12.404 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.404 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:12.404 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873059+0000", 2024-09-14T23:57:12.404 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:12.404 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.404 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.404 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:12.404 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873059+0000", 2024-09-14T23:57:12.405 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:12.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.405 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.405 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:12.405 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873060+0000", 2024-09-14T23:57:12.405 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7e-07 2024-09-14T23:57:12.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.405 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.405 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:12.405 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873061+0000", 2024-09-14T23:57:12.405 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.652e-06 2024-09-14T23:57:12.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.405 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.405 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:12.406 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873067+0000", 2024-09-14T23:57:12.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5570000000000003e-06 2024-09-14T23:57:12.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.406 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:12.406 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873071+0000", 2024-09-14T23:57:12.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.33e-06 2024-09-14T23:57:12.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.406 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:12.406 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.895536+0000", 2024-09-14T23:57:12.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.022464631999999998 2024-09-14T23:57:12.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.407 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:12.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.895617+0000", 2024-09-14T23:57:12.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1316000000000005e-05 2024-09-14T23:57:12.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.407 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:12.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.895766+0000", 2024-09-14T23:57:12.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000148386 2024-09-14T23:57:12.407 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:12.407 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:12.407 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:12.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.407 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30593 3.15 3:acd4c478:::rbd_data.1118fff5d5e0.0000000000000001:8 [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:57:12.408 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.873076+0000", 2024-09-14T23:57:12.408 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.468008852000001, 2024-09-14T23:57:12.408 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.02293823, 2024-09-14T23:57:12.408 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:12.408 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:12.408 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:12.408 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:12.408 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:12.408 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:12.408 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30593 2024-09-14T23:57:12.408 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.408 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:12.408 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.408 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:12.409 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873076+0000", 2024-09-14T23:57:12.409 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:12.409 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.409 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.409 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:12.409 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873076+0000", 2024-09-14T23:57:12.409 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:12.409 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.409 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.409 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:12.409 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873077+0000", 2024-09-14T23:57:12.409 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2399999999999999e-07 2024-09-14T23:57:12.409 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.409 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.410 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:12.410 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873078+0000", 2024-09-14T23:57:12.410 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6279999999999999e-06 2024-09-14T23:57:12.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.410 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.410 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:12.410 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873084+0000", 2024-09-14T23:57:12.410 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4940000000000003e-06 2024-09-14T23:57:12.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.410 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.410 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:12.410 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873089+0000", 2024-09-14T23:57:12.410 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8189999999999999e-06 2024-09-14T23:57:12.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.411 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.411 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:12.411 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.895827+0000", 2024-09-14T23:57:12.411 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.022738441000000002 2024-09-14T23:57:12.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.411 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.411 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:12.411 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.895911+0000", 2024-09-14T23:57:12.411 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4128999999999994e-05 2024-09-14T23:57:12.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.411 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.411 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:12.411 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.896015+0000", 2024-09-14T23:57:12.411 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000103395 2024-09-14T23:57:12.412 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:12.412 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:12.412 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:12.412 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.412 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.412 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30595 3.15 3:acd4c478:::rbd_data.1118fff5d5e0.0000000000000001:8 [read 3629056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:57:12.412 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.873244+0000", 2024-09-14T23:57:12.412 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.467841581999998, 2024-09-14T23:57:12.412 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.022932491999999999, 2024-09-14T23:57:12.412 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:12.412 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:12.412 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:12.412 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:12.412 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:12.413 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:12.413 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30595 2024-09-14T23:57:12.413 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.413 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:12.413 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.413 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:12.413 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873244+0000", 2024-09-14T23:57:12.413 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:12.413 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.413 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.413 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:12.413 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873244+0000", 2024-09-14T23:57:12.413 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:12.413 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.413 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.414 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:12.414 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873244+0000", 2024-09-14T23:57:12.414 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7199999999999999e-07 2024-09-14T23:57:12.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.414 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.414 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:12.414 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873247+0000", 2024-09-14T23:57:12.414 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.208e-06 2024-09-14T23:57:12.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.414 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.414 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:12.414 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873266+0000", 2024-09-14T23:57:12.415 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8689999999999999e-05 2024-09-14T23:57:12.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.415 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.415 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:12.415 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873274+0000", 2024-09-14T23:57:12.415 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.089e-06 2024-09-14T23:57:12.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.415 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.415 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:12.415 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.896055+0000", 2024-09-14T23:57:12.415 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.02278059 2024-09-14T23:57:12.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.415 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.415 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:12.416 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.896107+0000", 2024-09-14T23:57:12.416 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2645000000000002e-05 2024-09-14T23:57:12.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.416 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.416 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:12.416 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.896176+0000", 2024-09-14T23:57:12.416 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8697999999999999e-05 2024-09-14T23:57:12.416 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:12.416 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:12.416 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:12.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.416 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.416 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30597 3.15 3:acd4c478:::rbd_data.1118fff5d5e0.0000000000000001:8 [read 2641920~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:57:12.417 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.873284+0000", 2024-09-14T23:57:12.417 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.467801348999998, 2024-09-14T23:57:12.417 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.023132685, 2024-09-14T23:57:12.417 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:12.417 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:12.417 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:12.417 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:12.417 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:12.417 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:12.417 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30597 2024-09-14T23:57:12.417 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.417 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:12.417 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.417 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:12.418 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873284+0000", 2024-09-14T23:57:12.418 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:12.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.418 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:12.418 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873284+0000", 2024-09-14T23:57:12.418 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:12.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.418 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:12.418 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873284+0000", 2024-09-14T23:57:12.418 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.08e-07 2024-09-14T23:57:12.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.418 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:12.419 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873287+0000", 2024-09-14T23:57:12.419 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4269999999999998e-06 2024-09-14T23:57:12.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.419 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.419 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:12.419 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873293+0000", 2024-09-14T23:57:12.419 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3489999999999997e-06 2024-09-14T23:57:12.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.419 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.419 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:12.419 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873298+0000", 2024-09-14T23:57:12.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8559999999999998e-06 2024-09-14T23:57:12.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.420 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.420 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:12.420 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.896220+0000", 2024-09-14T23:57:12.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.022921725 2024-09-14T23:57:12.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.420 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.420 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:12.420 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.896313+0000", 2024-09-14T23:57:12.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3697999999999996e-05 2024-09-14T23:57:12.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.420 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.420 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:12.421 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.896416+0000", 2024-09-14T23:57:12.421 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000103222 2024-09-14T23:57:12.421 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:12.421 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:12.421 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:12.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.421 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.421 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30599 3.15 3:acd4c478:::rbd_data.1118fff5d5e0.0000000000000001:8 [read 3162112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:57:12.421 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.873304+0000", 2024-09-14T23:57:12.421 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.467781059, 2024-09-14T23:57:12.421 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.023260915, 2024-09-14T23:57:12.421 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:12.421 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:12.421 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:12.422 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:12.422 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:12.422 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:12.422 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30599 2024-09-14T23:57:12.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.422 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:12.422 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.422 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:12.422 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873304+0000", 2024-09-14T23:57:12.422 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:12.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.422 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.422 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:12.422 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873304+0000", 2024-09-14T23:57:12.422 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:12.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.423 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:12.423 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873304+0000", 2024-09-14T23:57:12.423 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6899999999999998e-07 2024-09-14T23:57:12.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.423 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:12.423 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873313+0000", 2024-09-14T23:57:12.423 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3450000000000006e-06 2024-09-14T23:57:12.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.423 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:12.423 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873319+0000", 2024-09-14T23:57:12.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3609999999999997e-06 2024-09-14T23:57:12.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:12.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873324+0000", 2024-09-14T23:57:12.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8300000000000003e-06 2024-09-14T23:57:12.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:12.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.896454+0000", 2024-09-14T23:57:12.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.023130064999999998 2024-09-14T23:57:12.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:12.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.896503+0000", 2024-09-14T23:57:12.425 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8900000000000003e-05 2024-09-14T23:57:12.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.425 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:12.425 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.896565+0000", 2024-09-14T23:57:12.425 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2045000000000007e-05 2024-09-14T23:57:12.425 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:12.425 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:12.425 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:12.428 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.428 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.428 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30601 3.15 3:acd4c478:::rbd_data.1118fff5d5e0.0000000000000001:8 [read 3223552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:57:12.429 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.873359+0000", 2024-09-14T23:57:12.429 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.467726545000001, 2024-09-14T23:57:12.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.023428353999999998, 2024-09-14T23:57:12.429 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:12.429 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:12.429 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:12.429 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:12.429 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:12.430 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:12.430 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30601 2024-09-14T23:57:12.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.430 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:12.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.430 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:12.430 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873359+0000", 2024-09-14T23:57:12.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:12.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.430 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:12.430 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873359+0000", 2024-09-14T23:57:12.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:12.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.431 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.431 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:12.431 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873359+0000", 2024-09-14T23:57:12.431 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7199999999999998e-07 2024-09-14T23:57:12.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.431 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.431 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:12.431 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873363+0000", 2024-09-14T23:57:12.431 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9219999999999999e-06 2024-09-14T23:57:12.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.431 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.431 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:12.431 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873375+0000", 2024-09-14T23:57:12.431 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2078e-05 2024-09-14T23:57:12.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.432 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:12.432 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873383+0000", 2024-09-14T23:57:12.432 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7069999999999992e-06 2024-09-14T23:57:12.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.432 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:12.432 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.896603+0000", 2024-09-14T23:57:12.432 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.023219758 2024-09-14T23:57:12.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.432 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:12.433 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.896693+0000", 2024-09-14T23:57:12.433 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0600999999999995e-05 2024-09-14T23:57:12.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.433 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:12.433 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.896787+0000", 2024-09-14T23:57:12.433 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3516000000000004e-05 2024-09-14T23:57:12.433 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:12.433 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:12.433 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:12.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.433 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30603 3.15 3:acd4c478:::rbd_data.1118fff5d5e0.0000000000000001:8 [read 3276800~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:57:12.433 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.873395+0000", 2024-09-14T23:57:12.434 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.467690533999999, 2024-09-14T23:57:12.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.023533291000000001, 2024-09-14T23:57:12.434 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:12.434 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:12.434 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:12.434 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:12.434 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:12.434 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:12.434 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30603 2024-09-14T23:57:12.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.434 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:12.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.434 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:12.434 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873395+0000", 2024-09-14T23:57:12.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:12.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.435 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:12.435 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873395+0000", 2024-09-14T23:57:12.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:12.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.435 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:12.435 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873395+0000", 2024-09-14T23:57:12.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9299999999999998e-07 2024-09-14T23:57:12.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.436 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:12.436 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873398+0000", 2024-09-14T23:57:12.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.481e-06 2024-09-14T23:57:12.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.436 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:12.436 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873405+0000", 2024-09-14T23:57:12.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.25e-06 2024-09-14T23:57:12.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.436 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:12.436 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873411+0000", 2024-09-14T23:57:12.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9359999999999996e-06 2024-09-14T23:57:12.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.437 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:12.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.896817+0000", 2024-09-14T23:57:12.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.023406347000000001 2024-09-14T23:57:12.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.437 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:12.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.896868+0000", 2024-09-14T23:57:12.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1258000000000003e-05 2024-09-14T23:57:12.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.437 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:12.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.896928+0000", 2024-09-14T23:57:12.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9426000000000003e-05 2024-09-14T23:57:12.438 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:12.438 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:12.438 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:12.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.438 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30605 3.15 3:acd4c478:::rbd_data.1118fff5d5e0.0000000000000001:8 [read 3338240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:57:12.438 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.873418+0000", 2024-09-14T23:57:12.438 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.467667251999998, 2024-09-14T23:57:12.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.023645488999999999, 2024-09-14T23:57:12.438 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:12.438 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:12.438 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:12.438 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:12.439 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:12.439 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:12.439 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30605 2024-09-14T23:57:12.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.439 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:12.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.439 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:12.439 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873418+0000", 2024-09-14T23:57:12.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:12.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.439 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:12.439 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873418+0000", 2024-09-14T23:57:12.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:12.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.440 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:12.440 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873418+0000", 2024-09-14T23:57:12.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5200000000000002e-07 2024-09-14T23:57:12.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.440 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:12.440 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873422+0000", 2024-09-14T23:57:12.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0450000000000002e-06 2024-09-14T23:57:12.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.440 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:12.440 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873429+0000", 2024-09-14T23:57:12.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5950000000000004e-06 2024-09-14T23:57:12.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:12.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873434+0000", 2024-09-14T23:57:12.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4389999999999999e-06 2024-09-14T23:57:12.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:12.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.896957+0000", 2024-09-14T23:57:12.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.023522568000000001 2024-09-14T23:57:12.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:12.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.897006+0000", 2024-09-14T23:57:12.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8925000000000003e-05 2024-09-14T23:57:12.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:12.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.897063+0000", 2024-09-14T23:57:12.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7465e-05 2024-09-14T23:57:12.442 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:12.442 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:12.442 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:12.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.442 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:30608 3.15 3:acd4c478:::rbd_data.1118fff5d5e0.0000000000000001:8 [read 3399680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T23:57:12.443 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:48.873441+0000", 2024-09-14T23:57:12.443 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.46764404, 2024-09-14T23:57:12.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.023846589000000001, 2024-09-14T23:57:12.443 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:12.443 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:12.443 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:12.443 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:12.443 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:12.443 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:12.443 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 30608 2024-09-14T23:57:12.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.443 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:12.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.443 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:12.443 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873441+0000", 2024-09-14T23:57:12.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:12.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:12.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873441+0000", 2024-09-14T23:57:12.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:12.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:12.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873442+0000", 2024-09-14T23:57:12.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2600000000000002e-07 2024-09-14T23:57:12.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:12.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873444+0000", 2024-09-14T23:57:12.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.238e-06 2024-09-14T23:57:12.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:12.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873451+0000", 2024-09-14T23:57:12.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.861e-06 2024-09-14T23:57:12.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:12.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.873456+0000", 2024-09-14T23:57:12.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1490000000000002e-06 2024-09-14T23:57:12.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.446 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:12.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.897094+0000", 2024-09-14T23:57:12.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.023638461 2024-09-14T23:57:12.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.446 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:12.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.897187+0000", 2024-09-14T23:57:12.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2763000000000006e-05 2024-09-14T23:57:12.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.446 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:12.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:48.897288+0000", 2024-09-14T23:57:12.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100591 2024-09-14T23:57:12.447 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:12.447 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:12.447 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:12.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.447 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:47283 3.15 3:ab6868a8:::rbd_data.1118350d5ae5.0000000000000216:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T23:57:12.447 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:58.454270+0000", 2024-09-14T23:57:12.447 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.88681549, 2024-09-14T23:57:12.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021341483000000001, 2024-09-14T23:57:12.447 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:12.447 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:12.448 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:57:12.448 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:12.448 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:12.448 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:12.448 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 47283 2024-09-14T23:57:12.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.448 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:12.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:12.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.454270+0000", 2024-09-14T23:57:12.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:12.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:12.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.454270+0000", 2024-09-14T23:57:12.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:12.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:12.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.454271+0000", 2024-09-14T23:57:12.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7460000000000001e-06 2024-09-14T23:57:12.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:12.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.454274+0000", 2024-09-14T23:57:12.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5339999999999998e-06 2024-09-14T23:57:12.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.450 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:12.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.454279+0000", 2024-09-14T23:57:12.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8489999999999997e-06 2024-09-14T23:57:12.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.450 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:12.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.454283+0000", 2024-09-14T23:57:12.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4390000000000002e-06 2024-09-14T23:57:12.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.450 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:12.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.475548+0000", 2024-09-14T23:57:12.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021264524 2024-09-14T23:57:12.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:12.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.475611+0000", 2024-09-14T23:57:12.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3391000000000003e-05 2024-09-14T23:57:12.451 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:12.451 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:12.451 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:12.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.451 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:47311 3.15 3:ab4845c3:::rbd_data.1118350d5ae5.0000000000000232:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T23:57:12.451 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:58.454366+0000", 2024-09-14T23:57:12.451 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.886719441, 2024-09-14T23:57:12.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021329957, 2024-09-14T23:57:12.452 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:12.452 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:12.452 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:57:12.452 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:12.452 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:12.452 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:12.452 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 47311 2024-09-14T23:57:12.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.452 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:12.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:12.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.454366+0000", 2024-09-14T23:57:12.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:12.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:12.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.454366+0000", 2024-09-14T23:57:12.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:12.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:12.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.454367+0000", 2024-09-14T23:57:12.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6390000000000001e-06 2024-09-14T23:57:12.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:12.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.454370+0000", 2024-09-14T23:57:12.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.678e-06 2024-09-14T23:57:12.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:12.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.454375+0000", 2024-09-14T23:57:12.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.242e-06 2024-09-14T23:57:12.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:12.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.454380+0000", 2024-09-14T23:57:12.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9609999999999996e-06 2024-09-14T23:57:12.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:12.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.475635+0000", 2024-09-14T23:57:12.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021254933 2024-09-14T23:57:12.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:12.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.475696+0000", 2024-09-14T23:57:12.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0504000000000002e-05 2024-09-14T23:57:12.455 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:12.455 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:12.455 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:12.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.455 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:47314 3.d 3:b30cd14a:::rbd_data.1118350d5ae5.0000000000000235:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T23:57:12.456 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:58.454405+0000", 2024-09-14T23:57:12.456 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.886680203999999, 2024-09-14T23:57:12.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021386354, 2024-09-14T23:57:12.456 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:12.456 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:12.456 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:57:12.456 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:12.456 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:12.456 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:12.456 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 47314 2024-09-14T23:57:12.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.456 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:12.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:12.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.454405+0000", 2024-09-14T23:57:12.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:12.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:12.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.454405+0000", 2024-09-14T23:57:12.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:12.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:12.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.454406+0000", 2024-09-14T23:57:12.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5579999999999999e-06 2024-09-14T23:57:12.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:12.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.454415+0000", 2024-09-14T23:57:12.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0220000000000004e-06 2024-09-14T23:57:12.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:12.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.454420+0000", 2024-09-14T23:57:12.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0719999999999999e-06 2024-09-14T23:57:12.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:12.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.454424+0000", 2024-09-14T23:57:12.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3479999999999997e-06 2024-09-14T23:57:12.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:12.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.475660+0000", 2024-09-14T23:57:12.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021235819999999999 2024-09-14T23:57:12.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:12.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.475791+0000", 2024-09-14T23:57:12.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013153399999999999 2024-09-14T23:57:12.459 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:12.459 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:12.459 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:12.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.460 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:56377 3.13 3:cb867fb0:::rbd_data.1118c4b3aebe.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:12.460 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.264421+0000", 2024-09-14T23:57:12.460 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 6.0766645270000001, 2024-09-14T23:57:12.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021606855000000001, 2024-09-14T23:57:12.460 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:12.460 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:12.460 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:12.460 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:12.460 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:12.460 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:12.460 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 56377 2024-09-14T23:57:12.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.460 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:12.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:12.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.264421+0000", 2024-09-14T23:57:12.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:12.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:12.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.264421+0000", 2024-09-14T23:57:12.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:12.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:12.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.264422+0000", 2024-09-14T23:57:12.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.012e-06 2024-09-14T23:57:12.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:12.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275783+0000", 2024-09-14T23:57:12.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.011361339999999999 2024-09-14T23:57:12.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:12.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275811+0000", 2024-09-14T23:57:12.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8231000000000001e-05 2024-09-14T23:57:12.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:12.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275824+0000", 2024-09-14T23:57:12.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2353e-05 2024-09-14T23:57:12.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:12.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275901+0000", 2024-09-14T23:57:12.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7152999999999998e-05 2024-09-14T23:57:12.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:12.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276010+0000", 2024-09-14T23:57:12.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000108942 2024-09-14T23:57:12.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 4", 2024-09-14T23:57:12.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276333+0000", 2024-09-14T23:57:12.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00032306000000000001 2024-09-14T23:57:12.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:57:12.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.283174+0000", 2024-09-14T23:57:12.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0068409819999999998 2024-09-14T23:57:12.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:12.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.285123+0000", 2024-09-14T23:57:12.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001949685 2024-09-14T23:57:12.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.469 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:12.469 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.285960+0000", 2024-09-14T23:57:12.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00083685599999999997 2024-09-14T23:57:12.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:12.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:12.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:12.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.286027+0000", 2024-09-14T23:57:12.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7241000000000002e-05 2024-09-14T23:57:12.470 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:12.470 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:12.470 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:12.470 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:12.470 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:12.470 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:12.471 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T23:57:12.472 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:12.470+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:12.473 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T23:57:12.573 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:12.572+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:12.582 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:12.582 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:57:12.582 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:57:12.582 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:12.591 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T23:57:12.598 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:57:12.598 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:57:12.615 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:57:12.616 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:57:12.632 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:57:12.632 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:57:12.648 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:57:12.649 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:57:12.665 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:57:12.665 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:57:12.674 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:12.672+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:12.682 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:57:12.683 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:57:12.699 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:12.699 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:12.699 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:57:12.699 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:57:12.708 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T23:57:12.716 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:57:12.717 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:57:12.775 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:12.773+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:12.823 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:12.823 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:12.832 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T23:57:12.875 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:12.874+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:12.977 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:12.975+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:12.981 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:12.981 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:12.990 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T23:57:13.078 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:13.077+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:13.106 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is unset 2024-09-14T23:57:13.115 INFO:teuthology.orchestra.run.smithi042.stdout:1963 write 0x4412e02 thru 0x4415f4b (0x314a bytes) 2024-09-14T23:57:13.122 INFO:teuthology.orchestra.run.smithi042.stdout:1964 read 0x302ea8c thru 0x3032b20 (0x4095 bytes) 2024-09-14T23:57:13.123 INFO:teuthology.orchestra.run.smithi042.stdout:1966 read 0x40e341a thru 0x40e4226 (0xe0d bytes) 2024-09-14T23:57:13.123 INFO:teuthology.orchestra.run.smithi042.stdout:1968 read 0x33d23b8 thru 0x33d344a (0x1093 bytes) 2024-09-14T23:57:13.124 INFO:teuthology.orchestra.run.smithi042.stdout:1970 trunc from 0x55ad70c to 0x3ec0b7a 2024-09-14T23:57:13.137 INFO:teuthology.orchestra.run.smithi042.stdout:1971 write 0x32ecdd8 thru 0x32f9210 (0xc439 bytes) 2024-09-14T23:57:13.145 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:13.145 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:13.150 INFO:teuthology.orchestra.run.smithi042.stdout:1973 write 0x5fde6c2 thru 0x5feb856 (0xd195 bytes) 2024-09-14T23:57:13.154 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T23:57:13.154 INFO:teuthology.orchestra.run.smithi042.stdout:1974 punch from 0x136436d to 0x136a13f, (0x5dd2 bytes) 2024-09-14T23:57:13.166 INFO:teuthology.orchestra.run.smithi042.stdout:1975 write 0xa573b27 thru 0xa57d0a9 (0x9583 bytes) 2024-09-14T23:57:13.168 INFO:teuthology.orchestra.run.smithi042.stdout:1976 read 0x7009237 thru 0x70161da (0xcfa4 bytes) 2024-09-14T23:57:13.168 INFO:teuthology.orchestra.run.smithi042.stdout:1977 punch from 0x606a854 to 0x606bff4, (0x17a0 bytes) 2024-09-14T23:57:13.168 INFO:teuthology.orchestra.run.smithi042.stdout:1978 punch from 0x883c416 to 0x8843302, (0x6eec bytes) 2024-09-14T23:57:13.175 INFO:teuthology.orchestra.run.smithi042.stdout:1979 trunc from 0xa57d0aa to 0x3ab9194 2024-09-14T23:57:13.179 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:13.177+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:13.220 INFO:teuthology.orchestra.run.smithi042.stdout:1981 writesame 0x2922566 thru 0x292dc9e data_size 0x9f(0xb739 bytes) 2024-09-14T23:57:13.221 INFO:teuthology.orchestra.run.smithi042.stdout:1986 write 0x355f804 thru 0x3560ad7 (0x12d4 bytes) 2024-09-14T23:57:13.230 INFO:teuthology.orchestra.run.smithi042.stdout:1987 write 0x51106b4 thru 0x511e32a (0xdc77 bytes) 2024-09-14T23:57:13.234 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:13.234 INFO:teuthology.orchestra.run.smithi042.stdout:1990 punch from 0x2cabb05 to 0x2cba380, (0xe87b bytes) 2024-09-14T23:57:13.234 INFO:teuthology.orchestra.run.smithi042.stdout:1992 read 0x44421e1 thru 0x444e9da (0xc7fa bytes) 2024-09-14T23:57:13.234 INFO:teuthology.orchestra.run.smithi042.stdout:1993 read 0x3979157 thru 0x39844f5 (0xb39f bytes) 2024-09-14T23:57:13.234 INFO:teuthology.orchestra.run.smithi042.stdout:1994 read 0x41d2c77 thru 0x41da566 (0x78f0 bytes) 2024-09-14T23:57:13.234 INFO:teuthology.orchestra.run.smithi042.stdout:1995 trunc from 0x511e32b to 0x4ec5265 2024-09-14T23:57:13.243 INFO:teuthology.orchestra.run.smithi042.stdout:1996 punch from 0x3026d2a to 0x302f94a, (0x8c20 bytes) 2024-09-14T23:57:13.252 INFO:teuthology.orchestra.run.smithi042.stdout:1998 writesame 0x8b15e02 thru 0x8b19f8b data_size 0x1(0x418a bytes) 2024-09-14T23:57:13.259 INFO:teuthology.orchestra.run.smithi042.stdout:1999 write 0x8f8e308 thru 0x8f9ba12 (0xd70b bytes) 2024-09-14T23:57:13.267 INFO:teuthology.orchestra.run.smithi042.stdout:2001 write 0xa8972e6 thru 0xa8a5aaa (0xe7c5 bytes) 2024-09-14T23:57:13.269 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:13.269 INFO:teuthology.orchestra.run.smithi042.stdout:2004 write 0x60b03ab thru 0x60bd674 (0xd2ca bytes) 2024-09-14T23:57:13.278 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:13.278 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:13.279 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:13.278+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:13.282 INFO:teuthology.orchestra.run.smithi042.stdout:2006 trunc from 0xa8a5aab to 0x1d58096 2024-09-14T23:57:13.290 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T23:57:13.316 INFO:teuthology.orchestra.run.smithi042.stdout:2007 write 0xdb18706 thru 0xdb1b750 (0x304b bytes) 2024-09-14T23:57:13.319 INFO:teuthology.orchestra.run.smithi042.stdout:2008 punch from 0x8c9dc59 to 0x8c9e486, (0x82d bytes) 2024-09-14T23:57:13.319 INFO:teuthology.orchestra.run.smithi042.stdout:2011 read 0xb9e89b7 thru 0xb9f74a3 (0xeaed bytes) 2024-09-14T23:57:13.320 INFO:teuthology.orchestra.run.smithi042.stdout:2013 writesame 0x286a83b thru 0x2872184 data_size 0x73(0x794a bytes) 2024-09-14T23:57:13.327 INFO:teuthology.orchestra.run.smithi042.stdout:2014 writesame 0xde616d9 thru 0xde67645 data_size 0x11(0x5f6d bytes) 2024-09-14T23:57:13.332 INFO:teuthology.orchestra.run.smithi042.stdout:2015 trunc from 0xde67646 to 0xa507e33 2024-09-14T23:57:13.346 INFO:teuthology.orchestra.run.smithi042.stdout:2016 write 0xc8fdcfc thru 0xc90c1d3 (0xe4d8 bytes) 2024-09-14T23:57:13.349 INFO:teuthology.orchestra.run.smithi042.stdout:2017 write 0xb3028f6 thru 0xb303f83 (0x168e bytes) 2024-09-14T23:57:13.352 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:13.352 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:13.352 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:13.352 INFO:teuthology.orchestra.run.smithi042.stdout:2022 writesame 0x3435dd3 thru 0x3439d6f data_size 0x1(0x3f9d bytes) 2024-09-14T23:57:13.361 INFO:teuthology.orchestra.run.smithi042.stdout:2024 read 0x281c948 thru 0x282a2cf (0xd988 bytes) 2024-09-14T23:57:13.362 INFO:teuthology.orchestra.run.smithi042.stdout:2025 read 0x9473a56 thru 0x94755d2 (0x1b7d bytes) 2024-09-14T23:57:13.362 INFO:teuthology.orchestra.run.smithi042.stdout:2026 write 0x148ff62 thru 0x1495028 (0x50c7 bytes) 2024-09-14T23:57:13.364 INFO:teuthology.orchestra.run.smithi042.stdout:2029 write 0x39ab71f thru 0x39b079f (0x5081 bytes) 2024-09-14T23:57:13.366 INFO:teuthology.orchestra.run.smithi042.stdout:2031 read 0xc6e5257 thru 0xc6ee0b0 (0x8e5a bytes) 2024-09-14T23:57:13.367 INFO:teuthology.orchestra.run.smithi042.stdout:2032 read 0x6eb2df3 thru 0x6ec241f (0xf62d bytes) 2024-09-14T23:57:13.367 INFO:teuthology.orchestra.run.smithi042.stdout:2033 punch from 0x95bc29c to 0x95c8d26, (0xca8a bytes) 2024-09-14T23:57:13.367 INFO:teuthology.orchestra.run.smithi042.stdout:2034 write 0xafb26e0 thru 0xafbc97c (0xa29d bytes) 2024-09-14T23:57:13.369 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:13.375 INFO:teuthology.orchestra.run.smithi042.stdout:2036 write 0xd695c5c thru 0xd6a0d1c (0xb0c1 bytes) 2024-09-14T23:57:13.378 INFO:teuthology.orchestra.run.smithi042.stdout:2039 write 0xb87b70a thru 0xb88b115 (0xfa0c bytes) 2024-09-14T23:57:13.380 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:13.379+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:13.381 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:13.381 INFO:teuthology.orchestra.run.smithi042.stdout:2043 write 0x845b681 thru 0x845bd4e (0x6ce bytes) 2024-09-14T23:57:13.383 INFO:teuthology.orchestra.run.smithi042.stdout:2045 write 0x437a397 thru 0x438485c (0xa4c6 bytes) 2024-09-14T23:57:13.384 INFO:teuthology.orchestra.run.smithi042.stdout:2046 read 0x35e2fb8 thru 0x35ea7fe (0x7847 bytes) 2024-09-14T23:57:13.385 INFO:teuthology.orchestra.run.smithi042.stdout:2047 write 0x3aa9d93 thru 0x3ab9280 (0xf4ee bytes) 2024-09-14T23:57:13.393 INFO:teuthology.orchestra.run.smithi042.stdout:2049 write 0xe4e86db thru 0xe4ef7da (0x7100 bytes) 2024-09-14T23:57:13.396 INFO:teuthology.orchestra.run.smithi042.stdout:2050 read 0xe01b530 thru 0xe026c8d (0xb75e bytes) 2024-09-14T23:57:13.396 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:13.408 INFO:teuthology.orchestra.run.smithi042.stdout:2052 trunc from 0xe4ef7db to 0x3904e3f 2024-09-14T23:57:13.435 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:13.435 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:13.445 INFO:teuthology.orchestra.run.smithi042.stdout:2053 writesame 0xd55c85c thru 0xd56ae2f data_size 0x6(0xe5d4 bytes) 2024-09-14T23:57:13.445 INFO:teuthology.orchestra.run.smithi042.stdout:2054 read 0xc9639a7 thru 0xc96f127 (0xb781 bytes) 2024-09-14T23:57:13.445 INFO:teuthology.orchestra.run.smithi042.stdout:2055 write 0x34d04e6 thru 0x34df472 (0xef8d bytes) 2024-09-14T23:57:13.447 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T23:57:13.447 INFO:teuthology.orchestra.run.smithi042.stdout:2059 punch from 0x784d4db to 0x7858214, (0xad39 bytes) 2024-09-14T23:57:13.450 INFO:teuthology.orchestra.run.smithi042.stdout:2060 trunc from 0xd56ae30 to 0x9c7d160 2024-09-14T23:57:13.457 INFO:teuthology.orchestra.run.smithi042.stdout:2062 write 0x1aa7ac2 thru 0x1ab643d (0xe97c bytes) 2024-09-14T23:57:13.459 INFO:teuthology.orchestra.run.smithi042.stdout:2063 read 0x96641b9 thru 0x9668ea3 (0x4ceb bytes) 2024-09-14T23:57:13.463 INFO:teuthology.orchestra.run.smithi042.stdout:2064 trunc from 0x9c7d160 to 0x63c5418 2024-09-14T23:57:13.473 INFO:teuthology.orchestra.run.smithi042.stdout:2065 read 0x17c79e9 thru 0x17d6deb (0xf403 bytes) 2024-09-14T23:57:13.474 INFO:teuthology.orchestra.run.smithi042.stdout:2066 trunc from 0x63c5418 to 0x629b805 2024-09-14T23:57:13.476 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:13.476 INFO:teuthology.orchestra.run.smithi042.stdout:2069 write 0x3da0642 thru 0x3da1ed1 (0x1890 bytes) 2024-09-14T23:57:13.481 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:13.480+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:13.482 INFO:teuthology.orchestra.run.smithi042.stdout:2070 trunc from 0x629b805 to 0x13e1380 2024-09-14T23:57:13.507 INFO:teuthology.orchestra.run.smithi042.stdout:2072 writesame 0x7340cdc thru 0x73424e8 data_size 0x1(0x180d bytes) 2024-09-14T23:57:13.508 INFO:teuthology.orchestra.run.smithi042.stdout:2073 write 0x2054747 thru 0x2057087 (0x2941 bytes) 2024-09-14T23:57:13.510 INFO:teuthology.orchestra.run.smithi042.stdout:2074 read 0x1da2124 thru 0x1dacbb4 (0xaa91 bytes) 2024-09-14T23:57:13.518 INFO:teuthology.orchestra.run.smithi042.stdout:2075 write 0x9f1de50 thru 0x9f1e16c (0x31d bytes) 2024-09-14T23:57:13.528 INFO:teuthology.orchestra.run.smithi042.stdout:2076 trunc from 0x9f1e16d to 0x493ed1a 2024-09-14T23:57:13.535 INFO:teuthology.orchestra.run.smithi042.stdout:2077 write 0x1e4f222 thru 0x1e5296d (0x374c bytes) 2024-09-14T23:57:13.538 INFO:teuthology.orchestra.run.smithi042.stdout:2078 punch from 0x248397b to 0x2492f2d, (0xf5b2 bytes) 2024-09-14T23:57:13.538 INFO:teuthology.orchestra.run.smithi042.stdout:2079 punch from 0x1909b34 to 0x1919951, (0xfe1d bytes) 2024-09-14T23:57:13.539 INFO:teuthology.orchestra.run.smithi042.stdout:2080 read 0x31337ba thru 0x3139db6 (0x65fd bytes) 2024-09-14T23:57:13.539 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:13.555 INFO:teuthology.orchestra.run.smithi042.stdout:2082 write 0xe9cf720 thru 0xe9dc149 (0xca2a bytes) 2024-09-14T23:57:13.558 INFO:teuthology.orchestra.run.smithi042.stdout:2083 read 0x76c5acd thru 0x76ceb23 (0x9057 bytes) 2024-09-14T23:57:13.559 INFO:teuthology.orchestra.run.smithi042.stdout:2084 write 0x9c21745 thru 0x9c2c638 (0xaef4 bytes) 2024-09-14T23:57:13.560 INFO:teuthology.orchestra.run.smithi042.stdout:2087 read 0x258369 thru 0x261993 (0x962b bytes) 2024-09-14T23:57:13.568 INFO:teuthology.orchestra.run.smithi042.stdout:2088 trunc from 0xe9dc14a to 0x7f00b86 2024-09-14T23:57:13.570 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:13.571 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:13.581 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T23:57:13.583 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:13.582+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:13.586 INFO:teuthology.orchestra.run.smithi042.stdout:2089 trunc from 0x7f00b86 to 0xe5631e7 2024-09-14T23:57:13.591 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:13.592 INFO:teuthology.orchestra.run.smithi042.stdout:2091 trunc from 0xe5631e7 to 0xd29245a 2024-09-14T23:57:13.598 INFO:teuthology.orchestra.run.smithi042.stdout:2092 read 0x48d4d81 thru 0x48e392b (0xebab bytes) 2024-09-14T23:57:13.599 INFO:teuthology.orchestra.run.smithi042.stdout:2093 read 0x74938d3 thru 0x7496679 (0x2da7 bytes) 2024-09-14T23:57:13.599 INFO:teuthology.orchestra.run.smithi042.stdout:2096 write 0xaae8cd3 thru 0xaaede41 (0x516f bytes) 2024-09-14T23:57:13.609 INFO:teuthology.orchestra.run.smithi042.stdout:2097 trunc from 0xd29245a to 0x64a5402 2024-09-14T23:57:13.615 INFO:teuthology.orchestra.run.smithi042.stdout:2099 write 0x365fcf1 thru 0x366243b (0x274b bytes) 2024-09-14T23:57:13.621 INFO:teuthology.orchestra.run.smithi042.stdout:2101 trunc from 0x64a5402 to 0x393bfb2 2024-09-14T23:57:13.628 INFO:teuthology.orchestra.run.smithi042.stdout:2102 read 0x1cf7f37 thru 0x1cff534 (0x75fe bytes) 2024-09-14T23:57:13.642 INFO:teuthology.orchestra.run.smithi042.stdout:2104 write 0xac41fc3 thru 0xac439ea (0x1a28 bytes) 2024-09-14T23:57:13.643 INFO:teuthology.orchestra.run.smithi042.stdout:2105 write 0x7197df4 thru 0x719e76e (0x697b bytes) 2024-09-14T23:57:13.645 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:13.645 INFO:teuthology.orchestra.run.smithi042.stdout:2107 write 0x8ae558 thru 0x8b5445 (0x6eee bytes) 2024-09-14T23:57:13.646 INFO:teuthology.orchestra.run.smithi042.stdout:2108 write 0x2854b2d thru 0x28553da (0x8ae bytes) 2024-09-14T23:57:13.647 INFO:teuthology.orchestra.run.smithi042.stdout:2110 write 0x81ef2b2 thru 0x81f889f (0x95ee bytes) 2024-09-14T23:57:13.649 INFO:teuthology.orchestra.run.smithi042.stdout:2111 read 0x7f848d thru 0x7f92f0 (0xe64 bytes) 2024-09-14T23:57:13.660 INFO:teuthology.orchestra.run.smithi042.stdout:2112 trunc from 0xac439eb to 0x1cc816a 2024-09-14T23:57:13.684 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:13.683+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:13.687 INFO:teuthology.orchestra.run.smithi042.stdout:2113 trunc from 0x1cc816a to 0xe7c65d7 2024-09-14T23:57:13.692 INFO:teuthology.orchestra.run.smithi042.stdout:2116 write 0x3e3cd1d thru 0x3e41a5c (0x4d40 bytes) 2024-09-14T23:57:13.700 INFO:teuthology.orchestra.run.smithi042.stdout:2117 write 0xece6c5b thru 0xecea972 (0x3d18 bytes) 2024-09-14T23:57:13.703 INFO:teuthology.orchestra.run.smithi042.stdout:2119 write 0x2c2b2d4 thru 0x2c387eb (0xd518 bytes) 2024-09-14T23:57:13.706 INFO:teuthology.orchestra.run.smithi042.stdout:2120 write 0xd307e79 thru 0xd30e4a2 (0x662a bytes) 2024-09-14T23:57:13.706 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:13.707 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:13.709 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:13.709 INFO:teuthology.orchestra.run.smithi042.stdout:2123 read 0x74dacea thru 0x74e7061 (0xc378 bytes) 2024-09-14T23:57:13.709 INFO:teuthology.orchestra.run.smithi042.stdout:2124 write 0x2693a93 thru 0x2696c54 (0x31c2 bytes) 2024-09-14T23:57:13.712 INFO:teuthology.orchestra.run.smithi042.stdout:2127 read 0x884457 thru 0x8909be (0xc568 bytes) 2024-09-14T23:57:13.712 INFO:teuthology.orchestra.run.smithi042.stdout:2129 read 0xda9d971 thru 0xdaa637c (0x8a0c bytes) 2024-09-14T23:57:13.712 INFO:teuthology.orchestra.run.smithi042.stdout:2130 writesame 0xe0e77a1 thru 0xe0e9c5f data_size 0x1(0x24bf bytes) 2024-09-14T23:57:13.716 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T23:57:13.718 INFO:teuthology.orchestra.run.smithi042.stdout:2131 clone 22 order 19 su 8192 sc 14 2024-09-14T23:57:13.785 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:13.784+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:13.824 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:13.824 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:13.833 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T23:57:13.886 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:13.885+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:13.981 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:13.981 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:13.987 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:13.986+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:13.990 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T23:57:14.088 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:14.087+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:14.106 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T23:57:14.106 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-14T23:57:14.144 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:14.144 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:14.154 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T23:57:14.195 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:14.194+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:14.203 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone21 from 0xecea973 (overlap 0x13e1380) to 0x1fcec0 2024-09-14T23:57:14.277 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:14.277 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:14.286 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T23:57:14.322 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-09-14T23:57:14.330 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:14.329+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:14.391 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:14.390+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:14.396 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:14.396 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:14.405 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T23:57:14.438 INFO:teuthology.orchestra.run.smithi042.stdout:2132 read 0x574c1c9 thru 0x574d7d0 (0x1608 bytes) 2024-09-14T23:57:14.439 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:14.439 INFO:teuthology.orchestra.run.smithi042.stdout:2134 write 0x4e4891e thru 0x4e574b7 (0xeb9a bytes) 2024-09-14T23:57:14.452 INFO:teuthology.orchestra.run.smithi042.stdout:2135 read 0x732bd87 thru 0x733a312 (0xe58c bytes) 2024-09-14T23:57:14.454 INFO:teuthology.orchestra.run.smithi042.stdout:2136 writesame 0x4e155fc thru 0x4e22a5d data_size 0x5(0xd462 bytes) 2024-09-14T23:57:14.462 INFO:teuthology.orchestra.run.smithi042.stdout:2137 clone 23 order 23 su 32768 sc 14 2024-09-14T23:57:14.492 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:14.491+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:14.555 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:14.555 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:14.564 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T23:57:14.593 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:14.592+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:14.671 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:14.671 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:14.679 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T23:57:14.694 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:14.692+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:14.728 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T23:57:14.786 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:14.787 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:14.795 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T23:57:14.796 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:14.794+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:14.857 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:57:14.858 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:57:14.875 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:57:14.875 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:57:14.891 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:57:14.891 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:57:14.895 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:14.894+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:14.907 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:57:14.907 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:57:14.924 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:57:14.924 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:57:14.941 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:57:14.941 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:57:14.944 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:14.944 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:57:14.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:57:14.944 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:57:14.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.944 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4284.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-14T23:57:14.944 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:57:14.944 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 52.417393144000002, 2024-09-14T23:57:14.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.004942086, 2024-09-14T23:57:14.945 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:14.945 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:14.945 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:14.945 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:14.945 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4284", 2024-09-14T23:57:14.945 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/4278347926", 2024-09-14T23:57:14.945 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1 2024-09-14T23:57:14.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.945 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:14.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:14.945 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:57:14.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:14.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.946 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:14.946 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:57:14.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:14.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.946 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:14.946 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524331+0000", 2024-09-14T23:57:14.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.383e-05 2024-09-14T23:57:14.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.947 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:14.947 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524339+0000", 2024-09-14T23:57:14.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8290000000000008e-06 2024-09-14T23:57:14.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.947 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:14.947 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524359+0000", 2024-09-14T23:57:14.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9998000000000001e-05 2024-09-14T23:57:14.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.947 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:14.947 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524373+0000", 2024-09-14T23:57:14.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4304000000000001e-05 2024-09-14T23:57:14.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.948 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:14.948 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524488+0000", 2024-09-14T23:57:14.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114663 2024-09-14T23:57:14.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.948 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:57:14.948 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524501+0000", 2024-09-14T23:57:14.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3349999999999999e-05 2024-09-14T23:57:14.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.948 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:14.949 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.526868+0000", 2024-09-14T23:57:14.949 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023670090000000002 2024-09-14T23:57:14.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:14.949 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.527041+0000", 2024-09-14T23:57:14.949 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000172597 2024-09-14T23:57:14.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-14T23:57:14.949 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.527384+0000", 2024-09-14T23:57:14.949 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00034309500000000002 2024-09-14T23:57:14.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.950 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:14.950 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529103+0000", 2024-09-14T23:57:14.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017185220000000001 2024-09-14T23:57:14.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.950 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:57:14.950 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529188+0000", 2024-09-14T23:57:14.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5591000000000001e-05 2024-09-14T23:57:14.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.950 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:14.950 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529227+0000", 2024-09-14T23:57:14.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8563000000000003e-05 2024-09-14T23:57:14.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.951 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:14.951 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529259+0000", 2024-09-14T23:57:14.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2734999999999997e-05 2024-09-14T23:57:14.951 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:14.951 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:14.951 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:14.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.951 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4284.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-14T23:57:14.951 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:23.530683+0000", 2024-09-14T23:57:14.951 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.411027478999998, 2024-09-14T23:57:14.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0036133110000000001, 2024-09-14T23:57:14.952 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:14.952 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:14.952 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:14.952 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:14.952 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4284", 2024-09-14T23:57:14.952 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/4278347926", 2024-09-14T23:57:14.952 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5 2024-09-14T23:57:14.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.952 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:14.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.953 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:14.953 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530683+0000", 2024-09-14T23:57:14.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:14.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.953 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:14.954 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530683+0000", 2024-09-14T23:57:14.954 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:14.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.954 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.954 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:14.954 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530685+0000", 2024-09-14T23:57:14.954 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.075e-06 2024-09-14T23:57:14.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.955 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:14.955 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530695+0000", 2024-09-14T23:57:14.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0261999999999999e-05 2024-09-14T23:57:14.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.955 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:14.955 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530715+0000", 2024-09-14T23:57:14.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9930000000000001e-05 2024-09-14T23:57:14.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:14.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530724+0000", 2024-09-14T23:57:14.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.901e-06 2024-09-14T23:57:14.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:14.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530800+0000", 2024-09-14T23:57:14.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5873999999999999e-05 2024-09-14T23:57:14.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-14T23:57:14.957 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530807+0000", 2024-09-14T23:57:14.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2389999999999996e-06 2024-09-14T23:57:14.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:14.957 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.532494+0000", 2024-09-14T23:57:14.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016871830000000001 2024-09-14T23:57:14.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:14.957 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.532599+0000", 2024-09-14T23:57:14.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010472 2024-09-14T23:57:14.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.958 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.958 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1", 2024-09-14T23:57:14.958 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.532715+0000", 2024-09-14T23:57:14.958 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011568199999999999 2024-09-14T23:57:14.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.958 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.958 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:57:14.958 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.533460+0000", 2024-09-14T23:57:14.958 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00074494999999999997 2024-09-14T23:57:14.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.958 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.958 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:14.958 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534230+0000", 2024-09-14T23:57:14.958 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00077036599999999998 2024-09-14T23:57:14.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.959 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.959 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:14.959 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534266+0000", 2024-09-14T23:57:14.959 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5725000000000001e-05 2024-09-14T23:57:14.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.959 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.959 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:14.959 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534296+0000", 2024-09-14T23:57:14.959 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0403999999999999e-05 2024-09-14T23:57:14.959 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:14.959 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:14.959 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:14.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.960 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4284.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-14T23:57:14.960 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:24.535160+0000", 2024-09-14T23:57:14.960 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 50.406550838000001, 2024-09-14T23:57:14.960 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033473650000000001, 2024-09-14T23:57:14.960 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:14.960 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:14.960 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:14.960 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:14.960 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4284", 2024-09-14T23:57:14.960 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/4278347926", 2024-09-14T23:57:14.961 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 7 2024-09-14T23:57:14.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.961 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:14.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:14.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535160+0000", 2024-09-14T23:57:14.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:14.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:14.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535160+0000", 2024-09-14T23:57:14.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:14.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.962 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:14.962 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535162+0000", 2024-09-14T23:57:14.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3369999999999998e-06 2024-09-14T23:57:14.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.962 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:14.962 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535171+0000", 2024-09-14T23:57:14.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7700000000000007e-06 2024-09-14T23:57:14.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.962 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:14.962 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535189+0000", 2024-09-14T23:57:14.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7938e-05 2024-09-14T23:57:14.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.963 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.963 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:14.963 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535197+0000", 2024-09-14T23:57:14.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5639999999999997e-06 2024-09-14T23:57:14.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.963 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.963 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:14.963 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535266+0000", 2024-09-14T23:57:14.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8926999999999999e-05 2024-09-14T23:57:14.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.963 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.963 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-14T23:57:14.963 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535271+0000", 2024-09-14T23:57:14.964 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0329999999999999e-06 2024-09-14T23:57:14.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.964 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:14.964 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.537064+0000", 2024-09-14T23:57:14.964 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001792929 2024-09-14T23:57:14.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.964 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:14.964 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.537131+0000", 2024-09-14T23:57:14.964 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6605000000000004e-05 2024-09-14T23:57:14.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.964 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1", 2024-09-14T23:57:14.965 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.537271+0000", 2024-09-14T23:57:14.965 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013992400000000001 2024-09-14T23:57:14.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.965 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.965 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:57:14.965 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.538093+0000", 2024-09-14T23:57:14.965 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00082210200000000001 2024-09-14T23:57:14.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.965 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.965 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:14.965 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.538439+0000", 2024-09-14T23:57:14.965 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00034595999999999997 2024-09-14T23:57:14.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.965 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:14.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.538476+0000", 2024-09-14T23:57:14.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6770000000000002e-05 2024-09-14T23:57:14.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:14.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.538507+0000", 2024-09-14T23:57:14.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1506e-05 2024-09-14T23:57:14.966 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:14.966 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:14.966 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:14.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.966 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T23:57:14.967 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:34.406877+0000", 2024-09-14T23:57:14.967 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 40.534833833, 2024-09-14T23:57:14.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030351169999999999, 2024-09-14T23:57:14.967 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:14.967 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:14.967 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:14.967 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:14.967 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:14.967 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:14.967 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 10 2024-09-14T23:57:14.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.967 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:14.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.967 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:14.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406877+0000", 2024-09-14T23:57:14.968 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:14.968 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.968 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.968 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:14.968 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406877+0000", 2024-09-14T23:57:14.968 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:14.968 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.968 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.968 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:14.968 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406878+0000", 2024-09-14T23:57:14.968 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.291e-06 2024-09-14T23:57:14.968 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.969 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.969 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:14.969 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406883+0000", 2024-09-14T23:57:14.969 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2399999999999998e-06 2024-09-14T23:57:14.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.969 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.969 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:14.969 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406891+0000", 2024-09-14T23:57:14.969 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5889999999999997e-06 2024-09-14T23:57:14.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.969 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.969 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:14.969 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406898+0000", 2024-09-14T23:57:14.969 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3599999999999998e-06 2024-09-14T23:57:14.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.970 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:14.970 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406948+0000", 2024-09-14T23:57:14.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9435999999999999e-05 2024-09-14T23:57:14.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.970 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-14T23:57:14.970 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406953+0000", 2024-09-14T23:57:14.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2809999999999999e-06 2024-09-14T23:57:14.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.970 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:14.970 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.408559+0000", 2024-09-14T23:57:14.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00160607 2024-09-14T23:57:14.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.971 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:14.971 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.408637+0000", 2024-09-14T23:57:14.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8504000000000006e-05 2024-09-14T23:57:14.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.971 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 4", 2024-09-14T23:57:14.971 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.408781+0000", 2024-09-14T23:57:14.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014374799999999999 2024-09-14T23:57:14.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.972 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.972 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:57:14.972 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.409659+0000", 2024-09-14T23:57:14.972 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000877781 2024-09-14T23:57:14.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.972 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.972 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:14.972 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.409822+0000", 2024-09-14T23:57:14.972 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016295400000000001 2024-09-14T23:57:14.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.972 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.972 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:14.973 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.409872+0000", 2024-09-14T23:57:14.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0022000000000003e-05 2024-09-14T23:57:14.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.973 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:14.973 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.409912+0000", 2024-09-14T23:57:14.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9841e-05 2024-09-14T23:57:14.973 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:14.973 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:14.973 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:14.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.973 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "MOSDPGPush(3.c 48/47 [PushOp(3:34183e86:::rbd_data.111888d8b54.0000000000000003:head, version: 37'112, data_included: [1473248~99616,1638400~65536,1835008~262144,2162688~65536,2293760~262144,2686976~65536,9737423~8043], data_size: 828555, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:34183e86:::rbd_data.111888d8b54.0000000000000003:head@37'112, size: 9745466, copy_subset: [(0, 9745466)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 9745466, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1473248, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-14T23:57:14.973 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:58.332069+0000", 2024-09-14T23:57:14.974 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 16.609641230000001, 2024-09-14T23:57:14.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0025177979999999999, 2024-09-14T23:57:14.974 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:14.974 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:14.974 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:14.974 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:14.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.974 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:14.974 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.332069+0000", 2024-09-14T23:57:14.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:14.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.975 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.975 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:14.975 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.332069+0000", 2024-09-14T23:57:14.975 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:14.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.975 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.975 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:14.975 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.332070+0000", 2024-09-14T23:57:14.975 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1200000000000003e-07 2024-09-14T23:57:14.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.975 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.975 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:14.975 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.332822+0000", 2024-09-14T23:57:14.975 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00075239299999999996 2024-09-14T23:57:14.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.976 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:14.976 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.332845+0000", 2024-09-14T23:57:14.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2436999999999999e-05 2024-09-14T23:57:14.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.976 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:14.976 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.332866+0000", 2024-09-14T23:57:14.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0874e-05 2024-09-14T23:57:14.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.976 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:14.976 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.332970+0000", 2024-09-14T23:57:14.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000104528 2024-09-14T23:57:14.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.977 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:14.977 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.332987+0000", 2024-09-14T23:57:14.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7073000000000001e-05 2024-09-14T23:57:14.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.977 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:14.977 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.334587+0000", 2024-09-14T23:57:14.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001599881 2024-09-14T23:57:14.977 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:14.977 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:14.978 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:14.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.978 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:50626 3.d e51/46 3:b5154cfe:::rbd_data.11183b9aac34.0000000000000004:head v 51'63, mlcod=45'61)", 2024-09-14T23:57:14.978 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:01.140129+0000", 2024-09-14T23:57:14.978 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.801581233, 2024-09-14T23:57:14.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0029883850000000001, 2024-09-14T23:57:14.978 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:14.978 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:14.978 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:14.979 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:14.979 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.979 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:14.979 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.140129+0000", 2024-09-14T23:57:14.979 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:14.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.979 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.979 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:14.979 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.140129+0000", 2024-09-14T23:57:14.979 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:14.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.979 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.980 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:14.980 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.140130+0000", 2024-09-14T23:57:14.980 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0529999999999999e-06 2024-09-14T23:57:14.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.980 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:14.980 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.140311+0000", 2024-09-14T23:57:14.980 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018054200000000001 2024-09-14T23:57:14.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.980 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:14.980 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.140325+0000", 2024-09-14T23:57:14.980 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.382e-05 2024-09-14T23:57:14.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.981 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.981 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:14.981 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.140332+0000", 2024-09-14T23:57:14.981 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5700000000000004e-06 2024-09-14T23:57:14.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.981 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.981 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:14.981 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.140392+0000", 2024-09-14T23:57:14.981 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9296000000000003e-05 2024-09-14T23:57:14.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.981 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.981 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:14.981 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.140436+0000", 2024-09-14T23:57:14.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4515999999999999e-05 2024-09-14T23:57:14.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.982 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:14.982 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.143075+0000", 2024-09-14T23:57:14.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0026385369999999998 2024-09-14T23:57:14.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.982 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:14.982 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.143118+0000", 2024-09-14T23:57:14.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3050999999999998e-05 2024-09-14T23:57:14.982 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:14.982 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:14.982 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:14.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.983 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.983 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:51277 3.d e52/46 3:b2645877:::rbd_data.11181975a1de.0000000000000001:head v 52'69, mlcod=51'66)", 2024-09-14T23:57:14.983 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:02.157938+0000", 2024-09-14T23:57:14.983 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 12.783772235000001, 2024-09-14T23:57:14.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0027517549999999998, 2024-09-14T23:57:14.983 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:14.983 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:14.983 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:14.983 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:14.983 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.983 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:14.983 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.157938+0000", 2024-09-14T23:57:14.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:14.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.984 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:14.984 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.157938+0000", 2024-09-14T23:57:14.984 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:14.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.984 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:14.984 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.157939+0000", 2024-09-14T23:57:14.984 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9199999999999999e-07 2024-09-14T23:57:14.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.984 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:14.984 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158543+0000", 2024-09-14T23:57:14.985 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00060333299999999995 2024-09-14T23:57:14.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.985 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.985 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:14.985 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158575+0000", 2024-09-14T23:57:14.985 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2801e-05 2024-09-14T23:57:14.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.985 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.985 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:14.985 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158586+0000", 2024-09-14T23:57:14.985 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.102e-05 2024-09-14T23:57:14.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.985 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.985 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:14.986 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158643+0000", 2024-09-14T23:57:14.986 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6320999999999998e-05 2024-09-14T23:57:14.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.986 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:14.986 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158693+0000", 2024-09-14T23:57:14.986 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0037000000000002e-05 2024-09-14T23:57:14.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.986 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:14.986 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.160641+0000", 2024-09-14T23:57:14.986 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0019485629999999999 2024-09-14T23:57:14.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.986 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:14.987 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.160690+0000", 2024-09-14T23:57:14.987 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8687999999999999e-05 2024-09-14T23:57:14.987 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:14.987 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:14.987 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:14.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.987 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:51275 3.2 e52/49 3:47a71653:::rbd_data.11181975a1de.0000000000000000:head v 52'118, mlcod=52'116)", 2024-09-14T23:57:14.987 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:02.158044+0000", 2024-09-14T23:57:14.987 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 12.783666663, 2024-09-14T23:57:14.987 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0028337200000000001, 2024-09-14T23:57:14.987 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:14.987 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:14.987 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:14.987 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:14.988 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.988 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:14.988 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158044+0000", 2024-09-14T23:57:14.988 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:14.988 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.988 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.988 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:14.988 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158044+0000", 2024-09-14T23:57:14.988 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:14.988 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.988 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.988 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:14.988 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158045+0000", 2024-09-14T23:57:14.989 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5700000000000002e-07 2024-09-14T23:57:14.989 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.989 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.989 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:14.989 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158637+0000", 2024-09-14T23:57:14.989 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00059278499999999997 2024-09-14T23:57:14.989 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.989 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.990 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:14.990 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158657+0000", 2024-09-14T23:57:14.990 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9422000000000001e-05 2024-09-14T23:57:14.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.990 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.990 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:14.990 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158675+0000", 2024-09-14T23:57:14.990 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8564999999999999e-05 2024-09-14T23:57:14.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.990 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.990 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:14.991 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158735+0000", 2024-09-14T23:57:14.991 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9904000000000001e-05 2024-09-14T23:57:14.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.991 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.991 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:14.991 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158757+0000", 2024-09-14T23:57:14.991 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1277e-05 2024-09-14T23:57:14.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.991 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.991 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:14.991 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.160845+0000", 2024-09-14T23:57:14.991 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0020881419999999999 2024-09-14T23:57:14.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.991 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.992 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:14.992 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.160878+0000", 2024-09-14T23:57:14.992 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2867999999999998e-05 2024-09-14T23:57:14.992 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:14.992 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:14.992 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:14.992 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.992 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.992 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56383 3.c e56/49 3:36b698b0:::rbd_data.1118c4b3aebe.0000000000000000:head v 56'194, mlcod=56'192)", 2024-09-14T23:57:14.992 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.270483+0000", 2024-09-14T23:57:14.992 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.6712276789999994, 2024-09-14T23:57:14.992 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0079685880000000004, 2024-09-14T23:57:14.992 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:14.992 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:14.993 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:14.993 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:14.993 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.993 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:14.993 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.270483+0000", 2024-09-14T23:57:14.993 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:14.993 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.993 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.993 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:14.993 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.270483+0000", 2024-09-14T23:57:14.993 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:14.993 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.993 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.993 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:14.994 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.270484+0000", 2024-09-14T23:57:14.994 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7400000000000002e-07 2024-09-14T23:57:14.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.994 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.994 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:14.994 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273680+0000", 2024-09-14T23:57:14.994 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031966680000000002 2024-09-14T23:57:14.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.994 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.994 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:14.994 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273697+0000", 2024-09-14T23:57:14.994 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6595e-05 2024-09-14T23:57:14.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.994 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.994 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:14.995 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273705+0000", 2024-09-14T23:57:14.995 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7530000000000001e-06 2024-09-14T23:57:14.995 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.995 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.995 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:14.995 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273763+0000", 2024-09-14T23:57:14.995 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8238e-05 2024-09-14T23:57:14.995 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.995 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.995 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:14.995 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273781+0000", 2024-09-14T23:57:14.996 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8281e-05 2024-09-14T23:57:14.996 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.996 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.996 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:14.996 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.278408+0000", 2024-09-14T23:57:14.996 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00462716 2024-09-14T23:57:14.996 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.996 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.996 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:14.996 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.278451+0000", 2024-09-14T23:57:14.996 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3019000000000003e-05 2024-09-14T23:57:14.996 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:14.997 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:14.997 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:14.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.997 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.997 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56384 3.c e56/49 3:36b698b0:::rbd_data.1118c4b3aebe.0000000000000000:head v 56'196, mlcod=56'192)", 2024-09-14T23:57:14.997 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:57:14.997 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.6671950070000001, 2024-09-14T23:57:14.997 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0046707529999999997, 2024-09-14T23:57:14.997 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:14.997 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:14.997 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:14.997 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:14.997 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.997 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:14.998 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:57:14.998 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:14.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.998 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:14.998 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:57:14.998 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:14.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.998 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:14.998 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:57:14.998 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1699999999999997e-07 2024-09-14T23:57:14.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.998 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:14.999 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274523+0000", 2024-09-14T23:57:14.999 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9129999999999997e-06 2024-09-14T23:57:14.999 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.999 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.999 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:14.999 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274535+0000", 2024-09-14T23:57:14.999 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1788e-05 2024-09-14T23:57:14.999 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.999 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.999 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:14.999 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274542+0000", 2024-09-14T23:57:14.999 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0539999999999997e-06 2024-09-14T23:57:14.999 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:14.999 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:14.999 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:15.000 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275889+0000", 2024-09-14T23:57:15.000 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00134669 2024-09-14T23:57:15.000 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.000 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.000 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:15.000 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275917+0000", 2024-09-14T23:57:15.000 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7664000000000001e-05 2024-09-14T23:57:15.000 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.000 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.000 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:15.000 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279134+0000", 2024-09-14T23:57:15.000 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0032178459999999999 2024-09-14T23:57:15.000 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.000 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:15.001 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279186+0000", 2024-09-14T23:57:15.001 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1881e-05 2024-09-14T23:57:15.001 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.001 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:15.001 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.001 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56381 3.1d e56/49 3:bcf9a8cf:::rbd_data.1118c4b3aebe.0000000000000006:head v 56'210, mlcod=56'208)", 2024-09-14T23:57:15.001 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.275462+0000", 2024-09-14T23:57:15.002 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.6662486770000005, 2024-09-14T23:57:15.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0071310710000000001, 2024-09-14T23:57:15.002 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:15.002 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:15.002 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:15.002 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:15.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.003 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:15.003 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275462+0000", 2024-09-14T23:57:15.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.003 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:15.003 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275462+0000", 2024-09-14T23:57:15.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.003 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:15.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275463+0000", 2024-09-14T23:57:15.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2699999999999999e-07 2024-09-14T23:57:15.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.004 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:15.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276779+0000", 2024-09-14T23:57:15.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0013163649999999999 2024-09-14T23:57:15.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.004 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:15.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276795+0000", 2024-09-14T23:57:15.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.59e-05 2024-09-14T23:57:15.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.005 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.005 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:15.005 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276806+0000", 2024-09-14T23:57:15.005 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1106e-05 2024-09-14T23:57:15.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.005 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.005 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:15.005 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276941+0000", 2024-09-14T23:57:15.005 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013477500000000001 2024-09-14T23:57:15.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.005 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.006 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:15.006 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276963+0000", 2024-09-14T23:57:15.006 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2419999999999999e-05 2024-09-14T23:57:15.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.006 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:15.006 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282546+0000", 2024-09-14T23:57:15.006 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0055827100000000003 2024-09-14T23:57:15.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.006 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:15.006 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282593+0000", 2024-09-14T23:57:15.006 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7067999999999997e-05 2024-09-14T23:57:15.006 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.007 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:15.007 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.007 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.007 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56387 3.c e56/49 3:30ee4ba0:::rbd_data.1118c4b3aebe.0000000000000004:head v 56'197, mlcod=56'192)", 2024-09-14T23:57:15.007 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.276498+0000", 2024-09-14T23:57:15.007 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.6652125820000006, 2024-09-14T23:57:15.007 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0092192639999999996, 2024-09-14T23:57:15.007 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:15.008 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:15.008 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:15.008 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:15.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:15.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276498+0000", 2024-09-14T23:57:15.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:15.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276498+0000", 2024-09-14T23:57:15.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.009 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:15.009 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276499+0000", 2024-09-14T23:57:15.009 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1100000000000004e-07 2024-09-14T23:57:15.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.009 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.009 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:15.009 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279646+0000", 2024-09-14T23:57:15.009 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031473220000000001 2024-09-14T23:57:15.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.009 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.009 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:15.009 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279672+0000", 2024-09-14T23:57:15.009 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6129999999999999e-05 2024-09-14T23:57:15.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.010 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.010 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:15.010 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279682+0000", 2024-09-14T23:57:15.010 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7170000000000003e-06 2024-09-14T23:57:15.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.010 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.010 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:15.010 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279742+0000", 2024-09-14T23:57:15.010 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9926999999999997e-05 2024-09-14T23:57:15.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.010 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.010 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:15.010 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279761+0000", 2024-09-14T23:57:15.010 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9402000000000002e-05 2024-09-14T23:57:15.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.011 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.011 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:15.011 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.285680+0000", 2024-09-14T23:57:15.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.005918881 2024-09-14T23:57:15.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.011 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.011 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:15.011 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.285717+0000", 2024-09-14T23:57:15.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6974000000000002e-05 2024-09-14T23:57:15.012 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.012 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:15.012 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.012 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56388 3.c e56/49 3:30ee4ba0:::rbd_data.1118c4b3aebe.0000000000000004:head v 56'199, mlcod=56'192)", 2024-09-14T23:57:15.012 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:57:15.012 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.6617209939999995, 2024-09-14T23:57:15.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0060596740000000001, 2024-09-14T23:57:15.012 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:15.013 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:15.013 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:15.013 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:15.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.013 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:15.013 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:57:15.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.013 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:15.013 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:57:15.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.014 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:15.014 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:57:15.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2200000000000003e-07 2024-09-14T23:57:15.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.014 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:15.014 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279995+0000", 2024-09-14T23:57:15.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5569999999999997e-06 2024-09-14T23:57:15.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.014 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:15.014 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280004+0000", 2024-09-14T23:57:15.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0080000000000008e-06 2024-09-14T23:57:15.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.015 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:15.015 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280010+0000", 2024-09-14T23:57:15.015 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5749999999999997e-06 2024-09-14T23:57:15.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.015 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:15.015 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282968+0000", 2024-09-14T23:57:15.015 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0029579390000000001 2024-09-14T23:57:15.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.016 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:15.016 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282998+0000", 2024-09-14T23:57:15.016 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9947999999999999e-05 2024-09-14T23:57:15.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.016 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.016 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:15.016 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.286012+0000", 2024-09-14T23:57:15.016 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030132900000000001 2024-09-14T23:57:15.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.016 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.016 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:15.016 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.286049+0000", 2024-09-14T23:57:15.016 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7534999999999999e-05 2024-09-14T23:57:15.016 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.017 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:15.017 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.017 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.017 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56382 3.1d e56/49 3:bcf9a8cf:::rbd_data.1118c4b3aebe.0000000000000006:head v 56'212, mlcod=56'208)", 2024-09-14T23:57:15.017 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.280285+0000", 2024-09-14T23:57:15.017 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.6614253110000003, 2024-09-14T23:57:15.017 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035393489999999998, 2024-09-14T23:57:15.017 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:15.017 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:15.017 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:15.017 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:15.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.018 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:15.018 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280285+0000", 2024-09-14T23:57:15.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.018 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:15.018 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280285+0000", 2024-09-14T23:57:15.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.018 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:15.018 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280286+0000", 2024-09-14T23:57:15.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5000000000000001e-07 2024-09-14T23:57:15.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:15.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280299+0000", 2024-09-14T23:57:15.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3312000000000001e-05 2024-09-14T23:57:15.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:15.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280311+0000", 2024-09-14T23:57:15.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1677e-05 2024-09-14T23:57:15.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:15.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280318+0000", 2024-09-14T23:57:15.020 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.742e-06 2024-09-14T23:57:15.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.020 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:15.020 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280374+0000", 2024-09-14T23:57:15.020 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.592e-05 2024-09-14T23:57:15.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.020 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:15.020 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280392+0000", 2024-09-14T23:57:15.020 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8490000000000001e-05 2024-09-14T23:57:15.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.020 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:15.021 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.283693+0000", 2024-09-14T23:57:15.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033012050000000002 2024-09-14T23:57:15.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.021 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:15.021 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.283825+0000", 2024-09-14T23:57:15.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013105300000000001 2024-09-14T23:57:15.021 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.021 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:15.021 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.021 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "pg_update_log_missing(3.c epoch 56/49 rep_tid 1493 entries 56'200 (0'0) error 3:370203a3:::rbd_data.1118c4b3aebe.000000000000000d:head by client.4376.0:56396 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 56'192)", 2024-09-14T23:57:15.021 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:57:15.022 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.6612008540000005, 2024-09-14T23:57:15.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057370099999999999, 2024-09-14T23:57:15.022 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:15.022 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:15.022 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:57:15.022 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:15.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.022 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:15.022 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:57:15.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.022 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:15.022 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:57:15.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.023 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:15.023 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:57:15.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0699999999999997e-07 2024-09-14T23:57:15.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.023 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:15.023 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280519+0000", 2024-09-14T23:57:15.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6260000000000001e-06 2024-09-14T23:57:15.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.023 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:15.024 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280541+0000", 2024-09-14T23:57:15.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1976000000000001e-05 2024-09-14T23:57:15.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.024 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:15.024 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280556+0000", 2024-09-14T23:57:15.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5075000000000001e-05 2024-09-14T23:57:15.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.024 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:15.024 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.285494+0000", 2024-09-14T23:57:15.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0049378690000000001 2024-09-14T23:57:15.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:15.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.286247+0000", 2024-09-14T23:57:15.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00075285700000000001 2024-09-14T23:57:15.025 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.025 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:15.025 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.025 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56400 3.1c e56/49 3:3f5b81ba:::rbd_data.1118c4b3aebe.000000000000000a:head v 56'236, mlcod=56'234)", 2024-09-14T23:57:15.025 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.295064+0000", 2024-09-14T23:57:15.025 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.6466460620000003, 2024-09-14T23:57:15.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0054331589999999999, 2024-09-14T23:57:15.025 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:15.026 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:15.026 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:15.026 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:15.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:15.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.295064+0000", 2024-09-14T23:57:15.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:15.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.295064+0000", 2024-09-14T23:57:15.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.027 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:15.027 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.295066+0000", 2024-09-14T23:57:15.027 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.502e-06 2024-09-14T23:57:15.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.027 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.027 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:15.027 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296403+0000", 2024-09-14T23:57:15.027 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001336554 2024-09-14T23:57:15.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.027 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.027 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:15.027 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296428+0000", 2024-09-14T23:57:15.027 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5179000000000001e-05 2024-09-14T23:57:15.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.028 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:15.028 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296440+0000", 2024-09-14T23:57:15.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2387e-05 2024-09-14T23:57:15.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.028 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:15.028 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296554+0000", 2024-09-14T23:57:15.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114269 2024-09-14T23:57:15.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.028 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:15.028 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296581+0000", 2024-09-14T23:57:15.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6673999999999999e-05 2024-09-14T23:57:15.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:15.029 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.300418+0000", 2024-09-14T23:57:15.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0038371249999999998 2024-09-14T23:57:15.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:15.029 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.300498+0000", 2024-09-14T23:57:15.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9468999999999994e-05 2024-09-14T23:57:15.029 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.029 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:15.029 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.030 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56401 3.1c e56/49 3:3f5b81ba:::rbd_data.1118c4b3aebe.000000000000000a:head v 56'238, mlcod=56'234)", 2024-09-14T23:57:15.030 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.297640+0000", 2024-09-14T23:57:15.030 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.6440702229999999, 2024-09-14T23:57:15.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031491919999999999, 2024-09-14T23:57:15.030 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:15.030 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:15.030 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:15.030 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:15.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:15.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.297640+0000", 2024-09-14T23:57:15.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.031 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:15.031 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.297640+0000", 2024-09-14T23:57:15.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.031 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:15.031 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.297641+0000", 2024-09-14T23:57:15.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9000000000000005e-07 2024-09-14T23:57:15.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.031 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:15.031 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.297649+0000", 2024-09-14T23:57:15.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4510000000000003e-06 2024-09-14T23:57:15.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:15.032 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.297663+0000", 2024-09-14T23:57:15.032 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4262e-05 2024-09-14T23:57:15.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:15.032 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.297679+0000", 2024-09-14T23:57:15.032 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5798e-05 2024-09-14T23:57:15.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:15.032 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.298408+0000", 2024-09-14T23:57:15.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00072923699999999998 2024-09-14T23:57:15.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.033 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:15.033 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.298430+0000", 2024-09-14T23:57:15.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1753999999999999e-05 2024-09-14T23:57:15.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.033 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:15.033 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.300749+0000", 2024-09-14T23:57:15.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023190590000000001 2024-09-14T23:57:15.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.034 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:15.034 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.300789+0000", 2024-09-14T23:57:15.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0640999999999999e-05 2024-09-14T23:57:15.034 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.034 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:15.034 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.034 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:61921 3.5 e57/49 3:a05d004f:::rbd_data.11188d356229.000000000000000b:head v 57'279, mlcod=57'277)", 2024-09-14T23:57:15.034 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:07.560389+0000", 2024-09-14T23:57:15.034 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.381321464, 2024-09-14T23:57:15.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0034956520000000001, 2024-09-14T23:57:15.034 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:15.034 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:15.035 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:15.035 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:15.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:15.035 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.560389+0000", 2024-09-14T23:57:15.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:15.035 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.560389+0000", 2024-09-14T23:57:15.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:15.035 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.560390+0000", 2024-09-14T23:57:15.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4109999999999999e-06 2024-09-14T23:57:15.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:15.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.561839+0000", 2024-09-14T23:57:15.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001448406 2024-09-14T23:57:15.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:15.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.561854+0000", 2024-09-14T23:57:15.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5426000000000001e-05 2024-09-14T23:57:15.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:15.037 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.561863+0000", 2024-09-14T23:57:15.037 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7509999999999997e-06 2024-09-14T23:57:15.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.037 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.037 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:15.037 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.561917+0000", 2024-09-14T23:57:15.037 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3531e-05 2024-09-14T23:57:15.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.037 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.037 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:15.037 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.561944+0000", 2024-09-14T23:57:15.037 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7528000000000001e-05 2024-09-14T23:57:15.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.038 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.038 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:15.038 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563843+0000", 2024-09-14T23:57:15.038 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001899361 2024-09-14T23:57:15.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.038 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.038 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:15.038 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563885+0000", 2024-09-14T23:57:15.038 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1238000000000001e-05 2024-09-14T23:57:15.038 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.039 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:15.039 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.039 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:61922 3.5 e57/49 3:a11ebf8e:::rbd_data.11188d356229.000000000000000c:head v 57'280, mlcod=57'277)", 2024-09-14T23:57:15.039 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:07.563144+0000", 2024-09-14T23:57:15.039 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.3785664100000004, 2024-09-14T23:57:15.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003977848, 2024-09-14T23:57:15.039 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:15.039 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:15.039 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:15.039 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:15.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:15.040 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563144+0000", 2024-09-14T23:57:15.040 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.040 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:15.040 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563144+0000", 2024-09-14T23:57:15.040 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.040 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:15.040 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563145+0000", 2024-09-14T23:57:15.040 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5400000000000001e-07 2024-09-14T23:57:15.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.041 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.041 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:15.041 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565056+0000", 2024-09-14T23:57:15.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0019108160000000001 2024-09-14T23:57:15.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.041 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.041 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:15.041 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565088+0000", 2024-09-14T23:57:15.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2586999999999998e-05 2024-09-14T23:57:15.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.041 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.041 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:15.041 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565100+0000", 2024-09-14T23:57:15.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1917e-05 2024-09-14T23:57:15.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.042 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.042 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:15.042 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565166+0000", 2024-09-14T23:57:15.042 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6245e-05 2024-09-14T23:57:15.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.042 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.042 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:15.042 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565190+0000", 2024-09-14T23:57:15.042 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3825999999999999e-05 2024-09-14T23:57:15.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.042 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.042 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:15.042 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.567080+0000", 2024-09-14T23:57:15.042 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001889541 2024-09-14T23:57:15.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.043 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:15.043 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.567122+0000", 2024-09-14T23:57:15.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2262000000000003e-05 2024-09-14T23:57:15.043 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.043 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:15.043 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.043 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:66564 3.c e60/49 3:33faa7ca:::rbd_data.111873792d9b.0000000000000003:head v 60'236, mlcod=57'233)", 2024-09-14T23:57:15.043 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:11.271596+0000", 2024-09-14T23:57:15.043 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.6701148319999999, 2024-09-14T23:57:15.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0060396099999999999, 2024-09-14T23:57:15.044 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:15.044 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:15.044 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:15.044 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:15.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.044 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:15.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.271596+0000", 2024-09-14T23:57:15.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.044 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:15.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.271596+0000", 2024-09-14T23:57:15.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.045 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.045 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:15.045 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.271597+0000", 2024-09-14T23:57:15.045 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2559999999999999e-06 2024-09-14T23:57:15.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.045 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.045 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:15.045 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.274854+0000", 2024-09-14T23:57:15.045 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0032574470000000001 2024-09-14T23:57:15.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.045 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.045 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:15.045 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.274874+0000", 2024-09-14T23:57:15.045 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9449999999999998e-05 2024-09-14T23:57:15.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.046 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.046 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:15.046 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.274883+0000", 2024-09-14T23:57:15.046 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5980000000000003e-06 2024-09-14T23:57:15.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.046 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.046 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:15.046 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.274977+0000", 2024-09-14T23:57:15.046 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3270999999999995e-05 2024-09-14T23:57:15.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.046 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.046 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:15.046 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.275000+0000", 2024-09-14T23:57:15.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3088e-05 2024-09-14T23:57:15.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.047 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:15.047 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.277590+0000", 2024-09-14T23:57:15.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0025898660000000001 2024-09-14T23:57:15.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.047 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:15.047 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.277635+0000", 2024-09-14T23:57:15.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5633999999999997e-05 2024-09-14T23:57:15.047 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.047 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:15.047 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.048 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.048 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:15.048 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:15.048 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:57:15.049 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:57:15.049 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:57:15.050 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T23:57:15.051 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:57:15.051 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:15.048+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:15.115 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.0 to 1 (10000) 2024-09-14T23:57:15.123 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T23:57:15.131 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-14T23:57:15.149 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:15.148+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:15.161 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:15.162 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:15.162 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:57:15.162 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:15.171 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T23:57:15.212 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone22 from 0xecea973 (overlap 0xecea973) to 0x17b10f1 2024-09-14T23:57:15.250 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:15.249+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:15.351 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:15.349+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:15.360 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:15.360 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:15.360 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:57:15.360 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:57:15.360 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:15.372 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T23:57:15.453 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:15.451+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:15.528 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:15.528 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:57:15.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:57:15.529 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:57:15.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.529 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68736 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 5906432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:15.529 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.264178+0000", 2024-09-14T23:57:15.529 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.26197429900000002, 2024-09-14T23:57:15.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.045913849999999999, 2024-09-14T23:57:15.529 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:15.529 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:15.529 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:15.529 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:15.529 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:15.530 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:15.530 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68736 2024-09-14T23:57:15.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.530 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:15.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.530 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:15.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264178+0000", 2024-09-14T23:57:15.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.530 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:15.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264178+0000", 2024-09-14T23:57:15.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:15.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264179+0000", 2024-09-14T23:57:15.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8599999999999999e-07 2024-09-14T23:57:15.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:15.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264181+0000", 2024-09-14T23:57:15.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.855e-06 2024-09-14T23:57:15.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:15.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264187+0000", 2024-09-14T23:57:15.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1539999999999999e-06 2024-09-14T23:57:15.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:15.532 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264192+0000", 2024-09-14T23:57:15.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6409999999999999e-06 2024-09-14T23:57:15.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:15.532 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.309931+0000", 2024-09-14T23:57:15.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.045738442999999997 2024-09-14T23:57:15.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.533 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:15.533 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310017+0000", 2024-09-14T23:57:15.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5829999999999996e-05 2024-09-14T23:57:15.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.533 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:15.533 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310092+0000", 2024-09-14T23:57:15.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5541000000000001e-05 2024-09-14T23:57:15.533 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.533 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:15.533 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.533 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68737 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 6021120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:15.534 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.264199+0000", 2024-09-14T23:57:15.534 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.261953834, 2024-09-14T23:57:15.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046063354000000001, 2024-09-14T23:57:15.534 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:15.534 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:15.534 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:15.534 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:15.534 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:15.534 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:15.534 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68737 2024-09-14T23:57:15.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.534 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:15.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:15.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264199+0000", 2024-09-14T23:57:15.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:15.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264199+0000", 2024-09-14T23:57:15.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:15.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264199+0000", 2024-09-14T23:57:15.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1600000000000002e-07 2024-09-14T23:57:15.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:15.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264210+0000", 2024-09-14T23:57:15.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0973e-05 2024-09-14T23:57:15.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:15.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264217+0000", 2024-09-14T23:57:15.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0659999999999998e-06 2024-09-14T23:57:15.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:15.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264223+0000", 2024-09-14T23:57:15.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2039999999999996e-06 2024-09-14T23:57:15.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:15.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310121+0000", 2024-09-14T23:57:15.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.045898436000000001 2024-09-14T23:57:15.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.538 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:15.538 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310194+0000", 2024-09-14T23:57:15.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2708999999999998e-05 2024-09-14T23:57:15.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.538 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:15.538 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310262+0000", 2024-09-14T23:57:15.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8549999999999999e-05 2024-09-14T23:57:15.538 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.538 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:15.538 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.539 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68738 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 6135808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:15.539 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.264234+0000", 2024-09-14T23:57:15.539 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.26191840399999999, 2024-09-14T23:57:15.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046212954000000001, 2024-09-14T23:57:15.539 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:15.539 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:15.539 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:15.539 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:15.539 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:15.539 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:15.540 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68738 2024-09-14T23:57:15.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.540 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:15.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:15.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264234+0000", 2024-09-14T23:57:15.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:15.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264234+0000", 2024-09-14T23:57:15.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:15.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264235+0000", 2024-09-14T23:57:15.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6600000000000002e-07 2024-09-14T23:57:15.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:15.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264237+0000", 2024-09-14T23:57:15.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3120000000000001e-06 2024-09-14T23:57:15.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:15.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264244+0000", 2024-09-14T23:57:15.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6680000000000004e-06 2024-09-14T23:57:15.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:15.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264255+0000", 2024-09-14T23:57:15.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.155e-05 2024-09-14T23:57:15.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.543 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:15.543 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310292+0000", 2024-09-14T23:57:15.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046037026000000002 2024-09-14T23:57:15.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.543 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:15.543 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310375+0000", 2024-09-14T23:57:15.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3039000000000002e-05 2024-09-14T23:57:15.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.543 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:15.543 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310447+0000", 2024-09-14T23:57:15.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1892999999999998e-05 2024-09-14T23:57:15.544 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.544 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:15.544 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.544 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68739 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 6250496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:15.544 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.264263+0000", 2024-09-14T23:57:15.544 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.26188984700000001, 2024-09-14T23:57:15.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046362974000000001, 2024-09-14T23:57:15.544 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:15.544 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:15.544 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:15.545 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:15.545 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:15.545 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:15.545 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68739 2024-09-14T23:57:15.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.545 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:15.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:15.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264263+0000", 2024-09-14T23:57:15.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:15.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264263+0000", 2024-09-14T23:57:15.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:15.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264263+0000", 2024-09-14T23:57:15.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8599999999999999e-07 2024-09-14T23:57:15.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:15.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264265+0000", 2024-09-14T23:57:15.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0020000000000001e-06 2024-09-14T23:57:15.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:15.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264272+0000", 2024-09-14T23:57:15.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4799999999999998e-06 2024-09-14T23:57:15.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:15.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264282+0000", 2024-09-14T23:57:15.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0244999999999999e-05 2024-09-14T23:57:15.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:15.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310478+0000", 2024-09-14T23:57:15.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046196070999999998 2024-09-14T23:57:15.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.548 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.548 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:15.548 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310554+0000", 2024-09-14T23:57:15.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6160000000000003e-05 2024-09-14T23:57:15.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.548 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.548 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:15.548 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310626+0000", 2024-09-14T23:57:15.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1630000000000004e-05 2024-09-14T23:57:15.548 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.548 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:15.548 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.549 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68740 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 6365184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:15.549 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.264289+0000", 2024-09-14T23:57:15.549 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.261863391, 2024-09-14T23:57:15.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046531222999999997, 2024-09-14T23:57:15.549 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:15.549 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:15.549 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:15.549 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:15.549 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:15.549 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:15.549 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68740 2024-09-14T23:57:15.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.550 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:15.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:15.550 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264289+0000", 2024-09-14T23:57:15.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:15.550 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264289+0000", 2024-09-14T23:57:15.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:15.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264290+0000", 2024-09-14T23:57:15.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4299999999999998e-07 2024-09-14T23:57:15.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:15.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264292+0000", 2024-09-14T23:57:15.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.181e-06 2024-09-14T23:57:15.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:15.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264298+0000", 2024-09-14T23:57:15.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1120000000000001e-06 2024-09-14T23:57:15.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.552 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.552 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:15.552 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264303+0000", 2024-09-14T23:57:15.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5839999999999996e-06 2024-09-14T23:57:15.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.552 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.552 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:15.552 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310654+0000", 2024-09-14T23:57:15.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046351046 2024-09-14T23:57:15.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.552 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.552 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:15.552 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310745+0000", 2024-09-14T23:57:15.553 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1027999999999997e-05 2024-09-14T23:57:15.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.553 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:15.553 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310821+0000", 2024-09-14T23:57:15.553 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5828999999999995e-05 2024-09-14T23:57:15.553 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.553 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:15.553 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.553 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68741 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 6479872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:15.553 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.264332+0000", 2024-09-14T23:57:15.553 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.26182042100000003, 2024-09-14T23:57:15.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046692392999999999, 2024-09-14T23:57:15.554 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:15.554 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:15.554 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:15.554 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:15.554 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:15.554 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:15.554 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68741 2024-09-14T23:57:15.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.554 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:15.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:15.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264332+0000", 2024-09-14T23:57:15.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:15.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264332+0000", 2024-09-14T23:57:15.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:15.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264333+0000", 2024-09-14T23:57:15.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2899999999999999e-07 2024-09-14T23:57:15.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:15.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264335+0000", 2024-09-14T23:57:15.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9870000000000002e-06 2024-09-14T23:57:15.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.556 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.556 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:15.556 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264340+0000", 2024-09-14T23:57:15.556 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0069999999999997e-06 2024-09-14T23:57:15.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.556 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.556 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:15.556 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264343+0000", 2024-09-14T23:57:15.556 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0960000000000001e-06 2024-09-14T23:57:15.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.556 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.556 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:15.557 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310855+0000", 2024-09-14T23:57:15.557 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046511915000000001 2024-09-14T23:57:15.557 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.557 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.557 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:15.557 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310946+0000", 2024-09-14T23:57:15.557 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1290000000000002e-05 2024-09-14T23:57:15.557 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.557 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.557 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:15.557 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311025+0000", 2024-09-14T23:57:15.557 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8769000000000004e-05 2024-09-14T23:57:15.557 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.557 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:15.558 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.558 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.558 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68742 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 6594560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:15.558 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.264347+0000", 2024-09-14T23:57:15.558 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.261805379, 2024-09-14T23:57:15.558 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046904279, 2024-09-14T23:57:15.558 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:15.559 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:15.559 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:15.559 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:15.559 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:15.559 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:15.559 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68742 2024-09-14T23:57:15.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.559 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:15.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:15.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264347+0000", 2024-09-14T23:57:15.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:15.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264347+0000", 2024-09-14T23:57:15.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:15.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264348+0000", 2024-09-14T23:57:15.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.22e-07 2024-09-14T23:57:15.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:15.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264349+0000", 2024-09-14T23:57:15.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2079999999999999e-06 2024-09-14T23:57:15.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.561 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:15.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264352+0000", 2024-09-14T23:57:15.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1640000000000001e-06 2024-09-14T23:57:15.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.561 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:15.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264354+0000", 2024-09-14T23:57:15.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3350000000000001e-06 2024-09-14T23:57:15.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:15.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311071+0000", 2024-09-14T23:57:15.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046716921000000002 2024-09-14T23:57:15.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:15.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311165+0000", 2024-09-14T23:57:15.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3900000000000006e-05 2024-09-14T23:57:15.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:15.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311252+0000", 2024-09-14T23:57:15.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6428999999999995e-05 2024-09-14T23:57:15.563 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.563 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:15.563 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.563 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68743 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 6709248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:15.563 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.264359+0000", 2024-09-14T23:57:15.563 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.26179374999999999, 2024-09-14T23:57:15.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047086817000000003, 2024-09-14T23:57:15.563 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:15.563 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:15.563 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:15.564 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:15.564 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:15.564 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:15.564 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68743 2024-09-14T23:57:15.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.564 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:15.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:15.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264359+0000", 2024-09-14T23:57:15.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:15.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264359+0000", 2024-09-14T23:57:15.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.565 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:15.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265672+0000", 2024-09-14T23:57:15.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0013131849999999999 2024-09-14T23:57:15.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.565 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:15.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265676+0000", 2024-09-14T23:57:15.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2030000000000002e-06 2024-09-14T23:57:15.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:15.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265685+0000", 2024-09-14T23:57:15.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7290000000000006e-06 2024-09-14T23:57:15.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:15.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265691+0000", 2024-09-14T23:57:15.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3779999999999998e-06 2024-09-14T23:57:15.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:15.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311283+0000", 2024-09-14T23:57:15.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.045591797000000003 2024-09-14T23:57:15.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.567 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:15.567 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311368+0000", 2024-09-14T23:57:15.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4444999999999995e-05 2024-09-14T23:57:15.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.567 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:15.567 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311446+0000", 2024-09-14T23:57:15.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8079999999999998e-05 2024-09-14T23:57:15.567 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.567 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:15.567 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.568 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68744 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 6823936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:15.568 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.265703+0000", 2024-09-14T23:57:15.568 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.260449975, 2024-09-14T23:57:15.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.045950957000000001, 2024-09-14T23:57:15.568 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:15.568 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:15.568 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:15.568 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:15.568 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:15.568 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:15.568 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68744 2024-09-14T23:57:15.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.569 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:15.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.569 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:15.569 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265703+0000", 2024-09-14T23:57:15.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.569 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:15.569 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265703+0000", 2024-09-14T23:57:15.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.569 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:15.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265703+0000", 2024-09-14T23:57:15.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9900000000000001e-07 2024-09-14T23:57:15.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.570 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:15.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265706+0000", 2024-09-14T23:57:15.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4329999999999998e-06 2024-09-14T23:57:15.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.570 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:15.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265712+0000", 2024-09-14T23:57:15.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8020000000000003e-06 2024-09-14T23:57:15.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:15.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265718+0000", 2024-09-14T23:57:15.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.13e-06 2024-09-14T23:57:15.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:15.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311494+0000", 2024-09-14T23:57:15.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.045776039999999997 2024-09-14T23:57:15.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:15.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311577+0000", 2024-09-14T23:57:15.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3057000000000001e-05 2024-09-14T23:57:15.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:15.572 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311654+0000", 2024-09-14T23:57:15.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6996000000000006e-05 2024-09-14T23:57:15.572 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.572 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:15.572 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.572 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68745 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 6938624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:15.572 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.265725+0000", 2024-09-14T23:57:15.572 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.260428145, 2024-09-14T23:57:15.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046101672000000003, 2024-09-14T23:57:15.573 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:15.573 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:15.573 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:15.573 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:15.573 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:15.573 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:15.573 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68745 2024-09-14T23:57:15.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.573 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:15.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.573 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:15.573 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265725+0000", 2024-09-14T23:57:15.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.574 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:15.574 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265725+0000", 2024-09-14T23:57:15.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.574 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:15.574 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265725+0000", 2024-09-14T23:57:15.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5299999999999999e-07 2024-09-14T23:57:15.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:15.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265727+0000", 2024-09-14T23:57:15.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.069e-06 2024-09-14T23:57:15.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:15.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265733+0000", 2024-09-14T23:57:15.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1510000000000001e-06 2024-09-14T23:57:15.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:15.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265738+0000", 2024-09-14T23:57:15.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7010000000000004e-06 2024-09-14T23:57:15.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:15.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311707+0000", 2024-09-14T23:57:15.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.045968602999999997 2024-09-14T23:57:15.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:15.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311776+0000", 2024-09-14T23:57:15.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9438999999999997e-05 2024-09-14T23:57:15.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:15.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311826+0000", 2024-09-14T23:57:15.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0256000000000001e-05 2024-09-14T23:57:15.577 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.577 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:15.577 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.577 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68746 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 7053312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:15.577 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.265744+0000", 2024-09-14T23:57:15.577 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.26040851399999998, 2024-09-14T23:57:15.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046203505999999998, 2024-09-14T23:57:15.577 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:15.577 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:15.577 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:15.577 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:15.578 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:15.578 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:15.578 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68746 2024-09-14T23:57:15.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.578 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:15.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:15.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265744+0000", 2024-09-14T23:57:15.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:15.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265744+0000", 2024-09-14T23:57:15.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:15.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265745+0000", 2024-09-14T23:57:15.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9200000000000002e-07 2024-09-14T23:57:15.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:15.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265746+0000", 2024-09-14T23:57:15.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8819999999999999e-06 2024-09-14T23:57:15.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:15.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265753+0000", 2024-09-14T23:57:15.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1990000000000003e-06 2024-09-14T23:57:15.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:15.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265760+0000", 2024-09-14T23:57:15.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9280000000000004e-06 2024-09-14T23:57:15.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:15.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311859+0000", 2024-09-14T23:57:15.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046099627999999997 2024-09-14T23:57:15.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:15.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311902+0000", 2024-09-14T23:57:15.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3239999999999999e-05 2024-09-14T23:57:15.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:15.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311948+0000", 2024-09-14T23:57:15.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5237000000000001e-05 2024-09-14T23:57:15.581 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.581 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:15.582 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.582 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68747 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 7168000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:15.582 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.265766+0000", 2024-09-14T23:57:15.582 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.26038635300000001, 2024-09-14T23:57:15.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046303703000000002, 2024-09-14T23:57:15.582 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:15.582 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:15.582 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:15.582 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:15.582 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:15.582 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:15.583 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68747 2024-09-14T23:57:15.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.583 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:15.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:15.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265766+0000", 2024-09-14T23:57:15.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:15.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265766+0000", 2024-09-14T23:57:15.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.584 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:15.584 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265767+0000", 2024-09-14T23:57:15.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9200000000000002e-07 2024-09-14T23:57:15.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.584 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:15.584 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265769+0000", 2024-09-14T23:57:15.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9369999999999998e-06 2024-09-14T23:57:15.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.584 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:15.584 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265775+0000", 2024-09-14T23:57:15.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9499999999999998e-06 2024-09-14T23:57:15.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:15.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265779+0000", 2024-09-14T23:57:15.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7979999999999996e-06 2024-09-14T23:57:15.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:15.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311977+0000", 2024-09-14T23:57:15.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046197437000000001 2024-09-14T23:57:15.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:15.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.312024+0000", 2024-09-14T23:57:15.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6780999999999998e-05 2024-09-14T23:57:15.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:15.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.312070+0000", 2024-09-14T23:57:15.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6408e-05 2024-09-14T23:57:15.586 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.586 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:15.586 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.586 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68748 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 7282688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:15.587 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.265785+0000", 2024-09-14T23:57:15.587 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.26036729199999997, 2024-09-14T23:57:15.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046405109999999999, 2024-09-14T23:57:15.587 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:15.587 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:15.587 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:15.587 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:15.587 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:15.587 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:15.587 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68748 2024-09-14T23:57:15.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.587 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:15.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:15.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265785+0000", 2024-09-14T23:57:15.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:15.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265785+0000", 2024-09-14T23:57:15.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:15.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265786+0000", 2024-09-14T23:57:15.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0900000000000002e-07 2024-09-14T23:57:15.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:15.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265796+0000", 2024-09-14T23:57:15.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8490000000000001e-06 2024-09-14T23:57:15.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:15.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265802+0000", 2024-09-14T23:57:15.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7309999999999996e-06 2024-09-14T23:57:15.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:15.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265812+0000", 2024-09-14T23:57:15.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8789999999999998e-06 2024-09-14T23:57:15.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:15.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.312098+0000", 2024-09-14T23:57:15.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046285862999999997 2024-09-14T23:57:15.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:15.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.312140+0000", 2024-09-14T23:57:15.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1718000000000003e-05 2024-09-14T23:57:15.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:15.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.312190+0000", 2024-09-14T23:57:15.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0661000000000002e-05 2024-09-14T23:57:15.591 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.591 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:15.591 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.591 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68749 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 7397376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:15.592 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.265821+0000", 2024-09-14T23:57:15.592 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.260331796, 2024-09-14T23:57:15.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046484708, 2024-09-14T23:57:15.592 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:15.592 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:15.592 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:15.592 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:15.592 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:15.592 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:15.592 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68749 2024-09-14T23:57:15.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.592 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:15.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:15.593 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265821+0000", 2024-09-14T23:57:15.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:15.593 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265821+0000", 2024-09-14T23:57:15.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:15.593 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265821+0000", 2024-09-14T23:57:15.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7300000000000001e-07 2024-09-14T23:57:15.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.594 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:15.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265829+0000", 2024-09-14T23:57:15.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8299999999999996e-06 2024-09-14T23:57:15.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.594 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:15.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265836+0000", 2024-09-14T23:57:15.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0709999999999997e-06 2024-09-14T23:57:15.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:15.595 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265841+0000", 2024-09-14T23:57:15.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1950000000000002e-06 2024-09-14T23:57:15.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:15.595 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.312218+0000", 2024-09-14T23:57:15.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046376873999999998 2024-09-14T23:57:15.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:15.595 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.312260+0000", 2024-09-14T23:57:15.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1693000000000002e-05 2024-09-14T23:57:15.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.596 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:15.596 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.312306+0000", 2024-09-14T23:57:15.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5571999999999997e-05 2024-09-14T23:57:15.596 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.596 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:15.596 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.596 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68750 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 7512064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:15.597 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.265863+0000", 2024-09-14T23:57:15.597 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.26029001699999998, 2024-09-14T23:57:15.597 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046634766000000001, 2024-09-14T23:57:15.597 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:15.597 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:15.597 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:15.597 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:15.597 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:15.597 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:15.597 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68750 2024-09-14T23:57:15.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.597 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:15.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.598 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:15.598 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265863+0000", 2024-09-14T23:57:15.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.598 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:15.598 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265863+0000", 2024-09-14T23:57:15.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.598 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:15.598 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265863+0000", 2024-09-14T23:57:15.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0900000000000002e-07 2024-09-14T23:57:15.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.599 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:15.599 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265865+0000", 2024-09-14T23:57:15.599 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9740000000000001e-06 2024-09-14T23:57:15.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.599 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:15.599 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265869+0000", 2024-09-14T23:57:15.599 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4159999999999997e-06 2024-09-14T23:57:15.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.599 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:15.599 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265873+0000", 2024-09-14T23:57:15.599 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1999999999999999e-06 2024-09-14T23:57:15.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.600 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:15.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.312334+0000", 2024-09-14T23:57:15.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046461737000000003 2024-09-14T23:57:15.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.600 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:15.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.312415+0000", 2024-09-14T23:57:15.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0370999999999993e-05 2024-09-14T23:57:15.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.600 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:15.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.312497+0000", 2024-09-14T23:57:15.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2658999999999996e-05 2024-09-14T23:57:15.601 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.601 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:15.601 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.601 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68751 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 7626752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:15.601 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.265877+0000", 2024-09-14T23:57:15.601 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.26027561599999999, 2024-09-14T23:57:15.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046759231999999998, 2024-09-14T23:57:15.601 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:15.601 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:15.601 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:15.601 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:15.601 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:15.602 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:15.602 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68751 2024-09-14T23:57:15.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.602 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:15.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.602 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:15.602 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265877+0000", 2024-09-14T23:57:15.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.602 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:15.602 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265877+0000", 2024-09-14T23:57:15.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:15.603 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.267673+0000", 2024-09-14T23:57:15.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017956389999999999 2024-09-14T23:57:15.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:15.603 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.267677+0000", 2024-09-14T23:57:15.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4900000000000002e-06 2024-09-14T23:57:15.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:15.603 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.267688+0000", 2024-09-14T23:57:15.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1121000000000001e-05 2024-09-14T23:57:15.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:15.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.267694+0000", 2024-09-14T23:57:15.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5520000000000004e-06 2024-09-14T23:57:15.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:15.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.312534+0000", 2024-09-14T23:57:15.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044839641 2024-09-14T23:57:15.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:15.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.312586+0000", 2024-09-14T23:57:15.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2629e-05 2024-09-14T23:57:15.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:15.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.312636+0000", 2024-09-14T23:57:15.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0160000000000001e-05 2024-09-14T23:57:15.605 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.605 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:15.605 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.605 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68885 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 7946240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:15.605 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.273765+0000", 2024-09-14T23:57:15.605 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.252388008, 2024-09-14T23:57:15.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.045854150000000003, 2024-09-14T23:57:15.606 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:15.606 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:15.606 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:15.606 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:15.606 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:15.606 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:15.606 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68885 2024-09-14T23:57:15.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.606 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:15.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.606 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:15.606 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273765+0000", 2024-09-14T23:57:15.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.607 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:15.607 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273765+0000", 2024-09-14T23:57:15.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.607 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:15.607 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273765+0000", 2024-09-14T23:57:15.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.84e-07 2024-09-14T23:57:15.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.607 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:15.607 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273767+0000", 2024-09-14T23:57:15.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3460000000000001e-06 2024-09-14T23:57:15.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:15.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273774+0000", 2024-09-14T23:57:15.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6420000000000001e-06 2024-09-14T23:57:15.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:15.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273779+0000", 2024-09-14T23:57:15.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.899e-06 2024-09-14T23:57:15.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:15.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.319535+0000", 2024-09-14T23:57:15.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.045755707999999999 2024-09-14T23:57:15.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:15.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.319577+0000", 2024-09-14T23:57:15.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2129999999999998e-05 2024-09-14T23:57:15.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:15.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.319619+0000", 2024-09-14T23:57:15.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2040999999999999e-05 2024-09-14T23:57:15.609 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.610 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:15.610 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.610 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68886 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 8060928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:15.610 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.273785+0000", 2024-09-14T23:57:15.610 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.25236721400000001, 2024-09-14T23:57:15.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.045962955, 2024-09-14T23:57:15.610 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:15.610 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:15.610 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:15.610 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:15.610 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:15.610 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:15.611 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68886 2024-09-14T23:57:15.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.611 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:15.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.611 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:15.611 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273785+0000", 2024-09-14T23:57:15.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.611 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:15.611 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273785+0000", 2024-09-14T23:57:15.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.612 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:15.612 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273786+0000", 2024-09-14T23:57:15.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9000000000000002e-07 2024-09-14T23:57:15.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.612 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:15.612 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273788+0000", 2024-09-14T23:57:15.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9139999999999998e-06 2024-09-14T23:57:15.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.612 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:15.612 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273794+0000", 2024-09-14T23:57:15.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.212e-06 2024-09-14T23:57:15.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:15.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273799+0000", 2024-09-14T23:57:15.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5580000000000002e-06 2024-09-14T23:57:15.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:15.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.319646+0000", 2024-09-14T23:57:15.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.045847951999999997 2024-09-14T23:57:15.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:15.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.319707+0000", 2024-09-14T23:57:15.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.029e-05 2024-09-14T23:57:15.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.614 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:15.614 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.319748+0000", 2024-09-14T23:57:15.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1638999999999999e-05 2024-09-14T23:57:15.614 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.614 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:15.614 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.614 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68887 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 8175616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:15.614 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.273805+0000", 2024-09-14T23:57:15.614 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.25234795500000001, 2024-09-14T23:57:15.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046062421999999999, 2024-09-14T23:57:15.615 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:15.615 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:15.615 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:15.615 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:15.615 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:15.615 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:15.615 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68887 2024-09-14T23:57:15.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.615 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:15.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.615 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:15.615 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273805+0000", 2024-09-14T23:57:15.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:15.616 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273805+0000", 2024-09-14T23:57:15.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:15.616 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273805+0000", 2024-09-14T23:57:15.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0699999999999998e-07 2024-09-14T23:57:15.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:15.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273815+0000", 2024-09-14T23:57:15.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7720000000000006e-06 2024-09-14T23:57:15.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:15.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273822+0000", 2024-09-14T23:57:15.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5850000000000002e-06 2024-09-14T23:57:15.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:15.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273833+0000", 2024-09-14T23:57:15.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0541e-05 2024-09-14T23:57:15.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:15.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.319779+0000", 2024-09-14T23:57:15.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.045946263000000001 2024-09-14T23:57:15.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:15.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.319824+0000", 2024-09-14T23:57:15.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4740000000000002e-05 2024-09-14T23:57:15.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:15.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.319867+0000", 2024-09-14T23:57:15.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3114000000000001e-05 2024-09-14T23:57:15.618 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.618 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:15.619 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.619 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68888 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 8290304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:15.619 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.273854+0000", 2024-09-14T23:57:15.619 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.25229881500000001, 2024-09-14T23:57:15.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046120520999999998, 2024-09-14T23:57:15.619 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:15.619 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:15.619 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:15.619 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:15.619 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:15.619 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:15.619 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68888 2024-09-14T23:57:15.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.620 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:15.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:15.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273854+0000", 2024-09-14T23:57:15.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:15.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273854+0000", 2024-09-14T23:57:15.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:15.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:15.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273854+0000", 2024-09-14T23:57:15.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.58e-07 2024-09-14T23:57:15.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:15.621 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273862+0000", 2024-09-14T23:57:15.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.52e-06 2024-09-14T23:57:15.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:15.621 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273869+0000", 2024-09-14T23:57:15.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1180000000000003e-06 2024-09-14T23:57:15.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:15.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273883+0000", 2024-09-14T23:57:15.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3709999999999999e-05 2024-09-14T23:57:15.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:15.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.319889+0000", 2024-09-14T23:57:15.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046005870999999997 2024-09-14T23:57:15.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:15.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.319930+0000", 2024-09-14T23:57:15.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1850000000000001e-05 2024-09-14T23:57:15.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:15.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:15.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:15.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.319974+0000", 2024-09-14T23:57:15.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3993999999999999e-05 2024-09-14T23:57:15.623 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.623 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:15.623 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.623 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:15.623 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:15.623 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:15.625 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T23:57:15.625 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:15.623+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:15.725 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:15.724+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:15.783 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:15.784 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:15.784 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:57:15.784 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:15.792 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T23:57:15.826 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:15.825+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:15.927 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:15.925+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:15.957 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:15.957 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:15.957 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:57:15.957 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:57:15.957 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:15.967 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T23:57:16.027 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:16.026+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:16.093 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:16.094 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:57:16.094 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:57:16.094 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:57:16.094 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.094 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69262 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 4800512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:16.094 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300811+0000", 2024-09-14T23:57:16.094 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.79017201199999998, 2024-09-14T23:57:16.094 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050403796000000001, 2024-09-14T23:57:16.094 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:16.094 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:16.094 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:16.095 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:16.095 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:16.095 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:16.095 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69262 2024-09-14T23:57:16.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.095 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:16.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:16.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300811+0000", 2024-09-14T23:57:16.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:16.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.096 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:16.096 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300811+0000", 2024-09-14T23:57:16.096 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:16.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.096 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:16.096 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300811+0000", 2024-09-14T23:57:16.096 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5700000000000002e-07 2024-09-14T23:57:16.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.096 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:16.096 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300815+0000", 2024-09-14T23:57:16.097 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.087e-06 2024-09-14T23:57:16.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.097 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.097 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:16.097 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300823+0000", 2024-09-14T23:57:16.097 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8720000000000002e-06 2024-09-14T23:57:16.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.097 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.097 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:16.097 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300830+0000", 2024-09-14T23:57:16.097 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0589999999999997e-06 2024-09-14T23:57:16.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.098 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.098 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:16.098 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351020+0000", 2024-09-14T23:57:16.098 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050189552999999998 2024-09-14T23:57:16.098 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.098 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.098 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:16.098 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351112+0000", 2024-09-14T23:57:16.098 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2615999999999996e-05 2024-09-14T23:57:16.098 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.098 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.099 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:16.099 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351214+0000", 2024-09-14T23:57:16.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101952 2024-09-14T23:57:16.099 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:16.099 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:16.099 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:16.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.099 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69263 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 4915200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:16.099 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300842+0000", 2024-09-14T23:57:16.099 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.79014050800000002, 2024-09-14T23:57:16.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050590637000000001, 2024-09-14T23:57:16.099 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:16.100 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:16.100 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:16.100 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:16.100 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:16.100 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:16.100 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69263 2024-09-14T23:57:16.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.101 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:16.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.101 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:16.101 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300842+0000", 2024-09-14T23:57:16.101 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:16.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.101 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:16.101 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300842+0000", 2024-09-14T23:57:16.101 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:16.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.102 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:16.102 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300843+0000", 2024-09-14T23:57:16.102 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4000000000000002e-07 2024-09-14T23:57:16.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.102 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.102 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:16.102 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300846+0000", 2024-09-14T23:57:16.102 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8720000000000002e-06 2024-09-14T23:57:16.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.102 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.102 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:16.102 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300853+0000", 2024-09-14T23:57:16.103 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0450000000000003e-06 2024-09-14T23:57:16.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.103 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.103 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:16.103 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300859+0000", 2024-09-14T23:57:16.103 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9510000000000003e-06 2024-09-14T23:57:16.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.103 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.103 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:16.103 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351260+0000", 2024-09-14T23:57:16.103 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050401654999999997 2024-09-14T23:57:16.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.103 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.104 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:16.104 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351353+0000", 2024-09-14T23:57:16.104 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2785e-05 2024-09-14T23:57:16.104 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.104 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.104 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:16.104 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351433+0000", 2024-09-14T23:57:16.104 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9789000000000005e-05 2024-09-14T23:57:16.104 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:16.104 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:16.104 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:16.104 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.104 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.105 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69264 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5029888~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:16.105 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300866+0000", 2024-09-14T23:57:16.105 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.79011657499999999, 2024-09-14T23:57:16.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050794063, 2024-09-14T23:57:16.105 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:16.105 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:16.105 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:16.105 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:16.105 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:16.105 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:16.105 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69264 2024-09-14T23:57:16.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.105 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:16.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.106 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:16.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300866+0000", 2024-09-14T23:57:16.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:16.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.106 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:16.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300866+0000", 2024-09-14T23:57:16.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:16.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.106 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:16.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300866+0000", 2024-09-14T23:57:16.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8799999999999998e-07 2024-09-14T23:57:16.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.107 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:16.107 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300869+0000", 2024-09-14T23:57:16.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0880000000000002e-06 2024-09-14T23:57:16.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.107 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:16.107 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300875+0000", 2024-09-14T23:57:16.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9280000000000004e-06 2024-09-14T23:57:16.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:16.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300882+0000", 2024-09-14T23:57:16.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4740000000000002e-06 2024-09-14T23:57:16.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:16.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351472+0000", 2024-09-14T23:57:16.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050590006999999999 2024-09-14T23:57:16.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:16.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351563+0000", 2024-09-14T23:57:16.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0762999999999998e-05 2024-09-14T23:57:16.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:16.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351660+0000", 2024-09-14T23:57:16.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7415000000000002e-05 2024-09-14T23:57:16.109 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:16.109 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:16.109 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:16.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.110 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.110 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69265 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5144576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:16.110 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300890+0000", 2024-09-14T23:57:16.110 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.79009307100000004, 2024-09-14T23:57:16.110 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050921389999999997, 2024-09-14T23:57:16.110 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:16.110 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:16.110 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:16.110 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:16.110 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:16.110 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:16.111 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69265 2024-09-14T23:57:16.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.111 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:16.111 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.111 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:16.111 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300890+0000", 2024-09-14T23:57:16.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:16.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.111 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.111 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:16.111 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300890+0000", 2024-09-14T23:57:16.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:16.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.111 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:16.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300890+0000", 2024-09-14T23:57:16.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2500000000000001e-07 2024-09-14T23:57:16.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:16.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300892+0000", 2024-09-14T23:57:16.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2469999999999998e-06 2024-09-14T23:57:16.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:16.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300899+0000", 2024-09-14T23:57:16.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5509999999999996e-06 2024-09-14T23:57:16.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.113 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:16.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300904+0000", 2024-09-14T23:57:16.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3299999999999998e-06 2024-09-14T23:57:16.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.113 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:16.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351707+0000", 2024-09-14T23:57:16.114 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050802673999999999 2024-09-14T23:57:16.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.114 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:16.114 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351755+0000", 2024-09-14T23:57:16.114 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8362000000000002e-05 2024-09-14T23:57:16.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.115 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:16.115 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351811+0000", 2024-09-14T23:57:16.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5801000000000003e-05 2024-09-14T23:57:16.115 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:16.115 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:16.115 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:16.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.115 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69266 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5259264~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:16.115 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300911+0000", 2024-09-14T23:57:16.115 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.79007199800000005, 2024-09-14T23:57:16.116 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051125089999999998, 2024-09-14T23:57:16.116 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:16.116 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:16.116 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:16.116 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:16.116 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:16.116 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:16.116 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69266 2024-09-14T23:57:16.116 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.116 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:16.116 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:16.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300911+0000", 2024-09-14T23:57:16.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:16.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:16.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300911+0000", 2024-09-14T23:57:16.121 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:16.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.121 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.121 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:16.121 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300911+0000", 2024-09-14T23:57:16.121 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-14T23:57:16.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.121 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.121 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:16.121 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300913+0000", 2024-09-14T23:57:16.121 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9300000000000002e-06 2024-09-14T23:57:16.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.122 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.122 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:16.122 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300919+0000", 2024-09-14T23:57:16.122 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.489e-06 2024-09-14T23:57:16.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.122 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.122 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:16.122 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300925+0000", 2024-09-14T23:57:16.122 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1719999999999998e-06 2024-09-14T23:57:16.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.122 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.122 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:16.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351847+0000", 2024-09-14T23:57:16.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050922621000000001 2024-09-14T23:57:16.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.123 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.123 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:16.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351938+0000", 2024-09-14T23:57:16.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0830999999999998e-05 2024-09-14T23:57:16.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.123 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.123 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:16.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352036+0000", 2024-09-14T23:57:16.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7676999999999993e-05 2024-09-14T23:57:16.124 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:16.124 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:16.124 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:16.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.124 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69267 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5373952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:16.124 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300945+0000", 2024-09-14T23:57:16.124 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.79003720300000002, 2024-09-14T23:57:16.124 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051217934, 2024-09-14T23:57:16.124 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:16.124 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:16.125 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:16.125 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:16.125 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:16.125 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:16.125 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69267 2024-09-14T23:57:16.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.125 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:16.125 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.125 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:16.125 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300945+0000", 2024-09-14T23:57:16.126 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:16.126 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.126 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.126 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:16.126 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300945+0000", 2024-09-14T23:57:16.126 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:16.126 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.126 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.126 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:16.126 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300946+0000", 2024-09-14T23:57:16.126 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2600000000000002e-07 2024-09-14T23:57:16.126 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.127 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.127 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:16.127 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300948+0000", 2024-09-14T23:57:16.127 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4940000000000002e-06 2024-09-14T23:57:16.127 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.127 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.127 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:16.127 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300956+0000", 2024-09-14T23:57:16.127 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.622e-06 2024-09-14T23:57:16.127 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.127 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.127 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:16.127 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300962+0000", 2024-09-14T23:57:16.128 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5790000000000003e-06 2024-09-14T23:57:16.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.128 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.128 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:16.128 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352063+0000", 2024-09-14T23:57:16.128 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051101281999999998 2024-09-14T23:57:16.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.128 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.128 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:16.128 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352110+0000", 2024-09-14T23:57:16.128 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7401000000000002e-05 2024-09-14T23:57:16.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.129 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.129 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:16.129 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352163+0000", 2024-09-14T23:57:16.129 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3029999999999999e-05 2024-09-14T23:57:16.129 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:16.129 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:16.129 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:16.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.129 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.129 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69268 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5488640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:16.129 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300969+0000", 2024-09-14T23:57:16.129 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.79001353100000005, 2024-09-14T23:57:16.130 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051413420000000001, 2024-09-14T23:57:16.130 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:16.130 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:16.130 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:16.130 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:16.130 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:16.130 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:16.130 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69268 2024-09-14T23:57:16.130 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.130 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:16.130 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.130 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:16.131 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300969+0000", 2024-09-14T23:57:16.131 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:16.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.131 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.131 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:16.131 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300969+0000", 2024-09-14T23:57:16.131 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:16.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.131 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.131 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:16.131 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300970+0000", 2024-09-14T23:57:16.131 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7899999999999999e-07 2024-09-14T23:57:16.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.132 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.132 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:16.132 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300982+0000", 2024-09-14T23:57:16.132 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2741e-05 2024-09-14T23:57:16.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.132 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.132 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:16.132 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300990+0000", 2024-09-14T23:57:16.132 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8169999999999999e-06 2024-09-14T23:57:16.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.133 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:16.133 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300995+0000", 2024-09-14T23:57:16.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2240000000000003e-06 2024-09-14T23:57:16.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.133 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:16.133 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352200+0000", 2024-09-14T23:57:16.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051204223 2024-09-14T23:57:16.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.133 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:16.133 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352285+0000", 2024-09-14T23:57:16.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5519999999999994e-05 2024-09-14T23:57:16.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:16.134 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352383+0000", 2024-09-14T23:57:16.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7516000000000006e-05 2024-09-14T23:57:16.134 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:16.134 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:16.134 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:16.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.134 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69269 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5603328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:16.135 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.301003+0000", 2024-09-14T23:57:16.135 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.789979349, 2024-09-14T23:57:16.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051497381000000002, 2024-09-14T23:57:16.135 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:16.135 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:16.135 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:16.135 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:16.135 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:16.135 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:16.135 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69269 2024-09-14T23:57:16.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.135 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:16.136 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.136 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:16.136 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.301003+0000", 2024-09-14T23:57:16.136 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:16.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.136 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.136 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:16.136 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.301003+0000", 2024-09-14T23:57:16.136 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:16.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.136 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.136 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:16.137 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302671+0000", 2024-09-14T23:57:16.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001667785 2024-09-14T23:57:16.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:16.137 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302676+0000", 2024-09-14T23:57:16.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6600000000000003e-06 2024-09-14T23:57:16.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:16.137 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302697+0000", 2024-09-14T23:57:16.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1288999999999999e-05 2024-09-14T23:57:16.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:16.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302716+0000", 2024-09-14T23:57:16.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9128999999999999e-05 2024-09-14T23:57:16.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:16.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352399+0000", 2024-09-14T23:57:16.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049682995000000001 2024-09-14T23:57:16.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.139 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:16.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352446+0000", 2024-09-14T23:57:16.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6341000000000002e-05 2024-09-14T23:57:16.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.139 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:16.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352501+0000", 2024-09-14T23:57:16.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5182000000000001e-05 2024-09-14T23:57:16.139 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:16.139 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:16.139 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:16.140 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.140 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.140 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69273 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6062080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:16.140 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302829+0000", 2024-09-14T23:57:16.140 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.78815378300000005, 2024-09-14T23:57:16.140 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050384311000000001, 2024-09-14T23:57:16.140 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:16.140 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:16.140 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:16.140 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:16.140 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:16.140 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:16.141 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69273 2024-09-14T23:57:16.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.141 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:16.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.141 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:16.141 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302829+0000", 2024-09-14T23:57:16.141 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:16.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.141 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:16.141 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302829+0000", 2024-09-14T23:57:16.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:16.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.142 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:16.142 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302829+0000", 2024-09-14T23:57:16.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7700000000000005e-07 2024-09-14T23:57:16.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:16.143 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302832+0000", 2024-09-14T23:57:16.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6970000000000002e-06 2024-09-14T23:57:16.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:16.143 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302840+0000", 2024-09-14T23:57:16.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5719999999999997e-06 2024-09-14T23:57:16.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.144 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:16.144 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302845+0000", 2024-09-14T23:57:16.144 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7939999999999999e-06 2024-09-14T23:57:16.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.144 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.144 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:16.144 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353111+0000", 2024-09-14T23:57:16.144 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050265588 2024-09-14T23:57:16.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.144 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.144 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:16.145 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353160+0000", 2024-09-14T23:57:16.145 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8859000000000001e-05 2024-09-14T23:57:16.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.145 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.145 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:16.145 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353213+0000", 2024-09-14T23:57:16.145 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3324e-05 2024-09-14T23:57:16.145 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:16.145 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:16.145 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:16.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.146 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69274 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6176768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:16.146 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302854+0000", 2024-09-14T23:57:16.146 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.78812868899999999, 2024-09-14T23:57:16.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050579526, 2024-09-14T23:57:16.146 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:16.146 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:16.146 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:16.146 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:16.146 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:16.146 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:16.146 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69274 2024-09-14T23:57:16.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.147 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:16.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.147 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:16.147 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302854+0000", 2024-09-14T23:57:16.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:16.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.147 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:16.147 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302854+0000", 2024-09-14T23:57:16.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:16.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.148 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:16.148 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302855+0000", 2024-09-14T23:57:16.148 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0200000000000001e-07 2024-09-14T23:57:16.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.148 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.148 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:16.148 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302857+0000", 2024-09-14T23:57:16.148 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.447e-06 2024-09-14T23:57:16.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.148 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.148 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:16.149 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302864+0000", 2024-09-14T23:57:16.149 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7730000000000002e-06 2024-09-14T23:57:16.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.149 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:16.149 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302869+0000", 2024-09-14T23:57:16.149 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3199999999999999e-06 2024-09-14T23:57:16.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.149 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:16.149 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353249+0000", 2024-09-14T23:57:16.149 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050379647999999999 2024-09-14T23:57:16.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:16.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353336+0000", 2024-09-14T23:57:16.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6767999999999993e-05 2024-09-14T23:57:16.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:16.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353433+0000", 2024-09-14T23:57:16.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7868000000000006e-05 2024-09-14T23:57:16.150 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:16.150 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:16.150 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:16.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.151 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69275 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6291456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:16.151 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302889+0000", 2024-09-14T23:57:16.151 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.78809407099999995, 2024-09-14T23:57:16.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050663118, 2024-09-14T23:57:16.151 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:16.151 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:16.151 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:16.151 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:16.151 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:16.151 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:16.151 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69275 2024-09-14T23:57:16.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.152 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:16.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.152 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:16.152 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302889+0000", 2024-09-14T23:57:16.152 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:16.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.152 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:16.152 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302889+0000", 2024-09-14T23:57:16.152 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:16.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.153 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.153 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:16.153 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302889+0000", 2024-09-14T23:57:16.153 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6399999999999998e-07 2024-09-14T23:57:16.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.153 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.153 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:16.153 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302891+0000", 2024-09-14T23:57:16.153 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.339e-06 2024-09-14T23:57:16.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.154 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.154 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:16.154 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302898+0000", 2024-09-14T23:57:16.154 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0380000000000002e-06 2024-09-14T23:57:16.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.154 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.154 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:16.154 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302904+0000", 2024-09-14T23:57:16.154 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2159999999999997e-06 2024-09-14T23:57:16.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.154 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:16.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353449+0000", 2024-09-14T23:57:16.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050545386999999997 2024-09-14T23:57:16.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:16.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353494+0000", 2024-09-14T23:57:16.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4592999999999998e-05 2024-09-14T23:57:16.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:16.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353552+0000", 2024-09-14T23:57:16.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8180999999999997e-05 2024-09-14T23:57:16.155 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:16.156 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:16.156 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:16.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.156 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69276 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6406144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:16.156 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302911+0000", 2024-09-14T23:57:16.156 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.78807165800000001, 2024-09-14T23:57:16.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050865753, 2024-09-14T23:57:16.156 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:16.156 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:16.156 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:16.156 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:16.156 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:16.157 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:16.157 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69276 2024-09-14T23:57:16.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.157 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:16.157 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.157 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:16.157 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302911+0000", 2024-09-14T23:57:16.157 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:16.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.157 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.157 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:16.157 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302911+0000", 2024-09-14T23:57:16.158 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:16.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.158 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.158 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:16.158 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302912+0000", 2024-09-14T23:57:16.158 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0800000000000004e-07 2024-09-14T23:57:16.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.158 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.158 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:16.158 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302914+0000", 2024-09-14T23:57:16.158 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9589999999999998e-06 2024-09-14T23:57:16.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.158 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.159 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:16.159 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302920+0000", 2024-09-14T23:57:16.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4579999999999998e-06 2024-09-14T23:57:16.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.159 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:16.159 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302925+0000", 2024-09-14T23:57:16.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9429999999999999e-06 2024-09-14T23:57:16.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.159 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:16.159 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353586+0000", 2024-09-14T23:57:16.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050660966000000002 2024-09-14T23:57:16.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.160 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.160 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:16.160 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353679+0000", 2024-09-14T23:57:16.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3163000000000002e-05 2024-09-14T23:57:16.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.160 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.160 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:16.160 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353777+0000", 2024-09-14T23:57:16.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7555999999999999e-05 2024-09-14T23:57:16.160 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:16.161 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:16.161 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:16.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.161 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.161 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69277 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6520832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:16.161 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302932+0000", 2024-09-14T23:57:16.161 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.78805057099999998, 2024-09-14T23:57:16.161 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050977674000000001, 2024-09-14T23:57:16.161 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:16.161 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:16.161 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:16.162 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:16.162 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:16.162 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:16.162 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69277 2024-09-14T23:57:16.162 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.162 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:16.162 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.162 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:16.162 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302932+0000", 2024-09-14T23:57:16.162 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:16.162 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.162 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.163 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:16.163 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302932+0000", 2024-09-14T23:57:16.163 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:16.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.163 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:16.163 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302932+0000", 2024-09-14T23:57:16.163 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3299999999999998e-07 2024-09-14T23:57:16.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.163 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:16.163 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302935+0000", 2024-09-14T23:57:16.164 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2139999999999999e-06 2024-09-14T23:57:16.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.164 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.164 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:16.164 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302941+0000", 2024-09-14T23:57:16.164 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5039999999999999e-06 2024-09-14T23:57:16.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.164 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.164 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:16.164 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302946+0000", 2024-09-14T23:57:16.164 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2079999999999999e-06 2024-09-14T23:57:16.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.164 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.165 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:16.165 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353805+0000", 2024-09-14T23:57:16.165 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050858588000000003 2024-09-14T23:57:16.165 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.165 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.165 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:16.165 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353849+0000", 2024-09-14T23:57:16.165 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4511000000000001e-05 2024-09-14T23:57:16.165 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.165 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.165 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:16.166 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353910+0000", 2024-09-14T23:57:16.166 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0316000000000003e-05 2024-09-14T23:57:16.166 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:16.166 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:16.166 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:16.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.166 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.166 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69278 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6635520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:16.166 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302953+0000", 2024-09-14T23:57:16.166 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.788029443, 2024-09-14T23:57:16.166 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051174059000000001, 2024-09-14T23:57:16.166 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:16.167 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:16.167 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:16.167 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:16.167 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:16.167 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:16.167 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69278 2024-09-14T23:57:16.167 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.167 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:16.167 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.167 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:16.167 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302953+0000", 2024-09-14T23:57:16.167 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:16.167 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.168 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.168 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:16.168 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302953+0000", 2024-09-14T23:57:16.168 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:16.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.168 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.168 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:16.168 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302954+0000", 2024-09-14T23:57:16.168 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5100000000000001e-07 2024-09-14T23:57:16.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.168 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.169 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:16.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302964+0000", 2024-09-14T23:57:16.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0475000000000001e-05 2024-09-14T23:57:16.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.169 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:16.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302970+0000", 2024-09-14T23:57:16.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3879999999999997e-06 2024-09-14T23:57:16.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.169 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:16.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302975+0000", 2024-09-14T23:57:16.170 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8409999999999999e-06 2024-09-14T23:57:16.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.170 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.170 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:16.170 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353944+0000", 2024-09-14T23:57:16.170 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050968952999999997 2024-09-14T23:57:16.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.170 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.170 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:16.170 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354031+0000", 2024-09-14T23:57:16.170 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6537000000000002e-05 2024-09-14T23:57:16.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.171 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.171 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:16.171 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354127+0000", 2024-09-14T23:57:16.171 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6514000000000005e-05 2024-09-14T23:57:16.171 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:16.171 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:16.171 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:16.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.171 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.171 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69279 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6750208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:16.171 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302984+0000", 2024-09-14T23:57:16.172 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.78799909199999996, 2024-09-14T23:57:16.172 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051258845999999997, 2024-09-14T23:57:16.172 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:16.172 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:16.172 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:16.172 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:16.172 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:16.172 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:16.172 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69279 2024-09-14T23:57:16.172 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.173 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:16.173 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.173 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:16.173 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302984+0000", 2024-09-14T23:57:16.173 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:16.173 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.173 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.173 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:16.173 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302984+0000", 2024-09-14T23:57:16.173 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:16.173 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.173 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.174 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:16.174 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303025+0000", 2024-09-14T23:57:16.174 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1690999999999998e-05 2024-09-14T23:57:16.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.174 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.174 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:16.174 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303029+0000", 2024-09-14T23:57:16.174 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6559999999999998e-06 2024-09-14T23:57:16.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.174 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.174 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:16.175 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303037+0000", 2024-09-14T23:57:16.175 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7950000000000008e-06 2024-09-14T23:57:16.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.175 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.175 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:16.175 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303043+0000", 2024-09-14T23:57:16.175 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9669999999999998e-06 2024-09-14T23:57:16.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.175 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.175 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:16.175 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354142+0000", 2024-09-14T23:57:16.175 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051099072000000002 2024-09-14T23:57:16.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.176 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.176 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:16.176 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354183+0000", 2024-09-14T23:57:16.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0834000000000002e-05 2024-09-14T23:57:16.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.176 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.176 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:16.176 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354242+0000", 2024-09-14T23:57:16.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9830999999999997e-05 2024-09-14T23:57:16.177 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:16.177 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:16.177 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:16.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.177 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69280 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6864896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:16.177 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.303052+0000", 2024-09-14T23:57:16.177 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.78793054900000004, 2024-09-14T23:57:16.177 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051406463999999999, 2024-09-14T23:57:16.177 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:16.177 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:16.178 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:16.178 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:16.178 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:16.178 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:16.178 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69280 2024-09-14T23:57:16.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.178 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:16.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.178 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:16.178 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303052+0000", 2024-09-14T23:57:16.178 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:16.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.179 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.179 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:16.179 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303052+0000", 2024-09-14T23:57:16.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:16.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.179 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.179 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:16.179 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304673+0000", 2024-09-14T23:57:16.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001621389 2024-09-14T23:57:16.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.179 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.179 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:16.179 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304681+0000", 2024-09-14T23:57:16.180 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8439999999999998e-06 2024-09-14T23:57:16.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.180 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.180 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:16.180 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304691+0000", 2024-09-14T23:57:16.180 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8339999999999994e-06 2024-09-14T23:57:16.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.180 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.180 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:16.180 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304698+0000", 2024-09-14T23:57:16.180 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3629999999999999e-06 2024-09-14T23:57:16.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.181 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.181 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:16.181 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354278+0000", 2024-09-14T23:57:16.181 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049580618 2024-09-14T23:57:16.181 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.181 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.181 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:16.181 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354364+0000", 2024-09-14T23:57:16.181 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6059999999999999e-05 2024-09-14T23:57:16.181 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.181 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.181 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:16.181 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354459+0000", 2024-09-14T23:57:16.182 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4356000000000003e-05 2024-09-14T23:57:16.182 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:16.182 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:16.182 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:16.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.182 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.182 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69285 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 7438336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:16.182 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.304792+0000", 2024-09-14T23:57:16.182 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.78619114000000001, 2024-09-14T23:57:16.182 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050398874000000003, 2024-09-14T23:57:16.182 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:16.183 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:16.183 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:16.183 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:16.183 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:16.183 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:16.183 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69285 2024-09-14T23:57:16.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.183 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:16.183 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.183 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:16.183 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304792+0000", 2024-09-14T23:57:16.184 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:16.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.184 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:16.184 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304792+0000", 2024-09-14T23:57:16.184 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:16.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.184 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:16.184 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304792+0000", 2024-09-14T23:57:16.184 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2800000000000003e-07 2024-09-14T23:57:16.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.185 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:16.185 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304794+0000", 2024-09-14T23:57:16.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8700000000000001e-06 2024-09-14T23:57:16.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.185 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.185 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:16.185 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304800+0000", 2024-09-14T23:57:16.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0889999999999996e-06 2024-09-14T23:57:16.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.185 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.185 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:16.185 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304805+0000", 2024-09-14T23:57:16.186 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7060000000000003e-06 2024-09-14T23:57:16.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.186 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.186 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:16.186 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355108+0000", 2024-09-14T23:57:16.186 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050303170000000001 2024-09-14T23:57:16.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.186 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.186 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:16.186 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355150+0000", 2024-09-14T23:57:16.186 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2271999999999998e-05 2024-09-14T23:57:16.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.187 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.187 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:16.187 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355190+0000", 2024-09-14T23:57:16.187 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0438999999999997e-05 2024-09-14T23:57:16.187 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:16.187 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:16.187 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:16.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.187 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.187 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69286 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 7553024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:16.187 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.304811+0000", 2024-09-14T23:57:16.187 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.78617177800000004, 2024-09-14T23:57:16.187 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050585512999999999, 2024-09-14T23:57:16.187 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:16.188 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:16.188 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:16.188 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:16.188 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:16.188 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:16.188 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69286 2024-09-14T23:57:16.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.188 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:16.188 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.188 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:16.188 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304811+0000", 2024-09-14T23:57:16.188 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:16.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.189 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.189 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:16.189 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304811+0000", 2024-09-14T23:57:16.189 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:16.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.189 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.189 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:16.189 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304811+0000", 2024-09-14T23:57:16.189 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.89e-07 2024-09-14T23:57:16.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.189 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.189 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:16.190 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304814+0000", 2024-09-14T23:57:16.190 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3199999999999998e-06 2024-09-14T23:57:16.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.190 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.190 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:16.190 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304821+0000", 2024-09-14T23:57:16.190 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5689999999999999e-06 2024-09-14T23:57:16.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.190 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.190 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:16.190 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304826+0000", 2024-09-14T23:57:16.190 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1200000000000001e-06 2024-09-14T23:57:16.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.191 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.191 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:16.191 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355226+0000", 2024-09-14T23:57:16.191 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050399653000000003 2024-09-14T23:57:16.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.191 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.191 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:16.191 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355314+0000", 2024-09-14T23:57:16.191 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8389999999999999e-05 2024-09-14T23:57:16.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.191 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.191 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:16.191 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355396+0000", 2024-09-14T23:57:16.191 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2071999999999996e-05 2024-09-14T23:57:16.192 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:16.192 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:16.192 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:16.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.192 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.192 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69287 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 7667712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:16.192 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.304833+0000", 2024-09-14T23:57:16.192 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.78614918, 2024-09-14T23:57:16.192 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050664740999999999, 2024-09-14T23:57:16.192 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:16.192 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:16.192 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:16.193 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:16.193 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:16.193 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:16.193 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69287 2024-09-14T23:57:16.193 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.193 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:16.193 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.193 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:16.193 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304833+0000", 2024-09-14T23:57:16.193 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:16.193 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.193 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.194 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:16.194 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304833+0000", 2024-09-14T23:57:16.194 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:16.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.194 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.194 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:16.194 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304834+0000", 2024-09-14T23:57:16.194 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4999999999999998e-07 2024-09-14T23:57:16.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.194 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.194 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:16.194 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304836+0000", 2024-09-14T23:57:16.194 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0279999999999999e-06 2024-09-14T23:57:16.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.195 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.195 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:16.195 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304842+0000", 2024-09-14T23:57:16.195 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8919999999999997e-06 2024-09-14T23:57:16.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.195 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.195 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:16.195 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304847+0000", 2024-09-14T23:57:16.195 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6160000000000001e-06 2024-09-14T23:57:16.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.195 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.195 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:16.195 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355411+0000", 2024-09-14T23:57:16.196 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050563540999999997 2024-09-14T23:57:16.196 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.196 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.196 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:16.196 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355459+0000", 2024-09-14T23:57:16.196 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7979000000000003e-05 2024-09-14T23:57:16.196 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.196 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.196 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:16.196 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355498+0000", 2024-09-14T23:57:16.196 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9335000000000002e-05 2024-09-14T23:57:16.196 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:16.197 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:16.197 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:16.197 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.197 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.197 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69288 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 7782400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:16.197 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.304854+0000", 2024-09-14T23:57:16.197 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.78612874200000005, 2024-09-14T23:57:16.197 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050855145999999997, 2024-09-14T23:57:16.197 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:16.197 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:16.197 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:16.197 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:16.197 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:16.198 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:16.198 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69288 2024-09-14T23:57:16.198 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.198 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:16.198 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.198 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:16.198 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304854+0000", 2024-09-14T23:57:16.198 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:16.198 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.198 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.198 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:16.198 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304854+0000", 2024-09-14T23:57:16.199 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:16.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.199 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.199 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:16.199 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.306670+0000", 2024-09-14T23:57:16.199 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00181631 2024-09-14T23:57:16.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.199 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.199 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:16.199 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.306675+0000", 2024-09-14T23:57:16.199 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0220000000000004e-06 2024-09-14T23:57:16.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.200 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.200 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:16.200 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.306684+0000", 2024-09-14T23:57:16.200 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7309999999999999e-06 2024-09-14T23:57:16.200 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.200 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.200 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:16.200 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.306689+0000", 2024-09-14T23:57:16.200 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6609999999999999e-06 2024-09-14T23:57:16.200 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.200 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.201 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:16.201 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355533+0000", 2024-09-14T23:57:16.201 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048844578 2024-09-14T23:57:16.201 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.201 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.201 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:16.201 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355622+0000", 2024-09-14T23:57:16.201 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9006999999999997e-05 2024-09-14T23:57:16.201 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:16.201 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:16.201 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:16.202 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355709+0000", 2024-09-14T23:57:16.202 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6836999999999995e-05 2024-09-14T23:57:16.202 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:16.202 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:16.202 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:16.202 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:16.202 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:16.202 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:16.203 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.1 to 1 (10000) 2024-09-14T23:57:16.205 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is unset 2024-09-14T23:57:16.205 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T23:57:16.206 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-14T23:57:16.206 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:16.202+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:16.305 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:16.303+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:16.406 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:16.404+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:16.410 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:16.411 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:57:16.411 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:57:16.411 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:16.425 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T23:57:16.506 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:16.504+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:16.573 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:16.573 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:57:16.573 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:57:16.573 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:57:16.573 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:16.584 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T23:57:16.607 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:16.606+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:16.708 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:16.706+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:16.776 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:16.776 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:57:16.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:57:16.776 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:57:16.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.776 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:43054 3.1f 3:ffbfe596:::rbd_data.1118350d5ae5.00000000000002eb:head [set-alloc-hint object_size 262144 write_size 262144,write 196608~22770] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T23:57:16.777 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:57:16.777 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.806380581999999, 2024-09-14T23:57:16.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.99973334400000002, 2024-09-14T23:57:16.777 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:16.777 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:16.777 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:16.777 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:16.777 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:16.777 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:16.777 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 43054 2024-09-14T23:57:16.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.777 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:16.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:16.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:57:16.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:16.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:16.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:57:16.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:16.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:16.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966552+0000", 2024-09-14T23:57:16.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3790000000000001e-06 2024-09-14T23:57:16.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:16.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966565+0000", 2024-09-14T23:57:16.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2459999999999999e-05 2024-09-14T23:57:16.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:16.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966576+0000", 2024-09-14T23:57:16.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1039000000000001e-05 2024-09-14T23:57:16.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:16.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966582+0000", 2024-09-14T23:57:16.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9660000000000001e-06 2024-09-14T23:57:16.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:16.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966625+0000", 2024-09-14T23:57:16.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3529999999999998e-05 2024-09-14T23:57:16.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.781 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:57:16.781 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966632+0000", 2024-09-14T23:57:16.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9809999999999997e-06 2024-09-14T23:57:16.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.781 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:16.781 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.964460+0000", 2024-09-14T23:57:16.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.99782768300000002 2024-09-14T23:57:16.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.781 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:16.781 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.964641+0000", 2024-09-14T23:57:16.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000181089 2024-09-14T23:57:16.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-14T23:57:16.782 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.964809+0000", 2024-09-14T23:57:16.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000168505 2024-09-14T23:57:16.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:57:16.782 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966030+0000", 2024-09-14T23:57:16.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012206929999999999 2024-09-14T23:57:16.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:16.782 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966181+0000", 2024-09-14T23:57:16.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000150569 2024-09-14T23:57:16.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:16.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966243+0000", 2024-09-14T23:57:16.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2331999999999999e-05 2024-09-14T23:57:16.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:16.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966284+0000", 2024-09-14T23:57:16.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1118000000000002e-05 2024-09-14T23:57:16.783 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:16.783 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:16.784 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:16.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.784 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:54673 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:16.784 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.167989+0000", 2024-09-14T23:57:16.784 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.604942231999999, 2024-09-14T23:57:16.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027580298, 2024-09-14T23:57:16.784 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:16.784 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:16.784 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:16.784 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:16.784 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:16.784 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:16.784 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54673 2024-09-14T23:57:16.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.785 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:16.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:16.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.167989+0000", 2024-09-14T23:57:16.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:16.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:16.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.167989+0000", 2024-09-14T23:57:16.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:16.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:16.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.167989+0000", 2024-09-14T23:57:16.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5499999999999998e-07 2024-09-14T23:57:16.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:16.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.167998+0000", 2024-09-14T23:57:16.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7369999999999995e-06 2024-09-14T23:57:16.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:16.787 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168005+0000", 2024-09-14T23:57:16.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8210000000000004e-06 2024-09-14T23:57:16.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:16.787 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168010+0000", 2024-09-14T23:57:16.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2510000000000002e-06 2024-09-14T23:57:16.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:16.787 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.195432+0000", 2024-09-14T23:57:16.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027421364 2024-09-14T23:57:16.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:16.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.195501+0000", 2024-09-14T23:57:16.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9611999999999997e-05 2024-09-14T23:57:16.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:16.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.195569+0000", 2024-09-14T23:57:16.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8058000000000005e-05 2024-09-14T23:57:16.789 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:16.789 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:16.789 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:16.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.789 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:54674 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 2637824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:16.789 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.168018+0000", 2024-09-14T23:57:16.790 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.604913588000001, 2024-09-14T23:57:16.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027716159000000001, 2024-09-14T23:57:16.790 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:16.790 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:16.790 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:16.790 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:16.790 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:16.790 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:16.790 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54674 2024-09-14T23:57:16.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.790 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:16.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:16.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168018+0000", 2024-09-14T23:57:16.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:16.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:16.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168018+0000", 2024-09-14T23:57:16.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:16.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:16.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168018+0000", 2024-09-14T23:57:16.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2600000000000002e-07 2024-09-14T23:57:16.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:16.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168020+0000", 2024-09-14T23:57:16.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9420000000000002e-06 2024-09-14T23:57:16.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:16.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168026+0000", 2024-09-14T23:57:16.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8880000000000002e-06 2024-09-14T23:57:16.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:16.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168031+0000", 2024-09-14T23:57:16.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.977e-06 2024-09-14T23:57:16.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:16.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.195599+0000", 2024-09-14T23:57:16.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027567758000000001 2024-09-14T23:57:16.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:16.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.195666+0000", 2024-09-14T23:57:16.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7102999999999998e-05 2024-09-14T23:57:16.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:16.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.195734+0000", 2024-09-14T23:57:16.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7865000000000002e-05 2024-09-14T23:57:16.794 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:16.794 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:16.794 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:16.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.794 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:54675 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:16.794 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.168039+0000", 2024-09-14T23:57:16.794 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.604892629, 2024-09-14T23:57:16.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027868378999999999, 2024-09-14T23:57:16.794 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:16.794 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:16.794 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:16.794 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:16.794 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:16.795 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:16.795 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54675 2024-09-14T23:57:16.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.795 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:16.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:16.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168039+0000", 2024-09-14T23:57:16.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:16.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:16.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168039+0000", 2024-09-14T23:57:16.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:16.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.796 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:16.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168039+0000", 2024-09-14T23:57:16.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3099999999999998e-07 2024-09-14T23:57:16.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.796 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:16.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168048+0000", 2024-09-14T23:57:16.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9460000000000004e-06 2024-09-14T23:57:16.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.796 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:16.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168055+0000", 2024-09-14T23:57:16.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5740000000000001e-06 2024-09-14T23:57:16.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:16.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168060+0000", 2024-09-14T23:57:16.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1909999999999999e-06 2024-09-14T23:57:16.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:16.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.195765+0000", 2024-09-14T23:57:16.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027705244 2024-09-14T23:57:16.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:16.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.195836+0000", 2024-09-14T23:57:16.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0642999999999994e-05 2024-09-14T23:57:16.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:16.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.195907+0000", 2024-09-14T23:57:16.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.135e-05 2024-09-14T23:57:16.798 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:16.798 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:16.798 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:16.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.798 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:54676 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 2998272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:16.798 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.168067+0000", 2024-09-14T23:57:16.799 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.604863895999999, 2024-09-14T23:57:16.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028002955999999999, 2024-09-14T23:57:16.799 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:16.799 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:16.799 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:16.799 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:16.799 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:16.799 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:16.799 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54676 2024-09-14T23:57:16.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.799 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:16.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:16.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168067+0000", 2024-09-14T23:57:16.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:16.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:16.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168067+0000", 2024-09-14T23:57:16.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:16.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:16.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168068+0000", 2024-09-14T23:57:16.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6899999999999998e-07 2024-09-14T23:57:16.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:16.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168070+0000", 2024-09-14T23:57:16.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9810000000000002e-06 2024-09-14T23:57:16.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:16.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168076+0000", 2024-09-14T23:57:16.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4280000000000001e-06 2024-09-14T23:57:16.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:16.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168091+0000", 2024-09-14T23:57:16.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5152e-05 2024-09-14T23:57:16.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:16.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.195936+0000", 2024-09-14T23:57:16.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027844892 2024-09-14T23:57:16.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:16.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196002+0000", 2024-09-14T23:57:16.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5633e-05 2024-09-14T23:57:16.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:16.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196070+0000", 2024-09-14T23:57:16.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8400999999999998e-05 2024-09-14T23:57:16.803 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:16.803 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:16.803 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:16.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.803 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:54677 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 3178496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:16.803 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.168099+0000", 2024-09-14T23:57:16.803 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.604832597, 2024-09-14T23:57:16.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028165816, 2024-09-14T23:57:16.803 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:16.803 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:16.803 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:16.803 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:16.804 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:16.804 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:16.804 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54677 2024-09-14T23:57:16.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.804 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:16.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:16.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168099+0000", 2024-09-14T23:57:16.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:16.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:16.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168099+0000", 2024-09-14T23:57:16.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:16.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:16.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168099+0000", 2024-09-14T23:57:16.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7400000000000003e-07 2024-09-14T23:57:16.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:16.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168102+0000", 2024-09-14T23:57:16.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2929999999999999e-06 2024-09-14T23:57:16.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:16.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168108+0000", 2024-09-14T23:57:16.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5429999999999999e-06 2024-09-14T23:57:16.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:16.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168113+0000", 2024-09-14T23:57:16.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8470000000000003e-06 2024-09-14T23:57:16.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:16.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196124+0000", 2024-09-14T23:57:16.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028011034000000001 2024-09-14T23:57:16.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:16.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196194+0000", 2024-09-14T23:57:16.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0314000000000005e-05 2024-09-14T23:57:16.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:16.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196264+0000", 2024-09-14T23:57:16.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0210999999999996e-05 2024-09-14T23:57:16.807 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:16.807 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:16.807 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:16.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.807 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:54678 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 3358720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:16.808 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.168144+0000", 2024-09-14T23:57:16.808 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.604787438000001, 2024-09-14T23:57:16.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028285267999999999, 2024-09-14T23:57:16.808 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:16.808 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:16.808 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:16.808 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:16.808 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:16.808 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:16.808 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54678 2024-09-14T23:57:16.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.808 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:16.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:16.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168144+0000", 2024-09-14T23:57:16.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:16.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:16.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168144+0000", 2024-09-14T23:57:16.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:16.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:16.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168144+0000", 2024-09-14T23:57:16.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5799999999999999e-07 2024-09-14T23:57:16.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:16.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168147+0000", 2024-09-14T23:57:16.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3580000000000001e-06 2024-09-14T23:57:16.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:16.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168154+0000", 2024-09-14T23:57:16.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1350000000000003e-06 2024-09-14T23:57:16.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:16.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168159+0000", 2024-09-14T23:57:16.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4160000000000003e-06 2024-09-14T23:57:16.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:16.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196294+0000", 2024-09-14T23:57:16.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028135050000000002 2024-09-14T23:57:16.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:16.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196363+0000", 2024-09-14T23:57:16.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8880000000000005e-05 2024-09-14T23:57:16.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:16.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196429+0000", 2024-09-14T23:57:16.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5870999999999993e-05 2024-09-14T23:57:16.812 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:16.812 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:16.812 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:16.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.812 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:54679 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:16.812 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.168167+0000", 2024-09-14T23:57:16.812 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.604764217, 2024-09-14T23:57:16.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028427019000000001, 2024-09-14T23:57:16.813 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:16.813 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:16.813 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:16.813 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:16.813 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:16.813 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:16.813 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54679 2024-09-14T23:57:16.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.813 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:16.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:16.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168167+0000", 2024-09-14T23:57:16.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:16.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:16.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168167+0000", 2024-09-14T23:57:16.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:16.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:16.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168168+0000", 2024-09-14T23:57:16.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.44e-07 2024-09-14T23:57:16.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:16.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168170+0000", 2024-09-14T23:57:16.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0870000000000001e-06 2024-09-14T23:57:16.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:16.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168176+0000", 2024-09-14T23:57:16.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9070000000000004e-06 2024-09-14T23:57:16.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:16.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168181+0000", 2024-09-14T23:57:16.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6579999999999999e-06 2024-09-14T23:57:16.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:16.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196458+0000", 2024-09-14T23:57:16.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028276431000000001 2024-09-14T23:57:16.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:16.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196527+0000", 2024-09-14T23:57:16.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9397e-05 2024-09-14T23:57:16.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:16.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196594+0000", 2024-09-14T23:57:16.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6895000000000002e-05 2024-09-14T23:57:16.817 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:16.817 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:16.817 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:16.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.818 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:54680 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 3719168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:16.818 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.168188+0000", 2024-09-14T23:57:16.818 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.604743011, 2024-09-14T23:57:16.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028579706999999999, 2024-09-14T23:57:16.818 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:16.818 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:16.818 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:16.818 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:16.818 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:16.818 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:16.818 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54680 2024-09-14T23:57:16.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.818 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:16.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:16.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168188+0000", 2024-09-14T23:57:16.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:16.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:16.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168188+0000", 2024-09-14T23:57:16.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:16.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:16.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168189+0000", 2024-09-14T23:57:16.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3099999999999998e-07 2024-09-14T23:57:16.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:16.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168191+0000", 2024-09-14T23:57:16.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0470000000000001e-06 2024-09-14T23:57:16.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:16.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168199+0000", 2024-09-14T23:57:16.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8809999999999996e-06 2024-09-14T23:57:16.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:16.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168204+0000", 2024-09-14T23:57:16.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.011e-06 2024-09-14T23:57:16.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:16.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196622+0000", 2024-09-14T23:57:16.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028418806000000001 2024-09-14T23:57:16.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:16.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196694+0000", 2024-09-14T23:57:16.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1124000000000005e-05 2024-09-14T23:57:16.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:16.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196768+0000", 2024-09-14T23:57:16.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4406999999999994e-05 2024-09-14T23:57:16.822 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:16.822 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:16.822 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:16.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.822 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:54681 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 3899392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:16.822 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.168212+0000", 2024-09-14T23:57:16.822 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.604719298999999, 2024-09-14T23:57:16.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028726643, 2024-09-14T23:57:16.822 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:16.822 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:16.822 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:16.823 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:16.823 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:16.823 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:16.823 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54681 2024-09-14T23:57:16.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.823 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:16.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:16.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168212+0000", 2024-09-14T23:57:16.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:16.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:16.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168212+0000", 2024-09-14T23:57:16.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:16.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:16.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168212+0000", 2024-09-14T23:57:16.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4900000000000001e-07 2024-09-14T23:57:16.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:16.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168214+0000", 2024-09-14T23:57:16.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0590000000000001e-06 2024-09-14T23:57:16.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:16.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168220+0000", 2024-09-14T23:57:16.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0209999999999996e-06 2024-09-14T23:57:16.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:16.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168227+0000", 2024-09-14T23:57:16.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4119999999999998e-06 2024-09-14T23:57:16.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:16.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196799+0000", 2024-09-14T23:57:16.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028572528 2024-09-14T23:57:16.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:16.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196866+0000", 2024-09-14T23:57:16.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7005e-05 2024-09-14T23:57:16.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:16.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196939+0000", 2024-09-14T23:57:16.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2169000000000006e-05 2024-09-14T23:57:16.826 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:16.827 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:16.827 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:16.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.827 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:54682 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 4079616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:16.827 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.168234+0000", 2024-09-14T23:57:16.827 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.60469775, 2024-09-14T23:57:16.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028872503000000001, 2024-09-14T23:57:16.827 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:16.827 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:16.827 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:16.827 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:16.827 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:16.828 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:16.828 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54682 2024-09-14T23:57:16.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.828 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:16.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:16.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168234+0000", 2024-09-14T23:57:16.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:16.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:16.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168234+0000", 2024-09-14T23:57:16.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:16.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:16.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168234+0000", 2024-09-14T23:57:16.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.58e-07 2024-09-14T23:57:16.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:16.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168236+0000", 2024-09-14T23:57:16.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.933e-06 2024-09-14T23:57:16.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:16.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168242+0000", 2024-09-14T23:57:16.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9970000000000004e-06 2024-09-14T23:57:16.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:16.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168247+0000", 2024-09-14T23:57:16.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9280000000000001e-06 2024-09-14T23:57:16.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:16.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196968+0000", 2024-09-14T23:57:16.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028721159999999999 2024-09-14T23:57:16.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:16.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.197040+0000", 2024-09-14T23:57:16.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1569999999999994e-05 2024-09-14T23:57:16.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:16.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.197106+0000", 2024-09-14T23:57:16.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6457000000000004e-05 2024-09-14T23:57:16.831 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:16.831 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:16.831 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:16.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.831 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:55239 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 3850240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:16.831 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.201093+0000", 2024-09-14T23:57:16.832 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.571838298999999, 2024-09-14T23:57:16.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027610927, 2024-09-14T23:57:16.832 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:16.832 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:16.832 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:16.832 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:16.832 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:16.832 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:16.832 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 55239 2024-09-14T23:57:16.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.832 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:16.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:16.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201093+0000", 2024-09-14T23:57:16.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:16.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:16.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201093+0000", 2024-09-14T23:57:16.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:16.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:16.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201093+0000", 2024-09-14T23:57:16.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5699999999999998e-07 2024-09-14T23:57:16.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:16.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201096+0000", 2024-09-14T23:57:16.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0990000000000001e-06 2024-09-14T23:57:16.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:16.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201102+0000", 2024-09-14T23:57:16.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5710000000000003e-06 2024-09-14T23:57:16.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:16.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201107+0000", 2024-09-14T23:57:16.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1019999999999996e-06 2024-09-14T23:57:16.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:16.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.228568+0000", 2024-09-14T23:57:16.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027461136000000001 2024-09-14T23:57:16.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:16.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.228638+0000", 2024-09-14T23:57:16.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0024999999999994e-05 2024-09-14T23:57:16.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:16.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.228704+0000", 2024-09-14T23:57:16.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5537000000000006e-05 2024-09-14T23:57:16.837 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:16.837 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:16.837 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:16.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.837 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:55240 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:16.837 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.201114+0000", 2024-09-14T23:57:16.837 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.571817062999999, 2024-09-14T23:57:16.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027752295, 2024-09-14T23:57:16.837 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:16.838 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:16.838 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:16.838 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:16.838 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:16.838 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:16.838 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 55240 2024-09-14T23:57:16.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.838 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:16.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:16.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201114+0000", 2024-09-14T23:57:16.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:16.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:16.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201114+0000", 2024-09-14T23:57:16.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:16.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:16.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201115+0000", 2024-09-14T23:57:16.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4200000000000001e-07 2024-09-14T23:57:16.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:16.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201117+0000", 2024-09-14T23:57:16.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.368e-06 2024-09-14T23:57:16.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:16.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201124+0000", 2024-09-14T23:57:16.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4999999999999996e-06 2024-09-14T23:57:16.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:16.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201136+0000", 2024-09-14T23:57:16.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.293e-05 2024-09-14T23:57:16.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:16.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.228736+0000", 2024-09-14T23:57:16.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027599597 2024-09-14T23:57:16.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:16.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.228806+0000", 2024-09-14T23:57:16.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9616999999999994e-05 2024-09-14T23:57:16.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:16.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.228866+0000", 2024-09-14T23:57:16.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0841000000000002e-05 2024-09-14T23:57:16.842 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:16.842 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:16.842 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:16.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.842 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:55349 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 868352~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:16.842 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.204621+0000", 2024-09-14T23:57:16.842 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.568309897000001, 2024-09-14T23:57:16.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030294596999999999, 2024-09-14T23:57:16.842 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:16.842 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:16.843 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:16.843 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:16.843 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:16.843 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:16.843 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 55349 2024-09-14T23:57:16.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.843 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:16.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:16.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204621+0000", 2024-09-14T23:57:16.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:16.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:16.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204621+0000", 2024-09-14T23:57:16.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:16.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:16.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204622+0000", 2024-09-14T23:57:16.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.58e-07 2024-09-14T23:57:16.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:16.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204624+0000", 2024-09-14T23:57:16.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.193e-06 2024-09-14T23:57:16.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:16.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204631+0000", 2024-09-14T23:57:16.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1910000000000003e-06 2024-09-14T23:57:16.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:16.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204637+0000", 2024-09-14T23:57:16.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5169999999999999e-06 2024-09-14T23:57:16.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:16.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.229653+0000", 2024-09-14T23:57:16.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025016283 2024-09-14T23:57:16.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:16.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.229721+0000", 2024-09-14T23:57:16.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8138000000000004e-05 2024-09-14T23:57:16.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:16.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.234916+0000", 2024-09-14T23:57:16.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.005194817 2024-09-14T23:57:16.847 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:16.847 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:16.847 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:16.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.847 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:55363 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 3145728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:16.847 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.204982+0000", 2024-09-14T23:57:16.847 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.567949041, 2024-09-14T23:57:16.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027646239999999999, 2024-09-14T23:57:16.848 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:16.848 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:16.848 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:16.848 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:16.848 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:16.848 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:16.848 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 55363 2024-09-14T23:57:16.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.848 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:16.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:16.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204982+0000", 2024-09-14T23:57:16.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:16.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:16.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204982+0000", 2024-09-14T23:57:16.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:16.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:16.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204983+0000", 2024-09-14T23:57:16.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2199999999999999e-07 2024-09-14T23:57:16.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:16.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204985+0000", 2024-09-14T23:57:16.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.261e-06 2024-09-14T23:57:16.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:16.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204991+0000", 2024-09-14T23:57:16.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7069999999999997e-06 2024-09-14T23:57:16.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:16.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204995+0000", 2024-09-14T23:57:16.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.853e-06 2024-09-14T23:57:16.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:16.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.232491+0000", 2024-09-14T23:57:16.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027495322999999999 2024-09-14T23:57:16.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:16.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.232563+0000", 2024-09-14T23:57:16.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1972999999999997e-05 2024-09-14T23:57:16.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:16.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.232628+0000", 2024-09-14T23:57:16.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5701e-05 2024-09-14T23:57:16.852 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:16.852 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:16.852 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:16.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.852 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:55364 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 3325952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:16.852 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.205003+0000", 2024-09-14T23:57:16.852 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.567928516, 2024-09-14T23:57:16.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027781963, 2024-09-14T23:57:16.853 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:16.853 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:16.853 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:16.853 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:16.853 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:16.853 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:16.853 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 55364 2024-09-14T23:57:16.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.853 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:16.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:16.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205003+0000", 2024-09-14T23:57:16.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:16.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:16.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205003+0000", 2024-09-14T23:57:16.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:16.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:16.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205003+0000", 2024-09-14T23:57:16.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2899999999999999e-07 2024-09-14T23:57:16.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:16.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205013+0000", 2024-09-14T23:57:16.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.611e-06 2024-09-14T23:57:16.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:16.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205020+0000", 2024-09-14T23:57:16.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2509999999999997e-06 2024-09-14T23:57:16.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:16.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205025+0000", 2024-09-14T23:57:16.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1399999999999999e-06 2024-09-14T23:57:16.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:16.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.232653+0000", 2024-09-14T23:57:16.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027627972000000001 2024-09-14T23:57:16.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:16.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.232723+0000", 2024-09-14T23:57:16.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9907999999999995e-05 2024-09-14T23:57:16.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:16.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.232785+0000", 2024-09-14T23:57:16.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1651999999999999e-05 2024-09-14T23:57:16.856 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:16.856 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:16.857 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:16.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.857 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:55365 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 3506176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:16.857 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.205033+0000", 2024-09-14T23:57:16.857 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.567898276999999, 2024-09-14T23:57:16.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027914906999999999, 2024-09-14T23:57:16.857 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:16.858 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:16.858 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:16.858 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:16.858 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:16.858 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:16.858 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 55365 2024-09-14T23:57:16.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.858 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:16.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:16.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205033+0000", 2024-09-14T23:57:16.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:16.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:16.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205033+0000", 2024-09-14T23:57:16.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:16.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:16.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205033+0000", 2024-09-14T23:57:16.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9900000000000001e-07 2024-09-14T23:57:16.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:16.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205036+0000", 2024-09-14T23:57:16.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3319999999999999e-06 2024-09-14T23:57:16.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:16.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205042+0000", 2024-09-14T23:57:16.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3049999999999998e-06 2024-09-14T23:57:16.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:16.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205047+0000", 2024-09-14T23:57:16.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0420000000000002e-06 2024-09-14T23:57:16.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:16.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.232809+0000", 2024-09-14T23:57:16.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027761917000000001 2024-09-14T23:57:16.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:16.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.232879+0000", 2024-09-14T23:57:16.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9597000000000005e-05 2024-09-14T23:57:16.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:16.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.232948+0000", 2024-09-14T23:57:16.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9214999999999994e-05 2024-09-14T23:57:16.861 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:16.861 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:16.861 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:16.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.862 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:55366 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 3686400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:16.862 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.205054+0000", 2024-09-14T23:57:16.862 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.56787709, 2024-09-14T23:57:16.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028049935000000002, 2024-09-14T23:57:16.862 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:16.862 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:16.862 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:16.862 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:16.862 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:16.862 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:16.863 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 55366 2024-09-14T23:57:16.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.863 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:16.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:16.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205054+0000", 2024-09-14T23:57:16.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:16.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:16.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205054+0000", 2024-09-14T23:57:16.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:16.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.864 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:16.864 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205055+0000", 2024-09-14T23:57:16.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9999999999999998e-07 2024-09-14T23:57:16.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.864 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:16.864 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205065+0000", 2024-09-14T23:57:16.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0489000000000001e-05 2024-09-14T23:57:16.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.864 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:16.864 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205073+0000", 2024-09-14T23:57:16.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9840000000000001e-06 2024-09-14T23:57:16.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:16.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205085+0000", 2024-09-14T23:57:16.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2052000000000001e-05 2024-09-14T23:57:16.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:16.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.232975+0000", 2024-09-14T23:57:16.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027889646000000001 2024-09-14T23:57:16.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:16.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.233039+0000", 2024-09-14T23:57:16.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3993999999999998e-05 2024-09-14T23:57:16.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:16.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.233104+0000", 2024-09-14T23:57:16.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5270000000000004e-05 2024-09-14T23:57:16.866 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:16.866 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:16.866 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:16.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.866 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:55367 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 3866624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:16.866 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.205098+0000", 2024-09-14T23:57:16.866 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.567833472, 2024-09-14T23:57:16.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028159871, 2024-09-14T23:57:16.867 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:16.867 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:16.867 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:16.867 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:16.867 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:16.867 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:16.867 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 55367 2024-09-14T23:57:16.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.867 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:16.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:16.868 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205098+0000", 2024-09-14T23:57:16.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:16.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.868 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:16.868 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205098+0000", 2024-09-14T23:57:16.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:16.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.868 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:16.869 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205098+0000", 2024-09-14T23:57:16.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4000000000000002e-07 2024-09-14T23:57:16.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.869 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:16.869 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205101+0000", 2024-09-14T23:57:16.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7860000000000001e-06 2024-09-14T23:57:16.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.869 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:16.869 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205109+0000", 2024-09-14T23:57:16.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4039999999999998e-06 2024-09-14T23:57:16.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:16.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205123+0000", 2024-09-14T23:57:16.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.42e-05 2024-09-14T23:57:16.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:16.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.233129+0000", 2024-09-14T23:57:16.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028006316999999999 2024-09-14T23:57:16.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:16.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.233196+0000", 2024-09-14T23:57:16.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7234e-05 2024-09-14T23:57:16.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:16.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.233258+0000", 2024-09-14T23:57:16.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1389999999999993e-05 2024-09-14T23:57:16.871 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:16.871 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:16.871 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:16.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.871 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:55368 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 4046848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:16.871 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.205132+0000", 2024-09-14T23:57:16.871 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.56779916, 2024-09-14T23:57:16.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028285467000000002, 2024-09-14T23:57:16.871 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:16.871 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:16.872 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:16.872 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:16.872 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:16.872 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:16.872 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 55368 2024-09-14T23:57:16.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.872 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:16.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:16.872 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205132+0000", 2024-09-14T23:57:16.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:16.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:16.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205132+0000", 2024-09-14T23:57:16.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:16.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.873 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:16.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205133+0000", 2024-09-14T23:57:16.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.13e-07 2024-09-14T23:57:16.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.873 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:16.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205135+0000", 2024-09-14T23:57:16.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5490000000000001e-06 2024-09-14T23:57:16.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:16.874 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205142+0000", 2024-09-14T23:57:16.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.007e-06 2024-09-14T23:57:16.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:16.874 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205147+0000", 2024-09-14T23:57:16.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.169e-06 2024-09-14T23:57:16.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:16.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.233281+0000", 2024-09-14T23:57:16.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028134059999999999 2024-09-14T23:57:16.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:16.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.233355+0000", 2024-09-14T23:57:16.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3120000000000005e-05 2024-09-14T23:57:16.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:16.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:16.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:16.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.233418+0000", 2024-09-14T23:57:16.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3048999999999999e-05 2024-09-14T23:57:16.875 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:16.875 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:16.876 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:16.876 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:16.876 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:16.876 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:16.877 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:16.876+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:16.878 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T23:57:16.978 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:16.977+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:17.031 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:17.031 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:57:17.031 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:57:17.031 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:17.040 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T23:57:17.052 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T23:57:17.085 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:17.082+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:17.125 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.2 to 1 (10000) 2024-09-14T23:57:17.147 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-14T23:57:17.183 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:17.183 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:57:17.183 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:57:17.183 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:57:17.183 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:17.184 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:17.181+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:17.195 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T23:57:17.244 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:57:17.244 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:57:17.264 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:57:17.264 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:57:17.280 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:17.277+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:17.284 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:57:17.284 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:57:17.302 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-09-14T23:57:17.306 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:57:17.307 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:57:17.312 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:17.312 INFO:teuthology.orchestra.run.smithi042.stdout:2139 write 0x1b8fcc2 thru 0x1b90b6c (0xeab bytes) 2024-09-14T23:57:17.339 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:57:17.339 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:57:17.372 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:57:17.373 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:57:17.381 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:17.380+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:17.385 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:17.385 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:17.398 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T23:57:17.408 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:57:17.408 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:57:17.418 INFO:teuthology.orchestra.run.smithi042.stdout:2140 read 0xdcd18bc thru 0xdcddea7 (0xc5ec bytes) 2024-09-14T23:57:17.418 INFO:teuthology.orchestra.run.smithi042.stdout:2142 punch from 0xca82140 to 0xca87c88, (0x5b48 bytes) 2024-09-14T23:57:17.418 INFO:teuthology.orchestra.run.smithi042.stdout:2143 punch from 0x1ac1157 to 0x1ac4d9e, (0x3c47 bytes) 2024-09-14T23:57:17.418 INFO:teuthology.orchestra.run.smithi042.stdout:2144 read 0xde54f1e thru 0xde5c5f5 (0x76d8 bytes) 2024-09-14T23:57:17.422 INFO:teuthology.orchestra.run.smithi042.stdout:2145 trunc from 0xecea973 to 0xaad32dd 2024-09-14T23:57:17.448 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:57:17.448 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:57:17.518 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:17.516+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:17.583 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:17.581+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:17.591 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:17.591 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:17.605 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T23:57:17.683 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:17.682+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:17.784 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:17.783+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:17.824 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:17.824 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:17.837 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T23:57:17.885 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:17.882+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:17.985 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:17.984+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:17.994 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:17.995 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:17.995 INFO:teuthology.orchestra.run.smithi042.stdout:2148 read 0x5c111c1 thru 0x5c1ebda (0xda1a bytes) 2024-09-14T23:57:17.997 INFO:teuthology.orchestra.run.smithi042.stdout:2149 trunc from 0xaad32dd to 0x8d7a6c3 2024-09-14T23:57:17.997 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:17.997 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:18.009 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T23:57:18.009 INFO:teuthology.orchestra.run.smithi042.stdout:2150 trunc from 0x8d7a6c3 to 0xb5b30d6 2024-09-14T23:57:18.014 INFO:teuthology.orchestra.run.smithi042.stdout:2151 writesame 0x1758ca6 thru 0x175e6f4 data_size 0x1(0x5a4f bytes) 2024-09-14T23:57:18.087 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:18.084+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:18.092 INFO:teuthology.orchestra.run.smithi042.stdout:2152 write 0xd3878f5 thru 0xd394e93 (0xd59f bytes) 2024-09-14T23:57:18.105 INFO:teuthology.orchestra.run.smithi042.stdout:2153 trunc from 0xd394e94 to 0x3b7517d 2024-09-14T23:57:18.135 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.3 to 1 (10000) 2024-09-14T23:57:18.156 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-14T23:57:18.173 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:18.174 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:18.187 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:18.186+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:18.190 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T23:57:18.206 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T23:57:18.289 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:18.287+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:18.389 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:18.388+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:18.407 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:18.407 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:18.419 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T23:57:18.490 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:18.489+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:18.591 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:18.590+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:18.604 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:18.604 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:18.615 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T23:57:18.651 INFO:teuthology.orchestra.run.smithi042.stdout:2154 trunc from 0x3b7517d to 0xae8299f 2024-09-14T23:57:18.666 INFO:teuthology.orchestra.run.smithi042.stdout:2155 write 0xdef7a5a thru 0xdef86bd (0xc64 bytes) 2024-09-14T23:57:18.670 INFO:teuthology.orchestra.run.smithi042.stdout:2156 punch from 0x9bb43f1 to 0x9bbc213, (0x7e22 bytes) 2024-09-14T23:57:18.670 INFO:teuthology.orchestra.run.smithi042.stdout:2159 punch from 0xbd02f41 to 0xbd063ae, (0x346d bytes) 2024-09-14T23:57:18.670 INFO:teuthology.orchestra.run.smithi042.stdout:2160 punch from 0xc51a4c5 to 0xc51b096, (0xbd1 bytes) 2024-09-14T23:57:18.670 INFO:teuthology.orchestra.run.smithi042.stdout:2162 writesame 0x277a9b0 thru 0x2782a96 data_size 0x1(0x80e7 bytes) 2024-09-14T23:57:18.681 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:18.681 INFO:teuthology.orchestra.run.smithi042.stdout:2164 writesame 0xb06296e thru 0xb070335 data_size 0x8a(0xd9c8 bytes) 2024-09-14T23:57:18.687 INFO:teuthology.orchestra.run.smithi042.stdout:2165 read 0x2a8cd5b thru 0x2a94b0d (0x7db3 bytes) 2024-09-14T23:57:18.687 INFO:teuthology.orchestra.run.smithi042.stdout:2167 write 0x1509814 thru 0x1512053 (0x8840 bytes) 2024-09-14T23:57:18.689 INFO:teuthology.orchestra.run.smithi042.stdout:2168 writesame 0xb5cbde6 thru 0xb5d6962 data_size 0x1(0xab7d bytes) 2024-09-14T23:57:18.692 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:18.690+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:18.703 INFO:teuthology.orchestra.run.smithi042.stdout:2171 read 0x124ac03 thru 0x125a0af (0xf4ad bytes) 2024-09-14T23:57:18.703 INFO:teuthology.orchestra.run.smithi042.stdout:2172 write 0x8f9dbd1 thru 0x8f9e012 (0x442 bytes) 2024-09-14T23:57:18.704 INFO:teuthology.orchestra.run.smithi042.stdout:2173 punch from 0xa0f11ce to 0xa100c46, (0xfa78 bytes) 2024-09-14T23:57:18.705 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:18.705 INFO:teuthology.orchestra.run.smithi042.stdout:2175 write 0x811937a thru 0x8120dc7 (0x7a4e bytes) 2024-09-14T23:57:18.708 INFO:teuthology.orchestra.run.smithi042.stdout:2176 trunc from 0xdef86be to 0xd6f34fd 2024-09-14T23:57:18.729 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:18.729 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:18.730 INFO:teuthology.orchestra.run.smithi042.stdout:2178 trunc from 0xd6f34fd to 0x699f92 2024-09-14T23:57:18.741 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T23:57:18.746 INFO:teuthology.orchestra.run.smithi042.stdout:2179 read 0x1fb1df thru 0x2006e6 (0x5508 bytes) 2024-09-14T23:57:18.769 INFO:teuthology.orchestra.run.smithi042.stdout:2181 writesame 0xe4ae4f9 thru 0xe4b2c0a data_size 0xb(0x4712 bytes) 2024-09-14T23:57:18.769 INFO:teuthology.orchestra.run.smithi042.stdout:2182 read 0x7ce7c96 thru 0x7ce9cb9 (0x2024 bytes) 2024-09-14T23:57:18.769 INFO:teuthology.orchestra.run.smithi042.stdout:2183 write 0x6b75c66 thru 0x6b7dd46 (0x80e1 bytes) 2024-09-14T23:57:18.770 INFO:teuthology.orchestra.run.smithi042.stdout:2184 read 0xe30d2d7 thru 0xe314d2a (0x7a54 bytes) 2024-09-14T23:57:18.781 INFO:teuthology.orchestra.run.smithi042.stdout:2185 trunc from 0xe4b2c0b to 0x50257f8 2024-09-14T23:57:18.793 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:18.792+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:18.795 INFO:teuthology.orchestra.run.smithi042.stdout:2186 trunc from 0x50257f8 to 0xa010514 2024-09-14T23:57:18.801 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:18.811 INFO:teuthology.orchestra.run.smithi042.stdout:2188 writesame 0xd1d1e31 thru 0xd1de955 data_size 0x5(0xcb25 bytes) 2024-09-14T23:57:18.811 INFO:teuthology.orchestra.run.smithi042.stdout:2189 read 0xc81444d thru 0xc822eea (0xea9e bytes) 2024-09-14T23:57:18.811 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:18.811 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:18.811 INFO:teuthology.orchestra.run.smithi042.stdout:2192 write 0xa28202c thru 0xa28dcdb (0xbcb0 bytes) 2024-09-14T23:57:18.815 INFO:teuthology.orchestra.run.smithi042.stdout:2194 punch from 0x83f662d to 0x8402414, (0xbde7 bytes) 2024-09-14T23:57:18.815 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:18.815 INFO:teuthology.orchestra.run.smithi042.stdout:2196 writesame 0x43d1ffb thru 0x43ddda1 data_size 0x1(0xbda7 bytes) 2024-09-14T23:57:18.841 INFO:teuthology.orchestra.run.smithi042.stdout:2197 write 0xec097fc thru 0xec0afc9 (0x17ce bytes) 2024-09-14T23:57:18.845 INFO:teuthology.orchestra.run.smithi042.stdout:2200 read 0xdd18017 thru 0xdd1d45d (0x5447 bytes) 2024-09-14T23:57:18.845 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:18.845 INFO:teuthology.orchestra.run.smithi042.stdout:2202 write 0x3bb9284 thru 0x3bc019b (0x6f18 bytes) 2024-09-14T23:57:18.846 INFO:teuthology.orchestra.run.smithi042.stdout:2203 write 0x2deb76f thru 0x2df0bd8 (0x546a bytes) 2024-09-14T23:57:18.847 INFO:teuthology.orchestra.run.smithi042.stdout:2204 read 0xb55608b thru 0xb55d2f9 (0x726f bytes) 2024-09-14T23:57:18.855 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:18.856 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:18.862 INFO:teuthology.orchestra.run.smithi042.stdout:2205 trunc from 0xec0afca to 0x2916f76 2024-09-14T23:57:18.864 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T23:57:18.874 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:18.874 INFO:teuthology.orchestra.run.smithi042.stdout:2207 clone 24 order 20 su 16384 sc 2 2024-09-14T23:57:18.894 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:18.891+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:18.995 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:18.994+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:19.011 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:19.012 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:19.020 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T23:57:19.096 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:19.094+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:19.139 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is set 2024-09-14T23:57:19.141 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.4 to 1 (10000) 2024-09-14T23:57:19.159 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-14T23:57:19.171 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:19.171 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:19.180 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T23:57:19.225 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:19.224+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:19.243 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone23 from 0x2916f76 (overlap 0x699f92) to 0x547fa8 2024-09-14T23:57:19.294 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:19.294 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:19.297 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:19.296+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:19.303 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T23:57:19.387 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-09-14T23:57:19.398 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:19.397+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:19.455 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:19.455 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:19.464 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T23:57:19.478 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T23:57:19.522 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:19.520+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:19.600 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:19.597+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:19.629 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:19.629 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:19.638 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T23:57:19.661 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:57:19.661 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:57:19.679 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:57:19.679 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:57:19.696 INFO:teuthology.orchestra.run.smithi042.stdout:2209 write 0xe50d06e thru 0xe51cc63 (0xfbf6 bytes) 2024-09-14T23:57:19.697 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:57:19.697 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:57:19.699 INFO:teuthology.orchestra.run.smithi042.stdout:2210 write 0xe2e58b5 thru 0xe2f0dad (0xb4f9 bytes) 2024-09-14T23:57:19.701 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:19.698+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:19.703 INFO:teuthology.orchestra.run.smithi042.stdout:2212 write 0x3ef2ef1 thru 0x3ef62bb (0x33cb bytes) 2024-09-14T23:57:19.707 INFO:teuthology.orchestra.run.smithi042.stdout:2213 read 0xc05e70d thru 0xc06b31c (0xcc10 bytes) 2024-09-14T23:57:19.707 INFO:teuthology.orchestra.run.smithi042.stdout:2214 read 0x339156 thru 0x342e55 (0x9d00 bytes) 2024-09-14T23:57:19.707 INFO:teuthology.orchestra.run.smithi042.stdout:2216 writesame 0xb1ac8e8 thru 0xb1b3d9c data_size 0x17(0x74b5 bytes) 2024-09-14T23:57:19.710 INFO:teuthology.orchestra.run.smithi042.stdout:2217 write 0x97ea22b thru 0x97ea733 (0x509 bytes) 2024-09-14T23:57:19.714 INFO:teuthology.orchestra.run.smithi042.stdout:2219 read 0x8e99611 thru 0x8ea8723 (0xf113 bytes) 2024-09-14T23:57:19.714 INFO:teuthology.orchestra.run.smithi042.stdout:2220 punch from 0x7808231 to 0x780c8af, (0x467e bytes) 2024-09-14T23:57:19.714 INFO:teuthology.orchestra.run.smithi042.stdout:2221 punch from 0xbd37369 to 0xbd44872, (0xd509 bytes) 2024-09-14T23:57:19.714 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:19.715 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:19.715 INFO:teuthology.orchestra.run.smithi042.stdout:2227 writesame 0xbb7be54 thru 0xbb8858a data_size 0x1(0xc737 bytes) 2024-09-14T23:57:19.716 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:57:19.717 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:57:19.732 INFO:teuthology.orchestra.run.smithi042.stdout:2228 write 0xcb988c thru 0xcbecf8 (0x546d bytes) 2024-09-14T23:57:19.735 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:57:19.736 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:57:19.741 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:19.741 INFO:teuthology.orchestra.run.smithi042.stdout:2231 read 0xa7a0248 thru 0xa7a9fc3 (0x9d7c bytes) 2024-09-14T23:57:19.742 INFO:teuthology.orchestra.run.smithi042.stdout:2232 write 0x426daa thru 0x4338f8 (0xcb4f bytes) 2024-09-14T23:57:19.757 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:19.757 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:19.757 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:57:19.757 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:57:19.766 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T23:57:19.769 INFO:teuthology.orchestra.run.smithi042.stdout:2233 trunc from 0xe51cc64 to 0x9063e6 2024-09-14T23:57:19.778 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:57:19.778 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:57:19.789 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:19.797 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:57:19.797 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:57:19.809 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:19.808+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:19.810 INFO:teuthology.orchestra.run.smithi042.stdout:2236 write 0xe7a3518 thru 0xe7a6e39 (0x3922 bytes) 2024-09-14T23:57:19.814 INFO:teuthology.orchestra.run.smithi042.stdout:2237 punch from 0xc34b218 to 0xc351b30, (0x6918 bytes) 2024-09-14T23:57:19.814 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:19.814 INFO:teuthology.orchestra.run.smithi042.stdout:2241 writesame 0xad60374 thru 0xad63d78 data_size 0x1(0x3a05 bytes) 2024-09-14T23:57:19.822 INFO:teuthology.orchestra.run.smithi042.stdout:2242 punch from 0x22c413e to 0x22c6a2e, (0x28f0 bytes) 2024-09-14T23:57:19.823 INFO:teuthology.orchestra.run.smithi042.stdout:2244 write 0xcaabe9d thru 0xcab38df (0x7a43 bytes) 2024-09-14T23:57:19.826 INFO:teuthology.orchestra.run.smithi042.stdout:2245 read 0xd1a2b95 thru 0xd1a416d (0x15d9 bytes) 2024-09-14T23:57:19.826 INFO:teuthology.orchestra.run.smithi042.stdout:2246 write 0x36da87b thru 0x36df1df (0x4965 bytes) 2024-09-14T23:57:19.829 INFO:teuthology.orchestra.run.smithi042.stdout:2248 write 0xdae880b thru 0xdaf262b (0x9e21 bytes) 2024-09-14T23:57:19.833 INFO:teuthology.orchestra.run.smithi042.stdout:2249 read 0xd202bdb thru 0xd209cdb (0x7101 bytes) 2024-09-14T23:57:19.833 INFO:teuthology.orchestra.run.smithi042.stdout:2251 write 0x1dddb64 thru 0x1dded39 (0x11d6 bytes) 2024-09-14T23:57:19.836 INFO:teuthology.orchestra.run.smithi042.stdout:2252 read 0x47e3b12 thru 0x47eacc5 (0x71b4 bytes) 2024-09-14T23:57:19.837 INFO:teuthology.orchestra.run.smithi042.stdout:2253 write 0x3ee7945 thru 0x3ef6c82 (0xf33e bytes) 2024-09-14T23:57:19.841 INFO:teuthology.orchestra.run.smithi042.stdout:2254 read 0x9c98dfe thru 0x9c9f657 (0x685a bytes) 2024-09-14T23:57:19.841 INFO:teuthology.orchestra.run.smithi042.stdout:2255 punch from 0x3ffec61 to 0x4003bb3, (0x4f52 bytes) 2024-09-14T23:57:19.854 INFO:teuthology.orchestra.run.smithi042.stdout:2256 trunc from 0xe7a6e3a to 0x2266d7c 2024-09-14T23:57:19.865 INFO:teuthology.orchestra.run.smithi042.stdout:2257 read 0xa0baf4 thru 0xa0da54 (0x1f61 bytes) 2024-09-14T23:57:19.866 INFO:teuthology.orchestra.run.smithi042.stdout:2259 read 0x6c4b94 thru 0x6d283b (0xdca8 bytes) 2024-09-14T23:57:19.884 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:19.885 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:57:19.885 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:57:19.885 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:57:19.885 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.885 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68736 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 5906432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:19.885 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.264178+0000", 2024-09-14T23:57:19.885 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.6182832779999998, 2024-09-14T23:57:19.885 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.045913849999999999, 2024-09-14T23:57:19.885 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:19.885 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:19.885 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:19.885 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:19.885 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:19.886 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:19.886 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68736 2024-09-14T23:57:19.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.886 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:19.886 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.886 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:19.886 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264178+0000", 2024-09-14T23:57:19.886 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:19.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.886 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.886 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:19.886 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264178+0000", 2024-09-14T23:57:19.886 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:19.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.887 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.887 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:19.887 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264179+0000", 2024-09-14T23:57:19.887 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8599999999999999e-07 2024-09-14T23:57:19.887 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.887 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.887 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:19.887 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264181+0000", 2024-09-14T23:57:19.887 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.855e-06 2024-09-14T23:57:19.887 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.887 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.887 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:19.887 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264187+0000", 2024-09-14T23:57:19.887 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1539999999999999e-06 2024-09-14T23:57:19.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.888 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.888 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:19.888 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264192+0000", 2024-09-14T23:57:19.888 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6409999999999999e-06 2024-09-14T23:57:19.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.888 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.888 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:19.888 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.309931+0000", 2024-09-14T23:57:19.888 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.045738442999999997 2024-09-14T23:57:19.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.888 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.888 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:19.888 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310017+0000", 2024-09-14T23:57:19.889 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5829999999999996e-05 2024-09-14T23:57:19.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.889 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.889 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:19.889 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310092+0000", 2024-09-14T23:57:19.889 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5541000000000001e-05 2024-09-14T23:57:19.889 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:19.889 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:19.889 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:19.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.889 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.889 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68737 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 6021120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:19.889 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.264199+0000", 2024-09-14T23:57:19.889 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.6182628130000003, 2024-09-14T23:57:19.890 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046063354000000001, 2024-09-14T23:57:19.890 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:19.890 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:19.890 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:19.890 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:19.890 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:19.890 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:19.890 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68737 2024-09-14T23:57:19.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.890 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:19.890 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.890 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:19.890 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264199+0000", 2024-09-14T23:57:19.890 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:19.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.891 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.891 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:19.891 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264199+0000", 2024-09-14T23:57:19.891 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:19.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.891 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.891 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:19.891 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264199+0000", 2024-09-14T23:57:19.891 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1600000000000002e-07 2024-09-14T23:57:19.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.891 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.891 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:19.891 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264210+0000", 2024-09-14T23:57:19.892 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0973e-05 2024-09-14T23:57:19.892 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.892 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.892 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:19.892 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264217+0000", 2024-09-14T23:57:19.892 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0659999999999998e-06 2024-09-14T23:57:19.892 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.892 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.892 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:19.892 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264223+0000", 2024-09-14T23:57:19.892 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2039999999999996e-06 2024-09-14T23:57:19.892 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.892 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.893 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:19.893 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310121+0000", 2024-09-14T23:57:19.893 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.045898436000000001 2024-09-14T23:57:19.893 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.893 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.893 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:19.893 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310194+0000", 2024-09-14T23:57:19.893 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2708999999999998e-05 2024-09-14T23:57:19.893 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.893 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.893 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:19.893 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310262+0000", 2024-09-14T23:57:19.893 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8549999999999999e-05 2024-09-14T23:57:19.893 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:19.893 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:19.894 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:19.894 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.894 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.894 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68738 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 6135808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:19.894 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.264234+0000", 2024-09-14T23:57:19.894 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.6182273829999998, 2024-09-14T23:57:19.894 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046212954000000001, 2024-09-14T23:57:19.894 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:19.894 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:19.894 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:19.894 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:19.894 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:19.894 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:19.894 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68738 2024-09-14T23:57:19.895 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.895 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:19.895 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.895 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:19.895 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264234+0000", 2024-09-14T23:57:19.895 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:19.895 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.895 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.895 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:19.895 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264234+0000", 2024-09-14T23:57:19.895 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:19.895 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.895 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.895 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:19.896 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264235+0000", 2024-09-14T23:57:19.896 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6600000000000002e-07 2024-09-14T23:57:19.896 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.896 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.896 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:19.896 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264237+0000", 2024-09-14T23:57:19.896 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3120000000000001e-06 2024-09-14T23:57:19.896 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.896 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.896 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:19.896 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264244+0000", 2024-09-14T23:57:19.896 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6680000000000004e-06 2024-09-14T23:57:19.896 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.896 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.897 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:19.897 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264255+0000", 2024-09-14T23:57:19.897 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.155e-05 2024-09-14T23:57:19.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.897 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.897 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:19.897 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310292+0000", 2024-09-14T23:57:19.897 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046037026000000002 2024-09-14T23:57:19.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.897 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.897 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:19.897 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310375+0000", 2024-09-14T23:57:19.897 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3039000000000002e-05 2024-09-14T23:57:19.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.898 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.898 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:19.898 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310447+0000", 2024-09-14T23:57:19.898 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1892999999999998e-05 2024-09-14T23:57:19.898 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:19.898 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:19.898 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:19.898 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.898 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.898 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68739 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 6250496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:19.898 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.264263+0000", 2024-09-14T23:57:19.898 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.6181988260000004, 2024-09-14T23:57:19.898 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046362974000000001, 2024-09-14T23:57:19.899 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:19.899 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:19.899 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:19.899 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:19.899 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:19.899 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:19.899 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68739 2024-09-14T23:57:19.899 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.899 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:19.899 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.899 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:19.899 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264263+0000", 2024-09-14T23:57:19.899 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:19.899 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.900 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.900 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:19.900 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264263+0000", 2024-09-14T23:57:19.900 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:19.900 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.900 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.900 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:19.900 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264263+0000", 2024-09-14T23:57:19.900 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8599999999999999e-07 2024-09-14T23:57:19.900 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.900 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.900 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:19.900 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264265+0000", 2024-09-14T23:57:19.900 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0020000000000001e-06 2024-09-14T23:57:19.901 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.901 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.901 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:19.901 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264272+0000", 2024-09-14T23:57:19.901 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4799999999999998e-06 2024-09-14T23:57:19.901 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.901 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.901 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:19.901 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264282+0000", 2024-09-14T23:57:19.901 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0244999999999999e-05 2024-09-14T23:57:19.901 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.901 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.901 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:19.901 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310478+0000", 2024-09-14T23:57:19.901 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046196070999999998 2024-09-14T23:57:19.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.902 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.902 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:19.902 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310554+0000", 2024-09-14T23:57:19.902 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6160000000000003e-05 2024-09-14T23:57:19.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.902 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.902 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:19.902 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310626+0000", 2024-09-14T23:57:19.902 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1630000000000004e-05 2024-09-14T23:57:19.902 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:19.902 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:19.902 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:19.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.902 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.903 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68740 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 6365184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:19.903 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.264289+0000", 2024-09-14T23:57:19.903 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.6181723699999999, 2024-09-14T23:57:19.903 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046531222999999997, 2024-09-14T23:57:19.903 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:19.903 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:19.903 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:19.903 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:19.903 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:19.903 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:19.903 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68740 2024-09-14T23:57:19.903 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.903 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:19.903 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.904 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:19.904 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264289+0000", 2024-09-14T23:57:19.904 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:19.904 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.904 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.904 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:19.904 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264289+0000", 2024-09-14T23:57:19.904 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:19.904 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.904 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.904 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:19.904 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264290+0000", 2024-09-14T23:57:19.904 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4299999999999998e-07 2024-09-14T23:57:19.904 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.905 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.905 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:19.905 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264292+0000", 2024-09-14T23:57:19.905 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.181e-06 2024-09-14T23:57:19.905 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.905 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.905 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:19.905 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264298+0000", 2024-09-14T23:57:19.905 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1120000000000001e-06 2024-09-14T23:57:19.905 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.905 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.905 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:19.905 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264303+0000", 2024-09-14T23:57:19.905 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5839999999999996e-06 2024-09-14T23:57:19.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.906 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.906 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:19.906 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310654+0000", 2024-09-14T23:57:19.906 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046351046 2024-09-14T23:57:19.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.906 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.906 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:19.906 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310745+0000", 2024-09-14T23:57:19.906 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1027999999999997e-05 2024-09-14T23:57:19.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.906 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.906 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:19.906 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310821+0000", 2024-09-14T23:57:19.907 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5828999999999995e-05 2024-09-14T23:57:19.907 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:19.907 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:19.907 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:19.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.907 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.907 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68741 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 6479872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:19.907 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.264332+0000", 2024-09-14T23:57:19.907 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.6181293999999999, 2024-09-14T23:57:19.907 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046692392999999999, 2024-09-14T23:57:19.907 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:19.907 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:19.907 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:19.907 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:19.907 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:19.908 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:19.908 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68741 2024-09-14T23:57:19.908 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.908 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:19.908 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.908 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:19.908 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264332+0000", 2024-09-14T23:57:19.908 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:19.908 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.908 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.908 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:19.908 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264332+0000", 2024-09-14T23:57:19.908 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:19.908 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.909 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.909 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:19.909 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264333+0000", 2024-09-14T23:57:19.909 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2899999999999999e-07 2024-09-14T23:57:19.909 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.909 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.909 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:19.909 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264335+0000", 2024-09-14T23:57:19.909 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9870000000000002e-06 2024-09-14T23:57:19.909 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.909 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.909 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:19.909 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264340+0000", 2024-09-14T23:57:19.909 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0069999999999997e-06 2024-09-14T23:57:19.910 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.910 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.910 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:19.910 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264343+0000", 2024-09-14T23:57:19.910 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0960000000000001e-06 2024-09-14T23:57:19.910 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.910 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.910 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:19.910 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310855+0000", 2024-09-14T23:57:19.910 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046511915000000001 2024-09-14T23:57:19.910 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.910 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.910 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:19.910 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310946+0000", 2024-09-14T23:57:19.910 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1290000000000002e-05 2024-09-14T23:57:19.911 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.911 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.911 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:19.911 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311025+0000", 2024-09-14T23:57:19.911 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8769000000000004e-05 2024-09-14T23:57:19.911 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:19.911 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:19.911 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:19.911 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.911 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.911 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68742 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 6594560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:19.911 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.264347+0000", 2024-09-14T23:57:19.911 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.6181143579999997, 2024-09-14T23:57:19.911 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046904279, 2024-09-14T23:57:19.911 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:19.912 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:19.912 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:19.912 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:19.912 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:19.912 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:19.912 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68742 2024-09-14T23:57:19.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.912 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:19.912 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.912 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:19.912 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264347+0000", 2024-09-14T23:57:19.912 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:19.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.912 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.913 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:19.913 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264347+0000", 2024-09-14T23:57:19.913 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:19.913 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.913 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.913 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:19.913 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264348+0000", 2024-09-14T23:57:19.913 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.22e-07 2024-09-14T23:57:19.913 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.913 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.913 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:19.913 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264349+0000", 2024-09-14T23:57:19.913 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2079999999999999e-06 2024-09-14T23:57:19.913 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.914 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.914 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:19.914 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264352+0000", 2024-09-14T23:57:19.914 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1640000000000001e-06 2024-09-14T23:57:19.914 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.914 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.914 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:19.914 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264354+0000", 2024-09-14T23:57:19.914 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3350000000000001e-06 2024-09-14T23:57:19.914 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.914 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.914 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:19.915 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311071+0000", 2024-09-14T23:57:19.915 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046716921000000002 2024-09-14T23:57:19.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.915 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.915 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:19.915 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311165+0000", 2024-09-14T23:57:19.915 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3900000000000006e-05 2024-09-14T23:57:19.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.915 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.915 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:19.915 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311252+0000", 2024-09-14T23:57:19.916 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6428999999999995e-05 2024-09-14T23:57:19.916 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:19.916 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:19.916 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:19.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.916 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.916 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68743 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 6709248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:19.916 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.264359+0000", 2024-09-14T23:57:19.916 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.6181027290000003, 2024-09-14T23:57:19.916 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047086817000000003, 2024-09-14T23:57:19.917 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:19.917 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:19.917 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:19.917 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:19.917 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:19.917 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:19.917 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68743 2024-09-14T23:57:19.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.917 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:19.917 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.917 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:19.917 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264359+0000", 2024-09-14T23:57:19.918 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:19.918 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.918 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.918 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:19.918 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264359+0000", 2024-09-14T23:57:19.918 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:19.918 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.918 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.918 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:19.918 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265672+0000", 2024-09-14T23:57:19.918 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0013131849999999999 2024-09-14T23:57:19.918 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.918 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.919 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:19.919 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265676+0000", 2024-09-14T23:57:19.919 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2030000000000002e-06 2024-09-14T23:57:19.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.919 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.919 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:19.919 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265685+0000", 2024-09-14T23:57:19.919 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7290000000000006e-06 2024-09-14T23:57:19.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.919 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.919 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:19.919 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265691+0000", 2024-09-14T23:57:19.919 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3779999999999998e-06 2024-09-14T23:57:19.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.920 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.920 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:19.920 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311283+0000", 2024-09-14T23:57:19.920 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.045591797000000003 2024-09-14T23:57:19.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.920 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.920 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:19.920 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311368+0000", 2024-09-14T23:57:19.920 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4444999999999995e-05 2024-09-14T23:57:19.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.920 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.921 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:19.921 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311446+0000", 2024-09-14T23:57:19.921 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8079999999999998e-05 2024-09-14T23:57:19.921 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:19.921 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:19.921 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:19.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.921 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.921 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68744 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 6823936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:19.921 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.265703+0000", 2024-09-14T23:57:19.921 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.6167589539999998, 2024-09-14T23:57:19.922 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.045950957000000001, 2024-09-14T23:57:19.922 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:19.922 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:19.922 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:19.922 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:19.922 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:19.922 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:19.922 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68744 2024-09-14T23:57:19.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.922 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:19.922 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.923 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:19.923 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265703+0000", 2024-09-14T23:57:19.923 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:19.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.923 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.923 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:19.923 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265703+0000", 2024-09-14T23:57:19.923 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:19.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.923 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.923 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:19.923 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265703+0000", 2024-09-14T23:57:19.923 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9900000000000001e-07 2024-09-14T23:57:19.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.924 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.924 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:19.924 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265706+0000", 2024-09-14T23:57:19.924 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4329999999999998e-06 2024-09-14T23:57:19.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.924 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.924 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:19.924 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265712+0000", 2024-09-14T23:57:19.924 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8020000000000003e-06 2024-09-14T23:57:19.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.924 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.924 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:19.925 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265718+0000", 2024-09-14T23:57:19.925 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.13e-06 2024-09-14T23:57:19.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.925 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.925 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:19.925 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311494+0000", 2024-09-14T23:57:19.925 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.045776039999999997 2024-09-14T23:57:19.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.925 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.925 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:19.925 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311577+0000", 2024-09-14T23:57:19.925 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3057000000000001e-05 2024-09-14T23:57:19.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.926 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.926 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:19.926 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311654+0000", 2024-09-14T23:57:19.926 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6996000000000006e-05 2024-09-14T23:57:19.926 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:19.926 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:19.926 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:19.926 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.926 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.926 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68745 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 6938624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:19.926 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.265725+0000", 2024-09-14T23:57:19.926 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.6167371240000001, 2024-09-14T23:57:19.927 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046101672000000003, 2024-09-14T23:57:19.927 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:19.927 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:19.927 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:19.927 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:19.927 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:19.927 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:19.927 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68745 2024-09-14T23:57:19.927 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.927 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:19.927 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.928 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:19.928 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265725+0000", 2024-09-14T23:57:19.928 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:19.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.928 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.928 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:19.928 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265725+0000", 2024-09-14T23:57:19.928 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:19.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.928 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.929 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:19.929 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265725+0000", 2024-09-14T23:57:19.929 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5299999999999999e-07 2024-09-14T23:57:19.929 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.929 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.929 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:19.929 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265727+0000", 2024-09-14T23:57:19.929 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.069e-06 2024-09-14T23:57:19.929 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.929 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.929 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:19.929 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265733+0000", 2024-09-14T23:57:19.930 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1510000000000001e-06 2024-09-14T23:57:19.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.930 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.930 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:19.930 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265738+0000", 2024-09-14T23:57:19.930 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7010000000000004e-06 2024-09-14T23:57:19.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.930 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.930 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:19.930 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311707+0000", 2024-09-14T23:57:19.930 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.045968602999999997 2024-09-14T23:57:19.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.931 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.931 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:19.931 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311776+0000", 2024-09-14T23:57:19.931 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9438999999999997e-05 2024-09-14T23:57:19.931 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.931 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.931 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:19.931 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311826+0000", 2024-09-14T23:57:19.931 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0256000000000001e-05 2024-09-14T23:57:19.931 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:19.931 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:19.931 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:19.932 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.932 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.932 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68746 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 7053312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:19.932 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.265744+0000", 2024-09-14T23:57:19.932 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.6167174930000003, 2024-09-14T23:57:19.932 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046203505999999998, 2024-09-14T23:57:19.932 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:19.932 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:19.932 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:19.932 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:19.932 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:19.932 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:19.933 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68746 2024-09-14T23:57:19.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.933 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:19.933 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.933 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:19.933 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265744+0000", 2024-09-14T23:57:19.933 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:19.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.933 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.933 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:19.933 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265744+0000", 2024-09-14T23:57:19.933 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:19.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.934 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.934 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:19.934 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265745+0000", 2024-09-14T23:57:19.934 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9200000000000002e-07 2024-09-14T23:57:19.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.934 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.934 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:19.934 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265746+0000", 2024-09-14T23:57:19.934 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8819999999999999e-06 2024-09-14T23:57:19.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.934 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:19.935 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265753+0000", 2024-09-14T23:57:19.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1990000000000003e-06 2024-09-14T23:57:19.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.935 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:19.935 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265760+0000", 2024-09-14T23:57:19.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9280000000000004e-06 2024-09-14T23:57:19.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.935 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:19.936 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311859+0000", 2024-09-14T23:57:19.936 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046099627999999997 2024-09-14T23:57:19.936 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.936 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.936 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:19.936 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311902+0000", 2024-09-14T23:57:19.936 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3239999999999999e-05 2024-09-14T23:57:19.936 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.936 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.936 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:19.936 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311948+0000", 2024-09-14T23:57:19.937 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5237000000000001e-05 2024-09-14T23:57:19.937 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:19.937 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:19.937 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:19.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.937 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.937 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68747 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 7168000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:19.937 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.265766+0000", 2024-09-14T23:57:19.937 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.6166953319999999, 2024-09-14T23:57:19.937 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046303703000000002, 2024-09-14T23:57:19.937 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:19.938 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:19.938 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:19.938 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:19.938 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:19.938 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:19.938 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68747 2024-09-14T23:57:19.938 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.938 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:19.938 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.938 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:19.938 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265766+0000", 2024-09-14T23:57:19.938 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:19.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.939 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:19.939 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265766+0000", 2024-09-14T23:57:19.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:19.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.939 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:19.939 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265767+0000", 2024-09-14T23:57:19.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9200000000000002e-07 2024-09-14T23:57:19.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:19.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265769+0000", 2024-09-14T23:57:19.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9369999999999998e-06 2024-09-14T23:57:19.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:19.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265775+0000", 2024-09-14T23:57:19.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9499999999999998e-06 2024-09-14T23:57:19.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:19.941 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265779+0000", 2024-09-14T23:57:19.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7979999999999996e-06 2024-09-14T23:57:19.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.941 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:19.941 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311977+0000", 2024-09-14T23:57:19.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046197437000000001 2024-09-14T23:57:19.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.941 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:19.941 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.312024+0000", 2024-09-14T23:57:19.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6780999999999998e-05 2024-09-14T23:57:19.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.942 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.942 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:19.942 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.312070+0000", 2024-09-14T23:57:19.942 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6408e-05 2024-09-14T23:57:19.942 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:19.942 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:19.942 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:19.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.942 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.942 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68748 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 7282688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:19.943 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.265785+0000", 2024-09-14T23:57:19.943 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.6166762710000002, 2024-09-14T23:57:19.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046405109999999999, 2024-09-14T23:57:19.943 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:19.943 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:19.943 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:19.943 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:19.943 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:19.943 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:19.943 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68748 2024-09-14T23:57:19.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.944 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:19.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:19.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265785+0000", 2024-09-14T23:57:19.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:19.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:19.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265785+0000", 2024-09-14T23:57:19.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:19.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:19.945 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265786+0000", 2024-09-14T23:57:19.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0900000000000002e-07 2024-09-14T23:57:19.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:19.945 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265796+0000", 2024-09-14T23:57:19.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8490000000000001e-06 2024-09-14T23:57:19.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.946 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:19.946 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265802+0000", 2024-09-14T23:57:19.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7309999999999996e-06 2024-09-14T23:57:19.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.946 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:19.946 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265812+0000", 2024-09-14T23:57:19.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8789999999999998e-06 2024-09-14T23:57:19.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.946 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:19.946 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.312098+0000", 2024-09-14T23:57:19.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046285862999999997 2024-09-14T23:57:19.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.947 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:19.947 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.312140+0000", 2024-09-14T23:57:19.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1718000000000003e-05 2024-09-14T23:57:19.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.947 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:19.947 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.312190+0000", 2024-09-14T23:57:19.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0661000000000002e-05 2024-09-14T23:57:19.948 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:19.948 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:19.948 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:19.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.948 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68749 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 7397376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:19.948 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.265821+0000", 2024-09-14T23:57:19.948 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.6166407750000005, 2024-09-14T23:57:19.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046484708, 2024-09-14T23:57:19.948 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:19.948 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:19.949 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:19.949 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:19.949 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:19.949 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:19.949 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68749 2024-09-14T23:57:19.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.949 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:19.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:19.949 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265821+0000", 2024-09-14T23:57:19.949 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:19.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.950 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:19.950 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265821+0000", 2024-09-14T23:57:19.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:19.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.950 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:19.950 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265821+0000", 2024-09-14T23:57:19.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7300000000000001e-07 2024-09-14T23:57:19.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.950 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:19.951 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265829+0000", 2024-09-14T23:57:19.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8299999999999996e-06 2024-09-14T23:57:19.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.951 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:19.951 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265836+0000", 2024-09-14T23:57:19.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0709999999999997e-06 2024-09-14T23:57:19.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.951 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:19.951 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265841+0000", 2024-09-14T23:57:19.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1950000000000002e-06 2024-09-14T23:57:19.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.952 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:19.952 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.312218+0000", 2024-09-14T23:57:19.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046376873999999998 2024-09-14T23:57:19.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.952 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:19.952 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.312260+0000", 2024-09-14T23:57:19.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1693000000000002e-05 2024-09-14T23:57:19.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.953 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:19.953 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.312306+0000", 2024-09-14T23:57:19.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5571999999999997e-05 2024-09-14T23:57:19.953 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:19.953 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:19.953 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:19.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.953 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68750 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 7512064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:19.953 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.265863+0000", 2024-09-14T23:57:19.953 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.6165989960000005, 2024-09-14T23:57:19.954 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046634766000000001, 2024-09-14T23:57:19.954 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:19.954 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:19.954 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:19.954 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:19.954 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:19.954 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:19.954 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68750 2024-09-14T23:57:19.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.954 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:19.954 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.954 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:19.955 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265863+0000", 2024-09-14T23:57:19.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:19.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.955 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:19.955 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265863+0000", 2024-09-14T23:57:19.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:19.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.955 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:19.955 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265863+0000", 2024-09-14T23:57:19.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0900000000000002e-07 2024-09-14T23:57:19.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:19.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265865+0000", 2024-09-14T23:57:19.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9740000000000001e-06 2024-09-14T23:57:19.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:19.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265869+0000", 2024-09-14T23:57:19.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4159999999999997e-06 2024-09-14T23:57:19.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:19.957 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265873+0000", 2024-09-14T23:57:19.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1999999999999999e-06 2024-09-14T23:57:19.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:19.957 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.312334+0000", 2024-09-14T23:57:19.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046461737000000003 2024-09-14T23:57:19.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:19.957 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.312415+0000", 2024-09-14T23:57:19.958 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0370999999999993e-05 2024-09-14T23:57:19.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.958 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.958 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:19.958 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.312497+0000", 2024-09-14T23:57:19.958 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2658999999999996e-05 2024-09-14T23:57:19.958 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:19.958 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:19.958 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:19.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.958 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.958 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68751 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 7626752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:19.958 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.265877+0000", 2024-09-14T23:57:19.959 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.616584595, 2024-09-14T23:57:19.959 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046759231999999998, 2024-09-14T23:57:19.959 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:19.959 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:19.959 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:19.959 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:19.959 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:19.959 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:19.959 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68751 2024-09-14T23:57:19.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.959 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:19.959 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.959 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:19.959 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265877+0000", 2024-09-14T23:57:19.960 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:19.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.960 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:19.960 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265877+0000", 2024-09-14T23:57:19.960 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:19.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.960 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:19.960 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.267673+0000", 2024-09-14T23:57:19.960 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017956389999999999 2024-09-14T23:57:19.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:19.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.267677+0000", 2024-09-14T23:57:19.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4900000000000002e-06 2024-09-14T23:57:19.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:19.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.267688+0000", 2024-09-14T23:57:19.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1121000000000001e-05 2024-09-14T23:57:19.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:19.962 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.267694+0000", 2024-09-14T23:57:19.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5520000000000004e-06 2024-09-14T23:57:19.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.962 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:19.962 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.312534+0000", 2024-09-14T23:57:19.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044839641 2024-09-14T23:57:19.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.962 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:19.962 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.312586+0000", 2024-09-14T23:57:19.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2629e-05 2024-09-14T23:57:19.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.963 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.963 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:19.963 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.312636+0000", 2024-09-14T23:57:19.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0160000000000001e-05 2024-09-14T23:57:19.963 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:19.963 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:19.963 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:19.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.963 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.963 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68885 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 7946240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:19.963 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.273765+0000", 2024-09-14T23:57:19.963 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.6086969870000001, 2024-09-14T23:57:19.964 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.045854150000000003, 2024-09-14T23:57:19.964 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:19.964 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:19.964 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:19.964 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:19.964 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:19.964 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:19.964 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68885 2024-09-14T23:57:19.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.964 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:19.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.965 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:19.965 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273765+0000", 2024-09-14T23:57:19.965 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:19.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.965 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.965 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:19.965 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273765+0000", 2024-09-14T23:57:19.965 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:19.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.965 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.965 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:19.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273765+0000", 2024-09-14T23:57:19.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.84e-07 2024-09-14T23:57:19.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:19.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273767+0000", 2024-09-14T23:57:19.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3460000000000001e-06 2024-09-14T23:57:19.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:19.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273774+0000", 2024-09-14T23:57:19.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6420000000000001e-06 2024-09-14T23:57:19.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.967 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:19.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273779+0000", 2024-09-14T23:57:19.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.899e-06 2024-09-14T23:57:19.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.967 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:19.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.319535+0000", 2024-09-14T23:57:19.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.045755707999999999 2024-09-14T23:57:19.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.967 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:19.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.319577+0000", 2024-09-14T23:57:19.968 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2129999999999998e-05 2024-09-14T23:57:19.968 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.968 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.968 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:19.968 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.319619+0000", 2024-09-14T23:57:19.968 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2040999999999999e-05 2024-09-14T23:57:19.968 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:19.968 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:19.968 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:19.968 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.968 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.968 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68886 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 8060928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:19.968 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.273785+0000", 2024-09-14T23:57:19.969 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.608676193, 2024-09-14T23:57:19.969 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.045962955, 2024-09-14T23:57:19.969 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:19.969 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:19.969 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:19.969 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:19.969 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:19.969 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:19.969 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68886 2024-09-14T23:57:19.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.969 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:19.969 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.970 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:19.970 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273785+0000", 2024-09-14T23:57:19.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:19.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.970 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:19.970 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273785+0000", 2024-09-14T23:57:19.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:19.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.970 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:19.970 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273786+0000", 2024-09-14T23:57:19.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9000000000000002e-07 2024-09-14T23:57:19.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.971 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:19.971 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273788+0000", 2024-09-14T23:57:19.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9139999999999998e-06 2024-09-14T23:57:19.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.971 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:19.971 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273794+0000", 2024-09-14T23:57:19.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.212e-06 2024-09-14T23:57:19.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.971 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:19.971 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273799+0000", 2024-09-14T23:57:19.972 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5580000000000002e-06 2024-09-14T23:57:19.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.972 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.972 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:19.972 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.319646+0000", 2024-09-14T23:57:19.972 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.045847951999999997 2024-09-14T23:57:19.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.972 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.972 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:19.972 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.319707+0000", 2024-09-14T23:57:19.972 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.029e-05 2024-09-14T23:57:19.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.973 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:19.973 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.319748+0000", 2024-09-14T23:57:19.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1638999999999999e-05 2024-09-14T23:57:19.973 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:19.973 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:19.973 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:19.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.973 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68887 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 8175616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:19.973 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.273805+0000", 2024-09-14T23:57:19.974 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.6086569339999999, 2024-09-14T23:57:19.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046062421999999999, 2024-09-14T23:57:19.974 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:19.974 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:19.974 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:19.974 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:19.974 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:19.974 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:19.974 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68887 2024-09-14T23:57:19.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.974 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:19.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.975 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:19.975 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273805+0000", 2024-09-14T23:57:19.975 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:19.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.975 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.975 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:19.975 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273805+0000", 2024-09-14T23:57:19.975 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:19.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.975 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.975 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:19.975 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273805+0000", 2024-09-14T23:57:19.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0699999999999998e-07 2024-09-14T23:57:19.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.976 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:19.976 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273815+0000", 2024-09-14T23:57:19.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7720000000000006e-06 2024-09-14T23:57:19.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.976 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:19.976 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273822+0000", 2024-09-14T23:57:19.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5850000000000002e-06 2024-09-14T23:57:19.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.977 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:19.977 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273833+0000", 2024-09-14T23:57:19.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0541e-05 2024-09-14T23:57:19.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.977 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:19.977 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.319779+0000", 2024-09-14T23:57:19.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.045946263000000001 2024-09-14T23:57:19.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.977 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:19.978 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.319824+0000", 2024-09-14T23:57:19.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4740000000000002e-05 2024-09-14T23:57:19.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.978 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:19.978 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.319867+0000", 2024-09-14T23:57:19.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3114000000000001e-05 2024-09-14T23:57:19.978 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:19.978 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:19.978 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:19.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.979 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68888 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 8290304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:19.979 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.273854+0000", 2024-09-14T23:57:19.979 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.6086077940000001, 2024-09-14T23:57:19.979 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046120520999999998, 2024-09-14T23:57:19.979 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:19.979 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:19.979 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:19.979 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:19.979 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:19.979 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:19.979 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68888 2024-09-14T23:57:19.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.980 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:19.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.980 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:19.980 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273854+0000", 2024-09-14T23:57:19.980 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:19.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.980 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:19.980 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273854+0000", 2024-09-14T23:57:19.980 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:19.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.981 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:19.981 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273854+0000", 2024-09-14T23:57:19.981 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.58e-07 2024-09-14T23:57:19.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.981 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.981 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:19.981 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273862+0000", 2024-09-14T23:57:19.981 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.52e-06 2024-09-14T23:57:19.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.981 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.981 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:19.981 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273869+0000", 2024-09-14T23:57:19.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1180000000000003e-06 2024-09-14T23:57:19.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.982 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:19.982 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273883+0000", 2024-09-14T23:57:19.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3709999999999999e-05 2024-09-14T23:57:19.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.982 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:19.982 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.319889+0000", 2024-09-14T23:57:19.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046005870999999997 2024-09-14T23:57:19.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.983 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.983 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:19.983 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.319930+0000", 2024-09-14T23:57:19.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1850000000000001e-05 2024-09-14T23:57:19.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:19.983 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:19.983 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:19.983 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.319974+0000", 2024-09-14T23:57:19.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3993999999999999e-05 2024-09-14T23:57:19.983 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:19.983 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:19.984 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:19.984 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:19.984 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:19.984 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:19.986 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T23:57:19.989 INFO:teuthology.orchestra.run.smithi042.stdout:2260 writesame 0xe2e391d thru 0xe2eb79b data_size 0x2f(0x7e7f bytes) 2024-09-14T23:57:19.989 INFO:teuthology.orchestra.run.smithi042.stdout:2261 writesame 0x5b77c11 thru 0x5b871ff data_size 0x1d(0xf5ef bytes) 2024-09-14T23:57:19.989 INFO:teuthology.orchestra.run.smithi042.stdout:2262 read 0xe42ce4 thru 0xe4fc22 (0xcf3f bytes) 2024-09-14T23:57:19.989 INFO:teuthology.orchestra.run.smithi042.stdout:2264 punch from 0x7102ae1 to 0x710841f, (0x593e bytes) 2024-09-14T23:57:19.989 INFO:teuthology.orchestra.run.smithi042.stdout:2266 read 0xdfbc94a thru 0xdfbf159 (0x2810 bytes) 2024-09-14T23:57:19.989 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:19.989 INFO:teuthology.orchestra.run.smithi042.stdout:2268 read 0x2ae8253 thru 0x2af3e07 (0xbbb5 bytes) 2024-09-14T23:57:19.989 INFO:teuthology.orchestra.run.smithi042.stdout:2269 write 0x8cfcc88 thru 0x8d0bcd5 (0xf04e bytes) 2024-09-14T23:57:19.989 INFO:teuthology.orchestra.run.smithi042.stdout:2270 write 0x296dbf9 thru 0x2971711 (0x3b19 bytes) 2024-09-14T23:57:19.989 INFO:teuthology.orchestra.run.smithi042.stdout:2274 write 0xc97e42 thru 0xc98807 (0x9c6 bytes) 2024-09-14T23:57:19.989 INFO:teuthology.orchestra.run.smithi042.stdout:2275 writesame 0x2f20c4 thru 0x2ff257 data_size 0x11(0xd194 bytes) 2024-09-14T23:57:19.990 INFO:teuthology.orchestra.run.smithi042.stdout:2276 writesame 0x772caf7 thru 0x772f8d1 data_size 0x27(0x2ddb bytes) 2024-09-14T23:57:19.990 INFO:teuthology.orchestra.run.smithi042.stdout:2277 read 0x63d9c91 thru 0x63e2eb0 (0x9220 bytes) 2024-09-14T23:57:19.990 INFO:teuthology.orchestra.run.smithi042.stdout:2279 read 0xa707db5 thru 0xa71228e (0xa4da bytes) 2024-09-14T23:57:19.990 INFO:teuthology.orchestra.run.smithi042.stdout:2281 punch from 0xa8be6e4 to 0xa8c4962, (0x627e bytes) 2024-09-14T23:57:19.990 INFO:teuthology.orchestra.run.smithi042.stdout:2282 trunc from 0xe2eb79c to 0x3559c26 2024-09-14T23:57:19.990 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:19.990 INFO:teuthology.orchestra.run.smithi042.stdout:2285 read 0xbd725b thru 0xbdb644 (0x43ea bytes) 2024-09-14T23:57:19.990 INFO:teuthology.orchestra.run.smithi042.stdout:2286 punch from 0x210a647 to 0x2113223, (0x8bdc bytes) 2024-09-14T23:57:19.990 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:19.990 INFO:teuthology.orchestra.run.smithi042.stdout:2288 read 0x733762 thru 0x73f991 (0xc230 bytes) 2024-09-14T23:57:19.990 INFO:teuthology.orchestra.run.smithi042.stdout:2289 punch from 0x305eabe to 0x305f5ad, (0xaef bytes) 2024-09-14T23:57:19.990 INFO:teuthology.orchestra.run.smithi042.stdout:2290 writesame 0x9cf614e thru 0x9cf7047 data_size 0x1b(0xefa bytes) 2024-09-14T23:57:19.991 INFO:teuthology.orchestra.run.smithi042.stdout:2291 punch from 0x5aa375b to 0x5aad559, (0x9dfe bytes) 2024-09-14T23:57:19.991 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:19.991 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:19.991 INFO:teuthology.orchestra.run.smithi042.stdout:2296 read 0x35fd343 thru 0x360244f (0x510d bytes) 2024-09-14T23:57:19.991 INFO:teuthology.orchestra.run.smithi042.stdout:2297 write 0x7b53dcf thru 0x7b59d35 (0x5f67 bytes) 2024-09-14T23:57:19.991 INFO:teuthology.orchestra.run.smithi042.stdout:2298 trunc from 0x9cf7048 to 0xbbd9e78 2024-09-14T23:57:19.991 INFO:teuthology.orchestra.run.smithi042.stdout:2301 punch from 0x73d3cc1 to 0x73dd5a8, (0x98e7 bytes) 2024-09-14T23:57:19.991 INFO:teuthology.orchestra.run.smithi042.stdout:2304 writesame 0xd262781 thru 0xd2655ed data_size 0x1(0x2e6d bytes) 2024-09-14T23:57:19.991 INFO:teuthology.orchestra.run.smithi042.stdout:2307 write 0x5f404f8 thru 0x5f47e2d (0x7936 bytes) 2024-09-14T23:57:19.991 INFO:teuthology.orchestra.run.smithi042.stdout:2308 read 0xb0c10c5 thru 0xb0c6304 (0x5240 bytes) 2024-09-14T23:57:19.991 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:19.991 INFO:teuthology.orchestra.run.smithi042.stdout:2310 read 0xb9aa910 thru 0xb9b8f07 (0xe5f8 bytes) 2024-09-14T23:57:19.992 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:19.982+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:19.992 INFO:teuthology.orchestra.run.smithi042.stdout:2311 trunc from 0xd2655ee to 0x7d8470 2024-09-14T23:57:20.019 INFO:teuthology.orchestra.run.smithi042.stdout:2312 trunc from 0x7d8470 to 0xc9ddcd5 2024-09-14T23:57:20.032 INFO:teuthology.orchestra.run.smithi042.stdout:2314 writesame 0xea7a75b thru 0xea87cad data_size 0x1(0xd553 bytes) 2024-09-14T23:57:20.033 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:20.033 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:20.033 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:20.033 INFO:teuthology.orchestra.run.smithi042.stdout:2321 punch from 0x4f907ea to 0x4f998ce, (0x90e4 bytes) 2024-09-14T23:57:20.033 INFO:teuthology.orchestra.run.smithi042.stdout:2322 write 0x96b96e9 thru 0x96c505c (0xb974 bytes) 2024-09-14T23:57:20.036 INFO:teuthology.orchestra.run.smithi042.stdout:2324 writesame 0xd4bff4a thru 0xd4c6d19 data_size 0x1c(0x6dd0 bytes) 2024-09-14T23:57:20.039 INFO:teuthology.orchestra.run.smithi042.stdout:2325 punch from 0xd20ded8 to 0xd216a67, (0x8b8f bytes) 2024-09-14T23:57:20.040 INFO:teuthology.orchestra.run.smithi042.stdout:2327 write 0x96c62ac thru 0x96cdeb6 (0x7c0b bytes) 2024-09-14T23:57:20.041 INFO:teuthology.orchestra.run.smithi042.stdout:2328 read 0x4088c3c thru 0x408e8ff (0x5cc4 bytes) 2024-09-14T23:57:20.042 INFO:teuthology.orchestra.run.smithi042.stdout:2331 punch from 0xd00890e to 0xd009471, (0xb63 bytes) 2024-09-14T23:57:20.042 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:20.042 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:20.042 INFO:teuthology.orchestra.run.smithi042.stdout:2337 read 0x65da2ee thru 0x65e9e9b (0xfbae bytes) 2024-09-14T23:57:20.042 INFO:teuthology.orchestra.run.smithi042.stdout:2338 writesame 0x25acc9c thru 0x25af706 data_size 0x1(0x2a6b bytes) 2024-09-14T23:57:20.049 INFO:teuthology.orchestra.run.smithi042.stdout:2339 read 0xe2e86af thru 0xe2ebe63 (0x37b5 bytes) 2024-09-14T23:57:20.049 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:20.049 INFO:teuthology.orchestra.run.smithi042.stdout:2341 writesame 0x451d284 thru 0x451e91b data_size 0xc(0x1698 bytes) 2024-09-14T23:57:20.058 INFO:teuthology.orchestra.run.smithi042.stdout:2342 trunc from 0xea87cae to 0x9a61cd0 2024-09-14T23:57:20.068 INFO:teuthology.orchestra.run.smithi042.stdout:2343 read 0x74476a thru 0x747ec6 (0x375d bytes) 2024-09-14T23:57:20.068 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:20.069 INFO:teuthology.orchestra.run.smithi042.stdout:2345 trunc from 0x9a61cd0 to 0xa1730e9 2024-09-14T23:57:20.074 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:20.074 INFO:teuthology.orchestra.run.smithi042.stdout:2347 punch from 0x885010 to 0x886251, (0x1241 bytes) 2024-09-14T23:57:20.074 INFO:teuthology.orchestra.run.smithi042.stdout:2349 punch from 0x8f788d6 to 0x8f84c7c, (0xc3a6 bytes) 2024-09-14T23:57:20.074 INFO:teuthology.orchestra.run.smithi042.stdout:2350 trunc from 0xa1730e9 to 0x9e085c4 2024-09-14T23:57:20.085 INFO:teuthology.orchestra.run.smithi042.stdout:2353 write 0xa5b0f26 thru 0xa5b183c (0x917 bytes) 2024-09-14T23:57:20.085 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:20.083+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:20.088 INFO:teuthology.orchestra.run.smithi042.stdout:2354 write 0x9e3d92b thru 0x9e4d4c8 (0xfb9e bytes) 2024-09-14T23:57:20.092 INFO:teuthology.orchestra.run.smithi042.stdout:2356 punch from 0x1aeea49 to 0x1af92ea, (0xa8a1 bytes) 2024-09-14T23:57:20.092 INFO:teuthology.orchestra.run.smithi042.stdout:2357 punch from 0x387894e to 0x388874d, (0xfdff bytes) 2024-09-14T23:57:20.092 INFO:teuthology.orchestra.run.smithi042.stdout:2358 flatten 2024-09-14T23:57:20.123 INFO:teuthology.orchestra.run.smithi042.stdout:2359 trunc from 0xa5b183d to 0x30b49b7 2024-09-14T23:57:20.142 INFO:teuthology.orchestra.run.smithi042.stdout:2360 trunc from 0x30b49b7 to 0xac04951 2024-09-14T23:57:20.144 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.6 to 1 (10000) 2024-09-14T23:57:20.146 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:20.146 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:20.146 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:57:20.146 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:20.150 INFO:teuthology.orchestra.run.smithi042.stdout:2363 read 0x2c22ecf thru 0x2c29bc6 (0x6cf8 bytes) 2024-09-14T23:57:20.156 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T23:57:20.159 INFO:teuthology.orchestra.run.smithi042.stdout:2364 write 0xc7473cf thru 0xc74bab7 (0x46e9 bytes) 2024-09-14T23:57:20.160 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-14T23:57:20.164 INFO:teuthology.orchestra.run.smithi042.stdout:2366 punch from 0x451746e to 0x451d222, (0x5db4 bytes) 2024-09-14T23:57:20.164 INFO:teuthology.orchestra.run.smithi042.stdout:2367 writesame 0x6a32cd2 thru 0x6a3998b data_size 0x3(0x6cba bytes) 2024-09-14T23:57:20.175 INFO:teuthology.orchestra.run.smithi042.stdout:2369 writesame 0xcab5e24 thru 0xcab808e data_size 0x21(0x226b bytes) 2024-09-14T23:57:20.176 INFO:teuthology.orchestra.run.smithi042.stdout:2371 write 0x9fb897c thru 0x9fbae4d (0x24d2 bytes) 2024-09-14T23:57:20.178 INFO:teuthology.orchestra.run.smithi042.stdout:2372 read 0x4006975 thru 0x400f451 (0x8add bytes) 2024-09-14T23:57:20.186 INFO:teuthology.orchestra.run.smithi042.stdout:2374 trunc from 0xcab808f to 0x63b008e 2024-09-14T23:57:20.187 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:20.184+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:20.197 INFO:teuthology.orchestra.run.smithi042.stdout:2375 read 0x2bc441a thru 0x2bc7b07 (0x36ee bytes) 2024-09-14T23:57:20.197 INFO:teuthology.orchestra.run.smithi042.stdout:2378 read 0x27b4631 thru 0x27c016a (0xbb3a bytes) 2024-09-14T23:57:20.197 INFO:teuthology.orchestra.run.smithi042.stdout:2379 write 0x189b791 thru 0x18a18fc (0x616c bytes) 2024-09-14T23:57:20.201 INFO:teuthology.orchestra.run.smithi042.stdout:2380 write 0x4d58619 thru 0x4d60163 (0x7b4b bytes) 2024-09-14T23:57:20.205 INFO:teuthology.orchestra.run.smithi042.stdout:2381 punch from 0x40c82ed to 0x40cd2be, (0x4fd1 bytes) 2024-09-14T23:57:20.208 INFO:teuthology.orchestra.run.smithi042.stdout:2383 trunc from 0x63b008e to 0x942e238 2024-09-14T23:57:20.213 INFO:teuthology.orchestra.run.smithi042.stdout:2384 write 0x91d9423 thru 0x91db63e (0x221c bytes) 2024-09-14T23:57:20.217 INFO:teuthology.orchestra.run.smithi042.stdout:2386 read 0x37e4ef0 thru 0x37e9452 (0x4563 bytes) 2024-09-14T23:57:20.217 INFO:teuthology.orchestra.run.smithi042.stdout:2387 read 0x560645c thru 0x56133a4 (0xcf49 bytes) 2024-09-14T23:57:20.217 INFO:teuthology.orchestra.run.smithi042.stdout:2388 write 0x75f090d thru 0x75ffea2 (0xf596 bytes) 2024-09-14T23:57:20.220 INFO:teuthology.orchestra.run.smithi042.stdout:2389 write 0x503f2cc thru 0x504cb02 (0xd837 bytes) 2024-09-14T23:57:20.224 INFO:teuthology.orchestra.run.smithi042.stdout:2390 read 0x3c55aa7 thru 0x3c5d2b9 (0x7813 bytes) 2024-09-14T23:57:20.228 INFO:teuthology.orchestra.run.smithi042.stdout:2391 trunc from 0x942e238 to 0xd23728a 2024-09-14T23:57:20.233 INFO:teuthology.orchestra.run.smithi042.stdout:2392 write 0x626025c thru 0x6262ff1 (0x2d96 bytes) 2024-09-14T23:57:20.237 INFO:teuthology.orchestra.run.smithi042.stdout:2394 read 0x86785f4 thru 0x867f096 (0x6aa3 bytes) 2024-09-14T23:57:20.237 INFO:teuthology.orchestra.run.smithi042.stdout:2396 read 0x8b06025 thru 0x8b11a49 (0xba25 bytes) 2024-09-14T23:57:20.237 INFO:teuthology.orchestra.run.smithi042.stdout:2397 write 0xc134221 thru 0xc14187d (0xd65d bytes) 2024-09-14T23:57:20.240 INFO:teuthology.orchestra.run.smithi042.stdout:2398 read 0x3ed6e5 thru 0x3ee744 (0x1060 bytes) 2024-09-14T23:57:20.241 INFO:teuthology.orchestra.run.smithi042.stdout:2399 read 0x4726a78 thru 0x472c6dd (0x5c66 bytes) 2024-09-14T23:57:20.241 INFO:teuthology.orchestra.run.smithi042.stdout:2400 read 0x50d72c0 thru 0x50e1bcf (0xa910 bytes) 2024-09-14T23:57:20.241 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:20.241 INFO:teuthology.orchestra.run.smithi042.stdout:2402 read 0xcea9a60 thru 0xceb4658 (0xabf9 bytes) 2024-09-14T23:57:20.242 INFO:teuthology.orchestra.run.smithi042.stdout:2403 write 0x3a557fb thru 0x3a58826 (0x302c bytes) 2024-09-14T23:57:20.248 INFO:teuthology.orchestra.run.smithi042.stdout:2405 trunc from 0xd23728a to 0xa8b863c 2024-09-14T23:57:20.258 INFO:teuthology.orchestra.run.smithi042.stdout:2407 punch from 0x19cd352 to 0x19cf634, (0x22e2 bytes) 2024-09-14T23:57:20.258 INFO:teuthology.orchestra.run.smithi042.stdout:2408 punch from 0x6ab9ee0 to 0x6ac94b1, (0xf5d1 bytes) 2024-09-14T23:57:20.259 INFO:teuthology.orchestra.run.smithi042.stdout:2409 write 0x13501e1 thru 0x1356c14 (0x6a34 bytes) 2024-09-14T23:57:20.266 INFO:teuthology.orchestra.run.smithi042.stdout:2410 trunc from 0xa8b863c to 0x6e06826 2024-09-14T23:57:20.279 INFO:teuthology.orchestra.run.smithi042.stdout:2411 trunc from 0x6e06826 to 0x3ce9329 2024-09-14T23:57:20.280 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:20.281 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:20.281 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:57:20.281 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:57:20.281 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:20.288 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:20.287+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:20.291 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T23:57:20.307 INFO:teuthology.orchestra.run.smithi042.stdout:2413 write 0xe89bfa5 thru 0xe8ab1da (0xf236 bytes) 2024-09-14T23:57:20.311 INFO:teuthology.orchestra.run.smithi042.stdout:2415 punch from 0x212f9c4 to 0x2133fce, (0x460a bytes) 2024-09-14T23:57:20.311 INFO:teuthology.orchestra.run.smithi042.stdout:2416 read 0x2c9aabf thru 0x2caa000 (0xf542 bytes) 2024-09-14T23:57:20.311 INFO:teuthology.orchestra.run.smithi042.stdout:2417 punch from 0x333d3cb to 0x3347c3e, (0xa873 bytes) 2024-09-14T23:57:20.311 INFO:teuthology.orchestra.run.smithi042.stdout:2420 write 0x4f5f2d9 thru 0x4f6306b (0x3d93 bytes) 2024-09-14T23:57:20.315 INFO:teuthology.orchestra.run.smithi042.stdout:2421 read 0xd5fb588 thru 0xd5ff30b (0x3d84 bytes) 2024-09-14T23:57:20.315 INFO:teuthology.orchestra.run.smithi042.stdout:2423 write 0x98afcf9 thru 0x98b2b9e (0x2ea6 bytes) 2024-09-14T23:57:20.319 INFO:teuthology.orchestra.run.smithi042.stdout:2424 clone 25 order 21 su 524288 sc 15 2024-09-14T23:57:20.389 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:20.388+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:20.441 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:20.442 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:57:20.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:57:20.442 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:57:20.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.442 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69262 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 4800512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:20.442 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300811+0000", 2024-09-14T23:57:20.442 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.1383861140000002, 2024-09-14T23:57:20.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050403796000000001, 2024-09-14T23:57:20.442 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:20.442 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:20.442 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:20.442 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:20.442 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:20.443 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:20.443 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69262 2024-09-14T23:57:20.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.443 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:20.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.443 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:20.443 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300811+0000", 2024-09-14T23:57:20.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:20.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.443 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:20.443 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300811+0000", 2024-09-14T23:57:20.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:20.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:20.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300811+0000", 2024-09-14T23:57:20.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5700000000000002e-07 2024-09-14T23:57:20.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:20.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300815+0000", 2024-09-14T23:57:20.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.087e-06 2024-09-14T23:57:20.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:20.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300823+0000", 2024-09-14T23:57:20.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8720000000000002e-06 2024-09-14T23:57:20.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:20.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300830+0000", 2024-09-14T23:57:20.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0589999999999997e-06 2024-09-14T23:57:20.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:20.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351020+0000", 2024-09-14T23:57:20.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050189552999999998 2024-09-14T23:57:20.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:20.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351112+0000", 2024-09-14T23:57:20.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2615999999999996e-05 2024-09-14T23:57:20.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.446 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:20.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351214+0000", 2024-09-14T23:57:20.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101952 2024-09-14T23:57:20.446 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:20.446 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:20.446 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:20.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.447 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69263 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 4915200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:20.447 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300842+0000", 2024-09-14T23:57:20.447 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.1383546100000004, 2024-09-14T23:57:20.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050590637000000001, 2024-09-14T23:57:20.447 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:20.447 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:20.447 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:20.447 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:20.447 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:20.447 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:20.447 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69263 2024-09-14T23:57:20.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.447 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:20.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:20.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300842+0000", 2024-09-14T23:57:20.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:20.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:20.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300842+0000", 2024-09-14T23:57:20.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:20.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:20.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300843+0000", 2024-09-14T23:57:20.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4000000000000002e-07 2024-09-14T23:57:20.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:20.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300846+0000", 2024-09-14T23:57:20.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8720000000000002e-06 2024-09-14T23:57:20.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:20.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300853+0000", 2024-09-14T23:57:20.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0450000000000003e-06 2024-09-14T23:57:20.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.450 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:20.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300859+0000", 2024-09-14T23:57:20.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9510000000000003e-06 2024-09-14T23:57:20.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.450 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:20.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351260+0000", 2024-09-14T23:57:20.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050401654999999997 2024-09-14T23:57:20.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.450 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:20.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351353+0000", 2024-09-14T23:57:20.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2785e-05 2024-09-14T23:57:20.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:20.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351433+0000", 2024-09-14T23:57:20.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9789000000000005e-05 2024-09-14T23:57:20.451 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:20.451 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:20.451 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:20.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.451 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69264 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5029888~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:20.451 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300866+0000", 2024-09-14T23:57:20.451 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.1383306769999999, 2024-09-14T23:57:20.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050794063, 2024-09-14T23:57:20.451 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:20.451 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:20.452 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:20.452 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:20.452 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:20.452 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:20.452 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69264 2024-09-14T23:57:20.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.452 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:20.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:20.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300866+0000", 2024-09-14T23:57:20.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:20.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:20.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300866+0000", 2024-09-14T23:57:20.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:20.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:20.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300866+0000", 2024-09-14T23:57:20.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8799999999999998e-07 2024-09-14T23:57:20.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:20.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300869+0000", 2024-09-14T23:57:20.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0880000000000002e-06 2024-09-14T23:57:20.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:20.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300875+0000", 2024-09-14T23:57:20.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9280000000000004e-06 2024-09-14T23:57:20.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:20.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300882+0000", 2024-09-14T23:57:20.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4740000000000002e-06 2024-09-14T23:57:20.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:20.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351472+0000", 2024-09-14T23:57:20.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050590006999999999 2024-09-14T23:57:20.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:20.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351563+0000", 2024-09-14T23:57:20.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0762999999999998e-05 2024-09-14T23:57:20.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:20.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351660+0000", 2024-09-14T23:57:20.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7415000000000002e-05 2024-09-14T23:57:20.455 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:20.455 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:20.456 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:20.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.456 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69265 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5144576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:20.456 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300890+0000", 2024-09-14T23:57:20.456 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.1383071730000003, 2024-09-14T23:57:20.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050921389999999997, 2024-09-14T23:57:20.456 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:20.456 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:20.456 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:20.456 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:20.456 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:20.456 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:20.456 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69265 2024-09-14T23:57:20.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.457 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:20.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:20.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300890+0000", 2024-09-14T23:57:20.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:20.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:20.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300890+0000", 2024-09-14T23:57:20.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:20.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:20.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300890+0000", 2024-09-14T23:57:20.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2500000000000001e-07 2024-09-14T23:57:20.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:20.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300892+0000", 2024-09-14T23:57:20.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2469999999999998e-06 2024-09-14T23:57:20.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:20.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300899+0000", 2024-09-14T23:57:20.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5509999999999996e-06 2024-09-14T23:57:20.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:20.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300904+0000", 2024-09-14T23:57:20.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3299999999999998e-06 2024-09-14T23:57:20.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:20.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351707+0000", 2024-09-14T23:57:20.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050802673999999999 2024-09-14T23:57:20.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:20.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351755+0000", 2024-09-14T23:57:20.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8362000000000002e-05 2024-09-14T23:57:20.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:20.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351811+0000", 2024-09-14T23:57:20.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5801000000000003e-05 2024-09-14T23:57:20.460 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:20.460 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:20.460 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:20.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.460 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69266 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5259264~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:20.460 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300911+0000", 2024-09-14T23:57:20.460 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.1382861000000002, 2024-09-14T23:57:20.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051125089999999998, 2024-09-14T23:57:20.461 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:20.461 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:20.461 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:20.461 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:20.461 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:20.461 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:20.461 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69266 2024-09-14T23:57:20.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.461 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:20.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:20.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300911+0000", 2024-09-14T23:57:20.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:20.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:20.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300911+0000", 2024-09-14T23:57:20.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:20.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:20.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300911+0000", 2024-09-14T23:57:20.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-14T23:57:20.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:20.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300913+0000", 2024-09-14T23:57:20.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9300000000000002e-06 2024-09-14T23:57:20.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:20.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300919+0000", 2024-09-14T23:57:20.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.489e-06 2024-09-14T23:57:20.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:20.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300925+0000", 2024-09-14T23:57:20.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1719999999999998e-06 2024-09-14T23:57:20.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:20.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351847+0000", 2024-09-14T23:57:20.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050922621000000001 2024-09-14T23:57:20.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:20.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351938+0000", 2024-09-14T23:57:20.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0830999999999998e-05 2024-09-14T23:57:20.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:20.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352036+0000", 2024-09-14T23:57:20.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7676999999999993e-05 2024-09-14T23:57:20.465 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:20.465 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:20.465 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:20.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.465 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69267 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5373952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:20.465 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300945+0000", 2024-09-14T23:57:20.465 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.1382513049999998, 2024-09-14T23:57:20.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051217934, 2024-09-14T23:57:20.465 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:20.465 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:20.465 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:20.465 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:20.465 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:20.466 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:20.466 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69267 2024-09-14T23:57:20.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.466 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:20.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:20.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300945+0000", 2024-09-14T23:57:20.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:20.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:20.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300945+0000", 2024-09-14T23:57:20.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:20.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:20.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300946+0000", 2024-09-14T23:57:20.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2600000000000002e-07 2024-09-14T23:57:20.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:20.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300948+0000", 2024-09-14T23:57:20.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4940000000000002e-06 2024-09-14T23:57:20.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:20.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300956+0000", 2024-09-14T23:57:20.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.622e-06 2024-09-14T23:57:20.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:20.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300962+0000", 2024-09-14T23:57:20.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5790000000000003e-06 2024-09-14T23:57:20.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:20.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352063+0000", 2024-09-14T23:57:20.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051101281999999998 2024-09-14T23:57:20.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:20.469 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352110+0000", 2024-09-14T23:57:20.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7401000000000002e-05 2024-09-14T23:57:20.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.469 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:20.469 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352163+0000", 2024-09-14T23:57:20.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3029999999999999e-05 2024-09-14T23:57:20.469 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:20.469 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:20.469 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:20.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.469 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69268 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5488640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:20.470 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300969+0000", 2024-09-14T23:57:20.470 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.1382276329999996, 2024-09-14T23:57:20.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051413420000000001, 2024-09-14T23:57:20.470 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:20.470 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:20.470 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:20.470 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:20.470 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:20.470 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:20.470 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69268 2024-09-14T23:57:20.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.470 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:20.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:20.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300969+0000", 2024-09-14T23:57:20.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:20.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:20.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300969+0000", 2024-09-14T23:57:20.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:20.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:20.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300970+0000", 2024-09-14T23:57:20.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7899999999999999e-07 2024-09-14T23:57:20.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:20.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300982+0000", 2024-09-14T23:57:20.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2741e-05 2024-09-14T23:57:20.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:20.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300990+0000", 2024-09-14T23:57:20.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8169999999999999e-06 2024-09-14T23:57:20.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:20.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300995+0000", 2024-09-14T23:57:20.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2240000000000003e-06 2024-09-14T23:57:20.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:20.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352200+0000", 2024-09-14T23:57:20.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051204223 2024-09-14T23:57:20.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:20.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352285+0000", 2024-09-14T23:57:20.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5519999999999994e-05 2024-09-14T23:57:20.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:20.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352383+0000", 2024-09-14T23:57:20.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7516000000000006e-05 2024-09-14T23:57:20.474 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:20.474 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:20.474 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:20.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.474 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69269 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5603328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:20.474 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.301003+0000", 2024-09-14T23:57:20.474 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.1381934510000002, 2024-09-14T23:57:20.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051497381000000002, 2024-09-14T23:57:20.474 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:20.474 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:20.474 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:20.474 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:20.474 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:20.475 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:20.475 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69269 2024-09-14T23:57:20.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.475 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:20.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:20.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.301003+0000", 2024-09-14T23:57:20.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:20.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:20.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.301003+0000", 2024-09-14T23:57:20.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:20.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:20.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302671+0000", 2024-09-14T23:57:20.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001667785 2024-09-14T23:57:20.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:20.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302676+0000", 2024-09-14T23:57:20.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6600000000000003e-06 2024-09-14T23:57:20.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:20.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302697+0000", 2024-09-14T23:57:20.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1288999999999999e-05 2024-09-14T23:57:20.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:20.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302716+0000", 2024-09-14T23:57:20.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9128999999999999e-05 2024-09-14T23:57:20.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:20.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352399+0000", 2024-09-14T23:57:20.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049682995000000001 2024-09-14T23:57:20.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:20.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352446+0000", 2024-09-14T23:57:20.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6341000000000002e-05 2024-09-14T23:57:20.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:20.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352501+0000", 2024-09-14T23:57:20.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5182000000000001e-05 2024-09-14T23:57:20.478 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:20.478 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:20.478 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:20.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.478 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69273 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6062080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:20.478 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302829+0000", 2024-09-14T23:57:20.479 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.1363678850000003, 2024-09-14T23:57:20.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050384311000000001, 2024-09-14T23:57:20.479 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:20.479 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:20.479 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:20.479 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:20.479 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:20.479 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:20.479 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69273 2024-09-14T23:57:20.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.479 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:20.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:20.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302829+0000", 2024-09-14T23:57:20.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:20.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:20.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302829+0000", 2024-09-14T23:57:20.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:20.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:20.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302829+0000", 2024-09-14T23:57:20.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7700000000000005e-07 2024-09-14T23:57:20.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:20.481 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302832+0000", 2024-09-14T23:57:20.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6970000000000002e-06 2024-09-14T23:57:20.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:20.481 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302840+0000", 2024-09-14T23:57:20.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5719999999999997e-06 2024-09-14T23:57:20.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:20.481 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302845+0000", 2024-09-14T23:57:20.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7939999999999999e-06 2024-09-14T23:57:20.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:20.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353111+0000", 2024-09-14T23:57:20.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050265588 2024-09-14T23:57:20.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:20.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353160+0000", 2024-09-14T23:57:20.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8859000000000001e-05 2024-09-14T23:57:20.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:20.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353213+0000", 2024-09-14T23:57:20.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3324e-05 2024-09-14T23:57:20.483 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:20.483 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:20.483 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:20.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.484 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69274 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6176768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:20.484 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302854+0000", 2024-09-14T23:57:20.484 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.1363427909999997, 2024-09-14T23:57:20.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050579526, 2024-09-14T23:57:20.484 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:20.484 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:20.484 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:20.484 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:20.484 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:20.484 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:20.484 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69274 2024-09-14T23:57:20.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.485 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:20.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:20.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302854+0000", 2024-09-14T23:57:20.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:20.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:20.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302854+0000", 2024-09-14T23:57:20.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:20.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:20.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302855+0000", 2024-09-14T23:57:20.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0200000000000001e-07 2024-09-14T23:57:20.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:20.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302857+0000", 2024-09-14T23:57:20.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.447e-06 2024-09-14T23:57:20.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:20.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302864+0000", 2024-09-14T23:57:20.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7730000000000002e-06 2024-09-14T23:57:20.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:20.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302869+0000", 2024-09-14T23:57:20.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3199999999999999e-06 2024-09-14T23:57:20.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:20.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353249+0000", 2024-09-14T23:57:20.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050379647999999999 2024-09-14T23:57:20.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:20.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353336+0000", 2024-09-14T23:57:20.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6767999999999993e-05 2024-09-14T23:57:20.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:20.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353433+0000", 2024-09-14T23:57:20.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7868000000000006e-05 2024-09-14T23:57:20.488 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:20.488 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:20.488 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:20.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.488 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69275 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6291456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:20.488 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302889+0000", 2024-09-14T23:57:20.488 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.1363081729999998, 2024-09-14T23:57:20.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050663118, 2024-09-14T23:57:20.489 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:20.489 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:20.489 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:20.489 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:20.489 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:20.489 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:20.489 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69275 2024-09-14T23:57:20.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.489 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:20.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:20.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302889+0000", 2024-09-14T23:57:20.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:20.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:20.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302889+0000", 2024-09-14T23:57:20.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:20.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:20.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302889+0000", 2024-09-14T23:57:20.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6399999999999998e-07 2024-09-14T23:57:20.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:20.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302891+0000", 2024-09-14T23:57:20.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.339e-06 2024-09-14T23:57:20.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:20.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302898+0000", 2024-09-14T23:57:20.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0380000000000002e-06 2024-09-14T23:57:20.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:20.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302904+0000", 2024-09-14T23:57:20.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2159999999999997e-06 2024-09-14T23:57:20.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:20.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353449+0000", 2024-09-14T23:57:20.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050545386999999997 2024-09-14T23:57:20.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:20.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353494+0000", 2024-09-14T23:57:20.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4592999999999998e-05 2024-09-14T23:57:20.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:20.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353552+0000", 2024-09-14T23:57:20.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8180999999999997e-05 2024-09-14T23:57:20.492 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:20.492 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:20.492 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:20.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.493 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69276 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6406144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:20.493 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302911+0000", 2024-09-14T23:57:20.493 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.1362857599999998, 2024-09-14T23:57:20.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050865753, 2024-09-14T23:57:20.493 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:20.493 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:20.493 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:20.493 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:20.493 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:20.494 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:20.494 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69276 2024-09-14T23:57:20.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.494 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:20.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:20.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302911+0000", 2024-09-14T23:57:20.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:20.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:20.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302911+0000", 2024-09-14T23:57:20.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:20.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:20.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302912+0000", 2024-09-14T23:57:20.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0800000000000004e-07 2024-09-14T23:57:20.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:20.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302914+0000", 2024-09-14T23:57:20.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9589999999999998e-06 2024-09-14T23:57:20.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:20.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302920+0000", 2024-09-14T23:57:20.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4579999999999998e-06 2024-09-14T23:57:20.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:20.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302925+0000", 2024-09-14T23:57:20.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9429999999999999e-06 2024-09-14T23:57:20.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:20.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353586+0000", 2024-09-14T23:57:20.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050660966000000002 2024-09-14T23:57:20.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:20.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353679+0000", 2024-09-14T23:57:20.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3163000000000002e-05 2024-09-14T23:57:20.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:20.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353777+0000", 2024-09-14T23:57:20.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7555999999999999e-05 2024-09-14T23:57:20.497 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:20.497 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:20.497 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:20.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.497 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69277 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6520832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:20.497 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302932+0000", 2024-09-14T23:57:20.498 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.1362646730000003, 2024-09-14T23:57:20.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050977674000000001, 2024-09-14T23:57:20.498 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:20.498 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:20.498 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:20.498 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:20.498 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:20.498 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:20.498 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69277 2024-09-14T23:57:20.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.498 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:20.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.498 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:20.498 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302932+0000", 2024-09-14T23:57:20.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:20.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.499 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:20.499 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302932+0000", 2024-09-14T23:57:20.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:20.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.499 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:20.499 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302932+0000", 2024-09-14T23:57:20.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3299999999999998e-07 2024-09-14T23:57:20.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.499 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:20.499 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302935+0000", 2024-09-14T23:57:20.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2139999999999999e-06 2024-09-14T23:57:20.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:20.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302941+0000", 2024-09-14T23:57:20.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5039999999999999e-06 2024-09-14T23:57:20.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:20.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302946+0000", 2024-09-14T23:57:20.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2079999999999999e-06 2024-09-14T23:57:20.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:20.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353805+0000", 2024-09-14T23:57:20.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050858588000000003 2024-09-14T23:57:20.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:20.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353849+0000", 2024-09-14T23:57:20.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4511000000000001e-05 2024-09-14T23:57:20.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:20.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353910+0000", 2024-09-14T23:57:20.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0316000000000003e-05 2024-09-14T23:57:20.501 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:20.501 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:20.502 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:20.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.502 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69278 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6635520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:20.502 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302953+0000", 2024-09-14T23:57:20.502 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.1362435450000001, 2024-09-14T23:57:20.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051174059000000001, 2024-09-14T23:57:20.502 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:20.502 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:20.502 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:20.502 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:20.502 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:20.502 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:20.502 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69278 2024-09-14T23:57:20.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.503 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:20.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:20.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302953+0000", 2024-09-14T23:57:20.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:20.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:20.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302953+0000", 2024-09-14T23:57:20.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:20.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:20.504 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302954+0000", 2024-09-14T23:57:20.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5100000000000001e-07 2024-09-14T23:57:20.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.504 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:20.504 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302964+0000", 2024-09-14T23:57:20.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0475000000000001e-05 2024-09-14T23:57:20.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.504 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:20.504 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302970+0000", 2024-09-14T23:57:20.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3879999999999997e-06 2024-09-14T23:57:20.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.504 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:20.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302975+0000", 2024-09-14T23:57:20.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8409999999999999e-06 2024-09-14T23:57:20.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:20.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353944+0000", 2024-09-14T23:57:20.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050968952999999997 2024-09-14T23:57:20.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:20.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354031+0000", 2024-09-14T23:57:20.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6537000000000002e-05 2024-09-14T23:57:20.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:20.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354127+0000", 2024-09-14T23:57:20.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6514000000000005e-05 2024-09-14T23:57:20.506 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:20.506 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:20.506 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:20.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.506 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69279 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6750208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:20.506 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302984+0000", 2024-09-14T23:57:20.506 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.1362131939999998, 2024-09-14T23:57:20.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051258845999999997, 2024-09-14T23:57:20.506 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:20.507 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:20.507 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:20.507 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:20.507 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:20.507 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:20.507 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69279 2024-09-14T23:57:20.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.507 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:20.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.507 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:20.507 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302984+0000", 2024-09-14T23:57:20.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:20.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:20.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302984+0000", 2024-09-14T23:57:20.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:20.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:20.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303025+0000", 2024-09-14T23:57:20.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1690999999999998e-05 2024-09-14T23:57:20.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:20.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303029+0000", 2024-09-14T23:57:20.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6559999999999998e-06 2024-09-14T23:57:20.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:20.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303037+0000", 2024-09-14T23:57:20.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7950000000000008e-06 2024-09-14T23:57:20.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:20.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303043+0000", 2024-09-14T23:57:20.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9669999999999998e-06 2024-09-14T23:57:20.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:20.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354142+0000", 2024-09-14T23:57:20.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051099072000000002 2024-09-14T23:57:20.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:20.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354183+0000", 2024-09-14T23:57:20.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0834000000000002e-05 2024-09-14T23:57:20.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:20.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354242+0000", 2024-09-14T23:57:20.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9830999999999997e-05 2024-09-14T23:57:20.510 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:20.511 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:20.511 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:20.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.511 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69280 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6864896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:20.511 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.303052+0000", 2024-09-14T23:57:20.511 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.1361446510000004, 2024-09-14T23:57:20.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051406463999999999, 2024-09-14T23:57:20.511 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:20.511 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:20.511 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:20.511 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:20.511 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:20.512 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:20.512 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69280 2024-09-14T23:57:20.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.512 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:20.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.512 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:20.512 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303052+0000", 2024-09-14T23:57:20.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:20.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.512 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:20.512 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303052+0000", 2024-09-14T23:57:20.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:20.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:20.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304673+0000", 2024-09-14T23:57:20.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001621389 2024-09-14T23:57:20.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:20.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304681+0000", 2024-09-14T23:57:20.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8439999999999998e-06 2024-09-14T23:57:20.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:20.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304691+0000", 2024-09-14T23:57:20.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8339999999999994e-06 2024-09-14T23:57:20.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:20.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304698+0000", 2024-09-14T23:57:20.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3629999999999999e-06 2024-09-14T23:57:20.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:20.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354278+0000", 2024-09-14T23:57:20.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049580618 2024-09-14T23:57:20.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:20.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354364+0000", 2024-09-14T23:57:20.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6059999999999999e-05 2024-09-14T23:57:20.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:20.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354459+0000", 2024-09-14T23:57:20.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4356000000000003e-05 2024-09-14T23:57:20.515 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:20.515 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:20.515 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:20.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.515 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69285 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 7438336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:20.516 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.304792+0000", 2024-09-14T23:57:20.516 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.1344052419999997, 2024-09-14T23:57:20.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050398874000000003, 2024-09-14T23:57:20.516 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:20.516 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:20.516 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:20.516 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:20.516 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:20.516 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:20.516 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69285 2024-09-14T23:57:20.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.516 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:20.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.516 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:20.517 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304792+0000", 2024-09-14T23:57:20.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:20.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:20.517 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304792+0000", 2024-09-14T23:57:20.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:20.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:20.517 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304792+0000", 2024-09-14T23:57:20.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2800000000000003e-07 2024-09-14T23:57:20.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:20.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304794+0000", 2024-09-14T23:57:20.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8700000000000001e-06 2024-09-14T23:57:20.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:20.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304800+0000", 2024-09-14T23:57:20.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0889999999999996e-06 2024-09-14T23:57:20.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:20.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304805+0000", 2024-09-14T23:57:20.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7060000000000003e-06 2024-09-14T23:57:20.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:20.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355108+0000", 2024-09-14T23:57:20.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050303170000000001 2024-09-14T23:57:20.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:20.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355150+0000", 2024-09-14T23:57:20.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2271999999999998e-05 2024-09-14T23:57:20.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:20.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355190+0000", 2024-09-14T23:57:20.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0438999999999997e-05 2024-09-14T23:57:20.519 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:20.519 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:20.520 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:20.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.520 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69286 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 7553024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:20.520 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.304811+0000", 2024-09-14T23:57:20.520 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.13438588, 2024-09-14T23:57:20.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050585512999999999, 2024-09-14T23:57:20.520 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:20.520 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:20.520 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:20.520 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:20.520 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:20.521 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:20.521 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69286 2024-09-14T23:57:20.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.521 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:20.521 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.521 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:20.521 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304811+0000", 2024-09-14T23:57:20.521 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:20.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.521 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.521 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:20.521 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304811+0000", 2024-09-14T23:57:20.521 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:20.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:20.522 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304811+0000", 2024-09-14T23:57:20.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.89e-07 2024-09-14T23:57:20.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:20.522 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304814+0000", 2024-09-14T23:57:20.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3199999999999998e-06 2024-09-14T23:57:20.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:20.522 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304821+0000", 2024-09-14T23:57:20.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5689999999999999e-06 2024-09-14T23:57:20.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:20.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304826+0000", 2024-09-14T23:57:20.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1200000000000001e-06 2024-09-14T23:57:20.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:20.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355226+0000", 2024-09-14T23:57:20.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050399653000000003 2024-09-14T23:57:20.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:20.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355314+0000", 2024-09-14T23:57:20.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8389999999999999e-05 2024-09-14T23:57:20.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:20.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355396+0000", 2024-09-14T23:57:20.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2071999999999996e-05 2024-09-14T23:57:20.524 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:20.524 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:20.524 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:20.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.524 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69287 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 7667712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:20.525 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.304833+0000", 2024-09-14T23:57:20.525 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.1343632819999998, 2024-09-14T23:57:20.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050664740999999999, 2024-09-14T23:57:20.525 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:20.525 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:20.525 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:20.525 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:20.525 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:20.525 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:20.525 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69287 2024-09-14T23:57:20.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.525 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:20.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.526 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:20.526 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304833+0000", 2024-09-14T23:57:20.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:20.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.526 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:20.526 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304833+0000", 2024-09-14T23:57:20.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:20.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.526 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:20.526 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304834+0000", 2024-09-14T23:57:20.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4999999999999998e-07 2024-09-14T23:57:20.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:20.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304836+0000", 2024-09-14T23:57:20.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0279999999999999e-06 2024-09-14T23:57:20.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:20.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304842+0000", 2024-09-14T23:57:20.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8919999999999997e-06 2024-09-14T23:57:20.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:20.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304847+0000", 2024-09-14T23:57:20.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6160000000000001e-06 2024-09-14T23:57:20.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:20.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355411+0000", 2024-09-14T23:57:20.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050563540999999997 2024-09-14T23:57:20.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:20.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355459+0000", 2024-09-14T23:57:20.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7979000000000003e-05 2024-09-14T23:57:20.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:20.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355498+0000", 2024-09-14T23:57:20.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9335000000000002e-05 2024-09-14T23:57:20.529 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:20.529 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:20.529 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:20.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.529 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69288 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 7782400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:20.529 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.304854+0000", 2024-09-14T23:57:20.529 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.1343428439999999, 2024-09-14T23:57:20.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050855145999999997, 2024-09-14T23:57:20.529 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:20.529 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:20.529 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:20.529 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:20.529 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:20.530 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:20.530 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69288 2024-09-14T23:57:20.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.530 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:20.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.530 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:20.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304854+0000", 2024-09-14T23:57:20.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:20.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.530 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:20.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304854+0000", 2024-09-14T23:57:20.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:20.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:20.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.306670+0000", 2024-09-14T23:57:20.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00181631 2024-09-14T23:57:20.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:20.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.306675+0000", 2024-09-14T23:57:20.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0220000000000004e-06 2024-09-14T23:57:20.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:20.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.306684+0000", 2024-09-14T23:57:20.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7309999999999999e-06 2024-09-14T23:57:20.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:20.532 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.306689+0000", 2024-09-14T23:57:20.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6609999999999999e-06 2024-09-14T23:57:20.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:20.532 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355533+0000", 2024-09-14T23:57:20.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048844578 2024-09-14T23:57:20.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:20.532 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355622+0000", 2024-09-14T23:57:20.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9006999999999997e-05 2024-09-14T23:57:20.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:20.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:20.533 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:20.533 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355709+0000", 2024-09-14T23:57:20.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6836999999999995e-05 2024-09-14T23:57:20.533 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:20.533 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:20.533 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:20.533 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:20.533 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:20.533 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:20.535 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:20.533+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:20.535 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T23:57:20.635 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:20.634+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:20.643 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:20.643 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:57:20.643 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:57:20.643 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:20.651 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T23:57:20.736 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:20.735+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:20.758 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:20.758 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:57:20.758 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:57:20.758 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:57:20.758 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:20.767 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T23:57:20.837 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:20.836+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:20.875 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:20.876 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:57:20.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:57:20.876 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:57:20.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.876 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:43054 3.1f 3:ffbfe596:::rbd_data.1118350d5ae5.00000000000002eb:head [set-alloc-hint object_size 262144 write_size 262144,write 196608~22770] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T23:57:20.876 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:57:20.876 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.906903455999998, 2024-09-14T23:57:20.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.99973334400000002, 2024-09-14T23:57:20.876 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:20.876 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:20.876 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:20.877 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:20.877 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:20.877 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:20.877 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 43054 2024-09-14T23:57:20.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.877 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:20.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:20.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:57:20.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:20.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:20.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:57:20.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:20.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:20.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966552+0000", 2024-09-14T23:57:20.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3790000000000001e-06 2024-09-14T23:57:20.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:20.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966565+0000", 2024-09-14T23:57:20.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2459999999999999e-05 2024-09-14T23:57:20.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:20.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966576+0000", 2024-09-14T23:57:20.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1039000000000001e-05 2024-09-14T23:57:20.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:20.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966582+0000", 2024-09-14T23:57:20.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9660000000000001e-06 2024-09-14T23:57:20.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:20.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966625+0000", 2024-09-14T23:57:20.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3529999999999998e-05 2024-09-14T23:57:20.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.880 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:57:20.880 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966632+0000", 2024-09-14T23:57:20.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9809999999999997e-06 2024-09-14T23:57:20.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.880 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:20.880 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.964460+0000", 2024-09-14T23:57:20.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.99782768300000002 2024-09-14T23:57:20.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.880 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:20.880 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.964641+0000", 2024-09-14T23:57:20.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000181089 2024-09-14T23:57:20.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-14T23:57:20.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.964809+0000", 2024-09-14T23:57:20.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000168505 2024-09-14T23:57:20.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:57:20.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966030+0000", 2024-09-14T23:57:20.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012206929999999999 2024-09-14T23:57:20.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:20.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966181+0000", 2024-09-14T23:57:20.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000150569 2024-09-14T23:57:20.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.882 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:20.882 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966243+0000", 2024-09-14T23:57:20.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2331999999999999e-05 2024-09-14T23:57:20.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.882 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:20.882 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966284+0000", 2024-09-14T23:57:20.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1118000000000002e-05 2024-09-14T23:57:20.882 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:20.882 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:20.882 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:20.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.882 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:54673 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:20.883 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.167989+0000", 2024-09-14T23:57:20.883 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.705465106, 2024-09-14T23:57:20.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027580298, 2024-09-14T23:57:20.883 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:20.883 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:20.883 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:20.883 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:20.883 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:20.883 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:20.883 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54673 2024-09-14T23:57:20.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.883 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:20.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.883 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:20.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.167989+0000", 2024-09-14T23:57:20.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:20.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:20.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.167989+0000", 2024-09-14T23:57:20.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:20.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:20.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.167989+0000", 2024-09-14T23:57:20.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5499999999999998e-07 2024-09-14T23:57:20.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.885 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:20.885 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.167998+0000", 2024-09-14T23:57:20.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7369999999999995e-06 2024-09-14T23:57:20.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.885 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:20.885 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168005+0000", 2024-09-14T23:57:20.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8210000000000004e-06 2024-09-14T23:57:20.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.885 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:20.885 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168010+0000", 2024-09-14T23:57:20.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2510000000000002e-06 2024-09-14T23:57:20.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:20.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.195432+0000", 2024-09-14T23:57:20.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027421364 2024-09-14T23:57:20.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:20.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.195501+0000", 2024-09-14T23:57:20.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9611999999999997e-05 2024-09-14T23:57:20.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:20.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.195569+0000", 2024-09-14T23:57:20.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8058000000000005e-05 2024-09-14T23:57:20.887 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:20.887 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:20.887 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:20.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.887 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:54674 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 2637824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:20.887 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.168018+0000", 2024-09-14T23:57:20.887 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.705436462, 2024-09-14T23:57:20.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027716159000000001, 2024-09-14T23:57:20.887 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:20.887 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:20.887 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:20.888 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:20.888 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:20.888 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:20.888 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54674 2024-09-14T23:57:20.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.888 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:20.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.888 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:20.888 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168018+0000", 2024-09-14T23:57:20.888 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:20.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.888 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:20.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168018+0000", 2024-09-14T23:57:20.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:20.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:20.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168018+0000", 2024-09-14T23:57:20.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2600000000000002e-07 2024-09-14T23:57:20.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:20.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168020+0000", 2024-09-14T23:57:20.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9420000000000002e-06 2024-09-14T23:57:20.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:20.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168026+0000", 2024-09-14T23:57:20.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8880000000000002e-06 2024-09-14T23:57:20.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:20.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168031+0000", 2024-09-14T23:57:20.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.977e-06 2024-09-14T23:57:20.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:20.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.195599+0000", 2024-09-14T23:57:20.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027567758000000001 2024-09-14T23:57:20.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:20.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.195666+0000", 2024-09-14T23:57:20.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7102999999999998e-05 2024-09-14T23:57:20.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:20.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.195734+0000", 2024-09-14T23:57:20.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7865000000000002e-05 2024-09-14T23:57:20.891 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:20.891 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:20.891 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:20.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.892 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:54675 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:20.892 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.168039+0000", 2024-09-14T23:57:20.892 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.705415502999999, 2024-09-14T23:57:20.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027868378999999999, 2024-09-14T23:57:20.892 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:20.892 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:20.892 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:20.892 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:20.892 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:20.892 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:20.892 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54675 2024-09-14T23:57:20.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.893 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:20.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:20.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168039+0000", 2024-09-14T23:57:20.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:20.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:20.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168039+0000", 2024-09-14T23:57:20.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:20.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:20.894 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168039+0000", 2024-09-14T23:57:20.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3099999999999998e-07 2024-09-14T23:57:20.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.894 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:20.894 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168048+0000", 2024-09-14T23:57:20.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9460000000000004e-06 2024-09-14T23:57:20.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.894 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:20.894 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168055+0000", 2024-09-14T23:57:20.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5740000000000001e-06 2024-09-14T23:57:20.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:20.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168060+0000", 2024-09-14T23:57:20.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1909999999999999e-06 2024-09-14T23:57:20.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:20.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.195765+0000", 2024-09-14T23:57:20.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027705244 2024-09-14T23:57:20.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:20.896 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.195836+0000", 2024-09-14T23:57:20.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0642999999999994e-05 2024-09-14T23:57:20.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.896 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:20.896 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.195907+0000", 2024-09-14T23:57:20.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.135e-05 2024-09-14T23:57:20.896 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:20.896 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:20.896 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:20.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.897 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:54676 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 2998272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:20.897 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.168067+0000", 2024-09-14T23:57:20.897 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.70538677, 2024-09-14T23:57:20.897 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028002955999999999, 2024-09-14T23:57:20.897 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:20.897 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:20.897 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:20.897 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:20.897 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:20.897 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:20.897 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54676 2024-09-14T23:57:20.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.897 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:20.898 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.898 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:20.898 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168067+0000", 2024-09-14T23:57:20.898 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:20.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.898 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.898 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:20.898 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168067+0000", 2024-09-14T23:57:20.898 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:20.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.898 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.898 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:20.898 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168068+0000", 2024-09-14T23:57:20.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6899999999999998e-07 2024-09-14T23:57:20.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.899 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:20.899 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168070+0000", 2024-09-14T23:57:20.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9810000000000002e-06 2024-09-14T23:57:20.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.899 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:20.899 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168076+0000", 2024-09-14T23:57:20.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4280000000000001e-06 2024-09-14T23:57:20.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.899 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:20.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168091+0000", 2024-09-14T23:57:20.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5152e-05 2024-09-14T23:57:20.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:20.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.195936+0000", 2024-09-14T23:57:20.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027844892 2024-09-14T23:57:20.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:20.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196002+0000", 2024-09-14T23:57:20.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5633e-05 2024-09-14T23:57:20.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:20.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196070+0000", 2024-09-14T23:57:20.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8400999999999998e-05 2024-09-14T23:57:20.901 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:20.901 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:20.901 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:20.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.901 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:54677 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 3178496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:20.901 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.168099+0000", 2024-09-14T23:57:20.901 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.705355471000001, 2024-09-14T23:57:20.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028165816, 2024-09-14T23:57:20.901 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:20.901 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:20.902 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:20.902 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:20.902 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:20.902 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:20.902 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54677 2024-09-14T23:57:20.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.902 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:20.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.902 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:20.902 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168099+0000", 2024-09-14T23:57:20.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:20.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.902 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:20.903 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168099+0000", 2024-09-14T23:57:20.903 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:20.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.903 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.903 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:20.903 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168099+0000", 2024-09-14T23:57:20.903 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7400000000000003e-07 2024-09-14T23:57:20.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.903 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.903 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:20.903 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168102+0000", 2024-09-14T23:57:20.903 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2929999999999999e-06 2024-09-14T23:57:20.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.903 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.904 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:20.904 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168108+0000", 2024-09-14T23:57:20.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5429999999999999e-06 2024-09-14T23:57:20.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.904 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:20.904 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168113+0000", 2024-09-14T23:57:20.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8470000000000003e-06 2024-09-14T23:57:20.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.904 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:20.904 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196124+0000", 2024-09-14T23:57:20.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028011034000000001 2024-09-14T23:57:20.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.905 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:20.905 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196194+0000", 2024-09-14T23:57:20.905 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0314000000000005e-05 2024-09-14T23:57:20.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.905 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:20.905 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196264+0000", 2024-09-14T23:57:20.905 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0210999999999996e-05 2024-09-14T23:57:20.905 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:20.905 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:20.905 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:20.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.906 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:54678 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 3358720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:20.906 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.168144+0000", 2024-09-14T23:57:20.906 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.705310312, 2024-09-14T23:57:20.906 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028285267999999999, 2024-09-14T23:57:20.906 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:20.906 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:20.906 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:20.906 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:20.906 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:20.906 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:20.906 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54678 2024-09-14T23:57:20.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.906 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:20.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.907 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:20.907 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168144+0000", 2024-09-14T23:57:20.907 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:20.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.907 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:20.907 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168144+0000", 2024-09-14T23:57:20.907 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:20.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.907 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:20.907 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168144+0000", 2024-09-14T23:57:20.907 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5799999999999999e-07 2024-09-14T23:57:20.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.908 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:20.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168147+0000", 2024-09-14T23:57:20.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3580000000000001e-06 2024-09-14T23:57:20.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.908 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:20.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168154+0000", 2024-09-14T23:57:20.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1350000000000003e-06 2024-09-14T23:57:20.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.908 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:20.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168159+0000", 2024-09-14T23:57:20.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4160000000000003e-06 2024-09-14T23:57:20.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:20.909 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196294+0000", 2024-09-14T23:57:20.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028135050000000002 2024-09-14T23:57:20.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:20.909 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196363+0000", 2024-09-14T23:57:20.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8880000000000005e-05 2024-09-14T23:57:20.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:20.910 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196429+0000", 2024-09-14T23:57:20.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5870999999999993e-05 2024-09-14T23:57:20.910 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:20.910 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:20.910 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:20.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.910 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.910 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:54679 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:20.910 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.168167+0000", 2024-09-14T23:57:20.910 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.705287091000001, 2024-09-14T23:57:20.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028427019000000001, 2024-09-14T23:57:20.910 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:20.910 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:20.910 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:20.911 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:20.911 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:20.911 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:20.911 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54679 2024-09-14T23:57:20.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.911 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:20.911 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.911 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:20.911 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168167+0000", 2024-09-14T23:57:20.911 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:20.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.911 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.911 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:20.911 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168167+0000", 2024-09-14T23:57:20.912 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:20.912 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.912 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.912 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:20.912 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168168+0000", 2024-09-14T23:57:20.912 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.44e-07 2024-09-14T23:57:20.912 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.912 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.912 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:20.912 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168170+0000", 2024-09-14T23:57:20.912 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0870000000000001e-06 2024-09-14T23:57:20.912 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.912 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.913 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:20.913 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168176+0000", 2024-09-14T23:57:20.913 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9070000000000004e-06 2024-09-14T23:57:20.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.913 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:20.913 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168181+0000", 2024-09-14T23:57:20.913 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6579999999999999e-06 2024-09-14T23:57:20.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.913 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:20.913 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196458+0000", 2024-09-14T23:57:20.913 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028276431000000001 2024-09-14T23:57:20.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.914 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:20.914 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196527+0000", 2024-09-14T23:57:20.914 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9397e-05 2024-09-14T23:57:20.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.914 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.914 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:20.914 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196594+0000", 2024-09-14T23:57:20.914 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6895000000000002e-05 2024-09-14T23:57:20.914 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:20.914 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:20.914 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:20.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.914 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.914 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:54680 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 3719168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:20.914 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.168188+0000", 2024-09-14T23:57:20.915 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.705265884999999, 2024-09-14T23:57:20.915 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028579706999999999, 2024-09-14T23:57:20.915 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:20.915 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:20.915 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:20.915 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:20.915 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:20.915 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:20.915 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54680 2024-09-14T23:57:20.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.915 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:20.916 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.916 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:20.916 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168188+0000", 2024-09-14T23:57:20.916 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:20.916 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.916 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.916 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:20.916 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168188+0000", 2024-09-14T23:57:20.916 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:20.916 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.916 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.916 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:20.917 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168189+0000", 2024-09-14T23:57:20.917 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3099999999999998e-07 2024-09-14T23:57:20.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.917 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.917 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:20.917 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168191+0000", 2024-09-14T23:57:20.917 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0470000000000001e-06 2024-09-14T23:57:20.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.917 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.917 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:20.917 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168199+0000", 2024-09-14T23:57:20.917 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8809999999999996e-06 2024-09-14T23:57:20.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.918 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.918 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:20.918 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168204+0000", 2024-09-14T23:57:20.918 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.011e-06 2024-09-14T23:57:20.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.918 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.918 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:20.918 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196622+0000", 2024-09-14T23:57:20.918 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028418806000000001 2024-09-14T23:57:20.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.918 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.918 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:20.918 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196694+0000", 2024-09-14T23:57:20.919 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1124000000000005e-05 2024-09-14T23:57:20.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.919 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.919 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:20.919 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196768+0000", 2024-09-14T23:57:20.919 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4406999999999994e-05 2024-09-14T23:57:20.919 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:20.919 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:20.919 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:20.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.919 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.919 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:54681 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 3899392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:20.919 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.168212+0000", 2024-09-14T23:57:20.919 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.705242173, 2024-09-14T23:57:20.920 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028726643, 2024-09-14T23:57:20.920 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:20.920 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:20.920 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:20.920 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:20.920 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:20.920 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:20.920 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54681 2024-09-14T23:57:20.920 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.920 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:20.920 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.920 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:20.920 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168212+0000", 2024-09-14T23:57:20.921 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:20.921 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.921 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.921 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:20.921 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168212+0000", 2024-09-14T23:57:20.921 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:20.921 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.921 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.921 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:20.921 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168212+0000", 2024-09-14T23:57:20.921 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4900000000000001e-07 2024-09-14T23:57:20.921 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.921 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.921 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:20.922 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168214+0000", 2024-09-14T23:57:20.922 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0590000000000001e-06 2024-09-14T23:57:20.922 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.922 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.922 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:20.922 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168220+0000", 2024-09-14T23:57:20.922 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0209999999999996e-06 2024-09-14T23:57:20.922 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.922 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.922 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:20.922 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168227+0000", 2024-09-14T23:57:20.922 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4119999999999998e-06 2024-09-14T23:57:20.923 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.923 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.923 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:20.923 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196799+0000", 2024-09-14T23:57:20.923 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028572528 2024-09-14T23:57:20.923 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.923 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.923 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:20.923 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196866+0000", 2024-09-14T23:57:20.923 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7005e-05 2024-09-14T23:57:20.923 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.923 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.923 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:20.923 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196939+0000", 2024-09-14T23:57:20.924 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2169000000000006e-05 2024-09-14T23:57:20.924 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:20.924 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:20.924 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:20.924 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.924 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.924 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:54682 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 4079616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:20.924 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.168234+0000", 2024-09-14T23:57:20.924 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.705220624000001, 2024-09-14T23:57:20.924 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028872503000000001, 2024-09-14T23:57:20.924 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:20.924 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:20.924 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:20.925 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:20.925 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:20.925 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:20.925 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54682 2024-09-14T23:57:20.925 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.925 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:20.925 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.925 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:20.925 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168234+0000", 2024-09-14T23:57:20.925 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:20.925 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.925 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.925 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:20.926 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168234+0000", 2024-09-14T23:57:20.926 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:20.926 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.926 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.926 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:20.926 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168234+0000", 2024-09-14T23:57:20.926 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.58e-07 2024-09-14T23:57:20.926 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.926 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.926 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:20.926 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168236+0000", 2024-09-14T23:57:20.926 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.933e-06 2024-09-14T23:57:20.926 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.926 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.927 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:20.927 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168242+0000", 2024-09-14T23:57:20.927 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9970000000000004e-06 2024-09-14T23:57:20.927 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.927 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.927 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:20.927 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168247+0000", 2024-09-14T23:57:20.927 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9280000000000001e-06 2024-09-14T23:57:20.927 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.927 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.927 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:20.927 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196968+0000", 2024-09-14T23:57:20.927 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028721159999999999 2024-09-14T23:57:20.927 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.928 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.928 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:20.928 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.197040+0000", 2024-09-14T23:57:20.928 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1569999999999994e-05 2024-09-14T23:57:20.928 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.928 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.928 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:20.928 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.197106+0000", 2024-09-14T23:57:20.928 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6457000000000004e-05 2024-09-14T23:57:20.928 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:20.928 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:20.928 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:20.928 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.928 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.929 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:55239 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 3850240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:20.929 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.201093+0000", 2024-09-14T23:57:20.929 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.672361173000001, 2024-09-14T23:57:20.929 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027610927, 2024-09-14T23:57:20.929 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:20.929 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:20.929 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:20.929 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:20.929 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:20.929 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:20.929 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 55239 2024-09-14T23:57:20.929 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.929 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:20.930 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.930 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:20.930 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201093+0000", 2024-09-14T23:57:20.930 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:20.930 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.930 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.930 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:20.930 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201093+0000", 2024-09-14T23:57:20.930 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:20.930 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.930 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.930 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:20.930 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201093+0000", 2024-09-14T23:57:20.930 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5699999999999998e-07 2024-09-14T23:57:20.931 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.931 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.931 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:20.931 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201096+0000", 2024-09-14T23:57:20.931 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0990000000000001e-06 2024-09-14T23:57:20.931 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.931 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.931 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:20.931 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201102+0000", 2024-09-14T23:57:20.931 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5710000000000003e-06 2024-09-14T23:57:20.931 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.931 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.931 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:20.931 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201107+0000", 2024-09-14T23:57:20.931 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1019999999999996e-06 2024-09-14T23:57:20.932 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.932 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.932 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:20.932 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.228568+0000", 2024-09-14T23:57:20.932 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027461136000000001 2024-09-14T23:57:20.932 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.932 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.932 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:20.932 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.228638+0000", 2024-09-14T23:57:20.932 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0024999999999994e-05 2024-09-14T23:57:20.932 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.932 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.932 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:20.933 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.228704+0000", 2024-09-14T23:57:20.933 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5537000000000006e-05 2024-09-14T23:57:20.933 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:20.933 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:20.933 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:20.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.933 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.933 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:55240 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:20.933 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.201114+0000", 2024-09-14T23:57:20.933 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.672339937, 2024-09-14T23:57:20.933 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027752295, 2024-09-14T23:57:20.933 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:20.933 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:20.934 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:20.934 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:20.934 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:20.934 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:20.934 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 55240 2024-09-14T23:57:20.934 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.934 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:20.934 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.934 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:20.934 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201114+0000", 2024-09-14T23:57:20.934 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:20.934 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.934 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.934 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:20.935 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201114+0000", 2024-09-14T23:57:20.935 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:20.935 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.935 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.935 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:20.935 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201115+0000", 2024-09-14T23:57:20.935 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4200000000000001e-07 2024-09-14T23:57:20.935 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.935 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.935 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:20.935 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201117+0000", 2024-09-14T23:57:20.935 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.368e-06 2024-09-14T23:57:20.935 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.936 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.936 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:20.936 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201124+0000", 2024-09-14T23:57:20.936 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4999999999999996e-06 2024-09-14T23:57:20.936 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.936 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.936 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:20.936 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201136+0000", 2024-09-14T23:57:20.936 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.293e-05 2024-09-14T23:57:20.936 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.936 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.936 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:20.936 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.228736+0000", 2024-09-14T23:57:20.937 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027599597 2024-09-14T23:57:20.937 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.937 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.937 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:20.937 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.228806+0000", 2024-09-14T23:57:20.937 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9616999999999994e-05 2024-09-14T23:57:20.937 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.937 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.937 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:20.937 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.228866+0000", 2024-09-14T23:57:20.937 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0841000000000002e-05 2024-09-14T23:57:20.937 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:20.937 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:20.938 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:20.938 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.938 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.938 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:55349 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 868352~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:20.938 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.204621+0000", 2024-09-14T23:57:20.938 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.668832771, 2024-09-14T23:57:20.938 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030294596999999999, 2024-09-14T23:57:20.938 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:20.938 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:20.938 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:20.938 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:20.938 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:20.938 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:20.939 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 55349 2024-09-14T23:57:20.939 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.939 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:20.939 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.939 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:20.939 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204621+0000", 2024-09-14T23:57:20.939 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:20.939 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.939 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.939 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:20.939 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204621+0000", 2024-09-14T23:57:20.939 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:20.939 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.939 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.939 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:20.940 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204622+0000", 2024-09-14T23:57:20.940 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.58e-07 2024-09-14T23:57:20.940 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.940 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.940 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:20.940 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204624+0000", 2024-09-14T23:57:20.940 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.193e-06 2024-09-14T23:57:20.940 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.940 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.940 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:20.940 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204631+0000", 2024-09-14T23:57:20.940 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1910000000000003e-06 2024-09-14T23:57:20.940 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.940 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.941 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:20.941 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204637+0000", 2024-09-14T23:57:20.941 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5169999999999999e-06 2024-09-14T23:57:20.941 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.941 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.941 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:20.941 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.229653+0000", 2024-09-14T23:57:20.941 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025016283 2024-09-14T23:57:20.941 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.941 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.941 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:20.941 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.229721+0000", 2024-09-14T23:57:20.941 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8138000000000004e-05 2024-09-14T23:57:20.941 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.942 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.942 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:20.942 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.234916+0000", 2024-09-14T23:57:20.942 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.005194817 2024-09-14T23:57:20.942 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:20.942 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:20.942 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:20.942 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.942 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.942 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:55363 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 3145728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:20.942 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.204982+0000", 2024-09-14T23:57:20.942 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.668471915, 2024-09-14T23:57:20.942 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027646239999999999, 2024-09-14T23:57:20.942 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:20.942 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:20.943 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:20.943 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:20.943 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:20.943 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:20.943 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 55363 2024-09-14T23:57:20.943 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.943 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:20.943 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.943 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:20.943 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204982+0000", 2024-09-14T23:57:20.943 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:20.943 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.943 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.944 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:20.944 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204982+0000", 2024-09-14T23:57:20.944 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:20.944 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.944 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.944 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:20.944 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204983+0000", 2024-09-14T23:57:20.944 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2199999999999999e-07 2024-09-14T23:57:20.944 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.944 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.944 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:20.944 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204985+0000", 2024-09-14T23:57:20.944 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.261e-06 2024-09-14T23:57:20.944 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.945 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.945 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:20.945 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204991+0000", 2024-09-14T23:57:20.945 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7069999999999997e-06 2024-09-14T23:57:20.945 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.945 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.945 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:20.945 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204995+0000", 2024-09-14T23:57:20.945 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.853e-06 2024-09-14T23:57:20.945 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.945 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.945 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:20.945 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.232491+0000", 2024-09-14T23:57:20.945 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027495322999999999 2024-09-14T23:57:20.946 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.946 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.946 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:20.946 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.232563+0000", 2024-09-14T23:57:20.946 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1972999999999997e-05 2024-09-14T23:57:20.946 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.946 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.946 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:20.946 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.232628+0000", 2024-09-14T23:57:20.946 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5701e-05 2024-09-14T23:57:20.946 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:20.946 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:20.946 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:20.946 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.946 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.947 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:55364 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 3325952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:20.947 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.205003+0000", 2024-09-14T23:57:20.947 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.66845139, 2024-09-14T23:57:20.947 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027781963, 2024-09-14T23:57:20.947 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:20.947 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:20.947 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:20.947 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:20.947 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:20.947 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:20.947 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 55364 2024-09-14T23:57:20.947 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.947 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:20.948 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.948 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:20.948 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205003+0000", 2024-09-14T23:57:20.948 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:20.948 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.948 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.948 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:20.948 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205003+0000", 2024-09-14T23:57:20.948 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:20.948 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.948 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.948 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:20.948 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205003+0000", 2024-09-14T23:57:20.949 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2899999999999999e-07 2024-09-14T23:57:20.949 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.949 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.949 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:20.949 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205013+0000", 2024-09-14T23:57:20.949 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.611e-06 2024-09-14T23:57:20.949 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.949 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.949 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:20.949 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205020+0000", 2024-09-14T23:57:20.949 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2509999999999997e-06 2024-09-14T23:57:20.949 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.949 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.950 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:20.950 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205025+0000", 2024-09-14T23:57:20.950 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1399999999999999e-06 2024-09-14T23:57:20.950 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.950 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.950 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:20.950 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.232653+0000", 2024-09-14T23:57:20.950 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027627972000000001 2024-09-14T23:57:20.950 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.950 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.950 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:20.950 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.232723+0000", 2024-09-14T23:57:20.950 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9907999999999995e-05 2024-09-14T23:57:20.950 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.951 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.951 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:20.951 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.232785+0000", 2024-09-14T23:57:20.951 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1651999999999999e-05 2024-09-14T23:57:20.951 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:20.951 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:20.951 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:20.951 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.951 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.951 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:55365 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 3506176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:20.951 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.205033+0000", 2024-09-14T23:57:20.951 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.668421151, 2024-09-14T23:57:20.951 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027914906999999999, 2024-09-14T23:57:20.951 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:20.952 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:20.952 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:20.952 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:20.952 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:20.952 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:20.952 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 55365 2024-09-14T23:57:20.952 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.952 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:20.952 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.952 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:20.952 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205033+0000", 2024-09-14T23:57:20.952 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:20.953 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.953 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.953 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:20.953 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205033+0000", 2024-09-14T23:57:20.953 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:20.953 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.953 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.953 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:20.953 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205033+0000", 2024-09-14T23:57:20.953 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9900000000000001e-07 2024-09-14T23:57:20.953 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.953 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.953 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:20.954 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205036+0000", 2024-09-14T23:57:20.954 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3319999999999999e-06 2024-09-14T23:57:20.954 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.954 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.954 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:20.954 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205042+0000", 2024-09-14T23:57:20.954 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3049999999999998e-06 2024-09-14T23:57:20.954 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.954 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.954 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:20.954 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205047+0000", 2024-09-14T23:57:20.954 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0420000000000002e-06 2024-09-14T23:57:20.954 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.955 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.955 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:20.955 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.232809+0000", 2024-09-14T23:57:20.955 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027761917000000001 2024-09-14T23:57:20.955 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.955 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.955 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:20.955 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.232879+0000", 2024-09-14T23:57:20.955 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9597000000000005e-05 2024-09-14T23:57:20.955 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.955 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.955 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:20.955 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.232948+0000", 2024-09-14T23:57:20.955 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9214999999999994e-05 2024-09-14T23:57:20.956 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:20.956 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:20.956 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:20.956 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.956 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.956 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:55366 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 3686400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:20.956 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.205054+0000", 2024-09-14T23:57:20.956 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.668399964000001, 2024-09-14T23:57:20.956 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028049935000000002, 2024-09-14T23:57:20.956 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:20.956 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:20.956 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:20.956 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:20.956 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:20.957 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:20.957 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 55366 2024-09-14T23:57:20.957 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.957 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:20.957 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.957 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:20.957 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205054+0000", 2024-09-14T23:57:20.957 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:20.957 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.957 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.957 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:20.957 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205054+0000", 2024-09-14T23:57:20.957 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:20.958 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.958 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.958 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:20.958 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205055+0000", 2024-09-14T23:57:20.958 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9999999999999998e-07 2024-09-14T23:57:20.958 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.958 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.958 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:20.958 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205065+0000", 2024-09-14T23:57:20.958 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0489000000000001e-05 2024-09-14T23:57:20.958 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.958 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.958 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:20.958 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205073+0000", 2024-09-14T23:57:20.959 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9840000000000001e-06 2024-09-14T23:57:20.959 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.959 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.959 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:20.959 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205085+0000", 2024-09-14T23:57:20.959 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2052000000000001e-05 2024-09-14T23:57:20.959 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.959 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.959 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:20.959 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.232975+0000", 2024-09-14T23:57:20.959 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027889646000000001 2024-09-14T23:57:20.959 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.959 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.960 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:20.960 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.233039+0000", 2024-09-14T23:57:20.960 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3993999999999998e-05 2024-09-14T23:57:20.960 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.960 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.960 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:20.960 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.233104+0000", 2024-09-14T23:57:20.960 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5270000000000004e-05 2024-09-14T23:57:20.960 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:20.960 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:20.960 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:20.960 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.960 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.960 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:55367 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 3866624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:20.961 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.205098+0000", 2024-09-14T23:57:20.961 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.668356345999999, 2024-09-14T23:57:20.961 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028159871, 2024-09-14T23:57:20.961 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:20.961 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:20.961 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:20.961 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:20.961 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:20.961 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:20.961 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 55367 2024-09-14T23:57:20.961 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.961 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:20.961 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.961 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:20.961 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205098+0000", 2024-09-14T23:57:20.962 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:20.962 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.962 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.962 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:20.962 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205098+0000", 2024-09-14T23:57:20.962 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:20.962 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.962 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.962 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:20.962 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205098+0000", 2024-09-14T23:57:20.962 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4000000000000002e-07 2024-09-14T23:57:20.962 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.962 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.963 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:20.963 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205101+0000", 2024-09-14T23:57:20.963 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7860000000000001e-06 2024-09-14T23:57:20.963 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.963 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.963 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:20.963 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205109+0000", 2024-09-14T23:57:20.963 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4039999999999998e-06 2024-09-14T23:57:20.963 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.963 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.963 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:20.963 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205123+0000", 2024-09-14T23:57:20.963 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.42e-05 2024-09-14T23:57:20.963 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.964 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.964 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:20.964 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.233129+0000", 2024-09-14T23:57:20.964 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028006316999999999 2024-09-14T23:57:20.964 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.964 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.964 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:20.964 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.233196+0000", 2024-09-14T23:57:20.964 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7234e-05 2024-09-14T23:57:20.964 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.964 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.964 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:20.964 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.233258+0000", 2024-09-14T23:57:20.965 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1389999999999993e-05 2024-09-14T23:57:20.965 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:20.965 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:20.965 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:20.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.965 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:55368 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 4046848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:20.965 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.205132+0000", 2024-09-14T23:57:20.965 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.668322033999999, 2024-09-14T23:57:20.965 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028285467000000002, 2024-09-14T23:57:20.965 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:20.965 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:20.965 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:20.965 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:20.966 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:20.966 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:20.966 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 55368 2024-09-14T23:57:20.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.966 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:20.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.966 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:20.966 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205132+0000", 2024-09-14T23:57:20.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:20.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.966 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:20.966 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205132+0000", 2024-09-14T23:57:20.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:20.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.967 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:20.967 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205133+0000", 2024-09-14T23:57:20.967 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.13e-07 2024-09-14T23:57:20.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.967 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:20.967 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205135+0000", 2024-09-14T23:57:20.967 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5490000000000001e-06 2024-09-14T23:57:20.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.967 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:20.967 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205142+0000", 2024-09-14T23:57:20.967 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.007e-06 2024-09-14T23:57:20.968 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.968 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.968 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:20.968 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205147+0000", 2024-09-14T23:57:20.968 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.169e-06 2024-09-14T23:57:20.968 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.968 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.968 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:20.968 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.233281+0000", 2024-09-14T23:57:20.968 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028134059999999999 2024-09-14T23:57:20.968 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.968 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.968 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:20.969 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.233355+0000", 2024-09-14T23:57:20.969 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3120000000000005e-05 2024-09-14T23:57:20.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:20.969 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:20.969 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:20.969 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.233418+0000", 2024-09-14T23:57:20.969 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3048999999999999e-05 2024-09-14T23:57:20.969 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:20.969 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:20.969 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:20.969 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:20.969 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:20.969 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:20.971 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T23:57:20.971 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:20.969+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:21.071 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:21.070+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:21.119 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:21.119 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:57:21.119 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:57:21.119 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:21.128 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T23:57:21.149 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.7 to 1 (10000) 2024-09-14T23:57:21.158 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T23:57:21.165 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-14T23:57:21.172 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:21.171+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:21.264 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:21.264 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:57:21.264 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:57:21.264 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:57:21.264 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:21.273 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:21.270+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:21.274 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T23:57:21.374 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:21.371+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:21.425 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-09-14T23:57:21.439 INFO:teuthology.orchestra.run.smithi042.stdout:2426 read 0x948c4d3 thru 0x949b3af (0xeedd bytes) 2024-09-14T23:57:21.440 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:21.440 INFO:teuthology.orchestra.run.smithi042.stdout:2428 read 0xab0ecf0 thru 0xab0f0d2 (0x3e3 bytes) 2024-09-14T23:57:21.440 INFO:teuthology.orchestra.run.smithi042.stdout:2429 write 0x73e1f0b thru 0x73e809c (0x6192 bytes) 2024-09-14T23:57:21.458 INFO:teuthology.orchestra.run.smithi042.stdout:2430 trunc from 0xe8ab1db to 0x523bc9d 2024-09-14T23:57:21.474 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:21.472+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:21.486 INFO:teuthology.orchestra.run.smithi042.stdout:2431 write 0x1b53e43 thru 0x1b5aeb8 (0x7076 bytes) 2024-09-14T23:57:21.492 INFO:teuthology.orchestra.run.smithi042.stdout:2432 read 0x23b0343 thru 0x23bc48f (0xc14d bytes) 2024-09-14T23:57:21.492 INFO:teuthology.orchestra.run.smithi042.stdout:2433 read 0x4f88179 thru 0x4f88d21 (0xba9 bytes) 2024-09-14T23:57:21.492 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:21.493 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:21.494 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:57:21.494 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:57:21.494 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:57:21.494 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.494 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5880 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:21.494 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375821+0000", 2024-09-14T23:57:21.494 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.114846792999998, 2024-09-14T23:57:21.494 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050482096999999997, 2024-09-14T23:57:21.494 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:21.494 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:21.494 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:21.494 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:21.494 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:21.495 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:21.495 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5880 2024-09-14T23:57:21.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.495 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:21.495 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.495 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:21.495 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375821+0000", 2024-09-14T23:57:21.495 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:21.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.495 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.495 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:21.495 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375821+0000", 2024-09-14T23:57:21.495 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:21.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.496 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.496 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:21.496 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375821+0000", 2024-09-14T23:57:21.496 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.28e-07 2024-09-14T23:57:21.496 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.496 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.496 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:21.496 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375822+0000", 2024-09-14T23:57:21.496 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7000000000000003e-07 2024-09-14T23:57:21.496 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.496 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.496 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:21.496 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375825+0000", 2024-09-14T23:57:21.497 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9560000000000002e-06 2024-09-14T23:57:21.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.497 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.497 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:21.497 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375827+0000", 2024-09-14T23:57:21.497 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3259999999999998e-06 2024-09-14T23:57:21.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.497 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.497 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:21.497 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426294+0000", 2024-09-14T23:57:21.497 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050466748999999998 2024-09-14T23:57:21.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.497 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.497 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:21.498 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426303+0000", 2024-09-14T23:57:21.498 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8680000000000005e-06 2024-09-14T23:57:21.498 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:21.498 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:21.498 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:21.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.498 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5881 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:21.498 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375830+0000", 2024-09-14T23:57:21.498 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.114837346999998, 2024-09-14T23:57:21.498 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050523831999999998, 2024-09-14T23:57:21.498 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:21.498 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:21.499 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:21.499 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:21.499 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:21.499 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:21.499 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5881 2024-09-14T23:57:21.499 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.499 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:21.499 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.499 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:21.499 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375830+0000", 2024-09-14T23:57:21.499 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:21.499 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.499 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.500 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:21.500 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375830+0000", 2024-09-14T23:57:21.500 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:21.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.500 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.500 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:21.500 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375830+0000", 2024-09-14T23:57:21.500 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.34e-07 2024-09-14T23:57:21.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.500 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.500 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:21.500 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375831+0000", 2024-09-14T23:57:21.500 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9599999999999998e-07 2024-09-14T23:57:21.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.501 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.501 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:21.501 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375834+0000", 2024-09-14T23:57:21.501 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0489999999999999e-06 2024-09-14T23:57:21.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.501 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.501 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:21.501 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375838+0000", 2024-09-14T23:57:21.501 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5319999999999998e-06 2024-09-14T23:57:21.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.501 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.501 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:21.501 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426342+0000", 2024-09-14T23:57:21.502 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050504391000000003 2024-09-14T23:57:21.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.502 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.502 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:21.502 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426354+0000", 2024-09-14T23:57:21.502 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.173e-05 2024-09-14T23:57:21.502 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:21.502 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:21.502 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:21.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.503 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.503 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5882 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:21.503 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:57:21.503 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.114826743000002, 2024-09-14T23:57:21.503 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050549666, 2024-09-14T23:57:21.503 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:21.503 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:21.503 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:21.503 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:21.503 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:21.503 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:21.504 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5882 2024-09-14T23:57:21.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.504 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:21.504 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.504 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:21.504 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:57:21.504 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:21.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.504 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.504 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:21.504 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:57:21.504 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:21.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.505 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.505 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:21.505 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:57:21.505 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8099999999999998e-07 2024-09-14T23:57:21.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.505 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.505 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:21.505 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375842+0000", 2024-09-14T23:57:21.505 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9900000000000009e-07 2024-09-14T23:57:21.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.505 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.505 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:21.505 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375845+0000", 2024-09-14T23:57:21.505 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.055e-06 2024-09-14T23:57:21.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.506 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:21.506 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375848+0000", 2024-09-14T23:57:21.506 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7609999999999999e-06 2024-09-14T23:57:21.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.506 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:21.506 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426381+0000", 2024-09-14T23:57:21.506 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050533392000000003 2024-09-14T23:57:21.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.506 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:21.506 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426390+0000", 2024-09-14T23:57:21.507 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0780000000000002e-06 2024-09-14T23:57:21.507 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:21.507 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:21.507 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:21.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.507 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.507 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5884 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:21.507 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:57:21.507 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.114816603000001, 2024-09-14T23:57:21.507 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050717439000000003, 2024-09-14T23:57:21.507 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:21.508 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:21.508 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:21.508 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:21.508 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:21.508 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:21.508 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5884 2024-09-14T23:57:21.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.508 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:21.508 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.508 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:21.508 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:57:21.508 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:21.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.509 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.509 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:21.509 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:57:21.509 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:21.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.509 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.509 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:21.509 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:57:21.509 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0800000000000001e-07 2024-09-14T23:57:21.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.509 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.509 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:21.509 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375852+0000", 2024-09-14T23:57:21.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.02e-06 2024-09-14T23:57:21.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.510 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:21.510 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375855+0000", 2024-09-14T23:57:21.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9900000000000002e-06 2024-09-14T23:57:21.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.510 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:21.510 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375857+0000", 2024-09-14T23:57:21.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3499999999999999e-06 2024-09-14T23:57:21.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.510 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:21.511 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426556+0000", 2024-09-14T23:57:21.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050698139000000003 2024-09-14T23:57:21.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.511 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:21.511 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426568+0000", 2024-09-14T23:57:21.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2731999999999999e-05 2024-09-14T23:57:21.511 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:21.511 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:21.511 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:21.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.511 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5885 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:21.512 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:57:21.512 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.114797578000001, 2024-09-14T23:57:21.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050737333000000003, 2024-09-14T23:57:21.512 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:21.512 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:21.512 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:21.512 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:21.512 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:21.512 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:21.512 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5885 2024-09-14T23:57:21.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.512 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:21.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.513 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:21.513 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:57:21.513 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:21.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.513 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:21.513 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:57:21.513 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:21.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.513 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:21.513 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:57:21.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5899999999999998e-07 2024-09-14T23:57:21.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:21.514 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375872+0000", 2024-09-14T23:57:21.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4219999999999999e-06 2024-09-14T23:57:21.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:21.514 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375878+0000", 2024-09-14T23:57:21.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4980000000000003e-06 2024-09-14T23:57:21.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.515 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:21.515 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375881+0000", 2024-09-14T23:57:21.515 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6110000000000001e-06 2024-09-14T23:57:21.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.515 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:21.515 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426596+0000", 2024-09-14T23:57:21.515 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050715320000000001 2024-09-14T23:57:21.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.515 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:21.515 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426607+0000", 2024-09-14T23:57:21.515 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1223000000000001e-05 2024-09-14T23:57:21.515 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:21.515 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:21.516 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:21.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.516 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5886 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:21.516 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:57:21.516 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.114783465000002, 2024-09-14T23:57:21.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050803395000000001, 2024-09-14T23:57:21.516 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:21.516 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:21.516 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:21.516 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:21.516 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:21.516 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:21.517 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5886 2024-09-14T23:57:21.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.517 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:21.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.517 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:21.517 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:57:21.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:21.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.517 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:21.517 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:57:21.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:21.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.518 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:21.518 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:57:21.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2700000000000001e-07 2024-09-14T23:57:21.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.518 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:21.518 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375886+0000", 2024-09-14T23:57:21.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7060000000000001e-06 2024-09-14T23:57:21.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.519 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:21.519 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375890+0000", 2024-09-14T23:57:21.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2259999999999999e-06 2024-09-14T23:57:21.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.519 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:21.519 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375893+0000", 2024-09-14T23:57:21.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0819999999999999e-06 2024-09-14T23:57:21.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.519 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:21.519 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426677+0000", 2024-09-14T23:57:21.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050784158000000003 2024-09-14T23:57:21.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.520 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:21.520 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426687+0000", 2024-09-14T23:57:21.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9960000000000005e-06 2024-09-14T23:57:21.520 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:21.520 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:21.520 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:21.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.520 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5887 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:21.520 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:57:21.520 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.114761379000001, 2024-09-14T23:57:21.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050833070000000001, 2024-09-14T23:57:21.521 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:21.521 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:21.521 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:21.521 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:21.521 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:21.521 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:21.521 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5887 2024-09-14T23:57:21.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.521 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:21.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.521 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:21.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:57:21.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:21.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:21.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:57:21.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:21.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:21.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:57:21.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3299999999999998e-07 2024-09-14T23:57:21.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:21.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375908+0000", 2024-09-14T23:57:21.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7689999999999999e-06 2024-09-14T23:57:21.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:21.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375912+0000", 2024-09-14T23:57:21.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.535e-06 2024-09-14T23:57:21.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:21.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375916+0000", 2024-09-14T23:57:21.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8979999999999998e-06 2024-09-14T23:57:21.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:21.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426729+0000", 2024-09-14T23:57:21.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050813453000000001 2024-09-14T23:57:21.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:21.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426739+0000", 2024-09-14T23:57:21.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0081999999999999e-05 2024-09-14T23:57:21.524 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:21.524 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:21.524 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:21.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.524 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5888 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:21.525 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:57:21.525 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.114748302000002, 2024-09-14T23:57:21.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050854652, 2024-09-14T23:57:21.525 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:21.525 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:21.525 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:21.525 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:21.525 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:21.525 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:21.525 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5888 2024-09-14T23:57:21.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.525 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:21.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:21.526 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:57:21.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:21.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.526 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:21.526 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:57:21.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:21.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.526 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:21.526 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:57:21.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1400000000000001e-07 2024-09-14T23:57:21.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:21.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375921+0000", 2024-09-14T23:57:21.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.119e-06 2024-09-14T23:57:21.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:21.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375928+0000", 2024-09-14T23:57:21.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3170000000000004e-06 2024-09-14T23:57:21.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:21.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375931+0000", 2024-09-14T23:57:21.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6970000000000002e-06 2024-09-14T23:57:21.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:21.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426768+0000", 2024-09-14T23:57:21.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050837355000000001 2024-09-14T23:57:21.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:21.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426774+0000", 2024-09-14T23:57:21.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9499999999999998e-06 2024-09-14T23:57:21.528 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:21.528 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:21.528 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:21.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.529 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5889 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:21.529 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:57:21.529 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.114731378000002, 2024-09-14T23:57:21.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050885446000000001, 2024-09-14T23:57:21.529 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:21.529 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:21.529 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:21.529 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:21.529 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:21.529 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:21.529 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5889 2024-09-14T23:57:21.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.530 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:21.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:21.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:57:21.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:21.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:21.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:57:21.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:21.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:21.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:57:21.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.36e-07 2024-09-14T23:57:21.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:21.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375940+0000", 2024-09-14T23:57:21.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.844e-06 2024-09-14T23:57:21.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:21.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375944+0000", 2024-09-14T23:57:21.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6720000000000002e-06 2024-09-14T23:57:21.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:21.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375946+0000", 2024-09-14T23:57:21.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5629999999999999e-06 2024-09-14T23:57:21.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:21.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426810+0000", 2024-09-14T23:57:21.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050863255000000003 2024-09-14T23:57:21.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:21.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426822+0000", 2024-09-14T23:57:21.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1875999999999999e-05 2024-09-14T23:57:21.533 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:21.533 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:21.533 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:21.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.533 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5890 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:21.533 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:57:21.533 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.114717566000003, 2024-09-14T23:57:21.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050917549999999999, 2024-09-14T23:57:21.534 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:21.534 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:21.534 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:21.534 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:21.534 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:21.534 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:21.534 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5890 2024-09-14T23:57:21.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.534 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:21.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:21.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:57:21.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:21.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:21.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:57:21.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:21.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:21.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:57:21.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4299999999999999e-07 2024-09-14T23:57:21.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:21.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375951+0000", 2024-09-14T23:57:21.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1000000000000001e-06 2024-09-14T23:57:21.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:21.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375956+0000", 2024-09-14T23:57:21.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3370000000000001e-06 2024-09-14T23:57:21.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:21.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375958+0000", 2024-09-14T23:57:21.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4820000000000001e-06 2024-09-14T23:57:21.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:21.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426858+0000", 2024-09-14T23:57:21.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050899666000000003 2024-09-14T23:57:21.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:21.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426867+0000", 2024-09-14T23:57:21.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7219999999999994e-06 2024-09-14T23:57:21.537 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:21.537 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:21.537 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:21.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.537 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5891 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:21.538 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:57:21.538 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.114706243999997, 2024-09-14T23:57:21.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050948067, 2024-09-14T23:57:21.538 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:21.538 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:21.538 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:21.538 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:21.538 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:21.538 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:21.538 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5891 2024-09-14T23:57:21.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.538 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:21.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:21.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:57:21.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:21.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:21.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:57:21.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:21.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:21.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:57:21.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0800000000000001e-07 2024-09-14T23:57:21.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:21.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375962+0000", 2024-09-14T23:57:21.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9600000000000008e-07 2024-09-14T23:57:21.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:21.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375970+0000", 2024-09-14T23:57:21.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1600000000000001e-06 2024-09-14T23:57:21.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:21.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375972+0000", 2024-09-14T23:57:21.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4849999999999999e-06 2024-09-14T23:57:21.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:21.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426902+0000", 2024-09-14T23:57:21.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050929813999999997 2024-09-14T23:57:21.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:21.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426909+0000", 2024-09-14T23:57:21.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4039999999999998e-06 2024-09-14T23:57:21.541 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:21.542 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:21.542 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:21.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.542 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5892 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:21.542 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:57:21.542 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.114691387000001, 2024-09-14T23:57:21.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050965036999999998, 2024-09-14T23:57:21.542 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:21.542 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:21.542 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:21.542 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:21.543 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:21.543 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:21.543 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5892 2024-09-14T23:57:21.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.543 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:21.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:21.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:57:21.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:21.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:21.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:57:21.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:21.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:21.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:57:21.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.34e-07 2024-09-14T23:57:21.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:21.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375978+0000", 2024-09-14T23:57:21.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1880000000000001e-06 2024-09-14T23:57:21.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:21.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375981+0000", 2024-09-14T23:57:21.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2399999999999999e-06 2024-09-14T23:57:21.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:21.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375984+0000", 2024-09-14T23:57:21.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9110000000000002e-06 2024-09-14T23:57:21.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:21.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426934+0000", 2024-09-14T23:57:21.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050949912 2024-09-14T23:57:21.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:21.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426941+0000", 2024-09-14T23:57:21.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5519999999999999e-06 2024-09-14T23:57:21.546 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:21.546 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:21.546 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:21.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.546 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5893 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:21.546 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:57:21.546 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.114680790000001, 2024-09-14T23:57:21.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050975131999999999, 2024-09-14T23:57:21.547 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:21.547 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:21.547 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:21.547 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:21.547 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:21.547 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:21.547 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5893 2024-09-14T23:57:21.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.547 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:21.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:21.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:57:21.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:21.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:21.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:57:21.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:21.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:21.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:57:21.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2999999999999999e-07 2024-09-14T23:57:21.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:21.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375988+0000", 2024-09-14T23:57:21.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0410000000000001e-06 2024-09-14T23:57:21.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:21.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375992+0000", 2024-09-14T23:57:21.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.104e-06 2024-09-14T23:57:21.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:21.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375995+0000", 2024-09-14T23:57:21.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4310000000000001e-06 2024-09-14T23:57:21.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:21.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426958+0000", 2024-09-14T23:57:21.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050963642000000003 2024-09-14T23:57:21.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:21.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426962+0000", 2024-09-14T23:57:21.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6839999999999998e-06 2024-09-14T23:57:21.550 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:21.550 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:21.551 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:21.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.551 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5894 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:21.551 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:57:21.551 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.114669390000003, 2024-09-14T23:57:21.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050978333000000001, 2024-09-14T23:57:21.551 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:21.551 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:21.551 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:21.551 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:21.551 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:21.551 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:21.551 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5894 2024-09-14T23:57:21.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.552 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:21.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:21.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:57:21.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:21.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:21.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:57:21.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:21.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:21.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:57:21.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.23e-07 2024-09-14T23:57:21.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:21.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375999+0000", 2024-09-14T23:57:21.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9800000000000002e-07 2024-09-14T23:57:21.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:21.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376007+0000", 2024-09-14T23:57:21.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8539999999999997e-06 2024-09-14T23:57:21.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:21.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376011+0000", 2024-09-14T23:57:21.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.399e-06 2024-09-14T23:57:21.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:21.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426973+0000", 2024-09-14T23:57:21.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050962454999999997 2024-09-14T23:57:21.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:21.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426976+0000", 2024-09-14T23:57:21.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4039999999999999e-06 2024-09-14T23:57:21.555 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:21.555 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:21.555 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:21.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.555 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5895 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:21.555 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376014+0000", 2024-09-14T23:57:21.555 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.114653046000001, 2024-09-14T23:57:21.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050975951999999998, 2024-09-14T23:57:21.556 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:21.556 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:21.556 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:21.556 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:21.556 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:21.556 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:21.556 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5895 2024-09-14T23:57:21.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.556 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:21.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:21.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376014+0000", 2024-09-14T23:57:21.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:21.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:21.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376014+0000", 2024-09-14T23:57:21.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:21.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:21.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376015+0000", 2024-09-14T23:57:21.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2999999999999999e-07 2024-09-14T23:57:21.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:21.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376016+0000", 2024-09-14T23:57:21.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1659999999999999e-06 2024-09-14T23:57:21.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:21.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376024+0000", 2024-09-14T23:57:21.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8930000000000005e-06 2024-09-14T23:57:21.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:21.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376026+0000", 2024-09-14T23:57:21.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6869999999999999e-06 2024-09-14T23:57:21.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:21.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426987+0000", 2024-09-14T23:57:21.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050960601000000001 2024-09-14T23:57:21.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:21.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426990+0000", 2024-09-14T23:57:21.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3749999999999999e-06 2024-09-14T23:57:21.559 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:21.559 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:21.559 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:21.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.560 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5896 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:21.560 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:57:21.560 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.114637309000003, 2024-09-14T23:57:21.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050975656000000001, 2024-09-14T23:57:21.560 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:21.560 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:21.560 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:21.560 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:21.560 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:21.560 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:21.560 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5896 2024-09-14T23:57:21.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.560 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:21.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:21.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:57:21.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:21.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:21.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:57:21.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:21.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:21.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:57:21.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5100000000000001e-07 2024-09-14T23:57:21.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:21.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376032+0000", 2024-09-14T23:57:21.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.141e-06 2024-09-14T23:57:21.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:21.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376036+0000", 2024-09-14T23:57:21.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.25e-06 2024-09-14T23:57:21.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:21.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376038+0000", 2024-09-14T23:57:21.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5469999999999999e-06 2024-09-14T23:57:21.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:21.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427002+0000", 2024-09-14T23:57:21.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050964029000000001 2024-09-14T23:57:21.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:21.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427006+0000", 2024-09-14T23:57:21.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4379999999999999e-06 2024-09-14T23:57:21.563 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:21.563 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:21.563 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:21.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.564 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5897 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:21.564 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:57:21.564 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.114625509, 2024-09-14T23:57:21.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050978001000000002, 2024-09-14T23:57:21.564 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:21.564 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:21.564 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:21.564 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:21.564 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:21.564 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:21.564 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5897 2024-09-14T23:57:21.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.565 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:21.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:21.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:57:21.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:21.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:21.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:57:21.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:21.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:21.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:57:21.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2600000000000001e-07 2024-09-14T23:57:21.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:21.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376043+0000", 2024-09-14T23:57:21.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.054e-06 2024-09-14T23:57:21.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:21.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376047+0000", 2024-09-14T23:57:21.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0659999999999997e-06 2024-09-14T23:57:21.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:21.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376050+0000", 2024-09-14T23:57:21.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.88e-06 2024-09-14T23:57:21.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:21.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427017+0000", 2024-09-14T23:57:21.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050966453000000002 2024-09-14T23:57:21.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:21.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427020+0000", 2024-09-14T23:57:21.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3220000000000001e-06 2024-09-14T23:57:21.568 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:21.568 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:21.568 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:21.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.568 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5898 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:21.568 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376053+0000", 2024-09-14T23:57:21.568 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.114614023999998, 2024-09-14T23:57:21.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050982024000000001, 2024-09-14T23:57:21.569 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:21.569 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:21.569 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:21.569 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:21.569 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:21.569 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:21.569 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5898 2024-09-14T23:57:21.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.569 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:21.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:21.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376053+0000", 2024-09-14T23:57:21.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:21.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:21.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376053+0000", 2024-09-14T23:57:21.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:21.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:21.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376278+0000", 2024-09-14T23:57:21.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022426400000000001 2024-09-14T23:57:21.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:21.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376282+0000", 2024-09-14T23:57:21.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4739999999999999e-06 2024-09-14T23:57:21.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:21.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376296+0000", 2024-09-14T23:57:21.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4229999999999999e-05 2024-09-14T23:57:21.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:21.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376303+0000", 2024-09-14T23:57:21.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2199999999999997e-06 2024-09-14T23:57:21.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:21.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427031+0000", 2024-09-14T23:57:21.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050728154999999997 2024-09-14T23:57:21.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:21.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427036+0000", 2024-09-14T23:57:21.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6809999999999997e-06 2024-09-14T23:57:21.572 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:21.572 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:21.572 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:21.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.572 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5899 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:21.572 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376338+0000", 2024-09-14T23:57:21.572 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.114329300000001, 2024-09-14T23:57:21.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050712100000000003, 2024-09-14T23:57:21.572 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:21.572 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:21.572 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:21.573 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:21.573 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:21.573 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:21.573 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5899 2024-09-14T23:57:21.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.573 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:21.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:21.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376338+0000", 2024-09-14T23:57:21.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:21.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:21.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376338+0000", 2024-09-14T23:57:21.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:21.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:21.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376339+0000", 2024-09-14T23:57:21.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9100000000000003e-07 2024-09-14T23:57:21.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:21.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376342+0000", 2024-09-14T23:57:21.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7e-06 2024-09-14T23:57:21.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:21.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376350+0000", 2024-09-14T23:57:21.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5280000000000003e-06 2024-09-14T23:57:21.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:21.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376356+0000", 2024-09-14T23:57:21.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6010000000000002e-06 2024-09-14T23:57:21.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:21.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427047+0000", 2024-09-14T23:57:21.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050690962999999999 2024-09-14T23:57:21.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:21.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427050+0000", 2024-09-14T23:57:21.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.517e-06 2024-09-14T23:57:21.576 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:21.576 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:21.576 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:21.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.577 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5900 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:21.577 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376366+0000", 2024-09-14T23:57:21.577 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.114301040999997, 2024-09-14T23:57:21.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050697858999999998, 2024-09-14T23:57:21.577 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:21.577 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:21.577 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:21.577 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:21.577 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:21.577 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:21.578 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5900 2024-09-14T23:57:21.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.578 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:21.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:21.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376366+0000", 2024-09-14T23:57:21.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:21.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:21.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376366+0000", 2024-09-14T23:57:21.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:21.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:21.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377650+0000", 2024-09-14T23:57:21.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001283834 2024-09-14T23:57:21.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:21.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377655+0000", 2024-09-14T23:57:21.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7230000000000004e-06 2024-09-14T23:57:21.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:21.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377667+0000", 2024-09-14T23:57:21.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1746000000000001e-05 2024-09-14T23:57:21.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:21.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377676+0000", 2024-09-14T23:57:21.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3619999999999995e-06 2024-09-14T23:57:21.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:21.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427061+0000", 2024-09-14T23:57:21.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049384755000000002 2024-09-14T23:57:21.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:21.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:21.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:21.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427064+0000", 2024-09-14T23:57:21.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.439e-06 2024-09-14T23:57:21.583 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:21.584 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:21.584 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:21.584 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:21.584 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:21.584 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:21.585 INFO:teuthology.orchestra.run.smithi042.stdout:2435 trunc from 0x523bc9d to 0xd5e71aa 2024-09-14T23:57:21.587 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T23:57:21.589 INFO:teuthology.orchestra.run.smithi042.stdout:2436 trunc from 0xd5e71aa to 0x8af2108 2024-09-14T23:57:21.589 INFO:teuthology.orchestra.run.smithi042.stdout:2438 punch from 0x893ccf6 to 0x89462c6, (0x95d0 bytes) 2024-09-14T23:57:21.590 INFO:teuthology.orchestra.run.smithi042.stdout:2439 write 0x458415b thru 0x45927cd (0xe673 bytes) 2024-09-14T23:57:21.590 INFO:teuthology.orchestra.run.smithi042.stdout:2440 write 0xbcf5d83 thru 0xbcf9b9c (0x3e1a bytes) 2024-09-14T23:57:21.590 INFO:teuthology.orchestra.run.smithi042.stdout:2441 writesame 0xc065f14 thru 0xc0678c9 data_size 0x3(0x19b6 bytes) 2024-09-14T23:57:21.590 INFO:teuthology.orchestra.run.smithi042.stdout:2442 trunc from 0xc0678ca to 0x6925489 2024-09-14T23:57:21.590 INFO:teuthology.orchestra.run.smithi042.stdout:2444 writesame 0x3b08f5a thru 0x3b15915 data_size 0x2(0xc9bc bytes) 2024-09-14T23:57:21.590 INFO:teuthology.orchestra.run.smithi042.stdout:2447 read 0x21a835a thru 0x21b59d7 (0xd67e bytes) 2024-09-14T23:57:21.590 INFO:teuthology.orchestra.run.smithi042.stdout:2448 punch from 0x5863fc7 to 0x586b7b6, (0x77ef bytes) 2024-09-14T23:57:21.590 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:21.591 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:21.591 INFO:teuthology.orchestra.run.smithi042.stdout:2451 writesame 0xe87ee39 thru 0xe881b6b data_size 0x39(0x2d33 bytes) 2024-09-14T23:57:21.591 INFO:teuthology.orchestra.run.smithi042.stdout:2452 read 0x580ae4c thru 0x5817802 (0xc9b7 bytes) 2024-09-14T23:57:21.591 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:21.583+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:21.597 INFO:teuthology.orchestra.run.smithi042.stdout:2453 trunc from 0xe881b6c to 0x4146b32 2024-09-14T23:57:21.629 INFO:teuthology.orchestra.run.smithi042.stdout:2456 writesame 0xea9e608 thru 0xeaa0073 data_size 0x26(0x1a6c bytes) 2024-09-14T23:57:21.629 INFO:teuthology.orchestra.run.smithi042.stdout:2457 write 0x781f3a5 thru 0x781ff12 (0xb6e bytes) 2024-09-14T23:57:21.631 INFO:teuthology.orchestra.run.smithi042.stdout:2458 read 0xc412c7c thru 0xc41d050 (0xa3d5 bytes) 2024-09-14T23:57:21.631 INFO:teuthology.orchestra.run.smithi042.stdout:2459 punch from 0x6397d31 to 0x639f8f2, (0x7bc1 bytes) 2024-09-14T23:57:21.637 INFO:teuthology.orchestra.run.smithi042.stdout:2461 trunc from 0xeaa0074 to 0xa5b1674 2024-09-14T23:57:21.649 INFO:teuthology.orchestra.run.smithi042.stdout:2462 write 0xbd274ad thru 0xbd3029b (0x8def bytes) 2024-09-14T23:57:21.652 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:21.652 INFO:teuthology.orchestra.run.smithi042.stdout:2464 writesame 0x6445c5d thru 0x6446c89 data_size 0x1(0x102d bytes) 2024-09-14T23:57:21.656 INFO:teuthology.orchestra.run.smithi042.stdout:2465 write 0x667c1da thru 0x66880f4 (0xbf1b bytes) 2024-09-14T23:57:21.660 INFO:teuthology.orchestra.run.smithi042.stdout:2466 read 0xa9a0ab6 thru 0xa9aa967 (0x9eb2 bytes) 2024-09-14T23:57:21.660 INFO:teuthology.orchestra.run.smithi042.stdout:2468 write 0x45e3ac3 thru 0x45e5bfd (0x213b bytes) 2024-09-14T23:57:21.673 INFO:teuthology.orchestra.run.smithi042.stdout:2469 trunc from 0xbd3029c to 0x188d61c 2024-09-14T23:57:21.686 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:21.684+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:21.701 INFO:teuthology.orchestra.run.smithi042.stdout:2470 read 0xae9b34 thru 0xaf7951 (0xde1e bytes) 2024-09-14T23:57:21.720 INFO:teuthology.orchestra.run.smithi042.stdout:2471 write 0xd39a3c7 thru 0xd3a3731 (0x936b bytes) 2024-09-14T23:57:21.729 INFO:teuthology.orchestra.run.smithi042.stdout:2472 writesame 0xeb511a0 thru 0xeb5c692 data_size 0x3(0xb4f3 bytes) 2024-09-14T23:57:21.729 INFO:teuthology.orchestra.run.smithi042.stdout:2473 writesame 0x2da4b94 thru 0x2da91e3 data_size 0x78(0x4650 bytes) 2024-09-14T23:57:21.730 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:21.730 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:57:21.730 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:57:21.730 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:21.732 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:21.732 INFO:teuthology.orchestra.run.smithi042.stdout:2475 write 0x8d05b48 thru 0x8d10283 (0xa73c bytes) 2024-09-14T23:57:21.735 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:21.735 INFO:teuthology.orchestra.run.smithi042.stdout:2478 write 0x36dee13 thru 0x36e282f (0x3a1d bytes) 2024-09-14T23:57:21.738 INFO:teuthology.orchestra.run.smithi042.stdout:2479 read 0x9be7d58 thru 0x9bf18fc (0x9ba5 bytes) 2024-09-14T23:57:21.740 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T23:57:21.741 INFO:teuthology.orchestra.run.smithi042.stdout:2480 trunc from 0xeb5c693 to 0xd1e07e6 2024-09-14T23:57:21.749 INFO:teuthology.orchestra.run.smithi042.stdout:2481 write 0x1dbd384 thru 0x1dc7d70 (0xa9ed bytes) 2024-09-14T23:57:21.751 INFO:teuthology.orchestra.run.smithi042.stdout:2482 write 0xa545343 thru 0xa5523f4 (0xd0b2 bytes) 2024-09-14T23:57:21.753 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:21.753 INFO:teuthology.orchestra.run.smithi042.stdout:2485 read 0xc14c52 thru 0xc18e7d (0x422c bytes) 2024-09-14T23:57:21.754 INFO:teuthology.orchestra.run.smithi042.stdout:2486 punch from 0x47418d5 to 0x474b4f4, (0x9c1f bytes) 2024-09-14T23:57:21.754 INFO:teuthology.orchestra.run.smithi042.stdout:2487 read 0x6e1b918 thru 0x6e2b7b2 (0xfe9b bytes) 2024-09-14T23:57:21.754 INFO:teuthology.orchestra.run.smithi042.stdout:2488 writesame 0xc8a088b thru 0xc8ac441 data_size 0x7(0xbbb7 bytes) 2024-09-14T23:57:21.759 INFO:teuthology.orchestra.run.smithi042.stdout:2490 write 0x5848ea1 thru 0x584e6af (0x580f bytes) 2024-09-14T23:57:21.761 INFO:teuthology.orchestra.run.smithi042.stdout:2491 read 0x8d3d775 thru 0x8d4c337 (0xebc3 bytes) 2024-09-14T23:57:21.762 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:21.762 INFO:teuthology.orchestra.run.smithi042.stdout:2494 write 0x40ec317 thru 0x40fa828 (0xe512 bytes) 2024-09-14T23:57:21.765 INFO:teuthology.orchestra.run.smithi042.stdout:2496 read 0x3ca4f9d thru 0x3cac2c5 (0x7329 bytes) 2024-09-14T23:57:21.769 INFO:teuthology.orchestra.run.smithi042.stdout:2497 trunc from 0xd1e07e6 to 0x99a62f3 2024-09-14T23:57:21.778 INFO:teuthology.orchestra.run.smithi042.stdout:2499 writesame 0x8c4d86e thru 0x8c52cf4 data_size 0x1(0x5487 bytes) 2024-09-14T23:57:21.786 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:21.785+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:21.790 INFO:teuthology.orchestra.run.smithi042.stdout:2500 read 0x8ec6d82 thru 0x8ecc2f9 (0x5578 bytes) 2024-09-14T23:57:21.790 INFO:teuthology.orchestra.run.smithi042.stdout:2501 write 0x5248f4c thru 0x5253401 (0xa4b6 bytes) 2024-09-14T23:57:21.794 INFO:teuthology.orchestra.run.smithi042.stdout:2504 punch from 0x1b8b681 to 0x1b8eb2f, (0x34ae bytes) 2024-09-14T23:57:21.794 INFO:teuthology.orchestra.run.smithi042.stdout:2506 read 0x1259125 thru 0x125c4ab (0x3387 bytes) 2024-09-14T23:57:21.795 INFO:teuthology.orchestra.run.smithi042.stdout:2507 read 0x1be6708 thru 0x1bf0306 (0x9bff bytes) 2024-09-14T23:57:21.795 INFO:teuthology.orchestra.run.smithi042.stdout:2509 read 0x518e23c thru 0x5193c6a (0x5a2f bytes) 2024-09-14T23:57:21.795 INFO:teuthology.orchestra.run.smithi042.stdout:2510 punch from 0x2c349a5 to 0x2c37890, (0x2eeb bytes) 2024-09-14T23:57:21.795 INFO:teuthology.orchestra.run.smithi042.stdout:2511 read 0x1015445 thru 0x101f0d4 (0x9c90 bytes) 2024-09-14T23:57:21.795 INFO:teuthology.orchestra.run.smithi042.stdout:2512 read 0x7fe6da2 thru 0x7ff53bb (0xe61a bytes) 2024-09-14T23:57:21.795 INFO:teuthology.orchestra.run.smithi042.stdout:2513 read 0x46abdb2 thru 0x46b5405 (0x9654 bytes) 2024-09-14T23:57:21.805 INFO:teuthology.orchestra.run.smithi042.stdout:2514 write 0xe21ba00 thru 0xe2226b3 (0x6cb4 bytes) 2024-09-14T23:57:21.808 INFO:teuthology.orchestra.run.smithi042.stdout:2515 write 0xd82516e thru 0xd82fb07 (0xa99a bytes) 2024-09-14T23:57:21.811 INFO:teuthology.orchestra.run.smithi042.stdout:2518 read 0x196ab7 thru 0x199c2d (0x3177 bytes) 2024-09-14T23:57:21.811 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:21.812 INFO:teuthology.orchestra.run.smithi042.stdout:2521 punch from 0x5749bd1 to 0x57576ea, (0xdb19 bytes) 2024-09-14T23:57:21.812 INFO:teuthology.orchestra.run.smithi042.stdout:2522 trunc from 0xe2226b4 to 0xe184cf1 2024-09-14T23:57:21.812 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T23:57:21.829 INFO:teuthology.orchestra.run.smithi042.stdout:2523 trunc from 0xe184cf1 to 0x41ae497 2024-09-14T23:57:21.852 INFO:teuthology.orchestra.run.smithi042.stdout:2524 write 0x5d84b68 thru 0x5d8961d (0x4ab6 bytes) 2024-09-14T23:57:21.856 INFO:teuthology.orchestra.run.smithi042.stdout:2525 read 0x5325f7e thru 0x53348d9 (0xe95c bytes) 2024-09-14T23:57:21.856 INFO:teuthology.orchestra.run.smithi042.stdout:2526 read 0x3c2c3d0 thru 0x3c2da34 (0x1665 bytes) 2024-09-14T23:57:21.856 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:21.857 INFO:teuthology.orchestra.run.smithi042.stdout:2531 trunc from 0x5d8961e to 0x7769fae 2024-09-14T23:57:21.869 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:21.869 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:57:21.869 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:57:21.869 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:57:21.869 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:21.871 INFO:teuthology.orchestra.run.smithi042.stdout:2532 writesame 0x9f713ca thru 0x9f7bf42 data_size 0x1(0xab79 bytes) 2024-09-14T23:57:21.881 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T23:57:21.884 INFO:teuthology.orchestra.run.smithi042.stdout:2533 writesame 0xea72ccc thru 0xea7e582 data_size 0x1(0xb8b7 bytes) 2024-09-14T23:57:21.884 INFO:teuthology.orchestra.run.smithi042.stdout:2534 read 0x559b22 thru 0x564a3a (0xaf19 bytes) 2024-09-14T23:57:21.884 INFO:teuthology.orchestra.run.smithi042.stdout:2535 punch from 0x39fb313 to 0x3a0067a, (0x5367 bytes) 2024-09-14T23:57:21.884 INFO:teuthology.orchestra.run.smithi042.stdout:2536 write 0xd78478 thru 0xd824f6 (0xa07f bytes) 2024-09-14T23:57:21.886 INFO:teuthology.orchestra.run.smithi042.stdout:2538 read 0x3fa75cc thru 0x3fb2303 (0xad38 bytes) 2024-09-14T23:57:21.886 INFO:teuthology.orchestra.run.smithi042.stdout:2540 write 0xdc3d87d thru 0xdc4af17 (0xd69b bytes) 2024-09-14T23:57:21.887 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:21.886+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:21.889 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:21.889 INFO:teuthology.orchestra.run.smithi042.stdout:2544 write 0xe1d8f36 thru 0xe1e8e13 (0xfede bytes) 2024-09-14T23:57:21.893 INFO:teuthology.orchestra.run.smithi042.stdout:2545 write 0xbb0b767 thru 0xbb16cce (0xb568 bytes) 2024-09-14T23:57:21.896 INFO:teuthology.orchestra.run.smithi042.stdout:2546 writesame 0x7c965d5 thru 0x7c9745d data_size 0x1(0xe89 bytes) 2024-09-14T23:57:21.900 INFO:teuthology.orchestra.run.smithi042.stdout:2548 write 0x5eb2657 thru 0x5eb9950 (0x72fa bytes) 2024-09-14T23:57:21.903 INFO:teuthology.orchestra.run.smithi042.stdout:2549 writesame 0x575776 thru 0x585047 data_size 0x1(0xf8d2 bytes) 2024-09-14T23:57:21.922 INFO:teuthology.orchestra.run.smithi042.stdout:2550 write 0xba2368d thru 0xba28226 (0x4b9a bytes) 2024-09-14T23:57:21.926 INFO:teuthology.orchestra.run.smithi042.stdout:2551 punch from 0x205a449 to 0x20619e7, (0x759e bytes) 2024-09-14T23:57:21.926 INFO:teuthology.orchestra.run.smithi042.stdout:2552 read 0x71f35ca thru 0x720347f (0xfeb6 bytes) 2024-09-14T23:57:21.926 INFO:teuthology.orchestra.run.smithi042.stdout:2553 write 0xe99afa9 thru 0xe99b190 (0x1e8 bytes) 2024-09-14T23:57:21.928 INFO:teuthology.orchestra.run.smithi042.stdout:2556 writesame 0x5eaf690 thru 0x5ebe719 data_size 0x42(0xf08a bytes) 2024-09-14T23:57:21.930 INFO:teuthology.orchestra.run.smithi042.stdout:2558 write 0xe977e3d thru 0xe9822ab (0xa46f bytes) 2024-09-14T23:57:21.933 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:21.933 INFO:teuthology.orchestra.run.smithi042.stdout:2560 read 0x60b4f09 thru 0x60bc1a2 (0x729a bytes) 2024-09-14T23:57:21.933 INFO:teuthology.orchestra.run.smithi042.stdout:2562 write 0x6f4642e thru 0x6f49836 (0x3409 bytes) 2024-09-14T23:57:21.936 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:21.936 INFO:teuthology.orchestra.run.smithi042.stdout:2564 write 0x27b82f6 thru 0x27bafcf (0x2cda bytes) 2024-09-14T23:57:21.943 INFO:teuthology.orchestra.run.smithi042.stdout:2565 writesame 0xeb64a76 thru 0xeb6dc60 data_size 0x5(0x91eb bytes) 2024-09-14T23:57:21.944 INFO:teuthology.orchestra.run.smithi042.stdout:2566 writesame 0xc717526 thru 0xc722147 data_size 0xb(0xac22 bytes) 2024-09-14T23:57:21.949 INFO:teuthology.orchestra.run.smithi042.stdout:2569 read 0x4bb32e8 thru 0x4bbd619 (0xa332 bytes) 2024-09-14T23:57:21.949 INFO:teuthology.orchestra.run.smithi042.stdout:2570 punch from 0xe146e02 to 0xe153753, (0xc951 bytes) 2024-09-14T23:57:21.949 INFO:teuthology.orchestra.run.smithi042.stdout:2571 write 0x3ad9c04 thru 0x3ae48bb (0xacb8 bytes) 2024-09-14T23:57:21.953 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:21.953 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:21.953 INFO:teuthology.orchestra.run.smithi042.stdout:2576 write 0xb5028d1 thru 0xb50c9c0 (0xa0f0 bytes) 2024-09-14T23:57:21.961 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:57:21.961 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:57:21.971 INFO:teuthology.orchestra.run.smithi042.stdout:2577 trunc from 0xeb6dc61 to 0x126e803 2024-09-14T23:57:21.985 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:57:21.985 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:57:21.987 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:21.986+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:22.004 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:57:22.004 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:57:22.006 INFO:teuthology.orchestra.run.smithi042.stdout:2579 write 0x5233301 thru 0x524155a (0xe25a bytes) 2024-09-14T23:57:22.013 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:22.013 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:22.018 INFO:teuthology.orchestra.run.smithi042.stdout:2580 write 0x91644a5 thru 0x9164643 (0x19f bytes) 2024-09-14T23:57:22.021 INFO:teuthology.orchestra.run.smithi042.stdout:2581 read 0x1b53f41 thru 0x1b5b044 (0x7104 bytes) 2024-09-14T23:57:22.021 INFO:teuthology.orchestra.run.smithi042.stdout:2583 read 0x50ed606 thru 0x50fc281 (0xec7c bytes) 2024-09-14T23:57:22.023 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T23:57:22.024 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:57:22.024 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:57:22.025 INFO:teuthology.orchestra.run.smithi042.stdout:2586 trunc from 0x9164644 to 0xbb5cc0f 2024-09-14T23:57:22.029 INFO:teuthology.orchestra.run.smithi042.stdout:2587 write 0x99c6c8b thru 0x99d48a6 (0xdc1c bytes) 2024-09-14T23:57:22.032 INFO:teuthology.orchestra.run.smithi042.stdout:2589 write 0x59b2b7d thru 0x59bf480 (0xc904 bytes) 2024-09-14T23:57:22.034 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:22.034 INFO:teuthology.orchestra.run.smithi042.stdout:2592 punch from 0x203aaa8 to 0x2045720, (0xac78 bytes) 2024-09-14T23:57:22.034 INFO:teuthology.orchestra.run.smithi042.stdout:2594 write 0x4dedc47 thru 0x4df3264 (0x561e bytes) 2024-09-14T23:57:22.037 INFO:teuthology.orchestra.run.smithi042.stdout:2595 read 0x726d0a2 thru 0x727b1d6 (0xe135 bytes) 2024-09-14T23:57:22.037 INFO:teuthology.orchestra.run.smithi042.stdout:2596 writesame 0x54b5871 thru 0x54baccc data_size 0x2(0x545c bytes) 2024-09-14T23:57:22.044 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:57:22.044 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:57:22.044 INFO:teuthology.orchestra.run.smithi042.stdout:2597 read 0x8a520d8 thru 0x8a60ef5 (0xee1e bytes) 2024-09-14T23:57:22.045 INFO:teuthology.orchestra.run.smithi042.stdout:2599 write 0x5270c thru 0x59a06 (0x72fb bytes) 2024-09-14T23:57:22.045 INFO:teuthology.orchestra.run.smithi042.stdout:2600 write 0x7f64850 thru 0x7f6715a (0x290b bytes) 2024-09-14T23:57:22.049 INFO:teuthology.orchestra.run.smithi042.stdout:2601 punch from 0x3783dff to 0x3788737, (0x4938 bytes) 2024-09-14T23:57:22.049 INFO:teuthology.orchestra.run.smithi042.stdout:2602 read 0xb480d12 thru 0xb48d785 (0xca74 bytes) 2024-09-14T23:57:22.049 INFO:teuthology.orchestra.run.smithi042.stdout:2603 read 0x2250677 thru 0x225d5ae (0xcf38 bytes) 2024-09-14T23:57:22.049 INFO:teuthology.orchestra.run.smithi042.stdout:2605 write 0x4be8096 thru 0x4bef265 (0x71d0 bytes) 2024-09-14T23:57:22.053 INFO:teuthology.orchestra.run.smithi042.stdout:2606 punch from 0x1c78ab0 to 0x1c88826, (0xfd76 bytes) 2024-09-14T23:57:22.053 INFO:teuthology.orchestra.run.smithi042.stdout:2608 punch from 0x1c42037 to 0x1c494fc, (0x74c5 bytes) 2024-09-14T23:57:22.053 INFO:teuthology.orchestra.run.smithi042.stdout:2610 read 0x5b77d40 thru 0x5b7a120 (0x23e1 bytes) 2024-09-14T23:57:22.053 INFO:teuthology.orchestra.run.smithi042.stdout:2612 write 0xb551ebd thru 0xb555e6c (0x3fb0 bytes) 2024-09-14T23:57:22.064 INFO:teuthology.orchestra.run.smithi042.stdout:2613 writesame 0xe2e4072 thru 0xe2e523e data_size 0x1f(0x11cd bytes) 2024-09-14T23:57:22.064 INFO:teuthology.orchestra.run.smithi042.stdout:2614 read 0x4c82e9a thru 0x4c8956c (0x66d3 bytes) 2024-09-14T23:57:22.064 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:57:22.065 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:57:22.065 INFO:teuthology.orchestra.run.smithi042.stdout:2615 write 0x6ed08a1 thru 0x6edfba1 (0xf301 bytes) 2024-09-14T23:57:22.067 INFO:teuthology.orchestra.run.smithi042.stdout:2616 write 0xb4b8b7d thru 0xb4ba0b1 (0x1535 bytes) 2024-09-14T23:57:22.071 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:22.075 INFO:teuthology.orchestra.run.smithi042.stdout:2619 trunc from 0xe2e523f to 0xa41a94c 2024-09-14T23:57:22.084 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:57:22.084 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:57:22.086 INFO:teuthology.orchestra.run.smithi042.stdout:2620 write 0x5a9dd66 thru 0x5aa5031 (0x72cc bytes) 2024-09-14T23:57:22.088 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:22.087+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:22.097 INFO:teuthology.orchestra.run.smithi042.stdout:2621 write 0xbbbeaa0 thru 0xbbcdae8 (0xf049 bytes) 2024-09-14T23:57:22.101 INFO:teuthology.orchestra.run.smithi042.stdout:2622 read 0x672639e thru 0x672e1a9 (0x7e0c bytes) 2024-09-14T23:57:22.102 INFO:teuthology.orchestra.run.smithi042.stdout:2625 punch from 0xa6396af to 0xa647708, (0xe059 bytes) 2024-09-14T23:57:22.102 INFO:teuthology.orchestra.run.smithi042.stdout:2628 read 0x7223537 thru 0x7229c28 (0x66f2 bytes) 2024-09-14T23:57:22.102 INFO:teuthology.orchestra.run.smithi042.stdout:2629 read 0x8dbaa6d thru 0x8dca39d (0xf931 bytes) 2024-09-14T23:57:22.102 INFO:teuthology.orchestra.run.smithi042.stdout:2632 read 0xb83f4a7 thru 0xb84d6ae (0xe208 bytes) 2024-09-14T23:57:22.102 INFO:teuthology.orchestra.run.smithi042.stdout:2633 read 0xadaf5df thru 0xadb5c2e (0x6650 bytes) 2024-09-14T23:57:22.102 INFO:teuthology.orchestra.run.smithi042.stdout:2634 write 0x92a496b thru 0x92b12dd (0xc973 bytes) 2024-09-14T23:57:22.104 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:57:22.104 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:57:22.112 INFO:teuthology.orchestra.run.smithi042.stdout:2635 trunc from 0xbbcdae9 to 0x5047ed3 2024-09-14T23:57:22.125 INFO:teuthology.orchestra.run.smithi042.stdout:2636 punch from 0x15e1d8d to 0x15f05ca, (0xe83d bytes) 2024-09-14T23:57:22.125 INFO:teuthology.orchestra.run.smithi042.stdout:2637 punch from 0x34c11ba to 0x34c14d8, (0x31e bytes) 2024-09-14T23:57:22.126 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:22.140 INFO:teuthology.orchestra.run.smithi042.stdout:2639 write 0xdf257bc thru 0xdf300f1 (0xa936 bytes) 2024-09-14T23:57:22.144 INFO:teuthology.orchestra.run.smithi042.stdout:2641 read 0xba0d1e2 thru 0xba1cf0b (0xfd2a bytes) 2024-09-14T23:57:22.144 INFO:teuthology.orchestra.run.smithi042.stdout:2643 write 0x523c264 thru 0x52493a5 (0xd142 bytes) 2024-09-14T23:57:22.145 INFO:teuthology.orchestra.run.smithi042.stdout:2644 writesame 0x9bfe541 thru 0x9c0883f data_size 0x7(0xa2ff bytes) 2024-09-14T23:57:22.147 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:22.147 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:22.151 INFO:teuthology.orchestra.run.smithi042.stdout:2645 writesame 0x8fc0fe2 thru 0x8fc8da4 data_size 0x2f(0x7dc3 bytes) 2024-09-14T23:57:22.154 INFO:teuthology.orchestra.run.smithi042.stdout:2646 read 0x779ed11 thru 0x77a9b26 (0xae16 bytes) 2024-09-14T23:57:22.157 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is set 2024-09-14T23:57:22.157 INFO:teuthology.orchestra.run.smithi042.stdout:2648 trunc from 0xdf300f2 to 0xb9ee8fe 2024-09-14T23:57:22.159 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T23:57:22.160 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.5 to 1 (10000) 2024-09-14T23:57:22.173 INFO:teuthology.orchestra.run.smithi042.stdout:2649 punch from 0x6bfc41d to 0x6c04f98, (0x8b7b bytes) 2024-09-14T23:57:22.173 INFO:teuthology.orchestra.run.smithi042.stdout:2650 read 0x53eb571 thru 0x53f556d (0x9ffd bytes) 2024-09-14T23:57:22.173 INFO:teuthology.orchestra.run.smithi042.stdout:2652 write 0x6dea499 thru 0x6dfa321 (0xfe89 bytes) 2024-09-14T23:57:22.176 INFO:teuthology.orchestra.run.smithi042.stdout:2655 punch from 0x587cb77 to 0x5880fb9, (0x4442 bytes) 2024-09-14T23:57:22.177 INFO:teuthology.orchestra.run.smithi042.stdout:2657 writesame 0x4694b36 thru 0x469e60f data_size 0x3(0x9ada bytes) 2024-09-14T23:57:22.178 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-14T23:57:22.178 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T23:57:22.185 INFO:teuthology.orchestra.run.smithi042.stdout:2659 read 0x31664e0 thru 0x3169213 (0x2d34 bytes) 2024-09-14T23:57:22.193 INFO:teuthology.orchestra.run.smithi042.stdout:2660 write 0xe066747 thru 0xe06dea5 (0x775f bytes) 2024-09-14T23:57:22.194 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:22.191+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:22.197 INFO:teuthology.orchestra.run.smithi042.stdout:2661 read 0xbf3e203 thru 0xbf40fdd (0x2ddb bytes) 2024-09-14T23:57:22.197 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:22.197 INFO:teuthology.orchestra.run.smithi042.stdout:2663 writesame 0x9b11528 thru 0x9b13c6e data_size 0x1(0x2747 bytes) 2024-09-14T23:57:22.275 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:22.275 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:22.284 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T23:57:22.325 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:22.323+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:22.390 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:22.389+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:22.432 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:22.432 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:22.441 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T23:57:22.477 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:57:22.478 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-14T23:57:22.489 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":99,"stamp":"2024-09-14T23:57:22.157002+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":623728736,"num_objects":1862,"num_object_clones":349,"num_object_copies":3724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1862,"num_whiteouts":979,"num_read":124066,"num_read_kb":508291,"num_write":10527,"num_write_kb":258604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":842,"num_bytes_recovered":37168158,"num_keys_recovered":121,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8996,"ondisk_log_size":8996,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":317788,"kb_used_data":47480,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754656932,"statfs":{"total":773094113280,"available":772768698368,"internally_reserved":0,"allocated":48619520,"data_stored":411219635,"data_compressed":29748525,"data_compressed_allocated":32854016,"data_compressed_original":400498446,"omap_allocated":12752,"internal_metadata":276614704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,20,52,78,65],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":45513001,"num_objects":498,"num_object_clones":43,"num_object_copies":996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":498,"num_whiteouts":421,"num_read":40628,"num_read_kb":145866,"num_write":1186,"num_write_kb":70896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.027405"},"pg_stats":[{"pgid":"3.1f","version":"67'428","reported_seq":2066,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-14T23:57:18.138713+0000","last_change":"2024-09-14T23:57:00.062727+0000","last_active":"2024-09-14T23:57:18.138713+0000","last_peered":"2024-09-14T23:57:18.138713+0000","last_clean":"2024-09-14T23:57:18.138713+0000","last_became_active":"2024-09-14T23:57:00.062116+0000","last_became_peered":"2024-09-14T23:57:00.062116+0000","last_unstale":"2024-09-14T23:57:18.138713+0000","last_undegraded":"2024-09-14T23:57:18.138713+0000","last_fullsized":"2024-09-14T23:57:18.138713+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:40:41.174328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015692659999999999,"stat_sum":{"num_bytes":23460714,"num_objects":66,"num_object_clones":11,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":34,"num_read":3498,"num_read_kb":8713,"num_write":533,"num_write_kb":19177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":2327658,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"68'182","reported_seq":1417,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.145376+0000","last_change":"2024-09-14T23:56:34.415748+0000","last_active":"2024-09-14T23:57:20.145376+0000","last_peered":"2024-09-14T23:57:20.145376+0000","last_clean":"2024-09-14T23:57:20.145376+0000","last_became_active":"2024-09-14T23:56:32.404385+0000","last_became_peered":"2024-09-14T23:56:32.404385+0000","last_unstale":"2024-09-14T23:57:20.145376+0000","last_undegraded":"2024-09-14T23:57:20.145376+0000","last_fullsized":"2024-09-14T23:57:20.145376+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:18:14.290947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010144430000000001,"stat_sum":{"num_bytes":13278889,"num_objects":64,"num_object_clones":15,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":31,"num_read":1245,"num_read_kb":9182,"num_write":169,"num_write_kb":8578,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"68'274","reported_seq":2745,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.147084+0000","last_change":"2024-09-14T23:57:00.062960+0000","last_active":"2024-09-14T23:57:20.147084+0000","last_peered":"2024-09-14T23:57:20.147084+0000","last_clean":"2024-09-14T23:57:20.147084+0000","last_became_active":"2024-09-14T23:57:00.062242+0000","last_became_peered":"2024-09-14T23:57:00.062242+0000","last_unstale":"2024-09-14T23:57:20.147084+0000","last_undegraded":"2024-09-14T23:57:20.147084+0000","last_fullsized":"2024-09-14T23:57:20.147084+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:44:00.321217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001148175,"stat_sum":{"num_bytes":29669808,"num_objects":67,"num_object_clones":15,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":34,"num_read":3214,"num_read_kb":13398,"num_write":249,"num_write_kb":11174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":1249963,"num_keys_recovered":34,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"68'297","reported_seq":3042,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.147123+0000","last_change":"2024-09-14T23:57:00.062694+0000","last_active":"2024-09-14T23:57:20.147123+0000","last_peered":"2024-09-14T23:57:20.147123+0000","last_clean":"2024-09-14T23:57:20.147123+0000","last_became_active":"2024-09-14T23:57:00.062126+0000","last_became_peered":"2024-09-14T23:57:00.062126+0000","last_unstale":"2024-09-14T23:57:20.147123+0000","last_undegraded":"2024-09-14T23:57:20.147123+0000","last_fullsized":"2024-09-14T23:57:20.147123+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:09:32.074708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095260199999999998,"stat_sum":{"num_bytes":9921574,"num_objects":52,"num_object_clones":7,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":30,"num_read":3528,"num_read_kb":10946,"num_write":351,"num_write_kb":5066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":2675491,"num_keys_recovered":40,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"65'143","reported_seq":608,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-14T23:57:18.181431+0000","last_change":"2024-09-14T23:56:34.401802+0000","last_active":"2024-09-14T23:57:18.181431+0000","last_peered":"2024-09-14T23:57:18.181431+0000","last_clean":"2024-09-14T23:57:18.181431+0000","last_became_active":"2024-09-14T23:56:32.398329+0000","last_became_peered":"2024-09-14T23:56:32.398329+0000","last_unstale":"2024-09-14T23:57:18.181431+0000","last_undegraded":"2024-09-14T23:57:18.181431+0000","last_fullsized":"2024-09-14T23:57:18.181431+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:28:01.396708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018313229999999999,"stat_sum":{"num_bytes":10931520,"num_objects":39,"num_object_clones":6,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":21,"num_read":391,"num_read_kb":7357,"num_write":129,"num_write_kb":4515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"68'231","reported_seq":2151,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.174636+0000","last_change":"2024-09-14T23:56:34.405093+0000","last_active":"2024-09-14T23:57:20.174636+0000","last_peered":"2024-09-14T23:57:20.174636+0000","last_clean":"2024-09-14T23:57:20.174636+0000","last_became_active":"2024-09-14T23:56:32.399655+0000","last_became_peered":"2024-09-14T23:56:32.399655+0000","last_unstale":"2024-09-14T23:57:20.174636+0000","last_undegraded":"2024-09-14T23:57:20.174636+0000","last_fullsized":"2024-09-14T23:57:20.174636+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:17:56.591978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096230199999999995,"stat_sum":{"num_bytes":15076848,"num_objects":71,"num_object_clones":15,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":43,"num_read":1922,"num_read_kb":10181,"num_write":217,"num_write_kb":13721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"66'202","reported_seq":1072,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-14T23:57:18.165959+0000","last_change":"2024-09-14T23:56:34.400617+0000","last_active":"2024-09-14T23:57:18.165959+0000","last_peered":"2024-09-14T23:57:18.165959+0000","last_clean":"2024-09-14T23:57:18.165959+0000","last_became_active":"2024-09-14T23:56:32.404364+0000","last_became_peered":"2024-09-14T23:56:32.404364+0000","last_unstale":"2024-09-14T23:57:18.165959+0000","last_undegraded":"2024-09-14T23:57:18.165959+0000","last_fullsized":"2024-09-14T23:57:18.165959+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:29:49.811207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001003264,"stat_sum":{"num_bytes":11096435,"num_objects":65,"num_object_clones":9,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":38,"num_read":967,"num_read_kb":4343,"num_write":188,"num_write_kb":3031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"68'244","reported_seq":2978,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.145133+0000","last_change":"2024-09-14T23:56:34.415021+0000","last_active":"2024-09-14T23:57:20.145133+0000","last_peered":"2024-09-14T23:57:20.145133+0000","last_clean":"2024-09-14T23:57:20.145133+0000","last_became_active":"2024-09-14T23:56:32.401799+0000","last_became_peered":"2024-09-14T23:56:32.401799+0000","last_unstale":"2024-09-14T23:57:20.145133+0000","last_undegraded":"2024-09-14T23:57:20.145133+0000","last_fullsized":"2024-09-14T23:57:20.145133+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":244,"log_dups_size":0,"ondisk_log_size":244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:01:57.971994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062985000000000005,"stat_sum":{"num_bytes":16572772,"num_objects":70,"num_object_clones":14,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":39,"num_read":3481,"num_read_kb":10912,"num_write":223,"num_write_kb":1906,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"66'186","reported_seq":1187,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-14T23:57:18.179656+0000","last_change":"2024-09-14T23:57:00.062824+0000","last_active":"2024-09-14T23:57:18.179656+0000","last_peered":"2024-09-14T23:57:18.179656+0000","last_clean":"2024-09-14T23:57:18.179656+0000","last_became_active":"2024-09-14T23:57:00.062304+0000","last_became_peered":"2024-09-14T23:57:00.062304+0000","last_unstale":"2024-09-14T23:57:18.179656+0000","last_undegraded":"2024-09-14T23:57:18.179656+0000","last_fullsized":"2024-09-14T23:57:18.179656+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:26:21.553898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014783859999999999,"stat_sum":{"num_bytes":7227960,"num_objects":48,"num_object_clones":9,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":24,"num_read":938,"num_read_kb":4320,"num_write":188,"num_write_kb":2947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":3345543,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"66'190","reported_seq":5531,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.145319+0000","last_change":"2024-09-14T23:57:13.115006+0000","last_active":"2024-09-14T23:57:20.145319+0000","last_peered":"2024-09-14T23:57:20.145319+0000","last_clean":"2024-09-14T23:57:20.145319+0000","last_became_active":"2024-09-14T23:57:13.114091+0000","last_became_peered":"2024-09-14T23:57:13.114091+0000","last_unstale":"2024-09-14T23:57:20.145319+0000","last_undegraded":"2024-09-14T23:57:20.145319+0000","last_fullsized":"2024-09-14T23:57:20.145319+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:33:49.799819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039416400000000002,"stat_sum":{"num_bytes":25452456,"num_objects":45,"num_object_clones":6,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":23,"num_read":5059,"num_read_kb":21361,"num_write":163,"num_write_kb":10062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":103,"num_bytes_recovered":3116309,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.15","version":"68'308","reported_seq":6939,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.145425+0000","last_change":"2024-09-14T23:56:34.399382+0000","last_active":"2024-09-14T23:57:20.145425+0000","last_peered":"2024-09-14T23:57:20.145425+0000","last_clean":"2024-09-14T23:57:20.145425+0000","last_became_active":"2024-09-14T23:56:32.405387+0000","last_became_peered":"2024-09-14T23:56:32.405387+0000","last_unstale":"2024-09-14T23:57:20.145425+0000","last_undegraded":"2024-09-14T23:57:20.145425+0000","last_fullsized":"2024-09-14T23:57:20.145425+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:17:00.235747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046328199999999997,"stat_sum":{"num_bytes":36776121,"num_objects":54,"num_object_clones":12,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":25,"num_read":7488,"num_read_kb":29301,"num_write":385,"num_write_kb":22562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"68'670","reported_seq":4989,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.147024+0000","last_change":"2024-09-14T23:56:34.404536+0000","last_active":"2024-09-14T23:57:20.147024+0000","last_peered":"2024-09-14T23:57:20.147024+0000","last_clean":"2024-09-14T23:57:20.147024+0000","last_became_active":"2024-09-14T23:56:32.403019+0000","last_became_peered":"2024-09-14T23:56:32.403019+0000","last_unstale":"2024-09-14T23:57:20.147024+0000","last_undegraded":"2024-09-14T23:57:20.147024+0000","last_fullsized":"2024-09-14T23:57:20.147024+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:36:36.583890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037505299999999997,"stat_sum":{"num_bytes":21573885,"num_objects":55,"num_object_clones":11,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":27,"num_read":7236,"num_read_kb":21793,"num_write":1131,"num_write_kb":5887,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":115,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.153322+0000","last_change":"2024-09-14T23:56:24.727061+0000","last_active":"2024-09-14T23:57:20.153322+0000","last_peered":"2024-09-14T23:57:20.153322+0000","last_clean":"2024-09-14T23:57:20.153322+0000","last_became_active":"2024-09-14T23:56:22.526915+0000","last_became_peered":"2024-09-14T23:56:22.526915+0000","last_unstale":"2024-09-14T23:57:20.153322+0000","last_undegraded":"2024-09-14T23:57:20.153322+0000","last_fullsized":"2024-09-14T23:57:20.153322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:43:15.359266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045105900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"66'247","reported_seq":2837,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-14T23:57:18.178135+0000","last_change":"2024-09-14T23:56:34.402234+0000","last_active":"2024-09-14T23:57:18.178135+0000","last_peered":"2024-09-14T23:57:18.178135+0000","last_clean":"2024-09-14T23:57:18.178135+0000","last_became_active":"2024-09-14T23:56:32.398664+0000","last_became_peered":"2024-09-14T23:56:32.398664+0000","last_unstale":"2024-09-14T23:57:18.178135+0000","last_undegraded":"2024-09-14T23:57:18.178135+0000","last_fullsized":"2024-09-14T23:57:18.178135+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:40:18.337645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012179999999999999,"stat_sum":{"num_bytes":14975451,"num_objects":48,"num_object_clones":6,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":23,"num_read":3872,"num_read_kb":13569,"num_write":244,"num_write_kb":4246,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":111,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-14T23:57:18.136220+0000","last_change":"2024-09-14T23:56:24.534632+0000","last_active":"2024-09-14T23:57:18.136220+0000","last_peered":"2024-09-14T23:57:18.136220+0000","last_clean":"2024-09-14T23:57:18.136220+0000","last_became_active":"2024-09-14T23:56:22.526355+0000","last_became_peered":"2024-09-14T23:56:22.526355+0000","last_unstale":"2024-09-14T23:57:18.136220+0000","last_undegraded":"2024-09-14T23:57:18.136220+0000","last_fullsized":"2024-09-14T23:57:18.136220+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:02:16.059627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049910000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"69'226","reported_seq":4798,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.189555+0000","last_change":"2024-09-14T23:56:34.416134+0000","last_active":"2024-09-14T23:57:20.189555+0000","last_peered":"2024-09-14T23:57:20.189555+0000","last_clean":"2024-09-14T23:57:20.189555+0000","last_became_active":"2024-09-14T23:56:32.404281+0000","last_became_peered":"2024-09-14T23:56:32.404281+0000","last_unstale":"2024-09-14T23:57:20.189555+0000","last_undegraded":"2024-09-14T23:57:20.189555+0000","last_fullsized":"2024-09-14T23:57:20.189555+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:37:18.248147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013337620000000001,"stat_sum":{"num_bytes":26122292,"num_objects":67,"num_object_clones":14,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":37,"num_read":5489,"num_read_kb":30567,"num_write":201,"num_write_kb":7732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":115,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.145540+0000","last_change":"2024-09-14T23:56:25.093112+0000","last_active":"2024-09-14T23:57:20.145540+0000","last_peered":"2024-09-14T23:57:20.145540+0000","last_clean":"2024-09-14T23:57:20.145540+0000","last_became_active":"2024-09-14T23:56:22.528211+0000","last_became_peered":"2024-09-14T23:56:22.528211+0000","last_unstale":"2024-09-14T23:57:20.145540+0000","last_undegraded":"2024-09-14T23:57:20.145540+0000","last_fullsized":"2024-09-14T23:57:20.145540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:14:11.495286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000459686,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"65'223","reported_seq":2461,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-14T23:57:18.174355+0000","last_change":"2024-09-14T23:56:34.401111+0000","last_active":"2024-09-14T23:57:18.174355+0000","last_peered":"2024-09-14T23:57:18.174355+0000","last_clean":"2024-09-14T23:57:18.174355+0000","last_became_active":"2024-09-14T23:56:32.401089+0000","last_became_peered":"2024-09-14T23:56:32.401089+0000","last_unstale":"2024-09-14T23:57:18.174355+0000","last_undegraded":"2024-09-14T23:57:18.174355+0000","last_fullsized":"2024-09-14T23:57:18.174355+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:39.493250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013033610000000001,"stat_sum":{"num_bytes":18464241,"num_objects":72,"num_object_clones":13,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":41,"num_read":2227,"num_read_kb":13887,"num_write":229,"num_write_kb":8679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":111,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-14T23:57:18.135987+0000","last_change":"2024-09-14T23:56:24.534510+0000","last_active":"2024-09-14T23:57:18.135987+0000","last_peered":"2024-09-14T23:57:18.135987+0000","last_clean":"2024-09-14T23:57:18.135987+0000","last_became_active":"2024-09-14T23:56:22.527611+0000","last_became_peered":"2024-09-14T23:56:22.527611+0000","last_unstale":"2024-09-14T23:57:18.135987+0000","last_undegraded":"2024-09-14T23:57:18.135987+0000","last_fullsized":"2024-09-14T23:57:18.135987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:15:45.606026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049916299999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"68'359","reported_seq":2290,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.145361+0000","last_change":"2024-09-14T23:57:00.065551+0000","last_active":"2024-09-14T23:57:20.145361+0000","last_peered":"2024-09-14T23:57:20.145361+0000","last_clean":"2024-09-14T23:57:20.145361+0000","last_became_active":"2024-09-14T23:57:00.064678+0000","last_became_peered":"2024-09-14T23:57:00.064678+0000","last_unstale":"2024-09-14T23:57:20.145361+0000","last_undegraded":"2024-09-14T23:57:20.145361+0000","last_fullsized":"2024-09-14T23:57:20.145361+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:54:22.755339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001272947,"stat_sum":{"num_bytes":12132806,"num_objects":74,"num_object_clones":15,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":39,"num_read":2794,"num_read_kb":11298,"num_write":430,"num_write_kb":8654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":3349587,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"69'223","reported_seq":2075,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.196275+0000","last_change":"2024-09-14T23:56:34.404751+0000","last_active":"2024-09-14T23:57:20.196275+0000","last_peered":"2024-09-14T23:57:20.196275+0000","last_clean":"2024-09-14T23:57:20.196275+0000","last_became_active":"2024-09-14T23:56:32.397883+0000","last_became_peered":"2024-09-14T23:56:32.397883+0000","last_unstale":"2024-09-14T23:57:20.196275+0000","last_undegraded":"2024-09-14T23:57:20.196275+0000","last_fullsized":"2024-09-14T23:57:20.196275+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:38:47.008304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048955799999999996,"stat_sum":{"num_bytes":20549548,"num_objects":64,"num_object_clones":11,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":30,"num_read":2766,"num_read_kb":10503,"num_write":191,"num_write_kb":4407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":122,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-14T23:57:18.136327+0000","last_change":"2024-09-14T23:56:57.052969+0000","last_active":"2024-09-14T23:57:18.136327+0000","last_peered":"2024-09-14T23:57:18.136327+0000","last_clean":"2024-09-14T23:57:18.136327+0000","last_became_active":"2024-09-14T23:56:57.050456+0000","last_became_peered":"2024-09-14T23:56:57.050456+0000","last_unstale":"2024-09-14T23:57:18.136327+0000","last_undegraded":"2024-09-14T23:57:18.136327+0000","last_fullsized":"2024-09-14T23:57:18.136327+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:57:07.644011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"65'216","reported_seq":6554,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-14T23:57:18.200629+0000","last_change":"2024-09-14T23:56:34.401159+0000","last_active":"2024-09-14T23:57:18.200629+0000","last_peered":"2024-09-14T23:57:18.200629+0000","last_clean":"2024-09-14T23:57:18.200629+0000","last_became_active":"2024-09-14T23:56:32.400278+0000","last_became_peered":"2024-09-14T23:56:32.400278+0000","last_unstale":"2024-09-14T23:57:18.200629+0000","last_undegraded":"2024-09-14T23:57:18.200629+0000","last_fullsized":"2024-09-14T23:57:18.200629+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:40:29.250653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082507200000000004,"stat_sum":{"num_bytes":25177953,"num_objects":50,"num_object_clones":6,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":27,"num_read":6413,"num_read_kb":20437,"num_write":207,"num_write_kb":6440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":107,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-14T23:57:16.149408+0000","last_change":"2024-09-14T23:56:24.909260+0000","last_active":"2024-09-14T23:57:16.149408+0000","last_peered":"2024-09-14T23:57:16.149408+0000","last_clean":"2024-09-14T23:57:16.149408+0000","last_became_active":"2024-09-14T23:56:22.822598+0000","last_became_peered":"2024-09-14T23:56:22.822598+0000","last_unstale":"2024-09-14T23:57:16.149408+0000","last_undegraded":"2024-09-14T23:57:16.149408+0000","last_fullsized":"2024-09-14T23:57:16.149408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:32:32.568028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026863400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"66'475","reported_seq":3535,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-14T23:57:18.137879+0000","last_change":"2024-09-14T23:56:34.401366+0000","last_active":"2024-09-14T23:57:18.137879+0000","last_peered":"2024-09-14T23:57:18.137879+0000","last_clean":"2024-09-14T23:57:18.137879+0000","last_became_active":"2024-09-14T23:56:32.397551+0000","last_became_peered":"2024-09-14T23:56:32.397551+0000","last_unstale":"2024-09-14T23:57:18.137879+0000","last_undegraded":"2024-09-14T23:57:18.137879+0000","last_fullsized":"2024-09-14T23:57:18.137879+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":475,"log_dups_size":0,"ondisk_log_size":475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:32:09.324100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001418816,"stat_sum":{"num_bytes":11751557,"num_objects":73,"num_object_clones":12,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":44,"num_read":5159,"num_read_kb":13539,"num_write":640,"num_write_kb":3710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":115,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.145188+0000","last_change":"2024-09-14T23:56:25.093090+0000","last_active":"2024-09-14T23:57:20.145188+0000","last_peered":"2024-09-14T23:57:20.145188+0000","last_clean":"2024-09-14T23:57:20.145188+0000","last_became_active":"2024-09-14T23:56:22.528115+0000","last_became_peered":"2024-09-14T23:56:22.528115+0000","last_unstale":"2024-09-14T23:57:20.145188+0000","last_undegraded":"2024-09-14T23:57:20.145188+0000","last_fullsized":"2024-09-14T23:57:20.145188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:57:36.610730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036520999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"69'376","reported_seq":2103,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.145288+0000","last_change":"2024-09-14T23:57:13.112422+0000","last_active":"2024-09-14T23:57:20.145288+0000","last_peered":"2024-09-14T23:57:20.145288+0000","last_clean":"2024-09-14T23:57:20.145288+0000","last_became_active":"2024-09-14T23:57:13.111521+0000","last_became_peered":"2024-09-14T23:57:13.111521+0000","last_unstale":"2024-09-14T23:57:20.145288+0000","last_undegraded":"2024-09-14T23:57:20.145288+0000","last_fullsized":"2024-09-14T23:57:20.145288+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:00:23.524552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073009599999999996,"stat_sum":{"num_bytes":11422919,"num_objects":76,"num_object_clones":19,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":38,"num_read":2052,"num_read_kb":7652,"num_write":458,"num_write_kb":5227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":7555714,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":105,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-14T23:57:16.149326+0000","last_change":"2024-09-14T23:56:57.054999+0000","last_active":"2024-09-14T23:57:16.149326+0000","last_peered":"2024-09-14T23:57:16.149326+0000","last_clean":"2024-09-14T23:57:16.149326+0000","last_became_active":"2024-09-14T23:56:57.050729+0000","last_became_peered":"2024-09-14T23:56:57.050729+0000","last_unstale":"2024-09-14T23:57:16.149326+0000","last_undegraded":"2024-09-14T23:57:16.149326+0000","last_fullsized":"2024-09-14T23:57:16.149326+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:23:11.727903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":266,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.145200+0000","last_change":"2024-09-14T23:56:15.332936+0000","last_active":"2024-09-14T23:57:20.145200+0000","last_peered":"2024-09-14T23:57:20.145200+0000","last_clean":"2024-09-14T23:57:20.145200+0000","last_became_active":"2024-09-14T23:56:15.331994+0000","last_became_peered":"2024-09-14T23:56:15.331994+0000","last_unstale":"2024-09-14T23:57:20.145200+0000","last_undegraded":"2024-09-14T23:57:20.145200+0000","last_fullsized":"2024-09-14T23:57:20.145200+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:14.322882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:14.322882+0000","last_clean_scrub_stamp":"2024-09-14T23:56:14.322882+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:36:12.639000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"68'155","reported_seq":2555,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.145238+0000","last_change":"2024-09-14T23:56:34.415037+0000","last_active":"2024-09-14T23:57:20.145238+0000","last_peered":"2024-09-14T23:57:20.145238+0000","last_clean":"2024-09-14T23:57:20.145238+0000","last_became_active":"2024-09-14T23:56:32.403572+0000","last_became_peered":"2024-09-14T23:56:32.403572+0000","last_unstale":"2024-09-14T23:57:20.145238+0000","last_undegraded":"2024-09-14T23:57:20.145238+0000","last_fullsized":"2024-09-14T23:57:20.145238+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:18:47.652619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052643700000000004,"stat_sum":{"num_bytes":13852047,"num_objects":39,"num_object_clones":6,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":21,"num_read":2418,"num_read_kb":9967,"num_write":151,"num_write_kb":7232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"68'159","reported_seq":1424,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.146990+0000","last_change":"2024-09-14T23:56:34.404977+0000","last_active":"2024-09-14T23:57:20.146990+0000","last_peered":"2024-09-14T23:57:20.146990+0000","last_clean":"2024-09-14T23:57:20.146990+0000","last_became_active":"2024-09-14T23:56:32.400017+0000","last_became_peered":"2024-09-14T23:56:32.400017+0000","last_unstale":"2024-09-14T23:57:20.146990+0000","last_undegraded":"2024-09-14T23:57:20.146990+0000","last_fullsized":"2024-09-14T23:57:20.146990+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:37:53.000677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068884800000000004,"stat_sum":{"num_bytes":20761281,"num_objects":54,"num_object_clones":11,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":26,"num_read":1220,"num_read_kb":10531,"num_write":162,"num_write_kb":14658,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"68'319","reported_seq":6124,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.153272+0000","last_change":"2024-09-14T23:56:34.498020+0000","last_active":"2024-09-14T23:57:20.153272+0000","last_peered":"2024-09-14T23:57:20.153272+0000","last_clean":"2024-09-14T23:57:20.153272+0000","last_became_active":"2024-09-14T23:56:32.402769+0000","last_became_peered":"2024-09-14T23:56:32.402769+0000","last_unstale":"2024-09-14T23:57:20.153272+0000","last_undegraded":"2024-09-14T23:57:20.153272+0000","last_fullsized":"2024-09-14T23:57:20.153272+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:33:42.371132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035483999999999998,"stat_sum":{"num_bytes":19544689,"num_objects":54,"num_object_clones":8,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":30,"num_read":6477,"num_read_kb":24154,"num_write":405,"num_write_kb":6787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"68'407","reported_seq":12302,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.145228+0000","last_change":"2024-09-14T23:56:34.415414+0000","last_active":"2024-09-14T23:57:20.145228+0000","last_peered":"2024-09-14T23:57:20.145228+0000","last_clean":"2024-09-14T23:57:20.145228+0000","last_became_active":"2024-09-14T23:56:32.403231+0000","last_became_peered":"2024-09-14T23:56:32.403231+0000","last_unstale":"2024-09-14T23:57:20.145228+0000","last_undegraded":"2024-09-14T23:57:20.145228+0000","last_fullsized":"2024-09-14T23:57:20.145228+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:07:07.850750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072791000000000002,"stat_sum":{"num_bytes":36361843,"num_objects":54,"num_object_clones":12,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":27,"num_read":12925,"num_read_kb":55485,"num_write":544,"num_write_kb":7110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"68'310","reported_seq":12157,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.145174+0000","last_change":"2024-09-14T23:57:00.066551+0000","last_active":"2024-09-14T23:57:20.145174+0000","last_peered":"2024-09-14T23:57:20.145174+0000","last_clean":"2024-09-14T23:57:20.145174+0000","last_became_active":"2024-09-14T23:57:00.065790+0000","last_became_peered":"2024-09-14T23:57:00.065790+0000","last_unstale":"2024-09-14T23:57:20.145174+0000","last_undegraded":"2024-09-14T23:57:20.145174+0000","last_fullsized":"2024-09-14T23:57:20.145174+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":310,"log_dups_size":0,"ondisk_log_size":310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:12:47.374150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092638099999999999,"stat_sum":{"num_bytes":46489726,"num_objects":57,"num_object_clones":13,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":26,"num_read":11352,"num_read_kb":50252,"num_write":402,"num_write_kb":18596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":7912941,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"68'164","reported_seq":919,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.145473+0000","last_change":"2024-09-14T23:57:10.147907+0000","last_active":"2024-09-14T23:57:20.145473+0000","last_peered":"2024-09-14T23:57:20.145473+0000","last_clean":"2024-09-14T23:57:20.145473+0000","last_became_active":"2024-09-14T23:57:10.098202+0000","last_became_peered":"2024-09-14T23:57:10.098202+0000","last_unstale":"2024-09-14T23:57:20.145473+0000","last_undegraded":"2024-09-14T23:57:20.145473+0000","last_fullsized":"2024-09-14T23:57:20.145473+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:37:04.579298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057774300000000003,"stat_sum":{"num_bytes":8653118,"num_objects":44,"num_object_clones":7,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":24,"num_read":606,"num_read_kb":6074,"num_write":175,"num_write_kb":4151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":5634914,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"68'205","reported_seq":3038,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.145393+0000","last_change":"2024-09-14T23:56:34.399894+0000","last_active":"2024-09-14T23:57:20.145393+0000","last_peered":"2024-09-14T23:57:20.145393+0000","last_clean":"2024-09-14T23:57:20.145393+0000","last_became_active":"2024-09-14T23:56:32.406251+0000","last_became_peered":"2024-09-14T23:56:32.406251+0000","last_unstale":"2024-09-14T23:57:20.145393+0000","last_undegraded":"2024-09-14T23:57:20.145393+0000","last_fullsized":"2024-09-14T23:57:20.145393+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:38:57.846896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073115799999999998,"stat_sum":{"num_bytes":22285447,"num_objects":64,"num_object_clones":13,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":32,"num_read":3565,"num_read_kb":15855,"num_write":203,"num_write_kb":10186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"68'266","reported_seq":3164,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.145425+0000","last_change":"2024-09-14T23:56:34.399966+0000","last_active":"2024-09-14T23:57:20.145425+0000","last_peered":"2024-09-14T23:57:20.145425+0000","last_clean":"2024-09-14T23:57:20.145425+0000","last_became_active":"2024-09-14T23:56:32.407109+0000","last_became_peered":"2024-09-14T23:56:32.407109+0000","last_unstale":"2024-09-14T23:57:20.145425+0000","last_undegraded":"2024-09-14T23:57:20.145425+0000","last_fullsized":"2024-09-14T23:57:20.145425+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":266,"log_dups_size":0,"ondisk_log_size":266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:04:11.394853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072766900000000002,"stat_sum":{"num_bytes":28667677,"num_objects":62,"num_object_clones":11,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":31,"num_read":4327,"num_read_kb":11372,"num_write":240,"num_write_kb":6108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"69'321","reported_seq":787,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.199463+0000","last_change":"2024-09-14T23:56:34.498033+0000","last_active":"2024-09-14T23:57:20.199463+0000","last_peered":"2024-09-14T23:57:20.199463+0000","last_clean":"2024-09-14T23:57:20.199463+0000","last_became_active":"2024-09-14T23:56:32.402973+0000","last_became_peered":"2024-09-14T23:56:32.402973+0000","last_unstale":"2024-09-14T23:57:20.199463+0000","last_undegraded":"2024-09-14T23:57:20.199463+0000","last_fullsized":"2024-09-14T23:57:20.199463+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:03:53.757298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035072199999999998,"stat_sum":{"num_bytes":17521363,"num_objects":55,"num_object_clones":8,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":31,"num_read":1299,"num_read_kb":8913,"num_write":444,"num_write_kb":5822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"69'212","reported_seq":4973,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.161932+0000","last_change":"2024-09-14T23:56:34.399387+0000","last_active":"2024-09-14T23:57:20.161932+0000","last_peered":"2024-09-14T23:57:20.161932+0000","last_clean":"2024-09-14T23:57:20.161932+0000","last_became_active":"2024-09-14T23:56:32.405457+0000","last_became_peered":"2024-09-14T23:56:32.405457+0000","last_unstale":"2024-09-14T23:57:20.161932+0000","last_undegraded":"2024-09-14T23:57:20.161932+0000","last_fullsized":"2024-09-14T23:57:20.161932+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:14:07.750188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048125700000000002,"stat_sum":{"num_bytes":25343151,"num_objects":59,"num_object_clones":12,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":31,"num_read":5012,"num_read_kb":23215,"num_write":194,"num_write_kb":5929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"65'124","reported_seq":979,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-14T23:57:18.138153+0000","last_change":"2024-09-14T23:56:34.402592+0000","last_active":"2024-09-14T23:57:18.138153+0000","last_peered":"2024-09-14T23:57:18.138153+0000","last_clean":"2024-09-14T23:57:18.138153+0000","last_became_active":"2024-09-14T23:56:32.402782+0000","last_became_peered":"2024-09-14T23:56:32.402782+0000","last_unstale":"2024-09-14T23:57:18.138153+0000","last_undegraded":"2024-09-14T23:57:18.138153+0000","last_fullsized":"2024-09-14T23:57:18.138153+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:37:43.204611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001446697,"stat_sum":{"num_bytes":6063276,"num_objects":42,"num_object_clones":9,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":25,"num_read":795,"num_read_kb":3821,"num_write":103,"num_write_kb":2113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"68'534","reported_seq":2466,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.145315+0000","last_change":"2024-09-14T23:56:34.399629+0000","last_active":"2024-09-14T23:57:20.145315+0000","last_peered":"2024-09-14T23:57:20.145315+0000","last_clean":"2024-09-14T23:57:20.145315+0000","last_became_active":"2024-09-14T23:56:32.407220+0000","last_became_peered":"2024-09-14T23:56:32.407220+0000","last_unstale":"2024-09-14T23:57:20.145315+0000","last_undegraded":"2024-09-14T23:57:20.145315+0000","last_fullsized":"2024-09-14T23:57:20.145315+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:06.294375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062462200000000002,"stat_sum":{"num_bytes":15958982,"num_objects":54,"num_object_clones":13,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":27,"num_read":4215,"num_read_kb":15293,"num_write":817,"num_write_kb":9773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":623138349,"num_objects":1858,"num_object_clones":349,"num_object_copies":3716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1858,"num_whiteouts":979,"num_read":123950,"num_read_kb":508191,"num_write":10366,"num_write_kb":256186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":838,"num_bytes_recovered":37168120,"num_keys_recovered":121,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":42799104,"data_stored":410842298,"data_compressed":30166154,"data_compressed_allocated":33320960,"data_compressed_original":402997006,"omap_allocated":0,"internal_metadata":0},"log_size":8876,"ondisk_log_size":8876,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64594,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509458,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47620,"kb_used_data":8108,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94324220,"statfs":{"total":96636764160,"available":96588001280,"internally_reserved":0,"allocated":8302592,"data_stored":72536130,"data_compressed":5521083,"data_compressed_allocated":6008832,"data_compressed_original":70944105,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509457,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39272,"kb_used_data":3920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94332568,"statfs":{"total":96636764160,"available":96596549632,"internally_reserved":0,"allocated":4014080,"data_stored":27265247,"data_compressed":2172617,"data_compressed_allocated":2424832,"data_compressed_original":26250104,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,20,52,78,65],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44756,"kb_used_data":5248,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94327084,"statfs":{"total":96636764160,"available":96590934016,"internally_reserved":0,"allocated":5373952,"data_stored":47685560,"data_compressed":3461412,"data_compressed_allocated":3665920,"data_compressed_original":46584663,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509458,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35484,"kb_used_data":8448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94336356,"statfs":{"total":96636764160,"available":96600428544,"internally_reserved":0,"allocated":8650752,"data_stored":72653434,"data_compressed":5273757,"data_compressed_allocated":6004736,"data_compressed_original":70769396,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509456,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35084,"kb_used_data":8060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94336756,"statfs":{"total":96636764160,"available":96600838144,"internally_reserved":0,"allocated":8253440,"data_stored":76461622,"data_compressed":5558873,"data_compressed_allocated":5922816,"data_compressed_original":74878412,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542160,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44864,"kb_used_data":5356,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94326976,"statfs":{"total":96636764160,"available":96590823424,"internally_reserved":0,"allocated":5484544,"data_stored":48782711,"data_compressed":3343763,"data_compressed_allocated":3710976,"data_compressed_original":47572484,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542159,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32628,"kb_used_data":5604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339212,"statfs":{"total":96636764160,"available":96603353088,"internally_reserved":0,"allocated":5738496,"data_stored":47222860,"data_compressed":3150008,"data_compressed_allocated":3670016,"data_compressed_original":45776066,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38080,"kb_used_data":2736,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333760,"statfs":{"total":96636764160,"available":96597770240,"internally_reserved":0,"allocated":2801664,"data_stored":18612071,"data_compressed":1267012,"data_compressed_allocated":1445888,"data_compressed_original":17723216,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32297,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32297,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4972544,"data_stored":47205858,"data_compressed":3196339,"data_compressed_allocated":3719168,"data_compressed_original":46152898,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4464640,"data_stored":44927070,"data_compressed":3155066,"data_compressed_allocated":3465216,"data_compressed_original":44066308,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2052096,"data_stored":18273746,"data_compressed":1267012,"data_compressed_allocated":1445888,"data_compressed_original":17723216,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":7770112,"data_stored":78234235,"data_compressed":5784686,"data_compressed_allocated":6189056,"data_compressed_original":76975564,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8122368,"data_stored":75331873,"data_compressed":5532679,"data_compressed_allocated":6320128,"data_compressed_original":73857780,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4595712,"data_stored":47337995,"data_compressed":3461412,"data_compressed_allocated":3665920,"data_compressed_original":46584663,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3244032,"data_stored":26925543,"data_compressed":2172617,"data_compressed_allocated":2424832,"data_compressed_original":26250104,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":7577600,"data_stored":72605978,"data_compressed":5596343,"data_compressed_allocated":6090752,"data_compressed_original":71386473,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T23:57:22.492 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T23:57:22.531 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:22.528+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:22.549 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:22.549 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:22.558 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T23:57:22.594 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:22.592+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:22.665 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:22.665 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:22.673 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T23:57:22.717 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:22.716+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:22.780 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:22.780 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:22.789 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T23:57:22.818 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:57:22.819 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-14T23:57:22.831 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:22.829+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:22.831 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":99,"stamp":"2024-09-14T23:57:22.157002+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":623728736,"num_objects":1862,"num_object_clones":349,"num_object_copies":3724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1862,"num_whiteouts":979,"num_read":124066,"num_read_kb":508291,"num_write":10527,"num_write_kb":258604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":842,"num_bytes_recovered":37168158,"num_keys_recovered":121,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8996,"ondisk_log_size":8996,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":317788,"kb_used_data":47480,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754656932,"statfs":{"total":773094113280,"available":772768698368,"internally_reserved":0,"allocated":48619520,"data_stored":411219635,"data_compressed":29748525,"data_compressed_allocated":32854016,"data_compressed_original":400498446,"omap_allocated":12752,"internal_metadata":276614704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,20,52,78,65],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":45513001,"num_objects":498,"num_object_clones":43,"num_object_copies":996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":498,"num_whiteouts":421,"num_read":40628,"num_read_kb":145866,"num_write":1186,"num_write_kb":70896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.027405"},"pg_stats":[{"pgid":"3.1f","version":"67'428","reported_seq":2066,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-14T23:57:18.138713+0000","last_change":"2024-09-14T23:57:00.062727+0000","last_active":"2024-09-14T23:57:18.138713+0000","last_peered":"2024-09-14T23:57:18.138713+0000","last_clean":"2024-09-14T23:57:18.138713+0000","last_became_active":"2024-09-14T23:57:00.062116+0000","last_became_peered":"2024-09-14T23:57:00.062116+0000","last_unstale":"2024-09-14T23:57:18.138713+0000","last_undegraded":"2024-09-14T23:57:18.138713+0000","last_fullsized":"2024-09-14T23:57:18.138713+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:40:41.174328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015692659999999999,"stat_sum":{"num_bytes":23460714,"num_objects":66,"num_object_clones":11,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":34,"num_read":3498,"num_read_kb":8713,"num_write":533,"num_write_kb":19177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":2327658,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"68'182","reported_seq":1417,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.145376+0000","last_change":"2024-09-14T23:56:34.415748+0000","last_active":"2024-09-14T23:57:20.145376+0000","last_peered":"2024-09-14T23:57:20.145376+0000","last_clean":"2024-09-14T23:57:20.145376+0000","last_became_active":"2024-09-14T23:56:32.404385+0000","last_became_peered":"2024-09-14T23:56:32.404385+0000","last_unstale":"2024-09-14T23:57:20.145376+0000","last_undegraded":"2024-09-14T23:57:20.145376+0000","last_fullsized":"2024-09-14T23:57:20.145376+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:18:14.290947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010144430000000001,"stat_sum":{"num_bytes":13278889,"num_objects":64,"num_object_clones":15,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":31,"num_read":1245,"num_read_kb":9182,"num_write":169,"num_write_kb":8578,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"68'274","reported_seq":2745,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.147084+0000","last_change":"2024-09-14T23:57:00.062960+0000","last_active":"2024-09-14T23:57:20.147084+0000","last_peered":"2024-09-14T23:57:20.147084+0000","last_clean":"2024-09-14T23:57:20.147084+0000","last_became_active":"2024-09-14T23:57:00.062242+0000","last_became_peered":"2024-09-14T23:57:00.062242+0000","last_unstale":"2024-09-14T23:57:20.147084+0000","last_undegraded":"2024-09-14T23:57:20.147084+0000","last_fullsized":"2024-09-14T23:57:20.147084+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:44:00.321217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001148175,"stat_sum":{"num_bytes":29669808,"num_objects":67,"num_object_clones":15,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":34,"num_read":3214,"num_read_kb":13398,"num_write":249,"num_write_kb":11174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":1249963,"num_keys_recovered":34,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"68'297","reported_seq":3042,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.147123+0000","last_change":"2024-09-14T23:57:00.062694+0000","last_active":"2024-09-14T23:57:20.147123+0000","last_peered":"2024-09-14T23:57:20.147123+0000","last_clean":"2024-09-14T23:57:20.147123+0000","last_became_active":"2024-09-14T23:57:00.062126+0000","last_became_peered":"2024-09-14T23:57:00.062126+0000","last_unstale":"2024-09-14T23:57:20.147123+0000","last_undegraded":"2024-09-14T23:57:20.147123+0000","last_fullsized":"2024-09-14T23:57:20.147123+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:09:32.074708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095260199999999998,"stat_sum":{"num_bytes":9921574,"num_objects":52,"num_object_clones":7,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":30,"num_read":3528,"num_read_kb":10946,"num_write":351,"num_write_kb":5066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":2675491,"num_keys_recovered":40,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"65'143","reported_seq":608,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-14T23:57:18.181431+0000","last_change":"2024-09-14T23:56:34.401802+0000","last_active":"2024-09-14T23:57:18.181431+0000","last_peered":"2024-09-14T23:57:18.181431+0000","last_clean":"2024-09-14T23:57:18.181431+0000","last_became_active":"2024-09-14T23:56:32.398329+0000","last_became_peered":"2024-09-14T23:56:32.398329+0000","last_unstale":"2024-09-14T23:57:18.181431+0000","last_undegraded":"2024-09-14T23:57:18.181431+0000","last_fullsized":"2024-09-14T23:57:18.181431+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:28:01.396708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018313229999999999,"stat_sum":{"num_bytes":10931520,"num_objects":39,"num_object_clones":6,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":21,"num_read":391,"num_read_kb":7357,"num_write":129,"num_write_kb":4515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"68'231","reported_seq":2151,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.174636+0000","last_change":"2024-09-14T23:56:34.405093+0000","last_active":"2024-09-14T23:57:20.174636+0000","last_peered":"2024-09-14T23:57:20.174636+0000","last_clean":"2024-09-14T23:57:20.174636+0000","last_became_active":"2024-09-14T23:56:32.399655+0000","last_became_peered":"2024-09-14T23:56:32.399655+0000","last_unstale":"2024-09-14T23:57:20.174636+0000","last_undegraded":"2024-09-14T23:57:20.174636+0000","last_fullsized":"2024-09-14T23:57:20.174636+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:17:56.591978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096230199999999995,"stat_sum":{"num_bytes":15076848,"num_objects":71,"num_object_clones":15,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":43,"num_read":1922,"num_read_kb":10181,"num_write":217,"num_write_kb":13721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"66'202","reported_seq":1072,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-14T23:57:18.165959+0000","last_change":"2024-09-14T23:56:34.400617+0000","last_active":"2024-09-14T23:57:18.165959+0000","last_peered":"2024-09-14T23:57:18.165959+0000","last_clean":"2024-09-14T23:57:18.165959+0000","last_became_active":"2024-09-14T23:56:32.404364+0000","last_became_peered":"2024-09-14T23:56:32.404364+0000","last_unstale":"2024-09-14T23:57:18.165959+0000","last_undegraded":"2024-09-14T23:57:18.165959+0000","last_fullsized":"2024-09-14T23:57:18.165959+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:29:49.811207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001003264,"stat_sum":{"num_bytes":11096435,"num_objects":65,"num_object_clones":9,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":38,"num_read":967,"num_read_kb":4343,"num_write":188,"num_write_kb":3031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"68'244","reported_seq":2978,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.145133+0000","last_change":"2024-09-14T23:56:34.415021+0000","last_active":"2024-09-14T23:57:20.145133+0000","last_peered":"2024-09-14T23:57:20.145133+0000","last_clean":"2024-09-14T23:57:20.145133+0000","last_became_active":"2024-09-14T23:56:32.401799+0000","last_became_peered":"2024-09-14T23:56:32.401799+0000","last_unstale":"2024-09-14T23:57:20.145133+0000","last_undegraded":"2024-09-14T23:57:20.145133+0000","last_fullsized":"2024-09-14T23:57:20.145133+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":244,"log_dups_size":0,"ondisk_log_size":244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:01:57.971994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062985000000000005,"stat_sum":{"num_bytes":16572772,"num_objects":70,"num_object_clones":14,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":39,"num_read":3481,"num_read_kb":10912,"num_write":223,"num_write_kb":1906,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"66'186","reported_seq":1187,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-14T23:57:18.179656+0000","last_change":"2024-09-14T23:57:00.062824+0000","last_active":"2024-09-14T23:57:18.179656+0000","last_peered":"2024-09-14T23:57:18.179656+0000","last_clean":"2024-09-14T23:57:18.179656+0000","last_became_active":"2024-09-14T23:57:00.062304+0000","last_became_peered":"2024-09-14T23:57:00.062304+0000","last_unstale":"2024-09-14T23:57:18.179656+0000","last_undegraded":"2024-09-14T23:57:18.179656+0000","last_fullsized":"2024-09-14T23:57:18.179656+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:26:21.553898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014783859999999999,"stat_sum":{"num_bytes":7227960,"num_objects":48,"num_object_clones":9,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":24,"num_read":938,"num_read_kb":4320,"num_write":188,"num_write_kb":2947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":3345543,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"66'190","reported_seq":5531,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.145319+0000","last_change":"2024-09-14T23:57:13.115006+0000","last_active":"2024-09-14T23:57:20.145319+0000","last_peered":"2024-09-14T23:57:20.145319+0000","last_clean":"2024-09-14T23:57:20.145319+0000","last_became_active":"2024-09-14T23:57:13.114091+0000","last_became_peered":"2024-09-14T23:57:13.114091+0000","last_unstale":"2024-09-14T23:57:20.145319+0000","last_undegraded":"2024-09-14T23:57:20.145319+0000","last_fullsized":"2024-09-14T23:57:20.145319+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:33:49.799819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039416400000000002,"stat_sum":{"num_bytes":25452456,"num_objects":45,"num_object_clones":6,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":23,"num_read":5059,"num_read_kb":21361,"num_write":163,"num_write_kb":10062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":103,"num_bytes_recovered":3116309,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.15","version":"68'308","reported_seq":6939,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.145425+0000","last_change":"2024-09-14T23:56:34.399382+0000","last_active":"2024-09-14T23:57:20.145425+0000","last_peered":"2024-09-14T23:57:20.145425+0000","last_clean":"2024-09-14T23:57:20.145425+0000","last_became_active":"2024-09-14T23:56:32.405387+0000","last_became_peered":"2024-09-14T23:56:32.405387+0000","last_unstale":"2024-09-14T23:57:20.145425+0000","last_undegraded":"2024-09-14T23:57:20.145425+0000","last_fullsized":"2024-09-14T23:57:20.145425+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:17:00.235747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046328199999999997,"stat_sum":{"num_bytes":36776121,"num_objects":54,"num_object_clones":12,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":25,"num_read":7488,"num_read_kb":29301,"num_write":385,"num_write_kb":22562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"68'670","reported_seq":4989,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.147024+0000","last_change":"2024-09-14T23:56:34.404536+0000","last_active":"2024-09-14T23:57:20.147024+0000","last_peered":"2024-09-14T23:57:20.147024+0000","last_clean":"2024-09-14T23:57:20.147024+0000","last_became_active":"2024-09-14T23:56:32.403019+0000","last_became_peered":"2024-09-14T23:56:32.403019+0000","last_unstale":"2024-09-14T23:57:20.147024+0000","last_undegraded":"2024-09-14T23:57:20.147024+0000","last_fullsized":"2024-09-14T23:57:20.147024+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:36:36.583890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037505299999999997,"stat_sum":{"num_bytes":21573885,"num_objects":55,"num_object_clones":11,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":27,"num_read":7236,"num_read_kb":21793,"num_write":1131,"num_write_kb":5887,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":115,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.153322+0000","last_change":"2024-09-14T23:56:24.727061+0000","last_active":"2024-09-14T23:57:20.153322+0000","last_peered":"2024-09-14T23:57:20.153322+0000","last_clean":"2024-09-14T23:57:20.153322+0000","last_became_active":"2024-09-14T23:56:22.526915+0000","last_became_peered":"2024-09-14T23:56:22.526915+0000","last_unstale":"2024-09-14T23:57:20.153322+0000","last_undegraded":"2024-09-14T23:57:20.153322+0000","last_fullsized":"2024-09-14T23:57:20.153322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:43:15.359266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045105900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"66'247","reported_seq":2837,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-14T23:57:18.178135+0000","last_change":"2024-09-14T23:56:34.402234+0000","last_active":"2024-09-14T23:57:18.178135+0000","last_peered":"2024-09-14T23:57:18.178135+0000","last_clean":"2024-09-14T23:57:18.178135+0000","last_became_active":"2024-09-14T23:56:32.398664+0000","last_became_peered":"2024-09-14T23:56:32.398664+0000","last_unstale":"2024-09-14T23:57:18.178135+0000","last_undegraded":"2024-09-14T23:57:18.178135+0000","last_fullsized":"2024-09-14T23:57:18.178135+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:40:18.337645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012179999999999999,"stat_sum":{"num_bytes":14975451,"num_objects":48,"num_object_clones":6,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":23,"num_read":3872,"num_read_kb":13569,"num_write":244,"num_write_kb":4246,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":111,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-14T23:57:18.136220+0000","last_change":"2024-09-14T23:56:24.534632+0000","last_active":"2024-09-14T23:57:18.136220+0000","last_peered":"2024-09-14T23:57:18.136220+0000","last_clean":"2024-09-14T23:57:18.136220+0000","last_became_active":"2024-09-14T23:56:22.526355+0000","last_became_peered":"2024-09-14T23:56:22.526355+0000","last_unstale":"2024-09-14T23:57:18.136220+0000","last_undegraded":"2024-09-14T23:57:18.136220+0000","last_fullsized":"2024-09-14T23:57:18.136220+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:02:16.059627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049910000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"69'226","reported_seq":4798,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.189555+0000","last_change":"2024-09-14T23:56:34.416134+0000","last_active":"2024-09-14T23:57:20.189555+0000","last_peered":"2024-09-14T23:57:20.189555+0000","last_clean":"2024-09-14T23:57:20.189555+0000","last_became_active":"2024-09-14T23:56:32.404281+0000","last_became_peered":"2024-09-14T23:56:32.404281+0000","last_unstale":"2024-09-14T23:57:20.189555+0000","last_undegraded":"2024-09-14T23:57:20.189555+0000","last_fullsized":"2024-09-14T23:57:20.189555+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:37:18.248147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013337620000000001,"stat_sum":{"num_bytes":26122292,"num_objects":67,"num_object_clones":14,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":37,"num_read":5489,"num_read_kb":30567,"num_write":201,"num_write_kb":7732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":115,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.145540+0000","last_change":"2024-09-14T23:56:25.093112+0000","last_active":"2024-09-14T23:57:20.145540+0000","last_peered":"2024-09-14T23:57:20.145540+0000","last_clean":"2024-09-14T23:57:20.145540+0000","last_became_active":"2024-09-14T23:56:22.528211+0000","last_became_peered":"2024-09-14T23:56:22.528211+0000","last_unstale":"2024-09-14T23:57:20.145540+0000","last_undegraded":"2024-09-14T23:57:20.145540+0000","last_fullsized":"2024-09-14T23:57:20.145540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:14:11.495286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000459686,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"65'223","reported_seq":2461,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-14T23:57:18.174355+0000","last_change":"2024-09-14T23:56:34.401111+0000","last_active":"2024-09-14T23:57:18.174355+0000","last_peered":"2024-09-14T23:57:18.174355+0000","last_clean":"2024-09-14T23:57:18.174355+0000","last_became_active":"2024-09-14T23:56:32.401089+0000","last_became_peered":"2024-09-14T23:56:32.401089+0000","last_unstale":"2024-09-14T23:57:18.174355+0000","last_undegraded":"2024-09-14T23:57:18.174355+0000","last_fullsized":"2024-09-14T23:57:18.174355+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:39.493250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013033610000000001,"stat_sum":{"num_bytes":18464241,"num_objects":72,"num_object_clones":13,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":41,"num_read":2227,"num_read_kb":13887,"num_write":229,"num_write_kb":8679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":111,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-14T23:57:18.135987+0000","last_change":"2024-09-14T23:56:24.534510+0000","last_active":"2024-09-14T23:57:18.135987+0000","last_peered":"2024-09-14T23:57:18.135987+0000","last_clean":"2024-09-14T23:57:18.135987+0000","last_became_active":"2024-09-14T23:56:22.527611+0000","last_became_peered":"2024-09-14T23:56:22.527611+0000","last_unstale":"2024-09-14T23:57:18.135987+0000","last_undegraded":"2024-09-14T23:57:18.135987+0000","last_fullsized":"2024-09-14T23:57:18.135987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:15:45.606026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049916299999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"68'359","reported_seq":2290,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.145361+0000","last_change":"2024-09-14T23:57:00.065551+0000","last_active":"2024-09-14T23:57:20.145361+0000","last_peered":"2024-09-14T23:57:20.145361+0000","last_clean":"2024-09-14T23:57:20.145361+0000","last_became_active":"2024-09-14T23:57:00.064678+0000","last_became_peered":"2024-09-14T23:57:00.064678+0000","last_unstale":"2024-09-14T23:57:20.145361+0000","last_undegraded":"2024-09-14T23:57:20.145361+0000","last_fullsized":"2024-09-14T23:57:20.145361+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:54:22.755339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001272947,"stat_sum":{"num_bytes":12132806,"num_objects":74,"num_object_clones":15,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":39,"num_read":2794,"num_read_kb":11298,"num_write":430,"num_write_kb":8654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":3349587,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"69'223","reported_seq":2075,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.196275+0000","last_change":"2024-09-14T23:56:34.404751+0000","last_active":"2024-09-14T23:57:20.196275+0000","last_peered":"2024-09-14T23:57:20.196275+0000","last_clean":"2024-09-14T23:57:20.196275+0000","last_became_active":"2024-09-14T23:56:32.397883+0000","last_became_peered":"2024-09-14T23:56:32.397883+0000","last_unstale":"2024-09-14T23:57:20.196275+0000","last_undegraded":"2024-09-14T23:57:20.196275+0000","last_fullsized":"2024-09-14T23:57:20.196275+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:38:47.008304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048955799999999996,"stat_sum":{"num_bytes":20549548,"num_objects":64,"num_object_clones":11,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":30,"num_read":2766,"num_read_kb":10503,"num_write":191,"num_write_kb":4407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":122,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-14T23:57:18.136327+0000","last_change":"2024-09-14T23:56:57.052969+0000","last_active":"2024-09-14T23:57:18.136327+0000","last_peered":"2024-09-14T23:57:18.136327+0000","last_clean":"2024-09-14T23:57:18.136327+0000","last_became_active":"2024-09-14T23:56:57.050456+0000","last_became_peered":"2024-09-14T23:56:57.050456+0000","last_unstale":"2024-09-14T23:57:18.136327+0000","last_undegraded":"2024-09-14T23:57:18.136327+0000","last_fullsized":"2024-09-14T23:57:18.136327+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:57:07.644011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"65'216","reported_seq":6554,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-14T23:57:18.200629+0000","last_change":"2024-09-14T23:56:34.401159+0000","last_active":"2024-09-14T23:57:18.200629+0000","last_peered":"2024-09-14T23:57:18.200629+0000","last_clean":"2024-09-14T23:57:18.200629+0000","last_became_active":"2024-09-14T23:56:32.400278+0000","last_became_peered":"2024-09-14T23:56:32.400278+0000","last_unstale":"2024-09-14T23:57:18.200629+0000","last_undegraded":"2024-09-14T23:57:18.200629+0000","last_fullsized":"2024-09-14T23:57:18.200629+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:40:29.250653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082507200000000004,"stat_sum":{"num_bytes":25177953,"num_objects":50,"num_object_clones":6,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":27,"num_read":6413,"num_read_kb":20437,"num_write":207,"num_write_kb":6440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":107,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-14T23:57:16.149408+0000","last_change":"2024-09-14T23:56:24.909260+0000","last_active":"2024-09-14T23:57:16.149408+0000","last_peered":"2024-09-14T23:57:16.149408+0000","last_clean":"2024-09-14T23:57:16.149408+0000","last_became_active":"2024-09-14T23:56:22.822598+0000","last_became_peered":"2024-09-14T23:56:22.822598+0000","last_unstale":"2024-09-14T23:57:16.149408+0000","last_undegraded":"2024-09-14T23:57:16.149408+0000","last_fullsized":"2024-09-14T23:57:16.149408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:32:32.568028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026863400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"66'475","reported_seq":3535,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-14T23:57:18.137879+0000","last_change":"2024-09-14T23:56:34.401366+0000","last_active":"2024-09-14T23:57:18.137879+0000","last_peered":"2024-09-14T23:57:18.137879+0000","last_clean":"2024-09-14T23:57:18.137879+0000","last_became_active":"2024-09-14T23:56:32.397551+0000","last_became_peered":"2024-09-14T23:56:32.397551+0000","last_unstale":"2024-09-14T23:57:18.137879+0000","last_undegraded":"2024-09-14T23:57:18.137879+0000","last_fullsized":"2024-09-14T23:57:18.137879+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":475,"log_dups_size":0,"ondisk_log_size":475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:32:09.324100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001418816,"stat_sum":{"num_bytes":11751557,"num_objects":73,"num_object_clones":12,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":44,"num_read":5159,"num_read_kb":13539,"num_write":640,"num_write_kb":3710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":115,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.145188+0000","last_change":"2024-09-14T23:56:25.093090+0000","last_active":"2024-09-14T23:57:20.145188+0000","last_peered":"2024-09-14T23:57:20.145188+0000","last_clean":"2024-09-14T23:57:20.145188+0000","last_became_active":"2024-09-14T23:56:22.528115+0000","last_became_peered":"2024-09-14T23:56:22.528115+0000","last_unstale":"2024-09-14T23:57:20.145188+0000","last_undegraded":"2024-09-14T23:57:20.145188+0000","last_fullsized":"2024-09-14T23:57:20.145188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:57:36.610730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036520999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"69'376","reported_seq":2103,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.145288+0000","last_change":"2024-09-14T23:57:13.112422+0000","last_active":"2024-09-14T23:57:20.145288+0000","last_peered":"2024-09-14T23:57:20.145288+0000","last_clean":"2024-09-14T23:57:20.145288+0000","last_became_active":"2024-09-14T23:57:13.111521+0000","last_became_peered":"2024-09-14T23:57:13.111521+0000","last_unstale":"2024-09-14T23:57:20.145288+0000","last_undegraded":"2024-09-14T23:57:20.145288+0000","last_fullsized":"2024-09-14T23:57:20.145288+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:00:23.524552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073009599999999996,"stat_sum":{"num_bytes":11422919,"num_objects":76,"num_object_clones":19,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":38,"num_read":2052,"num_read_kb":7652,"num_write":458,"num_write_kb":5227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":7555714,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":105,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-14T23:57:16.149326+0000","last_change":"2024-09-14T23:56:57.054999+0000","last_active":"2024-09-14T23:57:16.149326+0000","last_peered":"2024-09-14T23:57:16.149326+0000","last_clean":"2024-09-14T23:57:16.149326+0000","last_became_active":"2024-09-14T23:56:57.050729+0000","last_became_peered":"2024-09-14T23:56:57.050729+0000","last_unstale":"2024-09-14T23:57:16.149326+0000","last_undegraded":"2024-09-14T23:57:16.149326+0000","last_fullsized":"2024-09-14T23:57:16.149326+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:23:11.727903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":266,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.145200+0000","last_change":"2024-09-14T23:56:15.332936+0000","last_active":"2024-09-14T23:57:20.145200+0000","last_peered":"2024-09-14T23:57:20.145200+0000","last_clean":"2024-09-14T23:57:20.145200+0000","last_became_active":"2024-09-14T23:56:15.331994+0000","last_became_peered":"2024-09-14T23:56:15.331994+0000","last_unstale":"2024-09-14T23:57:20.145200+0000","last_undegraded":"2024-09-14T23:57:20.145200+0000","last_fullsized":"2024-09-14T23:57:20.145200+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:14.322882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:14.322882+0000","last_clean_scrub_stamp":"2024-09-14T23:56:14.322882+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:36:12.639000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"68'155","reported_seq":2555,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.145238+0000","last_change":"2024-09-14T23:56:34.415037+0000","last_active":"2024-09-14T23:57:20.145238+0000","last_peered":"2024-09-14T23:57:20.145238+0000","last_clean":"2024-09-14T23:57:20.145238+0000","last_became_active":"2024-09-14T23:56:32.403572+0000","last_became_peered":"2024-09-14T23:56:32.403572+0000","last_unstale":"2024-09-14T23:57:20.145238+0000","last_undegraded":"2024-09-14T23:57:20.145238+0000","last_fullsized":"2024-09-14T23:57:20.145238+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:18:47.652619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052643700000000004,"stat_sum":{"num_bytes":13852047,"num_objects":39,"num_object_clones":6,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":21,"num_read":2418,"num_read_kb":9967,"num_write":151,"num_write_kb":7232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"68'159","reported_seq":1424,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.146990+0000","last_change":"2024-09-14T23:56:34.404977+0000","last_active":"2024-09-14T23:57:20.146990+0000","last_peered":"2024-09-14T23:57:20.146990+0000","last_clean":"2024-09-14T23:57:20.146990+0000","last_became_active":"2024-09-14T23:56:32.400017+0000","last_became_peered":"2024-09-14T23:56:32.400017+0000","last_unstale":"2024-09-14T23:57:20.146990+0000","last_undegraded":"2024-09-14T23:57:20.146990+0000","last_fullsized":"2024-09-14T23:57:20.146990+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:37:53.000677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068884800000000004,"stat_sum":{"num_bytes":20761281,"num_objects":54,"num_object_clones":11,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":26,"num_read":1220,"num_read_kb":10531,"num_write":162,"num_write_kb":14658,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"68'319","reported_seq":6124,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.153272+0000","last_change":"2024-09-14T23:56:34.498020+0000","last_active":"2024-09-14T23:57:20.153272+0000","last_peered":"2024-09-14T23:57:20.153272+0000","last_clean":"2024-09-14T23:57:20.153272+0000","last_became_active":"2024-09-14T23:56:32.402769+0000","last_became_peered":"2024-09-14T23:56:32.402769+0000","last_unstale":"2024-09-14T23:57:20.153272+0000","last_undegraded":"2024-09-14T23:57:20.153272+0000","last_fullsized":"2024-09-14T23:57:20.153272+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:33:42.371132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035483999999999998,"stat_sum":{"num_bytes":19544689,"num_objects":54,"num_object_clones":8,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":30,"num_read":6477,"num_read_kb":24154,"num_write":405,"num_write_kb":6787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"68'407","reported_seq":12302,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.145228+0000","last_change":"2024-09-14T23:56:34.415414+0000","last_active":"2024-09-14T23:57:20.145228+0000","last_peered":"2024-09-14T23:57:20.145228+0000","last_clean":"2024-09-14T23:57:20.145228+0000","last_became_active":"2024-09-14T23:56:32.403231+0000","last_became_peered":"2024-09-14T23:56:32.403231+0000","last_unstale":"2024-09-14T23:57:20.145228+0000","last_undegraded":"2024-09-14T23:57:20.145228+0000","last_fullsized":"2024-09-14T23:57:20.145228+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:07:07.850750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072791000000000002,"stat_sum":{"num_bytes":36361843,"num_objects":54,"num_object_clones":12,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":27,"num_read":12925,"num_read_kb":55485,"num_write":544,"num_write_kb":7110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"68'310","reported_seq":12157,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.145174+0000","last_change":"2024-09-14T23:57:00.066551+0000","last_active":"2024-09-14T23:57:20.145174+0000","last_peered":"2024-09-14T23:57:20.145174+0000","last_clean":"2024-09-14T23:57:20.145174+0000","last_became_active":"2024-09-14T23:57:00.065790+0000","last_became_peered":"2024-09-14T23:57:00.065790+0000","last_unstale":"2024-09-14T23:57:20.145174+0000","last_undegraded":"2024-09-14T23:57:20.145174+0000","last_fullsized":"2024-09-14T23:57:20.145174+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":310,"log_dups_size":0,"ondisk_log_size":310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:12:47.374150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092638099999999999,"stat_sum":{"num_bytes":46489726,"num_objects":57,"num_object_clones":13,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":26,"num_read":11352,"num_read_kb":50252,"num_write":402,"num_write_kb":18596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":7912941,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"68'164","reported_seq":919,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.145473+0000","last_change":"2024-09-14T23:57:10.147907+0000","last_active":"2024-09-14T23:57:20.145473+0000","last_peered":"2024-09-14T23:57:20.145473+0000","last_clean":"2024-09-14T23:57:20.145473+0000","last_became_active":"2024-09-14T23:57:10.098202+0000","last_became_peered":"2024-09-14T23:57:10.098202+0000","last_unstale":"2024-09-14T23:57:20.145473+0000","last_undegraded":"2024-09-14T23:57:20.145473+0000","last_fullsized":"2024-09-14T23:57:20.145473+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:37:04.579298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057774300000000003,"stat_sum":{"num_bytes":8653118,"num_objects":44,"num_object_clones":7,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":24,"num_read":606,"num_read_kb":6074,"num_write":175,"num_write_kb":4151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":5634914,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"68'205","reported_seq":3038,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.145393+0000","last_change":"2024-09-14T23:56:34.399894+0000","last_active":"2024-09-14T23:57:20.145393+0000","last_peered":"2024-09-14T23:57:20.145393+0000","last_clean":"2024-09-14T23:57:20.145393+0000","last_became_active":"2024-09-14T23:56:32.406251+0000","last_became_peered":"2024-09-14T23:56:32.406251+0000","last_unstale":"2024-09-14T23:57:20.145393+0000","last_undegraded":"2024-09-14T23:57:20.145393+0000","last_fullsized":"2024-09-14T23:57:20.145393+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:38:57.846896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073115799999999998,"stat_sum":{"num_bytes":22285447,"num_objects":64,"num_object_clones":13,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":32,"num_read":3565,"num_read_kb":15855,"num_write":203,"num_write_kb":10186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"68'266","reported_seq":3164,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.145425+0000","last_change":"2024-09-14T23:56:34.399966+0000","last_active":"2024-09-14T23:57:20.145425+0000","last_peered":"2024-09-14T23:57:20.145425+0000","last_clean":"2024-09-14T23:57:20.145425+0000","last_became_active":"2024-09-14T23:56:32.407109+0000","last_became_peered":"2024-09-14T23:56:32.407109+0000","last_unstale":"2024-09-14T23:57:20.145425+0000","last_undegraded":"2024-09-14T23:57:20.145425+0000","last_fullsized":"2024-09-14T23:57:20.145425+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":266,"log_dups_size":0,"ondisk_log_size":266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:04:11.394853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072766900000000002,"stat_sum":{"num_bytes":28667677,"num_objects":62,"num_object_clones":11,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":31,"num_read":4327,"num_read_kb":11372,"num_write":240,"num_write_kb":6108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"69'321","reported_seq":787,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.199463+0000","last_change":"2024-09-14T23:56:34.498033+0000","last_active":"2024-09-14T23:57:20.199463+0000","last_peered":"2024-09-14T23:57:20.199463+0000","last_clean":"2024-09-14T23:57:20.199463+0000","last_became_active":"2024-09-14T23:56:32.402973+0000","last_became_peered":"2024-09-14T23:56:32.402973+0000","last_unstale":"2024-09-14T23:57:20.199463+0000","last_undegraded":"2024-09-14T23:57:20.199463+0000","last_fullsized":"2024-09-14T23:57:20.199463+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:03:53.757298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035072199999999998,"stat_sum":{"num_bytes":17521363,"num_objects":55,"num_object_clones":8,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":31,"num_read":1299,"num_read_kb":8913,"num_write":444,"num_write_kb":5822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"69'212","reported_seq":4973,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.161932+0000","last_change":"2024-09-14T23:56:34.399387+0000","last_active":"2024-09-14T23:57:20.161932+0000","last_peered":"2024-09-14T23:57:20.161932+0000","last_clean":"2024-09-14T23:57:20.161932+0000","last_became_active":"2024-09-14T23:56:32.405457+0000","last_became_peered":"2024-09-14T23:56:32.405457+0000","last_unstale":"2024-09-14T23:57:20.161932+0000","last_undegraded":"2024-09-14T23:57:20.161932+0000","last_fullsized":"2024-09-14T23:57:20.161932+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:14:07.750188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048125700000000002,"stat_sum":{"num_bytes":25343151,"num_objects":59,"num_object_clones":12,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":31,"num_read":5012,"num_read_kb":23215,"num_write":194,"num_write_kb":5929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"65'124","reported_seq":979,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-14T23:57:18.138153+0000","last_change":"2024-09-14T23:56:34.402592+0000","last_active":"2024-09-14T23:57:18.138153+0000","last_peered":"2024-09-14T23:57:18.138153+0000","last_clean":"2024-09-14T23:57:18.138153+0000","last_became_active":"2024-09-14T23:56:32.402782+0000","last_became_peered":"2024-09-14T23:56:32.402782+0000","last_unstale":"2024-09-14T23:57:18.138153+0000","last_undegraded":"2024-09-14T23:57:18.138153+0000","last_fullsized":"2024-09-14T23:57:18.138153+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:37:43.204611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001446697,"stat_sum":{"num_bytes":6063276,"num_objects":42,"num_object_clones":9,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":25,"num_read":795,"num_read_kb":3821,"num_write":103,"num_write_kb":2113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"68'534","reported_seq":2466,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.145315+0000","last_change":"2024-09-14T23:56:34.399629+0000","last_active":"2024-09-14T23:57:20.145315+0000","last_peered":"2024-09-14T23:57:20.145315+0000","last_clean":"2024-09-14T23:57:20.145315+0000","last_became_active":"2024-09-14T23:56:32.407220+0000","last_became_peered":"2024-09-14T23:56:32.407220+0000","last_unstale":"2024-09-14T23:57:20.145315+0000","last_undegraded":"2024-09-14T23:57:20.145315+0000","last_fullsized":"2024-09-14T23:57:20.145315+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:06.294375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062462200000000002,"stat_sum":{"num_bytes":15958982,"num_objects":54,"num_object_clones":13,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":27,"num_read":4215,"num_read_kb":15293,"num_write":817,"num_write_kb":9773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":623138349,"num_objects":1858,"num_object_clones":349,"num_object_copies":3716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1858,"num_whiteouts":979,"num_read":123950,"num_read_kb":508191,"num_write":10366,"num_write_kb":256186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":838,"num_bytes_recovered":37168120,"num_keys_recovered":121,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":42799104,"data_stored":410842298,"data_compressed":30166154,"data_compressed_allocated":33320960,"data_compressed_original":402997006,"omap_allocated":0,"internal_metadata":0},"log_size":8876,"ondisk_log_size":8876,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64594,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509458,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47620,"kb_used_data":8108,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94324220,"statfs":{"total":96636764160,"available":96588001280,"internally_reserved":0,"allocated":8302592,"data_stored":72536130,"data_compressed":5521083,"data_compressed_allocated":6008832,"data_compressed_original":70944105,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509457,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39272,"kb_used_data":3920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94332568,"statfs":{"total":96636764160,"available":96596549632,"internally_reserved":0,"allocated":4014080,"data_stored":27265247,"data_compressed":2172617,"data_compressed_allocated":2424832,"data_compressed_original":26250104,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,20,52,78,65],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44756,"kb_used_data":5248,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94327084,"statfs":{"total":96636764160,"available":96590934016,"internally_reserved":0,"allocated":5373952,"data_stored":47685560,"data_compressed":3461412,"data_compressed_allocated":3665920,"data_compressed_original":46584663,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509458,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35484,"kb_used_data":8448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94336356,"statfs":{"total":96636764160,"available":96600428544,"internally_reserved":0,"allocated":8650752,"data_stored":72653434,"data_compressed":5273757,"data_compressed_allocated":6004736,"data_compressed_original":70769396,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509456,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35084,"kb_used_data":8060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94336756,"statfs":{"total":96636764160,"available":96600838144,"internally_reserved":0,"allocated":8253440,"data_stored":76461622,"data_compressed":5558873,"data_compressed_allocated":5922816,"data_compressed_original":74878412,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542160,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44864,"kb_used_data":5356,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94326976,"statfs":{"total":96636764160,"available":96590823424,"internally_reserved":0,"allocated":5484544,"data_stored":48782711,"data_compressed":3343763,"data_compressed_allocated":3710976,"data_compressed_original":47572484,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542159,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32628,"kb_used_data":5604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339212,"statfs":{"total":96636764160,"available":96603353088,"internally_reserved":0,"allocated":5738496,"data_stored":47222860,"data_compressed":3150008,"data_compressed_allocated":3670016,"data_compressed_original":45776066,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38080,"kb_used_data":2736,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333760,"statfs":{"total":96636764160,"available":96597770240,"internally_reserved":0,"allocated":2801664,"data_stored":18612071,"data_compressed":1267012,"data_compressed_allocated":1445888,"data_compressed_original":17723216,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32297,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32297,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4972544,"data_stored":47205858,"data_compressed":3196339,"data_compressed_allocated":3719168,"data_compressed_original":46152898,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4464640,"data_stored":44927070,"data_compressed":3155066,"data_compressed_allocated":3465216,"data_compressed_original":44066308,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2052096,"data_stored":18273746,"data_compressed":1267012,"data_compressed_allocated":1445888,"data_compressed_original":17723216,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":7770112,"data_stored":78234235,"data_compressed":5784686,"data_compressed_allocated":6189056,"data_compressed_original":76975564,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8122368,"data_stored":75331873,"data_compressed":5532679,"data_compressed_allocated":6320128,"data_compressed_original":73857780,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4595712,"data_stored":47337995,"data_compressed":3461412,"data_compressed_allocated":3665920,"data_compressed_original":46584663,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3244032,"data_stored":26925543,"data_compressed":2172617,"data_compressed_allocated":2424832,"data_compressed_original":26250104,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":7577600,"data_stored":72605978,"data_compressed":5596343,"data_compressed_allocated":6090752,"data_compressed_original":71386473,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T23:57:22.833 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-14T23:57:22.833 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T23:57:22.833 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-14T23:57:22.833 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T23:57:22.896 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:22.895+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:22.937 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:22.937 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:22.945 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T23:57:23.029 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:23.028+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:23.093 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:23.093 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:23.099 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:23.096+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:23.101 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T23:57:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:57:23.164 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 72, 2024-09-14T23:57:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "21916fcc-f95c-4bfa-8b8b-11ca45845613", 2024-09-14T23:57:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: "created": "2024-09-14T23:56:03.167948+0000", 2024-09-14T23:57:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-09-14T23:57:22.961289+0000", 2024-09-14T23:57:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: "last_up_change": "2024-09-14T23:56:14.322882+0000", 2024-09-14T23:57:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: "last_in_change": "2024-09-14T23:56:56.035972+0000", 2024-09-14T23:57:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T23:57:23.164 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_num": 5806080, 2024-09-14T23:57:23.165 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_set": [ 2024-09-14T23:57:23.165 INFO:teuthology.orchestra.run.smithi042.stdout: "nodeep-scrub", 2024-09-14T23:57:23.165 INFO:teuthology.orchestra.run.smithi042.stdout: "noscrub", 2024-09-14T23:57:23.165 INFO:teuthology.orchestra.run.smithi042.stdout: "pglog_hardlimit", 2024-09-14T23:57:23.165 INFO:teuthology.orchestra.run.smithi042.stdout: "purged_snapdirs", 2024-09-14T23:57:23.165 INFO:teuthology.orchestra.run.smithi042.stdout: "recovery_deletes", 2024-09-14T23:57:23.165 INFO:teuthology.orchestra.run.smithi042.stdout: "sortbitwise" 2024-09-14T23:57:23.165 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-14T23:57:23.165 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_version": 6, 2024-09-14T23:57:23.165 INFO:teuthology.orchestra.run.smithi042.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T23:57:23.165 INFO:teuthology.orchestra.run.smithi042.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T23:57:23.165 INFO:teuthology.orchestra.run.smithi042.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T23:57:23.165 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_snapshot": "", 2024-09-14T23:57:23.166 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_max": 3, 2024-09-14T23:57:23.166 INFO:teuthology.orchestra.run.smithi042.stdout: "max_osd": 8, 2024-09-14T23:57:23.166 INFO:teuthology.orchestra.run.smithi042.stdout: "require_min_compat_client": "luminous", 2024-09-14T23:57:23.166 INFO:teuthology.orchestra.run.smithi042.stdout: "min_compat_client": "jewel", 2024-09-14T23:57:23.166 INFO:teuthology.orchestra.run.smithi042.stdout: "require_osd_release": "squid", 2024-09-14T23:57:23.166 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_crimson": false, 2024-09-14T23:57:23.166 INFO:teuthology.orchestra.run.smithi042.stdout: "pools": [ 2024-09-14T23:57:23.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.166 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 1, 2024-09-14T23:57:23.166 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": ".mgr", 2024-09-14T23:57:23.166 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-14T23:56:13.500197+0000", 2024-09-14T23:57:23.166 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-09-14T23:57:23.166 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool", 2024-09-14T23:57:23.167 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-14T23:57:23.167 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-14T23:57:23.167 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-14T23:57:23.167 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-14T23:57:23.167 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-14T23:57:23.167 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-14T23:57:23.167 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T23:57:23.167 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T23:57:23.167 INFO:teuthology.orchestra.run.smithi042.stdout: "is_stretch_pool": false, 2024-09-14T23:57:23.167 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-14T23:57:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-14T23:57:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 1, 2024-09-14T23:57:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 1, 2024-09-14T23:57:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 1, 2024-09-14T23:57:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 1, 2024-09-14T23:57:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 1, 2024-09-14T23:57:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-14T23:57:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-14T23:57:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-14T23:57:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-14T23:57:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-14T23:57:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-14T23:57:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-14T23:57:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "17", 2024-09-14T23:57:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "0", 2024-09-14T23:57:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T23:57:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T23:57:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-14T23:57:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-14T23:57:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 0, 2024-09-14T23:57:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 0, 2024-09-14T23:57:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-14T23:57:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-14T23:57:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-14T23:57:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-14T23:57:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-14T23:57:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-14T23:57:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-14T23:57:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-14T23:57:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-14T23:57:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-14T23:57:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-14T23:57:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T23:57:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T23:57:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T23:57:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-14T23:57:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-14T23:57:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-14T23:57:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-14T23:57:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-14T23:57:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-14T23:57:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-14T23:57:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-14T23:57:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-14T23:57:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-14T23:57:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T23:57:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-14T23:57:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-14T23:57:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-14T23:57:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-14T23:57:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-14T23:57:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: "options": { 2024-09-14T23:57:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_max": 32, 2024-09-14T23:57:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_min": 1 2024-09-14T23:57:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-14T23:57:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": {} 2024-09-14T23:57:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: "read_balance": { 2024-09-14T23:57:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: "score_type": "Fair distribution", 2024-09-14T23:57:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: "score_acting": 6.9000000953674316, 2024-09-14T23:57:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: "score_stable": 6.9000000953674316, 2024-09-14T23:57:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: "optimal_score": 0.28999999165534973, 2024-09-14T23:57:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_acting": 2, 2024-09-14T23:57:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_stable": 2, 2024-09-14T23:57:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity_weighted": 0.8712921142578125, 2024-09-14T23:57:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity": 1, 2024-09-14T23:57:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity_weighted": 0.87000000476837158 2024-09-14T23:57:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 2, 2024-09-14T23:57:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": "rbd", 2024-09-14T23:57:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-14T23:56:21.284705+0000", 2024-09-14T23:57:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 8193, 2024-09-14T23:57:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T23:57:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-14T23:57:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-14T23:57:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-14T23:57:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-14T23:57:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-14T23:57:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-14T23:57:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T23:57:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T23:57:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: "is_stretch_pool": false, 2024-09-14T23:57:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-14T23:57:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-14T23:57:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 8, 2024-09-14T23:57:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 8, 2024-09-14T23:57:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 8, 2024-09-14T23:57:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 8, 2024-09-14T23:57:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 8, 2024-09-14T23:57:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-14T23:57:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-14T23:57:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-14T23:57:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-14T23:57:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-14T23:57:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-14T23:57:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-14T23:57:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "21", 2024-09-14T23:57:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "0", 2024-09-14T23:57:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T23:57:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T23:57:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-14T23:57:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-14T23:57:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 2, 2024-09-14T23:57:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 21, 2024-09-14T23:57:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-14T23:57:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-14T23:57:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-14T23:57:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-14T23:57:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-14T23:57:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-14T23:57:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-14T23:57:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-14T23:57:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-14T23:57:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-14T23:57:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-14T23:57:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T23:57:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T23:57:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T23:57:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-14T23:57:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-14T23:57:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-14T23:57:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-14T23:57:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-14T23:57:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-14T23:57:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-14T23:57:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-14T23:57:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-14T23:57:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-14T23:57:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T23:57:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-14T23:57:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-14T23:57:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-14T23:57:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-14T23:57:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-14T23:57:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: "options": {}, 2024-09-14T23:57:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-14T23:57:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd": {} 2024-09-14T23:57:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: "read_balance": { 2024-09-14T23:57:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: "score_type": "Fair distribution", 2024-09-14T23:57:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: "score_acting": 2.630000114440918, 2024-09-14T23:57:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: "score_stable": 2.630000114440918, 2024-09-14T23:57:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: "optimal_score": 1, 2024-09-14T23:57:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_acting": 2.630000114440918, 2024-09-14T23:57:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_stable": 2.630000114440918, 2024-09-14T23:57:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity_weighted": 0.8712921142578125, 2024-09-14T23:57:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity": 1, 2024-09-14T23:57:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity_weighted": 0.87000000476837158 2024-09-14T23:57:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 3, 2024-09-14T23:57:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": "pool_client.0", 2024-09-14T23:57:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-14T23:56:30.861504+0000", 2024-09-14T23:57:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 8193, 2024-09-14T23:57:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T23:57:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-14T23:57:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-14T23:57:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-14T23:57:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-14T23:57:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-14T23:57:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-14T23:57:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T23:57:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T23:57:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: "is_stretch_pool": false, 2024-09-14T23:57:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-14T23:57:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-14T23:57:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 32, 2024-09-14T23:57:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 32, 2024-09-14T23:57:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 32, 2024-09-14T23:57:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 32, 2024-09-14T23:57:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 32, 2024-09-14T23:57:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-14T23:57:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-14T23:57:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-14T23:57:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-14T23:57:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-14T23:57:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-14T23:57:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-14T23:57:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "70", 2024-09-14T23:57:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "0", 2024-09-14T23:57:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T23:57:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T23:57:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-14T23:57:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-14T23:57:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 27, 2024-09-14T23:57:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 70, 2024-09-14T23:57:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-14T23:57:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-14T23:57:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-14T23:57:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-14T23:57:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-14T23:57:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-14T23:57:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-14T23:57:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-14T23:57:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-14T23:57:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-14T23:57:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-14T23:57:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T23:57:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T23:57:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T23:57:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-14T23:57:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-14T23:57:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-14T23:57:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-14T23:57:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-14T23:57:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-14T23:57:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-14T23:57:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-14T23:57:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-14T23:57:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-14T23:57:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T23:57:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-14T23:57:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-14T23:57:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-14T23:57:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-14T23:57:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-14T23:57:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: "options": {}, 2024-09-14T23:57:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-14T23:57:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd": {} 2024-09-14T23:57:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: "read_balance": { 2024-09-14T23:57:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: "score_type": "Fair distribution", 2024-09-14T23:57:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: "score_acting": 1.75, 2024-09-14T23:57:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: "score_stable": 1.75, 2024-09-14T23:57:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: "optimal_score": 1.1399999856948853, 2024-09-14T23:57:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_acting": 2, 2024-09-14T23:57:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_stable": 2, 2024-09-14T23:57:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity_weighted": 0.8712921142578125, 2024-09-14T23:57:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity": 1, 2024-09-14T23:57:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity_weighted": 0.87000000476837158 2024-09-14T23:57:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-14T23:57:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: "osds": [ 2024-09-14T23:57:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 0, 2024-09-14T23:57:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "db9d0c67-2d60-4592-9ad3-f99e5294f93f", 2024-09-14T23:57:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-14T23:57:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-14T23:57:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-14T23:57:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-14T23:57:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-14T23:57:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-14T23:57:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 14, 2024-09-14T23:57:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 49, 2024-09-14T23:57:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-14T23:57:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-14T23:57:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-14T23:57:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6816", 2024-09-14T23:57:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 314865173 2024-09-14T23:57:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6817", 2024-09-14T23:57:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 314865173 2024-09-14T23:57:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-14T23:57:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6818", 2024-09-14T23:57:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 314865173 2024-09-14T23:57:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6819", 2024-09-14T23:57:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 314865173 2024-09-14T23:57:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-14T23:57:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6822", 2024-09-14T23:57:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 314865173 2024-09-14T23:57:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.189 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:23.189 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6823", 2024-09-14T23:57:23.189 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 314865173 2024-09-14T23:57:23.189 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:23.189 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:23.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.189 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-14T23:57:23.189 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:23.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.189 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:23.189 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6820", 2024-09-14T23:57:23.189 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 314865173 2024-09-14T23:57:23.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.190 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.190 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:23.190 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6821", 2024-09-14T23:57:23.190 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 314865173 2024-09-14T23:57:23.190 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:23.190 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:23.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.190 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6817/314865173", 2024-09-14T23:57:23.190 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6819/314865173", 2024-09-14T23:57:23.190 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6823/314865173", 2024-09-14T23:57:23.190 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6821/314865173", 2024-09-14T23:57:23.190 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-14T23:57:23.190 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-14T23:57:23.190 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-14T23:57:23.190 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:23.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.191 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.191 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 1, 2024-09-14T23:57:23.191 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "6c1a2765-c591-4c57-8ab3-ee6645970aa5", 2024-09-14T23:57:23.191 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-14T23:57:23.191 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-14T23:57:23.191 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-14T23:57:23.191 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-14T23:57:23.191 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-14T23:57:23.191 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-14T23:57:23.191 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 14, 2024-09-14T23:57:23.191 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 46, 2024-09-14T23:57:23.191 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-14T23:57:23.192 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-14T23:57:23.192 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-14T23:57:23.192 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:23.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.192 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:23.192 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6800", 2024-09-14T23:57:23.192 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3197912310 2024-09-14T23:57:23.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.192 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:23.192 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6801", 2024-09-14T23:57:23.192 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3197912310 2024-09-14T23:57:23.192 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:23.192 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:23.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.193 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-14T23:57:23.193 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:23.193 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.193 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:23.193 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6802", 2024-09-14T23:57:23.193 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3197912310 2024-09-14T23:57:23.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.193 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.193 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:23.193 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6803", 2024-09-14T23:57:23.193 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3197912310 2024-09-14T23:57:23.193 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:23.193 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:23.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.194 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-14T23:57:23.194 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:23.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.194 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:23.194 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6806", 2024-09-14T23:57:23.194 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3197912310 2024-09-14T23:57:23.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.194 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:23.194 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6807", 2024-09-14T23:57:23.194 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3197912310 2024-09-14T23:57:23.194 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:23.194 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:23.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.195 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-14T23:57:23.195 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:23.195 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.195 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:23.195 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6804", 2024-09-14T23:57:23.195 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3197912310 2024-09-14T23:57:23.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.195 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.195 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:23.195 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6805", 2024-09-14T23:57:23.195 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3197912310 2024-09-14T23:57:23.195 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:23.195 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:23.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.196 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6801/3197912310", 2024-09-14T23:57:23.196 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6803/3197912310", 2024-09-14T23:57:23.196 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6807/3197912310", 2024-09-14T23:57:23.196 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6805/3197912310", 2024-09-14T23:57:23.196 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-14T23:57:23.196 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-14T23:57:23.196 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-14T23:57:23.196 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:23.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.196 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 2, 2024-09-14T23:57:23.196 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "930db893-7be6-4645-a7f1-e043c5ec95eb", 2024-09-14T23:57:23.196 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-14T23:57:23.196 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-14T23:57:23.197 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-14T23:57:23.197 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-14T23:57:23.197 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-14T23:57:23.197 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-14T23:57:23.197 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 13, 2024-09-14T23:57:23.197 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 46, 2024-09-14T23:57:23.197 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-14T23:57:23.197 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-14T23:57:23.197 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-14T23:57:23.197 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:23.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.197 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:23.197 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6808", 2024-09-14T23:57:23.197 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3241153660 2024-09-14T23:57:23.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.198 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:23.198 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6809", 2024-09-14T23:57:23.198 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3241153660 2024-09-14T23:57:23.198 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:23.198 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:23.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.198 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-14T23:57:23.198 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:23.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.198 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:23.198 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6810", 2024-09-14T23:57:23.198 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3241153660 2024-09-14T23:57:23.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.199 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:23.199 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6811", 2024-09-14T23:57:23.199 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3241153660 2024-09-14T23:57:23.199 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:23.199 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:23.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.199 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-14T23:57:23.199 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:23.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.199 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:23.199 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6814", 2024-09-14T23:57:23.199 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3241153660 2024-09-14T23:57:23.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.200 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:23.200 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6815", 2024-09-14T23:57:23.200 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3241153660 2024-09-14T23:57:23.200 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:23.200 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:23.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.200 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-14T23:57:23.200 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:23.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.200 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:23.200 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6812", 2024-09-14T23:57:23.200 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3241153660 2024-09-14T23:57:23.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.201 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:23.201 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6813", 2024-09-14T23:57:23.201 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3241153660 2024-09-14T23:57:23.201 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:23.201 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:23.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.201 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6809/3241153660", 2024-09-14T23:57:23.201 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6811/3241153660", 2024-09-14T23:57:23.201 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6815/3241153660", 2024-09-14T23:57:23.201 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6813/3241153660", 2024-09-14T23:57:23.201 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-14T23:57:23.201 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-14T23:57:23.202 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-14T23:57:23.202 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:23.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.202 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 3, 2024-09-14T23:57:23.202 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "45bb3b84-b1c4-4f87-a7d1-0eb77b81741b", 2024-09-14T23:57:23.202 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-14T23:57:23.202 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-14T23:57:23.202 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-14T23:57:23.202 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-14T23:57:23.202 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-14T23:57:23.202 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-14T23:57:23.202 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-14T23:57:23.202 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 71, 2024-09-14T23:57:23.202 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-14T23:57:23.203 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-14T23:57:23.203 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-14T23:57:23.203 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:23.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.203 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:23.203 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6824", 2024-09-14T23:57:23.203 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1736416326 2024-09-14T23:57:23.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.203 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:23.203 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6825", 2024-09-14T23:57:23.203 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1736416326 2024-09-14T23:57:23.203 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:23.203 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:23.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.204 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-14T23:57:23.204 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:23.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.204 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:23.204 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6826", 2024-09-14T23:57:23.204 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1736416326 2024-09-14T23:57:23.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.204 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:23.204 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6827", 2024-09-14T23:57:23.204 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1736416326 2024-09-14T23:57:23.204 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:23.204 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:23.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.205 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-14T23:57:23.205 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:23.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.205 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:23.205 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6830", 2024-09-14T23:57:23.205 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1736416326 2024-09-14T23:57:23.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.205 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:23.205 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6831", 2024-09-14T23:57:23.205 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1736416326 2024-09-14T23:57:23.205 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:23.205 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:23.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.206 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-14T23:57:23.206 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:23.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.206 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:23.206 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6828", 2024-09-14T23:57:23.206 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1736416326 2024-09-14T23:57:23.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.206 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:23.206 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6829", 2024-09-14T23:57:23.206 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1736416326 2024-09-14T23:57:23.206 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:23.207 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:23.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.207 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6825/1736416326", 2024-09-14T23:57:23.207 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6827/1736416326", 2024-09-14T23:57:23.207 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6831/1736416326", 2024-09-14T23:57:23.207 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6829/1736416326", 2024-09-14T23:57:23.207 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-14T23:57:23.207 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-14T23:57:23.207 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-14T23:57:23.207 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:23.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.207 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 4, 2024-09-14T23:57:23.207 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "a812e38f-c0f9-4d33-9373-1910fb09103c", 2024-09-14T23:57:23.208 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-14T23:57:23.208 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-14T23:57:23.208 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-14T23:57:23.208 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 0.9703369140625, 2024-09-14T23:57:23.208 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-14T23:57:23.208 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-14T23:57:23.208 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-14T23:57:23.208 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 49, 2024-09-14T23:57:23.208 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-14T23:57:23.208 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-14T23:57:23.208 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-14T23:57:23.208 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:23.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.209 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:23.209 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6808", 2024-09-14T23:57:23.209 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1755154427 2024-09-14T23:57:23.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.209 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:23.209 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6809", 2024-09-14T23:57:23.209 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1755154427 2024-09-14T23:57:23.209 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:23.209 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:23.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.209 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-14T23:57:23.209 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:23.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.209 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:23.210 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6810", 2024-09-14T23:57:23.210 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1755154427 2024-09-14T23:57:23.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.210 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:23.210 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6811", 2024-09-14T23:57:23.210 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1755154427 2024-09-14T23:57:23.210 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:23.210 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:23.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.210 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-14T23:57:23.210 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:23.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.210 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:23.211 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6814", 2024-09-14T23:57:23.211 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1755154427 2024-09-14T23:57:23.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.211 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:23.211 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6815", 2024-09-14T23:57:23.211 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1755154427 2024-09-14T23:57:23.211 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:23.211 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:23.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.211 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-14T23:57:23.211 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:23.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.211 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:23.212 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6812", 2024-09-14T23:57:23.212 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1755154427 2024-09-14T23:57:23.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.212 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:23.212 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6813", 2024-09-14T23:57:23.212 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1755154427 2024-09-14T23:57:23.212 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:23.212 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:23.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.212 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.43:6809/1755154427", 2024-09-14T23:57:23.212 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.43:6811/1755154427", 2024-09-14T23:57:23.212 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.43:6815/1755154427", 2024-09-14T23:57:23.212 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.43:6813/1755154427", 2024-09-14T23:57:23.213 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-14T23:57:23.213 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-14T23:57:23.213 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-14T23:57:23.213 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:23.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.213 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 5, 2024-09-14T23:57:23.213 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "0de87e5b-7a21-4fb4-9543-d96c065786f8", 2024-09-14T23:57:23.213 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-14T23:57:23.213 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-14T23:57:23.213 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-14T23:57:23.213 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 0, 2024-09-14T23:57:23.213 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-14T23:57:23.213 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-14T23:57:23.213 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-14T23:57:23.214 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 23, 2024-09-14T23:57:23.214 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-14T23:57:23.214 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-14T23:57:23.214 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-14T23:57:23.214 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:23.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.214 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:23.214 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6816", 2024-09-14T23:57:23.214 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3033769028 2024-09-14T23:57:23.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.214 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:23.214 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6817", 2024-09-14T23:57:23.214 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3033769028 2024-09-14T23:57:23.215 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:23.215 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:23.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.215 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-14T23:57:23.215 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:23.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.215 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:23.215 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6818", 2024-09-14T23:57:23.215 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3033769028 2024-09-14T23:57:23.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.215 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:23.215 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6819", 2024-09-14T23:57:23.215 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3033769028 2024-09-14T23:57:23.215 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:23.216 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:23.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.216 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-14T23:57:23.216 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:23.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.216 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:23.216 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6822", 2024-09-14T23:57:23.216 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3033769028 2024-09-14T23:57:23.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.216 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:23.216 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6823", 2024-09-14T23:57:23.216 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3033769028 2024-09-14T23:57:23.217 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:23.217 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:23.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.217 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-14T23:57:23.217 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:23.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.217 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:23.217 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6820", 2024-09-14T23:57:23.217 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3033769028 2024-09-14T23:57:23.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.217 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:23.217 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6821", 2024-09-14T23:57:23.217 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3033769028 2024-09-14T23:57:23.218 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:23.218 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:23.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.218 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.43:6817/3033769028", 2024-09-14T23:57:23.218 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.43:6819/3033769028", 2024-09-14T23:57:23.218 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.43:6823/3033769028", 2024-09-14T23:57:23.218 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.43:6821/3033769028", 2024-09-14T23:57:23.218 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-14T23:57:23.218 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-14T23:57:23.218 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-14T23:57:23.218 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:23.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.218 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 6, 2024-09-14T23:57:23.219 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "c58c1d59-ad02-40f0-98e3-9078ef7e622b", 2024-09-14T23:57:23.219 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-14T23:57:23.219 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-14T23:57:23.219 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-14T23:57:23.219 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-14T23:57:23.219 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-14T23:57:23.219 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-14T23:57:23.219 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-14T23:57:23.219 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 23, 2024-09-14T23:57:23.219 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-14T23:57:23.219 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-14T23:57:23.219 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-14T23:57:23.219 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:23.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.220 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:23.220 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6800", 2024-09-14T23:57:23.220 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3314282416 2024-09-14T23:57:23.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.220 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:23.220 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6801", 2024-09-14T23:57:23.220 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3314282416 2024-09-14T23:57:23.220 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:23.220 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:23.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.220 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-14T23:57:23.220 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:23.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.221 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:23.221 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6802", 2024-09-14T23:57:23.221 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3314282416 2024-09-14T23:57:23.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.221 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:23.221 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6803", 2024-09-14T23:57:23.221 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3314282416 2024-09-14T23:57:23.221 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:23.221 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:23.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.221 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-14T23:57:23.221 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:23.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.222 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:23.222 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6806", 2024-09-14T23:57:23.222 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3314282416 2024-09-14T23:57:23.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.222 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:23.222 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6807", 2024-09-14T23:57:23.222 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3314282416 2024-09-14T23:57:23.222 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:23.222 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:23.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.222 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-14T23:57:23.222 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:23.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.222 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:23.223 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6804", 2024-09-14T23:57:23.223 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3314282416 2024-09-14T23:57:23.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.223 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:23.223 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6805", 2024-09-14T23:57:23.223 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3314282416 2024-09-14T23:57:23.223 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:23.223 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:23.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.223 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.43:6801/3314282416", 2024-09-14T23:57:23.223 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.43:6803/3314282416", 2024-09-14T23:57:23.223 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.43:6807/3314282416", 2024-09-14T23:57:23.223 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.43:6805/3314282416", 2024-09-14T23:57:23.224 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-14T23:57:23.224 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-14T23:57:23.224 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-14T23:57:23.224 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:23.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.224 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 7, 2024-09-14T23:57:23.224 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "389bdb04-20e3-4a7a-8504-687e034bd7dc", 2024-09-14T23:57:23.224 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-14T23:57:23.224 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-14T23:57:23.224 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-14T23:57:23.224 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-14T23:57:23.224 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-14T23:57:23.225 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-14T23:57:23.225 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-14T23:57:23.225 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 71, 2024-09-14T23:57:23.225 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-14T23:57:23.225 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-14T23:57:23.225 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-14T23:57:23.225 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:23.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.225 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:23.225 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6824", 2024-09-14T23:57:23.225 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1481350118 2024-09-14T23:57:23.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.226 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:23.226 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6825", 2024-09-14T23:57:23.226 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1481350118 2024-09-14T23:57:23.226 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:23.226 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:23.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.226 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-14T23:57:23.226 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:23.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.226 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:23.226 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6826", 2024-09-14T23:57:23.226 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1481350118 2024-09-14T23:57:23.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.227 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:23.227 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6827", 2024-09-14T23:57:23.227 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1481350118 2024-09-14T23:57:23.227 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:23.227 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:23.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.227 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-14T23:57:23.227 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:23.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.227 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:23.227 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6830", 2024-09-14T23:57:23.227 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1481350118 2024-09-14T23:57:23.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.228 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:23.228 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6831", 2024-09-14T23:57:23.228 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1481350118 2024-09-14T23:57:23.228 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:23.228 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:23.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.228 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-14T23:57:23.228 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:23.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.228 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:23.228 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6828", 2024-09-14T23:57:23.228 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1481350118 2024-09-14T23:57:23.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.228 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:23.229 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6829", 2024-09-14T23:57:23.229 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1481350118 2024-09-14T23:57:23.229 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:23.229 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:23.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.229 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.43:6825/1481350118", 2024-09-14T23:57:23.229 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.43:6827/1481350118", 2024-09-14T23:57:23.229 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.43:6831/1481350118", 2024-09-14T23:57:23.229 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.43:6829/1481350118", 2024-09-14T23:57:23.229 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-14T23:57:23.229 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-14T23:57:23.229 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-14T23:57:23.229 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:23.229 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:23.230 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-14T23:57:23.230 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_xinfo": [ 2024-09-14T23:57:23.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.230 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 0, 2024-09-14T23:57:23.230 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-14T23:57:23.230 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-14T23:57:23.230 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-14T23:57:23.230 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-14T23:57:23.230 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-14T23:57:23.230 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-14T23:56:11.077810+0000", 2024-09-14T23:57:23.230 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-14T23:57:23.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.230 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 1, 2024-09-14T23:57:23.231 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-14T23:57:23.231 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-14T23:57:23.231 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-14T23:57:23.231 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-14T23:57:23.231 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-14T23:57:23.231 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-14T23:56:11.578838+0000", 2024-09-14T23:57:23.231 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-14T23:57:23.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.231 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 2, 2024-09-14T23:57:23.231 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-14T23:57:23.231 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-14T23:57:23.231 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-14T23:57:23.231 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-14T23:57:23.232 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-14T23:57:23.232 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-14T23:56:10.895490+0000", 2024-09-14T23:57:23.232 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-14T23:57:23.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.232 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 3, 2024-09-14T23:57:23.232 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-14T23:57:23.232 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-14T23:57:23.232 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-14T23:57:23.232 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-14T23:57:23.232 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-14T23:57:23.232 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-14T23:56:12.290821+0000", 2024-09-14T23:57:23.232 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-14T23:57:23.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.233 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 4, 2024-09-14T23:57:23.233 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-14T23:57:23.233 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-14T23:57:23.233 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-14T23:57:23.233 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-14T23:57:23.233 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-14T23:57:23.233 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-14T23:56:11.800581+0000", 2024-09-14T23:57:23.233 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-14T23:57:23.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.234 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 5, 2024-09-14T23:57:23.234 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-14T23:57:23.234 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-14T23:57:23.234 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-14T23:57:23.234 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-14T23:57:23.234 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-14T23:57:23.234 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-14T23:56:11.883155+0000", 2024-09-14T23:57:23.234 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-14T23:57:23.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.234 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 6, 2024-09-14T23:57:23.234 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-14T23:57:23.235 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-14T23:57:23.235 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-14T23:57:23.235 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-14T23:57:23.235 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-14T23:57:23.235 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-14T23:56:11.604790+0000", 2024-09-14T23:57:23.235 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-14T23:57:23.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.235 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 7, 2024-09-14T23:57:23.235 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-14T23:57:23.235 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-14T23:57:23.235 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-14T23:57:23.235 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-14T23:57:23.235 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-14T23:57:23.235 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-14T23:56:12.067759+0000", 2024-09-14T23:57:23.235 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-14T23:57:23.236 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:23.236 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-14T23:57:23.236 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_upmap": [], 2024-09-14T23:57:23.236 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_upmap_items": [], 2024-09-14T23:57:23.236 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_upmap_primaries": [], 2024-09-14T23:57:23.236 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_temp": [ 2024-09-14T23:57:23.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.236 INFO:teuthology.orchestra.run.smithi042.stdout: "pgid": "3.2", 2024-09-14T23:57:23.236 INFO:teuthology.orchestra.run.smithi042.stdout: "osds": [ 2024-09-14T23:57:23.236 INFO:teuthology.orchestra.run.smithi042.stdout: 3 2024-09-14T23:57:23.236 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:23.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.237 INFO:teuthology.orchestra.run.smithi042.stdout: "pgid": "3.d", 2024-09-14T23:57:23.237 INFO:teuthology.orchestra.run.smithi042.stdout: "osds": [ 2024-09-14T23:57:23.237 INFO:teuthology.orchestra.run.smithi042.stdout: 7 2024-09-14T23:57:23.237 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:23.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:23.237 INFO:teuthology.orchestra.run.smithi042.stdout: "pgid": "3.16", 2024-09-14T23:57:23.237 INFO:teuthology.orchestra.run.smithi042.stdout: "osds": [ 2024-09-14T23:57:23.237 INFO:teuthology.orchestra.run.smithi042.stdout: 7 2024-09-14T23:57:23.237 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:23.237 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:23.237 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-14T23:57:23.237 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_temp": [], 2024-09-14T23:57:23.237 INFO:teuthology.orchestra.run.smithi042.stdout: "blocklist": {}, 2024-09-14T23:57:23.238 INFO:teuthology.orchestra.run.smithi042.stdout: "range_blocklist": {}, 2024-09-14T23:57:23.238 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profiles": { 2024-09-14T23:57:23.238 INFO:teuthology.orchestra.run.smithi042.stdout: "default": { 2024-09-14T23:57:23.238 INFO:teuthology.orchestra.run.smithi042.stdout: "crush-failure-domain": "osd", 2024-09-14T23:57:23.238 INFO:teuthology.orchestra.run.smithi042.stdout: "k": "2", 2024-09-14T23:57:23.238 INFO:teuthology.orchestra.run.smithi042.stdout: "m": "1", 2024-09-14T23:57:23.238 INFO:teuthology.orchestra.run.smithi042.stdout: "plugin": "jerasure", 2024-09-14T23:57:23.238 INFO:teuthology.orchestra.run.smithi042.stdout: "technique": "reed_sol_van" 2024-09-14T23:57:23.238 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:23.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:23.238 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps_queue": [], 2024-09-14T23:57:23.238 INFO:teuthology.orchestra.run.smithi042.stdout: "new_removed_snaps": [], 2024-09-14T23:57:23.238 INFO:teuthology.orchestra.run.smithi042.stdout: "new_purged_snaps": [], 2024-09-14T23:57:23.238 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_node_flags": {}, 2024-09-14T23:57:23.239 INFO:teuthology.orchestra.run.smithi042.stdout: "device_class_flags": {}, 2024-09-14T23:57:23.239 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode": { 2024-09-14T23:57:23.239 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode_enabled": false, 2024-09-14T23:57:23.239 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_bucket_count": 0, 2024-09-14T23:57:23.239 INFO:teuthology.orchestra.run.smithi042.stdout: "degraded_stretch_mode": 0, 2024-09-14T23:57:23.239 INFO:teuthology.orchestra.run.smithi042.stdout: "recovering_stretch_mode": 0, 2024-09-14T23:57:23.239 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode_bucket": 0 2024-09-14T23:57:23.239 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:23.239 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:23.241 INFO:tasks.thrashosds.thrasher:j is {'epoch': 72, 'fsid': '21916fcc-f95c-4bfa-8b8b-11ca45845613', 'created': '2024-09-14T23:56:03.167948+0000', 'modified': '2024-09-14T23:57:22.961289+0000', 'last_up_change': '2024-09-14T23:56:14.322882+0000', 'last_in_change': '2024-09-14T23:56:56.035972+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T23:56:13.500197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8712921142578125, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8700000047683716}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T23:56:21.284705+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.630000114440918, 'score_stable': 2.630000114440918, 'optimal_score': 1, 'raw_score_acting': 2.630000114440918, 'raw_score_stable': 2.630000114440918, 'primary_affinity_weighted': 0.8712921142578125, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8700000047683716}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T23:56:30.861504+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 27, 'snap_epoch': 70, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8712921142578125, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8700000047683716}}], 'osds': [{'osd': 0, 'uuid': 'db9d0c67-2d60-4592-9ad3-f99e5294f93f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 49, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6816', 'nonce': 314865173}, {'type': 'v1', 'addr': '172.21.15.42:6817', 'nonce': 314865173}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6818', 'nonce': 314865173}, {'type': 'v1', 'addr': '172.21.15.42:6819', 'nonce': 314865173}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6822', 'nonce': 314865173}, {'type': 'v1', 'addr': '172.21.15.42:6823', 'nonce': 314865173}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6820', 'nonce': 314865173}, {'type': 'v1', 'addr': '172.21.15.42:6821', 'nonce': 314865173}]}, 'public_addr': '172.21.15.42:6817/314865173', 'cluster_addr': '172.21.15.42:6819/314865173', 'heartbeat_back_addr': '172.21.15.42:6823/314865173', 'heartbeat_front_addr': '172.21.15.42:6821/314865173', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '6c1a2765-c591-4c57-8ab3-ee6645970aa5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 46, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6800', 'nonce': 3197912310}, {'type': 'v1', 'addr': '172.21.15.42:6801', 'nonce': 3197912310}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6802', 'nonce': 3197912310}, {'type': 'v1', 'addr': '172.21.15.42:6803', 'nonce': 3197912310}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6806', 'nonce': 3197912310}, {'type': 'v1', 'addr': '172.21.15.42:6807', 'nonce': 3197912310}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6804', 'nonce': 3197912310}, {'type': 'v1', 'addr': '172.21.15.42:6805', 'nonce': 3197912310}]}, 'public_addr': '172.21.15.42:6801/3197912310', 'cluster_addr': '172.21.15.42:6803/3197912310', 'heartbeat_back_addr': '172.21.15.42:6807/3197912310', 'heartbeat_front_addr': '172.21.15.42:6805/3197912310', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '930db893-7be6-4645-a7f1-e043c5ec95eb', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 46, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6808', 'nonce': 3241153660}, {'type': 'v1', 'addr': '172.21.15.42:6809', 'nonce': 3241153660}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6810', 'nonce': 3241153660}, {'type': 'v1', 'addr': '172.21.15.42:6811', 'nonce': 3241153660}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6814', 'nonce': 3241153660}, {'type': 'v1', 'addr': '172.21.15.42:6815', 'nonce': 3241153660}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6812', 'nonce': 3241153660}, {'type': 'v1', 'addr': '172.21.15.42:6813', 'nonce': 3241153660}]}, 'public_addr': '172.21.15.42:6809/3241153660', 'cluster_addr': '172.21.15.42:6811/3241153660', 'heartbeat_back_addr': '172.21.15.42:6815/3241153660', 'heartbeat_front_addr': '172.21.15.42:6813/3241153660', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '45bb3b84-b1c4-4f87-a7d1-0eb77b81741b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 71, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6824', 'nonce': 1736416326}, {'type': 'v1', 'addr': '172.21.15.42:6825', 'nonce': 1736416326}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6826', 'nonce': 1736416326}, {'type': 'v1', 'addr': '172.21.15.42:6827', 'nonce': 1736416326}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6830', 'nonce': 1736416326}, {'type': 'v1', 'addr': '172.21.15.42:6831', 'nonce': 1736416326}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6828', 'nonce': 1736416326}, {'type': 'v1', 'addr': '172.21.15.42:6829', 'nonce': 1736416326}]}, 'public_addr': '172.21.15.42:6825/1736416326', 'cluster_addr': '172.21.15.42:6827/1736416326', 'heartbeat_back_addr': '172.21.15.42:6831/1736416326', 'heartbeat_front_addr': '172.21.15.42:6829/1736416326', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'a812e38f-c0f9-4d33-9373-1910fb09103c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.9703369140625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 49, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6808', 'nonce': 1755154427}, {'type': 'v1', 'addr': '172.21.15.43:6809', 'nonce': 1755154427}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6810', 'nonce': 1755154427}, {'type': 'v1', 'addr': '172.21.15.43:6811', 'nonce': 1755154427}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6814', 'nonce': 1755154427}, {'type': 'v1', 'addr': '172.21.15.43:6815', 'nonce': 1755154427}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6812', 'nonce': 1755154427}, {'type': 'v1', 'addr': '172.21.15.43:6813', 'nonce': 1755154427}]}, 'public_addr': '172.21.15.43:6809/1755154427', 'cluster_addr': '172.21.15.43:6811/1755154427', 'heartbeat_back_addr': '172.21.15.43:6815/1755154427', 'heartbeat_front_addr': '172.21.15.43:6813/1755154427', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '0de87e5b-7a21-4fb4-9543-d96c065786f8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6816', 'nonce': 3033769028}, {'type': 'v1', 'addr': '172.21.15.43:6817', 'nonce': 3033769028}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6818', 'nonce': 3033769028}, {'type': 'v1', 'addr': '172.21.15.43:6819', 'nonce': 3033769028}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6822', 'nonce': 3033769028}, {'type': 'v1', 'addr': '172.21.15.43:6823', 'nonce': 3033769028}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6820', 'nonce': 3033769028}, {'type': 'v1', 'addr': '172.21.15.43:6821', 'nonce': 3033769028}]}, 'public_addr': '172.21.15.43:6817/3033769028', 'cluster_addr': '172.21.15.43:6819/3033769028', 'heartbeat_back_addr': '172.21.15.43:6823/3033769028', 'heartbeat_front_addr': '172.21.15.43:6821/3033769028', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'c58c1d59-ad02-40f0-98e3-9078ef7e622b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6800', 'nonce': 3314282416}, {'type': 'v1', 'addr': '172.21.15.43:6801', 'nonce': 3314282416}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6802', 'nonce': 3314282416}, {'type': 'v1', 'addr': '172.21.15.43:6803', 'nonce': 3314282416}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6806', 'nonce': 3314282416}, {'type': 'v1', 'addr': '172.21.15.43:6807', 'nonce': 3314282416}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6804', 'nonce': 3314282416}, {'type': 'v1', 'addr': '172.21.15.43:6805', 'nonce': 3314282416}]}, 'public_addr': '172.21.15.43:6801/3314282416', 'cluster_addr': '172.21.15.43:6803/3314282416', 'heartbeat_back_addr': '172.21.15.43:6807/3314282416', 'heartbeat_front_addr': '172.21.15.43:6805/3314282416', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '389bdb04-20e3-4a7a-8504-687e034bd7dc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 71, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6824', 'nonce': 1481350118}, {'type': 'v1', 'addr': '172.21.15.43:6825', 'nonce': 1481350118}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6826', 'nonce': 1481350118}, {'type': 'v1', 'addr': '172.21.15.43:6827', 'nonce': 1481350118}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6830', 'nonce': 1481350118}, {'type': 'v1', 'addr': '172.21.15.43:6831', 'nonce': 1481350118}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6828', 'nonce': 1481350118}, {'type': 'v1', 'addr': '172.21.15.43:6829', 'nonce': 1481350118}]}, 'public_addr': '172.21.15.43:6825/1481350118', 'cluster_addr': '172.21.15.43:6827/1481350118', 'heartbeat_back_addr': '172.21.15.43:6831/1481350118', 'heartbeat_front_addr': '172.21.15.43:6829/1481350118', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T23:56:11.077810+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T23:56:11.578838+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T23:56:10.895490+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T23:56:12.290821+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T23:56:11.800581+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T23:56:11.883155+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T23:56:11.604790+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T23:56:12.067759+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [{'pgid': '3.2', 'osds': [3]}, {'pgid': '3.d', 'osds': [7]}, {'pgid': '3.16', 'osds': [7]}], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T23:57:23.241 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T23:57:23.242 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:23.239+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:23.250 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:23.250 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:23.258 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T23:57:23.341 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:23.339+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:23.366 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:23.366 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:23.375 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T23:57:23.442 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:23.440+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:23.482 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:23.482 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:23.491 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T23:57:23.543 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:23.541+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:23.572 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:57:23.572 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-14T23:57:23.584 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":101,"stamp":"2024-09-14T23:57:23.463890+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":640972685,"num_objects":1884,"num_object_clones":352,"num_object_copies":3768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1884,"num_whiteouts":979,"num_read":125943,"num_read_kb":516743,"num_write":11014,"num_write_kb":267928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":842,"num_bytes_recovered":37168158,"num_keys_recovered":121,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9318,"ondisk_log_size":9318,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":318536,"kb_used_data":48212,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754656184,"statfs":{"total":773094113280,"available":772767932416,"internally_reserved":0,"allocated":49369088,"data_stored":409461713,"data_compressed":30129852,"data_compressed_allocated":32817152,"data_compressed_original":398134894,"omap_allocated":12752,"internal_metadata":276614704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,20,52,78,65],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":64030230,"num_objects":376,"num_object_clones":36,"num_object_copies":752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":376,"num_whiteouts":295,"num_read":37956,"num_read_kb":142843,"num_write":1421,"num_write_kb":63176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001034"},"pg_stats":[{"pgid":"3.1f","version":"71'598","reported_seq":2445,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-14T23:57:22.964626+0000","last_change":"2024-09-14T23:57:00.062727+0000","last_active":"2024-09-14T23:57:22.964626+0000","last_peered":"2024-09-14T23:57:22.964626+0000","last_clean":"2024-09-14T23:57:22.964626+0000","last_became_active":"2024-09-14T23:57:00.062116+0000","last_became_peered":"2024-09-14T23:57:00.062116+0000","last_unstale":"2024-09-14T23:57:22.964626+0000","last_undegraded":"2024-09-14T23:57:22.964626+0000","last_fullsized":"2024-09-14T23:57:22.964626+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":598,"log_dups_size":0,"ondisk_log_size":598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:57:32.885487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015692659999999999,"stat_sum":{"num_bytes":21756842,"num_objects":70,"num_object_clones":13,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":34,"num_read":4685,"num_read_kb":13138,"num_write":814,"num_write_kb":20539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":2327658,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"68'182","reported_seq":1417,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.145376+0000","last_change":"2024-09-14T23:56:34.415748+0000","last_active":"2024-09-14T23:57:20.145376+0000","last_peered":"2024-09-14T23:57:20.145376+0000","last_clean":"2024-09-14T23:57:20.145376+0000","last_became_active":"2024-09-14T23:56:32.404385+0000","last_became_peered":"2024-09-14T23:56:32.404385+0000","last_unstale":"2024-09-14T23:57:20.145376+0000","last_undegraded":"2024-09-14T23:57:20.145376+0000","last_fullsized":"2024-09-14T23:57:20.145376+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:18:14.290947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010144430000000001,"stat_sum":{"num_bytes":13278889,"num_objects":64,"num_object_clones":15,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":31,"num_read":1245,"num_read_kb":9182,"num_write":169,"num_write_kb":8578,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"68'274","reported_seq":2745,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.147084+0000","last_change":"2024-09-14T23:57:00.062960+0000","last_active":"2024-09-14T23:57:20.147084+0000","last_peered":"2024-09-14T23:57:20.147084+0000","last_clean":"2024-09-14T23:57:20.147084+0000","last_became_active":"2024-09-14T23:57:00.062242+0000","last_became_peered":"2024-09-14T23:57:00.062242+0000","last_unstale":"2024-09-14T23:57:20.147084+0000","last_undegraded":"2024-09-14T23:57:20.147084+0000","last_fullsized":"2024-09-14T23:57:20.147084+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:44:00.321217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001148175,"stat_sum":{"num_bytes":29669808,"num_objects":67,"num_object_clones":15,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":34,"num_read":3214,"num_read_kb":13398,"num_write":249,"num_write_kb":11174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":1249963,"num_keys_recovered":34,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"68'297","reported_seq":3042,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.147123+0000","last_change":"2024-09-14T23:57:00.062694+0000","last_active":"2024-09-14T23:57:20.147123+0000","last_peered":"2024-09-14T23:57:20.147123+0000","last_clean":"2024-09-14T23:57:20.147123+0000","last_became_active":"2024-09-14T23:57:00.062126+0000","last_became_peered":"2024-09-14T23:57:00.062126+0000","last_unstale":"2024-09-14T23:57:20.147123+0000","last_undegraded":"2024-09-14T23:57:20.147123+0000","last_fullsized":"2024-09-14T23:57:20.147123+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:09:32.074708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095260199999999998,"stat_sum":{"num_bytes":9921574,"num_objects":52,"num_object_clones":7,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":30,"num_read":3528,"num_read_kb":10946,"num_write":351,"num_write_kb":5066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":2675491,"num_keys_recovered":40,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"70'155","reported_seq":632,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-14T23:57:22.964919+0000","last_change":"2024-09-14T23:56:34.401802+0000","last_active":"2024-09-14T23:57:22.964919+0000","last_peered":"2024-09-14T23:57:22.964919+0000","last_clean":"2024-09-14T23:57:22.964919+0000","last_became_active":"2024-09-14T23:56:32.398329+0000","last_became_peered":"2024-09-14T23:56:32.398329+0000","last_unstale":"2024-09-14T23:57:22.964919+0000","last_undegraded":"2024-09-14T23:57:22.964919+0000","last_fullsized":"2024-09-14T23:57:22.964919+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:30:27.363178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018313229999999999,"stat_sum":{"num_bytes":12504384,"num_objects":40,"num_object_clones":6,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":21,"num_read":393,"num_read_kb":7357,"num_write":141,"num_write_kb":4634,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"68'231","reported_seq":2151,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.174636+0000","last_change":"2024-09-14T23:56:34.405093+0000","last_active":"2024-09-14T23:57:20.174636+0000","last_peered":"2024-09-14T23:57:20.174636+0000","last_clean":"2024-09-14T23:57:20.174636+0000","last_became_active":"2024-09-14T23:56:32.399655+0000","last_became_peered":"2024-09-14T23:56:32.399655+0000","last_unstale":"2024-09-14T23:57:20.174636+0000","last_undegraded":"2024-09-14T23:57:20.174636+0000","last_fullsized":"2024-09-14T23:57:20.174636+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:17:56.591978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096230199999999995,"stat_sum":{"num_bytes":15076848,"num_objects":71,"num_object_clones":15,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":43,"num_read":1922,"num_read_kb":10181,"num_write":217,"num_write_kb":13721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"70'264","reported_seq":1155,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-14T23:57:22.968351+0000","last_change":"2024-09-14T23:56:34.400617+0000","last_active":"2024-09-14T23:57:22.968351+0000","last_peered":"2024-09-14T23:57:22.968351+0000","last_clean":"2024-09-14T23:57:22.968351+0000","last_became_active":"2024-09-14T23:56:32.404364+0000","last_became_peered":"2024-09-14T23:56:32.404364+0000","last_unstale":"2024-09-14T23:57:22.968351+0000","last_undegraded":"2024-09-14T23:57:22.968351+0000","last_fullsized":"2024-09-14T23:57:22.968351+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":264,"log_dups_size":0,"ondisk_log_size":264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:57:51.426540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001003264,"stat_sum":{"num_bytes":12669296,"num_objects":67,"num_object_clones":9,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":38,"num_read":1252,"num_read_kb":5599,"num_write":290,"num_write_kb":4210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"68'244","reported_seq":2978,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.145133+0000","last_change":"2024-09-14T23:56:34.415021+0000","last_active":"2024-09-14T23:57:20.145133+0000","last_peered":"2024-09-14T23:57:20.145133+0000","last_clean":"2024-09-14T23:57:20.145133+0000","last_became_active":"2024-09-14T23:56:32.401799+0000","last_became_peered":"2024-09-14T23:56:32.401799+0000","last_unstale":"2024-09-14T23:57:20.145133+0000","last_undegraded":"2024-09-14T23:57:20.145133+0000","last_fullsized":"2024-09-14T23:57:20.145133+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":244,"log_dups_size":0,"ondisk_log_size":244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:01:57.971994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062985000000000005,"stat_sum":{"num_bytes":16572772,"num_objects":70,"num_object_clones":14,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":39,"num_read":3481,"num_read_kb":10912,"num_write":223,"num_write_kb":1906,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"70'196","reported_seq":1213,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-14T23:57:22.964706+0000","last_change":"2024-09-14T23:57:00.062824+0000","last_active":"2024-09-14T23:57:22.964706+0000","last_peered":"2024-09-14T23:57:22.964706+0000","last_clean":"2024-09-14T23:57:22.964706+0000","last_became_active":"2024-09-14T23:57:00.062304+0000","last_became_peered":"2024-09-14T23:57:00.062304+0000","last_unstale":"2024-09-14T23:57:22.964706+0000","last_undegraded":"2024-09-14T23:57:22.964706+0000","last_fullsized":"2024-09-14T23:57:22.964706+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":196,"log_dups_size":0,"ondisk_log_size":196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:54:02.597826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014783859999999999,"stat_sum":{"num_bytes":9134901,"num_objects":50,"num_object_clones":9,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":24,"num_read":943,"num_read_kb":4329,"num_write":199,"num_write_kb":3550,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":3345543,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"66'190","reported_seq":5531,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.145319+0000","last_change":"2024-09-14T23:57:13.115006+0000","last_active":"2024-09-14T23:57:20.145319+0000","last_peered":"2024-09-14T23:57:20.145319+0000","last_clean":"2024-09-14T23:57:20.145319+0000","last_became_active":"2024-09-14T23:57:13.114091+0000","last_became_peered":"2024-09-14T23:57:13.114091+0000","last_unstale":"2024-09-14T23:57:20.145319+0000","last_undegraded":"2024-09-14T23:57:20.145319+0000","last_fullsized":"2024-09-14T23:57:20.145319+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:33:49.799819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039416400000000002,"stat_sum":{"num_bytes":25452456,"num_objects":45,"num_object_clones":6,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":23,"num_read":5059,"num_read_kb":21361,"num_write":163,"num_write_kb":10062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":103,"num_bytes_recovered":3116309,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.15","version":"68'308","reported_seq":6939,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.145425+0000","last_change":"2024-09-14T23:56:34.399382+0000","last_active":"2024-09-14T23:57:20.145425+0000","last_peered":"2024-09-14T23:57:20.145425+0000","last_clean":"2024-09-14T23:57:20.145425+0000","last_became_active":"2024-09-14T23:56:32.405387+0000","last_became_peered":"2024-09-14T23:56:32.405387+0000","last_unstale":"2024-09-14T23:57:20.145425+0000","last_undegraded":"2024-09-14T23:57:20.145425+0000","last_fullsized":"2024-09-14T23:57:20.145425+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:17:00.235747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046328199999999997,"stat_sum":{"num_bytes":36776121,"num_objects":54,"num_object_clones":12,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":25,"num_read":7488,"num_read_kb":29301,"num_write":385,"num_write_kb":22562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"68'670","reported_seq":4989,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.147024+0000","last_change":"2024-09-14T23:56:34.404536+0000","last_active":"2024-09-14T23:57:20.147024+0000","last_peered":"2024-09-14T23:57:20.147024+0000","last_clean":"2024-09-14T23:57:20.147024+0000","last_became_active":"2024-09-14T23:56:32.403019+0000","last_became_peered":"2024-09-14T23:56:32.403019+0000","last_unstale":"2024-09-14T23:57:20.147024+0000","last_undegraded":"2024-09-14T23:57:20.147024+0000","last_fullsized":"2024-09-14T23:57:20.147024+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:36:36.583890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037505299999999997,"stat_sum":{"num_bytes":21573885,"num_objects":55,"num_object_clones":11,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":27,"num_read":7236,"num_read_kb":21793,"num_write":1131,"num_write_kb":5887,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":115,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.153322+0000","last_change":"2024-09-14T23:56:24.727061+0000","last_active":"2024-09-14T23:57:20.153322+0000","last_peered":"2024-09-14T23:57:20.153322+0000","last_clean":"2024-09-14T23:57:20.153322+0000","last_became_active":"2024-09-14T23:56:22.526915+0000","last_became_peered":"2024-09-14T23:56:22.526915+0000","last_unstale":"2024-09-14T23:57:20.153322+0000","last_undegraded":"2024-09-14T23:57:20.153322+0000","last_fullsized":"2024-09-14T23:57:20.153322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:43:15.359266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045105900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"68'252","reported_seq":2859,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-14T23:57:22.965095+0000","last_change":"2024-09-14T23:56:34.402234+0000","last_active":"2024-09-14T23:57:22.965095+0000","last_peered":"2024-09-14T23:57:22.965095+0000","last_clean":"2024-09-14T23:57:22.965095+0000","last_became_active":"2024-09-14T23:56:32.398664+0000","last_became_peered":"2024-09-14T23:56:32.398664+0000","last_unstale":"2024-09-14T23:57:22.965095+0000","last_undegraded":"2024-09-14T23:57:22.965095+0000","last_fullsized":"2024-09-14T23:57:22.965095+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":252,"log_dups_size":0,"ondisk_log_size":252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:11:40.522531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012179999999999999,"stat_sum":{"num_bytes":15943243,"num_objects":49,"num_object_clones":6,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":23,"num_read":3879,"num_read_kb":13569,"num_write":249,"num_write_kb":4259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":121,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-14T23:57:22.968734+0000","last_change":"2024-09-14T23:56:24.534632+0000","last_active":"2024-09-14T23:57:22.968734+0000","last_peered":"2024-09-14T23:57:22.968734+0000","last_clean":"2024-09-14T23:57:22.968734+0000","last_became_active":"2024-09-14T23:56:22.526355+0000","last_became_peered":"2024-09-14T23:56:22.526355+0000","last_unstale":"2024-09-14T23:57:22.968734+0000","last_undegraded":"2024-09-14T23:57:22.968734+0000","last_fullsized":"2024-09-14T23:57:22.968734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:02:16.059627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049910000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"69'226","reported_seq":4798,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.189555+0000","last_change":"2024-09-14T23:56:34.416134+0000","last_active":"2024-09-14T23:57:20.189555+0000","last_peered":"2024-09-14T23:57:20.189555+0000","last_clean":"2024-09-14T23:57:20.189555+0000","last_became_active":"2024-09-14T23:56:32.404281+0000","last_became_peered":"2024-09-14T23:56:32.404281+0000","last_unstale":"2024-09-14T23:57:20.189555+0000","last_undegraded":"2024-09-14T23:57:20.189555+0000","last_fullsized":"2024-09-14T23:57:20.189555+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:37:18.248147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013337620000000001,"stat_sum":{"num_bytes":26122292,"num_objects":67,"num_object_clones":14,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":37,"num_read":5489,"num_read_kb":30567,"num_write":201,"num_write_kb":7732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":115,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.145540+0000","last_change":"2024-09-14T23:56:25.093112+0000","last_active":"2024-09-14T23:57:20.145540+0000","last_peered":"2024-09-14T23:57:20.145540+0000","last_clean":"2024-09-14T23:57:20.145540+0000","last_became_active":"2024-09-14T23:56:22.528211+0000","last_became_peered":"2024-09-14T23:56:22.528211+0000","last_unstale":"2024-09-14T23:57:20.145540+0000","last_undegraded":"2024-09-14T23:57:20.145540+0000","last_fullsized":"2024-09-14T23:57:20.145540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:14:11.495286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000459686,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"71'253","reported_seq":2510,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-14T23:57:22.968479+0000","last_change":"2024-09-14T23:56:34.401111+0000","last_active":"2024-09-14T23:57:22.968479+0000","last_peered":"2024-09-14T23:57:22.968479+0000","last_clean":"2024-09-14T23:57:22.968479+0000","last_became_active":"2024-09-14T23:56:32.401089+0000","last_became_peered":"2024-09-14T23:56:32.401089+0000","last_unstale":"2024-09-14T23:57:22.968479+0000","last_undegraded":"2024-09-14T23:57:22.968479+0000","last_fullsized":"2024-09-14T23:57:22.968479+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":253,"log_dups_size":0,"ondisk_log_size":253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:56:54.778547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013033610000000001,"stat_sum":{"num_bytes":24360121,"num_objects":78,"num_object_clones":13,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":41,"num_read":2241,"num_read_kb":13908,"num_write":260,"num_write_kb":9694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":121,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-14T23:57:22.968478+0000","last_change":"2024-09-14T23:56:24.534510+0000","last_active":"2024-09-14T23:57:22.968478+0000","last_peered":"2024-09-14T23:57:22.968478+0000","last_clean":"2024-09-14T23:57:22.968478+0000","last_became_active":"2024-09-14T23:56:22.527611+0000","last_became_peered":"2024-09-14T23:56:22.527611+0000","last_unstale":"2024-09-14T23:57:22.968478+0000","last_undegraded":"2024-09-14T23:57:22.968478+0000","last_fullsized":"2024-09-14T23:57:22.968478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:15:45.606026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049916299999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"68'359","reported_seq":2290,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.145361+0000","last_change":"2024-09-14T23:57:00.065551+0000","last_active":"2024-09-14T23:57:20.145361+0000","last_peered":"2024-09-14T23:57:20.145361+0000","last_clean":"2024-09-14T23:57:20.145361+0000","last_became_active":"2024-09-14T23:57:00.064678+0000","last_became_peered":"2024-09-14T23:57:00.064678+0000","last_unstale":"2024-09-14T23:57:20.145361+0000","last_undegraded":"2024-09-14T23:57:20.145361+0000","last_fullsized":"2024-09-14T23:57:20.145361+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:54:22.755339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001272947,"stat_sum":{"num_bytes":12132806,"num_objects":74,"num_object_clones":15,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":39,"num_read":2794,"num_read_kb":11298,"num_write":430,"num_write_kb":8654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":3349587,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"69'223","reported_seq":2075,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.196275+0000","last_change":"2024-09-14T23:56:34.404751+0000","last_active":"2024-09-14T23:57:20.196275+0000","last_peered":"2024-09-14T23:57:20.196275+0000","last_clean":"2024-09-14T23:57:20.196275+0000","last_became_active":"2024-09-14T23:56:32.397883+0000","last_became_peered":"2024-09-14T23:56:32.397883+0000","last_unstale":"2024-09-14T23:57:20.196275+0000","last_undegraded":"2024-09-14T23:57:20.196275+0000","last_fullsized":"2024-09-14T23:57:20.196275+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:38:47.008304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048955799999999996,"stat_sum":{"num_bytes":20549548,"num_objects":64,"num_object_clones":11,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":30,"num_read":2766,"num_read_kb":10503,"num_write":191,"num_write_kb":4407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":132,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-14T23:57:22.968436+0000","last_change":"2024-09-14T23:56:57.052969+0000","last_active":"2024-09-14T23:57:22.968436+0000","last_peered":"2024-09-14T23:57:22.968436+0000","last_clean":"2024-09-14T23:57:22.968436+0000","last_became_active":"2024-09-14T23:56:57.050456+0000","last_became_peered":"2024-09-14T23:56:57.050456+0000","last_unstale":"2024-09-14T23:57:22.968436+0000","last_undegraded":"2024-09-14T23:57:22.968436+0000","last_fullsized":"2024-09-14T23:57:22.968436+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:57:07.644011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"70'232","reported_seq":6919,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-14T23:57:22.968369+0000","last_change":"2024-09-14T23:56:34.401159+0000","last_active":"2024-09-14T23:57:22.968369+0000","last_peered":"2024-09-14T23:57:22.968369+0000","last_clean":"2024-09-14T23:57:22.968369+0000","last_became_active":"2024-09-14T23:56:32.400278+0000","last_became_peered":"2024-09-14T23:56:32.400278+0000","last_unstale":"2024-09-14T23:57:22.968369+0000","last_undegraded":"2024-09-14T23:57:22.968369+0000","last_fullsized":"2024-09-14T23:57:22.968369+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:44:11.048971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082507200000000004,"stat_sum":{"num_bytes":29831009,"num_objects":53,"num_object_clones":7,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":27,"num_read":6764,"num_read_kb":23142,"num_write":235,"num_write_kb":11315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":119,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-14T23:57:22.158579+0000","last_change":"2024-09-14T23:56:24.909260+0000","last_active":"2024-09-14T23:57:22.158579+0000","last_peered":"2024-09-14T23:57:22.158579+0000","last_clean":"2024-09-14T23:57:22.158579+0000","last_became_active":"2024-09-14T23:56:22.822598+0000","last_became_peered":"2024-09-14T23:56:22.822598+0000","last_unstale":"2024-09-14T23:57:22.158579+0000","last_undegraded":"2024-09-14T23:57:22.158579+0000","last_fullsized":"2024-09-14T23:57:22.158579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:32:32.568028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026863400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"69'480","reported_seq":3561,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-14T23:57:22.964631+0000","last_change":"2024-09-14T23:56:34.401366+0000","last_active":"2024-09-14T23:57:22.964631+0000","last_peered":"2024-09-14T23:57:22.964631+0000","last_clean":"2024-09-14T23:57:22.964631+0000","last_became_active":"2024-09-14T23:56:32.397551+0000","last_became_peered":"2024-09-14T23:56:32.397551+0000","last_unstale":"2024-09-14T23:57:22.964631+0000","last_undegraded":"2024-09-14T23:57:22.964631+0000","last_fullsized":"2024-09-14T23:57:22.964631+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":480,"log_dups_size":0,"ondisk_log_size":480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:54:54.801700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001418816,"stat_sum":{"num_bytes":12456069,"num_objects":74,"num_object_clones":12,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":44,"num_read":5172,"num_read_kb":13559,"num_write":645,"num_write_kb":3753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":115,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.145188+0000","last_change":"2024-09-14T23:56:25.093090+0000","last_active":"2024-09-14T23:57:20.145188+0000","last_peered":"2024-09-14T23:57:20.145188+0000","last_clean":"2024-09-14T23:57:20.145188+0000","last_became_active":"2024-09-14T23:56:22.528115+0000","last_became_peered":"2024-09-14T23:56:22.528115+0000","last_unstale":"2024-09-14T23:57:20.145188+0000","last_undegraded":"2024-09-14T23:57:20.145188+0000","last_fullsized":"2024-09-14T23:57:20.145188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:57:36.610730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036520999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"69'376","reported_seq":2103,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.145288+0000","last_change":"2024-09-14T23:57:13.112422+0000","last_active":"2024-09-14T23:57:20.145288+0000","last_peered":"2024-09-14T23:57:20.145288+0000","last_clean":"2024-09-14T23:57:20.145288+0000","last_became_active":"2024-09-14T23:57:13.111521+0000","last_became_peered":"2024-09-14T23:57:13.111521+0000","last_unstale":"2024-09-14T23:57:20.145288+0000","last_undegraded":"2024-09-14T23:57:20.145288+0000","last_fullsized":"2024-09-14T23:57:20.145288+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:00:23.524552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073009599999999996,"stat_sum":{"num_bytes":11422919,"num_objects":76,"num_object_clones":19,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":38,"num_read":2052,"num_read_kb":7652,"num_write":458,"num_write_kb":5227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":7555714,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":117,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-14T23:57:22.158524+0000","last_change":"2024-09-14T23:56:57.054999+0000","last_active":"2024-09-14T23:57:22.158524+0000","last_peered":"2024-09-14T23:57:22.158524+0000","last_clean":"2024-09-14T23:57:22.158524+0000","last_became_active":"2024-09-14T23:56:57.050729+0000","last_became_peered":"2024-09-14T23:56:57.050729+0000","last_unstale":"2024-09-14T23:57:22.158524+0000","last_undegraded":"2024-09-14T23:57:22.158524+0000","last_fullsized":"2024-09-14T23:57:22.158524+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:23:11.727903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":266,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.145200+0000","last_change":"2024-09-14T23:56:15.332936+0000","last_active":"2024-09-14T23:57:20.145200+0000","last_peered":"2024-09-14T23:57:20.145200+0000","last_clean":"2024-09-14T23:57:20.145200+0000","last_became_active":"2024-09-14T23:56:15.331994+0000","last_became_peered":"2024-09-14T23:56:15.331994+0000","last_unstale":"2024-09-14T23:57:20.145200+0000","last_undegraded":"2024-09-14T23:57:20.145200+0000","last_fullsized":"2024-09-14T23:57:20.145200+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:14.322882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:14.322882+0000","last_clean_scrub_stamp":"2024-09-14T23:56:14.322882+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:36:12.639000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"68'155","reported_seq":2555,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.145238+0000","last_change":"2024-09-14T23:56:34.415037+0000","last_active":"2024-09-14T23:57:20.145238+0000","last_peered":"2024-09-14T23:57:20.145238+0000","last_clean":"2024-09-14T23:57:20.145238+0000","last_became_active":"2024-09-14T23:56:32.403572+0000","last_became_peered":"2024-09-14T23:56:32.403572+0000","last_unstale":"2024-09-14T23:57:20.145238+0000","last_undegraded":"2024-09-14T23:57:20.145238+0000","last_fullsized":"2024-09-14T23:57:20.145238+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:18:47.652619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052643700000000004,"stat_sum":{"num_bytes":13852047,"num_objects":39,"num_object_clones":6,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":21,"num_read":2418,"num_read_kb":9967,"num_write":151,"num_write_kb":7232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"68'159","reported_seq":1424,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.146990+0000","last_change":"2024-09-14T23:56:34.404977+0000","last_active":"2024-09-14T23:57:20.146990+0000","last_peered":"2024-09-14T23:57:20.146990+0000","last_clean":"2024-09-14T23:57:20.146990+0000","last_became_active":"2024-09-14T23:56:32.400017+0000","last_became_peered":"2024-09-14T23:56:32.400017+0000","last_unstale":"2024-09-14T23:57:20.146990+0000","last_undegraded":"2024-09-14T23:57:20.146990+0000","last_fullsized":"2024-09-14T23:57:20.146990+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:37:53.000677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068884800000000004,"stat_sum":{"num_bytes":20761281,"num_objects":54,"num_object_clones":11,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":26,"num_read":1220,"num_read_kb":10531,"num_write":162,"num_write_kb":14658,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"68'319","reported_seq":6124,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.153272+0000","last_change":"2024-09-14T23:56:34.498020+0000","last_active":"2024-09-14T23:57:20.153272+0000","last_peered":"2024-09-14T23:57:20.153272+0000","last_clean":"2024-09-14T23:57:20.153272+0000","last_became_active":"2024-09-14T23:56:32.402769+0000","last_became_peered":"2024-09-14T23:56:32.402769+0000","last_unstale":"2024-09-14T23:57:20.153272+0000","last_undegraded":"2024-09-14T23:57:20.153272+0000","last_fullsized":"2024-09-14T23:57:20.153272+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:33:42.371132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035483999999999998,"stat_sum":{"num_bytes":19544689,"num_objects":54,"num_object_clones":8,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":30,"num_read":6477,"num_read_kb":24154,"num_write":405,"num_write_kb":6787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"68'407","reported_seq":12302,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.145228+0000","last_change":"2024-09-14T23:56:34.415414+0000","last_active":"2024-09-14T23:57:20.145228+0000","last_peered":"2024-09-14T23:57:20.145228+0000","last_clean":"2024-09-14T23:57:20.145228+0000","last_became_active":"2024-09-14T23:56:32.403231+0000","last_became_peered":"2024-09-14T23:56:32.403231+0000","last_unstale":"2024-09-14T23:57:20.145228+0000","last_undegraded":"2024-09-14T23:57:20.145228+0000","last_fullsized":"2024-09-14T23:57:20.145228+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:07:07.850750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072791000000000002,"stat_sum":{"num_bytes":36361843,"num_objects":54,"num_object_clones":12,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":27,"num_read":12925,"num_read_kb":55485,"num_write":544,"num_write_kb":7110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"68'310","reported_seq":12157,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.145174+0000","last_change":"2024-09-14T23:57:00.066551+0000","last_active":"2024-09-14T23:57:20.145174+0000","last_peered":"2024-09-14T23:57:20.145174+0000","last_clean":"2024-09-14T23:57:20.145174+0000","last_became_active":"2024-09-14T23:57:00.065790+0000","last_became_peered":"2024-09-14T23:57:00.065790+0000","last_unstale":"2024-09-14T23:57:20.145174+0000","last_undegraded":"2024-09-14T23:57:20.145174+0000","last_fullsized":"2024-09-14T23:57:20.145174+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":310,"log_dups_size":0,"ondisk_log_size":310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:12:47.374150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092638099999999999,"stat_sum":{"num_bytes":46489726,"num_objects":57,"num_object_clones":13,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":26,"num_read":11352,"num_read_kb":50252,"num_write":402,"num_write_kb":18596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":7912941,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"68'164","reported_seq":919,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.145473+0000","last_change":"2024-09-14T23:57:10.147907+0000","last_active":"2024-09-14T23:57:20.145473+0000","last_peered":"2024-09-14T23:57:20.145473+0000","last_clean":"2024-09-14T23:57:20.145473+0000","last_became_active":"2024-09-14T23:57:10.098202+0000","last_became_peered":"2024-09-14T23:57:10.098202+0000","last_unstale":"2024-09-14T23:57:20.145473+0000","last_undegraded":"2024-09-14T23:57:20.145473+0000","last_fullsized":"2024-09-14T23:57:20.145473+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:37:04.579298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057774300000000003,"stat_sum":{"num_bytes":8653118,"num_objects":44,"num_object_clones":7,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":24,"num_read":606,"num_read_kb":6074,"num_write":175,"num_write_kb":4151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":5634914,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"68'205","reported_seq":3038,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.145393+0000","last_change":"2024-09-14T23:56:34.399894+0000","last_active":"2024-09-14T23:57:20.145393+0000","last_peered":"2024-09-14T23:57:20.145393+0000","last_clean":"2024-09-14T23:57:20.145393+0000","last_became_active":"2024-09-14T23:56:32.406251+0000","last_became_peered":"2024-09-14T23:56:32.406251+0000","last_unstale":"2024-09-14T23:57:20.145393+0000","last_undegraded":"2024-09-14T23:57:20.145393+0000","last_fullsized":"2024-09-14T23:57:20.145393+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:38:57.846896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073115799999999998,"stat_sum":{"num_bytes":22285447,"num_objects":64,"num_object_clones":13,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":32,"num_read":3565,"num_read_kb":15855,"num_write":203,"num_write_kb":10186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"68'266","reported_seq":3164,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.145425+0000","last_change":"2024-09-14T23:56:34.399966+0000","last_active":"2024-09-14T23:57:20.145425+0000","last_peered":"2024-09-14T23:57:20.145425+0000","last_clean":"2024-09-14T23:57:20.145425+0000","last_became_active":"2024-09-14T23:56:32.407109+0000","last_became_peered":"2024-09-14T23:56:32.407109+0000","last_unstale":"2024-09-14T23:57:20.145425+0000","last_undegraded":"2024-09-14T23:57:20.145425+0000","last_fullsized":"2024-09-14T23:57:20.145425+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":266,"log_dups_size":0,"ondisk_log_size":266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:04:11.394853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072766900000000002,"stat_sum":{"num_bytes":28667677,"num_objects":62,"num_object_clones":11,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":31,"num_read":4327,"num_read_kb":11372,"num_write":240,"num_write_kb":6108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"69'321","reported_seq":787,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.199463+0000","last_change":"2024-09-14T23:56:34.498033+0000","last_active":"2024-09-14T23:57:20.199463+0000","last_peered":"2024-09-14T23:57:20.199463+0000","last_clean":"2024-09-14T23:57:20.199463+0000","last_became_active":"2024-09-14T23:56:32.402973+0000","last_became_peered":"2024-09-14T23:56:32.402973+0000","last_unstale":"2024-09-14T23:57:20.199463+0000","last_undegraded":"2024-09-14T23:57:20.199463+0000","last_fullsized":"2024-09-14T23:57:20.199463+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:03:53.757298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035072199999999998,"stat_sum":{"num_bytes":17521363,"num_objects":55,"num_object_clones":8,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":31,"num_read":1299,"num_read_kb":8913,"num_write":444,"num_write_kb":5822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"69'212","reported_seq":4973,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.161932+0000","last_change":"2024-09-14T23:56:34.399387+0000","last_active":"2024-09-14T23:57:20.161932+0000","last_peered":"2024-09-14T23:57:20.161932+0000","last_clean":"2024-09-14T23:57:20.161932+0000","last_became_active":"2024-09-14T23:56:32.405457+0000","last_became_peered":"2024-09-14T23:56:32.405457+0000","last_unstale":"2024-09-14T23:57:20.161932+0000","last_undegraded":"2024-09-14T23:57:20.161932+0000","last_fullsized":"2024-09-14T23:57:20.161932+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:14:07.750188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048125700000000002,"stat_sum":{"num_bytes":25343151,"num_objects":59,"num_object_clones":12,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":31,"num_read":5012,"num_read_kb":23215,"num_write":194,"num_write_kb":5929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"71'136","reported_seq":1012,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-14T23:57:22.964679+0000","last_change":"2024-09-14T23:56:34.402592+0000","last_active":"2024-09-14T23:57:22.964679+0000","last_peered":"2024-09-14T23:57:22.964679+0000","last_clean":"2024-09-14T23:57:22.964679+0000","last_became_active":"2024-09-14T23:56:32.402782+0000","last_became_peered":"2024-09-14T23:56:32.402782+0000","last_unstale":"2024-09-14T23:57:22.964679+0000","last_undegraded":"2024-09-14T23:57:22.964679+0000","last_fullsized":"2024-09-14T23:57:22.964679+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:31:15.301532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001446697,"stat_sum":{"num_bytes":7737191,"num_objects":44,"num_object_clones":9,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":25,"num_read":808,"num_read_kb":3837,"num_write":115,"num_write_kb":2228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"68'534","reported_seq":2466,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T23:57:20.145315+0000","last_change":"2024-09-14T23:56:34.399629+0000","last_active":"2024-09-14T23:57:20.145315+0000","last_peered":"2024-09-14T23:57:20.145315+0000","last_clean":"2024-09-14T23:57:20.145315+0000","last_became_active":"2024-09-14T23:56:32.407220+0000","last_became_peered":"2024-09-14T23:56:32.407220+0000","last_unstale":"2024-09-14T23:57:20.145315+0000","last_undegraded":"2024-09-14T23:57:20.145315+0000","last_fullsized":"2024-09-14T23:57:20.145315+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:06.294375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062462200000000002,"stat_sum":{"num_bytes":15958982,"num_objects":54,"num_object_clones":13,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":27,"num_read":4215,"num_read_kb":15293,"num_write":817,"num_write_kb":9773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":640382298,"num_objects":1880,"num_object_clones":352,"num_object_copies":3760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1880,"num_whiteouts":979,"num_read":125827,"num_read_kb":516643,"num_write":10853,"num_write_kb":265510,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":838,"num_bytes_recovered":37168120,"num_keys_recovered":121,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":43544576,"data_stored":411583131,"data_compressed":30657550,"data_compressed_allocated":33435648,"data_compressed_original":403189358,"omap_allocated":0,"internal_metadata":0},"log_size":9198,"ondisk_log_size":9198,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64594,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509458,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47620,"kb_used_data":8108,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94324220,"statfs":{"total":96636764160,"available":96588001280,"internally_reserved":0,"allocated":8302592,"data_stored":72536130,"data_compressed":5521083,"data_compressed_allocated":6008832,"data_compressed_original":70944105,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509457,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39272,"kb_used_data":3920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94332568,"statfs":{"total":96636764160,"available":96596549632,"internally_reserved":0,"allocated":4014080,"data_stored":27265247,"data_compressed":2172617,"data_compressed_allocated":2424832,"data_compressed_original":26250104,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,20,52,78,65],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44756,"kb_used_data":5248,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94327084,"statfs":{"total":96636764160,"available":96590934016,"internally_reserved":0,"allocated":5373952,"data_stored":47685560,"data_compressed":3461412,"data_compressed_allocated":3665920,"data_compressed_original":46584663,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509458,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35484,"kb_used_data":8448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94336356,"statfs":{"total":96636764160,"available":96600428544,"internally_reserved":0,"allocated":8650752,"data_stored":72653434,"data_compressed":5273757,"data_compressed_allocated":6004736,"data_compressed_original":70769396,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509456,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35084,"kb_used_data":8060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94336756,"statfs":{"total":96636764160,"available":96600838144,"internally_reserved":0,"allocated":8253440,"data_stored":76461622,"data_compressed":5558873,"data_compressed_allocated":5922816,"data_compressed_original":74878412,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542161,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45164,"kb_used_data":5656,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94326676,"statfs":{"total":96636764160,"available":96590516224,"internally_reserved":0,"allocated":5791744,"data_stored":47337809,"data_compressed":3430073,"data_compressed_allocated":3657728,"data_compressed_original":45835620,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542160,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32492,"kb_used_data":5464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339348,"statfs":{"total":96636764160,"available":96603492352,"internally_reserved":0,"allocated":5595136,"data_stored":43000395,"data_compressed":3079076,"data_compressed_allocated":3403776,"data_compressed_original":41483458,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38664,"kb_used_data":3308,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333176,"statfs":{"total":96636764160,"available":96597172224,"internally_reserved":0,"allocated":3387392,"data_stored":22521516,"data_compressed":1632961,"data_compressed_allocated":1728512,"data_compressed_original":21389136,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32297,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32297,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4706304,"data_stored":42036085,"data_compressed":3046779,"data_compressed_allocated":3358720,"data_compressed_original":40910018,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4964352,"data_stored":46963848,"data_compressed":3430073,"data_compressed_allocated":3657728,"data_compressed_original":45835620,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2564096,"data_stored":22147574,"data_compressed":1632961,"data_compressed_allocated":1728512,"data_compressed_original":21389136,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":7770112,"data_stored":78234235,"data_compressed":5784686,"data_compressed_allocated":6189056,"data_compressed_original":76975564,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8122368,"data_stored":75331873,"data_compressed":5532679,"data_compressed_allocated":6320128,"data_compressed_original":73857780,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4595712,"data_stored":47337995,"data_compressed":3461412,"data_compressed_allocated":3665920,"data_compressed_original":46584663,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3244032,"data_stored":26925543,"data_compressed":2172617,"data_compressed_allocated":2424832,"data_compressed_original":26250104,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":7577600,"data_stored":72605978,"data_compressed":5596343,"data_compressed_allocated":6090752,"data_compressed_original":71386473,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T23:57:23.586 INFO:tasks.thrashosds.thrasher:Setting 3.1e to [6, 1] 2024-09-14T23:57:23.586 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.1e', '6', '1'] 2024-09-14T23:57:23.586 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.1e 6 1 2024-09-14T23:57:23.598 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:23.598 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:23.607 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T23:57:23.668 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:23.666+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:23.714 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:23.714 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:23.723 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T23:57:23.766 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:23.765+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:23.829 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:23.829 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:23.838 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T23:57:23.846 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:23.844+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:23.947 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:23.946+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:23.964 INFO:teuthology.orchestra.run.smithi042.stderr:set 3.1e pg_upmap mapping to [6,1] 2024-09-14T23:57:23.988 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:23.988 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:23.997 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T23:57:24.063 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:24.061+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:24.124 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T23:57:24.127 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:24.127 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:24.136 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T23:57:24.178 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:24.175+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:24.178 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T23:57:24.250 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:24.249+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:24.256 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:57:24.256 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:57:24.273 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:57:24.273 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:57:24.290 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:24.291 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:57:24.291 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:57:24.291 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:57:24.291 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:24.291 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:57:24.291 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:57:24.301 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T23:57:24.309 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:57:24.309 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:57:24.326 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:57:24.326 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:57:24.343 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:57:24.343 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:57:24.351 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:24.349+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:24.360 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:57:24.360 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:57:24.377 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:57:24.377 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:57:24.419 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:24.419 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:57:24.419 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:57:24.419 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:57:24.419 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.420 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:43054 3.1f 3:ffbfe596:::rbd_data.1118350d5ae5.00000000000002eb:head [set-alloc-hint object_size 262144 write_size 262144,write 196608~22770] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T23:57:24.420 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:57:24.420 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 31.450597762000001, 2024-09-14T23:57:24.420 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.99973334400000002, 2024-09-14T23:57:24.420 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:24.420 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:24.420 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:24.420 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:24.420 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:24.420 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:24.420 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 43054 2024-09-14T23:57:24.420 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.420 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:24.420 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.421 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:24.421 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:57:24.421 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:24.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.421 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.421 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:24.421 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:57:24.421 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:24.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.421 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.422 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:24.422 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966552+0000", 2024-09-14T23:57:24.422 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3790000000000001e-06 2024-09-14T23:57:24.422 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.422 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.422 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:24.422 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966565+0000", 2024-09-14T23:57:24.422 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2459999999999999e-05 2024-09-14T23:57:24.422 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.422 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.422 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:24.422 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966576+0000", 2024-09-14T23:57:24.422 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1039000000000001e-05 2024-09-14T23:57:24.423 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.423 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.423 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:24.423 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966582+0000", 2024-09-14T23:57:24.423 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9660000000000001e-06 2024-09-14T23:57:24.423 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.423 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.423 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:24.423 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966625+0000", 2024-09-14T23:57:24.424 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3529999999999998e-05 2024-09-14T23:57:24.424 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.424 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.424 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:57:24.424 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966632+0000", 2024-09-14T23:57:24.424 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9809999999999997e-06 2024-09-14T23:57:24.424 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.424 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.424 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:24.424 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.964460+0000", 2024-09-14T23:57:24.424 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.99782768300000002 2024-09-14T23:57:24.424 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.424 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.424 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:24.425 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.964641+0000", 2024-09-14T23:57:24.425 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000181089 2024-09-14T23:57:24.425 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.425 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.425 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-14T23:57:24.425 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.964809+0000", 2024-09-14T23:57:24.425 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000168505 2024-09-14T23:57:24.425 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.425 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.425 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:57:24.425 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966030+0000", 2024-09-14T23:57:24.425 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012206929999999999 2024-09-14T23:57:24.425 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.425 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.426 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:24.426 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966181+0000", 2024-09-14T23:57:24.426 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000150569 2024-09-14T23:57:24.426 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.426 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.426 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:24.426 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966243+0000", 2024-09-14T23:57:24.426 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2331999999999999e-05 2024-09-14T23:57:24.426 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.426 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.426 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:24.426 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966284+0000", 2024-09-14T23:57:24.426 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1118000000000002e-05 2024-09-14T23:57:24.427 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:24.427 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:24.427 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:24.427 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.427 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.427 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:54673 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:24.427 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.167989+0000", 2024-09-14T23:57:24.427 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.249159412000001, 2024-09-14T23:57:24.427 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027580298, 2024-09-14T23:57:24.427 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:24.427 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:24.427 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:24.427 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:24.427 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:24.428 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:24.428 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54673 2024-09-14T23:57:24.428 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.428 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:24.428 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.428 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:24.428 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.167989+0000", 2024-09-14T23:57:24.428 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:24.428 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.428 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.428 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:24.428 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.167989+0000", 2024-09-14T23:57:24.428 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:24.428 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.429 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:24.429 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.167989+0000", 2024-09-14T23:57:24.429 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5499999999999998e-07 2024-09-14T23:57:24.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.429 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:24.429 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.167998+0000", 2024-09-14T23:57:24.429 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7369999999999995e-06 2024-09-14T23:57:24.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.429 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:24.429 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168005+0000", 2024-09-14T23:57:24.430 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8210000000000004e-06 2024-09-14T23:57:24.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.430 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:24.430 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168010+0000", 2024-09-14T23:57:24.430 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2510000000000002e-06 2024-09-14T23:57:24.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.430 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:24.430 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.195432+0000", 2024-09-14T23:57:24.430 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027421364 2024-09-14T23:57:24.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.430 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:24.431 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.195501+0000", 2024-09-14T23:57:24.431 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9611999999999997e-05 2024-09-14T23:57:24.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.431 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:24.431 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.195569+0000", 2024-09-14T23:57:24.431 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8058000000000005e-05 2024-09-14T23:57:24.431 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:24.431 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:24.431 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:24.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.431 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:54674 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 2637824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:24.431 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.168018+0000", 2024-09-14T23:57:24.431 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.249130768000001, 2024-09-14T23:57:24.432 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027716159000000001, 2024-09-14T23:57:24.432 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:24.432 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:24.432 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:24.432 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:24.432 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:24.432 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:24.432 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54674 2024-09-14T23:57:24.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.432 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:24.432 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.432 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:24.432 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168018+0000", 2024-09-14T23:57:24.433 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:24.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.433 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.433 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:24.433 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168018+0000", 2024-09-14T23:57:24.433 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:24.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.433 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.433 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:24.433 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168018+0000", 2024-09-14T23:57:24.433 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2600000000000002e-07 2024-09-14T23:57:24.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.433 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.433 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:24.434 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168020+0000", 2024-09-14T23:57:24.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9420000000000002e-06 2024-09-14T23:57:24.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.434 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:24.434 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168026+0000", 2024-09-14T23:57:24.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8880000000000002e-06 2024-09-14T23:57:24.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.434 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:24.434 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168031+0000", 2024-09-14T23:57:24.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.977e-06 2024-09-14T23:57:24.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.435 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:24.435 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.195599+0000", 2024-09-14T23:57:24.435 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027567758000000001 2024-09-14T23:57:24.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.435 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:24.435 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.195666+0000", 2024-09-14T23:57:24.435 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7102999999999998e-05 2024-09-14T23:57:24.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.435 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:24.435 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.195734+0000", 2024-09-14T23:57:24.435 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7865000000000002e-05 2024-09-14T23:57:24.436 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:24.436 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:24.436 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:24.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.436 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:54675 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:24.436 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.168039+0000", 2024-09-14T23:57:24.436 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.249109809, 2024-09-14T23:57:24.436 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027868378999999999, 2024-09-14T23:57:24.436 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:24.436 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:24.436 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:24.436 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:24.437 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:24.437 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:24.437 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54675 2024-09-14T23:57:24.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.437 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:24.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.437 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:24.437 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168039+0000", 2024-09-14T23:57:24.437 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:24.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.437 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:24.437 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168039+0000", 2024-09-14T23:57:24.437 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:24.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.438 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.438 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:24.438 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168039+0000", 2024-09-14T23:57:24.438 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3099999999999998e-07 2024-09-14T23:57:24.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.438 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.438 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:24.438 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168048+0000", 2024-09-14T23:57:24.438 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9460000000000004e-06 2024-09-14T23:57:24.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.438 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.438 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:24.438 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168055+0000", 2024-09-14T23:57:24.438 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5740000000000001e-06 2024-09-14T23:57:24.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.439 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.439 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:24.439 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168060+0000", 2024-09-14T23:57:24.439 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1909999999999999e-06 2024-09-14T23:57:24.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.439 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.439 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:24.439 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.195765+0000", 2024-09-14T23:57:24.439 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027705244 2024-09-14T23:57:24.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.439 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.439 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:24.439 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.195836+0000", 2024-09-14T23:57:24.439 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0642999999999994e-05 2024-09-14T23:57:24.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.440 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.440 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:24.440 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.195907+0000", 2024-09-14T23:57:24.440 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.135e-05 2024-09-14T23:57:24.440 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:24.440 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:24.440 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:24.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.440 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.440 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:54676 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 2998272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:24.440 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.168067+0000", 2024-09-14T23:57:24.440 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.249081076, 2024-09-14T23:57:24.440 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028002955999999999, 2024-09-14T23:57:24.440 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:24.441 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:24.441 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:24.441 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:24.441 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:24.441 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:24.441 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54676 2024-09-14T23:57:24.441 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.441 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:24.441 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.441 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:24.441 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168067+0000", 2024-09-14T23:57:24.441 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:24.441 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.441 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.442 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:24.442 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168067+0000", 2024-09-14T23:57:24.442 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:24.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.442 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.442 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:24.442 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168068+0000", 2024-09-14T23:57:24.442 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6899999999999998e-07 2024-09-14T23:57:24.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.442 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.442 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:24.442 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168070+0000", 2024-09-14T23:57:24.442 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9810000000000002e-06 2024-09-14T23:57:24.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.442 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.442 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:24.443 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168076+0000", 2024-09-14T23:57:24.443 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4280000000000001e-06 2024-09-14T23:57:24.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.443 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.443 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:24.443 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168091+0000", 2024-09-14T23:57:24.443 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5152e-05 2024-09-14T23:57:24.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.443 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.443 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:24.443 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.195936+0000", 2024-09-14T23:57:24.443 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027844892 2024-09-14T23:57:24.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.443 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.444 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:24.444 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196002+0000", 2024-09-14T23:57:24.444 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5633e-05 2024-09-14T23:57:24.444 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.444 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.444 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:24.444 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196070+0000", 2024-09-14T23:57:24.444 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8400999999999998e-05 2024-09-14T23:57:24.444 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:24.444 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:24.444 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:24.444 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.444 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.444 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:54677 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 3178496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:24.444 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.168099+0000", 2024-09-14T23:57:24.445 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.249049777, 2024-09-14T23:57:24.445 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028165816, 2024-09-14T23:57:24.445 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:24.445 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:24.445 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:24.445 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:24.445 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:24.445 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:24.445 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54677 2024-09-14T23:57:24.445 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.445 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:24.445 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.445 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:24.445 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168099+0000", 2024-09-14T23:57:24.446 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:24.446 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.446 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.446 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:24.446 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168099+0000", 2024-09-14T23:57:24.446 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:24.446 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.446 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.446 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:24.446 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168099+0000", 2024-09-14T23:57:24.446 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7400000000000003e-07 2024-09-14T23:57:24.446 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.446 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.447 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:24.447 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168102+0000", 2024-09-14T23:57:24.447 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2929999999999999e-06 2024-09-14T23:57:24.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.447 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.447 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:24.447 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168108+0000", 2024-09-14T23:57:24.447 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5429999999999999e-06 2024-09-14T23:57:24.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.447 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.447 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:24.447 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168113+0000", 2024-09-14T23:57:24.447 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8470000000000003e-06 2024-09-14T23:57:24.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.448 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.448 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:24.448 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196124+0000", 2024-09-14T23:57:24.448 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028011034000000001 2024-09-14T23:57:24.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.448 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.448 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:24.448 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196194+0000", 2024-09-14T23:57:24.448 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0314000000000005e-05 2024-09-14T23:57:24.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.448 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.448 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:24.449 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196264+0000", 2024-09-14T23:57:24.449 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0210999999999996e-05 2024-09-14T23:57:24.449 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:24.449 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:24.449 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:24.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.449 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.449 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:54678 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 3358720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:24.449 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.168144+0000", 2024-09-14T23:57:24.449 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.249004618000001, 2024-09-14T23:57:24.449 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028285267999999999, 2024-09-14T23:57:24.449 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:24.449 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:24.450 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:24.450 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:24.450 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:24.450 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:24.450 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54678 2024-09-14T23:57:24.450 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.450 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:24.450 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.450 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:24.450 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168144+0000", 2024-09-14T23:57:24.450 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:24.450 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.450 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.450 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:24.451 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168144+0000", 2024-09-14T23:57:24.451 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:24.451 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.451 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.451 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:24.451 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168144+0000", 2024-09-14T23:57:24.451 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5799999999999999e-07 2024-09-14T23:57:24.451 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.451 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.451 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:24.451 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168147+0000", 2024-09-14T23:57:24.451 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3580000000000001e-06 2024-09-14T23:57:24.451 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.451 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.452 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:24.452 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168154+0000", 2024-09-14T23:57:24.452 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1350000000000003e-06 2024-09-14T23:57:24.452 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.452 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.452 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:24.452 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168159+0000", 2024-09-14T23:57:24.452 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4160000000000003e-06 2024-09-14T23:57:24.452 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.452 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.452 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:24.452 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196294+0000", 2024-09-14T23:57:24.452 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028135050000000002 2024-09-14T23:57:24.452 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.453 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.453 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:24.453 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196363+0000", 2024-09-14T23:57:24.453 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8880000000000005e-05 2024-09-14T23:57:24.453 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.453 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.453 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:24.453 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196429+0000", 2024-09-14T23:57:24.453 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5870999999999993e-05 2024-09-14T23:57:24.453 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:24.453 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:24.453 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:24.453 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.454 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.454 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:54679 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:24.454 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.168167+0000", 2024-09-14T23:57:24.454 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.248981397000001, 2024-09-14T23:57:24.454 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028427019000000001, 2024-09-14T23:57:24.454 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:24.454 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:24.454 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:24.454 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:24.454 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:24.454 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:24.454 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54679 2024-09-14T23:57:24.454 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.454 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:24.455 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.455 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:24.455 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168167+0000", 2024-09-14T23:57:24.455 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:24.455 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.455 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.455 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:24.455 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168167+0000", 2024-09-14T23:57:24.455 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:24.455 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.455 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.455 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:24.455 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168168+0000", 2024-09-14T23:57:24.455 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.44e-07 2024-09-14T23:57:24.456 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.456 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.456 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:24.456 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168170+0000", 2024-09-14T23:57:24.456 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0870000000000001e-06 2024-09-14T23:57:24.456 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.456 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.456 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:24.456 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168176+0000", 2024-09-14T23:57:24.456 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9070000000000004e-06 2024-09-14T23:57:24.456 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.456 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.456 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:24.456 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168181+0000", 2024-09-14T23:57:24.456 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6579999999999999e-06 2024-09-14T23:57:24.457 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.457 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.457 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:24.457 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196458+0000", 2024-09-14T23:57:24.457 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028276431000000001 2024-09-14T23:57:24.457 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.457 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.457 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:24.457 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196527+0000", 2024-09-14T23:57:24.457 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9397e-05 2024-09-14T23:57:24.457 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.457 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.457 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:24.457 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196594+0000", 2024-09-14T23:57:24.458 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6895000000000002e-05 2024-09-14T23:57:24.458 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:24.458 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:24.458 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:24.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.458 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.458 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:54680 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 3719168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:24.458 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.168188+0000", 2024-09-14T23:57:24.458 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.248960190999998, 2024-09-14T23:57:24.458 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028579706999999999, 2024-09-14T23:57:24.458 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:24.458 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:24.458 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:24.458 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:24.458 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:24.459 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:24.459 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54680 2024-09-14T23:57:24.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.459 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:24.459 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.459 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:24.459 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168188+0000", 2024-09-14T23:57:24.459 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:24.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.459 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.459 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:24.459 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168188+0000", 2024-09-14T23:57:24.459 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:24.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.460 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.460 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:24.460 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168189+0000", 2024-09-14T23:57:24.460 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3099999999999998e-07 2024-09-14T23:57:24.460 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.460 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.460 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:24.460 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168191+0000", 2024-09-14T23:57:24.460 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0470000000000001e-06 2024-09-14T23:57:24.460 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.460 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.460 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:24.460 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168199+0000", 2024-09-14T23:57:24.460 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8809999999999996e-06 2024-09-14T23:57:24.460 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.460 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.461 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:24.461 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168204+0000", 2024-09-14T23:57:24.461 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.011e-06 2024-09-14T23:57:24.461 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.461 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.461 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:24.461 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196622+0000", 2024-09-14T23:57:24.461 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028418806000000001 2024-09-14T23:57:24.461 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.461 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.461 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:24.461 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196694+0000", 2024-09-14T23:57:24.461 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1124000000000005e-05 2024-09-14T23:57:24.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.462 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.462 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:24.462 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196768+0000", 2024-09-14T23:57:24.462 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4406999999999994e-05 2024-09-14T23:57:24.462 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:24.462 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:24.462 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:24.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.462 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.462 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:54681 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 3899392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:24.462 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.168212+0000", 2024-09-14T23:57:24.462 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.248936479000001, 2024-09-14T23:57:24.462 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028726643, 2024-09-14T23:57:24.463 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:24.463 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:24.463 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:24.463 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:24.463 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:24.463 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:24.463 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54681 2024-09-14T23:57:24.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.463 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:24.463 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.463 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:24.463 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168212+0000", 2024-09-14T23:57:24.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:24.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.464 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:24.464 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168212+0000", 2024-09-14T23:57:24.464 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:24.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.464 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:24.464 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168212+0000", 2024-09-14T23:57:24.464 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4900000000000001e-07 2024-09-14T23:57:24.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.464 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:24.464 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168214+0000", 2024-09-14T23:57:24.465 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0590000000000001e-06 2024-09-14T23:57:24.465 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.465 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.465 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:24.465 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168220+0000", 2024-09-14T23:57:24.465 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0209999999999996e-06 2024-09-14T23:57:24.465 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.465 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.465 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:24.465 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168227+0000", 2024-09-14T23:57:24.465 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4119999999999998e-06 2024-09-14T23:57:24.465 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.465 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.465 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:24.466 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196799+0000", 2024-09-14T23:57:24.466 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028572528 2024-09-14T23:57:24.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.466 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.466 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:24.466 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196866+0000", 2024-09-14T23:57:24.466 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7005e-05 2024-09-14T23:57:24.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.466 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.466 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:24.466 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196939+0000", 2024-09-14T23:57:24.466 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2169000000000006e-05 2024-09-14T23:57:24.466 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:24.466 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:24.467 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:24.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.467 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.467 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:54682 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 4079616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:24.467 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.168234+0000", 2024-09-14T23:57:24.467 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.248914930000002, 2024-09-14T23:57:24.467 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028872503000000001, 2024-09-14T23:57:24.467 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:24.467 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:24.467 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:24.467 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:24.467 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:24.467 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:24.467 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54682 2024-09-14T23:57:24.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.467 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:24.468 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.468 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:24.468 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168234+0000", 2024-09-14T23:57:24.468 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:24.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.468 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.468 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:24.468 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168234+0000", 2024-09-14T23:57:24.468 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:24.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.468 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.468 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:24.468 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168234+0000", 2024-09-14T23:57:24.468 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.58e-07 2024-09-14T23:57:24.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.469 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.469 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:24.469 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168236+0000", 2024-09-14T23:57:24.469 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.933e-06 2024-09-14T23:57:24.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.469 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.469 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:24.469 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168242+0000", 2024-09-14T23:57:24.469 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9970000000000004e-06 2024-09-14T23:57:24.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.469 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.469 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:24.469 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168247+0000", 2024-09-14T23:57:24.470 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9280000000000001e-06 2024-09-14T23:57:24.470 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.470 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.470 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:24.470 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196968+0000", 2024-09-14T23:57:24.470 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028721159999999999 2024-09-14T23:57:24.470 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.470 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.470 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:24.470 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.197040+0000", 2024-09-14T23:57:24.470 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1569999999999994e-05 2024-09-14T23:57:24.470 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.470 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.470 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:24.470 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.197106+0000", 2024-09-14T23:57:24.470 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6457000000000004e-05 2024-09-14T23:57:24.471 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:24.471 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:24.471 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:24.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.471 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.471 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:55239 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 3850240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:24.471 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.201093+0000", 2024-09-14T23:57:24.471 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.216055479000001, 2024-09-14T23:57:24.471 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027610927, 2024-09-14T23:57:24.471 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:24.471 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:24.471 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:24.471 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:24.471 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:24.471 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:24.472 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 55239 2024-09-14T23:57:24.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.472 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:24.472 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.472 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:24.472 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201093+0000", 2024-09-14T23:57:24.472 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:24.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.472 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.472 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:24.472 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201093+0000", 2024-09-14T23:57:24.472 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:24.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.472 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.473 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:24.473 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201093+0000", 2024-09-14T23:57:24.473 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5699999999999998e-07 2024-09-14T23:57:24.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.473 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:24.473 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201096+0000", 2024-09-14T23:57:24.473 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0990000000000001e-06 2024-09-14T23:57:24.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.473 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:24.473 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201102+0000", 2024-09-14T23:57:24.473 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5710000000000003e-06 2024-09-14T23:57:24.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.474 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.474 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:24.474 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201107+0000", 2024-09-14T23:57:24.474 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1019999999999996e-06 2024-09-14T23:57:24.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.474 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.474 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:24.474 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.228568+0000", 2024-09-14T23:57:24.474 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027461136000000001 2024-09-14T23:57:24.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.474 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.474 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:24.474 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.228638+0000", 2024-09-14T23:57:24.474 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0024999999999994e-05 2024-09-14T23:57:24.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.475 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.475 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:24.475 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.228704+0000", 2024-09-14T23:57:24.475 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5537000000000006e-05 2024-09-14T23:57:24.475 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:24.475 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:24.475 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:24.475 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.475 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.475 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:55240 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:24.475 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.201114+0000", 2024-09-14T23:57:24.475 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.216034242999999, 2024-09-14T23:57:24.475 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027752295, 2024-09-14T23:57:24.475 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:24.475 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:24.476 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:24.476 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:24.476 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:24.476 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:24.476 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 55240 2024-09-14T23:57:24.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.476 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:24.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.476 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:24.476 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201114+0000", 2024-09-14T23:57:24.476 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:24.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.476 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:24.477 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201114+0000", 2024-09-14T23:57:24.477 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:24.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.477 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.477 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:24.477 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201115+0000", 2024-09-14T23:57:24.477 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4200000000000001e-07 2024-09-14T23:57:24.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.477 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.477 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:24.477 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201117+0000", 2024-09-14T23:57:24.477 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.368e-06 2024-09-14T23:57:24.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.477 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.477 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:24.478 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201124+0000", 2024-09-14T23:57:24.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4999999999999996e-06 2024-09-14T23:57:24.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.478 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:24.478 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201136+0000", 2024-09-14T23:57:24.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.293e-05 2024-09-14T23:57:24.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.478 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:24.478 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.228736+0000", 2024-09-14T23:57:24.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027599597 2024-09-14T23:57:24.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.479 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:24.479 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.228806+0000", 2024-09-14T23:57:24.479 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9616999999999994e-05 2024-09-14T23:57:24.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.479 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:24.479 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.228866+0000", 2024-09-14T23:57:24.479 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0841000000000002e-05 2024-09-14T23:57:24.479 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:24.479 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:24.479 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:24.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.479 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:55349 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 868352~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:24.480 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.204621+0000", 2024-09-14T23:57:24.480 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.212527077000001, 2024-09-14T23:57:24.480 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030294596999999999, 2024-09-14T23:57:24.480 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:24.480 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:24.480 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:24.480 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:24.480 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:24.480 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:24.480 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 55349 2024-09-14T23:57:24.480 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.480 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:24.480 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.480 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:24.481 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204621+0000", 2024-09-14T23:57:24.481 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:24.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.481 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.481 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:24.481 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204621+0000", 2024-09-14T23:57:24.481 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:24.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.481 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.481 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:24.481 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204622+0000", 2024-09-14T23:57:24.481 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.58e-07 2024-09-14T23:57:24.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.481 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.481 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:24.482 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204624+0000", 2024-09-14T23:57:24.482 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.193e-06 2024-09-14T23:57:24.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.482 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.482 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:24.482 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204631+0000", 2024-09-14T23:57:24.482 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1910000000000003e-06 2024-09-14T23:57:24.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.482 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.482 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:24.482 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204637+0000", 2024-09-14T23:57:24.482 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5169999999999999e-06 2024-09-14T23:57:24.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.482 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.483 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:24.483 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.229653+0000", 2024-09-14T23:57:24.483 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025016283 2024-09-14T23:57:24.483 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.483 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.483 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:24.483 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.229721+0000", 2024-09-14T23:57:24.483 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8138000000000004e-05 2024-09-14T23:57:24.483 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.483 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.483 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:24.483 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.234916+0000", 2024-09-14T23:57:24.483 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.005194817 2024-09-14T23:57:24.483 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:24.483 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:24.484 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:24.484 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.484 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.484 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:55363 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 3145728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:24.484 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.204982+0000", 2024-09-14T23:57:24.484 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.212166221, 2024-09-14T23:57:24.484 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027646239999999999, 2024-09-14T23:57:24.484 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:24.484 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:24.484 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:24.484 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:24.484 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:24.484 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:24.484 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 55363 2024-09-14T23:57:24.484 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.485 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:24.485 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.485 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:24.485 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204982+0000", 2024-09-14T23:57:24.485 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:24.485 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.485 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.485 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:24.485 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204982+0000", 2024-09-14T23:57:24.485 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:24.485 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.485 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.485 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:24.485 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204983+0000", 2024-09-14T23:57:24.485 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2199999999999999e-07 2024-09-14T23:57:24.486 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.486 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.486 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:24.486 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204985+0000", 2024-09-14T23:57:24.486 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.261e-06 2024-09-14T23:57:24.486 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.486 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.486 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:24.486 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204991+0000", 2024-09-14T23:57:24.486 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7069999999999997e-06 2024-09-14T23:57:24.486 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.486 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.486 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:24.486 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204995+0000", 2024-09-14T23:57:24.486 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.853e-06 2024-09-14T23:57:24.486 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.487 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.487 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:24.487 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.232491+0000", 2024-09-14T23:57:24.487 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027495322999999999 2024-09-14T23:57:24.487 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.487 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.487 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:24.487 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.232563+0000", 2024-09-14T23:57:24.487 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1972999999999997e-05 2024-09-14T23:57:24.487 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.487 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.487 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:24.487 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.232628+0000", 2024-09-14T23:57:24.487 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5701e-05 2024-09-14T23:57:24.487 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:24.487 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:24.488 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:24.488 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.488 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.488 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:55364 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 3325952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:24.488 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.205003+0000", 2024-09-14T23:57:24.488 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.212145696, 2024-09-14T23:57:24.488 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027781963, 2024-09-14T23:57:24.488 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:24.488 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:24.488 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:24.488 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:24.488 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:24.488 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:24.488 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 55364 2024-09-14T23:57:24.489 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.489 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:24.489 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.489 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:24.489 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205003+0000", 2024-09-14T23:57:24.489 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:24.489 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.489 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.489 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:24.489 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205003+0000", 2024-09-14T23:57:24.489 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:24.489 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.489 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.489 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:24.490 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205003+0000", 2024-09-14T23:57:24.490 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2899999999999999e-07 2024-09-14T23:57:24.490 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.490 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.490 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:24.490 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205013+0000", 2024-09-14T23:57:24.490 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.611e-06 2024-09-14T23:57:24.490 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.490 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.490 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:24.490 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205020+0000", 2024-09-14T23:57:24.490 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2509999999999997e-06 2024-09-14T23:57:24.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.491 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.491 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:24.491 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205025+0000", 2024-09-14T23:57:24.491 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1399999999999999e-06 2024-09-14T23:57:24.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.491 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.491 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:24.491 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.232653+0000", 2024-09-14T23:57:24.491 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027627972000000001 2024-09-14T23:57:24.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.491 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.492 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:24.492 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.232723+0000", 2024-09-14T23:57:24.492 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9907999999999995e-05 2024-09-14T23:57:24.492 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.492 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.492 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:24.492 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.232785+0000", 2024-09-14T23:57:24.492 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1651999999999999e-05 2024-09-14T23:57:24.492 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:24.492 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:24.492 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:24.492 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.492 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.492 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:55365 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 3506176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:24.493 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.205033+0000", 2024-09-14T23:57:24.493 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.212115456999999, 2024-09-14T23:57:24.493 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027914906999999999, 2024-09-14T23:57:24.493 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:24.493 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:24.493 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:24.493 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:24.493 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:24.493 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:24.493 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 55365 2024-09-14T23:57:24.493 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.493 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:24.493 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.494 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:24.494 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205033+0000", 2024-09-14T23:57:24.494 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:24.494 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.494 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.494 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:24.494 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205033+0000", 2024-09-14T23:57:24.494 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:24.494 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.494 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.494 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:24.494 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205033+0000", 2024-09-14T23:57:24.494 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9900000000000001e-07 2024-09-14T23:57:24.494 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.495 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.495 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:24.495 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205036+0000", 2024-09-14T23:57:24.495 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3319999999999999e-06 2024-09-14T23:57:24.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.495 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.495 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:24.495 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205042+0000", 2024-09-14T23:57:24.495 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3049999999999998e-06 2024-09-14T23:57:24.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.495 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.495 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:24.495 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205047+0000", 2024-09-14T23:57:24.495 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0420000000000002e-06 2024-09-14T23:57:24.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.496 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.496 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:24.496 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.232809+0000", 2024-09-14T23:57:24.496 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027761917000000001 2024-09-14T23:57:24.496 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.496 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.496 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:24.496 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.232879+0000", 2024-09-14T23:57:24.496 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9597000000000005e-05 2024-09-14T23:57:24.496 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.496 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.496 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:24.496 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.232948+0000", 2024-09-14T23:57:24.496 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9214999999999994e-05 2024-09-14T23:57:24.497 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:24.497 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:24.497 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:24.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.497 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.497 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:55366 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 3686400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:24.497 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.205054+0000", 2024-09-14T23:57:24.497 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.212094270000001, 2024-09-14T23:57:24.497 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028049935000000002, 2024-09-14T23:57:24.497 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:24.497 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:24.497 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:24.497 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:24.497 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:24.498 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:24.498 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 55366 2024-09-14T23:57:24.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.498 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:24.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.498 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:24.498 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205054+0000", 2024-09-14T23:57:24.498 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:24.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.498 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:24.498 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205054+0000", 2024-09-14T23:57:24.498 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:24.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.499 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:24.499 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205055+0000", 2024-09-14T23:57:24.499 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9999999999999998e-07 2024-09-14T23:57:24.499 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.499 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.499 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:24.499 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205065+0000", 2024-09-14T23:57:24.499 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0489000000000001e-05 2024-09-14T23:57:24.499 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.499 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.499 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:24.499 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205073+0000", 2024-09-14T23:57:24.499 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9840000000000001e-06 2024-09-14T23:57:24.499 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.500 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.500 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:24.500 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205085+0000", 2024-09-14T23:57:24.500 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2052000000000001e-05 2024-09-14T23:57:24.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.500 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.500 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:24.500 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.232975+0000", 2024-09-14T23:57:24.500 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027889646000000001 2024-09-14T23:57:24.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.500 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.500 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:24.500 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.233039+0000", 2024-09-14T23:57:24.500 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3993999999999998e-05 2024-09-14T23:57:24.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.501 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.501 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:24.501 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.233104+0000", 2024-09-14T23:57:24.501 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5270000000000004e-05 2024-09-14T23:57:24.501 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:24.501 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:24.501 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:24.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.501 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.501 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:55367 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 3866624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:24.501 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.205098+0000", 2024-09-14T23:57:24.501 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.212050651999999, 2024-09-14T23:57:24.501 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028159871, 2024-09-14T23:57:24.502 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:24.502 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:24.502 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:24.502 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:24.502 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:24.502 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:24.502 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 55367 2024-09-14T23:57:24.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.502 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:24.502 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.502 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:24.502 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205098+0000", 2024-09-14T23:57:24.502 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:24.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.503 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.503 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:24.503 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205098+0000", 2024-09-14T23:57:24.503 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:24.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.503 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.503 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:24.503 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205098+0000", 2024-09-14T23:57:24.503 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4000000000000002e-07 2024-09-14T23:57:24.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.503 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.503 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:24.503 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205101+0000", 2024-09-14T23:57:24.503 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7860000000000001e-06 2024-09-14T23:57:24.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.504 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.504 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:24.504 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205109+0000", 2024-09-14T23:57:24.504 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4039999999999998e-06 2024-09-14T23:57:24.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.504 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.504 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:24.504 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205123+0000", 2024-09-14T23:57:24.504 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.42e-05 2024-09-14T23:57:24.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.504 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.504 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:24.504 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.233129+0000", 2024-09-14T23:57:24.505 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028006316999999999 2024-09-14T23:57:24.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.505 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.505 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:24.505 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.233196+0000", 2024-09-14T23:57:24.505 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7234e-05 2024-09-14T23:57:24.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.505 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.505 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:24.505 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.233258+0000", 2024-09-14T23:57:24.505 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1389999999999993e-05 2024-09-14T23:57:24.505 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:24.505 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:24.506 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:24.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.506 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:55368 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 4046848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:24.506 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.205132+0000", 2024-09-14T23:57:24.506 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.212016340000002, 2024-09-14T23:57:24.506 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028285467000000002, 2024-09-14T23:57:24.506 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:24.506 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:24.506 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:24.506 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:24.506 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:24.506 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:24.506 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 55368 2024-09-14T23:57:24.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.507 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:24.507 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.507 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:24.507 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205132+0000", 2024-09-14T23:57:24.507 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:24.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.507 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.507 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:24.507 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205132+0000", 2024-09-14T23:57:24.507 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:24.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.507 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.507 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:24.508 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205133+0000", 2024-09-14T23:57:24.508 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.13e-07 2024-09-14T23:57:24.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.508 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.508 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:24.508 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205135+0000", 2024-09-14T23:57:24.508 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5490000000000001e-06 2024-09-14T23:57:24.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.508 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.509 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:24.509 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205142+0000", 2024-09-14T23:57:24.509 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.007e-06 2024-09-14T23:57:24.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.509 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.509 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:24.509 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205147+0000", 2024-09-14T23:57:24.509 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.169e-06 2024-09-14T23:57:24.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.509 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.509 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:24.509 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.233281+0000", 2024-09-14T23:57:24.509 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028134059999999999 2024-09-14T23:57:24.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.510 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:24.510 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.233355+0000", 2024-09-14T23:57:24.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3120000000000005e-05 2024-09-14T23:57:24.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.510 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:24.510 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.233418+0000", 2024-09-14T23:57:24.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3048999999999999e-05 2024-09-14T23:57:24.510 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:24.510 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:24.510 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:24.511 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:24.511 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:24.511 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:24.512 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T23:57:24.513 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:24.511+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:24.613 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:24.610+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:24.661 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:24.661 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:57:24.661 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:57:24.661 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:24.670 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T23:57:24.713 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:24.712+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:24.814 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:24.811+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:24.817 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:24.817 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:57:24.817 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:57:24.817 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:57:24.817 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:24.825 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T23:57:24.915 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:24.914+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:24.948 INFO:teuthology.orchestra.run.smithi042.stdout:2664 punch from 0xd24591b to 0xd249747, (0x3e2c bytes) 2024-09-14T23:57:24.948 INFO:teuthology.orchestra.run.smithi042.stdout:2666 write 0xb07d598 thru 0xb08613d (0x8ba6 bytes) 2024-09-14T23:57:24.952 INFO:teuthology.orchestra.run.smithi042.stdout:2667 writesame 0x48e4348 thru 0x48ecddd data_size 0x51(0x8a96 bytes) 2024-09-14T23:57:24.958 INFO:teuthology.orchestra.run.smithi042.stdout:2668 read 0x5013900 thru 0x50229e4 (0xf0e5 bytes) 2024-09-14T23:57:24.958 INFO:teuthology.orchestra.run.smithi042.stdout:2671 read 0xc053eee thru 0xc0621eb (0xe2fe bytes) 2024-09-14T23:57:24.958 INFO:teuthology.orchestra.run.smithi042.stdout:2672 writesame 0xd3abbf5 thru 0xd3b4ff4 data_size 0x25(0x9400 bytes) 2024-09-14T23:57:24.962 INFO:teuthology.orchestra.run.smithi042.stdout:2673 writesame 0xce695a0 thru 0xce7695a data_size 0x1(0xd3bb bytes) 2024-09-14T23:57:24.965 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is unset 2024-09-14T23:57:24.987 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:24.987 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:57:24.987 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:57:24.988 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:57:24.988 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.988 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5880 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:24.988 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375821+0000", 2024-09-14T23:57:24.988 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.609665151999998, 2024-09-14T23:57:24.988 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050482096999999997, 2024-09-14T23:57:24.988 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:24.988 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:24.988 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:24.988 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:24.988 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:24.988 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:24.989 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5880 2024-09-14T23:57:24.989 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.989 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:24.989 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.989 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:24.989 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375821+0000", 2024-09-14T23:57:24.989 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:24.989 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.989 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.989 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:24.989 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375821+0000", 2024-09-14T23:57:24.989 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:24.989 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.989 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.990 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:24.990 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375821+0000", 2024-09-14T23:57:24.990 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.28e-07 2024-09-14T23:57:24.990 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.990 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.990 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:24.990 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375822+0000", 2024-09-14T23:57:24.990 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7000000000000003e-07 2024-09-14T23:57:24.990 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.990 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.990 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:24.990 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375825+0000", 2024-09-14T23:57:24.990 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9560000000000002e-06 2024-09-14T23:57:24.990 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.991 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.991 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:24.991 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375827+0000", 2024-09-14T23:57:24.991 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3259999999999998e-06 2024-09-14T23:57:24.991 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.991 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.991 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:24.991 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426294+0000", 2024-09-14T23:57:24.991 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050466748999999998 2024-09-14T23:57:24.991 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.991 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.991 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:24.991 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426303+0000", 2024-09-14T23:57:24.992 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8680000000000005e-06 2024-09-14T23:57:24.992 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:24.992 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:24.992 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:24.992 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.992 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.992 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5881 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:24.992 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375830+0000", 2024-09-14T23:57:24.992 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.609655705999998, 2024-09-14T23:57:24.992 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050523831999999998, 2024-09-14T23:57:24.992 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:24.992 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:24.992 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:24.992 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:24.993 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:24.993 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:24.993 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5881 2024-09-14T23:57:24.993 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.993 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:24.993 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.993 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:24.993 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375830+0000", 2024-09-14T23:57:24.993 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:24.993 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.993 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.993 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:24.993 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375830+0000", 2024-09-14T23:57:24.993 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:24.993 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.994 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.994 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:24.994 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375830+0000", 2024-09-14T23:57:24.994 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.34e-07 2024-09-14T23:57:24.994 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.994 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.994 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:24.994 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375831+0000", 2024-09-14T23:57:24.994 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9599999999999998e-07 2024-09-14T23:57:24.994 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.994 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.994 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:24.994 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375834+0000", 2024-09-14T23:57:24.994 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0489999999999999e-06 2024-09-14T23:57:24.995 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.995 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.995 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:24.995 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375838+0000", 2024-09-14T23:57:24.995 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5319999999999998e-06 2024-09-14T23:57:24.995 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.995 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.995 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:24.995 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426342+0000", 2024-09-14T23:57:24.995 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050504391000000003 2024-09-14T23:57:24.995 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.995 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.995 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:24.995 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426354+0000", 2024-09-14T23:57:24.995 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.173e-05 2024-09-14T23:57:24.996 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:24.996 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:24.996 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:24.996 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.996 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.996 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5882 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:24.996 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:57:24.996 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.609645102000002, 2024-09-14T23:57:24.996 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050549666, 2024-09-14T23:57:24.996 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:24.996 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:24.996 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:24.996 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:24.996 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:24.997 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:24.997 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5882 2024-09-14T23:57:24.997 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.997 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:24.997 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.997 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:24.997 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:57:24.997 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:24.997 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.997 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.997 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:24.997 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:57:24.997 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:24.997 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.998 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.998 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:24.998 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:57:24.998 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8099999999999998e-07 2024-09-14T23:57:24.998 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.998 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.998 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:24.998 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375842+0000", 2024-09-14T23:57:24.998 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9900000000000009e-07 2024-09-14T23:57:24.998 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.998 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.998 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:24.998 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375845+0000", 2024-09-14T23:57:24.998 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.055e-06 2024-09-14T23:57:24.999 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.999 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.999 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:24.999 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375848+0000", 2024-09-14T23:57:24.999 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7609999999999999e-06 2024-09-14T23:57:24.999 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.999 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.999 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:24.999 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426381+0000", 2024-09-14T23:57:24.999 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050533392000000003 2024-09-14T23:57:24.999 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:24.999 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:24.999 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:24.999 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426390+0000", 2024-09-14T23:57:25.000 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0780000000000002e-06 2024-09-14T23:57:25.000 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.000 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:25.000 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.000 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.000 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.000 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5884 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:25.000 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:57:25.000 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.609634962000001, 2024-09-14T23:57:25.000 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050717439000000003, 2024-09-14T23:57:25.000 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:25.000 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:25.000 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:25.000 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:25.001 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:25.001 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:25.001 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5884 2024-09-14T23:57:25.001 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.001 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:25.001 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.001 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:25.001 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:57:25.001 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.001 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.001 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.001 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:25.001 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:57:25.001 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.001 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.002 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.002 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:25.002 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:57:25.002 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0800000000000001e-07 2024-09-14T23:57:25.002 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.002 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.002 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:25.002 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375852+0000", 2024-09-14T23:57:25.002 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.02e-06 2024-09-14T23:57:25.002 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.002 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.002 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:25.002 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375855+0000", 2024-09-14T23:57:25.003 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9900000000000002e-06 2024-09-14T23:57:25.003 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.003 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.003 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:25.003 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375857+0000", 2024-09-14T23:57:25.003 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3499999999999999e-06 2024-09-14T23:57:25.003 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.003 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.003 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:25.003 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426556+0000", 2024-09-14T23:57:25.003 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050698139000000003 2024-09-14T23:57:25.003 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.003 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.003 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:25.003 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426568+0000", 2024-09-14T23:57:25.004 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2731999999999999e-05 2024-09-14T23:57:25.004 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.004 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:25.004 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.004 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.004 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.004 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5885 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:25.004 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:57:25.004 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.609615937000001, 2024-09-14T23:57:25.004 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050737333000000003, 2024-09-14T23:57:25.004 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:25.004 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:25.004 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:25.004 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:25.005 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:25.005 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:25.005 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5885 2024-09-14T23:57:25.005 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.005 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:25.005 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.005 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:25.005 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:57:25.005 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.005 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.005 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.005 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:25.005 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:57:25.005 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.006 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.006 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.006 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:25.006 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:57:25.006 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5899999999999998e-07 2024-09-14T23:57:25.006 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.006 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.006 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:25.006 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375872+0000", 2024-09-14T23:57:25.006 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4219999999999999e-06 2024-09-14T23:57:25.006 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.006 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.006 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:25.006 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375878+0000", 2024-09-14T23:57:25.007 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4980000000000003e-06 2024-09-14T23:57:25.007 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.007 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.007 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:25.007 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375881+0000", 2024-09-14T23:57:25.007 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6110000000000001e-06 2024-09-14T23:57:25.007 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.007 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.007 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:25.007 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426596+0000", 2024-09-14T23:57:25.007 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050715320000000001 2024-09-14T23:57:25.007 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.007 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.007 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:25.007 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426607+0000", 2024-09-14T23:57:25.008 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1223000000000001e-05 2024-09-14T23:57:25.008 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.008 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:25.008 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.008 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.008 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.008 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5886 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:25.008 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:57:25.008 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.609601824000002, 2024-09-14T23:57:25.008 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050803395000000001, 2024-09-14T23:57:25.008 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:25.008 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:25.008 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:25.008 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:25.009 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:25.009 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:25.009 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5886 2024-09-14T23:57:25.009 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.009 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:25.009 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.009 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:25.009 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:57:25.009 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.009 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.009 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.009 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:25.009 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:57:25.009 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.010 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.010 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.010 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:25.010 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:57:25.010 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2700000000000001e-07 2024-09-14T23:57:25.010 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.010 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.010 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:25.010 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375886+0000", 2024-09-14T23:57:25.010 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7060000000000001e-06 2024-09-14T23:57:25.010 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.010 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.010 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:25.010 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375890+0000", 2024-09-14T23:57:25.011 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2259999999999999e-06 2024-09-14T23:57:25.011 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.011 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.011 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:25.011 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375893+0000", 2024-09-14T23:57:25.011 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0819999999999999e-06 2024-09-14T23:57:25.011 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.011 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.011 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:25.011 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426677+0000", 2024-09-14T23:57:25.011 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050784158000000003 2024-09-14T23:57:25.011 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.011 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.011 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:25.012 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426687+0000", 2024-09-14T23:57:25.012 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9960000000000005e-06 2024-09-14T23:57:25.012 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.012 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:25.012 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.012 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.012 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5887 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:25.012 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:57:25.012 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.609579738000001, 2024-09-14T23:57:25.012 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050833070000000001, 2024-09-14T23:57:25.012 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:25.012 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:25.012 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:25.013 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:25.013 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:25.013 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:25.013 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5887 2024-09-14T23:57:25.013 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.013 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:25.013 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.013 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:25.013 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:57:25.013 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.013 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.013 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.013 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:25.013 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:57:25.013 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.014 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.014 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.014 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:25.014 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:57:25.014 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3299999999999998e-07 2024-09-14T23:57:25.014 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.014 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.014 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:25.014 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375908+0000", 2024-09-14T23:57:25.014 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7689999999999999e-06 2024-09-14T23:57:25.014 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.014 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.014 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:25.014 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375912+0000", 2024-09-14T23:57:25.015 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.535e-06 2024-09-14T23:57:25.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.015 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.015 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:25.015 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375916+0000", 2024-09-14T23:57:25.015 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8979999999999998e-06 2024-09-14T23:57:25.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.015 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.015 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:25.015 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426729+0000", 2024-09-14T23:57:25.015 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050813453000000001 2024-09-14T23:57:25.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.015 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.015 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:25.015 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426739+0000", 2024-09-14T23:57:25.016 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0081999999999999e-05 2024-09-14T23:57:25.016 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.016 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:25.016 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.016 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.016 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.016 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5888 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:25.016 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:57:25.016 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.609566661000002, 2024-09-14T23:57:25.016 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050854652, 2024-09-14T23:57:25.016 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:25.016 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:25.016 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:25.016 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:25.017 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:25.017 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:25.017 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5888 2024-09-14T23:57:25.017 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.017 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:25.017 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.017 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:25.017 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:57:25.017 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.017 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.017 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.017 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:25.017 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:57:25.017 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.018 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.018 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:25.018 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:57:25.018 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1400000000000001e-07 2024-09-14T23:57:25.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.018 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.018 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:25.018 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375921+0000", 2024-09-14T23:57:25.018 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.119e-06 2024-09-14T23:57:25.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.018 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.018 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:25.018 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375928+0000", 2024-09-14T23:57:25.018 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3170000000000004e-06 2024-09-14T23:57:25.019 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.019 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.019 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:25.019 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375931+0000", 2024-09-14T23:57:25.019 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6970000000000002e-06 2024-09-14T23:57:25.019 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.019 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.019 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:25.019 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426768+0000", 2024-09-14T23:57:25.019 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050837355000000001 2024-09-14T23:57:25.019 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.019 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.019 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:25.019 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426774+0000", 2024-09-14T23:57:25.020 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9499999999999998e-06 2024-09-14T23:57:25.020 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.020 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:25.020 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.020 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.020 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.020 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5889 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:25.020 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:57:25.020 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.609549737000002, 2024-09-14T23:57:25.020 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050885446000000001, 2024-09-14T23:57:25.020 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:25.020 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:25.020 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:25.020 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:25.021 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:25.021 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:25.021 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5889 2024-09-14T23:57:25.021 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.021 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:25.021 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.021 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:25.021 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:57:25.021 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.021 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.021 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.021 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:25.021 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:57:25.021 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.022 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.022 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.022 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:25.022 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:57:25.022 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.36e-07 2024-09-14T23:57:25.022 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.022 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.022 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:25.022 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375940+0000", 2024-09-14T23:57:25.022 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.844e-06 2024-09-14T23:57:25.022 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.022 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.022 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:25.023 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375944+0000", 2024-09-14T23:57:25.023 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6720000000000002e-06 2024-09-14T23:57:25.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.023 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.023 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:25.023 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375946+0000", 2024-09-14T23:57:25.023 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5629999999999999e-06 2024-09-14T23:57:25.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.023 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.023 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:25.023 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426810+0000", 2024-09-14T23:57:25.023 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050863255000000003 2024-09-14T23:57:25.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.023 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.024 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:25.024 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426822+0000", 2024-09-14T23:57:25.024 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1875999999999999e-05 2024-09-14T23:57:25.024 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.024 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:25.024 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.024 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.024 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.024 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5890 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:25.024 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:57:25.024 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.609535925000003, 2024-09-14T23:57:25.024 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050917549999999999, 2024-09-14T23:57:25.024 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:25.025 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:25.025 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:25.025 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:25.025 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:25.025 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:25.025 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5890 2024-09-14T23:57:25.025 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.025 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:25.025 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.025 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:25.025 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:57:25.025 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.025 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.026 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.026 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:25.026 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:57:25.026 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.026 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.026 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.026 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:25.026 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:57:25.026 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4299999999999999e-07 2024-09-14T23:57:25.026 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.026 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.026 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:25.026 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375951+0000", 2024-09-14T23:57:25.026 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1000000000000001e-06 2024-09-14T23:57:25.027 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.027 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.027 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:25.027 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375956+0000", 2024-09-14T23:57:25.027 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3370000000000001e-06 2024-09-14T23:57:25.027 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.027 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.027 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:25.027 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375958+0000", 2024-09-14T23:57:25.027 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4820000000000001e-06 2024-09-14T23:57:25.027 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.027 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.027 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:25.027 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426858+0000", 2024-09-14T23:57:25.028 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050899666000000003 2024-09-14T23:57:25.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.028 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.028 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:25.028 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426867+0000", 2024-09-14T23:57:25.028 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7219999999999994e-06 2024-09-14T23:57:25.028 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.028 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:25.028 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.028 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.028 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5891 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:25.028 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:57:25.028 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.609524602999997, 2024-09-14T23:57:25.029 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050948067, 2024-09-14T23:57:25.029 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:25.029 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:25.029 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:25.029 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:25.029 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:25.029 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:25.029 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5891 2024-09-14T23:57:25.029 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.029 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:25.029 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.029 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:25.029 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:57:25.029 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.030 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.030 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.030 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:25.030 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:57:25.030 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.030 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.030 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.030 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:25.030 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:57:25.030 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0800000000000001e-07 2024-09-14T23:57:25.030 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.030 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.030 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:25.030 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375962+0000", 2024-09-14T23:57:25.031 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9600000000000008e-07 2024-09-14T23:57:25.031 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.031 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.031 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:25.031 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375970+0000", 2024-09-14T23:57:25.031 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1600000000000001e-06 2024-09-14T23:57:25.031 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.031 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.031 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:25.031 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375972+0000", 2024-09-14T23:57:25.031 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4849999999999999e-06 2024-09-14T23:57:25.031 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.031 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.032 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:25.032 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426902+0000", 2024-09-14T23:57:25.032 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050929813999999997 2024-09-14T23:57:25.032 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.032 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.032 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:25.032 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426909+0000", 2024-09-14T23:57:25.032 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4039999999999998e-06 2024-09-14T23:57:25.032 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.032 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:25.032 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.032 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.032 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.032 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5892 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:25.033 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:57:25.033 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.609509746000001, 2024-09-14T23:57:25.033 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050965036999999998, 2024-09-14T23:57:25.033 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:25.033 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:25.033 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:25.033 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:25.033 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:25.033 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:25.033 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5892 2024-09-14T23:57:25.033 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.033 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:25.033 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.033 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:25.034 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:57:25.034 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.034 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.034 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:25.034 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:57:25.034 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.034 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.034 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:25.034 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:57:25.034 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.34e-07 2024-09-14T23:57:25.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.034 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.035 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:25.035 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375978+0000", 2024-09-14T23:57:25.035 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1880000000000001e-06 2024-09-14T23:57:25.035 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.035 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.035 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:25.035 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375981+0000", 2024-09-14T23:57:25.035 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2399999999999999e-06 2024-09-14T23:57:25.035 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.035 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.035 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:25.035 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375984+0000", 2024-09-14T23:57:25.035 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9110000000000002e-06 2024-09-14T23:57:25.035 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.035 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.036 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:25.036 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426934+0000", 2024-09-14T23:57:25.036 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050949912 2024-09-14T23:57:25.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.036 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.036 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:25.036 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426941+0000", 2024-09-14T23:57:25.036 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5519999999999999e-06 2024-09-14T23:57:25.036 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.036 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:25.036 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.036 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.036 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5893 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:25.037 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:57:25.037 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.609499149000001, 2024-09-14T23:57:25.037 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050975131999999999, 2024-09-14T23:57:25.037 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:25.037 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:25.037 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:25.037 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:25.037 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:25.037 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:25.037 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5893 2024-09-14T23:57:25.037 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.037 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:25.037 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.037 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:25.038 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:57:25.038 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.038 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.038 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.038 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:25.038 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:57:25.038 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.038 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.038 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.038 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:25.038 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:57:25.038 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2999999999999999e-07 2024-09-14T23:57:25.038 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.038 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.039 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:25.039 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375988+0000", 2024-09-14T23:57:25.039 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0410000000000001e-06 2024-09-14T23:57:25.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.039 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.039 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:25.039 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375992+0000", 2024-09-14T23:57:25.039 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.104e-06 2024-09-14T23:57:25.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.039 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.039 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:25.039 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375995+0000", 2024-09-14T23:57:25.039 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4310000000000001e-06 2024-09-14T23:57:25.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.040 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.040 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:25.040 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426958+0000", 2024-09-14T23:57:25.040 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050963642000000003 2024-09-14T23:57:25.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.040 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.040 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:25.040 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426962+0000", 2024-09-14T23:57:25.040 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6839999999999998e-06 2024-09-14T23:57:25.040 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.040 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:25.040 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.040 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.041 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5894 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:25.041 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:57:25.041 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.609487749000003, 2024-09-14T23:57:25.041 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050978333000000001, 2024-09-14T23:57:25.041 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:25.041 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:25.041 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:25.041 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:25.041 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:25.041 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:25.041 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5894 2024-09-14T23:57:25.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.041 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:25.041 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.042 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:25.042 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:57:25.042 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.042 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.042 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:25.042 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:57:25.042 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.042 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.042 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:25.042 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:57:25.042 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.23e-07 2024-09-14T23:57:25.043 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.043 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.043 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:25.043 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375999+0000", 2024-09-14T23:57:25.043 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9800000000000002e-07 2024-09-14T23:57:25.043 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.043 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.043 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:25.043 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376007+0000", 2024-09-14T23:57:25.043 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8539999999999997e-06 2024-09-14T23:57:25.043 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.043 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.043 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:25.044 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376011+0000", 2024-09-14T23:57:25.044 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.399e-06 2024-09-14T23:57:25.044 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.044 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.044 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:25.044 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426973+0000", 2024-09-14T23:57:25.044 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050962454999999997 2024-09-14T23:57:25.044 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.044 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.044 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:25.044 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426976+0000", 2024-09-14T23:57:25.044 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4039999999999999e-06 2024-09-14T23:57:25.044 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.044 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:25.045 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.045 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.045 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5895 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:25.045 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376014+0000", 2024-09-14T23:57:25.045 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.609471405000001, 2024-09-14T23:57:25.045 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050975951999999998, 2024-09-14T23:57:25.045 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:25.045 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:25.045 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:25.045 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:25.045 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:25.046 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:25.046 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5895 2024-09-14T23:57:25.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.046 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:25.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.046 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:25.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376014+0000", 2024-09-14T23:57:25.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.046 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:25.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376014+0000", 2024-09-14T23:57:25.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:25.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376015+0000", 2024-09-14T23:57:25.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2999999999999999e-07 2024-09-14T23:57:25.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:25.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376016+0000", 2024-09-14T23:57:25.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1659999999999999e-06 2024-09-14T23:57:25.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:25.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376024+0000", 2024-09-14T23:57:25.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8930000000000005e-06 2024-09-14T23:57:25.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.048 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:25.048 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376026+0000", 2024-09-14T23:57:25.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6869999999999999e-06 2024-09-14T23:57:25.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.048 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:25.048 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426987+0000", 2024-09-14T23:57:25.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050960601000000001 2024-09-14T23:57:25.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.048 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:25.048 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426990+0000", 2024-09-14T23:57:25.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3749999999999999e-06 2024-09-14T23:57:25.049 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.049 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:25.049 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.049 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5896 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:25.049 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:57:25.049 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.609455668000003, 2024-09-14T23:57:25.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050975656000000001, 2024-09-14T23:57:25.049 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:25.049 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:25.049 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:25.049 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:25.050 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:25.050 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:25.050 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5896 2024-09-14T23:57:25.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.050 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:25.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.050 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:25.050 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:57:25.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.050 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:25.050 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:57:25.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:25.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:57:25.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5100000000000001e-07 2024-09-14T23:57:25.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:25.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376032+0000", 2024-09-14T23:57:25.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.141e-06 2024-09-14T23:57:25.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:25.052 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376036+0000", 2024-09-14T23:57:25.052 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.25e-06 2024-09-14T23:57:25.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.052 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.052 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:25.052 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376038+0000", 2024-09-14T23:57:25.052 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5469999999999999e-06 2024-09-14T23:57:25.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.052 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.052 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:25.052 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427002+0000", 2024-09-14T23:57:25.052 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050964029000000001 2024-09-14T23:57:25.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.052 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.053 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:25.053 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427006+0000", 2024-09-14T23:57:25.053 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4379999999999999e-06 2024-09-14T23:57:25.053 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.053 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:25.053 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.053 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5897 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:25.053 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:57:25.053 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.609443868, 2024-09-14T23:57:25.053 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050978001000000002, 2024-09-14T23:57:25.053 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:25.053 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:25.054 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:25.054 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:25.054 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:25.054 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:25.054 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5897 2024-09-14T23:57:25.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.054 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:25.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.054 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:25.054 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:57:25.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.054 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:25.054 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:57:25.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:25.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:57:25.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2600000000000001e-07 2024-09-14T23:57:25.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:25.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376043+0000", 2024-09-14T23:57:25.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.054e-06 2024-09-14T23:57:25.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:25.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376047+0000", 2024-09-14T23:57:25.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0659999999999997e-06 2024-09-14T23:57:25.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:25.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376050+0000", 2024-09-14T23:57:25.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.88e-06 2024-09-14T23:57:25.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:25.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427017+0000", 2024-09-14T23:57:25.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050966453000000002 2024-09-14T23:57:25.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.057 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:25.057 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427020+0000", 2024-09-14T23:57:25.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3220000000000001e-06 2024-09-14T23:57:25.057 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.057 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:25.057 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.057 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5898 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:25.057 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376053+0000", 2024-09-14T23:57:25.057 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.609432382999998, 2024-09-14T23:57:25.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050982024000000001, 2024-09-14T23:57:25.057 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:25.057 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:25.057 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:25.058 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:25.058 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:25.058 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:25.058 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5898 2024-09-14T23:57:25.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.058 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:25.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.058 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:25.058 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376053+0000", 2024-09-14T23:57:25.058 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.058 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:25.058 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376053+0000", 2024-09-14T23:57:25.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:25.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376278+0000", 2024-09-14T23:57:25.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022426400000000001 2024-09-14T23:57:25.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:25.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376282+0000", 2024-09-14T23:57:25.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4739999999999999e-06 2024-09-14T23:57:25.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:25.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376296+0000", 2024-09-14T23:57:25.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4229999999999999e-05 2024-09-14T23:57:25.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:25.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376303+0000", 2024-09-14T23:57:25.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2199999999999997e-06 2024-09-14T23:57:25.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:25.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427031+0000", 2024-09-14T23:57:25.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050728154999999997 2024-09-14T23:57:25.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.061 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:25.061 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427036+0000", 2024-09-14T23:57:25.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6809999999999997e-06 2024-09-14T23:57:25.061 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.061 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:25.061 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.061 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5899 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:25.061 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376338+0000", 2024-09-14T23:57:25.061 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.609147659000001, 2024-09-14T23:57:25.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050712100000000003, 2024-09-14T23:57:25.061 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:25.061 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:25.061 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:25.062 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:25.062 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:25.062 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:25.062 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5899 2024-09-14T23:57:25.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.062 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:25.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.062 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:25.062 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376338+0000", 2024-09-14T23:57:25.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.062 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:25.063 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376338+0000", 2024-09-14T23:57:25.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:25.063 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376339+0000", 2024-09-14T23:57:25.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9100000000000003e-07 2024-09-14T23:57:25.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:25.063 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376342+0000", 2024-09-14T23:57:25.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7e-06 2024-09-14T23:57:25.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.064 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:25.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376350+0000", 2024-09-14T23:57:25.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5280000000000003e-06 2024-09-14T23:57:25.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.064 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:25.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376356+0000", 2024-09-14T23:57:25.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6010000000000002e-06 2024-09-14T23:57:25.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.064 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:25.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427047+0000", 2024-09-14T23:57:25.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050690962999999999 2024-09-14T23:57:25.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:25.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427050+0000", 2024-09-14T23:57:25.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.517e-06 2024-09-14T23:57:25.065 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.065 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:25.065 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.065 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5900 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:25.065 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376366+0000", 2024-09-14T23:57:25.065 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.609119399999997, 2024-09-14T23:57:25.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050697858999999998, 2024-09-14T23:57:25.066 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:25.066 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:25.066 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:25.066 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:25.066 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:25.066 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:25.066 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5900 2024-09-14T23:57:25.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.066 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:25.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:25.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376366+0000", 2024-09-14T23:57:25.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:25.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376366+0000", 2024-09-14T23:57:25.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:25.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377650+0000", 2024-09-14T23:57:25.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001283834 2024-09-14T23:57:25.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:25.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377655+0000", 2024-09-14T23:57:25.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7230000000000004e-06 2024-09-14T23:57:25.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:25.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377667+0000", 2024-09-14T23:57:25.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1746000000000001e-05 2024-09-14T23:57:25.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:25.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377676+0000", 2024-09-14T23:57:25.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3619999999999995e-06 2024-09-14T23:57:25.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:25.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427061+0000", 2024-09-14T23:57:25.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049384755000000002 2024-09-14T23:57:25.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:25.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427064+0000", 2024-09-14T23:57:25.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.439e-06 2024-09-14T23:57:25.069 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.069 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:25.069 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.069 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.069 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:25.069 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:25.070 INFO:teuthology.orchestra.run.smithi042.stdout:2675 read 0x251a9d6 thru 0x2526ec1 (0xc4ec bytes) 2024-09-14T23:57:25.071 INFO:teuthology.orchestra.run.smithi042.stdout:2676 write 0x46d3ef6 thru 0x46ddec9 (0x9fd4 bytes) 2024-09-14T23:57:25.072 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T23:57:25.074 INFO:teuthology.orchestra.run.smithi042.stdout:2677 read 0x196d959 thru 0x197a87b (0xcf23 bytes) 2024-09-14T23:57:25.074 INFO:teuthology.orchestra.run.smithi042.stdout:2678 punch from 0x720e1bf to 0x72131ee, (0x502f bytes) 2024-09-14T23:57:25.074 INFO:teuthology.orchestra.run.smithi042.stdout:2680 trunc from 0xe06dea6 to 0xd901af1 2024-09-14T23:57:25.074 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:25.074 INFO:teuthology.orchestra.run.smithi042.stdout:2683 trunc from 0xd901af1 to 0xbb66582 2024-09-14T23:57:25.074 INFO:teuthology.orchestra.run.smithi042.stdout:2684 read 0x58a893a thru 0x58add61 (0x5428 bytes) 2024-09-14T23:57:25.074 INFO:teuthology.orchestra.run.smithi042.stdout:2685 write 0x59fdca3 thru 0x5a01348 (0x36a6 bytes) 2024-09-14T23:57:25.074 INFO:teuthology.orchestra.run.smithi042.stdout:2686 read 0xab3838b thru 0xab3eecc (0x6b42 bytes) 2024-09-14T23:57:25.074 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:25.074 INFO:teuthology.orchestra.run.smithi042.stdout:2690 write 0x7b84316 thru 0x7b90683 (0xc36e bytes) 2024-09-14T23:57:25.075 INFO:teuthology.orchestra.run.smithi042.stdout:2692 writesame 0xa433b51 thru 0xa43c9ad data_size 0x5(0x8e5d bytes) 2024-09-14T23:57:25.075 INFO:teuthology.orchestra.run.smithi042.stdout:2693 writesame 0x6ef39ae thru 0x6ef78b4 data_size 0x5(0x3f07 bytes) 2024-09-14T23:57:25.075 INFO:teuthology.orchestra.run.smithi042.stdout:2694 punch from 0x255cc0f to 0x256857c, (0xb96d bytes) 2024-09-14T23:57:25.075 INFO:teuthology.orchestra.run.smithi042.stdout:2695 read 0x509328b thru 0x5097138 (0x3eae bytes) 2024-09-14T23:57:25.075 INFO:teuthology.orchestra.run.smithi042.stdout:2696 trunc from 0xbb66582 to 0xcb8489a 2024-09-14T23:57:25.075 INFO:teuthology.orchestra.run.smithi042.stdout:2697 punch from 0x4632502 to 0x4633443, (0xf41 bytes) 2024-09-14T23:57:25.075 INFO:teuthology.orchestra.run.smithi042.stdout:2698 trunc from 0xcb8489a to 0x4dda90f 2024-09-14T23:57:25.075 INFO:teuthology.orchestra.run.smithi042.stdout:2699 writesame 0x8dab2d thru 0x8e24fa data_size 0x3(0x79ce bytes) 2024-09-14T23:57:25.075 INFO:teuthology.orchestra.run.smithi042.stdout:2700 punch from 0x194a9c0 to 0x1952780, (0x7dc0 bytes) 2024-09-14T23:57:25.075 INFO:teuthology.orchestra.run.smithi042.stdout:2701 writesame 0x3ac35aa thru 0x3acbc9d data_size 0x6(0x86f4 bytes) 2024-09-14T23:57:25.076 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:25.069+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:25.089 INFO:teuthology.orchestra.run.smithi042.stdout:2702 write 0x90e9cb1 thru 0x90eafba (0x130a bytes) 2024-09-14T23:57:25.093 INFO:teuthology.orchestra.run.smithi042.stdout:2704 trunc from 0x90eafbb to 0x85566ad 2024-09-14T23:57:25.100 INFO:teuthology.orchestra.run.smithi042.stdout:2705 write 0x5fc8af0 thru 0x5fcadbb (0x22cc bytes) 2024-09-14T23:57:25.103 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:25.104 INFO:teuthology.orchestra.run.smithi042.stdout:2710 punch from 0x6bcf059 to 0x6bdc666, (0xd60d bytes) 2024-09-14T23:57:25.113 INFO:teuthology.orchestra.run.smithi042.stdout:2711 write 0x9fd7af6 thru 0x9fd7d96 (0x2a1 bytes) 2024-09-14T23:57:25.118 INFO:teuthology.orchestra.run.smithi042.stdout:2712 trunc from 0x9fd7d97 to 0x806d006 2024-09-14T23:57:25.140 INFO:teuthology.orchestra.run.smithi042.stdout:2713 trunc from 0x806d006 to 0x4fc907 2024-09-14T23:57:25.171 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:25.169+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:25.173 INFO:teuthology.orchestra.run.smithi042.stdout:2716 writesame 0x4fa210a thru 0x4fa66de data_size 0x3(0x45d5 bytes) 2024-09-14T23:57:25.173 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:25.173 INFO:teuthology.orchestra.run.smithi042.stdout:2718 read 0xfaa022 thru 0xfac302 (0x22e1 bytes) 2024-09-14T23:57:25.174 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:25.180 INFO:teuthology.orchestra.run.smithi042.stdout:2720 writesame 0x5e2794c thru 0x5e2bd03 data_size 0x16(0x43b8 bytes) 2024-09-14T23:57:25.180 INFO:teuthology.orchestra.run.smithi042.stdout:2721 read 0x53b5538 thru 0x53c3a52 (0xe51b bytes) 2024-09-14T23:57:25.194 INFO:teuthology.orchestra.run.smithi042.stdout:2723 write 0xd8a889b thru 0xd8aebe1 (0x6347 bytes) 2024-09-14T23:57:25.197 INFO:teuthology.orchestra.run.smithi042.stdout:2724 punch from 0x85b0942 to 0x85b9a21, (0x90df bytes) 2024-09-14T23:57:25.197 INFO:teuthology.orchestra.run.smithi042.stdout:2726 punch from 0x578d28c to 0x5794a2b, (0x779f bytes) 2024-09-14T23:57:25.198 INFO:teuthology.orchestra.run.smithi042.stdout:2727 write 0xb5cd95c thru 0xb5cdf2e (0x5d3 bytes) 2024-09-14T23:57:25.213 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:25.213 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:57:25.213 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:57:25.213 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:25.222 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T23:57:25.306 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:25.304+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:25.369 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:25.370 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:57:25.370 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:57:25.370 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:57:25.370 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:25.373 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:25.372+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:25.378 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T23:57:25.474 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:25.472+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:25.528 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:25.528 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:57:25.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:57:25.528 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:57:25.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.529 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:3578 3.4 e29/23 3:2298d860:::rbd_data.1118de1168b.0000000000000030:head v 29'15, mlcod=29'13)", 2024-09-14T23:57:25.529 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.036455+0000", 2024-09-14T23:57:25.529 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.489564043000001, 2024-09-14T23:57:25.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019449440000000001, 2024-09-14T23:57:25.529 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:25.529 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:25.529 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:25.529 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:25.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:25.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036455+0000", 2024-09-14T23:57:25.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:25.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036455+0000", 2024-09-14T23:57:25.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:25.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036456+0000", 2024-09-14T23:57:25.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0999999999999997e-07 2024-09-14T23:57:25.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:25.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036463+0000", 2024-09-14T23:57:25.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2200000000000003e-06 2024-09-14T23:57:25.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:25.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036478+0000", 2024-09-14T23:57:25.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4977000000000001e-05 2024-09-14T23:57:25.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:25.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036485+0000", 2024-09-14T23:57:25.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0559999999999999e-06 2024-09-14T23:57:25.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:25.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036693+0000", 2024-09-14T23:57:25.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000208051 2024-09-14T23:57:25.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:25.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036715+0000", 2024-09-14T23:57:25.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1330000000000001e-05 2024-09-14T23:57:25.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:25.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.038356+0000", 2024-09-14T23:57:25.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001641586 2024-09-14T23:57:25.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:25.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.038400+0000", 2024-09-14T23:57:25.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3914e-05 2024-09-14T23:57:25.533 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.533 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:25.533 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.533 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:3579 3.4 e29/23 3:27290fe6:::rbd_data.1118de1168b.0000000000000032:head v 29'16, mlcod=29'13)", 2024-09-14T23:57:25.533 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.037085+0000", 2024-09-14T23:57:25.533 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.488934452999999, 2024-09-14T23:57:25.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0020019220000000002, 2024-09-14T23:57:25.534 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:25.534 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:25.534 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:25.534 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:25.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:25.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037085+0000", 2024-09-14T23:57:25.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:25.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037085+0000", 2024-09-14T23:57:25.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:25.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037085+0000", 2024-09-14T23:57:25.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5499999999999997e-07 2024-09-14T23:57:25.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:25.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037091+0000", 2024-09-14T23:57:25.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7869999999999998e-06 2024-09-14T23:57:25.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:25.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037102+0000", 2024-09-14T23:57:25.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.031e-05 2024-09-14T23:57:25.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:25.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037116+0000", 2024-09-14T23:57:25.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4119e-05 2024-09-14T23:57:25.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:25.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037219+0000", 2024-09-14T23:57:25.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010372799999999999 2024-09-14T23:57:25.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:25.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037239+0000", 2024-09-14T23:57:25.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9993e-05 2024-09-14T23:57:25.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:25.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.039046+0000", 2024-09-14T23:57:25.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001806666 2024-09-14T23:57:25.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:25.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.039087+0000", 2024-09-14T23:57:25.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0564e-05 2024-09-14T23:57:25.538 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.538 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:25.538 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.538 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:3584 3.4 e29/23 3:2779453d:::rbd_data.1118de1168b.0000000000000034:head v 29'17, mlcod=29'13)", 2024-09-14T23:57:25.538 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.037588+0000", 2024-09-14T23:57:25.538 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.488431413999997, 2024-09-14T23:57:25.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001526828, 2024-09-14T23:57:25.538 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:25.539 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:25.539 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:25.539 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:25.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:25.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037588+0000", 2024-09-14T23:57:25.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:25.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037588+0000", 2024-09-14T23:57:25.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:25.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037588+0000", 2024-09-14T23:57:25.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1200000000000002e-07 2024-09-14T23:57:25.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:25.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037595+0000", 2024-09-14T23:57:25.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7580000000000003e-06 2024-09-14T23:57:25.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:25.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037609+0000", 2024-09-14T23:57:25.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4037e-05 2024-09-14T23:57:25.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:25.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037616+0000", 2024-09-14T23:57:25.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0509999999999999e-06 2024-09-14T23:57:25.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:25.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037787+0000", 2024-09-14T23:57:25.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017084899999999999 2024-09-14T23:57:25.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:25.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037806+0000", 2024-09-14T23:57:25.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9159000000000001e-05 2024-09-14T23:57:25.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:25.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.039089+0000", 2024-09-14T23:57:25.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012823909999999999 2024-09-14T23:57:25.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:25.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.039115+0000", 2024-09-14T23:57:25.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5871e-05 2024-09-14T23:57:25.542 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.542 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:25.542 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.543 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:18030 3.4 e33/23 3:225c4d41:::rbd_data.11187be34cc8.0000000000000001:head v 33'63, mlcod=33'61)", 2024-09-14T23:57:25.543 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:43.852748+0000", 2024-09-14T23:57:25.543 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 41.673271354999997, 2024-09-14T23:57:25.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001530696, 2024-09-14T23:57:25.543 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:25.543 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:25.543 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:25.543 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:25.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:25.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.852748+0000", 2024-09-14T23:57:25.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:25.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.852748+0000", 2024-09-14T23:57:25.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:25.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.852749+0000", 2024-09-14T23:57:25.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.164e-06 2024-09-14T23:57:25.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:25.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.852759+0000", 2024-09-14T23:57:25.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0265000000000001e-05 2024-09-14T23:57:25.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:25.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.852779+0000", 2024-09-14T23:57:25.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9247999999999999e-05 2024-09-14T23:57:25.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:25.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.852788+0000", 2024-09-14T23:57:25.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3260000000000002e-06 2024-09-14T23:57:25.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:25.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.853121+0000", 2024-09-14T23:57:25.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00033336699999999999 2024-09-14T23:57:25.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:25.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.853142+0000", 2024-09-14T23:57:25.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0500999999999999e-05 2024-09-14T23:57:25.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:25.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.854240+0000", 2024-09-14T23:57:25.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001098665 2024-09-14T23:57:25.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:25.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.854279+0000", 2024-09-14T23:57:25.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8160000000000001e-05 2024-09-14T23:57:25.547 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.547 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:25.547 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.547 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:32024 3.9 e39/23 3:94dc260b:::rbd_data.111814b482f8.0000000000000009:head v 39'45, mlcod=38'43)", 2024-09-14T23:57:25.547 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:49.061018+0000", 2024-09-14T23:57:25.547 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 36.465001114000003, 2024-09-14T23:57:25.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018482240000000001, 2024-09-14T23:57:25.547 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:25.547 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:25.547 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:25.547 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:25.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:25.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061018+0000", 2024-09-14T23:57:25.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:25.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061018+0000", 2024-09-14T23:57:25.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:25.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061018+0000", 2024-09-14T23:57:25.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9999999999999998e-07 2024-09-14T23:57:25.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:25.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061211+0000", 2024-09-14T23:57:25.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019206 2024-09-14T23:57:25.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:25.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061221+0000", 2024-09-14T23:57:25.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0659000000000001e-05 2024-09-14T23:57:25.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:25.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061230+0000", 2024-09-14T23:57:25.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4610000000000008e-06 2024-09-14T23:57:25.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:25.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061317+0000", 2024-09-14T23:57:25.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7514000000000003e-05 2024-09-14T23:57:25.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:25.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061338+0000", 2024-09-14T23:57:25.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0395999999999999e-05 2024-09-14T23:57:25.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:25.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.062830+0000", 2024-09-14T23:57:25.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0014921940000000001 2024-09-14T23:57:25.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:25.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.062866+0000", 2024-09-14T23:57:25.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6539999999999999e-05 2024-09-14T23:57:25.551 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.551 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:25.551 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.551 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:32420 3.9 e39/23 3:91dc34e7:::rbd_data.111814b482f8.000000000000001e:head v 39'46, mlcod=38'44)", 2024-09-14T23:57:25.551 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:49.086124+0000", 2024-09-14T23:57:25.551 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 36.439895614999998, 2024-09-14T23:57:25.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001686819, 2024-09-14T23:57:25.552 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:25.552 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:25.552 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:25.552 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:25.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:25.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086124+0000", 2024-09-14T23:57:25.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:25.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086124+0000", 2024-09-14T23:57:25.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:25.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086125+0000", 2024-09-14T23:57:25.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.361e-06 2024-09-14T23:57:25.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:25.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086605+0000", 2024-09-14T23:57:25.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00048026600000000003 2024-09-14T23:57:25.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:25.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086626+0000", 2024-09-14T23:57:25.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.037e-05 2024-09-14T23:57:25.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:25.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086635+0000", 2024-09-14T23:57:25.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5880000000000004e-06 2024-09-14T23:57:25.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:25.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086701+0000", 2024-09-14T23:57:25.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5905e-05 2024-09-14T23:57:25.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:25.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086727+0000", 2024-09-14T23:57:25.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5573000000000001e-05 2024-09-14T23:57:25.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:25.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.087767+0000", 2024-09-14T23:57:25.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001040537 2024-09-14T23:57:25.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:25.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.087810+0000", 2024-09-14T23:57:25.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3219000000000001e-05 2024-09-14T23:57:25.555 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.555 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:25.555 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.556 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:34296 3.9 e39/23 3:974a38c7:::rbd_data.111814b482f8.000000000000003a:head v 39'47, mlcod=39'45)", 2024-09-14T23:57:25.556 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:49.192105+0000", 2024-09-14T23:57:25.556 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 36.333913731999999, 2024-09-14T23:57:25.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001607516, 2024-09-14T23:57:25.556 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:25.556 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:25.556 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:25.556 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:25.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:25.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.192105+0000", 2024-09-14T23:57:25.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:25.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.192105+0000", 2024-09-14T23:57:25.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:25.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.192107+0000", 2024-09-14T23:57:25.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3549999999999999e-06 2024-09-14T23:57:25.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:25.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.192332+0000", 2024-09-14T23:57:25.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022486300000000001 2024-09-14T23:57:25.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:25.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.192349+0000", 2024-09-14T23:57:25.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6966e-05 2024-09-14T23:57:25.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:25.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.192358+0000", 2024-09-14T23:57:25.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8309999999999996e-06 2024-09-14T23:57:25.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:25.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.192422+0000", 2024-09-14T23:57:25.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3915e-05 2024-09-14T23:57:25.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:25.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.192450+0000", 2024-09-14T23:57:25.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7957e-05 2024-09-14T23:57:25.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:25.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.193662+0000", 2024-09-14T23:57:25.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012113270000000001 2024-09-14T23:57:25.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:25.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.193713+0000", 2024-09-14T23:57:25.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1301999999999998e-05 2024-09-14T23:57:25.561 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.561 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:25.561 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.561 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:50582 3.9 e51/23 3:92198e7a:::rbd_data.11183b9aac34.000000000000000a:head v 51'63, mlcod=50'60)", 2024-09-14T23:57:25.562 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:01.131541+0000", 2024-09-14T23:57:25.562 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 24.394478423999999, 2024-09-14T23:57:25.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001795547, 2024-09-14T23:57:25.562 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:25.562 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:25.562 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:25.562 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:25.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:25.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131541+0000", 2024-09-14T23:57:25.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:25.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131541+0000", 2024-09-14T23:57:25.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:25.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131541+0000", 2024-09-14T23:57:25.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7099999999999997e-07 2024-09-14T23:57:25.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:25.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131548+0000", 2024-09-14T23:57:25.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.708e-06 2024-09-14T23:57:25.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:25.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131570+0000", 2024-09-14T23:57:25.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1815999999999999e-05 2024-09-14T23:57:25.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:25.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131581+0000", 2024-09-14T23:57:25.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0933e-05 2024-09-14T23:57:25.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:25.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131660+0000", 2024-09-14T23:57:25.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9071000000000002e-05 2024-09-14T23:57:25.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:25.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131682+0000", 2024-09-14T23:57:25.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2534000000000001e-05 2024-09-14T23:57:25.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:25.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.133287+0000", 2024-09-14T23:57:25.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016045510000000001 2024-09-14T23:57:25.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:25.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.133336+0000", 2024-09-14T23:57:25.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9563e-05 2024-09-14T23:57:25.565 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.566 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:25.566 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.566 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:51182 3.9 e52/23 3:918e4fa0:::rbd_data.11181975a1de.000000000000007b:head v 52'74, mlcod=51'71)", 2024-09-14T23:57:25.566 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:02.138725+0000", 2024-09-14T23:57:25.566 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.387294687000001, 2024-09-14T23:57:25.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015321009999999999, 2024-09-14T23:57:25.566 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:25.566 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:25.566 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:25.566 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:25.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:25.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.138725+0000", 2024-09-14T23:57:25.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:25.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.138725+0000", 2024-09-14T23:57:25.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:25.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.138725+0000", 2024-09-14T23:57:25.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.22e-07 2024-09-14T23:57:25.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:25.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.138732+0000", 2024-09-14T23:57:25.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1779999999999997e-06 2024-09-14T23:57:25.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:25.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.138749+0000", 2024-09-14T23:57:25.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6500000000000001e-05 2024-09-14T23:57:25.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:25.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.138758+0000", 2024-09-14T23:57:25.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6239999999999997e-06 2024-09-14T23:57:25.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:25.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.138816+0000", 2024-09-14T23:57:25.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7939000000000002e-05 2024-09-14T23:57:25.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:25.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.138840+0000", 2024-09-14T23:57:25.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3856e-05 2024-09-14T23:57:25.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:25.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.140204+0000", 2024-09-14T23:57:25.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013641339999999999 2024-09-14T23:57:25.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:25.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.140257+0000", 2024-09-14T23:57:25.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2547999999999999e-05 2024-09-14T23:57:25.570 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.570 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:25.570 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.570 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:51281 3.0 e52/23 3:04e60ec2:::rbd_data.11181975a1de.0000000000000003:head v 52'113, mlcod=44'111)", 2024-09-14T23:57:25.570 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:02.158822+0000", 2024-09-14T23:57:25.570 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.367196927999998, 2024-09-14T23:57:25.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019860609999999999, 2024-09-14T23:57:25.570 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:25.570 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:25.571 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:25.571 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:25.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:25.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.158822+0000", 2024-09-14T23:57:25.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:25.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.158822+0000", 2024-09-14T23:57:25.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:25.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.158823+0000", 2024-09-14T23:57:25.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6800000000000001e-07 2024-09-14T23:57:25.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:25.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.159132+0000", 2024-09-14T23:57:25.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030974900000000001 2024-09-14T23:57:25.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:25.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.159146+0000", 2024-09-14T23:57:25.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3261000000000001e-05 2024-09-14T23:57:25.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:25.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.159155+0000", 2024-09-14T23:57:25.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7410000000000004e-06 2024-09-14T23:57:25.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:25.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.159221+0000", 2024-09-14T23:57:25.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5698999999999996e-05 2024-09-14T23:57:25.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:25.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.159244+0000", 2024-09-14T23:57:25.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3108e-05 2024-09-14T23:57:25.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:25.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.160767+0000", 2024-09-14T23:57:25.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015231120000000001 2024-09-14T23:57:25.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:25.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.160808+0000", 2024-09-14T23:57:25.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1022999999999997e-05 2024-09-14T23:57:25.574 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.574 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:25.574 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.574 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:56107 3.0 e56/23 3:00c47890:::rbd_data.1118c4b3aebe.0000000000000009:head v 56'125, mlcod=52'123)", 2024-09-14T23:57:25.575 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.236123+0000", 2024-09-14T23:57:25.575 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.289895729000001, 2024-09-14T23:57:25.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0061273220000000002, 2024-09-14T23:57:25.575 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:25.575 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:25.575 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:25.575 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:25.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:25.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.236123+0000", 2024-09-14T23:57:25.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:25.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.236123+0000", 2024-09-14T23:57:25.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:25.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.236125+0000", 2024-09-14T23:57:25.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.313e-06 2024-09-14T23:57:25.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:25.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237428+0000", 2024-09-14T23:57:25.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013029890000000001 2024-09-14T23:57:25.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:25.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237452+0000", 2024-09-14T23:57:25.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4090000000000001e-05 2024-09-14T23:57:25.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:25.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237462+0000", 2024-09-14T23:57:25.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0421999999999999e-05 2024-09-14T23:57:25.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:25.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237559+0000", 2024-09-14T23:57:25.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6378000000000005e-05 2024-09-14T23:57:25.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:25.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237583+0000", 2024-09-14T23:57:25.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4182e-05 2024-09-14T23:57:25.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:25.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.242205+0000", 2024-09-14T23:57:25.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0046217289999999998 2024-09-14T23:57:25.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:25.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.242251+0000", 2024-09-14T23:57:25.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6218999999999999e-05 2024-09-14T23:57:25.578 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.579 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:25.579 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.579 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:56108 3.0 e56/23 3:00c47890:::rbd_data.1118c4b3aebe.0000000000000009:head v 56'127, mlcod=52'123)", 2024-09-14T23:57:25.579 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.239693+0000", 2024-09-14T23:57:25.579 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.286326258999999, 2024-09-14T23:57:25.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0043496209999999997, 2024-09-14T23:57:25.579 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:25.579 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:25.579 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:25.579 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:25.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:25.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239693+0000", 2024-09-14T23:57:25.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:25.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239693+0000", 2024-09-14T23:57:25.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:25.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239694+0000", 2024-09-14T23:57:25.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0499999999999999e-06 2024-09-14T23:57:25.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:25.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239702+0000", 2024-09-14T23:57:25.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1170000000000004e-06 2024-09-14T23:57:25.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:25.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239716+0000", 2024-09-14T23:57:25.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4306e-05 2024-09-14T23:57:25.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:25.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239724+0000", 2024-09-14T23:57:25.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0399999999999993e-06 2024-09-14T23:57:25.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:25.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239778+0000", 2024-09-14T23:57:25.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4032e-05 2024-09-14T23:57:25.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:25.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239801+0000", 2024-09-14T23:57:25.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2770000000000001e-05 2024-09-14T23:57:25.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:25.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.243990+0000", 2024-09-14T23:57:25.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0041890909999999998 2024-09-14T23:57:25.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:25.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.244043+0000", 2024-09-14T23:57:25.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2215000000000001e-05 2024-09-14T23:57:25.583 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.583 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:25.583 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.583 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:56385 3.9 e56/23 3:963d31c7:::rbd_data.1118c4b3aebe.0000000000000003:head v 56'87, mlcod=53'85)", 2024-09-14T23:57:25.583 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.283936+0000", 2024-09-14T23:57:25.583 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.242083349000001, 2024-09-14T23:57:25.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0063042080000000004, 2024-09-14T23:57:25.583 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:25.583 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:25.584 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:25.584 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:25.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:25.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.283936+0000", 2024-09-14T23:57:25.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:25.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.283936+0000", 2024-09-14T23:57:25.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:25.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.283937+0000", 2024-09-14T23:57:25.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1179999999999999e-06 2024-09-14T23:57:25.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:25.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286338+0000", 2024-09-14T23:57:25.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024010149999999998 2024-09-14T23:57:25.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:25.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286360+0000", 2024-09-14T23:57:25.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2212e-05 2024-09-14T23:57:25.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:25.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286370+0000", 2024-09-14T23:57:25.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5850000000000006e-06 2024-09-14T23:57:25.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:25.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286443+0000", 2024-09-14T23:57:25.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2757999999999997e-05 2024-09-14T23:57:25.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:25.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286466+0000", 2024-09-14T23:57:25.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3844000000000001e-05 2024-09-14T23:57:25.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:25.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.290197+0000", 2024-09-14T23:57:25.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003730176 2024-09-14T23:57:25.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:25.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.290240+0000", 2024-09-14T23:57:25.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.35e-05 2024-09-14T23:57:25.587 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.587 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:25.587 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.587 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:56386 3.9 e56/23 3:963d31c7:::rbd_data.1118c4b3aebe.0000000000000003:head v 56'89, mlcod=53'85)", 2024-09-14T23:57:25.588 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.287578+0000", 2024-09-14T23:57:25.588 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.238441623, 2024-09-14T23:57:25.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033580250000000002, 2024-09-14T23:57:25.588 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:25.588 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:25.588 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:25.588 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:25.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:25.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287578+0000", 2024-09-14T23:57:25.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:25.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287578+0000", 2024-09-14T23:57:25.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:25.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287579+0000", 2024-09-14T23:57:25.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.64e-07 2024-09-14T23:57:25.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:25.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287586+0000", 2024-09-14T23:57:25.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5310000000000004e-06 2024-09-14T23:57:25.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:25.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287597+0000", 2024-09-14T23:57:25.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0903999999999999e-05 2024-09-14T23:57:25.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:25.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287605+0000", 2024-09-14T23:57:25.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7889999999999995e-06 2024-09-14T23:57:25.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:25.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.288437+0000", 2024-09-14T23:57:25.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000831914 2024-09-14T23:57:25.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:25.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.288460+0000", 2024-09-14T23:57:25.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3410000000000001e-05 2024-09-14T23:57:25.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:25.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.290883+0000", 2024-09-14T23:57:25.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024230599999999999 2024-09-14T23:57:25.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:25.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.290936+0000", 2024-09-14T23:57:25.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2453000000000001e-05 2024-09-14T23:57:25.591 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.591 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:25.592 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.592 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:60299 3.9 e57/23 3:96f26b6f:::rbd_data.11188d356229.0000000000000009:head v 57'98, mlcod=56'96)", 2024-09-14T23:57:25.592 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:07.418336+0000", 2024-09-14T23:57:25.592 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.107683130000002, 2024-09-14T23:57:25.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022211459999999998, 2024-09-14T23:57:25.592 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:25.592 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:25.592 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:25.592 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:25.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:25.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.418336+0000", 2024-09-14T23:57:25.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:25.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.418336+0000", 2024-09-14T23:57:25.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:25.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.418337+0000", 2024-09-14T23:57:25.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7199999999999999e-07 2024-09-14T23:57:25.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:25.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.419396+0000", 2024-09-14T23:57:25.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010598319999999999 2024-09-14T23:57:25.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:25.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.419407+0000", 2024-09-14T23:57:25.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0956e-05 2024-09-14T23:57:25.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:25.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.419417+0000", 2024-09-14T23:57:25.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0040999999999999e-05 2024-09-14T23:57:25.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:25.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.419468+0000", 2024-09-14T23:57:25.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0497000000000001e-05 2024-09-14T23:57:25.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:25.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.419485+0000", 2024-09-14T23:57:25.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6966e-05 2024-09-14T23:57:25.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:25.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.420527+0000", 2024-09-14T23:57:25.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001042133 2024-09-14T23:57:25.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:25.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.420557+0000", 2024-09-14T23:57:25.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0249000000000001e-05 2024-09-14T23:57:25.596 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.596 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:25.596 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.596 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:64263 3.4 e57/23 3:20c6a768:::rbd_data.11188d356229.0000000000000001:head v 57'165, mlcod=57'163)", 2024-09-14T23:57:25.596 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:07.999031+0000", 2024-09-14T23:57:25.596 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 17.526988475, 2024-09-14T23:57:25.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023815210000000002, 2024-09-14T23:57:25.596 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:25.596 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:25.597 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:25.597 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:25.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:25.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999031+0000", 2024-09-14T23:57:25.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:25.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999031+0000", 2024-09-14T23:57:25.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:25.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999032+0000", 2024-09-14T23:57:25.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.176e-06 2024-09-14T23:57:25.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:25.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999660+0000", 2024-09-14T23:57:25.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00062817099999999996 2024-09-14T23:57:25.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:25.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999683+0000", 2024-09-14T23:57:25.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2816999999999999e-05 2024-09-14T23:57:25.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:25.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999693+0000", 2024-09-14T23:57:25.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0088999999999999e-05 2024-09-14T23:57:25.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:25.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999767+0000", 2024-09-14T23:57:25.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4056999999999999e-05 2024-09-14T23:57:25.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:25.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999793+0000", 2024-09-14T23:57:25.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5573000000000001e-05 2024-09-14T23:57:25.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:25.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:08.001379+0000", 2024-09-14T23:57:25.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015859019999999999 2024-09-14T23:57:25.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:25.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:08.001412+0000", 2024-09-14T23:57:25.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3735999999999997e-05 2024-09-14T23:57:25.600 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.600 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:25.600 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.600 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:65427 3.4 e60/23 3:269fb893:::rbd_data.11188d356229.00000000000000d9:head v 60'179, mlcod=58'175)", 2024-09-14T23:57:25.601 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:11.105942+0000", 2024-09-14T23:57:25.601 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.420077514999999, 2024-09-14T23:57:25.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001562393, 2024-09-14T23:57:25.601 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:25.601 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:25.601 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:25.601 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:25.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:25.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.105942+0000", 2024-09-14T23:57:25.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:25.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.105942+0000", 2024-09-14T23:57:25.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:25.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.105942+0000", 2024-09-14T23:57:25.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0299999999999998e-07 2024-09-14T23:57:25.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:25.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.105965+0000", 2024-09-14T23:57:25.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2629e-05 2024-09-14T23:57:25.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:25.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.105977+0000", 2024-09-14T23:57:25.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2023e-05 2024-09-14T23:57:25.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:25.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.105986+0000", 2024-09-14T23:57:25.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9670000000000007e-06 2024-09-14T23:57:25.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:25.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.106091+0000", 2024-09-14T23:57:25.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010527900000000001 2024-09-14T23:57:25.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:25.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.106112+0000", 2024-09-14T23:57:25.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0258000000000002e-05 2024-09-14T23:57:25.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:25.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.107460+0000", 2024-09-14T23:57:25.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001348393 2024-09-14T23:57:25.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:25.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.107504+0000", 2024-09-14T23:57:25.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4141000000000003e-05 2024-09-14T23:57:25.605 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.605 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:25.605 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.605 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:66570 3.0 e60/23 3:055350b3:::rbd_data.111873792d9b.0000000000000005:head v 60'162, mlcod=57'159)", 2024-09-14T23:57:25.605 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:11.277517+0000", 2024-09-14T23:57:25.605 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.248502319, 2024-09-14T23:57:25.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003472199, 2024-09-14T23:57:25.605 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:25.606 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:25.606 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:25.606 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:25.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:25.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.277517+0000", 2024-09-14T23:57:25.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:25.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.277517+0000", 2024-09-14T23:57:25.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:25.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.277518+0000", 2024-09-14T23:57:25.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1379999999999999e-06 2024-09-14T23:57:25.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:25.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278504+0000", 2024-09-14T23:57:25.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00098617499999999994 2024-09-14T23:57:25.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:25.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278525+0000", 2024-09-14T23:57:25.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.09e-05 2024-09-14T23:57:25.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:25.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278534+0000", 2024-09-14T23:57:25.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6209999999999993e-06 2024-09-14T23:57:25.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:25.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278611+0000", 2024-09-14T23:57:25.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6926000000000002e-05 2024-09-14T23:57:25.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:25.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278641+0000", 2024-09-14T23:57:25.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0861999999999997e-05 2024-09-14T23:57:25.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:25.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.280953+0000", 2024-09-14T23:57:25.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023114239999999999 2024-09-14T23:57:25.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:25.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.280989+0000", 2024-09-14T23:57:25.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6152999999999998e-05 2024-09-14T23:57:25.609 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.609 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:25.609 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.610 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:70526 3.4 e64/23 3:2649db36:::rbd_data.111831f32e80.0000000000000058:head v 64'183, mlcod=64'181)", 2024-09-14T23:57:25.610 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:15.442503+0000", 2024-09-14T23:57:25.610 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.083516426999999, 2024-09-14T23:57:25.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001769837, 2024-09-14T23:57:25.610 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:25.610 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:25.610 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:25.610 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:25.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:25.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.442503+0000", 2024-09-14T23:57:25.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:25.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.442503+0000", 2024-09-14T23:57:25.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:25.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.442503+0000", 2024-09-14T23:57:25.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1099999999999995e-07 2024-09-14T23:57:25.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:25.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.442620+0000", 2024-09-14T23:57:25.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000116618 2024-09-14T23:57:25.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:25.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.442634+0000", 2024-09-14T23:57:25.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3984e-05 2024-09-14T23:57:25.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:25.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.442641+0000", 2024-09-14T23:57:25.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2540000000000003e-06 2024-09-14T23:57:25.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:25.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.442732+0000", 2024-09-14T23:57:25.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0480000000000001e-05 2024-09-14T23:57:25.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:25.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.442753+0000", 2024-09-14T23:57:25.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1571999999999999e-05 2024-09-14T23:57:25.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:25.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.444236+0000", 2024-09-14T23:57:25.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0014825439999999999 2024-09-14T23:57:25.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:25.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.444273+0000", 2024-09-14T23:57:25.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6674000000000002e-05 2024-09-14T23:57:25.613 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.613 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:25.614 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.614 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:106629 3.4 e70/23 3:240d85c6:::rbd_data.11187de8b0a5.0000000000000009:head v 70'233, mlcod=69'231)", 2024-09-14T23:57:25.614 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:21.489517+0000", 2024-09-14T23:57:25.614 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.0365017940000003, 2024-09-14T23:57:25.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016279840000000001, 2024-09-14T23:57:25.614 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:25.614 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:25.614 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:25.614 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:25.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:25.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.489517+0000", 2024-09-14T23:57:25.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:25.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.489517+0000", 2024-09-14T23:57:25.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:25.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:25.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.489518+0000", 2024-09-14T23:57:25.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.029e-06 2024-09-14T23:57:25.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:25.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.489840+0000", 2024-09-14T23:57:25.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00032164100000000001 2024-09-14T23:57:25.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:25.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.489859+0000", 2024-09-14T23:57:25.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8828999999999999e-05 2024-09-14T23:57:25.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:25.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.489869+0000", 2024-09-14T23:57:25.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6600000000000007e-06 2024-09-14T23:57:25.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:25.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.489934+0000", 2024-09-14T23:57:25.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5357000000000005e-05 2024-09-14T23:57:25.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:25.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.489951+0000", 2024-09-14T23:57:25.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6951000000000001e-05 2024-09-14T23:57:25.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:25.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.491108+0000", 2024-09-14T23:57:25.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011571940000000001 2024-09-14T23:57:25.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:25.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:25.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:25.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.491145+0000", 2024-09-14T23:57:25.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7323000000000002e-05 2024-09-14T23:57:25.618 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.618 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:25.618 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.618 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:25.618 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:25.618 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:25.620 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T23:57:25.620 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:25.618+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:25.720 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:25.719+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:25.769 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:25.769 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:57:25.769 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:57:25.769 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:25.777 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T23:57:25.821 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:25.819+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:25.885 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:25.885 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:57:25.885 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:57:25.885 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:57:25.885 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:25.894 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T23:57:25.921 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:25.920+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:25.981 INFO:teuthology.orchestra.run.smithi042.stdout:2728 read 0x4d96ad9 thru 0x4d99cbd (0x31e5 bytes) 2024-09-14T23:57:25.981 INFO:teuthology.orchestra.run.smithi042.stdout:2730 punch from 0xbb03521 to 0xbb0f3f2, (0xbed1 bytes) 2024-09-14T23:57:25.981 INFO:teuthology.orchestra.run.smithi042.stdout:2731 read 0x30de24 thru 0x31bafe (0xdcdb bytes) 2024-09-14T23:57:25.981 INFO:teuthology.orchestra.run.smithi042.stdout:2733 read 0x8d193fc thru 0x8d22243 (0x8e48 bytes) 2024-09-14T23:57:25.982 INFO:teuthology.orchestra.run.smithi042.stdout:2734 trunc from 0xd8aebe2 to 0xc76a8cb 2024-09-14T23:57:25.997 INFO:teuthology.orchestra.run.smithi042.stdout:2735 trunc from 0xc76a8cb to 0xaaaeacb 2024-09-14T23:57:26.008 INFO:teuthology.orchestra.run.smithi042.stdout:2737 read 0x20c085d thru 0x20ca5cf (0x9d73 bytes) 2024-09-14T23:57:26.008 INFO:teuthology.orchestra.run.smithi042.stdout:2738 punch from 0xa5c2d8 to 0xa67141, (0xae69 bytes) 2024-09-14T23:57:26.008 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:26.008 INFO:teuthology.orchestra.run.smithi042.stdout:2740 read 0x84e4737 thru 0x84f3034 (0xe8fe bytes) 2024-09-14T23:57:26.017 INFO:teuthology.orchestra.run.smithi042.stdout:2741 write 0xd8a6635 thru 0xd8ab3a0 (0x4d6c bytes) 2024-09-14T23:57:26.021 INFO:teuthology.orchestra.run.smithi042.stdout:2742 punch from 0x7fde959 to 0x7fe2aa5, (0x414c bytes) 2024-09-14T23:57:26.021 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:26.020+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:26.036 INFO:teuthology.orchestra.run.smithi042.stdout:2743 trunc from 0xd8ab3a1 to 0xf4b475 2024-09-14T23:57:26.063 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:26.063 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:57:26.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:57:26.063 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:57:26.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.063 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5180 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3883008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:26.063 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.250058+0000", 2024-09-14T23:57:26.064 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.810743197999997, 2024-09-14T23:57:26.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057351471000000001, 2024-09-14T23:57:26.064 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:26.064 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:26.064 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:26.064 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:26.064 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:26.064 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:26.064 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5180 2024-09-14T23:57:26.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.064 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:26.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:26.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250058+0000", 2024-09-14T23:57:26.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:26.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:26.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250058+0000", 2024-09-14T23:57:26.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:26.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:26.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250786+0000", 2024-09-14T23:57:26.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00072813399999999996 2024-09-14T23:57:26.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:26.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250789+0000", 2024-09-14T23:57:26.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.647e-06 2024-09-14T23:57:26.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:26.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250798+0000", 2024-09-14T23:57:26.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0000000000000002e-06 2024-09-14T23:57:26.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:26.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250803+0000", 2024-09-14T23:57:26.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.301e-06 2024-09-14T23:57:26.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:26.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.307234+0000", 2024-09-14T23:57:26.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056431165999999998 2024-09-14T23:57:26.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:26.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.307321+0000", 2024-09-14T23:57:26.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6831999999999998e-05 2024-09-14T23:57:26.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:26.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.307409+0000", 2024-09-14T23:57:26.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8391000000000001e-05 2024-09-14T23:57:26.068 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:26.068 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:26.069 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:26.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.069 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5184 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 4128768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:26.069 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.250851+0000", 2024-09-14T23:57:26.069 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.809949727000003, 2024-09-14T23:57:26.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057319038000000003, 2024-09-14T23:57:26.069 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:26.069 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:26.069 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:26.069 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:26.069 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:26.069 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:26.070 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5184 2024-09-14T23:57:26.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.070 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:26.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:26.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250851+0000", 2024-09-14T23:57:26.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:26.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:26.071 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250851+0000", 2024-09-14T23:57:26.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:26.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:26.071 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.251900+0000", 2024-09-14T23:57:26.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001048577 2024-09-14T23:57:26.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:26.071 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.251904+0000", 2024-09-14T23:57:26.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9580000000000001e-06 2024-09-14T23:57:26.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:26.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.251914+0000", 2024-09-14T23:57:26.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0186999999999999e-05 2024-09-14T23:57:26.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:26.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.251927+0000", 2024-09-14T23:57:26.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3593e-05 2024-09-14T23:57:26.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:26.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.308020+0000", 2024-09-14T23:57:26.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056092867999999997 2024-09-14T23:57:26.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:26.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.308094+0000", 2024-09-14T23:57:26.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3533000000000002e-05 2024-09-14T23:57:26.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:26.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.308170+0000", 2024-09-14T23:57:26.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6322000000000006e-05 2024-09-14T23:57:26.073 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:26.073 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:26.073 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:26.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.074 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5196 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 1925120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:26.074 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.252157+0000", 2024-09-14T23:57:26.074 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.808643613000001, 2024-09-14T23:57:26.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057285801999999997, 2024-09-14T23:57:26.074 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:26.074 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:26.074 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:26.074 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:26.074 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:26.074 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:26.074 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5196 2024-09-14T23:57:26.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.074 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:26.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:26.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.252157+0000", 2024-09-14T23:57:26.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:26.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:26.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.252157+0000", 2024-09-14T23:57:26.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:26.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:26.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253243+0000", 2024-09-14T23:57:26.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010861340000000001 2024-09-14T23:57:26.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:26.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253248+0000", 2024-09-14T23:57:26.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8799999999999999e-06 2024-09-14T23:57:26.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:26.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253271+0000", 2024-09-14T23:57:26.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2836e-05 2024-09-14T23:57:26.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:26.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253281+0000", 2024-09-14T23:57:26.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6520000000000001e-06 2024-09-14T23:57:26.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:26.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.309336+0000", 2024-09-14T23:57:26.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056054983000000003 2024-09-14T23:57:26.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:26.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.309389+0000", 2024-09-14T23:57:26.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3298999999999999e-05 2024-09-14T23:57:26.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:26.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.309443+0000", 2024-09-14T23:57:26.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4017999999999997e-05 2024-09-14T23:57:26.078 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:26.078 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:26.078 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:26.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.078 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5200 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2170880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:26.078 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253394+0000", 2024-09-14T23:57:26.078 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.807406835999998, 2024-09-14T23:57:26.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057428154000000002, 2024-09-14T23:57:26.078 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:26.078 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:26.079 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:26.079 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:26.079 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:26.079 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:26.079 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5200 2024-09-14T23:57:26.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.079 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:26.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:26.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253394+0000", 2024-09-14T23:57:26.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:26.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:26.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253394+0000", 2024-09-14T23:57:26.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:26.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:26.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253395+0000", 2024-09-14T23:57:26.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9100000000000004e-07 2024-09-14T23:57:26.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:26.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253397+0000", 2024-09-14T23:57:26.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2790000000000001e-06 2024-09-14T23:57:26.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:26.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253404+0000", 2024-09-14T23:57:26.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7959999999999998e-06 2024-09-14T23:57:26.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:26.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253409+0000", 2024-09-14T23:57:26.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1460000000000004e-06 2024-09-14T23:57:26.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:26.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310627+0000", 2024-09-14T23:57:26.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057218160999999997 2024-09-14T23:57:26.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:26.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310710+0000", 2024-09-14T23:57:26.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3102999999999994e-05 2024-09-14T23:57:26.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:26.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310822+0000", 2024-09-14T23:57:26.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000112178 2024-09-14T23:57:26.082 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:26.082 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:26.082 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:26.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.083 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5201 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2232320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:26.083 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:57:26.083 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.807382164000003, 2024-09-14T23:57:26.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057643228999999997, 2024-09-14T23:57:26.083 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:26.083 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:26.083 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:26.083 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:26.083 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:26.083 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:26.083 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5201 2024-09-14T23:57:26.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.084 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:26.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:26.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:57:26.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:26.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:26.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:57:26.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:26.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:26.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:57:26.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0699999999999998e-07 2024-09-14T23:57:26.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:26.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253422+0000", 2024-09-14T23:57:26.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.514e-06 2024-09-14T23:57:26.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:26.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253431+0000", 2024-09-14T23:57:26.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1919999999999994e-06 2024-09-14T23:57:26.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:26.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253436+0000", 2024-09-14T23:57:26.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5350000000000004e-06 2024-09-14T23:57:26.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:26.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310856+0000", 2024-09-14T23:57:26.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057419381999999998 2024-09-14T23:57:26.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:26.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310953+0000", 2024-09-14T23:57:26.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6774000000000006e-05 2024-09-14T23:57:26.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:26.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311062+0000", 2024-09-14T23:57:26.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000109425 2024-09-14T23:57:26.087 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:26.087 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:26.087 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:26.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.087 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5202 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2293760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:26.087 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:57:26.088 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.807357416000002, 2024-09-14T23:57:26.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057818255999999998, 2024-09-14T23:57:26.088 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:26.088 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:26.088 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:26.088 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:26.088 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:26.088 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:26.088 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5202 2024-09-14T23:57:26.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.088 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:26.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:26.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:57:26.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:26.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:26.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:57:26.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:26.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:26.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253573+0000", 2024-09-14T23:57:26.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012934000000000001 2024-09-14T23:57:26.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:26.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253576+0000", 2024-09-14T23:57:26.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5310000000000002e-06 2024-09-14T23:57:26.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:26.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253587+0000", 2024-09-14T23:57:26.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0522e-05 2024-09-14T23:57:26.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:26.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253595+0000", 2024-09-14T23:57:26.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8290000000000008e-06 2024-09-14T23:57:26.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:26.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311093+0000", 2024-09-14T23:57:26.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057498312000000003 2024-09-14T23:57:26.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:26.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311167+0000", 2024-09-14T23:57:26.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3582000000000001e-05 2024-09-14T23:57:26.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:26.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311262+0000", 2024-09-14T23:57:26.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5140000000000001e-05 2024-09-14T23:57:26.091 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:26.092 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:26.092 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:26.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.092 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5203 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2355200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:26.092 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:57:26.092 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.807194674000002, 2024-09-14T23:57:26.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057903938000000002, 2024-09-14T23:57:26.092 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:26.092 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:26.092 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:26.092 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:26.092 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:26.092 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:26.093 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5203 2024-09-14T23:57:26.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.093 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:26.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:26.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:57:26.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:26.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:26.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:57:26.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:26.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:26.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253902+0000", 2024-09-14T23:57:26.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029615400000000002 2024-09-14T23:57:26.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:26.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253906+0000", 2024-09-14T23:57:26.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9249999999999997e-06 2024-09-14T23:57:26.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:26.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253913+0000", 2024-09-14T23:57:26.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9099999999999999e-06 2024-09-14T23:57:26.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:26.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253917+0000", 2024-09-14T23:57:26.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9210000000000002e-06 2024-09-14T23:57:26.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:26.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311295+0000", 2024-09-14T23:57:26.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057378268000000003 2024-09-14T23:57:26.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:26.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311402+0000", 2024-09-14T23:57:26.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010644100000000001 2024-09-14T23:57:26.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:26.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311510+0000", 2024-09-14T23:57:26.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108319 2024-09-14T23:57:26.096 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:26.096 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:26.096 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:26.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.096 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5204 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2416640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:26.096 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:57:26.096 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.806876477000003, 2024-09-14T23:57:26.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057774860999999997, 2024-09-14T23:57:26.097 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:26.097 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:26.097 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:26.097 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:26.097 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:26.097 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:26.097 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5204 2024-09-14T23:57:26.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.097 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:26.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:26.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:57:26.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:26.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:26.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:57:26.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:26.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:26.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255903+0000", 2024-09-14T23:57:26.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001978807 2024-09-14T23:57:26.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:26.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255907+0000", 2024-09-14T23:57:26.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3610000000000001e-06 2024-09-14T23:57:26.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:26.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255914+0000", 2024-09-14T23:57:26.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.357e-06 2024-09-14T23:57:26.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:26.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255918+0000", 2024-09-14T23:57:26.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1779999999999996e-06 2024-09-14T23:57:26.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:26.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311541+0000", 2024-09-14T23:57:26.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055622697999999998 2024-09-14T23:57:26.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:26.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311614+0000", 2024-09-14T23:57:26.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3102999999999994e-05 2024-09-14T23:57:26.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:26.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311699+0000", 2024-09-14T23:57:26.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5357000000000003e-05 2024-09-14T23:57:26.100 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:26.100 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:26.100 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:26.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.101 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5210 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2785280~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:26.101 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256011+0000", 2024-09-14T23:57:26.101 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.804789839999998, 2024-09-14T23:57:26.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057273662000000003, 2024-09-14T23:57:26.101 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:26.101 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:26.101 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:26.101 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:26.101 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:26.101 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:26.101 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5210 2024-09-14T23:57:26.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.102 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:26.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:26.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256011+0000", 2024-09-14T23:57:26.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:26.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:26.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256011+0000", 2024-09-14T23:57:26.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:26.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:26.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256011+0000", 2024-09-14T23:57:26.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4999999999999999e-07 2024-09-14T23:57:26.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:26.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256013+0000", 2024-09-14T23:57:26.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3969999999999999e-06 2024-09-14T23:57:26.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:26.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256017+0000", 2024-09-14T23:57:26.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1409999999999998e-06 2024-09-14T23:57:26.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:26.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256019+0000", 2024-09-14T23:57:26.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6280000000000001e-06 2024-09-14T23:57:26.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:26.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313116+0000", 2024-09-14T23:57:26.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057096457000000003 2024-09-14T23:57:26.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:26.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313193+0000", 2024-09-14T23:57:26.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7439000000000002e-05 2024-09-14T23:57:26.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:26.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313285+0000", 2024-09-14T23:57:26.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1349999999999998e-05 2024-09-14T23:57:26.105 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:26.105 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:26.105 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:26.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.105 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5211 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2846720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:26.105 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:57:26.105 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.804778263000003, 2024-09-14T23:57:26.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057528151, 2024-09-14T23:57:26.105 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:26.106 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:26.106 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:26.106 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:26.106 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:26.106 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:26.106 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5211 2024-09-14T23:57:26.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.106 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:26.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:26.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:57:26.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:26.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:26.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:57:26.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:26.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:26.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:57:26.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1399999999999998e-07 2024-09-14T23:57:26.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:26.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256024+0000", 2024-09-14T23:57:26.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.114e-06 2024-09-14T23:57:26.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:26.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256028+0000", 2024-09-14T23:57:26.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0840000000000002e-06 2024-09-14T23:57:26.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:26.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256031+0000", 2024-09-14T23:57:26.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6819999999999999e-06 2024-09-14T23:57:26.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:26.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313317+0000", 2024-09-14T23:57:26.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057286295000000001 2024-09-14T23:57:26.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:26.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313403+0000", 2024-09-14T23:57:26.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6287999999999998e-05 2024-09-14T23:57:26.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:26.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313551+0000", 2024-09-14T23:57:26.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000147374 2024-09-14T23:57:26.109 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:26.109 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:26.109 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:26.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.110 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5212 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2908160~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:26.110 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:57:26.110 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.804767069, 2024-09-14T23:57:26.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057735679999999998, 2024-09-14T23:57:26.110 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:26.110 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:26.110 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:26.110 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:26.110 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:26.110 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:26.110 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5212 2024-09-14T23:57:26.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.110 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:26.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:26.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:57:26.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:26.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:26.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:57:26.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:26.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:26.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:57:26.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0900000000000001e-07 2024-09-14T23:57:26.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:26.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256035+0000", 2024-09-14T23:57:26.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1200000000000001e-06 2024-09-14T23:57:26.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:26.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256038+0000", 2024-09-14T23:57:26.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.05e-06 2024-09-14T23:57:26.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:26.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256041+0000", 2024-09-14T23:57:26.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.667e-06 2024-09-14T23:57:26.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:26.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313584+0000", 2024-09-14T23:57:26.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057543484999999998 2024-09-14T23:57:26.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:26.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313668+0000", 2024-09-14T23:57:26.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3807000000000006e-05 2024-09-14T23:57:26.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:26.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313770+0000", 2024-09-14T23:57:26.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010134200000000001 2024-09-14T23:57:26.114 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:26.114 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:26.114 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:26.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.114 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5213 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2969600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:26.114 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:57:26.114 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.804756982000001, 2024-09-14T23:57:26.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058145467999999999, 2024-09-14T23:57:26.114 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:26.114 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:26.114 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:26.115 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:26.115 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:26.115 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:26.115 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5213 2024-09-14T23:57:26.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.115 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:26.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:26.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:57:26.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:26.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:26.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:57:26.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:26.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:26.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257901+0000", 2024-09-14T23:57:26.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018567519999999999 2024-09-14T23:57:26.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:26.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257904+0000", 2024-09-14T23:57:26.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4850000000000001e-06 2024-09-14T23:57:26.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:26.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257912+0000", 2024-09-14T23:57:26.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2090000000000005e-06 2024-09-14T23:57:26.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:26.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257918+0000", 2024-09-14T23:57:26.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6899999999999997e-06 2024-09-14T23:57:26.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:26.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313816+0000", 2024-09-14T23:57:26.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055898075999999998 2024-09-14T23:57:26.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:26.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.314082+0000", 2024-09-14T23:57:26.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000265386 2024-09-14T23:57:26.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:26.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.314189+0000", 2024-09-14T23:57:26.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010787 2024-09-14T23:57:26.118 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:26.118 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:26.118 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:26.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.118 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5217 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3215360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:26.118 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257961+0000", 2024-09-14T23:57:26.119 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.802840003, 2024-09-14T23:57:26.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057320969999999999, 2024-09-14T23:57:26.119 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:26.119 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:26.119 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:26.119 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:26.119 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:26.119 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:26.119 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5217 2024-09-14T23:57:26.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.119 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:26.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:26.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257961+0000", 2024-09-14T23:57:26.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:26.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:26.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257961+0000", 2024-09-14T23:57:26.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:26.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:26.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257961+0000", 2024-09-14T23:57:26.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.22e-07 2024-09-14T23:57:26.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:26.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257962+0000", 2024-09-14T23:57:26.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1400000000000001e-06 2024-09-14T23:57:26.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:26.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257965+0000", 2024-09-14T23:57:26.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0929999999999999e-06 2024-09-14T23:57:26.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:26.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257968+0000", 2024-09-14T23:57:26.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5749999999999999e-06 2024-09-14T23:57:26.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:26.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315047+0000", 2024-09-14T23:57:26.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057078730000000001 2024-09-14T23:57:26.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:26.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315087+0000", 2024-09-14T23:57:26.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0806999999999997e-05 2024-09-14T23:57:26.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:26.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315282+0000", 2024-09-14T23:57:26.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000194403 2024-09-14T23:57:26.123 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:26.123 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:26.123 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:26.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.123 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5218 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3276800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:26.123 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:57:26.123 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.802829973000001, 2024-09-14T23:57:26.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057497557999999997, 2024-09-14T23:57:26.123 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:26.123 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:26.123 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:26.123 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:26.123 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:26.124 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:26.124 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5218 2024-09-14T23:57:26.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.124 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:26.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:26.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:57:26.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:26.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:26.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:57:26.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:26.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:26.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:57:26.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1899999999999999e-07 2024-09-14T23:57:26.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:26.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257972+0000", 2024-09-14T23:57:26.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.217e-06 2024-09-14T23:57:26.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:26.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257975+0000", 2024-09-14T23:57:26.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9359999999999999e-06 2024-09-14T23:57:26.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:26.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257978+0000", 2024-09-14T23:57:26.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1049999999999999e-06 2024-09-14T23:57:26.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:26.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315331+0000", 2024-09-14T23:57:26.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057352093999999999 2024-09-14T23:57:26.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:26.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315384+0000", 2024-09-14T23:57:26.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3780999999999999e-05 2024-09-14T23:57:26.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:26.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315468+0000", 2024-09-14T23:57:26.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4206e-05 2024-09-14T23:57:26.127 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:26.127 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:26.127 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:26.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.127 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5219 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3338240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:26.127 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:57:26.127 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.802819296999999, 2024-09-14T23:57:26.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057637764000000001, 2024-09-14T23:57:26.128 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:26.128 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:26.128 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:26.128 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:26.128 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:26.128 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:26.128 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5219 2024-09-14T23:57:26.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.128 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:26.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:26.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:57:26.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:26.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:26.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:57:26.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:26.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:26.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:57:26.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.03e-07 2024-09-14T23:57:26.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:26.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257983+0000", 2024-09-14T23:57:26.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9199999999999999e-07 2024-09-14T23:57:26.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:26.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257986+0000", 2024-09-14T23:57:26.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0860000000000002e-06 2024-09-14T23:57:26.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:26.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257989+0000", 2024-09-14T23:57:26.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6589999999999999e-06 2024-09-14T23:57:26.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:26.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315509+0000", 2024-09-14T23:57:26.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057520262000000003 2024-09-14T23:57:26.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:26.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315554+0000", 2024-09-14T23:57:26.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5015999999999997e-05 2024-09-14T23:57:26.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:26.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315619+0000", 2024-09-14T23:57:26.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5545999999999999e-05 2024-09-14T23:57:26.131 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:26.132 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:26.132 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:26.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.132 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5220 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3399680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:26.132 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:57:26.132 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.802809341, 2024-09-14T23:57:26.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057792128999999998, 2024-09-14T23:57:26.132 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:26.132 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:26.132 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:26.132 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:26.132 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:26.132 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:26.133 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5220 2024-09-14T23:57:26.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.133 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:26.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:26.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:57:26.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:26.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:26.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:57:26.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:26.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:26.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:57:26.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1799999999999999e-07 2024-09-14T23:57:26.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:26.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257993+0000", 2024-09-14T23:57:26.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6200000000000006e-07 2024-09-14T23:57:26.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:26.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257996+0000", 2024-09-14T23:57:26.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9050000000000001e-06 2024-09-14T23:57:26.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:26.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257998+0000", 2024-09-14T23:57:26.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5950000000000001e-06 2024-09-14T23:57:26.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:26.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315672+0000", 2024-09-14T23:57:26.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057673637999999999 2024-09-14T23:57:26.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:26.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315734+0000", 2024-09-14T23:57:26.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2216000000000002e-05 2024-09-14T23:57:26.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:26.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315784+0000", 2024-09-14T23:57:26.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9595000000000002e-05 2024-09-14T23:57:26.136 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:26.136 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:26.136 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:26.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.136 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5221 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3461120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:26.136 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:57:26.136 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.802799622999999, 2024-09-14T23:57:26.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058003557999999997, 2024-09-14T23:57:26.136 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:26.137 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:26.137 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:26.137 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:26.137 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:26.137 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:26.137 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5221 2024-09-14T23:57:26.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.137 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:26.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:26.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:57:26.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:26.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:26.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:57:26.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:26.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:26.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259905+0000", 2024-09-14T23:57:26.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019035759999999999 2024-09-14T23:57:26.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:26.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259915+0000", 2024-09-14T23:57:26.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0518e-05 2024-09-14T23:57:26.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:26.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259925+0000", 2024-09-14T23:57:26.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7180000000000008e-06 2024-09-14T23:57:26.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:26.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259930+0000", 2024-09-14T23:57:26.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4769999999999997e-06 2024-09-14T23:57:26.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:26.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315820+0000", 2024-09-14T23:57:26.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055889960000000002 2024-09-14T23:57:26.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:26.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315914+0000", 2024-09-14T23:57:26.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4625999999999998e-05 2024-09-14T23:57:26.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:26.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.316005+0000", 2024-09-14T23:57:26.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0682999999999999e-05 2024-09-14T23:57:26.140 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:26.140 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:26.140 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:26.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.141 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5230 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 4014080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:26.141 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.260060+0000", 2024-09-14T23:57:26.141 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.800740785000002, 2024-09-14T23:57:26.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057353556999999999, 2024-09-14T23:57:26.141 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:26.141 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:26.141 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:26.141 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:26.141 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:26.141 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:26.141 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5230 2024-09-14T23:57:26.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.141 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:26.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:26.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260060+0000", 2024-09-14T23:57:26.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:26.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:26.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260060+0000", 2024-09-14T23:57:26.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:26.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:26.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260060+0000", 2024-09-14T23:57:26.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.17e-07 2024-09-14T23:57:26.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:26.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260061+0000", 2024-09-14T23:57:26.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0300000000000001e-06 2024-09-14T23:57:26.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:26.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260064+0000", 2024-09-14T23:57:26.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9119999999999998e-06 2024-09-14T23:57:26.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:26.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260067+0000", 2024-09-14T23:57:26.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.678e-06 2024-09-14T23:57:26.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:26.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317269+0000", 2024-09-14T23:57:26.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057201771999999998 2024-09-14T23:57:26.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:26.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317342+0000", 2024-09-14T23:57:26.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2842000000000005e-05 2024-09-14T23:57:26.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:26.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317414+0000", 2024-09-14T23:57:26.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2106000000000004e-05 2024-09-14T23:57:26.145 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:26.145 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:26.145 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:26.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.145 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5231 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 4075520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:26.145 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:57:26.145 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.800730889, 2024-09-14T23:57:26.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057524733000000002, 2024-09-14T23:57:26.145 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:26.145 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:26.145 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:26.146 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:26.146 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:26.146 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:26.146 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5231 2024-09-14T23:57:26.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.146 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:26.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:26.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:57:26.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:26.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:26.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:57:26.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:26.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:26.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261904+0000", 2024-09-14T23:57:26.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018338790000000001 2024-09-14T23:57:26.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:26.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261910+0000", 2024-09-14T23:57:26.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9689999999999999e-06 2024-09-14T23:57:26.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:26.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261923+0000", 2024-09-14T23:57:26.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3061000000000001e-05 2024-09-14T23:57:26.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:26.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261931+0000", 2024-09-14T23:57:26.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3159999999999997e-06 2024-09-14T23:57:26.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:26.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317456+0000", 2024-09-14T23:57:26.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055525219000000001 2024-09-14T23:57:26.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:26.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317520+0000", 2024-09-14T23:57:26.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4010000000000005e-05 2024-09-14T23:57:26.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:26.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317595+0000", 2024-09-14T23:57:26.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4278999999999998e-05 2024-09-14T23:57:26.149 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:26.149 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:26.149 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:26.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.149 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5236 3.8 3:1071668d:::rbd_data.1118de1168b.0000000000000092:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:26.150 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:57:26.150 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.798717425, 2024-09-14T23:57:26.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057499462000000001, 2024-09-14T23:57:26.150 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:26.150 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:26.150 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:26.150 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:26.150 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:26.150 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:26.150 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5236 2024-09-14T23:57:26.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.150 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:26.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:26.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:57:26.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:26.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:26.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:57:26.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:26.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:26.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262084+0000", 2024-09-14T23:57:26.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3399999999999999e-07 2024-09-14T23:57:26.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:26.152 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262087+0000", 2024-09-14T23:57:26.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2310000000000001e-06 2024-09-14T23:57:26.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.152 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:26.152 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262095+0000", 2024-09-14T23:57:26.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3799999999999996e-06 2024-09-14T23:57:26.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.152 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:26.152 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262101+0000", 2024-09-14T23:57:26.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0009999999999998e-06 2024-09-14T23:57:26.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:26.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317625+0000", 2024-09-14T23:57:26.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055524111000000001 2024-09-14T23:57:26.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:26.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317702+0000", 2024-09-14T23:57:26.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7533999999999993e-05 2024-09-14T23:57:26.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-14T23:57:26.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317892+0000", 2024-09-14T23:57:26.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018991400000000001 2024-09-14T23:57:26.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:57:26.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319407+0000", 2024-09-14T23:57:26.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015145250000000001 2024-09-14T23:57:26.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:26.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319481+0000", 2024-09-14T23:57:26.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3826999999999996e-05 2024-09-14T23:57:26.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:26.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319541+0000", 2024-09-14T23:57:26.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0470999999999998e-05 2024-09-14T23:57:26.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:26.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:26.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:26.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319583+0000", 2024-09-14T23:57:26.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1934000000000002e-05 2024-09-14T23:57:26.155 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:26.155 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:26.155 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:26.155 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:26.155 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:26.155 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:26.157 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T23:57:26.157 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:26.155+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:26.257 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:26.256+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:26.265 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:26.265 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:26.265 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:57:26.266 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:26.274 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T23:57:26.358 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:26.357+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:26.381 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:26.381 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:26.381 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:57:26.381 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:57:26.381 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:26.389 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T23:57:26.390 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T23:57:26.459 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:26.458+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:26.498 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:26.498 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:57:26.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:57:26.499 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:57:26.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.499 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:3500 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T23:57:26.499 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:37.990193+0000", 2024-09-14T23:57:26.499 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 48.506069906, 2024-09-14T23:57:26.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99440750899999997, 2024-09-14T23:57:26.499 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:26.499 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:26.499 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:26.499 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:26.499 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:26.499 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:26.500 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3500 2024-09-14T23:57:26.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.500 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:26.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:26.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990193+0000", 2024-09-14T23:57:26.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:26.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:26.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990193+0000", 2024-09-14T23:57:26.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:26.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:26.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990194+0000", 2024-09-14T23:57:26.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5000000000000001e-07 2024-09-14T23:57:26.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:26.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990200+0000", 2024-09-14T23:57:26.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4389999999999997e-06 2024-09-14T23:57:26.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:26.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990215+0000", 2024-09-14T23:57:26.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4613999999999999e-05 2024-09-14T23:57:26.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:26.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990224+0000", 2024-09-14T23:57:26.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1740000000000006e-06 2024-09-14T23:57:26.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:26.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990283+0000", 2024-09-14T23:57:26.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9231999999999998e-05 2024-09-14T23:57:26.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:57:26.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990291+0000", 2024-09-14T23:57:26.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5900000000000002e-06 2024-09-14T23:57:26.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:26.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:38.984249+0000", 2024-09-14T23:57:26.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99395822700000003 2024-09-14T23:57:26.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:26.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:38.984448+0000", 2024-09-14T23:57:26.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00019855900000000001 2024-09-14T23:57:26.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:26.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:38.984601+0000", 2024-09-14T23:57:26.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000152824 2024-09-14T23:57:26.503 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:26.504 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:26.504 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:26.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.504 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4055 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1093632~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:26.504 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099066+0000", 2024-09-14T23:57:26.504 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.397197018, 2024-09-14T23:57:26.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044174782000000003, 2024-09-14T23:57:26.504 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:26.504 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:26.504 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:26.504 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:26.504 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:26.504 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:26.504 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4055 2024-09-14T23:57:26.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.505 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:26.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:26.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099066+0000", 2024-09-14T23:57:26.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:26.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:26.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099066+0000", 2024-09-14T23:57:26.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:26.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:26.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099066+0000", 2024-09-14T23:57:26.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3700000000000001e-07 2024-09-14T23:57:26.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:26.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099067+0000", 2024-09-14T23:57:26.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1379999999999999e-06 2024-09-14T23:57:26.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:26.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099073+0000", 2024-09-14T23:57:26.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4790000000000004e-06 2024-09-14T23:57:26.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.507 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:26.507 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099075+0000", 2024-09-14T23:57:26.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5320000000000001e-06 2024-09-14T23:57:26.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.507 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:26.507 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143147+0000", 2024-09-14T23:57:26.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044071676999999997 2024-09-14T23:57:26.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.507 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:26.507 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143186+0000", 2024-09-14T23:57:26.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8821e-05 2024-09-14T23:57:26.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:26.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143241+0000", 2024-09-14T23:57:26.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4797999999999999e-05 2024-09-14T23:57:26.508 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:26.508 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:26.508 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:26.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.508 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4056 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1155072~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:26.508 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099079+0000", 2024-09-14T23:57:26.508 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.397184303000003, 2024-09-14T23:57:26.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044295020999999997, 2024-09-14T23:57:26.508 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:26.508 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:26.509 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:26.509 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:26.509 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:26.509 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:26.509 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4056 2024-09-14T23:57:26.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.509 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:26.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:26.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099079+0000", 2024-09-14T23:57:26.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:26.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:26.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099079+0000", 2024-09-14T23:57:26.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:26.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:26.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099079+0000", 2024-09-14T23:57:26.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1500000000000001e-07 2024-09-14T23:57:26.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:26.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099080+0000", 2024-09-14T23:57:26.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0210000000000001e-06 2024-09-14T23:57:26.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:26.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099083+0000", 2024-09-14T23:57:26.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.174e-06 2024-09-14T23:57:26.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:26.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099086+0000", 2024-09-14T23:57:26.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.446e-06 2024-09-14T23:57:26.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:26.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143269+0000", 2024-09-14T23:57:26.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044183489999999999 2024-09-14T23:57:26.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:26.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143314+0000", 2024-09-14T23:57:26.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5135000000000001e-05 2024-09-14T23:57:26.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.512 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:26.512 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143374+0000", 2024-09-14T23:57:26.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9540000000000003e-05 2024-09-14T23:57:26.512 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:26.512 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:26.512 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:26.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.512 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4057 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1216512~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:26.512 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099088+0000", 2024-09-14T23:57:26.512 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.397174516, 2024-09-14T23:57:26.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044406119000000001, 2024-09-14T23:57:26.513 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:26.513 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:26.513 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:26.513 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:26.513 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:26.513 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:26.513 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4057 2024-09-14T23:57:26.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.513 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:26.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:26.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099088+0000", 2024-09-14T23:57:26.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:26.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:26.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099088+0000", 2024-09-14T23:57:26.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:26.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:26.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099089+0000", 2024-09-14T23:57:26.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.04e-07 2024-09-14T23:57:26.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:26.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099090+0000", 2024-09-14T23:57:26.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.026e-06 2024-09-14T23:57:26.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:26.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099093+0000", 2024-09-14T23:57:26.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0819999999999999e-06 2024-09-14T23:57:26.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:26.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099095+0000", 2024-09-14T23:57:26.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4930000000000001e-06 2024-09-14T23:57:26.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.516 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:26.516 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143400+0000", 2024-09-14T23:57:26.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044304698000000003 2024-09-14T23:57:26.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.516 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:26.516 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143441+0000", 2024-09-14T23:57:26.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1100000000000003e-05 2024-09-14T23:57:26.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.516 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:26.516 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143495+0000", 2024-09-14T23:57:26.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3516000000000001e-05 2024-09-14T23:57:26.517 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:26.517 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:26.517 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:26.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.517 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4058 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1277952~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:26.517 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099099+0000", 2024-09-14T23:57:26.517 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.397164025000002, 2024-09-14T23:57:26.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044514583000000003, 2024-09-14T23:57:26.517 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:26.517 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:26.517 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:26.517 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:26.517 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:26.518 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:26.518 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4058 2024-09-14T23:57:26.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.518 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:26.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:26.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099099+0000", 2024-09-14T23:57:26.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:26.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:26.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099099+0000", 2024-09-14T23:57:26.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:26.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:26.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099099+0000", 2024-09-14T23:57:26.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9999999999999999e-07 2024-09-14T23:57:26.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:26.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099100+0000", 2024-09-14T23:57:26.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7199999999999997e-07 2024-09-14T23:57:26.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:26.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099103+0000", 2024-09-14T23:57:26.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.095e-06 2024-09-14T23:57:26.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:26.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099106+0000", 2024-09-14T23:57:26.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4229999999999999e-06 2024-09-14T23:57:26.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:26.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143524+0000", 2024-09-14T23:57:26.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044417891000000001 2024-09-14T23:57:26.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:26.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143568+0000", 2024-09-14T23:57:26.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4060000000000002e-05 2024-09-14T23:57:26.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.521 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.521 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:26.521 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143614+0000", 2024-09-14T23:57:26.521 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5942000000000002e-05 2024-09-14T23:57:26.521 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:26.521 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:26.521 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:26.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.521 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.521 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4059 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1339392~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:26.521 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099114+0000", 2024-09-14T23:57:26.521 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.397149486000004, 2024-09-14T23:57:26.521 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044617194999999998, 2024-09-14T23:57:26.521 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:26.522 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:26.522 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:26.522 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:26.522 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:26.522 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:26.522 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4059 2024-09-14T23:57:26.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.522 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:26.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:26.522 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099114+0000", 2024-09-14T23:57:26.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:26.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:26.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099114+0000", 2024-09-14T23:57:26.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:26.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:26.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099114+0000", 2024-09-14T23:57:26.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5899999999999998e-07 2024-09-14T23:57:26.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:26.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099115+0000", 2024-09-14T23:57:26.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3510000000000001e-06 2024-09-14T23:57:26.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:26.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099119+0000", 2024-09-14T23:57:26.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.16e-06 2024-09-14T23:57:26.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:26.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099123+0000", 2024-09-14T23:57:26.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1339999999999997e-06 2024-09-14T23:57:26.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:26.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143643+0000", 2024-09-14T23:57:26.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044519232999999998 2024-09-14T23:57:26.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.525 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:26.525 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143686+0000", 2024-09-14T23:57:26.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3349000000000001e-05 2024-09-14T23:57:26.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.525 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:26.525 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143731+0000", 2024-09-14T23:57:26.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4709000000000002e-05 2024-09-14T23:57:26.525 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:26.525 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:26.525 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:26.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.526 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4060 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1400832~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:26.526 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099129+0000", 2024-09-14T23:57:26.526 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.397134262999998, 2024-09-14T23:57:26.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044721271999999999, 2024-09-14T23:57:26.526 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:26.526 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:26.526 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:26.526 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:26.526 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:26.526 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:26.526 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4060 2024-09-14T23:57:26.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.526 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:26.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:26.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099129+0000", 2024-09-14T23:57:26.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:26.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:26.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099129+0000", 2024-09-14T23:57:26.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:26.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:26.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099191+0000", 2024-09-14T23:57:26.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1805999999999997e-05 2024-09-14T23:57:26.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:26.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099193+0000", 2024-09-14T23:57:26.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2330000000000001e-06 2024-09-14T23:57:26.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:26.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099200+0000", 2024-09-14T23:57:26.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4200000000000001e-06 2024-09-14T23:57:26.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:26.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099204+0000", 2024-09-14T23:57:26.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7500000000000001e-06 2024-09-14T23:57:26.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.529 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:26.529 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143760+0000", 2024-09-14T23:57:26.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044556182 2024-09-14T23:57:26.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.529 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:26.529 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143804+0000", 2024-09-14T23:57:26.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4239000000000001e-05 2024-09-14T23:57:26.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.529 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:26.529 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143850+0000", 2024-09-14T23:57:26.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5642000000000001e-05 2024-09-14T23:57:26.530 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:26.530 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:26.530 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:26.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.530 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4061 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1462272~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:26.530 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099215+0000", 2024-09-14T23:57:26.530 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.397048138000002, 2024-09-14T23:57:26.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044745659, 2024-09-14T23:57:26.530 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:26.530 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:26.530 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:26.530 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:26.531 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:26.531 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:26.531 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4061 2024-09-14T23:57:26.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.531 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:26.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:26.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099215+0000", 2024-09-14T23:57:26.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:26.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:26.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099215+0000", 2024-09-14T23:57:26.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:26.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:26.532 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100672+0000", 2024-09-14T23:57:26.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00145689 2024-09-14T23:57:26.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:26.532 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100677+0000", 2024-09-14T23:57:26.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0139999999999998e-06 2024-09-14T23:57:26.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:26.532 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100689+0000", 2024-09-14T23:57:26.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.183e-05 2024-09-14T23:57:26.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.533 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:26.533 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100696+0000", 2024-09-14T23:57:26.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2599999999999999e-06 2024-09-14T23:57:26.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.533 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:26.533 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143877+0000", 2024-09-14T23:57:26.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043181279000000003 2024-09-14T23:57:26.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.533 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:26.533 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143919+0000", 2024-09-14T23:57:26.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1783000000000003e-05 2024-09-14T23:57:26.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:26.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143961+0000", 2024-09-14T23:57:26.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1603000000000001e-05 2024-09-14T23:57:26.534 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:26.534 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:26.534 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:26.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.534 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4070 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 2015232~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:26.534 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.100903+0000", 2024-09-14T23:57:26.534 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.395360429999997, 2024-09-14T23:57:26.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044103815999999997, 2024-09-14T23:57:26.535 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:26.535 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:26.535 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:26.535 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:26.535 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:26.535 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:26.535 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4070 2024-09-14T23:57:26.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.535 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:26.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:26.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100903+0000", 2024-09-14T23:57:26.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:26.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:26.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100903+0000", 2024-09-14T23:57:26.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:26.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:26.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.102671+0000", 2024-09-14T23:57:26.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001768279 2024-09-14T23:57:26.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:26.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.102675+0000", 2024-09-14T23:57:26.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9609999999999999e-06 2024-09-14T23:57:26.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:26.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.102684+0000", 2024-09-14T23:57:26.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.329e-06 2024-09-14T23:57:26.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:26.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.102689+0000", 2024-09-14T23:57:26.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7079999999999996e-06 2024-09-14T23:57:26.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:26.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.144912+0000", 2024-09-14T23:57:26.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.042223305000000003 2024-09-14T23:57:26.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.538 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:26.538 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.144960+0000", 2024-09-14T23:57:26.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7670000000000003e-05 2024-09-14T23:57:26.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.538 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:26.538 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.145006+0000", 2024-09-14T23:57:26.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6563999999999997e-05 2024-09-14T23:57:26.538 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:26.538 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:26.538 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:26.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.539 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4096 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1282048~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:26.539 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.104957+0000", 2024-09-14T23:57:26.539 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.391306256999997, 2024-09-14T23:57:26.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044093016999999998, 2024-09-14T23:57:26.539 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:26.539 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:26.539 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:26.539 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:26.539 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:26.539 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:26.539 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4096 2024-09-14T23:57:26.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.539 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:26.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:26.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104957+0000", 2024-09-14T23:57:26.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:26.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:26.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104957+0000", 2024-09-14T23:57:26.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:26.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:26.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104957+0000", 2024-09-14T23:57:26.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7400000000000003e-07 2024-09-14T23:57:26.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:26.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104960+0000", 2024-09-14T23:57:26.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3800000000000001e-06 2024-09-14T23:57:26.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:26.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104967+0000", 2024-09-14T23:57:26.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4030000000000001e-06 2024-09-14T23:57:26.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:26.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104973+0000", 2024-09-14T23:57:26.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7370000000000003e-06 2024-09-14T23:57:26.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:26.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.148883+0000", 2024-09-14T23:57:26.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043910124000000002 2024-09-14T23:57:26.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:26.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.148966+0000", 2024-09-14T23:57:26.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2868999999999996e-05 2024-09-14T23:57:26.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:26.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149050+0000", 2024-09-14T23:57:26.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3930000000000004e-05 2024-09-14T23:57:26.542 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:26.542 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:26.543 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:26.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.543 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4097 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1343488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:26.543 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.104983+0000", 2024-09-14T23:57:26.543 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.391280467999998, 2024-09-14T23:57:26.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044182104999999999, 2024-09-14T23:57:26.543 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:26.543 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:26.543 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:26.543 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:26.543 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:26.543 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:26.543 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4097 2024-09-14T23:57:26.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.544 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:26.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:26.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104983+0000", 2024-09-14T23:57:26.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:26.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:26.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104983+0000", 2024-09-14T23:57:26.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:26.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:26.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105207+0000", 2024-09-14T23:57:26.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000224687 2024-09-14T23:57:26.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:26.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105211+0000", 2024-09-14T23:57:26.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3050000000000001e-06 2024-09-14T23:57:26.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:26.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105217+0000", 2024-09-14T23:57:26.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8170000000000001e-06 2024-09-14T23:57:26.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:26.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105222+0000", 2024-09-14T23:57:26.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.267e-06 2024-09-14T23:57:26.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:26.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149079+0000", 2024-09-14T23:57:26.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043857581999999999 2024-09-14T23:57:26.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:26.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149122+0000", 2024-09-14T23:57:26.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2899999999999999e-05 2024-09-14T23:57:26.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:26.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149165+0000", 2024-09-14T23:57:26.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2546999999999998e-05 2024-09-14T23:57:26.547 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:26.547 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:26.547 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:26.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.547 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4098 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1404928~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:26.547 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.105242+0000", 2024-09-14T23:57:26.547 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.391020656000002, 2024-09-14T23:57:26.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044121104000000001, 2024-09-14T23:57:26.547 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:26.548 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:26.548 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:26.548 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:26.548 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:26.548 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:26.548 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4098 2024-09-14T23:57:26.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.548 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:26.548 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.548 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:26.548 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105242+0000", 2024-09-14T23:57:26.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:26.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:26.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105242+0000", 2024-09-14T23:57:26.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:26.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:26.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105243+0000", 2024-09-14T23:57:26.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0399999999999995e-07 2024-09-14T23:57:26.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:26.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105246+0000", 2024-09-14T23:57:26.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6309999999999999e-06 2024-09-14T23:57:26.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:26.550 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105253+0000", 2024-09-14T23:57:26.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0940000000000002e-06 2024-09-14T23:57:26.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:26.550 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105260+0000", 2024-09-14T23:57:26.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1799999999999999e-06 2024-09-14T23:57:26.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:26.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149201+0000", 2024-09-14T23:57:26.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043940629000000002 2024-09-14T23:57:26.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:26.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149283+0000", 2024-09-14T23:57:26.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2360000000000004e-05 2024-09-14T23:57:26.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:26.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149363+0000", 2024-09-14T23:57:26.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0506000000000005e-05 2024-09-14T23:57:26.551 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:26.551 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:26.552 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:26.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.552 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.552 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4099 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1466368~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:26.552 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.105270+0000", 2024-09-14T23:57:26.552 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.390993061000003, 2024-09-14T23:57:26.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044195433999999999, 2024-09-14T23:57:26.552 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:26.552 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:26.552 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:26.552 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:26.552 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:26.552 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:26.552 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4099 2024-09-14T23:57:26.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.553 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:26.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.553 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:26.553 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105270+0000", 2024-09-14T23:57:26.553 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:26.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.553 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:26.553 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105270+0000", 2024-09-14T23:57:26.553 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:26.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:26.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105429+0000", 2024-09-14T23:57:26.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015940299999999999 2024-09-14T23:57:26.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:26.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105433+0000", 2024-09-14T23:57:26.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3579999999999999e-06 2024-09-14T23:57:26.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:26.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105439+0000", 2024-09-14T23:57:26.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1759999999999998e-06 2024-09-14T23:57:26.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:26.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105445+0000", 2024-09-14T23:57:26.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9689999999999999e-06 2024-09-14T23:57:26.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:26.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149381+0000", 2024-09-14T23:57:26.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043936389999999999 2024-09-14T23:57:26.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:26.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149422+0000", 2024-09-14T23:57:26.556 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0865000000000003e-05 2024-09-14T23:57:26.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.556 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.556 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:26.556 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149465+0000", 2024-09-14T23:57:26.556 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3272999999999997e-05 2024-09-14T23:57:26.556 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:26.556 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:26.556 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:26.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.556 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.556 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4100 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1527808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:26.556 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.105454+0000", 2024-09-14T23:57:26.557 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.390808589999999, 2024-09-14T23:57:26.557 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044217411999999998, 2024-09-14T23:57:26.557 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:26.557 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:26.557 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:26.557 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:26.557 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:26.557 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:26.557 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4100 2024-09-14T23:57:26.557 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.557 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:26.557 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.557 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:26.557 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105454+0000", 2024-09-14T23:57:26.558 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:26.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.558 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.558 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:26.558 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105454+0000", 2024-09-14T23:57:26.558 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:26.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.558 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.558 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:26.558 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105578+0000", 2024-09-14T23:57:26.558 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012401799999999999 2024-09-14T23:57:26.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.558 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.558 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:26.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105583+0000", 2024-09-14T23:57:26.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1779999999999996e-06 2024-09-14T23:57:26.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:26.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105590+0000", 2024-09-14T23:57:26.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4279999999999999e-06 2024-09-14T23:57:26.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:26.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105596+0000", 2024-09-14T23:57:26.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0120000000000002e-06 2024-09-14T23:57:26.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:26.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149503+0000", 2024-09-14T23:57:26.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043906580000000001 2024-09-14T23:57:26.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:26.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149585+0000", 2024-09-14T23:57:26.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2356999999999998e-05 2024-09-14T23:57:26.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:26.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149672+0000", 2024-09-14T23:57:26.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6839e-05 2024-09-14T23:57:26.560 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:26.560 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:26.561 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:26.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.561 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4101 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1589248~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:26.561 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.105609+0000", 2024-09-14T23:57:26.561 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.390654050999999, 2024-09-14T23:57:26.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044175987999999999, 2024-09-14T23:57:26.561 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:26.561 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:26.561 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:26.561 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:26.561 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:26.561 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:26.561 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4101 2024-09-14T23:57:26.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.562 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:26.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:26.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105609+0000", 2024-09-14T23:57:26.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:26.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:26.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105609+0000", 2024-09-14T23:57:26.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:26.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:26.563 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.106671+0000", 2024-09-14T23:57:26.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010619270000000001 2024-09-14T23:57:26.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.563 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:26.563 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.106675+0000", 2024-09-14T23:57:26.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4429999999999996e-06 2024-09-14T23:57:26.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.563 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:26.563 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.106685+0000", 2024-09-14T23:57:26.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7429999999999997e-06 2024-09-14T23:57:26.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:26.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.106690+0000", 2024-09-14T23:57:26.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.293e-06 2024-09-14T23:57:26.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:26.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149700+0000", 2024-09-14T23:57:26.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043009512 2024-09-14T23:57:26.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:26.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149742+0000", 2024-09-14T23:57:26.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2083999999999999e-05 2024-09-14T23:57:26.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.565 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:26.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149785+0000", 2024-09-14T23:57:26.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2985999999999998e-05 2024-09-14T23:57:26.565 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:26.565 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:26.565 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:26.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.565 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4116 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 180224~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:26.565 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.107575+0000", 2024-09-14T23:57:26.565 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.388687582999999, 2024-09-14T23:57:26.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044157522999999997, 2024-09-14T23:57:26.566 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:26.566 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:26.566 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:26.566 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:26.566 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:26.566 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:26.566 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4116 2024-09-14T23:57:26.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.566 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:26.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:26.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.107575+0000", 2024-09-14T23:57:26.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:26.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.567 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:26.567 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.107575+0000", 2024-09-14T23:57:26.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:26.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.567 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:26.567 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108671+0000", 2024-09-14T23:57:26.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010954510000000001 2024-09-14T23:57:26.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.567 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:26.567 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108676+0000", 2024-09-14T23:57:26.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6639999999999997e-06 2024-09-14T23:57:26.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.568 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:26.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108685+0000", 2024-09-14T23:57:26.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1130000000000007e-06 2024-09-14T23:57:26.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.568 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:26.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108691+0000", 2024-09-14T23:57:26.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6019999999999996e-06 2024-09-14T23:57:26.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.568 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:26.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.151530+0000", 2024-09-14T23:57:26.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.042838345999999999 2024-09-14T23:57:26.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.569 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:26.569 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.151619+0000", 2024-09-14T23:57:26.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9302999999999995e-05 2024-09-14T23:57:26.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.569 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:26.569 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.151733+0000", 2024-09-14T23:57:26.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011404399999999999 2024-09-14T23:57:26.569 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:26.569 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:26.569 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:26.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.570 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4122 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 548864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:26.570 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.108842+0000", 2024-09-14T23:57:26.570 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.387420745, 2024-09-14T23:57:26.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044103631999999997, 2024-09-14T23:57:26.570 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:26.570 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:26.570 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:26.570 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:26.570 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:26.570 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:26.570 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4122 2024-09-14T23:57:26.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.571 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:26.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:26.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108842+0000", 2024-09-14T23:57:26.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:26.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:26.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108842+0000", 2024-09-14T23:57:26.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:26.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:26.572 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108843+0000", 2024-09-14T23:57:26.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7000000000000001e-07 2024-09-14T23:57:26.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:26.572 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108844+0000", 2024-09-14T23:57:26.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6479999999999999e-06 2024-09-14T23:57:26.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:26.572 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108848+0000", 2024-09-14T23:57:26.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8029999999999998e-06 2024-09-14T23:57:26.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.573 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:26.573 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108851+0000", 2024-09-14T23:57:26.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7939999999999998e-06 2024-09-14T23:57:26.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.573 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:26.573 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.152838+0000", 2024-09-14T23:57:26.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043987351000000001 2024-09-14T23:57:26.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.573 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:26.573 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.152885+0000", 2024-09-14T23:57:26.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7237000000000002e-05 2024-09-14T23:57:26.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.574 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:26.574 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.152946+0000", 2024-09-14T23:57:26.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0529000000000003e-05 2024-09-14T23:57:26.574 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:26.574 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:26.574 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:26.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.574 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4123 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 610304~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:26.574 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.108855+0000", 2024-09-14T23:57:26.574 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.387408416, 2024-09-14T23:57:26.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044321610999999997, 2024-09-14T23:57:26.574 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:26.575 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:26.575 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:26.575 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:26.575 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:26.575 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:26.575 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4123 2024-09-14T23:57:26.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.575 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:26.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:26.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108855+0000", 2024-09-14T23:57:26.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:26.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:26.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108855+0000", 2024-09-14T23:57:26.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:26.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:26.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.110671+0000", 2024-09-14T23:57:26.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018159719999999999 2024-09-14T23:57:26.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:26.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.110676+0000", 2024-09-14T23:57:26.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9450000000000001e-06 2024-09-14T23:57:26.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.577 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:26.577 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.110685+0000", 2024-09-14T23:57:26.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5359999999999998e-06 2024-09-14T23:57:26.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.577 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:26.577 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.110692+0000", 2024-09-14T23:57:26.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9280000000000004e-06 2024-09-14T23:57:26.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.577 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:26.577 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.152984+0000", 2024-09-14T23:57:26.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.042291677999999999 2024-09-14T23:57:26.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:26.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.153066+0000", 2024-09-14T23:57:26.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2657999999999994e-05 2024-09-14T23:57:26.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:26.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.153176+0000", 2024-09-14T23:57:26.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000109894 2024-09-14T23:57:26.578 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:26.578 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:26.578 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:26.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.579 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4136 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:26.579 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.111056+0000", 2024-09-14T23:57:26.579 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.385207268000002, 2024-09-14T23:57:26.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044097428000000001, 2024-09-14T23:57:26.579 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:26.579 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:26.579 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:26.579 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:26.579 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:26.579 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:26.579 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4136 2024-09-14T23:57:26.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.579 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:26.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:26.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.111056+0000", 2024-09-14T23:57:26.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:26.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:26.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.111056+0000", 2024-09-14T23:57:26.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:26.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:26.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.112671+0000", 2024-09-14T23:57:26.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016153999999999999 2024-09-14T23:57:26.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:26.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.112675+0000", 2024-09-14T23:57:26.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2830000000000003e-06 2024-09-14T23:57:26.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:26.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.112684+0000", 2024-09-14T23:57:26.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7800000000000006e-06 2024-09-14T23:57:26.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:26.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.112689+0000", 2024-09-14T23:57:26.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0000000000000004e-06 2024-09-14T23:57:26.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:26.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.155064+0000", 2024-09-14T23:57:26.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.042374423000000001 2024-09-14T23:57:26.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:26.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.155111+0000", 2024-09-14T23:57:26.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7367000000000002e-05 2024-09-14T23:57:26.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:26.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.155153+0000", 2024-09-14T23:57:26.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2175000000000002e-05 2024-09-14T23:57:26.583 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:26.583 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:26.583 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:26.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.583 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:46572 3.1d 3:b9e1f524:::rbd_header.1118cee06433:head [watch reconnect cookie 140670782499824 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-14T23:57:26.583 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:57.046868+0000", 2024-09-14T23:57:26.583 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 29.449395461999998, 2024-09-14T23:57:26.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0243412620000001, 2024-09-14T23:57:26.583 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:26.583 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:26.583 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:26.584 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:26.584 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:26.584 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:26.584 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 46572 2024-09-14T23:57:26.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.584 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:26.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.584 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:26.584 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046868+0000", 2024-09-14T23:57:26.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:26.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.584 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:26.584 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046868+0000", 2024-09-14T23:57:26.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:26.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:26.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046870+0000", 2024-09-14T23:57:26.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.204e-06 2024-09-14T23:57:26.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:26.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046877+0000", 2024-09-14T23:57:26.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5100000000000001e-06 2024-09-14T23:57:26.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:26.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046898+0000", 2024-09-14T23:57:26.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1149999999999999e-05 2024-09-14T23:57:26.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:26.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046911+0000", 2024-09-14T23:57:26.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2234e-05 2024-09-14T23:57:26.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:26.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046988+0000", 2024-09-14T23:57:26.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7445000000000001e-05 2024-09-14T23:57:26.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:57:26.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.047001+0000", 2024-09-14T23:57:26.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2951e-05 2024-09-14T23:57:26.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:26.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.069287+0000", 2024-09-14T23:57:26.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.022285748 2024-09-14T23:57:26.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for rw locks", 2024-09-14T23:57:26.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.069397+0000", 2024-09-14T23:57:26.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000110326 2024-09-14T23:57:26.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:26.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.069997+0000", 2024-09-14T23:57:26.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00059984399999999998 2024-09-14T23:57:26.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:26.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.070109+0000", 2024-09-14T23:57:26.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000112367 2024-09-14T23:57:26.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:26.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:26.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:26.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.071209+0000", 2024-09-14T23:57:26.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010994830000000001 2024-09-14T23:57:26.589 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:26.589 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:26.589 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:26.589 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:26.589 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:26.589 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:26.590 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:57:26.590 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:57:26.591 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:57:26.591 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:57:26.591 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T23:57:26.592 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:26.589+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:26.592 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:57:26.592 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:57:26.593 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:57:26.593 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:57:26.593 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:57:26.593 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:57:26.600 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:57:26.600 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:57:26.616 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:57:26.617 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:57:26.633 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:57:26.634 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:57:26.691 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:26.689+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:26.739 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:26.739 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:26.748 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T23:57:26.792 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:26.790+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:26.855 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:26.855 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:26.864 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T23:57:26.907 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:26.906+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:26.973 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:26.973 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:26.981 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T23:57:26.983 INFO:teuthology.orchestra.run.smithi042.stdout:2745 read 0x6d8a77 thru 0x6e258c (0x9b16 bytes) 2024-09-14T23:57:26.984 INFO:teuthology.orchestra.run.smithi042.stdout:2746 read 0xd43d88 thru 0xd4fb0d (0xbd86 bytes) 2024-09-14T23:57:26.986 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T23:57:26.993 INFO:teuthology.orchestra.run.smithi042.stdout:2747 write 0x4752881 thru 0x475c796 (0x9f16 bytes) 2024-09-14T23:57:26.994 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:26.991+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:26.997 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:27.006 INFO:teuthology.orchestra.run.smithi042.stdout:2750 write 0x78e64aa thru 0x78f5569 (0xf0c0 bytes) 2024-09-14T23:57:27.094 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:27.093+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:27.134 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:27.134 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:27.142 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T23:57:27.196 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:27.194+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:27.289 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:27.289 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:27.297 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:27.295+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:27.298 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T23:57:27.400 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:27.399+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:27.445 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:27.445 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:27.454 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T23:57:27.538 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:27.537+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:27.602 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:27.602 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:27.603 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:27.601+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:27.610 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T23:57:27.703 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:27.701+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:27.757 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:27.757 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:27.766 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T23:57:27.809 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:27.808+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:27.873 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:27.873 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:27.881 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T23:57:27.926 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:27.925+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:27.969 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is unset 2024-09-14T23:57:27.977 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:27.990 INFO:teuthology.orchestra.run.smithi042.stdout:2753 write 0xabc8803 thru 0xabd401d (0xb81b bytes) 2024-09-14T23:57:28.000 INFO:teuthology.orchestra.run.smithi042.stdout:2754 write 0xb707c10 thru 0xb717233 (0xf624 bytes) 2024-09-14T23:57:28.004 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:28.003+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:28.008 INFO:teuthology.orchestra.run.smithi042.stdout:2755 write 0xbc4b870 thru 0xbc57dc4 (0xc555 bytes) 2024-09-14T23:57:28.011 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:28.011 INFO:teuthology.orchestra.run.smithi042.stdout:2758 read 0xa643ad thru 0xa7431f (0xff73 bytes) 2024-09-14T23:57:28.012 INFO:teuthology.orchestra.run.smithi042.stdout:2759 read 0x7d93163 thru 0x7d94fcf (0x1e6d bytes) 2024-09-14T23:57:28.012 INFO:teuthology.orchestra.run.smithi042.stdout:2761 write 0x1abc6bf thru 0x1acaf20 (0xe862 bytes) 2024-09-14T23:57:28.013 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:28.013 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:28.013 INFO:teuthology.orchestra.run.smithi042.stdout:2762 write 0xaac7f5 thru 0xab7df1 (0xb5fd bytes) 2024-09-14T23:57:28.023 INFO:teuthology.orchestra.run.smithi042.stdout:2763 trunc from 0xbc57dc5 to 0x5c598c6 2024-09-14T23:57:28.024 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T23:57:28.033 INFO:teuthology.orchestra.run.smithi042.stdout:2764 write 0x1f4b6a3 thru 0x1f52d8a (0x76e8 bytes) 2024-09-14T23:57:28.036 INFO:teuthology.orchestra.run.smithi042.stdout:2765 write 0x19a132e thru 0x19a305d (0x1d30 bytes) 2024-09-14T23:57:28.037 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:28.037 INFO:teuthology.orchestra.run.smithi042.stdout:2768 write 0x3ea5574 thru 0x3ea8cdc (0x3769 bytes) 2024-09-14T23:57:28.040 INFO:teuthology.orchestra.run.smithi042.stdout:2769 read 0x30ae425 thru 0x30bb709 (0xd2e5 bytes) 2024-09-14T23:57:28.047 INFO:teuthology.orchestra.run.smithi042.stdout:2770 write 0x6d4fc1e thru 0x6d52835 (0x2c18 bytes) 2024-09-14T23:57:28.050 INFO:teuthology.orchestra.run.smithi042.stdout:2771 read 0x6587f80 thru 0x658aa03 (0x2a84 bytes) 2024-09-14T23:57:28.050 INFO:teuthology.orchestra.run.smithi042.stdout:2772 read 0x647a214 thru 0x64821b4 (0x7fa1 bytes) 2024-09-14T23:57:28.051 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:28.051 INFO:teuthology.orchestra.run.smithi042.stdout:2775 read 0x4c03ae4 thru 0x4c12e34 (0xf351 bytes) 2024-09-14T23:57:28.051 INFO:teuthology.orchestra.run.smithi042.stdout:2776 write 0x30f16d7 thru 0x30fb8fe (0xa228 bytes) 2024-09-14T23:57:28.054 INFO:teuthology.orchestra.run.smithi042.stdout:2778 write 0x6a0e8cf thru 0x6a16c70 (0x83a2 bytes) 2024-09-14T23:57:28.057 INFO:teuthology.orchestra.run.smithi042.stdout:2779 write 0x353fc3a thru 0x3542d41 (0x3108 bytes) 2024-09-14T23:57:28.062 INFO:teuthology.orchestra.run.smithi042.stdout:2780 trunc from 0x6d52836 to 0x4a57184 2024-09-14T23:57:28.073 INFO:teuthology.orchestra.run.smithi042.stdout:2781 read 0x484dec6 thru 0x485c079 (0xe1b4 bytes) 2024-09-14T23:57:28.075 INFO:teuthology.orchestra.run.smithi042.stdout:2782 trunc from 0x4a57184 to 0x6b8c140 2024-09-14T23:57:28.080 INFO:teuthology.orchestra.run.smithi042.stdout:2783 clone 26 order 24 su 1048576 sc 10 2024-09-14T23:57:28.106 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:28.105+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:28.145 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:28.145 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:28.154 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T23:57:28.238 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:28.236+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:28.301 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:28.301 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:28.307 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:28.306+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:28.310 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T23:57:28.408 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:28.407+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:28.457 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:28.457 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:28.467 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T23:57:28.509 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:28.508+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:28.573 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:28.573 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:28.582 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T23:57:28.628 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:28.624+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:28.645 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T23:57:28.690 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:28.691 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:28.700 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T23:57:28.743 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:28.741+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:28.770 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:57:28.770 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:57:28.788 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:57:28.788 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:57:28.812 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:28.809+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:28.821 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:57:28.821 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:57:28.829 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:28.829 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:57:28.829 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:57:28.829 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:28.842 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T23:57:28.847 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:57:28.848 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:57:28.865 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:57:28.865 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:57:28.885 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:57:28.885 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:57:28.902 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:57:28.902 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:57:28.913 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:28.912+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:28.919 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:57:28.919 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:57:28.957 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:28.957 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:57:28.957 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:57:28.957 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:57:28.957 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:28.965 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T23:57:28.980 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T23:57:28.980 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T23:57:28.980 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-14T23:57:28.981 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-14T23:57:28.981 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-14T23:57:28.981 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-14T23:57:29.014 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:29.013+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:29.048 INFO:teuthology.orchestra.run.smithi042.stdout:flattening image image_client.0-clone25 2024-09-14T23:57:29.115 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:29.113+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:29.121 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:29.121 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-14T23:57:29.121 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:29.122 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:29.123 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:57:29.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:57:29.123 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:57:29.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.123 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:3578 3.4 e29/23 3:2298d860:::rbd_data.1118de1168b.0000000000000030:head v 29'15, mlcod=29'13)", 2024-09-14T23:57:29.123 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.036455+0000", 2024-09-14T23:57:29.123 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 50.083697022999999, 2024-09-14T23:57:29.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019449440000000001, 2024-09-14T23:57:29.123 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:29.123 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:29.123 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:29.123 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:29.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:29.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036455+0000", 2024-09-14T23:57:29.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:29.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036455+0000", 2024-09-14T23:57:29.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:29.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036456+0000", 2024-09-14T23:57:29.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0999999999999997e-07 2024-09-14T23:57:29.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:29.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036463+0000", 2024-09-14T23:57:29.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2200000000000003e-06 2024-09-14T23:57:29.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:29.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036478+0000", 2024-09-14T23:57:29.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4977000000000001e-05 2024-09-14T23:57:29.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:29.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036485+0000", 2024-09-14T23:57:29.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0559999999999999e-06 2024-09-14T23:57:29.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:29.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036693+0000", 2024-09-14T23:57:29.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000208051 2024-09-14T23:57:29.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:29.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036715+0000", 2024-09-14T23:57:29.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1330000000000001e-05 2024-09-14T23:57:29.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:29.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.038356+0000", 2024-09-14T23:57:29.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001641586 2024-09-14T23:57:29.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:29.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.038400+0000", 2024-09-14T23:57:29.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3914e-05 2024-09-14T23:57:29.127 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.127 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:29.127 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.127 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:3579 3.4 e29/23 3:27290fe6:::rbd_data.1118de1168b.0000000000000032:head v 29'16, mlcod=29'13)", 2024-09-14T23:57:29.127 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.037085+0000", 2024-09-14T23:57:29.127 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 50.083067432999997, 2024-09-14T23:57:29.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0020019220000000002, 2024-09-14T23:57:29.128 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:29.128 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:29.128 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:29.128 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:29.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:29.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037085+0000", 2024-09-14T23:57:29.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:29.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037085+0000", 2024-09-14T23:57:29.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:29.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037085+0000", 2024-09-14T23:57:29.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5499999999999997e-07 2024-09-14T23:57:29.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:29.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037091+0000", 2024-09-14T23:57:29.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7869999999999998e-06 2024-09-14T23:57:29.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:29.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037102+0000", 2024-09-14T23:57:29.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.031e-05 2024-09-14T23:57:29.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:29.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037116+0000", 2024-09-14T23:57:29.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4119e-05 2024-09-14T23:57:29.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:29.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037219+0000", 2024-09-14T23:57:29.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010372799999999999 2024-09-14T23:57:29.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:29.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037239+0000", 2024-09-14T23:57:29.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9993e-05 2024-09-14T23:57:29.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:29.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.039046+0000", 2024-09-14T23:57:29.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001806666 2024-09-14T23:57:29.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:29.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.039087+0000", 2024-09-14T23:57:29.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0564e-05 2024-09-14T23:57:29.132 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.132 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:29.132 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.132 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:3584 3.4 e29/23 3:2779453d:::rbd_data.1118de1168b.0000000000000034:head v 29'17, mlcod=29'13)", 2024-09-14T23:57:29.132 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.037588+0000", 2024-09-14T23:57:29.132 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 50.082564394000002, 2024-09-14T23:57:29.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001526828, 2024-09-14T23:57:29.133 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:29.133 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:29.133 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:29.133 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:29.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:29.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037588+0000", 2024-09-14T23:57:29.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:29.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037588+0000", 2024-09-14T23:57:29.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:29.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037588+0000", 2024-09-14T23:57:29.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1200000000000002e-07 2024-09-14T23:57:29.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:29.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037595+0000", 2024-09-14T23:57:29.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7580000000000003e-06 2024-09-14T23:57:29.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:29.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037609+0000", 2024-09-14T23:57:29.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4037e-05 2024-09-14T23:57:29.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:29.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037616+0000", 2024-09-14T23:57:29.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0509999999999999e-06 2024-09-14T23:57:29.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:29.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037787+0000", 2024-09-14T23:57:29.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017084899999999999 2024-09-14T23:57:29.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:29.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037806+0000", 2024-09-14T23:57:29.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9159000000000001e-05 2024-09-14T23:57:29.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:29.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.039089+0000", 2024-09-14T23:57:29.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012823909999999999 2024-09-14T23:57:29.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:29.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.039115+0000", 2024-09-14T23:57:29.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5871e-05 2024-09-14T23:57:29.136 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.136 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:29.136 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.137 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:18030 3.4 e33/23 3:225c4d41:::rbd_data.11187be34cc8.0000000000000001:head v 33'63, mlcod=33'61)", 2024-09-14T23:57:29.137 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:43.852748+0000", 2024-09-14T23:57:29.137 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.267404335000002, 2024-09-14T23:57:29.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001530696, 2024-09-14T23:57:29.137 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:29.137 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:29.137 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:29.137 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:29.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:29.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.852748+0000", 2024-09-14T23:57:29.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:29.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.852748+0000", 2024-09-14T23:57:29.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:29.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.852749+0000", 2024-09-14T23:57:29.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.164e-06 2024-09-14T23:57:29.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:29.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.852759+0000", 2024-09-14T23:57:29.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0265000000000001e-05 2024-09-14T23:57:29.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:29.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.852779+0000", 2024-09-14T23:57:29.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9247999999999999e-05 2024-09-14T23:57:29.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:29.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.852788+0000", 2024-09-14T23:57:29.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3260000000000002e-06 2024-09-14T23:57:29.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:29.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.853121+0000", 2024-09-14T23:57:29.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00033336699999999999 2024-09-14T23:57:29.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:29.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.853142+0000", 2024-09-14T23:57:29.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0500999999999999e-05 2024-09-14T23:57:29.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:29.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.854240+0000", 2024-09-14T23:57:29.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001098665 2024-09-14T23:57:29.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:29.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:43.854279+0000", 2024-09-14T23:57:29.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8160000000000001e-05 2024-09-14T23:57:29.141 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.141 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:29.141 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.141 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:32024 3.9 e39/23 3:94dc260b:::rbd_data.111814b482f8.0000000000000009:head v 39'45, mlcod=38'43)", 2024-09-14T23:57:29.141 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:49.061018+0000", 2024-09-14T23:57:29.141 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 40.059134094000001, 2024-09-14T23:57:29.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018482240000000001, 2024-09-14T23:57:29.141 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:29.141 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:29.141 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:29.142 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:29.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:29.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061018+0000", 2024-09-14T23:57:29.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:29.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061018+0000", 2024-09-14T23:57:29.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:29.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061018+0000", 2024-09-14T23:57:29.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9999999999999998e-07 2024-09-14T23:57:29.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:29.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061211+0000", 2024-09-14T23:57:29.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019206 2024-09-14T23:57:29.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:29.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061221+0000", 2024-09-14T23:57:29.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0659000000000001e-05 2024-09-14T23:57:29.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:29.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061230+0000", 2024-09-14T23:57:29.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4610000000000008e-06 2024-09-14T23:57:29.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:29.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061317+0000", 2024-09-14T23:57:29.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7514000000000003e-05 2024-09-14T23:57:29.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:29.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061338+0000", 2024-09-14T23:57:29.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0395999999999999e-05 2024-09-14T23:57:29.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:29.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.062830+0000", 2024-09-14T23:57:29.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0014921940000000001 2024-09-14T23:57:29.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:29.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.062866+0000", 2024-09-14T23:57:29.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6539999999999999e-05 2024-09-14T23:57:29.146 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.146 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:29.146 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.146 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:32420 3.9 e39/23 3:91dc34e7:::rbd_data.111814b482f8.000000000000001e:head v 39'46, mlcod=38'44)", 2024-09-14T23:57:29.146 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:49.086124+0000", 2024-09-14T23:57:29.146 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 40.034028595000002, 2024-09-14T23:57:29.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001686819, 2024-09-14T23:57:29.146 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:29.146 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:29.146 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:29.146 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:29.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:29.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086124+0000", 2024-09-14T23:57:29.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:29.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086124+0000", 2024-09-14T23:57:29.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:29.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086125+0000", 2024-09-14T23:57:29.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.361e-06 2024-09-14T23:57:29.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:29.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086605+0000", 2024-09-14T23:57:29.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00048026600000000003 2024-09-14T23:57:29.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:29.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086626+0000", 2024-09-14T23:57:29.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.037e-05 2024-09-14T23:57:29.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:29.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086635+0000", 2024-09-14T23:57:29.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5880000000000004e-06 2024-09-14T23:57:29.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:29.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086701+0000", 2024-09-14T23:57:29.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5905e-05 2024-09-14T23:57:29.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:29.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086727+0000", 2024-09-14T23:57:29.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5573000000000001e-05 2024-09-14T23:57:29.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:29.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.087767+0000", 2024-09-14T23:57:29.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001040537 2024-09-14T23:57:29.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:29.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.087810+0000", 2024-09-14T23:57:29.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3219000000000001e-05 2024-09-14T23:57:29.150 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.150 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:29.150 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.150 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:34296 3.9 e39/23 3:974a38c7:::rbd_data.111814b482f8.000000000000003a:head v 39'47, mlcod=39'45)", 2024-09-14T23:57:29.150 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:49.192105+0000", 2024-09-14T23:57:29.150 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 39.928046711999997, 2024-09-14T23:57:29.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001607516, 2024-09-14T23:57:29.150 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:29.150 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:29.151 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:29.151 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:29.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:29.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.192105+0000", 2024-09-14T23:57:29.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:29.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.192105+0000", 2024-09-14T23:57:29.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:29.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.192107+0000", 2024-09-14T23:57:29.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3549999999999999e-06 2024-09-14T23:57:29.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.152 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:29.152 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.192332+0000", 2024-09-14T23:57:29.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022486300000000001 2024-09-14T23:57:29.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.152 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:29.152 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.192349+0000", 2024-09-14T23:57:29.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6966e-05 2024-09-14T23:57:29.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:29.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.192358+0000", 2024-09-14T23:57:29.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8309999999999996e-06 2024-09-14T23:57:29.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:29.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.192422+0000", 2024-09-14T23:57:29.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3915e-05 2024-09-14T23:57:29.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:29.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.192450+0000", 2024-09-14T23:57:29.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7957e-05 2024-09-14T23:57:29.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:29.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.193662+0000", 2024-09-14T23:57:29.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012113270000000001 2024-09-14T23:57:29.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:29.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.193713+0000", 2024-09-14T23:57:29.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1301999999999998e-05 2024-09-14T23:57:29.154 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.154 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:29.155 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.155 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:50582 3.9 e51/23 3:92198e7a:::rbd_data.11183b9aac34.000000000000000a:head v 51'63, mlcod=50'60)", 2024-09-14T23:57:29.155 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:01.131541+0000", 2024-09-14T23:57:29.155 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.988611404, 2024-09-14T23:57:29.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001795547, 2024-09-14T23:57:29.155 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:29.155 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:29.155 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:29.155 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:29.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:29.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131541+0000", 2024-09-14T23:57:29.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.156 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:29.156 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131541+0000", 2024-09-14T23:57:29.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.156 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:29.156 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131541+0000", 2024-09-14T23:57:29.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7099999999999997e-07 2024-09-14T23:57:29.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.156 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:29.156 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131548+0000", 2024-09-14T23:57:29.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.708e-06 2024-09-14T23:57:29.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.157 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:29.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131570+0000", 2024-09-14T23:57:29.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1815999999999999e-05 2024-09-14T23:57:29.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.157 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:29.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131581+0000", 2024-09-14T23:57:29.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0933e-05 2024-09-14T23:57:29.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.157 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:29.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131660+0000", 2024-09-14T23:57:29.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9071000000000002e-05 2024-09-14T23:57:29.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:29.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131682+0000", 2024-09-14T23:57:29.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2534000000000001e-05 2024-09-14T23:57:29.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:29.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.133287+0000", 2024-09-14T23:57:29.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016045510000000001 2024-09-14T23:57:29.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:29.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.133336+0000", 2024-09-14T23:57:29.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9563e-05 2024-09-14T23:57:29.159 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.159 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:29.159 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.159 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:51182 3.9 e52/23 3:918e4fa0:::rbd_data.11181975a1de.000000000000007b:head v 52'74, mlcod=51'71)", 2024-09-14T23:57:29.159 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:02.138725+0000", 2024-09-14T23:57:29.159 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.981427666999998, 2024-09-14T23:57:29.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015321009999999999, 2024-09-14T23:57:29.159 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:29.160 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:29.160 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:29.160 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:29.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:29.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.138725+0000", 2024-09-14T23:57:29.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:29.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.138725+0000", 2024-09-14T23:57:29.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.161 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:29.161 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.138725+0000", 2024-09-14T23:57:29.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.22e-07 2024-09-14T23:57:29.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.161 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:29.161 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.138732+0000", 2024-09-14T23:57:29.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1779999999999997e-06 2024-09-14T23:57:29.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.161 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:29.161 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.138749+0000", 2024-09-14T23:57:29.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6500000000000001e-05 2024-09-14T23:57:29.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:29.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.138758+0000", 2024-09-14T23:57:29.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6239999999999997e-06 2024-09-14T23:57:29.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:29.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.138816+0000", 2024-09-14T23:57:29.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7939000000000002e-05 2024-09-14T23:57:29.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:29.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.138840+0000", 2024-09-14T23:57:29.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3856e-05 2024-09-14T23:57:29.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:29.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.140204+0000", 2024-09-14T23:57:29.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013641339999999999 2024-09-14T23:57:29.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:29.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.140257+0000", 2024-09-14T23:57:29.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2547999999999999e-05 2024-09-14T23:57:29.163 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.163 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:29.163 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.163 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:51281 3.0 e52/23 3:04e60ec2:::rbd_data.11181975a1de.0000000000000003:head v 52'113, mlcod=44'111)", 2024-09-14T23:57:29.164 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:02.158822+0000", 2024-09-14T23:57:29.164 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.961329908, 2024-09-14T23:57:29.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019860609999999999, 2024-09-14T23:57:29.164 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:29.164 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:29.164 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:29.164 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:29.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.164 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:29.164 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.158822+0000", 2024-09-14T23:57:29.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.165 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:29.165 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.158822+0000", 2024-09-14T23:57:29.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.165 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.165 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:29.165 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.158823+0000", 2024-09-14T23:57:29.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6800000000000001e-07 2024-09-14T23:57:29.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.165 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.165 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:29.165 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.159132+0000", 2024-09-14T23:57:29.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030974900000000001 2024-09-14T23:57:29.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:29.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.159146+0000", 2024-09-14T23:57:29.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3261000000000001e-05 2024-09-14T23:57:29.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:29.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.159155+0000", 2024-09-14T23:57:29.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7410000000000004e-06 2024-09-14T23:57:29.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:29.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.159221+0000", 2024-09-14T23:57:29.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5698999999999996e-05 2024-09-14T23:57:29.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:29.167 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.159244+0000", 2024-09-14T23:57:29.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3108e-05 2024-09-14T23:57:29.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:29.167 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.160767+0000", 2024-09-14T23:57:29.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015231120000000001 2024-09-14T23:57:29.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:29.167 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.160808+0000", 2024-09-14T23:57:29.168 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1022999999999997e-05 2024-09-14T23:57:29.168 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.168 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:29.168 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.168 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:56107 3.0 e56/23 3:00c47890:::rbd_data.1118c4b3aebe.0000000000000009:head v 56'125, mlcod=52'123)", 2024-09-14T23:57:29.168 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.236123+0000", 2024-09-14T23:57:29.168 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.884028708999999, 2024-09-14T23:57:29.168 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0061273220000000002, 2024-09-14T23:57:29.168 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:29.168 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:29.168 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:29.168 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:29.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.169 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:29.169 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.236123+0000", 2024-09-14T23:57:29.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.169 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:29.169 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.236123+0000", 2024-09-14T23:57:29.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.169 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:29.169 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.236125+0000", 2024-09-14T23:57:29.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.313e-06 2024-09-14T23:57:29.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.170 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.170 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:29.170 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237428+0000", 2024-09-14T23:57:29.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013029890000000001 2024-09-14T23:57:29.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.170 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.170 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:29.170 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237452+0000", 2024-09-14T23:57:29.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4090000000000001e-05 2024-09-14T23:57:29.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.170 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.170 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:29.170 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237462+0000", 2024-09-14T23:57:29.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0421999999999999e-05 2024-09-14T23:57:29.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:29.171 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237559+0000", 2024-09-14T23:57:29.171 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6378000000000005e-05 2024-09-14T23:57:29.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:29.171 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237583+0000", 2024-09-14T23:57:29.171 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4182e-05 2024-09-14T23:57:29.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:29.172 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.242205+0000", 2024-09-14T23:57:29.172 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0046217289999999998 2024-09-14T23:57:29.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.172 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:29.172 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.242251+0000", 2024-09-14T23:57:29.172 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6218999999999999e-05 2024-09-14T23:57:29.172 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.172 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:29.172 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.172 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:56108 3.0 e56/23 3:00c47890:::rbd_data.1118c4b3aebe.0000000000000009:head v 56'127, mlcod=52'123)", 2024-09-14T23:57:29.172 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.239693+0000", 2024-09-14T23:57:29.173 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.880459239, 2024-09-14T23:57:29.173 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0043496209999999997, 2024-09-14T23:57:29.173 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:29.173 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:29.173 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:29.173 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:29.173 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.173 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:29.173 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239693+0000", 2024-09-14T23:57:29.173 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.173 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.173 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:29.174 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239693+0000", 2024-09-14T23:57:29.174 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.174 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.174 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:29.174 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239694+0000", 2024-09-14T23:57:29.174 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0499999999999999e-06 2024-09-14T23:57:29.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.174 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.174 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:29.174 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239702+0000", 2024-09-14T23:57:29.174 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1170000000000004e-06 2024-09-14T23:57:29.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.174 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.174 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:29.175 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239716+0000", 2024-09-14T23:57:29.175 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4306e-05 2024-09-14T23:57:29.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.175 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:29.175 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239724+0000", 2024-09-14T23:57:29.175 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0399999999999993e-06 2024-09-14T23:57:29.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.175 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:29.175 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239778+0000", 2024-09-14T23:57:29.175 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4032e-05 2024-09-14T23:57:29.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.176 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:29.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239801+0000", 2024-09-14T23:57:29.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2770000000000001e-05 2024-09-14T23:57:29.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.176 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:29.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.243990+0000", 2024-09-14T23:57:29.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0041890909999999998 2024-09-14T23:57:29.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.176 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:29.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.244043+0000", 2024-09-14T23:57:29.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2215000000000001e-05 2024-09-14T23:57:29.176 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.177 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:29.177 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.177 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.177 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.177 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:56385 3.9 e56/23 3:963d31c7:::rbd_data.1118c4b3aebe.0000000000000003:head v 56'87, mlcod=53'85)", 2024-09-14T23:57:29.177 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.283936+0000", 2024-09-14T23:57:29.177 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.836216328999999, 2024-09-14T23:57:29.177 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0063042080000000004, 2024-09-14T23:57:29.177 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:29.177 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:29.177 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:29.177 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:29.177 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.178 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:29.178 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.283936+0000", 2024-09-14T23:57:29.178 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.178 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.178 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:29.178 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.283936+0000", 2024-09-14T23:57:29.178 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.178 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.178 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:29.178 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.283937+0000", 2024-09-14T23:57:29.178 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1179999999999999e-06 2024-09-14T23:57:29.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.179 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.179 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:29.179 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286338+0000", 2024-09-14T23:57:29.179 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024010149999999998 2024-09-14T23:57:29.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.179 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.179 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:29.179 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286360+0000", 2024-09-14T23:57:29.179 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2212e-05 2024-09-14T23:57:29.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.179 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.179 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:29.179 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286370+0000", 2024-09-14T23:57:29.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5850000000000006e-06 2024-09-14T23:57:29.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:29.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286443+0000", 2024-09-14T23:57:29.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2757999999999997e-05 2024-09-14T23:57:29.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:29.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286466+0000", 2024-09-14T23:57:29.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3844000000000001e-05 2024-09-14T23:57:29.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:29.181 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.290197+0000", 2024-09-14T23:57:29.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003730176 2024-09-14T23:57:29.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.181 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.181 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:29.181 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.290240+0000", 2024-09-14T23:57:29.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.35e-05 2024-09-14T23:57:29.181 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.181 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:29.181 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.181 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.181 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:56386 3.9 e56/23 3:963d31c7:::rbd_data.1118c4b3aebe.0000000000000003:head v 56'89, mlcod=53'85)", 2024-09-14T23:57:29.181 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.287578+0000", 2024-09-14T23:57:29.182 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.832574603000001, 2024-09-14T23:57:29.182 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033580250000000002, 2024-09-14T23:57:29.182 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:29.182 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:29.182 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:29.182 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:29.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.182 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:29.182 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287578+0000", 2024-09-14T23:57:29.182 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.182 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:29.182 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287578+0000", 2024-09-14T23:57:29.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:29.183 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287579+0000", 2024-09-14T23:57:29.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.64e-07 2024-09-14T23:57:29.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:29.183 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287586+0000", 2024-09-14T23:57:29.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5310000000000004e-06 2024-09-14T23:57:29.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.184 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:29.184 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287597+0000", 2024-09-14T23:57:29.184 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0903999999999999e-05 2024-09-14T23:57:29.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.184 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.184 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:29.184 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287605+0000", 2024-09-14T23:57:29.184 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7889999999999995e-06 2024-09-14T23:57:29.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.184 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.184 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:29.184 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.288437+0000", 2024-09-14T23:57:29.184 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000831914 2024-09-14T23:57:29.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.185 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:29.185 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.288460+0000", 2024-09-14T23:57:29.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3410000000000001e-05 2024-09-14T23:57:29.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.185 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:29.185 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.290883+0000", 2024-09-14T23:57:29.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024230599999999999 2024-09-14T23:57:29.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.185 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:29.185 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.290936+0000", 2024-09-14T23:57:29.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2453000000000001e-05 2024-09-14T23:57:29.186 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.186 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:29.186 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.186 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:60299 3.9 e57/23 3:96f26b6f:::rbd_data.11188d356229.0000000000000009:head v 57'98, mlcod=56'96)", 2024-09-14T23:57:29.186 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:07.418336+0000", 2024-09-14T23:57:29.186 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 21.701816109999999, 2024-09-14T23:57:29.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022211459999999998, 2024-09-14T23:57:29.186 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:29.186 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:29.186 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:29.187 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:29.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:29.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.418336+0000", 2024-09-14T23:57:29.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:29.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.418336+0000", 2024-09-14T23:57:29.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:29.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.418337+0000", 2024-09-14T23:57:29.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7199999999999999e-07 2024-09-14T23:57:29.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.188 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:29.188 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.419396+0000", 2024-09-14T23:57:29.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010598319999999999 2024-09-14T23:57:29.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.188 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:29.188 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.419407+0000", 2024-09-14T23:57:29.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0956e-05 2024-09-14T23:57:29.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:29.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.419417+0000", 2024-09-14T23:57:29.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0040999999999999e-05 2024-09-14T23:57:29.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:29.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.419468+0000", 2024-09-14T23:57:29.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0497000000000001e-05 2024-09-14T23:57:29.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:29.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.419485+0000", 2024-09-14T23:57:29.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6966e-05 2024-09-14T23:57:29.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.190 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.190 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:29.190 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.420527+0000", 2024-09-14T23:57:29.190 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001042133 2024-09-14T23:57:29.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.190 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.190 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:29.190 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.420557+0000", 2024-09-14T23:57:29.190 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0249000000000001e-05 2024-09-14T23:57:29.190 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.190 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:29.190 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.191 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.191 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:64263 3.4 e57/23 3:20c6a768:::rbd_data.11188d356229.0000000000000001:head v 57'165, mlcod=57'163)", 2024-09-14T23:57:29.191 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:07.999031+0000", 2024-09-14T23:57:29.191 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 21.121121455000001, 2024-09-14T23:57:29.191 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023815210000000002, 2024-09-14T23:57:29.191 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:29.191 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:29.191 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:29.191 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:29.191 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.191 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:29.191 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999031+0000", 2024-09-14T23:57:29.191 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.192 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:29.192 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999031+0000", 2024-09-14T23:57:29.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.192 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:29.192 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999032+0000", 2024-09-14T23:57:29.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.176e-06 2024-09-14T23:57:29.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.192 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:29.192 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999660+0000", 2024-09-14T23:57:29.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00062817099999999996 2024-09-14T23:57:29.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.193 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.193 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:29.193 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999683+0000", 2024-09-14T23:57:29.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2816999999999999e-05 2024-09-14T23:57:29.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.193 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.193 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:29.193 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999693+0000", 2024-09-14T23:57:29.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0088999999999999e-05 2024-09-14T23:57:29.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.193 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.193 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:29.194 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999767+0000", 2024-09-14T23:57:29.194 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4056999999999999e-05 2024-09-14T23:57:29.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.194 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:29.194 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999793+0000", 2024-09-14T23:57:29.194 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5573000000000001e-05 2024-09-14T23:57:29.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.194 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:29.194 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:08.001379+0000", 2024-09-14T23:57:29.194 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015859019999999999 2024-09-14T23:57:29.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.195 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.195 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:29.195 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:08.001412+0000", 2024-09-14T23:57:29.195 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3735999999999997e-05 2024-09-14T23:57:29.195 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.195 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:29.195 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.195 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.195 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:65427 3.4 e60/23 3:269fb893:::rbd_data.11188d356229.00000000000000d9:head v 60'179, mlcod=58'175)", 2024-09-14T23:57:29.195 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:11.105942+0000", 2024-09-14T23:57:29.195 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.014210495, 2024-09-14T23:57:29.195 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001562393, 2024-09-14T23:57:29.195 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:29.195 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:29.196 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:29.196 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:29.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.196 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:29.196 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.105942+0000", 2024-09-14T23:57:29.196 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.196 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:29.196 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.105942+0000", 2024-09-14T23:57:29.196 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.196 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:29.197 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.105942+0000", 2024-09-14T23:57:29.197 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0299999999999998e-07 2024-09-14T23:57:29.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.197 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:29.197 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.105965+0000", 2024-09-14T23:57:29.197 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2629e-05 2024-09-14T23:57:29.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.197 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:29.197 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.105977+0000", 2024-09-14T23:57:29.197 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2023e-05 2024-09-14T23:57:29.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.198 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:29.198 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.105986+0000", 2024-09-14T23:57:29.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9670000000000007e-06 2024-09-14T23:57:29.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.198 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:29.198 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.106091+0000", 2024-09-14T23:57:29.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010527900000000001 2024-09-14T23:57:29.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.198 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:29.198 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.106112+0000", 2024-09-14T23:57:29.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0258000000000002e-05 2024-09-14T23:57:29.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.199 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:29.199 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.107460+0000", 2024-09-14T23:57:29.199 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001348393 2024-09-14T23:57:29.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.199 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:29.199 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.107504+0000", 2024-09-14T23:57:29.199 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4141000000000003e-05 2024-09-14T23:57:29.199 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.199 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:29.199 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.200 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:66570 3.0 e60/23 3:055350b3:::rbd_data.111873792d9b.0000000000000005:head v 60'162, mlcod=57'159)", 2024-09-14T23:57:29.200 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:11.277517+0000", 2024-09-14T23:57:29.200 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 17.842635299000001, 2024-09-14T23:57:29.200 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003472199, 2024-09-14T23:57:29.200 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:29.200 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:29.200 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:29.200 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:29.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.200 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:29.200 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.277517+0000", 2024-09-14T23:57:29.200 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.200 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:29.201 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.277517+0000", 2024-09-14T23:57:29.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.201 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:29.201 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.277518+0000", 2024-09-14T23:57:29.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1379999999999999e-06 2024-09-14T23:57:29.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.201 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:29.201 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278504+0000", 2024-09-14T23:57:29.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00098617499999999994 2024-09-14T23:57:29.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.202 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:29.202 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278525+0000", 2024-09-14T23:57:29.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.09e-05 2024-09-14T23:57:29.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.202 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:29.202 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278534+0000", 2024-09-14T23:57:29.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6209999999999993e-06 2024-09-14T23:57:29.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.202 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:29.202 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278611+0000", 2024-09-14T23:57:29.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6926000000000002e-05 2024-09-14T23:57:29.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.203 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:29.203 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278641+0000", 2024-09-14T23:57:29.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0861999999999997e-05 2024-09-14T23:57:29.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.203 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:29.203 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.280953+0000", 2024-09-14T23:57:29.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023114239999999999 2024-09-14T23:57:29.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.203 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:29.203 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.280989+0000", 2024-09-14T23:57:29.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6152999999999998e-05 2024-09-14T23:57:29.203 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.204 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:29.204 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.204 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:70526 3.4 e64/23 3:2649db36:::rbd_data.111831f32e80.0000000000000058:head v 64'183, mlcod=64'181)", 2024-09-14T23:57:29.204 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:15.442503+0000", 2024-09-14T23:57:29.204 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.677649407000001, 2024-09-14T23:57:29.204 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001769837, 2024-09-14T23:57:29.204 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:29.204 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:29.204 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:29.204 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:29.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.204 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:29.204 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.442503+0000", 2024-09-14T23:57:29.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:29.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.442503+0000", 2024-09-14T23:57:29.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:29.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.442503+0000", 2024-09-14T23:57:29.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1099999999999995e-07 2024-09-14T23:57:29.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:29.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.442620+0000", 2024-09-14T23:57:29.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000116618 2024-09-14T23:57:29.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:29.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.442634+0000", 2024-09-14T23:57:29.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3984e-05 2024-09-14T23:57:29.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:29.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.442641+0000", 2024-09-14T23:57:29.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2540000000000003e-06 2024-09-14T23:57:29.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:29.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.442732+0000", 2024-09-14T23:57:29.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0480000000000001e-05 2024-09-14T23:57:29.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.207 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:29.207 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.442753+0000", 2024-09-14T23:57:29.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1571999999999999e-05 2024-09-14T23:57:29.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.207 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:29.207 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.444236+0000", 2024-09-14T23:57:29.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0014825439999999999 2024-09-14T23:57:29.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.207 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:29.207 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.444273+0000", 2024-09-14T23:57:29.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6674000000000002e-05 2024-09-14T23:57:29.208 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.208 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:29.208 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.208 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:106629 3.4 e70/23 3:240d85c6:::rbd_data.11187de8b0a5.0000000000000009:head v 70'233, mlcod=69'231)", 2024-09-14T23:57:29.208 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:21.489517+0000", 2024-09-14T23:57:29.208 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 7.6306347739999998, 2024-09-14T23:57:29.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016279840000000001, 2024-09-14T23:57:29.208 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:29.208 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:29.208 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:29.208 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:29.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.209 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:29.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.489517+0000", 2024-09-14T23:57:29.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.209 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:29.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.489517+0000", 2024-09-14T23:57:29.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.209 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:29.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.489518+0000", 2024-09-14T23:57:29.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.029e-06 2024-09-14T23:57:29.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:29.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.489840+0000", 2024-09-14T23:57:29.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00032164100000000001 2024-09-14T23:57:29.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:29.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.489859+0000", 2024-09-14T23:57:29.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8828999999999999e-05 2024-09-14T23:57:29.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:29.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.489869+0000", 2024-09-14T23:57:29.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6600000000000007e-06 2024-09-14T23:57:29.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:29.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.489934+0000", 2024-09-14T23:57:29.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5357000000000005e-05 2024-09-14T23:57:29.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:29.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.489951+0000", 2024-09-14T23:57:29.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6951000000000001e-05 2024-09-14T23:57:29.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:29.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.491108+0000", 2024-09-14T23:57:29.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011571940000000001 2024-09-14T23:57:29.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.212 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:29.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.491145+0000", 2024-09-14T23:57:29.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7323000000000002e-05 2024-09-14T23:57:29.212 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.212 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:29.212 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.212 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.212 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:29.212 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:29.214 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T23:57:29.215 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-09-14T23:57:29.216 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:29.213+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:29.316 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:29.314+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:29.327 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:29.328 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:57:29.328 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:57:29.328 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:29.337 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T23:57:29.403 INFO:teuthology.orchestra.run.smithi042.stdout:2784 write 0x539d4a8 thru 0x53a6aa5 (0x95fe bytes) 2024-09-14T23:57:29.415 INFO:teuthology.orchestra.run.smithi042.stdout:2785 read 0x4cc70bf thru 0x4ccac4c (0x3b8e bytes) 2024-09-14T23:57:29.417 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:29.415+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:29.422 INFO:teuthology.orchestra.run.smithi042.stdout:2786 write 0x7a1f4ba thru 0x7a284d8 (0x901f bytes) 2024-09-14T23:57:29.431 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:29.431 INFO:teuthology.orchestra.run.smithi042.stdout:2788 read 0x738a809 thru 0x7390a14 (0x620c bytes) 2024-09-14T23:57:29.431 INFO:teuthology.orchestra.run.smithi042.stdout:2789 punch from 0x4a59655 to 0x4a66df6, (0xd7a1 bytes) 2024-09-14T23:57:29.434 INFO:teuthology.orchestra.run.smithi042.stdout:2791 trunc from 0x7a284d9 to 0xa26c5f9 2024-09-14T23:57:29.439 INFO:teuthology.orchestra.run.smithi042.stdout:2792 write 0x88cd517 thru 0x88cf6e7 (0x21d1 bytes) 2024-09-14T23:57:29.449 INFO:teuthology.orchestra.run.smithi042.stdout:2793 write 0xa833e78 thru 0xa843aa8 (0xfc31 bytes) 2024-09-14T23:57:29.456 INFO:teuthology.orchestra.run.smithi042.stdout:2794 trunc from 0xa843aa9 to 0xcf3f5d9 2024-09-14T23:57:29.470 INFO:teuthology.orchestra.run.smithi042.stdout:2796 write 0xebf70fa thru 0xec023bf (0xb2c6 bytes) 2024-09-14T23:57:29.474 INFO:teuthology.orchestra.run.smithi042.stdout:2797 read 0xe7ddd6d thru 0xe7de216 (0x4aa bytes) 2024-09-14T23:57:29.478 INFO:teuthology.orchestra.run.smithi042.stdout:2798 trunc from 0xec023c0 to 0xb5215b6 2024-09-14T23:57:29.487 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:29.487 INFO:teuthology.orchestra.run.smithi042.stdout:2801 read 0x5a99cdb thru 0x5a9fca2 (0x5fc8 bytes) 2024-09-14T23:57:29.487 INFO:teuthology.orchestra.run.smithi042.stdout:2805 read 0xa1b536d thru 0xa1b87fa (0x348e bytes) 2024-09-14T23:57:29.488 INFO:teuthology.orchestra.run.smithi042.stdout:2807 writesame 0x6739d3c thru 0x67473d6 data_size 0x1(0xd69b bytes) 2024-09-14T23:57:29.496 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:29.496 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:57:29.496 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:57:29.496 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:57:29.496 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:29.505 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T23:57:29.515 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:29.515 INFO:teuthology.orchestra.run.smithi042.stdout:2809 writesame 0x95d44f9 thru 0x95d7dab data_size 0x1(0x38b3 bytes) 2024-09-14T23:57:29.524 INFO:teuthology.orchestra.run.smithi042.stdout:2810 writesame 0x371fef3 thru 0x372d72f data_size 0x1(0xd83d bytes) 2024-09-14T23:57:29.525 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:29.522+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:29.553 INFO:teuthology.orchestra.run.smithi042.stdout:2811 punch from 0x7ebd050 to 0x7ec6382, (0x9332 bytes) 2024-09-14T23:57:29.562 INFO:teuthology.orchestra.run.smithi042.stdout:2813 write 0xe6946b7 thru 0xe698ca0 (0x45ea bytes) 2024-09-14T23:57:29.565 INFO:teuthology.orchestra.run.smithi042.stdout:2815 write 0xdf770d5 thru 0xdf7f96f (0x889b bytes) 2024-09-14T23:57:29.568 INFO:teuthology.orchestra.run.smithi042.stdout:2816 read 0x162612a thru 0x162829c (0x2173 bytes) 2024-09-14T23:57:29.569 INFO:teuthology.orchestra.run.smithi042.stdout:2817 punch from 0x2f545d to 0x304c7c, (0xf81f bytes) 2024-09-14T23:57:29.569 INFO:teuthology.orchestra.run.smithi042.stdout:2819 write 0xdeae235 thru 0xdebb860 (0xd62c bytes) 2024-09-14T23:57:29.572 INFO:teuthology.orchestra.run.smithi042.stdout:2820 read 0xa320651 thru 0xa326552 (0x5f02 bytes) 2024-09-14T23:57:29.572 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:29.572 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:29.572 INFO:teuthology.orchestra.run.smithi042.stdout:2824 punch from 0xddb77e8 to 0xddc0e82, (0x969a bytes) 2024-09-14T23:57:29.584 INFO:teuthology.orchestra.run.smithi042.stdout:2825 trunc from 0xe698ca1 to 0x3a2996c 2024-09-14T23:57:29.620 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:29.617+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:29.622 INFO:teuthology.orchestra.run.smithi042.stdout:2828 writesame 0xcd00e4e thru 0xcd109c1 data_size 0x85(0xfb74 bytes) 2024-09-14T23:57:29.622 INFO:teuthology.orchestra.run.smithi042.stdout:2829 read 0x976069c thru 0x9762e73 (0x27d8 bytes) 2024-09-14T23:57:29.623 INFO:teuthology.orchestra.run.smithi042.stdout:2831 writesame 0x8e723a thru 0x8e76f2 data_size 0xd(0x4b9 bytes) 2024-09-14T23:57:29.624 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:29.624 INFO:teuthology.orchestra.run.smithi042.stdout:2833 read 0x4e2d2a5 thru 0x4e2e3ec (0x1148 bytes) 2024-09-14T23:57:29.625 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:29.625 INFO:teuthology.orchestra.run.smithi042.stdout:2835 writesame 0xacb2363 thru 0xacb50d0 data_size 0x5(0x2d6e bytes) 2024-09-14T23:57:29.628 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:29.629 INFO:teuthology.orchestra.run.smithi042.stdout:2838 punch from 0x3ac78b0 to 0x3acfe5f, (0x85af bytes) 2024-09-14T23:57:29.629 INFO:teuthology.orchestra.run.smithi042.stdout:2840 write 0x5780007 thru 0x5789b8d (0x9b87 bytes) 2024-09-14T23:57:29.630 INFO:teuthology.orchestra.run.smithi042.stdout:2841 write 0x6d903fa thru 0x6d9f1dc (0xede3 bytes) 2024-09-14T23:57:29.637 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:29.637 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:57:29.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:57:29.638 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:57:29.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.638 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5180 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3883008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:29.638 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.250058+0000", 2024-09-14T23:57:29.638 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 50.385374173000002, 2024-09-14T23:57:29.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057351471000000001, 2024-09-14T23:57:29.638 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:29.638 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:29.638 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:29.638 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:29.638 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:29.638 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:29.638 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5180 2024-09-14T23:57:29.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.639 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:29.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:29.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250058+0000", 2024-09-14T23:57:29.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:29.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250058+0000", 2024-09-14T23:57:29.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:29.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250786+0000", 2024-09-14T23:57:29.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00072813399999999996 2024-09-14T23:57:29.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:29.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250789+0000", 2024-09-14T23:57:29.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.647e-06 2024-09-14T23:57:29.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:29.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250798+0000", 2024-09-14T23:57:29.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0000000000000002e-06 2024-09-14T23:57:29.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:29.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250803+0000", 2024-09-14T23:57:29.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.301e-06 2024-09-14T23:57:29.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:29.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.307234+0000", 2024-09-14T23:57:29.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056431165999999998 2024-09-14T23:57:29.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:29.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.307321+0000", 2024-09-14T23:57:29.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6831999999999998e-05 2024-09-14T23:57:29.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:29.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.307409+0000", 2024-09-14T23:57:29.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8391000000000001e-05 2024-09-14T23:57:29.642 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.642 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:29.642 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.642 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5184 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 4128768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:29.642 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.250851+0000", 2024-09-14T23:57:29.642 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 50.384580702000001, 2024-09-14T23:57:29.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057319038000000003, 2024-09-14T23:57:29.642 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:29.643 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:29.643 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:29.643 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:29.643 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:29.643 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:29.643 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5184 2024-09-14T23:57:29.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.643 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:29.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:29.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250851+0000", 2024-09-14T23:57:29.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:29.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250851+0000", 2024-09-14T23:57:29.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:29.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.251900+0000", 2024-09-14T23:57:29.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001048577 2024-09-14T23:57:29.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:29.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.251904+0000", 2024-09-14T23:57:29.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9580000000000001e-06 2024-09-14T23:57:29.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:29.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.251914+0000", 2024-09-14T23:57:29.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0186999999999999e-05 2024-09-14T23:57:29.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:29.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.251927+0000", 2024-09-14T23:57:29.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3593e-05 2024-09-14T23:57:29.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:29.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.308020+0000", 2024-09-14T23:57:29.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056092867999999997 2024-09-14T23:57:29.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:29.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.308094+0000", 2024-09-14T23:57:29.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3533000000000002e-05 2024-09-14T23:57:29.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:29.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.308170+0000", 2024-09-14T23:57:29.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6322000000000006e-05 2024-09-14T23:57:29.647 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.648 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:29.648 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.648 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5196 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 1925120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:29.648 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.252157+0000", 2024-09-14T23:57:29.648 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 50.383274587999999, 2024-09-14T23:57:29.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057285801999999997, 2024-09-14T23:57:29.648 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:29.648 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:29.648 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:29.648 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:29.648 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:29.648 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:29.649 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5196 2024-09-14T23:57:29.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.649 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:29.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:29.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.252157+0000", 2024-09-14T23:57:29.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:29.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.252157+0000", 2024-09-14T23:57:29.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.650 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:29.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253243+0000", 2024-09-14T23:57:29.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010861340000000001 2024-09-14T23:57:29.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.650 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:29.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253248+0000", 2024-09-14T23:57:29.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8799999999999999e-06 2024-09-14T23:57:29.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.650 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:29.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253271+0000", 2024-09-14T23:57:29.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2836e-05 2024-09-14T23:57:29.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.651 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:29.651 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253281+0000", 2024-09-14T23:57:29.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6520000000000001e-06 2024-09-14T23:57:29.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.651 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:29.651 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.309336+0000", 2024-09-14T23:57:29.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056054983000000003 2024-09-14T23:57:29.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.652 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:29.652 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.309389+0000", 2024-09-14T23:57:29.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3298999999999999e-05 2024-09-14T23:57:29.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.652 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:29.652 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.309443+0000", 2024-09-14T23:57:29.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4017999999999997e-05 2024-09-14T23:57:29.652 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.652 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:29.652 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.652 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5200 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2170880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:29.653 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253394+0000", 2024-09-14T23:57:29.653 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 50.382037811000004, 2024-09-14T23:57:29.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057428154000000002, 2024-09-14T23:57:29.653 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:29.653 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:29.653 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:29.653 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:29.653 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:29.653 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:29.653 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5200 2024-09-14T23:57:29.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.653 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:29.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.654 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:29.654 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253394+0000", 2024-09-14T23:57:29.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.654 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:29.654 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253394+0000", 2024-09-14T23:57:29.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.654 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:29.654 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253395+0000", 2024-09-14T23:57:29.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9100000000000004e-07 2024-09-14T23:57:29.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.655 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:29.655 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253397+0000", 2024-09-14T23:57:29.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2790000000000001e-06 2024-09-14T23:57:29.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.655 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:29.655 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253404+0000", 2024-09-14T23:57:29.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7959999999999998e-06 2024-09-14T23:57:29.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.655 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:29.656 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253409+0000", 2024-09-14T23:57:29.656 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1460000000000004e-06 2024-09-14T23:57:29.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.656 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.656 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:29.656 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310627+0000", 2024-09-14T23:57:29.656 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057218160999999997 2024-09-14T23:57:29.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.656 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.656 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:29.657 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310710+0000", 2024-09-14T23:57:29.657 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3102999999999994e-05 2024-09-14T23:57:29.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.657 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.657 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:29.657 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310822+0000", 2024-09-14T23:57:29.657 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000112178 2024-09-14T23:57:29.657 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.657 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:29.657 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.658 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5201 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2232320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:29.658 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:57:29.658 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 50.382013139000001, 2024-09-14T23:57:29.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057643228999999997, 2024-09-14T23:57:29.658 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:29.658 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:29.658 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:29.658 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:29.658 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:29.659 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:29.659 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5201 2024-09-14T23:57:29.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.659 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:29.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:29.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:57:29.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:29.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:57:29.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.660 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:29.660 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:57:29.660 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0699999999999998e-07 2024-09-14T23:57:29.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.660 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:29.660 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253422+0000", 2024-09-14T23:57:29.660 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.514e-06 2024-09-14T23:57:29.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.660 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:29.660 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253431+0000", 2024-09-14T23:57:29.660 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1919999999999994e-06 2024-09-14T23:57:29.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.661 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:29.661 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253436+0000", 2024-09-14T23:57:29.661 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5350000000000004e-06 2024-09-14T23:57:29.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.661 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:29.661 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310856+0000", 2024-09-14T23:57:29.661 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057419381999999998 2024-09-14T23:57:29.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.662 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:29.662 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310953+0000", 2024-09-14T23:57:29.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6774000000000006e-05 2024-09-14T23:57:29.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.662 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:29.662 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311062+0000", 2024-09-14T23:57:29.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000109425 2024-09-14T23:57:29.662 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.662 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:29.663 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.663 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5202 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2293760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:29.663 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:57:29.663 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 50.381988391, 2024-09-14T23:57:29.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057818255999999998, 2024-09-14T23:57:29.663 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:29.663 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:29.663 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:29.663 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:29.663 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:29.663 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:29.664 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5202 2024-09-14T23:57:29.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.664 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:29.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.664 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:29.664 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:57:29.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.664 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:29.664 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:57:29.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.665 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:29.665 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253573+0000", 2024-09-14T23:57:29.665 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012934000000000001 2024-09-14T23:57:29.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.665 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:29.665 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253576+0000", 2024-09-14T23:57:29.665 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5310000000000002e-06 2024-09-14T23:57:29.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.665 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:29.665 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253587+0000", 2024-09-14T23:57:29.665 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0522e-05 2024-09-14T23:57:29.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.666 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:29.666 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253595+0000", 2024-09-14T23:57:29.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8290000000000008e-06 2024-09-14T23:57:29.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.666 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:29.666 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311093+0000", 2024-09-14T23:57:29.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057498312000000003 2024-09-14T23:57:29.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.666 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:29.666 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311167+0000", 2024-09-14T23:57:29.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3582000000000001e-05 2024-09-14T23:57:29.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.667 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:29.667 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311262+0000", 2024-09-14T23:57:29.667 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5140000000000001e-05 2024-09-14T23:57:29.667 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.667 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:29.667 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.667 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5203 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2355200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:29.668 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:57:29.668 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 50.381825649, 2024-09-14T23:57:29.668 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057903938000000002, 2024-09-14T23:57:29.668 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:29.668 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:29.668 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:29.668 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:29.668 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:29.668 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:29.668 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5203 2024-09-14T23:57:29.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.668 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:29.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.669 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:29.669 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:57:29.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.669 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:29.669 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:57:29.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.669 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:29.669 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253902+0000", 2024-09-14T23:57:29.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029615400000000002 2024-09-14T23:57:29.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.670 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:29.670 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253906+0000", 2024-09-14T23:57:29.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9249999999999997e-06 2024-09-14T23:57:29.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.670 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:29.670 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253913+0000", 2024-09-14T23:57:29.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9099999999999999e-06 2024-09-14T23:57:29.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.670 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:29.670 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253917+0000", 2024-09-14T23:57:29.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9210000000000002e-06 2024-09-14T23:57:29.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.671 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:29.671 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311295+0000", 2024-09-14T23:57:29.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057378268000000003 2024-09-14T23:57:29.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.671 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:29.671 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311402+0000", 2024-09-14T23:57:29.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010644100000000001 2024-09-14T23:57:29.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.671 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:29.671 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311510+0000", 2024-09-14T23:57:29.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108319 2024-09-14T23:57:29.672 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.672 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:29.672 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.672 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5204 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2416640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:29.672 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:57:29.672 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 50.381507452000001, 2024-09-14T23:57:29.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057774860999999997, 2024-09-14T23:57:29.672 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:29.672 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:29.672 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:29.673 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:29.673 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:29.673 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:29.673 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5204 2024-09-14T23:57:29.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.673 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:29.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.673 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:29.673 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:57:29.673 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.673 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:29.674 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:57:29.674 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.674 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:29.674 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255903+0000", 2024-09-14T23:57:29.674 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001978807 2024-09-14T23:57:29.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.674 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:29.674 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255907+0000", 2024-09-14T23:57:29.674 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3610000000000001e-06 2024-09-14T23:57:29.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:29.675 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255914+0000", 2024-09-14T23:57:29.675 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.357e-06 2024-09-14T23:57:29.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:29.675 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255918+0000", 2024-09-14T23:57:29.675 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1779999999999996e-06 2024-09-14T23:57:29.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:29.675 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311541+0000", 2024-09-14T23:57:29.675 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055622697999999998 2024-09-14T23:57:29.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.676 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:29.676 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311614+0000", 2024-09-14T23:57:29.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3102999999999994e-05 2024-09-14T23:57:29.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.676 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:29.676 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311699+0000", 2024-09-14T23:57:29.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5357000000000003e-05 2024-09-14T23:57:29.676 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.676 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:29.676 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.677 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.677 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5210 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2785280~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:29.677 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256011+0000", 2024-09-14T23:57:29.677 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 50.379420815000003, 2024-09-14T23:57:29.677 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057273662000000003, 2024-09-14T23:57:29.677 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:29.677 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:29.677 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:29.677 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:29.677 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:29.677 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:29.677 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5210 2024-09-14T23:57:29.678 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.678 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:29.678 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.678 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:29.678 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256011+0000", 2024-09-14T23:57:29.678 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.678 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.678 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.678 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:29.678 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256011+0000", 2024-09-14T23:57:29.678 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.678 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.678 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.678 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:29.679 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256011+0000", 2024-09-14T23:57:29.679 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4999999999999999e-07 2024-09-14T23:57:29.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.679 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:29.679 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256013+0000", 2024-09-14T23:57:29.679 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3969999999999999e-06 2024-09-14T23:57:29.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.679 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:29.679 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256017+0000", 2024-09-14T23:57:29.679 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1409999999999998e-06 2024-09-14T23:57:29.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.679 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:29.680 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256019+0000", 2024-09-14T23:57:29.680 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6280000000000001e-06 2024-09-14T23:57:29.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.680 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:29.680 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313116+0000", 2024-09-14T23:57:29.680 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057096457000000003 2024-09-14T23:57:29.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.680 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:29.680 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313193+0000", 2024-09-14T23:57:29.680 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7439000000000002e-05 2024-09-14T23:57:29.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.680 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:29.681 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313285+0000", 2024-09-14T23:57:29.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1349999999999998e-05 2024-09-14T23:57:29.681 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.681 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:29.681 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.681 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5211 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2846720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:29.681 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:57:29.681 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 50.379409238000001, 2024-09-14T23:57:29.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057528151, 2024-09-14T23:57:29.681 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:29.681 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:29.681 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:29.682 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:29.682 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:29.682 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:29.682 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5211 2024-09-14T23:57:29.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.682 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:29.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.682 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:29.682 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:57:29.682 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.682 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:29.682 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:57:29.683 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.683 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.683 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:29.683 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:57:29.683 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1399999999999998e-07 2024-09-14T23:57:29.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.683 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.683 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:29.683 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256024+0000", 2024-09-14T23:57:29.683 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.114e-06 2024-09-14T23:57:29.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.684 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:29.684 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256028+0000", 2024-09-14T23:57:29.684 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0840000000000002e-06 2024-09-14T23:57:29.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.684 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:29.684 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256031+0000", 2024-09-14T23:57:29.684 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6819999999999999e-06 2024-09-14T23:57:29.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.684 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:29.684 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313317+0000", 2024-09-14T23:57:29.684 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057286295000000001 2024-09-14T23:57:29.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.685 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:29.685 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313403+0000", 2024-09-14T23:57:29.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6287999999999998e-05 2024-09-14T23:57:29.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.685 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:29.685 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313551+0000", 2024-09-14T23:57:29.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000147374 2024-09-14T23:57:29.685 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.685 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:29.685 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.686 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5212 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2908160~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:29.686 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:57:29.686 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 50.379398043999998, 2024-09-14T23:57:29.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057735679999999998, 2024-09-14T23:57:29.686 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:29.686 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:29.686 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:29.686 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:29.686 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:29.686 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:29.686 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5212 2024-09-14T23:57:29.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.687 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:29.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.687 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:29.687 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:57:29.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.687 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:29.687 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:57:29.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.687 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:29.687 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:57:29.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0900000000000001e-07 2024-09-14T23:57:29.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.688 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:29.688 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256035+0000", 2024-09-14T23:57:29.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1200000000000001e-06 2024-09-14T23:57:29.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.688 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:29.688 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256038+0000", 2024-09-14T23:57:29.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.05e-06 2024-09-14T23:57:29.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.688 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:29.688 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256041+0000", 2024-09-14T23:57:29.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.667e-06 2024-09-14T23:57:29.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.689 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:29.689 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313584+0000", 2024-09-14T23:57:29.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057543484999999998 2024-09-14T23:57:29.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.689 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:29.689 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313668+0000", 2024-09-14T23:57:29.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3807000000000006e-05 2024-09-14T23:57:29.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.689 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:29.689 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313770+0000", 2024-09-14T23:57:29.690 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010134200000000001 2024-09-14T23:57:29.690 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.690 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:29.690 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.690 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5213 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2969600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:29.690 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:57:29.690 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 50.379387956999999, 2024-09-14T23:57:29.690 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058145467999999999, 2024-09-14T23:57:29.690 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:29.690 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:29.690 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:29.690 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:29.690 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:29.691 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:29.691 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5213 2024-09-14T23:57:29.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.691 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:29.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.691 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:29.691 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:57:29.691 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.691 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:29.691 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:57:29.691 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.692 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.692 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.692 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:29.692 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257901+0000", 2024-09-14T23:57:29.692 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018567519999999999 2024-09-14T23:57:29.692 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.692 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.692 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:29.692 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257904+0000", 2024-09-14T23:57:29.692 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4850000000000001e-06 2024-09-14T23:57:29.692 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.692 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.692 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:29.692 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257912+0000", 2024-09-14T23:57:29.693 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2090000000000005e-06 2024-09-14T23:57:29.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.693 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.693 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:29.693 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257918+0000", 2024-09-14T23:57:29.693 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6899999999999997e-06 2024-09-14T23:57:29.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.693 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.693 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:29.693 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313816+0000", 2024-09-14T23:57:29.693 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055898075999999998 2024-09-14T23:57:29.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.693 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.693 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:29.693 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.314082+0000", 2024-09-14T23:57:29.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000265386 2024-09-14T23:57:29.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:29.694 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.314189+0000", 2024-09-14T23:57:29.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010787 2024-09-14T23:57:29.694 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.694 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:29.694 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.694 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5217 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3215360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:29.694 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257961+0000", 2024-09-14T23:57:29.694 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 50.377470977999998, 2024-09-14T23:57:29.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057320969999999999, 2024-09-14T23:57:29.695 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:29.695 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:29.695 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:29.695 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:29.695 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:29.695 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:29.695 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5217 2024-09-14T23:57:29.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.695 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:29.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.695 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:29.695 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257961+0000", 2024-09-14T23:57:29.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.696 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:29.696 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257961+0000", 2024-09-14T23:57:29.696 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.696 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:29.696 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257961+0000", 2024-09-14T23:57:29.696 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.22e-07 2024-09-14T23:57:29.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.696 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:29.696 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257962+0000", 2024-09-14T23:57:29.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1400000000000001e-06 2024-09-14T23:57:29.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.697 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:29.697 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257965+0000", 2024-09-14T23:57:29.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0929999999999999e-06 2024-09-14T23:57:29.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.697 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:29.698 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257968+0000", 2024-09-14T23:57:29.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5749999999999999e-06 2024-09-14T23:57:29.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.698 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:29.698 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315047+0000", 2024-09-14T23:57:29.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057078730000000001 2024-09-14T23:57:29.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.698 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:29.698 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315087+0000", 2024-09-14T23:57:29.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0806999999999997e-05 2024-09-14T23:57:29.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.699 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:29.699 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315282+0000", 2024-09-14T23:57:29.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000194403 2024-09-14T23:57:29.699 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.699 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:29.699 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.699 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5218 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3276800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:29.699 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:57:29.699 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 50.377460948, 2024-09-14T23:57:29.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057497557999999997, 2024-09-14T23:57:29.699 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:29.699 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:29.699 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:29.700 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:29.700 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:29.700 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:29.700 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5218 2024-09-14T23:57:29.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.700 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:29.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.700 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:29.700 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:57:29.700 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.701 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:29.701 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:57:29.701 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.701 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:29.701 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:57:29.701 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1899999999999999e-07 2024-09-14T23:57:29.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.701 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:29.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257972+0000", 2024-09-14T23:57:29.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.217e-06 2024-09-14T23:57:29.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:29.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257975+0000", 2024-09-14T23:57:29.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9359999999999999e-06 2024-09-14T23:57:29.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:29.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257978+0000", 2024-09-14T23:57:29.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1049999999999999e-06 2024-09-14T23:57:29.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:29.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315331+0000", 2024-09-14T23:57:29.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057352093999999999 2024-09-14T23:57:29.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.703 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:29.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315384+0000", 2024-09-14T23:57:29.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3780999999999999e-05 2024-09-14T23:57:29.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.703 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:29.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315468+0000", 2024-09-14T23:57:29.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4206e-05 2024-09-14T23:57:29.703 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.703 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:29.703 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.704 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5219 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3338240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:29.704 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:57:29.704 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 50.377450271999997, 2024-09-14T23:57:29.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057637764000000001, 2024-09-14T23:57:29.704 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:29.704 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:29.704 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:29.704 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:29.704 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:29.705 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:29.705 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5219 2024-09-14T23:57:29.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.705 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:29.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.705 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:29.705 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:57:29.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.705 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:29.705 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:57:29.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:29.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:57:29.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.03e-07 2024-09-14T23:57:29.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:29.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257983+0000", 2024-09-14T23:57:29.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9199999999999999e-07 2024-09-14T23:57:29.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:29.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257986+0000", 2024-09-14T23:57:29.707 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0860000000000002e-06 2024-09-14T23:57:29.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.707 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:29.707 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257989+0000", 2024-09-14T23:57:29.707 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6589999999999999e-06 2024-09-14T23:57:29.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.707 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:29.707 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315509+0000", 2024-09-14T23:57:29.707 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057520262000000003 2024-09-14T23:57:29.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.707 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:29.707 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315554+0000", 2024-09-14T23:57:29.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5015999999999997e-05 2024-09-14T23:57:29.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:29.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315619+0000", 2024-09-14T23:57:29.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5545999999999999e-05 2024-09-14T23:57:29.708 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.708 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:29.708 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.708 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5220 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3399680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:29.708 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:57:29.708 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 50.377440315999998, 2024-09-14T23:57:29.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057792128999999998, 2024-09-14T23:57:29.709 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:29.709 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:29.709 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:29.709 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:29.709 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:29.709 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:29.709 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5220 2024-09-14T23:57:29.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.709 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:29.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:29.709 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:57:29.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.710 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:29.710 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:57:29.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.710 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:29.710 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:57:29.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1799999999999999e-07 2024-09-14T23:57:29.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.710 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257993+0000", 2024-09-14T23:57:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6200000000000006e-07 2024-09-14T23:57:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257996+0000", 2024-09-14T23:57:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9050000000000001e-06 2024-09-14T23:57:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257998+0000", 2024-09-14T23:57:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5950000000000001e-06 2024-09-14T23:57:29.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.712 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.712 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:29.712 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315672+0000", 2024-09-14T23:57:29.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057673637999999999 2024-09-14T23:57:29.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.712 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.712 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:29.712 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315734+0000", 2024-09-14T23:57:29.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2216000000000002e-05 2024-09-14T23:57:29.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.712 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.712 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:29.712 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315784+0000", 2024-09-14T23:57:29.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9595000000000002e-05 2024-09-14T23:57:29.712 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.713 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:29.713 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.713 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5221 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3461120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:29.713 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:57:29.713 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 50.377430597999997, 2024-09-14T23:57:29.713 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058003557999999997, 2024-09-14T23:57:29.713 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:29.713 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:29.713 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:29.713 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:29.713 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5221 2024-09-14T23:57:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:57:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:57:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259905+0000", 2024-09-14T23:57:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019035759999999999 2024-09-14T23:57:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259915+0000", 2024-09-14T23:57:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0518e-05 2024-09-14T23:57:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259925+0000", 2024-09-14T23:57:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7180000000000008e-06 2024-09-14T23:57:29.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.716 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:29.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259930+0000", 2024-09-14T23:57:29.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4769999999999997e-06 2024-09-14T23:57:29.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.716 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:29.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315820+0000", 2024-09-14T23:57:29.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055889960000000002 2024-09-14T23:57:29.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.716 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:29.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315914+0000", 2024-09-14T23:57:29.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4625999999999998e-05 2024-09-14T23:57:29.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.717 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.717 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:29.717 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.316005+0000", 2024-09-14T23:57:29.717 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0682999999999999e-05 2024-09-14T23:57:29.717 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.717 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:29.717 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.717 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.717 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5230 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 4014080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:29.717 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.260060+0000", 2024-09-14T23:57:29.717 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 50.37537176, 2024-09-14T23:57:29.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057353556999999999, 2024-09-14T23:57:29.718 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:29.718 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:29.718 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:29.718 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:29.718 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:29.718 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:29.718 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5230 2024-09-14T23:57:29.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.718 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:29.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:29.718 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260060+0000", 2024-09-14T23:57:29.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:29.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260060+0000", 2024-09-14T23:57:29.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:29.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260060+0000", 2024-09-14T23:57:29.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.17e-07 2024-09-14T23:57:29.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:29.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260061+0000", 2024-09-14T23:57:29.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0300000000000001e-06 2024-09-14T23:57:29.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.720 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:29.720 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260064+0000", 2024-09-14T23:57:29.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9119999999999998e-06 2024-09-14T23:57:29.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.720 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:29.720 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260067+0000", 2024-09-14T23:57:29.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.678e-06 2024-09-14T23:57:29.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.720 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:29.720 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317269+0000", 2024-09-14T23:57:29.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057201771999999998 2024-09-14T23:57:29.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.721 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:29.721 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317342+0000", 2024-09-14T23:57:29.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2842000000000005e-05 2024-09-14T23:57:29.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.721 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:29.721 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317414+0000", 2024-09-14T23:57:29.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2106000000000004e-05 2024-09-14T23:57:29.721 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.721 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:29.721 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.722 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5231 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 4075520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:29.722 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:57:29.722 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 50.375361863999998, 2024-09-14T23:57:29.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057524733000000002, 2024-09-14T23:57:29.722 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:29.722 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:29.722 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:29.722 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:29.722 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:29.722 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:29.722 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5231 2024-09-14T23:57:29.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.722 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:29.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.722 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:29.723 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:57:29.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.723 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:29.723 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:57:29.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.723 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:29.723 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261904+0000", 2024-09-14T23:57:29.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018338790000000001 2024-09-14T23:57:29.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.723 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:29.724 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261910+0000", 2024-09-14T23:57:29.724 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9689999999999999e-06 2024-09-14T23:57:29.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.724 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.724 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:29.724 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261923+0000", 2024-09-14T23:57:29.724 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3061000000000001e-05 2024-09-14T23:57:29.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.724 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.724 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:29.724 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261931+0000", 2024-09-14T23:57:29.724 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3159999999999997e-06 2024-09-14T23:57:29.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.724 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.724 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:29.725 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317456+0000", 2024-09-14T23:57:29.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055525219000000001 2024-09-14T23:57:29.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.725 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:29.725 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317520+0000", 2024-09-14T23:57:29.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4010000000000005e-05 2024-09-14T23:57:29.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.725 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:29.725 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317595+0000", 2024-09-14T23:57:29.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4278999999999998e-05 2024-09-14T23:57:29.725 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5236 3.8 3:1071668d:::rbd_data.1118de1168b.0000000000000092:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:57:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 50.373348399999998, 2024-09-14T23:57:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057499462000000001, 2024-09-14T23:57:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:29.726 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:29.727 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:29.727 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5236 2024-09-14T23:57:29.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.727 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:29.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:29.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:57:29.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:29.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:57:29.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:29.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.728 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:29.728 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262084+0000", 2024-09-14T23:57:29.728 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3399999999999999e-07 2024-09-14T23:57:29.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.728 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:29.728 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262087+0000", 2024-09-14T23:57:29.728 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2310000000000001e-06 2024-09-14T23:57:29.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.728 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:29.728 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262095+0000", 2024-09-14T23:57:29.728 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3799999999999996e-06 2024-09-14T23:57:29.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.729 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.729 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:29.729 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262101+0000", 2024-09-14T23:57:29.729 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0009999999999998e-06 2024-09-14T23:57:29.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.729 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.729 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:29.729 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317625+0000", 2024-09-14T23:57:29.729 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055524111000000001 2024-09-14T23:57:29.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.729 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.729 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:29.729 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317702+0000", 2024-09-14T23:57:29.729 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7533999999999993e-05 2024-09-14T23:57:29.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.730 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-14T23:57:29.730 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317892+0000", 2024-09-14T23:57:29.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018991400000000001 2024-09-14T23:57:29.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.730 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:57:29.730 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319407+0000", 2024-09-14T23:57:29.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015145250000000001 2024-09-14T23:57:29.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.730 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:29.730 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319481+0000", 2024-09-14T23:57:29.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3826999999999996e-05 2024-09-14T23:57:29.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:29.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319541+0000", 2024-09-14T23:57:29.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0470999999999998e-05 2024-09-14T23:57:29.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:29.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:29.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:29.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319583+0000", 2024-09-14T23:57:29.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1934000000000002e-05 2024-09-14T23:57:29.731 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.731 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:29.731 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.731 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:29.732 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:29.732 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:29.733 INFO:teuthology.orchestra.run.smithi042.stdout:2842 write 0xe759837 thru 0xe75a927 (0x10f1 bytes) 2024-09-14T23:57:29.733 INFO:teuthology.orchestra.run.smithi042.stdout:2844 write 0x5b83b7a thru 0x5b8eafd (0xaf84 bytes) 2024-09-14T23:57:29.733 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:29.733 INFO:teuthology.orchestra.run.smithi042.stdout:2846 read 0xc9183b7 thru 0xc91d2e8 (0x4f32 bytes) 2024-09-14T23:57:29.733 INFO:teuthology.orchestra.run.smithi042.stdout:2847 read 0x322fc85 thru 0x323f561 (0xf8dd bytes) 2024-09-14T23:57:29.733 INFO:teuthology.orchestra.run.smithi042.stdout:2849 write 0xd64305b thru 0xd6486c6 (0x566c bytes) 2024-09-14T23:57:29.734 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T23:57:29.736 INFO:teuthology.orchestra.run.smithi042.stdout:2851 read 0xc3ec1e3 thru 0xc3f0705 (0x4523 bytes) 2024-09-14T23:57:29.736 INFO:teuthology.orchestra.run.smithi042.stdout:2852 read 0xaae648f thru 0xaaeaef9 (0x4a6b bytes) 2024-09-14T23:57:29.736 INFO:teuthology.orchestra.run.smithi042.stdout:2853 write 0xa33b2f0 thru 0xa3484f6 (0xd207 bytes) 2024-09-14T23:57:29.736 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:29.736 INFO:teuthology.orchestra.run.smithi042.stdout:2855 write 0xdd9dbe5 thru 0xdda38e6 (0x5d02 bytes) 2024-09-14T23:57:29.736 INFO:teuthology.orchestra.run.smithi042.stdout:2857 write 0x306340e thru 0x306e22b (0xae1e bytes) 2024-09-14T23:57:29.736 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:29.737 INFO:teuthology.orchestra.run.smithi042.stdout:2860 read 0x5e53b71 thru 0x5e5ae87 (0x7317 bytes) 2024-09-14T23:57:29.737 INFO:teuthology.orchestra.run.smithi042.stdout:2861 trunc from 0xe75a928 to 0x81ae31 2024-09-14T23:57:29.737 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:29.737 INFO:teuthology.orchestra.run.smithi042.stdout:2863 write 0x1578e04 thru 0x157bea2 (0x309f bytes) 2024-09-14T23:57:29.737 INFO:teuthology.orchestra.run.smithi042.stdout:2866 read 0x7b150b thru 0x7b2096 (0xb8c bytes) 2024-09-14T23:57:29.737 INFO:teuthology.orchestra.run.smithi042.stdout:2867 read 0xee8be1 thru 0xef6e4d (0xe26d bytes) 2024-09-14T23:57:29.737 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:29.737 INFO:teuthology.orchestra.run.smithi042.stdout:2870 read 0x135e0a7 thru 0x1367df1 (0x9d4b bytes) 2024-09-14T23:57:29.737 INFO:teuthology.orchestra.run.smithi042.stdout:2872 trunc from 0x157bea3 to 0x4514275 2024-09-14T23:57:29.737 INFO:teuthology.orchestra.run.smithi042.stdout:2873 writesame 0x29ac2ea thru 0x29b4932 data_size 0x7(0x8649 bytes) 2024-09-14T23:57:29.737 INFO:teuthology.orchestra.run.smithi042.stdout:2876 writesame 0xb811b74 thru 0xb81a217 data_size 0xe(0x86a4 bytes) 2024-09-14T23:57:29.737 INFO:teuthology.orchestra.run.smithi042.stdout:2877 read 0xabef89f thru 0xabf8e29 (0x958b bytes) 2024-09-14T23:57:29.737 INFO:teuthology.orchestra.run.smithi042.stdout:2880 writesame 0xc62b840 thru 0xc63644d data_size 0x9(0xac0e bytes) 2024-09-14T23:57:29.737 INFO:teuthology.orchestra.run.smithi042.stdout:2881 trunc from 0xc63644e to 0x2118f5f 2024-09-14T23:57:29.738 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:29.732+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:29.743 INFO:teuthology.orchestra.run.smithi042.stdout:2882 read 0x5ad400 thru 0x5b2165 (0x4d66 bytes) 2024-09-14T23:57:29.745 INFO:teuthology.orchestra.run.smithi042.stdout:2883 trunc from 0x2118f5f to 0x3b27244 2024-09-14T23:57:29.750 INFO:teuthology.orchestra.run.smithi042.stdout:2887 clone 27 order 18 su 131072 sc 4 2024-09-14T23:57:29.834 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:29.832+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:29.846 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:29.846 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:29.846 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:57:29.846 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:29.854 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T23:57:29.934 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:29.933+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:29.961 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:29.961 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:29.961 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:57:29.961 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:57:29.961 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:29.970 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T23:57:29.993 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T23:57:30.045 INFO:teuthology.orchestra.run.smithi042.stdout:flattening image image_client.0-clone26 2024-09-14T23:57:30.054 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:30.053+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:30.109 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-09-14T23:57:30.122 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:30.123 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:57:30.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:57:30.123 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:57:30.123 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.123 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:3500 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T23:57:30.123 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:37.990193+0000", 2024-09-14T23:57:30.123 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 52.130415423000002, 2024-09-14T23:57:30.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99440750899999997, 2024-09-14T23:57:30.123 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:30.123 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:30.123 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:30.123 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:30.123 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:30.123 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:30.124 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3500 2024-09-14T23:57:30.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.124 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:30.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.124 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:30.124 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990193+0000", 2024-09-14T23:57:30.124 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.124 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:30.124 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990193+0000", 2024-09-14T23:57:30.124 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.124 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:30.125 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990194+0000", 2024-09-14T23:57:30.125 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5000000000000001e-07 2024-09-14T23:57:30.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.125 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.125 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:30.125 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990200+0000", 2024-09-14T23:57:30.125 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4389999999999997e-06 2024-09-14T23:57:30.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.125 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.125 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:30.125 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990215+0000", 2024-09-14T23:57:30.125 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4613999999999999e-05 2024-09-14T23:57:30.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.125 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.125 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:30.126 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990224+0000", 2024-09-14T23:57:30.126 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1740000000000006e-06 2024-09-14T23:57:30.126 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.126 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.126 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:30.126 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990283+0000", 2024-09-14T23:57:30.126 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9231999999999998e-05 2024-09-14T23:57:30.126 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.126 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.126 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:57:30.126 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990291+0000", 2024-09-14T23:57:30.126 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5900000000000002e-06 2024-09-14T23:57:30.126 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.126 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.127 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:30.127 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:38.984249+0000", 2024-09-14T23:57:30.127 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99395822700000003 2024-09-14T23:57:30.127 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.127 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.127 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:30.127 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:38.984448+0000", 2024-09-14T23:57:30.127 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00019855900000000001 2024-09-14T23:57:30.127 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.127 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.127 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:30.127 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:38.984601+0000", 2024-09-14T23:57:30.127 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000152824 2024-09-14T23:57:30.127 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.128 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:30.128 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.128 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.128 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4055 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1093632~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:30.128 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099066+0000", 2024-09-14T23:57:30.128 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.021542535000002, 2024-09-14T23:57:30.128 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044174782000000003, 2024-09-14T23:57:30.128 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:30.128 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:30.128 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:30.128 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:30.128 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:30.129 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:30.129 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4055 2024-09-14T23:57:30.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.129 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:30.129 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.129 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:30.129 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099066+0000", 2024-09-14T23:57:30.129 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.129 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.129 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:30.129 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099066+0000", 2024-09-14T23:57:30.129 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.129 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.130 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:30.130 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099066+0000", 2024-09-14T23:57:30.130 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3700000000000001e-07 2024-09-14T23:57:30.130 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.130 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.130 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:30.130 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099067+0000", 2024-09-14T23:57:30.130 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1379999999999999e-06 2024-09-14T23:57:30.130 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.130 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.130 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:30.130 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099073+0000", 2024-09-14T23:57:30.130 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4790000000000004e-06 2024-09-14T23:57:30.130 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.130 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.131 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:30.131 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099075+0000", 2024-09-14T23:57:30.131 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5320000000000001e-06 2024-09-14T23:57:30.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.131 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.131 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:30.131 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143147+0000", 2024-09-14T23:57:30.131 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044071676999999997 2024-09-14T23:57:30.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.131 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.131 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:30.131 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143186+0000", 2024-09-14T23:57:30.131 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8821e-05 2024-09-14T23:57:30.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.131 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.132 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:30.132 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143241+0000", 2024-09-14T23:57:30.132 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4797999999999999e-05 2024-09-14T23:57:30.132 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.132 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:30.132 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.132 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.132 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4056 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1155072~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:30.132 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099079+0000", 2024-09-14T23:57:30.132 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.021529819999998, 2024-09-14T23:57:30.132 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044295020999999997, 2024-09-14T23:57:30.132 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:30.132 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:30.133 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:30.133 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:30.133 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:30.133 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:30.133 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4056 2024-09-14T23:57:30.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.133 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:30.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.133 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:30.133 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099079+0000", 2024-09-14T23:57:30.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.133 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:30.133 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099079+0000", 2024-09-14T23:57:30.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:30.134 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099079+0000", 2024-09-14T23:57:30.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1500000000000001e-07 2024-09-14T23:57:30.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:30.134 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099080+0000", 2024-09-14T23:57:30.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0210000000000001e-06 2024-09-14T23:57:30.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:30.134 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099083+0000", 2024-09-14T23:57:30.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.174e-06 2024-09-14T23:57:30.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.135 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:30.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099086+0000", 2024-09-14T23:57:30.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.446e-06 2024-09-14T23:57:30.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.135 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:30.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143269+0000", 2024-09-14T23:57:30.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044183489999999999 2024-09-14T23:57:30.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.135 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:30.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143314+0000", 2024-09-14T23:57:30.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5135000000000001e-05 2024-09-14T23:57:30.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.136 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.136 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:30.136 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143374+0000", 2024-09-14T23:57:30.136 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9540000000000003e-05 2024-09-14T23:57:30.136 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.136 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:30.136 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.136 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.136 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4057 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1216512~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:30.136 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099088+0000", 2024-09-14T23:57:30.136 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.021520033000002, 2024-09-14T23:57:30.136 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044406119000000001, 2024-09-14T23:57:30.136 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:30.137 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:30.137 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:30.137 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:30.137 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:30.137 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:30.137 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4057 2024-09-14T23:57:30.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.137 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:30.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:30.137 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099088+0000", 2024-09-14T23:57:30.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:30.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099088+0000", 2024-09-14T23:57:30.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:30.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099089+0000", 2024-09-14T23:57:30.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.04e-07 2024-09-14T23:57:30.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:30.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099090+0000", 2024-09-14T23:57:30.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.026e-06 2024-09-14T23:57:30.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.139 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:30.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099093+0000", 2024-09-14T23:57:30.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0819999999999999e-06 2024-09-14T23:57:30.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.139 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:30.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099095+0000", 2024-09-14T23:57:30.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4930000000000001e-06 2024-09-14T23:57:30.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.139 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:30.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143400+0000", 2024-09-14T23:57:30.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044304698000000003 2024-09-14T23:57:30.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.139 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:30.140 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143441+0000", 2024-09-14T23:57:30.140 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1100000000000003e-05 2024-09-14T23:57:30.140 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.140 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.140 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:30.140 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143495+0000", 2024-09-14T23:57:30.140 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3516000000000001e-05 2024-09-14T23:57:30.140 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.140 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:30.140 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.140 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.140 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.140 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4058 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1277952~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:30.140 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099099+0000", 2024-09-14T23:57:30.140 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.021509541999997, 2024-09-14T23:57:30.141 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044514583000000003, 2024-09-14T23:57:30.141 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:30.141 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:30.141 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:30.141 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:30.141 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:30.141 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:30.141 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4058 2024-09-14T23:57:30.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.141 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:30.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.141 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:30.141 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099099+0000", 2024-09-14T23:57:30.141 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.142 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:30.142 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099099+0000", 2024-09-14T23:57:30.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.142 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:30.142 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099099+0000", 2024-09-14T23:57:30.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9999999999999999e-07 2024-09-14T23:57:30.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.142 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:30.142 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099100+0000", 2024-09-14T23:57:30.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7199999999999997e-07 2024-09-14T23:57:30.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:30.143 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099103+0000", 2024-09-14T23:57:30.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.095e-06 2024-09-14T23:57:30.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:30.143 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099106+0000", 2024-09-14T23:57:30.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4229999999999999e-06 2024-09-14T23:57:30.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:30.143 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143524+0000", 2024-09-14T23:57:30.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044417891000000001 2024-09-14T23:57:30.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.144 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.144 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:30.144 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143568+0000", 2024-09-14T23:57:30.144 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4060000000000002e-05 2024-09-14T23:57:30.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.144 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.144 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:30.144 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143614+0000", 2024-09-14T23:57:30.144 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5942000000000002e-05 2024-09-14T23:57:30.144 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.144 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:30.144 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.144 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.145 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4059 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1339392~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:30.145 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099114+0000", 2024-09-14T23:57:30.145 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.021495002999998, 2024-09-14T23:57:30.145 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044617194999999998, 2024-09-14T23:57:30.145 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:30.145 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:30.145 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:30.145 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:30.145 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:30.145 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:30.145 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4059 2024-09-14T23:57:30.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.145 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:30.145 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.145 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:30.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099114+0000", 2024-09-14T23:57:30.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.146 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:30.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099114+0000", 2024-09-14T23:57:30.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.146 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:30.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099114+0000", 2024-09-14T23:57:30.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5899999999999998e-07 2024-09-14T23:57:30.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.146 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:30.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099115+0000", 2024-09-14T23:57:30.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3510000000000001e-06 2024-09-14T23:57:30.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.147 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:30.147 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099119+0000", 2024-09-14T23:57:30.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.16e-06 2024-09-14T23:57:30.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.147 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:30.147 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099123+0000", 2024-09-14T23:57:30.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1339999999999997e-06 2024-09-14T23:57:30.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.147 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:30.148 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143643+0000", 2024-09-14T23:57:30.148 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044519232999999998 2024-09-14T23:57:30.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.148 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.148 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:30.148 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143686+0000", 2024-09-14T23:57:30.148 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3349000000000001e-05 2024-09-14T23:57:30.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.148 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.148 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:30.148 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143731+0000", 2024-09-14T23:57:30.148 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4709000000000002e-05 2024-09-14T23:57:30.148 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.148 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:30.148 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.149 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4060 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1400832~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:30.149 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099129+0000", 2024-09-14T23:57:30.149 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.02147978, 2024-09-14T23:57:30.149 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044721271999999999, 2024-09-14T23:57:30.149 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:30.149 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:30.149 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:30.149 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:30.149 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:30.149 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:30.149 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4060 2024-09-14T23:57:30.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.149 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:30.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:30.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099129+0000", 2024-09-14T23:57:30.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:30.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099129+0000", 2024-09-14T23:57:30.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:30.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099191+0000", 2024-09-14T23:57:30.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1805999999999997e-05 2024-09-14T23:57:30.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:30.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099193+0000", 2024-09-14T23:57:30.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2330000000000001e-06 2024-09-14T23:57:30.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:30.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099200+0000", 2024-09-14T23:57:30.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4200000000000001e-06 2024-09-14T23:57:30.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.152 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:30.152 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099204+0000", 2024-09-14T23:57:30.152 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7500000000000001e-06 2024-09-14T23:57:30.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.152 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:30.152 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143760+0000", 2024-09-14T23:57:30.152 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044556182 2024-09-14T23:57:30.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.152 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:30.153 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143804+0000", 2024-09-14T23:57:30.153 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4239000000000001e-05 2024-09-14T23:57:30.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.153 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.153 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:30.153 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143850+0000", 2024-09-14T23:57:30.153 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5642000000000001e-05 2024-09-14T23:57:30.153 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.153 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:30.153 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.153 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.153 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4061 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1462272~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:30.153 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099215+0000", 2024-09-14T23:57:30.154 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.021393654999997, 2024-09-14T23:57:30.154 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044745659, 2024-09-14T23:57:30.154 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:30.154 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:30.154 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:30.154 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:30.154 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:30.154 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:30.154 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4061 2024-09-14T23:57:30.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.154 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:30.154 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.154 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:30.154 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099215+0000", 2024-09-14T23:57:30.154 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:30.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099215+0000", 2024-09-14T23:57:30.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:30.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100672+0000", 2024-09-14T23:57:30.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00145689 2024-09-14T23:57:30.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:30.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100677+0000", 2024-09-14T23:57:30.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0139999999999998e-06 2024-09-14T23:57:30.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.156 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:30.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100689+0000", 2024-09-14T23:57:30.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.183e-05 2024-09-14T23:57:30.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.156 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:30.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100696+0000", 2024-09-14T23:57:30.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2599999999999999e-06 2024-09-14T23:57:30.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.156 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:30.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143877+0000", 2024-09-14T23:57:30.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043181279000000003 2024-09-14T23:57:30.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.157 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.157 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:30.157 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143919+0000", 2024-09-14T23:57:30.157 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1783000000000003e-05 2024-09-14T23:57:30.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.157 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.157 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:30.157 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143961+0000", 2024-09-14T23:57:30.157 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1603000000000001e-05 2024-09-14T23:57:30.157 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.157 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:30.157 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.158 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.158 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4070 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 2015232~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:30.158 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.100903+0000", 2024-09-14T23:57:30.158 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.019705946999999, 2024-09-14T23:57:30.158 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044103815999999997, 2024-09-14T23:57:30.158 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:30.158 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:30.158 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:30.158 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:30.158 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:30.158 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:30.158 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4070 2024-09-14T23:57:30.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.158 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:30.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.159 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:30.159 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100903+0000", 2024-09-14T23:57:30.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.159 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:30.159 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100903+0000", 2024-09-14T23:57:30.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.159 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:30.159 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.102671+0000", 2024-09-14T23:57:30.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001768279 2024-09-14T23:57:30.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.160 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.160 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:30.160 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.102675+0000", 2024-09-14T23:57:30.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9609999999999999e-06 2024-09-14T23:57:30.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.160 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.160 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:30.160 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.102684+0000", 2024-09-14T23:57:30.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.329e-06 2024-09-14T23:57:30.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.160 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.160 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:30.160 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.102689+0000", 2024-09-14T23:57:30.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7079999999999996e-06 2024-09-14T23:57:30.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.161 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.161 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:30.161 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.144912+0000", 2024-09-14T23:57:30.161 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.042223305000000003 2024-09-14T23:57:30.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.161 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.161 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:30.161 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.144960+0000", 2024-09-14T23:57:30.161 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7670000000000003e-05 2024-09-14T23:57:30.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.161 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.161 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:30.161 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.145006+0000", 2024-09-14T23:57:30.161 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6563999999999997e-05 2024-09-14T23:57:30.162 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.162 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:30.162 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.162 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.162 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.162 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4096 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1282048~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:30.162 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.104957+0000", 2024-09-14T23:57:30.162 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.015651773999998, 2024-09-14T23:57:30.162 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044093016999999998, 2024-09-14T23:57:30.162 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:30.162 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:30.162 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:30.162 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:30.162 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:30.162 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:30.162 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4096 2024-09-14T23:57:30.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.163 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:30.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.163 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:30.163 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104957+0000", 2024-09-14T23:57:30.163 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.163 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:30.163 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104957+0000", 2024-09-14T23:57:30.163 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.163 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:30.164 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104957+0000", 2024-09-14T23:57:30.164 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7400000000000003e-07 2024-09-14T23:57:30.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.164 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.164 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:30.164 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104960+0000", 2024-09-14T23:57:30.164 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3800000000000001e-06 2024-09-14T23:57:30.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.164 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.164 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:30.164 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104967+0000", 2024-09-14T23:57:30.164 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4030000000000001e-06 2024-09-14T23:57:30.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.165 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.165 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:30.165 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104973+0000", 2024-09-14T23:57:30.165 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7370000000000003e-06 2024-09-14T23:57:30.165 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.165 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.165 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:30.165 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.148883+0000", 2024-09-14T23:57:30.165 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043910124000000002 2024-09-14T23:57:30.165 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.165 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.165 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:30.165 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.148966+0000", 2024-09-14T23:57:30.165 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2868999999999996e-05 2024-09-14T23:57:30.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.166 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.166 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:30.166 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149050+0000", 2024-09-14T23:57:30.166 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3930000000000004e-05 2024-09-14T23:57:30.166 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.166 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:30.166 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.166 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.166 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4097 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1343488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:30.166 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.104983+0000", 2024-09-14T23:57:30.166 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.015625985, 2024-09-14T23:57:30.166 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044182104999999999, 2024-09-14T23:57:30.167 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:30.167 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:30.167 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:30.167 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:30.167 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:30.167 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:30.167 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4097 2024-09-14T23:57:30.167 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.167 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:30.167 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.167 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:30.167 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104983+0000", 2024-09-14T23:57:30.167 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.167 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.168 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.168 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:30.168 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104983+0000", 2024-09-14T23:57:30.168 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.168 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.168 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:30.168 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105207+0000", 2024-09-14T23:57:30.168 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000224687 2024-09-14T23:57:30.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.168 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.168 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:30.168 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105211+0000", 2024-09-14T23:57:30.168 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3050000000000001e-06 2024-09-14T23:57:30.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.169 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:30.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105217+0000", 2024-09-14T23:57:30.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8170000000000001e-06 2024-09-14T23:57:30.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.169 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:30.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105222+0000", 2024-09-14T23:57:30.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.267e-06 2024-09-14T23:57:30.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.169 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:30.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149079+0000", 2024-09-14T23:57:30.170 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043857581999999999 2024-09-14T23:57:30.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.170 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.170 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:30.170 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149122+0000", 2024-09-14T23:57:30.170 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2899999999999999e-05 2024-09-14T23:57:30.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.170 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.170 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:30.170 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149165+0000", 2024-09-14T23:57:30.170 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2546999999999998e-05 2024-09-14T23:57:30.170 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.170 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:30.170 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.171 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.171 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4098 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1404928~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:30.171 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.105242+0000", 2024-09-14T23:57:30.171 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.015366172999997, 2024-09-14T23:57:30.171 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044121104000000001, 2024-09-14T23:57:30.171 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:30.171 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:30.171 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:30.171 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:30.171 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:30.171 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:30.171 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4098 2024-09-14T23:57:30.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.171 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:30.171 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.172 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:30.172 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105242+0000", 2024-09-14T23:57:30.172 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.172 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.172 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.172 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:30.172 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105242+0000", 2024-09-14T23:57:30.172 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.172 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.172 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.172 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:30.172 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105243+0000", 2024-09-14T23:57:30.172 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0399999999999995e-07 2024-09-14T23:57:30.172 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.173 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.173 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:30.173 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105246+0000", 2024-09-14T23:57:30.173 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6309999999999999e-06 2024-09-14T23:57:30.173 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.173 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.173 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:30.173 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105253+0000", 2024-09-14T23:57:30.173 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0940000000000002e-06 2024-09-14T23:57:30.173 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.173 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.173 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:30.173 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105260+0000", 2024-09-14T23:57:30.173 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1799999999999999e-06 2024-09-14T23:57:30.173 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.174 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.174 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:30.174 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149201+0000", 2024-09-14T23:57:30.174 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043940629000000002 2024-09-14T23:57:30.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.174 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.174 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:30.174 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149283+0000", 2024-09-14T23:57:30.174 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2360000000000004e-05 2024-09-14T23:57:30.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.174 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.174 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:30.174 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149363+0000", 2024-09-14T23:57:30.174 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0506000000000005e-05 2024-09-14T23:57:30.175 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.175 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:30.175 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.175 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.175 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4099 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1466368~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:30.175 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.105270+0000", 2024-09-14T23:57:30.175 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.015338577999998, 2024-09-14T23:57:30.175 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044195433999999999, 2024-09-14T23:57:30.175 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:30.175 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:30.175 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:30.175 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:30.175 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:30.176 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:30.176 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4099 2024-09-14T23:57:30.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.176 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:30.176 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.176 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:30.176 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105270+0000", 2024-09-14T23:57:30.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.176 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.176 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:30.176 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105270+0000", 2024-09-14T23:57:30.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.177 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:30.177 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105429+0000", 2024-09-14T23:57:30.177 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015940299999999999 2024-09-14T23:57:30.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.177 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:30.177 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105433+0000", 2024-09-14T23:57:30.177 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3579999999999999e-06 2024-09-14T23:57:30.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.177 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:30.177 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105439+0000", 2024-09-14T23:57:30.178 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1759999999999998e-06 2024-09-14T23:57:30.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.178 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:30.178 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105445+0000", 2024-09-14T23:57:30.178 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9689999999999999e-06 2024-09-14T23:57:30.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.178 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:30.178 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149381+0000", 2024-09-14T23:57:30.178 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043936389999999999 2024-09-14T23:57:30.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.178 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:30.179 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149422+0000", 2024-09-14T23:57:30.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0865000000000003e-05 2024-09-14T23:57:30.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.179 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.179 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:30.179 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149465+0000", 2024-09-14T23:57:30.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3272999999999997e-05 2024-09-14T23:57:30.179 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.179 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:30.179 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.179 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.179 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4100 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1527808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:30.179 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.105454+0000", 2024-09-14T23:57:30.179 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.015154107000001, 2024-09-14T23:57:30.180 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044217411999999998, 2024-09-14T23:57:30.180 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:30.180 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:30.180 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:30.180 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:30.180 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:30.180 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:30.180 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4100 2024-09-14T23:57:30.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.180 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:30.180 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.180 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:30.180 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105454+0000", 2024-09-14T23:57:30.180 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.181 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.181 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.181 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:30.181 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105454+0000", 2024-09-14T23:57:30.181 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.181 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.181 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.181 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:30.181 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105578+0000", 2024-09-14T23:57:30.181 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012401799999999999 2024-09-14T23:57:30.181 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.181 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.181 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:30.181 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105583+0000", 2024-09-14T23:57:30.182 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1779999999999996e-06 2024-09-14T23:57:30.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.182 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.182 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:30.182 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105590+0000", 2024-09-14T23:57:30.182 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4279999999999999e-06 2024-09-14T23:57:30.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.182 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.182 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:30.182 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105596+0000", 2024-09-14T23:57:30.182 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0120000000000002e-06 2024-09-14T23:57:30.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.182 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.182 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:30.183 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149503+0000", 2024-09-14T23:57:30.183 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043906580000000001 2024-09-14T23:57:30.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.183 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.183 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:30.183 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149585+0000", 2024-09-14T23:57:30.183 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2356999999999998e-05 2024-09-14T23:57:30.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.183 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.183 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:30.183 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149672+0000", 2024-09-14T23:57:30.183 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6839e-05 2024-09-14T23:57:30.183 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.183 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:30.183 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.184 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4101 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1589248~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:30.184 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.105609+0000", 2024-09-14T23:57:30.184 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.014999568, 2024-09-14T23:57:30.184 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044175987999999999, 2024-09-14T23:57:30.184 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:30.184 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:30.184 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:30.184 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:30.184 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:30.184 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:30.184 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4101 2024-09-14T23:57:30.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.184 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:30.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.185 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:30.185 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105609+0000", 2024-09-14T23:57:30.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.185 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.185 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:30.185 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105609+0000", 2024-09-14T23:57:30.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.185 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.185 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:30.185 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.106671+0000", 2024-09-14T23:57:30.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010619270000000001 2024-09-14T23:57:30.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.186 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.186 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:30.186 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.106675+0000", 2024-09-14T23:57:30.186 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4429999999999996e-06 2024-09-14T23:57:30.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.186 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.186 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:30.186 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.106685+0000", 2024-09-14T23:57:30.186 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7429999999999997e-06 2024-09-14T23:57:30.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.186 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.186 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:30.186 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.106690+0000", 2024-09-14T23:57:30.187 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.293e-06 2024-09-14T23:57:30.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.187 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.187 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:30.187 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149700+0000", 2024-09-14T23:57:30.187 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043009512 2024-09-14T23:57:30.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.187 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.187 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:30.187 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149742+0000", 2024-09-14T23:57:30.187 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2083999999999999e-05 2024-09-14T23:57:30.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.187 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.188 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:30.188 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149785+0000", 2024-09-14T23:57:30.188 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2985999999999998e-05 2024-09-14T23:57:30.188 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.188 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:30.188 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.188 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.188 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4116 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 180224~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:30.188 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.107575+0000", 2024-09-14T23:57:30.188 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.013033100000001, 2024-09-14T23:57:30.188 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044157522999999997, 2024-09-14T23:57:30.188 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:30.189 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:30.189 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:30.189 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:30.189 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:30.189 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:30.189 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4116 2024-09-14T23:57:30.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.189 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:30.189 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.189 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:30.189 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.107575+0000", 2024-09-14T23:57:30.189 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.189 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.190 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:30.190 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.107575+0000", 2024-09-14T23:57:30.190 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.190 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.190 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:30.190 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108671+0000", 2024-09-14T23:57:30.190 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010954510000000001 2024-09-14T23:57:30.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.190 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.190 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:30.190 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108676+0000", 2024-09-14T23:57:30.190 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6639999999999997e-06 2024-09-14T23:57:30.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.191 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.191 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:30.191 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108685+0000", 2024-09-14T23:57:30.191 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1130000000000007e-06 2024-09-14T23:57:30.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.191 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.191 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:30.191 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108691+0000", 2024-09-14T23:57:30.191 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6019999999999996e-06 2024-09-14T23:57:30.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.191 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.191 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:30.191 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.151530+0000", 2024-09-14T23:57:30.191 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.042838345999999999 2024-09-14T23:57:30.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.192 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.192 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:30.192 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.151619+0000", 2024-09-14T23:57:30.192 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9302999999999995e-05 2024-09-14T23:57:30.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.192 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.192 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:30.192 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.151733+0000", 2024-09-14T23:57:30.192 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011404399999999999 2024-09-14T23:57:30.192 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.192 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:30.192 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.192 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.193 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4122 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 548864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:30.193 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.108842+0000", 2024-09-14T23:57:30.193 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.011766262000002, 2024-09-14T23:57:30.193 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044103631999999997, 2024-09-14T23:57:30.193 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:30.193 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:30.193 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:30.193 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:30.193 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:30.193 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:30.194 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4122 2024-09-14T23:57:30.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.194 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:30.194 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.194 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:30.194 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108842+0000", 2024-09-14T23:57:30.194 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.194 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.194 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:30.194 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108842+0000", 2024-09-14T23:57:30.194 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.195 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.195 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:30.195 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108843+0000", 2024-09-14T23:57:30.195 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7000000000000001e-07 2024-09-14T23:57:30.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.195 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.195 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:30.195 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108844+0000", 2024-09-14T23:57:30.195 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6479999999999999e-06 2024-09-14T23:57:30.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.195 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.195 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:30.195 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108848+0000", 2024-09-14T23:57:30.195 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8029999999999998e-06 2024-09-14T23:57:30.196 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.196 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.196 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:30.196 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108851+0000", 2024-09-14T23:57:30.196 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7939999999999998e-06 2024-09-14T23:57:30.196 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.196 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.196 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:30.196 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.152838+0000", 2024-09-14T23:57:30.196 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043987351000000001 2024-09-14T23:57:30.196 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.196 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.196 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:30.196 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.152885+0000", 2024-09-14T23:57:30.197 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7237000000000002e-05 2024-09-14T23:57:30.197 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.197 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.197 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:30.197 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.152946+0000", 2024-09-14T23:57:30.197 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0529000000000003e-05 2024-09-14T23:57:30.197 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.197 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:30.197 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.197 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.197 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.197 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4123 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 610304~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:30.197 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.108855+0000", 2024-09-14T23:57:30.197 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.011753933000001, 2024-09-14T23:57:30.198 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044321610999999997, 2024-09-14T23:57:30.198 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:30.198 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:30.198 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:30.198 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:30.198 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:30.198 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:30.198 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4123 2024-09-14T23:57:30.198 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.198 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:30.198 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.198 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:30.198 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108855+0000", 2024-09-14T23:57:30.198 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.198 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.198 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.199 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:30.199 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108855+0000", 2024-09-14T23:57:30.199 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.199 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.199 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:30.199 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.110671+0000", 2024-09-14T23:57:30.199 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018159719999999999 2024-09-14T23:57:30.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.199 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.199 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:30.199 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.110676+0000", 2024-09-14T23:57:30.199 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9450000000000001e-06 2024-09-14T23:57:30.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.199 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.200 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:30.200 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.110685+0000", 2024-09-14T23:57:30.200 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5359999999999998e-06 2024-09-14T23:57:30.200 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.200 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.200 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:30.200 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.110692+0000", 2024-09-14T23:57:30.200 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9280000000000004e-06 2024-09-14T23:57:30.200 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.200 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.200 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:30.200 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.152984+0000", 2024-09-14T23:57:30.200 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.042291677999999999 2024-09-14T23:57:30.200 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.201 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.201 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:30.201 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.153066+0000", 2024-09-14T23:57:30.201 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2657999999999994e-05 2024-09-14T23:57:30.201 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.201 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.201 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:30.201 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.153176+0000", 2024-09-14T23:57:30.201 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000109894 2024-09-14T23:57:30.201 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.201 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:30.201 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.201 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.201 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.202 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4136 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:30.202 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.111056+0000", 2024-09-14T23:57:30.202 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.009552784999997, 2024-09-14T23:57:30.202 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044097428000000001, 2024-09-14T23:57:30.202 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:30.202 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:30.202 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:30.202 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:30.202 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:30.202 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:30.202 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4136 2024-09-14T23:57:30.202 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.202 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:30.202 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.202 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:30.202 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.111056+0000", 2024-09-14T23:57:30.203 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.203 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.203 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:30.203 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.111056+0000", 2024-09-14T23:57:30.203 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.203 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.203 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:30.203 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.112671+0000", 2024-09-14T23:57:30.203 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016153999999999999 2024-09-14T23:57:30.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.203 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.203 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:30.204 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.112675+0000", 2024-09-14T23:57:30.204 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2830000000000003e-06 2024-09-14T23:57:30.204 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.204 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.204 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:30.204 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.112684+0000", 2024-09-14T23:57:30.204 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7800000000000006e-06 2024-09-14T23:57:30.204 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.204 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.204 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:30.204 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.112689+0000", 2024-09-14T23:57:30.204 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0000000000000004e-06 2024-09-14T23:57:30.204 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.204 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.205 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:30.205 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.155064+0000", 2024-09-14T23:57:30.205 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.042374423000000001 2024-09-14T23:57:30.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.205 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.205 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:30.205 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.155111+0000", 2024-09-14T23:57:30.205 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7367000000000002e-05 2024-09-14T23:57:30.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.205 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.205 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:30.205 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.155153+0000", 2024-09-14T23:57:30.205 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2175000000000002e-05 2024-09-14T23:57:30.205 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.206 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:30.206 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.206 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.206 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.206 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:46572 3.1d 3:b9e1f524:::rbd_header.1118cee06433:head [watch reconnect cookie 140670782499824 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-14T23:57:30.206 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:57.046868+0000", 2024-09-14T23:57:30.206 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 33.073740979, 2024-09-14T23:57:30.206 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0243412620000001, 2024-09-14T23:57:30.206 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:30.206 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:30.206 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:30.206 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:30.206 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:30.206 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:30.207 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 46572 2024-09-14T23:57:30.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.207 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:30.207 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.207 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:30.207 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046868+0000", 2024-09-14T23:57:30.207 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.207 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.207 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:30.207 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046868+0000", 2024-09-14T23:57:30.207 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.207 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.207 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:30.208 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046870+0000", 2024-09-14T23:57:30.208 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.204e-06 2024-09-14T23:57:30.208 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.208 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.208 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:30.208 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046877+0000", 2024-09-14T23:57:30.208 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5100000000000001e-06 2024-09-14T23:57:30.208 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.208 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.208 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:30.208 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046898+0000", 2024-09-14T23:57:30.208 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1149999999999999e-05 2024-09-14T23:57:30.208 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.208 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.208 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:30.209 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046911+0000", 2024-09-14T23:57:30.209 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2234e-05 2024-09-14T23:57:30.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.209 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.209 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:30.209 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046988+0000", 2024-09-14T23:57:30.209 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7445000000000001e-05 2024-09-14T23:57:30.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.209 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.209 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:57:30.209 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.047001+0000", 2024-09-14T23:57:30.209 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2951e-05 2024-09-14T23:57:30.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.209 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.210 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:30.210 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.069287+0000", 2024-09-14T23:57:30.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.022285748 2024-09-14T23:57:30.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.210 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for rw locks", 2024-09-14T23:57:30.210 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.069397+0000", 2024-09-14T23:57:30.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000110326 2024-09-14T23:57:30.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.210 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:30.210 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.069997+0000", 2024-09-14T23:57:30.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00059984399999999998 2024-09-14T23:57:30.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.211 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:30.211 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.070109+0000", 2024-09-14T23:57:30.211 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000112367 2024-09-14T23:57:30.211 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.211 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.211 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:30.211 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.071209+0000", 2024-09-14T23:57:30.211 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010994830000000001 2024-09-14T23:57:30.211 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.211 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:30.211 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.211 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.211 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:30.211 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:30.213 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T23:57:30.213 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:30.211+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:30.232 INFO:teuthology.orchestra.run.smithi042.stdout:2888 trunc from 0x3b27244 to 0xdf54e9 2024-09-14T23:57:30.268 INFO:teuthology.orchestra.run.smithi042.stdout:2889 write 0xedbf2d9 thru 0xedc6fc8 (0x7cf0 bytes) 2024-09-14T23:57:30.272 INFO:teuthology.orchestra.run.smithi042.stdout:2890 write 0x108c9f4 thru 0x108eebb (0x24c8 bytes) 2024-09-14T23:57:30.290 INFO:teuthology.orchestra.run.smithi042.stdout:2891 trunc from 0xedc6fc9 to 0x325a87d 2024-09-14T23:57:30.300 INFO:teuthology.orchestra.run.smithi042.stdout:2893 read 0x4b7cb5 thru 0x4c6fca (0xf316 bytes) 2024-09-14T23:57:30.301 INFO:teuthology.orchestra.run.smithi042.stdout:2894 read 0x315168d thru 0x315e8a1 (0xd215 bytes) 2024-09-14T23:57:30.301 INFO:teuthology.orchestra.run.smithi042.stdout:2896 punch from 0x11d934f to 0x11e276b, (0x941c bytes) 2024-09-14T23:57:30.301 INFO:teuthology.orchestra.run.smithi042.stdout:2897 clone 28 order 21 su 262144 sc 4 2024-09-14T23:57:30.313 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:30.312+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:30.366 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:30.366 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:30.366 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:57:30.366 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:30.375 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T23:57:30.415 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:30.413+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:30.482 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:30.483 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:30.483 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:57:30.483 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:57:30.483 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:30.491 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T23:57:30.535 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:30.534+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:30.600 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:30.600 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:57:30.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:57:30.601 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:57:30.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.601 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4284.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-14T23:57:30.601 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:57:30.601 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 68.073340559000002, 2024-09-14T23:57:30.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.004942086, 2024-09-14T23:57:30.601 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:30.601 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:30.601 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:30.601 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:30.602 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4284", 2024-09-14T23:57:30.602 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/4278347926", 2024-09-14T23:57:30.602 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1 2024-09-14T23:57:30.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.602 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:30.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.602 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:30.602 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:57:30.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:30.603 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:57:30.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:30.603 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524331+0000", 2024-09-14T23:57:30.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.383e-05 2024-09-14T23:57:30.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:30.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524339+0000", 2024-09-14T23:57:30.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8290000000000008e-06 2024-09-14T23:57:30.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:30.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524359+0000", 2024-09-14T23:57:30.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9998000000000001e-05 2024-09-14T23:57:30.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:30.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524373+0000", 2024-09-14T23:57:30.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4304000000000001e-05 2024-09-14T23:57:30.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:30.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524488+0000", 2024-09-14T23:57:30.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114663 2024-09-14T23:57:30.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.606 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:57:30.606 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524501+0000", 2024-09-14T23:57:30.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3349999999999999e-05 2024-09-14T23:57:30.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.606 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:30.606 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.526868+0000", 2024-09-14T23:57:30.724 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023670090000000002 2024-09-14T23:57:30.725 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.725 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.725 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:30.725 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.527041+0000", 2024-09-14T23:57:30.725 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000172597 2024-09-14T23:57:30.725 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.725 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.725 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-14T23:57:30.725 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.527384+0000", 2024-09-14T23:57:30.725 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00034309500000000002 2024-09-14T23:57:30.725 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.725 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.726 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:30.726 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529103+0000", 2024-09-14T23:57:30.726 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017185220000000001 2024-09-14T23:57:30.726 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.726 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.726 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:57:30.726 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529188+0000", 2024-09-14T23:57:30.726 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5591000000000001e-05 2024-09-14T23:57:30.726 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.726 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.726 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:30.726 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529227+0000", 2024-09-14T23:57:30.726 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8563000000000003e-05 2024-09-14T23:57:30.726 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.727 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.727 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:30.727 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529259+0000", 2024-09-14T23:57:30.727 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2734999999999997e-05 2024-09-14T23:57:30.727 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.727 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:30.727 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.727 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.727 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.727 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4284.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-14T23:57:30.727 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:23.530683+0000", 2024-09-14T23:57:30.727 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 67.066974893999998, 2024-09-14T23:57:30.727 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0036133110000000001, 2024-09-14T23:57:30.727 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:30.728 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:30.728 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:30.728 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:30.728 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4284", 2024-09-14T23:57:30.728 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/4278347926", 2024-09-14T23:57:30.728 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5 2024-09-14T23:57:30.728 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.728 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:30.728 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.728 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:30.728 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530683+0000", 2024-09-14T23:57:30.728 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.728 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.728 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.729 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:30.729 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530683+0000", 2024-09-14T23:57:30.729 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.729 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.729 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.729 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:30.729 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530685+0000", 2024-09-14T23:57:30.729 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.075e-06 2024-09-14T23:57:30.729 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.729 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.729 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:30.730 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530695+0000", 2024-09-14T23:57:30.730 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0261999999999999e-05 2024-09-14T23:57:30.730 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.730 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.730 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:30.730 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530715+0000", 2024-09-14T23:57:30.730 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9930000000000001e-05 2024-09-14T23:57:30.730 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.730 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.730 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:30.730 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530724+0000", 2024-09-14T23:57:30.730 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.901e-06 2024-09-14T23:57:30.731 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.731 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.731 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:30.731 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530800+0000", 2024-09-14T23:57:30.731 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5873999999999999e-05 2024-09-14T23:57:30.731 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.731 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.731 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-14T23:57:30.731 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530807+0000", 2024-09-14T23:57:30.731 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2389999999999996e-06 2024-09-14T23:57:30.731 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.731 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.731 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:30.731 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.532494+0000", 2024-09-14T23:57:30.732 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016871830000000001 2024-09-14T23:57:30.732 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.732 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.732 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:30.732 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.532599+0000", 2024-09-14T23:57:30.732 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010472 2024-09-14T23:57:30.732 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.732 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.732 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1", 2024-09-14T23:57:30.732 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.532715+0000", 2024-09-14T23:57:30.732 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011568199999999999 2024-09-14T23:57:30.732 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.732 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.732 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:57:30.733 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.533460+0000", 2024-09-14T23:57:30.733 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00074494999999999997 2024-09-14T23:57:30.733 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.733 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.733 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:30.733 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534230+0000", 2024-09-14T23:57:30.733 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00077036599999999998 2024-09-14T23:57:30.733 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.733 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.733 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:30.733 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534266+0000", 2024-09-14T23:57:30.733 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5725000000000001e-05 2024-09-14T23:57:30.733 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.733 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.734 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:30.734 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534296+0000", 2024-09-14T23:57:30.734 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0403999999999999e-05 2024-09-14T23:57:30.734 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.734 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:30.734 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.734 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.734 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.734 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4284.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-14T23:57:30.734 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:24.535160+0000", 2024-09-14T23:57:30.734 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.062498253000001, 2024-09-14T23:57:30.734 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033473650000000001, 2024-09-14T23:57:30.735 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:30.735 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:30.735 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:30.735 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:30.735 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4284", 2024-09-14T23:57:30.735 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/4278347926", 2024-09-14T23:57:30.735 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 7 2024-09-14T23:57:30.735 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.735 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:30.735 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.735 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:30.735 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535160+0000", 2024-09-14T23:57:30.735 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.736 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.736 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.736 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:30.736 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535160+0000", 2024-09-14T23:57:30.736 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.736 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.736 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.736 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:30.736 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535162+0000", 2024-09-14T23:57:30.736 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3369999999999998e-06 2024-09-14T23:57:30.736 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.736 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.736 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:30.736 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535171+0000", 2024-09-14T23:57:30.737 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7700000000000007e-06 2024-09-14T23:57:30.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.737 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.737 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:30.737 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535189+0000", 2024-09-14T23:57:30.737 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7938e-05 2024-09-14T23:57:30.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.737 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.737 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:30.737 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535197+0000", 2024-09-14T23:57:30.737 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5639999999999997e-06 2024-09-14T23:57:30.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.737 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.737 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:30.738 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535266+0000", 2024-09-14T23:57:30.738 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8926999999999999e-05 2024-09-14T23:57:30.738 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.738 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.738 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-14T23:57:30.738 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535271+0000", 2024-09-14T23:57:30.738 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0329999999999999e-06 2024-09-14T23:57:30.738 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.738 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.738 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:30.738 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.537064+0000", 2024-09-14T23:57:30.738 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001792929 2024-09-14T23:57:30.738 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.738 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.739 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:30.739 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.537131+0000", 2024-09-14T23:57:30.739 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6605000000000004e-05 2024-09-14T23:57:30.739 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.739 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.739 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1", 2024-09-14T23:57:30.739 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.537271+0000", 2024-09-14T23:57:30.739 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013992400000000001 2024-09-14T23:57:30.739 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.739 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.739 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:57:30.739 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.538093+0000", 2024-09-14T23:57:30.739 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00082210200000000001 2024-09-14T23:57:30.740 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.740 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.740 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:30.740 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.538439+0000", 2024-09-14T23:57:30.740 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00034595999999999997 2024-09-14T23:57:30.740 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.740 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.740 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:30.740 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.538476+0000", 2024-09-14T23:57:30.740 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6770000000000002e-05 2024-09-14T23:57:30.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.741 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.741 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:30.741 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.538507+0000", 2024-09-14T23:57:30.741 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1506e-05 2024-09-14T23:57:30.741 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.741 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:30.741 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.741 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.742 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T23:57:30.742 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:34.406877+0000", 2024-09-14T23:57:30.742 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 56.190781248, 2024-09-14T23:57:30.742 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030351169999999999, 2024-09-14T23:57:30.742 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:30.742 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:30.742 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:30.742 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:30.742 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:30.742 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:30.742 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 10 2024-09-14T23:57:30.742 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.743 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:30.743 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.743 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:30.743 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406877+0000", 2024-09-14T23:57:30.743 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.743 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.743 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.743 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:30.743 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406877+0000", 2024-09-14T23:57:30.743 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.743 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.743 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.743 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:30.743 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406878+0000", 2024-09-14T23:57:30.744 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.291e-06 2024-09-14T23:57:30.744 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.744 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.744 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:30.744 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406883+0000", 2024-09-14T23:57:30.744 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2399999999999998e-06 2024-09-14T23:57:30.744 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.744 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.744 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:30.744 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406891+0000", 2024-09-14T23:57:30.744 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5889999999999997e-06 2024-09-14T23:57:30.744 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.744 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.744 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:30.745 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406898+0000", 2024-09-14T23:57:30.745 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3599999999999998e-06 2024-09-14T23:57:30.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.745 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.745 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:30.745 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406948+0000", 2024-09-14T23:57:30.745 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9435999999999999e-05 2024-09-14T23:57:30.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.745 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.745 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-14T23:57:30.745 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406953+0000", 2024-09-14T23:57:30.745 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2809999999999999e-06 2024-09-14T23:57:30.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.745 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.746 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:30.746 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.408559+0000", 2024-09-14T23:57:30.746 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00160607 2024-09-14T23:57:30.746 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.746 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.746 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:30.746 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.408637+0000", 2024-09-14T23:57:30.746 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8504000000000006e-05 2024-09-14T23:57:30.746 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.746 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.746 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 4", 2024-09-14T23:57:30.746 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.408781+0000", 2024-09-14T23:57:30.746 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014374799999999999 2024-09-14T23:57:30.746 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.747 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.747 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:57:30.747 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.409659+0000", 2024-09-14T23:57:30.747 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000877781 2024-09-14T23:57:30.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.747 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.747 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:30.747 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.409822+0000", 2024-09-14T23:57:30.747 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016295400000000001 2024-09-14T23:57:30.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.747 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.747 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:30.747 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.409872+0000", 2024-09-14T23:57:30.747 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0022000000000003e-05 2024-09-14T23:57:30.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.748 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.748 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:30.748 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.409912+0000", 2024-09-14T23:57:30.748 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9841e-05 2024-09-14T23:57:30.748 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.748 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:30.748 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.748 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.748 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.748 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "MOSDPGPush(3.c 48/47 [PushOp(3:34183e86:::rbd_data.111888d8b54.0000000000000003:head, version: 37'112, data_included: [1473248~99616,1638400~65536,1835008~262144,2162688~65536,2293760~262144,2686976~65536,9737423~8043], data_size: 828555, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:34183e86:::rbd_data.111888d8b54.0000000000000003:head@37'112, size: 9745466, copy_subset: [(0, 9745466)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 9745466, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1473248, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-14T23:57:30.748 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:58.332069+0000", 2024-09-14T23:57:30.749 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 32.265588645000001, 2024-09-14T23:57:30.749 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0025177979999999999, 2024-09-14T23:57:30.749 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:30.749 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:30.749 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:30.749 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:30.749 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.749 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:30.749 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.332069+0000", 2024-09-14T23:57:30.749 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.750 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.750 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:30.750 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.332069+0000", 2024-09-14T23:57:30.750 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.750 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.750 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:30.750 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.332070+0000", 2024-09-14T23:57:30.750 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1200000000000003e-07 2024-09-14T23:57:30.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.750 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.750 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:30.750 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.332822+0000", 2024-09-14T23:57:30.751 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00075239299999999996 2024-09-14T23:57:30.751 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.751 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.751 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:30.751 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.332845+0000", 2024-09-14T23:57:30.751 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2436999999999999e-05 2024-09-14T23:57:30.751 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.751 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.751 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:30.751 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.332866+0000", 2024-09-14T23:57:30.751 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0874e-05 2024-09-14T23:57:30.751 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.751 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.751 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:30.752 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.332970+0000", 2024-09-14T23:57:30.752 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000104528 2024-09-14T23:57:30.752 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.752 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.752 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:30.752 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.332987+0000", 2024-09-14T23:57:30.752 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7073000000000001e-05 2024-09-14T23:57:30.752 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.752 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.752 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:30.752 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.334587+0000", 2024-09-14T23:57:30.752 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001599881 2024-09-14T23:57:30.752 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.752 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:30.753 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.753 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.753 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.753 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:50626 3.d e51/46 3:b5154cfe:::rbd_data.11183b9aac34.0000000000000004:head v 51'63, mlcod=45'61)", 2024-09-14T23:57:30.753 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:01.140129+0000", 2024-09-14T23:57:30.753 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 29.457528648, 2024-09-14T23:57:30.753 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0029883850000000001, 2024-09-14T23:57:30.753 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:30.753 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:30.753 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:30.754 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:30.754 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.754 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:30.754 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.140129+0000", 2024-09-14T23:57:30.754 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.754 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.754 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:30.754 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.140129+0000", 2024-09-14T23:57:30.754 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.754 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.754 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:30.755 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.140130+0000", 2024-09-14T23:57:30.755 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0529999999999999e-06 2024-09-14T23:57:30.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.755 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:30.755 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.140311+0000", 2024-09-14T23:57:30.755 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018054200000000001 2024-09-14T23:57:30.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.755 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:30.755 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.140325+0000", 2024-09-14T23:57:30.755 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.382e-05 2024-09-14T23:57:30.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.756 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:30.756 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.140332+0000", 2024-09-14T23:57:30.756 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5700000000000004e-06 2024-09-14T23:57:30.756 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.756 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.756 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:30.756 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.140392+0000", 2024-09-14T23:57:30.756 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9296000000000003e-05 2024-09-14T23:57:30.756 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.756 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.756 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:30.756 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.140436+0000", 2024-09-14T23:57:30.756 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4515999999999999e-05 2024-09-14T23:57:30.756 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.757 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.757 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:30.757 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.143075+0000", 2024-09-14T23:57:30.757 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0026385369999999998 2024-09-14T23:57:30.757 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.757 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.757 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:30.757 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.143118+0000", 2024-09-14T23:57:30.757 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3050999999999998e-05 2024-09-14T23:57:30.757 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.757 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:30.757 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.757 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.757 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.758 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:51277 3.d e52/46 3:b2645877:::rbd_data.11181975a1de.0000000000000001:head v 52'69, mlcod=51'66)", 2024-09-14T23:57:30.758 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:02.157938+0000", 2024-09-14T23:57:30.758 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 28.439719650000001, 2024-09-14T23:57:30.758 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0027517549999999998, 2024-09-14T23:57:30.758 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:30.758 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:30.758 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:30.758 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:30.758 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.758 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:30.758 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.157938+0000", 2024-09-14T23:57:30.758 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.758 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.758 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.758 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:30.759 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.157938+0000", 2024-09-14T23:57:30.759 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.759 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.759 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.759 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:30.759 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.157939+0000", 2024-09-14T23:57:30.759 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9199999999999999e-07 2024-09-14T23:57:30.759 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.759 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.759 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:30.759 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158543+0000", 2024-09-14T23:57:30.759 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00060333299999999995 2024-09-14T23:57:30.759 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.759 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.760 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:30.760 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158575+0000", 2024-09-14T23:57:30.760 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2801e-05 2024-09-14T23:57:30.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.760 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.760 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:30.760 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158586+0000", 2024-09-14T23:57:30.760 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.102e-05 2024-09-14T23:57:30.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.760 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.760 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:30.760 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158643+0000", 2024-09-14T23:57:30.760 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6320999999999998e-05 2024-09-14T23:57:30.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.761 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.761 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:30.761 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158693+0000", 2024-09-14T23:57:30.761 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0037000000000002e-05 2024-09-14T23:57:30.761 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.761 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.761 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:30.761 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.160641+0000", 2024-09-14T23:57:30.761 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0019485629999999999 2024-09-14T23:57:30.761 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.761 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.761 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:30.761 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.160690+0000", 2024-09-14T23:57:30.761 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8687999999999999e-05 2024-09-14T23:57:30.762 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.762 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:30.762 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.762 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.762 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:51275 3.2 e52/49 3:47a71653:::rbd_data.11181975a1de.0000000000000000:head v 52'118, mlcod=52'116)", 2024-09-14T23:57:30.762 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:02.158044+0000", 2024-09-14T23:57:30.762 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 28.439614078000002, 2024-09-14T23:57:30.762 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0028337200000000001, 2024-09-14T23:57:30.762 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:30.762 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:30.762 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:30.762 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:30.762 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.762 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:30.763 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158044+0000", 2024-09-14T23:57:30.763 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.763 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:30.763 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158044+0000", 2024-09-14T23:57:30.763 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.763 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:30.763 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158045+0000", 2024-09-14T23:57:30.763 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5700000000000002e-07 2024-09-14T23:57:30.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.764 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:30.764 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158637+0000", 2024-09-14T23:57:30.764 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00059278499999999997 2024-09-14T23:57:30.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.764 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.764 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:30.764 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158657+0000", 2024-09-14T23:57:30.764 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9422000000000001e-05 2024-09-14T23:57:30.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.764 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.764 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:30.764 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158675+0000", 2024-09-14T23:57:30.764 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8564999999999999e-05 2024-09-14T23:57:30.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.765 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.765 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:30.765 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158735+0000", 2024-09-14T23:57:30.765 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9904000000000001e-05 2024-09-14T23:57:30.765 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.765 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.765 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:30.765 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158757+0000", 2024-09-14T23:57:30.765 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1277e-05 2024-09-14T23:57:30.765 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.765 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.765 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:30.765 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.160845+0000", 2024-09-14T23:57:30.765 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0020881419999999999 2024-09-14T23:57:30.766 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.766 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.766 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:30.766 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.160878+0000", 2024-09-14T23:57:30.766 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2867999999999998e-05 2024-09-14T23:57:30.766 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.766 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:30.766 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.766 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.766 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.766 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56383 3.c e56/49 3:36b698b0:::rbd_data.1118c4b3aebe.0000000000000000:head v 56'194, mlcod=56'192)", 2024-09-14T23:57:30.766 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.270483+0000", 2024-09-14T23:57:30.766 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.327175094000001, 2024-09-14T23:57:30.766 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0079685880000000004, 2024-09-14T23:57:30.767 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:30.767 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:30.767 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:30.767 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:30.767 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.767 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:30.767 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.270483+0000", 2024-09-14T23:57:30.767 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.767 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.767 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:30.767 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.270483+0000", 2024-09-14T23:57:30.767 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.768 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.768 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:30.768 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.270484+0000", 2024-09-14T23:57:30.768 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7400000000000002e-07 2024-09-14T23:57:30.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.768 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.768 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:30.768 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273680+0000", 2024-09-14T23:57:30.768 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031966680000000002 2024-09-14T23:57:30.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.768 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.768 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:30.768 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273697+0000", 2024-09-14T23:57:30.768 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6595e-05 2024-09-14T23:57:30.769 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.769 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.769 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:30.769 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273705+0000", 2024-09-14T23:57:30.769 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7530000000000001e-06 2024-09-14T23:57:30.769 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.769 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.769 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:30.769 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273763+0000", 2024-09-14T23:57:30.769 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8238e-05 2024-09-14T23:57:30.769 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.769 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.769 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:30.769 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273781+0000", 2024-09-14T23:57:30.770 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8281e-05 2024-09-14T23:57:30.770 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.770 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.770 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:30.770 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.278408+0000", 2024-09-14T23:57:30.770 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00462716 2024-09-14T23:57:30.770 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.770 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.770 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:30.770 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.278451+0000", 2024-09-14T23:57:30.770 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3019000000000003e-05 2024-09-14T23:57:30.770 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.770 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:30.770 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.771 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.771 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.771 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56384 3.c e56/49 3:36b698b0:::rbd_data.1118c4b3aebe.0000000000000000:head v 56'196, mlcod=56'192)", 2024-09-14T23:57:30.771 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:57:30.771 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.323142422, 2024-09-14T23:57:30.771 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0046707529999999997, 2024-09-14T23:57:30.771 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:30.771 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:30.771 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:30.771 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:30.771 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.771 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:30.771 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:57:30.771 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.772 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.772 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.772 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:30.772 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:57:30.772 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.772 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.772 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.772 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:30.772 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:57:30.772 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1699999999999997e-07 2024-09-14T23:57:30.772 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.772 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.772 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:30.772 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274523+0000", 2024-09-14T23:57:30.773 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9129999999999997e-06 2024-09-14T23:57:30.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.773 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.773 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:30.773 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274535+0000", 2024-09-14T23:57:30.773 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1788e-05 2024-09-14T23:57:30.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.773 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.773 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:30.773 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274542+0000", 2024-09-14T23:57:30.773 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0539999999999997e-06 2024-09-14T23:57:30.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.773 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.773 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:30.773 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275889+0000", 2024-09-14T23:57:30.774 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00134669 2024-09-14T23:57:30.774 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.774 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.774 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:30.774 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275917+0000", 2024-09-14T23:57:30.774 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7664000000000001e-05 2024-09-14T23:57:30.774 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.774 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.774 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:30.774 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279134+0000", 2024-09-14T23:57:30.774 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0032178459999999999 2024-09-14T23:57:30.774 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.774 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.774 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:30.775 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279186+0000", 2024-09-14T23:57:30.775 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1881e-05 2024-09-14T23:57:30.775 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.775 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:30.775 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.775 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.775 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.775 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56381 3.1d e56/49 3:bcf9a8cf:::rbd_data.1118c4b3aebe.0000000000000006:head v 56'210, mlcod=56'208)", 2024-09-14T23:57:30.775 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.275462+0000", 2024-09-14T23:57:30.775 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.322196091999999, 2024-09-14T23:57:30.775 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0071310710000000001, 2024-09-14T23:57:30.775 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:30.775 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:30.775 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:30.775 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:30.776 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.776 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:30.776 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275462+0000", 2024-09-14T23:57:30.776 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.776 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.776 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:30.776 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275462+0000", 2024-09-14T23:57:30.776 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.776 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.776 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:30.776 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275463+0000", 2024-09-14T23:57:30.776 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2699999999999999e-07 2024-09-14T23:57:30.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.777 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.777 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:30.777 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276779+0000", 2024-09-14T23:57:30.777 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0013163649999999999 2024-09-14T23:57:30.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.777 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.777 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:30.777 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276795+0000", 2024-09-14T23:57:30.777 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.59e-05 2024-09-14T23:57:30.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.777 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.777 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:30.777 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276806+0000", 2024-09-14T23:57:30.778 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1106e-05 2024-09-14T23:57:30.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.778 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.778 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:30.778 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276941+0000", 2024-09-14T23:57:30.778 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013477500000000001 2024-09-14T23:57:30.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.778 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.778 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:30.778 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276963+0000", 2024-09-14T23:57:30.778 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2419999999999999e-05 2024-09-14T23:57:30.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.778 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.778 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:30.779 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282546+0000", 2024-09-14T23:57:30.779 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0055827100000000003 2024-09-14T23:57:30.779 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.779 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.779 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:30.779 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282593+0000", 2024-09-14T23:57:30.779 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7067999999999997e-05 2024-09-14T23:57:30.779 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.779 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:30.779 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.779 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.779 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.779 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56387 3.c e56/49 3:30ee4ba0:::rbd_data.1118c4b3aebe.0000000000000004:head v 56'197, mlcod=56'192)", 2024-09-14T23:57:30.779 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.276498+0000", 2024-09-14T23:57:30.779 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.321159996999999, 2024-09-14T23:57:30.780 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0092192639999999996, 2024-09-14T23:57:30.780 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:30.780 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:30.780 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:30.780 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:30.780 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.780 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:30.780 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276498+0000", 2024-09-14T23:57:30.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.781 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:30.781 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276498+0000", 2024-09-14T23:57:30.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.781 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:30.781 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276499+0000", 2024-09-14T23:57:30.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1100000000000004e-07 2024-09-14T23:57:30.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.781 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:30.782 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279646+0000", 2024-09-14T23:57:30.782 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031473220000000001 2024-09-14T23:57:30.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.782 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:30.782 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279672+0000", 2024-09-14T23:57:30.782 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6129999999999999e-05 2024-09-14T23:57:30.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.782 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:30.782 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279682+0000", 2024-09-14T23:57:30.782 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7170000000000003e-06 2024-09-14T23:57:30.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.783 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:30.783 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279742+0000", 2024-09-14T23:57:30.783 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9926999999999997e-05 2024-09-14T23:57:30.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.783 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.783 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:30.783 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279761+0000", 2024-09-14T23:57:30.783 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9402000000000002e-05 2024-09-14T23:57:30.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.783 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.783 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:30.783 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.285680+0000", 2024-09-14T23:57:30.783 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.005918881 2024-09-14T23:57:30.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.784 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.784 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:30.784 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.285717+0000", 2024-09-14T23:57:30.784 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6974000000000002e-05 2024-09-14T23:57:30.784 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.784 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:30.784 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.784 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.784 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.784 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56388 3.c e56/49 3:30ee4ba0:::rbd_data.1118c4b3aebe.0000000000000004:head v 56'199, mlcod=56'192)", 2024-09-14T23:57:30.784 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:57:30.784 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.317668408999999, 2024-09-14T23:57:30.784 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0060596740000000001, 2024-09-14T23:57:30.784 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:30.785 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:30.785 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:30.785 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:30.785 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.785 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:30.785 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:57:30.785 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.785 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.785 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.785 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:30.785 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:57:30.785 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.785 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.785 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.785 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:30.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:57:30.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2200000000000003e-07 2024-09-14T23:57:30.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:30.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279995+0000", 2024-09-14T23:57:30.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5569999999999997e-06 2024-09-14T23:57:30.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:30.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280004+0000", 2024-09-14T23:57:30.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0080000000000008e-06 2024-09-14T23:57:30.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.787 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:30.787 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280010+0000", 2024-09-14T23:57:30.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5749999999999997e-06 2024-09-14T23:57:30.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.787 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:30.787 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282968+0000", 2024-09-14T23:57:30.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0029579390000000001 2024-09-14T23:57:30.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.787 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:30.787 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282998+0000", 2024-09-14T23:57:30.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9947999999999999e-05 2024-09-14T23:57:30.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.788 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:30.788 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.286012+0000", 2024-09-14T23:57:30.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030132900000000001 2024-09-14T23:57:30.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.788 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:30.788 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.286049+0000", 2024-09-14T23:57:30.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7534999999999999e-05 2024-09-14T23:57:30.788 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.788 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:30.788 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.789 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56382 3.1d e56/49 3:bcf9a8cf:::rbd_data.1118c4b3aebe.0000000000000006:head v 56'212, mlcod=56'208)", 2024-09-14T23:57:30.789 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.280285+0000", 2024-09-14T23:57:30.789 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.317372725999999, 2024-09-14T23:57:30.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035393489999999998, 2024-09-14T23:57:30.789 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:30.789 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:30.789 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:30.789 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:30.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.789 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:30.789 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280285+0000", 2024-09-14T23:57:30.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:30.790 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280285+0000", 2024-09-14T23:57:30.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:30.790 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280286+0000", 2024-09-14T23:57:30.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5000000000000001e-07 2024-09-14T23:57:30.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:30.790 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280299+0000", 2024-09-14T23:57:30.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3312000000000001e-05 2024-09-14T23:57:30.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.791 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:30.791 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280311+0000", 2024-09-14T23:57:30.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1677e-05 2024-09-14T23:57:30.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.791 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:30.791 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280318+0000", 2024-09-14T23:57:30.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.742e-06 2024-09-14T23:57:30.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.791 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:30.791 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280374+0000", 2024-09-14T23:57:30.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.592e-05 2024-09-14T23:57:30.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.792 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:30.792 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280392+0000", 2024-09-14T23:57:30.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8490000000000001e-05 2024-09-14T23:57:30.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.792 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:30.792 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.283693+0000", 2024-09-14T23:57:30.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033012050000000002 2024-09-14T23:57:30.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.793 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:30.793 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.283825+0000", 2024-09-14T23:57:30.793 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013105300000000001 2024-09-14T23:57:30.793 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.793 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:30.793 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.793 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "pg_update_log_missing(3.c epoch 56/49 rep_tid 1493 entries 56'200 (0'0) error 3:370203a3:::rbd_data.1118c4b3aebe.000000000000000d:head by client.4376.0:56396 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 56'192)", 2024-09-14T23:57:30.793 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:57:30.793 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.317148269, 2024-09-14T23:57:30.793 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057370099999999999, 2024-09-14T23:57:30.793 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:30.793 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:30.794 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:57:30.794 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:30.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.794 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:30.794 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:57:30.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.794 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:30.794 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:57:30.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.794 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:30.794 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:57:30.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0699999999999997e-07 2024-09-14T23:57:30.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:30.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280519+0000", 2024-09-14T23:57:30.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6260000000000001e-06 2024-09-14T23:57:30.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:30.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280541+0000", 2024-09-14T23:57:30.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1976000000000001e-05 2024-09-14T23:57:30.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:30.796 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280556+0000", 2024-09-14T23:57:30.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5075000000000001e-05 2024-09-14T23:57:30.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.796 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:30.796 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.285494+0000", 2024-09-14T23:57:30.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0049378690000000001 2024-09-14T23:57:30.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.796 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:30.796 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.286247+0000", 2024-09-14T23:57:30.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00075285700000000001 2024-09-14T23:57:30.796 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.796 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:30.797 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.797 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56400 3.1c e56/49 3:3f5b81ba:::rbd_data.1118c4b3aebe.000000000000000a:head v 56'236, mlcod=56'234)", 2024-09-14T23:57:30.797 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.295064+0000", 2024-09-14T23:57:30.797 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.302593476999999, 2024-09-14T23:57:30.797 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0054331589999999999, 2024-09-14T23:57:30.797 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:30.797 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:30.797 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:30.797 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:30.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.797 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:30.798 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.295064+0000", 2024-09-14T23:57:30.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.798 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:30.798 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.295064+0000", 2024-09-14T23:57:30.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.798 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:30.798 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.295066+0000", 2024-09-14T23:57:30.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.502e-06 2024-09-14T23:57:30.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.799 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:30.799 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296403+0000", 2024-09-14T23:57:30.799 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001336554 2024-09-14T23:57:30.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.799 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.799 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:30.799 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296428+0000", 2024-09-14T23:57:30.799 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5179000000000001e-05 2024-09-14T23:57:30.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.799 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.799 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:30.799 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296440+0000", 2024-09-14T23:57:30.799 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2387e-05 2024-09-14T23:57:30.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:30.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296554+0000", 2024-09-14T23:57:30.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114269 2024-09-14T23:57:30.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:30.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296581+0000", 2024-09-14T23:57:30.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6673999999999999e-05 2024-09-14T23:57:30.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:30.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.300418+0000", 2024-09-14T23:57:30.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0038371249999999998 2024-09-14T23:57:30.801 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.801 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.801 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:30.801 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.300498+0000", 2024-09-14T23:57:30.801 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9468999999999994e-05 2024-09-14T23:57:30.801 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.801 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:30.801 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.801 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.801 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.801 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56401 3.1c e56/49 3:3f5b81ba:::rbd_data.1118c4b3aebe.000000000000000a:head v 56'238, mlcod=56'234)", 2024-09-14T23:57:30.801 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.297640+0000", 2024-09-14T23:57:30.801 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.300017638, 2024-09-14T23:57:30.802 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031491919999999999, 2024-09-14T23:57:30.802 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:30.802 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:30.802 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:30.802 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:30.802 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.802 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:30.802 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.297640+0000", 2024-09-14T23:57:30.802 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.802 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.802 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:30.802 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.297640+0000", 2024-09-14T23:57:30.802 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.803 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:30.803 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.297641+0000", 2024-09-14T23:57:30.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9000000000000005e-07 2024-09-14T23:57:30.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.803 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:30.803 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.297649+0000", 2024-09-14T23:57:30.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4510000000000003e-06 2024-09-14T23:57:30.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.803 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:30.803 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.297663+0000", 2024-09-14T23:57:30.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4262e-05 2024-09-14T23:57:30.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:30.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.297679+0000", 2024-09-14T23:57:30.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5798e-05 2024-09-14T23:57:30.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:30.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.298408+0000", 2024-09-14T23:57:30.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00072923699999999998 2024-09-14T23:57:30.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:30.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.298430+0000", 2024-09-14T23:57:30.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1753999999999999e-05 2024-09-14T23:57:30.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.805 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.805 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:30.805 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.300749+0000", 2024-09-14T23:57:30.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023190590000000001 2024-09-14T23:57:30.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.805 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.805 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:30.805 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.300789+0000", 2024-09-14T23:57:30.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0640999999999999e-05 2024-09-14T23:57:30.805 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.805 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:30.805 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.806 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.806 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:61921 3.5 e57/49 3:a05d004f:::rbd_data.11188d356229.000000000000000b:head v 57'279, mlcod=57'277)", 2024-09-14T23:57:30.806 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:07.560389+0000", 2024-09-14T23:57:30.806 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.037268878999999, 2024-09-14T23:57:30.806 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0034956520000000001, 2024-09-14T23:57:30.806 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:30.806 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:30.806 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:30.806 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:30.806 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.806 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:30.806 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.560389+0000", 2024-09-14T23:57:30.806 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.807 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:30.807 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.560389+0000", 2024-09-14T23:57:30.807 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.807 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:30.807 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.560390+0000", 2024-09-14T23:57:30.807 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4109999999999999e-06 2024-09-14T23:57:30.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.807 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:30.807 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.561839+0000", 2024-09-14T23:57:30.807 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001448406 2024-09-14T23:57:30.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.808 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.808 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:30.808 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.561854+0000", 2024-09-14T23:57:30.808 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5426000000000001e-05 2024-09-14T23:57:30.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.808 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.808 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:30.808 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.561863+0000", 2024-09-14T23:57:30.808 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7509999999999997e-06 2024-09-14T23:57:30.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.808 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.808 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:30.808 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.561917+0000", 2024-09-14T23:57:30.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3531e-05 2024-09-14T23:57:30.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.809 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:30.809 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.561944+0000", 2024-09-14T23:57:30.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7528000000000001e-05 2024-09-14T23:57:30.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.809 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:30.809 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563843+0000", 2024-09-14T23:57:30.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001899361 2024-09-14T23:57:30.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.809 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:30.810 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563885+0000", 2024-09-14T23:57:30.810 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1238000000000001e-05 2024-09-14T23:57:30.810 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.810 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:30.810 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.810 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.810 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:61922 3.5 e57/49 3:a11ebf8e:::rbd_data.11188d356229.000000000000000c:head v 57'280, mlcod=57'277)", 2024-09-14T23:57:30.810 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:07.563144+0000", 2024-09-14T23:57:30.810 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.034513825000001, 2024-09-14T23:57:30.810 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003977848, 2024-09-14T23:57:30.810 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:30.810 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:30.810 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:30.811 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:30.811 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.811 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:30.811 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563144+0000", 2024-09-14T23:57:30.811 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.811 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.811 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.811 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:30.811 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563144+0000", 2024-09-14T23:57:30.811 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.811 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.811 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.811 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:30.811 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563145+0000", 2024-09-14T23:57:30.811 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5400000000000001e-07 2024-09-14T23:57:30.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.812 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.812 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:30.812 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565056+0000", 2024-09-14T23:57:30.812 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0019108160000000001 2024-09-14T23:57:30.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.812 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.812 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:30.812 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565088+0000", 2024-09-14T23:57:30.812 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2586999999999998e-05 2024-09-14T23:57:30.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.812 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.812 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:30.813 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565100+0000", 2024-09-14T23:57:30.813 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1917e-05 2024-09-14T23:57:30.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.813 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.813 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:30.813 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565166+0000", 2024-09-14T23:57:30.813 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6245e-05 2024-09-14T23:57:30.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.813 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.813 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:30.813 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565190+0000", 2024-09-14T23:57:30.813 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3825999999999999e-05 2024-09-14T23:57:30.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.813 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.814 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:30.814 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.567080+0000", 2024-09-14T23:57:30.814 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001889541 2024-09-14T23:57:30.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.814 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.814 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:30.814 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.567122+0000", 2024-09-14T23:57:30.814 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2262000000000003e-05 2024-09-14T23:57:30.814 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.814 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:30.814 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.814 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.814 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:66564 3.c e60/49 3:33faa7ca:::rbd_data.111873792d9b.0000000000000003:head v 60'236, mlcod=57'233)", 2024-09-14T23:57:30.815 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:11.271596+0000", 2024-09-14T23:57:30.815 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.326062246999999, 2024-09-14T23:57:30.815 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0060396099999999999, 2024-09-14T23:57:30.815 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:30.815 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:30.815 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:30.815 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:30.815 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.815 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:30.815 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.271596+0000", 2024-09-14T23:57:30.815 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.815 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.815 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.816 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:30.816 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.271596+0000", 2024-09-14T23:57:30.816 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:30.816 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.816 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.816 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:30.816 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.271597+0000", 2024-09-14T23:57:30.816 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2559999999999999e-06 2024-09-14T23:57:30.816 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.816 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.816 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:30.816 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.274854+0000", 2024-09-14T23:57:30.816 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0032574470000000001 2024-09-14T23:57:30.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.817 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.817 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:30.817 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.274874+0000", 2024-09-14T23:57:30.817 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9449999999999998e-05 2024-09-14T23:57:30.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.817 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.817 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:30.817 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.274883+0000", 2024-09-14T23:57:30.817 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5980000000000003e-06 2024-09-14T23:57:30.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.817 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.817 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:30.817 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.274977+0000", 2024-09-14T23:57:30.818 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3270999999999995e-05 2024-09-14T23:57:30.818 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.818 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.818 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:30.818 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.275000+0000", 2024-09-14T23:57:30.818 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3088e-05 2024-09-14T23:57:30.818 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.818 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.818 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:30.818 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.277590+0000", 2024-09-14T23:57:30.818 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0025898660000000001 2024-09-14T23:57:30.818 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:30.818 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:30.818 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:30.819 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.277635+0000", 2024-09-14T23:57:30.819 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5633999999999997e-05 2024-09-14T23:57:30.819 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.819 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:30.819 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.819 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:30.819 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:30.819 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:30.821 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T23:57:30.821 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:30.819+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:30.922 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:30.920+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:30.931 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T23:57:30.977 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:30.978 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:30.978 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:57:30.978 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:30.987 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T23:57:30.990 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is set 2024-09-14T23:57:31.031 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:31.030+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:31.052 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone27 from 0x325a87d (overlap 0xdf54e9) to 0x0 2024-09-14T23:57:31.091 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:57:31.091 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:57:31.110 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:57:31.110 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:57:31.122 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:31.122 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:31.122 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:57:31.122 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:57:31.122 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:31.123 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:31.122+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:31.128 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:57:31.128 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:57:31.131 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T23:57:31.145 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-09-14T23:57:31.145 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:57:31.145 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:57:31.163 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:57:31.163 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:57:31.181 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:57:31.181 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:57:31.198 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:57:31.198 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:57:31.215 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:57:31.215 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:57:31.224 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:31.223+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:31.225 INFO:teuthology.orchestra.run.smithi042.stdout:2898 write 0x6216a38 thru 0x621ddaf (0x7378 bytes) 2024-09-14T23:57:31.229 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:31.229 INFO:teuthology.orchestra.run.smithi042.stdout:2900 write 0x4f429ed thru 0x4f4befb (0x950f bytes) 2024-09-14T23:57:31.232 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:31.232 INFO:teuthology.orchestra.run.smithi042.stdout:2903 read 0x20c36c7 thru 0x20cb016 (0x7950 bytes) 2024-09-14T23:57:31.232 INFO:teuthology.orchestra.run.smithi042.stdout:2906 read 0x30c86af thru 0x30d690b (0xe25d bytes) 2024-09-14T23:57:31.240 INFO:teuthology.orchestra.run.smithi042.stdout:2907 write 0x8283122 thru 0x82925b9 (0xf498 bytes) 2024-09-14T23:57:31.243 INFO:teuthology.orchestra.run.smithi042.stdout:2908 read 0x6bd3a38 thru 0x6bdd344 (0x990d bytes) 2024-09-14T23:57:31.243 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:31.250 INFO:teuthology.orchestra.run.smithi042.stdout:2911 write 0x9be629d thru 0x9be7fc3 (0x1d27 bytes) 2024-09-14T23:57:31.264 INFO:teuthology.orchestra.run.smithi042.stdout:2914 write 0xe8293d9 thru 0xe8352c1 (0xbee9 bytes) 2024-09-14T23:57:31.267 INFO:teuthology.orchestra.run.smithi042.stdout:2915 read 0xdfeaf36 thru 0xdffa90d (0xf9d8 bytes) 2024-09-14T23:57:31.268 INFO:teuthology.orchestra.run.smithi042.stdout:2916 punch from 0xb4c673d to 0xb4c8559, (0x1e1c bytes) 2024-09-14T23:57:31.268 INFO:teuthology.orchestra.run.smithi042.stdout:2917 punch from 0x1729544 to 0x1738f8e, (0xfa4a bytes) 2024-09-14T23:57:31.268 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:31.268 INFO:teuthology.orchestra.run.smithi042.stdout:2919 read 0x66667b4 thru 0x6675e66 (0xf6b3 bytes) 2024-09-14T23:57:31.280 INFO:teuthology.orchestra.run.smithi042.stdout:2920 trunc from 0xe8352c2 to 0x33ab9cf 2024-09-14T23:57:31.282 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:31.282 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:31.293 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T23:57:31.296 INFO:teuthology.orchestra.run.smithi042.stdout:2921 read 0x11c5fb9 thru 0x11c6f3e (0xf86 bytes) 2024-09-14T23:57:31.296 INFO:teuthology.orchestra.run.smithi042.stdout:2923 read 0x1978be4 thru 0x197a844 (0x1c61 bytes) 2024-09-14T23:57:31.296 INFO:teuthology.orchestra.run.smithi042.stdout:2924 punch from 0x1082c95 to 0x1089413, (0x677e bytes) 2024-09-14T23:57:31.296 INFO:teuthology.orchestra.run.smithi042.stdout:2925 read 0x23f52c0 thru 0x2400b94 (0xb8d5 bytes) 2024-09-14T23:57:31.302 INFO:teuthology.orchestra.run.smithi042.stdout:2927 writesame 0x45971f6 thru 0x45a69eb data_size 0x11(0xf7f6 bytes) 2024-09-14T23:57:31.308 INFO:teuthology.orchestra.run.smithi042.stdout:2928 write 0x58f961f thru 0x5903877 (0xa259 bytes) 2024-09-14T23:57:31.312 INFO:teuthology.orchestra.run.smithi042.stdout:2930 read 0x3e77fbb thru 0x3e7de3b (0x5e81 bytes) 2024-09-14T23:57:31.312 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:31.312 INFO:teuthology.orchestra.run.smithi042.stdout:2932 writesame 0x15a1da9 thru 0x15abba8 data_size 0x80(0x9e00 bytes) 2024-09-14T23:57:31.316 INFO:teuthology.orchestra.run.smithi042.stdout:2933 punch from 0x179deb0 to 0x17ab46d, (0xd5bd bytes) 2024-09-14T23:57:31.316 INFO:teuthology.orchestra.run.smithi042.stdout:2934 writesame 0x6ff7f6 thru 0x701b1a data_size 0x1(0x2325 bytes) 2024-09-14T23:57:31.330 INFO:teuthology.orchestra.run.smithi042.stdout:2936 read 0x55115cc thru 0x5516c90 (0x56c5 bytes) 2024-09-14T23:57:31.331 INFO:teuthology.orchestra.run.smithi042.stdout:2938 punch from 0x20a7c63 to 0x20b643b, (0xe7d8 bytes) 2024-09-14T23:57:31.331 INFO:teuthology.orchestra.run.smithi042.stdout:2939 read 0x18968db thru 0x18a0919 (0xa03f bytes) 2024-09-14T23:57:31.337 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:31.336+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:31.345 INFO:teuthology.orchestra.run.smithi042.stdout:2940 write 0xd2b7ac6 thru 0xd2c5599 (0xdad4 bytes) 2024-09-14T23:57:31.355 INFO:teuthology.orchestra.run.smithi042.stdout:2941 write 0xd69b589 thru 0xd6a7cde (0xc756 bytes) 2024-09-14T23:57:31.371 INFO:teuthology.orchestra.run.smithi042.stdout:2942 trunc from 0xd6a7cdf to 0x1392c45 2024-09-14T23:57:31.396 INFO:teuthology.orchestra.run.smithi042.stdout:2943 write 0x37ea53a thru 0x37f26fc (0x81c3 bytes) 2024-09-14T23:57:31.399 INFO:teuthology.orchestra.run.smithi042.stdout:2944 read 0xe05e3e thru 0xe06bf2 (0xdb5 bytes) 2024-09-14T23:57:31.399 INFO:teuthology.orchestra.run.smithi042.stdout:2945 punch from 0x12c30ca to 0x12cd5ac, (0xa4e2 bytes) 2024-09-14T23:57:31.410 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:31.410 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:31.413 INFO:teuthology.orchestra.run.smithi042.stdout:2946 writesame 0xb480767 thru 0xb480b17 data_size 0x5(0x3b1 bytes) 2024-09-14T23:57:31.413 INFO:teuthology.orchestra.run.smithi042.stdout:2947 read 0xa9a6ca5 thru 0xa9aee90 (0x81ec bytes) 2024-09-14T23:57:31.413 INFO:teuthology.orchestra.run.smithi042.stdout:2948 punch from 0x85ca3be to 0x85d3336, (0x8f78 bytes) 2024-09-14T23:57:31.413 INFO:teuthology.orchestra.run.smithi042.stdout:2949 write 0x12b1997 thru 0x12b1ddc (0x446 bytes) 2024-09-14T23:57:31.414 INFO:teuthology.orchestra.run.smithi042.stdout:2951 write 0xac30c8c thru 0xac33482 (0x27f7 bytes) 2024-09-14T23:57:31.418 INFO:teuthology.orchestra.run.smithi042.stdout:2952 read 0x3e08e1c thru 0x3e16789 (0xd96e bytes) 2024-09-14T23:57:31.418 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:31.419 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T23:57:31.425 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:31.424+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:31.427 INFO:teuthology.orchestra.run.smithi042.stdout:2955 writesame 0xebeb6a8 thru 0xebf606b data_size 0x35(0xa9c4 bytes) 2024-09-14T23:57:31.427 INFO:teuthology.orchestra.run.smithi042.stdout:2956 write 0x15a02df thru 0x15af307 (0xf029 bytes) 2024-09-14T23:57:31.429 INFO:teuthology.orchestra.run.smithi042.stdout:2958 write 0x597bd0a thru 0x59848ae (0x8ba5 bytes) 2024-09-14T23:57:31.433 INFO:teuthology.orchestra.run.smithi042.stdout:2959 read 0x543d7cd thru 0x544620c (0x8a40 bytes) 2024-09-14T23:57:31.433 INFO:teuthology.orchestra.run.smithi042.stdout:2960 write 0x6a467ec thru 0x6a4a815 (0x402a bytes) 2024-09-14T23:57:31.435 INFO:teuthology.orchestra.run.smithi042.stdout:2961 read 0xae09d9a thru 0xae0e767 (0x49ce bytes) 2024-09-14T23:57:31.436 INFO:teuthology.orchestra.run.smithi042.stdout:2963 read 0x28a67d thru 0x2945d5 (0x9f59 bytes) 2024-09-14T23:57:31.436 INFO:teuthology.orchestra.run.smithi042.stdout:2964 writesame 0xba8a003 thru 0xba981f5 data_size 0x3(0xe1f3 bytes) 2024-09-14T23:57:31.457 INFO:teuthology.orchestra.run.smithi042.stdout:2965 trunc from 0xebf606c to 0x44af363 2024-09-14T23:57:31.467 INFO:teuthology.orchestra.run.smithi042.stdout:2966 read 0x350882c thru 0x3514edc (0xc6b1 bytes) 2024-09-14T23:57:31.476 INFO:teuthology.orchestra.run.smithi042.stdout:2967 writesame 0x71d3d21 thru 0x71db2b8 data_size 0x6a(0x7598 bytes) 2024-09-14T23:57:31.476 INFO:teuthology.orchestra.run.smithi042.stdout:2968 write 0x3510671 thru 0x3516085 (0x5a15 bytes) 2024-09-14T23:57:31.479 INFO:teuthology.orchestra.run.smithi042.stdout:2969 read 0x313a656 thru 0x313ced6 (0x2881 bytes) 2024-09-14T23:57:31.480 INFO:teuthology.orchestra.run.smithi042.stdout:2970 write 0x43d8781 thru 0x43de694 (0x5f14 bytes) 2024-09-14T23:57:31.486 INFO:teuthology.orchestra.run.smithi042.stdout:2971 trunc from 0x71db2b9 to 0x9dcdec2 2024-09-14T23:57:31.497 INFO:teuthology.orchestra.run.smithi042.stdout:2972 trunc from 0x9dcdec2 to 0x5b8e02f 2024-09-14T23:57:31.502 INFO:teuthology.orchestra.run.smithi042.stdout:2975 read 0x31f26f4 thru 0x31f5d3c (0x3649 bytes) 2024-09-14T23:57:31.508 INFO:teuthology.orchestra.run.smithi042.stdout:2976 write 0x64ee4fe thru 0x64fb37b (0xce7e bytes) 2024-09-14T23:57:31.511 INFO:teuthology.orchestra.run.smithi042.stdout:2977 read 0x4eec443 thru 0x4efb095 (0xec53 bytes) 2024-09-14T23:57:31.512 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:31.512 INFO:teuthology.orchestra.run.smithi042.stdout:2980 punch from 0x2e95764 to 0x2ea3b34, (0xe3d0 bytes) 2024-09-14T23:57:31.512 INFO:teuthology.orchestra.run.smithi042.stdout:2981 punch from 0x5ef761c to 0x5f0140b, (0x9def bytes) 2024-09-14T23:57:31.512 INFO:teuthology.orchestra.run.smithi042.stdout:2982 trunc from 0x64fb37c to 0x5dc56d6 2024-09-14T23:57:31.520 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:31.521 INFO:teuthology.orchestra.run.smithi042.stdout:2986 punch from 0x2f20c5c to 0x2f2802b, (0x73cf bytes) 2024-09-14T23:57:31.521 INFO:teuthology.orchestra.run.smithi042.stdout:2988 write 0x2784878 thru 0x278b377 (0x6b00 bytes) 2024-09-14T23:57:31.523 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:31.523 INFO:teuthology.orchestra.run.smithi042.stdout:2994 read 0x1850fae thru 0x185e8c0 (0xd913 bytes) 2024-09-14T23:57:31.523 INFO:teuthology.orchestra.run.smithi042.stdout:2995 clone 29 order 25 su 2097152 sc 14 2024-09-14T23:57:31.527 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:31.525+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:31.571 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:31.572 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:31.581 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T23:57:31.628 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:31.627+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:31.687 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:31.687 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:31.696 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T23:57:31.729 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:31.728+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:31.805 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:31.805 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:31.814 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T23:57:31.830 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:31.829+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:31.921 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:31.921 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:31.930 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T23:57:31.931 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:31.930+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:32.032 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:32.031+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:32.045 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:32.045 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:32.055 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T23:57:32.066 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone28 from 0x5dc56d6 (overlap 0x1392c45) to 0x5df3ae 2024-09-14T23:57:32.133 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:32.131+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:32.142 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-09-14T23:57:32.153 INFO:teuthology.orchestra.run.smithi042.stdout:2997 write 0x6c7b855 thru 0x6c83a70 (0x821c bytes) 2024-09-14T23:57:32.157 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:32.157 INFO:teuthology.orchestra.run.smithi042.stdout:3001 punch from 0x3b720b6 to 0x3b813d7, (0xf321 bytes) 2024-09-14T23:57:32.164 INFO:teuthology.orchestra.run.smithi042.stdout:3002 trunc from 0x6c83a71 to 0x92864b 2024-09-14T23:57:32.188 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:32.189 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:32.203 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T23:57:32.208 INFO:teuthology.orchestra.run.smithi042.stdout:3003 writesame 0xc7a9dcb thru 0xc7b79b5 data_size 0x1(0xdbeb bytes) 2024-09-14T23:57:32.210 INFO:teuthology.orchestra.run.smithi042.stdout:3005 write 0x9d3038c thru 0x9d3c65a (0xc2cf bytes) 2024-09-14T23:57:32.212 INFO:teuthology.orchestra.run.smithi042.stdout:3006 read 0x128d98a thru 0x1293ec5 (0x653c bytes) 2024-09-14T23:57:32.212 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:32.213 INFO:teuthology.orchestra.run.smithi042.stdout:3008 read 0x283b333 thru 0x283d31e (0x1fec bytes) 2024-09-14T23:57:32.213 INFO:teuthology.orchestra.run.smithi042.stdout:3009 write 0x9a0d4c1 thru 0x9a1394b (0x648b bytes) 2024-09-14T23:57:32.215 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:32.215 INFO:teuthology.orchestra.run.smithi042.stdout:3012 read 0x370f442 thru 0x371ccd2 (0xd891 bytes) 2024-09-14T23:57:32.215 INFO:teuthology.orchestra.run.smithi042.stdout:3014 write 0x57d490f thru 0x57e2759 (0xde4b bytes) 2024-09-14T23:57:32.217 INFO:teuthology.orchestra.run.smithi042.stdout:3015 writesame 0x9b74d6b thru 0x9b785d1 data_size 0x1(0x3867 bytes) 2024-09-14T23:57:32.225 INFO:teuthology.orchestra.run.smithi042.stdout:3016 read 0x35e7e7 thru 0x35ff7b (0x1795 bytes) 2024-09-14T23:57:32.228 INFO:teuthology.orchestra.run.smithi042.stdout:3017 trunc from 0xc7b79b6 to 0x9dcfa5d 2024-09-14T23:57:32.233 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:32.232+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:32.236 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:32.236 INFO:teuthology.orchestra.run.smithi042.stdout:3021 read 0x4ed88d4 thru 0x4edeefc (0x6629 bytes) 2024-09-14T23:57:32.236 INFO:teuthology.orchestra.run.smithi042.stdout:3022 writesame 0x27929d7 thru 0x279eff8 data_size 0x7(0xc622 bytes) 2024-09-14T23:57:32.244 INFO:teuthology.orchestra.run.smithi042.stdout:3023 trunc from 0x9dcfa5d to 0xc777236 2024-09-14T23:57:32.250 INFO:teuthology.orchestra.run.smithi042.stdout:3024 read 0x71d5c9c thru 0x71dadfe (0x5163 bytes) 2024-09-14T23:57:32.252 INFO:teuthology.orchestra.run.smithi042.stdout:3027 trunc from 0xc777236 to 0xdd92797 2024-09-14T23:57:32.257 INFO:teuthology.orchestra.run.smithi042.stdout:3028 write 0x44da000 thru 0x44de320 (0x4321 bytes) 2024-09-14T23:57:32.260 INFO:teuthology.orchestra.run.smithi042.stdout:3029 writesame 0x3bad6b3 thru 0x3bae244 data_size 0x1(0xb92 bytes) 2024-09-14T23:57:32.262 INFO:teuthology.orchestra.run.smithi042.stdout:3031 punch from 0xc265b16 to 0xc26794d, (0x1e37 bytes) 2024-09-14T23:57:32.262 INFO:teuthology.orchestra.run.smithi042.stdout:3033 write 0xaa53f22 thru 0xaa5dc4a (0x9d29 bytes) 2024-09-14T23:57:32.263 INFO:teuthology.orchestra.run.smithi042.stdout:3034 read 0x115541f thru 0x11651fb (0xfddd bytes) 2024-09-14T23:57:32.264 INFO:teuthology.orchestra.run.smithi042.stdout:3035 write 0x36e394e thru 0x36eb9b3 (0x8066 bytes) 2024-09-14T23:57:32.267 INFO:teuthology.orchestra.run.smithi042.stdout:3036 read 0x43c2435 thru 0x43c625c (0x3e28 bytes) 2024-09-14T23:57:32.267 INFO:teuthology.orchestra.run.smithi042.stdout:3038 read 0xd429401 thru 0xd42d684 (0x4284 bytes) 2024-09-14T23:57:32.267 INFO:teuthology.orchestra.run.smithi042.stdout:3039 read 0x8459a9a thru 0x8464f56 (0xb4bd bytes) 2024-09-14T23:57:32.268 INFO:teuthology.orchestra.run.smithi042.stdout:3041 write 0x4707b8e thru 0x470f2c0 (0x7733 bytes) 2024-09-14T23:57:32.269 INFO:teuthology.orchestra.run.smithi042.stdout:3042 punch from 0x16b8bbb to 0x16c7943, (0xed88 bytes) 2024-09-14T23:57:32.275 INFO:teuthology.orchestra.run.smithi042.stdout:3044 trunc from 0xdd92797 to 0x8300a92 2024-09-14T23:57:32.284 INFO:teuthology.orchestra.run.smithi042.stdout:3045 punch from 0x616b4f to 0x61d68a, (0x6b3b bytes) 2024-09-14T23:57:32.284 INFO:teuthology.orchestra.run.smithi042.stdout:3046 read 0x51c385d thru 0x51cc7ce (0x8f72 bytes) 2024-09-14T23:57:32.293 INFO:teuthology.orchestra.run.smithi042.stdout:3047 write 0xb2ec191 thru 0xb2ec837 (0x6a7 bytes) 2024-09-14T23:57:32.294 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:32.295 INFO:teuthology.orchestra.run.smithi042.stdout:3049 read 0x7f28c7c thru 0x7f2fa4c (0x6dd1 bytes) 2024-09-14T23:57:32.295 INFO:teuthology.orchestra.run.smithi042.stdout:3050 read 0x4db0db2 thru 0x4db70ad (0x62fc bytes) 2024-09-14T23:57:32.303 INFO:teuthology.orchestra.run.smithi042.stdout:3051 trunc from 0xb2ec838 to 0x3acaea1 2024-09-14T23:57:32.313 INFO:teuthology.orchestra.run.smithi042.stdout:3053 read 0x26754ed thru 0x267bc6d (0x6781 bytes) 2024-09-14T23:57:32.319 INFO:teuthology.orchestra.run.smithi042.stdout:3054 trunc from 0x3acaea1 to 0x8c6ff6f 2024-09-14T23:57:32.324 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:32.324 INFO:teuthology.orchestra.run.smithi042.stdout:3056 read 0xc4f8fe thru 0xc54c89 (0x538c bytes) 2024-09-14T23:57:32.325 INFO:teuthology.orchestra.run.smithi042.stdout:3057 punch from 0x7328955 to 0x732db1a, (0x51c5 bytes) 2024-09-14T23:57:32.332 INFO:teuthology.orchestra.run.smithi042.stdout:3058 write 0x96ce277 thru 0x96d9e09 (0xbb93 bytes) 2024-09-14T23:57:32.334 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:32.332+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:32.336 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:32.336 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:32.343 INFO:teuthology.orchestra.run.smithi042.stdout:3059 trunc from 0x96d9e0a to 0xd6fd8a 2024-09-14T23:57:32.351 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T23:57:32.355 INFO:teuthology.orchestra.run.smithi042.stdout:3060 punch from 0xc8bfa2 to 0xc91a7b, (0x5ad9 bytes) 2024-09-14T23:57:32.367 INFO:teuthology.orchestra.run.smithi042.stdout:3061 write 0x6acf4d1 thru 0x6ad208c (0x2bbc bytes) 2024-09-14T23:57:32.370 INFO:teuthology.orchestra.run.smithi042.stdout:3063 writesame 0x5fc42ca thru 0x5fcc364 data_size 0x29(0x809b bytes) 2024-09-14T23:57:32.373 INFO:teuthology.orchestra.run.smithi042.stdout:3064 trunc from 0x6ad208d to 0x7a15d49 2024-09-14T23:57:32.378 INFO:teuthology.orchestra.run.smithi042.stdout:3066 writesame 0x415b9f7 thru 0x416b2cd data_size 0x1(0xf8d7 bytes) 2024-09-14T23:57:32.408 INFO:teuthology.orchestra.run.smithi042.stdout:3067 punch from 0x640356e to 0x6412460, (0xeef2 bytes) 2024-09-14T23:57:32.408 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:32.408 INFO:teuthology.orchestra.run.smithi042.stdout:3069 punch from 0x4fa81c0 to 0x4fb10af, (0x8eef bytes) 2024-09-14T23:57:32.408 INFO:teuthology.orchestra.run.smithi042.stdout:3071 read 0x365dcef thru 0x3663f32 (0x6244 bytes) 2024-09-14T23:57:32.408 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:32.409 INFO:teuthology.orchestra.run.smithi042.stdout:3074 write 0x753d53c thru 0x75453b5 (0x7e7a bytes) 2024-09-14T23:57:32.409 INFO:teuthology.orchestra.run.smithi042.stdout:3075 punch from 0x6d904fa to 0x6d9839b, (0x7ea1 bytes) 2024-09-14T23:57:32.409 INFO:teuthology.orchestra.run.smithi042.stdout:3077 read 0x1d19578 thru 0x1d1c8fa (0x3383 bytes) 2024-09-14T23:57:32.411 INFO:teuthology.orchestra.run.smithi042.stdout:3078 trunc from 0x7a15d49 to 0x6266327 2024-09-14T23:57:32.423 INFO:teuthology.orchestra.run.smithi042.stdout:3082 trunc from 0x6266327 to 0x2b37970 2024-09-14T23:57:32.433 INFO:teuthology.orchestra.run.smithi042.stdout:3083 trunc from 0x2b37970 to 0x4a2ac2e 2024-09-14T23:57:32.435 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:32.433+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:32.437 INFO:teuthology.orchestra.run.smithi042.stdout:3084 read 0x3d2d927 thru 0x3d36881 (0x8f5b bytes) 2024-09-14T23:57:32.448 INFO:teuthology.orchestra.run.smithi042.stdout:3086 write 0x96b10fc thru 0x96befeb (0xdef0 bytes) 2024-09-14T23:57:32.459 INFO:teuthology.orchestra.run.smithi042.stdout:3087 trunc from 0x96befec to 0x17ac833 2024-09-14T23:57:32.466 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:32.466 INFO:teuthology.orchestra.run.smithi042.stdout:3090 read 0x720fd4 thru 0x728404 (0x7431 bytes) 2024-09-14T23:57:32.467 INFO:teuthology.orchestra.run.smithi042.stdout:3091 punch from 0x11d29fb to 0x11d6f15, (0x451a bytes) 2024-09-14T23:57:32.481 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:32.481 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:32.487 INFO:teuthology.orchestra.run.smithi042.stdout:3092 writesame 0xee12478 thru 0xee14c31 data_size 0x2d(0x27ba bytes) 2024-09-14T23:57:32.487 INFO:teuthology.orchestra.run.smithi042.stdout:3095 read 0xd009a89 thru 0xd009bb9 (0x131 bytes) 2024-09-14T23:57:32.487 INFO:teuthology.orchestra.run.smithi042.stdout:3096 write 0x4a8702 thru 0x4b1b21 (0x9420 bytes) 2024-09-14T23:57:32.489 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:32.489 INFO:teuthology.orchestra.run.smithi042.stdout:3098 punch from 0x21676ca to 0x216c42c, (0x4d62 bytes) 2024-09-14T23:57:32.490 INFO:teuthology.orchestra.run.smithi042.stdout:3100 read 0x9dce0cc thru 0x9dda321 (0xc256 bytes) 2024-09-14T23:57:32.490 INFO:teuthology.orchestra.run.smithi042.stdout:3101 read 0x8b5013b thru 0x8b58620 (0x84e6 bytes) 2024-09-14T23:57:32.490 INFO:teuthology.orchestra.run.smithi042.stdout:3102 punch from 0x85b60b to 0x86074b, (0x5140 bytes) 2024-09-14T23:57:32.490 INFO:teuthology.orchestra.run.smithi042.stdout:3103 punch from 0xc963e03 to 0xc96865c, (0x4859 bytes) 2024-09-14T23:57:32.490 INFO:teuthology.orchestra.run.smithi042.stdout:3104 write 0x320d8de thru 0x3212780 (0x4ea3 bytes) 2024-09-14T23:57:32.491 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T23:57:32.491 INFO:teuthology.orchestra.run.smithi042.stdout:3105 write 0x469a023 thru 0x469e310 (0x42ee bytes) 2024-09-14T23:57:32.493 INFO:teuthology.orchestra.run.smithi042.stdout:3106 punch from 0x2240a16 to 0x2246091, (0x567b bytes) 2024-09-14T23:57:32.504 INFO:teuthology.orchestra.run.smithi042.stdout:3107 trunc from 0xee14c32 to 0x4c14c70 2024-09-14T23:57:32.513 INFO:teuthology.orchestra.run.smithi042.stdout:3108 write 0x3bb63ea thru 0x3bbf941 (0x9558 bytes) 2024-09-14T23:57:32.514 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:32.514 INFO:teuthology.orchestra.run.smithi042.stdout:3110 read 0x518ae5 thru 0x51fcb5 (0x71d1 bytes) 2024-09-14T23:57:32.515 INFO:teuthology.orchestra.run.smithi042.stdout:3111 read 0x27ae531 thru 0x27b1942 (0x3412 bytes) 2024-09-14T23:57:32.515 INFO:teuthology.orchestra.run.smithi042.stdout:3112 read 0x3fcff2d thru 0x3fdab64 (0xac38 bytes) 2024-09-14T23:57:32.515 INFO:teuthology.orchestra.run.smithi042.stdout:3113 write 0x25e8b05 thru 0x25f4918 (0xbe14 bytes) 2024-09-14T23:57:32.530 INFO:teuthology.orchestra.run.smithi042.stdout:3114 writesame 0xbeda022 thru 0xbee2246 data_size 0x1(0x8225 bytes) 2024-09-14T23:57:32.531 INFO:teuthology.orchestra.run.smithi042.stdout:3117 writesame 0x835c384 thru 0x8362ef9 data_size 0x69(0x6b76 bytes) 2024-09-14T23:57:32.534 INFO:teuthology.orchestra.run.smithi042.stdout:3118 read 0x5eeca5c thru 0x5efbe14 (0xf3b9 bytes) 2024-09-14T23:57:32.535 INFO:teuthology.orchestra.run.smithi042.stdout:3119 read 0x10c773 thru 0x112102 (0x5990 bytes) 2024-09-14T23:57:32.535 INFO:teuthology.orchestra.run.smithi042.stdout:3120 read 0xa9096c2 thru 0xa90d191 (0x3ad0 bytes) 2024-09-14T23:57:32.535 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:32.536 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:32.536 INFO:teuthology.orchestra.run.smithi042.stdout:3124 write 0x6cbbb87 thru 0x6cc77b1 (0xbc2b bytes) 2024-09-14T23:57:32.536 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:32.534+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:32.538 INFO:teuthology.orchestra.run.smithi042.stdout:3126 writesame 0x16b54b6 thru 0x16c36b5 data_size 0x80(0xe200 bytes) 2024-09-14T23:57:32.553 INFO:teuthology.orchestra.run.smithi042.stdout:3127 trunc from 0xbee2247 to 0xa9c1b2 2024-09-14T23:57:32.568 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:32.573 INFO:teuthology.orchestra.run.smithi042.stdout:3129 trunc from 0xa9c1b2 to 0x511872a 2024-09-14T23:57:32.587 INFO:teuthology.orchestra.run.smithi042.stdout:3130 write 0x6e4aa18 thru 0x6e4faa3 (0x508c bytes) 2024-09-14T23:57:32.590 INFO:teuthology.orchestra.run.smithi042.stdout:3131 read 0x6664b4e thru 0x66742bf (0xf772 bytes) 2024-09-14T23:57:32.590 INFO:teuthology.orchestra.run.smithi042.stdout:3132 write 0x2b7c611 thru 0x2b7d6aa (0x109a bytes) 2024-09-14T23:57:32.593 INFO:teuthology.orchestra.run.smithi042.stdout:3134 punch from 0x2480f01 to 0x2490749, (0xf848 bytes) 2024-09-14T23:57:32.604 INFO:teuthology.orchestra.run.smithi042.stdout:3135 writesame 0xc42f65b thru 0xc43a544 data_size 0x33(0xaeea bytes) 2024-09-14T23:57:32.615 INFO:teuthology.orchestra.run.smithi042.stdout:3136 trunc from 0xc43a545 to 0x385d145 2024-09-14T23:57:32.622 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:32.622 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:32.626 INFO:teuthology.orchestra.run.smithi042.stdout:3140 trunc from 0x385d145 to 0xccb641 2024-09-14T23:57:32.635 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T23:57:32.636 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:32.635+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:32.638 INFO:teuthology.orchestra.run.smithi042.stdout:3141 read 0x3ca0ec thru 0x3d578b (0xb6a0 bytes) 2024-09-14T23:57:32.656 INFO:teuthology.orchestra.run.smithi042.stdout:3142 writesame 0xc053903 thru 0xc05cf1e data_size 0x1a(0x961c bytes) 2024-09-14T23:57:32.663 INFO:teuthology.orchestra.run.smithi042.stdout:3143 writesame 0xdc0cf0c thru 0xdc1af5c data_size 0x5(0xe051 bytes) 2024-09-14T23:57:32.663 INFO:teuthology.orchestra.run.smithi042.stdout:3144 read 0x292cbd5 thru 0x292d485 (0x8b1 bytes) 2024-09-14T23:57:32.664 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:32.664 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:32.664 INFO:teuthology.orchestra.run.smithi042.stdout:3149 read 0xb7991d0 thru 0xb79a777 (0x15a8 bytes) 2024-09-14T23:57:32.664 INFO:teuthology.orchestra.run.smithi042.stdout:3150 read 0xa8d5281 thru 0xa8e1a7e (0xc7fe bytes) 2024-09-14T23:57:32.664 INFO:teuthology.orchestra.run.smithi042.stdout:3152 read 0x1a50896 thru 0x1a50f5c (0x6c7 bytes) 2024-09-14T23:57:32.664 INFO:teuthology.orchestra.run.smithi042.stdout:3153 write 0xb07d09b thru 0xb08596a (0x88d0 bytes) 2024-09-14T23:57:32.665 INFO:teuthology.orchestra.run.smithi042.stdout:3154 write 0x958eae9 thru 0x958ed7c (0x294 bytes) 2024-09-14T23:57:32.666 INFO:teuthology.orchestra.run.smithi042.stdout:3155 read 0x3fe9e3c thru 0x3ff9534 (0xf6f9 bytes) 2024-09-14T23:57:32.667 INFO:teuthology.orchestra.run.smithi042.stdout:3156 write 0xc2811d7 thru 0xc28afa5 (0x9dcf bytes) 2024-09-14T23:57:32.670 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:32.680 INFO:teuthology.orchestra.run.smithi042.stdout:3159 trunc from 0xdc1af5d to 0x42bebb7 2024-09-14T23:57:32.689 INFO:teuthology.orchestra.run.smithi042.stdout:3160 write 0x2a75a6c thru 0x2a7d36f (0x7904 bytes) 2024-09-14T23:57:32.691 INFO:teuthology.orchestra.run.smithi042.stdout:3161 write 0x3dbba23 thru 0x3dbbd1e (0x2fc bytes) 2024-09-14T23:57:32.705 INFO:teuthology.orchestra.run.smithi042.stdout:3162 write 0xae09a5e thru 0xae0fe2e (0x63d1 bytes) 2024-09-14T23:57:32.713 INFO:teuthology.orchestra.run.smithi042.stdout:3163 write 0xc009fd1 thru 0xc00e074 (0x40a4 bytes) 2024-09-14T23:57:32.716 INFO:teuthology.orchestra.run.smithi042.stdout:3165 punch from 0x10b4a89 to 0x10c1227, (0xc79e bytes) 2024-09-14T23:57:32.729 INFO:teuthology.orchestra.run.smithi042.stdout:3167 trunc from 0xc00e075 to 0x7b6d83 2024-09-14T23:57:32.737 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:32.736+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:32.744 INFO:teuthology.orchestra.run.smithi042.stdout:3168 read 0x121f19 thru 0x126573 (0x465b bytes) 2024-09-14T23:57:32.745 INFO:teuthology.orchestra.run.smithi042.stdout:3169 punch from 0x4443cc to 0x44e95e, (0xa592 bytes) 2024-09-14T23:57:32.745 INFO:teuthology.orchestra.run.smithi042.stdout:3171 punch from 0x1d478c to 0x1dae7d, (0x66f1 bytes) 2024-09-14T23:57:32.745 INFO:teuthology.orchestra.run.smithi042.stdout:3172 read 0x3110c3 thru 0x3131f3 (0x2131 bytes) 2024-09-14T23:57:32.745 INFO:teuthology.orchestra.run.smithi042.stdout:3174 read 0x28c58f thru 0x28e943 (0x23b5 bytes) 2024-09-14T23:57:32.746 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:32.746 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:32.746 INFO:teuthology.orchestra.run.smithi042.stdout:3177 punch from 0x2e641b to 0x2f306e, (0xcc53 bytes) 2024-09-14T23:57:32.746 INFO:teuthology.orchestra.run.smithi042.stdout:3178 punch from 0x5f81cc to 0x5fd60a, (0x543e bytes) 2024-09-14T23:57:32.754 INFO:teuthology.orchestra.run.smithi042.stdout:3179 write 0x3625cc6 thru 0x3634fd6 (0xf311 bytes) 2024-09-14T23:57:32.767 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:32.767 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:32.772 INFO:teuthology.orchestra.run.smithi042.stdout:3180 write 0xc3c5ec6 thru 0xc3cc70d (0x6848 bytes) 2024-09-14T23:57:32.773 INFO:teuthology.orchestra.run.smithi042.stdout:3181 read 0x24b11a6 thru 0x24b3749 (0x25a4 bytes) 2024-09-14T23:57:32.773 INFO:teuthology.orchestra.run.smithi042.stdout:3182 writesame 0xb034bec thru 0xb040315 data_size 0x1e(0xb72a bytes) 2024-09-14T23:57:32.777 INFO:teuthology.orchestra.run.smithi042.stdout:3184 punch from 0x974da89 to 0x97533be, (0x5935 bytes) 2024-09-14T23:57:32.778 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T23:57:32.778 INFO:teuthology.orchestra.run.smithi042.stdout:3185 write 0x7affb17 thru 0x7b08575 (0x8a5f bytes) 2024-09-14T23:57:32.780 INFO:teuthology.orchestra.run.smithi042.stdout:3187 read 0x2184e03 thru 0x21911d5 (0xc3d3 bytes) 2024-09-14T23:57:32.780 INFO:teuthology.orchestra.run.smithi042.stdout:3188 write 0xf14521 thru 0xf1f48a (0xaf6a bytes) 2024-09-14T23:57:32.783 INFO:teuthology.orchestra.run.smithi042.stdout:3190 write 0x26c53c3 thru 0x26cdf16 (0x8b54 bytes) 2024-09-14T23:57:32.785 INFO:teuthology.orchestra.run.smithi042.stdout:3191 punch from 0x36dc093 to 0x36e74af, (0xb41c bytes) 2024-09-14T23:57:32.785 INFO:teuthology.orchestra.run.smithi042.stdout:3192 read 0x88e0fb8 thru 0x88e3df9 (0x2e42 bytes) 2024-09-14T23:57:32.785 INFO:teuthology.orchestra.run.smithi042.stdout:3194 read 0x8fd8110 thru 0x8fe06a3 (0x8594 bytes) 2024-09-14T23:57:32.785 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:32.785 INFO:teuthology.orchestra.run.smithi042.stdout:3198 write 0xb834ebd thru 0xb83bf79 (0x70bd bytes) 2024-09-14T23:57:32.788 INFO:teuthology.orchestra.run.smithi042.stdout:3200 read 0x8618e0a thru 0x861b076 (0x226d bytes) 2024-09-14T23:57:32.788 INFO:teuthology.orchestra.run.smithi042.stdout:3201 read 0x22758dd thru 0x227ea97 (0x91bb bytes) 2024-09-14T23:57:32.788 INFO:teuthology.orchestra.run.smithi042.stdout:3203 write 0x34d76a thru 0x355290 (0x7b27 bytes) 2024-09-14T23:57:32.789 INFO:teuthology.orchestra.run.smithi042.stdout:3205 punch from 0x9538ffd to 0x9540066, (0x7069 bytes) 2024-09-14T23:57:32.790 INFO:teuthology.orchestra.run.smithi042.stdout:3206 write 0xacf5335 thru 0xacffaec (0xa7b8 bytes) 2024-09-14T23:57:32.791 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:32.791 INFO:teuthology.orchestra.run.smithi042.stdout:3208 read 0x249dd7 thru 0x258f53 (0xf17d bytes) 2024-09-14T23:57:32.794 INFO:teuthology.orchestra.run.smithi042.stdout:3210 trunc from 0xc3cc70e to 0xdf79ef3 2024-09-14T23:57:32.799 INFO:teuthology.orchestra.run.smithi042.stdout:3212 write 0xc78aea2 thru 0xc791b3e (0x6c9d bytes) 2024-09-14T23:57:32.801 INFO:teuthology.orchestra.run.smithi042.stdout:3213 punch from 0x5f60a08 to 0x5f66355, (0x594d bytes) 2024-09-14T23:57:32.801 INFO:teuthology.orchestra.run.smithi042.stdout:3214 writesame 0x7f9ed53 thru 0x7fa3bf2 data_size 0x4a(0x4ea0 bytes) 2024-09-14T23:57:32.802 INFO:teuthology.orchestra.run.smithi042.stdout:3217 writesame 0xc2f3c1c thru 0xc3025af data_size 0x63(0xe994 bytes) 2024-09-14T23:57:32.804 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:32.804 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:32.804 INFO:teuthology.orchestra.run.smithi042.stdout:3221 write 0x3eb48a4 thru 0x3eb75b5 (0x2d12 bytes) 2024-09-14T23:57:32.820 INFO:teuthology.orchestra.run.smithi042.stdout:3222 trunc from 0xdf79ef3 to 0x791636 2024-09-14T23:57:32.837 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:32.836+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:32.849 INFO:teuthology.orchestra.run.smithi042.stdout:3223 write 0x6880944 thru 0x688598c (0x5049 bytes) 2024-09-14T23:57:32.852 INFO:teuthology.orchestra.run.smithi042.stdout:3224 read 0x2d210f1 thru 0x2d21369 (0x279 bytes) 2024-09-14T23:57:32.852 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:32.852 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:32.852 INFO:teuthology.orchestra.run.smithi042.stdout:3227 read 0x44f8702 thru 0x44fd3f2 (0x4cf1 bytes) 2024-09-14T23:57:32.853 INFO:teuthology.orchestra.run.smithi042.stdout:3228 punch from 0x436e58b to 0x4376ce5, (0x875a bytes) 2024-09-14T23:57:32.853 INFO:teuthology.orchestra.run.smithi042.stdout:3229 read 0x5fbd276 thru 0x5fc5592 (0x831d bytes) 2024-09-14T23:57:32.853 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:32.853 INFO:teuthology.orchestra.run.smithi042.stdout:3231 read 0x61770c0 thru 0x617ae63 (0x3da4 bytes) 2024-09-14T23:57:32.858 INFO:teuthology.orchestra.run.smithi042.stdout:3232 trunc from 0x688598d to 0x10d5e62 2024-09-14T23:57:32.868 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:32.888 INFO:teuthology.orchestra.run.smithi042.stdout:3235 writesame 0xea65087 thru 0xea6a6c7 data_size 0x1(0x5641 bytes) 2024-09-14T23:57:32.888 INFO:teuthology.orchestra.run.smithi042.stdout:3237 write 0x3b1ddab thru 0x3b260f5 (0x834b bytes) 2024-09-14T23:57:32.898 INFO:teuthology.orchestra.run.smithi042.stdout:3238 trunc from 0xea6a6c8 to 0x71257bb 2024-09-14T23:57:32.905 INFO:teuthology.orchestra.run.smithi042.stdout:3240 write 0x55f988d thru 0x5604bbb (0xb32f bytes) 2024-09-14T23:57:32.907 INFO:teuthology.orchestra.run.smithi042.stdout:3241 punch from 0xe84f87 to 0xe910a7, (0xc120 bytes) 2024-09-14T23:57:32.907 INFO:teuthology.orchestra.run.smithi042.stdout:3244 read 0x52a08f6 thru 0x52a1181 (0x88c bytes) 2024-09-14T23:57:32.908 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:32.909 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:32.922 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T23:57:32.922 INFO:teuthology.orchestra.run.smithi042.stdout:3245 write 0xe480ec4 thru 0xe4890c6 (0x8203 bytes) 2024-09-14T23:57:32.923 INFO:teuthology.orchestra.run.smithi042.stdout:3246 read 0xe3b2e27 thru 0xe3be04f (0xb229 bytes) 2024-09-14T23:57:32.923 INFO:teuthology.orchestra.run.smithi042.stdout:3247 write 0x47e5177 thru 0x47ef9d7 (0xa861 bytes) 2024-09-14T23:57:32.928 INFO:teuthology.orchestra.run.smithi042.stdout:3249 read 0x2dd3270 thru 0x2de13c5 (0xe156 bytes) 2024-09-14T23:57:32.928 INFO:teuthology.orchestra.run.smithi042.stdout:3250 write 0xa8faa31 thru 0xa9058e5 (0xaeb5 bytes) 2024-09-14T23:57:32.928 INFO:teuthology.orchestra.run.smithi042.stdout:3252 writesame 0x3bf9630 thru 0x3bf986d data_size 0x7(0x23e bytes) 2024-09-14T23:57:32.929 INFO:teuthology.orchestra.run.smithi042.stdout:3253 write 0x5d2037c thru 0x5d2c138 (0xbdbd bytes) 2024-09-14T23:57:32.933 INFO:teuthology.orchestra.run.smithi042.stdout:3254 read 0x82bd905 thru 0x82cc3a0 (0xea9c bytes) 2024-09-14T23:57:32.933 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:32.933 INFO:teuthology.orchestra.run.smithi042.stdout:3256 write 0x4e7bf38 thru 0x4e82df3 (0x6ebc bytes) 2024-09-14T23:57:32.936 INFO:teuthology.orchestra.run.smithi042.stdout:3257 punch from 0x481fb04 to 0x4823b4d, (0x4049 bytes) 2024-09-14T23:57:32.936 INFO:teuthology.orchestra.run.smithi042.stdout:3258 write 0x1037afd thru 0x103c02c (0x4530 bytes) 2024-09-14T23:57:32.938 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:32.937+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:32.939 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:32.939 INFO:teuthology.orchestra.run.smithi042.stdout:3262 read 0xa7aeae0 thru 0xa7be211 (0xf732 bytes) 2024-09-14T23:57:32.939 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:32.941 INFO:teuthology.orchestra.run.smithi042.stdout:3264 trunc from 0xe4890c7 to 0xc335bf9 2024-09-14T23:57:32.949 INFO:teuthology.orchestra.run.smithi042.stdout:3265 read 0x4f36267 thru 0x4f3d46e (0x7208 bytes) 2024-09-14T23:57:32.949 INFO:teuthology.orchestra.run.smithi042.stdout:3266 read 0xbeb9441 thru 0xbec51a9 (0xbd69 bytes) 2024-09-14T23:57:32.950 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:32.950 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:32.950 INFO:teuthology.orchestra.run.smithi042.stdout:3270 writesame 0xbbb7737 thru 0xbbc0dfe data_size 0x64(0x96c8 bytes) 2024-09-14T23:57:32.952 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:32.952 INFO:teuthology.orchestra.run.smithi042.stdout:3272 clone 30 order 19 su 8192 sc 15 2024-09-14T23:57:33.011 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T23:57:33.044 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:33.044 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:33.044 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:33.043+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:33.054 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T23:57:33.073 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone29 from 0xc335bf9 (overlap 0x791636) to 0x54c2bf 2024-09-14T23:57:33.139 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:33.138+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:33.169 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-09-14T23:57:33.185 INFO:teuthology.orchestra.run.smithi042.stdout:3273 write 0xc21c43d thru 0xc22752c (0xb0f0 bytes) 2024-09-14T23:57:33.211 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:33.214 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:33.214 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:33.218 INFO:teuthology.orchestra.run.smithi042.stdout:3277 writesame 0xd85ef95 thru 0xd860291 data_size 0x1(0x12fd bytes) 2024-09-14T23:57:33.218 INFO:teuthology.orchestra.run.smithi042.stdout:3278 write 0xbcde14a thru 0xbcec237 (0xe0ee bytes) 2024-09-14T23:57:33.225 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T23:57:33.228 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T23:57:33.240 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:33.240 INFO:teuthology.orchestra.run.smithi042.stdout:3280 writesame 0x1924b11 thru 0x1929197 data_size 0x17(0x4687 bytes) 2024-09-14T23:57:33.241 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:33.239+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:33.254 INFO:teuthology.orchestra.run.smithi042.stdout:3281 writesame 0x15a43bf thru 0x15a8c8e data_size 0x28(0x48d0 bytes) 2024-09-14T23:57:33.271 INFO:teuthology.orchestra.run.smithi042.stdout:3283 write 0x22e5263 thru 0x22e6139 (0xed7 bytes) 2024-09-14T23:57:33.287 INFO:teuthology.orchestra.run.smithi042.stdout:3284 write 0x323a477 thru 0x3242483 (0x800d bytes) 2024-09-14T23:57:33.315 INFO:teuthology.orchestra.run.smithi042.stdout:3285 write 0x3d6da5f thru 0x3d7302d (0x55cf bytes) 2024-09-14T23:57:33.338 INFO:teuthology.orchestra.run.smithi042.stdout:3289 read 0x85f4f19 thru 0x85f6a64 (0x1b4c bytes) 2024-09-14T23:57:33.338 INFO:teuthology.orchestra.run.smithi042.stdout:3293 read 0x1d3b2f7 thru 0x1d45014 (0x9d1e bytes) 2024-09-14T23:57:33.338 INFO:teuthology.orchestra.run.smithi042.stdout:3295 read 0x1658ad5 thru 0x165f5fe (0x6b2a bytes) 2024-09-14T23:57:33.341 INFO:teuthology.orchestra.run.smithi042.stdout:3296 trunc from 0xd860292 to 0xb824658 2024-09-14T23:57:33.341 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:33.340+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:33.407 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:33.407 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:33.421 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T23:57:33.445 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:57:33.445 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:57:33.445 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:33.444+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:33.448 INFO:teuthology.orchestra.run.smithi042.stdout:3297 read 0x5d1588e thru 0x5d1bb5c (0x62cf bytes) 2024-09-14T23:57:33.449 INFO:teuthology.orchestra.run.smithi042.stdout:3298 punch from 0x6d1602a to 0x6d2349a, (0xd470 bytes) 2024-09-14T23:57:33.449 INFO:teuthology.orchestra.run.smithi042.stdout:3299 read 0x454c8d2 thru 0x454f227 (0x2956 bytes) 2024-09-14T23:57:33.449 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:33.449 INFO:teuthology.orchestra.run.smithi042.stdout:3302 writesame 0xafe950a thru 0xafecfdd data_size 0x1e(0x3ad4 bytes) 2024-09-14T23:57:33.463 INFO:teuthology.orchestra.run.smithi042.stdout:3303 write 0x1c6a27b thru 0x1c6ddbb (0x3b41 bytes) 2024-09-14T23:57:33.470 INFO:teuthology.orchestra.run.smithi042.stdout:3304 read 0xbd834c thru 0xbdc815 (0x44ca bytes) 2024-09-14T23:57:33.470 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:57:33.471 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:57:33.472 INFO:teuthology.orchestra.run.smithi042.stdout:3305 trunc from 0xb824658 to 0xcf990ba 2024-09-14T23:57:33.482 INFO:teuthology.orchestra.run.smithi042.stdout:3307 write 0xd33cebe thru 0xd341d6b (0x4eae bytes) 2024-09-14T23:57:33.486 INFO:teuthology.orchestra.run.smithi042.stdout:3309 write 0x9d45143 thru 0x9d4a064 (0x4f22 bytes) 2024-09-14T23:57:33.489 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:57:33.489 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:57:33.503 INFO:teuthology.orchestra.run.smithi042.stdout:3310 writesame 0x4dc05e5 thru 0x4dc97bb data_size 0x39(0x91d7 bytes) 2024-09-14T23:57:33.512 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:57:33.512 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:57:33.524 INFO:teuthology.orchestra.run.smithi042.stdout:3311 write 0xc66798b thru 0xc6737d7 (0xbe4d bytes) 2024-09-14T23:57:33.533 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:57:33.533 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:57:33.535 INFO:teuthology.orchestra.run.smithi042.stdout:3312 writesame 0xb6e7a1b thru 0xb6e8bca data_size 0x8(0x11b0 bytes) 2024-09-14T23:57:33.536 INFO:teuthology.orchestra.run.smithi042.stdout:3313 read 0x72ebe86 thru 0x72fa22b (0xe3a6 bytes) 2024-09-14T23:57:33.537 INFO:teuthology.orchestra.run.smithi042.stdout:3314 read 0x9c6792e thru 0x9c74b55 (0xd228 bytes) 2024-09-14T23:57:33.539 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:33.539 INFO:teuthology.orchestra.run.smithi042.stdout:3317 write 0x1b0927b thru 0x1b15799 (0xc51f bytes) 2024-09-14T23:57:33.544 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:33.544 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:33.544 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:57:33.544 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:33.544 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:33.543+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:33.552 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T23:57:33.561 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:57:33.561 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:57:33.562 INFO:teuthology.orchestra.run.smithi042.stdout:3319 read 0x698bfaf thru 0x698d5ee (0x1640 bytes) 2024-09-14T23:57:33.577 INFO:teuthology.orchestra.run.smithi042.stdout:3320 trunc from 0xd341d6c to 0x117316 2024-09-14T23:57:33.579 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:57:33.579 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:57:33.609 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:57:33.609 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:57:33.644 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:33.643+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:33.703 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:33.703 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:33.703 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:57:33.703 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:57:33.703 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:33.713 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T23:57:33.748 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:33.744+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:33.750 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:33.750 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:33.764 INFO:teuthology.orchestra.run.smithi042.stdout:3324 trunc from 0x117316 to 0xcb26cd6 2024-09-14T23:57:33.771 INFO:teuthology.orchestra.run.smithi042.stdout:3325 punch from 0x5602a63 to 0x561047d, (0xda1a bytes) 2024-09-14T23:57:33.771 INFO:teuthology.orchestra.run.smithi042.stdout:3326 read 0x7ca34be thru 0x7caa9bb (0x74fe bytes) 2024-09-14T23:57:33.771 INFO:teuthology.orchestra.run.smithi042.stdout:3328 read 0x176a3b thru 0x180c50 (0xa216 bytes) 2024-09-14T23:57:33.773 INFO:teuthology.orchestra.run.smithi042.stdout:3329 trunc from 0xcb26cd6 to 0xad23738 2024-09-14T23:57:33.780 INFO:teuthology.orchestra.run.smithi042.stdout:3330 write 0x451586d thru 0x451e778 (0x8f0c bytes) 2024-09-14T23:57:33.788 INFO:teuthology.orchestra.run.smithi042.stdout:3331 read 0x5b1e4e7 thru 0x5b276a8 (0x91c2 bytes) 2024-09-14T23:57:33.788 INFO:teuthology.orchestra.run.smithi042.stdout:3332 write 0x1d7f2be thru 0x1d84ce7 (0x5a2a bytes) 2024-09-14T23:57:33.794 INFO:teuthology.orchestra.run.smithi042.stdout:3335 write 0x4b1664a thru 0x4b20937 (0xa2ee bytes) 2024-09-14T23:57:33.802 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:33.802 INFO:teuthology.orchestra.run.smithi042.stdout:3337 writesame 0x97c85e9 thru 0x97d2b09 data_size 0x21(0xa521 bytes) 2024-09-14T23:57:33.809 INFO:teuthology.orchestra.run.smithi042.stdout:3338 writesame 0x8b9f309 thru 0x8ba3329 data_size 0x1(0x4021 bytes) 2024-09-14T23:57:33.817 INFO:teuthology.orchestra.run.smithi042.stdout:3339 punch from 0x2ffc473 to 0x3000ec3, (0x4a50 bytes) 2024-09-14T23:57:33.828 INFO:teuthology.orchestra.run.smithi042.stdout:3340 trunc from 0xad23738 to 0x34bae6 2024-09-14T23:57:33.847 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:33.845+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:33.847 INFO:teuthology.orchestra.run.smithi042.stdout:3341 write 0xbb2db thru 0xc7036 (0xbd5c bytes) 2024-09-14T23:57:33.849 INFO:teuthology.orchestra.run.smithi042.stdout:3342 read 0x1e92cf thru 0x1f1fe6 (0x8d18 bytes) 2024-09-14T23:57:33.857 INFO:teuthology.orchestra.run.smithi042.stdout:3343 write 0xb22449 thru 0xb2fb0d (0xd6c5 bytes) 2024-09-14T23:57:33.865 INFO:teuthology.orchestra.run.smithi042.stdout:3344 read 0x1976e0 thru 0x19fcfe (0x861f bytes) 2024-09-14T23:57:33.884 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:33.884 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:57:33.884 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:57:33.884 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:57:33.884 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.884 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:3500 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T23:57:33.884 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:37.990193+0000", 2024-09-14T23:57:33.884 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 55.891140022999998, 2024-09-14T23:57:33.885 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99440750899999997, 2024-09-14T23:57:33.885 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:33.885 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:33.885 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:33.885 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:33.885 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:33.885 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:33.885 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3500 2024-09-14T23:57:33.885 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.885 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:33.885 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.885 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:33.885 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990193+0000", 2024-09-14T23:57:33.885 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:33.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.886 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.886 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:33.886 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990193+0000", 2024-09-14T23:57:33.886 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:33.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.886 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.886 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:33.886 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990194+0000", 2024-09-14T23:57:33.886 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5000000000000001e-07 2024-09-14T23:57:33.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.886 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.886 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:33.886 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990200+0000", 2024-09-14T23:57:33.887 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4389999999999997e-06 2024-09-14T23:57:33.887 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.887 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.887 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:33.887 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990215+0000", 2024-09-14T23:57:33.887 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4613999999999999e-05 2024-09-14T23:57:33.887 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.887 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.887 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:33.887 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990224+0000", 2024-09-14T23:57:33.887 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1740000000000006e-06 2024-09-14T23:57:33.887 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.887 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.888 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:33.888 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990283+0000", 2024-09-14T23:57:33.888 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9231999999999998e-05 2024-09-14T23:57:33.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.888 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.888 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:57:33.888 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990291+0000", 2024-09-14T23:57:33.888 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5900000000000002e-06 2024-09-14T23:57:33.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.888 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.888 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:33.888 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:38.984249+0000", 2024-09-14T23:57:33.888 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99395822700000003 2024-09-14T23:57:33.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.889 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.889 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:33.889 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:38.984448+0000", 2024-09-14T23:57:33.889 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00019855900000000001 2024-09-14T23:57:33.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.889 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.889 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:33.889 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:38.984601+0000", 2024-09-14T23:57:33.889 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000152824 2024-09-14T23:57:33.889 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:33.889 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:33.889 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:33.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.889 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.890 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4055 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1093632~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:33.890 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099066+0000", 2024-09-14T23:57:33.890 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 54.782267134999998, 2024-09-14T23:57:33.890 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044174782000000003, 2024-09-14T23:57:33.890 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:33.890 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:33.890 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:33.890 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:33.890 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:33.890 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:33.890 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4055 2024-09-14T23:57:33.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.890 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:33.891 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.891 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:33.891 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099066+0000", 2024-09-14T23:57:33.891 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:33.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.891 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.891 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:33.891 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099066+0000", 2024-09-14T23:57:33.891 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:33.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.891 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.891 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:33.891 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099066+0000", 2024-09-14T23:57:33.892 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3700000000000001e-07 2024-09-14T23:57:33.892 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.892 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.892 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:33.892 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099067+0000", 2024-09-14T23:57:33.892 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1379999999999999e-06 2024-09-14T23:57:33.892 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.892 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.892 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:33.892 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099073+0000", 2024-09-14T23:57:33.892 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4790000000000004e-06 2024-09-14T23:57:33.892 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.892 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.893 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:33.893 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099075+0000", 2024-09-14T23:57:33.893 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5320000000000001e-06 2024-09-14T23:57:33.893 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.893 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.893 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:33.893 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143147+0000", 2024-09-14T23:57:33.893 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044071676999999997 2024-09-14T23:57:33.893 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.893 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.893 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:33.893 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143186+0000", 2024-09-14T23:57:33.893 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8821e-05 2024-09-14T23:57:33.894 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.894 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.894 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:33.894 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143241+0000", 2024-09-14T23:57:33.894 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4797999999999999e-05 2024-09-14T23:57:33.894 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:33.894 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:33.894 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:33.894 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.894 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.894 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4056 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1155072~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:33.894 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099079+0000", 2024-09-14T23:57:33.894 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 54.782254420000001, 2024-09-14T23:57:33.895 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044295020999999997, 2024-09-14T23:57:33.895 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:33.895 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:33.895 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:33.895 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:33.895 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:33.895 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:33.895 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4056 2024-09-14T23:57:33.895 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.895 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:33.895 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.895 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:33.895 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099079+0000", 2024-09-14T23:57:33.896 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:33.896 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.896 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.896 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:33.896 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099079+0000", 2024-09-14T23:57:33.896 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:33.896 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.896 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.896 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:33.896 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099079+0000", 2024-09-14T23:57:33.896 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1500000000000001e-07 2024-09-14T23:57:33.896 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.896 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.897 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:33.897 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099080+0000", 2024-09-14T23:57:33.897 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0210000000000001e-06 2024-09-14T23:57:33.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.897 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.897 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:33.897 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099083+0000", 2024-09-14T23:57:33.897 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.174e-06 2024-09-14T23:57:33.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.897 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.897 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:33.897 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099086+0000", 2024-09-14T23:57:33.897 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.446e-06 2024-09-14T23:57:33.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.898 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.898 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:33.898 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143269+0000", 2024-09-14T23:57:33.898 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044183489999999999 2024-09-14T23:57:33.898 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.898 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.898 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:33.898 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143314+0000", 2024-09-14T23:57:33.898 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5135000000000001e-05 2024-09-14T23:57:33.898 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.898 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.898 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:33.898 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143374+0000", 2024-09-14T23:57:33.899 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9540000000000003e-05 2024-09-14T23:57:33.899 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:33.899 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:33.899 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:33.899 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.899 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.899 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4057 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1216512~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:33.899 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099088+0000", 2024-09-14T23:57:33.899 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 54.782244632999998, 2024-09-14T23:57:33.899 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044406119000000001, 2024-09-14T23:57:33.899 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:33.899 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:33.899 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:33.899 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:33.900 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:33.900 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:33.900 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4057 2024-09-14T23:57:33.900 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.900 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:33.900 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.900 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:33.900 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099088+0000", 2024-09-14T23:57:33.900 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:33.900 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.900 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.900 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:33.900 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099088+0000", 2024-09-14T23:57:33.901 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:33.901 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.901 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.901 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:33.901 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099089+0000", 2024-09-14T23:57:33.901 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.04e-07 2024-09-14T23:57:33.901 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.901 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.901 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:33.901 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099090+0000", 2024-09-14T23:57:33.901 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.026e-06 2024-09-14T23:57:33.901 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.901 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.902 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:33.902 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099093+0000", 2024-09-14T23:57:33.902 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0819999999999999e-06 2024-09-14T23:57:33.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.902 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.902 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:33.902 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099095+0000", 2024-09-14T23:57:33.902 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4930000000000001e-06 2024-09-14T23:57:33.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.902 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.902 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:33.902 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143400+0000", 2024-09-14T23:57:33.902 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044304698000000003 2024-09-14T23:57:33.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.903 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.903 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:33.903 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143441+0000", 2024-09-14T23:57:33.903 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1100000000000003e-05 2024-09-14T23:57:33.903 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.903 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.903 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:33.903 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143495+0000", 2024-09-14T23:57:33.903 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3516000000000001e-05 2024-09-14T23:57:33.903 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:33.903 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:33.903 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:33.903 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.904 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.904 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4058 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1277952~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:33.904 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099099+0000", 2024-09-14T23:57:33.904 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 54.782234142, 2024-09-14T23:57:33.904 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044514583000000003, 2024-09-14T23:57:33.904 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:33.904 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:33.904 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:33.904 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:33.904 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:33.904 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:33.904 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4058 2024-09-14T23:57:33.904 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.905 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:33.905 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.905 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:33.905 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099099+0000", 2024-09-14T23:57:33.905 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:33.905 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.905 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.905 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:33.905 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099099+0000", 2024-09-14T23:57:33.905 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:33.905 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.905 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.905 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:33.906 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099099+0000", 2024-09-14T23:57:33.906 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9999999999999999e-07 2024-09-14T23:57:33.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.906 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.906 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:33.906 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099100+0000", 2024-09-14T23:57:33.906 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7199999999999997e-07 2024-09-14T23:57:33.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.906 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.906 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:33.906 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099103+0000", 2024-09-14T23:57:33.906 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.095e-06 2024-09-14T23:57:33.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.907 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.907 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:33.907 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099106+0000", 2024-09-14T23:57:33.907 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4229999999999999e-06 2024-09-14T23:57:33.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.907 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.907 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:33.907 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143524+0000", 2024-09-14T23:57:33.907 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044417891000000001 2024-09-14T23:57:33.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.908 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.908 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:33.908 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143568+0000", 2024-09-14T23:57:33.908 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4060000000000002e-05 2024-09-14T23:57:33.908 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.908 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.908 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:33.908 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143614+0000", 2024-09-14T23:57:33.908 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5942000000000002e-05 2024-09-14T23:57:33.908 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:33.908 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:33.908 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:33.908 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.909 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.909 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4059 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1339392~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:33.909 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099114+0000", 2024-09-14T23:57:33.909 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 54.782219603000001, 2024-09-14T23:57:33.909 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044617194999999998, 2024-09-14T23:57:33.909 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:33.909 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:33.909 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:33.909 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:33.909 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:33.909 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:33.909 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4059 2024-09-14T23:57:33.909 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.909 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:33.910 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.910 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:33.910 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099114+0000", 2024-09-14T23:57:33.910 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:33.910 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.910 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.910 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:33.910 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099114+0000", 2024-09-14T23:57:33.910 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:33.910 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.910 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.910 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:33.910 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099114+0000", 2024-09-14T23:57:33.910 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5899999999999998e-07 2024-09-14T23:57:33.911 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.911 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.911 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:33.911 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099115+0000", 2024-09-14T23:57:33.911 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3510000000000001e-06 2024-09-14T23:57:33.911 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.911 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.911 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:33.911 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099119+0000", 2024-09-14T23:57:33.911 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.16e-06 2024-09-14T23:57:33.911 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.911 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.911 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:33.911 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099123+0000", 2024-09-14T23:57:33.911 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1339999999999997e-06 2024-09-14T23:57:33.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.912 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.912 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:33.912 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143643+0000", 2024-09-14T23:57:33.912 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044519232999999998 2024-09-14T23:57:33.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.912 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.912 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:33.912 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143686+0000", 2024-09-14T23:57:33.912 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3349000000000001e-05 2024-09-14T23:57:33.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.912 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.912 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:33.912 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143731+0000", 2024-09-14T23:57:33.913 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4709000000000002e-05 2024-09-14T23:57:33.913 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:33.913 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:33.913 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:33.913 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.913 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.913 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4060 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1400832~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:33.913 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099129+0000", 2024-09-14T23:57:33.913 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 54.782204380000003, 2024-09-14T23:57:33.913 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044721271999999999, 2024-09-14T23:57:33.913 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:33.913 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:33.913 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:33.914 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:33.914 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:33.914 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:33.914 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4060 2024-09-14T23:57:33.914 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.914 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:33.914 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.914 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:33.914 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099129+0000", 2024-09-14T23:57:33.914 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:33.914 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.914 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.914 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:33.914 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099129+0000", 2024-09-14T23:57:33.915 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:33.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.915 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.915 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:33.915 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099191+0000", 2024-09-14T23:57:33.915 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1805999999999997e-05 2024-09-14T23:57:33.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.915 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.915 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:33.915 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099193+0000", 2024-09-14T23:57:33.916 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2330000000000001e-06 2024-09-14T23:57:33.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.916 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.916 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:33.916 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099200+0000", 2024-09-14T23:57:33.916 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4200000000000001e-06 2024-09-14T23:57:33.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.916 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.916 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:33.916 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099204+0000", 2024-09-14T23:57:33.916 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7500000000000001e-06 2024-09-14T23:57:33.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.917 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.917 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:33.917 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143760+0000", 2024-09-14T23:57:33.917 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044556182 2024-09-14T23:57:33.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.917 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.917 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:33.917 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143804+0000", 2024-09-14T23:57:33.917 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4239000000000001e-05 2024-09-14T23:57:33.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.917 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.917 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:33.917 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143850+0000", 2024-09-14T23:57:33.917 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5642000000000001e-05 2024-09-14T23:57:33.918 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:33.918 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:33.918 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:33.918 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.918 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.918 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4061 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1462272~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:33.918 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099215+0000", 2024-09-14T23:57:33.918 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 54.782118255, 2024-09-14T23:57:33.918 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044745659, 2024-09-14T23:57:33.918 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:33.918 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:33.918 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:33.918 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:33.919 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:33.919 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:33.919 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4061 2024-09-14T23:57:33.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.919 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:33.919 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.919 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:33.919 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099215+0000", 2024-09-14T23:57:33.919 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:33.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.919 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.919 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:33.919 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099215+0000", 2024-09-14T23:57:33.919 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:33.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.920 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.920 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:33.920 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100672+0000", 2024-09-14T23:57:33.920 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00145689 2024-09-14T23:57:33.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.920 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.920 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:33.920 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100677+0000", 2024-09-14T23:57:33.920 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0139999999999998e-06 2024-09-14T23:57:33.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.920 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.920 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:33.920 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100689+0000", 2024-09-14T23:57:33.920 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.183e-05 2024-09-14T23:57:33.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.921 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.921 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:33.921 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100696+0000", 2024-09-14T23:57:33.921 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2599999999999999e-06 2024-09-14T23:57:33.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.921 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.921 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:33.921 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143877+0000", 2024-09-14T23:57:33.921 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043181279000000003 2024-09-14T23:57:33.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.921 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.921 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:33.921 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143919+0000", 2024-09-14T23:57:33.922 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1783000000000003e-05 2024-09-14T23:57:33.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.922 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.922 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:33.922 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143961+0000", 2024-09-14T23:57:33.922 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1603000000000001e-05 2024-09-14T23:57:33.922 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:33.922 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:33.922 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:33.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.922 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.922 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4070 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 2015232~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:33.922 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.100903+0000", 2024-09-14T23:57:33.922 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 54.780430547000002, 2024-09-14T23:57:33.923 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044103815999999997, 2024-09-14T23:57:33.923 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:33.923 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:33.923 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:33.923 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:33.923 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:33.923 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:33.923 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4070 2024-09-14T23:57:33.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.923 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:33.923 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.923 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:33.923 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100903+0000", 2024-09-14T23:57:33.923 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:33.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.924 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.924 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:33.924 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100903+0000", 2024-09-14T23:57:33.924 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:33.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.924 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.924 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:33.924 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.102671+0000", 2024-09-14T23:57:33.924 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001768279 2024-09-14T23:57:33.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.924 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.924 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:33.924 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.102675+0000", 2024-09-14T23:57:33.925 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9609999999999999e-06 2024-09-14T23:57:33.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.925 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.925 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:33.925 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.102684+0000", 2024-09-14T23:57:33.925 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.329e-06 2024-09-14T23:57:33.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.925 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.925 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:33.925 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.102689+0000", 2024-09-14T23:57:33.925 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7079999999999996e-06 2024-09-14T23:57:33.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.925 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.925 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:33.926 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.144912+0000", 2024-09-14T23:57:33.926 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.042223305000000003 2024-09-14T23:57:33.926 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.926 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.926 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:33.926 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.144960+0000", 2024-09-14T23:57:33.926 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7670000000000003e-05 2024-09-14T23:57:33.926 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.926 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.926 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:33.926 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.145006+0000", 2024-09-14T23:57:33.926 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6563999999999997e-05 2024-09-14T23:57:33.926 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:33.926 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:33.927 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:33.927 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.927 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.927 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4096 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1282048~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:33.927 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.104957+0000", 2024-09-14T23:57:33.927 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 54.776376374000002, 2024-09-14T23:57:33.927 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044093016999999998, 2024-09-14T23:57:33.927 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:33.927 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:33.927 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:33.927 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:33.927 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:33.927 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:33.927 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4096 2024-09-14T23:57:33.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.928 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:33.928 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.928 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:33.928 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104957+0000", 2024-09-14T23:57:33.928 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:33.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.928 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.928 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:33.928 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104957+0000", 2024-09-14T23:57:33.928 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:33.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.928 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.929 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:33.929 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104957+0000", 2024-09-14T23:57:33.929 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7400000000000003e-07 2024-09-14T23:57:33.929 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.929 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.929 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:33.929 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104960+0000", 2024-09-14T23:57:33.929 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3800000000000001e-06 2024-09-14T23:57:33.929 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.929 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.929 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:33.929 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104967+0000", 2024-09-14T23:57:33.929 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4030000000000001e-06 2024-09-14T23:57:33.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.930 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.930 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:33.930 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104973+0000", 2024-09-14T23:57:33.930 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7370000000000003e-06 2024-09-14T23:57:33.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.930 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.930 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:33.930 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.148883+0000", 2024-09-14T23:57:33.930 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043910124000000002 2024-09-14T23:57:33.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.930 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.930 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:33.930 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.148966+0000", 2024-09-14T23:57:33.931 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2868999999999996e-05 2024-09-14T23:57:33.931 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.931 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.931 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:33.931 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149050+0000", 2024-09-14T23:57:33.931 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3930000000000004e-05 2024-09-14T23:57:33.931 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:33.931 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:33.931 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:33.931 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.931 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.931 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4097 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1343488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:33.931 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.104983+0000", 2024-09-14T23:57:33.931 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 54.776350585000003, 2024-09-14T23:57:33.932 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044182104999999999, 2024-09-14T23:57:33.932 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:33.932 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:33.932 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:33.932 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:33.932 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:33.932 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:33.932 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4097 2024-09-14T23:57:33.932 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.932 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:33.932 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.932 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:33.932 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104983+0000", 2024-09-14T23:57:33.932 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:33.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.933 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.933 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:33.933 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104983+0000", 2024-09-14T23:57:33.933 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:33.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.933 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.933 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:33.933 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105207+0000", 2024-09-14T23:57:33.933 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000224687 2024-09-14T23:57:33.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.933 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.933 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:33.933 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105211+0000", 2024-09-14T23:57:33.934 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3050000000000001e-06 2024-09-14T23:57:33.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.934 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.934 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:33.934 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105217+0000", 2024-09-14T23:57:33.934 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8170000000000001e-06 2024-09-14T23:57:33.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.934 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.934 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:33.934 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105222+0000", 2024-09-14T23:57:33.934 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.267e-06 2024-09-14T23:57:33.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.934 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.934 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:33.935 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149079+0000", 2024-09-14T23:57:33.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043857581999999999 2024-09-14T23:57:33.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.935 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:33.935 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149122+0000", 2024-09-14T23:57:33.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2899999999999999e-05 2024-09-14T23:57:33.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.935 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:33.935 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149165+0000", 2024-09-14T23:57:33.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2546999999999998e-05 2024-09-14T23:57:33.935 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:33.935 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:33.936 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:33.936 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.936 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.936 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4098 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1404928~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:33.936 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.105242+0000", 2024-09-14T23:57:33.936 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 54.776090773, 2024-09-14T23:57:33.936 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044121104000000001, 2024-09-14T23:57:33.936 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:33.936 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:33.936 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:33.936 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:33.936 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:33.936 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:33.937 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4098 2024-09-14T23:57:33.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.937 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:33.937 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.937 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:33.937 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105242+0000", 2024-09-14T23:57:33.937 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:33.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.937 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.937 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:33.937 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105242+0000", 2024-09-14T23:57:33.937 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:33.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.938 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.938 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:33.938 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105243+0000", 2024-09-14T23:57:33.938 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0399999999999995e-07 2024-09-14T23:57:33.938 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.938 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.938 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:33.938 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105246+0000", 2024-09-14T23:57:33.938 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6309999999999999e-06 2024-09-14T23:57:33.938 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.938 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.938 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:33.938 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105253+0000", 2024-09-14T23:57:33.938 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0940000000000002e-06 2024-09-14T23:57:33.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.939 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:33.939 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105260+0000", 2024-09-14T23:57:33.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1799999999999999e-06 2024-09-14T23:57:33.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.939 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:33.939 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149201+0000", 2024-09-14T23:57:33.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043940629000000002 2024-09-14T23:57:33.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.939 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:33.939 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149283+0000", 2024-09-14T23:57:33.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2360000000000004e-05 2024-09-14T23:57:33.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:33.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149363+0000", 2024-09-14T23:57:33.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0506000000000005e-05 2024-09-14T23:57:33.940 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:33.940 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:33.940 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:33.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.940 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4099 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1466368~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:33.940 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.105270+0000", 2024-09-14T23:57:33.940 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 54.776063178000001, 2024-09-14T23:57:33.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044195433999999999, 2024-09-14T23:57:33.941 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:33.941 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:33.941 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:33.941 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:33.941 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:33.941 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:33.941 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4099 2024-09-14T23:57:33.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.941 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:33.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.941 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:33.941 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105270+0000", 2024-09-14T23:57:33.942 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:33.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.942 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.942 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:33.942 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105270+0000", 2024-09-14T23:57:33.942 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:33.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.942 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.942 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:33.942 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105429+0000", 2024-09-14T23:57:33.942 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015940299999999999 2024-09-14T23:57:33.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.942 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.942 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:33.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105433+0000", 2024-09-14T23:57:33.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3579999999999999e-06 2024-09-14T23:57:33.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.943 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:33.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105439+0000", 2024-09-14T23:57:33.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1759999999999998e-06 2024-09-14T23:57:33.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.943 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:33.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105445+0000", 2024-09-14T23:57:33.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9689999999999999e-06 2024-09-14T23:57:33.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:33.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149381+0000", 2024-09-14T23:57:33.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043936389999999999 2024-09-14T23:57:33.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:33.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149422+0000", 2024-09-14T23:57:33.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0865000000000003e-05 2024-09-14T23:57:33.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:33.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149465+0000", 2024-09-14T23:57:33.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3272999999999997e-05 2024-09-14T23:57:33.944 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:33.945 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:33.945 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:33.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.945 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4100 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1527808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:33.945 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.105454+0000", 2024-09-14T23:57:33.945 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 54.775878706999997, 2024-09-14T23:57:33.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044217411999999998, 2024-09-14T23:57:33.945 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:33.945 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:33.945 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:33.945 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:33.945 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:33.946 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:33.946 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4100 2024-09-14T23:57:33.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.946 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:33.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.946 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:33.946 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105454+0000", 2024-09-14T23:57:33.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:33.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.946 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:33.946 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105454+0000", 2024-09-14T23:57:33.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:33.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.947 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:33.947 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105578+0000", 2024-09-14T23:57:33.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012401799999999999 2024-09-14T23:57:33.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.947 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:33.947 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105583+0000", 2024-09-14T23:57:33.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1779999999999996e-06 2024-09-14T23:57:33.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.947 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:33.947 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105590+0000", 2024-09-14T23:57:33.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4279999999999999e-06 2024-09-14T23:57:33.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.948 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:33.948 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105596+0000", 2024-09-14T23:57:33.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0120000000000002e-06 2024-09-14T23:57:33.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.948 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:33.948 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149503+0000", 2024-09-14T23:57:33.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043906580000000001 2024-09-14T23:57:33.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.948 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:33.949 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149585+0000", 2024-09-14T23:57:33.949 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2356999999999998e-05 2024-09-14T23:57:33.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:33.949 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149672+0000", 2024-09-14T23:57:33.949 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6839e-05 2024-09-14T23:57:33.949 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:33.949 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:33.949 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:33.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.949 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4101 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1589248~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:33.950 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.105609+0000", 2024-09-14T23:57:33.950 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 54.775724167999996, 2024-09-14T23:57:33.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044175987999999999, 2024-09-14T23:57:33.950 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:33.950 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:33.950 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:33.950 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:33.950 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:33.950 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:33.950 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4101 2024-09-14T23:57:33.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.951 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:33.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.952 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:33.952 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105609+0000", 2024-09-14T23:57:33.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:33.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.952 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:33.952 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105609+0000", 2024-09-14T23:57:33.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:33.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.952 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:33.952 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.106671+0000", 2024-09-14T23:57:33.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010619270000000001 2024-09-14T23:57:33.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.953 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:33.953 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.106675+0000", 2024-09-14T23:57:33.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4429999999999996e-06 2024-09-14T23:57:33.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.953 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:33.953 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.106685+0000", 2024-09-14T23:57:33.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7429999999999997e-06 2024-09-14T23:57:33.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.953 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:33.954 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.106690+0000", 2024-09-14T23:57:33.954 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.293e-06 2024-09-14T23:57:33.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.954 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.954 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:33.954 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149700+0000", 2024-09-14T23:57:33.954 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043009512 2024-09-14T23:57:33.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.954 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.954 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:33.954 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149742+0000", 2024-09-14T23:57:33.954 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2083999999999999e-05 2024-09-14T23:57:33.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.954 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.955 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:33.955 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149785+0000", 2024-09-14T23:57:33.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2985999999999998e-05 2024-09-14T23:57:33.955 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:33.955 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:33.955 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:33.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.955 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4116 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 180224~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:33.955 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.107575+0000", 2024-09-14T23:57:33.955 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 54.773757699999997, 2024-09-14T23:57:33.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044157522999999997, 2024-09-14T23:57:33.955 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:33.955 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:33.956 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:33.956 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:33.956 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:33.956 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:33.956 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4116 2024-09-14T23:57:33.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.956 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:33.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:33.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.107575+0000", 2024-09-14T23:57:33.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:33.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:33.957 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.107575+0000", 2024-09-14T23:57:33.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:33.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:33.957 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108671+0000", 2024-09-14T23:57:33.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010954510000000001 2024-09-14T23:57:33.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:33.957 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108676+0000", 2024-09-14T23:57:33.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6639999999999997e-06 2024-09-14T23:57:33.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.958 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.958 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:33.958 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108685+0000", 2024-09-14T23:57:33.958 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1130000000000007e-06 2024-09-14T23:57:33.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.958 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.958 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:33.958 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108691+0000", 2024-09-14T23:57:33.958 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6019999999999996e-06 2024-09-14T23:57:33.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.958 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.958 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:33.958 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.151530+0000", 2024-09-14T23:57:33.958 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.042838345999999999 2024-09-14T23:57:33.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.959 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.959 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:33.959 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.151619+0000", 2024-09-14T23:57:33.959 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9302999999999995e-05 2024-09-14T23:57:33.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.959 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.959 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:33.959 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.151733+0000", 2024-09-14T23:57:33.959 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011404399999999999 2024-09-14T23:57:33.959 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:33.959 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:33.959 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:33.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.960 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4122 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 548864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:33.960 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.108842+0000", 2024-09-14T23:57:33.960 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 54.772490861999998, 2024-09-14T23:57:33.960 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044103631999999997, 2024-09-14T23:57:33.960 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:33.960 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:33.960 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:33.960 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:33.960 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:33.960 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:33.960 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4122 2024-09-14T23:57:33.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.960 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:33.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:33.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108842+0000", 2024-09-14T23:57:33.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:33.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:33.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108842+0000", 2024-09-14T23:57:33.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:33.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:33.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108843+0000", 2024-09-14T23:57:33.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7000000000000001e-07 2024-09-14T23:57:33.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.962 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:33.962 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108844+0000", 2024-09-14T23:57:33.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6479999999999999e-06 2024-09-14T23:57:33.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.962 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:33.962 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108848+0000", 2024-09-14T23:57:33.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8029999999999998e-06 2024-09-14T23:57:33.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.963 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:33.963 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108851+0000", 2024-09-14T23:57:33.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7939999999999998e-06 2024-09-14T23:57:33.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.963 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.963 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:33.963 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.152838+0000", 2024-09-14T23:57:33.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043987351000000001 2024-09-14T23:57:33.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.963 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.963 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:33.963 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.152885+0000", 2024-09-14T23:57:33.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7237000000000002e-05 2024-09-14T23:57:33.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.964 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:33.964 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.152946+0000", 2024-09-14T23:57:33.964 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0529000000000003e-05 2024-09-14T23:57:33.964 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:33.964 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:33.964 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:33.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.964 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4123 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 610304~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:33.964 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.108855+0000", 2024-09-14T23:57:33.964 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 54.772478532999997, 2024-09-14T23:57:33.964 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044321610999999997, 2024-09-14T23:57:33.964 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:33.965 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:33.965 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:33.965 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:33.965 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:33.965 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:33.965 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4123 2024-09-14T23:57:33.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.965 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:33.965 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.965 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:33.965 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108855+0000", 2024-09-14T23:57:33.965 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:33.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.965 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:33.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108855+0000", 2024-09-14T23:57:33.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:33.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:33.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.110671+0000", 2024-09-14T23:57:33.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018159719999999999 2024-09-14T23:57:33.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:33.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.110676+0000", 2024-09-14T23:57:33.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9450000000000001e-06 2024-09-14T23:57:33.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.967 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:33.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.110685+0000", 2024-09-14T23:57:33.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5359999999999998e-06 2024-09-14T23:57:33.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.967 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:33.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.110692+0000", 2024-09-14T23:57:33.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9280000000000004e-06 2024-09-14T23:57:33.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.967 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:33.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.152984+0000", 2024-09-14T23:57:33.968 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.042291677999999999 2024-09-14T23:57:33.968 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.968 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.968 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:33.968 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.153066+0000", 2024-09-14T23:57:33.968 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2657999999999994e-05 2024-09-14T23:57:33.968 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.968 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.968 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:33.968 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.153176+0000", 2024-09-14T23:57:33.968 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000109894 2024-09-14T23:57:33.968 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:33.968 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:33.968 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:33.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.969 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.969 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4136 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:33.969 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.111056+0000", 2024-09-14T23:57:33.969 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 54.770277385, 2024-09-14T23:57:33.969 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044097428000000001, 2024-09-14T23:57:33.969 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:33.969 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:33.969 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:33.969 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:33.969 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:33.969 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:33.969 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4136 2024-09-14T23:57:33.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.970 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:33.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.970 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:33.970 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.111056+0000", 2024-09-14T23:57:33.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:33.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.970 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:33.970 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.111056+0000", 2024-09-14T23:57:33.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:33.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.970 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:33.970 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.112671+0000", 2024-09-14T23:57:33.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016153999999999999 2024-09-14T23:57:33.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.971 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:33.971 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.112675+0000", 2024-09-14T23:57:33.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2830000000000003e-06 2024-09-14T23:57:33.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.971 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:33.971 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.112684+0000", 2024-09-14T23:57:33.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7800000000000006e-06 2024-09-14T23:57:33.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.971 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:33.972 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.112689+0000", 2024-09-14T23:57:33.972 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0000000000000004e-06 2024-09-14T23:57:33.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.972 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.972 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:33.972 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.155064+0000", 2024-09-14T23:57:33.972 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.042374423000000001 2024-09-14T23:57:33.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.972 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.972 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:33.972 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.155111+0000", 2024-09-14T23:57:33.972 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7367000000000002e-05 2024-09-14T23:57:33.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.973 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:33.973 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.155153+0000", 2024-09-14T23:57:33.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2175000000000002e-05 2024-09-14T23:57:33.973 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:33.973 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:33.973 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:33.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.973 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:46572 3.1d 3:b9e1f524:::rbd_header.1118cee06433:head [watch reconnect cookie 140670782499824 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-14T23:57:33.973 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:57.046868+0000", 2024-09-14T23:57:33.973 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.834465579000003, 2024-09-14T23:57:33.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0243412620000001, 2024-09-14T23:57:33.973 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:33.974 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:33.974 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:33.974 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:33.974 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:33.974 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:33.974 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 46572 2024-09-14T23:57:33.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.974 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:33.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.974 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:33.974 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046868+0000", 2024-09-14T23:57:33.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:33.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.975 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:33.975 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046868+0000", 2024-09-14T23:57:33.975 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:33.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.975 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.975 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:33.975 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046870+0000", 2024-09-14T23:57:33.975 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.204e-06 2024-09-14T23:57:33.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.975 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.975 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:33.975 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046877+0000", 2024-09-14T23:57:33.975 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5100000000000001e-06 2024-09-14T23:57:33.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.976 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:33.976 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046898+0000", 2024-09-14T23:57:33.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1149999999999999e-05 2024-09-14T23:57:33.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.976 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:33.976 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046911+0000", 2024-09-14T23:57:33.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2234e-05 2024-09-14T23:57:33.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.976 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:33.976 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046988+0000", 2024-09-14T23:57:33.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7445000000000001e-05 2024-09-14T23:57:33.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.977 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:57:33.977 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.047001+0000", 2024-09-14T23:57:33.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2951e-05 2024-09-14T23:57:33.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.977 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:33.977 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.069287+0000", 2024-09-14T23:57:33.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.022285748 2024-09-14T23:57:33.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.977 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for rw locks", 2024-09-14T23:57:33.978 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.069397+0000", 2024-09-14T23:57:33.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000110326 2024-09-14T23:57:33.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.978 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:33.978 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.069997+0000", 2024-09-14T23:57:33.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00059984399999999998 2024-09-14T23:57:33.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.978 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:33.978 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.070109+0000", 2024-09-14T23:57:33.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000112367 2024-09-14T23:57:33.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:33.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:33.979 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:33.979 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.071209+0000", 2024-09-14T23:57:33.979 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010994830000000001 2024-09-14T23:57:33.979 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:33.979 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:33.979 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:33.979 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:33.979 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:33.979 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:33.981 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T23:57:33.982 INFO:teuthology.orchestra.run.smithi042.stdout:3345 write 0x974bd53 thru 0x975acb9 (0xef67 bytes) 2024-09-14T23:57:33.982 INFO:teuthology.orchestra.run.smithi042.stdout:3346 read 0x71b44ea thru 0x71bf385 (0xae9c bytes) 2024-09-14T23:57:33.982 INFO:teuthology.orchestra.run.smithi042.stdout:3349 writesame 0x55b8def thru 0x55bda77 data_size 0x9(0x4c89 bytes) 2024-09-14T23:57:33.982 INFO:teuthology.orchestra.run.smithi042.stdout:3350 read 0x7f096a5 thru 0x7f0e1cc (0x4b28 bytes) 2024-09-14T23:57:33.982 INFO:teuthology.orchestra.run.smithi042.stdout:3351 read 0x9f5e0b thru 0x9fcdbb (0x6fb1 bytes) 2024-09-14T23:57:33.982 INFO:teuthology.orchestra.run.smithi042.stdout:3352 write 0xab636e9 thru 0xab63b03 (0x41b bytes) 2024-09-14T23:57:33.982 INFO:teuthology.orchestra.run.smithi042.stdout:3354 read 0xb8ee2a thru 0xb97cdb (0x8eb2 bytes) 2024-09-14T23:57:33.982 INFO:teuthology.orchestra.run.smithi042.stdout:3355 clone 31 order 18 su 2048 sc 14 2024-09-14T23:57:33.983 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:33.979+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:34.003 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is set 2024-09-14T23:57:34.082 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:34.080+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:34.090 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone30 from 0xab63b04 (overlap 0x117316) to 0xa7840 2024-09-14T23:57:34.105 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:34.106 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:57:34.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.106 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:114604 3.1c e83/49 3:3cc7f21d:::rbd_object_map.11186acd4a72:head v 83'505, mlcod=83'501)", 2024-09-14T23:57:34.106 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:34.103603+0000", 2024-09-14T23:57:34.106 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.00090501500000000001, 2024-09-14T23:57:34.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000931367, 2024-09-14T23:57:34.106 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:34.106 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:34.106 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:34.106 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:34.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.106 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:34.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:34.103603+0000", 2024-09-14T23:57:34.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:34.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.107 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:34.107 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:34.103603+0000", 2024-09-14T23:57:34.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:34.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.107 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:34.107 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:34.103603+0000", 2024-09-14T23:57:34.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8100000000000002e-07 2024-09-14T23:57:34.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.107 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:34.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:34.103610+0000", 2024-09-14T23:57:34.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.054e-06 2024-09-14T23:57:34.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:34.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:34.103626+0000", 2024-09-14T23:57:34.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6124999999999999e-05 2024-09-14T23:57:34.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:34.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:34.103633+0000", 2024-09-14T23:57:34.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.126e-06 2024-09-14T23:57:34.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:34.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:34.103695+0000", 2024-09-14T23:57:34.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2545000000000005e-05 2024-09-14T23:57:34.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:34.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:34.103718+0000", 2024-09-14T23:57:34.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2764000000000001e-05 2024-09-14T23:57:34.109 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:34.109 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:34.110 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:34.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.110 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.110 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:114605 3.1c e83/49 3:3cc7f21d:::rbd_object_map.11186acd4a72:head v 83'506, mlcod=83'502)", 2024-09-14T23:57:34.110 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:34.104324+0000", 2024-09-14T23:57:34.110 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.000184091, 2024-09-14T23:57:34.110 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00027733100000000002, 2024-09-14T23:57:34.110 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:34.110 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:34.110 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:34.110 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:34.110 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.110 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:34.111 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:34.104324+0000", 2024-09-14T23:57:34.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:34.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.111 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.111 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:34.111 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:34.104324+0000", 2024-09-14T23:57:34.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:34.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.111 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.111 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:34.111 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:34.104324+0000", 2024-09-14T23:57:34.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8899999999999999e-07 2024-09-14T23:57:34.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.111 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:34.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:34.104331+0000", 2024-09-14T23:57:34.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1029999999999998e-06 2024-09-14T23:57:34.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:34.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:34.104341+0000", 2024-09-14T23:57:34.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0312000000000001e-05 2024-09-14T23:57:34.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:34.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:34.104348+0000", 2024-09-14T23:57:34.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7900000000000002e-06 2024-09-14T23:57:34.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.113 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:34.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:34.104394+0000", 2024-09-14T23:57:34.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6504000000000001e-05 2024-09-14T23:57:34.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.113 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:34.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:34.104412+0000", 2024-09-14T23:57:34.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8229000000000001e-05 2024-09-14T23:57:34.113 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:34.113 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:34.113 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:34.113 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:34.113 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-14T23:57:34.114 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 2 2024-09-14T23:57:34.114 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:34.117 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T23:57:34.183 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:34.181+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:34.201 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-09-14T23:57:34.212 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:34.212 INFO:teuthology.orchestra.run.smithi042.stdout:3357 read 0x2d1651a thru 0x2d1e5a0 (0x8087 bytes) 2024-09-14T23:57:34.213 INFO:teuthology.orchestra.run.smithi042.stdout:3358 punch from 0xa16ad3b to 0xa1765e6, (0xb8ab bytes) 2024-09-14T23:57:34.214 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T23:57:34.215 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T23:57:34.215 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-14T23:57:34.215 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T23:57:34.217 INFO:teuthology.orchestra.run.smithi042.stdout:3359 read 0x4c02346 thru 0x4c107ed (0xe4a8 bytes) 2024-09-14T23:57:34.224 INFO:teuthology.orchestra.run.smithi042.stdout:3360 trunc from 0xab63b04 to 0x4125a52 2024-09-14T23:57:34.245 INFO:teuthology.orchestra.run.smithi042.stdout:3361 read 0x1abfa67 thru 0x1ac5352 (0x58ec bytes) 2024-09-14T23:57:34.246 INFO:teuthology.orchestra.run.smithi042.stdout:3362 read 0x11a8dd thru 0x11eb49 (0x426d bytes) 2024-09-14T23:57:34.254 INFO:teuthology.orchestra.run.smithi042.stdout:3364 write 0x44f781a thru 0x4503f81 (0xc768 bytes) 2024-09-14T23:57:34.277 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:34.277 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:34.277 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:57:34.277 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:57:34.277 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:34.283 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:34.282+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:34.286 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T23:57:34.287 INFO:teuthology.orchestra.run.smithi042.stdout:3365 write 0xeaa9c25 thru 0xeab0187 (0x6563 bytes) 2024-09-14T23:57:34.304 INFO:teuthology.orchestra.run.smithi042.stdout:3366 write 0x3d32450 thru 0x3d3df70 (0xbb21 bytes) 2024-09-14T23:57:34.322 INFO:teuthology.orchestra.run.smithi042.stdout:3369 trunc from 0xeab0188 to 0xdbeec7d 2024-09-14T23:57:34.333 INFO:teuthology.orchestra.run.smithi042.stdout:3370 punch from 0x871c068 to 0x872b8ce, (0xf866 bytes) 2024-09-14T23:57:34.333 INFO:teuthology.orchestra.run.smithi042.stdout:3371 write 0xaa373 thru 0xb91cf (0xee5d bytes) 2024-09-14T23:57:34.385 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:34.383+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:34.404 INFO:teuthology.orchestra.run.smithi042.stdout:3373 write 0x77925b0 thru 0x7799fdf (0x7a30 bytes) 2024-09-14T23:57:34.428 INFO:teuthology.orchestra.run.smithi042.stdout:3374 trunc from 0xdbeec7d to 0x900d6c8 2024-09-14T23:57:34.436 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:34.436 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:57:34.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:57:34.436 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:57:34.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.436 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4284.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-14T23:57:34.436 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:57:34.436 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 71.909475135999998, 2024-09-14T23:57:34.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.004942086, 2024-09-14T23:57:34.437 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:34.437 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:34.437 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:34.437 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:34.437 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4284", 2024-09-14T23:57:34.437 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/4278347926", 2024-09-14T23:57:34.437 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1 2024-09-14T23:57:34.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.437 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:34.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.437 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:34.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:57:34.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:34.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:34.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:57:34.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:34.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:34.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524331+0000", 2024-09-14T23:57:34.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.383e-05 2024-09-14T23:57:34.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.439 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:34.439 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524339+0000", 2024-09-14T23:57:34.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8290000000000008e-06 2024-09-14T23:57:34.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.439 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:34.439 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524359+0000", 2024-09-14T23:57:34.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9998000000000001e-05 2024-09-14T23:57:34.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.440 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:34.440 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524373+0000", 2024-09-14T23:57:34.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4304000000000001e-05 2024-09-14T23:57:34.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.440 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:34.440 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524488+0000", 2024-09-14T23:57:34.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114663 2024-09-14T23:57:34.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.440 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:57:34.440 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524501+0000", 2024-09-14T23:57:34.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3349999999999999e-05 2024-09-14T23:57:34.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:34.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.526868+0000", 2024-09-14T23:57:34.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023670090000000002 2024-09-14T23:57:34.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:34.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.527041+0000", 2024-09-14T23:57:34.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000172597 2024-09-14T23:57:34.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-14T23:57:34.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.527384+0000", 2024-09-14T23:57:34.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00034309500000000002 2024-09-14T23:57:34.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:34.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529103+0000", 2024-09-14T23:57:34.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017185220000000001 2024-09-14T23:57:34.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:57:34.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529188+0000", 2024-09-14T23:57:34.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5591000000000001e-05 2024-09-14T23:57:34.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.443 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:34.443 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529227+0000", 2024-09-14T23:57:34.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8563000000000003e-05 2024-09-14T23:57:34.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.443 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:34.443 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529259+0000", 2024-09-14T23:57:34.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2734999999999997e-05 2024-09-14T23:57:34.443 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:34.443 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:34.443 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:34.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.444 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4284.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-14T23:57:34.444 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:23.530683+0000", 2024-09-14T23:57:34.444 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 70.903109470999993, 2024-09-14T23:57:34.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0036133110000000001, 2024-09-14T23:57:34.444 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:34.444 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:34.444 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:34.444 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:34.444 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4284", 2024-09-14T23:57:34.444 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/4278347926", 2024-09-14T23:57:34.444 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5 2024-09-14T23:57:34.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.445 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:34.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:34.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530683+0000", 2024-09-14T23:57:34.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:34.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:34.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530683+0000", 2024-09-14T23:57:34.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:34.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.446 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:34.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530685+0000", 2024-09-14T23:57:34.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.075e-06 2024-09-14T23:57:34.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.446 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:34.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530695+0000", 2024-09-14T23:57:34.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0261999999999999e-05 2024-09-14T23:57:34.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.446 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:34.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530715+0000", 2024-09-14T23:57:34.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9930000000000001e-05 2024-09-14T23:57:34.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:34.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530724+0000", 2024-09-14T23:57:34.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.901e-06 2024-09-14T23:57:34.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:34.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530800+0000", 2024-09-14T23:57:34.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5873999999999999e-05 2024-09-14T23:57:34.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-14T23:57:34.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530807+0000", 2024-09-14T23:57:34.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2389999999999996e-06 2024-09-14T23:57:34.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:34.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.532494+0000", 2024-09-14T23:57:34.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016871830000000001 2024-09-14T23:57:34.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:34.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.532599+0000", 2024-09-14T23:57:34.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010472 2024-09-14T23:57:34.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1", 2024-09-14T23:57:34.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.532715+0000", 2024-09-14T23:57:34.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011568199999999999 2024-09-14T23:57:34.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:57:34.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.533460+0000", 2024-09-14T23:57:34.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00074494999999999997 2024-09-14T23:57:34.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:34.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534230+0000", 2024-09-14T23:57:34.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00077036599999999998 2024-09-14T23:57:34.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:34.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534266+0000", 2024-09-14T23:57:34.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5725000000000001e-05 2024-09-14T23:57:34.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.450 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:34.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534296+0000", 2024-09-14T23:57:34.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0403999999999999e-05 2024-09-14T23:57:34.450 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:34.450 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:34.450 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:34.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.450 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4284.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-14T23:57:34.450 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:24.535160+0000", 2024-09-14T23:57:34.450 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 69.898632829999997, 2024-09-14T23:57:34.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033473650000000001, 2024-09-14T23:57:34.451 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:34.451 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:34.451 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:34.451 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:34.451 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4284", 2024-09-14T23:57:34.451 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/4278347926", 2024-09-14T23:57:34.451 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 7 2024-09-14T23:57:34.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.451 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:34.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:34.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535160+0000", 2024-09-14T23:57:34.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:34.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:34.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535160+0000", 2024-09-14T23:57:34.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:34.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:34.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535162+0000", 2024-09-14T23:57:34.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3369999999999998e-06 2024-09-14T23:57:34.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:34.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535171+0000", 2024-09-14T23:57:34.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7700000000000007e-06 2024-09-14T23:57:34.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:34.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535189+0000", 2024-09-14T23:57:34.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7938e-05 2024-09-14T23:57:34.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:34.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535197+0000", 2024-09-14T23:57:34.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5639999999999997e-06 2024-09-14T23:57:34.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:34.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535266+0000", 2024-09-14T23:57:34.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8926999999999999e-05 2024-09-14T23:57:34.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-14T23:57:34.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535271+0000", 2024-09-14T23:57:34.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0329999999999999e-06 2024-09-14T23:57:34.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:34.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.537064+0000", 2024-09-14T23:57:34.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001792929 2024-09-14T23:57:34.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:34.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.537131+0000", 2024-09-14T23:57:34.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6605000000000004e-05 2024-09-14T23:57:34.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1", 2024-09-14T23:57:34.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.537271+0000", 2024-09-14T23:57:34.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013992400000000001 2024-09-14T23:57:34.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:57:34.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.538093+0000", 2024-09-14T23:57:34.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00082210200000000001 2024-09-14T23:57:34.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:34.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.538439+0000", 2024-09-14T23:57:34.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00034595999999999997 2024-09-14T23:57:34.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.456 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:34.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.538476+0000", 2024-09-14T23:57:34.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6770000000000002e-05 2024-09-14T23:57:34.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.456 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:34.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.538507+0000", 2024-09-14T23:57:34.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1506e-05 2024-09-14T23:57:34.456 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:34.457 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:34.457 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:34.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.457 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T23:57:34.457 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:34.406877+0000", 2024-09-14T23:57:34.457 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 60.026915825000003, 2024-09-14T23:57:34.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030351169999999999, 2024-09-14T23:57:34.457 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:34.457 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:34.457 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:34.457 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:34.457 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:34.458 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:34.458 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 10 2024-09-14T23:57:34.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.458 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:34.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:34.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406877+0000", 2024-09-14T23:57:34.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:34.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:34.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406877+0000", 2024-09-14T23:57:34.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:34.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:34.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406878+0000", 2024-09-14T23:57:34.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.291e-06 2024-09-14T23:57:34.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:34.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406883+0000", 2024-09-14T23:57:34.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2399999999999998e-06 2024-09-14T23:57:34.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:34.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406891+0000", 2024-09-14T23:57:34.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5889999999999997e-06 2024-09-14T23:57:34.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:34.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406898+0000", 2024-09-14T23:57:34.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3599999999999998e-06 2024-09-14T23:57:34.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:34.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406948+0000", 2024-09-14T23:57:34.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9435999999999999e-05 2024-09-14T23:57:34.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-14T23:57:34.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406953+0000", 2024-09-14T23:57:34.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2809999999999999e-06 2024-09-14T23:57:34.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:34.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.408559+0000", 2024-09-14T23:57:34.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00160607 2024-09-14T23:57:34.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:34.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.408637+0000", 2024-09-14T23:57:34.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8504000000000006e-05 2024-09-14T23:57:34.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 4", 2024-09-14T23:57:34.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.408781+0000", 2024-09-14T23:57:34.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014374799999999999 2024-09-14T23:57:34.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:57:34.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.409659+0000", 2024-09-14T23:57:34.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000877781 2024-09-14T23:57:34.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:34.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.409822+0000", 2024-09-14T23:57:34.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016295400000000001 2024-09-14T23:57:34.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:34.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.409872+0000", 2024-09-14T23:57:34.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0022000000000003e-05 2024-09-14T23:57:34.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:34.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.409912+0000", 2024-09-14T23:57:34.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9841e-05 2024-09-14T23:57:34.463 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:34.463 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:34.463 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:34.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.463 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "MOSDPGPush(3.c 48/47 [PushOp(3:34183e86:::rbd_data.111888d8b54.0000000000000003:head, version: 37'112, data_included: [1473248~99616,1638400~65536,1835008~262144,2162688~65536,2293760~262144,2686976~65536,9737423~8043], data_size: 828555, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:34183e86:::rbd_data.111888d8b54.0000000000000003:head@37'112, size: 9745466, copy_subset: [(0, 9745466)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 9745466, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1473248, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-14T23:57:34.463 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:58.332069+0000", 2024-09-14T23:57:34.463 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.101723221999997, 2024-09-14T23:57:34.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0025177979999999999, 2024-09-14T23:57:34.464 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:34.464 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:34.464 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:34.464 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:34.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:34.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.332069+0000", 2024-09-14T23:57:34.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:34.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:34.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.332069+0000", 2024-09-14T23:57:34.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:34.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:34.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.332070+0000", 2024-09-14T23:57:34.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1200000000000003e-07 2024-09-14T23:57:34.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:34.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.332822+0000", 2024-09-14T23:57:34.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00075239299999999996 2024-09-14T23:57:34.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:34.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.332845+0000", 2024-09-14T23:57:34.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2436999999999999e-05 2024-09-14T23:57:34.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:34.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.332866+0000", 2024-09-14T23:57:34.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0874e-05 2024-09-14T23:57:34.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:34.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.332970+0000", 2024-09-14T23:57:34.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000104528 2024-09-14T23:57:34.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:34.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.332987+0000", 2024-09-14T23:57:34.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7073000000000001e-05 2024-09-14T23:57:34.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:34.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.334587+0000", 2024-09-14T23:57:34.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001599881 2024-09-14T23:57:34.467 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:34.467 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:34.467 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:34.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.467 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:50626 3.d e51/46 3:b5154cfe:::rbd_data.11183b9aac34.0000000000000004:head v 51'63, mlcod=45'61)", 2024-09-14T23:57:34.468 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:01.140129+0000", 2024-09-14T23:57:34.468 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 33.293663225000003, 2024-09-14T23:57:34.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0029883850000000001, 2024-09-14T23:57:34.468 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:34.468 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:34.468 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:34.468 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:34.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:34.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.140129+0000", 2024-09-14T23:57:34.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:34.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:34.469 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.140129+0000", 2024-09-14T23:57:34.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:34.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.469 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:34.469 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.140130+0000", 2024-09-14T23:57:34.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0529999999999999e-06 2024-09-14T23:57:34.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.469 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:34.469 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.140311+0000", 2024-09-14T23:57:34.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018054200000000001 2024-09-14T23:57:34.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:34.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.140325+0000", 2024-09-14T23:57:34.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.382e-05 2024-09-14T23:57:34.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:34.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.140332+0000", 2024-09-14T23:57:34.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5700000000000004e-06 2024-09-14T23:57:34.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:34.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.140392+0000", 2024-09-14T23:57:34.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9296000000000003e-05 2024-09-14T23:57:34.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:34.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.140436+0000", 2024-09-14T23:57:34.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4515999999999999e-05 2024-09-14T23:57:34.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:34.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.143075+0000", 2024-09-14T23:57:34.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0026385369999999998 2024-09-14T23:57:34.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:34.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.143118+0000", 2024-09-14T23:57:34.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3050999999999998e-05 2024-09-14T23:57:34.471 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:34.472 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:34.472 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:34.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.472 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:51277 3.d e52/46 3:b2645877:::rbd_data.11181975a1de.0000000000000001:head v 52'69, mlcod=51'66)", 2024-09-14T23:57:34.472 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:02.157938+0000", 2024-09-14T23:57:34.472 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 32.275854227000004, 2024-09-14T23:57:34.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0027517549999999998, 2024-09-14T23:57:34.472 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:34.472 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:34.472 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:34.472 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:34.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:34.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.157938+0000", 2024-09-14T23:57:34.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:34.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:34.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.157938+0000", 2024-09-14T23:57:34.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:34.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:34.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.157939+0000", 2024-09-14T23:57:34.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9199999999999999e-07 2024-09-14T23:57:34.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:34.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158543+0000", 2024-09-14T23:57:34.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00060333299999999995 2024-09-14T23:57:34.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:34.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158575+0000", 2024-09-14T23:57:34.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2801e-05 2024-09-14T23:57:34.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:34.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158586+0000", 2024-09-14T23:57:34.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.102e-05 2024-09-14T23:57:34.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:34.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158643+0000", 2024-09-14T23:57:34.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6320999999999998e-05 2024-09-14T23:57:34.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:34.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158693+0000", 2024-09-14T23:57:34.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0037000000000002e-05 2024-09-14T23:57:34.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:34.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.160641+0000", 2024-09-14T23:57:34.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0019485629999999999 2024-09-14T23:57:34.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:34.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.160690+0000", 2024-09-14T23:57:34.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8687999999999999e-05 2024-09-14T23:57:34.476 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:34.476 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:34.476 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:34.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.476 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:51275 3.2 e52/49 3:47a71653:::rbd_data.11181975a1de.0000000000000000:head v 52'118, mlcod=52'116)", 2024-09-14T23:57:34.476 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:02.158044+0000", 2024-09-14T23:57:34.476 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 32.275748655000001, 2024-09-14T23:57:34.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0028337200000000001, 2024-09-14T23:57:34.476 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:34.477 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:34.477 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:34.477 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:34.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:34.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158044+0000", 2024-09-14T23:57:34.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:34.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:34.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158044+0000", 2024-09-14T23:57:34.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:34.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:34.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158045+0000", 2024-09-14T23:57:34.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5700000000000002e-07 2024-09-14T23:57:34.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:34.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158637+0000", 2024-09-14T23:57:34.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00059278499999999997 2024-09-14T23:57:34.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:34.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158657+0000", 2024-09-14T23:57:34.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9422000000000001e-05 2024-09-14T23:57:34.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:34.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158675+0000", 2024-09-14T23:57:34.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8564999999999999e-05 2024-09-14T23:57:34.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:34.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158735+0000", 2024-09-14T23:57:34.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9904000000000001e-05 2024-09-14T23:57:34.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:34.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.158757+0000", 2024-09-14T23:57:34.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1277e-05 2024-09-14T23:57:34.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:34.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.160845+0000", 2024-09-14T23:57:34.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0020881419999999999 2024-09-14T23:57:34.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:34.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:02.160878+0000", 2024-09-14T23:57:34.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2867999999999998e-05 2024-09-14T23:57:34.480 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:34.480 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:34.481 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:34.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.481 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56383 3.c e56/49 3:36b698b0:::rbd_data.1118c4b3aebe.0000000000000000:head v 56'194, mlcod=56'192)", 2024-09-14T23:57:34.481 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.270483+0000", 2024-09-14T23:57:34.481 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 28.163309671, 2024-09-14T23:57:34.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0079685880000000004, 2024-09-14T23:57:34.481 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:34.481 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:34.481 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:34.481 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:34.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:34.481 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.270483+0000", 2024-09-14T23:57:34.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:34.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:34.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.270483+0000", 2024-09-14T23:57:34.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:34.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:34.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.270484+0000", 2024-09-14T23:57:34.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7400000000000002e-07 2024-09-14T23:57:34.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:34.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273680+0000", 2024-09-14T23:57:34.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031966680000000002 2024-09-14T23:57:34.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:34.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273697+0000", 2024-09-14T23:57:34.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6595e-05 2024-09-14T23:57:34.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:34.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273705+0000", 2024-09-14T23:57:34.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7530000000000001e-06 2024-09-14T23:57:34.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:34.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273763+0000", 2024-09-14T23:57:34.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8238e-05 2024-09-14T23:57:34.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:34.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273781+0000", 2024-09-14T23:57:34.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8281e-05 2024-09-14T23:57:34.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:34.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.278408+0000", 2024-09-14T23:57:34.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00462716 2024-09-14T23:57:34.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:34.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.278451+0000", 2024-09-14T23:57:34.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3019000000000003e-05 2024-09-14T23:57:34.485 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:34.485 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:34.485 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:34.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.485 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56384 3.c e56/49 3:36b698b0:::rbd_data.1118c4b3aebe.0000000000000000:head v 56'196, mlcod=56'192)", 2024-09-14T23:57:34.485 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:57:34.485 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 28.159276998999999, 2024-09-14T23:57:34.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0046707529999999997, 2024-09-14T23:57:34.486 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:34.486 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:34.486 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:34.486 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:34.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:34.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:57:34.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:34.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:34.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:57:34.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:34.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:34.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:57:34.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1699999999999997e-07 2024-09-14T23:57:34.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:34.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274523+0000", 2024-09-14T23:57:34.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9129999999999997e-06 2024-09-14T23:57:34.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:34.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274535+0000", 2024-09-14T23:57:34.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1788e-05 2024-09-14T23:57:34.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:34.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274542+0000", 2024-09-14T23:57:34.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0539999999999997e-06 2024-09-14T23:57:34.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:34.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275889+0000", 2024-09-14T23:57:34.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00134669 2024-09-14T23:57:34.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:34.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275917+0000", 2024-09-14T23:57:34.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7664000000000001e-05 2024-09-14T23:57:34.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:34.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279134+0000", 2024-09-14T23:57:34.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0032178459999999999 2024-09-14T23:57:34.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:34.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279186+0000", 2024-09-14T23:57:34.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1881e-05 2024-09-14T23:57:34.489 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:34.489 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:34.489 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:34.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.489 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56381 3.1d e56/49 3:bcf9a8cf:::rbd_data.1118c4b3aebe.0000000000000006:head v 56'210, mlcod=56'208)", 2024-09-14T23:57:34.490 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.275462+0000", 2024-09-14T23:57:34.490 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 28.158330669000001, 2024-09-14T23:57:34.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0071310710000000001, 2024-09-14T23:57:34.490 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:34.490 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:34.490 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:34.490 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:34.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:34.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275462+0000", 2024-09-14T23:57:34.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:34.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:34.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275462+0000", 2024-09-14T23:57:34.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:34.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:34.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275463+0000", 2024-09-14T23:57:34.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2699999999999999e-07 2024-09-14T23:57:34.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:34.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276779+0000", 2024-09-14T23:57:34.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0013163649999999999 2024-09-14T23:57:34.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:34.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276795+0000", 2024-09-14T23:57:34.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.59e-05 2024-09-14T23:57:34.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:34.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276806+0000", 2024-09-14T23:57:34.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1106e-05 2024-09-14T23:57:34.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:34.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276941+0000", 2024-09-14T23:57:34.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013477500000000001 2024-09-14T23:57:34.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:34.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276963+0000", 2024-09-14T23:57:34.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2419999999999999e-05 2024-09-14T23:57:34.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:34.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282546+0000", 2024-09-14T23:57:34.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0055827100000000003 2024-09-14T23:57:34.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:34.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282593+0000", 2024-09-14T23:57:34.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7067999999999997e-05 2024-09-14T23:57:34.493 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:34.493 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:34.494 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:34.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.494 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56387 3.c e56/49 3:30ee4ba0:::rbd_data.1118c4b3aebe.0000000000000004:head v 56'197, mlcod=56'192)", 2024-09-14T23:57:34.494 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.276498+0000", 2024-09-14T23:57:34.494 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 28.157294574000002, 2024-09-14T23:57:34.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0092192639999999996, 2024-09-14T23:57:34.494 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:34.494 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:34.494 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:34.494 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:34.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:34.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276498+0000", 2024-09-14T23:57:34.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:34.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:34.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276498+0000", 2024-09-14T23:57:34.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:34.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:34.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276499+0000", 2024-09-14T23:57:34.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1100000000000004e-07 2024-09-14T23:57:34.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:34.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279646+0000", 2024-09-14T23:57:34.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031473220000000001 2024-09-14T23:57:34.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:34.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279672+0000", 2024-09-14T23:57:34.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6129999999999999e-05 2024-09-14T23:57:34.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:34.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279682+0000", 2024-09-14T23:57:34.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7170000000000003e-06 2024-09-14T23:57:34.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:34.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279742+0000", 2024-09-14T23:57:34.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9926999999999997e-05 2024-09-14T23:57:34.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:34.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279761+0000", 2024-09-14T23:57:34.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9402000000000002e-05 2024-09-14T23:57:34.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:34.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.285680+0000", 2024-09-14T23:57:34.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.005918881 2024-09-14T23:57:34.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:34.498 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.285717+0000", 2024-09-14T23:57:34.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6974000000000002e-05 2024-09-14T23:57:34.498 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:34.498 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:34.498 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:34.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.498 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56388 3.c e56/49 3:30ee4ba0:::rbd_data.1118c4b3aebe.0000000000000004:head v 56'199, mlcod=56'192)", 2024-09-14T23:57:34.498 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:57:34.498 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 28.153802985999999, 2024-09-14T23:57:34.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0060596740000000001, 2024-09-14T23:57:34.498 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:34.498 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:34.498 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:34.499 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:34.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.499 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:34.499 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:57:34.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:34.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.499 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:34.499 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:57:34.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:34.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.499 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:34.499 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:57:34.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2200000000000003e-07 2024-09-14T23:57:34.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:34.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279995+0000", 2024-09-14T23:57:34.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5569999999999997e-06 2024-09-14T23:57:34.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:34.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280004+0000", 2024-09-14T23:57:34.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0080000000000008e-06 2024-09-14T23:57:34.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:34.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280010+0000", 2024-09-14T23:57:34.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5749999999999997e-06 2024-09-14T23:57:34.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:34.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282968+0000", 2024-09-14T23:57:34.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0029579390000000001 2024-09-14T23:57:34.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:34.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282998+0000", 2024-09-14T23:57:34.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9947999999999999e-05 2024-09-14T23:57:34.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:34.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.286012+0000", 2024-09-14T23:57:34.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030132900000000001 2024-09-14T23:57:34.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:34.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.286049+0000", 2024-09-14T23:57:34.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7534999999999999e-05 2024-09-14T23:57:34.502 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:34.502 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:34.502 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:34.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.503 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56382 3.1d e56/49 3:bcf9a8cf:::rbd_data.1118c4b3aebe.0000000000000006:head v 56'212, mlcod=56'208)", 2024-09-14T23:57:34.503 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.280285+0000", 2024-09-14T23:57:34.503 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 28.153507303000001, 2024-09-14T23:57:34.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035393489999999998, 2024-09-14T23:57:34.503 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:34.503 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:34.503 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:34.503 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:34.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:34.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280285+0000", 2024-09-14T23:57:34.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:34.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.504 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:34.504 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280285+0000", 2024-09-14T23:57:34.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:34.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.504 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:34.504 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280286+0000", 2024-09-14T23:57:34.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5000000000000001e-07 2024-09-14T23:57:34.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.504 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:34.504 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280299+0000", 2024-09-14T23:57:34.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3312000000000001e-05 2024-09-14T23:57:34.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:34.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280311+0000", 2024-09-14T23:57:34.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1677e-05 2024-09-14T23:57:34.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:34.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280318+0000", 2024-09-14T23:57:34.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.742e-06 2024-09-14T23:57:34.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:34.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280374+0000", 2024-09-14T23:57:34.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.592e-05 2024-09-14T23:57:34.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:34.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280392+0000", 2024-09-14T23:57:34.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8490000000000001e-05 2024-09-14T23:57:34.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:34.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.283693+0000", 2024-09-14T23:57:34.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033012050000000002 2024-09-14T23:57:34.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:34.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.283825+0000", 2024-09-14T23:57:34.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013105300000000001 2024-09-14T23:57:34.507 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:34.507 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:34.507 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:34.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.507 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "pg_update_log_missing(3.c epoch 56/49 rep_tid 1493 entries 56'200 (0'0) error 3:370203a3:::rbd_data.1118c4b3aebe.000000000000000d:head by client.4376.0:56396 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 56'192)", 2024-09-14T23:57:34.507 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:57:34.507 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 28.153282846, 2024-09-14T23:57:34.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057370099999999999, 2024-09-14T23:57:34.507 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:34.507 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:34.507 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:57:34.507 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:34.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:34.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:57:34.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:34.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:34.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:57:34.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:34.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:34.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:57:34.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0699999999999997e-07 2024-09-14T23:57:34.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:34.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280519+0000", 2024-09-14T23:57:34.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6260000000000001e-06 2024-09-14T23:57:34.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:34.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280541+0000", 2024-09-14T23:57:34.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1976000000000001e-05 2024-09-14T23:57:34.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:34.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280556+0000", 2024-09-14T23:57:34.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5075000000000001e-05 2024-09-14T23:57:34.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:34.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.285494+0000", 2024-09-14T23:57:34.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0049378690000000001 2024-09-14T23:57:34.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:34.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.286247+0000", 2024-09-14T23:57:34.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00075285700000000001 2024-09-14T23:57:34.510 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:34.510 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:34.510 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:34.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.510 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56400 3.1c e56/49 3:3f5b81ba:::rbd_data.1118c4b3aebe.000000000000000a:head v 56'236, mlcod=56'234)", 2024-09-14T23:57:34.510 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.295064+0000", 2024-09-14T23:57:34.510 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 28.138728054000001, 2024-09-14T23:57:34.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0054331589999999999, 2024-09-14T23:57:34.511 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:34.511 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:34.511 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:34.511 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:34.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:34.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.295064+0000", 2024-09-14T23:57:34.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:34.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:34.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.295064+0000", 2024-09-14T23:57:34.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:34.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.512 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:34.512 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.295066+0000", 2024-09-14T23:57:34.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.502e-06 2024-09-14T23:57:34.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.512 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:34.512 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296403+0000", 2024-09-14T23:57:34.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001336554 2024-09-14T23:57:34.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.512 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:34.512 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296428+0000", 2024-09-14T23:57:34.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5179000000000001e-05 2024-09-14T23:57:34.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:34.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296440+0000", 2024-09-14T23:57:34.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2387e-05 2024-09-14T23:57:34.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:34.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296554+0000", 2024-09-14T23:57:34.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114269 2024-09-14T23:57:34.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:34.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296581+0000", 2024-09-14T23:57:34.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6673999999999999e-05 2024-09-14T23:57:34.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:34.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.300418+0000", 2024-09-14T23:57:34.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0038371249999999998 2024-09-14T23:57:34.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:34.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.300498+0000", 2024-09-14T23:57:34.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9468999999999994e-05 2024-09-14T23:57:34.514 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:34.514 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:34.514 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:34.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.515 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56401 3.1c e56/49 3:3f5b81ba:::rbd_data.1118c4b3aebe.000000000000000a:head v 56'238, mlcod=56'234)", 2024-09-14T23:57:34.515 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.297640+0000", 2024-09-14T23:57:34.515 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 28.136152214999999, 2024-09-14T23:57:34.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031491919999999999, 2024-09-14T23:57:34.515 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:34.515 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:34.515 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:34.515 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:34.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:34.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.297640+0000", 2024-09-14T23:57:34.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:34.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:34.516 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.297640+0000", 2024-09-14T23:57:34.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:34.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.516 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:34.516 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.297641+0000", 2024-09-14T23:57:34.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9000000000000005e-07 2024-09-14T23:57:34.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.516 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:34.516 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.297649+0000", 2024-09-14T23:57:34.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4510000000000003e-06 2024-09-14T23:57:34.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:34.517 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.297663+0000", 2024-09-14T23:57:34.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4262e-05 2024-09-14T23:57:34.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:34.517 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.297679+0000", 2024-09-14T23:57:34.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5798e-05 2024-09-14T23:57:34.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:34.517 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.298408+0000", 2024-09-14T23:57:34.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00072923699999999998 2024-09-14T23:57:34.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:34.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.298430+0000", 2024-09-14T23:57:34.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1753999999999999e-05 2024-09-14T23:57:34.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:34.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.300749+0000", 2024-09-14T23:57:34.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023190590000000001 2024-09-14T23:57:34.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:34.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.300789+0000", 2024-09-14T23:57:34.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0640999999999999e-05 2024-09-14T23:57:34.519 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:34.519 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:34.519 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:34.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.519 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:61921 3.5 e57/49 3:a05d004f:::rbd_data.11188d356229.000000000000000b:head v 57'279, mlcod=57'277)", 2024-09-14T23:57:34.519 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:07.560389+0000", 2024-09-14T23:57:34.519 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 26.873403455999998, 2024-09-14T23:57:34.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0034956520000000001, 2024-09-14T23:57:34.519 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:34.519 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:34.519 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:34.519 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:34.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:34.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.560389+0000", 2024-09-14T23:57:34.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:34.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:34.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.560389+0000", 2024-09-14T23:57:34.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:34.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:34.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.560390+0000", 2024-09-14T23:57:34.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4109999999999999e-06 2024-09-14T23:57:34.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.521 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.521 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:34.521 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.561839+0000", 2024-09-14T23:57:34.521 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001448406 2024-09-14T23:57:34.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.521 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.521 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:34.521 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.561854+0000", 2024-09-14T23:57:34.521 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5426000000000001e-05 2024-09-14T23:57:34.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.521 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.521 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:34.521 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.561863+0000", 2024-09-14T23:57:34.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7509999999999997e-06 2024-09-14T23:57:34.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:34.522 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.561917+0000", 2024-09-14T23:57:34.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3531e-05 2024-09-14T23:57:34.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:34.522 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.561944+0000", 2024-09-14T23:57:34.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7528000000000001e-05 2024-09-14T23:57:34.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:34.522 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563843+0000", 2024-09-14T23:57:34.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001899361 2024-09-14T23:57:34.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:34.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563885+0000", 2024-09-14T23:57:34.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1238000000000001e-05 2024-09-14T23:57:34.523 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:34.523 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:34.523 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:34.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.523 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:61922 3.5 e57/49 3:a11ebf8e:::rbd_data.11188d356229.000000000000000c:head v 57'280, mlcod=57'277)", 2024-09-14T23:57:34.523 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:07.563144+0000", 2024-09-14T23:57:34.523 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 26.870648402, 2024-09-14T23:57:34.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003977848, 2024-09-14T23:57:34.524 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:34.524 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:34.524 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:34.524 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:34.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:34.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563144+0000", 2024-09-14T23:57:34.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:34.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:34.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563144+0000", 2024-09-14T23:57:34.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:34.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.525 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:34.525 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563145+0000", 2024-09-14T23:57:34.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5400000000000001e-07 2024-09-14T23:57:34.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.525 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:34.525 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565056+0000", 2024-09-14T23:57:34.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0019108160000000001 2024-09-14T23:57:34.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.525 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:34.525 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565088+0000", 2024-09-14T23:57:34.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2586999999999998e-05 2024-09-14T23:57:34.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.526 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:34.526 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565100+0000", 2024-09-14T23:57:34.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1917e-05 2024-09-14T23:57:34.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.526 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:34.526 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565166+0000", 2024-09-14T23:57:34.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6245e-05 2024-09-14T23:57:34.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.526 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:34.526 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565190+0000", 2024-09-14T23:57:34.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3825999999999999e-05 2024-09-14T23:57:34.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:34.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.567080+0000", 2024-09-14T23:57:34.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001889541 2024-09-14T23:57:34.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:34.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.567122+0000", 2024-09-14T23:57:34.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2262000000000003e-05 2024-09-14T23:57:34.527 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:34.527 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:34.527 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:34.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.528 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:66564 3.c e60/49 3:33faa7ca:::rbd_data.111873792d9b.0000000000000003:head v 60'236, mlcod=57'233)", 2024-09-14T23:57:34.528 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:11.271596+0000", 2024-09-14T23:57:34.528 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.162196823999999, 2024-09-14T23:57:34.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0060396099999999999, 2024-09-14T23:57:34.528 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:34.528 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:34.528 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:34.528 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:34.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:34.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.271596+0000", 2024-09-14T23:57:34.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:34.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.529 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:34.529 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.271596+0000", 2024-09-14T23:57:34.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:34.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.529 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:34.529 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.271597+0000", 2024-09-14T23:57:34.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2559999999999999e-06 2024-09-14T23:57:34.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.529 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:34.529 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.274854+0000", 2024-09-14T23:57:34.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0032574470000000001 2024-09-14T23:57:34.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.530 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:34.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.274874+0000", 2024-09-14T23:57:34.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9449999999999998e-05 2024-09-14T23:57:34.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.530 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:34.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.274883+0000", 2024-09-14T23:57:34.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5980000000000003e-06 2024-09-14T23:57:34.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.530 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:34.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.274977+0000", 2024-09-14T23:57:34.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3270999999999995e-05 2024-09-14T23:57:34.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:34.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.275000+0000", 2024-09-14T23:57:34.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3088e-05 2024-09-14T23:57:34.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:34.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.277590+0000", 2024-09-14T23:57:34.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0025898660000000001 2024-09-14T23:57:34.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:34.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:34.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:34.532 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.277635+0000", 2024-09-14T23:57:34.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5633999999999997e-05 2024-09-14T23:57:34.532 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:34.532 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:34.532 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:34.532 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:34.532 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:34.532 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:34.534 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T23:57:34.535 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:34.532+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:34.535 INFO:teuthology.orchestra.run.smithi042.stdout:3375 write 0xb1f5b06 thru 0xb1fe75f (0x8c5a bytes) 2024-09-14T23:57:34.535 INFO:teuthology.orchestra.run.smithi042.stdout:3376 punch from 0xa4559f0 to 0xa460cfb, (0xb30b bytes) 2024-09-14T23:57:34.535 INFO:teuthology.orchestra.run.smithi042.stdout:3378 punch from 0x7d7ce39 to 0x7d7d11e, (0x2e5 bytes) 2024-09-14T23:57:34.535 INFO:teuthology.orchestra.run.smithi042.stdout:3379 write 0x5427f4 thru 0x544d24 (0x2531 bytes) 2024-09-14T23:57:34.535 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:34.535 INFO:teuthology.orchestra.run.smithi042.stdout:3381 write 0x57ff7c3 thru 0x580cb1f (0xd35d bytes) 2024-09-14T23:57:34.535 INFO:teuthology.orchestra.run.smithi042.stdout:3382 writesame 0x6ddfa38 thru 0x6dea4b1 data_size 0x1(0xaa7a bytes) 2024-09-14T23:57:34.535 INFO:teuthology.orchestra.run.smithi042.stdout:3384 read 0xae8d0bb thru 0xae8e3dd (0x1323 bytes) 2024-09-14T23:57:34.536 INFO:teuthology.orchestra.run.smithi042.stdout:3386 trunc from 0xb1fe760 to 0x2012cf0 2024-09-14T23:57:34.576 INFO:teuthology.orchestra.run.smithi042.stdout:3389 read 0x1a90bbd thru 0x1a95744 (0x4b88 bytes) 2024-09-14T23:57:34.577 INFO:teuthology.orchestra.run.smithi042.stdout:3390 write 0x1edb541 thru 0x1ee93fb (0xdebb bytes) 2024-09-14T23:57:34.593 INFO:teuthology.orchestra.run.smithi042.stdout:3391 write 0x1021825 thru 0x1022b37 (0x1313 bytes) 2024-09-14T23:57:34.616 INFO:teuthology.orchestra.run.smithi042.stdout:3392 writesame 0xb2556b1 thru 0xb257ea6 data_size 0x16(0x27f6 bytes) 2024-09-14T23:57:34.616 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:34.622 INFO:teuthology.orchestra.run.smithi042.stdout:3395 write 0xc012146 thru 0xc019693 (0x754e bytes) 2024-09-14T23:57:34.634 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:34.633+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:34.635 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:57:34.635 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:34.636 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 83, 2024-09-14T23:57:34.636 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "21916fcc-f95c-4bfa-8b8b-11ca45845613", 2024-09-14T23:57:34.636 INFO:teuthology.orchestra.run.smithi042.stdout: "created": "2024-09-14T23:56:03.167948+0000", 2024-09-14T23:57:34.636 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-09-14T23:57:33.998921+0000", 2024-09-14T23:57:34.636 INFO:teuthology.orchestra.run.smithi042.stdout: "last_up_change": "2024-09-14T23:56:14.322882+0000", 2024-09-14T23:57:34.636 INFO:teuthology.orchestra.run.smithi042.stdout: "last_in_change": "2024-09-14T23:56:56.035972+0000", 2024-09-14T23:57:34.636 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T23:57:34.636 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_num": 5806080, 2024-09-14T23:57:34.636 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_set": [ 2024-09-14T23:57:34.636 INFO:teuthology.orchestra.run.smithi042.stdout: "nodeep-scrub", 2024-09-14T23:57:34.636 INFO:teuthology.orchestra.run.smithi042.stdout: "noscrub", 2024-09-14T23:57:34.637 INFO:teuthology.orchestra.run.smithi042.stdout: "pglog_hardlimit", 2024-09-14T23:57:34.637 INFO:teuthology.orchestra.run.smithi042.stdout: "purged_snapdirs", 2024-09-14T23:57:34.637 INFO:teuthology.orchestra.run.smithi042.stdout: "recovery_deletes", 2024-09-14T23:57:34.637 INFO:teuthology.orchestra.run.smithi042.stdout: "sortbitwise" 2024-09-14T23:57:34.637 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-14T23:57:34.637 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_version": 6, 2024-09-14T23:57:34.637 INFO:teuthology.orchestra.run.smithi042.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T23:57:34.637 INFO:teuthology.orchestra.run.smithi042.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T23:57:34.637 INFO:teuthology.orchestra.run.smithi042.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T23:57:34.637 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_snapshot": "", 2024-09-14T23:57:34.638 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_max": 3, 2024-09-14T23:57:34.638 INFO:teuthology.orchestra.run.smithi042.stdout: "max_osd": 8, 2024-09-14T23:57:34.638 INFO:teuthology.orchestra.run.smithi042.stdout: "require_min_compat_client": "luminous", 2024-09-14T23:57:34.638 INFO:teuthology.orchestra.run.smithi042.stdout: "min_compat_client": "luminous", 2024-09-14T23:57:34.638 INFO:teuthology.orchestra.run.smithi042.stdout: "require_osd_release": "squid", 2024-09-14T23:57:34.638 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_crimson": false, 2024-09-14T23:57:34.638 INFO:teuthology.orchestra.run.smithi042.stdout: "pools": [ 2024-09-14T23:57:34.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.638 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 1, 2024-09-14T23:57:34.638 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": ".mgr", 2024-09-14T23:57:34.638 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-14T23:56:13.500197+0000", 2024-09-14T23:57:34.638 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-09-14T23:57:34.638 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool", 2024-09-14T23:57:34.639 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-14T23:57:34.639 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-14T23:57:34.639 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-14T23:57:34.639 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-14T23:57:34.639 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-14T23:57:34.639 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-14T23:57:34.639 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T23:57:34.639 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T23:57:34.639 INFO:teuthology.orchestra.run.smithi042.stdout: "is_stretch_pool": false, 2024-09-14T23:57:34.639 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-14T23:57:34.639 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-14T23:57:34.639 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 1, 2024-09-14T23:57:34.639 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 1, 2024-09-14T23:57:34.640 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 1, 2024-09-14T23:57:34.640 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 1, 2024-09-14T23:57:34.640 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 1, 2024-09-14T23:57:34.640 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-14T23:57:34.640 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-14T23:57:34.640 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-14T23:57:34.640 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-14T23:57:34.640 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-14T23:57:34.640 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-14T23:57:34.640 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-14T23:57:34.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.641 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "17", 2024-09-14T23:57:34.641 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "0", 2024-09-14T23:57:34.641 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T23:57:34.641 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T23:57:34.641 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-14T23:57:34.641 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-14T23:57:34.641 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 0, 2024-09-14T23:57:34.641 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 0, 2024-09-14T23:57:34.641 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-14T23:57:34.641 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-14T23:57:34.641 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-14T23:57:34.641 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-14T23:57:34.641 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-14T23:57:34.642 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-14T23:57:34.642 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-14T23:57:34.642 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-14T23:57:34.642 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-14T23:57:34.642 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-14T23:57:34.642 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-14T23:57:34.642 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T23:57:34.642 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T23:57:34.642 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T23:57:34.642 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-14T23:57:34.642 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-14T23:57:34.642 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-14T23:57:34.642 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-14T23:57:34.643 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-14T23:57:34.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.643 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-14T23:57:34.643 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-14T23:57:34.643 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-14T23:57:34.643 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-14T23:57:34.643 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-14T23:57:34.643 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T23:57:34.643 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-14T23:57:34.643 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-14T23:57:34.643 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-14T23:57:34.643 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-14T23:57:34.643 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-14T23:57:34.644 INFO:teuthology.orchestra.run.smithi042.stdout: "options": { 2024-09-14T23:57:34.644 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_max": 32, 2024-09-14T23:57:34.644 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_min": 1 2024-09-14T23:57:34.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.644 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-14T23:57:34.644 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": {} 2024-09-14T23:57:34.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.644 INFO:teuthology.orchestra.run.smithi042.stdout: "read_balance": { 2024-09-14T23:57:34.644 INFO:teuthology.orchestra.run.smithi042.stdout: "score_type": "Fair distribution", 2024-09-14T23:57:34.644 INFO:teuthology.orchestra.run.smithi042.stdout: "score_acting": 6.9000000953674316, 2024-09-14T23:57:34.644 INFO:teuthology.orchestra.run.smithi042.stdout: "score_stable": 6.9000000953674316, 2024-09-14T23:57:34.644 INFO:teuthology.orchestra.run.smithi042.stdout: "optimal_score": 0.28999999165534973, 2024-09-14T23:57:34.645 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_acting": 2, 2024-09-14T23:57:34.645 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_stable": 2, 2024-09-14T23:57:34.645 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity_weighted": 0.8712921142578125, 2024-09-14T23:57:34.645 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity": 1, 2024-09-14T23:57:34.645 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity_weighted": 0.87000000476837158 2024-09-14T23:57:34.645 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:34.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.645 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 2, 2024-09-14T23:57:34.645 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": "rbd", 2024-09-14T23:57:34.645 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-14T23:56:21.284705+0000", 2024-09-14T23:57:34.645 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 8193, 2024-09-14T23:57:34.645 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T23:57:34.646 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-14T23:57:34.646 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-14T23:57:34.646 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-14T23:57:34.646 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-14T23:57:34.646 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-14T23:57:34.646 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-14T23:57:34.646 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T23:57:34.646 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T23:57:34.646 INFO:teuthology.orchestra.run.smithi042.stdout: "is_stretch_pool": false, 2024-09-14T23:57:34.646 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-14T23:57:34.646 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-14T23:57:34.646 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 8, 2024-09-14T23:57:34.646 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 8, 2024-09-14T23:57:34.646 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 8, 2024-09-14T23:57:34.647 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 8, 2024-09-14T23:57:34.647 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 8, 2024-09-14T23:57:34.647 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-14T23:57:34.647 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-14T23:57:34.647 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-14T23:57:34.647 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-14T23:57:34.647 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-14T23:57:34.647 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-14T23:57:34.647 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-14T23:57:34.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.647 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "21", 2024-09-14T23:57:34.647 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "0", 2024-09-14T23:57:34.647 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T23:57:34.647 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T23:57:34.647 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-14T23:57:34.648 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-14T23:57:34.648 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 2, 2024-09-14T23:57:34.648 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 21, 2024-09-14T23:57:34.648 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-14T23:57:34.648 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-14T23:57:34.648 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-14T23:57:34.648 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-14T23:57:34.648 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-14T23:57:34.648 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-14T23:57:34.648 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-14T23:57:34.648 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-14T23:57:34.648 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-14T23:57:34.648 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-14T23:57:34.649 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-14T23:57:34.649 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T23:57:34.649 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T23:57:34.649 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T23:57:34.649 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-14T23:57:34.649 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-14T23:57:34.649 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-14T23:57:34.649 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-14T23:57:34.649 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-14T23:57:34.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.649 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-14T23:57:34.649 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-14T23:57:34.649 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-14T23:57:34.649 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-14T23:57:34.650 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-14T23:57:34.650 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T23:57:34.650 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-14T23:57:34.650 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-14T23:57:34.650 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-14T23:57:34.650 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-14T23:57:34.650 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-14T23:57:34.650 INFO:teuthology.orchestra.run.smithi042.stdout: "options": {}, 2024-09-14T23:57:34.650 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-14T23:57:34.650 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd": {} 2024-09-14T23:57:34.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.650 INFO:teuthology.orchestra.run.smithi042.stdout: "read_balance": { 2024-09-14T23:57:34.650 INFO:teuthology.orchestra.run.smithi042.stdout: "score_type": "Fair distribution", 2024-09-14T23:57:34.650 INFO:teuthology.orchestra.run.smithi042.stdout: "score_acting": 2.630000114440918, 2024-09-14T23:57:34.651 INFO:teuthology.orchestra.run.smithi042.stdout: "score_stable": 2.630000114440918, 2024-09-14T23:57:34.651 INFO:teuthology.orchestra.run.smithi042.stdout: "optimal_score": 1, 2024-09-14T23:57:34.651 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_acting": 2.630000114440918, 2024-09-14T23:57:34.651 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_stable": 2.630000114440918, 2024-09-14T23:57:34.651 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity_weighted": 0.8712921142578125, 2024-09-14T23:57:34.651 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity": 1, 2024-09-14T23:57:34.651 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity_weighted": 0.87000000476837158 2024-09-14T23:57:34.651 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:34.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.651 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 3, 2024-09-14T23:57:34.651 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": "pool_client.0", 2024-09-14T23:57:34.651 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-14T23:56:30.861504+0000", 2024-09-14T23:57:34.651 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 8193, 2024-09-14T23:57:34.651 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T23:57:34.652 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-14T23:57:34.652 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-14T23:57:34.652 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-14T23:57:34.652 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-14T23:57:34.652 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-14T23:57:34.652 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-14T23:57:34.652 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T23:57:34.652 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T23:57:34.652 INFO:teuthology.orchestra.run.smithi042.stdout: "is_stretch_pool": false, 2024-09-14T23:57:34.652 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-14T23:57:34.652 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-14T23:57:34.652 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 32, 2024-09-14T23:57:34.652 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 32, 2024-09-14T23:57:34.653 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 32, 2024-09-14T23:57:34.653 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 32, 2024-09-14T23:57:34.653 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 32, 2024-09-14T23:57:34.653 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-14T23:57:34.653 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-14T23:57:34.653 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-14T23:57:34.653 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-14T23:57:34.653 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-14T23:57:34.653 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-14T23:57:34.653 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-14T23:57:34.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.653 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "83", 2024-09-14T23:57:34.654 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "0", 2024-09-14T23:57:34.654 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T23:57:34.654 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T23:57:34.654 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-14T23:57:34.654 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-14T23:57:34.654 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 33, 2024-09-14T23:57:34.654 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 83, 2024-09-14T23:57:34.654 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-14T23:57:34.654 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-14T23:57:34.654 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-14T23:57:34.655 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-14T23:57:34.655 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-14T23:57:34.655 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-14T23:57:34.655 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-14T23:57:34.655 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-14T23:57:34.655 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-14T23:57:34.655 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-14T23:57:34.655 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-14T23:57:34.655 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T23:57:34.655 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T23:57:34.655 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T23:57:34.655 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-14T23:57:34.655 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-14T23:57:34.655 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-14T23:57:34.655 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-14T23:57:34.656 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-14T23:57:34.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.656 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-14T23:57:34.656 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-14T23:57:34.656 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-14T23:57:34.656 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-14T23:57:34.656 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-14T23:57:34.656 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T23:57:34.656 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-14T23:57:34.656 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-14T23:57:34.656 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-14T23:57:34.656 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-14T23:57:34.656 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-14T23:57:34.657 INFO:teuthology.orchestra.run.smithi042.stdout: "options": {}, 2024-09-14T23:57:34.657 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-14T23:57:34.657 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd": {} 2024-09-14T23:57:34.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.657 INFO:teuthology.orchestra.run.smithi042.stdout: "read_balance": { 2024-09-14T23:57:34.657 INFO:teuthology.orchestra.run.smithi042.stdout: "score_type": "Fair distribution", 2024-09-14T23:57:34.657 INFO:teuthology.orchestra.run.smithi042.stdout: "score_acting": 1.5399999618530273, 2024-09-14T23:57:34.657 INFO:teuthology.orchestra.run.smithi042.stdout: "score_stable": 1.5399999618530273, 2024-09-14T23:57:34.657 INFO:teuthology.orchestra.run.smithi042.stdout: "optimal_score": 1.1399999856948853, 2024-09-14T23:57:34.657 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_acting": 1.75, 2024-09-14T23:57:34.657 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_stable": 1.75, 2024-09-14T23:57:34.657 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity_weighted": 0.8712921142578125, 2024-09-14T23:57:34.657 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity": 1, 2024-09-14T23:57:34.657 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity_weighted": 0.87000000476837158 2024-09-14T23:57:34.658 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:34.658 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:34.658 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-14T23:57:34.658 INFO:teuthology.orchestra.run.smithi042.stdout: "osds": [ 2024-09-14T23:57:34.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.658 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 0, 2024-09-14T23:57:34.658 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "db9d0c67-2d60-4592-9ad3-f99e5294f93f", 2024-09-14T23:57:34.658 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-14T23:57:34.658 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-14T23:57:34.658 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-14T23:57:34.658 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-14T23:57:34.658 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-14T23:57:34.658 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-14T23:57:34.658 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 14, 2024-09-14T23:57:34.658 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 49, 2024-09-14T23:57:34.659 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-14T23:57:34.659 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-14T23:57:34.659 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-14T23:57:34.659 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:34.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.659 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:34.659 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6816", 2024-09-14T23:57:34.659 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 314865173 2024-09-14T23:57:34.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.659 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:34.659 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6817", 2024-09-14T23:57:34.659 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 314865173 2024-09-14T23:57:34.660 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:34.660 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:34.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.660 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-14T23:57:34.660 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:34.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.660 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:34.660 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6818", 2024-09-14T23:57:34.660 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 314865173 2024-09-14T23:57:34.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.660 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:34.660 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6819", 2024-09-14T23:57:34.660 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 314865173 2024-09-14T23:57:34.661 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:34.661 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:34.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.661 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-14T23:57:34.661 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:34.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.661 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:34.661 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6822", 2024-09-14T23:57:34.661 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 314865173 2024-09-14T23:57:34.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.661 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:34.661 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6823", 2024-09-14T23:57:34.661 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 314865173 2024-09-14T23:57:34.662 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:34.662 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:34.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.662 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-14T23:57:34.662 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:34.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.662 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:34.662 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6820", 2024-09-14T23:57:34.662 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 314865173 2024-09-14T23:57:34.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.662 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:34.662 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6821", 2024-09-14T23:57:34.662 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 314865173 2024-09-14T23:57:34.663 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:34.663 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:34.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.663 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6817/314865173", 2024-09-14T23:57:34.663 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6819/314865173", 2024-09-14T23:57:34.663 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6823/314865173", 2024-09-14T23:57:34.663 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6821/314865173", 2024-09-14T23:57:34.663 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-14T23:57:34.663 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-14T23:57:34.663 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-14T23:57:34.663 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:34.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.663 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 1, 2024-09-14T23:57:34.664 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "6c1a2765-c591-4c57-8ab3-ee6645970aa5", 2024-09-14T23:57:34.664 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-14T23:57:34.664 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-14T23:57:34.664 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-14T23:57:34.664 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-14T23:57:34.664 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-14T23:57:34.664 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-14T23:57:34.664 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 14, 2024-09-14T23:57:34.664 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 46, 2024-09-14T23:57:34.664 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-14T23:57:34.664 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-14T23:57:34.664 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-14T23:57:34.664 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:34.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.665 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:34.665 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6800", 2024-09-14T23:57:34.665 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3197912310 2024-09-14T23:57:34.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.665 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:34.665 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6801", 2024-09-14T23:57:34.665 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3197912310 2024-09-14T23:57:34.665 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:34.665 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:34.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.665 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-14T23:57:34.665 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:34.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.666 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:34.666 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6802", 2024-09-14T23:57:34.666 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3197912310 2024-09-14T23:57:34.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.666 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:34.666 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6803", 2024-09-14T23:57:34.666 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3197912310 2024-09-14T23:57:34.666 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:34.666 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:34.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.666 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-14T23:57:34.666 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:34.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.667 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:34.667 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6806", 2024-09-14T23:57:34.667 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3197912310 2024-09-14T23:57:34.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.667 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:34.667 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6807", 2024-09-14T23:57:34.667 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3197912310 2024-09-14T23:57:34.667 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:34.667 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:34.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.667 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-14T23:57:34.667 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:34.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.668 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:34.668 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6804", 2024-09-14T23:57:34.668 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3197912310 2024-09-14T23:57:34.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.668 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:34.668 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6805", 2024-09-14T23:57:34.668 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3197912310 2024-09-14T23:57:34.668 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:34.668 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:34.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.668 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6801/3197912310", 2024-09-14T23:57:34.668 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6803/3197912310", 2024-09-14T23:57:34.668 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6807/3197912310", 2024-09-14T23:57:34.669 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6805/3197912310", 2024-09-14T23:57:34.669 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-14T23:57:34.669 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-14T23:57:34.669 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-14T23:57:34.669 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:34.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.669 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 2, 2024-09-14T23:57:34.669 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "930db893-7be6-4645-a7f1-e043c5ec95eb", 2024-09-14T23:57:34.669 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-14T23:57:34.669 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-14T23:57:34.669 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-14T23:57:34.669 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-14T23:57:34.669 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-14T23:57:34.670 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-14T23:57:34.670 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 13, 2024-09-14T23:57:34.670 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 46, 2024-09-14T23:57:34.670 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-14T23:57:34.670 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-14T23:57:34.670 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-14T23:57:34.670 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:34.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.670 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:34.670 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6808", 2024-09-14T23:57:34.670 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3241153660 2024-09-14T23:57:34.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.670 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:34.671 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6809", 2024-09-14T23:57:34.671 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3241153660 2024-09-14T23:57:34.671 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:34.671 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:34.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.671 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-14T23:57:34.671 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:34.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.671 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:34.671 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6810", 2024-09-14T23:57:34.671 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3241153660 2024-09-14T23:57:34.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.672 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:34.672 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6811", 2024-09-14T23:57:34.672 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3241153660 2024-09-14T23:57:34.672 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:34.672 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:34.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.672 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-14T23:57:34.672 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:34.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.672 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:34.672 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6814", 2024-09-14T23:57:34.672 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3241153660 2024-09-14T23:57:34.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.673 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:34.673 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6815", 2024-09-14T23:57:34.673 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3241153660 2024-09-14T23:57:34.673 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:34.673 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:34.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.673 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-14T23:57:34.673 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:34.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.673 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:34.673 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6812", 2024-09-14T23:57:34.673 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3241153660 2024-09-14T23:57:34.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.674 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:34.674 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6813", 2024-09-14T23:57:34.674 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3241153660 2024-09-14T23:57:34.674 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:34.674 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:34.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.674 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6809/3241153660", 2024-09-14T23:57:34.674 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6811/3241153660", 2024-09-14T23:57:34.674 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6815/3241153660", 2024-09-14T23:57:34.674 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6813/3241153660", 2024-09-14T23:57:34.674 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-14T23:57:34.674 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-14T23:57:34.674 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-14T23:57:34.674 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:34.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.675 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 3, 2024-09-14T23:57:34.675 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "45bb3b84-b1c4-4f87-a7d1-0eb77b81741b", 2024-09-14T23:57:34.675 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-14T23:57:34.675 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-14T23:57:34.675 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-14T23:57:34.675 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-14T23:57:34.675 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-14T23:57:34.675 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-14T23:57:34.675 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-14T23:57:34.675 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 75, 2024-09-14T23:57:34.675 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-14T23:57:34.675 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-14T23:57:34.675 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-14T23:57:34.675 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:34.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.676 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:34.676 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6824", 2024-09-14T23:57:34.676 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1736416326 2024-09-14T23:57:34.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.676 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:34.676 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6825", 2024-09-14T23:57:34.676 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1736416326 2024-09-14T23:57:34.676 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:34.676 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:34.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.676 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-14T23:57:34.677 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:34.677 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.677 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:34.677 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6826", 2024-09-14T23:57:34.677 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1736416326 2024-09-14T23:57:34.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.677 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.677 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:34.677 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6827", 2024-09-14T23:57:34.677 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1736416326 2024-09-14T23:57:34.677 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:34.677 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:34.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.677 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-14T23:57:34.678 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:34.678 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.678 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:34.678 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6830", 2024-09-14T23:57:34.678 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1736416326 2024-09-14T23:57:34.678 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.678 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.678 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:34.678 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6831", 2024-09-14T23:57:34.678 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1736416326 2024-09-14T23:57:34.678 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:34.678 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:34.678 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.679 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-14T23:57:34.679 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:34.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.679 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:34.679 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6828", 2024-09-14T23:57:34.679 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1736416326 2024-09-14T23:57:34.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.679 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:34.679 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6829", 2024-09-14T23:57:34.679 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1736416326 2024-09-14T23:57:34.679 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:34.679 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:34.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.679 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6825/1736416326", 2024-09-14T23:57:34.679 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6827/1736416326", 2024-09-14T23:57:34.680 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6831/1736416326", 2024-09-14T23:57:34.680 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6829/1736416326", 2024-09-14T23:57:34.680 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-14T23:57:34.680 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-14T23:57:34.680 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-14T23:57:34.680 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:34.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.680 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 4, 2024-09-14T23:57:34.680 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "a812e38f-c0f9-4d33-9373-1910fb09103c", 2024-09-14T23:57:34.680 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-14T23:57:34.680 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-14T23:57:34.680 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-14T23:57:34.680 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 0.9703369140625, 2024-09-14T23:57:34.680 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-14T23:57:34.681 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-14T23:57:34.681 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-14T23:57:34.681 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 49, 2024-09-14T23:57:34.681 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-14T23:57:34.681 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-14T23:57:34.681 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-14T23:57:34.681 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:34.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.681 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:34.681 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6808", 2024-09-14T23:57:34.681 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1755154427 2024-09-14T23:57:34.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.681 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:34.682 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6809", 2024-09-14T23:57:34.682 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1755154427 2024-09-14T23:57:34.682 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:34.682 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:34.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.682 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-14T23:57:34.682 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:34.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.682 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:34.682 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6810", 2024-09-14T23:57:34.682 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1755154427 2024-09-14T23:57:34.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.683 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:34.683 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6811", 2024-09-14T23:57:34.683 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1755154427 2024-09-14T23:57:34.683 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:34.683 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:34.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.683 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-14T23:57:34.683 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:34.683 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.683 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:34.683 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6814", 2024-09-14T23:57:34.683 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1755154427 2024-09-14T23:57:34.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.683 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.684 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:34.684 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6815", 2024-09-14T23:57:34.684 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1755154427 2024-09-14T23:57:34.684 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:34.684 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:34.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.684 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-14T23:57:34.684 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:34.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.684 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:34.684 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6812", 2024-09-14T23:57:34.684 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1755154427 2024-09-14T23:57:34.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.685 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:34.685 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6813", 2024-09-14T23:57:34.685 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1755154427 2024-09-14T23:57:34.685 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:34.685 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:34.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.685 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.43:6809/1755154427", 2024-09-14T23:57:34.685 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.43:6811/1755154427", 2024-09-14T23:57:34.685 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.43:6815/1755154427", 2024-09-14T23:57:34.685 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.43:6813/1755154427", 2024-09-14T23:57:34.685 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-14T23:57:34.685 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-14T23:57:34.685 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-14T23:57:34.686 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:34.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.686 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 5, 2024-09-14T23:57:34.686 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "0de87e5b-7a21-4fb4-9543-d96c065786f8", 2024-09-14T23:57:34.686 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-14T23:57:34.686 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-14T23:57:34.686 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-14T23:57:34.686 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 0, 2024-09-14T23:57:34.686 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-14T23:57:34.686 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-14T23:57:34.686 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-14T23:57:34.686 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 23, 2024-09-14T23:57:34.686 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-14T23:57:34.686 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-14T23:57:34.687 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-14T23:57:34.687 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:34.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.687 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:34.687 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6816", 2024-09-14T23:57:34.687 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3033769028 2024-09-14T23:57:34.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.687 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:34.687 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6817", 2024-09-14T23:57:34.687 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3033769028 2024-09-14T23:57:34.687 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:34.687 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:34.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.687 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-14T23:57:34.687 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:34.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.688 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:34.688 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6818", 2024-09-14T23:57:34.688 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3033769028 2024-09-14T23:57:34.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.688 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:34.688 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6819", 2024-09-14T23:57:34.688 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3033769028 2024-09-14T23:57:34.688 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:34.688 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:34.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.688 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-14T23:57:34.688 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:34.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.688 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:34.689 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6822", 2024-09-14T23:57:34.689 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3033769028 2024-09-14T23:57:34.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.689 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:34.689 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6823", 2024-09-14T23:57:34.689 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3033769028 2024-09-14T23:57:34.689 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:34.689 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:34.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.689 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-14T23:57:34.689 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:34.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.689 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:34.690 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6820", 2024-09-14T23:57:34.690 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3033769028 2024-09-14T23:57:34.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.690 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:34.690 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6821", 2024-09-14T23:57:34.690 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3033769028 2024-09-14T23:57:34.690 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:34.690 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:34.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.690 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.43:6817/3033769028", 2024-09-14T23:57:34.690 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.43:6819/3033769028", 2024-09-14T23:57:34.690 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.43:6823/3033769028", 2024-09-14T23:57:34.690 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.43:6821/3033769028", 2024-09-14T23:57:34.691 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-14T23:57:34.691 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-14T23:57:34.691 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-14T23:57:34.691 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:34.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.691 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 6, 2024-09-14T23:57:34.691 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "c58c1d59-ad02-40f0-98e3-9078ef7e622b", 2024-09-14T23:57:34.691 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-14T23:57:34.691 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-14T23:57:34.691 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-14T23:57:34.691 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-14T23:57:34.691 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-14T23:57:34.691 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-14T23:57:34.692 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-14T23:57:34.692 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 76, 2024-09-14T23:57:34.692 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-14T23:57:34.692 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-14T23:57:34.692 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-14T23:57:34.692 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:34.692 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.692 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:34.692 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6800", 2024-09-14T23:57:34.692 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3314282416 2024-09-14T23:57:34.692 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.692 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.693 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:34.693 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6801", 2024-09-14T23:57:34.693 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3314282416 2024-09-14T23:57:34.693 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:34.693 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:34.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.693 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-14T23:57:34.693 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:34.693 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.693 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:34.693 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6802", 2024-09-14T23:57:34.693 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3314282416 2024-09-14T23:57:34.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.694 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:34.694 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6803", 2024-09-14T23:57:34.694 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3314282416 2024-09-14T23:57:34.694 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:34.694 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:34.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.694 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-14T23:57:34.694 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:34.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.694 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:34.694 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6806", 2024-09-14T23:57:34.695 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3314282416 2024-09-14T23:57:34.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.695 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:34.695 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6807", 2024-09-14T23:57:34.695 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3314282416 2024-09-14T23:57:34.695 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:34.695 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:34.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.695 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-14T23:57:34.695 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:34.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.695 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:34.696 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6804", 2024-09-14T23:57:34.696 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3314282416 2024-09-14T23:57:34.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.696 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:34.696 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6805", 2024-09-14T23:57:34.696 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3314282416 2024-09-14T23:57:34.696 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:34.696 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:34.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.696 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.43:6801/3314282416", 2024-09-14T23:57:34.696 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.43:6803/3314282416", 2024-09-14T23:57:34.696 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.43:6807/3314282416", 2024-09-14T23:57:34.696 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.43:6805/3314282416", 2024-09-14T23:57:34.697 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-14T23:57:34.697 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-14T23:57:34.697 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-14T23:57:34.697 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:34.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.697 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 7, 2024-09-14T23:57:34.697 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "389bdb04-20e3-4a7a-8504-687e034bd7dc", 2024-09-14T23:57:34.697 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-14T23:57:34.697 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-14T23:57:34.697 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-14T23:57:34.697 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-14T23:57:34.697 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-14T23:57:34.697 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-14T23:57:34.698 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-14T23:57:34.698 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 75, 2024-09-14T23:57:34.698 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-14T23:57:34.698 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-14T23:57:34.698 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-14T23:57:34.698 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:34.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.698 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:34.698 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6824", 2024-09-14T23:57:34.698 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1481350118 2024-09-14T23:57:34.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.698 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:34.699 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6825", 2024-09-14T23:57:34.699 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1481350118 2024-09-14T23:57:34.699 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:34.699 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:34.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.699 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-14T23:57:34.699 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:34.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.699 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:34.699 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6826", 2024-09-14T23:57:34.699 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1481350118 2024-09-14T23:57:34.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.699 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:34.699 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6827", 2024-09-14T23:57:34.700 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1481350118 2024-09-14T23:57:34.700 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:34.700 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:34.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.700 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-14T23:57:34.700 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:34.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.700 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:34.700 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6830", 2024-09-14T23:57:34.700 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1481350118 2024-09-14T23:57:34.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.700 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:34.700 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6831", 2024-09-14T23:57:34.700 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1481350118 2024-09-14T23:57:34.701 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:34.701 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:34.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.701 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-14T23:57:34.701 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:34.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.701 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:34.701 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6828", 2024-09-14T23:57:34.701 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1481350118 2024-09-14T23:57:34.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.701 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:34.701 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6829", 2024-09-14T23:57:34.701 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1481350118 2024-09-14T23:57:34.701 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:34.702 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:34.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.702 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.43:6825/1481350118", 2024-09-14T23:57:34.702 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.43:6827/1481350118", 2024-09-14T23:57:34.702 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.43:6831/1481350118", 2024-09-14T23:57:34.702 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.43:6829/1481350118", 2024-09-14T23:57:34.702 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-14T23:57:34.702 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-14T23:57:34.702 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-14T23:57:34.702 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:34.702 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:34.702 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-14T23:57:34.702 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_xinfo": [ 2024-09-14T23:57:34.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.703 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 0, 2024-09-14T23:57:34.703 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-14T23:57:34.703 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-14T23:57:34.703 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-14T23:57:34.703 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-14T23:57:34.703 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-14T23:57:34.703 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-14T23:56:11.077810+0000", 2024-09-14T23:57:34.703 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-14T23:57:34.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.703 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 1, 2024-09-14T23:57:34.703 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-14T23:57:34.703 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-14T23:57:34.703 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-14T23:57:34.704 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-14T23:57:34.704 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-14T23:57:34.704 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-14T23:56:11.578838+0000", 2024-09-14T23:57:34.704 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-14T23:57:34.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.704 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 2, 2024-09-14T23:57:34.704 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-14T23:57:34.704 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-14T23:57:34.704 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-14T23:57:34.704 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-14T23:57:34.704 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-14T23:57:34.704 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-14T23:56:10.895490+0000", 2024-09-14T23:57:34.704 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-14T23:57:34.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.705 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 3, 2024-09-14T23:57:34.705 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-14T23:57:34.705 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-14T23:57:34.705 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-14T23:57:34.705 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-14T23:57:34.705 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-14T23:57:34.705 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-14T23:56:12.290821+0000", 2024-09-14T23:57:34.705 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-14T23:57:34.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.705 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 4, 2024-09-14T23:57:34.705 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-14T23:57:34.705 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-14T23:57:34.706 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-14T23:57:34.706 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-14T23:57:34.706 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-14T23:57:34.706 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-14T23:56:11.800581+0000", 2024-09-14T23:57:34.706 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-14T23:57:34.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.706 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 5, 2024-09-14T23:57:34.706 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-14T23:57:34.706 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-14T23:57:34.706 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-14T23:57:34.706 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-14T23:57:34.706 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-14T23:57:34.706 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-14T23:56:11.883155+0000", 2024-09-14T23:57:34.706 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-14T23:57:34.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.707 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 6, 2024-09-14T23:57:34.707 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-14T23:57:34.707 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-14T23:57:34.707 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-14T23:57:34.707 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-14T23:57:34.707 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-14T23:57:34.707 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-14T23:56:11.604790+0000", 2024-09-14T23:57:34.707 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-14T23:57:34.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.707 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 7, 2024-09-14T23:57:34.707 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-14T23:57:34.707 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-14T23:57:34.708 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-14T23:57:34.708 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-14T23:57:34.708 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-14T23:57:34.708 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-14T23:56:12.067759+0000", 2024-09-14T23:57:34.708 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-14T23:57:34.708 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:34.708 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-14T23:57:34.708 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_upmap": [ 2024-09-14T23:57:34.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:34.708 INFO:teuthology.orchestra.run.smithi042.stdout: "pgid": "3.1e", 2024-09-14T23:57:34.708 INFO:teuthology.orchestra.run.smithi042.stdout: "osds": [ 2024-09-14T23:57:34.708 INFO:teuthology.orchestra.run.smithi042.stdout: 6, 2024-09-14T23:57:34.708 INFO:teuthology.orchestra.run.smithi042.stdout: 1 2024-09-14T23:57:34.708 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:34.708 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:34.709 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-14T23:57:34.709 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_upmap_items": [], 2024-09-14T23:57:34.709 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_upmap_primaries": [], 2024-09-14T23:57:34.709 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_temp": [], 2024-09-14T23:57:34.709 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_temp": [], 2024-09-14T23:57:34.709 INFO:teuthology.orchestra.run.smithi042.stdout: "blocklist": {}, 2024-09-14T23:57:34.709 INFO:teuthology.orchestra.run.smithi042.stdout: "range_blocklist": {}, 2024-09-14T23:57:34.709 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profiles": { 2024-09-14T23:57:34.709 INFO:teuthology.orchestra.run.smithi042.stdout: "default": { 2024-09-14T23:57:34.709 INFO:teuthology.orchestra.run.smithi042.stdout: "crush-failure-domain": "osd", 2024-09-14T23:57:34.709 INFO:teuthology.orchestra.run.smithi042.stdout: "k": "2", 2024-09-14T23:57:34.709 INFO:teuthology.orchestra.run.smithi042.stdout: "m": "1", 2024-09-14T23:57:34.709 INFO:teuthology.orchestra.run.smithi042.stdout: "plugin": "jerasure", 2024-09-14T23:57:34.710 INFO:teuthology.orchestra.run.smithi042.stdout: "technique": "reed_sol_van" 2024-09-14T23:57:34.710 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:34.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:34.710 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps_queue": [], 2024-09-14T23:57:34.710 INFO:teuthology.orchestra.run.smithi042.stdout: "new_removed_snaps": [], 2024-09-14T23:57:34.710 INFO:teuthology.orchestra.run.smithi042.stdout: "new_purged_snaps": [], 2024-09-14T23:57:34.710 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_node_flags": {}, 2024-09-14T23:57:34.710 INFO:teuthology.orchestra.run.smithi042.stdout: "device_class_flags": {}, 2024-09-14T23:57:34.710 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode": { 2024-09-14T23:57:34.710 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode_enabled": false, 2024-09-14T23:57:34.710 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_bucket_count": 0, 2024-09-14T23:57:34.710 INFO:teuthology.orchestra.run.smithi042.stdout: "degraded_stretch_mode": 0, 2024-09-14T23:57:34.710 INFO:teuthology.orchestra.run.smithi042.stdout: "recovering_stretch_mode": 0, 2024-09-14T23:57:34.710 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode_bucket": 0 2024-09-14T23:57:34.710 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:34.710 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:34.711 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:34.711 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:34.711 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:57:34.711 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:34.711 INFO:teuthology.orchestra.run.smithi042.stdout:3397 writesame 0x6c44767 thru 0x6c47970 data_size 0xa(0x320a bytes) 2024-09-14T23:57:34.713 INFO:tasks.thrashosds.thrasher:j is {'epoch': 83, 'fsid': '21916fcc-f95c-4bfa-8b8b-11ca45845613', 'created': '2024-09-14T23:56:03.167948+0000', 'modified': '2024-09-14T23:57:33.998921+0000', 'last_up_change': '2024-09-14T23:56:14.322882+0000', 'last_in_change': '2024-09-14T23:56:56.035972+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T23:56:13.500197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8712921142578125, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8700000047683716}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T23:56:21.284705+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.630000114440918, 'score_stable': 2.630000114440918, 'optimal_score': 1, 'raw_score_acting': 2.630000114440918, 'raw_score_stable': 2.630000114440918, 'primary_affinity_weighted': 0.8712921142578125, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8700000047683716}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T23:56:30.861504+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 33, 'snap_epoch': 83, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5399999618530273, 'score_stable': 1.5399999618530273, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.8712921142578125, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8700000047683716}}], 'osds': [{'osd': 0, 'uuid': 'db9d0c67-2d60-4592-9ad3-f99e5294f93f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 49, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6816', 'nonce': 314865173}, {'type': 'v1', 'addr': '172.21.15.42:6817', 'nonce': 314865173}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6818', 'nonce': 314865173}, {'type': 'v1', 'addr': '172.21.15.42:6819', 'nonce': 314865173}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6822', 'nonce': 314865173}, {'type': 'v1', 'addr': '172.21.15.42:6823', 'nonce': 314865173}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6820', 'nonce': 314865173}, {'type': 'v1', 'addr': '172.21.15.42:6821', 'nonce': 314865173}]}, 'public_addr': '172.21.15.42:6817/314865173', 'cluster_addr': '172.21.15.42:6819/314865173', 'heartbeat_back_addr': '172.21.15.42:6823/314865173', 'heartbeat_front_addr': '172.21.15.42:6821/314865173', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '6c1a2765-c591-4c57-8ab3-ee6645970aa5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 46, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6800', 'nonce': 3197912310}, {'type': 'v1', 'addr': '172.21.15.42:6801', 'nonce': 3197912310}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6802', 'nonce': 3197912310}, {'type': 'v1', 'addr': '172.21.15.42:6803', 'nonce': 3197912310}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6806', 'nonce': 3197912310}, {'type': 'v1', 'addr': '172.21.15.42:6807', 'nonce': 3197912310}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6804', 'nonce': 3197912310}, {'type': 'v1', 'addr': '172.21.15.42:6805', 'nonce': 3197912310}]}, 'public_addr': '172.21.15.42:6801/3197912310', 'cluster_addr': '172.21.15.42:6803/3197912310', 'heartbeat_back_addr': '172.21.15.42:6807/3197912310', 'heartbeat_front_addr': '172.21.15.42:6805/3197912310', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '930db893-7be6-4645-a7f1-e043c5ec95eb', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 46, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6808', 'nonce': 3241153660}, {'type': 'v1', 'addr': '172.21.15.42:6809', 'nonce': 3241153660}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6810', 'nonce': 3241153660}, {'type': 'v1', 'addr': '172.21.15.42:6811', 'nonce': 3241153660}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6814', 'nonce': 3241153660}, {'type': 'v1', 'addr': '172.21.15.42:6815', 'nonce': 3241153660}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6812', 'nonce': 3241153660}, {'type': 'v1', 'addr': '172.21.15.42:6813', 'nonce': 3241153660}]}, 'public_addr': '172.21.15.42:6809/3241153660', 'cluster_addr': '172.21.15.42:6811/3241153660', 'heartbeat_back_addr': '172.21.15.42:6815/3241153660', 'heartbeat_front_addr': '172.21.15.42:6813/3241153660', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '45bb3b84-b1c4-4f87-a7d1-0eb77b81741b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 75, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6824', 'nonce': 1736416326}, {'type': 'v1', 'addr': '172.21.15.42:6825', 'nonce': 1736416326}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6826', 'nonce': 1736416326}, {'type': 'v1', 'addr': '172.21.15.42:6827', 'nonce': 1736416326}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6830', 'nonce': 1736416326}, {'type': 'v1', 'addr': '172.21.15.42:6831', 'nonce': 1736416326}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6828', 'nonce': 1736416326}, {'type': 'v1', 'addr': '172.21.15.42:6829', 'nonce': 1736416326}]}, 'public_addr': '172.21.15.42:6825/1736416326', 'cluster_addr': '172.21.15.42:6827/1736416326', 'heartbeat_back_addr': '172.21.15.42:6831/1736416326', 'heartbeat_front_addr': '172.21.15.42:6829/1736416326', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'a812e38f-c0f9-4d33-9373-1910fb09103c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.9703369140625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 49, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6808', 'nonce': 1755154427}, {'type': 'v1', 'addr': '172.21.15.43:6809', 'nonce': 1755154427}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6810', 'nonce': 1755154427}, {'type': 'v1', 'addr': '172.21.15.43:6811', 'nonce': 1755154427}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6814', 'nonce': 1755154427}, {'type': 'v1', 'addr': '172.21.15.43:6815', 'nonce': 1755154427}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6812', 'nonce': 1755154427}, {'type': 'v1', 'addr': '172.21.15.43:6813', 'nonce': 1755154427}]}, 'public_addr': '172.21.15.43:6809/1755154427', 'cluster_addr': '172.21.15.43:6811/1755154427', 'heartbeat_back_addr': '172.21.15.43:6815/1755154427', 'heartbeat_front_addr': '172.21.15.43:6813/1755154427', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '0de87e5b-7a21-4fb4-9543-d96c065786f8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6816', 'nonce': 3033769028}, {'type': 'v1', 'addr': '172.21.15.43:6817', 'nonce': 3033769028}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6818', 'nonce': 3033769028}, {'type': 'v1', 'addr': '172.21.15.43:6819', 'nonce': 3033769028}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6822', 'nonce': 3033769028}, {'type': 'v1', 'addr': '172.21.15.43:6823', 'nonce': 3033769028}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6820', 'nonce': 3033769028}, {'type': 'v1', 'addr': '172.21.15.43:6821', 'nonce': 3033769028}]}, 'public_addr': '172.21.15.43:6817/3033769028', 'cluster_addr': '172.21.15.43:6819/3033769028', 'heartbeat_back_addr': '172.21.15.43:6823/3033769028', 'heartbeat_front_addr': '172.21.15.43:6821/3033769028', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'c58c1d59-ad02-40f0-98e3-9078ef7e622b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 76, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6800', 'nonce': 3314282416}, {'type': 'v1', 'addr': '172.21.15.43:6801', 'nonce': 3314282416}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6802', 'nonce': 3314282416}, {'type': 'v1', 'addr': '172.21.15.43:6803', 'nonce': 3314282416}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6806', 'nonce': 3314282416}, {'type': 'v1', 'addr': '172.21.15.43:6807', 'nonce': 3314282416}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6804', 'nonce': 3314282416}, {'type': 'v1', 'addr': '172.21.15.43:6805', 'nonce': 3314282416}]}, 'public_addr': '172.21.15.43:6801/3314282416', 'cluster_addr': '172.21.15.43:6803/3314282416', 'heartbeat_back_addr': '172.21.15.43:6807/3314282416', 'heartbeat_front_addr': '172.21.15.43:6805/3314282416', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '389bdb04-20e3-4a7a-8504-687e034bd7dc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 75, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6824', 'nonce': 1481350118}, {'type': 'v1', 'addr': '172.21.15.43:6825', 'nonce': 1481350118}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6826', 'nonce': 1481350118}, {'type': 'v1', 'addr': '172.21.15.43:6827', 'nonce': 1481350118}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6830', 'nonce': 1481350118}, {'type': 'v1', 'addr': '172.21.15.43:6831', 'nonce': 1481350118}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6828', 'nonce': 1481350118}, {'type': 'v1', 'addr': '172.21.15.43:6829', 'nonce': 1481350118}]}, 'public_addr': '172.21.15.43:6825/1481350118', 'cluster_addr': '172.21.15.43:6827/1481350118', 'heartbeat_back_addr': '172.21.15.43:6831/1481350118', 'heartbeat_front_addr': '172.21.15.43:6829/1481350118', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T23:56:11.077810+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T23:56:11.578838+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T23:56:10.895490+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T23:56:12.290821+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T23:56:11.800581+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T23:56:11.883155+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T23:56:11.604790+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T23:56:12.067759+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.1e', 'osds': [6, 1]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T23:57:34.713 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T23:57:34.714 INFO:teuthology.orchestra.run.smithi042.stdout:3398 read 0x7846250 thru 0x78510ff (0xaeb0 bytes) 2024-09-14T23:57:34.714 INFO:teuthology.orchestra.run.smithi042.stdout:3399 write 0xa59d330 thru 0xa5a9844 (0xc515 bytes) 2024-09-14T23:57:34.714 INFO:teuthology.orchestra.run.smithi042.stdout:3402 read 0x2f593bb thru 0x2f5953a (0x180 bytes) 2024-09-14T23:57:34.715 INFO:teuthology.orchestra.run.smithi042.stdout:3403 read 0xb19b57d thru 0xb1a1119 (0x5b9d bytes) 2024-09-14T23:57:34.715 INFO:teuthology.orchestra.run.smithi042.stdout:3404 punch from 0xbebbe69 to 0xbec2278, (0x640f bytes) 2024-09-14T23:57:34.715 INFO:teuthology.orchestra.run.smithi042.stdout:3405 read 0x50da7ee thru 0x50e3a6c (0x927f bytes) 2024-09-14T23:57:34.715 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:34.715 INFO:teuthology.orchestra.run.smithi042.stdout:3407 punch from 0x8f99860 to 0x8f9cff8, (0x3798 bytes) 2024-09-14T23:57:34.715 INFO:teuthology.orchestra.run.smithi042.stdout:3409 write 0x96bb591 thru 0x96bf070 (0x3ae0 bytes) 2024-09-14T23:57:34.715 INFO:teuthology.orchestra.run.smithi042.stdout:3410 punch from 0x859591e to 0x8597c72, (0x2354 bytes) 2024-09-14T23:57:34.715 INFO:teuthology.orchestra.run.smithi042.stdout:3411 clone 32 order 25 su 16777216 sc 4 2024-09-14T23:57:34.716 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T23:57:34.757 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:34.756+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:34.836 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:34.835+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:34.864 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:34.864 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:34.864 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:57:34.864 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:57:34.864 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:34.872 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T23:57:34.937 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:34.936+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:35.024 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:35.024 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:57:35.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:57:35.024 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:57:35.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.024 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68736 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 5906432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:35.024 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.264178+0000", 2024-09-14T23:57:35.024 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.757581082000002, 2024-09-14T23:57:35.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.045913849999999999, 2024-09-14T23:57:35.025 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:35.025 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:35.025 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:35.025 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:35.025 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:35.025 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:35.025 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68736 2024-09-14T23:57:35.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.026 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:35.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:35.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264178+0000", 2024-09-14T23:57:35.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:35.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264178+0000", 2024-09-14T23:57:35.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.027 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.027 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:35.027 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264179+0000", 2024-09-14T23:57:35.027 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8599999999999999e-07 2024-09-14T23:57:35.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.027 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.027 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:35.027 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264181+0000", 2024-09-14T23:57:35.027 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.855e-06 2024-09-14T23:57:35.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.028 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:35.028 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264187+0000", 2024-09-14T23:57:35.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1539999999999999e-06 2024-09-14T23:57:35.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.028 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:35.028 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264192+0000", 2024-09-14T23:57:35.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6409999999999999e-06 2024-09-14T23:57:35.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:35.029 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.309931+0000", 2024-09-14T23:57:35.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.045738442999999997 2024-09-14T23:57:35.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:35.029 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310017+0000", 2024-09-14T23:57:35.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5829999999999996e-05 2024-09-14T23:57:35.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:35.029 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310092+0000", 2024-09-14T23:57:35.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5541000000000001e-05 2024-09-14T23:57:35.029 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.030 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:35.030 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.030 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68737 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 6021120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:35.030 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.264199+0000", 2024-09-14T23:57:35.030 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.757560616999999, 2024-09-14T23:57:35.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046063354000000001, 2024-09-14T23:57:35.030 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:35.030 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:35.030 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:35.030 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:35.030 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:35.030 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:35.031 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68737 2024-09-14T23:57:35.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.031 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:35.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.031 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:35.031 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264199+0000", 2024-09-14T23:57:35.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.031 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:35.031 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264199+0000", 2024-09-14T23:57:35.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:35.032 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264199+0000", 2024-09-14T23:57:35.032 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1600000000000002e-07 2024-09-14T23:57:35.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:35.032 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264210+0000", 2024-09-14T23:57:35.032 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0973e-05 2024-09-14T23:57:35.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:35.032 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264217+0000", 2024-09-14T23:57:35.032 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0659999999999998e-06 2024-09-14T23:57:35.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.033 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:35.033 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264223+0000", 2024-09-14T23:57:35.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2039999999999996e-06 2024-09-14T23:57:35.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.034 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:35.034 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310121+0000", 2024-09-14T23:57:35.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.045898436000000001 2024-09-14T23:57:35.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.034 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:35.034 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310194+0000", 2024-09-14T23:57:35.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2708999999999998e-05 2024-09-14T23:57:35.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.034 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:35.034 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310262+0000", 2024-09-14T23:57:35.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8549999999999999e-05 2024-09-14T23:57:35.035 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.035 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:35.035 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.035 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68738 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 6135808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:35.035 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.264234+0000", 2024-09-14T23:57:35.035 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.757525186999999, 2024-09-14T23:57:35.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046212954000000001, 2024-09-14T23:57:35.035 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:35.035 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:35.035 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:35.035 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:35.035 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:35.036 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:35.036 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68738 2024-09-14T23:57:35.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.036 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:35.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:35.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264234+0000", 2024-09-14T23:57:35.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:35.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264234+0000", 2024-09-14T23:57:35.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.037 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.037 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:35.037 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264235+0000", 2024-09-14T23:57:35.037 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6600000000000002e-07 2024-09-14T23:57:35.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.037 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.037 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:35.037 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264237+0000", 2024-09-14T23:57:35.037 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3120000000000001e-06 2024-09-14T23:57:35.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.037 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.037 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:35.037 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264244+0000", 2024-09-14T23:57:35.038 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6680000000000004e-06 2024-09-14T23:57:35.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.038 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.038 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:35.038 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264255+0000", 2024-09-14T23:57:35.038 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.155e-05 2024-09-14T23:57:35.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.038 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.038 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:35.038 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310292+0000", 2024-09-14T23:57:35.038 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046037026000000002 2024-09-14T23:57:35.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.038 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.038 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:35.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310375+0000", 2024-09-14T23:57:35.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3039000000000002e-05 2024-09-14T23:57:35.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:35.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310447+0000", 2024-09-14T23:57:35.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1892999999999998e-05 2024-09-14T23:57:35.039 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.039 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:35.039 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.039 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68739 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 6250496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:35.040 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.264263+0000", 2024-09-14T23:57:35.040 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.757496629999999, 2024-09-14T23:57:35.040 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046362974000000001, 2024-09-14T23:57:35.040 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:35.040 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:35.040 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:35.040 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:35.040 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:35.040 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:35.040 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68739 2024-09-14T23:57:35.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.040 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:35.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.041 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:35.041 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264263+0000", 2024-09-14T23:57:35.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.041 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.041 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:35.041 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264263+0000", 2024-09-14T23:57:35.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.041 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.041 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:35.041 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264263+0000", 2024-09-14T23:57:35.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8599999999999999e-07 2024-09-14T23:57:35.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.042 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.042 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:35.042 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264265+0000", 2024-09-14T23:57:35.042 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0020000000000001e-06 2024-09-14T23:57:35.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.042 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.042 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:35.042 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264272+0000", 2024-09-14T23:57:35.042 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4799999999999998e-06 2024-09-14T23:57:35.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.042 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.042 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:35.042 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264282+0000", 2024-09-14T23:57:35.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0244999999999999e-05 2024-09-14T23:57:35.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.043 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:35.043 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310478+0000", 2024-09-14T23:57:35.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046196070999999998 2024-09-14T23:57:35.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.043 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:35.043 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310554+0000", 2024-09-14T23:57:35.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6160000000000003e-05 2024-09-14T23:57:35.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.044 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:35.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310626+0000", 2024-09-14T23:57:35.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1630000000000004e-05 2024-09-14T23:57:35.044 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.044 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:35.044 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.044 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68740 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 6365184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:35.044 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.264289+0000", 2024-09-14T23:57:35.044 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.757470174000002, 2024-09-14T23:57:35.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046531222999999997, 2024-09-14T23:57:35.045 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:35.045 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:35.045 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:35.045 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:35.045 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:35.045 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:35.045 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68740 2024-09-14T23:57:35.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.045 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:35.045 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.045 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:35.045 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264289+0000", 2024-09-14T23:57:35.045 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.046 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.046 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:35.046 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264289+0000", 2024-09-14T23:57:35.046 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.046 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.046 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:35.046 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264290+0000", 2024-09-14T23:57:35.046 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4299999999999998e-07 2024-09-14T23:57:35.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.046 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.046 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:35.046 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264292+0000", 2024-09-14T23:57:35.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.181e-06 2024-09-14T23:57:35.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.047 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:35.047 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264298+0000", 2024-09-14T23:57:35.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1120000000000001e-06 2024-09-14T23:57:35.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.047 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:35.047 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264303+0000", 2024-09-14T23:57:35.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5839999999999996e-06 2024-09-14T23:57:35.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.047 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:35.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310654+0000", 2024-09-14T23:57:35.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046351046 2024-09-14T23:57:35.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.048 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.048 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:35.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310745+0000", 2024-09-14T23:57:35.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1027999999999997e-05 2024-09-14T23:57:35.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.048 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.048 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:35.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310821+0000", 2024-09-14T23:57:35.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5828999999999995e-05 2024-09-14T23:57:35.048 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.049 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:35.049 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.049 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.049 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68741 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 6479872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:35.049 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.264332+0000", 2024-09-14T23:57:35.049 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.757427203999999, 2024-09-14T23:57:35.049 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046692392999999999, 2024-09-14T23:57:35.049 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:35.049 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:35.049 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:35.049 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:35.049 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:35.050 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:35.050 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68741 2024-09-14T23:57:35.050 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.050 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:35.050 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.050 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:35.050 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264332+0000", 2024-09-14T23:57:35.050 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.050 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.050 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.050 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:35.050 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264332+0000", 2024-09-14T23:57:35.050 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.050 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.051 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.051 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:35.051 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264333+0000", 2024-09-14T23:57:35.051 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2899999999999999e-07 2024-09-14T23:57:35.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.051 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.051 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:35.051 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264335+0000", 2024-09-14T23:57:35.051 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9870000000000002e-06 2024-09-14T23:57:35.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.051 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.051 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:35.051 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264340+0000", 2024-09-14T23:57:35.051 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0069999999999997e-06 2024-09-14T23:57:35.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.051 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.052 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:35.052 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264343+0000", 2024-09-14T23:57:35.052 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0960000000000001e-06 2024-09-14T23:57:35.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.052 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.052 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:35.052 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310855+0000", 2024-09-14T23:57:35.052 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046511915000000001 2024-09-14T23:57:35.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.052 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.052 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:35.052 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310946+0000", 2024-09-14T23:57:35.052 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1290000000000002e-05 2024-09-14T23:57:35.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.052 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.053 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:35.053 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311025+0000", 2024-09-14T23:57:35.053 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8769000000000004e-05 2024-09-14T23:57:35.053 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.053 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:35.053 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.053 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68742 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 6594560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:35.053 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.264347+0000", 2024-09-14T23:57:35.053 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.757412162000001, 2024-09-14T23:57:35.053 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046904279, 2024-09-14T23:57:35.053 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:35.053 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:35.053 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:35.054 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:35.054 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:35.054 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:35.054 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68742 2024-09-14T23:57:35.054 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.054 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:35.054 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.054 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:35.054 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264347+0000", 2024-09-14T23:57:35.054 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.054 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.054 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.054 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:35.054 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264347+0000", 2024-09-14T23:57:35.055 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.055 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.055 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.055 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:35.055 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264348+0000", 2024-09-14T23:57:35.055 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.22e-07 2024-09-14T23:57:35.055 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.055 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.055 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:35.055 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264349+0000", 2024-09-14T23:57:35.055 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2079999999999999e-06 2024-09-14T23:57:35.055 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.055 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.055 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:35.056 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264352+0000", 2024-09-14T23:57:35.056 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1640000000000001e-06 2024-09-14T23:57:35.056 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.056 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.056 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:35.056 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264354+0000", 2024-09-14T23:57:35.056 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3350000000000001e-06 2024-09-14T23:57:35.056 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.056 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.056 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:35.056 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311071+0000", 2024-09-14T23:57:35.056 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046716921000000002 2024-09-14T23:57:35.056 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.056 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.056 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:35.057 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311165+0000", 2024-09-14T23:57:35.057 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3900000000000006e-05 2024-09-14T23:57:35.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.057 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.057 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:35.057 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311252+0000", 2024-09-14T23:57:35.057 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6428999999999995e-05 2024-09-14T23:57:35.057 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.057 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:35.057 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.057 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.057 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68743 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 6709248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:35.057 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.264359+0000", 2024-09-14T23:57:35.057 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.757400532999998, 2024-09-14T23:57:35.058 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047086817000000003, 2024-09-14T23:57:35.058 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:35.058 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:35.058 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:35.058 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:35.058 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:35.058 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:35.058 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68743 2024-09-14T23:57:35.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.058 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:35.058 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.058 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:35.058 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264359+0000", 2024-09-14T23:57:35.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.059 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:35.059 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264359+0000", 2024-09-14T23:57:35.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.059 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:35.059 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265672+0000", 2024-09-14T23:57:35.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0013131849999999999 2024-09-14T23:57:35.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.059 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:35.059 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265676+0000", 2024-09-14T23:57:35.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2030000000000002e-06 2024-09-14T23:57:35.060 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.060 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.060 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:35.060 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265685+0000", 2024-09-14T23:57:35.060 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7290000000000006e-06 2024-09-14T23:57:35.060 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.060 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.060 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:35.060 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265691+0000", 2024-09-14T23:57:35.060 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3779999999999998e-06 2024-09-14T23:57:35.060 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.060 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.060 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:35.060 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311283+0000", 2024-09-14T23:57:35.060 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.045591797000000003 2024-09-14T23:57:35.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.061 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.061 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:35.061 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311368+0000", 2024-09-14T23:57:35.061 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4444999999999995e-05 2024-09-14T23:57:35.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.061 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.061 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:35.061 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311446+0000", 2024-09-14T23:57:35.061 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8079999999999998e-05 2024-09-14T23:57:35.061 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.061 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:35.061 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.062 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.062 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68744 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 6823936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:35.062 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.265703+0000", 2024-09-14T23:57:35.062 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.756056758, 2024-09-14T23:57:35.062 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.045950957000000001, 2024-09-14T23:57:35.062 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:35.062 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:35.062 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:35.062 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:35.062 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:35.062 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:35.062 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68744 2024-09-14T23:57:35.062 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.062 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:35.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:35.063 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265703+0000", 2024-09-14T23:57:35.063 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:35.063 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265703+0000", 2024-09-14T23:57:35.063 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:35.063 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265703+0000", 2024-09-14T23:57:35.063 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9900000000000001e-07 2024-09-14T23:57:35.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:35.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265706+0000", 2024-09-14T23:57:35.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4329999999999998e-06 2024-09-14T23:57:35.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:35.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265712+0000", 2024-09-14T23:57:35.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8020000000000003e-06 2024-09-14T23:57:35.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:35.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265718+0000", 2024-09-14T23:57:35.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.13e-06 2024-09-14T23:57:35.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.065 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.065 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:35.065 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311494+0000", 2024-09-14T23:57:35.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.045776039999999997 2024-09-14T23:57:35.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.065 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.065 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:35.065 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311577+0000", 2024-09-14T23:57:35.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3057000000000001e-05 2024-09-14T23:57:35.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.065 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.065 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:35.065 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311654+0000", 2024-09-14T23:57:35.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6996000000000006e-05 2024-09-14T23:57:35.066 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.066 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:35.066 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.066 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.066 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.066 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68745 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 6938624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:35.066 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.265725+0000", 2024-09-14T23:57:35.066 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.756034927999998, 2024-09-14T23:57:35.066 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046101672000000003, 2024-09-14T23:57:35.066 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:35.066 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:35.066 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:35.066 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:35.066 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:35.066 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:35.067 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68745 2024-09-14T23:57:35.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.067 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:35.067 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.067 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:35.067 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265725+0000", 2024-09-14T23:57:35.067 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.067 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.067 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:35.067 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265725+0000", 2024-09-14T23:57:35.067 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.067 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.067 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:35.067 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265725+0000", 2024-09-14T23:57:35.068 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5299999999999999e-07 2024-09-14T23:57:35.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.068 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.068 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:35.068 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265727+0000", 2024-09-14T23:57:35.068 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.069e-06 2024-09-14T23:57:35.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.068 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.068 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:35.068 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265733+0000", 2024-09-14T23:57:35.068 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1510000000000001e-06 2024-09-14T23:57:35.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.068 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.068 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:35.069 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265738+0000", 2024-09-14T23:57:35.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7010000000000004e-06 2024-09-14T23:57:35.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.069 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.069 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:35.069 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311707+0000", 2024-09-14T23:57:35.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.045968602999999997 2024-09-14T23:57:35.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.069 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.069 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:35.069 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311776+0000", 2024-09-14T23:57:35.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9438999999999997e-05 2024-09-14T23:57:35.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.070 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:35.070 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311826+0000", 2024-09-14T23:57:35.070 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0256000000000001e-05 2024-09-14T23:57:35.070 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.070 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:35.070 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.070 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68746 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 7053312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:35.070 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.265744+0000", 2024-09-14T23:57:35.070 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.756015297000001, 2024-09-14T23:57:35.070 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046203505999999998, 2024-09-14T23:57:35.071 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:35.071 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:35.071 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:35.071 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:35.071 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:35.071 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:35.071 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68746 2024-09-14T23:57:35.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.071 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:35.071 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.071 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:35.071 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265744+0000", 2024-09-14T23:57:35.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.071 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.072 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:35.072 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265744+0000", 2024-09-14T23:57:35.072 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.072 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.072 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:35.072 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265745+0000", 2024-09-14T23:57:35.072 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9200000000000002e-07 2024-09-14T23:57:35.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.072 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.072 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:35.072 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265746+0000", 2024-09-14T23:57:35.072 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8819999999999999e-06 2024-09-14T23:57:35.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.073 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.073 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:35.073 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265753+0000", 2024-09-14T23:57:35.073 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1990000000000003e-06 2024-09-14T23:57:35.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.073 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.073 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:35.073 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265760+0000", 2024-09-14T23:57:35.073 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9280000000000004e-06 2024-09-14T23:57:35.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.073 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.073 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:35.073 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311859+0000", 2024-09-14T23:57:35.074 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046099627999999997 2024-09-14T23:57:35.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.074 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.074 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:35.074 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311902+0000", 2024-09-14T23:57:35.074 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3239999999999999e-05 2024-09-14T23:57:35.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.074 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.074 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:35.074 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311948+0000", 2024-09-14T23:57:35.074 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5237000000000001e-05 2024-09-14T23:57:35.074 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.074 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:35.074 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.075 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.075 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68747 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 7168000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:35.075 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.265766+0000", 2024-09-14T23:57:35.075 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.755993136000001, 2024-09-14T23:57:35.075 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046303703000000002, 2024-09-14T23:57:35.075 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:35.075 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:35.075 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:35.075 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:35.075 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:35.075 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:35.075 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68747 2024-09-14T23:57:35.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.076 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:35.076 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.076 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:35.076 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265766+0000", 2024-09-14T23:57:35.076 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.076 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.076 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:35.076 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265766+0000", 2024-09-14T23:57:35.076 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.076 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.076 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:35.076 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265767+0000", 2024-09-14T23:57:35.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9200000000000002e-07 2024-09-14T23:57:35.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.077 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:35.077 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265769+0000", 2024-09-14T23:57:35.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9369999999999998e-06 2024-09-14T23:57:35.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.077 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:35.077 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265775+0000", 2024-09-14T23:57:35.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9499999999999998e-06 2024-09-14T23:57:35.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.078 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.078 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:35.078 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265779+0000", 2024-09-14T23:57:35.078 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7979999999999996e-06 2024-09-14T23:57:35.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.078 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.078 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:35.078 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311977+0000", 2024-09-14T23:57:35.078 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046197437000000001 2024-09-14T23:57:35.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.078 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.078 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:35.078 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.312024+0000", 2024-09-14T23:57:35.078 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6780999999999998e-05 2024-09-14T23:57:35.079 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.079 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.079 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:35.079 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.312070+0000", 2024-09-14T23:57:35.079 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6408e-05 2024-09-14T23:57:35.079 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.079 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:35.079 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.079 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.079 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.079 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68748 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 7282688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:35.079 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.265785+0000", 2024-09-14T23:57:35.079 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.755974075000001, 2024-09-14T23:57:35.079 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046405109999999999, 2024-09-14T23:57:35.080 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:35.080 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:35.080 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:35.080 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:35.080 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:35.080 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:35.080 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68748 2024-09-14T23:57:35.080 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.080 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:35.080 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.080 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:35.080 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265785+0000", 2024-09-14T23:57:35.080 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.081 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.081 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:35.081 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265785+0000", 2024-09-14T23:57:35.081 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.081 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.081 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:35.081 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265786+0000", 2024-09-14T23:57:35.081 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0900000000000002e-07 2024-09-14T23:57:35.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.081 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.081 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:35.082 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265796+0000", 2024-09-14T23:57:35.082 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8490000000000001e-06 2024-09-14T23:57:35.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.082 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.082 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:35.082 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265802+0000", 2024-09-14T23:57:35.082 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7309999999999996e-06 2024-09-14T23:57:35.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.082 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.082 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:35.082 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265812+0000", 2024-09-14T23:57:35.082 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8789999999999998e-06 2024-09-14T23:57:35.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.083 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.083 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:35.083 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.312098+0000", 2024-09-14T23:57:35.083 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046285862999999997 2024-09-14T23:57:35.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.083 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.083 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:35.083 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.312140+0000", 2024-09-14T23:57:35.083 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1718000000000003e-05 2024-09-14T23:57:35.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.083 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.083 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:35.083 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.312190+0000", 2024-09-14T23:57:35.083 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0661000000000002e-05 2024-09-14T23:57:35.084 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.084 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:35.084 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.084 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.084 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.084 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68749 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 7397376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:35.084 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.265821+0000", 2024-09-14T23:57:35.084 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.755938578999999, 2024-09-14T23:57:35.084 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046484708, 2024-09-14T23:57:35.084 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:35.084 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:35.085 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:35.085 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:35.085 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:35.085 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:35.085 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68749 2024-09-14T23:57:35.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.085 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:35.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.085 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:35.085 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265821+0000", 2024-09-14T23:57:35.085 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.085 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:35.086 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265821+0000", 2024-09-14T23:57:35.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.086 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.086 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:35.086 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265821+0000", 2024-09-14T23:57:35.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7300000000000001e-07 2024-09-14T23:57:35.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.086 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.086 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:35.086 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265829+0000", 2024-09-14T23:57:35.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8299999999999996e-06 2024-09-14T23:57:35.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.086 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.087 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:35.087 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265836+0000", 2024-09-14T23:57:35.087 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0709999999999997e-06 2024-09-14T23:57:35.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.087 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:35.087 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265841+0000", 2024-09-14T23:57:35.087 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1950000000000002e-06 2024-09-14T23:57:35.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.087 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:35.087 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.312218+0000", 2024-09-14T23:57:35.087 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046376873999999998 2024-09-14T23:57:35.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.088 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.088 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:35.088 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.312260+0000", 2024-09-14T23:57:35.088 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1693000000000002e-05 2024-09-14T23:57:35.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.088 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.088 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:35.088 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.312306+0000", 2024-09-14T23:57:35.088 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5571999999999997e-05 2024-09-14T23:57:35.088 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.088 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:35.088 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.088 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.089 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68750 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 7512064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:35.089 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.265863+0000", 2024-09-14T23:57:35.089 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.755896799999999, 2024-09-14T23:57:35.089 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046634766000000001, 2024-09-14T23:57:35.089 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:35.089 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:35.089 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:35.089 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:35.089 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:35.089 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:35.089 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68750 2024-09-14T23:57:35.089 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.089 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:35.089 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.090 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:35.090 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265863+0000", 2024-09-14T23:57:35.090 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.090 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.090 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:35.090 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265863+0000", 2024-09-14T23:57:35.090 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.090 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.090 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:35.090 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265863+0000", 2024-09-14T23:57:35.090 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0900000000000002e-07 2024-09-14T23:57:35.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:35.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265865+0000", 2024-09-14T23:57:35.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9740000000000001e-06 2024-09-14T23:57:35.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:35.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265869+0000", 2024-09-14T23:57:35.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4159999999999997e-06 2024-09-14T23:57:35.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:35.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265873+0000", 2024-09-14T23:57:35.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1999999999999999e-06 2024-09-14T23:57:35.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.092 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.092 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:35.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.312334+0000", 2024-09-14T23:57:35.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046461737000000003 2024-09-14T23:57:35.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.092 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.092 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:35.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.312415+0000", 2024-09-14T23:57:35.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0370999999999993e-05 2024-09-14T23:57:35.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.092 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.093 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:35.093 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.312497+0000", 2024-09-14T23:57:35.093 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2658999999999996e-05 2024-09-14T23:57:35.093 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.093 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:35.093 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.093 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.093 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.093 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68751 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 7626752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:35.093 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.265877+0000", 2024-09-14T23:57:35.093 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.755882399000001, 2024-09-14T23:57:35.093 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046759231999999998, 2024-09-14T23:57:35.093 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:35.094 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:35.094 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:35.094 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:35.094 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:35.094 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:35.094 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68751 2024-09-14T23:57:35.094 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.094 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:35.094 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.094 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:35.094 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265877+0000", 2024-09-14T23:57:35.094 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.094 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.094 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.094 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:35.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265877+0000", 2024-09-14T23:57:35.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:35.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.267673+0000", 2024-09-14T23:57:35.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017956389999999999 2024-09-14T23:57:35.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:35.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.267677+0000", 2024-09-14T23:57:35.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4900000000000002e-06 2024-09-14T23:57:35.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:35.096 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.267688+0000", 2024-09-14T23:57:35.096 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1121000000000001e-05 2024-09-14T23:57:35.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.096 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:35.096 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.267694+0000", 2024-09-14T23:57:35.096 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5520000000000004e-06 2024-09-14T23:57:35.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.096 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:35.096 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.312534+0000", 2024-09-14T23:57:35.096 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044839641 2024-09-14T23:57:35.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.097 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.097 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:35.097 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.312586+0000", 2024-09-14T23:57:35.097 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2629e-05 2024-09-14T23:57:35.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.097 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.097 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:35.097 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.312636+0000", 2024-09-14T23:57:35.097 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0160000000000001e-05 2024-09-14T23:57:35.097 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.097 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:35.097 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.097 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.098 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68886 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 8060928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:35.098 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.273785+0000", 2024-09-14T23:57:35.098 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.747973996999999, 2024-09-14T23:57:35.098 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.045962955, 2024-09-14T23:57:35.098 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:35.098 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:35.098 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:35.098 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:35.098 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:35.098 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:35.098 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68886 2024-09-14T23:57:35.098 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.098 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:35.098 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.099 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:35.099 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273785+0000", 2024-09-14T23:57:35.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.099 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:35.099 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273785+0000", 2024-09-14T23:57:35.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.099 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:35.099 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273786+0000", 2024-09-14T23:57:35.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9000000000000002e-07 2024-09-14T23:57:35.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.100 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.100 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:35.100 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273788+0000", 2024-09-14T23:57:35.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9139999999999998e-06 2024-09-14T23:57:35.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.100 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.100 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:35.100 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273794+0000", 2024-09-14T23:57:35.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.212e-06 2024-09-14T23:57:35.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.100 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.100 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:35.100 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273799+0000", 2024-09-14T23:57:35.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5580000000000002e-06 2024-09-14T23:57:35.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.101 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:35.101 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.319646+0000", 2024-09-14T23:57:35.101 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.045847951999999997 2024-09-14T23:57:35.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.101 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:35.101 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.319707+0000", 2024-09-14T23:57:35.101 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.029e-05 2024-09-14T23:57:35.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.101 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:35.101 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.319748+0000", 2024-09-14T23:57:35.102 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1638999999999999e-05 2024-09-14T23:57:35.102 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.102 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:35.102 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.102 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.102 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68887 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 8175616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:35.102 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.273805+0000", 2024-09-14T23:57:35.102 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.747954738000001, 2024-09-14T23:57:35.102 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046062421999999999, 2024-09-14T23:57:35.102 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:35.102 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:35.102 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:35.102 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:35.103 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:35.103 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:35.103 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68887 2024-09-14T23:57:35.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.103 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:35.103 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.103 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:35.103 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273805+0000", 2024-09-14T23:57:35.103 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.103 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.103 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:35.103 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273805+0000", 2024-09-14T23:57:35.103 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.104 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.104 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.104 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:35.104 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273805+0000", 2024-09-14T23:57:35.104 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0699999999999998e-07 2024-09-14T23:57:35.104 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.104 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.104 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:35.104 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273815+0000", 2024-09-14T23:57:35.104 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7720000000000006e-06 2024-09-14T23:57:35.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.105 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:35.105 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273822+0000", 2024-09-14T23:57:35.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5850000000000002e-06 2024-09-14T23:57:35.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.105 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:35.105 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273833+0000", 2024-09-14T23:57:35.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0541e-05 2024-09-14T23:57:35.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.105 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:35.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.319779+0000", 2024-09-14T23:57:35.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.045946263000000001 2024-09-14T23:57:35.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.106 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:35.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.319824+0000", 2024-09-14T23:57:35.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4740000000000002e-05 2024-09-14T23:57:35.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.106 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:35.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.319867+0000", 2024-09-14T23:57:35.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3114000000000001e-05 2024-09-14T23:57:35.106 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.106 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:35.107 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.107 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68888 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 8290304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:35.107 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.273854+0000", 2024-09-14T23:57:35.107 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.747905597999999, 2024-09-14T23:57:35.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046120520999999998, 2024-09-14T23:57:35.107 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:35.107 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:35.107 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:35.107 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:35.107 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:35.107 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:35.108 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68888 2024-09-14T23:57:35.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.108 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:35.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:35.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273854+0000", 2024-09-14T23:57:35.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:35.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273854+0000", 2024-09-14T23:57:35.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:35.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273854+0000", 2024-09-14T23:57:35.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.58e-07 2024-09-14T23:57:35.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:35.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273862+0000", 2024-09-14T23:57:35.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.52e-06 2024-09-14T23:57:35.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:35.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273869+0000", 2024-09-14T23:57:35.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1180000000000003e-06 2024-09-14T23:57:35.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.110 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.110 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:35.110 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273883+0000", 2024-09-14T23:57:35.110 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3709999999999999e-05 2024-09-14T23:57:35.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.110 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.110 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:35.110 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.319889+0000", 2024-09-14T23:57:35.110 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046005870999999997 2024-09-14T23:57:35.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.110 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.110 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:35.110 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.319930+0000", 2024-09-14T23:57:35.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1850000000000001e-05 2024-09-14T23:57:35.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.111 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.111 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:35.111 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.319974+0000", 2024-09-14T23:57:35.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3993999999999999e-05 2024-09-14T23:57:35.111 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.111 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:35.111 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.111 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.111 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:107472 3.1e 3:7bf21588:::rbd_data.11187de8b0a5.000000000000005d:head [set-alloc-hint object_size 2097152 write_size 2097152,write 317788~1491] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-14T23:57:35.111 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:25.198493+0000", 2024-09-14T23:57:35.112 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.8232664960000005, 2024-09-14T23:57:35.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.78175149300000002, 2024-09-14T23:57:35.112 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:35.112 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:35.112 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:35.112 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:35.112 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:35.112 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:35.112 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 107472 2024-09-14T23:57:35.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.112 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:35.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:35.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198493+0000", 2024-09-14T23:57:35.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.113 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:35.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198493+0000", 2024-09-14T23:57:35.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.113 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:35.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198495+0000", 2024-09-14T23:57:35.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.826e-06 2024-09-14T23:57:35.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.114 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:35.114 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198503+0000", 2024-09-14T23:57:35.114 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0150000000000003e-06 2024-09-14T23:57:35.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.114 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:35.114 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198521+0000", 2024-09-14T23:57:35.114 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8374000000000001e-05 2024-09-14T23:57:35.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.114 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:35.114 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198533+0000", 2024-09-14T23:57:35.114 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1593000000000001e-05 2024-09-14T23:57:35.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.115 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:35.115 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198603+0000", 2024-09-14T23:57:35.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0279999999999998e-05 2024-09-14T23:57:35.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.115 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:57:35.115 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198617+0000", 2024-09-14T23:57:35.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3517e-05 2024-09-14T23:57:35.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.115 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:35.115 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.977067+0000", 2024-09-14T23:57:35.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.77845065800000002 2024-09-14T23:57:35.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.116 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for missing object", 2024-09-14T23:57:35.116 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.977173+0000", 2024-09-14T23:57:35.116 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010543999999999999 2024-09-14T23:57:35.116 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.116 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.116 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:35.116 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.978737+0000", 2024-09-14T23:57:35.116 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001564078 2024-09-14T23:57:35.116 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.116 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.116 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:35.116 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.978845+0000", 2024-09-14T23:57:35.116 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000108643 2024-09-14T23:57:35.116 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1", 2024-09-14T23:57:35.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.978964+0000", 2024-09-14T23:57:35.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000119085 2024-09-14T23:57:35.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:35.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980109+0000", 2024-09-14T23:57:35.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0011442170000000001 2024-09-14T23:57:35.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:57:35.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980130+0000", 2024-09-14T23:57:35.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1766000000000001e-05 2024-09-14T23:57:35.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.118 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.118 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:35.118 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980171+0000", 2024-09-14T23:57:35.118 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0627999999999998e-05 2024-09-14T23:57:35.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.118 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.118 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:35.118 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980244+0000", 2024-09-14T23:57:35.118 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3373000000000004e-05 2024-09-14T23:57:35.118 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.118 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:35.118 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.118 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.119 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:35.119 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:35.121 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:57:35.121 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-14T23:57:35.122 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T23:57:35.123 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":118,"stamp":"2024-09-14T23:57:35.004527+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":716714006,"num_objects":1947,"num_object_clones":362,"num_object_copies":3894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1947,"num_whiteouts":995,"num_read":131402,"num_read_kb":549217,"num_write":12114,"num_write_kb":286232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1171,"num_bytes_recovered":61248546,"num_keys_recovered":133,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10277,"ondisk_log_size":10277,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":324992,"kb_used_data":53280,"kb_used_omap":12,"kb_used_meta":271539,"kb_avail":754649728,"statfs":{"total":773094113280,"available":772761321472,"internally_reserved":0,"allocated":54558720,"data_stored":441846744,"data_compressed":33228805,"data_compressed_allocated":36159488,"data_compressed_original":429318226,"omap_allocated":12752,"internal_metadata":278056496},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":38781147,"num_objects":35,"num_object_clones":10,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":16,"num_read":2318,"num_read_kb":10764,"num_write":394,"num_write_kb":7280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.017782"},"pg_stats":[{"pgid":"3.1f","version":"81'694","reported_seq":2578,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T23:57:33.145761+0000","last_change":"2024-09-14T23:57:00.062727+0000","last_active":"2024-09-14T23:57:33.145761+0000","last_peered":"2024-09-14T23:57:33.145761+0000","last_clean":"2024-09-14T23:57:33.145761+0000","last_became_active":"2024-09-14T23:57:00.062116+0000","last_became_peered":"2024-09-14T23:57:00.062116+0000","last_unstale":"2024-09-14T23:57:33.145761+0000","last_undegraded":"2024-09-14T23:57:33.145761+0000","last_fullsized":"2024-09-14T23:57:33.145761+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":694,"log_dups_size":0,"ondisk_log_size":694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:10:59.654769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015692659999999999,"stat_sum":{"num_bytes":23260872,"num_objects":74,"num_object_clones":14,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":36,"num_read":5123,"num_read_kb":14512,"num_write":964,"num_write_kb":20837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":2327658,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"77'192","reported_seq":1630,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-14T23:57:30.034411+0000","last_change":"2024-09-14T23:57:27.976532+0000","last_active":"2024-09-14T23:57:30.034411+0000","last_peered":"2024-09-14T23:57:30.034411+0000","last_clean":"2024-09-14T23:57:30.034411+0000","last_became_active":"2024-09-14T23:57:27.974745+0000","last_became_peered":"2024-09-14T23:57:27.974745+0000","last_unstale":"2024-09-14T23:57:30.034411+0000","last_undegraded":"2024-09-14T23:57:30.034411+0000","last_fullsized":"2024-09-14T23:57:30.034411+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":192,"log_dups_size":0,"ondisk_log_size":192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:58:52.427483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010144430000000001,"stat_sum":{"num_bytes":13278889,"num_objects":64,"num_object_clones":15,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":31,"num_read":1251,"num_read_kb":9567,"num_write":179,"num_write_kb":8677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":4573164,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"78'284","reported_seq":2792,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-14T23:57:29.986752+0000","last_change":"2024-09-14T23:57:00.062960+0000","last_active":"2024-09-14T23:57:29.986752+0000","last_peered":"2024-09-14T23:57:29.986752+0000","last_clean":"2024-09-14T23:57:29.986752+0000","last_became_active":"2024-09-14T23:57:00.062242+0000","last_became_peered":"2024-09-14T23:57:00.062242+0000","last_unstale":"2024-09-14T23:57:29.986752+0000","last_undegraded":"2024-09-14T23:57:29.986752+0000","last_fullsized":"2024-09-14T23:57:29.986752+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:40:53.918326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001148175,"stat_sum":{"num_bytes":30718384,"num_objects":68,"num_object_clones":15,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":34,"num_read":3233,"num_read_kb":15860,"num_write":259,"num_write_kb":11285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":1249963,"num_keys_recovered":34,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"79'325","reported_seq":3112,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-14T23:57:30.021052+0000","last_change":"2024-09-14T23:57:00.062694+0000","last_active":"2024-09-14T23:57:30.021052+0000","last_peered":"2024-09-14T23:57:30.021052+0000","last_clean":"2024-09-14T23:57:30.021052+0000","last_became_active":"2024-09-14T23:57:00.062126+0000","last_became_peered":"2024-09-14T23:57:00.062126+0000","last_unstale":"2024-09-14T23:57:30.021052+0000","last_undegraded":"2024-09-14T23:57:30.021052+0000","last_fullsized":"2024-09-14T23:57:30.021052+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:24:01.389497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095260199999999998,"stat_sum":{"num_bytes":16315269,"num_objects":54,"num_object_clones":7,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":30,"num_read":3562,"num_read_kb":10990,"num_write":382,"num_write_kb":5958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":2675491,"num_keys_recovered":40,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"82'304","reported_seq":808,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T23:57:33.098113+0000","last_change":"2024-09-14T23:56:34.401802+0000","last_active":"2024-09-14T23:57:33.098113+0000","last_peered":"2024-09-14T23:57:33.098113+0000","last_clean":"2024-09-14T23:57:33.098113+0000","last_became_active":"2024-09-14T23:56:32.398329+0000","last_became_peered":"2024-09-14T23:56:32.398329+0000","last_unstale":"2024-09-14T23:57:33.098113+0000","last_undegraded":"2024-09-14T23:57:33.098113+0000","last_fullsized":"2024-09-14T23:57:33.098113+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":304,"log_dups_size":0,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:14:50.429434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018313229999999999,"stat_sum":{"num_bytes":10931555,"num_objects":42,"num_object_clones":6,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":23,"num_read":1079,"num_read_kb":7902,"num_write":378,"num_write_kb":4934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"74'245","reported_seq":2315,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-14T23:57:30.097472+0000","last_change":"2024-09-14T23:56:34.405093+0000","last_active":"2024-09-14T23:57:30.097472+0000","last_peered":"2024-09-14T23:57:30.097472+0000","last_clean":"2024-09-14T23:57:30.097472+0000","last_became_active":"2024-09-14T23:56:32.399655+0000","last_became_peered":"2024-09-14T23:56:32.399655+0000","last_unstale":"2024-09-14T23:57:30.097472+0000","last_undegraded":"2024-09-14T23:57:30.097472+0000","last_fullsized":"2024-09-14T23:57:30.097472+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":245,"log_dups_size":0,"ondisk_log_size":245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:58:29.734250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096230199999999995,"stat_sum":{"num_bytes":15076848,"num_objects":71,"num_object_clones":15,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":43,"num_read":2052,"num_read_kb":10722,"num_write":231,"num_write_kb":13952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"81'270","reported_seq":1187,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T23:57:32.996538+0000","last_change":"2024-09-14T23:56:34.400617+0000","last_active":"2024-09-14T23:57:32.996538+0000","last_peered":"2024-09-14T23:57:32.996538+0000","last_clean":"2024-09-14T23:57:32.996538+0000","last_became_active":"2024-09-14T23:56:32.404364+0000","last_became_peered":"2024-09-14T23:56:32.404364+0000","last_unstale":"2024-09-14T23:57:32.996538+0000","last_undegraded":"2024-09-14T23:57:32.996538+0000","last_fullsized":"2024-09-14T23:57:32.996538+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":270,"log_dups_size":0,"ondisk_log_size":270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:58:30.262256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001003264,"stat_sum":{"num_bytes":12145008,"num_objects":67,"num_object_clones":9,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":39,"num_read":1254,"num_read_kb":6623,"num_write":294,"num_write_kb":4231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"77'251","reported_seq":3013,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-14T23:57:29.986660+0000","last_change":"2024-09-14T23:56:34.415021+0000","last_active":"2024-09-14T23:57:29.986660+0000","last_peered":"2024-09-14T23:57:29.986660+0000","last_clean":"2024-09-14T23:57:29.986660+0000","last_became_active":"2024-09-14T23:56:32.401799+0000","last_became_peered":"2024-09-14T23:56:32.401799+0000","last_unstale":"2024-09-14T23:57:29.986660+0000","last_undegraded":"2024-09-14T23:57:29.986660+0000","last_fullsized":"2024-09-14T23:57:29.986660+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":251,"log_dups_size":0,"ondisk_log_size":251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:47:58.776776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062985000000000005,"stat_sum":{"num_bytes":16912111,"num_objects":71,"num_object_clones":14,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":39,"num_read":3487,"num_read_kb":10972,"num_write":230,"num_write_kb":1980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"80'211","reported_seq":1260,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T23:57:32.997392+0000","last_change":"2024-09-14T23:57:00.062824+0000","last_active":"2024-09-14T23:57:32.997392+0000","last_peered":"2024-09-14T23:57:32.997392+0000","last_clean":"2024-09-14T23:57:32.997392+0000","last_became_active":"2024-09-14T23:57:00.062304+0000","last_became_peered":"2024-09-14T23:57:00.062304+0000","last_unstale":"2024-09-14T23:57:32.997392+0000","last_undegraded":"2024-09-14T23:57:32.997392+0000","last_fullsized":"2024-09-14T23:57:32.997392+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:23:16.775003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014783859999999999,"stat_sum":{"num_bytes":8741685,"num_objects":55,"num_object_clones":10,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":28,"num_read":958,"num_read_kb":5490,"num_write":209,"num_write_kb":3731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":3345543,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"79'213","reported_seq":5745,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-14T23:57:30.099942+0000","last_change":"2024-09-14T23:57:26.973933+0000","last_active":"2024-09-14T23:57:30.099942+0000","last_peered":"2024-09-14T23:57:30.099942+0000","last_clean":"2024-09-14T23:57:30.099942+0000","last_became_active":"2024-09-14T23:57:26.973212+0000","last_became_peered":"2024-09-14T23:57:26.973212+0000","last_unstale":"2024-09-14T23:57:30.099942+0000","last_undegraded":"2024-09-14T23:57:30.099942+0000","last_fullsized":"2024-09-14T23:57:30.099942+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:06:50.402930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039416400000000002,"stat_sum":{"num_bytes":26861480,"num_objects":48,"num_object_clones":6,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":23,"num_read":5136,"num_read_kb":22951,"num_write":184,"num_write_kb":10712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":13520480,"num_keys_recovered":48,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.15","version":"70'311","reported_seq":6986,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-14T23:57:29.995082+0000","last_change":"2024-09-14T23:56:34.399382+0000","last_active":"2024-09-14T23:57:29.995082+0000","last_peered":"2024-09-14T23:57:29.995082+0000","last_clean":"2024-09-14T23:57:29.995082+0000","last_became_active":"2024-09-14T23:56:32.405387+0000","last_became_peered":"2024-09-14T23:56:32.405387+0000","last_unstale":"2024-09-14T23:57:29.995082+0000","last_undegraded":"2024-09-14T23:57:29.995082+0000","last_fullsized":"2024-09-14T23:57:29.995082+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:19:15.959433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046328199999999997,"stat_sum":{"num_bytes":37120185,"num_objects":55,"num_object_clones":12,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":25,"num_read":7533,"num_read_kb":32696,"num_write":388,"num_write_kb":22616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"79'680","reported_seq":5032,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-14T23:57:30.026353+0000","last_change":"2024-09-14T23:56:34.404536+0000","last_active":"2024-09-14T23:57:30.026353+0000","last_peered":"2024-09-14T23:57:30.026353+0000","last_clean":"2024-09-14T23:57:30.026353+0000","last_became_active":"2024-09-14T23:56:32.403019+0000","last_became_peered":"2024-09-14T23:56:32.403019+0000","last_unstale":"2024-09-14T23:57:30.026353+0000","last_undegraded":"2024-09-14T23:57:30.026353+0000","last_fullsized":"2024-09-14T23:57:30.026353+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:24:38.033684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037505299999999997,"stat_sum":{"num_bytes":22930896,"num_objects":57,"num_object_clones":11,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":27,"num_read":7253,"num_read_kb":23899,"num_write":1141,"num_write_kb":5963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":135,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-14T23:57:30.034331+0000","last_change":"2024-09-14T23:56:24.727061+0000","last_active":"2024-09-14T23:57:30.034331+0000","last_peered":"2024-09-14T23:57:30.034331+0000","last_clean":"2024-09-14T23:57:30.034331+0000","last_became_active":"2024-09-14T23:56:22.526915+0000","last_became_peered":"2024-09-14T23:56:22.526915+0000","last_unstale":"2024-09-14T23:57:30.034331+0000","last_undegraded":"2024-09-14T23:57:30.034331+0000","last_fullsized":"2024-09-14T23:57:30.034331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:43:15.359266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045105900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"82'279","reported_seq":2911,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T23:57:33.081980+0000","last_change":"2024-09-14T23:56:34.402234+0000","last_active":"2024-09-14T23:57:33.081980+0000","last_peered":"2024-09-14T23:57:33.081980+0000","last_clean":"2024-09-14T23:57:33.081980+0000","last_became_active":"2024-09-14T23:56:32.398664+0000","last_became_peered":"2024-09-14T23:56:32.398664+0000","last_unstale":"2024-09-14T23:57:33.081980+0000","last_undegraded":"2024-09-14T23:57:33.081980+0000","last_fullsized":"2024-09-14T23:57:33.081980+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":279,"log_dups_size":0,"ondisk_log_size":279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:15:59.729258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012179999999999999,"stat_sum":{"num_bytes":23930756,"num_objects":52,"num_object_clones":7,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":24,"num_read":3898,"num_read_kb":13569,"num_write":265,"num_write_kb":4569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":141,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T23:57:32.996803+0000","last_change":"2024-09-14T23:56:24.534632+0000","last_active":"2024-09-14T23:57:32.996803+0000","last_peered":"2024-09-14T23:57:32.996803+0000","last_clean":"2024-09-14T23:57:32.996803+0000","last_became_active":"2024-09-14T23:56:22.526355+0000","last_became_peered":"2024-09-14T23:56:22.526355+0000","last_unstale":"2024-09-14T23:57:32.996803+0000","last_undegraded":"2024-09-14T23:57:32.996803+0000","last_fullsized":"2024-09-14T23:57:32.996803+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:02:16.059627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049910000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"79'280","reported_seq":4888,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-14T23:57:29.994801+0000","last_change":"2024-09-14T23:56:34.416134+0000","last_active":"2024-09-14T23:57:29.994801+0000","last_peered":"2024-09-14T23:57:29.994801+0000","last_clean":"2024-09-14T23:57:29.994801+0000","last_became_active":"2024-09-14T23:56:32.404281+0000","last_became_peered":"2024-09-14T23:56:32.404281+0000","last_unstale":"2024-09-14T23:57:29.994801+0000","last_undegraded":"2024-09-14T23:57:29.994801+0000","last_fullsized":"2024-09-14T23:57:29.994801+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":280,"log_dups_size":0,"ondisk_log_size":280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:02:17.637168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013337620000000001,"stat_sum":{"num_bytes":25401077,"num_objects":67,"num_object_clones":14,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":37,"num_read":5741,"num_read_kb":30765,"num_write":281,"num_write_kb":7929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":135,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-14T23:57:29.995196+0000","last_change":"2024-09-14T23:56:25.093112+0000","last_active":"2024-09-14T23:57:29.995196+0000","last_peered":"2024-09-14T23:57:29.995196+0000","last_clean":"2024-09-14T23:57:29.995196+0000","last_became_active":"2024-09-14T23:56:22.528211+0000","last_became_peered":"2024-09-14T23:56:22.528211+0000","last_unstale":"2024-09-14T23:57:29.995196+0000","last_undegraded":"2024-09-14T23:57:29.995196+0000","last_fullsized":"2024-09-14T23:57:29.995196+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:14:11.495286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000459686,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"82'314","reported_seq":2869,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T23:57:33.166275+0000","last_change":"2024-09-14T23:56:34.401111+0000","last_active":"2024-09-14T23:57:33.166275+0000","last_peered":"2024-09-14T23:57:33.166275+0000","last_clean":"2024-09-14T23:57:33.166275+0000","last_became_active":"2024-09-14T23:56:32.401089+0000","last_became_peered":"2024-09-14T23:56:32.401089+0000","last_unstale":"2024-09-14T23:57:33.166275+0000","last_undegraded":"2024-09-14T23:57:33.166275+0000","last_fullsized":"2024-09-14T23:57:33.166275+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:00:27.976218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013033610000000001,"stat_sum":{"num_bytes":22476573,"num_objects":83,"num_object_clones":15,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":45,"num_read":3059,"num_read_kb":16006,"num_write":310,"num_write_kb":10123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":141,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T23:57:32.996683+0000","last_change":"2024-09-14T23:56:24.534510+0000","last_active":"2024-09-14T23:57:32.996683+0000","last_peered":"2024-09-14T23:57:32.996683+0000","last_clean":"2024-09-14T23:57:32.996683+0000","last_became_active":"2024-09-14T23:56:22.527611+0000","last_became_peered":"2024-09-14T23:56:22.527611+0000","last_unstale":"2024-09-14T23:57:32.996683+0000","last_undegraded":"2024-09-14T23:57:32.996683+0000","last_fullsized":"2024-09-14T23:57:32.996683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:15:45.606026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049916299999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"79'364","reported_seq":2327,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-14T23:57:30.087191+0000","last_change":"2024-09-14T23:57:00.065551+0000","last_active":"2024-09-14T23:57:30.087191+0000","last_peered":"2024-09-14T23:57:30.087191+0000","last_clean":"2024-09-14T23:57:30.087191+0000","last_became_active":"2024-09-14T23:57:00.064678+0000","last_became_peered":"2024-09-14T23:57:00.064678+0000","last_unstale":"2024-09-14T23:57:30.087191+0000","last_undegraded":"2024-09-14T23:57:30.087191+0000","last_fullsized":"2024-09-14T23:57:30.087191+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:53:42.376840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001272947,"stat_sum":{"num_bytes":12132915,"num_objects":76,"num_object_clones":15,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":39,"num_read":2812,"num_read_kb":11305,"num_write":435,"num_write_kb":8709,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":3349587,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"78'259","reported_seq":2252,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-14T23:57:29.986645+0000","last_change":"2024-09-14T23:56:34.404751+0000","last_active":"2024-09-14T23:57:29.986645+0000","last_peered":"2024-09-14T23:57:29.986645+0000","last_clean":"2024-09-14T23:57:29.986645+0000","last_became_active":"2024-09-14T23:56:32.397883+0000","last_became_peered":"2024-09-14T23:56:32.397883+0000","last_unstale":"2024-09-14T23:57:29.986645+0000","last_undegraded":"2024-09-14T23:57:29.986645+0000","last_fullsized":"2024-09-14T23:57:29.986645+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":259,"log_dups_size":0,"ondisk_log_size":259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:38:58.817802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048955799999999996,"stat_sum":{"num_bytes":27889580,"num_objects":66,"num_object_clones":11,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":30,"num_read":3144,"num_read_kb":10813,"num_write":228,"num_write_kb":5761,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":152,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T23:57:32.996652+0000","last_change":"2024-09-14T23:56:57.052969+0000","last_active":"2024-09-14T23:57:32.996652+0000","last_peered":"2024-09-14T23:57:32.996652+0000","last_clean":"2024-09-14T23:57:32.996652+0000","last_became_active":"2024-09-14T23:56:57.050456+0000","last_became_peered":"2024-09-14T23:56:57.050456+0000","last_unstale":"2024-09-14T23:57:32.996652+0000","last_undegraded":"2024-09-14T23:57:32.996652+0000","last_fullsized":"2024-09-14T23:57:32.996652+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:57:07.644011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"82'308","reported_seq":7054,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T23:57:33.079458+0000","last_change":"2024-09-14T23:56:34.401159+0000","last_active":"2024-09-14T23:57:33.079458+0000","last_peered":"2024-09-14T23:57:33.079458+0000","last_clean":"2024-09-14T23:57:33.079458+0000","last_became_active":"2024-09-14T23:56:32.400278+0000","last_became_peered":"2024-09-14T23:56:32.400278+0000","last_unstale":"2024-09-14T23:57:33.079458+0000","last_undegraded":"2024-09-14T23:57:33.079458+0000","last_fullsized":"2024-09-14T23:57:33.079458+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:46:26.056952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082507200000000004,"stat_sum":{"num_bytes":54459136,"num_objects":60,"num_object_clones":10,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":28,"num_read":6859,"num_read_kb":24352,"num_write":281,"num_write_kb":15530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":139,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-14T23:57:32.040684+0000","last_change":"2024-09-14T23:56:24.909260+0000","last_active":"2024-09-14T23:57:32.040684+0000","last_peered":"2024-09-14T23:57:32.040684+0000","last_clean":"2024-09-14T23:57:32.040684+0000","last_became_active":"2024-09-14T23:56:22.822598+0000","last_became_peered":"2024-09-14T23:56:22.822598+0000","last_unstale":"2024-09-14T23:57:32.040684+0000","last_undegraded":"2024-09-14T23:57:32.040684+0000","last_fullsized":"2024-09-14T23:57:32.040684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:32:32.568028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026863400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"82'522","reported_seq":3848,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T23:57:33.169138+0000","last_change":"2024-09-14T23:56:34.401366+0000","last_active":"2024-09-14T23:57:33.169138+0000","last_peered":"2024-09-14T23:57:33.169138+0000","last_clean":"2024-09-14T23:57:33.169138+0000","last_became_active":"2024-09-14T23:56:32.397551+0000","last_became_peered":"2024-09-14T23:56:32.397551+0000","last_unstale":"2024-09-14T23:57:33.169138+0000","last_undegraded":"2024-09-14T23:57:33.169138+0000","last_fullsized":"2024-09-14T23:57:33.169138+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":522,"log_dups_size":0,"ondisk_log_size":522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:32:51.740992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001418816,"stat_sum":{"num_bytes":13857035,"num_objects":77,"num_object_clones":13,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":45,"num_read":5822,"num_read_kb":14163,"num_write":684,"num_write_kb":3852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":135,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-14T23:57:29.994971+0000","last_change":"2024-09-14T23:56:25.093090+0000","last_active":"2024-09-14T23:57:29.994971+0000","last_peered":"2024-09-14T23:57:29.994971+0000","last_clean":"2024-09-14T23:57:29.994971+0000","last_became_active":"2024-09-14T23:56:22.528115+0000","last_became_peered":"2024-09-14T23:56:22.528115+0000","last_unstale":"2024-09-14T23:57:29.994971+0000","last_undegraded":"2024-09-14T23:57:29.994971+0000","last_fullsized":"2024-09-14T23:57:29.994971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:57:36.610730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036520999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"78'386","reported_seq":2370,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-14T23:57:29.986998+0000","last_change":"2024-09-14T23:57:26.975545+0000","last_active":"2024-09-14T23:57:29.986998+0000","last_peered":"2024-09-14T23:57:29.986998+0000","last_clean":"2024-09-14T23:57:29.986998+0000","last_became_active":"2024-09-14T23:57:26.974022+0000","last_became_peered":"2024-09-14T23:57:26.974022+0000","last_unstale":"2024-09-14T23:57:29.986998+0000","last_undegraded":"2024-09-14T23:57:29.986998+0000","last_fullsized":"2024-09-14T23:57:29.986998+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:23:48.051158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073009599999999996,"stat_sum":{"num_bytes":13302760,"num_objects":77,"num_object_clones":19,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":38,"num_read":2068,"num_read_kb":8846,"num_write":470,"num_write_kb":5864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":297,"num_bytes_recovered":12389647,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":137,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-14T23:57:32.040728+0000","last_change":"2024-09-14T23:56:57.054999+0000","last_active":"2024-09-14T23:57:32.040728+0000","last_peered":"2024-09-14T23:57:32.040728+0000","last_clean":"2024-09-14T23:57:32.040728+0000","last_became_active":"2024-09-14T23:56:57.050729+0000","last_became_peered":"2024-09-14T23:56:57.050729+0000","last_unstale":"2024-09-14T23:57:32.040728+0000","last_undegraded":"2024-09-14T23:57:32.040728+0000","last_fullsized":"2024-09-14T23:57:32.040728+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:23:11.727903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":286,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-14T23:57:29.994995+0000","last_change":"2024-09-14T23:56:15.332936+0000","last_active":"2024-09-14T23:57:29.994995+0000","last_peered":"2024-09-14T23:57:29.994995+0000","last_clean":"2024-09-14T23:57:29.994995+0000","last_became_active":"2024-09-14T23:56:15.331994+0000","last_became_peered":"2024-09-14T23:56:15.331994+0000","last_unstale":"2024-09-14T23:57:29.994995+0000","last_undegraded":"2024-09-14T23:57:29.994995+0000","last_fullsized":"2024-09-14T23:57:29.994995+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:14.322882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:14.322882+0000","last_clean_scrub_stamp":"2024-09-14T23:56:14.322882+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:36:12.639000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"78'168","reported_seq":2600,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-14T23:57:29.986748+0000","last_change":"2024-09-14T23:56:34.415037+0000","last_active":"2024-09-14T23:57:29.986748+0000","last_peered":"2024-09-14T23:57:29.986748+0000","last_clean":"2024-09-14T23:57:29.986748+0000","last_became_active":"2024-09-14T23:56:32.403572+0000","last_became_peered":"2024-09-14T23:56:32.403572+0000","last_unstale":"2024-09-14T23:57:29.986748+0000","last_undegraded":"2024-09-14T23:57:29.986748+0000","last_fullsized":"2024-09-14T23:57:29.986748+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:32:24.049063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052643700000000004,"stat_sum":{"num_bytes":17731067,"num_objects":41,"num_object_clones":6,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":21,"num_read":2434,"num_read_kb":10377,"num_write":166,"num_write_kb":8860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"78'170","reported_seq":1461,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-14T23:57:29.986553+0000","last_change":"2024-09-14T23:56:34.404977+0000","last_active":"2024-09-14T23:57:29.986553+0000","last_peered":"2024-09-14T23:57:29.986553+0000","last_clean":"2024-09-14T23:57:29.986553+0000","last_became_active":"2024-09-14T23:56:32.400017+0000","last_became_peered":"2024-09-14T23:56:32.400017+0000","last_unstale":"2024-09-14T23:57:29.986553+0000","last_undegraded":"2024-09-14T23:57:29.986553+0000","last_fullsized":"2024-09-14T23:57:29.986553+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:05:26.356467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068884800000000004,"stat_sum":{"num_bytes":21118108,"num_objects":55,"num_object_clones":11,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":26,"num_read":1223,"num_read_kb":11328,"num_write":172,"num_write_kb":14779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"79'334","reported_seq":6169,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-14T23:57:30.045628+0000","last_change":"2024-09-14T23:56:34.498020+0000","last_active":"2024-09-14T23:57:30.045628+0000","last_peered":"2024-09-14T23:57:30.045628+0000","last_clean":"2024-09-14T23:57:30.045628+0000","last_became_active":"2024-09-14T23:56:32.402769+0000","last_became_peered":"2024-09-14T23:56:32.402769+0000","last_unstale":"2024-09-14T23:57:30.045628+0000","last_undegraded":"2024-09-14T23:57:30.045628+0000","last_fullsized":"2024-09-14T23:57:30.045628+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":334,"log_dups_size":0,"ondisk_log_size":334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:01:59.074599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035483999999999998,"stat_sum":{"num_bytes":25159184,"num_objects":57,"num_object_clones":8,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":30,"num_read":6497,"num_read_kb":25203,"num_write":423,"num_write_kb":8413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"78'415","reported_seq":12334,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-14T23:57:29.986763+0000","last_change":"2024-09-14T23:56:34.415414+0000","last_active":"2024-09-14T23:57:29.986763+0000","last_peered":"2024-09-14T23:57:29.986763+0000","last_clean":"2024-09-14T23:57:29.986763+0000","last_became_active":"2024-09-14T23:56:32.403231+0000","last_became_peered":"2024-09-14T23:56:32.403231+0000","last_unstale":"2024-09-14T23:57:29.986763+0000","last_undegraded":"2024-09-14T23:57:29.986763+0000","last_fullsized":"2024-09-14T23:57:29.986763+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:26:16.476412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072791000000000002,"stat_sum":{"num_bytes":36361843,"num_objects":54,"num_object_clones":12,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":27,"num_read":12928,"num_read_kb":55488,"num_write":552,"num_write_kb":7190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"70'316","reported_seq":12186,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-14T23:57:29.986644+0000","last_change":"2024-09-14T23:57:00.066551+0000","last_active":"2024-09-14T23:57:29.986644+0000","last_peered":"2024-09-14T23:57:29.986644+0000","last_clean":"2024-09-14T23:57:29.986644+0000","last_became_active":"2024-09-14T23:57:00.065790+0000","last_became_peered":"2024-09-14T23:57:00.065790+0000","last_unstale":"2024-09-14T23:57:29.986644+0000","last_undegraded":"2024-09-14T23:57:29.986644+0000","last_fullsized":"2024-09-14T23:57:29.986644+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":316,"log_dups_size":0,"ondisk_log_size":316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:32:52.161561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092638099999999999,"stat_sum":{"num_bytes":46489726,"num_objects":57,"num_object_clones":13,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":26,"num_read":11352,"num_read_kb":50252,"num_write":408,"num_write_kb":18634,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":7912941,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"76'170","reported_seq":1105,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-14T23:57:29.995174+0000","last_change":"2024-09-14T23:57:26.973929+0000","last_active":"2024-09-14T23:57:29.995174+0000","last_peered":"2024-09-14T23:57:29.995174+0000","last_clean":"2024-09-14T23:57:29.995174+0000","last_became_active":"2024-09-14T23:57:26.973189+0000","last_became_peered":"2024-09-14T23:57:26.973189+0000","last_unstale":"2024-09-14T23:57:29.995174+0000","last_undegraded":"2024-09-14T23:57:29.995174+0000","last_fullsized":"2024-09-14T23:57:29.995174+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:08:32.879125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057774300000000003,"stat_sum":{"num_bytes":9701694,"num_objects":45,"num_object_clones":7,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":24,"num_read":618,"num_read_kb":7162,"num_write":182,"num_write_kb":4666,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":9904034,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"79'292","reported_seq":3634,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-14T23:57:30.132195+0000","last_change":"2024-09-14T23:56:34.399894+0000","last_active":"2024-09-14T23:57:30.132195+0000","last_peered":"2024-09-14T23:57:30.132195+0000","last_clean":"2024-09-14T23:57:30.132195+0000","last_became_active":"2024-09-14T23:56:32.406251+0000","last_became_peered":"2024-09-14T23:56:32.406251+0000","last_unstale":"2024-09-14T23:57:30.132195+0000","last_undegraded":"2024-09-14T23:57:30.132195+0000","last_fullsized":"2024-09-14T23:57:30.132195+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":292,"log_dups_size":0,"ondisk_log_size":292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:13:02.054063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073115799999999998,"stat_sum":{"num_bytes":24001781,"num_objects":67,"num_object_clones":13,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":32,"num_read":4928,"num_read_kb":18700,"num_write":291,"num_write_kb":10868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"78'312","reported_seq":3253,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-14T23:57:29.995246+0000","last_change":"2024-09-14T23:56:34.399966+0000","last_active":"2024-09-14T23:57:29.995246+0000","last_peered":"2024-09-14T23:57:29.995246+0000","last_clean":"2024-09-14T23:57:29.995246+0000","last_became_active":"2024-09-14T23:56:32.407109+0000","last_became_peered":"2024-09-14T23:56:32.407109+0000","last_unstale":"2024-09-14T23:57:29.995246+0000","last_undegraded":"2024-09-14T23:57:29.995246+0000","last_fullsized":"2024-09-14T23:57:29.995246+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:23:21.014878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072766900000000002,"stat_sum":{"num_bytes":29716269,"num_objects":64,"num_object_clones":11,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":31,"num_read":4370,"num_read_kb":13505,"num_write":286,"num_write_kb":6514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"77'369","reported_seq":867,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-14T23:57:30.034471+0000","last_change":"2024-09-14T23:56:34.498033+0000","last_active":"2024-09-14T23:57:30.034471+0000","last_peered":"2024-09-14T23:57:30.034471+0000","last_clean":"2024-09-14T23:57:30.034471+0000","last_became_active":"2024-09-14T23:56:32.402973+0000","last_became_peered":"2024-09-14T23:56:32.402973+0000","last_unstale":"2024-09-14T23:57:30.034471+0000","last_undegraded":"2024-09-14T23:57:30.034471+0000","last_fullsized":"2024-09-14T23:57:30.034471+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:26:55.347746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035072199999999998,"stat_sum":{"num_bytes":18752284,"num_objects":57,"num_object_clones":8,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":31,"num_read":1475,"num_read_kb":9132,"num_write":515,"num_write_kb":6040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"78'228","reported_seq":5034,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-14T23:57:30.018344+0000","last_change":"2024-09-14T23:56:34.399387+0000","last_active":"2024-09-14T23:57:30.018344+0000","last_peered":"2024-09-14T23:57:30.018344+0000","last_clean":"2024-09-14T23:57:30.018344+0000","last_became_active":"2024-09-14T23:56:32.405457+0000","last_became_peered":"2024-09-14T23:56:32.405457+0000","last_unstale":"2024-09-14T23:57:30.018344+0000","last_undegraded":"2024-09-14T23:57:30.018344+0000","last_fullsized":"2024-09-14T23:57:30.018344+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":228,"log_dups_size":0,"ondisk_log_size":228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:54:02.307799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048125700000000002,"stat_sum":{"num_bytes":34529826,"num_objects":63,"num_object_clones":12,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":31,"num_read":5071,"num_read_kb":23340,"num_write":212,"num_write_kb":6603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"81'153","reported_seq":1059,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T23:57:32.997325+0000","last_change":"2024-09-14T23:56:34.402592+0000","last_active":"2024-09-14T23:57:32.997325+0000","last_peered":"2024-09-14T23:57:32.997325+0000","last_clean":"2024-09-14T23:57:32.997325+0000","last_became_active":"2024-09-14T23:56:32.402782+0000","last_became_peered":"2024-09-14T23:56:32.402782+0000","last_unstale":"2024-09-14T23:57:32.997325+0000","last_undegraded":"2024-09-14T23:57:32.997325+0000","last_fullsized":"2024-09-14T23:57:32.997325+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:12:45.766825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001446697,"stat_sum":{"num_bytes":8859841,"num_objects":45,"num_object_clones":10,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":25,"num_read":816,"num_read_kb":6921,"num_write":132,"num_write_kb":4183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"74'538","reported_seq":2504,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-14T23:57:29.995056+0000","last_change":"2024-09-14T23:56:34.399629+0000","last_active":"2024-09-14T23:57:29.995056+0000","last_peered":"2024-09-14T23:57:29.995056+0000","last_clean":"2024-09-14T23:57:29.995056+0000","last_became_active":"2024-09-14T23:56:32.407220+0000","last_became_peered":"2024-09-14T23:56:32.407220+0000","last_unstale":"2024-09-14T23:57:29.995056+0000","last_undegraded":"2024-09-14T23:57:29.995056+0000","last_fullsized":"2024-09-14T23:57:29.995056+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":538,"log_dups_size":0,"ondisk_log_size":538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:24:28.126680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062462200000000002,"stat_sum":{"num_bytes":15958982,"num_objects":54,"num_object_clones":13,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":27,"num_read":4250,"num_read_kb":15706,"num_write":821,"num_write_kb":9851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":716123619,"num_objects":1943,"num_object_clones":362,"num_object_copies":3886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1943,"num_whiteouts":995,"num_read":131286,"num_read_kb":549117,"num_write":11953,"num_write_kb":283814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1167,"num_bytes_recovered":61248508,"num_keys_recovered":133,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":48201728,"data_stored":448439686,"data_compressed":34110048,"data_compressed_allocated":36667392,"data_compressed_original":438675619,"omap_allocated":0,"internal_metadata":0},"log_size":10157,"ondisk_log_size":10157,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64594,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509460,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35764,"kb_used_data":8740,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94336076,"statfs":{"total":96636764160,"available":96600141824,"internally_reserved":0,"allocated":8949760,"data_stored":75878606,"data_compressed":5815401,"data_compressed_allocated":6361088,"data_compressed_original":74089833,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509459,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38472,"kb_used_data":3116,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333368,"statfs":{"total":96636764160,"available":96597368832,"internally_reserved":0,"allocated":3190784,"data_stored":18259409,"data_compressed":1537105,"data_compressed_allocated":1716224,"data_compressed_original":17374072,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509458,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34832,"kb_used_data":7808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337008,"statfs":{"total":96636764160,"available":96601096192,"internally_reserved":0,"allocated":7995392,"data_stored":72583583,"data_compressed":5440787,"data_compressed_allocated":5652480,"data_compressed_original":70992407,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509460,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58788,"kb_used_data":9556,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94313052,"statfs":{"total":96636764160,"available":96576565248,"internally_reserved":0,"allocated":9785344,"data_stored":80137907,"data_compressed":5927733,"data_compressed_allocated":6799360,"data_compressed_original":78015220,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509458,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35004,"kb_used_data":7976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94336836,"statfs":{"total":96636764160,"available":96600920064,"internally_reserved":0,"allocated":8167424,"data_stored":73541429,"data_compressed":5275863,"data_compressed_allocated":5746688,"data_compressed_original":71908812,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542163,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45468,"kb_used_data":5956,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94326372,"statfs":{"total":96636764160,"available":96590204928,"internally_reserved":0,"allocated":6098944,"data_stored":44708321,"data_compressed":3407586,"data_compressed_allocated":3760128,"data_compressed_original":43089907,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542162,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37884,"kb_used_data":6700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94333956,"statfs":{"total":96636764160,"available":96597970944,"internally_reserved":0,"allocated":6860800,"data_stored":53738365,"data_compressed":4096055,"data_compressed_allocated":4288512,"data_compressed_original":51948624,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38780,"kb_used_data":3428,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333060,"statfs":{"total":96636764160,"available":96597053440,"internally_reserved":0,"allocated":3510272,"data_stored":22999124,"data_compressed":1728275,"data_compressed_allocated":1835008,"data_compressed_original":21899351,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32297,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32297,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5861376,"data_stored":54081416,"data_compressed":4218520,"data_compressed_allocated":4243456,"data_compressed_original":52697839,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5480448,"data_stored":48226512,"data_compressed":3708020,"data_compressed_allocated":3756032,"data_compressed_original":46614949,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3059712,"data_stored":26303793,"data_compressed":2030855,"data_compressed_allocated":2207744,"data_compressed_original":25532503,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":7426048,"data_stored":74160275,"data_compressed":5414412,"data_compressed_allocated":5906432,"data_compressed_original":72941004,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8892416,"data_stored":79734775,"data_compressed":5928133,"data_compressed_allocated":6803456,"data_compressed_original":78023412,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":7090176,"data_stored":72167944,"data_compressed":5440787,"data_compressed_allocated":5652480,"data_compressed_original":70992407,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2412544,"data_stored":18892312,"data_compressed":1586217,"data_compressed_allocated":1781760,"data_compressed_original":18357112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":7979008,"data_stored":74872659,"data_compressed":5783104,"data_compressed_allocated":6316032,"data_compressed_original":73516393,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T23:57:35.125 INFO:tasks.thrashosds.thrasher:Setting 3.d to [3, 6, 2, 0] 2024-09-14T23:57:35.125 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.d', '3', '6', '2', '0'] 2024-09-14T23:57:35.126 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.d 3 6 2 0 2024-09-14T23:57:35.126 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone31 from 0xc019694 (overlap 0x2012cf0) to 0x33514b 2024-09-14T23:57:35.126 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:35.118+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:35.220 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:35.219+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:35.287 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:35.287 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:35.287 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:57:35.287 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:35.299 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T23:57:35.321 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:35.319+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:35.396 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-09-14T23:57:35.400 INFO:teuthology.orchestra.run.smithi042.stdout:3413 write 0x91972c thru 0x919fa8 (0x87d bytes) 2024-09-14T23:57:35.415 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:35.415 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:35.415 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:57:35.415 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:57:35.415 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:35.422 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:35.420+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:35.429 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T23:57:35.522 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:35.521+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:35.584 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:35.584 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:57:35.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:57:35.584 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:57:35.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.585 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69262 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 4800512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:35.585 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300811+0000", 2024-09-14T23:57:35.585 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 20.281221598999998, 2024-09-14T23:57:35.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050403796000000001, 2024-09-14T23:57:35.585 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:35.585 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:35.585 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:35.585 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:35.585 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:35.585 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:35.585 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69262 2024-09-14T23:57:35.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.585 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:35.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:35.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300811+0000", 2024-09-14T23:57:35.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:35.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300811+0000", 2024-09-14T23:57:35.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:35.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300811+0000", 2024-09-14T23:57:35.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5700000000000002e-07 2024-09-14T23:57:35.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:35.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300815+0000", 2024-09-14T23:57:35.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.087e-06 2024-09-14T23:57:35.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:35.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300823+0000", 2024-09-14T23:57:35.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8720000000000002e-06 2024-09-14T23:57:35.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:35.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300830+0000", 2024-09-14T23:57:35.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0589999999999997e-06 2024-09-14T23:57:35.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:35.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351020+0000", 2024-09-14T23:57:35.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050189552999999998 2024-09-14T23:57:35.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:35.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351112+0000", 2024-09-14T23:57:35.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2615999999999996e-05 2024-09-14T23:57:35.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:35.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351214+0000", 2024-09-14T23:57:35.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101952 2024-09-14T23:57:35.589 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.590 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:35.590 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.590 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69263 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 4915200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:35.591 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300842+0000", 2024-09-14T23:57:35.591 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 20.281190094999999, 2024-09-14T23:57:35.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050590637000000001, 2024-09-14T23:57:35.591 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:35.591 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:35.591 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:35.591 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:35.592 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:35.592 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:35.592 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69263 2024-09-14T23:57:35.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.592 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:35.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:35.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300842+0000", 2024-09-14T23:57:35.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:35.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300842+0000", 2024-09-14T23:57:35.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:35.593 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300843+0000", 2024-09-14T23:57:35.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4000000000000002e-07 2024-09-14T23:57:35.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:35.593 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300846+0000", 2024-09-14T23:57:35.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8720000000000002e-06 2024-09-14T23:57:35.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:35.593 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300853+0000", 2024-09-14T23:57:35.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0450000000000003e-06 2024-09-14T23:57:35.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.594 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:35.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300859+0000", 2024-09-14T23:57:35.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9510000000000003e-06 2024-09-14T23:57:35.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.594 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:35.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351260+0000", 2024-09-14T23:57:35.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050401654999999997 2024-09-14T23:57:35.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.594 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:35.595 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351353+0000", 2024-09-14T23:57:35.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2785e-05 2024-09-14T23:57:35.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:35.595 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351433+0000", 2024-09-14T23:57:35.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9789000000000005e-05 2024-09-14T23:57:35.595 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.595 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:35.595 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.595 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69264 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5029888~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:35.595 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300866+0000", 2024-09-14T23:57:35.595 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 20.281166162000002, 2024-09-14T23:57:35.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050794063, 2024-09-14T23:57:35.596 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:35.596 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:35.596 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:35.596 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:35.596 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:35.596 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:35.596 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69264 2024-09-14T23:57:35.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.596 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:35.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.596 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:35.596 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300866+0000", 2024-09-14T23:57:35.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.598 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:35.598 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300866+0000", 2024-09-14T23:57:35.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.598 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:35.598 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300866+0000", 2024-09-14T23:57:35.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8799999999999998e-07 2024-09-14T23:57:35.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.598 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:35.599 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300869+0000", 2024-09-14T23:57:35.599 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0880000000000002e-06 2024-09-14T23:57:35.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.599 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:35.599 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300875+0000", 2024-09-14T23:57:35.599 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9280000000000004e-06 2024-09-14T23:57:35.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.599 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:35.599 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300882+0000", 2024-09-14T23:57:35.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4740000000000002e-06 2024-09-14T23:57:35.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.600 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:35.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351472+0000", 2024-09-14T23:57:35.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050590006999999999 2024-09-14T23:57:35.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.600 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:35.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351563+0000", 2024-09-14T23:57:35.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0762999999999998e-05 2024-09-14T23:57:35.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:35.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351660+0000", 2024-09-14T23:57:35.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7415000000000002e-05 2024-09-14T23:57:35.601 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.601 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:35.601 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.601 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69265 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5144576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:35.602 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300890+0000", 2024-09-14T23:57:35.602 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 20.281142658, 2024-09-14T23:57:35.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050921389999999997, 2024-09-14T23:57:35.602 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:35.602 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:35.602 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:35.602 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:35.602 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:35.602 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:35.602 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69265 2024-09-14T23:57:35.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.602 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:35.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.602 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:35.603 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300890+0000", 2024-09-14T23:57:35.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:35.603 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300890+0000", 2024-09-14T23:57:35.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:35.603 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300890+0000", 2024-09-14T23:57:35.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2500000000000001e-07 2024-09-14T23:57:35.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:35.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300892+0000", 2024-09-14T23:57:35.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2469999999999998e-06 2024-09-14T23:57:35.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:35.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300899+0000", 2024-09-14T23:57:35.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5509999999999996e-06 2024-09-14T23:57:35.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:35.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300904+0000", 2024-09-14T23:57:35.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3299999999999998e-06 2024-09-14T23:57:35.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:35.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351707+0000", 2024-09-14T23:57:35.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050802673999999999 2024-09-14T23:57:35.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:35.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351755+0000", 2024-09-14T23:57:35.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8362000000000002e-05 2024-09-14T23:57:35.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:35.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351811+0000", 2024-09-14T23:57:35.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5801000000000003e-05 2024-09-14T23:57:35.606 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.606 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:35.606 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.606 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69266 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5259264~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:35.606 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300911+0000", 2024-09-14T23:57:35.606 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 20.281121585000001, 2024-09-14T23:57:35.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051125089999999998, 2024-09-14T23:57:35.606 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:35.606 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:35.606 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:35.606 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:35.607 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:35.607 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:35.607 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69266 2024-09-14T23:57:35.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.607 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:35.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.607 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:35.607 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300911+0000", 2024-09-14T23:57:35.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.607 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:35.607 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300911+0000", 2024-09-14T23:57:35.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:35.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300911+0000", 2024-09-14T23:57:35.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-14T23:57:35.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:35.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300913+0000", 2024-09-14T23:57:35.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9300000000000002e-06 2024-09-14T23:57:35.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:35.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300919+0000", 2024-09-14T23:57:35.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.489e-06 2024-09-14T23:57:35.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:35.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300925+0000", 2024-09-14T23:57:35.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1719999999999998e-06 2024-09-14T23:57:35.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:35.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351847+0000", 2024-09-14T23:57:35.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050922621000000001 2024-09-14T23:57:35.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:35.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351938+0000", 2024-09-14T23:57:35.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0830999999999998e-05 2024-09-14T23:57:35.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:35.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352036+0000", 2024-09-14T23:57:35.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7676999999999993e-05 2024-09-14T23:57:35.610 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.610 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:35.610 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.610 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69267 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5373952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:35.611 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300945+0000", 2024-09-14T23:57:35.611 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 20.28108679, 2024-09-14T23:57:35.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051217934, 2024-09-14T23:57:35.611 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:35.611 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:35.611 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:35.611 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:35.611 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:35.611 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:35.611 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69267 2024-09-14T23:57:35.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.611 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:35.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.611 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:35.611 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300945+0000", 2024-09-14T23:57:35.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.612 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:35.612 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300945+0000", 2024-09-14T23:57:35.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.612 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:35.612 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300946+0000", 2024-09-14T23:57:35.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2600000000000002e-07 2024-09-14T23:57:35.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:35.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300948+0000", 2024-09-14T23:57:35.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4940000000000002e-06 2024-09-14T23:57:35.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:35.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300956+0000", 2024-09-14T23:57:35.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.622e-06 2024-09-14T23:57:35.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:35.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300962+0000", 2024-09-14T23:57:35.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5790000000000003e-06 2024-09-14T23:57:35.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.614 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:35.614 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352063+0000", 2024-09-14T23:57:35.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051101281999999998 2024-09-14T23:57:35.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.614 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:35.614 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352110+0000", 2024-09-14T23:57:35.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7401000000000002e-05 2024-09-14T23:57:35.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.614 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:35.614 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352163+0000", 2024-09-14T23:57:35.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3029999999999999e-05 2024-09-14T23:57:35.615 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.615 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:35.615 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.615 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69268 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5488640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:35.615 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300969+0000", 2024-09-14T23:57:35.615 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 20.281063117999999, 2024-09-14T23:57:35.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051413420000000001, 2024-09-14T23:57:35.615 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:35.615 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:35.615 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:35.615 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:35.615 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:35.615 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:35.616 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69268 2024-09-14T23:57:35.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.616 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:35.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:35.616 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300969+0000", 2024-09-14T23:57:35.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:35.616 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300969+0000", 2024-09-14T23:57:35.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:35.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300970+0000", 2024-09-14T23:57:35.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7899999999999999e-07 2024-09-14T23:57:35.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:35.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300982+0000", 2024-09-14T23:57:35.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2741e-05 2024-09-14T23:57:35.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:35.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300990+0000", 2024-09-14T23:57:35.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8169999999999999e-06 2024-09-14T23:57:35.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:35.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300995+0000", 2024-09-14T23:57:35.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2240000000000003e-06 2024-09-14T23:57:35.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:35.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352200+0000", 2024-09-14T23:57:35.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051204223 2024-09-14T23:57:35.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:35.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352285+0000", 2024-09-14T23:57:35.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5519999999999994e-05 2024-09-14T23:57:35.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.619 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:35.619 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352383+0000", 2024-09-14T23:57:35.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7516000000000006e-05 2024-09-14T23:57:35.619 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.619 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:35.619 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.619 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69269 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5603328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:35.619 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.301003+0000", 2024-09-14T23:57:35.620 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 20.281028935999998, 2024-09-14T23:57:35.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051497381000000002, 2024-09-14T23:57:35.620 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:35.620 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:35.620 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:35.620 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:35.620 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:35.620 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:35.620 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69269 2024-09-14T23:57:35.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.620 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:35.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:35.621 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.301003+0000", 2024-09-14T23:57:35.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:35.621 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.301003+0000", 2024-09-14T23:57:35.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:35.621 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302671+0000", 2024-09-14T23:57:35.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001667785 2024-09-14T23:57:35.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:35.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302676+0000", 2024-09-14T23:57:35.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6600000000000003e-06 2024-09-14T23:57:35.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:35.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302697+0000", 2024-09-14T23:57:35.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1288999999999999e-05 2024-09-14T23:57:35.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:35.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302716+0000", 2024-09-14T23:57:35.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9128999999999999e-05 2024-09-14T23:57:35.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:35.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352399+0000", 2024-09-14T23:57:35.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049682995000000001 2024-09-14T23:57:35.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:35.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352446+0000", 2024-09-14T23:57:35.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6341000000000002e-05 2024-09-14T23:57:35.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:35.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352501+0000", 2024-09-14T23:57:35.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5182000000000001e-05 2024-09-14T23:57:35.624 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.624 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:35.624 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.624 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69273 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6062080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:35.624 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302829+0000", 2024-09-14T23:57:35.624 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 20.279203370000001, 2024-09-14T23:57:35.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050384311000000001, 2024-09-14T23:57:35.624 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:35.624 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:35.624 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:35.624 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:35.624 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:35.625 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:35.625 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69273 2024-09-14T23:57:35.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.625 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:35.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.625 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:35.625 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302829+0000", 2024-09-14T23:57:35.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.625 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:35.625 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302829+0000", 2024-09-14T23:57:35.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:35.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302829+0000", 2024-09-14T23:57:35.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7700000000000005e-07 2024-09-14T23:57:35.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:35.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302832+0000", 2024-09-14T23:57:35.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6970000000000002e-06 2024-09-14T23:57:35.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:35.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302840+0000", 2024-09-14T23:57:35.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5719999999999997e-06 2024-09-14T23:57:35.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:35.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302845+0000", 2024-09-14T23:57:35.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7939999999999999e-06 2024-09-14T23:57:35.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:35.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353111+0000", 2024-09-14T23:57:35.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050265588 2024-09-14T23:57:35.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:35.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353160+0000", 2024-09-14T23:57:35.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8859000000000001e-05 2024-09-14T23:57:35.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:35.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353213+0000", 2024-09-14T23:57:35.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3324e-05 2024-09-14T23:57:35.628 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.628 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:35.628 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.628 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69274 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6176768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:35.628 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302854+0000", 2024-09-14T23:57:35.628 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 20.279178276, 2024-09-14T23:57:35.629 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050579526, 2024-09-14T23:57:35.629 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:35.629 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:35.629 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:35.629 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:35.629 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:35.629 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:35.629 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69274 2024-09-14T23:57:35.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.629 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:35.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.629 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:35.629 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302854+0000", 2024-09-14T23:57:35.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:35.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302854+0000", 2024-09-14T23:57:35.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:35.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302855+0000", 2024-09-14T23:57:35.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0200000000000001e-07 2024-09-14T23:57:35.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:35.631 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302857+0000", 2024-09-14T23:57:35.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.447e-06 2024-09-14T23:57:35.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.631 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:35.631 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302864+0000", 2024-09-14T23:57:35.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7730000000000002e-06 2024-09-14T23:57:35.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.631 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:35.631 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302869+0000", 2024-09-14T23:57:35.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3199999999999999e-06 2024-09-14T23:57:35.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.632 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:35.632 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353249+0000", 2024-09-14T23:57:35.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050379647999999999 2024-09-14T23:57:35.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.632 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:35.632 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353336+0000", 2024-09-14T23:57:35.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6767999999999993e-05 2024-09-14T23:57:35.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.632 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:35.632 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353433+0000", 2024-09-14T23:57:35.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7868000000000006e-05 2024-09-14T23:57:35.633 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.633 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:35.633 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.633 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69275 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6291456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:35.633 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302889+0000", 2024-09-14T23:57:35.633 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 20.279143657999999, 2024-09-14T23:57:35.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050663118, 2024-09-14T23:57:35.633 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:35.633 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:35.633 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:35.633 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:35.633 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:35.634 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:35.634 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69275 2024-09-14T23:57:35.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.634 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:35.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:35.634 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302889+0000", 2024-09-14T23:57:35.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:35.634 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302889+0000", 2024-09-14T23:57:35.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:35.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302889+0000", 2024-09-14T23:57:35.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6399999999999998e-07 2024-09-14T23:57:35.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:35.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302891+0000", 2024-09-14T23:57:35.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.339e-06 2024-09-14T23:57:35.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:35.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302898+0000", 2024-09-14T23:57:35.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0380000000000002e-06 2024-09-14T23:57:35.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:35.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302904+0000", 2024-09-14T23:57:35.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2159999999999997e-06 2024-09-14T23:57:35.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:35.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353449+0000", 2024-09-14T23:57:35.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050545386999999997 2024-09-14T23:57:35.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:35.637 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353494+0000", 2024-09-14T23:57:35.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4592999999999998e-05 2024-09-14T23:57:35.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.637 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:35.637 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353552+0000", 2024-09-14T23:57:35.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8180999999999997e-05 2024-09-14T23:57:35.637 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.637 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:35.637 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.638 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69276 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6406144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:35.638 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302911+0000", 2024-09-14T23:57:35.638 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 20.279121244999999, 2024-09-14T23:57:35.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050865753, 2024-09-14T23:57:35.638 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:35.638 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:35.638 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:35.638 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:35.638 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:35.638 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:35.638 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69276 2024-09-14T23:57:35.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.639 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:35.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:35.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302911+0000", 2024-09-14T23:57:35.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:35.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302911+0000", 2024-09-14T23:57:35.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:35.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302912+0000", 2024-09-14T23:57:35.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0800000000000004e-07 2024-09-14T23:57:35.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:35.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302914+0000", 2024-09-14T23:57:35.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9589999999999998e-06 2024-09-14T23:57:35.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:35.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302920+0000", 2024-09-14T23:57:35.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4579999999999998e-06 2024-09-14T23:57:35.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:35.642 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302925+0000", 2024-09-14T23:57:35.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9429999999999999e-06 2024-09-14T23:57:35.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:35.642 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353586+0000", 2024-09-14T23:57:35.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050660966000000002 2024-09-14T23:57:35.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:35.642 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353679+0000", 2024-09-14T23:57:35.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3163000000000002e-05 2024-09-14T23:57:35.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:35.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353777+0000", 2024-09-14T23:57:35.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7555999999999999e-05 2024-09-14T23:57:35.643 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.643 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:35.643 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.643 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69277 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6520832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:35.643 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302932+0000", 2024-09-14T23:57:35.643 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 20.279100157999999, 2024-09-14T23:57:35.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050977674000000001, 2024-09-14T23:57:35.644 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:35.644 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:35.644 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:35.644 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:35.644 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:35.644 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:35.644 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69277 2024-09-14T23:57:35.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.644 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:35.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:35.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302932+0000", 2024-09-14T23:57:35.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:35.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302932+0000", 2024-09-14T23:57:35.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:35.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302932+0000", 2024-09-14T23:57:35.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3299999999999998e-07 2024-09-14T23:57:35.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:35.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302935+0000", 2024-09-14T23:57:35.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2139999999999999e-06 2024-09-14T23:57:35.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:35.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302941+0000", 2024-09-14T23:57:35.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5039999999999999e-06 2024-09-14T23:57:35.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:35.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302946+0000", 2024-09-14T23:57:35.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2079999999999999e-06 2024-09-14T23:57:35.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:35.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353805+0000", 2024-09-14T23:57:35.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050858588000000003 2024-09-14T23:57:35.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:35.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353849+0000", 2024-09-14T23:57:35.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4511000000000001e-05 2024-09-14T23:57:35.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:35.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353910+0000", 2024-09-14T23:57:35.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0316000000000003e-05 2024-09-14T23:57:35.648 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.648 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:35.648 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.648 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69278 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6635520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:35.648 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302953+0000", 2024-09-14T23:57:35.648 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 20.279079029999998, 2024-09-14T23:57:35.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051174059000000001, 2024-09-14T23:57:35.649 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:35.649 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:35.649 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:35.649 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:35.649 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:35.649 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:35.649 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69278 2024-09-14T23:57:35.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.649 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:35.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:35.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302953+0000", 2024-09-14T23:57:35.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:35.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302953+0000", 2024-09-14T23:57:35.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:35.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302954+0000", 2024-09-14T23:57:35.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5100000000000001e-07 2024-09-14T23:57:35.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:35.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302964+0000", 2024-09-14T23:57:35.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0475000000000001e-05 2024-09-14T23:57:35.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:35.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302970+0000", 2024-09-14T23:57:35.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3879999999999997e-06 2024-09-14T23:57:35.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:35.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302975+0000", 2024-09-14T23:57:35.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8409999999999999e-06 2024-09-14T23:57:35.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.652 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:35.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353944+0000", 2024-09-14T23:57:35.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050968952999999997 2024-09-14T23:57:35.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.652 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:35.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354031+0000", 2024-09-14T23:57:35.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6537000000000002e-05 2024-09-14T23:57:35.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.652 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:35.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354127+0000", 2024-09-14T23:57:35.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6514000000000005e-05 2024-09-14T23:57:35.653 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.653 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:35.653 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.653 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69279 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6750208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:35.653 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302984+0000", 2024-09-14T23:57:35.653 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 20.279048678999999, 2024-09-14T23:57:35.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051258845999999997, 2024-09-14T23:57:35.653 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:35.653 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:35.653 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:35.653 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:35.653 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:35.654 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:35.654 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69279 2024-09-14T23:57:35.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.654 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:35.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:35.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302984+0000", 2024-09-14T23:57:35.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:35.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302984+0000", 2024-09-14T23:57:35.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.655 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:35.655 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303025+0000", 2024-09-14T23:57:35.655 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1690999999999998e-05 2024-09-14T23:57:35.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.655 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:35.655 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303029+0000", 2024-09-14T23:57:35.655 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6559999999999998e-06 2024-09-14T23:57:35.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.655 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:35.655 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303037+0000", 2024-09-14T23:57:35.655 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7950000000000008e-06 2024-09-14T23:57:35.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:35.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303043+0000", 2024-09-14T23:57:35.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9669999999999998e-06 2024-09-14T23:57:35.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:35.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354142+0000", 2024-09-14T23:57:35.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051099072000000002 2024-09-14T23:57:35.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:35.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354183+0000", 2024-09-14T23:57:35.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0834000000000002e-05 2024-09-14T23:57:35.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:35.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354242+0000", 2024-09-14T23:57:35.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9830999999999997e-05 2024-09-14T23:57:35.657 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.657 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:35.657 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.657 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69280 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6864896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:35.657 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.303052+0000", 2024-09-14T23:57:35.657 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 20.278980136000001, 2024-09-14T23:57:35.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051406463999999999, 2024-09-14T23:57:35.657 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:35.657 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:35.657 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:35.658 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:35.658 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:35.658 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:35.658 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69280 2024-09-14T23:57:35.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.658 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:35.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:35.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303052+0000", 2024-09-14T23:57:35.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:35.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303052+0000", 2024-09-14T23:57:35.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.659 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:35.659 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304673+0000", 2024-09-14T23:57:35.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001621389 2024-09-14T23:57:35.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.659 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:35.659 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304681+0000", 2024-09-14T23:57:35.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8439999999999998e-06 2024-09-14T23:57:35.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.659 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:35.659 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304691+0000", 2024-09-14T23:57:35.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8339999999999994e-06 2024-09-14T23:57:35.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:35.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304698+0000", 2024-09-14T23:57:35.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3629999999999999e-06 2024-09-14T23:57:35.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:35.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354278+0000", 2024-09-14T23:57:35.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049580618 2024-09-14T23:57:35.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:35.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354364+0000", 2024-09-14T23:57:35.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6059999999999999e-05 2024-09-14T23:57:35.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:35.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354459+0000", 2024-09-14T23:57:35.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4356000000000003e-05 2024-09-14T23:57:35.661 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.661 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:35.661 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.661 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69285 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 7438336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:35.661 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.304792+0000", 2024-09-14T23:57:35.661 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 20.277240726999999, 2024-09-14T23:57:35.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050398874000000003, 2024-09-14T23:57:35.662 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:35.662 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:35.662 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:35.662 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:35.662 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:35.662 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:35.662 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69285 2024-09-14T23:57:35.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.662 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:35.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:35.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304792+0000", 2024-09-14T23:57:35.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.663 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:35.663 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304792+0000", 2024-09-14T23:57:35.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.663 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:35.663 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304792+0000", 2024-09-14T23:57:35.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2800000000000003e-07 2024-09-14T23:57:35.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.663 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:35.663 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304794+0000", 2024-09-14T23:57:35.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8700000000000001e-06 2024-09-14T23:57:35.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.664 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.664 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:35.664 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304800+0000", 2024-09-14T23:57:35.664 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0889999999999996e-06 2024-09-14T23:57:35.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.664 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.664 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:35.664 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304805+0000", 2024-09-14T23:57:35.664 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7060000000000003e-06 2024-09-14T23:57:35.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.664 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.664 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:35.664 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355108+0000", 2024-09-14T23:57:35.665 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050303170000000001 2024-09-14T23:57:35.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.665 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.665 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:35.665 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355150+0000", 2024-09-14T23:57:35.665 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2271999999999998e-05 2024-09-14T23:57:35.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.665 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.665 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:35.665 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355190+0000", 2024-09-14T23:57:35.665 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0438999999999997e-05 2024-09-14T23:57:35.665 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.665 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:35.666 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.666 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69286 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 7553024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:35.666 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.304811+0000", 2024-09-14T23:57:35.666 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 20.277221364999999, 2024-09-14T23:57:35.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050585512999999999, 2024-09-14T23:57:35.666 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:35.666 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:35.666 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:35.666 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:35.666 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:35.666 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:35.667 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69286 2024-09-14T23:57:35.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.667 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:35.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:35.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304811+0000", 2024-09-14T23:57:35.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:35.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304811+0000", 2024-09-14T23:57:35.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.668 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.668 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:35.668 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304811+0000", 2024-09-14T23:57:35.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.89e-07 2024-09-14T23:57:35.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.668 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.668 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:35.668 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304814+0000", 2024-09-14T23:57:35.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3199999999999998e-06 2024-09-14T23:57:35.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.668 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.668 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:35.668 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304821+0000", 2024-09-14T23:57:35.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5689999999999999e-06 2024-09-14T23:57:35.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.669 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:35.669 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304826+0000", 2024-09-14T23:57:35.669 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1200000000000001e-06 2024-09-14T23:57:35.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.669 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:35.669 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355226+0000", 2024-09-14T23:57:35.669 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050399653000000003 2024-09-14T23:57:35.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.669 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:35.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355314+0000", 2024-09-14T23:57:35.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8389999999999999e-05 2024-09-14T23:57:35.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:35.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355396+0000", 2024-09-14T23:57:35.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2071999999999996e-05 2024-09-14T23:57:35.670 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.670 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:35.670 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.670 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69287 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 7667712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:35.670 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.304833+0000", 2024-09-14T23:57:35.671 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 20.277198767000002, 2024-09-14T23:57:35.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050664740999999999, 2024-09-14T23:57:35.671 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:35.671 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:35.671 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:35.671 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:35.671 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:35.671 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:35.671 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69287 2024-09-14T23:57:35.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.671 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:35.671 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.671 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:35.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304833+0000", 2024-09-14T23:57:35.672 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.672 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.672 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:35.672 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304833+0000", 2024-09-14T23:57:35.672 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.672 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.672 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:35.672 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304834+0000", 2024-09-14T23:57:35.672 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4999999999999998e-07 2024-09-14T23:57:35.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.672 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.673 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:35.673 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304836+0000", 2024-09-14T23:57:35.673 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0279999999999999e-06 2024-09-14T23:57:35.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.673 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.673 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:35.673 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304842+0000", 2024-09-14T23:57:35.673 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8919999999999997e-06 2024-09-14T23:57:35.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.673 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.673 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:35.673 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304847+0000", 2024-09-14T23:57:35.674 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6160000000000001e-06 2024-09-14T23:57:35.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.674 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:35.674 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355411+0000", 2024-09-14T23:57:35.674 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050563540999999997 2024-09-14T23:57:35.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.674 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:35.674 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355459+0000", 2024-09-14T23:57:35.674 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7979000000000003e-05 2024-09-14T23:57:35.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.675 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:35.675 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355498+0000", 2024-09-14T23:57:35.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9335000000000002e-05 2024-09-14T23:57:35.675 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.675 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:35.675 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.675 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69288 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 7782400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:35.675 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.304854+0000", 2024-09-14T23:57:35.675 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 20.277178329000002, 2024-09-14T23:57:35.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050855145999999997, 2024-09-14T23:57:35.675 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:35.675 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:35.675 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:35.676 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:35.676 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:35.676 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:35.676 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69288 2024-09-14T23:57:35.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.676 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:35.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.676 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:35.676 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304854+0000", 2024-09-14T23:57:35.676 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.676 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:35.676 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304854+0000", 2024-09-14T23:57:35.677 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:35.677 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.677 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.677 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:35.677 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.306670+0000", 2024-09-14T23:57:35.677 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00181631 2024-09-14T23:57:35.677 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.677 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.677 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:35.677 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.306675+0000", 2024-09-14T23:57:35.677 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0220000000000004e-06 2024-09-14T23:57:35.677 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.677 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.677 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:35.677 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.306684+0000", 2024-09-14T23:57:35.678 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7309999999999999e-06 2024-09-14T23:57:35.678 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.678 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.678 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:35.678 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.306689+0000", 2024-09-14T23:57:35.678 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6609999999999999e-06 2024-09-14T23:57:35.678 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.678 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.678 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:35.678 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355533+0000", 2024-09-14T23:57:35.678 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048844578 2024-09-14T23:57:35.678 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.678 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.678 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:35.679 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355622+0000", 2024-09-14T23:57:35.679 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9006999999999997e-05 2024-09-14T23:57:35.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:35.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:35.679 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:35.679 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355709+0000", 2024-09-14T23:57:35.679 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6836999999999995e-05 2024-09-14T23:57:35.679 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.679 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:35.679 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.679 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:35.679 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:35.679 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:35.680 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T23:57:35.682 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:35.679+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:35.682 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T23:57:35.781 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:35.780+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:35.796 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:35.796 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:57:35.796 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:57:35.796 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:35.803 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T23:57:35.832 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:57:35.832 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:57:35.849 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:57:35.850 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:57:35.868 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:57:35.868 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:57:35.882 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:35.882+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:35.886 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:57:35.886 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:57:35.906 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:57:35.906 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:57:35.925 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:57:35.926 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:57:35.944 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:35.944 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:35.945 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:57:35.945 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:57:35.953 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T23:57:35.962 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:57:35.963 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:57:35.966 INFO:teuthology.orchestra.run.smithi042.stdout:3415 trunc from 0xc019694 to 0x87b36fc 2024-09-14T23:57:35.978 INFO:teuthology.orchestra.run.smithi042.stdout:3416 read 0x222b2b1 thru 0x2239b2a (0xe87a bytes) 2024-09-14T23:57:35.983 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:35.982+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:35.984 INFO:teuthology.orchestra.run.smithi042.stdout:3417 trunc from 0x87b36fc to 0x4a8825a 2024-09-14T23:57:36.015 INFO:teuthology.orchestra.run.smithi042.stderr:set 3.d pg_upmap_items mapping to [3->6,2->0] 2024-09-14T23:57:36.018 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T23:57:36.081 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:36.081 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:36.085 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:36.084+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:36.092 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T23:57:36.185 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:36.184+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:36.204 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:36.204 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:36.213 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T23:57:36.265 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:36.265 INFO:teuthology.orchestra.run.smithi042.stdout:3420 read 0x17c8e45 thru 0x17cca1c (0x3bd8 bytes) 2024-09-14T23:57:36.282 INFO:teuthology.orchestra.run.smithi042.stdout:3421 write 0xdf86218 thru 0xdf8f9d9 (0x97c2 bytes) 2024-09-14T23:57:36.286 INFO:teuthology.orchestra.run.smithi042.stdout:3422 read 0x2d61d92 thru 0x2d6ebe7 (0xce56 bytes) 2024-09-14T23:57:36.287 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:36.285+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:36.287 INFO:teuthology.orchestra.run.smithi042.stdout:3423 trunc from 0xdf8f9da to 0xe4cd61a 2024-09-14T23:57:36.301 INFO:teuthology.orchestra.run.smithi042.stdout:3424 trunc from 0xe4cd61a to 0x682738d 2024-09-14T23:57:36.312 INFO:teuthology.orchestra.run.smithi042.stdout:3425 trunc from 0x682738d to 0x68b9709 2024-09-14T23:57:36.317 INFO:teuthology.orchestra.run.smithi042.stdout:3426 read 0x444d886 thru 0x4457dc3 (0xa53e bytes) 2024-09-14T23:57:36.319 INFO:teuthology.orchestra.run.smithi042.stdout:3427 read 0x129bde4 thru 0x12a7f84 (0xc1a1 bytes) 2024-09-14T23:57:36.329 INFO:teuthology.orchestra.run.smithi042.stdout:3428 writesame 0x904b967 thru 0x904babb data_size 0x1(0x155 bytes) 2024-09-14T23:57:36.342 INFO:teuthology.orchestra.run.smithi042.stdout:3429 writesame 0xeab848e thru 0xeabf1a3 data_size 0x1(0x6d16 bytes) 2024-09-14T23:57:36.342 INFO:teuthology.orchestra.run.smithi042.stdout:3430 writesame 0x5f23bad thru 0x5f2aa5c data_size 0x4d(0x6eb0 bytes) 2024-09-14T23:57:36.344 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:36.344 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:36.344 INFO:teuthology.orchestra.run.smithi042.stdout:3433 writesame 0x956e9a9 thru 0x9570585 data_size 0x1(0x1bdd bytes) 2024-09-14T23:57:36.356 INFO:teuthology.orchestra.run.smithi042.stdout:3434 trunc from 0xeabf1a4 to 0x8833e44 2024-09-14T23:57:36.371 INFO:teuthology.orchestra.run.smithi042.stdout:3435 trunc from 0x8833e44 to 0xd097cf3 2024-09-14T23:57:36.376 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:36.376 INFO:teuthology.orchestra.run.smithi042.stdout:3437 read 0x2bef0cd thru 0x2bfeccd (0xfc01 bytes) 2024-09-14T23:57:36.377 INFO:teuthology.orchestra.run.smithi042.stdout:3438 read 0x3da4624 thru 0x3db11c0 (0xcb9d bytes) 2024-09-14T23:57:36.377 INFO:teuthology.orchestra.run.smithi042.stdout:3440 read 0x8200ff2 thru 0x8210d96 (0xfda5 bytes) 2024-09-14T23:57:36.377 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:36.378 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:36.388 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:36.387+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:36.390 INFO:teuthology.orchestra.run.smithi042.stdout:3442 trunc from 0xd097cf3 to 0x1832f9b 2024-09-14T23:57:36.392 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T23:57:36.401 INFO:teuthology.orchestra.run.smithi042.stdout:3443 punch from 0xe665b9 to 0xe750af, (0xeaf6 bytes) 2024-09-14T23:57:36.402 INFO:teuthology.orchestra.run.smithi042.stdout:3444 punch from 0x138ab73 to 0x139883f, (0xdccc bytes) 2024-09-14T23:57:36.404 INFO:teuthology.orchestra.run.smithi042.stdout:3445 trunc from 0x1832f9b to 0x755585 2024-09-14T23:57:36.415 INFO:teuthology.orchestra.run.smithi042.stdout:3446 punch from 0x4dbca1 to 0x4e46e9, (0x8a48 bytes) 2024-09-14T23:57:36.429 INFO:teuthology.orchestra.run.smithi042.stdout:3447 trunc from 0x755585 to 0xc9ed3a2 2024-09-14T23:57:36.434 INFO:teuthology.orchestra.run.smithi042.stdout:3448 writesame 0x326b87b thru 0x3273163 data_size 0x1(0x78e9 bytes) 2024-09-14T23:57:36.453 INFO:teuthology.orchestra.run.smithi042.stdout:3449 trunc from 0xc9ed3a2 to 0xaac0a31 2024-09-14T23:57:36.459 INFO:teuthology.orchestra.run.smithi042.stdout:3450 write 0x976fdda thru 0x977c985 (0xcbac bytes) 2024-09-14T23:57:36.462 INFO:teuthology.orchestra.run.smithi042.stdout:3451 write 0x6e60b85 thru 0x6e67379 (0x67f5 bytes) 2024-09-14T23:57:36.465 INFO:teuthology.orchestra.run.smithi042.stdout:3453 write 0xa710d1c thru 0xa71f017 (0xe2fc bytes) 2024-09-14T23:57:36.475 INFO:teuthology.orchestra.run.smithi042.stdout:3455 write 0xbc98009 thru 0xbca58d8 (0xd8d0 bytes) 2024-09-14T23:57:36.478 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:36.478 INFO:teuthology.orchestra.run.smithi042.stdout:3457 punch from 0x6e40a1c to 0x6e468d1, (0x5eb5 bytes) 2024-09-14T23:57:36.478 INFO:teuthology.orchestra.run.smithi042.stdout:3458 read 0x283bdbc thru 0x284132f (0x5574 bytes) 2024-09-14T23:57:36.479 INFO:teuthology.orchestra.run.smithi042.stdout:3459 read 0x2603384 thru 0x260a6ad (0x732a bytes) 2024-09-14T23:57:36.479 INFO:teuthology.orchestra.run.smithi042.stdout:3460 punch from 0x7c5c320 to 0x7c5ed8d, (0x2a6d bytes) 2024-09-14T23:57:36.479 INFO:teuthology.orchestra.run.smithi042.stdout:3461 read 0x5edc3d3 thru 0x5edd981 (0x15af bytes) 2024-09-14T23:57:36.479 INFO:teuthology.orchestra.run.smithi042.stdout:3463 punch from 0x35dbaec to 0x35e8ab7, (0xcfcb bytes) 2024-09-14T23:57:36.479 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:36.479 INFO:teuthology.orchestra.run.smithi042.stdout:3465 clone 33 order 18 su 32768 sc 11 2024-09-14T23:57:36.488 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:36.487+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:36.516 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:36.516 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:36.525 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T23:57:36.589 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:36.588+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:36.671 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:36.672 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:36.680 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T23:57:36.690 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:36.689+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:36.787 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:36.787 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:36.792 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:36.790+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:36.796 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T23:57:36.893 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:36.892+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:36.942 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:36.943 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:36.951 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T23:57:36.995 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:36.994+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:37.016 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is unset 2024-09-14T23:57:37.067 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:37.067 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:37.076 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T23:57:37.094 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:37.093+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:37.117 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone32 from 0xbca58d9 (overlap 0x755585) to 0x233341 2024-09-14T23:57:37.196 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:37.194+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:37.242 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:37.242 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:37.250 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #31, image image_client.0-clone31 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent32 2024-09-14T23:57:37.252 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T23:57:37.260 INFO:teuthology.orchestra.run.smithi042.stdout:3468 read 0xb86b3bd thru 0xb87a3b0 (0xeff4 bytes) 2024-09-14T23:57:37.262 INFO:teuthology.orchestra.run.smithi042.stdout:3469 write 0x1824fae thru 0x182f072 (0xa0c5 bytes) 2024-09-14T23:57:37.268 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:37.269 INFO:teuthology.orchestra.run.smithi042.stdout:3472 punch from 0x7c5f691 to 0x7c64df7, (0x5766 bytes) 2024-09-14T23:57:37.269 INFO:teuthology.orchestra.run.smithi042.stdout:3474 read 0xb2cc4ae thru 0xb2d5e62 (0x99b5 bytes) 2024-09-14T23:57:37.269 INFO:teuthology.orchestra.run.smithi042.stdout:3475 write 0x9187fd1 thru 0x918e939 (0x6969 bytes) 2024-09-14T23:57:37.275 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:37.275 INFO:teuthology.orchestra.run.smithi042.stdout:3477 write 0x36e5b71 thru 0x36efe3e (0xa2ce bytes) 2024-09-14T23:57:37.287 INFO:teuthology.orchestra.run.smithi042.stdout:3478 trunc from 0xbca58d9 to 0x5d4d517 2024-09-14T23:57:37.297 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:37.296+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:37.304 INFO:teuthology.orchestra.run.smithi042.stdout:3479 read 0x2407793 thru 0x240fe9c (0x870a bytes) 2024-09-14T23:57:37.304 INFO:teuthology.orchestra.run.smithi042.stdout:3480 read 0x14fd672 thru 0x1508508 (0xae97 bytes) 2024-09-14T23:57:37.307 INFO:teuthology.orchestra.run.smithi042.stdout:3481 trunc from 0x5d4d517 to 0x7d96d7d 2024-09-14T23:57:37.312 INFO:teuthology.orchestra.run.smithi042.stdout:3482 read 0x692206b thru 0x692e776 (0xc70c bytes) 2024-09-14T23:57:37.312 INFO:teuthology.orchestra.run.smithi042.stdout:3484 write 0x1f066ef thru 0x1f1337b (0xcc8d bytes) 2024-09-14T23:57:37.318 INFO:teuthology.orchestra.run.smithi042.stdout:3485 write 0x70e87cc thru 0x70f5362 (0xcb97 bytes) 2024-09-14T23:57:37.328 INFO:teuthology.orchestra.run.smithi042.stdout:3486 writesame 0x89b8208 thru 0x89be3ea data_size 0x1(0x61e3 bytes) 2024-09-14T23:57:37.328 INFO:teuthology.orchestra.run.smithi042.stdout:3487 write 0x4ce1be9 thru 0x4ce6c2f (0x5047 bytes) 2024-09-14T23:57:37.332 INFO:teuthology.orchestra.run.smithi042.stdout:3489 read 0x4373b33 thru 0x4376275 (0x2743 bytes) 2024-09-14T23:57:37.332 INFO:teuthology.orchestra.run.smithi042.stdout:3493 read 0x2b6fed7 thru 0x2b7a7a5 (0xa8cf bytes) 2024-09-14T23:57:37.335 INFO:teuthology.orchestra.run.smithi042.stdout:3494 trunc from 0x89be3eb to 0xa5921b9 2024-09-14T23:57:37.346 INFO:teuthology.orchestra.run.smithi042.stdout:3497 write 0xb10a6e8 thru 0xb10f6d5 (0x4fee bytes) 2024-09-14T23:57:37.349 INFO:teuthology.orchestra.run.smithi042.stdout:3498 write 0x1937c45 thru 0x19413cb (0x9787 bytes) 2024-09-14T23:57:37.354 INFO:teuthology.orchestra.run.smithi042.stdout:3501 read 0x9b51cf8 thru 0x9b5c711 (0xaa1a bytes) 2024-09-14T23:57:37.354 INFO:teuthology.orchestra.run.smithi042.stdout:3504 write 0x7793b0c thru 0x779a285 (0x677a bytes) 2024-09-14T23:57:37.358 INFO:teuthology.orchestra.run.smithi042.stdout:3506 writesame 0x6b9c5d8 thru 0x6ba8e36 data_size 0x1(0xc85f bytes) 2024-09-14T23:57:37.386 INFO:teuthology.orchestra.run.smithi042.stdout:3509 punch from 0x333a812 to 0x33458c3, (0xb0b1 bytes) 2024-09-14T23:57:37.386 INFO:teuthology.orchestra.run.smithi042.stdout:3510 read 0x4450850 thru 0x445a090 (0x9841 bytes) 2024-09-14T23:57:37.386 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:37.387 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:37.387 INFO:teuthology.orchestra.run.smithi042.stdout:3511 clone 34 order 23 su 65536 sc 2 2024-09-14T23:57:37.396 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T23:57:37.437 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:37.436+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:37.499 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:37.497+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:37.504 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:37.504 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:37.513 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T23:57:37.599 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:37.598+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:37.661 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:37.661 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:37.669 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T23:57:37.700 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:37.699+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:37.776 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:37.777 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:37.785 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T23:57:37.802 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:37.800+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:37.893 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:37.893 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:37.902 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T23:57:37.903 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:37.902+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:37.978 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T23:57:38.004 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:38.003+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:38.009 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:38.009 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:38.018 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T23:57:38.105 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:38.104+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:38.119 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:57:38.120 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:57:38.124 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone33 from 0xb10f6d6 (overlap 0x5d4d517) to 0x12a9104 2024-09-14T23:57:38.148 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:57:38.148 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:57:38.173 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:57:38.174 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:57:38.191 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:38.191 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:38.197 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:57:38.198 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:57:38.205 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T23:57:38.206 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:38.205+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:38.307 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:38.305+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:38.361 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:38.361 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:38.373 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T23:57:38.407 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:38.406+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:38.425 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:57:38.426 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:57:38.451 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:57:38.451 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:57:38.470 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:57:38.470 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:57:38.487 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:57:38.487 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:57:38.509 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:38.509 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:38.509 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:57:38.509 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:57:38.509 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:38.518 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T23:57:38.540 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:38.538+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:38.590 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #32, image image_client.0-clone32 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent33 2024-09-14T23:57:38.601 INFO:teuthology.orchestra.run.smithi042.stdout:3513 read 0x1565419 thru 0x1566ec1 (0x1aa9 bytes) 2024-09-14T23:57:38.602 INFO:teuthology.orchestra.run.smithi042.stdout:3514 write 0x6e5c31d thru 0x6e63beb (0x78cf bytes) 2024-09-14T23:57:38.609 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:38.607+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:38.611 INFO:teuthology.orchestra.run.smithi042.stdout:3515 punch from 0x5cf0d1 to 0x5d1d47, (0x2c76 bytes) 2024-09-14T23:57:38.611 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:38.612 INFO:teuthology.orchestra.run.smithi042.stdout:3518 write 0x7524e09 thru 0x75264fd (0x16f5 bytes) 2024-09-14T23:57:38.615 INFO:teuthology.orchestra.run.smithi042.stdout:3520 write 0x4e9c5a7 thru 0x4ea70cd (0xab27 bytes) 2024-09-14T23:57:38.639 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:38.639 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:57:38.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:57:38.639 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:57:38.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.639 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68736 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 5906432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:38.640 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.264178+0000", 2024-09-14T23:57:38.640 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.37200172, 2024-09-14T23:57:38.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.045913849999999999, 2024-09-14T23:57:38.640 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:38.640 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:38.640 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:38.640 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:38.640 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:38.640 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:38.640 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68736 2024-09-14T23:57:38.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.640 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:38.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:38.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264178+0000", 2024-09-14T23:57:38.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:38.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:38.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264178+0000", 2024-09-14T23:57:38.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:38.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:38.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264179+0000", 2024-09-14T23:57:38.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8599999999999999e-07 2024-09-14T23:57:38.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:38.642 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264181+0000", 2024-09-14T23:57:38.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.855e-06 2024-09-14T23:57:38.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:38.642 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264187+0000", 2024-09-14T23:57:38.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1539999999999999e-06 2024-09-14T23:57:38.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:38.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264192+0000", 2024-09-14T23:57:38.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6409999999999999e-06 2024-09-14T23:57:38.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:38.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.309931+0000", 2024-09-14T23:57:38.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.045738442999999997 2024-09-14T23:57:38.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:38.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310017+0000", 2024-09-14T23:57:38.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5829999999999996e-05 2024-09-14T23:57:38.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:38.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310092+0000", 2024-09-14T23:57:38.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5541000000000001e-05 2024-09-14T23:57:38.644 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:38.644 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:38.644 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:38.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.644 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68737 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 6021120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:38.644 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.264199+0000", 2024-09-14T23:57:38.644 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.371981255000001, 2024-09-14T23:57:38.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046063354000000001, 2024-09-14T23:57:38.644 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:38.644 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:38.645 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:38.645 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:38.645 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:38.645 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:38.645 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68737 2024-09-14T23:57:38.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.645 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:38.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:38.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264199+0000", 2024-09-14T23:57:38.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:38.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:38.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264199+0000", 2024-09-14T23:57:38.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:38.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:38.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264199+0000", 2024-09-14T23:57:38.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1600000000000002e-07 2024-09-14T23:57:38.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:38.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264210+0000", 2024-09-14T23:57:38.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0973e-05 2024-09-14T23:57:38.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:38.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264217+0000", 2024-09-14T23:57:38.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0659999999999998e-06 2024-09-14T23:57:38.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:38.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264223+0000", 2024-09-14T23:57:38.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2039999999999996e-06 2024-09-14T23:57:38.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:38.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310121+0000", 2024-09-14T23:57:38.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.045898436000000001 2024-09-14T23:57:38.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:38.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310194+0000", 2024-09-14T23:57:38.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2708999999999998e-05 2024-09-14T23:57:38.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:38.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310262+0000", 2024-09-14T23:57:38.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8549999999999999e-05 2024-09-14T23:57:38.648 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:38.648 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:38.648 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:38.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.648 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68738 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 6135808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:38.648 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.264234+0000", 2024-09-14T23:57:38.648 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.371945825000001, 2024-09-14T23:57:38.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046212954000000001, 2024-09-14T23:57:38.648 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:38.649 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:38.649 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:38.649 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:38.649 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:38.649 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:38.649 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68738 2024-09-14T23:57:38.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.649 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:38.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:38.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264234+0000", 2024-09-14T23:57:38.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:38.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:38.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264234+0000", 2024-09-14T23:57:38.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:38.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:38.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264235+0000", 2024-09-14T23:57:38.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6600000000000002e-07 2024-09-14T23:57:38.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:38.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264237+0000", 2024-09-14T23:57:38.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3120000000000001e-06 2024-09-14T23:57:38.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:38.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264244+0000", 2024-09-14T23:57:38.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6680000000000004e-06 2024-09-14T23:57:38.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:38.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264255+0000", 2024-09-14T23:57:38.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.155e-05 2024-09-14T23:57:38.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:38.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310292+0000", 2024-09-14T23:57:38.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046037026000000002 2024-09-14T23:57:38.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:38.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310375+0000", 2024-09-14T23:57:38.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3039000000000002e-05 2024-09-14T23:57:38.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.652 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:38.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310447+0000", 2024-09-14T23:57:38.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1892999999999998e-05 2024-09-14T23:57:38.652 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:38.652 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:38.652 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:38.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.652 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68739 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 6250496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:38.652 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.264263+0000", 2024-09-14T23:57:38.652 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.371917268000001, 2024-09-14T23:57:38.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046362974000000001, 2024-09-14T23:57:38.653 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:38.653 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:38.653 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:38.653 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:38.653 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:38.653 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:38.653 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68739 2024-09-14T23:57:38.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.653 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:38.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:38.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264263+0000", 2024-09-14T23:57:38.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:38.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:38.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264263+0000", 2024-09-14T23:57:38.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:38.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:38.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264263+0000", 2024-09-14T23:57:38.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8599999999999999e-07 2024-09-14T23:57:38.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:38.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264265+0000", 2024-09-14T23:57:38.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0020000000000001e-06 2024-09-14T23:57:38.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.655 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:38.655 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264272+0000", 2024-09-14T23:57:38.655 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4799999999999998e-06 2024-09-14T23:57:38.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.655 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:38.655 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264282+0000", 2024-09-14T23:57:38.655 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0244999999999999e-05 2024-09-14T23:57:38.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.655 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:38.655 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310478+0000", 2024-09-14T23:57:38.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046196070999999998 2024-09-14T23:57:38.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:38.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310554+0000", 2024-09-14T23:57:38.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6160000000000003e-05 2024-09-14T23:57:38.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:38.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310626+0000", 2024-09-14T23:57:38.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1630000000000004e-05 2024-09-14T23:57:38.656 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:38.656 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:38.656 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:38.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.657 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68740 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 6365184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:38.657 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.264289+0000", 2024-09-14T23:57:38.657 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.371890812, 2024-09-14T23:57:38.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046531222999999997, 2024-09-14T23:57:38.657 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:38.657 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:38.657 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:38.657 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:38.657 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:38.657 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:38.657 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68740 2024-09-14T23:57:38.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.658 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:38.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:38.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264289+0000", 2024-09-14T23:57:38.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:38.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:38.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264289+0000", 2024-09-14T23:57:38.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:38.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:38.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264290+0000", 2024-09-14T23:57:38.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4299999999999998e-07 2024-09-14T23:57:38.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.659 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:38.659 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264292+0000", 2024-09-14T23:57:38.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.181e-06 2024-09-14T23:57:38.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.659 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:38.659 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264298+0000", 2024-09-14T23:57:38.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1120000000000001e-06 2024-09-14T23:57:38.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.659 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:38.659 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264303+0000", 2024-09-14T23:57:38.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5839999999999996e-06 2024-09-14T23:57:38.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:38.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310654+0000", 2024-09-14T23:57:38.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046351046 2024-09-14T23:57:38.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:38.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310745+0000", 2024-09-14T23:57:38.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1027999999999997e-05 2024-09-14T23:57:38.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:38.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310821+0000", 2024-09-14T23:57:38.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5828999999999995e-05 2024-09-14T23:57:38.661 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:38.661 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:38.661 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:38.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.661 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68741 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 6479872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:38.661 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.264332+0000", 2024-09-14T23:57:38.661 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.371847842000001, 2024-09-14T23:57:38.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046692392999999999, 2024-09-14T23:57:38.661 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:38.661 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:38.661 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:38.662 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:38.662 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:38.662 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:38.662 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68741 2024-09-14T23:57:38.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.662 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:38.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:38.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264332+0000", 2024-09-14T23:57:38.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:38.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:38.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264332+0000", 2024-09-14T23:57:38.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:38.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.663 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:38.663 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264333+0000", 2024-09-14T23:57:38.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2899999999999999e-07 2024-09-14T23:57:38.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.663 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:38.663 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264335+0000", 2024-09-14T23:57:38.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9870000000000002e-06 2024-09-14T23:57:38.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.663 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:38.664 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264340+0000", 2024-09-14T23:57:38.664 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0069999999999997e-06 2024-09-14T23:57:38.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.664 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.664 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:38.664 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264343+0000", 2024-09-14T23:57:38.664 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0960000000000001e-06 2024-09-14T23:57:38.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.664 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.664 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:38.664 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310855+0000", 2024-09-14T23:57:38.664 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046511915000000001 2024-09-14T23:57:38.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.664 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.664 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:38.665 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.310946+0000", 2024-09-14T23:57:38.665 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1290000000000002e-05 2024-09-14T23:57:38.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.665 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.665 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:38.665 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311025+0000", 2024-09-14T23:57:38.665 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8769000000000004e-05 2024-09-14T23:57:38.665 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:38.665 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:38.665 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:38.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.665 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.665 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68742 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 6594560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:38.665 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.264347+0000", 2024-09-14T23:57:38.666 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.3718328, 2024-09-14T23:57:38.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046904279, 2024-09-14T23:57:38.666 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:38.666 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:38.666 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:38.666 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:38.666 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:38.666 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:38.666 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68742 2024-09-14T23:57:38.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.666 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:38.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:38.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264347+0000", 2024-09-14T23:57:38.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:38.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:38.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264347+0000", 2024-09-14T23:57:38.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:38.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:38.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264348+0000", 2024-09-14T23:57:38.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.22e-07 2024-09-14T23:57:38.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:38.668 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264349+0000", 2024-09-14T23:57:38.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2079999999999999e-06 2024-09-14T23:57:38.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.668 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.668 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:38.668 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264352+0000", 2024-09-14T23:57:38.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1640000000000001e-06 2024-09-14T23:57:38.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.668 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.668 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:38.668 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264354+0000", 2024-09-14T23:57:38.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3350000000000001e-06 2024-09-14T23:57:38.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.668 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.669 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:38.669 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311071+0000", 2024-09-14T23:57:38.669 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046716921000000002 2024-09-14T23:57:38.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.669 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:38.669 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311165+0000", 2024-09-14T23:57:38.669 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3900000000000006e-05 2024-09-14T23:57:38.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.669 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:38.669 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311252+0000", 2024-09-14T23:57:38.669 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6428999999999995e-05 2024-09-14T23:57:38.670 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:38.670 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:38.670 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:38.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.670 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68743 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 6709248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:38.670 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.264359+0000", 2024-09-14T23:57:38.670 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.371821171000001, 2024-09-14T23:57:38.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047086817000000003, 2024-09-14T23:57:38.670 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:38.670 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:38.670 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:38.670 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:38.671 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:38.671 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:38.671 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68743 2024-09-14T23:57:38.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.671 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:38.671 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.671 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:38.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264359+0000", 2024-09-14T23:57:38.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:38.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.671 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.671 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:38.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.264359+0000", 2024-09-14T23:57:38.672 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:38.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.672 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.672 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:38.672 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265672+0000", 2024-09-14T23:57:38.672 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0013131849999999999 2024-09-14T23:57:38.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.672 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.672 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:38.672 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265676+0000", 2024-09-14T23:57:38.672 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2030000000000002e-06 2024-09-14T23:57:38.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.672 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.673 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:38.673 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265685+0000", 2024-09-14T23:57:38.673 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7290000000000006e-06 2024-09-14T23:57:38.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.673 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.673 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:38.673 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265691+0000", 2024-09-14T23:57:38.673 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3779999999999998e-06 2024-09-14T23:57:38.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.673 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.673 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:38.673 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311283+0000", 2024-09-14T23:57:38.673 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.045591797000000003 2024-09-14T23:57:38.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.674 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:38.674 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311368+0000", 2024-09-14T23:57:38.674 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4444999999999995e-05 2024-09-14T23:57:38.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.674 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:38.674 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311446+0000", 2024-09-14T23:57:38.674 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8079999999999998e-05 2024-09-14T23:57:38.674 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:38.674 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:38.674 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:38.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.675 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68744 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 6823936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:38.675 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.265703+0000", 2024-09-14T23:57:38.675 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.370477395999998, 2024-09-14T23:57:38.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.045950957000000001, 2024-09-14T23:57:38.675 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:38.675 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:38.675 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:38.675 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:38.675 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:38.675 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:38.675 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68744 2024-09-14T23:57:38.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.676 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:38.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.676 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:38.676 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265703+0000", 2024-09-14T23:57:38.676 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:38.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.676 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:38.676 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265703+0000", 2024-09-14T23:57:38.676 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:38.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.676 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:38.677 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265703+0000", 2024-09-14T23:57:38.677 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9900000000000001e-07 2024-09-14T23:57:38.677 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.677 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.677 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:38.677 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265706+0000", 2024-09-14T23:57:38.677 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4329999999999998e-06 2024-09-14T23:57:38.677 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.677 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.677 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:38.677 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265712+0000", 2024-09-14T23:57:38.677 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8020000000000003e-06 2024-09-14T23:57:38.677 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.678 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.678 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:38.678 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265718+0000", 2024-09-14T23:57:38.678 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.13e-06 2024-09-14T23:57:38.678 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.678 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.678 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:38.678 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311494+0000", 2024-09-14T23:57:38.678 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.045776039999999997 2024-09-14T23:57:38.678 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.678 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.678 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:38.678 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311577+0000", 2024-09-14T23:57:38.678 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3057000000000001e-05 2024-09-14T23:57:38.678 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.679 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:38.679 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311654+0000", 2024-09-14T23:57:38.679 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6996000000000006e-05 2024-09-14T23:57:38.679 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:38.679 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:38.679 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:38.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.679 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68745 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 6938624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:38.679 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.265725+0000", 2024-09-14T23:57:38.679 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.370455566, 2024-09-14T23:57:38.679 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046101672000000003, 2024-09-14T23:57:38.679 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:38.680 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:38.680 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:38.680 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:38.680 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:38.680 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:38.680 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68745 2024-09-14T23:57:38.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.680 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:38.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.680 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:38.680 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265725+0000", 2024-09-14T23:57:38.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:38.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.681 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:38.681 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265725+0000", 2024-09-14T23:57:38.681 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:38.681 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.681 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.681 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:38.681 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265725+0000", 2024-09-14T23:57:38.681 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5299999999999999e-07 2024-09-14T23:57:38.681 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.681 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.681 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:38.681 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265727+0000", 2024-09-14T23:57:38.681 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.069e-06 2024-09-14T23:57:38.682 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.682 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.682 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:38.682 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265733+0000", 2024-09-14T23:57:38.682 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1510000000000001e-06 2024-09-14T23:57:38.682 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.682 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.682 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:38.682 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265738+0000", 2024-09-14T23:57:38.682 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7010000000000004e-06 2024-09-14T23:57:38.682 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.682 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.682 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:38.682 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311707+0000", 2024-09-14T23:57:38.683 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.045968602999999997 2024-09-14T23:57:38.683 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.683 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.683 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:38.683 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311776+0000", 2024-09-14T23:57:38.683 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9438999999999997e-05 2024-09-14T23:57:38.683 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.683 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.683 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:38.683 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311826+0000", 2024-09-14T23:57:38.683 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0256000000000001e-05 2024-09-14T23:57:38.683 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:38.683 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:38.683 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:38.683 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.684 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.684 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68746 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 7053312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:38.684 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.265744+0000", 2024-09-14T23:57:38.684 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.370435935, 2024-09-14T23:57:38.684 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046203505999999998, 2024-09-14T23:57:38.684 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:38.684 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:38.684 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:38.684 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:38.684 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:38.684 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:38.684 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68746 2024-09-14T23:57:38.684 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.685 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:38.685 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.685 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:38.685 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265744+0000", 2024-09-14T23:57:38.685 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:38.685 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.685 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.685 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:38.685 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265744+0000", 2024-09-14T23:57:38.685 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:38.685 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.685 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.685 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:38.685 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265745+0000", 2024-09-14T23:57:38.685 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9200000000000002e-07 2024-09-14T23:57:38.686 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.686 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.686 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:38.686 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265746+0000", 2024-09-14T23:57:38.686 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8819999999999999e-06 2024-09-14T23:57:38.686 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.686 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.686 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:38.686 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265753+0000", 2024-09-14T23:57:38.686 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1990000000000003e-06 2024-09-14T23:57:38.686 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.686 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.686 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:38.686 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265760+0000", 2024-09-14T23:57:38.687 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9280000000000004e-06 2024-09-14T23:57:38.687 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.687 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.687 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:38.687 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311859+0000", 2024-09-14T23:57:38.687 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046099627999999997 2024-09-14T23:57:38.687 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.687 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.687 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:38.687 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311902+0000", 2024-09-14T23:57:38.687 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3239999999999999e-05 2024-09-14T23:57:38.687 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.687 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.687 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:38.687 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311948+0000", 2024-09-14T23:57:38.688 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5237000000000001e-05 2024-09-14T23:57:38.688 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:38.688 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:38.688 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:38.688 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.688 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.688 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68747 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 7168000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:38.688 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.265766+0000", 2024-09-14T23:57:38.688 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.370413773999999, 2024-09-14T23:57:38.688 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046303703000000002, 2024-09-14T23:57:38.688 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:38.688 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:38.688 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:38.688 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:38.688 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:38.689 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:38.689 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68747 2024-09-14T23:57:38.689 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.689 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:38.689 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.689 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:38.689 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265766+0000", 2024-09-14T23:57:38.689 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:38.689 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.689 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.689 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:38.689 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265766+0000", 2024-09-14T23:57:38.689 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:38.689 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.689 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.690 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:38.690 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265767+0000", 2024-09-14T23:57:38.690 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9200000000000002e-07 2024-09-14T23:57:38.690 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.690 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.690 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:38.690 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265769+0000", 2024-09-14T23:57:38.690 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9369999999999998e-06 2024-09-14T23:57:38.690 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.690 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.690 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:38.690 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265775+0000", 2024-09-14T23:57:38.690 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9499999999999998e-06 2024-09-14T23:57:38.690 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.690 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.691 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:38.691 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265779+0000", 2024-09-14T23:57:38.691 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7979999999999996e-06 2024-09-14T23:57:38.691 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.691 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.691 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:38.691 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.311977+0000", 2024-09-14T23:57:38.691 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046197437000000001 2024-09-14T23:57:38.691 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.691 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.691 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:38.691 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.312024+0000", 2024-09-14T23:57:38.691 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6780999999999998e-05 2024-09-14T23:57:38.691 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.691 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.691 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:38.692 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.312070+0000", 2024-09-14T23:57:38.692 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6408e-05 2024-09-14T23:57:38.692 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:38.692 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:38.692 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:38.692 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.692 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.692 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68748 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 7282688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:38.692 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.265785+0000", 2024-09-14T23:57:38.692 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.370394713, 2024-09-14T23:57:38.692 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046405109999999999, 2024-09-14T23:57:38.692 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:38.692 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:38.693 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:38.693 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:38.693 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:38.693 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:38.693 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68748 2024-09-14T23:57:38.693 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.693 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:38.693 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.693 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:38.693 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265785+0000", 2024-09-14T23:57:38.693 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:38.693 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.693 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.693 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:38.694 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265785+0000", 2024-09-14T23:57:38.694 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:38.694 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.694 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.694 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:38.694 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265786+0000", 2024-09-14T23:57:38.694 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0900000000000002e-07 2024-09-14T23:57:38.694 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.694 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.694 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:38.694 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265796+0000", 2024-09-14T23:57:38.694 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8490000000000001e-06 2024-09-14T23:57:38.694 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.694 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.695 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:38.695 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265802+0000", 2024-09-14T23:57:38.695 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7309999999999996e-06 2024-09-14T23:57:38.695 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.695 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.695 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:38.695 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265812+0000", 2024-09-14T23:57:38.695 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8789999999999998e-06 2024-09-14T23:57:38.695 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.695 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.695 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:38.695 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.312098+0000", 2024-09-14T23:57:38.695 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046285862999999997 2024-09-14T23:57:38.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.696 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.696 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:38.696 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.312140+0000", 2024-09-14T23:57:38.696 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1718000000000003e-05 2024-09-14T23:57:38.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.696 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.696 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:38.696 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.312190+0000", 2024-09-14T23:57:38.696 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0661000000000002e-05 2024-09-14T23:57:38.696 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:38.697 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:38.697 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:38.697 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.697 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.697 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68749 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 7397376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:38.697 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.265821+0000", 2024-09-14T23:57:38.697 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.370359217000001, 2024-09-14T23:57:38.697 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046484708, 2024-09-14T23:57:38.697 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:38.697 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:38.697 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:38.697 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:38.697 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:38.698 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:38.698 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68749 2024-09-14T23:57:38.698 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.698 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:38.698 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.698 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:38.698 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265821+0000", 2024-09-14T23:57:38.698 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:38.698 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.698 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.698 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:38.698 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265821+0000", 2024-09-14T23:57:38.698 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:38.698 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.698 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.698 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:38.699 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265821+0000", 2024-09-14T23:57:38.699 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7300000000000001e-07 2024-09-14T23:57:38.699 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.699 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.699 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:38.699 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265829+0000", 2024-09-14T23:57:38.699 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8299999999999996e-06 2024-09-14T23:57:38.699 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.699 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.699 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:38.699 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265836+0000", 2024-09-14T23:57:38.699 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0709999999999997e-06 2024-09-14T23:57:38.699 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.699 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.700 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:38.700 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265841+0000", 2024-09-14T23:57:38.700 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1950000000000002e-06 2024-09-14T23:57:38.700 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.700 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.700 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:38.700 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.312218+0000", 2024-09-14T23:57:38.700 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046376873999999998 2024-09-14T23:57:38.700 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.700 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.700 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:38.700 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.312260+0000", 2024-09-14T23:57:38.700 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1693000000000002e-05 2024-09-14T23:57:38.700 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.700 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.701 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:38.701 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.312306+0000", 2024-09-14T23:57:38.701 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5571999999999997e-05 2024-09-14T23:57:38.701 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:38.701 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:38.701 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:38.701 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.701 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.701 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68750 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 7512064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:38.701 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.265863+0000", 2024-09-14T23:57:38.701 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.370317438000001, 2024-09-14T23:57:38.701 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046634766000000001, 2024-09-14T23:57:38.701 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:38.702 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:38.702 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:38.702 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:38.702 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:38.702 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:38.702 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68750 2024-09-14T23:57:38.702 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.702 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:38.702 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.702 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:38.702 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265863+0000", 2024-09-14T23:57:38.702 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:38.702 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.702 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.702 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:38.703 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265863+0000", 2024-09-14T23:57:38.703 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:38.703 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.703 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.703 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:38.703 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265863+0000", 2024-09-14T23:57:38.703 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0900000000000002e-07 2024-09-14T23:57:38.703 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.703 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.703 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:38.703 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265865+0000", 2024-09-14T23:57:38.703 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9740000000000001e-06 2024-09-14T23:57:38.704 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.704 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.704 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:38.704 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265869+0000", 2024-09-14T23:57:38.704 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4159999999999997e-06 2024-09-14T23:57:38.704 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.704 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.704 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:38.704 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265873+0000", 2024-09-14T23:57:38.704 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1999999999999999e-06 2024-09-14T23:57:38.704 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.704 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.704 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:38.704 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.312334+0000", 2024-09-14T23:57:38.705 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046461737000000003 2024-09-14T23:57:38.705 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.705 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.705 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:38.705 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.312415+0000", 2024-09-14T23:57:38.705 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0370999999999993e-05 2024-09-14T23:57:38.705 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.705 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.705 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:38.705 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.312497+0000", 2024-09-14T23:57:38.705 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2658999999999996e-05 2024-09-14T23:57:38.705 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:38.705 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:38.705 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:38.706 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.706 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.706 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68751 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 7626752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:38.706 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.265877+0000", 2024-09-14T23:57:38.706 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.370303036999999, 2024-09-14T23:57:38.706 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046759231999999998, 2024-09-14T23:57:38.706 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:38.706 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:38.706 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:38.706 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:38.706 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:38.707 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:38.707 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68751 2024-09-14T23:57:38.707 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.707 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:38.707 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.707 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:38.707 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265877+0000", 2024-09-14T23:57:38.707 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:38.707 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.707 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.707 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:38.707 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.265877+0000", 2024-09-14T23:57:38.708 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:38.708 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.708 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.708 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:38.708 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.267673+0000", 2024-09-14T23:57:38.708 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017956389999999999 2024-09-14T23:57:38.708 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.708 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.708 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:38.708 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.267677+0000", 2024-09-14T23:57:38.708 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4900000000000002e-06 2024-09-14T23:57:38.708 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.708 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.709 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:38.709 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.267688+0000", 2024-09-14T23:57:38.709 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1121000000000001e-05 2024-09-14T23:57:38.709 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.709 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.709 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:38.709 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.267694+0000", 2024-09-14T23:57:38.709 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5520000000000004e-06 2024-09-14T23:57:38.709 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.709 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.709 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:38.709 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.312534+0000", 2024-09-14T23:57:38.709 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044839641 2024-09-14T23:57:38.710 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.710 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.710 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:38.710 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.312586+0000", 2024-09-14T23:57:38.710 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2629e-05 2024-09-14T23:57:38.710 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.710 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.710 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:38.710 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.312636+0000", 2024-09-14T23:57:38.710 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0160000000000001e-05 2024-09-14T23:57:38.710 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:38.710 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:38.710 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:38.710 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.711 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.711 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68886 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 8060928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:38.711 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.273785+0000", 2024-09-14T23:57:38.711 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.362394635000001, 2024-09-14T23:57:38.711 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.045962955, 2024-09-14T23:57:38.711 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:38.711 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:38.711 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:38.711 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:38.711 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:38.711 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:38.711 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68886 2024-09-14T23:57:38.711 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.712 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:38.712 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.712 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:38.712 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273785+0000", 2024-09-14T23:57:38.712 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:38.712 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.712 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.712 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:38.712 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273785+0000", 2024-09-14T23:57:38.712 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:38.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.713 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.713 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:38.713 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273786+0000", 2024-09-14T23:57:38.713 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9000000000000002e-07 2024-09-14T23:57:38.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.713 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.713 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:38.713 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273788+0000", 2024-09-14T23:57:38.713 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9139999999999998e-06 2024-09-14T23:57:38.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.713 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.713 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:38.713 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273794+0000", 2024-09-14T23:57:38.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.212e-06 2024-09-14T23:57:38.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.714 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.714 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:38.714 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273799+0000", 2024-09-14T23:57:38.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5580000000000002e-06 2024-09-14T23:57:38.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.714 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.714 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:38.714 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.319646+0000", 2024-09-14T23:57:38.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.045847951999999997 2024-09-14T23:57:38.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.714 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.714 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:38.715 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.319707+0000", 2024-09-14T23:57:38.715 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.029e-05 2024-09-14T23:57:38.715 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.715 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.715 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:38.715 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.319748+0000", 2024-09-14T23:57:38.715 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1638999999999999e-05 2024-09-14T23:57:38.715 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:38.715 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:38.715 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:38.715 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.715 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.715 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68887 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 8175616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:38.715 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.273805+0000", 2024-09-14T23:57:38.715 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.362375375999999, 2024-09-14T23:57:38.716 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046062421999999999, 2024-09-14T23:57:38.716 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:38.716 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:38.716 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:38.716 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:38.716 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:38.716 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:38.716 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68887 2024-09-14T23:57:38.716 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.716 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:38.716 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.716 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:38.716 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273805+0000", 2024-09-14T23:57:38.716 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:38.717 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.717 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.717 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:38.717 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273805+0000", 2024-09-14T23:57:38.717 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:38.717 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.717 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.717 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:38.717 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273805+0000", 2024-09-14T23:57:38.717 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0699999999999998e-07 2024-09-14T23:57:38.717 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.717 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.717 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:38.718 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273815+0000", 2024-09-14T23:57:38.718 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7720000000000006e-06 2024-09-14T23:57:38.718 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.718 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.718 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:38.718 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273822+0000", 2024-09-14T23:57:38.718 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5850000000000002e-06 2024-09-14T23:57:38.718 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.718 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.718 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:38.718 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273833+0000", 2024-09-14T23:57:38.719 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0541e-05 2024-09-14T23:57:38.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.719 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.719 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:38.719 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.319779+0000", 2024-09-14T23:57:38.719 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.045946263000000001 2024-09-14T23:57:38.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.719 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.719 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:38.719 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.319824+0000", 2024-09-14T23:57:38.719 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4740000000000002e-05 2024-09-14T23:57:38.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.719 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.720 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:38.720 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.319867+0000", 2024-09-14T23:57:38.720 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3114000000000001e-05 2024-09-14T23:57:38.720 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:38.720 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:38.720 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:38.720 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.720 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.720 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:68888 3.a 3:51896f17:::rbd_data.11181e486ee6.0000000000000003:18 [read 8290304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:38.720 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.273854+0000", 2024-09-14T23:57:38.720 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.362326236000001, 2024-09-14T23:57:38.720 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046120520999999998, 2024-09-14T23:57:38.720 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:38.720 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:38.721 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:38.721 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:38.721 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:38.721 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:38.721 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 68888 2024-09-14T23:57:38.721 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.721 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:38.721 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.721 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:38.721 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273854+0000", 2024-09-14T23:57:38.721 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:38.721 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.721 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.722 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:38.722 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273854+0000", 2024-09-14T23:57:38.722 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:38.722 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.722 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.722 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:38.722 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273854+0000", 2024-09-14T23:57:38.722 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.58e-07 2024-09-14T23:57:38.722 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.722 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.722 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:38.722 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273862+0000", 2024-09-14T23:57:38.722 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.52e-06 2024-09-14T23:57:38.723 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.723 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.723 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:38.723 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273869+0000", 2024-09-14T23:57:38.723 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1180000000000003e-06 2024-09-14T23:57:38.723 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.723 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.723 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:38.723 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.273883+0000", 2024-09-14T23:57:38.723 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3709999999999999e-05 2024-09-14T23:57:38.724 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.724 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.724 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:38.724 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.319889+0000", 2024-09-14T23:57:38.724 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046005870999999997 2024-09-14T23:57:38.724 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.724 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.724 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:38.724 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.319930+0000", 2024-09-14T23:57:38.724 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1850000000000001e-05 2024-09-14T23:57:38.724 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.724 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.724 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:38.724 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.319974+0000", 2024-09-14T23:57:38.725 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3993999999999999e-05 2024-09-14T23:57:38.725 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:38.725 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:38.725 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:38.725 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.725 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.725 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:107472 3.1e 3:7bf21588:::rbd_data.11187de8b0a5.000000000000005d:head [set-alloc-hint object_size 2097152 write_size 2097152,write 317788~1491] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-14T23:57:38.725 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:25.198493+0000", 2024-09-14T23:57:38.725 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.437687134000001, 2024-09-14T23:57:38.725 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.78175149300000002, 2024-09-14T23:57:38.725 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:38.726 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:38.726 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:38.726 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:38.726 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:38.726 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:38.726 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 107472 2024-09-14T23:57:38.726 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.726 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:38.726 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.726 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:38.726 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198493+0000", 2024-09-14T23:57:38.726 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:38.726 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.726 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.726 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:38.727 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198493+0000", 2024-09-14T23:57:38.727 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:38.727 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.727 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.727 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:38.727 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198495+0000", 2024-09-14T23:57:38.727 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.826e-06 2024-09-14T23:57:38.727 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.727 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.727 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:38.727 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198503+0000", 2024-09-14T23:57:38.727 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0150000000000003e-06 2024-09-14T23:57:38.727 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.728 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.728 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:38.728 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198521+0000", 2024-09-14T23:57:38.728 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8374000000000001e-05 2024-09-14T23:57:38.728 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.728 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.728 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:38.728 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198533+0000", 2024-09-14T23:57:38.728 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1593000000000001e-05 2024-09-14T23:57:38.728 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.728 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.728 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:38.729 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198603+0000", 2024-09-14T23:57:38.729 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0279999999999998e-05 2024-09-14T23:57:38.729 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.729 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.729 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:57:38.729 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198617+0000", 2024-09-14T23:57:38.729 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3517e-05 2024-09-14T23:57:38.729 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.729 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.729 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:38.729 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.977067+0000", 2024-09-14T23:57:38.729 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.77845065800000002 2024-09-14T23:57:38.729 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.729 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.730 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for missing object", 2024-09-14T23:57:38.730 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.977173+0000", 2024-09-14T23:57:38.730 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010543999999999999 2024-09-14T23:57:38.730 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.730 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.730 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:38.730 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.978737+0000", 2024-09-14T23:57:38.730 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001564078 2024-09-14T23:57:38.730 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.730 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.730 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:38.730 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.978845+0000", 2024-09-14T23:57:38.730 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000108643 2024-09-14T23:57:38.730 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.731 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.731 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1", 2024-09-14T23:57:38.731 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.978964+0000", 2024-09-14T23:57:38.731 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000119085 2024-09-14T23:57:38.731 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.731 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.731 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:38.731 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980109+0000", 2024-09-14T23:57:38.731 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0011442170000000001 2024-09-14T23:57:38.731 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.731 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.731 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:57:38.731 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980130+0000", 2024-09-14T23:57:38.731 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1766000000000001e-05 2024-09-14T23:57:38.731 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.732 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.732 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:38.732 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980171+0000", 2024-09-14T23:57:38.732 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0627999999999998e-05 2024-09-14T23:57:38.732 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:38.732 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:38.732 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:38.732 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980244+0000", 2024-09-14T23:57:38.732 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3373000000000004e-05 2024-09-14T23:57:38.732 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:38.732 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:38.732 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:38.732 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:38.733 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:38.733 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:38.734 INFO:teuthology.orchestra.run.smithi042.stdout:3521 write 0x5e498fe thru 0x5e596c6 (0xfdc9 bytes) 2024-09-14T23:57:38.734 INFO:teuthology.orchestra.run.smithi042.stdout:3522 read 0x737a488 thru 0x737e41c (0x3f95 bytes) 2024-09-14T23:57:38.734 INFO:teuthology.orchestra.run.smithi042.stdout:3523 write 0x9a171f4 thru 0x9a17efe (0xd0b bytes) 2024-09-14T23:57:38.735 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:38.732+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:38.735 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T23:57:38.738 INFO:teuthology.orchestra.run.smithi042.stdout:3525 writesame 0xc659e60 thru 0xc663d62 data_size 0x3(0x9f03 bytes) 2024-09-14T23:57:38.745 INFO:teuthology.orchestra.run.smithi042.stdout:3527 write 0xde30045 thru 0xde3d73d (0xd6f9 bytes) 2024-09-14T23:57:38.748 INFO:teuthology.orchestra.run.smithi042.stdout:3528 writesame 0xb114ab4 thru 0xb115f66 data_size 0x1(0x14b3 bytes) 2024-09-14T23:57:38.753 INFO:teuthology.orchestra.run.smithi042.stdout:3531 read 0x9b5b793 thru 0x9b5e2ad (0x2b1b bytes) 2024-09-14T23:57:38.753 INFO:teuthology.orchestra.run.smithi042.stdout:3532 read 0x1591421 thru 0x1598741 (0x7321 bytes) 2024-09-14T23:57:38.757 INFO:teuthology.orchestra.run.smithi042.stdout:3534 trunc from 0xde3d73e to 0xa38d283 2024-09-14T23:57:38.771 INFO:teuthology.orchestra.run.smithi042.stdout:3536 read 0x7534d8e thru 0x75381c2 (0x3435 bytes) 2024-09-14T23:57:38.776 INFO:teuthology.orchestra.run.smithi042.stdout:3537 trunc from 0xa38d283 to 0x60fea3e 2024-09-14T23:57:38.795 INFO:teuthology.orchestra.run.smithi042.stdout:3539 trunc from 0x60fea3e to 0xdbd3d39 2024-09-14T23:57:38.800 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:38.801 INFO:teuthology.orchestra.run.smithi042.stdout:3542 write 0xc327e3e thru 0xc32b61d (0x37e0 bytes) 2024-09-14T23:57:38.804 INFO:teuthology.orchestra.run.smithi042.stdout:3543 read 0x943bccb thru 0x944477b (0x8ab1 bytes) 2024-09-14T23:57:38.804 INFO:teuthology.orchestra.run.smithi042.stdout:3544 write 0xd17ad2f thru 0xd189dee (0xf0c0 bytes) 2024-09-14T23:57:38.821 INFO:teuthology.orchestra.run.smithi042.stdout:3545 trunc from 0xdbd3d39 to 0x1ff655b 2024-09-14T23:57:38.834 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:38.833+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:38.853 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:38.853 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:38.853 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:57:38.853 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:38.858 INFO:teuthology.orchestra.run.smithi042.stdout:3546 punch from 0x1fb4803 to 0x1fc191c, (0xd119 bytes) 2024-09-14T23:57:38.858 INFO:teuthology.orchestra.run.smithi042.stdout:3548 read 0x1dd4195 thru 0x1dd59fc (0x1868 bytes) 2024-09-14T23:57:38.859 INFO:teuthology.orchestra.run.smithi042.stdout:3549 read 0x23d8af thru 0x2460bd (0x880f bytes) 2024-09-14T23:57:38.864 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T23:57:38.878 INFO:teuthology.orchestra.run.smithi042.stdout:3550 write 0xd563fbe thru 0xd5669e9 (0x2a2c bytes) 2024-09-14T23:57:38.881 INFO:teuthology.orchestra.run.smithi042.stdout:3553 read 0x6a34f0a thru 0x6a399f9 (0x4af0 bytes) 2024-09-14T23:57:38.882 INFO:teuthology.orchestra.run.smithi042.stdout:3554 punch from 0x674ae78 to 0x674aec4, (0x4c bytes) 2024-09-14T23:57:38.882 INFO:teuthology.orchestra.run.smithi042.stdout:3556 read 0x9d10dd3 thru 0x9d19e00 (0x902e bytes) 2024-09-14T23:57:38.882 INFO:teuthology.orchestra.run.smithi042.stdout:3557 punch from 0x9364488 to 0x936f80d, (0xb385 bytes) 2024-09-14T23:57:38.892 INFO:teuthology.orchestra.run.smithi042.stdout:3558 trunc from 0xd5669ea to 0x4210f10 2024-09-14T23:57:38.902 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:38.902 INFO:teuthology.orchestra.run.smithi042.stdout:3560 writesame 0x4c5762 thru 0x4d2d78 data_size 0x1(0xd617 bytes) 2024-09-14T23:57:38.949 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:38.948+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:39.032 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T23:57:39.036 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:39.035+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:39.038 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:39.038 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:39.038 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:57:39.038 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:57:39.038 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:39.047 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T23:57:39.051 INFO:teuthology.orchestra.run.smithi042.stdout:3561 trunc from 0x4210f10 to 0xbb1c34 2024-09-14T23:57:39.064 INFO:teuthology.orchestra.run.smithi042.stdout:3563 read 0xa802e2 thru 0xa900b0 (0xfdcf bytes) 2024-09-14T23:57:39.083 INFO:teuthology.orchestra.run.smithi042.stdout:3564 write 0xc8ff857 thru 0xc9050a2 (0x584c bytes) 2024-09-14T23:57:39.086 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:39.086 INFO:teuthology.orchestra.run.smithi042.stdout:3571 read 0xc26c863 thru 0xc26e8d2 (0x2070 bytes) 2024-09-14T23:57:39.087 INFO:teuthology.orchestra.run.smithi042.stdout:3572 write 0x8f1cde1 thru 0x8f1db06 (0xd26 bytes) 2024-09-14T23:57:39.090 INFO:teuthology.orchestra.run.smithi042.stdout:3574 punch from 0x44f32a2 to 0x44f4fe8, (0x1d46 bytes) 2024-09-14T23:57:39.090 INFO:teuthology.orchestra.run.smithi042.stdout:3577 read 0xa281282 thru 0xa284e59 (0x3bd8 bytes) 2024-09-14T23:57:39.090 INFO:teuthology.orchestra.run.smithi042.stdout:3578 punch from 0xa08114a to 0xa08cc59, (0xbb0f bytes) 2024-09-14T23:57:39.091 INFO:teuthology.orchestra.run.smithi042.stdout:3580 read 0x76665c2 thru 0x7673e19 (0xd858 bytes) 2024-09-14T23:57:39.091 INFO:teuthology.orchestra.run.smithi042.stdout:3581 read 0x5eb713f thru 0x5ec2b37 (0xb9f9 bytes) 2024-09-14T23:57:39.091 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:39.091 INFO:teuthology.orchestra.run.smithi042.stdout:3586 write 0x783914a thru 0x7841696 (0x854d bytes) 2024-09-14T23:57:39.105 INFO:teuthology.orchestra.run.smithi042.stdout:3587 trunc from 0xc9050a3 to 0x2cdfaeb 2024-09-14T23:57:39.133 INFO:teuthology.orchestra.run.smithi042.stdout:3588 write 0xd51fa9d thru 0xd520dfc (0x1360 bytes) 2024-09-14T23:57:39.136 INFO:teuthology.orchestra.run.smithi042.stdout:3590 read 0xaba1a6c thru 0xaba67ad (0x4d42 bytes) 2024-09-14T23:57:39.137 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:39.135+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:39.144 INFO:teuthology.orchestra.run.smithi042.stdout:3593 trunc from 0xd520dfd to 0x66ac63e 2024-09-14T23:57:39.160 INFO:teuthology.orchestra.run.smithi042.stdout:3594 trunc from 0x66ac63e to 0x10a306a 2024-09-14T23:57:39.166 INFO:teuthology.orchestra.run.smithi042.stdout:3595 read 0x8f6f8b thru 0x8fb2a2 (0x4318 bytes) 2024-09-14T23:57:39.166 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:39.178 INFO:teuthology.orchestra.run.smithi042.stdout:3597 writesame 0x5690350 thru 0x569e349 data_size 0x1(0xdffa bytes) 2024-09-14T23:57:39.178 INFO:teuthology.orchestra.run.smithi042.stdout:3598 clone 35 order 23 su 1048576 sc 8 2024-09-14T23:57:39.201 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:39.202 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:57:39.202 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:57:39.202 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:57:39.202 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.202 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69262 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 4800512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:39.202 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300811+0000", 2024-09-14T23:57:39.203 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.897544438000001, 2024-09-14T23:57:39.203 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050403796000000001, 2024-09-14T23:57:39.203 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:39.203 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:39.203 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:39.203 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:39.203 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:39.204 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:39.204 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69262 2024-09-14T23:57:39.204 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.204 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:39.204 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.205 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:39.205 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300811+0000", 2024-09-14T23:57:39.205 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:39.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.205 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.205 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:39.206 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300811+0000", 2024-09-14T23:57:39.206 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:39.206 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.206 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.206 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:39.206 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300811+0000", 2024-09-14T23:57:39.207 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5700000000000002e-07 2024-09-14T23:57:39.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.207 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.207 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:39.207 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300815+0000", 2024-09-14T23:57:39.207 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.087e-06 2024-09-14T23:57:39.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.208 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.208 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:39.208 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300823+0000", 2024-09-14T23:57:39.208 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8720000000000002e-06 2024-09-14T23:57:39.208 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.208 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.208 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:39.209 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300830+0000", 2024-09-14T23:57:39.209 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0589999999999997e-06 2024-09-14T23:57:39.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.209 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.209 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:39.210 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351020+0000", 2024-09-14T23:57:39.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050189552999999998 2024-09-14T23:57:39.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.210 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:39.210 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351112+0000", 2024-09-14T23:57:39.211 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2615999999999996e-05 2024-09-14T23:57:39.211 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.211 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.211 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:39.211 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351214+0000", 2024-09-14T23:57:39.212 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101952 2024-09-14T23:57:39.212 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:39.212 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:39.212 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:39.212 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.212 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.213 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69263 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 4915200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:39.213 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300842+0000", 2024-09-14T23:57:39.213 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.897512934000002, 2024-09-14T23:57:39.213 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050590637000000001, 2024-09-14T23:57:39.213 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:39.214 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:39.214 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:39.214 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:39.214 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:39.214 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:39.214 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69263 2024-09-14T23:57:39.215 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.215 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:39.215 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.215 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:39.215 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300842+0000", 2024-09-14T23:57:39.215 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:39.215 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.215 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.216 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:39.216 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300842+0000", 2024-09-14T23:57:39.216 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:39.216 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.216 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.216 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:39.216 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300843+0000", 2024-09-14T23:57:39.216 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4000000000000002e-07 2024-09-14T23:57:39.217 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.217 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.217 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:39.217 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300846+0000", 2024-09-14T23:57:39.217 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8720000000000002e-06 2024-09-14T23:57:39.217 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.217 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.217 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:39.218 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300853+0000", 2024-09-14T23:57:39.218 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0450000000000003e-06 2024-09-14T23:57:39.218 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.218 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.218 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:39.218 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300859+0000", 2024-09-14T23:57:39.218 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9510000000000003e-06 2024-09-14T23:57:39.219 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.219 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.219 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:39.219 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351260+0000", 2024-09-14T23:57:39.219 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050401654999999997 2024-09-14T23:57:39.219 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.219 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.220 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:39.220 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351353+0000", 2024-09-14T23:57:39.220 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2785e-05 2024-09-14T23:57:39.220 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.220 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.220 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:39.221 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351433+0000", 2024-09-14T23:57:39.221 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9789000000000005e-05 2024-09-14T23:57:39.221 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:39.221 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:39.221 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:39.221 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.222 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.222 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69264 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5029888~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:39.222 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300866+0000", 2024-09-14T23:57:39.222 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.897489001, 2024-09-14T23:57:39.222 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050794063, 2024-09-14T23:57:39.222 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:39.222 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:39.223 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:39.223 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:39.223 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:39.223 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:39.223 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69264 2024-09-14T23:57:39.223 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.223 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:39.223 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.223 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:39.223 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300866+0000", 2024-09-14T23:57:39.223 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:39.224 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.224 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.224 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:39.224 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300866+0000", 2024-09-14T23:57:39.224 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:39.224 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.224 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.224 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:39.224 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300866+0000", 2024-09-14T23:57:39.224 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8799999999999998e-07 2024-09-14T23:57:39.224 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.224 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.224 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:39.225 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300869+0000", 2024-09-14T23:57:39.225 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0880000000000002e-06 2024-09-14T23:57:39.225 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.225 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.225 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:39.225 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300875+0000", 2024-09-14T23:57:39.225 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9280000000000004e-06 2024-09-14T23:57:39.225 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.225 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.225 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:39.225 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300882+0000", 2024-09-14T23:57:39.225 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4740000000000002e-06 2024-09-14T23:57:39.225 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.225 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.226 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:39.226 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351472+0000", 2024-09-14T23:57:39.226 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050590006999999999 2024-09-14T23:57:39.226 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.226 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.226 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:39.226 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351563+0000", 2024-09-14T23:57:39.226 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0762999999999998e-05 2024-09-14T23:57:39.226 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.226 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.226 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:39.226 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351660+0000", 2024-09-14T23:57:39.226 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7415000000000002e-05 2024-09-14T23:57:39.226 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:39.227 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:39.227 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:39.227 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.227 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.227 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69265 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5144576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:39.227 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300890+0000", 2024-09-14T23:57:39.227 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.897465496999999, 2024-09-14T23:57:39.227 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050921389999999997, 2024-09-14T23:57:39.227 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:39.227 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:39.227 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:39.227 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:39.227 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:39.227 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:39.228 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69265 2024-09-14T23:57:39.228 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.228 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:39.228 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.228 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:39.228 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300890+0000", 2024-09-14T23:57:39.228 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:39.228 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.228 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.228 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:39.228 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300890+0000", 2024-09-14T23:57:39.228 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:39.228 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.229 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.229 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:39.229 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300890+0000", 2024-09-14T23:57:39.229 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2500000000000001e-07 2024-09-14T23:57:39.229 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.229 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.229 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:39.229 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300892+0000", 2024-09-14T23:57:39.229 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2469999999999998e-06 2024-09-14T23:57:39.229 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.229 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.230 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:39.230 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300899+0000", 2024-09-14T23:57:39.230 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5509999999999996e-06 2024-09-14T23:57:39.230 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.230 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.230 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:39.230 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300904+0000", 2024-09-14T23:57:39.230 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3299999999999998e-06 2024-09-14T23:57:39.230 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.230 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.230 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:39.230 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351707+0000", 2024-09-14T23:57:39.230 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050802673999999999 2024-09-14T23:57:39.230 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.231 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.231 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:39.231 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351755+0000", 2024-09-14T23:57:39.231 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8362000000000002e-05 2024-09-14T23:57:39.231 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.231 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.231 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:39.231 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351811+0000", 2024-09-14T23:57:39.231 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5801000000000003e-05 2024-09-14T23:57:39.231 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:39.231 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:39.231 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:39.231 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.232 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.232 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69266 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5259264~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:39.232 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300911+0000", 2024-09-14T23:57:39.232 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.897444424, 2024-09-14T23:57:39.232 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051125089999999998, 2024-09-14T23:57:39.232 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:39.232 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:39.232 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:39.232 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:39.232 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:39.232 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:39.232 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69266 2024-09-14T23:57:39.232 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.233 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:39.233 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.233 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:39.233 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300911+0000", 2024-09-14T23:57:39.233 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:39.233 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.233 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.233 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:39.233 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300911+0000", 2024-09-14T23:57:39.234 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:39.234 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.234 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.234 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:39.234 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300911+0000", 2024-09-14T23:57:39.234 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-14T23:57:39.235 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.235 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.235 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:39.235 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300913+0000", 2024-09-14T23:57:39.235 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9300000000000002e-06 2024-09-14T23:57:39.235 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.236 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.236 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:39.236 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300919+0000", 2024-09-14T23:57:39.236 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.489e-06 2024-09-14T23:57:39.236 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.236 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.237 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:39.237 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300925+0000", 2024-09-14T23:57:39.237 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1719999999999998e-06 2024-09-14T23:57:39.237 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.237 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.237 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:39.237 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351847+0000", 2024-09-14T23:57:39.237 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050922621000000001 2024-09-14T23:57:39.237 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.237 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.237 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:39.237 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351938+0000", 2024-09-14T23:57:39.237 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0830999999999998e-05 2024-09-14T23:57:39.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.238 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.238 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:39.238 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352036+0000", 2024-09-14T23:57:39.238 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7676999999999993e-05 2024-09-14T23:57:39.238 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:39.238 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:39.238 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:39.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.238 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.238 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69267 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5373952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:39.238 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300945+0000", 2024-09-14T23:57:39.238 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.897409628999998, 2024-09-14T23:57:39.238 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051217934, 2024-09-14T23:57:39.239 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:39.239 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:39.239 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:39.239 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:39.239 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:39.239 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:39.239 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69267 2024-09-14T23:57:39.239 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.239 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:39.239 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.239 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:39.239 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300945+0000", 2024-09-14T23:57:39.239 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:39.239 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.240 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.240 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:39.240 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300945+0000", 2024-09-14T23:57:39.240 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:39.240 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.240 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.240 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:39.240 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300946+0000", 2024-09-14T23:57:39.240 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2600000000000002e-07 2024-09-14T23:57:39.240 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.240 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.240 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:39.240 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300948+0000", 2024-09-14T23:57:39.240 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4940000000000002e-06 2024-09-14T23:57:39.241 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.241 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.241 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:39.241 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300956+0000", 2024-09-14T23:57:39.241 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.622e-06 2024-09-14T23:57:39.241 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.241 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.241 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:39.241 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300962+0000", 2024-09-14T23:57:39.241 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5790000000000003e-06 2024-09-14T23:57:39.241 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.241 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.242 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:39.242 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352063+0000", 2024-09-14T23:57:39.242 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051101281999999998 2024-09-14T23:57:39.242 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.242 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.242 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:39.242 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352110+0000", 2024-09-14T23:57:39.242 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7401000000000002e-05 2024-09-14T23:57:39.242 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.242 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.242 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:39.243 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352163+0000", 2024-09-14T23:57:39.243 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3029999999999999e-05 2024-09-14T23:57:39.243 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:39.243 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:39.243 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:39.243 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.243 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.243 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69268 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5488640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:39.243 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300969+0000", 2024-09-14T23:57:39.243 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.897385957000001, 2024-09-14T23:57:39.243 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051413420000000001, 2024-09-14T23:57:39.243 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:39.243 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:39.243 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:39.243 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:39.244 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:39.244 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:39.244 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69268 2024-09-14T23:57:39.244 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.244 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:39.244 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.244 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:39.244 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300969+0000", 2024-09-14T23:57:39.244 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:39.244 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.244 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.244 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:39.244 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300969+0000", 2024-09-14T23:57:39.244 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:39.245 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.245 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.245 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:39.245 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300970+0000", 2024-09-14T23:57:39.245 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7899999999999999e-07 2024-09-14T23:57:39.245 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.245 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.245 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:39.245 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300982+0000", 2024-09-14T23:57:39.245 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2741e-05 2024-09-14T23:57:39.245 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.245 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.245 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:39.246 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300990+0000", 2024-09-14T23:57:39.246 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8169999999999999e-06 2024-09-14T23:57:39.246 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.246 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.246 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:39.246 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300995+0000", 2024-09-14T23:57:39.246 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2240000000000003e-06 2024-09-14T23:57:39.246 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.246 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.246 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:39.246 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352200+0000", 2024-09-14T23:57:39.246 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051204223 2024-09-14T23:57:39.247 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.247 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.247 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:39.247 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352285+0000", 2024-09-14T23:57:39.247 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5519999999999994e-05 2024-09-14T23:57:39.248 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.248 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.248 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:39.248 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352383+0000", 2024-09-14T23:57:39.248 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7516000000000006e-05 2024-09-14T23:57:39.248 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:39.249 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:39.249 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:39.249 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.249 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.249 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69269 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5603328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:39.249 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.301003+0000", 2024-09-14T23:57:39.250 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.897351775000001, 2024-09-14T23:57:39.250 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051497381000000002, 2024-09-14T23:57:39.250 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:39.250 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:39.250 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:39.250 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:39.250 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:39.251 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:39.251 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69269 2024-09-14T23:57:39.251 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.251 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:39.251 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.251 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:39.251 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.301003+0000", 2024-09-14T23:57:39.252 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:39.252 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.252 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.252 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:39.252 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.301003+0000", 2024-09-14T23:57:39.252 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:39.253 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.253 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.253 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:39.253 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302671+0000", 2024-09-14T23:57:39.253 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001667785 2024-09-14T23:57:39.253 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.253 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.254 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:39.254 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302676+0000", 2024-09-14T23:57:39.254 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6600000000000003e-06 2024-09-14T23:57:39.254 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.254 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.254 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:39.254 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302697+0000", 2024-09-14T23:57:39.255 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1288999999999999e-05 2024-09-14T23:57:39.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.255 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:39.255 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302716+0000", 2024-09-14T23:57:39.255 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9128999999999999e-05 2024-09-14T23:57:39.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.256 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.256 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:39.256 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352399+0000", 2024-09-14T23:57:39.256 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049682995000000001 2024-09-14T23:57:39.256 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.256 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.257 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:39.257 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352446+0000", 2024-09-14T23:57:39.257 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6341000000000002e-05 2024-09-14T23:57:39.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.257 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.258 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:39.258 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352501+0000", 2024-09-14T23:57:39.258 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5182000000000001e-05 2024-09-14T23:57:39.258 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:39.258 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:39.258 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:39.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.259 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.259 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69273 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6062080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:39.259 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302829+0000", 2024-09-14T23:57:39.259 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.895526209, 2024-09-14T23:57:39.260 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050384311000000001, 2024-09-14T23:57:39.260 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:39.260 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:39.260 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:39.260 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:39.260 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:39.260 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:39.261 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69273 2024-09-14T23:57:39.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.261 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:39.261 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.261 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:39.261 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302829+0000", 2024-09-14T23:57:39.261 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:39.262 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.262 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.262 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:39.262 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302829+0000", 2024-09-14T23:57:39.262 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:39.262 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.263 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:39.263 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302829+0000", 2024-09-14T23:57:39.263 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7700000000000005e-07 2024-09-14T23:57:39.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.263 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:39.264 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302832+0000", 2024-09-14T23:57:39.264 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6970000000000002e-06 2024-09-14T23:57:39.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.264 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.264 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:39.265 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302840+0000", 2024-09-14T23:57:39.265 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5719999999999997e-06 2024-09-14T23:57:39.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.265 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:39.265 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302845+0000", 2024-09-14T23:57:39.266 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7939999999999999e-06 2024-09-14T23:57:39.266 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.266 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.266 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:39.266 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353111+0000", 2024-09-14T23:57:39.266 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050265588 2024-09-14T23:57:39.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.267 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.267 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:39.267 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353160+0000", 2024-09-14T23:57:39.267 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8859000000000001e-05 2024-09-14T23:57:39.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.268 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.268 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:39.268 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353213+0000", 2024-09-14T23:57:39.268 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3324e-05 2024-09-14T23:57:39.268 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:39.269 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:39.269 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:39.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.269 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.269 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69274 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6176768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:39.270 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302854+0000", 2024-09-14T23:57:39.270 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.895501114999998, 2024-09-14T23:57:39.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050579526, 2024-09-14T23:57:39.270 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:39.270 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:39.270 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:39.270 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:39.270 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:39.270 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:39.270 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69274 2024-09-14T23:57:39.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.270 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:39.270 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.271 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:39.271 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302854+0000", 2024-09-14T23:57:39.271 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:39.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.271 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.271 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:39.271 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302854+0000", 2024-09-14T23:57:39.271 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:39.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.271 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.271 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:39.271 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302855+0000", 2024-09-14T23:57:39.271 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0200000000000001e-07 2024-09-14T23:57:39.272 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.272 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.272 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:39.272 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302857+0000", 2024-09-14T23:57:39.272 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.447e-06 2024-09-14T23:57:39.272 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.272 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.272 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:39.272 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302864+0000", 2024-09-14T23:57:39.272 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7730000000000002e-06 2024-09-14T23:57:39.272 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.272 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.273 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:39.273 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302869+0000", 2024-09-14T23:57:39.273 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3199999999999999e-06 2024-09-14T23:57:39.273 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.273 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.273 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:39.273 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353249+0000", 2024-09-14T23:57:39.273 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050379647999999999 2024-09-14T23:57:39.273 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.273 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.274 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:39.274 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353336+0000", 2024-09-14T23:57:39.274 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6767999999999993e-05 2024-09-14T23:57:39.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.274 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:39.274 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353433+0000", 2024-09-14T23:57:39.274 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7868000000000006e-05 2024-09-14T23:57:39.274 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:39.274 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:39.274 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:39.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.274 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69275 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6291456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:39.275 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302889+0000", 2024-09-14T23:57:39.275 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.895466497000001, 2024-09-14T23:57:39.275 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050663118, 2024-09-14T23:57:39.275 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:39.275 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:39.275 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:39.275 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:39.275 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:39.275 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:39.275 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69275 2024-09-14T23:57:39.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.275 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:39.275 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.275 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:39.276 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302889+0000", 2024-09-14T23:57:39.276 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:39.276 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.276 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.276 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:39.276 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302889+0000", 2024-09-14T23:57:39.276 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:39.276 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.276 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.276 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:39.276 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302889+0000", 2024-09-14T23:57:39.276 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6399999999999998e-07 2024-09-14T23:57:39.276 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.277 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.277 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:39.277 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302891+0000", 2024-09-14T23:57:39.277 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.339e-06 2024-09-14T23:57:39.277 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.277 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.277 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:39.277 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302898+0000", 2024-09-14T23:57:39.277 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0380000000000002e-06 2024-09-14T23:57:39.277 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.277 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.277 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:39.277 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302904+0000", 2024-09-14T23:57:39.278 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2159999999999997e-06 2024-09-14T23:57:39.278 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.278 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.278 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:39.278 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353449+0000", 2024-09-14T23:57:39.278 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050545386999999997 2024-09-14T23:57:39.278 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.278 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.278 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:39.278 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353494+0000", 2024-09-14T23:57:39.278 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4592999999999998e-05 2024-09-14T23:57:39.278 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.278 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.278 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:39.279 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353552+0000", 2024-09-14T23:57:39.279 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8180999999999997e-05 2024-09-14T23:57:39.279 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:39.279 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:39.279 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:39.279 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.279 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.279 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69276 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6406144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:39.279 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302911+0000", 2024-09-14T23:57:39.279 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.895444084000001, 2024-09-14T23:57:39.279 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050865753, 2024-09-14T23:57:39.279 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:39.279 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:39.280 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:39.280 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:39.280 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:39.280 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:39.280 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69276 2024-09-14T23:57:39.280 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.280 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:39.281 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.281 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:39.281 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302911+0000", 2024-09-14T23:57:39.281 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:39.281 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.282 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.282 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:39.282 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302911+0000", 2024-09-14T23:57:39.282 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:39.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.282 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.283 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:39.283 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302912+0000", 2024-09-14T23:57:39.283 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0800000000000004e-07 2024-09-14T23:57:39.283 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.283 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.283 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:39.284 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302914+0000", 2024-09-14T23:57:39.284 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9589999999999998e-06 2024-09-14T23:57:39.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.284 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.284 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:39.285 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302920+0000", 2024-09-14T23:57:39.285 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4579999999999998e-06 2024-09-14T23:57:39.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.285 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.285 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:39.285 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302925+0000", 2024-09-14T23:57:39.286 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9429999999999999e-06 2024-09-14T23:57:39.286 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.286 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.286 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:39.286 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353586+0000", 2024-09-14T23:57:39.286 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050660966000000002 2024-09-14T23:57:39.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.287 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.287 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:39.287 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353679+0000", 2024-09-14T23:57:39.287 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3163000000000002e-05 2024-09-14T23:57:39.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.288 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.288 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:39.288 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353777+0000", 2024-09-14T23:57:39.288 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7555999999999999e-05 2024-09-14T23:57:39.288 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:39.288 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:39.289 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:39.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.289 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.289 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69277 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6520832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:39.289 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302932+0000", 2024-09-14T23:57:39.289 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.895422997000001, 2024-09-14T23:57:39.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050977674000000001, 2024-09-14T23:57:39.290 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:39.290 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:39.290 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:39.290 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:39.290 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:39.291 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:39.291 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69277 2024-09-14T23:57:39.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.291 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:39.291 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.291 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:39.291 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302932+0000", 2024-09-14T23:57:39.291 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:39.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.291 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.292 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:39.292 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302932+0000", 2024-09-14T23:57:39.292 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:39.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.292 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.292 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:39.292 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302932+0000", 2024-09-14T23:57:39.293 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3299999999999998e-07 2024-09-14T23:57:39.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.293 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.293 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:39.293 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302935+0000", 2024-09-14T23:57:39.293 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2139999999999999e-06 2024-09-14T23:57:39.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.294 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.294 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:39.294 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302941+0000", 2024-09-14T23:57:39.294 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5039999999999999e-06 2024-09-14T23:57:39.294 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.294 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.294 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:39.295 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302946+0000", 2024-09-14T23:57:39.295 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2079999999999999e-06 2024-09-14T23:57:39.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.295 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.295 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:39.295 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353805+0000", 2024-09-14T23:57:39.295 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050858588000000003 2024-09-14T23:57:39.296 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.296 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.296 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:39.296 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353849+0000", 2024-09-14T23:57:39.296 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4511000000000001e-05 2024-09-14T23:57:39.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.297 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.297 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:39.297 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353910+0000", 2024-09-14T23:57:39.297 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0316000000000003e-05 2024-09-14T23:57:39.298 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:39.298 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:39.298 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:39.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.298 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.299 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69278 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6635520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:39.299 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302953+0000", 2024-09-14T23:57:39.299 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.895401869000001, 2024-09-14T23:57:39.299 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051174059000000001, 2024-09-14T23:57:39.299 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:39.299 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:39.299 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:39.300 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:39.300 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:39.300 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:39.300 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69278 2024-09-14T23:57:39.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.300 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:39.300 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.301 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:39.301 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302953+0000", 2024-09-14T23:57:39.301 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:39.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.301 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.301 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:39.302 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302953+0000", 2024-09-14T23:57:39.302 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:39.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.302 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.302 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:39.302 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302954+0000", 2024-09-14T23:57:39.303 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5100000000000001e-07 2024-09-14T23:57:39.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.303 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:39.303 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302964+0000", 2024-09-14T23:57:39.303 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0475000000000001e-05 2024-09-14T23:57:39.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.304 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.304 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:39.304 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302970+0000", 2024-09-14T23:57:39.304 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3879999999999997e-06 2024-09-14T23:57:39.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.304 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.305 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:39.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302975+0000", 2024-09-14T23:57:39.305 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8409999999999999e-06 2024-09-14T23:57:39.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.306 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:39.306 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353944+0000", 2024-09-14T23:57:39.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050968952999999997 2024-09-14T23:57:39.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.306 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.306 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:39.307 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354031+0000", 2024-09-14T23:57:39.307 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6537000000000002e-05 2024-09-14T23:57:39.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.307 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.307 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:39.308 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354127+0000", 2024-09-14T23:57:39.308 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6514000000000005e-05 2024-09-14T23:57:39.308 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:39.308 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:39.308 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:39.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.308 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.308 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69279 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6750208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:39.308 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302984+0000", 2024-09-14T23:57:39.308 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.895371518000001, 2024-09-14T23:57:39.308 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051258845999999997, 2024-09-14T23:57:39.308 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:39.309 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:39.309 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:39.309 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:39.309 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:39.309 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:39.309 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69279 2024-09-14T23:57:39.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.309 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:39.309 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.309 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:39.309 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302984+0000", 2024-09-14T23:57:39.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:39.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.310 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:39.310 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302984+0000", 2024-09-14T23:57:39.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:39.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.310 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:39.310 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303025+0000", 2024-09-14T23:57:39.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1690999999999998e-05 2024-09-14T23:57:39.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.310 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:39.310 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303029+0000", 2024-09-14T23:57:39.311 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6559999999999998e-06 2024-09-14T23:57:39.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.311 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.311 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:39.311 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303037+0000", 2024-09-14T23:57:39.311 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7950000000000008e-06 2024-09-14T23:57:39.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.311 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.311 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:39.311 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303043+0000", 2024-09-14T23:57:39.311 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9669999999999998e-06 2024-09-14T23:57:39.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.311 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.312 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:39.312 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354142+0000", 2024-09-14T23:57:39.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051099072000000002 2024-09-14T23:57:39.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.312 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:39.312 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354183+0000", 2024-09-14T23:57:39.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0834000000000002e-05 2024-09-14T23:57:39.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.312 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:39.312 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354242+0000", 2024-09-14T23:57:39.313 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9830999999999997e-05 2024-09-14T23:57:39.313 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:39.313 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:39.313 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:39.313 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.313 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.313 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69280 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6864896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:39.313 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.303052+0000", 2024-09-14T23:57:39.313 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.895302975, 2024-09-14T23:57:39.313 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051406463999999999, 2024-09-14T23:57:39.313 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:39.313 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:39.313 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:39.314 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:39.314 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:39.314 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:39.314 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69280 2024-09-14T23:57:39.314 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.314 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:39.314 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.314 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:39.314 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303052+0000", 2024-09-14T23:57:39.314 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:39.314 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.314 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.314 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:39.314 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303052+0000", 2024-09-14T23:57:39.315 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:39.315 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.315 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.315 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:39.315 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304673+0000", 2024-09-14T23:57:39.315 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001621389 2024-09-14T23:57:39.315 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.315 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.315 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:39.315 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304681+0000", 2024-09-14T23:57:39.315 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8439999999999998e-06 2024-09-14T23:57:39.315 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.315 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.316 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:39.316 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304691+0000", 2024-09-14T23:57:39.316 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8339999999999994e-06 2024-09-14T23:57:39.316 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.316 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.316 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:39.316 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304698+0000", 2024-09-14T23:57:39.316 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3629999999999999e-06 2024-09-14T23:57:39.316 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.316 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.316 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:39.316 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354278+0000", 2024-09-14T23:57:39.316 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049580618 2024-09-14T23:57:39.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.317 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.317 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:39.317 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354364+0000", 2024-09-14T23:57:39.317 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6059999999999999e-05 2024-09-14T23:57:39.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.317 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.317 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:39.317 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354459+0000", 2024-09-14T23:57:39.317 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4356000000000003e-05 2024-09-14T23:57:39.317 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:39.317 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:39.317 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:39.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.318 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.318 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69285 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 7438336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:39.318 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.304792+0000", 2024-09-14T23:57:39.318 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.893563566000001, 2024-09-14T23:57:39.318 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050398874000000003, 2024-09-14T23:57:39.319 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:39.319 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:39.319 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:39.319 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:39.319 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:39.319 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:39.320 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69285 2024-09-14T23:57:39.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.320 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:39.320 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.320 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:39.321 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304792+0000", 2024-09-14T23:57:39.321 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:39.321 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.321 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.321 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:39.321 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304792+0000", 2024-09-14T23:57:39.322 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:39.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.322 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.322 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:39.322 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304792+0000", 2024-09-14T23:57:39.323 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2800000000000003e-07 2024-09-14T23:57:39.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.323 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.323 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:39.323 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304794+0000", 2024-09-14T23:57:39.323 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8700000000000001e-06 2024-09-14T23:57:39.324 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.324 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.324 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:39.324 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304800+0000", 2024-09-14T23:57:39.324 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0889999999999996e-06 2024-09-14T23:57:39.324 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.325 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.325 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:39.325 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304805+0000", 2024-09-14T23:57:39.325 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7060000000000003e-06 2024-09-14T23:57:39.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.325 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.326 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:39.326 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355108+0000", 2024-09-14T23:57:39.326 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050303170000000001 2024-09-14T23:57:39.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.326 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.326 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:39.327 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355150+0000", 2024-09-14T23:57:39.327 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2271999999999998e-05 2024-09-14T23:57:39.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.327 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.327 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:39.327 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355190+0000", 2024-09-14T23:57:39.327 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0438999999999997e-05 2024-09-14T23:57:39.327 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:39.327 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:39.327 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:39.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.328 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.328 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69286 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 7553024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:39.328 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.304811+0000", 2024-09-14T23:57:39.328 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.893544204000001, 2024-09-14T23:57:39.328 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050585512999999999, 2024-09-14T23:57:39.328 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:39.328 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:39.328 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:39.328 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:39.328 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:39.328 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:39.328 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69286 2024-09-14T23:57:39.328 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.328 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:39.329 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.329 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:39.329 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304811+0000", 2024-09-14T23:57:39.329 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:39.329 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.329 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.329 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:39.329 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304811+0000", 2024-09-14T23:57:39.329 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:39.329 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.329 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.329 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:39.329 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304811+0000", 2024-09-14T23:57:39.329 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.89e-07 2024-09-14T23:57:39.330 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.330 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.330 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:39.330 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304814+0000", 2024-09-14T23:57:39.330 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3199999999999998e-06 2024-09-14T23:57:39.330 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.330 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.330 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:39.330 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304821+0000", 2024-09-14T23:57:39.330 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5689999999999999e-06 2024-09-14T23:57:39.330 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.330 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.330 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:39.330 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304826+0000", 2024-09-14T23:57:39.331 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1200000000000001e-06 2024-09-14T23:57:39.331 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.331 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.331 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:39.331 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355226+0000", 2024-09-14T23:57:39.331 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050399653000000003 2024-09-14T23:57:39.331 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.331 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.331 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:39.331 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355314+0000", 2024-09-14T23:57:39.331 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8389999999999999e-05 2024-09-14T23:57:39.331 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.331 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.331 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:39.332 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355396+0000", 2024-09-14T23:57:39.332 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2071999999999996e-05 2024-09-14T23:57:39.332 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:39.332 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:39.332 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:39.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.332 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.332 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69287 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 7667712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:39.332 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.304833+0000", 2024-09-14T23:57:39.332 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.893521606, 2024-09-14T23:57:39.332 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050664740999999999, 2024-09-14T23:57:39.332 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:39.332 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:39.333 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:39.333 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:39.333 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:39.333 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:39.333 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69287 2024-09-14T23:57:39.333 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.333 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:39.333 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.333 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:39.333 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304833+0000", 2024-09-14T23:57:39.333 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:39.333 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.333 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.333 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:39.334 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304833+0000", 2024-09-14T23:57:39.334 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:39.334 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.334 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.334 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:39.334 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304834+0000", 2024-09-14T23:57:39.334 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4999999999999998e-07 2024-09-14T23:57:39.334 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.334 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.334 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:39.334 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304836+0000", 2024-09-14T23:57:39.334 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0279999999999999e-06 2024-09-14T23:57:39.334 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.334 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.335 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:39.335 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304842+0000", 2024-09-14T23:57:39.335 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8919999999999997e-06 2024-09-14T23:57:39.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.335 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.335 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:39.335 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304847+0000", 2024-09-14T23:57:39.335 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6160000000000001e-06 2024-09-14T23:57:39.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.335 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.335 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:39.335 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355411+0000", 2024-09-14T23:57:39.335 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050563540999999997 2024-09-14T23:57:39.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.336 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.336 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:39.336 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355459+0000", 2024-09-14T23:57:39.336 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7979000000000003e-05 2024-09-14T23:57:39.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.336 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.336 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:39.336 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355498+0000", 2024-09-14T23:57:39.336 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9335000000000002e-05 2024-09-14T23:57:39.336 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:39.336 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:39.336 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:39.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.337 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.337 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69288 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 7782400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:39.337 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.304854+0000", 2024-09-14T23:57:39.337 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.893501168, 2024-09-14T23:57:39.337 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050855145999999997, 2024-09-14T23:57:39.337 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:39.337 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:39.337 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:39.337 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:39.337 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:39.337 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:39.337 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69288 2024-09-14T23:57:39.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.337 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:39.338 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.338 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:39.338 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304854+0000", 2024-09-14T23:57:39.338 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:39.338 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.338 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.338 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:39.338 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304854+0000", 2024-09-14T23:57:39.338 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:39.338 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.338 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.338 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:39.338 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.306670+0000", 2024-09-14T23:57:39.338 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00181631 2024-09-14T23:57:39.339 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.339 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.339 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:39.339 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.306675+0000", 2024-09-14T23:57:39.339 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0220000000000004e-06 2024-09-14T23:57:39.339 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.339 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.339 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:39.339 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.306684+0000", 2024-09-14T23:57:39.339 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7309999999999999e-06 2024-09-14T23:57:39.339 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.339 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.339 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:39.339 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.306689+0000", 2024-09-14T23:57:39.340 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6609999999999999e-06 2024-09-14T23:57:39.340 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.340 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.340 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:39.340 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355533+0000", 2024-09-14T23:57:39.340 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048844578 2024-09-14T23:57:39.340 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.340 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.340 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:39.340 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355622+0000", 2024-09-14T23:57:39.340 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9006999999999997e-05 2024-09-14T23:57:39.340 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:39.340 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:39.340 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:39.341 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355709+0000", 2024-09-14T23:57:39.341 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6836999999999995e-05 2024-09-14T23:57:39.341 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:39.341 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:39.341 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:39.341 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:39.341 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:39.341 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:39.342 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:39.341+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:39.343 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T23:57:39.444 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:39.442+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:39.450 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:39.450 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:57:39.450 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:57:39.450 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:39.459 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T23:57:39.545 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:39.544+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:39.566 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:39.566 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:57:39.566 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:57:39.566 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:57:39.567 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:39.575 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T23:57:39.646 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:39.645+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:39.684 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:39.684 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:57:39.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:57:39.685 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:57:39.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.685 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:43054 3.1f 3:ffbfe596:::rbd_data.1118350d5ae5.00000000000002eb:head [set-alloc-hint object_size 262144 write_size 262144,write 196608~22770] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T23:57:39.685 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:57:39.685 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.715634573000003, 2024-09-14T23:57:39.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.99973334400000002, 2024-09-14T23:57:39.685 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:39.685 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:39.685 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:39.685 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:39.685 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:39.685 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:39.686 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 43054 2024-09-14T23:57:39.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.686 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:39.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.686 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:39.686 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:57:39.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:39.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.686 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:39.686 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:57:39.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:39.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.687 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:39.687 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966552+0000", 2024-09-14T23:57:39.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3790000000000001e-06 2024-09-14T23:57:39.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.687 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:39.687 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966565+0000", 2024-09-14T23:57:39.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2459999999999999e-05 2024-09-14T23:57:39.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.687 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:39.687 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966576+0000", 2024-09-14T23:57:39.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1039000000000001e-05 2024-09-14T23:57:39.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.688 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:39.688 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966582+0000", 2024-09-14T23:57:39.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9660000000000001e-06 2024-09-14T23:57:39.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.688 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:39.688 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966625+0000", 2024-09-14T23:57:39.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3529999999999998e-05 2024-09-14T23:57:39.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.688 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:57:39.688 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966632+0000", 2024-09-14T23:57:39.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9809999999999997e-06 2024-09-14T23:57:39.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.689 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:39.689 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.964460+0000", 2024-09-14T23:57:39.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.99782768300000002 2024-09-14T23:57:39.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.689 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:39.689 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.964641+0000", 2024-09-14T23:57:39.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000181089 2024-09-14T23:57:39.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.690 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-14T23:57:39.690 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.964809+0000", 2024-09-14T23:57:39.690 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000168505 2024-09-14T23:57:39.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.690 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:57:39.690 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966030+0000", 2024-09-14T23:57:39.690 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012206929999999999 2024-09-14T23:57:39.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.690 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:39.690 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966181+0000", 2024-09-14T23:57:39.690 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000150569 2024-09-14T23:57:39.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.691 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:39.691 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966243+0000", 2024-09-14T23:57:39.691 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2331999999999999e-05 2024-09-14T23:57:39.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.691 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:39.691 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966284+0000", 2024-09-14T23:57:39.691 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1118000000000002e-05 2024-09-14T23:57:39.691 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:39.691 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:39.691 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:39.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.692 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:54673 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:39.692 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.167989+0000", 2024-09-14T23:57:39.692 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 33.514196222999999, 2024-09-14T23:57:39.692 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027580298, 2024-09-14T23:57:39.692 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:39.692 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:39.692 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:39.692 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:39.692 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:39.692 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:39.692 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54673 2024-09-14T23:57:39.692 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.692 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:39.693 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.693 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:39.693 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.167989+0000", 2024-09-14T23:57:39.693 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:39.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.693 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.693 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:39.693 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.167989+0000", 2024-09-14T23:57:39.693 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:39.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.693 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.693 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:39.693 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.167989+0000", 2024-09-14T23:57:39.693 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5499999999999998e-07 2024-09-14T23:57:39.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:39.694 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.167998+0000", 2024-09-14T23:57:39.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7369999999999995e-06 2024-09-14T23:57:39.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:39.694 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168005+0000", 2024-09-14T23:57:39.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8210000000000004e-06 2024-09-14T23:57:39.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:39.695 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168010+0000", 2024-09-14T23:57:39.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2510000000000002e-06 2024-09-14T23:57:39.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.695 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:39.695 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.195432+0000", 2024-09-14T23:57:39.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027421364 2024-09-14T23:57:39.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.695 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:39.695 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.195501+0000", 2024-09-14T23:57:39.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9611999999999997e-05 2024-09-14T23:57:39.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.696 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:39.696 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.195569+0000", 2024-09-14T23:57:39.696 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8058000000000005e-05 2024-09-14T23:57:39.696 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:39.696 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:39.696 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:39.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.696 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:54674 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 2637824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:39.696 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.168018+0000", 2024-09-14T23:57:39.696 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 33.514167579000002, 2024-09-14T23:57:39.696 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027716159000000001, 2024-09-14T23:57:39.696 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:39.696 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:39.697 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:39.697 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:39.697 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:39.697 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:39.697 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54674 2024-09-14T23:57:39.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.697 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:39.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.697 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:39.697 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168018+0000", 2024-09-14T23:57:39.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:39.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.697 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:39.698 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168018+0000", 2024-09-14T23:57:39.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:39.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.698 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:39.698 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168018+0000", 2024-09-14T23:57:39.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2600000000000002e-07 2024-09-14T23:57:39.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.698 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:39.698 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168020+0000", 2024-09-14T23:57:39.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9420000000000002e-06 2024-09-14T23:57:39.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.699 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:39.699 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168026+0000", 2024-09-14T23:57:39.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8880000000000002e-06 2024-09-14T23:57:39.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.699 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:39.699 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168031+0000", 2024-09-14T23:57:39.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.977e-06 2024-09-14T23:57:39.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.699 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:39.699 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.195599+0000", 2024-09-14T23:57:39.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027567758000000001 2024-09-14T23:57:39.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.700 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:39.700 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.195666+0000", 2024-09-14T23:57:39.700 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7102999999999998e-05 2024-09-14T23:57:39.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.700 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:39.700 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.195734+0000", 2024-09-14T23:57:39.700 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7865000000000002e-05 2024-09-14T23:57:39.700 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:39.700 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:39.700 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:39.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.701 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:54675 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:39.701 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.168039+0000", 2024-09-14T23:57:39.701 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 33.514146619999998, 2024-09-14T23:57:39.701 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027868378999999999, 2024-09-14T23:57:39.701 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:39.701 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:39.701 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:39.701 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:39.701 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:39.701 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:39.701 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54675 2024-09-14T23:57:39.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.701 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:39.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:39.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168039+0000", 2024-09-14T23:57:39.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:39.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:39.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168039+0000", 2024-09-14T23:57:39.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:39.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:39.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168039+0000", 2024-09-14T23:57:39.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3099999999999998e-07 2024-09-14T23:57:39.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.703 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:39.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168048+0000", 2024-09-14T23:57:39.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9460000000000004e-06 2024-09-14T23:57:39.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.703 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:39.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168055+0000", 2024-09-14T23:57:39.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5740000000000001e-06 2024-09-14T23:57:39.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.703 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:39.704 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168060+0000", 2024-09-14T23:57:39.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1909999999999999e-06 2024-09-14T23:57:39.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.704 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:39.704 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.195765+0000", 2024-09-14T23:57:39.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027705244 2024-09-14T23:57:39.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.704 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:39.704 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.195836+0000", 2024-09-14T23:57:39.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0642999999999994e-05 2024-09-14T23:57:39.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.705 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:39.705 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.195907+0000", 2024-09-14T23:57:39.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.135e-05 2024-09-14T23:57:39.705 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:39.705 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:39.705 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:39.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.705 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:54676 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 2998272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:39.705 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.168067+0000", 2024-09-14T23:57:39.705 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 33.514117886999998, 2024-09-14T23:57:39.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028002955999999999, 2024-09-14T23:57:39.705 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:39.706 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:39.706 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:39.706 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:39.706 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:39.706 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:39.706 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54676 2024-09-14T23:57:39.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.706 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:39.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:39.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168067+0000", 2024-09-14T23:57:39.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:39.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.707 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:39.707 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168067+0000", 2024-09-14T23:57:39.707 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:39.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.707 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:39.707 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168068+0000", 2024-09-14T23:57:39.707 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6899999999999998e-07 2024-09-14T23:57:39.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.707 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:39.707 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168070+0000", 2024-09-14T23:57:39.707 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9810000000000002e-06 2024-09-14T23:57:39.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:39.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168076+0000", 2024-09-14T23:57:39.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4280000000000001e-06 2024-09-14T23:57:39.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:39.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168091+0000", 2024-09-14T23:57:39.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5152e-05 2024-09-14T23:57:39.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:39.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.195936+0000", 2024-09-14T23:57:39.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027844892 2024-09-14T23:57:39.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:39.709 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196002+0000", 2024-09-14T23:57:39.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5633e-05 2024-09-14T23:57:39.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:39.709 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196070+0000", 2024-09-14T23:57:39.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8400999999999998e-05 2024-09-14T23:57:39.709 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:39.709 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:39.709 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:39.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.710 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:54677 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 3178496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:39.710 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.168099+0000", 2024-09-14T23:57:39.710 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 33.514086587999998, 2024-09-14T23:57:39.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028165816, 2024-09-14T23:57:39.710 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:39.710 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:39.710 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:39.710 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:39.710 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:39.710 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:39.710 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54677 2024-09-14T23:57:39.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.711 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:39.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:39.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168099+0000", 2024-09-14T23:57:39.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:39.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:39.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168099+0000", 2024-09-14T23:57:39.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:39.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:39.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168099+0000", 2024-09-14T23:57:39.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7400000000000003e-07 2024-09-14T23:57:39.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.712 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.712 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:39.712 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168102+0000", 2024-09-14T23:57:39.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2929999999999999e-06 2024-09-14T23:57:39.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.712 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.712 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:39.712 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168108+0000", 2024-09-14T23:57:39.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5429999999999999e-06 2024-09-14T23:57:39.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.712 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.712 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:39.713 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168113+0000", 2024-09-14T23:57:39.713 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8470000000000003e-06 2024-09-14T23:57:39.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.713 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:39.713 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196124+0000", 2024-09-14T23:57:39.713 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028011034000000001 2024-09-14T23:57:39.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.714 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:39.714 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196194+0000", 2024-09-14T23:57:39.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0314000000000005e-05 2024-09-14T23:57:39.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.714 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:39.714 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196264+0000", 2024-09-14T23:57:39.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0210999999999996e-05 2024-09-14T23:57:39.714 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:39.714 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:39.714 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:39.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.715 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:54678 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 3358720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:39.715 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.168144+0000", 2024-09-14T23:57:39.715 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 33.514041429000002, 2024-09-14T23:57:39.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028285267999999999, 2024-09-14T23:57:39.715 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:39.715 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:39.715 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:39.715 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:39.715 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:39.715 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:39.715 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54678 2024-09-14T23:57:39.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.715 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:39.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.716 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:39.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168144+0000", 2024-09-14T23:57:39.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:39.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.716 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:39.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168144+0000", 2024-09-14T23:57:39.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:39.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.716 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:39.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168144+0000", 2024-09-14T23:57:39.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5799999999999999e-07 2024-09-14T23:57:39.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.717 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.717 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:39.717 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168147+0000", 2024-09-14T23:57:39.717 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3580000000000001e-06 2024-09-14T23:57:39.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.717 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.717 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:39.717 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168154+0000", 2024-09-14T23:57:39.717 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1350000000000003e-06 2024-09-14T23:57:39.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.717 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.717 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:39.717 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168159+0000", 2024-09-14T23:57:39.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4160000000000003e-06 2024-09-14T23:57:39.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:39.718 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196294+0000", 2024-09-14T23:57:39.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028135050000000002 2024-09-14T23:57:39.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:39.718 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196363+0000", 2024-09-14T23:57:39.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8880000000000005e-05 2024-09-14T23:57:39.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:39.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196429+0000", 2024-09-14T23:57:39.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5870999999999993e-05 2024-09-14T23:57:39.719 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:39.719 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:39.719 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:39.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.720 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:54679 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:39.720 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.168167+0000", 2024-09-14T23:57:39.720 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 33.514018208000003, 2024-09-14T23:57:39.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028427019000000001, 2024-09-14T23:57:39.720 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:39.720 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:39.720 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:39.720 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:39.720 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:39.720 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:39.720 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54679 2024-09-14T23:57:39.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.720 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:39.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.721 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:39.721 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168167+0000", 2024-09-14T23:57:39.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:39.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.721 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:39.721 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168167+0000", 2024-09-14T23:57:39.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:39.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.721 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:39.721 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168168+0000", 2024-09-14T23:57:39.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.44e-07 2024-09-14T23:57:39.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.722 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:39.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168170+0000", 2024-09-14T23:57:39.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0870000000000001e-06 2024-09-14T23:57:39.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.722 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:39.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168176+0000", 2024-09-14T23:57:39.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9070000000000004e-06 2024-09-14T23:57:39.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.722 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:39.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168181+0000", 2024-09-14T23:57:39.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6579999999999999e-06 2024-09-14T23:57:39.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.723 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:39.723 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196458+0000", 2024-09-14T23:57:39.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028276431000000001 2024-09-14T23:57:39.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.723 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:39.723 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196527+0000", 2024-09-14T23:57:39.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9397e-05 2024-09-14T23:57:39.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.723 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:39.723 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196594+0000", 2024-09-14T23:57:39.724 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6895000000000002e-05 2024-09-14T23:57:39.724 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:39.724 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:39.724 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:39.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.724 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.724 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:54680 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 3719168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:39.724 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.168188+0000", 2024-09-14T23:57:39.724 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 33.513997001999996, 2024-09-14T23:57:39.724 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028579706999999999, 2024-09-14T23:57:39.724 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:39.724 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:39.724 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:39.725 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:39.725 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:39.725 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:39.725 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54680 2024-09-14T23:57:39.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.725 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:39.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.725 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:39.725 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168188+0000", 2024-09-14T23:57:39.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:39.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.725 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:39.725 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168188+0000", 2024-09-14T23:57:39.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:39.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.726 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:39.726 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168189+0000", 2024-09-14T23:57:39.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3099999999999998e-07 2024-09-14T23:57:39.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.726 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:39.726 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168191+0000", 2024-09-14T23:57:39.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0470000000000001e-06 2024-09-14T23:57:39.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.726 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:39.726 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168199+0000", 2024-09-14T23:57:39.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8809999999999996e-06 2024-09-14T23:57:39.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:39.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168204+0000", 2024-09-14T23:57:39.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.011e-06 2024-09-14T23:57:39.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:39.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196622+0000", 2024-09-14T23:57:39.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028418806000000001 2024-09-14T23:57:39.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:39.728 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196694+0000", 2024-09-14T23:57:39.728 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1124000000000005e-05 2024-09-14T23:57:39.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.728 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:39.728 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196768+0000", 2024-09-14T23:57:39.728 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4406999999999994e-05 2024-09-14T23:57:39.728 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:39.728 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:39.728 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:39.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.728 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:54681 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 3899392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:39.729 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.168212+0000", 2024-09-14T23:57:39.729 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 33.513973290000003, 2024-09-14T23:57:39.729 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028726643, 2024-09-14T23:57:39.729 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:39.729 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:39.729 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:39.729 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:39.729 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:39.729 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:39.729 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54681 2024-09-14T23:57:39.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.730 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:39.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.730 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:39.730 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168212+0000", 2024-09-14T23:57:39.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:39.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.730 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:39.730 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168212+0000", 2024-09-14T23:57:39.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:39.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.730 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:39.730 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168212+0000", 2024-09-14T23:57:39.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4900000000000001e-07 2024-09-14T23:57:39.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:39.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168214+0000", 2024-09-14T23:57:39.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0590000000000001e-06 2024-09-14T23:57:39.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:39.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168220+0000", 2024-09-14T23:57:39.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0209999999999996e-06 2024-09-14T23:57:39.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:39.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168227+0000", 2024-09-14T23:57:39.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4119999999999998e-06 2024-09-14T23:57:39.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.732 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:39.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196799+0000", 2024-09-14T23:57:39.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028572528 2024-09-14T23:57:39.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.732 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:39.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196866+0000", 2024-09-14T23:57:39.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7005e-05 2024-09-14T23:57:39.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.733 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:39.733 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196939+0000", 2024-09-14T23:57:39.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2169000000000006e-05 2024-09-14T23:57:39.733 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:39.733 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:39.733 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:39.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.733 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:54682 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 4079616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:39.733 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.168234+0000", 2024-09-14T23:57:39.733 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 33.513951741, 2024-09-14T23:57:39.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028872503000000001, 2024-09-14T23:57:39.733 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:39.734 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:39.734 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:39.734 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:39.734 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:39.734 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:39.734 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54682 2024-09-14T23:57:39.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.734 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:39.734 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.736 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:39.736 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168234+0000", 2024-09-14T23:57:39.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:39.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.736 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.736 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:39.736 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168234+0000", 2024-09-14T23:57:39.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:39.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.736 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.736 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:39.736 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168234+0000", 2024-09-14T23:57:39.737 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.58e-07 2024-09-14T23:57:39.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.737 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:39.737 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168236+0000", 2024-09-14T23:57:39.737 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.933e-06 2024-09-14T23:57:39.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.737 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:39.737 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168242+0000", 2024-09-14T23:57:39.737 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9970000000000004e-06 2024-09-14T23:57:39.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.738 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.738 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:39.738 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168247+0000", 2024-09-14T23:57:39.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9280000000000001e-06 2024-09-14T23:57:39.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.738 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.738 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:39.738 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196968+0000", 2024-09-14T23:57:39.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028721159999999999 2024-09-14T23:57:39.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.738 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.738 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:39.738 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.197040+0000", 2024-09-14T23:57:39.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1569999999999994e-05 2024-09-14T23:57:39.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.739 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:39.739 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.197106+0000", 2024-09-14T23:57:39.739 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6457000000000004e-05 2024-09-14T23:57:39.739 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:39.739 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:39.739 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:39.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.739 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:55239 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 3850240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:39.739 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.201093+0000", 2024-09-14T23:57:39.739 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 33.481092289999999, 2024-09-14T23:57:39.739 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027610927, 2024-09-14T23:57:39.740 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:39.740 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:39.740 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:39.740 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:39.740 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:39.740 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:39.740 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 55239 2024-09-14T23:57:39.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.740 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:39.740 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.740 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:39.740 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201093+0000", 2024-09-14T23:57:39.740 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:39.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.741 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:39.741 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201093+0000", 2024-09-14T23:57:39.741 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:39.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.741 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:39.741 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201093+0000", 2024-09-14T23:57:39.741 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5699999999999998e-07 2024-09-14T23:57:39.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.741 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:39.741 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201096+0000", 2024-09-14T23:57:39.741 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0990000000000001e-06 2024-09-14T23:57:39.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.742 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:39.742 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201102+0000", 2024-09-14T23:57:39.742 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5710000000000003e-06 2024-09-14T23:57:39.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.742 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:39.742 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201107+0000", 2024-09-14T23:57:39.742 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1019999999999996e-06 2024-09-14T23:57:39.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.742 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:39.742 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.228568+0000", 2024-09-14T23:57:39.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027461136000000001 2024-09-14T23:57:39.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:39.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.228638+0000", 2024-09-14T23:57:39.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0024999999999994e-05 2024-09-14T23:57:39.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:39.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.228704+0000", 2024-09-14T23:57:39.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5537000000000006e-05 2024-09-14T23:57:39.743 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:39.743 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:39.743 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:39.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.744 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:55240 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:39.744 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.201114+0000", 2024-09-14T23:57:39.744 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 33.481071053999997, 2024-09-14T23:57:39.744 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027752295, 2024-09-14T23:57:39.744 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:39.744 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:39.744 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:39.744 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:39.744 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:39.744 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:39.744 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 55240 2024-09-14T23:57:39.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.745 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:39.745 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.745 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:39.745 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201114+0000", 2024-09-14T23:57:39.745 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:39.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.745 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.745 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:39.745 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201114+0000", 2024-09-14T23:57:39.745 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:39.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.745 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.745 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:39.746 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201115+0000", 2024-09-14T23:57:39.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4200000000000001e-07 2024-09-14T23:57:39.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.746 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:39.746 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201117+0000", 2024-09-14T23:57:39.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.368e-06 2024-09-14T23:57:39.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.746 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:39.746 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201124+0000", 2024-09-14T23:57:39.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4999999999999996e-06 2024-09-14T23:57:39.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.747 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:39.747 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201136+0000", 2024-09-14T23:57:39.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.293e-05 2024-09-14T23:57:39.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.747 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:39.747 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.228736+0000", 2024-09-14T23:57:39.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027599597 2024-09-14T23:57:39.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.747 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:39.747 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.228806+0000", 2024-09-14T23:57:39.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9616999999999994e-05 2024-09-14T23:57:39.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.748 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:39.748 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.228866+0000", 2024-09-14T23:57:39.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0841000000000002e-05 2024-09-14T23:57:39.748 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:39.748 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:39.748 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:39.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.748 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:55349 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 868352~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:39.748 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.204621+0000", 2024-09-14T23:57:39.748 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 33.477563887999999, 2024-09-14T23:57:39.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030294596999999999, 2024-09-14T23:57:39.749 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:39.749 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:39.749 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:39.749 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:39.749 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:39.749 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:39.749 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 55349 2024-09-14T23:57:39.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.749 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:39.749 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.749 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:39.749 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204621+0000", 2024-09-14T23:57:39.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:39.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.750 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.750 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:39.750 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204621+0000", 2024-09-14T23:57:39.750 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:39.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.750 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.750 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:39.750 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204622+0000", 2024-09-14T23:57:39.750 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.58e-07 2024-09-14T23:57:39.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.750 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.750 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:39.750 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204624+0000", 2024-09-14T23:57:39.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.193e-06 2024-09-14T23:57:39.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.751 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:39.751 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204631+0000", 2024-09-14T23:57:39.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1910000000000003e-06 2024-09-14T23:57:39.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.751 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:39.751 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204637+0000", 2024-09-14T23:57:39.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5169999999999999e-06 2024-09-14T23:57:39.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.751 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:39.751 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.229653+0000", 2024-09-14T23:57:39.752 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025016283 2024-09-14T23:57:39.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.752 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.752 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:39.752 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.229721+0000", 2024-09-14T23:57:39.752 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8138000000000004e-05 2024-09-14T23:57:39.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.752 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.752 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:39.752 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.234916+0000", 2024-09-14T23:57:39.752 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.005194817 2024-09-14T23:57:39.752 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:39.752 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:39.753 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:39.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.753 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:55363 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 3145728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:39.753 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.204982+0000", 2024-09-14T23:57:39.753 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 33.477203031999998, 2024-09-14T23:57:39.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027646239999999999, 2024-09-14T23:57:39.753 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:39.753 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:39.754 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:39.754 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:39.754 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:39.754 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:39.754 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 55363 2024-09-14T23:57:39.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.754 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:39.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.754 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:39.754 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204982+0000", 2024-09-14T23:57:39.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:39.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.755 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:39.755 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204982+0000", 2024-09-14T23:57:39.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:39.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.755 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:39.755 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204983+0000", 2024-09-14T23:57:39.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2199999999999999e-07 2024-09-14T23:57:39.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.755 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:39.755 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204985+0000", 2024-09-14T23:57:39.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.261e-06 2024-09-14T23:57:39.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.756 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:39.756 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204991+0000", 2024-09-14T23:57:39.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7069999999999997e-06 2024-09-14T23:57:39.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.756 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:39.756 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204995+0000", 2024-09-14T23:57:39.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.853e-06 2024-09-14T23:57:39.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.756 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:39.756 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.232491+0000", 2024-09-14T23:57:39.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027495322999999999 2024-09-14T23:57:39.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.757 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:39.757 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.232563+0000", 2024-09-14T23:57:39.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1972999999999997e-05 2024-09-14T23:57:39.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.757 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:39.757 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.232628+0000", 2024-09-14T23:57:39.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5701e-05 2024-09-14T23:57:39.757 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:39.757 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:39.757 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:39.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.758 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:55364 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 3325952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:39.758 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.205003+0000", 2024-09-14T23:57:39.758 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 33.477182507000002, 2024-09-14T23:57:39.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027781963, 2024-09-14T23:57:39.758 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:39.758 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:39.758 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:39.758 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:39.758 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:39.758 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:39.758 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 55364 2024-09-14T23:57:39.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.758 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:39.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:39.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205003+0000", 2024-09-14T23:57:39.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:39.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:39.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205003+0000", 2024-09-14T23:57:39.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:39.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:39.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205003+0000", 2024-09-14T23:57:39.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2899999999999999e-07 2024-09-14T23:57:39.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.760 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:39.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205013+0000", 2024-09-14T23:57:39.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.611e-06 2024-09-14T23:57:39.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.760 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:39.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205020+0000", 2024-09-14T23:57:39.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2509999999999997e-06 2024-09-14T23:57:39.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.760 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:39.761 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205025+0000", 2024-09-14T23:57:39.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1399999999999999e-06 2024-09-14T23:57:39.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.761 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:39.761 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.232653+0000", 2024-09-14T23:57:39.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027627972000000001 2024-09-14T23:57:39.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.761 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:39.761 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.232723+0000", 2024-09-14T23:57:39.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9907999999999995e-05 2024-09-14T23:57:39.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:39.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.232785+0000", 2024-09-14T23:57:39.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1651999999999999e-05 2024-09-14T23:57:39.762 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:39.762 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:39.762 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:39.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.762 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:55365 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 3506176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:39.762 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.205033+0000", 2024-09-14T23:57:39.762 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 33.477152267999998, 2024-09-14T23:57:39.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027914906999999999, 2024-09-14T23:57:39.762 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:39.763 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:39.763 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:39.763 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:39.763 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:39.763 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:39.763 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 55365 2024-09-14T23:57:39.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.763 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:39.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:39.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205033+0000", 2024-09-14T23:57:39.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:39.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.764 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:39.764 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205033+0000", 2024-09-14T23:57:39.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:39.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.764 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:39.764 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205033+0000", 2024-09-14T23:57:39.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9900000000000001e-07 2024-09-14T23:57:39.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.764 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:39.764 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205036+0000", 2024-09-14T23:57:39.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3319999999999999e-06 2024-09-14T23:57:39.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.765 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:39.765 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205042+0000", 2024-09-14T23:57:39.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3049999999999998e-06 2024-09-14T23:57:39.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.765 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:39.765 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205047+0000", 2024-09-14T23:57:39.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0420000000000002e-06 2024-09-14T23:57:39.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.765 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:39.765 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.232809+0000", 2024-09-14T23:57:39.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027761917000000001 2024-09-14T23:57:39.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:39.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.232879+0000", 2024-09-14T23:57:39.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9597000000000005e-05 2024-09-14T23:57:39.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:39.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.232948+0000", 2024-09-14T23:57:39.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9214999999999994e-05 2024-09-14T23:57:39.766 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:39.766 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:39.766 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:39.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.767 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:55366 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 3686400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:39.767 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.205054+0000", 2024-09-14T23:57:39.767 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 33.477131081000003, 2024-09-14T23:57:39.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028049935000000002, 2024-09-14T23:57:39.767 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:39.767 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:39.767 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:39.767 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:39.767 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:39.767 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:39.767 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 55366 2024-09-14T23:57:39.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.767 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:39.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:39.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205054+0000", 2024-09-14T23:57:39.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:39.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:39.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205054+0000", 2024-09-14T23:57:39.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:39.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:39.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205055+0000", 2024-09-14T23:57:39.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9999999999999998e-07 2024-09-14T23:57:39.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.769 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:39.769 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205065+0000", 2024-09-14T23:57:39.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0489000000000001e-05 2024-09-14T23:57:39.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.769 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:39.769 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205073+0000", 2024-09-14T23:57:39.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9840000000000001e-06 2024-09-14T23:57:39.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.769 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:39.770 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205085+0000", 2024-09-14T23:57:39.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2052000000000001e-05 2024-09-14T23:57:39.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.770 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:39.770 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.232975+0000", 2024-09-14T23:57:39.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027889646000000001 2024-09-14T23:57:39.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.770 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:39.770 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.233039+0000", 2024-09-14T23:57:39.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3993999999999998e-05 2024-09-14T23:57:39.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.770 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:39.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.233104+0000", 2024-09-14T23:57:39.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5270000000000004e-05 2024-09-14T23:57:39.771 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:39.771 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:39.771 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:39.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.771 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:55367 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 3866624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:39.771 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.205098+0000", 2024-09-14T23:57:39.771 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 33.477087462999997, 2024-09-14T23:57:39.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028159871, 2024-09-14T23:57:39.771 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:39.771 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:39.771 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:39.772 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:39.772 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:39.772 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:39.772 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 55367 2024-09-14T23:57:39.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.772 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:39.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:39.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205098+0000", 2024-09-14T23:57:39.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:39.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:39.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205098+0000", 2024-09-14T23:57:39.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:39.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:39.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205098+0000", 2024-09-14T23:57:39.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4000000000000002e-07 2024-09-14T23:57:39.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:39.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205101+0000", 2024-09-14T23:57:39.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7860000000000001e-06 2024-09-14T23:57:39.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:39.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205109+0000", 2024-09-14T23:57:39.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4039999999999998e-06 2024-09-14T23:57:39.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:39.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205123+0000", 2024-09-14T23:57:39.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.42e-05 2024-09-14T23:57:39.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:39.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.233129+0000", 2024-09-14T23:57:39.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028006316999999999 2024-09-14T23:57:39.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:39.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.233196+0000", 2024-09-14T23:57:39.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7234e-05 2024-09-14T23:57:39.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:39.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.233258+0000", 2024-09-14T23:57:39.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1389999999999993e-05 2024-09-14T23:57:39.775 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:39.776 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:39.776 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:39.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.776 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:55368 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 4046848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:39.776 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.205132+0000", 2024-09-14T23:57:39.776 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 33.477053151, 2024-09-14T23:57:39.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028285467000000002, 2024-09-14T23:57:39.776 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:39.776 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:39.776 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:39.776 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:39.776 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:39.776 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:39.777 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 55368 2024-09-14T23:57:39.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.777 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:39.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.777 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:39.777 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205132+0000", 2024-09-14T23:57:39.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:39.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.777 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:39.777 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205132+0000", 2024-09-14T23:57:39.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:39.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:39.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205133+0000", 2024-09-14T23:57:39.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.13e-07 2024-09-14T23:57:39.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:39.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205135+0000", 2024-09-14T23:57:39.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5490000000000001e-06 2024-09-14T23:57:39.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:39.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205142+0000", 2024-09-14T23:57:39.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.007e-06 2024-09-14T23:57:39.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:39.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205147+0000", 2024-09-14T23:57:39.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.169e-06 2024-09-14T23:57:39.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:39.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.233281+0000", 2024-09-14T23:57:39.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028134059999999999 2024-09-14T23:57:39.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:39.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.233355+0000", 2024-09-14T23:57:39.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3120000000000005e-05 2024-09-14T23:57:39.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:39.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:39.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:39.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.233418+0000", 2024-09-14T23:57:39.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3048999999999999e-05 2024-09-14T23:57:39.780 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:39.780 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:39.780 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:39.780 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:39.780 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:39.780 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:39.782 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T23:57:39.782 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:39.780+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:39.882 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:39.881+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:39.930 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:39.930 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:57:39.930 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:57:39.930 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:39.938 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T23:57:40.022 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:40.021+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:40.028 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is unset 2024-09-14T23:57:40.088 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:40.087+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:40.095 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:40.095 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:57:40.095 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:57:40.095 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:57:40.095 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:40.104 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T23:57:40.187 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:40.186+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:40.256 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:40.256 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:57:40.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:57:40.256 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:57:40.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.257 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5880 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:40.257 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375821+0000", 2024-09-14T23:57:40.257 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 60.877430855999997, 2024-09-14T23:57:40.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050482096999999997, 2024-09-14T23:57:40.258 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:40.258 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:40.258 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:40.258 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:40.258 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:40.258 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:40.259 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5880 2024-09-14T23:57:40.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.259 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:40.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.259 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:40.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375821+0000", 2024-09-14T23:57:40.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:40.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:40.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375821+0000", 2024-09-14T23:57:40.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:40.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:40.261 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375821+0000", 2024-09-14T23:57:40.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.28e-07 2024-09-14T23:57:40.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:40.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375822+0000", 2024-09-14T23:57:40.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7000000000000003e-07 2024-09-14T23:57:40.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.263 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:40.263 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375825+0000", 2024-09-14T23:57:40.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9560000000000002e-06 2024-09-14T23:57:40.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:40.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375827+0000", 2024-09-14T23:57:40.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3259999999999998e-06 2024-09-14T23:57:40.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:40.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426294+0000", 2024-09-14T23:57:40.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050466748999999998 2024-09-14T23:57:40.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.265 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:40.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426303+0000", 2024-09-14T23:57:40.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8680000000000005e-06 2024-09-14T23:57:40.265 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:40.265 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:40.265 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:40.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.266 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5881 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:40.266 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375830+0000", 2024-09-14T23:57:40.266 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 60.877421409999997, 2024-09-14T23:57:40.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050523831999999998, 2024-09-14T23:57:40.266 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:40.266 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:40.266 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:40.266 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:40.266 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:40.266 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:40.266 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5881 2024-09-14T23:57:40.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.267 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:40.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.267 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:40.267 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375830+0000", 2024-09-14T23:57:40.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:40.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.267 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:40.267 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375830+0000", 2024-09-14T23:57:40.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:40.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.267 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:40.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375830+0000", 2024-09-14T23:57:40.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.34e-07 2024-09-14T23:57:40.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:40.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375831+0000", 2024-09-14T23:57:40.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9599999999999998e-07 2024-09-14T23:57:40.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:40.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375834+0000", 2024-09-14T23:57:40.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0489999999999999e-06 2024-09-14T23:57:40.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:40.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375838+0000", 2024-09-14T23:57:40.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5319999999999998e-06 2024-09-14T23:57:40.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:40.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426342+0000", 2024-09-14T23:57:40.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050504391000000003 2024-09-14T23:57:40.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:40.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426354+0000", 2024-09-14T23:57:40.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.173e-05 2024-09-14T23:57:40.270 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:40.270 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:40.270 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:40.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.270 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5882 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:40.270 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:57:40.270 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 60.877410806, 2024-09-14T23:57:40.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050549666, 2024-09-14T23:57:40.270 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:40.270 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:40.270 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:40.270 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:40.270 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:40.271 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:40.271 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5882 2024-09-14T23:57:40.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.271 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:40.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:40.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:57:40.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:40.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:40.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:57:40.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:40.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:40.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:57:40.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8099999999999998e-07 2024-09-14T23:57:40.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:40.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375842+0000", 2024-09-14T23:57:40.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9900000000000009e-07 2024-09-14T23:57:40.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:40.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375845+0000", 2024-09-14T23:57:40.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.055e-06 2024-09-14T23:57:40.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:40.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375848+0000", 2024-09-14T23:57:40.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7609999999999999e-06 2024-09-14T23:57:40.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:40.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426381+0000", 2024-09-14T23:57:40.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050533392000000003 2024-09-14T23:57:40.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:40.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426390+0000", 2024-09-14T23:57:40.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0780000000000002e-06 2024-09-14T23:57:40.274 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:40.274 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:40.274 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:40.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.274 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5884 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:40.274 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:57:40.274 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 60.877400666, 2024-09-14T23:57:40.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050717439000000003, 2024-09-14T23:57:40.275 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:40.275 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:40.275 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:40.275 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:40.275 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:40.275 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:40.275 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5884 2024-09-14T23:57:40.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.275 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:40.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.276 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:40.276 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:57:40.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:40.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.276 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:40.276 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:57:40.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:40.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.276 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:40.276 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:57:40.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0800000000000001e-07 2024-09-14T23:57:40.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:40.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375852+0000", 2024-09-14T23:57:40.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.02e-06 2024-09-14T23:57:40.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:40.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375855+0000", 2024-09-14T23:57:40.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9900000000000002e-06 2024-09-14T23:57:40.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:40.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375857+0000", 2024-09-14T23:57:40.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3499999999999999e-06 2024-09-14T23:57:40.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:40.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426556+0000", 2024-09-14T23:57:40.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050698139000000003 2024-09-14T23:57:40.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:40.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426568+0000", 2024-09-14T23:57:40.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2731999999999999e-05 2024-09-14T23:57:40.279 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:40.279 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:40.279 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:40.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.279 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5885 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:40.279 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:57:40.279 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 60.877381640999999, 2024-09-14T23:57:40.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050737333000000003, 2024-09-14T23:57:40.279 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:40.279 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:40.279 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:40.280 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:40.280 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:40.280 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:40.280 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5885 2024-09-14T23:57:40.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.280 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:40.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.280 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:40.280 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:57:40.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:40.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.280 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:40.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:57:40.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:40.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:40.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:57:40.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5899999999999998e-07 2024-09-14T23:57:40.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:40.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375872+0000", 2024-09-14T23:57:40.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4219999999999999e-06 2024-09-14T23:57:40.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:40.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375878+0000", 2024-09-14T23:57:40.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4980000000000003e-06 2024-09-14T23:57:40.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:40.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375881+0000", 2024-09-14T23:57:40.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6110000000000001e-06 2024-09-14T23:57:40.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:40.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426596+0000", 2024-09-14T23:57:40.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050715320000000001 2024-09-14T23:57:40.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:40.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426607+0000", 2024-09-14T23:57:40.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1223000000000001e-05 2024-09-14T23:57:40.283 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:40.283 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:40.283 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:40.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.284 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5886 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:40.284 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:57:40.284 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 60.877367528000001, 2024-09-14T23:57:40.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050803395000000001, 2024-09-14T23:57:40.284 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:40.284 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:40.284 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:40.284 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:40.284 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:40.284 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:40.284 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5886 2024-09-14T23:57:40.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.285 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:40.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.285 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:40.285 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:57:40.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:40.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.285 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:40.285 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:57:40.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:40.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:40.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:57:40.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2700000000000001e-07 2024-09-14T23:57:40.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:40.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375886+0000", 2024-09-14T23:57:40.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7060000000000001e-06 2024-09-14T23:57:40.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.287 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:40.287 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375890+0000", 2024-09-14T23:57:40.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2259999999999999e-06 2024-09-14T23:57:40.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.287 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:40.287 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375893+0000", 2024-09-14T23:57:40.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0819999999999999e-06 2024-09-14T23:57:40.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.287 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:40.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426677+0000", 2024-09-14T23:57:40.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050784158000000003 2024-09-14T23:57:40.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:40.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426687+0000", 2024-09-14T23:57:40.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9960000000000005e-06 2024-09-14T23:57:40.288 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:40.288 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:40.288 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:40.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.289 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5887 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:40.289 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:57:40.289 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 60.877345441999999, 2024-09-14T23:57:40.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050833070000000001, 2024-09-14T23:57:40.289 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:40.289 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:40.289 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:40.289 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:40.289 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:40.289 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:40.290 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5887 2024-09-14T23:57:40.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.290 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:40.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:40.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:57:40.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:40.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:40.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:57:40.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:40.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:40.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:57:40.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3299999999999998e-07 2024-09-14T23:57:40.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:40.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375908+0000", 2024-09-14T23:57:40.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7689999999999999e-06 2024-09-14T23:57:40.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:40.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375912+0000", 2024-09-14T23:57:40.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.535e-06 2024-09-14T23:57:40.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.292 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:40.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375916+0000", 2024-09-14T23:57:40.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8979999999999998e-06 2024-09-14T23:57:40.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.292 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:40.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426729+0000", 2024-09-14T23:57:40.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050813453000000001 2024-09-14T23:57:40.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.292 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:40.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426739+0000", 2024-09-14T23:57:40.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0081999999999999e-05 2024-09-14T23:57:40.293 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:40.293 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:40.293 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:40.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.293 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5888 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:40.293 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:57:40.293 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 60.877332365000001, 2024-09-14T23:57:40.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050854652, 2024-09-14T23:57:40.293 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:40.293 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:40.294 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:40.294 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:40.294 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:40.294 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:40.294 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5888 2024-09-14T23:57:40.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.294 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:40.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:40.294 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:57:40.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:40.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.295 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:40.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:57:40.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:40.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.295 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:40.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:57:40.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1400000000000001e-07 2024-09-14T23:57:40.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.296 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:40.296 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375921+0000", 2024-09-14T23:57:40.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.119e-06 2024-09-14T23:57:40.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.296 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:40.296 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375928+0000", 2024-09-14T23:57:40.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3170000000000004e-06 2024-09-14T23:57:40.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:40.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375931+0000", 2024-09-14T23:57:40.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6970000000000002e-06 2024-09-14T23:57:40.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.298 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:40.298 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426768+0000", 2024-09-14T23:57:40.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050837355000000001 2024-09-14T23:57:40.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.298 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.298 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:40.298 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426774+0000", 2024-09-14T23:57:40.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9499999999999998e-06 2024-09-14T23:57:40.298 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:40.299 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:40.299 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:40.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.299 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5889 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:40.299 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:57:40.299 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 60.877315441, 2024-09-14T23:57:40.299 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050885446000000001, 2024-09-14T23:57:40.300 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:40.300 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:40.300 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:40.300 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:40.300 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:40.300 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:40.300 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5889 2024-09-14T23:57:40.300 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.301 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:40.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.301 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:40.301 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:57:40.301 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:40.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.301 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:40.301 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:57:40.302 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:40.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.302 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.302 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:40.302 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:57:40.302 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.36e-07 2024-09-14T23:57:40.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.302 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.303 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:40.303 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375940+0000", 2024-09-14T23:57:40.303 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.844e-06 2024-09-14T23:57:40.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.303 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.303 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:40.303 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375944+0000", 2024-09-14T23:57:40.303 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6720000000000002e-06 2024-09-14T23:57:40.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.304 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.304 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:40.304 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375946+0000", 2024-09-14T23:57:40.304 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5629999999999999e-06 2024-09-14T23:57:40.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.304 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.304 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:40.304 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426810+0000", 2024-09-14T23:57:40.305 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050863255000000003 2024-09-14T23:57:40.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.305 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.305 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:40.305 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426822+0000", 2024-09-14T23:57:40.305 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1875999999999999e-05 2024-09-14T23:57:40.305 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:40.305 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:40.306 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:40.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.306 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.306 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5890 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:40.306 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:57:40.306 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 60.877301629000002, 2024-09-14T23:57:40.306 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050917549999999999, 2024-09-14T23:57:40.306 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:40.306 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:40.307 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:40.307 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:40.307 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:40.307 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:40.307 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5890 2024-09-14T23:57:40.307 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.307 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:40.307 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.307 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:40.308 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:57:40.308 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:40.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.308 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.308 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:40.308 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:57:40.308 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:40.309 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.309 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.309 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:40.309 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:57:40.309 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4299999999999999e-07 2024-09-14T23:57:40.309 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.309 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.309 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:40.310 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375951+0000", 2024-09-14T23:57:40.310 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1000000000000001e-06 2024-09-14T23:57:40.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.310 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:40.310 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375956+0000", 2024-09-14T23:57:40.310 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3370000000000001e-06 2024-09-14T23:57:40.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.311 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:40.311 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375958+0000", 2024-09-14T23:57:40.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4820000000000001e-06 2024-09-14T23:57:40.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.311 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:40.311 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426858+0000", 2024-09-14T23:57:40.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050899666000000003 2024-09-14T23:57:40.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.311 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:40.312 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426867+0000", 2024-09-14T23:57:40.312 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7219999999999994e-06 2024-09-14T23:57:40.312 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:40.312 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:40.312 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:40.312 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.312 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.312 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5891 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:40.312 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:57:40.312 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 60.877290307000003, 2024-09-14T23:57:40.312 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050948067, 2024-09-14T23:57:40.312 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:40.312 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:40.313 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:40.313 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:40.313 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:40.313 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:40.313 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5891 2024-09-14T23:57:40.313 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.313 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:40.313 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.313 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:40.313 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:57:40.313 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:40.313 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.313 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.313 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:40.314 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:57:40.314 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:40.314 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.314 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.314 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:40.314 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:57:40.314 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0800000000000001e-07 2024-09-14T23:57:40.314 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.314 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.314 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:40.315 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375962+0000", 2024-09-14T23:57:40.315 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9600000000000008e-07 2024-09-14T23:57:40.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.315 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:40.315 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375970+0000", 2024-09-14T23:57:40.315 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1600000000000001e-06 2024-09-14T23:57:40.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.315 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:40.315 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375972+0000", 2024-09-14T23:57:40.315 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4849999999999999e-06 2024-09-14T23:57:40.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.316 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.316 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:40.316 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426902+0000", 2024-09-14T23:57:40.316 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050929813999999997 2024-09-14T23:57:40.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.316 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.316 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:40.316 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426909+0000", 2024-09-14T23:57:40.316 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4039999999999998e-06 2024-09-14T23:57:40.316 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:40.316 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:40.316 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:40.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.317 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.317 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5892 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:40.317 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:57:40.317 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 60.877275449999999, 2024-09-14T23:57:40.317 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050965036999999998, 2024-09-14T23:57:40.317 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:40.317 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:40.317 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:40.317 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:40.317 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:40.317 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:40.317 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5892 2024-09-14T23:57:40.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.318 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:40.318 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.318 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:40.318 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:57:40.318 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:40.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.318 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.318 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:40.318 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:57:40.318 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:40.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.318 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.318 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:40.318 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:57:40.319 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.34e-07 2024-09-14T23:57:40.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.319 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.319 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:40.319 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375978+0000", 2024-09-14T23:57:40.319 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1880000000000001e-06 2024-09-14T23:57:40.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.319 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.319 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:40.319 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375981+0000", 2024-09-14T23:57:40.319 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2399999999999999e-06 2024-09-14T23:57:40.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.319 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.320 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:40.320 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375984+0000", 2024-09-14T23:57:40.320 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9110000000000002e-06 2024-09-14T23:57:40.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.320 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.320 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:40.320 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426934+0000", 2024-09-14T23:57:40.320 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050949912 2024-09-14T23:57:40.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.320 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.320 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:40.320 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426941+0000", 2024-09-14T23:57:40.320 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5519999999999999e-06 2024-09-14T23:57:40.321 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:40.321 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:40.321 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:40.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.321 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.321 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5893 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:40.321 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:57:40.321 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 60.877264853, 2024-09-14T23:57:40.321 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050975131999999999, 2024-09-14T23:57:40.321 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:40.322 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:40.322 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:40.322 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:40.322 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:40.322 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:40.322 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5893 2024-09-14T23:57:40.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.322 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:40.322 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.322 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:40.322 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:57:40.322 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:40.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.322 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.323 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:40.323 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:57:40.323 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:40.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.323 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.323 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:40.323 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:57:40.323 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2999999999999999e-07 2024-09-14T23:57:40.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.323 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.323 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:40.323 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375988+0000", 2024-09-14T23:57:40.323 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0410000000000001e-06 2024-09-14T23:57:40.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.324 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:40.324 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375992+0000", 2024-09-14T23:57:40.324 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.104e-06 2024-09-14T23:57:40.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.324 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:40.324 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375995+0000", 2024-09-14T23:57:40.324 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4310000000000001e-06 2024-09-14T23:57:40.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.324 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:40.324 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426958+0000", 2024-09-14T23:57:40.325 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050963642000000003 2024-09-14T23:57:40.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.325 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:40.325 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426962+0000", 2024-09-14T23:57:40.325 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6839999999999998e-06 2024-09-14T23:57:40.325 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:40.325 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:40.325 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:40.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.325 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5894 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:40.326 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:57:40.326 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 60.877253453000002, 2024-09-14T23:57:40.326 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050978333000000001, 2024-09-14T23:57:40.326 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:40.326 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:40.326 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:40.326 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:40.326 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:40.326 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:40.326 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5894 2024-09-14T23:57:40.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.326 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:40.326 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.326 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:40.326 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:57:40.327 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:40.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.327 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.327 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:40.327 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:57:40.327 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:40.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.327 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.327 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:40.327 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:57:40.327 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.23e-07 2024-09-14T23:57:40.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.327 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.327 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:40.328 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375999+0000", 2024-09-14T23:57:40.328 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9800000000000002e-07 2024-09-14T23:57:40.328 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.328 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.328 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:40.328 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376007+0000", 2024-09-14T23:57:40.328 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8539999999999997e-06 2024-09-14T23:57:40.328 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.328 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.328 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:40.328 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376011+0000", 2024-09-14T23:57:40.328 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.399e-06 2024-09-14T23:57:40.328 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.328 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.329 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:40.329 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426973+0000", 2024-09-14T23:57:40.329 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050962454999999997 2024-09-14T23:57:40.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.329 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.329 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:40.329 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426976+0000", 2024-09-14T23:57:40.329 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4039999999999999e-06 2024-09-14T23:57:40.329 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:40.329 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:40.329 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:40.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.329 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.329 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5895 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:40.330 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376014+0000", 2024-09-14T23:57:40.330 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 60.877237108999999, 2024-09-14T23:57:40.330 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050975951999999998, 2024-09-14T23:57:40.330 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:40.330 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:40.330 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:40.330 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:40.330 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:40.330 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:40.330 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5895 2024-09-14T23:57:40.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.330 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:40.330 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.330 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:40.331 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376014+0000", 2024-09-14T23:57:40.331 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:40.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.331 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.331 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:40.331 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376014+0000", 2024-09-14T23:57:40.331 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:40.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.331 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.331 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:40.331 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376015+0000", 2024-09-14T23:57:40.331 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2999999999999999e-07 2024-09-14T23:57:40.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.332 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.332 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:40.332 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376016+0000", 2024-09-14T23:57:40.332 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1659999999999999e-06 2024-09-14T23:57:40.332 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.332 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.332 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:40.332 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376024+0000", 2024-09-14T23:57:40.332 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8930000000000005e-06 2024-09-14T23:57:40.332 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.332 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.332 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:40.332 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376026+0000", 2024-09-14T23:57:40.332 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6869999999999999e-06 2024-09-14T23:57:40.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.333 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.333 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:40.333 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426987+0000", 2024-09-14T23:57:40.333 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050960601000000001 2024-09-14T23:57:40.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.333 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.333 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:40.333 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426990+0000", 2024-09-14T23:57:40.333 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3749999999999999e-06 2024-09-14T23:57:40.333 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:40.333 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:40.333 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:40.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.334 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.334 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5896 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:40.334 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:57:40.334 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 60.877221372000001, 2024-09-14T23:57:40.334 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050975656000000001, 2024-09-14T23:57:40.334 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:40.334 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:40.334 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:40.334 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:40.334 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:40.334 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:40.334 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5896 2024-09-14T23:57:40.334 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.335 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:40.335 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.335 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:40.335 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:57:40.335 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:40.335 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.335 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.335 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:40.335 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:57:40.335 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:40.335 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.336 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:40.336 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:57:40.336 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5100000000000001e-07 2024-09-14T23:57:40.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.336 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:40.336 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376032+0000", 2024-09-14T23:57:40.336 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.141e-06 2024-09-14T23:57:40.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.336 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:40.336 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376036+0000", 2024-09-14T23:57:40.337 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.25e-06 2024-09-14T23:57:40.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.337 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.337 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:40.337 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376038+0000", 2024-09-14T23:57:40.337 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5469999999999999e-06 2024-09-14T23:57:40.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.337 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.337 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:40.337 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427002+0000", 2024-09-14T23:57:40.337 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050964029000000001 2024-09-14T23:57:40.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.337 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.338 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:40.338 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427006+0000", 2024-09-14T23:57:40.338 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4379999999999999e-06 2024-09-14T23:57:40.338 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:40.338 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:40.338 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:40.338 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.338 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.338 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5897 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:40.338 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:57:40.338 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 60.877209571999998, 2024-09-14T23:57:40.338 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050978001000000002, 2024-09-14T23:57:40.338 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:40.338 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:40.339 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:40.339 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:40.339 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:40.339 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:40.339 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5897 2024-09-14T23:57:40.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.339 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:40.339 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.339 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:40.339 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:57:40.339 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:40.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.340 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.340 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:40.340 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:57:40.340 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:40.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.340 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.340 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:40.340 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:57:40.340 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2600000000000001e-07 2024-09-14T23:57:40.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.340 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.340 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:40.341 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376043+0000", 2024-09-14T23:57:40.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.054e-06 2024-09-14T23:57:40.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.341 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.341 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:40.341 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376047+0000", 2024-09-14T23:57:40.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0659999999999997e-06 2024-09-14T23:57:40.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.341 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.341 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:40.341 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376050+0000", 2024-09-14T23:57:40.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.88e-06 2024-09-14T23:57:40.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.342 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.342 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:40.342 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427017+0000", 2024-09-14T23:57:40.342 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050966453000000002 2024-09-14T23:57:40.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.342 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.342 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:40.342 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427020+0000", 2024-09-14T23:57:40.342 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3220000000000001e-06 2024-09-14T23:57:40.342 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:40.342 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:40.342 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:40.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.343 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5898 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:40.343 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376053+0000", 2024-09-14T23:57:40.343 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 60.877198086999996, 2024-09-14T23:57:40.343 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050982024000000001, 2024-09-14T23:57:40.343 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:40.343 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:40.343 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:40.343 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:40.343 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:40.343 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:40.343 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5898 2024-09-14T23:57:40.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.344 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:40.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.344 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:40.344 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376053+0000", 2024-09-14T23:57:40.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:40.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.344 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:40.344 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376053+0000", 2024-09-14T23:57:40.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:40.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.344 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:40.345 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376278+0000", 2024-09-14T23:57:40.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022426400000000001 2024-09-14T23:57:40.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:40.345 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376282+0000", 2024-09-14T23:57:40.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4739999999999999e-06 2024-09-14T23:57:40.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:40.345 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376296+0000", 2024-09-14T23:57:40.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4229999999999999e-05 2024-09-14T23:57:40.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.346 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:40.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376303+0000", 2024-09-14T23:57:40.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2199999999999997e-06 2024-09-14T23:57:40.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.346 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:40.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427031+0000", 2024-09-14T23:57:40.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050728154999999997 2024-09-14T23:57:40.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.347 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:40.347 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427036+0000", 2024-09-14T23:57:40.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6809999999999997e-06 2024-09-14T23:57:40.347 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:40.347 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:40.347 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:40.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.347 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5899 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:40.347 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376338+0000", 2024-09-14T23:57:40.347 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 60.876913363, 2024-09-14T23:57:40.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050712100000000003, 2024-09-14T23:57:40.347 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:40.347 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:40.348 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:40.348 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:40.348 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:40.348 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:40.348 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5899 2024-09-14T23:57:40.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.348 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:40.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.348 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:40.348 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376338+0000", 2024-09-14T23:57:40.348 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:40.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.348 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:40.349 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376338+0000", 2024-09-14T23:57:40.349 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:40.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.349 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:40.349 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376339+0000", 2024-09-14T23:57:40.349 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9100000000000003e-07 2024-09-14T23:57:40.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.349 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:40.349 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376342+0000", 2024-09-14T23:57:40.349 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7e-06 2024-09-14T23:57:40.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.350 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:40.350 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376350+0000", 2024-09-14T23:57:40.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5280000000000003e-06 2024-09-14T23:57:40.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.350 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:40.350 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376356+0000", 2024-09-14T23:57:40.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6010000000000002e-06 2024-09-14T23:57:40.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.350 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:40.350 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427047+0000", 2024-09-14T23:57:40.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050690962999999999 2024-09-14T23:57:40.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.351 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:40.351 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427050+0000", 2024-09-14T23:57:40.351 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.517e-06 2024-09-14T23:57:40.351 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:40.351 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:40.351 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:40.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.351 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5900 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:40.351 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376366+0000", 2024-09-14T23:57:40.351 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 60.876885104000003, 2024-09-14T23:57:40.352 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050697858999999998, 2024-09-14T23:57:40.352 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:40.352 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:40.352 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:40.352 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:40.352 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:40.352 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:40.352 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5900 2024-09-14T23:57:40.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.352 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:40.352 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.352 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:40.352 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376366+0000", 2024-09-14T23:57:40.353 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:40.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:40.353 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376366+0000", 2024-09-14T23:57:40.353 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:40.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:40.353 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377650+0000", 2024-09-14T23:57:40.353 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001283834 2024-09-14T23:57:40.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:40.353 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377655+0000", 2024-09-14T23:57:40.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7230000000000004e-06 2024-09-14T23:57:40.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.354 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:40.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377667+0000", 2024-09-14T23:57:40.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1746000000000001e-05 2024-09-14T23:57:40.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.354 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:40.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377676+0000", 2024-09-14T23:57:40.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3619999999999995e-06 2024-09-14T23:57:40.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.355 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:40.355 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427061+0000", 2024-09-14T23:57:40.355 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049384755000000002 2024-09-14T23:57:40.355 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:40.355 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:40.355 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:40.355 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427064+0000", 2024-09-14T23:57:40.355 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.439e-06 2024-09-14T23:57:40.355 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:40.355 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:40.355 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:40.355 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:40.355 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:40.355 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:40.357 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T23:57:40.357 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone34 from 0x569e34a (overlap 0xbb1c34) to 0x4ad814 2024-09-14T23:57:40.358 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:40.355+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:40.457 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:40.456+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:40.494 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #33, image image_client.0-clone33 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent34 2024-09-14T23:57:40.498 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T23:57:40.513 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:40.513 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:57:40.514 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:57:40.514 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:40.523 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T23:57:40.558 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:40.557+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:40.617 INFO:teuthology.orchestra.run.smithi042.stdout:3599 write 0x81aa901 thru 0x81b3e74 (0x9574 bytes) 2024-09-14T23:57:40.629 INFO:teuthology.orchestra.run.smithi042.stdout:3601 writesame 0xb1fb287 thru 0xb20b110 data_size 0x1f(0xfe8a bytes) 2024-09-14T23:57:40.636 INFO:teuthology.orchestra.run.smithi042.stdout:3603 write 0xba8e859 thru 0xba8f3f3 (0xb9b bytes) 2024-09-14T23:57:40.638 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:57:40.638 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:57:40.639 INFO:teuthology.orchestra.run.smithi042.stdout:3605 punch from 0x6d6cd79 to 0x6d73559, (0x67e0 bytes) 2024-09-14T23:57:40.639 INFO:teuthology.orchestra.run.smithi042.stdout:3606 punch from 0x85305e9 to 0x8535b32, (0x5549 bytes) 2024-09-14T23:57:40.651 INFO:teuthology.orchestra.run.smithi042.stdout:3607 trunc from 0xba8f3f4 to 0x1404fe0 2024-09-14T23:57:40.658 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:57:40.658 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:57:40.659 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:40.658+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:40.684 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:57:40.684 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:57:40.693 INFO:teuthology.orchestra.run.smithi042.stdout:3608 trunc from 0x1404fe0 to 0xd5211e4 2024-09-14T23:57:40.693 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:40.694 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:40.702 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:57:40.703 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:57:40.704 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T23:57:40.710 INFO:teuthology.orchestra.run.smithi042.stdout:3609 trunc from 0xd5211e4 to 0x5afe3b2 2024-09-14T23:57:40.715 INFO:teuthology.orchestra.run.smithi042.stdout:3610 read 0xcfa717 thru 0xcfb8e8 (0x11d2 bytes) 2024-09-14T23:57:40.716 INFO:teuthology.orchestra.run.smithi042.stdout:3612 punch from 0x39f63a7 to 0x39f769b, (0x12f4 bytes) 2024-09-14T23:57:40.716 INFO:teuthology.orchestra.run.smithi042.stdout:3615 writesame 0x5177eb6 thru 0x51789fb data_size 0x6(0xb46 bytes) 2024-09-14T23:57:40.719 INFO:teuthology.orchestra.run.smithi042.stdout:3618 read 0x27e9584 thru 0x27f6cef (0xd76c bytes) 2024-09-14T23:57:40.719 INFO:teuthology.orchestra.run.smithi042.stdout:3620 read 0x14376d6 thru 0x1439e34 (0x275f bytes) 2024-09-14T23:57:40.722 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:57:40.723 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:57:40.727 INFO:teuthology.orchestra.run.smithi042.stdout:3622 writesame 0x7940ffd thru 0x7944133 data_size 0x1(0x3137 bytes) 2024-09-14T23:57:40.728 INFO:teuthology.orchestra.run.smithi042.stdout:3623 write 0x5387d76 thru 0x5390f61 (0x91ec bytes) 2024-09-14T23:57:40.731 INFO:teuthology.orchestra.run.smithi042.stdout:3625 read 0x356080f thru 0x356ce6f (0xc661 bytes) 2024-09-14T23:57:40.732 INFO:teuthology.orchestra.run.smithi042.stdout:3627 punch from 0x317f688 to 0x318cd80, (0xd6f8 bytes) 2024-09-14T23:57:40.732 INFO:teuthology.orchestra.run.smithi042.stdout:3628 write 0x44515a2 thru 0x445c443 (0xaea2 bytes) 2024-09-14T23:57:40.735 INFO:teuthology.orchestra.run.smithi042.stdout:3629 writesame 0x3e8cb5b thru 0x3e94a4f data_size 0x1(0x7ef5 bytes) 2024-09-14T23:57:40.750 INFO:teuthology.orchestra.run.smithi042.stdout:3630 write 0x20d4691 thru 0x20d917a (0x4aea bytes) 2024-09-14T23:57:40.752 INFO:teuthology.orchestra.run.smithi042.stdout:3633 write 0x6bc6d77 thru 0x6bcf2e0 (0x856a bytes) 2024-09-14T23:57:40.754 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:40.754 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:40.754 INFO:teuthology.orchestra.run.smithi042.stdout:3636 read 0x789a739 thru 0x78a8e3e (0xe706 bytes) 2024-09-14T23:57:40.754 INFO:teuthology.orchestra.run.smithi042.stdout:3637 read 0x4593d6a thru 0x459c54f (0x87e6 bytes) 2024-09-14T23:57:40.754 INFO:teuthology.orchestra.run.smithi042.stdout:3638 write 0x5b70451 thru 0x5b7b959 (0xb509 bytes) 2024-09-14T23:57:40.756 INFO:teuthology.orchestra.run.smithi042.stdout:3639 read 0x71f4cf4 thru 0x71fc7c2 (0x7acf bytes) 2024-09-14T23:57:40.756 INFO:teuthology.orchestra.run.smithi042.stdout:3640 punch from 0x5151983 to 0x5151b22, (0x19f bytes) 2024-09-14T23:57:40.760 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:40.759+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:40.763 INFO:teuthology.orchestra.run.smithi042.stdout:3641 trunc from 0x7944134 to 0x1fc9103 2024-09-14T23:57:40.786 INFO:teuthology.orchestra.run.smithi042.stdout:3642 trunc from 0x1fc9103 to 0x8d8398a 2024-09-14T23:57:40.791 INFO:teuthology.orchestra.run.smithi042.stdout:3643 read 0x20ca2cb thru 0x20d53de (0xb114 bytes) 2024-09-14T23:57:40.791 INFO:teuthology.orchestra.run.smithi042.stdout:3644 read 0x1084e8e thru 0x1090e3d (0xbfb0 bytes) 2024-09-14T23:57:40.804 INFO:teuthology.orchestra.run.smithi042.stdout:3645 write 0xd782d30 thru 0xd78315b (0x42c bytes) 2024-09-14T23:57:40.819 INFO:teuthology.orchestra.run.smithi042.stdout:3646 trunc from 0xd78315c to 0x4557b8d 2024-09-14T23:57:40.829 INFO:teuthology.orchestra.run.smithi042.stdout:3647 read 0x716531 thru 0x724625 (0xe0f5 bytes) 2024-09-14T23:57:40.834 INFO:teuthology.orchestra.run.smithi042.stdout:3648 trunc from 0x4557b8d to 0xd9b5c1 2024-09-14T23:57:40.848 INFO:teuthology.orchestra.run.smithi042.stdout:3649 write 0xfc368b thru 0xfc4223 (0xb99 bytes) 2024-09-14T23:57:40.849 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:40.849 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:40.850 INFO:teuthology.orchestra.run.smithi042.stdout:3652 trunc from 0xfc4224 to 0x21827ae 2024-09-14T23:57:40.855 INFO:teuthology.orchestra.run.smithi042.stdout:3653 read 0x3ebfd4 thru 0x3f6692 (0xa6bf bytes) 2024-09-14T23:57:40.858 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T23:57:40.862 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:40.861+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:40.869 INFO:teuthology.orchestra.run.smithi042.stdout:3654 write 0x8d145a6 thru 0x8d182c1 (0x3d1c bytes) 2024-09-14T23:57:40.879 INFO:teuthology.orchestra.run.smithi042.stdout:3655 write 0x99128b6 thru 0x991b3cd (0x8b18 bytes) 2024-09-14T23:57:40.882 INFO:teuthology.orchestra.run.smithi042.stdout:3656 read 0x2a8edbb thru 0x2a91b8b (0x2dd1 bytes) 2024-09-14T23:57:40.883 INFO:teuthology.orchestra.run.smithi042.stdout:3657 punch from 0x5b88d38 to 0x5b8f6c9, (0x6991 bytes) 2024-09-14T23:57:40.883 INFO:teuthology.orchestra.run.smithi042.stdout:3658 punch from 0x91efc86 to 0x91f7de8, (0x8162 bytes) 2024-09-14T23:57:40.883 INFO:teuthology.orchestra.run.smithi042.stdout:3659 trunc from 0x991b3ce to 0xa607408 2024-09-14T23:57:40.889 INFO:teuthology.orchestra.run.smithi042.stdout:3660 read 0x2fdcfab thru 0x2fe3712 (0x6768 bytes) 2024-09-14T23:57:40.889 INFO:teuthology.orchestra.run.smithi042.stdout:3661 read 0xf5a657 thru 0xf68ed4 (0xe87e bytes) 2024-09-14T23:57:40.889 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:40.889 INFO:teuthology.orchestra.run.smithi042.stdout:3665 punch from 0x57b02fd to 0x57b6ddf, (0x6ae2 bytes) 2024-09-14T23:57:40.889 INFO:teuthology.orchestra.run.smithi042.stdout:3666 writesame 0x8856cbb thru 0x8865d0b data_size 0x1(0xf051 bytes) 2024-09-14T23:57:40.915 INFO:teuthology.orchestra.run.smithi042.stdout:3667 punch from 0xa3d365d to 0xa3d532a, (0x1ccd bytes) 2024-09-14T23:57:40.915 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:40.915 INFO:teuthology.orchestra.run.smithi042.stdout:3671 punch from 0x9b3f9af to 0x9b46cf8, (0x7349 bytes) 2024-09-14T23:57:40.916 INFO:teuthology.orchestra.run.smithi042.stdout:3672 trunc from 0xa607408 to 0xb0ddef3 2024-09-14T23:57:40.921 INFO:teuthology.orchestra.run.smithi042.stdout:3673 punch from 0x77afd82 to 0x77b0fca, (0x1248 bytes) 2024-09-14T23:57:40.921 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:40.929 INFO:teuthology.orchestra.run.smithi042.stdout:3675 write 0xc48a3fd thru 0xc493cf5 (0x98f9 bytes) 2024-09-14T23:57:40.932 INFO:teuthology.orchestra.run.smithi042.stdout:3677 writesame 0x7126caa thru 0x712b1f4 data_size 0x49(0x454b bytes) 2024-09-14T23:57:40.936 INFO:teuthology.orchestra.run.smithi042.stdout:3678 read 0x16cb0be thru 0x16d7a30 (0xc973 bytes) 2024-09-14T23:57:40.937 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:40.942 INFO:teuthology.orchestra.run.smithi042.stdout:3683 write 0xc70f735 thru 0xc7134ec (0x3db8 bytes) 2024-09-14T23:57:40.943 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:57:40.944 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:57:40.945 INFO:teuthology.orchestra.run.smithi042.stdout:3684 read 0x9d23103 thru 0x9d30ff1 (0xdeef bytes) 2024-09-14T23:57:40.946 INFO:teuthology.orchestra.run.smithi042.stdout:3685 writesame 0xa8739fd thru 0xa87c465 data_size 0x1f(0x8a69 bytes) 2024-09-14T23:57:40.947 INFO:teuthology.orchestra.run.smithi042.stdout:3686 read 0xba8141e thru 0xba827cf (0x13b2 bytes) 2024-09-14T23:57:40.956 INFO:teuthology.orchestra.run.smithi042.stdout:3687 trunc from 0xc7134ed to 0x5f53594 2024-09-14T23:57:40.963 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:57:40.963 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:57:40.963 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:40.961+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:40.981 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:57:40.981 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:57:40.986 INFO:teuthology.orchestra.run.smithi042.stdout:3688 writesame 0xdf7318a thru 0xdf793f9 data_size 0x90(0x6270 bytes) 2024-09-14T23:57:40.986 INFO:teuthology.orchestra.run.smithi042.stdout:3689 punch from 0x1b36a71 to 0x1b3a5a5, (0x3b34 bytes) 2024-09-14T23:57:40.987 INFO:teuthology.orchestra.run.smithi042.stdout:3690 punch from 0x372a813 to 0x37320b1, (0x789e bytes) 2024-09-14T23:57:40.993 INFO:teuthology.orchestra.run.smithi042.stdout:3691 write 0xe16e497 thru 0xe1700d7 (0x1c41 bytes) 2024-09-14T23:57:40.996 INFO:teuthology.orchestra.run.smithi042.stdout:3692 punch from 0x666cda0 to 0x666d808, (0xa68 bytes) 2024-09-14T23:57:40.996 INFO:teuthology.orchestra.run.smithi042.stdout:3693 writesame 0x64b3a9b thru 0x64b6f26 data_size 0x26(0x348c bytes) 2024-09-14T23:57:40.999 INFO:teuthology.orchestra.run.smithi042.stdout:3694 write 0x8dee18c thru 0x8df24f9 (0x436e bytes) 2024-09-14T23:57:41.002 INFO:teuthology.orchestra.run.smithi042.stdout:3695 write 0x6d32293 thru 0x6d344ab (0x2219 bytes) 2024-09-14T23:57:41.005 INFO:teuthology.orchestra.run.smithi042.stdout:3697 write 0x316f194 thru 0x317e841 (0xf6ae bytes) 2024-09-14T23:57:41.006 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:41.006 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:41.007 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:41.007 INFO:teuthology.orchestra.run.smithi042.stdout:3699 write 0xafba27e thru 0xafc0422 (0x61a5 bytes) 2024-09-14T23:57:41.009 INFO:teuthology.orchestra.run.smithi042.stdout:3701 punch from 0xa71359b to 0xa71e110, (0xab75 bytes) 2024-09-14T23:57:41.009 INFO:teuthology.orchestra.run.smithi042.stdout:3702 write 0x5c54d4d thru 0x5c5cd6c (0x8020 bytes) 2024-09-14T23:57:41.011 INFO:teuthology.orchestra.run.smithi042.stdout:3703 write 0xd0b7499 thru 0xd0c5478 (0xdfe0 bytes) 2024-09-14T23:57:41.016 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T23:57:41.021 INFO:teuthology.orchestra.run.smithi042.stdout:3704 trunc from 0xe1700d8 to 0x8133d78 2024-09-14T23:57:41.030 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T23:57:41.030 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T23:57:41.030 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-14T23:57:41.030 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T23:57:41.037 INFO:teuthology.orchestra.run.smithi042.stdout:3706 read 0x1ab5fe9 thru 0x1ab6970 (0x988 bytes) 2024-09-14T23:57:41.037 INFO:teuthology.orchestra.run.smithi042.stdout:3708 read 0x1198a45 thru 0x1198d28 (0x2e4 bytes) 2024-09-14T23:57:41.037 INFO:teuthology.orchestra.run.smithi042.stdout:3709 read 0x6c14d62 thru 0x6c1e470 (0x970f bytes) 2024-09-14T23:57:41.038 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:41.044 INFO:teuthology.orchestra.run.smithi042.stdout:3711 writesame 0x8d4bef6 thru 0x8d5a970 data_size 0x6b(0xea7b bytes) 2024-09-14T23:57:41.056 INFO:teuthology.orchestra.run.smithi042.stdout:3713 write 0xdfadb79 thru 0xdfb0c45 (0x30cd bytes) 2024-09-14T23:57:41.059 INFO:teuthology.orchestra.run.smithi042.stdout:3717 write 0x6d47cf9 thru 0x6d489b7 (0xcbf bytes) 2024-09-14T23:57:41.061 INFO:teuthology.orchestra.run.smithi042.stdout:3718 write 0xdf44237 thru 0xdf50155 (0xbf1f bytes) 2024-09-14T23:57:41.063 INFO:teuthology.orchestra.run.smithi042.stdout:3719 write 0x49b7d7d thru 0x49c3c25 (0xbea9 bytes) 2024-09-14T23:57:41.063 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:41.062+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:41.065 INFO:teuthology.orchestra.run.smithi042.stdout:3720 write 0x65c8a10 thru 0x65cd9cd (0x4fbe bytes) 2024-09-14T23:57:41.066 INFO:teuthology.orchestra.run.smithi042.stdout:3722 write 0xa642c40 thru 0xa64c1c0 (0x9581 bytes) 2024-09-14T23:57:41.070 INFO:teuthology.orchestra.run.smithi042.stdout:3723 write 0x9d67851 thru 0x9d6943e (0x1bee bytes) 2024-09-14T23:57:41.085 INFO:teuthology.orchestra.run.smithi042.stdout:3724 trunc from 0xdfb0c46 to 0x580a965 2024-09-14T23:57:41.096 INFO:teuthology.orchestra.run.smithi042.stdout:3725 read 0x10dc03 thru 0x11dbbf (0xffbd bytes) 2024-09-14T23:57:41.102 INFO:teuthology.orchestra.run.smithi042.stdout:3727 writesame 0x5c2a6f5 thru 0x5c32682 data_size 0x1d(0x7f8e bytes) 2024-09-14T23:57:41.104 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:41.111 INFO:teuthology.orchestra.run.smithi042.stdout:3729 write 0x7e8832d thru 0x7e89ca0 (0x1974 bytes) 2024-09-14T23:57:41.116 INFO:teuthology.orchestra.run.smithi042.stdout:3731 writesame 0x35d0d6f thru 0x35d1797 data_size 0x11(0xa29 bytes) 2024-09-14T23:57:41.116 INFO:teuthology.orchestra.run.smithi042.stdout:3733 punch from 0x1984aba to 0x1991f5a, (0xd4a0 bytes) 2024-09-14T23:57:41.126 INFO:teuthology.orchestra.run.smithi042.stdout:3734 write 0xbdc4ae1 thru 0xbdcf0e2 (0xa602 bytes) 2024-09-14T23:57:41.129 INFO:teuthology.orchestra.run.smithi042.stdout:3735 write 0x4b5bdc2 thru 0x4b64690 (0x88cf bytes) 2024-09-14T23:57:41.133 INFO:teuthology.orchestra.run.smithi042.stdout:3736 write 0x1557e0e thru 0x15645e1 (0xc7d4 bytes) 2024-09-14T23:57:41.134 INFO:teuthology.orchestra.run.smithi042.stdout:3737 writesame 0xa903d98 thru 0xa9058b0 data_size 0x1(0x1b19 bytes) 2024-09-14T23:57:41.140 INFO:teuthology.orchestra.run.smithi042.stdout:3738 read 0x5548009 thru 0x554a4ee (0x24e6 bytes) 2024-09-14T23:57:41.140 INFO:teuthology.orchestra.run.smithi042.stdout:3739 read 0xb1d47e4 thru 0xb1dbf7b (0x7798 bytes) 2024-09-14T23:57:41.142 INFO:teuthology.orchestra.run.smithi042.stdout:3741 trunc from 0xbdcf0e3 to 0xd6b8855 2024-09-14T23:57:41.148 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:41.148 INFO:teuthology.orchestra.run.smithi042.stdout:3743 writesame 0x433689a thru 0x43373b6 data_size 0x1(0xb1d bytes) 2024-09-14T23:57:41.155 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:41.155 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:41.156 INFO:teuthology.orchestra.run.smithi042.stdout:3745 trunc from 0xd6b8855 to 0x8bc8a6e 2024-09-14T23:57:41.164 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:41.164 INFO:teuthology.orchestra.run.smithi042.stdout:3748 trunc from 0x8bc8a6e to 0x90af262 2024-09-14T23:57:41.165 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:41.163+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:41.168 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T23:57:41.169 INFO:teuthology.orchestra.run.smithi042.stdout:3750 read 0x74bf649 thru 0x74c1a4c (0x2404 bytes) 2024-09-14T23:57:41.170 INFO:teuthology.orchestra.run.smithi042.stdout:3751 read 0xa549cf thru 0xa568a0 (0x1ed2 bytes) 2024-09-14T23:57:41.173 INFO:teuthology.orchestra.run.smithi042.stdout:3752 trunc from 0x90af262 to 0xb8e090a 2024-09-14T23:57:41.179 INFO:teuthology.orchestra.run.smithi042.stdout:3753 write 0xa58f935 thru 0xa59b2b9 (0xb985 bytes) 2024-09-14T23:57:41.180 INFO:teuthology.orchestra.run.smithi042.stdout:3754 read 0x95cf848 thru 0x95d9828 (0x9fe1 bytes) 2024-09-14T23:57:41.181 INFO:teuthology.orchestra.run.smithi042.stdout:3755 write 0x38aafa1 thru 0x38bacf3 (0xfd53 bytes) 2024-09-14T23:57:41.193 INFO:teuthology.orchestra.run.smithi042.stdout:3756 trunc from 0xb8e090a to 0x2abb8ac 2024-09-14T23:57:41.206 INFO:teuthology.orchestra.run.smithi042.stdout:3757 read 0x1cb012e thru 0x1cb08d0 (0x7a3 bytes) 2024-09-14T23:57:41.223 INFO:teuthology.orchestra.run.smithi042.stdout:3758 write 0xc0fce2e thru 0xc100bf6 (0x3dc9 bytes) 2024-09-14T23:57:41.227 INFO:teuthology.orchestra.run.smithi042.stdout:3759 write 0x172bd4c thru 0x17371da (0xb48f bytes) 2024-09-14T23:57:41.237 INFO:teuthology.orchestra.run.smithi042.stdout:3760 write 0xdbc6ccb thru 0xdbcb566 (0x489c bytes) 2024-09-14T23:57:41.240 INFO:teuthology.orchestra.run.smithi042.stdout:3761 writesame 0x9809ae8 thru 0x980d21f data_size 0x3e(0x3738 bytes) 2024-09-14T23:57:41.251 INFO:teuthology.orchestra.run.smithi042.stdout:3762 trunc from 0xdbcb567 to 0x63b4ca6 2024-09-14T23:57:41.261 INFO:teuthology.orchestra.run.smithi042.stdout:3763 punch from 0x294e23b to 0x295defa, (0xfcbf bytes) 2024-09-14T23:57:41.262 INFO:teuthology.orchestra.run.smithi042.stdout:3764 trunc from 0x63b4ca6 to 0x5869306 2024-09-14T23:57:41.266 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:41.264+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:41.287 INFO:teuthology.orchestra.run.smithi042.stdout:3765 write 0xec30d9b thru 0xec37539 (0x679f bytes) 2024-09-14T23:57:41.290 INFO:teuthology.orchestra.run.smithi042.stdout:3767 write 0xa8107d0 thru 0xa81601a (0x584b bytes) 2024-09-14T23:57:41.293 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:41.293 INFO:teuthology.orchestra.run.smithi042.stdout:3769 write 0x32a363 thru 0x3390ad (0xed4b bytes) 2024-09-14T23:57:41.296 INFO:teuthology.orchestra.run.smithi042.stdout:3772 trunc from 0xec3753a to 0xd6ce663 2024-09-14T23:57:41.299 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:41.299 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:41.305 INFO:teuthology.orchestra.run.smithi042.stdout:3775 punch from 0x3695c90 to 0x3699058, (0x33c8 bytes) 2024-09-14T23:57:41.305 INFO:teuthology.orchestra.run.smithi042.stdout:3776 punch from 0xc74855a to 0xc74a4d8, (0x1f7e bytes) 2024-09-14T23:57:41.305 INFO:teuthology.orchestra.run.smithi042.stdout:3777 writesame 0x29eecd2 thru 0x29f48a1 data_size 0x68(0x5bd0 bytes) 2024-09-14T23:57:41.306 INFO:teuthology.orchestra.run.smithi042.stdout:3778 read 0x61ad81f thru 0x61aed42 (0x1524 bytes) 2024-09-14T23:57:41.307 INFO:teuthology.orchestra.run.smithi042.stdout:3779 trunc from 0xd6ce663 to 0xe26a910 2024-09-14T23:57:41.309 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T23:57:41.313 INFO:teuthology.orchestra.run.smithi042.stdout:3780 read 0x6885691 thru 0x68926cd (0xd03d bytes) 2024-09-14T23:57:41.313 INFO:teuthology.orchestra.run.smithi042.stdout:3783 write 0x5972bf8 thru 0x5981e6d (0xf276 bytes) 2024-09-14T23:57:41.320 INFO:teuthology.orchestra.run.smithi042.stdout:3785 write 0x16a59e4 thru 0x16b4b7c (0xf199 bytes) 2024-09-14T23:57:41.321 INFO:teuthology.orchestra.run.smithi042.stdout:3786 punch from 0x2145978 to 0x21503a3, (0xaa2b bytes) 2024-09-14T23:57:41.321 INFO:teuthology.orchestra.run.smithi042.stdout:3787 read 0xc6b5ad9 thru 0xc6b717f (0x16a7 bytes) 2024-09-14T23:57:41.321 INFO:teuthology.orchestra.run.smithi042.stdout:3788 writesame 0x81cca00 thru 0x81d2294 data_size 0x1(0x5895 bytes) 2024-09-14T23:57:41.333 INFO:teuthology.orchestra.run.smithi042.stdout:3789 write 0x655eaf1 thru 0x656bc59 (0xd169 bytes) 2024-09-14T23:57:41.336 INFO:teuthology.orchestra.run.smithi042.stdout:3790 writesame 0x5b7c220 thru 0x5b7f816 data_size 0xf(0x35f7 bytes) 2024-09-14T23:57:41.339 INFO:teuthology.orchestra.run.smithi042.stdout:3791 write 0x79d29f6 thru 0x79e1523 (0xeb2e bytes) 2024-09-14T23:57:41.341 INFO:teuthology.orchestra.run.smithi042.stdout:3792 punch from 0x2f9ada7 to 0x2fa81ca, (0xd423 bytes) 2024-09-14T23:57:41.341 INFO:teuthology.orchestra.run.smithi042.stdout:3793 read 0xdd988d7 thru 0xdda248f (0x9bb9 bytes) 2024-09-14T23:57:41.341 INFO:teuthology.orchestra.run.smithi042.stdout:3794 writesame 0x60ddb1a thru 0x60e9a04 data_size 0x73(0xbeeb bytes) 2024-09-14T23:57:41.344 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:41.344 INFO:teuthology.orchestra.run.smithi042.stdout:3796 write 0x9214492 thru 0x9218981 (0x44f0 bytes) 2024-09-14T23:57:41.347 INFO:teuthology.orchestra.run.smithi042.stdout:3797 write 0x5182375 thru 0x5188f88 (0x6c14 bytes) 2024-09-14T23:57:41.348 INFO:teuthology.orchestra.run.smithi042.stdout:3800 write 0xaf7004d thru 0xaf738fb (0x38af bytes) 2024-09-14T23:57:41.351 INFO:teuthology.orchestra.run.smithi042.stdout:3801 read 0xaaea680 thru 0xaaf14e7 (0x6e68 bytes) 2024-09-14T23:57:41.352 INFO:teuthology.orchestra.run.smithi042.stdout:3803 write 0x683b2bc thru 0x684544d (0xa192 bytes) 2024-09-14T23:57:41.359 INFO:teuthology.orchestra.run.smithi042.stdout:3804 trunc from 0xe26a910 to 0x8b991e2 2024-09-14T23:57:41.366 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:41.365+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:41.377 INFO:teuthology.orchestra.run.smithi042.stdout:3806 write 0x9178833 thru 0x91837e5 (0xafb3 bytes) 2024-09-14T23:57:41.379 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:57:41.379 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:41.379 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 88, 2024-09-14T23:57:41.379 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "21916fcc-f95c-4bfa-8b8b-11ca45845613", 2024-09-14T23:57:41.379 INFO:teuthology.orchestra.run.smithi042.stdout: "created": "2024-09-14T23:56:03.167948+0000", 2024-09-14T23:57:41.380 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-09-14T23:57:40.021278+0000", 2024-09-14T23:57:41.380 INFO:teuthology.orchestra.run.smithi042.stdout: "last_up_change": "2024-09-14T23:56:14.322882+0000", 2024-09-14T23:57:41.380 INFO:teuthology.orchestra.run.smithi042.stdout: "last_in_change": "2024-09-14T23:56:56.035972+0000", 2024-09-14T23:57:41.380 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T23:57:41.380 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_num": 5799936, 2024-09-14T23:57:41.380 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_set": [ 2024-09-14T23:57:41.380 INFO:teuthology.orchestra.run.smithi042.stdout: "pglog_hardlimit", 2024-09-14T23:57:41.380 INFO:teuthology.orchestra.run.smithi042.stdout: "purged_snapdirs", 2024-09-14T23:57:41.380 INFO:teuthology.orchestra.run.smithi042.stdout: "recovery_deletes", 2024-09-14T23:57:41.380 INFO:teuthology.orchestra.run.smithi042.stdout: "sortbitwise" 2024-09-14T23:57:41.380 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-14T23:57:41.380 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_version": 6, 2024-09-14T23:57:41.380 INFO:teuthology.orchestra.run.smithi042.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T23:57:41.381 INFO:teuthology.orchestra.run.smithi042.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T23:57:41.381 INFO:teuthology.orchestra.run.smithi042.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T23:57:41.381 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_snapshot": "", 2024-09-14T23:57:41.381 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_max": 3, 2024-09-14T23:57:41.381 INFO:teuthology.orchestra.run.smithi042.stdout: "max_osd": 8, 2024-09-14T23:57:41.381 INFO:teuthology.orchestra.run.smithi042.stdout: "require_min_compat_client": "luminous", 2024-09-14T23:57:41.381 INFO:teuthology.orchestra.run.smithi042.stdout: "min_compat_client": "luminous", 2024-09-14T23:57:41.381 INFO:teuthology.orchestra.run.smithi042.stdout: "require_osd_release": "squid", 2024-09-14T23:57:41.381 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_crimson": false, 2024-09-14T23:57:41.381 INFO:teuthology.orchestra.run.smithi042.stdout: "pools": [ 2024-09-14T23:57:41.381 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.381 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 1, 2024-09-14T23:57:41.381 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": ".mgr", 2024-09-14T23:57:41.381 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-14T23:56:13.500197+0000", 2024-09-14T23:57:41.382 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-09-14T23:57:41.382 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool", 2024-09-14T23:57:41.382 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-14T23:57:41.382 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-14T23:57:41.382 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-14T23:57:41.382 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-14T23:57:41.382 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-14T23:57:41.382 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-14T23:57:41.382 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T23:57:41.382 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T23:57:41.382 INFO:teuthology.orchestra.run.smithi042.stdout: "is_stretch_pool": false, 2024-09-14T23:57:41.382 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-14T23:57:41.382 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-14T23:57:41.382 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 1, 2024-09-14T23:57:41.383 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 1, 2024-09-14T23:57:41.383 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 1, 2024-09-14T23:57:41.383 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 1, 2024-09-14T23:57:41.383 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 1, 2024-09-14T23:57:41.383 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-14T23:57:41.383 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-14T23:57:41.383 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-14T23:57:41.383 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-14T23:57:41.383 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-14T23:57:41.383 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-14T23:57:41.383 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-14T23:57:41.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.383 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "17", 2024-09-14T23:57:41.383 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "0", 2024-09-14T23:57:41.384 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T23:57:41.384 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T23:57:41.384 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-14T23:57:41.384 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-14T23:57:41.384 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 0, 2024-09-14T23:57:41.384 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 0, 2024-09-14T23:57:41.384 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-14T23:57:41.384 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-14T23:57:41.384 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-14T23:57:41.384 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-14T23:57:41.384 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-14T23:57:41.384 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-14T23:57:41.384 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-14T23:57:41.384 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-14T23:57:41.385 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-14T23:57:41.385 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-14T23:57:41.385 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-14T23:57:41.385 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T23:57:41.385 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T23:57:41.385 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T23:57:41.385 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-14T23:57:41.385 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-14T23:57:41.385 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-14T23:57:41.385 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-14T23:57:41.385 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-14T23:57:41.385 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.385 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-14T23:57:41.385 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-14T23:57:41.386 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-14T23:57:41.386 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-14T23:57:41.386 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-14T23:57:41.386 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T23:57:41.386 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-14T23:57:41.386 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-14T23:57:41.386 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-14T23:57:41.386 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-14T23:57:41.386 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-14T23:57:41.386 INFO:teuthology.orchestra.run.smithi042.stdout: "options": { 2024-09-14T23:57:41.387 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_max": 32, 2024-09-14T23:57:41.387 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_min": 1 2024-09-14T23:57:41.387 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.387 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-14T23:57:41.387 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": {} 2024-09-14T23:57:41.387 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.387 INFO:teuthology.orchestra.run.smithi042.stdout: "read_balance": { 2024-09-14T23:57:41.387 INFO:teuthology.orchestra.run.smithi042.stdout: "score_type": "Fair distribution", 2024-09-14T23:57:41.387 INFO:teuthology.orchestra.run.smithi042.stdout: "score_acting": 6.9000000953674316, 2024-09-14T23:57:41.387 INFO:teuthology.orchestra.run.smithi042.stdout: "score_stable": 6.9000000953674316, 2024-09-14T23:57:41.387 INFO:teuthology.orchestra.run.smithi042.stdout: "optimal_score": 0.28999999165534973, 2024-09-14T23:57:41.387 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_acting": 2, 2024-09-14T23:57:41.388 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_stable": 2, 2024-09-14T23:57:41.388 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity_weighted": 0.8712921142578125, 2024-09-14T23:57:41.388 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity": 1, 2024-09-14T23:57:41.388 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity_weighted": 0.87000000476837158 2024-09-14T23:57:41.388 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:41.388 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.388 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.388 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 2, 2024-09-14T23:57:41.388 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": "rbd", 2024-09-14T23:57:41.388 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-14T23:56:21.284705+0000", 2024-09-14T23:57:41.388 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 8193, 2024-09-14T23:57:41.388 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T23:57:41.388 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-14T23:57:41.388 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-14T23:57:41.389 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-14T23:57:41.389 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-14T23:57:41.389 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-14T23:57:41.389 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-14T23:57:41.389 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T23:57:41.389 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T23:57:41.389 INFO:teuthology.orchestra.run.smithi042.stdout: "is_stretch_pool": false, 2024-09-14T23:57:41.389 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-14T23:57:41.389 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-14T23:57:41.389 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 8, 2024-09-14T23:57:41.389 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 8, 2024-09-14T23:57:41.389 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 8, 2024-09-14T23:57:41.389 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 8, 2024-09-14T23:57:41.389 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 8, 2024-09-14T23:57:41.390 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-14T23:57:41.390 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-14T23:57:41.390 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-14T23:57:41.390 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-14T23:57:41.390 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-14T23:57:41.390 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-14T23:57:41.390 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-14T23:57:41.390 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.390 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "21", 2024-09-14T23:57:41.390 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "0", 2024-09-14T23:57:41.390 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T23:57:41.390 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T23:57:41.390 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-14T23:57:41.390 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-14T23:57:41.391 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 2, 2024-09-14T23:57:41.391 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 21, 2024-09-14T23:57:41.391 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-14T23:57:41.391 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-14T23:57:41.391 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-14T23:57:41.391 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-14T23:57:41.391 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-14T23:57:41.391 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-14T23:57:41.391 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-14T23:57:41.391 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-14T23:57:41.391 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-14T23:57:41.391 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-14T23:57:41.391 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-14T23:57:41.391 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T23:57:41.391 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T23:57:41.392 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T23:57:41.392 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-14T23:57:41.392 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-14T23:57:41.392 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-14T23:57:41.392 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-14T23:57:41.392 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-14T23:57:41.392 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.392 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-14T23:57:41.392 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-14T23:57:41.392 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-14T23:57:41.392 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-14T23:57:41.392 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-14T23:57:41.392 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T23:57:41.392 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-14T23:57:41.393 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-14T23:57:41.393 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-14T23:57:41.393 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-14T23:57:41.393 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-14T23:57:41.393 INFO:teuthology.orchestra.run.smithi042.stdout: "options": {}, 2024-09-14T23:57:41.393 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-14T23:57:41.393 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd": {} 2024-09-14T23:57:41.393 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.393 INFO:teuthology.orchestra.run.smithi042.stdout: "read_balance": { 2024-09-14T23:57:41.393 INFO:teuthology.orchestra.run.smithi042.stdout: "score_type": "Fair distribution", 2024-09-14T23:57:41.393 INFO:teuthology.orchestra.run.smithi042.stdout: "score_acting": 2.630000114440918, 2024-09-14T23:57:41.393 INFO:teuthology.orchestra.run.smithi042.stdout: "score_stable": 2.630000114440918, 2024-09-14T23:57:41.393 INFO:teuthology.orchestra.run.smithi042.stdout: "optimal_score": 1, 2024-09-14T23:57:41.393 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_acting": 2.630000114440918, 2024-09-14T23:57:41.394 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_stable": 2.630000114440918, 2024-09-14T23:57:41.394 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity_weighted": 0.8712921142578125, 2024-09-14T23:57:41.394 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity": 1, 2024-09-14T23:57:41.394 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity_weighted": 0.87000000476837158 2024-09-14T23:57:41.394 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:41.394 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.394 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.394 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 3, 2024-09-14T23:57:41.394 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": "pool_client.0", 2024-09-14T23:57:41.394 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-14T23:56:30.861504+0000", 2024-09-14T23:57:41.394 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 8193, 2024-09-14T23:57:41.394 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T23:57:41.394 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-14T23:57:41.394 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-14T23:57:41.394 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-14T23:57:41.395 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-14T23:57:41.395 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-14T23:57:41.395 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-14T23:57:41.395 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T23:57:41.395 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T23:57:41.395 INFO:teuthology.orchestra.run.smithi042.stdout: "is_stretch_pool": false, 2024-09-14T23:57:41.395 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-14T23:57:41.395 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-14T23:57:41.395 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 32, 2024-09-14T23:57:41.395 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 32, 2024-09-14T23:57:41.396 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 32, 2024-09-14T23:57:41.396 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 32, 2024-09-14T23:57:41.396 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 32, 2024-09-14T23:57:41.396 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-14T23:57:41.396 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-14T23:57:41.396 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-14T23:57:41.396 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-14T23:57:41.396 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-14T23:57:41.396 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-14T23:57:41.396 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-14T23:57:41.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.397 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "88", 2024-09-14T23:57:41.397 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "0", 2024-09-14T23:57:41.397 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T23:57:41.397 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T23:57:41.397 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-14T23:57:41.397 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-14T23:57:41.397 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 37, 2024-09-14T23:57:41.397 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 88, 2024-09-14T23:57:41.397 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-14T23:57:41.397 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-14T23:57:41.397 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-14T23:57:41.397 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-14T23:57:41.397 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-14T23:57:41.397 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-14T23:57:41.398 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-14T23:57:41.398 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-14T23:57:41.398 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-14T23:57:41.398 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-14T23:57:41.398 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-14T23:57:41.398 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T23:57:41.398 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T23:57:41.398 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T23:57:41.398 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-14T23:57:41.398 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-14T23:57:41.398 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-14T23:57:41.398 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-14T23:57:41.398 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-14T23:57:41.398 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.398 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-14T23:57:41.399 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-14T23:57:41.399 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-14T23:57:41.399 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-14T23:57:41.399 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-14T23:57:41.399 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T23:57:41.399 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-14T23:57:41.399 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-14T23:57:41.399 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-14T23:57:41.399 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-14T23:57:41.399 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-14T23:57:41.399 INFO:teuthology.orchestra.run.smithi042.stdout: "options": {}, 2024-09-14T23:57:41.399 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-14T23:57:41.399 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd": {} 2024-09-14T23:57:41.399 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.400 INFO:teuthology.orchestra.run.smithi042.stdout: "read_balance": { 2024-09-14T23:57:41.400 INFO:teuthology.orchestra.run.smithi042.stdout: "score_type": "Fair distribution", 2024-09-14T23:57:41.400 INFO:teuthology.orchestra.run.smithi042.stdout: "score_acting": 1.5399999618530273, 2024-09-14T23:57:41.400 INFO:teuthology.orchestra.run.smithi042.stdout: "score_stable": 1.5399999618530273, 2024-09-14T23:57:41.400 INFO:teuthology.orchestra.run.smithi042.stdout: "optimal_score": 1.1399999856948853, 2024-09-14T23:57:41.400 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_acting": 1.75, 2024-09-14T23:57:41.400 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_stable": 1.75, 2024-09-14T23:57:41.400 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity_weighted": 0.8712921142578125, 2024-09-14T23:57:41.400 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity": 1, 2024-09-14T23:57:41.400 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity_weighted": 0.87000000476837158 2024-09-14T23:57:41.400 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:41.400 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:41.400 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-14T23:57:41.400 INFO:teuthology.orchestra.run.smithi042.stdout: "osds": [ 2024-09-14T23:57:41.401 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.401 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 0, 2024-09-14T23:57:41.401 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "db9d0c67-2d60-4592-9ad3-f99e5294f93f", 2024-09-14T23:57:41.401 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-14T23:57:41.401 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-14T23:57:41.401 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-14T23:57:41.401 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-14T23:57:41.401 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-14T23:57:41.401 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-14T23:57:41.401 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 14, 2024-09-14T23:57:41.401 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 49, 2024-09-14T23:57:41.401 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-14T23:57:41.401 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-14T23:57:41.401 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-14T23:57:41.402 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:41.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.402 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:41.402 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6816", 2024-09-14T23:57:41.402 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 314865173 2024-09-14T23:57:41.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.402 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:41.402 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6817", 2024-09-14T23:57:41.402 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 314865173 2024-09-14T23:57:41.402 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:41.402 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:41.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.402 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-14T23:57:41.403 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:41.403 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.403 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:41.403 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6818", 2024-09-14T23:57:41.403 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 314865173 2024-09-14T23:57:41.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.403 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.403 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:41.403 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6819", 2024-09-14T23:57:41.403 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 314865173 2024-09-14T23:57:41.403 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:41.403 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:41.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.403 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-14T23:57:41.404 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:41.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.404 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:41.404 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6822", 2024-09-14T23:57:41.404 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 314865173 2024-09-14T23:57:41.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.404 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:41.404 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6823", 2024-09-14T23:57:41.404 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 314865173 2024-09-14T23:57:41.404 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:41.404 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:41.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.404 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-14T23:57:41.405 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:41.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.405 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:41.405 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6820", 2024-09-14T23:57:41.405 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 314865173 2024-09-14T23:57:41.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.405 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:41.405 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6821", 2024-09-14T23:57:41.405 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 314865173 2024-09-14T23:57:41.405 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:41.405 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:41.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.405 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6817/314865173", 2024-09-14T23:57:41.406 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6819/314865173", 2024-09-14T23:57:41.406 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6823/314865173", 2024-09-14T23:57:41.406 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6821/314865173", 2024-09-14T23:57:41.406 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-14T23:57:41.406 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-14T23:57:41.406 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-14T23:57:41.406 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:41.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.406 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.406 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 1, 2024-09-14T23:57:41.406 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "6c1a2765-c591-4c57-8ab3-ee6645970aa5", 2024-09-14T23:57:41.406 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-14T23:57:41.406 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-14T23:57:41.406 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-14T23:57:41.406 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-14T23:57:41.407 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-14T23:57:41.407 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-14T23:57:41.407 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 14, 2024-09-14T23:57:41.407 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 46, 2024-09-14T23:57:41.407 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-14T23:57:41.407 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-14T23:57:41.407 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-14T23:57:41.407 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:41.407 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.407 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:41.407 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6800", 2024-09-14T23:57:41.407 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3197912310 2024-09-14T23:57:41.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.407 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.408 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:41.408 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6801", 2024-09-14T23:57:41.408 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3197912310 2024-09-14T23:57:41.408 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:41.408 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:41.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.408 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-14T23:57:41.408 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:41.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.408 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:41.408 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6802", 2024-09-14T23:57:41.408 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3197912310 2024-09-14T23:57:41.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.409 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:41.409 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6803", 2024-09-14T23:57:41.409 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3197912310 2024-09-14T23:57:41.409 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:41.409 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:41.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.409 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-14T23:57:41.409 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:41.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.409 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:41.409 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6806", 2024-09-14T23:57:41.410 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3197912310 2024-09-14T23:57:41.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.410 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.410 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:41.410 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6807", 2024-09-14T23:57:41.410 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3197912310 2024-09-14T23:57:41.410 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:41.410 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:41.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.410 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-14T23:57:41.410 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:41.410 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.410 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:41.410 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6804", 2024-09-14T23:57:41.411 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3197912310 2024-09-14T23:57:41.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.411 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.411 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:41.411 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6805", 2024-09-14T23:57:41.411 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3197912310 2024-09-14T23:57:41.411 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:41.411 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:41.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.411 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6801/3197912310", 2024-09-14T23:57:41.411 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6803/3197912310", 2024-09-14T23:57:41.411 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6807/3197912310", 2024-09-14T23:57:41.411 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6805/3197912310", 2024-09-14T23:57:41.411 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-14T23:57:41.411 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-14T23:57:41.412 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-14T23:57:41.412 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:41.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.412 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.412 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 2, 2024-09-14T23:57:41.412 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "930db893-7be6-4645-a7f1-e043c5ec95eb", 2024-09-14T23:57:41.412 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-14T23:57:41.412 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-14T23:57:41.412 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-14T23:57:41.412 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-14T23:57:41.412 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-14T23:57:41.412 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-14T23:57:41.412 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 13, 2024-09-14T23:57:41.412 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 46, 2024-09-14T23:57:41.412 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-14T23:57:41.413 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-14T23:57:41.413 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-14T23:57:41.413 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:41.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.413 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:41.413 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6808", 2024-09-14T23:57:41.413 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3241153660 2024-09-14T23:57:41.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.413 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:41.413 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6809", 2024-09-14T23:57:41.413 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3241153660 2024-09-14T23:57:41.413 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:41.413 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:41.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.414 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-14T23:57:41.414 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:41.414 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.414 INFO:teuthology.orchestra.run.smithi042.stdout:3807 write 0xe90e1c thru 0xe9ac91 (0x9e76 bytes) 2024-09-14T23:57:41.415 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:41.415 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6810", 2024-09-14T23:57:41.415 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3241153660 2024-09-14T23:57:41.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.415 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.415 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:41.415 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6811", 2024-09-14T23:57:41.415 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3241153660 2024-09-14T23:57:41.415 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:41.415 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:41.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.415 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-14T23:57:41.416 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:41.416 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.416 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:41.416 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6814", 2024-09-14T23:57:41.416 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3241153660 2024-09-14T23:57:41.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.416 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.416 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:41.416 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6815", 2024-09-14T23:57:41.416 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3241153660 2024-09-14T23:57:41.416 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:41.416 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:41.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.416 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-14T23:57:41.417 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:41.417 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.417 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:41.417 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6812", 2024-09-14T23:57:41.417 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3241153660 2024-09-14T23:57:41.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.417 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.417 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:41.417 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6813", 2024-09-14T23:57:41.417 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3241153660 2024-09-14T23:57:41.417 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:41.417 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:41.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.417 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6809/3241153660", 2024-09-14T23:57:41.418 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6811/3241153660", 2024-09-14T23:57:41.418 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6815/3241153660", 2024-09-14T23:57:41.418 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6813/3241153660", 2024-09-14T23:57:41.418 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-14T23:57:41.418 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-14T23:57:41.418 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-14T23:57:41.418 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:41.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.418 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 3, 2024-09-14T23:57:41.418 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "45bb3b84-b1c4-4f87-a7d1-0eb77b81741b", 2024-09-14T23:57:41.418 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-14T23:57:41.418 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-14T23:57:41.418 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-14T23:57:41.419 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-14T23:57:41.419 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-14T23:57:41.419 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-14T23:57:41.419 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-14T23:57:41.419 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 75, 2024-09-14T23:57:41.419 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-14T23:57:41.419 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-14T23:57:41.419 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-14T23:57:41.419 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:41.419 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.419 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:41.419 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6824", 2024-09-14T23:57:41.419 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1736416326 2024-09-14T23:57:41.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.420 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.420 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:41.420 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6825", 2024-09-14T23:57:41.420 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1736416326 2024-09-14T23:57:41.420 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:41.420 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:41.420 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.420 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-14T23:57:41.420 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:41.421 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.421 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:41.421 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6826", 2024-09-14T23:57:41.421 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1736416326 2024-09-14T23:57:41.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.421 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.421 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:41.421 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6827", 2024-09-14T23:57:41.421 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1736416326 2024-09-14T23:57:41.421 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:41.421 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:41.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.421 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-14T23:57:41.422 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:41.422 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.422 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:41.422 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6830", 2024-09-14T23:57:41.422 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1736416326 2024-09-14T23:57:41.422 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.422 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.422 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:41.422 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6831", 2024-09-14T23:57:41.422 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1736416326 2024-09-14T23:57:41.422 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:41.422 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:41.422 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.423 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-14T23:57:41.423 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:41.423 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.423 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:41.423 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6828", 2024-09-14T23:57:41.423 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1736416326 2024-09-14T23:57:41.423 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.423 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.423 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:41.423 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6829", 2024-09-14T23:57:41.423 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1736416326 2024-09-14T23:57:41.423 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:41.423 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:41.423 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.424 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6825/1736416326", 2024-09-14T23:57:41.424 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6827/1736416326", 2024-09-14T23:57:41.424 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6831/1736416326", 2024-09-14T23:57:41.424 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6829/1736416326", 2024-09-14T23:57:41.424 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-14T23:57:41.424 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-14T23:57:41.424 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-14T23:57:41.424 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:41.424 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.424 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.424 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 4, 2024-09-14T23:57:41.424 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "a812e38f-c0f9-4d33-9373-1910fb09103c", 2024-09-14T23:57:41.424 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-14T23:57:41.424 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-14T23:57:41.425 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-14T23:57:41.425 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 0.9703369140625, 2024-09-14T23:57:41.425 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-14T23:57:41.425 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-14T23:57:41.425 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-14T23:57:41.425 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 49, 2024-09-14T23:57:41.425 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-14T23:57:41.425 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-14T23:57:41.425 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-14T23:57:41.425 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:41.425 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.425 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:41.425 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6808", 2024-09-14T23:57:41.425 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1755154427 2024-09-14T23:57:41.425 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.426 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.426 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:41.426 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6809", 2024-09-14T23:57:41.426 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1755154427 2024-09-14T23:57:41.426 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:41.426 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:41.426 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.426 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-14T23:57:41.426 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:41.426 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.426 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:41.426 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6810", 2024-09-14T23:57:41.426 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1755154427 2024-09-14T23:57:41.426 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.427 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.427 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:41.427 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6811", 2024-09-14T23:57:41.427 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1755154427 2024-09-14T23:57:41.427 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:41.427 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:41.427 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.427 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-14T23:57:41.427 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:41.427 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.427 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:41.427 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6814", 2024-09-14T23:57:41.427 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1755154427 2024-09-14T23:57:41.427 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.428 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.428 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:41.428 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6815", 2024-09-14T23:57:41.428 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1755154427 2024-09-14T23:57:41.428 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:41.428 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:41.428 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.428 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-14T23:57:41.428 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:41.428 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.428 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:41.428 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6812", 2024-09-14T23:57:41.428 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1755154427 2024-09-14T23:57:41.428 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.429 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:41.429 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6813", 2024-09-14T23:57:41.429 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1755154427 2024-09-14T23:57:41.429 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:41.429 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:41.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.429 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.43:6809/1755154427", 2024-09-14T23:57:41.429 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.43:6811/1755154427", 2024-09-14T23:57:41.429 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.43:6815/1755154427", 2024-09-14T23:57:41.429 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.43:6813/1755154427", 2024-09-14T23:57:41.429 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-14T23:57:41.429 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-14T23:57:41.429 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-14T23:57:41.429 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:41.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.430 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 5, 2024-09-14T23:57:41.430 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "0de87e5b-7a21-4fb4-9543-d96c065786f8", 2024-09-14T23:57:41.430 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-14T23:57:41.430 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-14T23:57:41.430 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-14T23:57:41.430 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 0, 2024-09-14T23:57:41.430 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-14T23:57:41.430 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-14T23:57:41.430 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-14T23:57:41.430 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 23, 2024-09-14T23:57:41.430 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-14T23:57:41.430 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-14T23:57:41.431 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-14T23:57:41.431 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:41.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.431 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:41.431 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6816", 2024-09-14T23:57:41.431 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3033769028 2024-09-14T23:57:41.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.431 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:41.431 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6817", 2024-09-14T23:57:41.431 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3033769028 2024-09-14T23:57:41.431 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:41.431 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:41.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.431 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-14T23:57:41.432 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:41.432 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.432 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:41.432 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6818", 2024-09-14T23:57:41.432 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3033769028 2024-09-14T23:57:41.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.432 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.432 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:41.432 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6819", 2024-09-14T23:57:41.432 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3033769028 2024-09-14T23:57:41.432 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:41.432 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:41.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.432 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-14T23:57:41.433 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:41.433 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.433 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:41.433 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6822", 2024-09-14T23:57:41.433 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3033769028 2024-09-14T23:57:41.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.433 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.433 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:41.433 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6823", 2024-09-14T23:57:41.433 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3033769028 2024-09-14T23:57:41.433 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:41.433 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:41.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.434 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-14T23:57:41.434 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:41.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.434 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:41.434 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6820", 2024-09-14T23:57:41.434 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3033769028 2024-09-14T23:57:41.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.434 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:41.434 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6821", 2024-09-14T23:57:41.434 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3033769028 2024-09-14T23:57:41.434 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:41.434 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:41.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.435 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.43:6817/3033769028", 2024-09-14T23:57:41.435 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.43:6819/3033769028", 2024-09-14T23:57:41.435 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.43:6823/3033769028", 2024-09-14T23:57:41.435 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.43:6821/3033769028", 2024-09-14T23:57:41.435 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-14T23:57:41.435 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-14T23:57:41.435 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-14T23:57:41.435 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:41.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.435 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 6, 2024-09-14T23:57:41.435 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "c58c1d59-ad02-40f0-98e3-9078ef7e622b", 2024-09-14T23:57:41.435 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-14T23:57:41.435 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-14T23:57:41.435 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-14T23:57:41.436 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-14T23:57:41.436 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-14T23:57:41.436 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-14T23:57:41.436 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-14T23:57:41.436 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 76, 2024-09-14T23:57:41.436 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-14T23:57:41.436 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-14T23:57:41.436 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-14T23:57:41.436 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:41.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.436 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:41.436 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6800", 2024-09-14T23:57:41.436 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3314282416 2024-09-14T23:57:41.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.437 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:41.437 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6801", 2024-09-14T23:57:41.437 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3314282416 2024-09-14T23:57:41.437 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:41.437 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:41.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.437 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-14T23:57:41.437 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:41.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.437 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:41.438 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6802", 2024-09-14T23:57:41.438 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3314282416 2024-09-14T23:57:41.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.438 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.438 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:41.438 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6803", 2024-09-14T23:57:41.438 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3314282416 2024-09-14T23:57:41.438 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:41.438 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:41.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.438 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-14T23:57:41.438 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:41.438 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.438 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:41.439 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6806", 2024-09-14T23:57:41.439 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3314282416 2024-09-14T23:57:41.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.439 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.439 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:41.439 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6807", 2024-09-14T23:57:41.439 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3314282416 2024-09-14T23:57:41.439 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:41.439 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:41.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.439 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-14T23:57:41.439 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:41.439 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.439 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:41.440 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6804", 2024-09-14T23:57:41.440 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3314282416 2024-09-14T23:57:41.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.440 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.440 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:41.440 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6805", 2024-09-14T23:57:41.440 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3314282416 2024-09-14T23:57:41.440 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:41.440 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:41.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.440 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.43:6801/3314282416", 2024-09-14T23:57:41.440 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.43:6803/3314282416", 2024-09-14T23:57:41.440 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.43:6807/3314282416", 2024-09-14T23:57:41.441 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.43:6805/3314282416", 2024-09-14T23:57:41.441 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-14T23:57:41.441 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-14T23:57:41.441 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-14T23:57:41.441 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:41.441 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.441 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.441 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 7, 2024-09-14T23:57:41.441 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "389bdb04-20e3-4a7a-8504-687e034bd7dc", 2024-09-14T23:57:41.441 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-14T23:57:41.442 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-14T23:57:41.442 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-14T23:57:41.442 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-14T23:57:41.442 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-14T23:57:41.442 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-14T23:57:41.442 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-14T23:57:41.442 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 75, 2024-09-14T23:57:41.442 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-14T23:57:41.442 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-14T23:57:41.442 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-14T23:57:41.442 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:41.442 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.442 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:41.443 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6824", 2024-09-14T23:57:41.443 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1481350118 2024-09-14T23:57:41.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.443 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.443 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:41.443 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6825", 2024-09-14T23:57:41.443 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1481350118 2024-09-14T23:57:41.443 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:41.443 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:41.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.443 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-14T23:57:41.443 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:41.443 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.443 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:41.444 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6826", 2024-09-14T23:57:41.444 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1481350118 2024-09-14T23:57:41.444 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.444 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.444 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:41.444 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6827", 2024-09-14T23:57:41.444 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1481350118 2024-09-14T23:57:41.444 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:41.444 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:41.444 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.444 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-14T23:57:41.444 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:41.444 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.445 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:41.445 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6830", 2024-09-14T23:57:41.445 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1481350118 2024-09-14T23:57:41.445 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.445 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.445 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:41.445 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6831", 2024-09-14T23:57:41.445 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1481350118 2024-09-14T23:57:41.445 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:41.445 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:41.445 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.445 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-14T23:57:41.445 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:41.445 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.445 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:41.446 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6828", 2024-09-14T23:57:41.446 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1481350118 2024-09-14T23:57:41.446 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.446 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.446 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:41.446 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6829", 2024-09-14T23:57:41.446 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1481350118 2024-09-14T23:57:41.446 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:41.446 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:41.446 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.446 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.43:6825/1481350118", 2024-09-14T23:57:41.446 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.43:6827/1481350118", 2024-09-14T23:57:41.446 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.43:6831/1481350118", 2024-09-14T23:57:41.446 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.43:6829/1481350118", 2024-09-14T23:57:41.447 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-14T23:57:41.447 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-14T23:57:41.447 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-14T23:57:41.447 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:41.447 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:41.447 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-14T23:57:41.447 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_xinfo": [ 2024-09-14T23:57:41.447 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.447 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 0, 2024-09-14T23:57:41.447 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-14T23:57:41.447 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-14T23:57:41.447 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-14T23:57:41.447 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-14T23:57:41.447 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-14T23:57:41.447 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-14T23:56:11.077810+0000", 2024-09-14T23:57:41.448 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-14T23:57:41.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.448 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.448 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 1, 2024-09-14T23:57:41.448 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-14T23:57:41.448 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-14T23:57:41.448 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-14T23:57:41.448 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-14T23:57:41.448 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-14T23:57:41.448 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-14T23:56:11.578838+0000", 2024-09-14T23:57:41.448 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-14T23:57:41.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.448 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.449 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 2, 2024-09-14T23:57:41.449 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-14T23:57:41.449 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-14T23:57:41.449 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-14T23:57:41.449 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-14T23:57:41.449 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-14T23:57:41.449 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-14T23:56:10.895490+0000", 2024-09-14T23:57:41.449 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-14T23:57:41.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.449 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.449 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 3, 2024-09-14T23:57:41.449 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-14T23:57:41.449 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-14T23:57:41.450 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-14T23:57:41.450 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-14T23:57:41.450 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-14T23:57:41.450 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-14T23:56:12.290821+0000", 2024-09-14T23:57:41.450 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-14T23:57:41.450 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.450 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.450 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 4, 2024-09-14T23:57:41.450 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-14T23:57:41.450 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-14T23:57:41.450 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-14T23:57:41.450 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-14T23:57:41.450 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-14T23:57:41.450 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-14T23:56:11.800581+0000", 2024-09-14T23:57:41.451 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-14T23:57:41.451 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.451 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.451 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 5, 2024-09-14T23:57:41.451 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-14T23:57:41.451 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-14T23:57:41.451 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-14T23:57:41.451 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-14T23:57:41.451 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-14T23:57:41.451 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-14T23:56:11.883155+0000", 2024-09-14T23:57:41.451 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-14T23:57:41.451 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.451 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.451 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 6, 2024-09-14T23:57:41.452 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-14T23:57:41.452 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-14T23:57:41.452 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-14T23:57:41.452 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-14T23:57:41.452 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-14T23:57:41.452 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-14T23:56:11.604790+0000", 2024-09-14T23:57:41.452 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-14T23:57:41.452 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.452 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.452 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 7, 2024-09-14T23:57:41.452 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-14T23:57:41.452 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-14T23:57:41.452 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-14T23:57:41.453 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-14T23:57:41.453 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-14T23:57:41.453 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-14T23:56:12.067759+0000", 2024-09-14T23:57:41.453 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-14T23:57:41.453 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:41.453 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-14T23:57:41.453 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_upmap": [ 2024-09-14T23:57:41.453 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:41.453 INFO:teuthology.orchestra.run.smithi042.stdout: "pgid": "3.1e", 2024-09-14T23:57:41.453 INFO:teuthology.orchestra.run.smithi042.stdout: "osds": [ 2024-09-14T23:57:41.453 INFO:teuthology.orchestra.run.smithi042.stdout: 6, 2024-09-14T23:57:41.453 INFO:teuthology.orchestra.run.smithi042.stdout: 1 2024-09-14T23:57:41.453 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:41.453 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:41.454 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-14T23:57:41.454 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_upmap_items": [], 2024-09-14T23:57:41.454 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_upmap_primaries": [], 2024-09-14T23:57:41.454 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_temp": [], 2024-09-14T23:57:41.454 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_temp": [], 2024-09-14T23:57:41.454 INFO:teuthology.orchestra.run.smithi042.stdout: "blocklist": {}, 2024-09-14T23:57:41.454 INFO:teuthology.orchestra.run.smithi042.stdout: "range_blocklist": {}, 2024-09-14T23:57:41.454 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profiles": { 2024-09-14T23:57:41.454 INFO:teuthology.orchestra.run.smithi042.stdout: "default": { 2024-09-14T23:57:41.454 INFO:teuthology.orchestra.run.smithi042.stdout: "crush-failure-domain": "osd", 2024-09-14T23:57:41.454 INFO:teuthology.orchestra.run.smithi042.stdout: "k": "2", 2024-09-14T23:57:41.454 INFO:teuthology.orchestra.run.smithi042.stdout: "m": "1", 2024-09-14T23:57:41.454 INFO:teuthology.orchestra.run.smithi042.stdout: "plugin": "jerasure", 2024-09-14T23:57:41.454 INFO:teuthology.orchestra.run.smithi042.stdout: "technique": "reed_sol_van" 2024-09-14T23:57:41.455 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:41.455 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:41.455 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps_queue": [], 2024-09-14T23:57:41.455 INFO:teuthology.orchestra.run.smithi042.stdout: "new_removed_snaps": [], 2024-09-14T23:57:41.455 INFO:teuthology.orchestra.run.smithi042.stdout: "new_purged_snaps": [], 2024-09-14T23:57:41.455 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_node_flags": {}, 2024-09-14T23:57:41.455 INFO:teuthology.orchestra.run.smithi042.stdout: "device_class_flags": {}, 2024-09-14T23:57:41.455 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode": { 2024-09-14T23:57:41.455 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode_enabled": false, 2024-09-14T23:57:41.455 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_bucket_count": 0, 2024-09-14T23:57:41.455 INFO:teuthology.orchestra.run.smithi042.stdout: "degraded_stretch_mode": 0, 2024-09-14T23:57:41.455 INFO:teuthology.orchestra.run.smithi042.stdout: "recovering_stretch_mode": 0, 2024-09-14T23:57:41.455 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode_bucket": 0 2024-09-14T23:57:41.455 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:41.456 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:41.456 INFO:teuthology.orchestra.run.smithi042.stdout:3808 trunc from 0x91837e6 to 0xfade4a 2024-09-14T23:57:41.457 INFO:tasks.thrashosds.thrasher:j is {'epoch': 88, 'fsid': '21916fcc-f95c-4bfa-8b8b-11ca45845613', 'created': '2024-09-14T23:56:03.167948+0000', 'modified': '2024-09-14T23:57:40.021278+0000', 'last_up_change': '2024-09-14T23:56:14.322882+0000', 'last_in_change': '2024-09-14T23:56:56.035972+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T23:56:13.500197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8712921142578125, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8700000047683716}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T23:56:21.284705+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.630000114440918, 'score_stable': 2.630000114440918, 'optimal_score': 1, 'raw_score_acting': 2.630000114440918, 'raw_score_stable': 2.630000114440918, 'primary_affinity_weighted': 0.8712921142578125, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8700000047683716}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T23:56:30.861504+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 37, 'snap_epoch': 88, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5399999618530273, 'score_stable': 1.5399999618530273, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.8712921142578125, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8700000047683716}}], 'osds': [{'osd': 0, 'uuid': 'db9d0c67-2d60-4592-9ad3-f99e5294f93f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 49, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6816', 'nonce': 314865173}, {'type': 'v1', 'addr': '172.21.15.42:6817', 'nonce': 314865173}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6818', 'nonce': 314865173}, {'type': 'v1', 'addr': '172.21.15.42:6819', 'nonce': 314865173}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6822', 'nonce': 314865173}, {'type': 'v1', 'addr': '172.21.15.42:6823', 'nonce': 314865173}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6820', 'nonce': 314865173}, {'type': 'v1', 'addr': '172.21.15.42:6821', 'nonce': 314865173}]}, 'public_addr': '172.21.15.42:6817/314865173', 'cluster_addr': '172.21.15.42:6819/314865173', 'heartbeat_back_addr': '172.21.15.42:6823/314865173', 'heartbeat_front_addr': '172.21.15.42:6821/314865173', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '6c1a2765-c591-4c57-8ab3-ee6645970aa5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 46, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6800', 'nonce': 3197912310}, {'type': 'v1', 'addr': '172.21.15.42:6801', 'nonce': 3197912310}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6802', 'nonce': 3197912310}, {'type': 'v1', 'addr': '172.21.15.42:6803', 'nonce': 3197912310}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6806', 'nonce': 3197912310}, {'type': 'v1', 'addr': '172.21.15.42:6807', 'nonce': 3197912310}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6804', 'nonce': 3197912310}, {'type': 'v1', 'addr': '172.21.15.42:6805', 'nonce': 3197912310}]}, 'public_addr': '172.21.15.42:6801/3197912310', 'cluster_addr': '172.21.15.42:6803/3197912310', 'heartbeat_back_addr': '172.21.15.42:6807/3197912310', 'heartbeat_front_addr': '172.21.15.42:6805/3197912310', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '930db893-7be6-4645-a7f1-e043c5ec95eb', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 46, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6808', 'nonce': 3241153660}, {'type': 'v1', 'addr': '172.21.15.42:6809', 'nonce': 3241153660}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6810', 'nonce': 3241153660}, {'type': 'v1', 'addr': '172.21.15.42:6811', 'nonce': 3241153660}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6814', 'nonce': 3241153660}, {'type': 'v1', 'addr': '172.21.15.42:6815', 'nonce': 3241153660}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6812', 'nonce': 3241153660}, {'type': 'v1', 'addr': '172.21.15.42:6813', 'nonce': 3241153660}]}, 'public_addr': '172.21.15.42:6809/3241153660', 'cluster_addr': '172.21.15.42:6811/3241153660', 'heartbeat_back_addr': '172.21.15.42:6815/3241153660', 'heartbeat_front_addr': '172.21.15.42:6813/3241153660', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '45bb3b84-b1c4-4f87-a7d1-0eb77b81741b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 75, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6824', 'nonce': 1736416326}, {'type': 'v1', 'addr': '172.21.15.42:6825', 'nonce': 1736416326}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6826', 'nonce': 1736416326}, {'type': 'v1', 'addr': '172.21.15.42:6827', 'nonce': 1736416326}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6830', 'nonce': 1736416326}, {'type': 'v1', 'addr': '172.21.15.42:6831', 'nonce': 1736416326}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6828', 'nonce': 1736416326}, {'type': 'v1', 'addr': '172.21.15.42:6829', 'nonce': 1736416326}]}, 'public_addr': '172.21.15.42:6825/1736416326', 'cluster_addr': '172.21.15.42:6827/1736416326', 'heartbeat_back_addr': '172.21.15.42:6831/1736416326', 'heartbeat_front_addr': '172.21.15.42:6829/1736416326', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'a812e38f-c0f9-4d33-9373-1910fb09103c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.9703369140625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 49, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6808', 'nonce': 1755154427}, {'type': 'v1', 'addr': '172.21.15.43:6809', 'nonce': 1755154427}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6810', 'nonce': 1755154427}, {'type': 'v1', 'addr': '172.21.15.43:6811', 'nonce': 1755154427}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6814', 'nonce': 1755154427}, {'type': 'v1', 'addr': '172.21.15.43:6815', 'nonce': 1755154427}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6812', 'nonce': 1755154427}, {'type': 'v1', 'addr': '172.21.15.43:6813', 'nonce': 1755154427}]}, 'public_addr': '172.21.15.43:6809/1755154427', 'cluster_addr': '172.21.15.43:6811/1755154427', 'heartbeat_back_addr': '172.21.15.43:6815/1755154427', 'heartbeat_front_addr': '172.21.15.43:6813/1755154427', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '0de87e5b-7a21-4fb4-9543-d96c065786f8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6816', 'nonce': 3033769028}, {'type': 'v1', 'addr': '172.21.15.43:6817', 'nonce': 3033769028}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6818', 'nonce': 3033769028}, {'type': 'v1', 'addr': '172.21.15.43:6819', 'nonce': 3033769028}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6822', 'nonce': 3033769028}, {'type': 'v1', 'addr': '172.21.15.43:6823', 'nonce': 3033769028}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6820', 'nonce': 3033769028}, {'type': 'v1', 'addr': '172.21.15.43:6821', 'nonce': 3033769028}]}, 'public_addr': '172.21.15.43:6817/3033769028', 'cluster_addr': '172.21.15.43:6819/3033769028', 'heartbeat_back_addr': '172.21.15.43:6823/3033769028', 'heartbeat_front_addr': '172.21.15.43:6821/3033769028', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'c58c1d59-ad02-40f0-98e3-9078ef7e622b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 76, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6800', 'nonce': 3314282416}, {'type': 'v1', 'addr': '172.21.15.43:6801', 'nonce': 3314282416}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6802', 'nonce': 3314282416}, {'type': 'v1', 'addr': '172.21.15.43:6803', 'nonce': 3314282416}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6806', 'nonce': 3314282416}, {'type': 'v1', 'addr': '172.21.15.43:6807', 'nonce': 3314282416}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6804', 'nonce': 3314282416}, {'type': 'v1', 'addr': '172.21.15.43:6805', 'nonce': 3314282416}]}, 'public_addr': '172.21.15.43:6801/3314282416', 'cluster_addr': '172.21.15.43:6803/3314282416', 'heartbeat_back_addr': '172.21.15.43:6807/3314282416', 'heartbeat_front_addr': '172.21.15.43:6805/3314282416', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '389bdb04-20e3-4a7a-8504-687e034bd7dc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 75, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6824', 'nonce': 1481350118}, {'type': 'v1', 'addr': '172.21.15.43:6825', 'nonce': 1481350118}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6826', 'nonce': 1481350118}, {'type': 'v1', 'addr': '172.21.15.43:6827', 'nonce': 1481350118}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6830', 'nonce': 1481350118}, {'type': 'v1', 'addr': '172.21.15.43:6831', 'nonce': 1481350118}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6828', 'nonce': 1481350118}, {'type': 'v1', 'addr': '172.21.15.43:6829', 'nonce': 1481350118}]}, 'public_addr': '172.21.15.43:6825/1481350118', 'cluster_addr': '172.21.15.43:6827/1481350118', 'heartbeat_back_addr': '172.21.15.43:6831/1481350118', 'heartbeat_front_addr': '172.21.15.43:6829/1481350118', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T23:56:11.077810+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T23:56:11.578838+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T23:56:10.895490+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T23:56:12.290821+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T23:56:11.800581+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T23:56:11.883155+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T23:56:11.604790+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T23:56:12.067759+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.1e', 'osds': [6, 1]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T23:57:41.458 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T23:57:41.459 INFO:teuthology.orchestra.run.smithi042.stdout:3809 punch from 0x4f61b0 to 0x4fd0a7, (0x6ef7 bytes) 2024-09-14T23:57:41.459 INFO:teuthology.orchestra.run.smithi042.stdout:3810 read 0x162bd0 thru 0x16abdb (0x800c bytes) 2024-09-14T23:57:41.459 INFO:teuthology.orchestra.run.smithi042.stdout:3811 trunc from 0xfade4a to 0x923a992 2024-09-14T23:57:41.460 INFO:teuthology.orchestra.run.smithi042.stdout:3813 trunc from 0x923a992 to 0x466b7ab 2024-09-14T23:57:41.460 INFO:teuthology.orchestra.run.smithi042.stdout:3814 trunc from 0x466b7ab to 0x54db76f 2024-09-14T23:57:41.460 INFO:teuthology.orchestra.run.smithi042.stdout:3817 read 0x4530898 thru 0x45312ee (0xa57 bytes) 2024-09-14T23:57:41.460 INFO:teuthology.orchestra.run.smithi042.stdout:3818 punch from 0x1253d8b to 0x125a8bb, (0x6b30 bytes) 2024-09-14T23:57:41.460 INFO:teuthology.orchestra.run.smithi042.stdout:3819 write 0xf8eb8c thru 0xf9647b (0x78f0 bytes) 2024-09-14T23:57:41.460 INFO:teuthology.orchestra.run.smithi042.stdout:3820 write 0xb413f0a thru 0xb417c5e (0x3d55 bytes) 2024-09-14T23:57:41.460 INFO:teuthology.orchestra.run.smithi042.stdout:3821 read 0x71682d4 thru 0x7175783 (0xd4b0 bytes) 2024-09-14T23:57:41.460 INFO:teuthology.orchestra.run.smithi042.stdout:3822 write 0xaa5270 thru 0xaac194 (0x6f25 bytes) 2024-09-14T23:57:41.460 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:41.460 INFO:teuthology.orchestra.run.smithi042.stdout:3825 read 0x221ed34 thru 0x222d321 (0xe5ee bytes) 2024-09-14T23:57:41.461 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:41.461 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:41.461 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T23:57:41.463 INFO:teuthology.orchestra.run.smithi042.stdout:3826 trunc from 0xb417c5f to 0x45094b4 2024-09-14T23:57:41.485 INFO:teuthology.orchestra.run.smithi042.stdout:3827 write 0xa3a8955 thru 0xa3b3945 (0xaff1 bytes) 2024-09-14T23:57:41.485 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:41.484+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:41.488 INFO:teuthology.orchestra.run.smithi042.stdout:3829 writesame 0xe44b55 thru 0xe47d6f data_size 0x1(0x321b bytes) 2024-09-14T23:57:41.501 INFO:teuthology.orchestra.run.smithi042.stdout:3833 write 0xa542ed5 thru 0xa54d45e (0xa58a bytes) 2024-09-14T23:57:41.511 INFO:teuthology.orchestra.run.smithi042.stdout:3834 write 0xab60e44 thru 0xab6cf5e (0xc11b bytes) 2024-09-14T23:57:41.512 INFO:teuthology.orchestra.run.smithi042.stdout:3835 write 0x351c30 thru 0x35a069 (0x843a bytes) 2024-09-14T23:57:41.514 INFO:teuthology.orchestra.run.smithi042.stdout:3836 read 0x4c66bb3 thru 0x4c6d82b (0x6c79 bytes) 2024-09-14T23:57:41.515 INFO:teuthology.orchestra.run.smithi042.stdout:3838 trunc from 0xab6cf5f to 0xa058f80 2024-09-14T23:57:41.526 INFO:teuthology.orchestra.run.smithi042.stdout:3839 trunc from 0xa058f80 to 0xcf12e20 2024-09-14T23:57:41.532 INFO:teuthology.orchestra.run.smithi042.stdout:3840 read 0x4eeb458 thru 0x4ef9ac0 (0xe669 bytes) 2024-09-14T23:57:41.532 INFO:teuthology.orchestra.run.smithi042.stdout:3841 write 0xa12ba64 thru 0xa12df6a (0x2507 bytes) 2024-09-14T23:57:41.535 INFO:teuthology.orchestra.run.smithi042.stdout:3842 write 0x252b3 thru 0x28cb2 (0x3a00 bytes) 2024-09-14T23:57:41.536 INFO:teuthology.orchestra.run.smithi042.stdout:3844 write 0xa3102a4 thru 0xa310ebf (0xc1c bytes) 2024-09-14T23:57:41.538 INFO:teuthology.orchestra.run.smithi042.stdout:3845 writesame 0x2a0ee2a thru 0x2a18f7b data_size 0x3(0xa152 bytes) 2024-09-14T23:57:41.545 INFO:teuthology.orchestra.run.smithi042.stdout:3846 write 0x5176303 thru 0x517e98e (0x868c bytes) 2024-09-14T23:57:41.557 INFO:teuthology.orchestra.run.smithi042.stdout:3847 trunc from 0xcf12e20 to 0x59d8eeb 2024-09-14T23:57:41.568 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:41.568 INFO:teuthology.orchestra.run.smithi042.stdout:3850 write 0x1758f47 thru 0x1760530 (0x75ea bytes) 2024-09-14T23:57:41.569 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:41.567+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:41.569 INFO:teuthology.orchestra.run.smithi042.stdout:3851 write 0x33bdd72 thru 0x33c9877 (0xbb06 bytes) 2024-09-14T23:57:41.571 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:41.571 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:41.571 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:41.574 INFO:teuthology.orchestra.run.smithi042.stdout:3860 trunc from 0x59d8eeb to 0x84ead95 2024-09-14T23:57:41.582 INFO:teuthology.orchestra.run.smithi042.stdout:3861 trunc from 0x84ead95 to 0x6c14e85 2024-09-14T23:57:41.593 INFO:teuthology.orchestra.run.smithi042.stdout:3862 trunc from 0x6c14e85 to 0xa673309 2024-09-14T23:57:41.598 INFO:teuthology.orchestra.run.smithi042.stdout:3864 punch from 0x39029d8 to 0x390f6e1, (0xcd09 bytes) 2024-09-14T23:57:41.606 INFO:teuthology.orchestra.run.smithi042.stdout:3865 trunc from 0xa673309 to 0x3f87192 2024-09-14T23:57:41.615 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:41.616 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:41.617 INFO:teuthology.orchestra.run.smithi042.stdout:3866 writesame 0x88aa4e thru 0x8916b0 data_size 0x3(0x6c63 bytes) 2024-09-14T23:57:41.622 INFO:teuthology.orchestra.run.smithi042.stdout:3867 read 0x3c3a3fb thru 0x3c44588 (0xa18e bytes) 2024-09-14T23:57:41.625 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T23:57:41.632 INFO:teuthology.orchestra.run.smithi042.stdout:3868 trunc from 0x3f87192 to 0xbdf9eb6 2024-09-14T23:57:41.644 INFO:teuthology.orchestra.run.smithi042.stdout:3869 write 0xc3abb13 thru 0xc3b3200 (0x76ee bytes) 2024-09-14T23:57:41.647 INFO:teuthology.orchestra.run.smithi042.stdout:3870 clone 36 order 24 su 2097152 sc 12 2024-09-14T23:57:41.670 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:41.668+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:41.734 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:41.734 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:41.742 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T23:57:41.777 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:57:41.778 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-14T23:57:41.785 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:41.784+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:41.789 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":126,"stamp":"2024-09-14T23:57:41.467214+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":883464000,"num_objects":2711,"num_object_clones":545,"num_object_copies":5422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2711,"num_whiteouts":1502,"num_read":157923,"num_read_kb":650659,"num_write":15253,"num_write_kb":366510,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1171,"num_bytes_recovered":61248546,"num_keys_recovered":133,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13120,"ondisk_log_size":13120,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":357004,"kb_used_data":64500,"kb_used_omap":12,"kb_used_meta":292339,"kb_avail":754617716,"statfs":{"total":773094113280,"available":772728541184,"internally_reserved":0,"allocated":66048000,"data_stored":540326955,"data_compressed":40990843,"data_compressed_allocated":43220992,"data_compressed_original":523940602,"omap_allocated":12752,"internal_metadata":299355696},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":94172344,"num_objects":510,"num_object_clones":127,"num_object_copies":1020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":510,"num_whiteouts":404,"num_read":13838,"num_read_kb":56987,"num_write":1393,"num_write_kb":61566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.455412"},"pg_stats":[{"pgid":"3.1f","version":"87'793","reported_seq":3048,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T23:57:38.186630+0000","last_change":"2024-09-14T23:57:00.062727+0000","last_active":"2024-09-14T23:57:38.186630+0000","last_peered":"2024-09-14T23:57:38.186630+0000","last_clean":"2024-09-14T23:57:38.186630+0000","last_became_active":"2024-09-14T23:57:00.062116+0000","last_became_peered":"2024-09-14T23:57:00.062116+0000","last_unstale":"2024-09-14T23:57:38.186630+0000","last_undegraded":"2024-09-14T23:57:38.186630+0000","last_fullsized":"2024-09-14T23:57:38.186630+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":793,"log_dups_size":0,"ondisk_log_size":793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:18:52.325381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015692659999999999,"stat_sum":{"num_bytes":24041195,"num_objects":93,"num_object_clones":19,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":47,"num_read":5757,"num_read_kb":15474,"num_write":1083,"num_write_kb":21200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":2327658,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"87'263","reported_seq":2423,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T23:57:40.032307+0000","last_change":"2024-09-14T23:57:27.976532+0000","last_active":"2024-09-14T23:57:40.032307+0000","last_peered":"2024-09-14T23:57:40.032307+0000","last_clean":"2024-09-14T23:57:40.032307+0000","last_became_active":"2024-09-14T23:57:27.974745+0000","last_became_peered":"2024-09-14T23:57:27.974745+0000","last_unstale":"2024-09-14T23:57:40.032307+0000","last_undegraded":"2024-09-14T23:57:40.032307+0000","last_fullsized":"2024-09-14T23:57:40.032307+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":263,"log_dups_size":0,"ondisk_log_size":263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:46:22.880900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010144430000000001,"stat_sum":{"num_bytes":15616247,"num_objects":90,"num_object_clones":24,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":46,"num_read":1982,"num_read_kb":10226,"num_write":230,"num_write_kb":8862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":4573164,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"87'344","reported_seq":2879,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T23:57:40.027409+0000","last_change":"2024-09-14T23:57:00.062960+0000","last_active":"2024-09-14T23:57:40.027409+0000","last_peered":"2024-09-14T23:57:40.027409+0000","last_clean":"2024-09-14T23:57:40.027409+0000","last_became_active":"2024-09-14T23:57:00.062242+0000","last_became_peered":"2024-09-14T23:57:00.062242+0000","last_unstale":"2024-09-14T23:57:40.027409+0000","last_undegraded":"2024-09-14T23:57:40.027409+0000","last_fullsized":"2024-09-14T23:57:40.027409+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:02:46.937458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001148175,"stat_sum":{"num_bytes":32029104,"num_objects":93,"num_object_clones":20,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":54,"num_read":3263,"num_read_kb":15936,"num_write":299,"num_write_kb":11335,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":1249963,"num_keys_recovered":34,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"87'572","reported_seq":4062,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T23:57:40.027493+0000","last_change":"2024-09-14T23:57:00.062694+0000","last_active":"2024-09-14T23:57:40.027493+0000","last_peered":"2024-09-14T23:57:40.027493+0000","last_clean":"2024-09-14T23:57:40.027493+0000","last_became_active":"2024-09-14T23:57:00.062126+0000","last_became_peered":"2024-09-14T23:57:00.062126+0000","last_unstale":"2024-09-14T23:57:40.027493+0000","last_undegraded":"2024-09-14T23:57:40.027493+0000","last_fullsized":"2024-09-14T23:57:40.027493+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:13:14.540098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095260199999999998,"stat_sum":{"num_bytes":19625316,"num_objects":81,"num_object_clones":12,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":49,"num_read":5333,"num_read_kb":16297,"num_write":778,"num_write_kb":6607,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":2675491,"num_keys_recovered":40,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"87'347","reported_seq":1211,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T23:57:38.135471+0000","last_change":"2024-09-14T23:56:34.401802+0000","last_active":"2024-09-14T23:57:38.135471+0000","last_peered":"2024-09-14T23:57:38.135471+0000","last_clean":"2024-09-14T23:57:38.135471+0000","last_became_active":"2024-09-14T23:56:32.398329+0000","last_became_peered":"2024-09-14T23:56:32.398329+0000","last_unstale":"2024-09-14T23:57:38.135471+0000","last_undegraded":"2024-09-14T23:57:38.135471+0000","last_fullsized":"2024-09-14T23:57:38.135471+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":347,"log_dups_size":0,"ondisk_log_size":347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:06:54.579318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018313229999999999,"stat_sum":{"num_bytes":12395890,"num_objects":60,"num_object_clones":13,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":30,"num_read":1448,"num_read_kb":8686,"num_write":414,"num_write_kb":5260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"87'324","reported_seq":3830,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T23:57:40.027166+0000","last_change":"2024-09-14T23:56:34.405093+0000","last_active":"2024-09-14T23:57:40.027166+0000","last_peered":"2024-09-14T23:57:40.027166+0000","last_clean":"2024-09-14T23:57:40.027166+0000","last_became_active":"2024-09-14T23:56:32.399655+0000","last_became_peered":"2024-09-14T23:56:32.399655+0000","last_unstale":"2024-09-14T23:57:40.027166+0000","last_undegraded":"2024-09-14T23:57:40.027166+0000","last_fullsized":"2024-09-14T23:57:40.027166+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:26:33.390359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096230199999999995,"stat_sum":{"num_bytes":55196034,"num_objects":94,"num_object_clones":20,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":59,"num_read":3702,"num_read_kb":29152,"num_write":292,"num_write_kb":14594,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"87'311","reported_seq":1260,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T23:57:38.137161+0000","last_change":"2024-09-14T23:56:34.400617+0000","last_active":"2024-09-14T23:57:38.137161+0000","last_peered":"2024-09-14T23:57:38.137161+0000","last_clean":"2024-09-14T23:57:38.137161+0000","last_became_active":"2024-09-14T23:56:32.404364+0000","last_became_peered":"2024-09-14T23:56:32.404364+0000","last_unstale":"2024-09-14T23:57:38.137161+0000","last_undegraded":"2024-09-14T23:57:38.137161+0000","last_fullsized":"2024-09-14T23:57:38.137161+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:46:54.547217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001003264,"stat_sum":{"num_bytes":20458230,"num_objects":79,"num_object_clones":11,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":47,"num_read":1313,"num_read_kb":6981,"num_write":325,"num_write_kb":4351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"87'363","reported_seq":4109,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T23:57:40.030643+0000","last_change":"2024-09-14T23:56:34.415021+0000","last_active":"2024-09-14T23:57:40.030643+0000","last_peered":"2024-09-14T23:57:40.030643+0000","last_clean":"2024-09-14T23:57:40.030643+0000","last_became_active":"2024-09-14T23:56:32.401799+0000","last_became_peered":"2024-09-14T23:56:32.401799+0000","last_unstale":"2024-09-14T23:57:40.030643+0000","last_undegraded":"2024-09-14T23:57:40.030643+0000","last_fullsized":"2024-09-14T23:57:40.030643+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:58:42.220920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062985000000000005,"stat_sum":{"num_bytes":35381697,"num_objects":114,"num_object_clones":26,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":114,"num_whiteouts":66,"num_read":4517,"num_read_kb":20470,"num_write":304,"num_write_kb":6569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"87'251","reported_seq":1664,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T23:57:38.134135+0000","last_change":"2024-09-14T23:57:00.062824+0000","last_active":"2024-09-14T23:57:38.134135+0000","last_peered":"2024-09-14T23:57:38.134135+0000","last_clean":"2024-09-14T23:57:38.134135+0000","last_became_active":"2024-09-14T23:57:00.062304+0000","last_became_peered":"2024-09-14T23:57:00.062304+0000","last_unstale":"2024-09-14T23:57:38.134135+0000","last_undegraded":"2024-09-14T23:57:38.134135+0000","last_fullsized":"2024-09-14T23:57:38.134135+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":251,"log_dups_size":0,"ondisk_log_size":251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:46:48.439266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014783859999999999,"stat_sum":{"num_bytes":9767773,"num_objects":72,"num_object_clones":17,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":37,"num_read":1330,"num_read_kb":5841,"num_write":240,"num_write_kb":3859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":3345543,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"87'261","reported_seq":6072,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T23:57:38.579516+0000","last_change":"2024-09-14T23:57:26.973933+0000","last_active":"2024-09-14T23:57:38.579516+0000","last_peered":"2024-09-14T23:57:38.579516+0000","last_clean":"2024-09-14T23:57:38.579516+0000","last_became_active":"2024-09-14T23:57:26.973212+0000","last_became_peered":"2024-09-14T23:57:26.973212+0000","last_unstale":"2024-09-14T23:57:38.579516+0000","last_undegraded":"2024-09-14T23:57:38.579516+0000","last_fullsized":"2024-09-14T23:57:38.579516+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":261,"log_dups_size":0,"ondisk_log_size":261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:28:56.664350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039416400000000002,"stat_sum":{"num_bytes":27774888,"num_objects":66,"num_object_clones":9,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":36,"num_read":5993,"num_read_kb":24142,"num_write":219,"num_write_kb":10972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":13520480,"num_keys_recovered":48,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.15","version":"87'379","reported_seq":7079,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T23:57:38.438806+0000","last_change":"2024-09-14T23:56:34.399382+0000","last_active":"2024-09-14T23:57:38.438806+0000","last_peered":"2024-09-14T23:57:38.438806+0000","last_clean":"2024-09-14T23:57:38.438806+0000","last_became_active":"2024-09-14T23:56:32.405387+0000","last_became_peered":"2024-09-14T23:56:32.405387+0000","last_unstale":"2024-09-14T23:57:38.438806+0000","last_undegraded":"2024-09-14T23:57:38.438806+0000","last_fullsized":"2024-09-14T23:57:38.438806+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:33:00.154344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046328199999999997,"stat_sum":{"num_bytes":37492921,"num_objects":81,"num_object_clones":14,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":48,"num_read":7565,"num_read_kb":32733,"num_write":434,"num_write_kb":22868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"87'795","reported_seq":5535,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T23:57:40.027501+0000","last_change":"2024-09-14T23:56:34.404536+0000","last_active":"2024-09-14T23:57:40.027501+0000","last_peered":"2024-09-14T23:57:40.027501+0000","last_clean":"2024-09-14T23:57:40.027501+0000","last_became_active":"2024-09-14T23:56:32.403019+0000","last_became_peered":"2024-09-14T23:56:32.403019+0000","last_unstale":"2024-09-14T23:57:40.027501+0000","last_undegraded":"2024-09-14T23:57:40.027501+0000","last_fullsized":"2024-09-14T23:57:40.027501+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":795,"log_dups_size":0,"ondisk_log_size":795,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:24:26.415960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037505299999999997,"stat_sum":{"num_bytes":25295409,"num_objects":103,"num_object_clones":23,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":57,"num_read":7750,"num_read_kb":25319,"num_write":1254,"num_write_kb":14208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":153,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T23:57:40.032254+0000","last_change":"2024-09-14T23:56:24.727061+0000","last_active":"2024-09-14T23:57:40.032254+0000","last_peered":"2024-09-14T23:57:40.032254+0000","last_clean":"2024-09-14T23:57:40.032254+0000","last_became_active":"2024-09-14T23:56:22.526915+0000","last_became_peered":"2024-09-14T23:56:22.526915+0000","last_unstale":"2024-09-14T23:57:40.032254+0000","last_undegraded":"2024-09-14T23:57:40.032254+0000","last_fullsized":"2024-09-14T23:57:40.032254+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:43:15.359266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045105900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"87'315","reported_seq":3130,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T23:57:38.159758+0000","last_change":"2024-09-14T23:56:34.402234+0000","last_active":"2024-09-14T23:57:38.159758+0000","last_peered":"2024-09-14T23:57:38.159758+0000","last_clean":"2024-09-14T23:57:38.159758+0000","last_became_active":"2024-09-14T23:56:32.398664+0000","last_became_peered":"2024-09-14T23:56:32.398664+0000","last_unstale":"2024-09-14T23:57:38.159758+0000","last_undegraded":"2024-09-14T23:57:38.159758+0000","last_fullsized":"2024-09-14T23:57:38.159758+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:06:52.553990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012179999999999999,"stat_sum":{"num_bytes":23991763,"num_objects":63,"num_object_clones":9,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":31,"num_read":4088,"num_read_kb":14952,"num_write":295,"num_write_kb":4884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":151,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T23:57:38.021988+0000","last_change":"2024-09-14T23:56:24.534632+0000","last_active":"2024-09-14T23:57:38.021988+0000","last_peered":"2024-09-14T23:57:38.021988+0000","last_clean":"2024-09-14T23:57:38.021988+0000","last_became_active":"2024-09-14T23:56:22.526355+0000","last_became_peered":"2024-09-14T23:56:22.526355+0000","last_unstale":"2024-09-14T23:57:38.021988+0000","last_undegraded":"2024-09-14T23:57:38.021988+0000","last_fullsized":"2024-09-14T23:57:38.021988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:02:16.059627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049910000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"87'351","reported_seq":6051,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T23:57:40.030630+0000","last_change":"2024-09-14T23:56:34.416134+0000","last_active":"2024-09-14T23:57:40.030630+0000","last_peered":"2024-09-14T23:57:40.030630+0000","last_clean":"2024-09-14T23:57:40.030630+0000","last_became_active":"2024-09-14T23:56:32.404281+0000","last_became_peered":"2024-09-14T23:56:32.404281+0000","last_unstale":"2024-09-14T23:57:40.030630+0000","last_undegraded":"2024-09-14T23:57:40.030630+0000","last_fullsized":"2024-09-14T23:57:40.030630+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:17:21.618861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013337620000000001,"stat_sum":{"num_bytes":33254849,"num_objects":102,"num_object_clones":23,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":59,"num_read":6857,"num_read_kb":32764,"num_write":334,"num_write_kb":16483,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":151,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T23:57:38.021815+0000","last_change":"2024-09-14T23:56:25.093112+0000","last_active":"2024-09-14T23:57:38.021815+0000","last_peered":"2024-09-14T23:57:38.021815+0000","last_clean":"2024-09-14T23:57:38.021815+0000","last_became_active":"2024-09-14T23:56:22.528211+0000","last_became_peered":"2024-09-14T23:56:22.528211+0000","last_unstale":"2024-09-14T23:57:38.021815+0000","last_undegraded":"2024-09-14T23:57:38.021815+0000","last_fullsized":"2024-09-14T23:57:38.021815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:14:11.495286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000459686,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"87'354","reported_seq":3794,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T23:57:38.145004+0000","last_change":"2024-09-14T23:56:34.401111+0000","last_active":"2024-09-14T23:57:38.145004+0000","last_peered":"2024-09-14T23:57:38.145004+0000","last_clean":"2024-09-14T23:57:38.145004+0000","last_became_active":"2024-09-14T23:56:32.401089+0000","last_became_peered":"2024-09-14T23:56:32.401089+0000","last_unstale":"2024-09-14T23:57:38.145004+0000","last_undegraded":"2024-09-14T23:57:38.145004+0000","last_fullsized":"2024-09-14T23:57:38.145004+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:21:58.313139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013033610000000001,"stat_sum":{"num_bytes":23154943,"num_objects":94,"num_object_clones":20,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":50,"num_read":4194,"num_read_kb":16669,"num_write":343,"num_write_kb":10183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":151,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T23:57:38.021998+0000","last_change":"2024-09-14T23:56:24.534510+0000","last_active":"2024-09-14T23:57:38.021998+0000","last_peered":"2024-09-14T23:57:38.021998+0000","last_clean":"2024-09-14T23:57:38.021998+0000","last_became_active":"2024-09-14T23:56:22.527611+0000","last_became_peered":"2024-09-14T23:56:22.527611+0000","last_unstale":"2024-09-14T23:57:38.021998+0000","last_undegraded":"2024-09-14T23:57:38.021998+0000","last_fullsized":"2024-09-14T23:57:38.021998+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:15:45.606026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049916299999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"87'495","reported_seq":2969,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T23:57:40.030718+0000","last_change":"2024-09-14T23:57:00.065551+0000","last_active":"2024-09-14T23:57:40.030718+0000","last_peered":"2024-09-14T23:57:40.030718+0000","last_clean":"2024-09-14T23:57:40.030718+0000","last_became_active":"2024-09-14T23:57:00.064678+0000","last_became_peered":"2024-09-14T23:57:00.064678+0000","last_unstale":"2024-09-14T23:57:40.030718+0000","last_undegraded":"2024-09-14T23:57:40.030718+0000","last_fullsized":"2024-09-14T23:57:40.030718+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:32:56.375125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001272947,"stat_sum":{"num_bytes":13421153,"num_objects":106,"num_object_clones":21,"num_object_copies":212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":106,"num_whiteouts":60,"num_read":4261,"num_read_kb":12858,"num_write":546,"num_write_kb":9122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":3349587,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"87'331","reported_seq":2752,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T23:57:40.027348+0000","last_change":"2024-09-14T23:56:34.404751+0000","last_active":"2024-09-14T23:57:40.027348+0000","last_peered":"2024-09-14T23:57:40.027348+0000","last_clean":"2024-09-14T23:57:40.027348+0000","last_became_active":"2024-09-14T23:56:32.397883+0000","last_became_peered":"2024-09-14T23:56:32.397883+0000","last_unstale":"2024-09-14T23:57:40.027348+0000","last_undegraded":"2024-09-14T23:57:40.027348+0000","last_fullsized":"2024-09-14T23:57:40.027348+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:10:48.481194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048955799999999996,"stat_sum":{"num_bytes":37123928,"num_objects":95,"num_object_clones":19,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":49,"num_read":3581,"num_read_kb":13350,"num_write":282,"num_write_kb":14352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":162,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T23:57:38.022025+0000","last_change":"2024-09-14T23:56:57.052969+0000","last_active":"2024-09-14T23:57:38.022025+0000","last_peered":"2024-09-14T23:57:38.022025+0000","last_clean":"2024-09-14T23:57:38.022025+0000","last_became_active":"2024-09-14T23:56:57.050456+0000","last_became_peered":"2024-09-14T23:56:57.050456+0000","last_unstale":"2024-09-14T23:57:38.022025+0000","last_undegraded":"2024-09-14T23:57:38.022025+0000","last_fullsized":"2024-09-14T23:57:38.022025+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:57:07.644011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"87'352","reported_seq":7870,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T23:57:38.151477+0000","last_change":"2024-09-14T23:56:34.401159+0000","last_active":"2024-09-14T23:57:38.151477+0000","last_peered":"2024-09-14T23:57:38.151477+0000","last_clean":"2024-09-14T23:57:38.151477+0000","last_became_active":"2024-09-14T23:56:32.400278+0000","last_became_peered":"2024-09-14T23:56:32.400278+0000","last_unstale":"2024-09-14T23:57:38.151477+0000","last_undegraded":"2024-09-14T23:57:38.151477+0000","last_fullsized":"2024-09-14T23:57:38.151477+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:44:47.851257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082507200000000004,"stat_sum":{"num_bytes":55980030,"num_objects":79,"num_object_clones":16,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":38,"num_read":7637,"num_read_kb":34438,"num_write":316,"num_write_kb":15840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":149,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-14T23:57:37.030738+0000","last_change":"2024-09-14T23:56:24.909260+0000","last_active":"2024-09-14T23:57:37.030738+0000","last_peered":"2024-09-14T23:57:37.030738+0000","last_clean":"2024-09-14T23:57:37.030738+0000","last_became_active":"2024-09-14T23:56:22.822598+0000","last_became_peered":"2024-09-14T23:56:22.822598+0000","last_unstale":"2024-09-14T23:57:37.030738+0000","last_undegraded":"2024-09-14T23:57:37.030738+0000","last_fullsized":"2024-09-14T23:57:37.030738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:32:32.568028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026863400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"87'553","reported_seq":3992,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T23:57:38.179392+0000","last_change":"2024-09-14T23:56:34.401366+0000","last_active":"2024-09-14T23:57:38.179392+0000","last_peered":"2024-09-14T23:57:38.179392+0000","last_clean":"2024-09-14T23:57:38.179392+0000","last_became_active":"2024-09-14T23:56:32.397551+0000","last_became_peered":"2024-09-14T23:56:32.397551+0000","last_unstale":"2024-09-14T23:57:38.179392+0000","last_undegraded":"2024-09-14T23:57:38.179392+0000","last_fullsized":"2024-09-14T23:57:38.179392+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":553,"log_dups_size":0,"ondisk_log_size":553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:25:02.919990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001418816,"stat_sum":{"num_bytes":13935751,"num_objects":84,"num_object_clones":14,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":51,"num_read":6173,"num_read_kb":14453,"num_write":710,"num_write_kb":3934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":151,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T23:57:38.021468+0000","last_change":"2024-09-14T23:56:25.093090+0000","last_active":"2024-09-14T23:57:38.021468+0000","last_peered":"2024-09-14T23:57:38.021468+0000","last_clean":"2024-09-14T23:57:38.021468+0000","last_became_active":"2024-09-14T23:56:22.528115+0000","last_became_peered":"2024-09-14T23:56:22.528115+0000","last_unstale":"2024-09-14T23:57:38.021468+0000","last_undegraded":"2024-09-14T23:57:38.021468+0000","last_fullsized":"2024-09-14T23:57:38.021468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:57:36.610730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036520999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"87'467","reported_seq":3775,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T23:57:40.030624+0000","last_change":"2024-09-14T23:57:26.975545+0000","last_active":"2024-09-14T23:57:40.030624+0000","last_peered":"2024-09-14T23:57:40.030624+0000","last_clean":"2024-09-14T23:57:40.030624+0000","last_became_active":"2024-09-14T23:57:26.974022+0000","last_became_peered":"2024-09-14T23:57:26.974022+0000","last_unstale":"2024-09-14T23:57:40.030624+0000","last_undegraded":"2024-09-14T23:57:40.030624+0000","last_fullsized":"2024-09-14T23:57:40.030624+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":467,"log_dups_size":0,"ondisk_log_size":467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:58:22.063912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073009599999999996,"stat_sum":{"num_bytes":14296937,"num_objects":98,"num_object_clones":23,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":53,"num_read":3786,"num_read_kb":11514,"num_write":534,"num_write_kb":6247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":297,"num_bytes_recovered":12389647,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":147,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-14T23:57:37.030747+0000","last_change":"2024-09-14T23:56:57.054999+0000","last_active":"2024-09-14T23:57:37.030747+0000","last_peered":"2024-09-14T23:57:37.030747+0000","last_clean":"2024-09-14T23:57:37.030747+0000","last_became_active":"2024-09-14T23:56:57.050729+0000","last_became_peered":"2024-09-14T23:56:57.050729+0000","last_unstale":"2024-09-14T23:57:37.030747+0000","last_undegraded":"2024-09-14T23:57:37.030747+0000","last_fullsized":"2024-09-14T23:57:37.030747+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:23:11.727903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":302,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T23:57:38.021480+0000","last_change":"2024-09-14T23:56:15.332936+0000","last_active":"2024-09-14T23:57:38.021480+0000","last_peered":"2024-09-14T23:57:38.021480+0000","last_clean":"2024-09-14T23:57:38.021480+0000","last_became_active":"2024-09-14T23:56:15.331994+0000","last_became_peered":"2024-09-14T23:56:15.331994+0000","last_unstale":"2024-09-14T23:57:38.021480+0000","last_undegraded":"2024-09-14T23:57:38.021480+0000","last_fullsized":"2024-09-14T23:57:38.021480+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:14.322882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:14.322882+0000","last_clean_scrub_stamp":"2024-09-14T23:56:14.322882+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:36:12.639000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"87'314","reported_seq":4002,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T23:57:40.030552+0000","last_change":"2024-09-14T23:56:34.415037+0000","last_active":"2024-09-14T23:57:40.030552+0000","last_peered":"2024-09-14T23:57:40.030552+0000","last_clean":"2024-09-14T23:57:40.030552+0000","last_became_active":"2024-09-14T23:56:32.403572+0000","last_became_peered":"2024-09-14T23:56:32.403572+0000","last_unstale":"2024-09-14T23:57:40.030552+0000","last_undegraded":"2024-09-14T23:57:40.030552+0000","last_fullsized":"2024-09-14T23:57:40.030552+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:46:22.262567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052643700000000004,"stat_sum":{"num_bytes":30046727,"num_objects":75,"num_object_clones":15,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":44,"num_read":4091,"num_read_kb":15767,"num_write":275,"num_write_kb":9560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"87'225","reported_seq":2062,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T23:57:40.027212+0000","last_change":"2024-09-14T23:56:34.404977+0000","last_active":"2024-09-14T23:57:40.027212+0000","last_peered":"2024-09-14T23:57:40.027212+0000","last_clean":"2024-09-14T23:57:40.027212+0000","last_became_active":"2024-09-14T23:56:32.400017+0000","last_became_peered":"2024-09-14T23:56:32.400017+0000","last_unstale":"2024-09-14T23:57:40.027212+0000","last_undegraded":"2024-09-14T23:57:40.027212+0000","last_fullsized":"2024-09-14T23:57:40.027212+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:06:26.325962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068884800000000004,"stat_sum":{"num_bytes":22078705,"num_objects":79,"num_object_clones":16,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":44,"num_read":1773,"num_read_kb":12127,"num_write":208,"num_write_kb":15086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"87'628","reported_seq":7031,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T23:57:40.032309+0000","last_change":"2024-09-14T23:56:34.498020+0000","last_active":"2024-09-14T23:57:40.032309+0000","last_peered":"2024-09-14T23:57:40.032309+0000","last_clean":"2024-09-14T23:57:40.032309+0000","last_became_active":"2024-09-14T23:56:32.402769+0000","last_became_peered":"2024-09-14T23:56:32.402769+0000","last_unstale":"2024-09-14T23:57:40.032309+0000","last_undegraded":"2024-09-14T23:57:40.032309+0000","last_fullsized":"2024-09-14T23:57:40.032309+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":628,"log_dups_size":0,"ondisk_log_size":628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:44:00.529861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035483999999999998,"stat_sum":{"num_bytes":26133279,"num_objects":80,"num_object_clones":12,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":48,"num_read":8473,"num_read_kb":29914,"num_write":927,"num_write_kb":9066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"87'488","reported_seq":12957,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T23:57:40.030563+0000","last_change":"2024-09-14T23:56:34.415414+0000","last_active":"2024-09-14T23:57:40.030563+0000","last_peered":"2024-09-14T23:57:40.030563+0000","last_clean":"2024-09-14T23:57:40.030563+0000","last_became_active":"2024-09-14T23:56:32.403231+0000","last_became_peered":"2024-09-14T23:56:32.403231+0000","last_unstale":"2024-09-14T23:57:40.030563+0000","last_undegraded":"2024-09-14T23:57:40.030563+0000","last_fullsized":"2024-09-14T23:57:40.030563+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:51:19.458429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072791000000000002,"stat_sum":{"num_bytes":37209942,"num_objects":74,"num_object_clones":17,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":39,"num_read":13484,"num_read_kb":56086,"num_write":607,"num_write_kb":8013,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"87'410","reported_seq":13783,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T23:57:40.030579+0000","last_change":"2024-09-14T23:57:00.066551+0000","last_active":"2024-09-14T23:57:40.030579+0000","last_peered":"2024-09-14T23:57:40.030579+0000","last_clean":"2024-09-14T23:57:40.030579+0000","last_became_active":"2024-09-14T23:57:00.065790+0000","last_became_peered":"2024-09-14T23:57:00.065790+0000","last_unstale":"2024-09-14T23:57:40.030579+0000","last_undegraded":"2024-09-14T23:57:40.030579+0000","last_fullsized":"2024-09-14T23:57:40.030579+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:45:15.013312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092638099999999999,"stat_sum":{"num_bytes":63356825,"num_objects":89,"num_object_clones":21,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":48,"num_read":12886,"num_read_kb":57927,"num_write":523,"num_write_kb":33646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":7912941,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"87'248","reported_seq":1549,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T23:57:39.135557+0000","last_change":"2024-09-14T23:57:26.973929+0000","last_active":"2024-09-14T23:57:39.135557+0000","last_peered":"2024-09-14T23:57:39.135557+0000","last_clean":"2024-09-14T23:57:39.135557+0000","last_became_active":"2024-09-14T23:57:26.973189+0000","last_became_peered":"2024-09-14T23:57:26.973189+0000","last_unstale":"2024-09-14T23:57:39.135557+0000","last_undegraded":"2024-09-14T23:57:39.135557+0000","last_fullsized":"2024-09-14T23:57:39.135557+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":248,"log_dups_size":0,"ondisk_log_size":248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:12:28.811282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057774300000000003,"stat_sum":{"num_bytes":10423332,"num_objects":68,"num_object_clones":12,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":40,"num_read":995,"num_read_kb":7951,"num_write":247,"num_write_kb":5246,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":9904034,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"87'353","reported_seq":4462,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T23:57:38.559120+0000","last_change":"2024-09-14T23:56:34.399894+0000","last_active":"2024-09-14T23:57:38.559120+0000","last_peered":"2024-09-14T23:57:38.559120+0000","last_clean":"2024-09-14T23:57:38.559120+0000","last_became_active":"2024-09-14T23:56:32.406251+0000","last_became_peered":"2024-09-14T23:56:32.406251+0000","last_unstale":"2024-09-14T23:57:38.559120+0000","last_undegraded":"2024-09-14T23:57:38.559120+0000","last_fullsized":"2024-09-14T23:57:38.559120+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:27:42.156092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073115799999999998,"stat_sum":{"num_bytes":24380697,"num_objects":81,"num_object_clones":16,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":41,"num_read":6181,"num_read_kb":19840,"num_write":340,"num_write_kb":11017,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"87'378","reported_seq":3362,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T23:57:39.086538+0000","last_change":"2024-09-14T23:56:34.399966+0000","last_active":"2024-09-14T23:57:39.086538+0000","last_peered":"2024-09-14T23:57:39.086538+0000","last_clean":"2024-09-14T23:57:39.086538+0000","last_became_active":"2024-09-14T23:56:32.407109+0000","last_became_peered":"2024-09-14T23:56:32.407109+0000","last_unstale":"2024-09-14T23:57:39.086538+0000","last_undegraded":"2024-09-14T23:57:39.086538+0000","last_fullsized":"2024-09-14T23:57:39.086538+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:03.015051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072766900000000002,"stat_sum":{"num_bytes":31857291,"num_objects":93,"num_object_clones":18,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":49,"num_read":4434,"num_read_kb":14064,"num_write":342,"num_write_kb":14909,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"88'591","reported_seq":2662,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T23:57:40.032566+0000","last_change":"2024-09-14T23:56:34.498033+0000","last_active":"2024-09-14T23:57:40.032566+0000","last_peered":"2024-09-14T23:57:40.032566+0000","last_clean":"2024-09-14T23:57:40.032566+0000","last_became_active":"2024-09-14T23:56:32.402973+0000","last_became_peered":"2024-09-14T23:56:32.402973+0000","last_unstale":"2024-09-14T23:57:40.032566+0000","last_undegraded":"2024-09-14T23:57:40.032566+0000","last_fullsized":"2024-09-14T23:57:40.032566+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:27:11.687918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035072199999999998,"stat_sum":{"num_bytes":28334329,"num_objects":94,"num_object_clones":15,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":56,"num_read":3690,"num_read_kb":19639,"num_write":1000,"num_write_kb":20841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"87'344","reported_seq":5575,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T23:57:38.537066+0000","last_change":"2024-09-14T23:56:34.399387+0000","last_active":"2024-09-14T23:57:38.537066+0000","last_peered":"2024-09-14T23:57:38.537066+0000","last_clean":"2024-09-14T23:57:38.537066+0000","last_became_active":"2024-09-14T23:56:32.405457+0000","last_became_peered":"2024-09-14T23:56:32.405457+0000","last_unstale":"2024-09-14T23:57:38.537066+0000","last_undegraded":"2024-09-14T23:57:38.537066+0000","last_fullsized":"2024-09-14T23:57:38.537066+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:29:53.710458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048125700000000002,"stat_sum":{"num_bytes":35105406,"num_objects":85,"num_object_clones":16,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":46,"num_read":5799,"num_read_kb":25238,"num_write":357,"num_write_kb":6847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"86'193","reported_seq":1462,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T23:57:38.021848+0000","last_change":"2024-09-14T23:56:34.402592+0000","last_active":"2024-09-14T23:57:38.021848+0000","last_peered":"2024-09-14T23:57:38.021848+0000","last_clean":"2024-09-14T23:57:38.021848+0000","last_became_active":"2024-09-14T23:56:32.402782+0000","last_became_peered":"2024-09-14T23:56:32.402782+0000","last_unstale":"2024-09-14T23:57:38.021848+0000","last_undegraded":"2024-09-14T23:57:38.021848+0000","last_fullsized":"2024-09-14T23:57:38.021848+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:12.456579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001446697,"stat_sum":{"num_bytes":23133342,"num_objects":57,"num_object_clones":15,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":31,"num_read":1179,"num_read_kb":7295,"num_write":165,"num_write_kb":4341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"87'607","reported_seq":2607,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T23:57:38.765995+0000","last_change":"2024-09-14T23:56:34.399629+0000","last_active":"2024-09-14T23:57:38.765995+0000","last_peered":"2024-09-14T23:57:38.765995+0000","last_clean":"2024-09-14T23:57:38.765995+0000","last_became_active":"2024-09-14T23:56:32.407220+0000","last_became_peered":"2024-09-14T23:56:32.407220+0000","last_unstale":"2024-09-14T23:57:38.765995+0000","last_undegraded":"2024-09-14T23:57:38.765995+0000","last_fullsized":"2024-09-14T23:57:38.765995+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":607,"log_dups_size":0,"ondisk_log_size":607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:48:33.348254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062462200000000002,"stat_sum":{"num_bytes":20579680,"num_objects":85,"num_object_clones":19,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":49,"num_read":4292,"num_read_kb":22456,"num_write":869,"num_write_kb":13790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":882873613,"num_objects":2707,"num_object_clones":545,"num_object_copies":5414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2707,"num_whiteouts":1502,"num_read":157807,"num_read_kb":650559,"num_write":15092,"num_write_kb":364092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1167,"num_bytes_recovered":61248508,"num_keys_recovered":133,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57802752,"data_stored":535403709,"data_compressed":40926249,"data_compressed_allocated":43130880,"data_compressed_original":522793722,"omap_allocated":0,"internal_metadata":0},"log_size":13000,"ondisk_log_size":13000,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64594,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509462,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37584,"kb_used_data":10560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94334256,"statfs":{"total":96636764160,"available":96598278144,"internally_reserved":0,"allocated":10813440,"data_stored":95921992,"data_compressed":7424560,"data_compressed_allocated":7733248,"data_compressed_original":93697716,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509461,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43992,"kb_used_data":4484,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94327848,"statfs":{"total":96636764160,"available":96591716352,"internally_reserved":0,"allocated":4591616,"data_stored":28054137,"data_compressed":2264756,"data_compressed_allocated":2711552,"data_compressed_original":26809406,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509460,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39840,"kb_used_data":8656,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94332000,"statfs":{"total":96636764160,"available":96595968000,"internally_reserved":0,"allocated":8863744,"data_stored":78176660,"data_compressed":5912700,"data_compressed_allocated":6070272,"data_compressed_original":76306509,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509462,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65012,"kb_used_data":11620,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94306828,"statfs":{"total":96636764160,"available":96570191872,"internally_reserved":0,"allocated":11898880,"data_stored":97820923,"data_compressed":7209328,"data_compressed_allocated":8019968,"data_compressed_original":94935155,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509460,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41092,"kb_used_data":9900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94330748,"statfs":{"total":96636764160,"available":96594685952,"internally_reserved":0,"allocated":10137600,"data_stored":92573875,"data_compressed":6769539,"data_compressed_allocated":7163904,"data_compressed_original":90516746,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542164,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50488,"kb_used_data":6820,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94321352,"statfs":{"total":96636764160,"available":96585064448,"internally_reserved":0,"allocated":6983680,"data_stored":50566668,"data_compressed":3881414,"data_compressed_allocated":3883008,"data_compressed_original":48212389,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542163,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39388,"kb_used_data":8204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94332452,"statfs":{"total":96636764160,"available":96596430848,"internally_reserved":0,"allocated":8400896,"data_stored":66072696,"data_compressed":5152498,"data_compressed_allocated":5394432,"data_compressed_original":63886992,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39608,"kb_used_data":4256,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94332232,"statfs":{"total":96636764160,"available":96596205568,"internally_reserved":0,"allocated":4358144,"data_stored":31140004,"data_compressed":2376048,"data_compressed_allocated":2244608,"data_compressed_original":29575689,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32297,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32297,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":7331840,"data_stored":65017640,"data_compressed":5120201,"data_compressed_allocated":5349376,"data_compressed_original":63313552,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5971968,"data_stored":50101921,"data_compressed":3881414,"data_compressed_allocated":3883008,"data_compressed_original":48212389,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3362816,"data_stored":30681448,"data_compressed":2376048,"data_compressed_allocated":2244608,"data_compressed_original":29575689,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":9117696,"data_stored":92102975,"data_compressed":6769539,"data_compressed_allocated":7163904,"data_compressed_original":90516746,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":10878976,"data_stored":97350023,"data_compressed":7209328,"data_compressed_allocated":8019968,"data_compressed_original":94935155,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":7839744,"data_stored":77705741,"data_compressed":5912700,"data_compressed_allocated":6070272,"data_compressed_original":76306509,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3571712,"data_stored":27583237,"data_compressed":2264756,"data_compressed_allocated":2711552,"data_compressed_original":26809406,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":9728000,"data_stored":94860724,"data_compressed":7392263,"data_compressed_allocated":7688192,"data_compressed_original":93124276,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T23:57:41.791 INFO:tasks.thrashosds.thrasher:Setting 3.1e to [7, 0, 1, 5] 2024-09-14T23:57:41.791 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.1e', '7', '0', '1', '5'] 2024-09-14T23:57:41.792 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.1e 7 0 1 5 2024-09-14T23:57:41.849 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:41.849 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:41.858 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T23:57:41.899 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:41.898+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:41.972 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:41.970+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:42.006 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:42.006 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:42.014 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T23:57:42.048 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T23:57:42.099 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:42.098+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:42.150 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone35 from 0xc3b3201 (overlap 0xd9b5c1) to 0x82a040 2024-09-14T23:57:42.170 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:42.170 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:42.173 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:42.172+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:42.179 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T23:57:42.274 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:42.273+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:42.314 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #34, image image_client.0-clone34 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent35 2024-09-14T23:57:42.327 INFO:teuthology.orchestra.run.smithi042.stdout:3871 read 0xab278c9 thru 0xab3033e (0x8a76 bytes) 2024-09-14T23:57:42.328 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:42.328 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:42.329 INFO:teuthology.orchestra.run.smithi042.stdout:3875 punch from 0x89382f4 to 0x893de3f, (0x5b4b bytes) 2024-09-14T23:57:42.332 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:42.333 INFO:teuthology.orchestra.run.smithi042.stdout:3878 write 0x86826fd thru 0x8689093 (0x6997 bytes) 2024-09-14T23:57:42.338 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T23:57:42.338 INFO:teuthology.orchestra.run.smithi042.stdout:3879 write 0x90718dd thru 0x90777e6 (0x5f0a bytes) 2024-09-14T23:57:42.346 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:42.346 INFO:teuthology.orchestra.run.smithi042.stdout:3882 read 0x4448d93 thru 0x4458509 (0xf777 bytes) 2024-09-14T23:57:42.346 INFO:teuthology.orchestra.run.smithi042.stdout:3885 punch from 0x45c03da to 0x45cb4dc, (0xb102 bytes) 2024-09-14T23:57:42.357 INFO:teuthology.orchestra.run.smithi042.stdout:3887 trunc from 0xc3b3201 to 0x20eb306 2024-09-14T23:57:42.375 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:42.373+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:42.393 INFO:teuthology.orchestra.run.smithi042.stdout:3888 writesame 0x8e2cc61 thru 0x8e33587 data_size 0x9(0x6927 bytes) 2024-09-14T23:57:42.402 INFO:teuthology.orchestra.run.smithi042.stdout:3889 trunc from 0x8e33588 to 0x25cbf96 2024-09-14T23:57:42.416 INFO:teuthology.orchestra.run.smithi042.stdout:3890 write 0x312da7f thru 0x31353bf (0x7941 bytes) 2024-09-14T23:57:42.418 INFO:teuthology.orchestra.run.smithi042.stdout:3891 trunc from 0x31353c0 to 0x22cfe25 2024-09-14T23:57:42.426 INFO:teuthology.orchestra.run.smithi042.stdout:3892 punch from 0x216a977 to 0x216bc0e, (0x1297 bytes) 2024-09-14T23:57:42.426 INFO:teuthology.orchestra.run.smithi042.stdout:3893 punch from 0x14f1d9d to 0x14f4270, (0x24d3 bytes) 2024-09-14T23:57:42.431 INFO:teuthology.orchestra.run.smithi042.stdout:3894 write 0x2a2804f thru 0x2a2d2cc (0x527e bytes) 2024-09-14T23:57:42.432 INFO:teuthology.orchestra.run.smithi042.stdout:3896 write 0x8a870b thru 0x8ac7c7 (0x40bd bytes) 2024-09-14T23:57:42.434 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:42.434 INFO:teuthology.orchestra.run.smithi042.stdout:3899 read 0x1f923cf thru 0x1fa18a0 (0xf4d2 bytes) 2024-09-14T23:57:42.434 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:42.435 INFO:teuthology.orchestra.run.smithi042.stdout:3901 read 0x1002201 thru 0x1010613 (0xe413 bytes) 2024-09-14T23:57:42.449 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:42.449 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:42.449 INFO:teuthology.orchestra.run.smithi042.stdout:3902 write 0xa6992e2 thru 0xa6a7865 (0xe584 bytes) 2024-09-14T23:57:42.451 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:42.451 INFO:teuthology.orchestra.run.smithi042.stdout:3906 read 0x2976e9f thru 0x29808a3 (0x9a05 bytes) 2024-09-14T23:57:42.451 INFO:teuthology.orchestra.run.smithi042.stdout:3907 read 0x846273a thru 0x846a640 (0x7f07 bytes) 2024-09-14T23:57:42.452 INFO:teuthology.orchestra.run.smithi042.stdout:3909 punch from 0x6ef3232 to 0x6efa892, (0x7660 bytes) 2024-09-14T23:57:42.453 INFO:teuthology.orchestra.run.smithi042.stdout:3912 trunc from 0xa6a7866 to 0xc0c575d 2024-09-14T23:57:42.458 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T23:57:42.460 INFO:teuthology.orchestra.run.smithi042.stdout:3913 read 0x9e8a681 thru 0x9e8af3f (0x8bf bytes) 2024-09-14T23:57:42.461 INFO:teuthology.orchestra.run.smithi042.stdout:3915 punch from 0x8c45045 to 0x8c52f14, (0xdecf bytes) 2024-09-14T23:57:42.466 INFO:teuthology.orchestra.run.smithi042.stdout:3916 trunc from 0xc0c575d to 0x7761b0d 2024-09-14T23:57:42.475 INFO:teuthology.orchestra.run.smithi042.stdout:3918 trunc from 0x7761b0d to 0x887403b 2024-09-14T23:57:42.475 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:42.474+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:42.480 INFO:teuthology.orchestra.run.smithi042.stdout:3920 read 0x1a6f597 thru 0x1a73196 (0x3c00 bytes) 2024-09-14T23:57:42.480 INFO:teuthology.orchestra.run.smithi042.stdout:3921 punch from 0x7c614ed to 0x7c6448c, (0x2f9f bytes) 2024-09-14T23:57:42.487 INFO:teuthology.orchestra.run.smithi042.stdout:3922 trunc from 0x887403b to 0x23abf75 2024-09-14T23:57:42.510 INFO:teuthology.orchestra.run.smithi042.stdout:3923 write 0xa9b1c9d thru 0xa9be54a (0xc8ae bytes) 2024-09-14T23:57:42.512 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:42.518 INFO:teuthology.orchestra.run.smithi042.stdout:3926 write 0xb0a26fa thru 0xb0acb38 (0xa43f bytes) 2024-09-14T23:57:42.520 INFO:teuthology.orchestra.run.smithi042.stdout:3927 write 0x1d742fd thru 0x1d79972 (0x5676 bytes) 2024-09-14T23:57:42.521 INFO:teuthology.orchestra.run.smithi042.stdout:3929 clone 37 order 25 su 262144 sc 3 2024-09-14T23:57:42.567 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:42.567 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:42.575 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T23:57:42.577 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:42.575+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:42.677 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:42.676+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:42.683 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:42.683 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:42.692 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T23:57:42.778 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:42.777+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:42.840 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:42.840 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:42.848 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T23:57:42.879 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:42.878+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:42.980 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:42.979+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:42.995 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T23:57:42.996 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:42.996 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:43.005 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T23:57:43.034 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is set 2024-09-14T23:57:43.039 INFO:teuthology.orchestra.run.smithi042.stderr:set 3.1e pg_upmap_items mapping to [7->0,1->5] 2024-09-14T23:57:43.099 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:43.097+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:43.144 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:57:43.145 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:57:43.157 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:43.157 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:57:43.157 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:57:43.157 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:43.163 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:57:43.163 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:57:43.167 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T23:57:43.173 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone36 from 0xb0acb39 (overlap 0x20eb306) to 0x1075983 2024-09-14T23:57:43.187 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:57:43.187 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:57:43.188 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:43.185+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:43.209 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:57:43.209 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:57:43.228 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:57:43.228 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:57:43.248 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:57:43.248 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:57:43.266 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:57:43.266 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:57:43.283 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:43.281+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:43.300 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:43.300 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:57:43.300 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:57:43.300 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:57:43.300 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:43.311 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T23:57:43.370 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #35, image image_client.0-clone35 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent36 2024-09-14T23:57:43.384 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:43.383+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:43.463 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:43.463 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:57:43.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:57:43.463 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:57:43.463 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.463 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:43054 3.1f 3:ffbfe596:::rbd_data.1118350d5ae5.00000000000002eb:head [set-alloc-hint object_size 262144 write_size 262144,write 196608~22770] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T23:57:43.463 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:57:43.463 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 50.493893247000003, 2024-09-14T23:57:43.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.99973334400000002, 2024-09-14T23:57:43.463 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:43.463 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:43.464 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:43.464 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:43.464 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:43.464 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:43.464 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 43054 2024-09-14T23:57:43.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.464 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:43.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.464 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:43.464 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:57:43.464 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:43.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.465 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:43.465 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:57:43.465 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:43.465 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.465 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.465 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:43.465 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966552+0000", 2024-09-14T23:57:43.465 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3790000000000001e-06 2024-09-14T23:57:43.465 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.465 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.465 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:43.465 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966565+0000", 2024-09-14T23:57:43.465 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2459999999999999e-05 2024-09-14T23:57:43.465 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.466 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.466 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:43.466 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966576+0000", 2024-09-14T23:57:43.466 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1039000000000001e-05 2024-09-14T23:57:43.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.466 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.466 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:43.466 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966582+0000", 2024-09-14T23:57:43.466 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9660000000000001e-06 2024-09-14T23:57:43.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.466 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.466 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:43.466 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966625+0000", 2024-09-14T23:57:43.467 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3529999999999998e-05 2024-09-14T23:57:43.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.467 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.467 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:57:43.467 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966632+0000", 2024-09-14T23:57:43.467 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9809999999999997e-06 2024-09-14T23:57:43.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.467 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.467 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:43.468 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.964460+0000", 2024-09-14T23:57:43.468 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.99782768300000002 2024-09-14T23:57:43.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.468 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.468 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:43.468 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.964641+0000", 2024-09-14T23:57:43.468 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000181089 2024-09-14T23:57:43.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.468 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.468 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-14T23:57:43.468 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.964809+0000", 2024-09-14T23:57:43.468 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000168505 2024-09-14T23:57:43.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.469 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.469 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:57:43.469 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966030+0000", 2024-09-14T23:57:43.469 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012206929999999999 2024-09-14T23:57:43.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.469 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.469 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:43.469 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966181+0000", 2024-09-14T23:57:43.469 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000150569 2024-09-14T23:57:43.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.469 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.469 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:43.469 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966243+0000", 2024-09-14T23:57:43.470 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2331999999999999e-05 2024-09-14T23:57:43.470 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.470 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.470 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:43.470 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966284+0000", 2024-09-14T23:57:43.470 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1118000000000002e-05 2024-09-14T23:57:43.470 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:43.470 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:43.470 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:43.470 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.470 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.470 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:54673 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:43.470 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.167989+0000", 2024-09-14T23:57:43.470 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.292454896999999, 2024-09-14T23:57:43.471 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027580298, 2024-09-14T23:57:43.471 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:43.471 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:43.471 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:43.471 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:43.471 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:43.471 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:43.471 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54673 2024-09-14T23:57:43.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.471 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:43.471 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.471 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:43.471 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.167989+0000", 2024-09-14T23:57:43.471 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:43.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.472 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.472 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:43.472 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.167989+0000", 2024-09-14T23:57:43.472 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:43.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.472 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.472 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:43.472 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.167989+0000", 2024-09-14T23:57:43.472 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5499999999999998e-07 2024-09-14T23:57:43.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.472 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.472 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:43.472 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.167998+0000", 2024-09-14T23:57:43.473 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7369999999999995e-06 2024-09-14T23:57:43.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.473 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:43.473 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168005+0000", 2024-09-14T23:57:43.473 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8210000000000004e-06 2024-09-14T23:57:43.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.473 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:43.473 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168010+0000", 2024-09-14T23:57:43.473 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2510000000000002e-06 2024-09-14T23:57:43.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.473 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:43.474 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.195432+0000", 2024-09-14T23:57:43.474 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027421364 2024-09-14T23:57:43.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.474 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.474 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:43.474 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.195501+0000", 2024-09-14T23:57:43.474 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9611999999999997e-05 2024-09-14T23:57:43.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.474 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.474 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:43.474 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.195569+0000", 2024-09-14T23:57:43.474 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8058000000000005e-05 2024-09-14T23:57:43.474 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:43.474 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:43.474 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:43.475 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.475 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.475 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:54674 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 2637824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:43.475 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.168018+0000", 2024-09-14T23:57:43.475 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.292426253000002, 2024-09-14T23:57:43.475 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027716159000000001, 2024-09-14T23:57:43.475 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:43.475 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:43.475 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:43.475 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:43.475 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:43.475 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:43.475 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54674 2024-09-14T23:57:43.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.476 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:43.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.476 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:43.476 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168018+0000", 2024-09-14T23:57:43.476 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:43.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.476 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:43.476 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168018+0000", 2024-09-14T23:57:43.476 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:43.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.476 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:43.477 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168018+0000", 2024-09-14T23:57:43.477 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2600000000000002e-07 2024-09-14T23:57:43.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.477 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.477 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:43.477 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168020+0000", 2024-09-14T23:57:43.477 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9420000000000002e-06 2024-09-14T23:57:43.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.477 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.477 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:43.478 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168026+0000", 2024-09-14T23:57:43.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8880000000000002e-06 2024-09-14T23:57:43.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.478 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:43.478 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168031+0000", 2024-09-14T23:57:43.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.977e-06 2024-09-14T23:57:43.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.478 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:43.478 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.195599+0000", 2024-09-14T23:57:43.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027567758000000001 2024-09-14T23:57:43.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.479 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:43.479 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.195666+0000", 2024-09-14T23:57:43.479 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7102999999999998e-05 2024-09-14T23:57:43.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.479 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:43.479 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.195734+0000", 2024-09-14T23:57:43.479 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7865000000000002e-05 2024-09-14T23:57:43.479 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:43.479 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:43.479 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:43.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.480 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.480 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:54675 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:43.480 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.168039+0000", 2024-09-14T23:57:43.480 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.292405293999998, 2024-09-14T23:57:43.480 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027868378999999999, 2024-09-14T23:57:43.480 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:43.480 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:43.480 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:43.480 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:43.480 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:43.480 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:43.480 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54675 2024-09-14T23:57:43.480 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.481 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:43.481 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.481 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:43.481 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168039+0000", 2024-09-14T23:57:43.481 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:43.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.481 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.481 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:43.481 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168039+0000", 2024-09-14T23:57:43.481 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:43.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.481 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.481 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:43.482 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168039+0000", 2024-09-14T23:57:43.482 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3099999999999998e-07 2024-09-14T23:57:43.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.482 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.482 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:43.482 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168048+0000", 2024-09-14T23:57:43.482 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9460000000000004e-06 2024-09-14T23:57:43.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.482 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.482 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:43.482 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168055+0000", 2024-09-14T23:57:43.482 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5740000000000001e-06 2024-09-14T23:57:43.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.482 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.482 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:43.483 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168060+0000", 2024-09-14T23:57:43.483 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1909999999999999e-06 2024-09-14T23:57:43.483 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.483 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.483 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:43.483 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.195765+0000", 2024-09-14T23:57:43.483 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027705244 2024-09-14T23:57:43.483 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.483 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.483 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:43.483 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.195836+0000", 2024-09-14T23:57:43.483 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0642999999999994e-05 2024-09-14T23:57:43.483 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.483 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.484 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:43.484 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.195907+0000", 2024-09-14T23:57:43.484 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.135e-05 2024-09-14T23:57:43.484 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:43.484 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:43.484 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:43.484 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.484 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.484 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:54676 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 2998272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:43.484 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.168067+0000", 2024-09-14T23:57:43.484 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.292376560999998, 2024-09-14T23:57:43.484 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028002955999999999, 2024-09-14T23:57:43.484 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:43.484 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:43.485 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:43.485 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:43.485 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:43.485 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:43.485 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54676 2024-09-14T23:57:43.485 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.485 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:43.485 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.485 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:43.485 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168067+0000", 2024-09-14T23:57:43.485 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:43.485 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.485 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.486 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:43.486 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168067+0000", 2024-09-14T23:57:43.486 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:43.486 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.486 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.486 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:43.486 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168068+0000", 2024-09-14T23:57:43.486 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6899999999999998e-07 2024-09-14T23:57:43.486 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.486 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.486 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:43.486 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168070+0000", 2024-09-14T23:57:43.486 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9810000000000002e-06 2024-09-14T23:57:43.486 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.487 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.487 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:43.487 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168076+0000", 2024-09-14T23:57:43.487 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4280000000000001e-06 2024-09-14T23:57:43.487 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.487 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.487 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:43.487 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168091+0000", 2024-09-14T23:57:43.487 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5152e-05 2024-09-14T23:57:43.487 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.487 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.487 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:43.487 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.195936+0000", 2024-09-14T23:57:43.488 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027844892 2024-09-14T23:57:43.488 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.488 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.488 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:43.488 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196002+0000", 2024-09-14T23:57:43.488 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5633e-05 2024-09-14T23:57:43.488 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.488 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.488 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:43.489 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196070+0000", 2024-09-14T23:57:43.489 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8400999999999998e-05 2024-09-14T23:57:43.489 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:43.489 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:43.489 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:43.489 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.489 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.489 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:54677 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 3178496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:43.489 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.168099+0000", 2024-09-14T23:57:43.489 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.292345261999998, 2024-09-14T23:57:43.489 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028165816, 2024-09-14T23:57:43.489 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:43.489 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:43.489 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:43.490 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:43.490 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:43.490 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:43.490 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54677 2024-09-14T23:57:43.490 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.490 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:43.490 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.490 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:43.490 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168099+0000", 2024-09-14T23:57:43.490 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:43.490 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.490 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.490 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:43.490 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168099+0000", 2024-09-14T23:57:43.491 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:43.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.491 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.491 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:43.491 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168099+0000", 2024-09-14T23:57:43.491 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7400000000000003e-07 2024-09-14T23:57:43.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.491 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.491 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:43.491 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168102+0000", 2024-09-14T23:57:43.491 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2929999999999999e-06 2024-09-14T23:57:43.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.491 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.491 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:43.491 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168108+0000", 2024-09-14T23:57:43.492 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5429999999999999e-06 2024-09-14T23:57:43.492 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.492 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.492 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:43.492 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168113+0000", 2024-09-14T23:57:43.492 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8470000000000003e-06 2024-09-14T23:57:43.492 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.492 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.492 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:43.492 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196124+0000", 2024-09-14T23:57:43.492 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028011034000000001 2024-09-14T23:57:43.492 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.492 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.493 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:43.493 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196194+0000", 2024-09-14T23:57:43.493 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0314000000000005e-05 2024-09-14T23:57:43.493 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.493 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.493 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:43.493 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196264+0000", 2024-09-14T23:57:43.493 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0210999999999996e-05 2024-09-14T23:57:43.493 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:43.493 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:43.493 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:43.493 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.493 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.493 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:54678 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 3358720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:43.494 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.168144+0000", 2024-09-14T23:57:43.494 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.292300103000002, 2024-09-14T23:57:43.494 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028285267999999999, 2024-09-14T23:57:43.494 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:43.494 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:43.494 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:43.494 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:43.494 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:43.494 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:43.494 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54678 2024-09-14T23:57:43.494 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.494 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:43.494 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.494 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:43.494 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168144+0000", 2024-09-14T23:57:43.495 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:43.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.495 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.495 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:43.495 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168144+0000", 2024-09-14T23:57:43.495 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:43.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.495 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.495 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:43.495 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168144+0000", 2024-09-14T23:57:43.495 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5799999999999999e-07 2024-09-14T23:57:43.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.496 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.496 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:43.496 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168147+0000", 2024-09-14T23:57:43.496 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3580000000000001e-06 2024-09-14T23:57:43.496 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.496 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.496 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:43.496 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168154+0000", 2024-09-14T23:57:43.496 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1350000000000003e-06 2024-09-14T23:57:43.496 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.496 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.497 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:43.497 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168159+0000", 2024-09-14T23:57:43.497 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4160000000000003e-06 2024-09-14T23:57:43.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.497 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.497 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:43.497 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196294+0000", 2024-09-14T23:57:43.497 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028135050000000002 2024-09-14T23:57:43.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.497 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.497 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:43.497 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196363+0000", 2024-09-14T23:57:43.497 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8880000000000005e-05 2024-09-14T23:57:43.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.498 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:43.498 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196429+0000", 2024-09-14T23:57:43.498 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5870999999999993e-05 2024-09-14T23:57:43.498 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:43.498 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:43.498 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:43.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.498 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:54679 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:43.498 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.168167+0000", 2024-09-14T23:57:43.498 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.292276882000003, 2024-09-14T23:57:43.498 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028427019000000001, 2024-09-14T23:57:43.499 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:43.499 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:43.499 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:43.499 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:43.499 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:43.499 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:43.499 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54679 2024-09-14T23:57:43.499 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.499 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:43.499 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.499 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:43.499 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168167+0000", 2024-09-14T23:57:43.499 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:43.499 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.500 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.500 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:43.500 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168167+0000", 2024-09-14T23:57:43.500 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:43.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.500 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.500 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:43.500 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168168+0000", 2024-09-14T23:57:43.500 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.44e-07 2024-09-14T23:57:43.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.500 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.500 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:43.500 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168170+0000", 2024-09-14T23:57:43.501 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0870000000000001e-06 2024-09-14T23:57:43.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.501 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.501 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:43.501 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168176+0000", 2024-09-14T23:57:43.501 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9070000000000004e-06 2024-09-14T23:57:43.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.501 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.501 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:43.501 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168181+0000", 2024-09-14T23:57:43.501 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6579999999999999e-06 2024-09-14T23:57:43.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.501 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.501 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:43.502 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196458+0000", 2024-09-14T23:57:43.502 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028276431000000001 2024-09-14T23:57:43.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.502 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.502 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:43.502 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196527+0000", 2024-09-14T23:57:43.502 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9397e-05 2024-09-14T23:57:43.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.502 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.502 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:43.502 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196594+0000", 2024-09-14T23:57:43.502 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6895000000000002e-05 2024-09-14T23:57:43.502 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:43.502 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:43.503 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:43.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.503 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.503 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:54680 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 3719168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:43.503 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.168188+0000", 2024-09-14T23:57:43.503 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.292255676000003, 2024-09-14T23:57:43.503 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028579706999999999, 2024-09-14T23:57:43.503 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:43.503 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:43.503 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:43.503 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:43.503 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:43.503 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:43.503 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54680 2024-09-14T23:57:43.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.504 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:43.504 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.504 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:43.504 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168188+0000", 2024-09-14T23:57:43.504 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:43.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.504 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.504 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:43.504 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168188+0000", 2024-09-14T23:57:43.504 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:43.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.504 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.504 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:43.505 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168189+0000", 2024-09-14T23:57:43.506 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3099999999999998e-07 2024-09-14T23:57:43.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.506 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:43.506 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168191+0000", 2024-09-14T23:57:43.506 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0470000000000001e-06 2024-09-14T23:57:43.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.506 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:43.506 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168199+0000", 2024-09-14T23:57:43.507 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8809999999999996e-06 2024-09-14T23:57:43.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.507 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.507 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:43.507 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168204+0000", 2024-09-14T23:57:43.507 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.011e-06 2024-09-14T23:57:43.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.507 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.507 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:43.507 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196622+0000", 2024-09-14T23:57:43.507 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028418806000000001 2024-09-14T23:57:43.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.507 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.507 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:43.508 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196694+0000", 2024-09-14T23:57:43.508 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1124000000000005e-05 2024-09-14T23:57:43.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.508 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.508 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:43.508 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196768+0000", 2024-09-14T23:57:43.508 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4406999999999994e-05 2024-09-14T23:57:43.508 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:43.508 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:43.508 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:43.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.508 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.508 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:54681 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 3899392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:43.509 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.168212+0000", 2024-09-14T23:57:43.509 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.292231964000003, 2024-09-14T23:57:43.509 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028726643, 2024-09-14T23:57:43.509 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:43.510 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:43.510 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:43.510 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:43.510 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:43.510 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:43.510 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54681 2024-09-14T23:57:43.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.510 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:43.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.510 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:43.510 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168212+0000", 2024-09-14T23:57:43.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:43.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.511 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:43.511 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168212+0000", 2024-09-14T23:57:43.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:43.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.511 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:43.512 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168212+0000", 2024-09-14T23:57:43.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4900000000000001e-07 2024-09-14T23:57:43.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.513 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:43.513 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168214+0000", 2024-09-14T23:57:43.513 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0590000000000001e-06 2024-09-14T23:57:43.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:43.514 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168220+0000", 2024-09-14T23:57:43.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0209999999999996e-06 2024-09-14T23:57:43.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:43.515 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168227+0000", 2024-09-14T23:57:43.515 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4119999999999998e-06 2024-09-14T23:57:43.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.515 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:43.516 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196799+0000", 2024-09-14T23:57:43.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028572528 2024-09-14T23:57:43.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.516 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:43.516 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196866+0000", 2024-09-14T23:57:43.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7005e-05 2024-09-14T23:57:43.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.517 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:43.517 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196939+0000", 2024-09-14T23:57:43.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2169000000000006e-05 2024-09-14T23:57:43.518 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:43.518 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:43.518 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:43.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.519 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:54682 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 4079616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:43.519 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.168234+0000", 2024-09-14T23:57:43.519 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.292210415, 2024-09-14T23:57:43.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028872503000000001, 2024-09-14T23:57:43.519 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:43.519 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:43.520 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:43.520 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:43.520 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:43.520 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:43.520 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 54682 2024-09-14T23:57:43.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.521 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:43.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.521 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:43.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168234+0000", 2024-09-14T23:57:43.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:43.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:43.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168234+0000", 2024-09-14T23:57:43.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:43.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:43.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168234+0000", 2024-09-14T23:57:43.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.58e-07 2024-09-14T23:57:43.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:43.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168236+0000", 2024-09-14T23:57:43.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.933e-06 2024-09-14T23:57:43.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:43.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168242+0000", 2024-09-14T23:57:43.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9970000000000004e-06 2024-09-14T23:57:43.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.526 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:43.526 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.168247+0000", 2024-09-14T23:57:43.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9280000000000001e-06 2024-09-14T23:57:43.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.526 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:43.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.196968+0000", 2024-09-14T23:57:43.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028721159999999999 2024-09-14T23:57:43.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:43.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.197040+0000", 2024-09-14T23:57:43.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1569999999999994e-05 2024-09-14T23:57:43.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:43.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.197106+0000", 2024-09-14T23:57:43.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6457000000000004e-05 2024-09-14T23:57:43.529 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:43.529 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:43.529 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:43.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.530 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:55239 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 3850240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:43.530 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.201093+0000", 2024-09-14T23:57:43.530 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.259350963999999, 2024-09-14T23:57:43.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027610927, 2024-09-14T23:57:43.530 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:43.531 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:43.531 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:43.531 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:43.531 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:43.531 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:43.532 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 55239 2024-09-14T23:57:43.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.532 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:43.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:43.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201093+0000", 2024-09-14T23:57:43.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:43.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:43.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201093+0000", 2024-09-14T23:57:43.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:43.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:43.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201093+0000", 2024-09-14T23:57:43.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5699999999999998e-07 2024-09-14T23:57:43.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:43.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201096+0000", 2024-09-14T23:57:43.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0990000000000001e-06 2024-09-14T23:57:43.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:43.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201102+0000", 2024-09-14T23:57:43.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5710000000000003e-06 2024-09-14T23:57:43.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:43.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201107+0000", 2024-09-14T23:57:43.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1019999999999996e-06 2024-09-14T23:57:43.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:43.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.228568+0000", 2024-09-14T23:57:43.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027461136000000001 2024-09-14T23:57:43.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:43.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.228638+0000", 2024-09-14T23:57:43.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0024999999999994e-05 2024-09-14T23:57:43.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:43.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.228704+0000", 2024-09-14T23:57:43.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5537000000000006e-05 2024-09-14T23:57:43.540 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:43.540 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:43.540 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:43.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.541 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:55240 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:43.541 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.201114+0000", 2024-09-14T23:57:43.541 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.259329727999997, 2024-09-14T23:57:43.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027752295, 2024-09-14T23:57:43.542 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:43.542 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:43.542 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:43.542 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:43.542 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:43.542 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:43.543 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 55240 2024-09-14T23:57:43.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.543 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:43.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:43.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201114+0000", 2024-09-14T23:57:43.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:43.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:43.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201114+0000", 2024-09-14T23:57:43.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:43.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:43.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201115+0000", 2024-09-14T23:57:43.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4200000000000001e-07 2024-09-14T23:57:43.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:43.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201117+0000", 2024-09-14T23:57:43.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.368e-06 2024-09-14T23:57:43.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:43.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201124+0000", 2024-09-14T23:57:43.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4999999999999996e-06 2024-09-14T23:57:43.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:43.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.201136+0000", 2024-09-14T23:57:43.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.293e-05 2024-09-14T23:57:43.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:43.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.228736+0000", 2024-09-14T23:57:43.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027599597 2024-09-14T23:57:43.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:43.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.228806+0000", 2024-09-14T23:57:43.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9616999999999994e-05 2024-09-14T23:57:43.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:43.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.228866+0000", 2024-09-14T23:57:43.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0841000000000002e-05 2024-09-14T23:57:43.549 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:43.549 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:43.550 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:43.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.550 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:55349 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 868352~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:43.550 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.204621+0000", 2024-09-14T23:57:43.550 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.255822561999999, 2024-09-14T23:57:43.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.030294596999999999, 2024-09-14T23:57:43.550 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:43.550 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:43.551 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:43.551 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:43.551 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:43.551 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:43.551 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 55349 2024-09-14T23:57:43.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.551 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:43.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:43.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204621+0000", 2024-09-14T23:57:43.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:43.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:43.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204621+0000", 2024-09-14T23:57:43.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:43.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:43.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204622+0000", 2024-09-14T23:57:43.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.58e-07 2024-09-14T23:57:43.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:43.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204624+0000", 2024-09-14T23:57:43.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.193e-06 2024-09-14T23:57:43.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:43.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204631+0000", 2024-09-14T23:57:43.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1910000000000003e-06 2024-09-14T23:57:43.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:43.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204637+0000", 2024-09-14T23:57:43.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5169999999999999e-06 2024-09-14T23:57:43.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:43.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.229653+0000", 2024-09-14T23:57:43.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.025016283 2024-09-14T23:57:43.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:43.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.229721+0000", 2024-09-14T23:57:43.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8138000000000004e-05 2024-09-14T23:57:43.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:43.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.234916+0000", 2024-09-14T23:57:43.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.005194817 2024-09-14T23:57:43.556 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:43.557 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:43.557 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:43.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.557 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:55363 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 3145728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:43.557 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.204982+0000", 2024-09-14T23:57:43.557 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.255461705999998, 2024-09-14T23:57:43.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027646239999999999, 2024-09-14T23:57:43.558 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:43.558 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:43.558 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:43.558 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:43.559 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:43.559 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:43.559 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 55363 2024-09-14T23:57:43.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.559 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:43.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:43.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204982+0000", 2024-09-14T23:57:43.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:43.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:43.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204982+0000", 2024-09-14T23:57:43.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:43.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:43.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204983+0000", 2024-09-14T23:57:43.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2199999999999999e-07 2024-09-14T23:57:43.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:43.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204985+0000", 2024-09-14T23:57:43.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.261e-06 2024-09-14T23:57:43.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:43.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204991+0000", 2024-09-14T23:57:43.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7069999999999997e-06 2024-09-14T23:57:43.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:43.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.204995+0000", 2024-09-14T23:57:43.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.853e-06 2024-09-14T23:57:43.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:43.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.232491+0000", 2024-09-14T23:57:43.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027495322999999999 2024-09-14T23:57:43.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:43.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.232563+0000", 2024-09-14T23:57:43.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1972999999999997e-05 2024-09-14T23:57:43.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:43.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.232628+0000", 2024-09-14T23:57:43.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5701e-05 2024-09-14T23:57:43.564 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:43.564 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:43.564 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:43.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.564 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:55364 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 3325952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:43.564 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.205003+0000", 2024-09-14T23:57:43.564 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.255441181000002, 2024-09-14T23:57:43.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027781963, 2024-09-14T23:57:43.564 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:43.564 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:43.564 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:43.565 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:43.565 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:43.565 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:43.565 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 55364 2024-09-14T23:57:43.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.565 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:43.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:43.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205003+0000", 2024-09-14T23:57:43.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:43.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:43.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205003+0000", 2024-09-14T23:57:43.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:43.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:43.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205003+0000", 2024-09-14T23:57:43.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2899999999999999e-07 2024-09-14T23:57:43.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:43.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205013+0000", 2024-09-14T23:57:43.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.611e-06 2024-09-14T23:57:43.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:43.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205020+0000", 2024-09-14T23:57:43.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2509999999999997e-06 2024-09-14T23:57:43.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:43.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205025+0000", 2024-09-14T23:57:43.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1399999999999999e-06 2024-09-14T23:57:43.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:43.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.232653+0000", 2024-09-14T23:57:43.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027627972000000001 2024-09-14T23:57:43.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:43.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.232723+0000", 2024-09-14T23:57:43.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9907999999999995e-05 2024-09-14T23:57:43.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:43.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.232785+0000", 2024-09-14T23:57:43.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1651999999999999e-05 2024-09-14T23:57:43.570 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:43.570 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:43.571 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:43.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.571 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:55365 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 3506176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:43.571 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.205033+0000", 2024-09-14T23:57:43.571 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.255410941999997, 2024-09-14T23:57:43.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027914906999999999, 2024-09-14T23:57:43.571 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:43.571 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:43.571 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:43.571 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:43.571 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:43.571 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:43.571 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 55365 2024-09-14T23:57:43.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.572 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:43.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:43.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205033+0000", 2024-09-14T23:57:43.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:43.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:43.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205033+0000", 2024-09-14T23:57:43.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:43.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:43.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205033+0000", 2024-09-14T23:57:43.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9900000000000001e-07 2024-09-14T23:57:43.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:43.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205036+0000", 2024-09-14T23:57:43.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3319999999999999e-06 2024-09-14T23:57:43.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:43.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205042+0000", 2024-09-14T23:57:43.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3049999999999998e-06 2024-09-14T23:57:43.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:43.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205047+0000", 2024-09-14T23:57:43.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0420000000000002e-06 2024-09-14T23:57:43.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:43.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.232809+0000", 2024-09-14T23:57:43.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027761917000000001 2024-09-14T23:57:43.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:43.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.232879+0000", 2024-09-14T23:57:43.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9597000000000005e-05 2024-09-14T23:57:43.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:43.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.232948+0000", 2024-09-14T23:57:43.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9214999999999994e-05 2024-09-14T23:57:43.575 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:43.575 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:43.575 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:43.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.575 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:55366 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 3686400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:43.575 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.205054+0000", 2024-09-14T23:57:43.575 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.255389755000003, 2024-09-14T23:57:43.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028049935000000002, 2024-09-14T23:57:43.576 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:43.576 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:43.576 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:43.576 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:43.576 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:43.576 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:43.576 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 55366 2024-09-14T23:57:43.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.576 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:43.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:43.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205054+0000", 2024-09-14T23:57:43.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:43.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:43.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205054+0000", 2024-09-14T23:57:43.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:43.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:43.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205055+0000", 2024-09-14T23:57:43.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9999999999999998e-07 2024-09-14T23:57:43.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:43.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205065+0000", 2024-09-14T23:57:43.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0489000000000001e-05 2024-09-14T23:57:43.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:43.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205073+0000", 2024-09-14T23:57:43.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9840000000000001e-06 2024-09-14T23:57:43.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:43.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205085+0000", 2024-09-14T23:57:43.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2052000000000001e-05 2024-09-14T23:57:43.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:43.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.232975+0000", 2024-09-14T23:57:43.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.027889646000000001 2024-09-14T23:57:43.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:43.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.233039+0000", 2024-09-14T23:57:43.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3993999999999998e-05 2024-09-14T23:57:43.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:43.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.233104+0000", 2024-09-14T23:57:43.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5270000000000004e-05 2024-09-14T23:57:43.580 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:43.580 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:43.580 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:43.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.580 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:55367 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 3866624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:43.580 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.205098+0000", 2024-09-14T23:57:43.580 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.255346136999997, 2024-09-14T23:57:43.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028159871, 2024-09-14T23:57:43.580 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:43.580 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:43.580 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:43.580 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:43.580 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:43.581 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:43.581 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 55367 2024-09-14T23:57:43.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.581 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:43.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:43.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205098+0000", 2024-09-14T23:57:43.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:43.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:43.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205098+0000", 2024-09-14T23:57:43.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:43.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:43.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205098+0000", 2024-09-14T23:57:43.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4000000000000002e-07 2024-09-14T23:57:43.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:43.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205101+0000", 2024-09-14T23:57:43.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7860000000000001e-06 2024-09-14T23:57:43.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:43.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205109+0000", 2024-09-14T23:57:43.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4039999999999998e-06 2024-09-14T23:57:43.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:43.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205123+0000", 2024-09-14T23:57:43.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.42e-05 2024-09-14T23:57:43.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:43.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.233129+0000", 2024-09-14T23:57:43.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028006316999999999 2024-09-14T23:57:43.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:43.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.233196+0000", 2024-09-14T23:57:43.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7234e-05 2024-09-14T23:57:43.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:43.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.233258+0000", 2024-09-14T23:57:43.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1389999999999993e-05 2024-09-14T23:57:43.584 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:43.584 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:43.584 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:43.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.585 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:55368 3.1 3:812f3a36:::rbd_data.1118c4788fd8.0000000000000004:13 [read 4046848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T23:57:43.585 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.205132+0000", 2024-09-14T23:57:43.585 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.255311825, 2024-09-14T23:57:43.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028285467000000002, 2024-09-14T23:57:43.585 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:43.585 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:43.585 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:43.585 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:43.585 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:43.585 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:43.585 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 55368 2024-09-14T23:57:43.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.585 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:43.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:43.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205132+0000", 2024-09-14T23:57:43.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:43.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:43.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205132+0000", 2024-09-14T23:57:43.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:43.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:43.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205133+0000", 2024-09-14T23:57:43.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.13e-07 2024-09-14T23:57:43.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:43.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205135+0000", 2024-09-14T23:57:43.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5490000000000001e-06 2024-09-14T23:57:43.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:43.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205142+0000", 2024-09-14T23:57:43.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.007e-06 2024-09-14T23:57:43.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:43.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.205147+0000", 2024-09-14T23:57:43.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.169e-06 2024-09-14T23:57:43.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:43.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.233281+0000", 2024-09-14T23:57:43.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.028134059999999999 2024-09-14T23:57:43.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:43.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.233355+0000", 2024-09-14T23:57:43.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3120000000000005e-05 2024-09-14T23:57:43.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:43.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:43.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:43.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.233418+0000", 2024-09-14T23:57:43.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3048999999999999e-05 2024-09-14T23:57:43.589 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:43.589 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:43.589 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:43.589 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:43.589 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:43.589 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:43.591 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T23:57:43.591 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:57:43.592 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:57:43.592 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:43.589+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:43.691 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:43.690+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:43.739 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:43.739 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:57:43.739 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:57:43.739 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:43.748 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T23:57:43.831 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:43.830+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:43.894 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:43.893+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:43.895 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:43.895 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:57:43.895 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:57:43.895 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:57:43.895 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:43.904 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T23:57:43.995 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:43.994+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:44.057 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:44.057 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:57:44.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:57:44.057 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:57:44.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.057 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5880 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:44.057 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375821+0000", 2024-09-14T23:57:44.057 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 64.679223558999993, 2024-09-14T23:57:44.058 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050482096999999997, 2024-09-14T23:57:44.058 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:44.058 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:44.058 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:44.058 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:44.058 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:44.058 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:44.058 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5880 2024-09-14T23:57:44.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.058 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:44.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.058 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:44.058 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375821+0000", 2024-09-14T23:57:44.058 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:44.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375821+0000", 2024-09-14T23:57:44.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:44.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375821+0000", 2024-09-14T23:57:44.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.28e-07 2024-09-14T23:57:44.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:44.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375822+0000", 2024-09-14T23:57:44.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7000000000000003e-07 2024-09-14T23:57:44.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:44.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375825+0000", 2024-09-14T23:57:44.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9560000000000002e-06 2024-09-14T23:57:44.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:44.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375827+0000", 2024-09-14T23:57:44.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3259999999999998e-06 2024-09-14T23:57:44.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.061 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:44.061 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426294+0000", 2024-09-14T23:57:44.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050466748999999998 2024-09-14T23:57:44.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.062 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:44.062 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426303+0000", 2024-09-14T23:57:44.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8680000000000005e-06 2024-09-14T23:57:44.062 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.062 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:44.062 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.062 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5881 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:44.062 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375830+0000", 2024-09-14T23:57:44.062 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 64.679214113, 2024-09-14T23:57:44.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050523831999999998, 2024-09-14T23:57:44.062 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:44.062 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:44.063 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:44.063 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:44.063 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:44.063 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:44.063 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5881 2024-09-14T23:57:44.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.063 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:44.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:44.063 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375830+0000", 2024-09-14T23:57:44.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:44.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375830+0000", 2024-09-14T23:57:44.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.064 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:44.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375830+0000", 2024-09-14T23:57:44.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.34e-07 2024-09-14T23:57:44.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.064 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:44.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375831+0000", 2024-09-14T23:57:44.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9599999999999998e-07 2024-09-14T23:57:44.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:44.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375834+0000", 2024-09-14T23:57:44.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0489999999999999e-06 2024-09-14T23:57:44.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:44.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375838+0000", 2024-09-14T23:57:44.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5319999999999998e-06 2024-09-14T23:57:44.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:44.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426342+0000", 2024-09-14T23:57:44.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050504391000000003 2024-09-14T23:57:44.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:44.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426354+0000", 2024-09-14T23:57:44.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.173e-05 2024-09-14T23:57:44.066 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.066 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:44.066 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.066 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5882 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:44.067 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:57:44.067 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 64.679203509000004, 2024-09-14T23:57:44.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050549666, 2024-09-14T23:57:44.067 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:44.067 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:44.067 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:44.067 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:44.067 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:44.067 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:44.067 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5882 2024-09-14T23:57:44.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.067 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:44.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:44.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:57:44.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:44.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:57:44.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:44.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:57:44.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8099999999999998e-07 2024-09-14T23:57:44.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:44.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375842+0000", 2024-09-14T23:57:44.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9900000000000009e-07 2024-09-14T23:57:44.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:44.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375845+0000", 2024-09-14T23:57:44.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.055e-06 2024-09-14T23:57:44.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:44.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375848+0000", 2024-09-14T23:57:44.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7609999999999999e-06 2024-09-14T23:57:44.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:44.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426381+0000", 2024-09-14T23:57:44.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050533392000000003 2024-09-14T23:57:44.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:44.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426390+0000", 2024-09-14T23:57:44.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0780000000000002e-06 2024-09-14T23:57:44.070 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.071 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:44.071 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.071 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5884 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:44.071 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:57:44.071 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 64.679193369000004, 2024-09-14T23:57:44.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050717439000000003, 2024-09-14T23:57:44.071 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:44.071 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:44.071 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:44.071 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:44.071 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:44.071 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:44.072 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5884 2024-09-14T23:57:44.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.072 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:44.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:44.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:57:44.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:44.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:57:44.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:44.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:57:44.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0800000000000001e-07 2024-09-14T23:57:44.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:44.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375852+0000", 2024-09-14T23:57:44.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.02e-06 2024-09-14T23:57:44.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:44.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375855+0000", 2024-09-14T23:57:44.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9900000000000002e-06 2024-09-14T23:57:44.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:44.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375857+0000", 2024-09-14T23:57:44.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3499999999999999e-06 2024-09-14T23:57:44.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:44.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426556+0000", 2024-09-14T23:57:44.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050698139000000003 2024-09-14T23:57:44.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:44.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426568+0000", 2024-09-14T23:57:44.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2731999999999999e-05 2024-09-14T23:57:44.075 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.075 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:44.075 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.075 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5885 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:44.075 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:57:44.075 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 64.679174344000003, 2024-09-14T23:57:44.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050737333000000003, 2024-09-14T23:57:44.075 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:44.075 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:44.076 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:44.076 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:44.076 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:44.076 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:44.076 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5885 2024-09-14T23:57:44.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.076 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:44.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:44.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:57:44.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:44.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:57:44.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:44.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:57:44.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5899999999999998e-07 2024-09-14T23:57:44.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:44.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375872+0000", 2024-09-14T23:57:44.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4219999999999999e-06 2024-09-14T23:57:44.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:44.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375878+0000", 2024-09-14T23:57:44.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4980000000000003e-06 2024-09-14T23:57:44.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:44.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375881+0000", 2024-09-14T23:57:44.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6110000000000001e-06 2024-09-14T23:57:44.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:44.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426596+0000", 2024-09-14T23:57:44.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050715320000000001 2024-09-14T23:57:44.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:44.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426607+0000", 2024-09-14T23:57:44.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1223000000000001e-05 2024-09-14T23:57:44.080 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.080 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:44.080 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.080 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5886 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:44.080 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:57:44.081 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 64.679160230999997, 2024-09-14T23:57:44.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050803395000000001, 2024-09-14T23:57:44.081 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:44.081 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:44.081 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:44.081 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:44.082 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:44.082 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:44.082 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5886 2024-09-14T23:57:44.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.082 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:44.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:44.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:57:44.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:44.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:57:44.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:44.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:57:44.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2700000000000001e-07 2024-09-14T23:57:44.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:44.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375886+0000", 2024-09-14T23:57:44.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7060000000000001e-06 2024-09-14T23:57:44.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:44.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375890+0000", 2024-09-14T23:57:44.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2259999999999999e-06 2024-09-14T23:57:44.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:44.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375893+0000", 2024-09-14T23:57:44.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0819999999999999e-06 2024-09-14T23:57:44.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:44.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426677+0000", 2024-09-14T23:57:44.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050784158000000003 2024-09-14T23:57:44.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:44.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426687+0000", 2024-09-14T23:57:44.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9960000000000005e-06 2024-09-14T23:57:44.088 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.088 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:44.088 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.089 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5887 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:44.089 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:57:44.089 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 64.679138144999996, 2024-09-14T23:57:44.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050833070000000001, 2024-09-14T23:57:44.089 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:44.089 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:44.090 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:44.090 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:44.090 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:44.090 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:44.090 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5887 2024-09-14T23:57:44.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.091 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:44.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:44.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:57:44.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:44.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:57:44.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:44.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:57:44.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3299999999999998e-07 2024-09-14T23:57:44.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:44.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375908+0000", 2024-09-14T23:57:44.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7689999999999999e-06 2024-09-14T23:57:44.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:44.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375912+0000", 2024-09-14T23:57:44.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.535e-06 2024-09-14T23:57:44.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:44.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375916+0000", 2024-09-14T23:57:44.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8979999999999998e-06 2024-09-14T23:57:44.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:44.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426729+0000", 2024-09-14T23:57:44.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050813453000000001 2024-09-14T23:57:44.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:44.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426739+0000", 2024-09-14T23:57:44.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0081999999999999e-05 2024-09-14T23:57:44.095 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.095 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:44.095 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.096 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5888 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:44.096 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:57:44.096 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 64.679125068000005, 2024-09-14T23:57:44.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050854652, 2024-09-14T23:57:44.096 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:44.096 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:44.096 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:44.096 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:44.096 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:44.096 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:44.096 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5888 2024-09-14T23:57:44.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.097 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:44.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:44.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:57:44.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:44.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:57:44.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:44.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:57:44.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1400000000000001e-07 2024-09-14T23:57:44.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:44.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375921+0000", 2024-09-14T23:57:44.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.119e-06 2024-09-14T23:57:44.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:44.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375928+0000", 2024-09-14T23:57:44.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3170000000000004e-06 2024-09-14T23:57:44.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:44.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375931+0000", 2024-09-14T23:57:44.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6970000000000002e-06 2024-09-14T23:57:44.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:44.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426768+0000", 2024-09-14T23:57:44.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050837355000000001 2024-09-14T23:57:44.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:44.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426774+0000", 2024-09-14T23:57:44.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9499999999999998e-06 2024-09-14T23:57:44.100 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.100 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:44.100 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.100 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5889 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:44.100 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:57:44.100 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 64.679108143999997, 2024-09-14T23:57:44.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050885446000000001, 2024-09-14T23:57:44.101 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:44.101 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:44.101 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:44.101 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:44.101 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:44.101 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:44.101 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5889 2024-09-14T23:57:44.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.101 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:44.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:44.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:57:44.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:44.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:57:44.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:44.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:57:44.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.36e-07 2024-09-14T23:57:44.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:44.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375940+0000", 2024-09-14T23:57:44.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.844e-06 2024-09-14T23:57:44.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:44.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375944+0000", 2024-09-14T23:57:44.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6720000000000002e-06 2024-09-14T23:57:44.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:44.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375946+0000", 2024-09-14T23:57:44.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5629999999999999e-06 2024-09-14T23:57:44.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:44.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426810+0000", 2024-09-14T23:57:44.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050863255000000003 2024-09-14T23:57:44.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:44.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426822+0000", 2024-09-14T23:57:44.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1875999999999999e-05 2024-09-14T23:57:44.105 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.105 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:44.105 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.105 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5890 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:44.105 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:57:44.105 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 64.679094332000005, 2024-09-14T23:57:44.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050917549999999999, 2024-09-14T23:57:44.106 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:44.106 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:44.106 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:44.106 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:44.106 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:44.106 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:44.106 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5890 2024-09-14T23:57:44.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.106 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:44.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:44.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:57:44.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:44.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:57:44.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:44.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:57:44.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4299999999999999e-07 2024-09-14T23:57:44.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:44.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375951+0000", 2024-09-14T23:57:44.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1000000000000001e-06 2024-09-14T23:57:44.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:44.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375956+0000", 2024-09-14T23:57:44.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3370000000000001e-06 2024-09-14T23:57:44.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:44.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375958+0000", 2024-09-14T23:57:44.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4820000000000001e-06 2024-09-14T23:57:44.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:44.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426858+0000", 2024-09-14T23:57:44.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050899666000000003 2024-09-14T23:57:44.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:44.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426867+0000", 2024-09-14T23:57:44.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7219999999999994e-06 2024-09-14T23:57:44.109 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.109 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:44.109 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.110 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5891 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:44.110 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:57:44.110 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 64.679083009999999, 2024-09-14T23:57:44.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050948067, 2024-09-14T23:57:44.110 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:44.110 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:44.110 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:44.110 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:44.110 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:44.111 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:44.111 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5891 2024-09-14T23:57:44.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.111 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:44.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:44.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:57:44.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:44.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:57:44.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:44.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:57:44.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0800000000000001e-07 2024-09-14T23:57:44.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:44.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375962+0000", 2024-09-14T23:57:44.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9600000000000008e-07 2024-09-14T23:57:44.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:44.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375970+0000", 2024-09-14T23:57:44.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1600000000000001e-06 2024-09-14T23:57:44.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:44.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375972+0000", 2024-09-14T23:57:44.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4849999999999999e-06 2024-09-14T23:57:44.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:44.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426902+0000", 2024-09-14T23:57:44.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050929813999999997 2024-09-14T23:57:44.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:44.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426909+0000", 2024-09-14T23:57:44.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4039999999999998e-06 2024-09-14T23:57:44.114 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.114 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:44.114 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.114 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5892 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:44.114 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:57:44.114 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 64.679068153000003, 2024-09-14T23:57:44.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050965036999999998, 2024-09-14T23:57:44.115 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:44.115 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:44.115 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:44.115 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:44.115 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:44.115 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:44.115 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5892 2024-09-14T23:57:44.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.115 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:44.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:44.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:57:44.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:44.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:57:44.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:44.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:57:44.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.34e-07 2024-09-14T23:57:44.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:44.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375978+0000", 2024-09-14T23:57:44.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1880000000000001e-06 2024-09-14T23:57:44.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:44.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375981+0000", 2024-09-14T23:57:44.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2399999999999999e-06 2024-09-14T23:57:44.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:44.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375984+0000", 2024-09-14T23:57:44.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9110000000000002e-06 2024-09-14T23:57:44.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:44.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426934+0000", 2024-09-14T23:57:44.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050949912 2024-09-14T23:57:44.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:44.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426941+0000", 2024-09-14T23:57:44.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5519999999999999e-06 2024-09-14T23:57:44.119 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.119 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:44.119 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.119 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5893 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:44.119 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:57:44.119 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 64.679057556000004, 2024-09-14T23:57:44.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050975131999999999, 2024-09-14T23:57:44.119 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:44.119 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:44.119 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:44.119 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:44.119 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:44.120 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:44.120 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5893 2024-09-14T23:57:44.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.120 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:44.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:44.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:57:44.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:44.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:57:44.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:44.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:57:44.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2999999999999999e-07 2024-09-14T23:57:44.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:44.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375988+0000", 2024-09-14T23:57:44.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0410000000000001e-06 2024-09-14T23:57:44.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:44.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375992+0000", 2024-09-14T23:57:44.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.104e-06 2024-09-14T23:57:44.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:44.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375995+0000", 2024-09-14T23:57:44.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4310000000000001e-06 2024-09-14T23:57:44.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:44.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426958+0000", 2024-09-14T23:57:44.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050963642000000003 2024-09-14T23:57:44.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:44.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426962+0000", 2024-09-14T23:57:44.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6839999999999998e-06 2024-09-14T23:57:44.123 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.123 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:44.123 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.123 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5894 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:44.124 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:57:44.124 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 64.679046155999998, 2024-09-14T23:57:44.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050978333000000001, 2024-09-14T23:57:44.124 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:44.124 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:44.124 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:44.124 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:44.124 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:44.124 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:44.124 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5894 2024-09-14T23:57:44.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.124 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:44.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:44.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:57:44.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:44.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:57:44.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:44.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:57:44.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.23e-07 2024-09-14T23:57:44.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:44.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375999+0000", 2024-09-14T23:57:44.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9800000000000002e-07 2024-09-14T23:57:44.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:44.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376007+0000", 2024-09-14T23:57:44.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8539999999999997e-06 2024-09-14T23:57:44.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:44.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376011+0000", 2024-09-14T23:57:44.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.399e-06 2024-09-14T23:57:44.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:44.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426973+0000", 2024-09-14T23:57:44.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050962454999999997 2024-09-14T23:57:44.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:44.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426976+0000", 2024-09-14T23:57:44.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4039999999999999e-06 2024-09-14T23:57:44.128 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.128 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:44.128 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.128 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5895 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:44.128 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376014+0000", 2024-09-14T23:57:44.128 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 64.679029811999996, 2024-09-14T23:57:44.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050975951999999998, 2024-09-14T23:57:44.128 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:44.128 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:44.128 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:44.129 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:44.129 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:44.129 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:44.129 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5895 2024-09-14T23:57:44.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.129 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:44.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:44.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376014+0000", 2024-09-14T23:57:44.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:44.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376014+0000", 2024-09-14T23:57:44.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:44.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376015+0000", 2024-09-14T23:57:44.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2999999999999999e-07 2024-09-14T23:57:44.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:44.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376016+0000", 2024-09-14T23:57:44.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1659999999999999e-06 2024-09-14T23:57:44.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:44.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376024+0000", 2024-09-14T23:57:44.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8930000000000005e-06 2024-09-14T23:57:44.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:44.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376026+0000", 2024-09-14T23:57:44.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6869999999999999e-06 2024-09-14T23:57:44.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:44.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426987+0000", 2024-09-14T23:57:44.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050960601000000001 2024-09-14T23:57:44.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:44.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426990+0000", 2024-09-14T23:57:44.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3749999999999999e-06 2024-09-14T23:57:44.132 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.132 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:44.132 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.133 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5896 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:44.133 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:57:44.133 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 64.679014074999998, 2024-09-14T23:57:44.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050975656000000001, 2024-09-14T23:57:44.133 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:44.133 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:44.133 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:44.133 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:44.133 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:44.133 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:44.133 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5896 2024-09-14T23:57:44.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.134 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:44.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:44.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:57:44.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:44.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:57:44.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:44.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:57:44.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5100000000000001e-07 2024-09-14T23:57:44.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:44.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376032+0000", 2024-09-14T23:57:44.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.141e-06 2024-09-14T23:57:44.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:44.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376036+0000", 2024-09-14T23:57:44.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.25e-06 2024-09-14T23:57:44.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:44.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376038+0000", 2024-09-14T23:57:44.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5469999999999999e-06 2024-09-14T23:57:44.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:44.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427002+0000", 2024-09-14T23:57:44.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050964029000000001 2024-09-14T23:57:44.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:44.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427006+0000", 2024-09-14T23:57:44.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4379999999999999e-06 2024-09-14T23:57:44.138 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.138 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:44.138 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.138 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5897 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:44.138 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:57:44.138 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 64.679002275000002, 2024-09-14T23:57:44.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050978001000000002, 2024-09-14T23:57:44.138 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:44.138 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:44.139 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:44.139 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:44.139 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:44.139 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:44.139 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5897 2024-09-14T23:57:44.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.139 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:44.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:44.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:57:44.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:44.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:57:44.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:44.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:57:44.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2600000000000001e-07 2024-09-14T23:57:44.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:44.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376043+0000", 2024-09-14T23:57:44.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.054e-06 2024-09-14T23:57:44.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:44.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376047+0000", 2024-09-14T23:57:44.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0659999999999997e-06 2024-09-14T23:57:44.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:44.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376050+0000", 2024-09-14T23:57:44.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.88e-06 2024-09-14T23:57:44.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:44.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427017+0000", 2024-09-14T23:57:44.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050966453000000002 2024-09-14T23:57:44.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:44.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427020+0000", 2024-09-14T23:57:44.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3220000000000001e-06 2024-09-14T23:57:44.142 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.142 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:44.142 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.143 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5898 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:44.143 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376053+0000", 2024-09-14T23:57:44.143 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 64.67899079, 2024-09-14T23:57:44.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050982024000000001, 2024-09-14T23:57:44.143 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:44.143 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:44.143 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:44.143 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:44.143 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:44.143 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:44.143 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5898 2024-09-14T23:57:44.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.144 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:44.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:44.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376053+0000", 2024-09-14T23:57:44.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:44.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376053+0000", 2024-09-14T23:57:44.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:44.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376278+0000", 2024-09-14T23:57:44.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022426400000000001 2024-09-14T23:57:44.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:44.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376282+0000", 2024-09-14T23:57:44.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4739999999999999e-06 2024-09-14T23:57:44.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:44.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376296+0000", 2024-09-14T23:57:44.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4229999999999999e-05 2024-09-14T23:57:44.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:44.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376303+0000", 2024-09-14T23:57:44.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2199999999999997e-06 2024-09-14T23:57:44.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:44.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427031+0000", 2024-09-14T23:57:44.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050728154999999997 2024-09-14T23:57:44.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:44.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427036+0000", 2024-09-14T23:57:44.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6809999999999997e-06 2024-09-14T23:57:44.147 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.147 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:44.148 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.148 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5899 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:44.148 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376338+0000", 2024-09-14T23:57:44.148 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 64.678706066000004, 2024-09-14T23:57:44.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050712100000000003, 2024-09-14T23:57:44.148 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:44.148 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:44.148 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:44.149 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:44.149 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:44.149 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:44.149 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5899 2024-09-14T23:57:44.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.149 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:44.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:44.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376338+0000", 2024-09-14T23:57:44.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:44.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376338+0000", 2024-09-14T23:57:44.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:44.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376339+0000", 2024-09-14T23:57:44.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9100000000000003e-07 2024-09-14T23:57:44.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:44.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376342+0000", 2024-09-14T23:57:44.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7e-06 2024-09-14T23:57:44.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:44.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376350+0000", 2024-09-14T23:57:44.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5280000000000003e-06 2024-09-14T23:57:44.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:44.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376356+0000", 2024-09-14T23:57:44.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6010000000000002e-06 2024-09-14T23:57:44.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:44.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427047+0000", 2024-09-14T23:57:44.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050690962999999999 2024-09-14T23:57:44.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.152 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:44.152 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427050+0000", 2024-09-14T23:57:44.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.517e-06 2024-09-14T23:57:44.152 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.152 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:44.152 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.153 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5900 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:44.153 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376366+0000", 2024-09-14T23:57:44.153 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 64.678677807, 2024-09-14T23:57:44.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050697858999999998, 2024-09-14T23:57:44.153 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:44.153 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:44.153 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:44.153 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:44.153 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:44.153 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:44.153 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5900 2024-09-14T23:57:44.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.153 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:44.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:44.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376366+0000", 2024-09-14T23:57:44.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:44.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376366+0000", 2024-09-14T23:57:44.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:44.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377650+0000", 2024-09-14T23:57:44.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001283834 2024-09-14T23:57:44.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:44.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377655+0000", 2024-09-14T23:57:44.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7230000000000004e-06 2024-09-14T23:57:44.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:44.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377667+0000", 2024-09-14T23:57:44.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1746000000000001e-05 2024-09-14T23:57:44.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:44.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377676+0000", 2024-09-14T23:57:44.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3619999999999995e-06 2024-09-14T23:57:44.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.156 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:44.156 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427061+0000", 2024-09-14T23:57:44.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049384755000000002 2024-09-14T23:57:44.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.156 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:44.156 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427064+0000", 2024-09-14T23:57:44.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.439e-06 2024-09-14T23:57:44.156 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.156 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:44.156 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.157 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.157 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:44.157 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:44.158 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T23:57:44.159 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:44.157+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:44.259 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:44.257+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:44.267 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:44.268 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:57:44.268 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:57:44.268 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:44.276 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T23:57:44.360 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:44.358+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:44.425 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:44.425 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:57:44.425 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:57:44.425 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:57:44.425 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:44.435 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T23:57:44.460 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:44.459+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:44.543 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:44.544 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:57:44.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:57:44.544 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:57:44.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.544 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:3578 3.4 e29/23 3:2298d860:::rbd_data.1118de1168b.0000000000000030:head v 29'15, mlcod=29'13)", 2024-09-14T23:57:44.544 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.036455+0000", 2024-09-14T23:57:44.544 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.504905237000003, 2024-09-14T23:57:44.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019449440000000001, 2024-09-14T23:57:44.544 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:44.544 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:44.544 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:44.545 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:44.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:44.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036455+0000", 2024-09-14T23:57:44.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:44.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036455+0000", 2024-09-14T23:57:44.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:44.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036456+0000", 2024-09-14T23:57:44.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0999999999999997e-07 2024-09-14T23:57:44.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:44.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036463+0000", 2024-09-14T23:57:44.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2200000000000003e-06 2024-09-14T23:57:44.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:44.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036478+0000", 2024-09-14T23:57:44.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4977000000000001e-05 2024-09-14T23:57:44.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:44.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036485+0000", 2024-09-14T23:57:44.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0559999999999999e-06 2024-09-14T23:57:44.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:44.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036693+0000", 2024-09-14T23:57:44.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000208051 2024-09-14T23:57:44.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:44.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036715+0000", 2024-09-14T23:57:44.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1330000000000001e-05 2024-09-14T23:57:44.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:44.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.038356+0000", 2024-09-14T23:57:44.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001641586 2024-09-14T23:57:44.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:44.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.038400+0000", 2024-09-14T23:57:44.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3914e-05 2024-09-14T23:57:44.548 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.548 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:44.549 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.549 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:3579 3.4 e29/23 3:27290fe6:::rbd_data.1118de1168b.0000000000000032:head v 29'16, mlcod=29'13)", 2024-09-14T23:57:44.549 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.037085+0000", 2024-09-14T23:57:44.549 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.504275647, 2024-09-14T23:57:44.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0020019220000000002, 2024-09-14T23:57:44.549 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:44.549 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:44.549 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:44.549 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:44.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:44.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037085+0000", 2024-09-14T23:57:44.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:44.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037085+0000", 2024-09-14T23:57:44.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:44.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037085+0000", 2024-09-14T23:57:44.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5499999999999997e-07 2024-09-14T23:57:44.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:44.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037091+0000", 2024-09-14T23:57:44.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7869999999999998e-06 2024-09-14T23:57:44.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:44.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037102+0000", 2024-09-14T23:57:44.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.031e-05 2024-09-14T23:57:44.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:44.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037116+0000", 2024-09-14T23:57:44.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4119e-05 2024-09-14T23:57:44.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:44.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037219+0000", 2024-09-14T23:57:44.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010372799999999999 2024-09-14T23:57:44.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:44.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037239+0000", 2024-09-14T23:57:44.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9993e-05 2024-09-14T23:57:44.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:44.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.039046+0000", 2024-09-14T23:57:44.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001806666 2024-09-14T23:57:44.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:44.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.039087+0000", 2024-09-14T23:57:44.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0564e-05 2024-09-14T23:57:44.553 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.553 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:44.553 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.553 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:32024 3.9 e39/23 3:94dc260b:::rbd_data.111814b482f8.0000000000000009:head v 39'45, mlcod=38'43)", 2024-09-14T23:57:44.553 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:49.061018+0000", 2024-09-14T23:57:44.553 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 55.480342307999997, 2024-09-14T23:57:44.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018482240000000001, 2024-09-14T23:57:44.554 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:44.554 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:44.554 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:44.554 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:44.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:44.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061018+0000", 2024-09-14T23:57:44.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:44.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061018+0000", 2024-09-14T23:57:44.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:44.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061018+0000", 2024-09-14T23:57:44.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9999999999999998e-07 2024-09-14T23:57:44.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:44.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061211+0000", 2024-09-14T23:57:44.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019206 2024-09-14T23:57:44.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:44.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061221+0000", 2024-09-14T23:57:44.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0659000000000001e-05 2024-09-14T23:57:44.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:44.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061230+0000", 2024-09-14T23:57:44.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4610000000000008e-06 2024-09-14T23:57:44.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:44.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061317+0000", 2024-09-14T23:57:44.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7514000000000003e-05 2024-09-14T23:57:44.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:44.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061338+0000", 2024-09-14T23:57:44.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0395999999999999e-05 2024-09-14T23:57:44.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:44.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.062830+0000", 2024-09-14T23:57:44.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0014921940000000001 2024-09-14T23:57:44.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:44.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.062866+0000", 2024-09-14T23:57:44.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6539999999999999e-05 2024-09-14T23:57:44.557 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.557 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:44.558 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.558 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:32420 3.9 e39/23 3:91dc34e7:::rbd_data.111814b482f8.000000000000001e:head v 39'46, mlcod=38'44)", 2024-09-14T23:57:44.558 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:49.086124+0000", 2024-09-14T23:57:44.558 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 55.455236808999999, 2024-09-14T23:57:44.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001686819, 2024-09-14T23:57:44.558 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:44.558 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:44.558 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:44.558 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:44.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:44.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086124+0000", 2024-09-14T23:57:44.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:44.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086124+0000", 2024-09-14T23:57:44.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:44.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086125+0000", 2024-09-14T23:57:44.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.361e-06 2024-09-14T23:57:44.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:44.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086605+0000", 2024-09-14T23:57:44.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00048026600000000003 2024-09-14T23:57:44.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:44.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086626+0000", 2024-09-14T23:57:44.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.037e-05 2024-09-14T23:57:44.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:44.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086635+0000", 2024-09-14T23:57:44.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5880000000000004e-06 2024-09-14T23:57:44.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:44.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086701+0000", 2024-09-14T23:57:44.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5905e-05 2024-09-14T23:57:44.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:44.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086727+0000", 2024-09-14T23:57:44.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5573000000000001e-05 2024-09-14T23:57:44.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:44.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.087767+0000", 2024-09-14T23:57:44.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001040537 2024-09-14T23:57:44.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:44.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.087810+0000", 2024-09-14T23:57:44.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3219000000000001e-05 2024-09-14T23:57:44.562 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.562 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:44.562 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.562 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:34296 3.9 e39/23 3:974a38c7:::rbd_data.111814b482f8.000000000000003a:head v 39'47, mlcod=39'45)", 2024-09-14T23:57:44.562 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:49.192105+0000", 2024-09-14T23:57:44.562 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 55.349254926, 2024-09-14T23:57:44.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001607516, 2024-09-14T23:57:44.562 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:44.562 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:44.563 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:44.563 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:44.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:44.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.192105+0000", 2024-09-14T23:57:44.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:44.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.192105+0000", 2024-09-14T23:57:44.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:44.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.192107+0000", 2024-09-14T23:57:44.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3549999999999999e-06 2024-09-14T23:57:44.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:44.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.192332+0000", 2024-09-14T23:57:44.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022486300000000001 2024-09-14T23:57:44.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:44.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.192349+0000", 2024-09-14T23:57:44.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6966e-05 2024-09-14T23:57:44.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:44.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.192358+0000", 2024-09-14T23:57:44.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8309999999999996e-06 2024-09-14T23:57:44.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:44.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.192422+0000", 2024-09-14T23:57:44.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3915e-05 2024-09-14T23:57:44.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:44.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.192450+0000", 2024-09-14T23:57:44.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7957e-05 2024-09-14T23:57:44.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:44.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.193662+0000", 2024-09-14T23:57:44.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012113270000000001 2024-09-14T23:57:44.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:44.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.193713+0000", 2024-09-14T23:57:44.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1301999999999998e-05 2024-09-14T23:57:44.567 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.567 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:44.567 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.567 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:50582 3.9 e51/23 3:92198e7a:::rbd_data.11183b9aac34.000000000000000a:head v 51'63, mlcod=50'60)", 2024-09-14T23:57:44.567 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:01.131541+0000", 2024-09-14T23:57:44.567 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 43.409819618, 2024-09-14T23:57:44.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001795547, 2024-09-14T23:57:44.568 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:44.568 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:44.568 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:44.568 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:44.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:44.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131541+0000", 2024-09-14T23:57:44.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:44.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131541+0000", 2024-09-14T23:57:44.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:44.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131541+0000", 2024-09-14T23:57:44.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7099999999999997e-07 2024-09-14T23:57:44.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:44.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131548+0000", 2024-09-14T23:57:44.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.708e-06 2024-09-14T23:57:44.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:44.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131570+0000", 2024-09-14T23:57:44.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1815999999999999e-05 2024-09-14T23:57:44.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:44.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131581+0000", 2024-09-14T23:57:44.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0933e-05 2024-09-14T23:57:44.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:44.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131660+0000", 2024-09-14T23:57:44.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9071000000000002e-05 2024-09-14T23:57:44.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:44.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131682+0000", 2024-09-14T23:57:44.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2534000000000001e-05 2024-09-14T23:57:44.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:44.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.133287+0000", 2024-09-14T23:57:44.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016045510000000001 2024-09-14T23:57:44.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:44.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.133336+0000", 2024-09-14T23:57:44.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9563e-05 2024-09-14T23:57:44.571 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.571 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:44.572 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.572 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:51182 3.9 e52/23 3:918e4fa0:::rbd_data.11181975a1de.000000000000007b:head v 52'74, mlcod=51'71)", 2024-09-14T23:57:44.572 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:02.138725+0000", 2024-09-14T23:57:44.572 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.402635881000002, 2024-09-14T23:57:44.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015321009999999999, 2024-09-14T23:57:44.572 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:44.572 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:44.572 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:44.572 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:44.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:44.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.138725+0000", 2024-09-14T23:57:44.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:44.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.138725+0000", 2024-09-14T23:57:44.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:44.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.138725+0000", 2024-09-14T23:57:44.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.22e-07 2024-09-14T23:57:44.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:44.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.138732+0000", 2024-09-14T23:57:44.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1779999999999997e-06 2024-09-14T23:57:44.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:44.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.138749+0000", 2024-09-14T23:57:44.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6500000000000001e-05 2024-09-14T23:57:44.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:44.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.138758+0000", 2024-09-14T23:57:44.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6239999999999997e-06 2024-09-14T23:57:44.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:44.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.138816+0000", 2024-09-14T23:57:44.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7939000000000002e-05 2024-09-14T23:57:44.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:44.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.138840+0000", 2024-09-14T23:57:44.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3856e-05 2024-09-14T23:57:44.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:44.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.140204+0000", 2024-09-14T23:57:44.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013641339999999999 2024-09-14T23:57:44.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:44.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.140257+0000", 2024-09-14T23:57:44.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2547999999999999e-05 2024-09-14T23:57:44.576 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.576 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:44.576 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.576 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:51281 3.0 e52/23 3:04e60ec2:::rbd_data.11181975a1de.0000000000000003:head v 52'113, mlcod=44'111)", 2024-09-14T23:57:44.576 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:02.158822+0000", 2024-09-14T23:57:44.576 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.382538122, 2024-09-14T23:57:44.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019860609999999999, 2024-09-14T23:57:44.576 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:44.576 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:44.577 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:44.577 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:44.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:44.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.158822+0000", 2024-09-14T23:57:44.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:44.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.158822+0000", 2024-09-14T23:57:44.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:44.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.158823+0000", 2024-09-14T23:57:44.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6800000000000001e-07 2024-09-14T23:57:44.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:44.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.159132+0000", 2024-09-14T23:57:44.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030974900000000001 2024-09-14T23:57:44.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:44.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.159146+0000", 2024-09-14T23:57:44.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3261000000000001e-05 2024-09-14T23:57:44.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:44.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.159155+0000", 2024-09-14T23:57:44.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7410000000000004e-06 2024-09-14T23:57:44.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:44.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.159221+0000", 2024-09-14T23:57:44.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5698999999999996e-05 2024-09-14T23:57:44.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:44.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.159244+0000", 2024-09-14T23:57:44.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3108e-05 2024-09-14T23:57:44.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:44.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.160767+0000", 2024-09-14T23:57:44.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015231120000000001 2024-09-14T23:57:44.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:44.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.160808+0000", 2024-09-14T23:57:44.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1022999999999997e-05 2024-09-14T23:57:44.580 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.580 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:44.581 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.581 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:56107 3.0 e56/23 3:00c47890:::rbd_data.1118c4b3aebe.0000000000000009:head v 56'125, mlcod=52'123)", 2024-09-14T23:57:44.581 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.236123+0000", 2024-09-14T23:57:44.581 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 38.305236923000002, 2024-09-14T23:57:44.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0061273220000000002, 2024-09-14T23:57:44.581 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:44.581 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:44.581 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:44.581 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:44.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:44.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.236123+0000", 2024-09-14T23:57:44.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:44.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.236123+0000", 2024-09-14T23:57:44.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:44.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.236125+0000", 2024-09-14T23:57:44.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.313e-06 2024-09-14T23:57:44.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:44.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237428+0000", 2024-09-14T23:57:44.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013029890000000001 2024-09-14T23:57:44.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:44.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237452+0000", 2024-09-14T23:57:44.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4090000000000001e-05 2024-09-14T23:57:44.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:44.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237462+0000", 2024-09-14T23:57:44.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0421999999999999e-05 2024-09-14T23:57:44.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:44.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237559+0000", 2024-09-14T23:57:44.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6378000000000005e-05 2024-09-14T23:57:44.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:44.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237583+0000", 2024-09-14T23:57:44.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4182e-05 2024-09-14T23:57:44.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:44.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.242205+0000", 2024-09-14T23:57:44.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0046217289999999998 2024-09-14T23:57:44.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:44.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.242251+0000", 2024-09-14T23:57:44.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6218999999999999e-05 2024-09-14T23:57:44.585 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.585 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:44.585 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.586 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:56108 3.0 e56/23 3:00c47890:::rbd_data.1118c4b3aebe.0000000000000009:head v 56'127, mlcod=52'123)", 2024-09-14T23:57:44.586 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.239693+0000", 2024-09-14T23:57:44.586 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 38.301667453, 2024-09-14T23:57:44.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0043496209999999997, 2024-09-14T23:57:44.586 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:44.586 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:44.586 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:44.586 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:44.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:44.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239693+0000", 2024-09-14T23:57:44.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:44.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239693+0000", 2024-09-14T23:57:44.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:44.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239694+0000", 2024-09-14T23:57:44.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0499999999999999e-06 2024-09-14T23:57:44.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:44.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239702+0000", 2024-09-14T23:57:44.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1170000000000004e-06 2024-09-14T23:57:44.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:44.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239716+0000", 2024-09-14T23:57:44.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4306e-05 2024-09-14T23:57:44.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:44.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239724+0000", 2024-09-14T23:57:44.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0399999999999993e-06 2024-09-14T23:57:44.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:44.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239778+0000", 2024-09-14T23:57:44.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4032e-05 2024-09-14T23:57:44.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:44.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239801+0000", 2024-09-14T23:57:44.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2770000000000001e-05 2024-09-14T23:57:44.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:44.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.243990+0000", 2024-09-14T23:57:44.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0041890909999999998 2024-09-14T23:57:44.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:44.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.244043+0000", 2024-09-14T23:57:44.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2215000000000001e-05 2024-09-14T23:57:44.590 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.590 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:44.590 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.590 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:56385 3.9 e56/23 3:963d31c7:::rbd_data.1118c4b3aebe.0000000000000003:head v 56'87, mlcod=53'85)", 2024-09-14T23:57:44.592 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.283936+0000", 2024-09-14T23:57:44.592 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 38.257424542999999, 2024-09-14T23:57:44.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0063042080000000004, 2024-09-14T23:57:44.592 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:44.592 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:44.592 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:44.593 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:44.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:44.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.283936+0000", 2024-09-14T23:57:44.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:44.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.283936+0000", 2024-09-14T23:57:44.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:44.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.283937+0000", 2024-09-14T23:57:44.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1179999999999999e-06 2024-09-14T23:57:44.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:44.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286338+0000", 2024-09-14T23:57:44.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024010149999999998 2024-09-14T23:57:44.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:44.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286360+0000", 2024-09-14T23:57:44.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2212e-05 2024-09-14T23:57:44.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:44.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286370+0000", 2024-09-14T23:57:44.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5850000000000006e-06 2024-09-14T23:57:44.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:44.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286443+0000", 2024-09-14T23:57:44.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2757999999999997e-05 2024-09-14T23:57:44.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:44.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286466+0000", 2024-09-14T23:57:44.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3844000000000001e-05 2024-09-14T23:57:44.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:44.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.290197+0000", 2024-09-14T23:57:44.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003730176 2024-09-14T23:57:44.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:44.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.290240+0000", 2024-09-14T23:57:44.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.35e-05 2024-09-14T23:57:44.596 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.596 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:44.596 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.597 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:56386 3.9 e56/23 3:963d31c7:::rbd_data.1118c4b3aebe.0000000000000003:head v 56'89, mlcod=53'85)", 2024-09-14T23:57:44.597 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.287578+0000", 2024-09-14T23:57:44.597 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 38.253782817000001, 2024-09-14T23:57:44.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033580250000000002, 2024-09-14T23:57:44.597 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:44.597 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:44.597 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:44.597 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:44.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:44.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287578+0000", 2024-09-14T23:57:44.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:44.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287578+0000", 2024-09-14T23:57:44.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:44.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287579+0000", 2024-09-14T23:57:44.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.64e-07 2024-09-14T23:57:44.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:44.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287586+0000", 2024-09-14T23:57:44.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5310000000000004e-06 2024-09-14T23:57:44.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:44.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287597+0000", 2024-09-14T23:57:44.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0903999999999999e-05 2024-09-14T23:57:44.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:44.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287605+0000", 2024-09-14T23:57:44.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7889999999999995e-06 2024-09-14T23:57:44.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:44.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.288437+0000", 2024-09-14T23:57:44.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000831914 2024-09-14T23:57:44.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:44.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.288460+0000", 2024-09-14T23:57:44.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3410000000000001e-05 2024-09-14T23:57:44.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:44.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.290883+0000", 2024-09-14T23:57:44.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024230599999999999 2024-09-14T23:57:44.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:44.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.290936+0000", 2024-09-14T23:57:44.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2453000000000001e-05 2024-09-14T23:57:44.601 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.601 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:44.601 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.601 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:60299 3.9 e57/23 3:96f26b6f:::rbd_data.11188d356229.0000000000000009:head v 57'98, mlcod=56'96)", 2024-09-14T23:57:44.601 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:07.418336+0000", 2024-09-14T23:57:44.601 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.123024323999999, 2024-09-14T23:57:44.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022211459999999998, 2024-09-14T23:57:44.601 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:44.602 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:44.602 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:44.602 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:44.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:44.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.418336+0000", 2024-09-14T23:57:44.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:44.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.418336+0000", 2024-09-14T23:57:44.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:44.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.418337+0000", 2024-09-14T23:57:44.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7199999999999999e-07 2024-09-14T23:57:44.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:44.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.419396+0000", 2024-09-14T23:57:44.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010598319999999999 2024-09-14T23:57:44.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:44.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.419407+0000", 2024-09-14T23:57:44.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0956e-05 2024-09-14T23:57:44.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:44.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.419417+0000", 2024-09-14T23:57:44.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0040999999999999e-05 2024-09-14T23:57:44.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:44.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.419468+0000", 2024-09-14T23:57:44.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0497000000000001e-05 2024-09-14T23:57:44.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:44.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.419485+0000", 2024-09-14T23:57:44.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6966e-05 2024-09-14T23:57:44.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:44.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.420527+0000", 2024-09-14T23:57:44.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001042133 2024-09-14T23:57:44.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:44.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.420557+0000", 2024-09-14T23:57:44.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0249000000000001e-05 2024-09-14T23:57:44.606 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.606 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:44.606 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.606 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:64263 3.4 e57/23 3:20c6a768:::rbd_data.11188d356229.0000000000000001:head v 57'165, mlcod=57'163)", 2024-09-14T23:57:44.606 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:07.999031+0000", 2024-09-14T23:57:44.606 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 36.542329668999997, 2024-09-14T23:57:44.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023815210000000002, 2024-09-14T23:57:44.606 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:44.606 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:44.606 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:44.606 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:44.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:44.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999031+0000", 2024-09-14T23:57:44.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:44.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999031+0000", 2024-09-14T23:57:44.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:44.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999032+0000", 2024-09-14T23:57:44.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.176e-06 2024-09-14T23:57:44.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:44.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999660+0000", 2024-09-14T23:57:44.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00062817099999999996 2024-09-14T23:57:44.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:44.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999683+0000", 2024-09-14T23:57:44.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2816999999999999e-05 2024-09-14T23:57:44.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:44.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999693+0000", 2024-09-14T23:57:44.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0088999999999999e-05 2024-09-14T23:57:44.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:44.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999767+0000", 2024-09-14T23:57:44.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4056999999999999e-05 2024-09-14T23:57:44.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:44.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999793+0000", 2024-09-14T23:57:44.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5573000000000001e-05 2024-09-14T23:57:44.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:44.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:08.001379+0000", 2024-09-14T23:57:44.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015859019999999999 2024-09-14T23:57:44.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:44.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:08.001412+0000", 2024-09-14T23:57:44.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3735999999999997e-05 2024-09-14T23:57:44.611 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.611 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:44.611 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.611 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:65427 3.4 e60/23 3:269fb893:::rbd_data.11188d356229.00000000000000d9:head v 60'179, mlcod=58'175)", 2024-09-14T23:57:44.611 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:11.105942+0000", 2024-09-14T23:57:44.611 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 33.435418708999997, 2024-09-14T23:57:44.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001562393, 2024-09-14T23:57:44.612 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:44.612 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:44.612 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:44.612 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:44.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:44.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.105942+0000", 2024-09-14T23:57:44.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:44.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.105942+0000", 2024-09-14T23:57:44.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:44.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.105942+0000", 2024-09-14T23:57:44.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0299999999999998e-07 2024-09-14T23:57:44.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:44.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.105965+0000", 2024-09-14T23:57:44.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2629e-05 2024-09-14T23:57:44.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:44.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.105977+0000", 2024-09-14T23:57:44.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2023e-05 2024-09-14T23:57:44.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:44.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.105986+0000", 2024-09-14T23:57:44.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9670000000000007e-06 2024-09-14T23:57:44.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:44.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.106091+0000", 2024-09-14T23:57:44.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010527900000000001 2024-09-14T23:57:44.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:44.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.106112+0000", 2024-09-14T23:57:44.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0258000000000002e-05 2024-09-14T23:57:44.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:44.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.107460+0000", 2024-09-14T23:57:44.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001348393 2024-09-14T23:57:44.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:44.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.107504+0000", 2024-09-14T23:57:44.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4141000000000003e-05 2024-09-14T23:57:44.619 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.620 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:44.620 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.620 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:66570 3.0 e60/23 3:055350b3:::rbd_data.111873792d9b.0000000000000005:head v 60'162, mlcod=57'159)", 2024-09-14T23:57:44.620 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:11.277517+0000", 2024-09-14T23:57:44.621 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 33.263843512999998, 2024-09-14T23:57:44.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003472199, 2024-09-14T23:57:44.621 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:44.621 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:44.621 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:44.621 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:44.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:44.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.277517+0000", 2024-09-14T23:57:44.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:44.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.277517+0000", 2024-09-14T23:57:44.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:44.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.277518+0000", 2024-09-14T23:57:44.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1379999999999999e-06 2024-09-14T23:57:44.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:44.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278504+0000", 2024-09-14T23:57:44.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00098617499999999994 2024-09-14T23:57:44.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:44.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278525+0000", 2024-09-14T23:57:44.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.09e-05 2024-09-14T23:57:44.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:44.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278534+0000", 2024-09-14T23:57:44.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6209999999999993e-06 2024-09-14T23:57:44.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:44.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278611+0000", 2024-09-14T23:57:44.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6926000000000002e-05 2024-09-14T23:57:44.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:44.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278641+0000", 2024-09-14T23:57:44.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0861999999999997e-05 2024-09-14T23:57:44.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:44.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.280953+0000", 2024-09-14T23:57:44.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023114239999999999 2024-09-14T23:57:44.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:44.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.280989+0000", 2024-09-14T23:57:44.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6152999999999998e-05 2024-09-14T23:57:44.630 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.630 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:44.631 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.631 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:70526 3.4 e64/23 3:2649db36:::rbd_data.111831f32e80.0000000000000058:head v 64'183, mlcod=64'181)", 2024-09-14T23:57:44.631 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:15.442503+0000", 2024-09-14T23:57:44.632 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 29.098857621000001, 2024-09-14T23:57:44.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001769837, 2024-09-14T23:57:44.632 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:44.632 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:44.632 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:44.633 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:44.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:44.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.442503+0000", 2024-09-14T23:57:44.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:44.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.442503+0000", 2024-09-14T23:57:44.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:44.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.442503+0000", 2024-09-14T23:57:44.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1099999999999995e-07 2024-09-14T23:57:44.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:44.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.442620+0000", 2024-09-14T23:57:44.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000116618 2024-09-14T23:57:44.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:44.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.442634+0000", 2024-09-14T23:57:44.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3984e-05 2024-09-14T23:57:44.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:44.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.442641+0000", 2024-09-14T23:57:44.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2540000000000003e-06 2024-09-14T23:57:44.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:44.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.442732+0000", 2024-09-14T23:57:44.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0480000000000001e-05 2024-09-14T23:57:44.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:44.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.442753+0000", 2024-09-14T23:57:44.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1571999999999999e-05 2024-09-14T23:57:44.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:44.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.444236+0000", 2024-09-14T23:57:44.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0014825439999999999 2024-09-14T23:57:44.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:44.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.444273+0000", 2024-09-14T23:57:44.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6674000000000002e-05 2024-09-14T23:57:44.641 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.641 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:44.641 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.642 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:106629 3.4 e70/23 3:240d85c6:::rbd_data.11187de8b0a5.0000000000000009:head v 70'233, mlcod=69'231)", 2024-09-14T23:57:44.642 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:21.489517+0000", 2024-09-14T23:57:44.642 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.051842988000001, 2024-09-14T23:57:44.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016279840000000001, 2024-09-14T23:57:44.642 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:44.642 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:44.642 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:44.642 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:44.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:44.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.489517+0000", 2024-09-14T23:57:44.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:44.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.489517+0000", 2024-09-14T23:57:44.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:44.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.489518+0000", 2024-09-14T23:57:44.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.029e-06 2024-09-14T23:57:44.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:44.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.489840+0000", 2024-09-14T23:57:44.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00032164100000000001 2024-09-14T23:57:44.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:44.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.489859+0000", 2024-09-14T23:57:44.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8828999999999999e-05 2024-09-14T23:57:44.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:44.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.489869+0000", 2024-09-14T23:57:44.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6600000000000007e-06 2024-09-14T23:57:44.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:44.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.489934+0000", 2024-09-14T23:57:44.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5357000000000005e-05 2024-09-14T23:57:44.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:44.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.489951+0000", 2024-09-14T23:57:44.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6951000000000001e-05 2024-09-14T23:57:44.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:44.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.491108+0000", 2024-09-14T23:57:44.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011571940000000001 2024-09-14T23:57:44.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:44.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.491145+0000", 2024-09-14T23:57:44.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7323000000000002e-05 2024-09-14T23:57:44.646 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.646 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:44.647 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.647 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:108739 3.9 e80/23 3:9372b409:::rbd_data.111867e26b91.0000000000000005:head v 80'172, mlcod=78'170)", 2024-09-14T23:57:44.647 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:31.064099+0000", 2024-09-14T23:57:44.647 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.477261735000001, 2024-09-14T23:57:44.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001688174, 2024-09-14T23:57:44.647 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:44.647 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:44.647 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:44.647 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:44.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.648 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:44.648 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:31.064099+0000", 2024-09-14T23:57:44.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.648 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:44.648 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:31.064099+0000", 2024-09-14T23:57:44.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.648 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:44.648 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:31.064099+0000", 2024-09-14T23:57:44.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.44e-07 2024-09-14T23:57:44.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:44.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:31.064453+0000", 2024-09-14T23:57:44.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00035374899999999999 2024-09-14T23:57:44.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:44.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:31.064468+0000", 2024-09-14T23:57:44.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4476e-05 2024-09-14T23:57:44.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:44.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:31.064475+0000", 2024-09-14T23:57:44.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9289999999999998e-06 2024-09-14T23:57:44.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.650 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:44.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:31.064530+0000", 2024-09-14T23:57:44.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4401000000000003e-05 2024-09-14T23:57:44.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.650 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:44.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:31.064551+0000", 2024-09-14T23:57:44.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0996e-05 2024-09-14T23:57:44.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.651 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:44.651 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:31.065748+0000", 2024-09-14T23:57:44.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011967449999999999 2024-09-14T23:57:44.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.651 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:44.651 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:31.065787+0000", 2024-09-14T23:57:44.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9233999999999997e-05 2024-09-14T23:57:44.651 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.651 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:44.651 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.652 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "pg_update_log_missing(3.9 epoch 84/23 rep_tid 2234 entries 84'200 (0'0) error 3:93b58b1d:::rbd_data.1118417a1b0b.000000000000001b:head by client.4376.0:118637 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 83'197)", 2024-09-14T23:57:44.652 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:35.103156+0000", 2024-09-14T23:57:44.652 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.4382042399999992, 2024-09-14T23:57:44.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019852009999999998, 2024-09-14T23:57:44.652 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:44.652 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:44.652 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:44.652 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:44.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.652 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:44.652 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:35.103156+0000", 2024-09-14T23:57:44.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.653 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:44.653 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:35.103156+0000", 2024-09-14T23:57:44.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:44.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.653 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:44.653 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:35.103157+0000", 2024-09-14T23:57:44.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1969999999999999e-06 2024-09-14T23:57:44.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.653 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:44.653 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:35.103170+0000", 2024-09-14T23:57:44.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2921e-05 2024-09-14T23:57:44.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.654 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:44.654 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:35.103202+0000", 2024-09-14T23:57:44.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1992000000000001e-05 2024-09-14T23:57:44.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.654 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:44.654 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:35.103220+0000", 2024-09-14T23:57:44.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7312999999999998e-05 2024-09-14T23:57:44.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.654 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:44.654 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:35.103305+0000", 2024-09-14T23:57:44.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4975000000000005e-05 2024-09-14T23:57:44.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:44.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:44.655 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:44.655 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:35.105141+0000", 2024-09-14T23:57:44.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018368029999999999 2024-09-14T23:57:44.655 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.655 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:44.655 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.655 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:44.655 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:44.655 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:44.657 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T23:57:44.657 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:44.655+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:44.758 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:44.756+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:44.766 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:44.766 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:57:44.766 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:57:44.766 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:44.774 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T23:57:44.859 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:44.857+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:44.922 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:44.922 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:57:44.922 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:57:44.922 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:57:44.923 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:44.931 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T23:57:44.973 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:44.971+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:45.057 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T23:57:45.061 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:45.059+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:45.085 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:45.085 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:57:45.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:57:45.085 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:57:45.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.085 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5180 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3883008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:45.085 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.250058+0000", 2024-09-14T23:57:45.086 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.832569286999998, 2024-09-14T23:57:45.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057351471000000001, 2024-09-14T23:57:45.086 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:45.086 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:45.086 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:45.086 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:45.086 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:45.086 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:45.086 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5180 2024-09-14T23:57:45.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.086 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:45.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:45.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250058+0000", 2024-09-14T23:57:45.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:45.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:45.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250058+0000", 2024-09-14T23:57:45.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:45.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:45.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250786+0000", 2024-09-14T23:57:45.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00072813399999999996 2024-09-14T23:57:45.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:45.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250789+0000", 2024-09-14T23:57:45.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.647e-06 2024-09-14T23:57:45.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:45.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250798+0000", 2024-09-14T23:57:45.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0000000000000002e-06 2024-09-14T23:57:45.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:45.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250803+0000", 2024-09-14T23:57:45.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.301e-06 2024-09-14T23:57:45.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:45.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.307234+0000", 2024-09-14T23:57:45.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056431165999999998 2024-09-14T23:57:45.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:45.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.307321+0000", 2024-09-14T23:57:45.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6831999999999998e-05 2024-09-14T23:57:45.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:45.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.307409+0000", 2024-09-14T23:57:45.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8391000000000001e-05 2024-09-14T23:57:45.091 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:45.091 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:45.091 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:45.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.092 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5184 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 4128768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:45.092 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.250851+0000", 2024-09-14T23:57:45.092 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.831775816000004, 2024-09-14T23:57:45.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057319038000000003, 2024-09-14T23:57:45.092 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:45.092 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:45.093 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:45.093 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:45.093 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:45.093 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:45.093 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5184 2024-09-14T23:57:45.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.093 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:45.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:45.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250851+0000", 2024-09-14T23:57:45.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:45.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:45.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250851+0000", 2024-09-14T23:57:45.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:45.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:45.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.251900+0000", 2024-09-14T23:57:45.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001048577 2024-09-14T23:57:45.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:45.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.251904+0000", 2024-09-14T23:57:45.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9580000000000001e-06 2024-09-14T23:57:45.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:45.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.251914+0000", 2024-09-14T23:57:45.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0186999999999999e-05 2024-09-14T23:57:45.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:45.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.251927+0000", 2024-09-14T23:57:45.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3593e-05 2024-09-14T23:57:45.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:45.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.308020+0000", 2024-09-14T23:57:45.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056092867999999997 2024-09-14T23:57:45.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:45.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.308094+0000", 2024-09-14T23:57:45.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3533000000000002e-05 2024-09-14T23:57:45.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:45.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.308170+0000", 2024-09-14T23:57:45.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6322000000000006e-05 2024-09-14T23:57:45.099 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:45.099 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:45.099 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:45.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.099 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5196 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 1925120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:45.099 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.252157+0000", 2024-09-14T23:57:45.100 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.830469702000002, 2024-09-14T23:57:45.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057285801999999997, 2024-09-14T23:57:45.100 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:45.100 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:45.100 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:45.100 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:45.100 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:45.100 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:45.101 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5196 2024-09-14T23:57:45.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.101 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:45.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:45.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.252157+0000", 2024-09-14T23:57:45.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:45.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:45.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.252157+0000", 2024-09-14T23:57:45.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:45.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:45.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253243+0000", 2024-09-14T23:57:45.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010861340000000001 2024-09-14T23:57:45.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:45.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253248+0000", 2024-09-14T23:57:45.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8799999999999999e-06 2024-09-14T23:57:45.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:45.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253271+0000", 2024-09-14T23:57:45.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2836e-05 2024-09-14T23:57:45.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:45.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253281+0000", 2024-09-14T23:57:45.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6520000000000001e-06 2024-09-14T23:57:45.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:45.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.309336+0000", 2024-09-14T23:57:45.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056054983000000003 2024-09-14T23:57:45.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:45.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.309389+0000", 2024-09-14T23:57:45.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3298999999999999e-05 2024-09-14T23:57:45.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:45.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.309443+0000", 2024-09-14T23:57:45.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4017999999999997e-05 2024-09-14T23:57:45.106 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:45.106 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:45.106 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:45.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.106 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5200 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2170880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:45.106 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253394+0000", 2024-09-14T23:57:45.106 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.829232924999999, 2024-09-14T23:57:45.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057428154000000002, 2024-09-14T23:57:45.106 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:45.106 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:45.106 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:45.107 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:45.107 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:45.107 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:45.107 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5200 2024-09-14T23:57:45.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.107 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:45.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:45.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253394+0000", 2024-09-14T23:57:45.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:45.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:45.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253394+0000", 2024-09-14T23:57:45.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:45.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:45.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253395+0000", 2024-09-14T23:57:45.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9100000000000004e-07 2024-09-14T23:57:45.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:45.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253397+0000", 2024-09-14T23:57:45.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2790000000000001e-06 2024-09-14T23:57:45.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:45.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253404+0000", 2024-09-14T23:57:45.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7959999999999998e-06 2024-09-14T23:57:45.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:45.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253409+0000", 2024-09-14T23:57:45.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1460000000000004e-06 2024-09-14T23:57:45.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:45.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310627+0000", 2024-09-14T23:57:45.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057218160999999997 2024-09-14T23:57:45.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:45.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310710+0000", 2024-09-14T23:57:45.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3102999999999994e-05 2024-09-14T23:57:45.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:45.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310822+0000", 2024-09-14T23:57:45.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000112178 2024-09-14T23:57:45.110 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:45.110 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:45.110 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:45.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.110 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5201 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2232320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:45.111 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:57:45.111 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.829208253000004, 2024-09-14T23:57:45.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057643228999999997, 2024-09-14T23:57:45.111 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:45.111 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:45.111 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:45.111 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:45.111 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:45.111 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:45.111 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5201 2024-09-14T23:57:45.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.112 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:45.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:45.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:57:45.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:45.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:45.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:57:45.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:45.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:45.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:57:45.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0699999999999998e-07 2024-09-14T23:57:45.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:45.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253422+0000", 2024-09-14T23:57:45.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.514e-06 2024-09-14T23:57:45.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:45.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253431+0000", 2024-09-14T23:57:45.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1919999999999994e-06 2024-09-14T23:57:45.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:45.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253436+0000", 2024-09-14T23:57:45.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5350000000000004e-06 2024-09-14T23:57:45.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:45.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310856+0000", 2024-09-14T23:57:45.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057419381999999998 2024-09-14T23:57:45.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:45.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310953+0000", 2024-09-14T23:57:45.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6774000000000006e-05 2024-09-14T23:57:45.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:45.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311062+0000", 2024-09-14T23:57:45.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000109425 2024-09-14T23:57:45.115 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:45.115 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:45.115 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:45.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.115 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5202 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2293760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:45.115 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:57:45.115 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.829183505000003, 2024-09-14T23:57:45.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057818255999999998, 2024-09-14T23:57:45.115 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:45.115 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:45.116 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:45.116 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:45.116 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:45.116 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:45.116 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5202 2024-09-14T23:57:45.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.116 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:45.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:45.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:57:45.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:45.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:45.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:57:45.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:45.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:45.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253573+0000", 2024-09-14T23:57:45.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012934000000000001 2024-09-14T23:57:45.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:45.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253576+0000", 2024-09-14T23:57:45.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5310000000000002e-06 2024-09-14T23:57:45.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:45.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253587+0000", 2024-09-14T23:57:45.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0522e-05 2024-09-14T23:57:45.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:45.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253595+0000", 2024-09-14T23:57:45.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8290000000000008e-06 2024-09-14T23:57:45.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:45.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311093+0000", 2024-09-14T23:57:45.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057498312000000003 2024-09-14T23:57:45.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:45.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311167+0000", 2024-09-14T23:57:45.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3582000000000001e-05 2024-09-14T23:57:45.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:45.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311262+0000", 2024-09-14T23:57:45.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5140000000000001e-05 2024-09-14T23:57:45.120 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:45.120 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:45.120 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:45.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.120 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5203 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2355200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:45.120 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:57:45.121 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.829020763000003, 2024-09-14T23:57:45.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057903938000000002, 2024-09-14T23:57:45.121 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:45.121 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:45.121 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:45.121 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:45.121 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:45.121 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:45.121 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5203 2024-09-14T23:57:45.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.121 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:45.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:45.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:57:45.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:45.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:45.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:57:45.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:45.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:45.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253902+0000", 2024-09-14T23:57:45.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029615400000000002 2024-09-14T23:57:45.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:45.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253906+0000", 2024-09-14T23:57:45.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9249999999999997e-06 2024-09-14T23:57:45.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:45.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253913+0000", 2024-09-14T23:57:45.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9099999999999999e-06 2024-09-14T23:57:45.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:45.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253917+0000", 2024-09-14T23:57:45.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9210000000000002e-06 2024-09-14T23:57:45.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:45.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311295+0000", 2024-09-14T23:57:45.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057378268000000003 2024-09-14T23:57:45.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:45.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311402+0000", 2024-09-14T23:57:45.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010644100000000001 2024-09-14T23:57:45.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:45.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311510+0000", 2024-09-14T23:57:45.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108319 2024-09-14T23:57:45.125 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:45.125 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:45.125 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:45.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.125 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5204 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2416640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:45.125 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:57:45.125 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.828702566000004, 2024-09-14T23:57:45.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057774860999999997, 2024-09-14T23:57:45.125 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:45.125 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:45.125 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:45.125 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:45.125 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:45.126 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:45.126 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5204 2024-09-14T23:57:45.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.126 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:45.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:45.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:57:45.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:45.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:45.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:57:45.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:45.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:45.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255903+0000", 2024-09-14T23:57:45.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001978807 2024-09-14T23:57:45.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:45.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255907+0000", 2024-09-14T23:57:45.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3610000000000001e-06 2024-09-14T23:57:45.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:45.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255914+0000", 2024-09-14T23:57:45.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.357e-06 2024-09-14T23:57:45.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:45.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255918+0000", 2024-09-14T23:57:45.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1779999999999996e-06 2024-09-14T23:57:45.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:45.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311541+0000", 2024-09-14T23:57:45.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055622697999999998 2024-09-14T23:57:45.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:45.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311614+0000", 2024-09-14T23:57:45.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3102999999999994e-05 2024-09-14T23:57:45.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:45.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311699+0000", 2024-09-14T23:57:45.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5357000000000003e-05 2024-09-14T23:57:45.129 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:45.129 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:45.130 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:45.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.130 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5211 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2846720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:45.130 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:57:45.130 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.826604352000004, 2024-09-14T23:57:45.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057528151, 2024-09-14T23:57:45.130 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:45.130 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:45.130 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:45.130 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:45.130 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:45.130 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:45.130 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5211 2024-09-14T23:57:45.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.131 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:45.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:45.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:57:45.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:45.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:45.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:57:45.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:45.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:45.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:57:45.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1399999999999998e-07 2024-09-14T23:57:45.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:45.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256024+0000", 2024-09-14T23:57:45.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.114e-06 2024-09-14T23:57:45.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:45.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256028+0000", 2024-09-14T23:57:45.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0840000000000002e-06 2024-09-14T23:57:45.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:45.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256031+0000", 2024-09-14T23:57:45.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6819999999999999e-06 2024-09-14T23:57:45.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:45.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313317+0000", 2024-09-14T23:57:45.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057286295000000001 2024-09-14T23:57:45.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:45.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313403+0000", 2024-09-14T23:57:45.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6287999999999998e-05 2024-09-14T23:57:45.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:45.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313551+0000", 2024-09-14T23:57:45.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000147374 2024-09-14T23:57:45.134 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:45.134 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:45.134 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:45.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.134 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5212 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2908160~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:45.134 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:57:45.134 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.826593157999994, 2024-09-14T23:57:45.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057735679999999998, 2024-09-14T23:57:45.134 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:45.134 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:45.135 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:45.135 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:45.135 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:45.135 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:45.135 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5212 2024-09-14T23:57:45.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.135 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:45.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:45.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:57:45.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:45.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:45.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:57:45.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:45.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:45.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:57:45.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0900000000000001e-07 2024-09-14T23:57:45.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:45.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256035+0000", 2024-09-14T23:57:45.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1200000000000001e-06 2024-09-14T23:57:45.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:45.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256038+0000", 2024-09-14T23:57:45.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.05e-06 2024-09-14T23:57:45.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:45.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256041+0000", 2024-09-14T23:57:45.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.667e-06 2024-09-14T23:57:45.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:45.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313584+0000", 2024-09-14T23:57:45.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057543484999999998 2024-09-14T23:57:45.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:45.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313668+0000", 2024-09-14T23:57:45.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3807000000000006e-05 2024-09-14T23:57:45.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:45.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313770+0000", 2024-09-14T23:57:45.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010134200000000001 2024-09-14T23:57:45.138 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:45.138 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:45.138 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:45.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.139 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5213 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2969600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:45.139 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:57:45.139 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.826583071000002, 2024-09-14T23:57:45.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058145467999999999, 2024-09-14T23:57:45.139 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:45.139 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:45.139 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:45.139 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:45.139 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:45.139 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:45.140 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5213 2024-09-14T23:57:45.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.140 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:45.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:45.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:57:45.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:45.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:45.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:57:45.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:45.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:45.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257901+0000", 2024-09-14T23:57:45.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018567519999999999 2024-09-14T23:57:45.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:45.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257904+0000", 2024-09-14T23:57:45.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4850000000000001e-06 2024-09-14T23:57:45.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:45.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257912+0000", 2024-09-14T23:57:45.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2090000000000005e-06 2024-09-14T23:57:45.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:45.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257918+0000", 2024-09-14T23:57:45.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6899999999999997e-06 2024-09-14T23:57:45.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:45.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313816+0000", 2024-09-14T23:57:45.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055898075999999998 2024-09-14T23:57:45.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:45.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.314082+0000", 2024-09-14T23:57:45.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000265386 2024-09-14T23:57:45.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:45.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.314189+0000", 2024-09-14T23:57:45.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010787 2024-09-14T23:57:45.144 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:45.144 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:45.144 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:45.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.145 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5217 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3215360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:45.145 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257961+0000", 2024-09-14T23:57:45.145 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.824666092000001, 2024-09-14T23:57:45.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057320969999999999, 2024-09-14T23:57:45.145 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:45.145 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:45.145 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:45.145 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:45.145 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:45.145 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:45.145 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5217 2024-09-14T23:57:45.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.145 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:45.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:45.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257961+0000", 2024-09-14T23:57:45.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:45.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:45.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257961+0000", 2024-09-14T23:57:45.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:45.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:45.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257961+0000", 2024-09-14T23:57:45.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.22e-07 2024-09-14T23:57:45.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:45.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257962+0000", 2024-09-14T23:57:45.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1400000000000001e-06 2024-09-14T23:57:45.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:45.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257965+0000", 2024-09-14T23:57:45.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0929999999999999e-06 2024-09-14T23:57:45.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:45.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257968+0000", 2024-09-14T23:57:45.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5749999999999999e-06 2024-09-14T23:57:45.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:45.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315047+0000", 2024-09-14T23:57:45.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057078730000000001 2024-09-14T23:57:45.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:45.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315087+0000", 2024-09-14T23:57:45.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0806999999999997e-05 2024-09-14T23:57:45.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:45.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315282+0000", 2024-09-14T23:57:45.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000194403 2024-09-14T23:57:45.149 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:45.149 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:45.149 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:45.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.149 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5218 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3276800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:45.149 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:57:45.149 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.824656062000003, 2024-09-14T23:57:45.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057497557999999997, 2024-09-14T23:57:45.149 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:45.149 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:45.149 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:45.150 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:45.150 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:45.150 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:45.150 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5218 2024-09-14T23:57:45.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.150 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:45.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:45.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:57:45.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:45.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:45.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:57:45.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:45.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:45.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:57:45.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1899999999999999e-07 2024-09-14T23:57:45.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:45.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257972+0000", 2024-09-14T23:57:45.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.217e-06 2024-09-14T23:57:45.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:45.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257975+0000", 2024-09-14T23:57:45.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9359999999999999e-06 2024-09-14T23:57:45.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.152 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:45.152 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257978+0000", 2024-09-14T23:57:45.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1049999999999999e-06 2024-09-14T23:57:45.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.152 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:45.152 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315331+0000", 2024-09-14T23:57:45.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057352093999999999 2024-09-14T23:57:45.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:45.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315384+0000", 2024-09-14T23:57:45.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3780999999999999e-05 2024-09-14T23:57:45.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:45.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315468+0000", 2024-09-14T23:57:45.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4206e-05 2024-09-14T23:57:45.153 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:45.153 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:45.153 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:45.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.154 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5219 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3338240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:45.154 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:57:45.154 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.824645386, 2024-09-14T23:57:45.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057637764000000001, 2024-09-14T23:57:45.154 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:45.154 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:45.154 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:45.154 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:45.154 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:45.154 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:45.154 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5219 2024-09-14T23:57:45.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.154 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:45.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:45.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:57:45.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:45.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:45.156 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:57:45.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:45.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.156 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:45.156 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:57:45.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.03e-07 2024-09-14T23:57:45.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.156 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:45.156 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257983+0000", 2024-09-14T23:57:45.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9199999999999999e-07 2024-09-14T23:57:45.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.157 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:45.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257986+0000", 2024-09-14T23:57:45.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0860000000000002e-06 2024-09-14T23:57:45.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.157 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:45.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257989+0000", 2024-09-14T23:57:45.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6589999999999999e-06 2024-09-14T23:57:45.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.157 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:45.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315509+0000", 2024-09-14T23:57:45.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057520262000000003 2024-09-14T23:57:45.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:45.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315554+0000", 2024-09-14T23:57:45.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5015999999999997e-05 2024-09-14T23:57:45.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:45.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315619+0000", 2024-09-14T23:57:45.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5545999999999999e-05 2024-09-14T23:57:45.158 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:45.158 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:45.158 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:45.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.159 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5220 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3399680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:45.159 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:57:45.159 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.824635430000001, 2024-09-14T23:57:45.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057792128999999998, 2024-09-14T23:57:45.159 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:45.159 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:45.159 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:45.159 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:45.159 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:45.159 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:45.159 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5220 2024-09-14T23:57:45.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.159 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:45.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:45.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:57:45.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:45.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:45.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:57:45.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:45.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:45.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:57:45.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1799999999999999e-07 2024-09-14T23:57:45.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.161 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:45.161 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257993+0000", 2024-09-14T23:57:45.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6200000000000006e-07 2024-09-14T23:57:45.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.161 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:45.161 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257996+0000", 2024-09-14T23:57:45.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9050000000000001e-06 2024-09-14T23:57:45.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.161 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:45.161 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257998+0000", 2024-09-14T23:57:45.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5950000000000001e-06 2024-09-14T23:57:45.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:45.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315672+0000", 2024-09-14T23:57:45.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057673637999999999 2024-09-14T23:57:45.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:45.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315734+0000", 2024-09-14T23:57:45.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2216000000000002e-05 2024-09-14T23:57:45.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:45.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315784+0000", 2024-09-14T23:57:45.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9595000000000002e-05 2024-09-14T23:57:45.163 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:45.163 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:45.163 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:45.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.163 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5221 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3461120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:45.163 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:57:45.163 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.824625712, 2024-09-14T23:57:45.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058003557999999997, 2024-09-14T23:57:45.163 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:45.163 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:45.163 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:45.163 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:45.163 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:45.164 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:45.164 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5221 2024-09-14T23:57:45.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.164 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:45.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.164 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:45.164 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:57:45.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:45.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.164 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:45.164 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:57:45.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:45.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.165 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.165 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:45.165 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259905+0000", 2024-09-14T23:57:45.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019035759999999999 2024-09-14T23:57:45.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.165 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.165 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:45.165 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259915+0000", 2024-09-14T23:57:45.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0518e-05 2024-09-14T23:57:45.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.165 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.165 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:45.165 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259925+0000", 2024-09-14T23:57:45.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7180000000000008e-06 2024-09-14T23:57:45.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:45.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259930+0000", 2024-09-14T23:57:45.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4769999999999997e-06 2024-09-14T23:57:45.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:45.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315820+0000", 2024-09-14T23:57:45.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055889960000000002 2024-09-14T23:57:45.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:45.167 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315914+0000", 2024-09-14T23:57:45.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4625999999999998e-05 2024-09-14T23:57:45.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:45.167 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.316005+0000", 2024-09-14T23:57:45.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0682999999999999e-05 2024-09-14T23:57:45.167 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:45.167 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:45.167 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:45.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.168 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5230 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 4014080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:45.168 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.260060+0000", 2024-09-14T23:57:45.168 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.822566874000003, 2024-09-14T23:57:45.168 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057353556999999999, 2024-09-14T23:57:45.168 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:45.168 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:45.168 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:45.168 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:45.168 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:45.168 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:45.169 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5230 2024-09-14T23:57:45.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.169 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:45.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.169 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:45.169 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260060+0000", 2024-09-14T23:57:45.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:45.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.169 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:45.169 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260060+0000", 2024-09-14T23:57:45.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:45.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.170 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:45.170 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260060+0000", 2024-09-14T23:57:45.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.17e-07 2024-09-14T23:57:45.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.170 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.170 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:45.170 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260061+0000", 2024-09-14T23:57:45.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0300000000000001e-06 2024-09-14T23:57:45.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.170 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.170 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:45.170 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260064+0000", 2024-09-14T23:57:45.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9119999999999998e-06 2024-09-14T23:57:45.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:45.171 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260067+0000", 2024-09-14T23:57:45.171 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.678e-06 2024-09-14T23:57:45.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:45.171 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317269+0000", 2024-09-14T23:57:45.171 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057201771999999998 2024-09-14T23:57:45.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:45.171 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317342+0000", 2024-09-14T23:57:45.171 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2842000000000005e-05 2024-09-14T23:57:45.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.172 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:45.172 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317414+0000", 2024-09-14T23:57:45.172 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2106000000000004e-05 2024-09-14T23:57:45.172 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:45.172 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:45.172 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:45.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.172 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5231 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 4075520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:45.172 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:57:45.172 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.822556977999994, 2024-09-14T23:57:45.172 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057524733000000002, 2024-09-14T23:57:45.172 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:45.173 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:45.173 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:45.173 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:45.173 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:45.173 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:45.173 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5231 2024-09-14T23:57:45.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.173 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:45.173 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.173 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:45.173 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:57:45.173 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:45.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.174 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.174 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:45.174 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:57:45.174 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:45.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.174 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.174 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:45.174 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261904+0000", 2024-09-14T23:57:45.174 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018338790000000001 2024-09-14T23:57:45.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.174 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.174 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:45.174 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261910+0000", 2024-09-14T23:57:45.174 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9689999999999999e-06 2024-09-14T23:57:45.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.175 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:45.175 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261923+0000", 2024-09-14T23:57:45.175 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3061000000000001e-05 2024-09-14T23:57:45.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.175 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:45.175 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261931+0000", 2024-09-14T23:57:45.175 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3159999999999997e-06 2024-09-14T23:57:45.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.176 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:45.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317456+0000", 2024-09-14T23:57:45.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055525219000000001 2024-09-14T23:57:45.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.176 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:45.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317520+0000", 2024-09-14T23:57:45.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4010000000000005e-05 2024-09-14T23:57:45.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.177 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:45.177 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317595+0000", 2024-09-14T23:57:45.177 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4278999999999998e-05 2024-09-14T23:57:45.177 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:45.177 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:45.177 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:45.177 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.177 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.177 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5236 3.8 3:1071668d:::rbd_data.1118de1168b.0000000000000092:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:45.177 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:57:45.177 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.820543513999993, 2024-09-14T23:57:45.177 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057499462000000001, 2024-09-14T23:57:45.177 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:45.177 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:45.178 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:45.178 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:45.178 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:45.178 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:45.178 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5236 2024-09-14T23:57:45.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.178 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:45.178 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.178 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:45.178 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:57:45.178 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:45.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.178 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.179 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:45.179 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:57:45.179 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:45.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.179 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.179 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:45.179 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262084+0000", 2024-09-14T23:57:45.179 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3399999999999999e-07 2024-09-14T23:57:45.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.179 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.179 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:45.179 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262087+0000", 2024-09-14T23:57:45.179 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2310000000000001e-06 2024-09-14T23:57:45.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:45.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262095+0000", 2024-09-14T23:57:45.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3799999999999996e-06 2024-09-14T23:57:45.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:45.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262101+0000", 2024-09-14T23:57:45.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0009999999999998e-06 2024-09-14T23:57:45.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:45.181 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317625+0000", 2024-09-14T23:57:45.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055524111000000001 2024-09-14T23:57:45.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.181 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.181 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:45.181 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317702+0000", 2024-09-14T23:57:45.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7533999999999993e-05 2024-09-14T23:57:45.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.181 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.181 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-14T23:57:45.181 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317892+0000", 2024-09-14T23:57:45.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018991400000000001 2024-09-14T23:57:45.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.182 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:57:45.182 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319407+0000", 2024-09-14T23:57:45.182 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015145250000000001 2024-09-14T23:57:45.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.182 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:45.182 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319481+0000", 2024-09-14T23:57:45.182 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3826999999999996e-05 2024-09-14T23:57:45.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.182 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:45.183 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319541+0000", 2024-09-14T23:57:45.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0470999999999998e-05 2024-09-14T23:57:45.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:45.183 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319583+0000", 2024-09-14T23:57:45.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1934000000000002e-05 2024-09-14T23:57:45.183 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:45.183 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:45.183 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:45.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.183 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:132622 3.1e 3.7a50b9e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-14T23:57:45.183 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:43.371285+0000", 2024-09-14T23:57:45.183 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 1.711341612, 2024-09-14T23:57:45.184 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.67037987200000004, 2024-09-14T23:57:45.184 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:45.184 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:45.184 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:45.184 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:45.184 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:45.184 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:45.184 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 132622 2024-09-14T23:57:45.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.184 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:45.184 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.184 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:45.184 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371285+0000", 2024-09-14T23:57:45.184 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:45.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.185 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:45.185 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371285+0000", 2024-09-14T23:57:45.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:45.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.185 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:45.185 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371286+0000", 2024-09-14T23:57:45.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6400000000000002e-07 2024-09-14T23:57:45.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.185 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:45.186 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371289+0000", 2024-09-14T23:57:45.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2059999999999999e-06 2024-09-14T23:57:45.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.186 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:45.186 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371299+0000", 2024-09-14T23:57:45.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0288e-05 2024-09-14T23:57:45.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.186 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:45.186 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371306+0000", 2024-09-14T23:57:45.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1650000000000003e-06 2024-09-14T23:57:45.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:45.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371617+0000", 2024-09-14T23:57:45.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000311043 2024-09-14T23:57:45.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:57:45.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371625+0000", 2024-09-14T23:57:45.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.833e-06 2024-09-14T23:57:45.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:45.188 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.041651+0000", 2024-09-14T23:57:45.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.67002580199999995 2024-09-14T23:57:45.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:45.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:45.188 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:45.188 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.041665+0000", 2024-09-14T23:57:45.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3971000000000001e-05 2024-09-14T23:57:45.188 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:45.188 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:45.188 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:45.188 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:45.188 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:45.188 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:45.190 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T23:57:45.191 INFO:teuthology.orchestra.run.smithi042.stdout:3930 punch from 0x5f30f2b to 0x5f3260d, (0x16e2 bytes) 2024-09-14T23:57:45.191 INFO:teuthology.orchestra.run.smithi042.stdout:3932 write 0x698e49d thru 0x6994608 (0x616c bytes) 2024-09-14T23:57:45.191 INFO:teuthology.orchestra.run.smithi042.stdout:3934 read 0x13b5f10 thru 0x13b8d2d (0x2e1e bytes) 2024-09-14T23:57:45.191 INFO:teuthology.orchestra.run.smithi042.stdout:3935 writesame 0x1573272 thru 0x15771e7 data_size 0x1(0x3f76 bytes) 2024-09-14T23:57:45.191 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:45.188+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:45.202 INFO:teuthology.orchestra.run.smithi042.stdout:3936 write 0x9bba9e3 thru 0x9bbedde (0x43fc bytes) 2024-09-14T23:57:45.213 INFO:teuthology.orchestra.run.smithi042.stdout:3937 read 0x5af33e thru 0x5b2b39 (0x37fc bytes) 2024-09-14T23:57:45.214 INFO:teuthology.orchestra.run.smithi042.stdout:3938 read 0x6f38643 thru 0x6f40dd2 (0x8790 bytes) 2024-09-14T23:57:45.224 INFO:teuthology.orchestra.run.smithi042.stdout:3939 writesame 0xd4b8fd0 thru 0xd4c2124 data_size 0x7(0x9155 bytes) 2024-09-14T23:57:45.224 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:45.226 INFO:teuthology.orchestra.run.smithi042.stdout:3941 trunc from 0xd4c2125 to 0xb06d3ca 2024-09-14T23:57:45.243 INFO:teuthology.orchestra.run.smithi042.stdout:3942 read 0x70eb8a7 thru 0x70ee740 (0x2e9a bytes) 2024-09-14T23:57:45.243 INFO:teuthology.orchestra.run.smithi042.stdout:3943 read 0x89a536b thru 0x89a730e (0x1fa4 bytes) 2024-09-14T23:57:45.243 INFO:teuthology.orchestra.run.smithi042.stdout:3945 read 0x769968 thru 0x76d29a (0x3933 bytes) 2024-09-14T23:57:45.244 INFO:teuthology.orchestra.run.smithi042.stdout:3946 writesame 0x52eb32e thru 0x52ef48c data_size 0x1(0x415f bytes) 2024-09-14T23:57:45.255 INFO:teuthology.orchestra.run.smithi042.stdout:3947 read 0x3b3e891 thru 0x3b43859 (0x4fc9 bytes) 2024-09-14T23:57:45.266 INFO:teuthology.orchestra.run.smithi042.stdout:3949 trunc from 0xb06d3ca to 0x2232c49 2024-09-14T23:57:45.291 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:45.289+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:45.317 INFO:teuthology.orchestra.run.smithi042.stdout:3951 writesame 0x7d24c5a thru 0x7d29152 data_size 0x1(0x44f9 bytes) 2024-09-14T23:57:45.318 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:45.318 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:45.319 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:57:45.319 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:45.326 INFO:teuthology.orchestra.run.smithi042.stdout:3952 write 0x9c43a1d thru 0x9c49d20 (0x6304 bytes) 2024-09-14T23:57:45.328 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T23:57:45.331 INFO:teuthology.orchestra.run.smithi042.stdout:3954 trunc from 0x9c49d21 to 0xae7a3ba 2024-09-14T23:57:45.337 INFO:teuthology.orchestra.run.smithi042.stdout:3955 punch from 0xa58f5e9 to 0xa59d54f, (0xdf66 bytes) 2024-09-14T23:57:45.337 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:45.337 INFO:teuthology.orchestra.run.smithi042.stdout:3957 write 0x3d0cd1b thru 0x3d18222 (0xb508 bytes) 2024-09-14T23:57:45.346 INFO:teuthology.orchestra.run.smithi042.stdout:3958 trunc from 0xae7a3ba to 0x530a7d1 2024-09-14T23:57:45.359 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:45.359 INFO:teuthology.orchestra.run.smithi042.stdout:3961 read 0x106675c thru 0x106efb7 (0x885c bytes) 2024-09-14T23:57:45.363 INFO:teuthology.orchestra.run.smithi042.stdout:3965 trunc from 0x530a7d1 to 0x86c8134 2024-09-14T23:57:45.369 INFO:teuthology.orchestra.run.smithi042.stdout:3966 write 0x821366 thru 0x8284a2 (0x713d bytes) 2024-09-14T23:57:45.383 INFO:teuthology.orchestra.run.smithi042.stdout:3967 write 0xd57a169 thru 0xd582fb6 (0x8e4e bytes) 2024-09-14T23:57:45.388 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:45.388 INFO:teuthology.orchestra.run.smithi042.stdout:3969 read 0xad2eb2 thru 0xad661a (0x3769 bytes) 2024-09-14T23:57:45.388 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:45.390 INFO:teuthology.orchestra.run.smithi042.stdout:3971 trunc from 0xd582fb7 to 0xbba30e6 2024-09-14T23:57:45.391 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:45.389+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:45.401 INFO:teuthology.orchestra.run.smithi042.stdout:3973 read 0x7e172fc thru 0x7e1ce87 (0x5b8c bytes) 2024-09-14T23:57:45.401 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:45.401 INFO:teuthology.orchestra.run.smithi042.stdout:3975 punch from 0xa03a37 to 0xa05e58, (0x2421 bytes) 2024-09-14T23:57:45.401 INFO:teuthology.orchestra.run.smithi042.stdout:3978 write 0x9ea2537 thru 0x9ea4a64 (0x252e bytes) 2024-09-14T23:57:45.404 INFO:teuthology.orchestra.run.smithi042.stdout:3979 trunc from 0xbba30e6 to 0xc4109df 2024-09-14T23:57:45.410 INFO:teuthology.orchestra.run.smithi042.stdout:3980 write 0x2140cba thru 0x21505b2 (0xf8f9 bytes) 2024-09-14T23:57:45.412 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:45.413 INFO:teuthology.orchestra.run.smithi042.stdout:3982 write 0x94c8c3b thru 0x94cdba6 (0x4f6c bytes) 2024-09-14T23:57:45.414 INFO:teuthology.orchestra.run.smithi042.stdout:3983 punch from 0x1ccd620 to 0x1cdb905, (0xe2e5 bytes) 2024-09-14T23:57:45.414 INFO:teuthology.orchestra.run.smithi042.stdout:3984 write 0x1d93048 thru 0x1d9de4f (0xae08 bytes) 2024-09-14T23:57:45.416 INFO:teuthology.orchestra.run.smithi042.stdout:3985 write 0x3b47853 thru 0x3b52f46 (0xb6f4 bytes) 2024-09-14T23:57:45.417 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:45.423 INFO:teuthology.orchestra.run.smithi042.stdout:3987 writesame 0xca28efb thru 0xca333bd data_size 0x1(0xa4c3 bytes) 2024-09-14T23:57:45.423 INFO:teuthology.orchestra.run.smithi042.stdout:3990 write 0x8b942b1 thru 0x8b9e420 (0xa170 bytes) 2024-09-14T23:57:45.426 INFO:teuthology.orchestra.run.smithi042.stdout:3991 punch from 0x4c91066 to 0x4c9e819, (0xd7b3 bytes) 2024-09-14T23:57:45.426 INFO:teuthology.orchestra.run.smithi042.stdout:3992 writesame 0x5e70c10 thru 0x5e7542b data_size 0x47(0x481c bytes) 2024-09-14T23:57:45.428 INFO:teuthology.orchestra.run.smithi042.stdout:3993 read 0x9c56afe thru 0x9c5da77 (0x6f7a bytes) 2024-09-14T23:57:45.428 INFO:teuthology.orchestra.run.smithi042.stdout:3994 write 0x11a765a thru 0x11aa42d (0x2dd4 bytes) 2024-09-14T23:57:45.430 INFO:teuthology.orchestra.run.smithi042.stdout:3995 read 0xb1e82b6 thru 0xb1f5756 (0xd4a1 bytes) 2024-09-14T23:57:45.430 INFO:teuthology.orchestra.run.smithi042.stdout:3997 read 0x615dc63 thru 0x616d928 (0xfcc6 bytes) 2024-09-14T23:57:45.430 INFO:teuthology.orchestra.run.smithi042.stdout:3998 punch from 0x5e90ffb to 0x5ea0b88, (0xfb8d bytes) 2024-09-14T23:57:45.430 INFO:teuthology.orchestra.run.smithi042.stdout:3999 punch from 0xc330ab1 to 0xc33c619, (0xbb68 bytes) 2024-09-14T23:57:45.431 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:45.435 INFO:teuthology.orchestra.run.smithi042.stdout:4001 write 0xcc2e93d thru 0xcc312e6 (0x29aa bytes) 2024-09-14T23:57:45.441 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:45.441 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:45.441 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:57:45.441 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:57:45.442 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:45.443 INFO:teuthology.orchestra.run.smithi042.stdout:4002 trunc from 0xcc312e7 to 0x86aab99 2024-09-14T23:57:45.452 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T23:57:45.454 INFO:teuthology.orchestra.run.smithi042.stdout:4003 write 0x49327ec thru 0x493f86d (0xd082 bytes) 2024-09-14T23:57:45.459 INFO:teuthology.orchestra.run.smithi042.stdout:4004 trunc from 0x86aab99 to 0x52ad78d 2024-09-14T23:57:45.471 INFO:teuthology.orchestra.run.smithi042.stdout:4005 read 0x4380fa6 thru 0x43822bb (0x1316 bytes) 2024-09-14T23:57:45.483 INFO:teuthology.orchestra.run.smithi042.stdout:4006 writesame 0x9c08094 thru 0x9c0dd7a data_size 0x1(0x5ce7 bytes) 2024-09-14T23:57:45.483 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:45.483 INFO:teuthology.orchestra.run.smithi042.stdout:4008 read 0xa15c95 thru 0xa22c4a (0xcfb6 bytes) 2024-09-14T23:57:45.483 INFO:teuthology.orchestra.run.smithi042.stdout:4009 read 0x975c6b2 thru 0x97689e7 (0xc336 bytes) 2024-09-14T23:57:45.483 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:45.483 INFO:teuthology.orchestra.run.smithi042.stdout:4013 read 0x6c3aafb thru 0x6c3aafc (0x2 bytes) 2024-09-14T23:57:45.491 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:45.490+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:45.493 INFO:teuthology.orchestra.run.smithi042.stdout:4014 write 0xdd76355 thru 0xdd76fd8 (0xc84 bytes) 2024-09-14T23:57:45.495 INFO:teuthology.orchestra.run.smithi042.stdout:4015 write 0x7f2140c thru 0x7f2e42b (0xd020 bytes) 2024-09-14T23:57:45.498 INFO:teuthology.orchestra.run.smithi042.stdout:4016 write 0xbbee44d thru 0xbbf2aa3 (0x4657 bytes) 2024-09-14T23:57:45.499 INFO:teuthology.orchestra.run.smithi042.stdout:4019 read 0x4d253b6 thru 0x4d2660c (0x1257 bytes) 2024-09-14T23:57:45.500 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:45.500 INFO:teuthology.orchestra.run.smithi042.stdout:4022 read 0x3b7df57 thru 0x3b86573 (0x861d bytes) 2024-09-14T23:57:45.500 INFO:teuthology.orchestra.run.smithi042.stdout:4023 write 0xab50414 thru 0xab57dca (0x79b7 bytes) 2024-09-14T23:57:45.502 INFO:teuthology.orchestra.run.smithi042.stdout:4024 read 0x55b643f thru 0x55c4305 (0xdec7 bytes) 2024-09-14T23:57:45.502 INFO:teuthology.orchestra.run.smithi042.stdout:4025 write 0xa1e1e95 thru 0xa1edee1 (0xc04d bytes) 2024-09-14T23:57:45.505 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:45.505 INFO:teuthology.orchestra.run.smithi042.stdout:4027 writesame 0x2da9b36 thru 0x2dae332 data_size 0x1(0x47fd bytes) 2024-09-14T23:57:45.515 INFO:teuthology.orchestra.run.smithi042.stdout:4028 read 0x6ce72d1 thru 0x6cf5aa7 (0xe7d7 bytes) 2024-09-14T23:57:45.515 INFO:teuthology.orchestra.run.smithi042.stdout:4031 writesame 0x53f93e9 thru 0x5407464 data_size 0x6(0xe07c bytes) 2024-09-14T23:57:45.520 INFO:teuthology.orchestra.run.smithi042.stdout:4032 trunc from 0xdd76fd9 to 0xe7990ac 2024-09-14T23:57:45.525 INFO:teuthology.orchestra.run.smithi042.stdout:4033 write 0x95012b7 thru 0x95032bb (0x2005 bytes) 2024-09-14T23:57:45.527 INFO:teuthology.orchestra.run.smithi042.stdout:4034 writesame 0x309bfae thru 0x30a541e data_size 0x3(0x9471 bytes) 2024-09-14T23:57:45.548 INFO:teuthology.orchestra.run.smithi042.stdout:4035 trunc from 0xe7990ac to 0x1b9226a 2024-09-14T23:57:45.562 INFO:teuthology.orchestra.run.smithi042.stdout:4036 trunc from 0x1b9226a to 0xa5c4b 2024-09-14T23:57:45.581 INFO:teuthology.orchestra.run.smithi042.stdout:4037 write 0x4c73928 thru 0x4c7b10f (0x77e8 bytes) 2024-09-14T23:57:45.592 INFO:teuthology.orchestra.run.smithi042.stdout:4038 trunc from 0x4c7b110 to 0xcf4cc2d 2024-09-14T23:57:45.593 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T23:57:45.593 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:45.591+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:45.598 INFO:teuthology.orchestra.run.smithi042.stdout:4039 read 0x99d1ba7 thru 0x99d9164 (0x75be bytes) 2024-09-14T23:57:45.599 INFO:teuthology.orchestra.run.smithi042.stdout:4040 read 0x389fc59 thru 0x38a7845 (0x7bed bytes) 2024-09-14T23:57:45.599 INFO:teuthology.orchestra.run.smithi042.stdout:4041 read 0x393e36c thru 0x39453cb (0x7060 bytes) 2024-09-14T23:57:45.599 INFO:teuthology.orchestra.run.smithi042.stdout:4042 writesame 0x256b48 thru 0x25952f data_size 0x12(0x29e8 bytes) 2024-09-14T23:57:45.600 INFO:teuthology.orchestra.run.smithi042.stdout:4043 read 0x1e22574 thru 0x1e31b5e (0xf5eb bytes) 2024-09-14T23:57:45.607 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:45.607 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:57:45.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:57:45.607 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:57:45.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.607 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:3500 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T23:57:45.608 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:37.990193+0000", 2024-09-14T23:57:45.608 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 67.613884338999995, 2024-09-14T23:57:45.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99440750899999997, 2024-09-14T23:57:45.608 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:45.608 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:45.608 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:45.608 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:45.608 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:45.608 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:45.608 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3500 2024-09-14T23:57:45.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.608 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:45.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:45.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990193+0000", 2024-09-14T23:57:45.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:45.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:45.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990193+0000", 2024-09-14T23:57:45.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:45.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:45.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990194+0000", 2024-09-14T23:57:45.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5000000000000001e-07 2024-09-14T23:57:45.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:45.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990200+0000", 2024-09-14T23:57:45.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4389999999999997e-06 2024-09-14T23:57:45.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:45.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990215+0000", 2024-09-14T23:57:45.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4613999999999999e-05 2024-09-14T23:57:45.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:45.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990224+0000", 2024-09-14T23:57:45.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1740000000000006e-06 2024-09-14T23:57:45.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.611 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:45.611 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990283+0000", 2024-09-14T23:57:45.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9231999999999998e-05 2024-09-14T23:57:45.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.611 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:57:45.611 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990291+0000", 2024-09-14T23:57:45.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5900000000000002e-06 2024-09-14T23:57:45.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.612 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:45.612 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:38.984249+0000", 2024-09-14T23:57:45.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99395822700000003 2024-09-14T23:57:45.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.612 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:45.612 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:38.984448+0000", 2024-09-14T23:57:45.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00019855900000000001 2024-09-14T23:57:45.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.612 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:45.612 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:38.984601+0000", 2024-09-14T23:57:45.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000152824 2024-09-14T23:57:45.613 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:45.613 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:45.613 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:45.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.613 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4055 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1093632~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:45.613 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099066+0000", 2024-09-14T23:57:45.613 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.505011451000001, 2024-09-14T23:57:45.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044174782000000003, 2024-09-14T23:57:45.613 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:45.613 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:45.613 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:45.613 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:45.613 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:45.614 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:45.614 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4055 2024-09-14T23:57:45.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.614 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:45.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.614 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:45.614 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099066+0000", 2024-09-14T23:57:45.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:45.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:45.616 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099066+0000", 2024-09-14T23:57:45.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:45.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:45.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099066+0000", 2024-09-14T23:57:45.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3700000000000001e-07 2024-09-14T23:57:45.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:45.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099067+0000", 2024-09-14T23:57:45.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1379999999999999e-06 2024-09-14T23:57:45.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:45.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099073+0000", 2024-09-14T23:57:45.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4790000000000004e-06 2024-09-14T23:57:45.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:45.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099075+0000", 2024-09-14T23:57:45.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5320000000000001e-06 2024-09-14T23:57:45.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:45.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143147+0000", 2024-09-14T23:57:45.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044071676999999997 2024-09-14T23:57:45.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:45.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143186+0000", 2024-09-14T23:57:45.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8821e-05 2024-09-14T23:57:45.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.619 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:45.619 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143241+0000", 2024-09-14T23:57:45.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4797999999999999e-05 2024-09-14T23:57:45.619 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:45.619 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:45.619 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:45.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.619 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4056 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1155072~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:45.619 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099079+0000", 2024-09-14T23:57:45.619 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.504998736000005, 2024-09-14T23:57:45.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044295020999999997, 2024-09-14T23:57:45.619 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:45.620 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:45.620 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:45.620 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:45.620 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:45.620 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:45.620 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4056 2024-09-14T23:57:45.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.620 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:45.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:45.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099079+0000", 2024-09-14T23:57:45.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:45.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:45.621 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099079+0000", 2024-09-14T23:57:45.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:45.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:45.621 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099079+0000", 2024-09-14T23:57:45.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1500000000000001e-07 2024-09-14T23:57:45.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:45.621 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099080+0000", 2024-09-14T23:57:45.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0210000000000001e-06 2024-09-14T23:57:45.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:45.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099083+0000", 2024-09-14T23:57:45.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.174e-06 2024-09-14T23:57:45.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:45.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099086+0000", 2024-09-14T23:57:45.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.446e-06 2024-09-14T23:57:45.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:45.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143269+0000", 2024-09-14T23:57:45.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044183489999999999 2024-09-14T23:57:45.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:45.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143314+0000", 2024-09-14T23:57:45.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5135000000000001e-05 2024-09-14T23:57:45.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:45.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143374+0000", 2024-09-14T23:57:45.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9540000000000003e-05 2024-09-14T23:57:45.623 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:45.623 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:45.623 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:45.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.624 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4057 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1216512~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:45.624 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099088+0000", 2024-09-14T23:57:45.624 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.504988948999994, 2024-09-14T23:57:45.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044406119000000001, 2024-09-14T23:57:45.624 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:45.624 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:45.624 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:45.624 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:45.624 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:45.624 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:45.624 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4057 2024-09-14T23:57:45.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.625 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:45.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.625 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:45.625 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099088+0000", 2024-09-14T23:57:45.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:45.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.625 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:45.625 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099088+0000", 2024-09-14T23:57:45.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:45.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.625 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:45.625 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099089+0000", 2024-09-14T23:57:45.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.04e-07 2024-09-14T23:57:45.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:45.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099090+0000", 2024-09-14T23:57:45.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.026e-06 2024-09-14T23:57:45.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:45.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099093+0000", 2024-09-14T23:57:45.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0819999999999999e-06 2024-09-14T23:57:45.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:45.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099095+0000", 2024-09-14T23:57:45.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4930000000000001e-06 2024-09-14T23:57:45.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:45.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143400+0000", 2024-09-14T23:57:45.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044304698000000003 2024-09-14T23:57:45.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:45.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143441+0000", 2024-09-14T23:57:45.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1100000000000003e-05 2024-09-14T23:57:45.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:45.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143495+0000", 2024-09-14T23:57:45.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3516000000000001e-05 2024-09-14T23:57:45.628 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:45.628 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:45.628 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:45.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.628 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4058 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1277952~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:45.628 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099099+0000", 2024-09-14T23:57:45.628 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.504978457999997, 2024-09-14T23:57:45.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044514583000000003, 2024-09-14T23:57:45.628 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:45.628 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:45.629 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:45.629 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:45.629 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:45.629 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:45.629 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4058 2024-09-14T23:57:45.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.629 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:45.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.629 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:45.629 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099099+0000", 2024-09-14T23:57:45.629 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:45.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.629 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:45.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099099+0000", 2024-09-14T23:57:45.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:45.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:45.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099099+0000", 2024-09-14T23:57:45.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9999999999999999e-07 2024-09-14T23:57:45.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:45.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099100+0000", 2024-09-14T23:57:45.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7199999999999997e-07 2024-09-14T23:57:45.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.631 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:45.631 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099103+0000", 2024-09-14T23:57:45.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.095e-06 2024-09-14T23:57:45.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.631 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:45.631 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099106+0000", 2024-09-14T23:57:45.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4229999999999999e-06 2024-09-14T23:57:45.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.631 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:45.631 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143524+0000", 2024-09-14T23:57:45.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044417891000000001 2024-09-14T23:57:45.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.632 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:45.632 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143568+0000", 2024-09-14T23:57:45.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4060000000000002e-05 2024-09-14T23:57:45.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.632 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:45.632 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143614+0000", 2024-09-14T23:57:45.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5942000000000002e-05 2024-09-14T23:57:45.632 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:45.632 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:45.632 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:45.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.633 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4059 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1339392~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:45.633 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099114+0000", 2024-09-14T23:57:45.633 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.504963919000005, 2024-09-14T23:57:45.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044617194999999998, 2024-09-14T23:57:45.633 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:45.633 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:45.633 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:45.633 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:45.633 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:45.633 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:45.633 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4059 2024-09-14T23:57:45.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.634 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:45.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:45.634 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099114+0000", 2024-09-14T23:57:45.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:45.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:45.634 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099114+0000", 2024-09-14T23:57:45.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:45.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:45.634 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099114+0000", 2024-09-14T23:57:45.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5899999999999998e-07 2024-09-14T23:57:45.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:45.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099115+0000", 2024-09-14T23:57:45.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3510000000000001e-06 2024-09-14T23:57:45.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:45.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099119+0000", 2024-09-14T23:57:45.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.16e-06 2024-09-14T23:57:45.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:45.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099123+0000", 2024-09-14T23:57:45.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1339999999999997e-06 2024-09-14T23:57:45.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:45.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143643+0000", 2024-09-14T23:57:45.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044519232999999998 2024-09-14T23:57:45.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:45.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143686+0000", 2024-09-14T23:57:45.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3349000000000001e-05 2024-09-14T23:57:45.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:45.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143731+0000", 2024-09-14T23:57:45.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4709000000000002e-05 2024-09-14T23:57:45.637 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:45.637 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:45.637 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:45.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.637 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4060 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1400832~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:45.637 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099129+0000", 2024-09-14T23:57:45.637 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.504948696, 2024-09-14T23:57:45.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044721271999999999, 2024-09-14T23:57:45.637 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:45.637 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:45.637 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:45.638 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:45.638 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:45.638 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:45.638 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4060 2024-09-14T23:57:45.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.638 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:45.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.638 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:45.638 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099129+0000", 2024-09-14T23:57:45.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:45.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.638 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:45.638 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099129+0000", 2024-09-14T23:57:45.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:45.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:45.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099191+0000", 2024-09-14T23:57:45.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1805999999999997e-05 2024-09-14T23:57:45.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:45.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099193+0000", 2024-09-14T23:57:45.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2330000000000001e-06 2024-09-14T23:57:45.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:45.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099200+0000", 2024-09-14T23:57:45.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4200000000000001e-06 2024-09-14T23:57:45.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:45.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099204+0000", 2024-09-14T23:57:45.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7500000000000001e-06 2024-09-14T23:57:45.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:45.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143760+0000", 2024-09-14T23:57:45.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044556182 2024-09-14T23:57:45.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:45.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143804+0000", 2024-09-14T23:57:45.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4239000000000001e-05 2024-09-14T23:57:45.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:45.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143850+0000", 2024-09-14T23:57:45.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5642000000000001e-05 2024-09-14T23:57:45.641 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:45.641 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:45.641 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:45.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.642 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4061 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1462272~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:45.642 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099215+0000", 2024-09-14T23:57:45.642 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.504862571000004, 2024-09-14T23:57:45.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044745659, 2024-09-14T23:57:45.642 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:45.642 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:45.642 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:45.642 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:45.642 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:45.642 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:45.642 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4061 2024-09-14T23:57:45.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.642 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:45.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:45.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099215+0000", 2024-09-14T23:57:45.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:45.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:45.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099215+0000", 2024-09-14T23:57:45.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:45.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:45.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100672+0000", 2024-09-14T23:57:45.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00145689 2024-09-14T23:57:45.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:45.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100677+0000", 2024-09-14T23:57:45.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0139999999999998e-06 2024-09-14T23:57:45.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:45.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100689+0000", 2024-09-14T23:57:45.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.183e-05 2024-09-14T23:57:45.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:45.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100696+0000", 2024-09-14T23:57:45.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2599999999999999e-06 2024-09-14T23:57:45.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:45.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143877+0000", 2024-09-14T23:57:45.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043181279000000003 2024-09-14T23:57:45.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:45.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143919+0000", 2024-09-14T23:57:45.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1783000000000003e-05 2024-09-14T23:57:45.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:45.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143961+0000", 2024-09-14T23:57:45.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1603000000000001e-05 2024-09-14T23:57:45.646 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:45.646 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:45.646 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:45.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.646 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4070 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 2015232~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:45.646 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.100903+0000", 2024-09-14T23:57:45.646 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.503174862999998, 2024-09-14T23:57:45.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044103815999999997, 2024-09-14T23:57:45.646 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:45.646 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:45.646 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:45.647 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:45.647 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:45.647 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:45.647 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4070 2024-09-14T23:57:45.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.647 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:45.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:45.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100903+0000", 2024-09-14T23:57:45.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:45.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:45.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100903+0000", 2024-09-14T23:57:45.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:45.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:45.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.102671+0000", 2024-09-14T23:57:45.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001768279 2024-09-14T23:57:45.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:45.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.102675+0000", 2024-09-14T23:57:45.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9609999999999999e-06 2024-09-14T23:57:45.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:45.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.102684+0000", 2024-09-14T23:57:45.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.329e-06 2024-09-14T23:57:45.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:45.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.102689+0000", 2024-09-14T23:57:45.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7079999999999996e-06 2024-09-14T23:57:45.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:45.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.144912+0000", 2024-09-14T23:57:45.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.042223305000000003 2024-09-14T23:57:45.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:45.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.144960+0000", 2024-09-14T23:57:45.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7670000000000003e-05 2024-09-14T23:57:45.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:45.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.145006+0000", 2024-09-14T23:57:45.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6563999999999997e-05 2024-09-14T23:57:45.650 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:45.650 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:45.650 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:45.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.651 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4096 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1282048~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:45.651 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.104957+0000", 2024-09-14T23:57:45.651 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.499120689999998, 2024-09-14T23:57:45.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044093016999999998, 2024-09-14T23:57:45.651 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:45.651 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:45.651 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:45.651 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:45.651 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:45.651 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:45.651 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4096 2024-09-14T23:57:45.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.651 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:45.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.652 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:45.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104957+0000", 2024-09-14T23:57:45.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:45.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.652 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:45.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104957+0000", 2024-09-14T23:57:45.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:45.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.652 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:45.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104957+0000", 2024-09-14T23:57:45.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7400000000000003e-07 2024-09-14T23:57:45.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:45.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104960+0000", 2024-09-14T23:57:45.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3800000000000001e-06 2024-09-14T23:57:45.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:45.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104967+0000", 2024-09-14T23:57:45.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4030000000000001e-06 2024-09-14T23:57:45.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:45.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104973+0000", 2024-09-14T23:57:45.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7370000000000003e-06 2024-09-14T23:57:45.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:45.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.148883+0000", 2024-09-14T23:57:45.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043910124000000002 2024-09-14T23:57:45.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:45.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.148966+0000", 2024-09-14T23:57:45.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2868999999999996e-05 2024-09-14T23:57:45.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:45.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149050+0000", 2024-09-14T23:57:45.655 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3930000000000004e-05 2024-09-14T23:57:45.655 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:45.655 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:45.655 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:45.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.655 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4097 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1343488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:45.655 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.104983+0000", 2024-09-14T23:57:45.655 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.499094901000007, 2024-09-14T23:57:45.655 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044182104999999999, 2024-09-14T23:57:45.655 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:45.655 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:45.655 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:45.655 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:45.656 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:45.656 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:45.656 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4097 2024-09-14T23:57:45.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.656 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:45.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:45.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104983+0000", 2024-09-14T23:57:45.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:45.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:45.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104983+0000", 2024-09-14T23:57:45.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:45.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:45.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105207+0000", 2024-09-14T23:57:45.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000224687 2024-09-14T23:57:45.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:45.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105211+0000", 2024-09-14T23:57:45.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3050000000000001e-06 2024-09-14T23:57:45.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:45.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105217+0000", 2024-09-14T23:57:45.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8170000000000001e-06 2024-09-14T23:57:45.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:45.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105222+0000", 2024-09-14T23:57:45.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.267e-06 2024-09-14T23:57:45.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:45.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149079+0000", 2024-09-14T23:57:45.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043857581999999999 2024-09-14T23:57:45.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:45.659 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149122+0000", 2024-09-14T23:57:45.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2899999999999999e-05 2024-09-14T23:57:45.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.659 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:45.659 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149165+0000", 2024-09-14T23:57:45.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2546999999999998e-05 2024-09-14T23:57:45.659 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:45.659 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:45.659 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:45.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.659 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4098 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1404928~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:45.659 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.105242+0000", 2024-09-14T23:57:45.660 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.498835088999996, 2024-09-14T23:57:45.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044121104000000001, 2024-09-14T23:57:45.660 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:45.660 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:45.660 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:45.660 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:45.660 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:45.660 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:45.660 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4098 2024-09-14T23:57:45.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.660 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:45.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:45.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105242+0000", 2024-09-14T23:57:45.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:45.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:45.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105242+0000", 2024-09-14T23:57:45.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:45.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:45.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105243+0000", 2024-09-14T23:57:45.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0399999999999995e-07 2024-09-14T23:57:45.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:45.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105246+0000", 2024-09-14T23:57:45.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6309999999999999e-06 2024-09-14T23:57:45.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:45.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105253+0000", 2024-09-14T23:57:45.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0940000000000002e-06 2024-09-14T23:57:45.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:45.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105260+0000", 2024-09-14T23:57:45.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1799999999999999e-06 2024-09-14T23:57:45.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.663 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:45.663 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149201+0000", 2024-09-14T23:57:45.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043940629000000002 2024-09-14T23:57:45.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.663 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:45.663 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149283+0000", 2024-09-14T23:57:45.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2360000000000004e-05 2024-09-14T23:57:45.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.663 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:45.663 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149363+0000", 2024-09-14T23:57:45.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0506000000000005e-05 2024-09-14T23:57:45.664 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:45.664 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:45.664 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:45.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.664 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.664 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4099 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1466368~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:45.664 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.105270+0000", 2024-09-14T23:57:45.664 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.498807494000005, 2024-09-14T23:57:45.664 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044195433999999999, 2024-09-14T23:57:45.664 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:45.664 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:45.664 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:45.664 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:45.664 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:45.665 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:45.665 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4099 2024-09-14T23:57:45.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.665 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:45.665 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.665 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:45.665 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105270+0000", 2024-09-14T23:57:45.665 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:45.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.665 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.665 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:45.665 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105270+0000", 2024-09-14T23:57:45.665 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:45.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:45.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105429+0000", 2024-09-14T23:57:45.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015940299999999999 2024-09-14T23:57:45.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:45.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105433+0000", 2024-09-14T23:57:45.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3579999999999999e-06 2024-09-14T23:57:45.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:45.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105439+0000", 2024-09-14T23:57:45.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1759999999999998e-06 2024-09-14T23:57:45.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:45.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105445+0000", 2024-09-14T23:57:45.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9689999999999999e-06 2024-09-14T23:57:45.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:45.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149381+0000", 2024-09-14T23:57:45.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043936389999999999 2024-09-14T23:57:45.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.668 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:45.668 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149422+0000", 2024-09-14T23:57:45.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0865000000000003e-05 2024-09-14T23:57:45.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.668 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.668 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:45.668 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149465+0000", 2024-09-14T23:57:45.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3272999999999997e-05 2024-09-14T23:57:45.668 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:45.668 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:45.668 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:45.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.668 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.668 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4100 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1527808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:45.669 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.105454+0000", 2024-09-14T23:57:45.669 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.498623022999993, 2024-09-14T23:57:45.669 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044217411999999998, 2024-09-14T23:57:45.669 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:45.669 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:45.669 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:45.669 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:45.669 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:45.669 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:45.669 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4100 2024-09-14T23:57:45.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.669 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:45.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:45.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105454+0000", 2024-09-14T23:57:45.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:45.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:45.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105454+0000", 2024-09-14T23:57:45.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:45.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:45.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105578+0000", 2024-09-14T23:57:45.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012401799999999999 2024-09-14T23:57:45.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.671 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.671 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:45.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105583+0000", 2024-09-14T23:57:45.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1779999999999996e-06 2024-09-14T23:57:45.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.671 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.671 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:45.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105590+0000", 2024-09-14T23:57:45.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4279999999999999e-06 2024-09-14T23:57:45.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.671 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.671 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:45.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105596+0000", 2024-09-14T23:57:45.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0120000000000002e-06 2024-09-14T23:57:45.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.672 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.672 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:45.672 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149503+0000", 2024-09-14T23:57:45.672 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043906580000000001 2024-09-14T23:57:45.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.672 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.672 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:45.672 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149585+0000", 2024-09-14T23:57:45.672 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2356999999999998e-05 2024-09-14T23:57:45.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.672 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.672 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:45.672 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149672+0000", 2024-09-14T23:57:45.673 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6839e-05 2024-09-14T23:57:45.673 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:45.673 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:45.673 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:45.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.673 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.673 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4101 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1589248~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:45.673 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.105609+0000", 2024-09-14T23:57:45.673 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.498468484, 2024-09-14T23:57:45.673 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044175987999999999, 2024-09-14T23:57:45.673 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:45.673 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:45.673 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:45.673 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:45.673 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:45.674 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:45.674 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4101 2024-09-14T23:57:45.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.674 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:45.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.674 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:45.674 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105609+0000", 2024-09-14T23:57:45.674 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:45.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.674 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:45.674 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105609+0000", 2024-09-14T23:57:45.674 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:45.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.675 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:45.675 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.106671+0000", 2024-09-14T23:57:45.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010619270000000001 2024-09-14T23:57:45.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.675 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:45.675 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.106675+0000", 2024-09-14T23:57:45.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4429999999999996e-06 2024-09-14T23:57:45.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.675 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:45.675 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.106685+0000", 2024-09-14T23:57:45.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7429999999999997e-06 2024-09-14T23:57:45.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.676 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:45.676 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.106690+0000", 2024-09-14T23:57:45.676 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.293e-06 2024-09-14T23:57:45.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.676 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:45.676 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149700+0000", 2024-09-14T23:57:45.676 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043009512 2024-09-14T23:57:45.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.676 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:45.676 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149742+0000", 2024-09-14T23:57:45.676 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2083999999999999e-05 2024-09-14T23:57:45.677 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.677 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.677 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:45.677 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149785+0000", 2024-09-14T23:57:45.677 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2985999999999998e-05 2024-09-14T23:57:45.677 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:45.677 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:45.677 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:45.677 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.677 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.677 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4116 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 180224~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:45.677 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.107575+0000", 2024-09-14T23:57:45.677 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.496502015999994, 2024-09-14T23:57:45.677 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044157522999999997, 2024-09-14T23:57:45.678 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:45.678 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:45.678 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:45.678 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:45.678 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:45.678 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:45.678 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4116 2024-09-14T23:57:45.678 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.678 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:45.678 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.678 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:45.678 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.107575+0000", 2024-09-14T23:57:45.678 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:45.678 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.679 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:45.679 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.107575+0000", 2024-09-14T23:57:45.679 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:45.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.679 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:45.679 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108671+0000", 2024-09-14T23:57:45.679 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010954510000000001 2024-09-14T23:57:45.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.679 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:45.679 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108676+0000", 2024-09-14T23:57:45.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6639999999999997e-06 2024-09-14T23:57:45.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.680 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:45.680 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108685+0000", 2024-09-14T23:57:45.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1130000000000007e-06 2024-09-14T23:57:45.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.680 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:45.680 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108691+0000", 2024-09-14T23:57:45.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6019999999999996e-06 2024-09-14T23:57:45.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.680 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:45.681 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.151530+0000", 2024-09-14T23:57:45.681 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.042838345999999999 2024-09-14T23:57:45.681 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.681 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.681 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:45.681 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.151619+0000", 2024-09-14T23:57:45.681 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9302999999999995e-05 2024-09-14T23:57:45.681 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.681 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.681 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:45.681 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.151733+0000", 2024-09-14T23:57:45.681 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011404399999999999 2024-09-14T23:57:45.681 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:45.682 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:45.682 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:45.682 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.682 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.682 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4122 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 548864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:45.682 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.108842+0000", 2024-09-14T23:57:45.682 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.495235178000001, 2024-09-14T23:57:45.682 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044103631999999997, 2024-09-14T23:57:45.682 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:45.682 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:45.682 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:45.682 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:45.682 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:45.682 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:45.682 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4122 2024-09-14T23:57:45.683 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.683 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:45.683 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.683 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:45.683 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108842+0000", 2024-09-14T23:57:45.683 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:45.683 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.683 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.683 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:45.683 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108842+0000", 2024-09-14T23:57:45.683 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:45.683 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.683 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.684 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:45.684 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108843+0000", 2024-09-14T23:57:45.684 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7000000000000001e-07 2024-09-14T23:57:45.684 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.684 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.684 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:45.684 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108844+0000", 2024-09-14T23:57:45.684 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6479999999999999e-06 2024-09-14T23:57:45.684 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.684 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.684 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:45.684 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108848+0000", 2024-09-14T23:57:45.684 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8029999999999998e-06 2024-09-14T23:57:45.685 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.685 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.685 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:45.685 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108851+0000", 2024-09-14T23:57:45.685 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7939999999999998e-06 2024-09-14T23:57:45.685 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.685 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.685 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:45.685 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.152838+0000", 2024-09-14T23:57:45.685 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043987351000000001 2024-09-14T23:57:45.685 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.685 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.685 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:45.685 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.152885+0000", 2024-09-14T23:57:45.686 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7237000000000002e-05 2024-09-14T23:57:45.686 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.686 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.686 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:45.686 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.152946+0000", 2024-09-14T23:57:45.686 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0529000000000003e-05 2024-09-14T23:57:45.686 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:45.686 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:45.686 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:45.686 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.686 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.686 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4123 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 610304~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:45.686 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.108855+0000", 2024-09-14T23:57:45.686 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.495222849000001, 2024-09-14T23:57:45.687 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044321610999999997, 2024-09-14T23:57:45.687 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:45.687 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:45.687 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:45.687 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:45.687 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:45.687 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:45.687 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4123 2024-09-14T23:57:45.687 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.687 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:45.687 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.687 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:45.687 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108855+0000", 2024-09-14T23:57:45.687 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:45.688 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.688 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.688 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:45.688 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108855+0000", 2024-09-14T23:57:45.688 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:45.688 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.688 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.688 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:45.688 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.110671+0000", 2024-09-14T23:57:45.688 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018159719999999999 2024-09-14T23:57:45.688 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.688 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.688 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:45.688 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.110676+0000", 2024-09-14T23:57:45.689 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9450000000000001e-06 2024-09-14T23:57:45.689 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.689 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.689 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:45.689 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.110685+0000", 2024-09-14T23:57:45.689 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5359999999999998e-06 2024-09-14T23:57:45.689 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.689 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.689 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:45.689 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.110692+0000", 2024-09-14T23:57:45.689 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9280000000000004e-06 2024-09-14T23:57:45.689 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.689 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.689 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:45.689 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.152984+0000", 2024-09-14T23:57:45.690 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.042291677999999999 2024-09-14T23:57:45.690 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.690 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.690 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:45.690 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.153066+0000", 2024-09-14T23:57:45.690 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2657999999999994e-05 2024-09-14T23:57:45.690 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.690 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.690 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:45.690 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.153176+0000", 2024-09-14T23:57:45.690 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000109894 2024-09-14T23:57:45.690 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:45.690 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:45.690 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:45.691 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.691 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.691 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4136 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:45.691 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.111056+0000", 2024-09-14T23:57:45.691 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.493021701000004, 2024-09-14T23:57:45.691 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044097428000000001, 2024-09-14T23:57:45.691 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:45.691 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:45.691 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:45.691 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:45.691 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:45.691 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:45.691 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4136 2024-09-14T23:57:45.691 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.691 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:45.692 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.692 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:45.692 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.111056+0000", 2024-09-14T23:57:45.692 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:45.692 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.692 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.692 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:45.692 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.111056+0000", 2024-09-14T23:57:45.692 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:45.692 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.692 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.692 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:45.692 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.112671+0000", 2024-09-14T23:57:45.692 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016153999999999999 2024-09-14T23:57:45.693 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.693 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.693 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:45.693 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.112675+0000", 2024-09-14T23:57:45.693 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2830000000000003e-06 2024-09-14T23:57:45.693 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.693 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.693 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:45.693 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.112684+0000", 2024-09-14T23:57:45.693 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7800000000000006e-06 2024-09-14T23:57:45.693 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.693 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.693 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:45.694 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.112689+0000", 2024-09-14T23:57:45.694 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0000000000000004e-06 2024-09-14T23:57:45.694 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.694 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.694 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:45.694 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.155064+0000", 2024-09-14T23:57:45.694 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.042374423000000001 2024-09-14T23:57:45.694 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.694 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.694 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:45.694 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.155111+0000", 2024-09-14T23:57:45.694 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7367000000000002e-05 2024-09-14T23:57:45.694 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.694 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.695 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:45.695 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.155153+0000", 2024-09-14T23:57:45.695 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2175000000000002e-05 2024-09-14T23:57:45.695 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:45.695 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:45.695 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:45.695 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.695 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.695 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:46572 3.1d 3:b9e1f524:::rbd_header.1118cee06433:head [watch reconnect cookie 140670782499824 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-14T23:57:45.695 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:57.046868+0000", 2024-09-14T23:57:45.695 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 48.557209895, 2024-09-14T23:57:45.695 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0243412620000001, 2024-09-14T23:57:45.695 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:45.695 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:45.695 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:45.696 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:45.696 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:45.696 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:45.696 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 46572 2024-09-14T23:57:45.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.696 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:45.696 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.696 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:45.696 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046868+0000", 2024-09-14T23:57:45.697 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:45.697 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.697 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.697 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:45.697 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046868+0000", 2024-09-14T23:57:45.697 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:45.697 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.697 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.697 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:45.697 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046870+0000", 2024-09-14T23:57:45.697 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.204e-06 2024-09-14T23:57:45.697 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.697 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.697 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:45.698 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046877+0000", 2024-09-14T23:57:45.698 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5100000000000001e-06 2024-09-14T23:57:45.698 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.698 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.698 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:45.698 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046898+0000", 2024-09-14T23:57:45.698 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1149999999999999e-05 2024-09-14T23:57:45.698 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.698 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.698 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:45.698 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046911+0000", 2024-09-14T23:57:45.698 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2234e-05 2024-09-14T23:57:45.698 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.698 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.699 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:45.699 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046988+0000", 2024-09-14T23:57:45.699 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7445000000000001e-05 2024-09-14T23:57:45.699 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.699 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.699 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:57:45.699 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.047001+0000", 2024-09-14T23:57:45.699 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2951e-05 2024-09-14T23:57:45.699 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.699 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.699 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:45.699 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.069287+0000", 2024-09-14T23:57:45.699 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.022285748 2024-09-14T23:57:45.699 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.699 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.700 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for rw locks", 2024-09-14T23:57:45.700 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.069397+0000", 2024-09-14T23:57:45.700 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000110326 2024-09-14T23:57:45.700 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.700 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.700 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:45.700 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.069997+0000", 2024-09-14T23:57:45.700 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00059984399999999998 2024-09-14T23:57:45.700 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.700 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.700 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:45.700 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.070109+0000", 2024-09-14T23:57:45.700 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000112367 2024-09-14T23:57:45.700 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:45.701 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:45.701 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:45.701 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.071209+0000", 2024-09-14T23:57:45.701 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010994830000000001 2024-09-14T23:57:45.701 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:45.701 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:45.701 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:45.701 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:45.701 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:45.701 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:45.703 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T23:57:45.704 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:45.700+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:45.704 INFO:teuthology.orchestra.run.smithi042.stdout:4044 trunc from 0xcf4cc2d to 0x1e33a08 2024-09-14T23:57:45.705 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:45.705 INFO:teuthology.orchestra.run.smithi042.stdout:4047 trunc from 0x1e33a08 to 0x42f3668 2024-09-14T23:57:45.705 INFO:teuthology.orchestra.run.smithi042.stdout:4048 read 0xc44f5e thru 0xc504a1 (0xb544 bytes) 2024-09-14T23:57:45.705 INFO:teuthology.orchestra.run.smithi042.stdout:4049 trunc from 0x42f3668 to 0xe96d9cf 2024-09-14T23:57:45.705 INFO:teuthology.orchestra.run.smithi042.stdout:4050 read 0xd30e520 thru 0xd31ccf9 (0xe7da bytes) 2024-09-14T23:57:45.705 INFO:teuthology.orchestra.run.smithi042.stdout:4054 write 0xbf4ffb0 thru 0xbf53204 (0x3255 bytes) 2024-09-14T23:57:45.705 INFO:teuthology.orchestra.run.smithi042.stdout:4056 punch from 0x5fa46b6 to 0x5fb02cf, (0xbc19 bytes) 2024-09-14T23:57:45.705 INFO:teuthology.orchestra.run.smithi042.stdout:4057 read 0xc8e5d7e thru 0xc8ef384 (0x9607 bytes) 2024-09-14T23:57:45.705 INFO:teuthology.orchestra.run.smithi042.stdout:4058 write 0xbdb4975 thru 0xbdc231d (0xd9a9 bytes) 2024-09-14T23:57:45.705 INFO:teuthology.orchestra.run.smithi042.stdout:4059 writesame 0xe0029fe thru 0xe0040eb data_size 0x5(0x16ee bytes) 2024-09-14T23:57:45.705 INFO:teuthology.orchestra.run.smithi042.stdout:4062 write 0x504033a thru 0x504c6b5 (0xc37c bytes) 2024-09-14T23:57:45.705 INFO:teuthology.orchestra.run.smithi042.stdout:4063 trunc from 0xe96d9cf to 0x25adda 2024-09-14T23:57:45.705 INFO:teuthology.orchestra.run.smithi042.stdout:4066 trunc from 0x25adda to 0x66dee81 2024-09-14T23:57:45.714 INFO:teuthology.orchestra.run.smithi042.stdout:4067 write 0xe09fd4f thru 0xe0a994b (0x9bfd bytes) 2024-09-14T23:57:45.717 INFO:teuthology.orchestra.run.smithi042.stdout:4069 writesame 0xba495b1 thru 0xba4ad6c data_size 0xe(0x17bc bytes) 2024-09-14T23:57:45.721 INFO:teuthology.orchestra.run.smithi042.stdout:4070 write 0x22816b6 thru 0x228fb1e (0xe469 bytes) 2024-09-14T23:57:45.723 INFO:teuthology.orchestra.run.smithi042.stdout:4071 read 0xdc8632b thru 0xdc8fc56 (0x992c bytes) 2024-09-14T23:57:45.723 INFO:teuthology.orchestra.run.smithi042.stdout:4072 writesame 0xd4f84a2 thru 0xd5017e1 data_size 0x13(0x9340 bytes) 2024-09-14T23:57:45.727 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:45.727 INFO:teuthology.orchestra.run.smithi042.stdout:4075 punch from 0x7af4c71 to 0x7b00e69, (0xc1f8 bytes) 2024-09-14T23:57:45.727 INFO:teuthology.orchestra.run.smithi042.stdout:4076 writesame 0x917317e thru 0x9177cc5 data_size 0x49(0x4b48 bytes) 2024-09-14T23:57:45.732 INFO:teuthology.orchestra.run.smithi042.stdout:4077 read 0x4c9853f thru 0x4ca3061 (0xab23 bytes) 2024-09-14T23:57:45.732 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:45.732 INFO:teuthology.orchestra.run.smithi042.stdout:4081 read 0x91fe93e thru 0x91fed0f (0x3d2 bytes) 2024-09-14T23:57:45.733 INFO:teuthology.orchestra.run.smithi042.stdout:4082 write 0x1c4d321 thru 0x1c4f122 (0x1e02 bytes) 2024-09-14T23:57:45.736 INFO:teuthology.orchestra.run.smithi042.stdout:4083 trunc from 0xe0a994c to 0xb3b00f3 2024-09-14T23:57:45.742 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:57:45.742 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:57:45.748 INFO:teuthology.orchestra.run.smithi042.stdout:4084 write 0x8a56ff7 thru 0x8a5c9b2 (0x59bc bytes) 2024-09-14T23:57:45.749 INFO:teuthology.orchestra.run.smithi042.stdout:4087 punch from 0x3f71d74 to 0x3f77a88, (0x5d14 bytes) 2024-09-14T23:57:45.749 INFO:teuthology.orchestra.run.smithi042.stdout:4088 punch from 0x3f9d8b9 to 0x3fa18b6, (0x3ffd bytes) 2024-09-14T23:57:45.749 INFO:teuthology.orchestra.run.smithi042.stdout:4089 punch from 0xad0b6ce to 0xad0d37c, (0x1cae bytes) 2024-09-14T23:57:45.749 INFO:teuthology.orchestra.run.smithi042.stdout:4090 write 0x673d2a4 thru 0x6743dfa (0x6b57 bytes) 2024-09-14T23:57:45.751 INFO:teuthology.orchestra.run.smithi042.stdout:4092 read 0x60a74c1 thru 0x60ae7a9 (0x72e9 bytes) 2024-09-14T23:57:45.751 INFO:teuthology.orchestra.run.smithi042.stdout:4093 read 0xa1a6183 thru 0xa1afd4a (0x9bc8 bytes) 2024-09-14T23:57:45.751 INFO:teuthology.orchestra.run.smithi042.stdout:4095 write 0x36bb554 thru 0x36c3912 (0x83bf bytes) 2024-09-14T23:57:45.753 INFO:teuthology.orchestra.run.smithi042.stdout:4096 write 0x1eb842d thru 0x1ebe481 (0x6055 bytes) 2024-09-14T23:57:45.755 INFO:teuthology.orchestra.run.smithi042.stdout:4097 read 0x1687b8b thru 0x168da89 (0x5eff bytes) 2024-09-14T23:57:45.755 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:45.755 INFO:teuthology.orchestra.run.smithi042.stdout:4099 writesame 0x8b87c70 thru 0x8b93fcf data_size 0x30(0xc360 bytes) 2024-09-14T23:57:45.762 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:57:45.762 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:57:45.764 INFO:teuthology.orchestra.run.smithi042.stdout:4102 writesame 0xb89d094 thru 0xb8a40b0 data_size 0x9(0x701d bytes) 2024-09-14T23:57:45.765 INFO:teuthology.orchestra.run.smithi042.stdout:4103 writesame 0x5129d4b thru 0x5131215 data_size 0x1(0x74cb bytes) 2024-09-14T23:57:45.779 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:57:45.779 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:57:45.783 INFO:teuthology.orchestra.run.smithi042.stdout:4104 trunc from 0xb8a40b1 to 0x9410819 2024-09-14T23:57:45.790 INFO:teuthology.orchestra.run.smithi042.stdout:4105 punch from 0x60d8e26 to 0x60d9920, (0xafa bytes) 2024-09-14T23:57:45.790 INFO:teuthology.orchestra.run.smithi042.stdout:4106 read 0x5ea4f44 thru 0x5eb272d (0xd7ea bytes) 2024-09-14T23:57:45.790 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:45.790 INFO:teuthology.orchestra.run.smithi042.stdout:4109 write 0x316252e thru 0x317138a (0xee5d bytes) 2024-09-14T23:57:45.792 INFO:teuthology.orchestra.run.smithi042.stdout:4111 writesame 0x81ae3eb thru 0x81b4b2c data_size 0x1(0x6742 bytes) 2024-09-14T23:57:45.801 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:57:45.801 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:57:45.804 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:45.802+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:45.805 INFO:teuthology.orchestra.run.smithi042.stdout:4112 read 0x709cf12 thru 0x70a1694 (0x4783 bytes) 2024-09-14T23:57:45.805 INFO:teuthology.orchestra.run.smithi042.stdout:4114 write 0x1658b94 thru 0x1663ef4 (0xb361 bytes) 2024-09-14T23:57:45.814 INFO:teuthology.orchestra.run.smithi042.stdout:4115 trunc from 0x9410819 to 0xeaeda85 2024-09-14T23:57:45.820 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:57:45.821 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:57:45.821 INFO:teuthology.orchestra.run.smithi042.stdout:4116 write 0xd6370c5 thru 0xd644962 (0xd89e bytes) 2024-09-14T23:57:45.824 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:45.825 INFO:teuthology.orchestra.run.smithi042.stdout:4119 read 0x2a8dd68 thru 0x2a91acf (0x3d68 bytes) 2024-09-14T23:57:45.825 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:45.826 INFO:teuthology.orchestra.run.smithi042.stdout:4121 punch from 0x331b133 to 0x331c6d7, (0x15a4 bytes) 2024-09-14T23:57:45.826 INFO:teuthology.orchestra.run.smithi042.stdout:4123 write 0x6697a0f thru 0x66a272a (0xad1c bytes) 2024-09-14T23:57:45.826 INFO:teuthology.orchestra.run.smithi042.stdout:4125 write 0xe3b9c19 thru 0xe3c26c2 (0x8aaa bytes) 2024-09-14T23:57:45.828 INFO:teuthology.orchestra.run.smithi042.stdout:4126 write 0x38b5696 thru 0x38bb0a7 (0x5a12 bytes) 2024-09-14T23:57:45.839 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:57:45.840 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:57:45.844 INFO:teuthology.orchestra.run.smithi042.stdout:4127 trunc from 0xeaeda85 to 0x282ddd2 2024-09-14T23:57:45.856 INFO:teuthology.orchestra.run.smithi042.stdout:4128 read 0x10559ef thru 0x105f935 (0x9f47 bytes) 2024-09-14T23:57:45.857 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:45.857 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:45.857 INFO:teuthology.orchestra.run.smithi042.stdout:4129 read 0xf9b5bc thru 0xfa8c94 (0xd6d9 bytes) 2024-09-14T23:57:45.861 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:57:45.862 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:57:45.866 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T23:57:45.874 INFO:teuthology.orchestra.run.smithi042.stdout:4130 writesame 0xc716410 thru 0xc717afe data_size 0x13(0x16ef bytes) 2024-09-14T23:57:45.874 INFO:teuthology.orchestra.run.smithi042.stdout:4132 read 0xc399957 thru 0xc3a2b15 (0x91bf bytes) 2024-09-14T23:57:45.874 INFO:teuthology.orchestra.run.smithi042.stdout:4133 read 0x1f94398 thru 0x1fa4121 (0xfd8a bytes) 2024-09-14T23:57:45.875 INFO:teuthology.orchestra.run.smithi042.stdout:4134 punch from 0x1a75c12 to 0x1a80920, (0xad0e bytes) 2024-09-14T23:57:45.875 INFO:teuthology.orchestra.run.smithi042.stdout:4135 write 0x57e6242 thru 0x57ee7ac (0x856b bytes) 2024-09-14T23:57:45.877 INFO:teuthology.orchestra.run.smithi042.stdout:4136 writesame 0x8c255c3 thru 0x8c340c5 data_size 0x1(0xeb03 bytes) 2024-09-14T23:57:45.881 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:57:45.881 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:57:45.907 INFO:teuthology.orchestra.run.smithi042.stdout:4138 write 0x346fba8 thru 0x34780d0 (0x8529 bytes) 2024-09-14T23:57:45.908 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:45.906+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:45.917 INFO:teuthology.orchestra.run.smithi042.stdout:4140 write 0xec9b8d9 thru 0xeca3726 (0x7e4e bytes) 2024-09-14T23:57:45.920 INFO:teuthology.orchestra.run.smithi042.stdout:4141 write 0x38f3d39 thru 0x38f4538 (0x800 bytes) 2024-09-14T23:57:45.922 INFO:teuthology.orchestra.run.smithi042.stdout:4143 read 0xc3df8c5 thru 0xc3eb724 (0xbe60 bytes) 2024-09-14T23:57:45.922 INFO:teuthology.orchestra.run.smithi042.stdout:4144 write 0xadb4837 thru 0xadb9599 (0x4d63 bytes) 2024-09-14T23:57:45.925 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:45.925 INFO:teuthology.orchestra.run.smithi042.stdout:4146 read 0xe00815d thru 0xe00c366 (0x420a bytes) 2024-09-14T23:57:45.925 INFO:teuthology.orchestra.run.smithi042.stdout:4148 punch from 0x6934e4c to 0x693e514, (0x96c8 bytes) 2024-09-14T23:57:45.925 INFO:teuthology.orchestra.run.smithi042.stdout:4149 write 0x9a3a0a4 thru 0x9a487ef (0xe74c bytes) 2024-09-14T23:57:45.926 INFO:teuthology.orchestra.run.smithi042.stdout:4151 writesame 0x91c0691 thru 0x91c9083 data_size 0x7(0x89f3 bytes) 2024-09-14T23:57:45.929 INFO:teuthology.orchestra.run.smithi042.stdout:4153 write 0xbd3d135 thru 0xbd45e80 (0x8d4c bytes) 2024-09-14T23:57:45.945 INFO:teuthology.orchestra.run.smithi042.stdout:4154 trunc from 0xeca3727 to 0x3c27727 2024-09-14T23:57:45.956 INFO:teuthology.orchestra.run.smithi042.stdout:4156 punch from 0x15b29ff to 0x15b632e, (0x392f bytes) 2024-09-14T23:57:45.958 INFO:teuthology.orchestra.run.smithi042.stdout:4159 trunc from 0x3c27727 to 0x4ea8a7f 2024-09-14T23:57:45.974 INFO:teuthology.orchestra.run.smithi042.stdout:4160 write 0x973ff97 thru 0x97436a5 (0x370f bytes) 2024-09-14T23:57:45.977 INFO:teuthology.orchestra.run.smithi042.stdout:4161 write 0x161fb15 thru 0x162aea0 (0xb38c bytes) 2024-09-14T23:57:45.978 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:45.979 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:45.988 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T23:57:45.989 INFO:teuthology.orchestra.run.smithi042.stdout:4162 write 0xc1743a7 thru 0xc17befb (0x7b55 bytes) 2024-09-14T23:57:45.991 INFO:teuthology.orchestra.run.smithi042.stdout:4164 writesame 0xaf2fe94 thru 0xaf32d84 data_size 0x1(0x2ef1 bytes) 2024-09-14T23:57:45.998 INFO:teuthology.orchestra.run.smithi042.stdout:4165 trunc from 0xc17befc to 0xe054ce1 2024-09-14T23:57:46.004 INFO:teuthology.orchestra.run.smithi042.stdout:4166 writesame 0x8388c37 thru 0x8396c9f data_size 0x1d(0xe069 bytes) 2024-09-14T23:57:46.006 INFO:teuthology.orchestra.run.smithi042.stdout:4167 punch from 0x1496b8f to 0x14a4f5c, (0xe3cd bytes) 2024-09-14T23:57:46.006 INFO:teuthology.orchestra.run.smithi042.stdout:4168 write 0x8195073 thru 0x819fbdb (0xab69 bytes) 2024-09-14T23:57:46.007 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:46.008 INFO:teuthology.orchestra.run.smithi042.stdout:4170 write 0xaae41b9 thru 0xaaf237c (0xe1c4 bytes) 2024-09-14T23:57:46.019 INFO:teuthology.orchestra.run.smithi042.stdout:4171 trunc from 0xe054ce1 to 0x516a3a7 2024-09-14T23:57:46.030 INFO:teuthology.orchestra.run.smithi042.stdout:4172 read 0x3d0eaa8 thru 0x3d0f09d (0x5f6 bytes) 2024-09-14T23:57:46.033 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:46.031+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:46.044 INFO:teuthology.orchestra.run.smithi042.stdout:4173 write 0xc4bd0d0 thru 0xc4beb0e (0x1a3f bytes) 2024-09-14T23:57:46.047 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is set 2024-09-14T23:57:46.049 INFO:teuthology.orchestra.run.smithi042.stdout:4174 write 0x9472778 thru 0x947b26d (0x8af6 bytes) 2024-09-14T23:57:46.054 INFO:teuthology.orchestra.run.smithi042.stdout:4175 read 0x43548ac thru 0x43573e5 (0x2b3a bytes) 2024-09-14T23:57:46.055 INFO:teuthology.orchestra.run.smithi042.stdout:4176 write 0x9f82f1a thru 0x9f847b2 (0x1899 bytes) 2024-09-14T23:57:46.068 INFO:teuthology.orchestra.run.smithi042.stdout:4177 trunc from 0xc4beb0f to 0x23cc207 2024-09-14T23:57:46.081 INFO:teuthology.orchestra.run.smithi042.stdout:4179 read 0xfcecc3 thru 0xfd1d42 (0x3080 bytes) 2024-09-14T23:57:46.098 INFO:teuthology.orchestra.run.smithi042.stdout:4181 writesame 0xc0ed97c thru 0xc0fba6a data_size 0x1(0xe0ef bytes) 2024-09-14T23:57:46.100 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:46.101 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:46.105 INFO:teuthology.orchestra.run.smithi042.stdout:4182 trunc from 0xc0fba6b to 0x6811a56 2024-09-14T23:57:46.107 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:46.105+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:46.109 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T23:57:46.124 INFO:teuthology.orchestra.run.smithi042.stdout:4183 writesame 0xd50fe6d thru 0xd51685a data_size 0x5b(0x69ee bytes) 2024-09-14T23:57:46.124 INFO:teuthology.orchestra.run.smithi042.stdout:4184 read 0x6f3ca26 thru 0x6f40135 (0x3710 bytes) 2024-09-14T23:57:46.124 INFO:teuthology.orchestra.run.smithi042.stdout:4185 writesame 0x65c6592 thru 0x65d4434 data_size 0x1(0xdea3 bytes) 2024-09-14T23:57:46.151 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:46.151 INFO:teuthology.orchestra.run.smithi042.stdout:4187 write 0xcb1707f thru 0xcb1893f (0x18c1 bytes) 2024-09-14T23:57:46.154 INFO:teuthology.orchestra.run.smithi042.stdout:4188 write 0x705d2fc thru 0x7060271 (0x2f76 bytes) 2024-09-14T23:57:46.155 INFO:teuthology.orchestra.run.smithi042.stdout:4189 read 0x4f35aa8 thru 0x4f3c40a (0x6963 bytes) 2024-09-14T23:57:46.156 INFO:teuthology.orchestra.run.smithi042.stdout:4190 read 0x106b007 thru 0x106bc9a (0xc94 bytes) 2024-09-14T23:57:46.156 INFO:teuthology.orchestra.run.smithi042.stdout:4192 write 0xcd7928 thru 0xce4b77 (0xd250 bytes) 2024-09-14T23:57:46.158 INFO:teuthology.orchestra.run.smithi042.stdout:4193 write 0x1dccc65 thru 0x1dcf94d (0x2ce9 bytes) 2024-09-14T23:57:46.160 INFO:teuthology.orchestra.run.smithi042.stdout:4194 read 0x3ef7148 thru 0x3f00b1d (0x99d6 bytes) 2024-09-14T23:57:46.160 INFO:teuthology.orchestra.run.smithi042.stdout:4195 write 0x9bea18a thru 0x9bf92ab (0xf122 bytes) 2024-09-14T23:57:46.162 INFO:teuthology.orchestra.run.smithi042.stdout:4196 writesame 0x9dbb0a7 thru 0x9dc8a25 data_size 0x1(0xd97f bytes) 2024-09-14T23:57:46.183 INFO:teuthology.orchestra.run.smithi042.stdout:4197 read 0x5b09139 thru 0x5b0b28c (0x2154 bytes) 2024-09-14T23:57:46.184 INFO:teuthology.orchestra.run.smithi042.stdout:4201 write 0x9e3a7c0 thru 0x9e3fcff (0x5540 bytes) 2024-09-14T23:57:46.185 INFO:teuthology.orchestra.run.smithi042.stdout:4202 read 0x9849afd thru 0x984dd01 (0x4205 bytes) 2024-09-14T23:57:46.186 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:46.186 INFO:teuthology.orchestra.run.smithi042.stdout:4205 write 0x3d9e739 thru 0x3da7e06 (0x96ce bytes) 2024-09-14T23:57:46.187 INFO:teuthology.orchestra.run.smithi042.stdout:4207 read 0x62e2ce0 thru 0x62f2a96 (0xfdb7 bytes) 2024-09-14T23:57:46.188 INFO:teuthology.orchestra.run.smithi042.stdout:4208 read 0x466ed6d thru 0x4679f44 (0xb1d8 bytes) 2024-09-14T23:57:46.188 INFO:teuthology.orchestra.run.smithi042.stdout:4209 write 0x25b3f75 thru 0x25b8c2a (0x4cb6 bytes) 2024-09-14T23:57:46.190 INFO:teuthology.orchestra.run.smithi042.stdout:4210 read 0x27ebe1e thru 0x27edb13 (0x1cf6 bytes) 2024-09-14T23:57:46.190 INFO:teuthology.orchestra.run.smithi042.stdout:4211 read 0x44115f5 thru 0x4417d95 (0x67a1 bytes) 2024-09-14T23:57:46.190 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:46.197 INFO:teuthology.orchestra.run.smithi042.stdout:4214 writesame 0xe046880 thru 0xe052f91 data_size 0x53(0xc712 bytes) 2024-09-14T23:57:46.197 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:46.197 INFO:teuthology.orchestra.run.smithi042.stdout:4216 write 0xcc5ff06 thru 0xcc69cb5 (0x9db0 bytes) 2024-09-14T23:57:46.205 INFO:teuthology.orchestra.run.smithi042.stdout:4217 write 0xe20d383 thru 0xe21ccfb (0xf979 bytes) 2024-09-14T23:57:46.206 INFO:teuthology.orchestra.run.smithi042.stdout:4218 punch from 0x4375d1a to 0x4380420, (0xa706 bytes) 2024-09-14T23:57:46.207 INFO:teuthology.orchestra.run.smithi042.stdout:4219 read 0x2e04999 thru 0x2e0aa1d (0x6085 bytes) 2024-09-14T23:57:46.207 INFO:teuthology.orchestra.run.smithi042.stdout:4221 writesame 0x6c84d96 thru 0x6c89ff1 data_size 0x2a(0x525c bytes) 2024-09-14T23:57:46.207 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:46.205+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:46.208 INFO:teuthology.orchestra.run.smithi042.stdout:4222 trunc from 0xe21ccfc to 0xe11787f 2024-09-14T23:57:46.215 INFO:teuthology.orchestra.run.smithi042.stdout:4223 write 0x2721fdd thru 0x272b8c6 (0x98ea bytes) 2024-09-14T23:57:46.217 INFO:teuthology.orchestra.run.smithi042.stdout:4224 write 0xa533a5c thru 0xa5348eb (0xe90 bytes) 2024-09-14T23:57:46.218 INFO:teuthology.orchestra.run.smithi042.stdout:4225 punch from 0x1f3de30 to 0x1f45ba9, (0x7d79 bytes) 2024-09-14T23:57:46.218 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:46.221 INFO:teuthology.orchestra.run.smithi042.stdout:4228 trunc from 0xe11787f to 0xb5e1d5b 2024-09-14T23:57:46.241 INFO:teuthology.orchestra.run.smithi042.stdout:4230 write 0xe272156 thru 0xe279074 (0x6f1f bytes) 2024-09-14T23:57:46.244 INFO:teuthology.orchestra.run.smithi042.stdout:4233 read 0xbf47be9 thru 0xbf4b261 (0x3679 bytes) 2024-09-14T23:57:46.244 INFO:teuthology.orchestra.run.smithi042.stdout:4234 read 0xbc499f2 thru 0xbc4c678 (0x2c87 bytes) 2024-09-14T23:57:46.244 INFO:teuthology.orchestra.run.smithi042.stdout:4235 writesame 0xb36904f thru 0xb371994 data_size 0x3(0x8946 bytes) 2024-09-14T23:57:46.262 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:46.262 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:46.264 INFO:teuthology.orchestra.run.smithi042.stdout:4236 trunc from 0xe279075 to 0x179abc3 2024-09-14T23:57:46.272 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T23:57:46.296 INFO:teuthology.orchestra.run.smithi042.stdout:4238 writesame 0xe61e827 thru 0xe62b65c data_size 0x5(0xce36 bytes) 2024-09-14T23:57:46.308 INFO:teuthology.orchestra.run.smithi042.stdout:4242 trunc from 0xe62b65d to 0x3bc0bc3 2024-09-14T23:57:46.313 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:46.311+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:46.323 INFO:teuthology.orchestra.run.smithi042.stdout:4243 write 0x5af5966 thru 0x5afac78 (0x5313 bytes) 2024-09-14T23:57:46.334 INFO:teuthology.orchestra.run.smithi042.stdout:4244 write 0x96fe901 thru 0x970532d (0x6a2d bytes) 2024-09-14T23:57:46.338 INFO:teuthology.orchestra.run.smithi042.stdout:4245 punch from 0x134549d to 0x1354b6e, (0xf6d1 bytes) 2024-09-14T23:57:46.338 INFO:teuthology.orchestra.run.smithi042.stdout:4247 write 0x48583b9 thru 0x4860935 (0x857d bytes) 2024-09-14T23:57:46.340 INFO:teuthology.orchestra.run.smithi042.stdout:4248 punch from 0xe102d to 0xeebeb, (0xdbbe bytes) 2024-09-14T23:57:46.340 INFO:teuthology.orchestra.run.smithi042.stdout:4250 read 0x8da3b6c thru 0x8db0e56 (0xd2eb bytes) 2024-09-14T23:57:46.340 INFO:teuthology.orchestra.run.smithi042.stdout:4251 read 0x64fb624 thru 0x65012a4 (0x5c81 bytes) 2024-09-14T23:57:46.340 INFO:teuthology.orchestra.run.smithi042.stdout:4256 read 0x5dd67bd thru 0x5ddc0f4 (0x5938 bytes) 2024-09-14T23:57:46.341 INFO:teuthology.orchestra.run.smithi042.stdout:4259 read 0x23b29d2 thru 0x23bf177 (0xc7a6 bytes) 2024-09-14T23:57:46.341 INFO:teuthology.orchestra.run.smithi042.stdout:4260 read 0x85a6d9c thru 0x85ad1ee (0x6453 bytes) 2024-09-14T23:57:46.341 INFO:teuthology.orchestra.run.smithi042.stdout:4261 read 0x927e07f thru 0x9286ee6 (0x8e68 bytes) 2024-09-14T23:57:46.341 INFO:teuthology.orchestra.run.smithi042.stdout:4262 clone 38 order 22 su 65536 sc 5 2024-09-14T23:57:46.409 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:46.407+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:46.421 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:46.421 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:46.430 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T23:57:46.514 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:46.512+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:46.578 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:46.578 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:46.587 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T23:57:46.611 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:46.610+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:46.712 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:46.711+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:46.735 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:46.735 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:46.744 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T23:57:46.813 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:46.811+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:46.851 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:46.851 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:46.859 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T23:57:46.943 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:46.941+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:47.007 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:47.007 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:47.016 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:47.013+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:47.016 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T23:57:47.116 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:47.115+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:47.149 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:47.150 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:47.160 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T23:57:47.193 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone37 from 0x970532e (overlap 0xa5c4b) to 0x95310 2024-09-14T23:57:47.218 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:47.216+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:47.278 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:47.278 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:47.287 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T23:57:47.328 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:47.326+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:47.383 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #36, image image_client.0-clone36 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent37 2024-09-14T23:57:47.408 INFO:teuthology.orchestra.run.smithi042.stdout:4263 read 0x2541d86 thru 0x2551258 (0xf4d3 bytes) 2024-09-14T23:57:47.418 INFO:teuthology.orchestra.run.smithi042.stdout:4264 write 0x9b10081 thru 0x9b12efa (0x2e7a bytes) 2024-09-14T23:57:47.420 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:47.418+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:47.424 INFO:teuthology.orchestra.run.smithi042.stdout:4265 punch from 0x2925e09 to 0x292ad64, (0x4f5b bytes) 2024-09-14T23:57:47.424 INFO:teuthology.orchestra.run.smithi042.stdout:4267 read 0x65af598 thru 0x65bbb69 (0xc5d2 bytes) 2024-09-14T23:57:47.424 INFO:teuthology.orchestra.run.smithi042.stdout:4269 read 0x4fc739f thru 0x4fcb506 (0x4168 bytes) 2024-09-14T23:57:47.424 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:47.434 INFO:teuthology.orchestra.run.smithi042.stdout:4271 write 0xd5e9806 thru 0xd5ea6ff (0xefa bytes) 2024-09-14T23:57:47.437 INFO:teuthology.orchestra.run.smithi042.stdout:4272 read 0x3ed47db thru 0x3ed5de3 (0x1609 bytes) 2024-09-14T23:57:47.437 INFO:teuthology.orchestra.run.smithi042.stdout:4273 write 0x8aca571 thru 0x8ad69fb (0xc48b bytes) 2024-09-14T23:57:47.446 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:47.446 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:47.458 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T23:57:47.459 INFO:teuthology.orchestra.run.smithi042.stdout:4275 trunc from 0xd5ea700 to 0x5cdca8e 2024-09-14T23:57:47.489 INFO:teuthology.orchestra.run.smithi042.stdout:4276 punch from 0x5632523 to 0x563a786, (0x8263 bytes) 2024-09-14T23:57:47.489 INFO:teuthology.orchestra.run.smithi042.stdout:4277 read 0x160a913 thru 0x1610d45 (0x6433 bytes) 2024-09-14T23:57:47.489 INFO:teuthology.orchestra.run.smithi042.stdout:4278 write 0x4f0308f thru 0x4f0f42e (0xc3a0 bytes) 2024-09-14T23:57:47.506 INFO:teuthology.orchestra.run.smithi042.stdout:4279 writesame 0x82b707d thru 0x82c31d6 data_size 0x1(0xc15a bytes) 2024-09-14T23:57:47.506 INFO:teuthology.orchestra.run.smithi042.stdout:4280 read 0x1fbaf35 thru 0x1fbc381 (0x144d bytes) 2024-09-14T23:57:47.506 INFO:teuthology.orchestra.run.smithi042.stdout:4281 read 0x198a91e thru 0x1992190 (0x7873 bytes) 2024-09-14T23:57:47.507 INFO:teuthology.orchestra.run.smithi042.stdout:4282 writesame 0x33e5dbd thru 0x33e6311 data_size 0x5(0x555 bytes) 2024-09-14T23:57:47.514 INFO:teuthology.orchestra.run.smithi042.stdout:4283 read 0x260e5a3 thru 0x261853e (0x9f9c bytes) 2024-09-14T23:57:47.514 INFO:teuthology.orchestra.run.smithi042.stdout:4284 writesame 0x1f3fe82 thru 0x1f4485c data_size 0x25(0x49db bytes) 2024-09-14T23:57:47.521 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:47.520+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:47.527 INFO:teuthology.orchestra.run.smithi042.stdout:4285 write 0x1ec1ff7 thru 0x1ecfa7f (0xda89 bytes) 2024-09-14T23:57:47.535 INFO:teuthology.orchestra.run.smithi042.stdout:4286 read 0x64b5dde thru 0x64c1e3c (0xc05f bytes) 2024-09-14T23:57:47.548 INFO:teuthology.orchestra.run.smithi042.stdout:4287 write 0xdbe3fad thru 0xdbe7ac7 (0x3b1b bytes) 2024-09-14T23:57:47.551 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:47.564 INFO:teuthology.orchestra.run.smithi042.stdout:4293 trunc from 0xdbe7ac8 to 0x28675ca 2024-09-14T23:57:47.592 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:47.593 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:47.602 INFO:teuthology.orchestra.run.smithi042.stdout:4294 write 0x3e721ef thru 0x3e7eb5d (0xc96f bytes) 2024-09-14T23:57:47.603 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T23:57:47.606 INFO:teuthology.orchestra.run.smithi042.stdout:4295 punch from 0x39ab1ef to 0x39b9e14, (0xec25 bytes) 2024-09-14T23:57:47.606 INFO:teuthology.orchestra.run.smithi042.stdout:4296 writesame 0x228e891 thru 0x2292d66 data_size 0x16(0x44d6 bytes) 2024-09-14T23:57:47.614 INFO:teuthology.orchestra.run.smithi042.stdout:4297 read 0x247c17f thru 0x2489f0e (0xdd90 bytes) 2024-09-14T23:57:47.622 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:47.620+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:47.623 INFO:teuthology.orchestra.run.smithi042.stdout:4298 write 0x70d60d3 thru 0x70e15d1 (0xb4ff bytes) 2024-09-14T23:57:47.642 INFO:teuthology.orchestra.run.smithi042.stdout:4299 write 0xecb33e3 thru 0xecb8a00 (0x561e bytes) 2024-09-14T23:57:47.645 INFO:teuthology.orchestra.run.smithi042.stdout:4300 writesame 0x15a7763 thru 0x15b0416 data_size 0xa(0x8cb4 bytes) 2024-09-14T23:57:47.670 INFO:teuthology.orchestra.run.smithi042.stdout:4301 trunc from 0xecb8a01 to 0xb55ddb 2024-09-14T23:57:47.705 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:47.705 INFO:teuthology.orchestra.run.smithi042.stdout:4303 read 0x2b6806 thru 0x2ba67c (0x3e77 bytes) 2024-09-14T23:57:47.720 INFO:teuthology.orchestra.run.smithi042.stdout:4304 write 0x947c99a thru 0x9481e2a (0x5491 bytes) 2024-09-14T23:57:47.723 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:47.721+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:47.724 INFO:teuthology.orchestra.run.smithi042.stdout:4305 punch from 0x6730891 to 0x673a5a3, (0x9d12 bytes) 2024-09-14T23:57:47.734 INFO:teuthology.orchestra.run.smithi042.stdout:4307 writesame 0xe30a862 thru 0xe313a86 data_size 0x9(0x9225 bytes) 2024-09-14T23:57:47.735 INFO:teuthology.orchestra.run.smithi042.stdout:4308 writesame 0x6738e7e thru 0x6747f3b data_size 0x5(0xf0be bytes) 2024-09-14T23:57:47.736 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:47.736 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:47.748 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T23:57:47.755 INFO:teuthology.orchestra.run.smithi042.stdout:4309 trunc from 0xe313a87 to 0x240118b 2024-09-14T23:57:47.773 INFO:teuthology.orchestra.run.smithi042.stdout:4310 trunc from 0x240118b to 0x938050e 2024-09-14T23:57:47.780 INFO:teuthology.orchestra.run.smithi042.stdout:4313 writesame 0x5a46657 thru 0x5a479d0 data_size 0x9(0x137a bytes) 2024-09-14T23:57:47.784 INFO:teuthology.orchestra.run.smithi042.stdout:4314 trunc from 0x938050e to 0xa7310c0 2024-09-14T23:57:47.790 INFO:teuthology.orchestra.run.smithi042.stdout:4315 punch from 0x24228f6 to 0x2432599, (0xfca3 bytes) 2024-09-14T23:57:47.790 INFO:teuthology.orchestra.run.smithi042.stdout:4317 write 0xa3e620 thru 0xa40d4b (0x272c bytes) 2024-09-14T23:57:47.792 INFO:teuthology.orchestra.run.smithi042.stdout:4318 write 0x73bf14f thru 0x73cee06 (0xfcb8 bytes) 2024-09-14T23:57:47.804 INFO:teuthology.orchestra.run.smithi042.stdout:4322 write 0xcfb085a thru 0xcfbbc45 (0xb3ec bytes) 2024-09-14T23:57:47.807 INFO:teuthology.orchestra.run.smithi042.stdout:4324 punch from 0x512dc36 to 0x5138fd7, (0xb3a1 bytes) 2024-09-14T23:57:47.807 INFO:teuthology.orchestra.run.smithi042.stdout:4325 read 0x5f7a16a thru 0x5f826f0 (0x8587 bytes) 2024-09-14T23:57:47.807 INFO:teuthology.orchestra.run.smithi042.stdout:4326 write 0x128f676 thru 0x1293f7f (0x490a bytes) 2024-09-14T23:57:47.809 INFO:teuthology.orchestra.run.smithi042.stdout:4327 read 0x6dda87 thru 0x6e60e2 (0x865c bytes) 2024-09-14T23:57:47.809 INFO:teuthology.orchestra.run.smithi042.stdout:4329 read 0xb0dd6cd thru 0xb0df303 (0x1c37 bytes) 2024-09-14T23:57:47.809 INFO:teuthology.orchestra.run.smithi042.stdout:4330 read 0x6b3efc0 thru 0x6b46214 (0x7255 bytes) 2024-09-14T23:57:47.815 INFO:teuthology.orchestra.run.smithi042.stdout:4331 writesame 0xd038a57 thru 0xd03b114 data_size 0x26(0x26be bytes) 2024-09-14T23:57:47.815 INFO:teuthology.orchestra.run.smithi042.stdout:4332 punch from 0x953c923 to 0x9544f12, (0x85ef bytes) 2024-09-14T23:57:47.815 INFO:teuthology.orchestra.run.smithi042.stdout:4333 writesame 0xc940cbb thru 0xc942baa data_size 0x24(0x1ef0 bytes) 2024-09-14T23:57:47.818 INFO:teuthology.orchestra.run.smithi042.stdout:4334 write 0xc725001 thru 0xc7334e3 (0xe4e3 bytes) 2024-09-14T23:57:47.821 INFO:teuthology.orchestra.run.smithi042.stdout:4336 read 0x5bed359 thru 0x5bfca71 (0xf719 bytes) 2024-09-14T23:57:47.825 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:47.823+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:47.828 INFO:teuthology.orchestra.run.smithi042.stdout:4337 trunc from 0xd03b115 to 0x731ebfb 2024-09-14T23:57:47.839 INFO:teuthology.orchestra.run.smithi042.stdout:4339 write 0x110f1bd thru 0x111d3e5 (0xe229 bytes) 2024-09-14T23:57:47.848 INFO:teuthology.orchestra.run.smithi042.stdout:4340 writesame 0x9733d7f thru 0x9734aff data_size 0x1(0xd81 bytes) 2024-09-14T23:57:47.848 INFO:teuthology.orchestra.run.smithi042.stdout:4341 read 0x5cbc963 thru 0x5cc5ad0 (0x916e bytes) 2024-09-14T23:57:47.848 INFO:teuthology.orchestra.run.smithi042.stdout:4344 write 0x9217a5a thru 0x921d354 (0x58fb bytes) 2024-09-14T23:57:47.851 INFO:teuthology.orchestra.run.smithi042.stdout:4345 read 0x60cd947 thru 0x60cf488 (0x1b42 bytes) 2024-09-14T23:57:47.851 INFO:teuthology.orchestra.run.smithi042.stdout:4347 writesame 0x6ae3e62 thru 0x6af16fd data_size 0x6(0xd89c bytes) 2024-09-14T23:57:47.866 INFO:teuthology.orchestra.run.smithi042.stdout:4348 writesame 0xbc03afc thru 0xbc115ca data_size 0x5(0xdacf bytes) 2024-09-14T23:57:47.867 INFO:teuthology.orchestra.run.smithi042.stdout:4349 trunc from 0xbc115cb to 0xa76fad5 2024-09-14T23:57:47.873 INFO:teuthology.orchestra.run.smithi042.stdout:4350 read 0x49a9e4c thru 0x49b3805 (0x99ba bytes) 2024-09-14T23:57:47.873 INFO:teuthology.orchestra.run.smithi042.stdout:4351 read 0x162cda5 thru 0x162f04b (0x22a7 bytes) 2024-09-14T23:57:47.873 INFO:teuthology.orchestra.run.smithi042.stdout:4352 read 0x5d7465e thru 0x5d7b31f (0x6cc2 bytes) 2024-09-14T23:57:47.873 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:47.875 INFO:teuthology.orchestra.run.smithi042.stdout:4356 trunc from 0xa76fad5 to 0x7b9418b 2024-09-14T23:57:47.880 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:47.881 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:47.885 INFO:teuthology.orchestra.run.smithi042.stdout:4359 read 0x6d5a111 thru 0x6d5cce9 (0x2bd9 bytes) 2024-09-14T23:57:47.886 INFO:teuthology.orchestra.run.smithi042.stdout:4360 trunc from 0x7b9418b to 0x6991bb5 2024-09-14T23:57:47.892 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T23:57:47.893 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T23:57:47.903 INFO:teuthology.orchestra.run.smithi042.stdout:4361 write 0xa9be092 thru 0xa9c7281 (0x91f0 bytes) 2024-09-14T23:57:47.907 INFO:teuthology.orchestra.run.smithi042.stdout:4363 punch from 0x7a5ca1f to 0x7a678b2, (0xae93 bytes) 2024-09-14T23:57:47.907 INFO:teuthology.orchestra.run.smithi042.stdout:4365 punch from 0xa3ded7b to 0xa3e44b9, (0x573e bytes) 2024-09-14T23:57:47.907 INFO:teuthology.orchestra.run.smithi042.stdout:4369 read 0x55e9fc8 thru 0x55f3b88 (0x9bc1 bytes) 2024-09-14T23:57:47.907 INFO:teuthology.orchestra.run.smithi042.stdout:4370 punch from 0x52c20d5 to 0x52c9baa, (0x7ad5 bytes) 2024-09-14T23:57:47.907 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:47.907 INFO:teuthology.orchestra.run.smithi042.stdout:4372 write 0x4b85dbd thru 0x4b8b80c (0x5a50 bytes) 2024-09-14T23:57:47.911 INFO:teuthology.orchestra.run.smithi042.stdout:4373 write 0x3190b thru 0x3d6b7 (0xbdad bytes) 2024-09-14T23:57:47.913 INFO:teuthology.orchestra.run.smithi042.stdout:4374 write 0x86e2f23 thru 0x86e4653 (0x1731 bytes) 2024-09-14T23:57:47.921 INFO:teuthology.orchestra.run.smithi042.stdout:4375 write 0xaa25f6b thru 0xaa2c4f0 (0x6586 bytes) 2024-09-14T23:57:47.925 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:47.923+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:47.934 INFO:teuthology.orchestra.run.smithi042.stdout:4376 writesame 0xe1efb79 thru 0xe1fabfb data_size 0x1(0xb083 bytes) 2024-09-14T23:57:47.934 INFO:teuthology.orchestra.run.smithi042.stdout:4378 read 0xdd60133 thru 0xdd6c2c5 (0xc193 bytes) 2024-09-14T23:57:47.934 INFO:teuthology.orchestra.run.smithi042.stdout:4379 read 0x6ff7939 thru 0x7002452 (0xab1a bytes) 2024-09-14T23:57:47.950 INFO:teuthology.orchestra.run.smithi042.stdout:4380 trunc from 0xe1fabfc to 0xd3e5a0 2024-09-14T23:57:47.963 INFO:teuthology.orchestra.run.smithi042.stdout:4382 punch from 0x15be1d to 0x16279f, (0x6982 bytes) 2024-09-14T23:57:47.975 INFO:teuthology.orchestra.run.smithi042.stdout:4383 trunc from 0xd3e5a0 to 0xb062765 2024-09-14T23:57:47.988 INFO:teuthology.orchestra.run.smithi042.stdout:4384 trunc from 0xb062765 to 0x4e6f396 2024-09-14T23:57:47.993 INFO:teuthology.orchestra.run.smithi042.stdout:4388 write 0x476851e thru 0x476d6c8 (0x51ab bytes) 2024-09-14T23:57:48.001 INFO:teuthology.orchestra.run.smithi042.stdout:4389 trunc from 0x4e6f396 to 0xaa098a 2024-09-14T23:57:48.012 INFO:teuthology.orchestra.run.smithi042.stdout:4390 read 0x53fdd0 thru 0x54d7fe (0xda2f bytes) 2024-09-14T23:57:48.023 INFO:teuthology.orchestra.run.smithi042.stdout:4391 write 0x4bd46c6 thru 0x4be33d6 (0xed11 bytes) 2024-09-14T23:57:48.027 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:48.025+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:48.027 INFO:teuthology.orchestra.run.smithi042.stdout:4392 read 0x2613e0b thru 0x2622aa9 (0xec9f bytes) 2024-09-14T23:57:48.030 INFO:teuthology.orchestra.run.smithi042.stdout:4393 trunc from 0x4be33d7 to 0x77ccbb3 2024-09-14T23:57:48.033 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:48.033 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:48.036 INFO:teuthology.orchestra.run.smithi042.stdout:4394 punch from 0x15d4288 to 0x15ddd64, (0x9adc bytes) 2024-09-14T23:57:48.046 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T23:57:48.052 INFO:teuthology.orchestra.run.smithi042.stdout:4395 write 0xee47427 thru 0xee49267 (0x1e41 bytes) 2024-09-14T23:57:48.055 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:48.055 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:48.055 INFO:teuthology.orchestra.run.smithi042.stdout:4399 write 0x8f873ee thru 0x8f92840 (0xb453 bytes) 2024-09-14T23:57:48.058 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T23:57:48.058 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T23:57:48.058 INFO:tasks.thrashosds.thrasher:inject_pause on osd.6 2024-09-14T23:57:48.058 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-14T23:57:48.058 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-14T23:57:48.059 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok config set filestore_inject_stall 3 2024-09-14T23:57:48.059 INFO:teuthology.orchestra.run.smithi042.stdout:4400 read 0xd0fbc33 thru 0xd0fe2fd (0x26cb bytes) 2024-09-14T23:57:48.059 INFO:teuthology.orchestra.run.smithi042.stdout:4401 writesame 0xceeb1c1 thru 0xcef48b6 data_size 0x39(0x96f6 bytes) 2024-09-14T23:57:48.065 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:57:48.065 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:57:48.066 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T23:57:48.066 INFO:teuthology.orchestra.run.smithi042.stdout:4402 trunc from 0xee49268 to 0xd09c4d7 2024-09-14T23:57:48.084 INFO:teuthology.orchestra.run.smithi042.stdout:4403 trunc from 0xd09c4d7 to 0x97c9509 2024-09-14T23:57:48.089 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:57:48.089 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:57:48.097 INFO:teuthology.orchestra.run.smithi042.stdout:4404 read 0x92dcd35 thru 0x92e8e0b (0xc0d7 bytes) 2024-09-14T23:57:48.104 INFO:teuthology.orchestra.run.smithi042.stdout:4405 trunc from 0x97c9509 to 0x40a7e6a 2024-09-14T23:57:48.111 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:57:48.111 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:57:48.116 INFO:teuthology.orchestra.run.smithi042.stdout:4406 read 0x25417c0 thru 0x255028a (0xeacb bytes) 2024-09-14T23:57:48.128 INFO:teuthology.orchestra.run.smithi042.stdout:4408 write 0x8f5b492 thru 0x8f5ea18 (0x3587 bytes) 2024-09-14T23:57:48.128 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:48.126+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:48.130 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:57:48.130 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:57:48.132 INFO:teuthology.orchestra.run.smithi042.stdout:4409 read 0x27b8c7b thru 0x27c58e2 (0xcc68 bytes) 2024-09-14T23:57:48.132 INFO:teuthology.orchestra.run.smithi042.stdout:4410 punch from 0x4a788f2 to 0x4a7f4dd, (0x6beb bytes) 2024-09-14T23:57:48.132 INFO:teuthology.orchestra.run.smithi042.stdout:4411 read 0x100eb84 thru 0x100f47c (0x8f9 bytes) 2024-09-14T23:57:48.138 INFO:teuthology.orchestra.run.smithi042.stdout:4412 trunc from 0x8f5ea19 to 0xd32a077 2024-09-14T23:57:48.144 INFO:teuthology.orchestra.run.smithi042.stdout:4413 trunc from 0xd32a077 to 0xcbc10cb 2024-09-14T23:57:48.150 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:48.150 INFO:teuthology.orchestra.run.smithi042.stdout:4416 writesame 0x8a00fe5 thru 0x8a0fc1d data_size 0x35(0xec39 bytes) 2024-09-14T23:57:48.154 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:57:48.154 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:57:48.169 INFO:teuthology.orchestra.run.smithi042.stdout:4417 trunc from 0xcbc10cb to 0xe9d7a7 2024-09-14T23:57:48.170 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:48.170 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-14T23:57:48.171 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:48.173 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:57:48.173 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:57:48.181 INFO:teuthology.orchestra.run.smithi042.stdout:4418 read 0x644ed2 thru 0x64b473 (0x65a2 bytes) 2024-09-14T23:57:48.184 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:48.184 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:57:48.185 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:57:48.185 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:57:48.185 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:48.188 INFO:teuthology.orchestra.run.smithi042.stdout:4419 writesame 0x1f6602e thru 0x1f6fdfc data_size 0x1(0x9dcf bytes) 2024-09-14T23:57:48.188 INFO:teuthology.orchestra.run.smithi042.stdout:4420 punch from 0xf6f43f to 0xf76e48, (0x7a09 bytes) 2024-09-14T23:57:48.188 INFO:teuthology.orchestra.run.smithi042.stdout:4421 read 0x5d4305 thru 0x5dcf5e (0x8c5a bytes) 2024-09-14T23:57:48.188 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:48.190 INFO:teuthology.orchestra.run.smithi042.stdout:4423 trunc from 0x1f6fdfd to 0x3c0febc 2024-09-14T23:57:48.195 INFO:teuthology.orchestra.run.smithi042.stdout:4424 punch from 0x1bf5d2 to 0x1cea24, (0xf452 bytes) 2024-09-14T23:57:48.195 INFO:teuthology.orchestra.run.smithi042.stdout:4425 read 0x5aa9fc thru 0x5b0148 (0x574d bytes) 2024-09-14T23:57:48.197 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T23:57:48.198 INFO:teuthology.orchestra.run.smithi042.stdout:4426 trunc from 0x3c0febc to 0x1d7a225 2024-09-14T23:57:48.207 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:57:48.207 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:57:48.219 INFO:teuthology.orchestra.run.smithi042.stdout:4427 writesame 0x9989a49 thru 0x998cb9f data_size 0x1(0x3157 bytes) 2024-09-14T23:57:48.219 INFO:teuthology.orchestra.run.smithi042.stdout:4428 read 0x74b355 thru 0x75616d (0xae19 bytes) 2024-09-14T23:57:48.219 INFO:teuthology.orchestra.run.smithi042.stdout:4429 read 0x5488023 thru 0x5496abe (0xea9c bytes) 2024-09-14T23:57:48.219 INFO:teuthology.orchestra.run.smithi042.stdout:4430 write 0x4c4c80 thru 0x4cb3ab (0x672c bytes) 2024-09-14T23:57:48.221 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:48.228 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:57:48.229 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:57:48.229 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:48.227+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:48.231 INFO:teuthology.orchestra.run.smithi042.stdout:4432 write 0xbaba521 thru 0xbac72da (0xcdba bytes) 2024-09-14T23:57:48.236 INFO:teuthology.orchestra.run.smithi042.stdout:4433 read 0x16afd08 thru 0x16b6acb (0x6dc4 bytes) 2024-09-14T23:57:48.236 INFO:teuthology.orchestra.run.smithi042.stdout:4434 punch from 0x41a8d00 to 0x41b6d13, (0xe013 bytes) 2024-09-14T23:57:48.236 INFO:teuthology.orchestra.run.smithi042.stdout:4435 punch from 0xb8cef4c to 0xb8d0d3f, (0x1df3 bytes) 2024-09-14T23:57:48.236 INFO:teuthology.orchestra.run.smithi042.stdout:4436 write 0xa27244b thru 0xa278d67 (0x691d bytes) 2024-09-14T23:57:48.241 INFO:teuthology.orchestra.run.smithi042.stdout:4437 trunc from 0xbac72db to 0xa77db40 2024-09-14T23:57:48.253 INFO:teuthology.orchestra.run.smithi042.stdout:4438 write 0x1ee8d93 thru 0x1eec481 (0x36ef bytes) 2024-09-14T23:57:48.266 INFO:teuthology.orchestra.run.smithi042.stdout:4439 trunc from 0xa77db40 to 0x2df6ba0 2024-09-14T23:57:48.277 INFO:teuthology.orchestra.run.smithi042.stdout:4441 punch from 0xdfa145 to 0xe038c9, (0x9784 bytes) 2024-09-14T23:57:48.277 INFO:teuthology.orchestra.run.smithi042.stdout:4443 read 0x182d806 thru 0x182ee0f (0x160a bytes) 2024-09-14T23:57:48.277 INFO:teuthology.orchestra.run.smithi042.stdout:4444 writesame 0x903760 thru 0x90e716 data_size 0x1(0xafb7 bytes) 2024-09-14T23:57:48.303 INFO:teuthology.orchestra.run.smithi042.stdout:4447 writesame 0xe492ac thru 0xe4e5b5 data_size 0x9(0x530a bytes) 2024-09-14T23:57:48.305 INFO:teuthology.orchestra.run.smithi042.stdout:4450 clone 39 order 21 su 65536 sc 14 2024-09-14T23:57:48.328 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:48.328 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:57:48.328 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:57:48.328 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:57:48.328 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.328 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:3578 3.4 e29/23 3:2298d860:::rbd_data.1118de1168b.0000000000000030:head v 29'15, mlcod=29'13)", 2024-09-14T23:57:48.329 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.036455+0000", 2024-09-14T23:57:48.329 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 69.289447658, 2024-09-14T23:57:48.329 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019449440000000001, 2024-09-14T23:57:48.329 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:48.329 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:48.329 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:48.329 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:48.329 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.329 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:48.329 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036455+0000", 2024-09-14T23:57:48.329 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.329 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.330 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:48.330 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036455+0000", 2024-09-14T23:57:48.330 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.330 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.330 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:48.330 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036456+0000", 2024-09-14T23:57:48.330 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0999999999999997e-07 2024-09-14T23:57:48.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.330 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.330 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:48.330 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036463+0000", 2024-09-14T23:57:48.330 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2200000000000003e-06 2024-09-14T23:57:48.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.331 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.331 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:48.331 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036478+0000", 2024-09-14T23:57:48.331 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4977000000000001e-05 2024-09-14T23:57:48.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.331 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.331 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:48.331 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036485+0000", 2024-09-14T23:57:48.331 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0559999999999999e-06 2024-09-14T23:57:48.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.331 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.331 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:48.331 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036693+0000", 2024-09-14T23:57:48.331 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000208051 2024-09-14T23:57:48.332 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.332 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.332 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:48.332 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036715+0000", 2024-09-14T23:57:48.332 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1330000000000001e-05 2024-09-14T23:57:48.332 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.332 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.332 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:48.332 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.038356+0000", 2024-09-14T23:57:48.332 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001641586 2024-09-14T23:57:48.332 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.332 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.333 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:48.333 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.038400+0000", 2024-09-14T23:57:48.333 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3914e-05 2024-09-14T23:57:48.333 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.333 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:48.333 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.333 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.333 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:3579 3.4 e29/23 3:27290fe6:::rbd_data.1118de1168b.0000000000000032:head v 29'16, mlcod=29'13)", 2024-09-14T23:57:48.333 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.037085+0000", 2024-09-14T23:57:48.333 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 69.288818067999998, 2024-09-14T23:57:48.333 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0020019220000000002, 2024-09-14T23:57:48.333 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:48.333 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:48.334 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:48.334 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:48.334 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.334 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:48.334 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037085+0000", 2024-09-14T23:57:48.334 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.334 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.334 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.334 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:48.334 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037085+0000", 2024-09-14T23:57:48.334 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.334 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.334 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.334 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:48.335 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037085+0000", 2024-09-14T23:57:48.335 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5499999999999997e-07 2024-09-14T23:57:48.335 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.335 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.335 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:48.335 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037091+0000", 2024-09-14T23:57:48.335 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7869999999999998e-06 2024-09-14T23:57:48.335 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.335 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.335 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:48.335 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037102+0000", 2024-09-14T23:57:48.335 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.031e-05 2024-09-14T23:57:48.335 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.336 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:48.336 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037116+0000", 2024-09-14T23:57:48.336 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4119e-05 2024-09-14T23:57:48.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.336 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:48.336 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037219+0000", 2024-09-14T23:57:48.336 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010372799999999999 2024-09-14T23:57:48.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.336 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:48.336 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037239+0000", 2024-09-14T23:57:48.337 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9993e-05 2024-09-14T23:57:48.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.337 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.337 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:48.337 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.039046+0000", 2024-09-14T23:57:48.337 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001806666 2024-09-14T23:57:48.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.337 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.337 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:48.337 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.039087+0000", 2024-09-14T23:57:48.337 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0564e-05 2024-09-14T23:57:48.337 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.337 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:48.337 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.338 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.338 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.338 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:32024 3.9 e39/23 3:94dc260b:::rbd_data.111814b482f8.0000000000000009:head v 39'45, mlcod=38'43)", 2024-09-14T23:57:48.338 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:49.061018+0000", 2024-09-14T23:57:48.338 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 59.264884729000002, 2024-09-14T23:57:48.338 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018482240000000001, 2024-09-14T23:57:48.338 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:48.338 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:48.338 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:48.338 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:48.338 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.338 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:48.338 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061018+0000", 2024-09-14T23:57:48.338 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.339 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.339 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:48.339 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061018+0000", 2024-09-14T23:57:48.339 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.339 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.339 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:48.339 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061018+0000", 2024-09-14T23:57:48.339 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9999999999999998e-07 2024-09-14T23:57:48.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.339 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.339 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:48.339 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061211+0000", 2024-09-14T23:57:48.340 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019206 2024-09-14T23:57:48.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.340 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.340 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:48.340 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061221+0000", 2024-09-14T23:57:48.340 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0659000000000001e-05 2024-09-14T23:57:48.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.340 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.340 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:48.340 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061230+0000", 2024-09-14T23:57:48.340 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4610000000000008e-06 2024-09-14T23:57:48.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.340 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.341 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:48.341 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061317+0000", 2024-09-14T23:57:48.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7514000000000003e-05 2024-09-14T23:57:48.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.341 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.341 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:48.341 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061338+0000", 2024-09-14T23:57:48.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0395999999999999e-05 2024-09-14T23:57:48.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.341 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.341 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:48.341 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.062830+0000", 2024-09-14T23:57:48.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0014921940000000001 2024-09-14T23:57:48.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.342 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.342 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:48.342 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.062866+0000", 2024-09-14T23:57:48.342 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6539999999999999e-05 2024-09-14T23:57:48.342 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.342 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:48.342 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.342 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.342 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:32420 3.9 e39/23 3:91dc34e7:::rbd_data.111814b482f8.000000000000001e:head v 39'46, mlcod=38'44)", 2024-09-14T23:57:48.342 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:49.086124+0000", 2024-09-14T23:57:48.342 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 59.239779230000003, 2024-09-14T23:57:48.342 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001686819, 2024-09-14T23:57:48.342 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:48.343 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:48.343 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:48.343 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:48.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.343 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:48.343 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086124+0000", 2024-09-14T23:57:48.343 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.343 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:48.343 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086124+0000", 2024-09-14T23:57:48.343 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.344 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:48.344 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086125+0000", 2024-09-14T23:57:48.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.361e-06 2024-09-14T23:57:48.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.344 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:48.344 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086605+0000", 2024-09-14T23:57:48.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00048026600000000003 2024-09-14T23:57:48.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.344 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:48.344 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086626+0000", 2024-09-14T23:57:48.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.037e-05 2024-09-14T23:57:48.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:48.345 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086635+0000", 2024-09-14T23:57:48.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5880000000000004e-06 2024-09-14T23:57:48.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:48.345 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086701+0000", 2024-09-14T23:57:48.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5905e-05 2024-09-14T23:57:48.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:48.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086727+0000", 2024-09-14T23:57:48.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5573000000000001e-05 2024-09-14T23:57:48.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.346 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:48.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.087767+0000", 2024-09-14T23:57:48.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001040537 2024-09-14T23:57:48.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.346 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:48.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.087810+0000", 2024-09-14T23:57:48.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3219000000000001e-05 2024-09-14T23:57:48.346 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.347 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:48.347 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.347 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:34296 3.9 e39/23 3:974a38c7:::rbd_data.111814b482f8.000000000000003a:head v 39'47, mlcod=39'45)", 2024-09-14T23:57:48.347 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:49.192105+0000", 2024-09-14T23:57:48.347 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 59.133797346999998, 2024-09-14T23:57:48.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001607516, 2024-09-14T23:57:48.347 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:48.347 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:48.347 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:48.347 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:48.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.347 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:48.348 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.192105+0000", 2024-09-14T23:57:48.348 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.348 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:48.348 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.192105+0000", 2024-09-14T23:57:48.348 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.348 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:48.348 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.192107+0000", 2024-09-14T23:57:48.348 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3549999999999999e-06 2024-09-14T23:57:48.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.349 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:48.349 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.192332+0000", 2024-09-14T23:57:48.349 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022486300000000001 2024-09-14T23:57:48.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.349 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:48.349 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.192349+0000", 2024-09-14T23:57:48.349 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6966e-05 2024-09-14T23:57:48.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.349 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:48.349 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.192358+0000", 2024-09-14T23:57:48.349 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8309999999999996e-06 2024-09-14T23:57:48.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.350 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:48.350 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.192422+0000", 2024-09-14T23:57:48.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3915e-05 2024-09-14T23:57:48.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.350 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:48.350 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.192450+0000", 2024-09-14T23:57:48.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7957e-05 2024-09-14T23:57:48.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.350 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:48.350 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.193662+0000", 2024-09-14T23:57:48.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012113270000000001 2024-09-14T23:57:48.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.351 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:48.351 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.193713+0000", 2024-09-14T23:57:48.351 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1301999999999998e-05 2024-09-14T23:57:48.351 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.351 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:48.351 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.351 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:50582 3.9 e51/23 3:92198e7a:::rbd_data.11183b9aac34.000000000000000a:head v 51'63, mlcod=50'60)", 2024-09-14T23:57:48.351 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:01.131541+0000", 2024-09-14T23:57:48.351 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 47.194362038999998, 2024-09-14T23:57:48.351 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001795547, 2024-09-14T23:57:48.351 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:48.351 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:48.351 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:48.352 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:48.352 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.352 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:48.352 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131541+0000", 2024-09-14T23:57:48.352 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.352 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.352 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:48.352 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131541+0000", 2024-09-14T23:57:48.352 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.352 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.352 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:48.352 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131541+0000", 2024-09-14T23:57:48.353 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7099999999999997e-07 2024-09-14T23:57:48.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:48.353 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131548+0000", 2024-09-14T23:57:48.353 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.708e-06 2024-09-14T23:57:48.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:48.353 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131570+0000", 2024-09-14T23:57:48.353 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1815999999999999e-05 2024-09-14T23:57:48.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.354 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:48.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131581+0000", 2024-09-14T23:57:48.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0933e-05 2024-09-14T23:57:48.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.354 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:48.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131660+0000", 2024-09-14T23:57:48.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9071000000000002e-05 2024-09-14T23:57:48.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.354 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:48.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131682+0000", 2024-09-14T23:57:48.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2534000000000001e-05 2024-09-14T23:57:48.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.355 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.355 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:48.355 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.133287+0000", 2024-09-14T23:57:48.355 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016045510000000001 2024-09-14T23:57:48.355 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.355 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.355 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:48.355 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.133336+0000", 2024-09-14T23:57:48.355 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9563e-05 2024-09-14T23:57:48.355 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.355 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:48.355 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.355 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.356 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.356 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:51182 3.9 e52/23 3:918e4fa0:::rbd_data.11181975a1de.000000000000007b:head v 52'74, mlcod=51'71)", 2024-09-14T23:57:48.356 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:02.138725+0000", 2024-09-14T23:57:48.356 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.187178302, 2024-09-14T23:57:48.356 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015321009999999999, 2024-09-14T23:57:48.356 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:48.356 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:48.356 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:48.356 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:48.356 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.356 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:48.356 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.138725+0000", 2024-09-14T23:57:48.356 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.356 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.357 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.357 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:48.357 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.138725+0000", 2024-09-14T23:57:48.357 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.357 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.357 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.357 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:48.357 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.138725+0000", 2024-09-14T23:57:48.357 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.22e-07 2024-09-14T23:57:48.357 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.357 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.357 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:48.357 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.138732+0000", 2024-09-14T23:57:48.358 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1779999999999997e-06 2024-09-14T23:57:48.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.358 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:48.358 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.138749+0000", 2024-09-14T23:57:48.358 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6500000000000001e-05 2024-09-14T23:57:48.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.358 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:48.358 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.138758+0000", 2024-09-14T23:57:48.359 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6239999999999997e-06 2024-09-14T23:57:48.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.359 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.359 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:48.359 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.138816+0000", 2024-09-14T23:57:48.359 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7939000000000002e-05 2024-09-14T23:57:48.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.359 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.359 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:48.359 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.138840+0000", 2024-09-14T23:57:48.359 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3856e-05 2024-09-14T23:57:48.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.359 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.360 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:48.360 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.140204+0000", 2024-09-14T23:57:48.360 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013641339999999999 2024-09-14T23:57:48.360 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.360 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.360 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:48.360 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.140257+0000", 2024-09-14T23:57:48.360 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2547999999999999e-05 2024-09-14T23:57:48.360 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.360 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:48.360 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.360 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.361 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.361 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:51281 3.0 e52/23 3:04e60ec2:::rbd_data.11181975a1de.0000000000000003:head v 52'113, mlcod=44'111)", 2024-09-14T23:57:48.361 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:02.158822+0000", 2024-09-14T23:57:48.361 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.167080542999997, 2024-09-14T23:57:48.361 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019860609999999999, 2024-09-14T23:57:48.361 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:48.361 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:48.361 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:48.361 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:48.361 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.361 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:48.361 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.158822+0000", 2024-09-14T23:57:48.361 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.361 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.362 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.362 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:48.362 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.158822+0000", 2024-09-14T23:57:48.362 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.362 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.362 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.362 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:48.362 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.158823+0000", 2024-09-14T23:57:48.362 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6800000000000001e-07 2024-09-14T23:57:48.362 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.362 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.362 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:48.362 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.159132+0000", 2024-09-14T23:57:48.363 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030974900000000001 2024-09-14T23:57:48.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.363 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.363 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:48.363 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.159146+0000", 2024-09-14T23:57:48.363 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3261000000000001e-05 2024-09-14T23:57:48.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.363 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.363 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:48.363 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.159155+0000", 2024-09-14T23:57:48.363 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7410000000000004e-06 2024-09-14T23:57:48.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.363 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.363 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:48.364 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.159221+0000", 2024-09-14T23:57:48.364 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5698999999999996e-05 2024-09-14T23:57:48.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.364 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.364 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:48.364 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.159244+0000", 2024-09-14T23:57:48.364 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3108e-05 2024-09-14T23:57:48.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.364 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.364 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:48.364 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.160767+0000", 2024-09-14T23:57:48.364 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015231120000000001 2024-09-14T23:57:48.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.365 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.365 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:48.365 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.160808+0000", 2024-09-14T23:57:48.365 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1022999999999997e-05 2024-09-14T23:57:48.365 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.365 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:48.365 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.365 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.365 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.365 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:56107 3.0 e56/23 3:00c47890:::rbd_data.1118c4b3aebe.0000000000000009:head v 56'125, mlcod=52'123)", 2024-09-14T23:57:48.365 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.236123+0000", 2024-09-14T23:57:48.365 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.089779344, 2024-09-14T23:57:48.365 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0061273220000000002, 2024-09-14T23:57:48.366 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:48.366 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:48.366 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:48.366 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:48.366 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.366 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:48.366 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.236123+0000", 2024-09-14T23:57:48.366 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.366 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.366 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.366 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:48.366 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.236123+0000", 2024-09-14T23:57:48.366 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.366 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.367 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.367 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:48.367 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.236125+0000", 2024-09-14T23:57:48.367 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.313e-06 2024-09-14T23:57:48.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.367 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.367 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:48.367 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237428+0000", 2024-09-14T23:57:48.367 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013029890000000001 2024-09-14T23:57:48.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.367 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.367 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:48.367 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237452+0000", 2024-09-14T23:57:48.367 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4090000000000001e-05 2024-09-14T23:57:48.368 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.368 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.368 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:48.368 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237462+0000", 2024-09-14T23:57:48.368 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0421999999999999e-05 2024-09-14T23:57:48.368 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.368 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.368 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:48.368 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237559+0000", 2024-09-14T23:57:48.368 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6378000000000005e-05 2024-09-14T23:57:48.368 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.368 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.368 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:48.368 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237583+0000", 2024-09-14T23:57:48.369 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4182e-05 2024-09-14T23:57:48.369 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.369 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.369 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:48.369 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.242205+0000", 2024-09-14T23:57:48.369 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0046217289999999998 2024-09-14T23:57:48.369 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.369 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.369 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:48.369 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.242251+0000", 2024-09-14T23:57:48.369 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6218999999999999e-05 2024-09-14T23:57:48.369 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.369 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:48.370 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.370 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.370 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.370 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:56108 3.0 e56/23 3:00c47890:::rbd_data.1118c4b3aebe.0000000000000009:head v 56'127, mlcod=52'123)", 2024-09-14T23:57:48.370 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.239693+0000", 2024-09-14T23:57:48.370 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.086209873999998, 2024-09-14T23:57:48.370 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0043496209999999997, 2024-09-14T23:57:48.370 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:48.370 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:48.370 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:48.370 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:48.370 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.370 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:48.370 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239693+0000", 2024-09-14T23:57:48.371 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.371 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.371 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.371 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:48.371 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239693+0000", 2024-09-14T23:57:48.371 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.371 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.371 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.371 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:48.371 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239694+0000", 2024-09-14T23:57:48.371 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0499999999999999e-06 2024-09-14T23:57:48.371 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.371 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.371 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:48.372 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239702+0000", 2024-09-14T23:57:48.372 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1170000000000004e-06 2024-09-14T23:57:48.372 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.372 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.372 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:48.372 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239716+0000", 2024-09-14T23:57:48.372 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4306e-05 2024-09-14T23:57:48.372 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.372 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.372 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:48.372 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239724+0000", 2024-09-14T23:57:48.372 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0399999999999993e-06 2024-09-14T23:57:48.372 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.373 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.373 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:48.373 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239778+0000", 2024-09-14T23:57:48.373 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4032e-05 2024-09-14T23:57:48.373 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.373 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.373 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:48.373 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239801+0000", 2024-09-14T23:57:48.373 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2770000000000001e-05 2024-09-14T23:57:48.373 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.373 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.373 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:48.373 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.243990+0000", 2024-09-14T23:57:48.374 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0041890909999999998 2024-09-14T23:57:48.374 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.374 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.374 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:48.374 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.244043+0000", 2024-09-14T23:57:48.374 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2215000000000001e-05 2024-09-14T23:57:48.374 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.374 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:48.374 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.374 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.374 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.374 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:56385 3.9 e56/23 3:963d31c7:::rbd_data.1118c4b3aebe.0000000000000003:head v 56'87, mlcod=53'85)", 2024-09-14T23:57:48.374 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.283936+0000", 2024-09-14T23:57:48.374 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.041966963999997, 2024-09-14T23:57:48.375 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0063042080000000004, 2024-09-14T23:57:48.375 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:48.375 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:48.375 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:48.375 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:48.375 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.375 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:48.375 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.283936+0000", 2024-09-14T23:57:48.375 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.375 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.375 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.375 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:48.375 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.283936+0000", 2024-09-14T23:57:48.375 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.375 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.376 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.376 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:48.376 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.283937+0000", 2024-09-14T23:57:48.376 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1179999999999999e-06 2024-09-14T23:57:48.376 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.376 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.376 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:48.376 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286338+0000", 2024-09-14T23:57:48.376 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024010149999999998 2024-09-14T23:57:48.376 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.376 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.376 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:48.376 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286360+0000", 2024-09-14T23:57:48.377 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2212e-05 2024-09-14T23:57:48.377 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.377 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.377 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:48.377 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286370+0000", 2024-09-14T23:57:48.377 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5850000000000006e-06 2024-09-14T23:57:48.377 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.377 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.377 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:48.377 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286443+0000", 2024-09-14T23:57:48.377 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2757999999999997e-05 2024-09-14T23:57:48.377 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.377 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.378 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:48.378 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286466+0000", 2024-09-14T23:57:48.378 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3844000000000001e-05 2024-09-14T23:57:48.378 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.378 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.378 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:48.378 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.290197+0000", 2024-09-14T23:57:48.378 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003730176 2024-09-14T23:57:48.378 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.378 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.378 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:48.378 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.290240+0000", 2024-09-14T23:57:48.378 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.35e-05 2024-09-14T23:57:48.378 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.379 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:48.379 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.379 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.379 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.379 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:56386 3.9 e56/23 3:963d31c7:::rbd_data.1118c4b3aebe.0000000000000003:head v 56'89, mlcod=53'85)", 2024-09-14T23:57:48.379 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.287578+0000", 2024-09-14T23:57:48.379 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.038325237999999, 2024-09-14T23:57:48.379 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033580250000000002, 2024-09-14T23:57:48.379 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:48.379 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:48.379 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:48.379 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:48.379 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.379 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:48.380 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287578+0000", 2024-09-14T23:57:48.380 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.380 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.380 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.380 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:48.380 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287578+0000", 2024-09-14T23:57:48.380 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.380 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.380 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.380 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:48.380 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287579+0000", 2024-09-14T23:57:48.380 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.64e-07 2024-09-14T23:57:48.380 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.381 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.381 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:48.381 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287586+0000", 2024-09-14T23:57:48.381 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5310000000000004e-06 2024-09-14T23:57:48.381 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.381 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.381 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:48.381 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287597+0000", 2024-09-14T23:57:48.381 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0903999999999999e-05 2024-09-14T23:57:48.381 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.381 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.381 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:48.381 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287605+0000", 2024-09-14T23:57:48.382 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7889999999999995e-06 2024-09-14T23:57:48.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.382 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.382 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:48.382 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.288437+0000", 2024-09-14T23:57:48.382 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000831914 2024-09-14T23:57:48.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.382 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.382 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:48.382 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.288460+0000", 2024-09-14T23:57:48.382 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3410000000000001e-05 2024-09-14T23:57:48.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.382 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.383 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:48.383 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.290883+0000", 2024-09-14T23:57:48.383 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024230599999999999 2024-09-14T23:57:48.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.383 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.383 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:48.383 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.290936+0000", 2024-09-14T23:57:48.383 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2453000000000001e-05 2024-09-14T23:57:48.383 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.383 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:48.383 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.383 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.383 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:60299 3.9 e57/23 3:96f26b6f:::rbd_data.11188d356229.0000000000000009:head v 57'98, mlcod=56'96)", 2024-09-14T23:57:48.384 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:07.418336+0000", 2024-09-14T23:57:48.384 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 40.907566744999997, 2024-09-14T23:57:48.384 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022211459999999998, 2024-09-14T23:57:48.384 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:48.384 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:48.384 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:48.384 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:48.384 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.384 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:48.384 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.418336+0000", 2024-09-14T23:57:48.384 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.384 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.384 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.384 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:48.384 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.418336+0000", 2024-09-14T23:57:48.385 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.385 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.385 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.385 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:48.385 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.418337+0000", 2024-09-14T23:57:48.385 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7199999999999999e-07 2024-09-14T23:57:48.385 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.385 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.385 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:48.385 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.419396+0000", 2024-09-14T23:57:48.385 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010598319999999999 2024-09-14T23:57:48.385 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.385 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.386 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:48.386 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.419407+0000", 2024-09-14T23:57:48.386 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0956e-05 2024-09-14T23:57:48.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.386 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.386 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:48.386 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.419417+0000", 2024-09-14T23:57:48.386 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0040999999999999e-05 2024-09-14T23:57:48.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.386 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.386 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:48.386 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.419468+0000", 2024-09-14T23:57:48.386 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0497000000000001e-05 2024-09-14T23:57:48.387 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.387 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.387 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:48.387 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.419485+0000", 2024-09-14T23:57:48.387 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6966e-05 2024-09-14T23:57:48.387 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.387 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.387 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:48.387 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.420527+0000", 2024-09-14T23:57:48.387 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001042133 2024-09-14T23:57:48.387 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.387 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.387 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:48.388 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.420557+0000", 2024-09-14T23:57:48.388 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0249000000000001e-05 2024-09-14T23:57:48.388 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.388 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:48.388 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.388 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.388 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.388 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:64263 3.4 e57/23 3:20c6a768:::rbd_data.11188d356229.0000000000000001:head v 57'165, mlcod=57'163)", 2024-09-14T23:57:48.388 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:07.999031+0000", 2024-09-14T23:57:48.388 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 40.326872090000002, 2024-09-14T23:57:48.388 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023815210000000002, 2024-09-14T23:57:48.388 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:48.388 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:48.389 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:48.389 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:48.389 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.389 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:48.389 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999031+0000", 2024-09-14T23:57:48.389 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.389 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.389 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.389 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:48.389 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999031+0000", 2024-09-14T23:57:48.389 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.389 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.389 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.389 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:48.390 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999032+0000", 2024-09-14T23:57:48.390 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.176e-06 2024-09-14T23:57:48.390 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.390 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.390 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:48.390 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999660+0000", 2024-09-14T23:57:48.390 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00062817099999999996 2024-09-14T23:57:48.390 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.390 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.390 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:48.390 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999683+0000", 2024-09-14T23:57:48.390 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2816999999999999e-05 2024-09-14T23:57:48.390 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.391 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.391 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:48.391 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999693+0000", 2024-09-14T23:57:48.391 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0088999999999999e-05 2024-09-14T23:57:48.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.391 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.391 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:48.391 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999767+0000", 2024-09-14T23:57:48.391 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4056999999999999e-05 2024-09-14T23:57:48.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.391 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.391 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:48.391 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999793+0000", 2024-09-14T23:57:48.391 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5573000000000001e-05 2024-09-14T23:57:48.392 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.392 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.392 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:48.392 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:08.001379+0000", 2024-09-14T23:57:48.392 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015859019999999999 2024-09-14T23:57:48.392 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.392 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.392 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:48.392 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:08.001412+0000", 2024-09-14T23:57:48.392 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3735999999999997e-05 2024-09-14T23:57:48.392 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.392 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:48.392 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.393 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.393 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.393 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:65427 3.4 e60/23 3:269fb893:::rbd_data.11188d356229.00000000000000d9:head v 60'179, mlcod=58'175)", 2024-09-14T23:57:48.393 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:11.105942+0000", 2024-09-14T23:57:48.393 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.219961130000002, 2024-09-14T23:57:48.393 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001562393, 2024-09-14T23:57:48.393 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:48.393 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:48.393 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:48.393 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:48.393 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.393 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:48.393 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.105942+0000", 2024-09-14T23:57:48.394 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.394 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.394 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.394 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:48.394 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.105942+0000", 2024-09-14T23:57:48.394 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.394 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.394 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.394 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:48.394 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.105942+0000", 2024-09-14T23:57:48.394 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0299999999999998e-07 2024-09-14T23:57:48.394 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.394 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.394 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:48.394 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.105965+0000", 2024-09-14T23:57:48.395 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2629e-05 2024-09-14T23:57:48.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.395 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:48.395 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.105977+0000", 2024-09-14T23:57:48.395 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2023e-05 2024-09-14T23:57:48.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.395 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:48.395 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.105986+0000", 2024-09-14T23:57:48.395 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9670000000000007e-06 2024-09-14T23:57:48.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.396 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:48.396 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.106091+0000", 2024-09-14T23:57:48.396 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010527900000000001 2024-09-14T23:57:48.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.396 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.396 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:48.396 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.106112+0000", 2024-09-14T23:57:48.396 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0258000000000002e-05 2024-09-14T23:57:48.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.396 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.396 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:48.396 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.107460+0000", 2024-09-14T23:57:48.396 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001348393 2024-09-14T23:57:48.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.397 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.397 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:48.397 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.107504+0000", 2024-09-14T23:57:48.397 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4141000000000003e-05 2024-09-14T23:57:48.397 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.397 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:48.397 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.397 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.397 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.397 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:66570 3.0 e60/23 3:055350b3:::rbd_data.111873792d9b.0000000000000005:head v 60'162, mlcod=57'159)", 2024-09-14T23:57:48.397 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:11.277517+0000", 2024-09-14T23:57:48.397 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.048385934000002, 2024-09-14T23:57:48.397 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003472199, 2024-09-14T23:57:48.398 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:48.398 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:48.398 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:48.398 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:48.398 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.398 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:48.398 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.277517+0000", 2024-09-14T23:57:48.398 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.398 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.398 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.398 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:48.398 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.277517+0000", 2024-09-14T23:57:48.398 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.398 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.399 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.399 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:48.399 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.277518+0000", 2024-09-14T23:57:48.399 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1379999999999999e-06 2024-09-14T23:57:48.399 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.399 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.399 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:48.399 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278504+0000", 2024-09-14T23:57:48.399 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00098617499999999994 2024-09-14T23:57:48.399 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.399 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.399 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:48.399 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278525+0000", 2024-09-14T23:57:48.399 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.09e-05 2024-09-14T23:57:48.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.400 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:48.400 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278534+0000", 2024-09-14T23:57:48.400 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6209999999999993e-06 2024-09-14T23:57:48.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.400 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:48.400 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278611+0000", 2024-09-14T23:57:48.400 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6926000000000002e-05 2024-09-14T23:57:48.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.400 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:48.401 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278641+0000", 2024-09-14T23:57:48.401 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0861999999999997e-05 2024-09-14T23:57:48.401 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.401 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.401 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:48.401 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.280953+0000", 2024-09-14T23:57:48.401 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023114239999999999 2024-09-14T23:57:48.401 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.401 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.401 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:48.401 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.280989+0000", 2024-09-14T23:57:48.401 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6152999999999998e-05 2024-09-14T23:57:48.401 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.401 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:48.402 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.402 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:70526 3.4 e64/23 3:2649db36:::rbd_data.111831f32e80.0000000000000058:head v 64'183, mlcod=64'181)", 2024-09-14T23:57:48.402 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:15.442503+0000", 2024-09-14T23:57:48.402 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.883400041999998, 2024-09-14T23:57:48.402 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001769837, 2024-09-14T23:57:48.402 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:48.402 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:48.402 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:48.402 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:48.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.402 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:48.402 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.442503+0000", 2024-09-14T23:57:48.403 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.403 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.403 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:48.403 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.442503+0000", 2024-09-14T23:57:48.403 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.403 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.403 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:48.403 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.442503+0000", 2024-09-14T23:57:48.403 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1099999999999995e-07 2024-09-14T23:57:48.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.403 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.403 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:48.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.442620+0000", 2024-09-14T23:57:48.404 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000116618 2024-09-14T23:57:48.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.404 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:48.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.442634+0000", 2024-09-14T23:57:48.404 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3984e-05 2024-09-14T23:57:48.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.404 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:48.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.442641+0000", 2024-09-14T23:57:48.404 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2540000000000003e-06 2024-09-14T23:57:48.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.405 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:48.405 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.442732+0000", 2024-09-14T23:57:48.405 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0480000000000001e-05 2024-09-14T23:57:48.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.405 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:48.405 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.442753+0000", 2024-09-14T23:57:48.405 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1571999999999999e-05 2024-09-14T23:57:48.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.405 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:48.406 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.444236+0000", 2024-09-14T23:57:48.406 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0014825439999999999 2024-09-14T23:57:48.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.406 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.406 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:48.406 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.444273+0000", 2024-09-14T23:57:48.406 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6674000000000002e-05 2024-09-14T23:57:48.406 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.406 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:48.406 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.406 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.406 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:106629 3.4 e70/23 3:240d85c6:::rbd_data.11187de8b0a5.0000000000000009:head v 70'233, mlcod=69'231)", 2024-09-14T23:57:48.407 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:21.489517+0000", 2024-09-14T23:57:48.407 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.836385409000002, 2024-09-14T23:57:48.407 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016279840000000001, 2024-09-14T23:57:48.407 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:48.407 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:48.407 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:48.407 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:48.407 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.407 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:48.407 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.489517+0000", 2024-09-14T23:57:48.407 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.407 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.407 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:48.408 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.489517+0000", 2024-09-14T23:57:48.408 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.408 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:48.408 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.489518+0000", 2024-09-14T23:57:48.408 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.029e-06 2024-09-14T23:57:48.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.408 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:48.408 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.489840+0000", 2024-09-14T23:57:48.408 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00032164100000000001 2024-09-14T23:57:48.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.408 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:48.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.489859+0000", 2024-09-14T23:57:48.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8828999999999999e-05 2024-09-14T23:57:48.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:48.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.489869+0000", 2024-09-14T23:57:48.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6600000000000007e-06 2024-09-14T23:57:48.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:48.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.489934+0000", 2024-09-14T23:57:48.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5357000000000005e-05 2024-09-14T23:57:48.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:48.410 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.489951+0000", 2024-09-14T23:57:48.410 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6951000000000001e-05 2024-09-14T23:57:48.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.410 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.410 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:48.410 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.491108+0000", 2024-09-14T23:57:48.410 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011571940000000001 2024-09-14T23:57:48.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.410 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.410 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:48.410 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.491145+0000", 2024-09-14T23:57:48.410 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7323000000000002e-05 2024-09-14T23:57:48.410 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.410 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:48.410 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.411 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.411 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:108739 3.9 e80/23 3:9372b409:::rbd_data.111867e26b91.0000000000000005:head v 80'172, mlcod=78'170)", 2024-09-14T23:57:48.411 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:31.064099+0000", 2024-09-14T23:57:48.411 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 17.261804156, 2024-09-14T23:57:48.411 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001688174, 2024-09-14T23:57:48.411 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:48.411 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:48.411 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:48.411 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:48.411 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.411 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:48.411 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:31.064099+0000", 2024-09-14T23:57:48.411 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.412 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.412 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:48.412 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:31.064099+0000", 2024-09-14T23:57:48.412 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.412 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.412 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:48.412 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:31.064099+0000", 2024-09-14T23:57:48.412 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.44e-07 2024-09-14T23:57:48.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.412 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.412 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:48.412 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:31.064453+0000", 2024-09-14T23:57:48.412 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00035374899999999999 2024-09-14T23:57:48.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.413 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:48.413 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:31.064468+0000", 2024-09-14T23:57:48.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4476e-05 2024-09-14T23:57:48.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.413 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:48.413 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:31.064475+0000", 2024-09-14T23:57:48.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9289999999999998e-06 2024-09-14T23:57:48.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.413 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:48.414 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:31.064530+0000", 2024-09-14T23:57:48.414 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4401000000000003e-05 2024-09-14T23:57:48.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.414 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.414 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:48.414 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:31.064551+0000", 2024-09-14T23:57:48.414 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0996e-05 2024-09-14T23:57:48.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.414 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.414 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:48.414 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:31.065748+0000", 2024-09-14T23:57:48.414 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011967449999999999 2024-09-14T23:57:48.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.415 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.415 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:48.415 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:31.065787+0000", 2024-09-14T23:57:48.415 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9233999999999997e-05 2024-09-14T23:57:48.415 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.415 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:48.415 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.415 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.415 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "pg_update_log_missing(3.9 epoch 84/23 rep_tid 2234 entries 84'200 (0'0) error 3:93b58b1d:::rbd_data.1118417a1b0b.000000000000001b:head by client.4376.0:118637 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 83'197)", 2024-09-14T23:57:48.415 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:35.103156+0000", 2024-09-14T23:57:48.415 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.222746661, 2024-09-14T23:57:48.415 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019852009999999998, 2024-09-14T23:57:48.415 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:48.416 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:48.416 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:48.416 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:48.416 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.416 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:48.416 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:35.103156+0000", 2024-09-14T23:57:48.416 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.416 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.416 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:48.416 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:35.103156+0000", 2024-09-14T23:57:48.416 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.417 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.417 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:48.417 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:35.103157+0000", 2024-09-14T23:57:48.417 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1969999999999999e-06 2024-09-14T23:57:48.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.417 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.417 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:48.417 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:35.103170+0000", 2024-09-14T23:57:48.417 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2921e-05 2024-09-14T23:57:48.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.417 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.417 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:48.417 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:35.103202+0000", 2024-09-14T23:57:48.417 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1992000000000001e-05 2024-09-14T23:57:48.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.418 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:48.418 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:35.103220+0000", 2024-09-14T23:57:48.418 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7312999999999998e-05 2024-09-14T23:57:48.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.418 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:48.418 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:35.103305+0000", 2024-09-14T23:57:48.418 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4975000000000005e-05 2024-09-14T23:57:48.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.418 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:48.419 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:35.105141+0000", 2024-09-14T23:57:48.419 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018368029999999999 2024-09-14T23:57:48.419 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.419 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:48.419 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.419 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.419 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:48.419 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:48.421 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T23:57:48.421 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:48.419+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:48.521 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:48.519+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:48.567 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:48.567 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:57:48.567 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:57:48.567 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:48.576 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T23:57:48.622 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:48.621+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:48.683 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:48.683 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:57:48.684 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:57:48.684 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:57:48.684 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:48.692 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T23:57:48.724 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:48.721+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:48.824 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:48.823+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:48.841 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:48.841 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:57:48.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:57:48.841 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:57:48.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.841 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5180 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3883008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:48.842 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.250058+0000", 2024-09-14T23:57:48.842 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 69.589087524000007, 2024-09-14T23:57:48.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057351471000000001, 2024-09-14T23:57:48.842 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:48.842 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:48.842 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:48.842 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:48.842 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:48.842 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:48.842 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5180 2024-09-14T23:57:48.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.842 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:48.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:48.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250058+0000", 2024-09-14T23:57:48.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:48.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250058+0000", 2024-09-14T23:57:48.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:48.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250786+0000", 2024-09-14T23:57:48.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00072813399999999996 2024-09-14T23:57:48.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:48.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250789+0000", 2024-09-14T23:57:48.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.647e-06 2024-09-14T23:57:48.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:48.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250798+0000", 2024-09-14T23:57:48.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0000000000000002e-06 2024-09-14T23:57:48.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:48.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250803+0000", 2024-09-14T23:57:48.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.301e-06 2024-09-14T23:57:48.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:48.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.307234+0000", 2024-09-14T23:57:48.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056431165999999998 2024-09-14T23:57:48.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:48.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.307321+0000", 2024-09-14T23:57:48.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6831999999999998e-05 2024-09-14T23:57:48.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:48.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.307409+0000", 2024-09-14T23:57:48.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.8391000000000001e-05 2024-09-14T23:57:48.846 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.846 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:48.846 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.846 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5184 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 4128768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:48.846 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.250851+0000", 2024-09-14T23:57:48.846 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 69.588294052999998, 2024-09-14T23:57:48.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057319038000000003, 2024-09-14T23:57:48.846 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:48.846 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:48.847 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:48.847 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:48.847 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:48.847 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:48.847 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5184 2024-09-14T23:57:48.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.847 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:48.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:48.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250851+0000", 2024-09-14T23:57:48.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:48.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.250851+0000", 2024-09-14T23:57:48.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:48.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.251900+0000", 2024-09-14T23:57:48.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001048577 2024-09-14T23:57:48.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:48.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.251904+0000", 2024-09-14T23:57:48.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9580000000000001e-06 2024-09-14T23:57:48.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:48.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.251914+0000", 2024-09-14T23:57:48.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0186999999999999e-05 2024-09-14T23:57:48.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:48.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.251927+0000", 2024-09-14T23:57:48.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3593e-05 2024-09-14T23:57:48.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:48.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.308020+0000", 2024-09-14T23:57:48.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056092867999999997 2024-09-14T23:57:48.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:48.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.308094+0000", 2024-09-14T23:57:48.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3533000000000002e-05 2024-09-14T23:57:48.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:48.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.308170+0000", 2024-09-14T23:57:48.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6322000000000006e-05 2024-09-14T23:57:48.850 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.851 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:48.851 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.851 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5196 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 1925120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:48.851 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.252157+0000", 2024-09-14T23:57:48.851 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 69.586987938999997, 2024-09-14T23:57:48.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057285801999999997, 2024-09-14T23:57:48.851 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:48.851 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:48.851 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:48.851 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:48.851 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:48.851 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:48.852 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5196 2024-09-14T23:57:48.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.852 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:48.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:48.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.252157+0000", 2024-09-14T23:57:48.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:48.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.252157+0000", 2024-09-14T23:57:48.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:48.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253243+0000", 2024-09-14T23:57:48.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010861340000000001 2024-09-14T23:57:48.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:48.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253248+0000", 2024-09-14T23:57:48.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8799999999999999e-06 2024-09-14T23:57:48.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:48.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253271+0000", 2024-09-14T23:57:48.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2836e-05 2024-09-14T23:57:48.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:48.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253281+0000", 2024-09-14T23:57:48.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6520000000000001e-06 2024-09-14T23:57:48.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:48.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.309336+0000", 2024-09-14T23:57:48.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056054983000000003 2024-09-14T23:57:48.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:48.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.309389+0000", 2024-09-14T23:57:48.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3298999999999999e-05 2024-09-14T23:57:48.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:48.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.309443+0000", 2024-09-14T23:57:48.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4017999999999997e-05 2024-09-14T23:57:48.855 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.855 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:48.855 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.856 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5200 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2170880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:48.856 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253394+0000", 2024-09-14T23:57:48.856 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 69.585751161999994, 2024-09-14T23:57:48.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057428154000000002, 2024-09-14T23:57:48.856 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:48.856 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:48.856 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:48.856 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:48.856 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:48.857 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:48.857 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5200 2024-09-14T23:57:48.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.857 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:48.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:48.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253394+0000", 2024-09-14T23:57:48.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:48.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253394+0000", 2024-09-14T23:57:48.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:48.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253395+0000", 2024-09-14T23:57:48.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9100000000000004e-07 2024-09-14T23:57:48.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:48.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253397+0000", 2024-09-14T23:57:48.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2790000000000001e-06 2024-09-14T23:57:48.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:48.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253404+0000", 2024-09-14T23:57:48.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7959999999999998e-06 2024-09-14T23:57:48.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:48.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253409+0000", 2024-09-14T23:57:48.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1460000000000004e-06 2024-09-14T23:57:48.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:48.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310627+0000", 2024-09-14T23:57:48.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057218160999999997 2024-09-14T23:57:48.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:48.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310710+0000", 2024-09-14T23:57:48.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3102999999999994e-05 2024-09-14T23:57:48.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:48.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310822+0000", 2024-09-14T23:57:48.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000112178 2024-09-14T23:57:48.862 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.862 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:48.862 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.862 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5201 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2232320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:48.863 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:57:48.863 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 69.585726489999999, 2024-09-14T23:57:48.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057643228999999997, 2024-09-14T23:57:48.863 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:48.863 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:48.863 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:48.863 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:48.863 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:48.863 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:48.863 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5201 2024-09-14T23:57:48.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.863 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:48.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:48.864 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:57:48.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.864 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:48.864 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:57:48.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.864 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:48.864 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:57:48.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0699999999999998e-07 2024-09-14T23:57:48.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:48.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253422+0000", 2024-09-14T23:57:48.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.514e-06 2024-09-14T23:57:48.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:48.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253431+0000", 2024-09-14T23:57:48.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1919999999999994e-06 2024-09-14T23:57:48.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:48.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253436+0000", 2024-09-14T23:57:48.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5350000000000004e-06 2024-09-14T23:57:48.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:48.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310856+0000", 2024-09-14T23:57:48.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057419381999999998 2024-09-14T23:57:48.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:48.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310953+0000", 2024-09-14T23:57:48.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6774000000000006e-05 2024-09-14T23:57:48.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:48.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311062+0000", 2024-09-14T23:57:48.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000109425 2024-09-14T23:57:48.867 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.867 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:48.867 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.867 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5202 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2293760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:48.867 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:57:48.867 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 69.585701741999998, 2024-09-14T23:57:48.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057818255999999998, 2024-09-14T23:57:48.867 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:48.867 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:48.867 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:48.867 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:48.867 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:48.868 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:48.868 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5202 2024-09-14T23:57:48.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.868 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:48.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.868 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:48.868 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:57:48.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.868 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:48.868 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:57:48.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.869 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:48.869 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253573+0000", 2024-09-14T23:57:48.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012934000000000001 2024-09-14T23:57:48.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.869 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:48.869 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253576+0000", 2024-09-14T23:57:48.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5310000000000002e-06 2024-09-14T23:57:48.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.869 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:48.869 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253587+0000", 2024-09-14T23:57:48.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0522e-05 2024-09-14T23:57:48.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:48.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253595+0000", 2024-09-14T23:57:48.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8290000000000008e-06 2024-09-14T23:57:48.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:48.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311093+0000", 2024-09-14T23:57:48.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057498312000000003 2024-09-14T23:57:48.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:48.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311167+0000", 2024-09-14T23:57:48.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3582000000000001e-05 2024-09-14T23:57:48.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:48.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311262+0000", 2024-09-14T23:57:48.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5140000000000001e-05 2024-09-14T23:57:48.871 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.871 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:48.871 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.871 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5203 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2355200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:48.871 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:57:48.871 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 69.585538999999997, 2024-09-14T23:57:48.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057903938000000002, 2024-09-14T23:57:48.872 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:48.872 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:48.872 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:48.872 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:48.872 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:48.872 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:48.872 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5203 2024-09-14T23:57:48.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.872 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:48.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:48.872 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:57:48.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.873 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:48.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:57:48.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.873 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:48.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253902+0000", 2024-09-14T23:57:48.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029615400000000002 2024-09-14T23:57:48.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.873 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:48.874 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253906+0000", 2024-09-14T23:57:48.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9249999999999997e-06 2024-09-14T23:57:48.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:48.874 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253913+0000", 2024-09-14T23:57:48.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9099999999999999e-06 2024-09-14T23:57:48.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:48.874 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253917+0000", 2024-09-14T23:57:48.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9210000000000002e-06 2024-09-14T23:57:48.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:48.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311295+0000", 2024-09-14T23:57:48.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057378268000000003 2024-09-14T23:57:48.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:48.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311402+0000", 2024-09-14T23:57:48.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010644100000000001 2024-09-14T23:57:48.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:48.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311510+0000", 2024-09-14T23:57:48.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108319 2024-09-14T23:57:48.875 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.876 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:48.876 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.876 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5204 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2416640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:48.876 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:57:48.876 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 69.585220802999999, 2024-09-14T23:57:48.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057774860999999997, 2024-09-14T23:57:48.876 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:48.876 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:48.876 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:48.876 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:48.876 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:48.877 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:48.877 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5204 2024-09-14T23:57:48.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.877 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:48.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:48.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:57:48.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:48.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:57:48.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:48.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255903+0000", 2024-09-14T23:57:48.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001978807 2024-09-14T23:57:48.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:48.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255907+0000", 2024-09-14T23:57:48.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3610000000000001e-06 2024-09-14T23:57:48.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:48.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255914+0000", 2024-09-14T23:57:48.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.357e-06 2024-09-14T23:57:48.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:48.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255918+0000", 2024-09-14T23:57:48.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1779999999999996e-06 2024-09-14T23:57:48.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:48.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311541+0000", 2024-09-14T23:57:48.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055622697999999998 2024-09-14T23:57:48.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:48.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311614+0000", 2024-09-14T23:57:48.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3102999999999994e-05 2024-09-14T23:57:48.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.880 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:48.880 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311699+0000", 2024-09-14T23:57:48.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5357000000000003e-05 2024-09-14T23:57:48.880 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.880 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:48.880 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.880 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5211 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2846720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:48.880 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:57:48.881 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 69.583122588999998, 2024-09-14T23:57:48.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057528151, 2024-09-14T23:57:48.881 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:48.881 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:48.881 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:48.881 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:48.881 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:48.881 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:48.881 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5211 2024-09-14T23:57:48.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.881 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:48.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:48.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:57:48.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.882 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:48.882 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:57:48.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.882 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:48.882 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:57:48.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1399999999999998e-07 2024-09-14T23:57:48.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.882 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:48.883 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256024+0000", 2024-09-14T23:57:48.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.114e-06 2024-09-14T23:57:48.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.883 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:48.883 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256028+0000", 2024-09-14T23:57:48.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0840000000000002e-06 2024-09-14T23:57:48.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.883 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:48.883 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256031+0000", 2024-09-14T23:57:48.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6819999999999999e-06 2024-09-14T23:57:48.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:48.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313317+0000", 2024-09-14T23:57:48.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057286295000000001 2024-09-14T23:57:48.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:48.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313403+0000", 2024-09-14T23:57:48.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6287999999999998e-05 2024-09-14T23:57:48.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:48.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313551+0000", 2024-09-14T23:57:48.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000147374 2024-09-14T23:57:48.884 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.885 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:48.885 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.885 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5212 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2908160~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:48.885 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:57:48.885 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 69.583111395000003, 2024-09-14T23:57:48.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057735679999999998, 2024-09-14T23:57:48.885 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:48.885 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:48.885 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:48.885 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:48.885 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:48.886 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:48.886 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5212 2024-09-14T23:57:48.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.886 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:48.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:48.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:57:48.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:48.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:57:48.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:48.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:57:48.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0900000000000001e-07 2024-09-14T23:57:48.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:48.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256035+0000", 2024-09-14T23:57:48.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1200000000000001e-06 2024-09-14T23:57:48.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:48.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256038+0000", 2024-09-14T23:57:48.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.05e-06 2024-09-14T23:57:48.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.888 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:48.888 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256041+0000", 2024-09-14T23:57:48.888 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.667e-06 2024-09-14T23:57:48.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.888 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:48.888 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313584+0000", 2024-09-14T23:57:48.888 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057543484999999998 2024-09-14T23:57:48.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.888 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:48.888 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313668+0000", 2024-09-14T23:57:48.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3807000000000006e-05 2024-09-14T23:57:48.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:48.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313770+0000", 2024-09-14T23:57:48.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010134200000000001 2024-09-14T23:57:48.889 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.889 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:48.889 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.889 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5213 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2969600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:48.889 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:57:48.889 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 69.583101307999996, 2024-09-14T23:57:48.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058145467999999999, 2024-09-14T23:57:48.890 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:48.890 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:48.890 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:48.890 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:48.890 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:48.890 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:48.890 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5213 2024-09-14T23:57:48.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.890 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:48.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:48.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:57:48.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:48.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:57:48.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:48.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257901+0000", 2024-09-14T23:57:48.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018567519999999999 2024-09-14T23:57:48.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:48.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257904+0000", 2024-09-14T23:57:48.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4850000000000001e-06 2024-09-14T23:57:48.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:48.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257912+0000", 2024-09-14T23:57:48.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2090000000000005e-06 2024-09-14T23:57:48.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:48.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257918+0000", 2024-09-14T23:57:48.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6899999999999997e-06 2024-09-14T23:57:48.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:48.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313816+0000", 2024-09-14T23:57:48.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055898075999999998 2024-09-14T23:57:48.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:48.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.314082+0000", 2024-09-14T23:57:48.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000265386 2024-09-14T23:57:48.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:48.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.314189+0000", 2024-09-14T23:57:48.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010787 2024-09-14T23:57:48.893 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.893 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:48.893 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.894 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5217 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3215360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:48.894 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257961+0000", 2024-09-14T23:57:48.894 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 69.581184328999996, 2024-09-14T23:57:48.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057320969999999999, 2024-09-14T23:57:48.894 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:48.894 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:48.894 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:48.894 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:48.894 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:48.894 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:48.894 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5217 2024-09-14T23:57:48.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.895 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:48.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:48.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257961+0000", 2024-09-14T23:57:48.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:48.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257961+0000", 2024-09-14T23:57:48.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:48.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257961+0000", 2024-09-14T23:57:48.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.22e-07 2024-09-14T23:57:48.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.896 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:48.896 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257962+0000", 2024-09-14T23:57:48.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1400000000000001e-06 2024-09-14T23:57:48.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.896 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:48.896 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257965+0000", 2024-09-14T23:57:48.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0929999999999999e-06 2024-09-14T23:57:48.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.896 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:48.896 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257968+0000", 2024-09-14T23:57:48.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5749999999999999e-06 2024-09-14T23:57:48.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.897 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:48.897 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315047+0000", 2024-09-14T23:57:48.897 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057078730000000001 2024-09-14T23:57:48.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.897 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:48.897 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315087+0000", 2024-09-14T23:57:48.897 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0806999999999997e-05 2024-09-14T23:57:48.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.897 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:48.897 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315282+0000", 2024-09-14T23:57:48.897 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000194403 2024-09-14T23:57:48.898 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.898 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:48.898 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.898 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.898 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5218 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3276800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:48.898 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:57:48.898 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 69.581174298999997, 2024-09-14T23:57:48.898 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057497557999999997, 2024-09-14T23:57:48.898 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:48.898 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:48.898 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:48.898 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:48.898 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:48.899 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:48.899 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5218 2024-09-14T23:57:48.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.899 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:48.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.899 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:48.899 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:57:48.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.899 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:48.899 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:57:48.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.899 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:48.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:57:48.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1899999999999999e-07 2024-09-14T23:57:48.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:48.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257972+0000", 2024-09-14T23:57:48.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.217e-06 2024-09-14T23:57:48.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:48.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257975+0000", 2024-09-14T23:57:48.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9359999999999999e-06 2024-09-14T23:57:48.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:48.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257978+0000", 2024-09-14T23:57:48.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1049999999999999e-06 2024-09-14T23:57:48.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:48.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315331+0000", 2024-09-14T23:57:48.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057352093999999999 2024-09-14T23:57:48.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:48.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315384+0000", 2024-09-14T23:57:48.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3780999999999999e-05 2024-09-14T23:57:48.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.902 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:48.902 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315468+0000", 2024-09-14T23:57:48.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4206e-05 2024-09-14T23:57:48.902 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.902 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:48.902 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.902 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5219 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3338240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:48.902 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:57:48.902 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 69.581163622999995, 2024-09-14T23:57:48.903 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057637764000000001, 2024-09-14T23:57:48.903 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:48.903 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:48.903 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:48.903 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:48.903 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:48.903 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:48.903 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5219 2024-09-14T23:57:48.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.903 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:48.903 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.903 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:48.903 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:57:48.903 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.903 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.904 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:48.904 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:57:48.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.904 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:48.904 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:57:48.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.03e-07 2024-09-14T23:57:48.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.904 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:48.904 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257983+0000", 2024-09-14T23:57:48.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9199999999999999e-07 2024-09-14T23:57:48.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.905 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:48.905 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257986+0000", 2024-09-14T23:57:48.905 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0860000000000002e-06 2024-09-14T23:57:48.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.905 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:48.905 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257989+0000", 2024-09-14T23:57:48.905 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6589999999999999e-06 2024-09-14T23:57:48.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.905 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:48.905 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315509+0000", 2024-09-14T23:57:48.905 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057520262000000003 2024-09-14T23:57:48.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.906 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.906 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:48.906 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315554+0000", 2024-09-14T23:57:48.906 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5015999999999997e-05 2024-09-14T23:57:48.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.906 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.906 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:48.906 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315619+0000", 2024-09-14T23:57:48.906 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5545999999999999e-05 2024-09-14T23:57:48.906 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.906 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:48.906 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.906 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.906 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5220 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3399680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:48.907 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:57:48.907 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 69.581153666999995, 2024-09-14T23:57:48.907 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057792128999999998, 2024-09-14T23:57:48.907 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:48.907 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:48.907 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:48.907 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:48.907 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:48.907 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:48.907 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5220 2024-09-14T23:57:48.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.907 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:48.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.907 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:48.907 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:57:48.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.908 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:48.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:57:48.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.908 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:48.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:57:48.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1799999999999999e-07 2024-09-14T23:57:48.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.908 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:48.909 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257993+0000", 2024-09-14T23:57:48.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6200000000000006e-07 2024-09-14T23:57:48.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:48.909 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257996+0000", 2024-09-14T23:57:48.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9050000000000001e-06 2024-09-14T23:57:48.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:48.909 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257998+0000", 2024-09-14T23:57:48.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5950000000000001e-06 2024-09-14T23:57:48.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:48.910 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315672+0000", 2024-09-14T23:57:48.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057673637999999999 2024-09-14T23:57:48.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.910 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.910 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:48.910 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315734+0000", 2024-09-14T23:57:48.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2216000000000002e-05 2024-09-14T23:57:48.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.910 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.910 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:48.910 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315784+0000", 2024-09-14T23:57:48.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9595000000000002e-05 2024-09-14T23:57:48.910 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.910 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:48.910 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.911 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.911 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5221 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3461120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:48.911 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:57:48.911 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 69.581143948999994, 2024-09-14T23:57:48.911 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058003557999999997, 2024-09-14T23:57:48.911 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:48.911 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:48.911 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:48.911 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:48.911 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:48.911 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:48.911 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5221 2024-09-14T23:57:48.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.911 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:48.912 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.912 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:48.912 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:57:48.912 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.912 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.912 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.912 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:48.912 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:57:48.912 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.912 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.912 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.912 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:48.912 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259905+0000", 2024-09-14T23:57:48.912 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019035759999999999 2024-09-14T23:57:48.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.913 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:48.913 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259915+0000", 2024-09-14T23:57:48.913 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0518e-05 2024-09-14T23:57:48.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.913 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:48.913 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259925+0000", 2024-09-14T23:57:48.913 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7180000000000008e-06 2024-09-14T23:57:48.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.913 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:48.913 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259930+0000", 2024-09-14T23:57:48.914 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4769999999999997e-06 2024-09-14T23:57:48.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.914 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.914 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:48.914 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315820+0000", 2024-09-14T23:57:48.914 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055889960000000002 2024-09-14T23:57:48.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.914 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.914 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:48.914 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315914+0000", 2024-09-14T23:57:48.914 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4625999999999998e-05 2024-09-14T23:57:48.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.914 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.915 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:48.915 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.316005+0000", 2024-09-14T23:57:48.915 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0682999999999999e-05 2024-09-14T23:57:48.915 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.915 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:48.915 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.915 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.915 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5230 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 4014080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:48.915 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.260060+0000", 2024-09-14T23:57:48.915 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 69.579085110999998, 2024-09-14T23:57:48.915 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057353556999999999, 2024-09-14T23:57:48.915 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:48.915 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:48.916 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:48.916 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:48.916 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:48.916 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:48.916 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5230 2024-09-14T23:57:48.916 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.916 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:48.916 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.916 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:48.916 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260060+0000", 2024-09-14T23:57:48.916 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.916 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.916 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.916 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:48.917 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260060+0000", 2024-09-14T23:57:48.917 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.917 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.917 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:48.917 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260060+0000", 2024-09-14T23:57:48.917 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.17e-07 2024-09-14T23:57:48.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.917 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.917 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:48.917 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260061+0000", 2024-09-14T23:57:48.917 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0300000000000001e-06 2024-09-14T23:57:48.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.917 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.918 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:48.918 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260064+0000", 2024-09-14T23:57:48.918 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9119999999999998e-06 2024-09-14T23:57:48.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.918 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.918 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:48.918 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260067+0000", 2024-09-14T23:57:48.918 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.678e-06 2024-09-14T23:57:48.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.918 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.918 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:48.918 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317269+0000", 2024-09-14T23:57:48.918 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057201771999999998 2024-09-14T23:57:48.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.919 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.919 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:48.919 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317342+0000", 2024-09-14T23:57:48.919 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2842000000000005e-05 2024-09-14T23:57:48.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.919 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.919 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:48.919 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317414+0000", 2024-09-14T23:57:48.919 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2106000000000004e-05 2024-09-14T23:57:48.919 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.919 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:48.919 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.919 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.920 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5231 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 4075520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:48.920 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:57:48.920 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 69.579075215000003, 2024-09-14T23:57:48.920 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057524733000000002, 2024-09-14T23:57:48.920 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:48.920 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:48.920 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:48.920 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:48.920 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:48.920 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:48.920 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5231 2024-09-14T23:57:48.920 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.920 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:48.920 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.921 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:48.921 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:57:48.921 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.921 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.921 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.921 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:48.921 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:57:48.921 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.921 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.921 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.921 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:48.921 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261904+0000", 2024-09-14T23:57:48.921 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018338790000000001 2024-09-14T23:57:48.921 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.921 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.922 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:48.922 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261910+0000", 2024-09-14T23:57:48.922 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9689999999999999e-06 2024-09-14T23:57:48.922 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.922 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.922 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:48.922 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261923+0000", 2024-09-14T23:57:48.922 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3061000000000001e-05 2024-09-14T23:57:48.922 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.922 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.922 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:48.922 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261931+0000", 2024-09-14T23:57:48.922 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3159999999999997e-06 2024-09-14T23:57:48.922 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.922 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.923 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:48.923 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317456+0000", 2024-09-14T23:57:48.923 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055525219000000001 2024-09-14T23:57:48.923 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.923 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.923 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:48.923 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317520+0000", 2024-09-14T23:57:48.923 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4010000000000005e-05 2024-09-14T23:57:48.923 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.923 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.923 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:48.923 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317595+0000", 2024-09-14T23:57:48.923 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4278999999999998e-05 2024-09-14T23:57:48.923 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.923 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:48.924 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.924 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.924 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.924 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5236 3.8 3:1071668d:::rbd_data.1118de1168b.0000000000000092:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:48.924 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:57:48.924 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 69.577061751000002, 2024-09-14T23:57:48.924 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057499462000000001, 2024-09-14T23:57:48.924 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:48.924 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:48.924 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:48.924 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:48.924 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:48.924 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:48.924 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5236 2024-09-14T23:57:48.925 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.925 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:48.925 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.925 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:48.925 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:57:48.925 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.925 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.925 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.925 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:48.925 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:57:48.925 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.925 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.925 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.925 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:48.925 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262084+0000", 2024-09-14T23:57:48.926 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3399999999999999e-07 2024-09-14T23:57:48.926 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.926 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.926 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:48.926 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262087+0000", 2024-09-14T23:57:48.926 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2310000000000001e-06 2024-09-14T23:57:48.926 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.926 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.926 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:48.926 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262095+0000", 2024-09-14T23:57:48.926 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3799999999999996e-06 2024-09-14T23:57:48.926 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.926 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.926 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:48.926 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262101+0000", 2024-09-14T23:57:48.927 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0009999999999998e-06 2024-09-14T23:57:48.927 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.927 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.927 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:48.927 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317625+0000", 2024-09-14T23:57:48.927 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055524111000000001 2024-09-14T23:57:48.927 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.927 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.927 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:48.927 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317702+0000", 2024-09-14T23:57:48.927 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7533999999999993e-05 2024-09-14T23:57:48.927 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.927 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.927 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-14T23:57:48.928 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317892+0000", 2024-09-14T23:57:48.928 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018991400000000001 2024-09-14T23:57:48.928 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.928 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.928 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:57:48.928 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319407+0000", 2024-09-14T23:57:48.928 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015145250000000001 2024-09-14T23:57:48.928 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.928 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.928 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:48.928 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319481+0000", 2024-09-14T23:57:48.928 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3826999999999996e-05 2024-09-14T23:57:48.928 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.929 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.929 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:48.929 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319541+0000", 2024-09-14T23:57:48.929 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0470999999999998e-05 2024-09-14T23:57:48.929 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.929 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.929 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:48.929 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319583+0000", 2024-09-14T23:57:48.929 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1934000000000002e-05 2024-09-14T23:57:48.929 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.929 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:48.929 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.929 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.929 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.930 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:132622 3.1e 3.7a50b9e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-14T23:57:48.930 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:43.371285+0000", 2024-09-14T23:57:48.930 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.4678598489999999, 2024-09-14T23:57:48.930 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.67037987200000004, 2024-09-14T23:57:48.930 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:48.930 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:48.930 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:57:48.930 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:48.930 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:48.930 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:48.930 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 132622 2024-09-14T23:57:48.930 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.930 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:48.930 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.930 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:48.930 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371285+0000", 2024-09-14T23:57:48.931 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.931 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.931 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.931 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:48.931 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371285+0000", 2024-09-14T23:57:48.931 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:48.931 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.931 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.931 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:48.931 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371286+0000", 2024-09-14T23:57:48.931 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6400000000000002e-07 2024-09-14T23:57:48.931 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.931 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.931 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:48.932 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371289+0000", 2024-09-14T23:57:48.932 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2059999999999999e-06 2024-09-14T23:57:48.932 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.932 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.932 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:48.932 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371299+0000", 2024-09-14T23:57:48.932 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0288e-05 2024-09-14T23:57:48.932 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.932 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.932 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:48.932 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371306+0000", 2024-09-14T23:57:48.932 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1650000000000003e-06 2024-09-14T23:57:48.932 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.932 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.932 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:48.933 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371617+0000", 2024-09-14T23:57:48.933 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000311043 2024-09-14T23:57:48.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.933 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.933 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:57:48.933 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371625+0000", 2024-09-14T23:57:48.933 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.833e-06 2024-09-14T23:57:48.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.933 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.933 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:48.933 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.041651+0000", 2024-09-14T23:57:48.933 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.67002580199999995 2024-09-14T23:57:48.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:48.933 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:48.934 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:48.934 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.041665+0000", 2024-09-14T23:57:48.934 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3971000000000001e-05 2024-09-14T23:57:48.934 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.934 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:48.934 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.934 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:48.934 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:48.934 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:48.936 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T23:57:48.936 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:48.934+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:49.036 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:49.034+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:49.054 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:49.054 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:49.055 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:57:49.055 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:49.058 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is unset 2024-09-14T23:57:49.064 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T23:57:49.136 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:49.135+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:49.177 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:49.177 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:49.177 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:57:49.177 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:57:49.177 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:49.186 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T23:57:49.200 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone38 from 0x2df6ba0 (overlap 0xaa098a) to 0x0 2024-09-14T23:57:49.237 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:49.236+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:49.300 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:49.300 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:57:49.300 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:57:49.300 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:57:49.301 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.301 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:3500 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T23:57:49.301 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:37.990193+0000", 2024-09-14T23:57:49.301 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 71.307715247999994, 2024-09-14T23:57:49.301 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99440750899999997, 2024-09-14T23:57:49.301 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:49.301 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:49.301 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:49.301 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:49.301 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:49.301 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:49.301 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3500 2024-09-14T23:57:49.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.301 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:49.302 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.302 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:49.302 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990193+0000", 2024-09-14T23:57:49.302 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.302 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.302 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:49.302 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990193+0000", 2024-09-14T23:57:49.302 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.303 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:49.303 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990194+0000", 2024-09-14T23:57:49.303 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5000000000000001e-07 2024-09-14T23:57:49.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.303 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:49.303 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990200+0000", 2024-09-14T23:57:49.303 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4389999999999997e-06 2024-09-14T23:57:49.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.303 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:49.303 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990215+0000", 2024-09-14T23:57:49.304 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4613999999999999e-05 2024-09-14T23:57:49.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.304 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.304 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:49.304 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990224+0000", 2024-09-14T23:57:49.304 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1740000000000006e-06 2024-09-14T23:57:49.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.304 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.304 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:49.304 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990283+0000", 2024-09-14T23:57:49.304 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9231999999999998e-05 2024-09-14T23:57:49.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.304 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.304 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:57:49.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990291+0000", 2024-09-14T23:57:49.305 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5900000000000002e-06 2024-09-14T23:57:49.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.305 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:49.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:38.984249+0000", 2024-09-14T23:57:49.305 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99395822700000003 2024-09-14T23:57:49.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.305 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:49.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:38.984448+0000", 2024-09-14T23:57:49.305 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00019855900000000001 2024-09-14T23:57:49.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.306 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:49.306 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:38.984601+0000", 2024-09-14T23:57:49.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000152824 2024-09-14T23:57:49.306 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.306 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:49.306 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.306 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.306 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4055 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1093632~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:49.306 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099066+0000", 2024-09-14T23:57:49.306 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 70.19884236, 2024-09-14T23:57:49.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044174782000000003, 2024-09-14T23:57:49.306 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:49.306 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:49.307 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:49.307 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:49.307 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:49.307 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:49.307 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4055 2024-09-14T23:57:49.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.307 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:49.307 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.307 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:49.307 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099066+0000", 2024-09-14T23:57:49.307 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.307 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.308 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:49.308 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099066+0000", 2024-09-14T23:57:49.308 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.308 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.308 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:49.308 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099066+0000", 2024-09-14T23:57:49.308 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3700000000000001e-07 2024-09-14T23:57:49.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.308 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.308 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:49.308 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099067+0000", 2024-09-14T23:57:49.308 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1379999999999999e-06 2024-09-14T23:57:49.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.309 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.309 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:49.309 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099073+0000", 2024-09-14T23:57:49.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4790000000000004e-06 2024-09-14T23:57:49.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.309 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.309 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:49.309 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099075+0000", 2024-09-14T23:57:49.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5320000000000001e-06 2024-09-14T23:57:49.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.309 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.309 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:49.309 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143147+0000", 2024-09-14T23:57:49.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044071676999999997 2024-09-14T23:57:49.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.310 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:49.310 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143186+0000", 2024-09-14T23:57:49.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8821e-05 2024-09-14T23:57:49.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.310 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:49.310 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143241+0000", 2024-09-14T23:57:49.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4797999999999999e-05 2024-09-14T23:57:49.310 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.310 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:49.310 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.311 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4056 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1155072~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:49.311 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099079+0000", 2024-09-14T23:57:49.311 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 70.198829645000004, 2024-09-14T23:57:49.311 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044295020999999997, 2024-09-14T23:57:49.311 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:49.311 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:49.311 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:49.311 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:49.311 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:49.311 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:49.311 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4056 2024-09-14T23:57:49.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.312 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:49.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.312 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:49.312 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099079+0000", 2024-09-14T23:57:49.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.312 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:49.313 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099079+0000", 2024-09-14T23:57:49.313 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.313 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.313 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.313 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:49.313 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099079+0000", 2024-09-14T23:57:49.313 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1500000000000001e-07 2024-09-14T23:57:49.313 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.313 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.313 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:49.313 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099080+0000", 2024-09-14T23:57:49.313 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0210000000000001e-06 2024-09-14T23:57:49.313 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.313 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.314 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:49.314 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099083+0000", 2024-09-14T23:57:49.314 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.174e-06 2024-09-14T23:57:49.314 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.314 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.314 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:49.314 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099086+0000", 2024-09-14T23:57:49.314 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.446e-06 2024-09-14T23:57:49.314 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.314 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.314 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:49.314 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143269+0000", 2024-09-14T23:57:49.315 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044183489999999999 2024-09-14T23:57:49.315 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.315 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.315 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:49.315 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143314+0000", 2024-09-14T23:57:49.315 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5135000000000001e-05 2024-09-14T23:57:49.315 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.316 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.316 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:49.316 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143374+0000", 2024-09-14T23:57:49.316 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9540000000000003e-05 2024-09-14T23:57:49.316 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.316 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:49.316 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.316 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.316 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.316 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4057 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1216512~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:49.316 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099088+0000", 2024-09-14T23:57:49.316 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 70.198819857999993, 2024-09-14T23:57:49.317 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044406119000000001, 2024-09-14T23:57:49.317 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:49.317 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:49.317 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:49.317 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:49.317 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:49.317 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:49.317 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4057 2024-09-14T23:57:49.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.317 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:49.317 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.317 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:49.317 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099088+0000", 2024-09-14T23:57:49.317 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.318 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.318 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:49.318 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099088+0000", 2024-09-14T23:57:49.318 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.318 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.318 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:49.318 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099089+0000", 2024-09-14T23:57:49.318 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.04e-07 2024-09-14T23:57:49.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.318 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.318 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:49.319 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099090+0000", 2024-09-14T23:57:49.319 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.026e-06 2024-09-14T23:57:49.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.319 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.319 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:49.319 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099093+0000", 2024-09-14T23:57:49.319 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0819999999999999e-06 2024-09-14T23:57:49.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.319 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.319 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:49.319 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099095+0000", 2024-09-14T23:57:49.319 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4930000000000001e-06 2024-09-14T23:57:49.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.319 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.319 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:49.320 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143400+0000", 2024-09-14T23:57:49.320 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044304698000000003 2024-09-14T23:57:49.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.320 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.320 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:49.320 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143441+0000", 2024-09-14T23:57:49.320 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1100000000000003e-05 2024-09-14T23:57:49.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.320 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.320 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:49.320 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143495+0000", 2024-09-14T23:57:49.320 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3516000000000001e-05 2024-09-14T23:57:49.320 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.321 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:49.321 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.321 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.321 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.321 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4058 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1277952~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:49.321 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099099+0000", 2024-09-14T23:57:49.321 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 70.198809366999996, 2024-09-14T23:57:49.321 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044514583000000003, 2024-09-14T23:57:49.321 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:49.321 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:49.321 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:49.321 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:49.321 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:49.321 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:49.322 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4058 2024-09-14T23:57:49.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.322 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:49.322 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.322 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:49.322 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099099+0000", 2024-09-14T23:57:49.322 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.322 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.322 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:49.322 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099099+0000", 2024-09-14T23:57:49.322 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.322 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.323 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:49.323 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099099+0000", 2024-09-14T23:57:49.323 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9999999999999999e-07 2024-09-14T23:57:49.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.323 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.323 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:49.323 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099100+0000", 2024-09-14T23:57:49.323 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7199999999999997e-07 2024-09-14T23:57:49.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.323 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.323 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:49.323 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099103+0000", 2024-09-14T23:57:49.323 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.095e-06 2024-09-14T23:57:49.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.324 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.324 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:49.324 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099106+0000", 2024-09-14T23:57:49.324 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4229999999999999e-06 2024-09-14T23:57:49.324 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.324 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.324 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:49.324 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143524+0000", 2024-09-14T23:57:49.324 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044417891000000001 2024-09-14T23:57:49.324 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.324 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.324 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:49.324 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143568+0000", 2024-09-14T23:57:49.324 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4060000000000002e-05 2024-09-14T23:57:49.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.325 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.325 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:49.325 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143614+0000", 2024-09-14T23:57:49.325 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5942000000000002e-05 2024-09-14T23:57:49.325 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.325 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:49.325 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.326 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.326 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4059 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1339392~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:49.326 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099114+0000", 2024-09-14T23:57:49.326 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 70.198794828000004, 2024-09-14T23:57:49.326 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044617194999999998, 2024-09-14T23:57:49.326 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:49.326 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:49.326 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:49.326 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:49.326 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:49.326 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:49.326 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4059 2024-09-14T23:57:49.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.327 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:49.327 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.327 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:49.327 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099114+0000", 2024-09-14T23:57:49.327 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.327 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.327 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:49.327 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099114+0000", 2024-09-14T23:57:49.327 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.327 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.327 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:49.328 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099114+0000", 2024-09-14T23:57:49.328 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5899999999999998e-07 2024-09-14T23:57:49.328 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.328 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.328 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:49.328 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099115+0000", 2024-09-14T23:57:49.328 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3510000000000001e-06 2024-09-14T23:57:49.328 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.328 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.328 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:49.328 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099119+0000", 2024-09-14T23:57:49.328 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.16e-06 2024-09-14T23:57:49.328 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.329 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.329 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:49.329 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099123+0000", 2024-09-14T23:57:49.329 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1339999999999997e-06 2024-09-14T23:57:49.329 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.329 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.329 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:49.329 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143643+0000", 2024-09-14T23:57:49.329 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044519232999999998 2024-09-14T23:57:49.329 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.329 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.329 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:49.329 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143686+0000", 2024-09-14T23:57:49.330 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3349000000000001e-05 2024-09-14T23:57:49.330 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.330 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.330 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:49.330 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143731+0000", 2024-09-14T23:57:49.330 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4709000000000002e-05 2024-09-14T23:57:49.330 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.330 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:49.330 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.330 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.330 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.330 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4060 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1400832~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:49.330 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099129+0000", 2024-09-14T23:57:49.330 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 70.198779604999999, 2024-09-14T23:57:49.331 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044721271999999999, 2024-09-14T23:57:49.331 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:49.331 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:49.331 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:49.331 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:49.331 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:49.331 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:49.331 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4060 2024-09-14T23:57:49.331 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.331 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:49.331 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.331 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:49.331 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099129+0000", 2024-09-14T23:57:49.331 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.332 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.332 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:49.332 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099129+0000", 2024-09-14T23:57:49.332 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.332 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.332 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:49.332 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099191+0000", 2024-09-14T23:57:49.332 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1805999999999997e-05 2024-09-14T23:57:49.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.332 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.332 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:49.333 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099193+0000", 2024-09-14T23:57:49.333 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2330000000000001e-06 2024-09-14T23:57:49.333 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.333 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.333 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:49.333 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099200+0000", 2024-09-14T23:57:49.333 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4200000000000001e-06 2024-09-14T23:57:49.333 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.333 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.333 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:49.334 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099204+0000", 2024-09-14T23:57:49.334 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7500000000000001e-06 2024-09-14T23:57:49.334 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.334 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.334 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:49.334 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143760+0000", 2024-09-14T23:57:49.334 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044556182 2024-09-14T23:57:49.334 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.334 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.334 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:49.334 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143804+0000", 2024-09-14T23:57:49.334 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4239000000000001e-05 2024-09-14T23:57:49.334 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.334 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.335 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:49.335 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143850+0000", 2024-09-14T23:57:49.335 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5642000000000001e-05 2024-09-14T23:57:49.335 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.335 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:49.335 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.335 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.335 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4061 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1462272~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:49.335 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099215+0000", 2024-09-14T23:57:49.335 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 70.198693480000003, 2024-09-14T23:57:49.335 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044745659, 2024-09-14T23:57:49.335 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:49.336 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:49.336 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:49.336 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:49.336 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:49.336 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:49.336 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4061 2024-09-14T23:57:49.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.336 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:49.336 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.336 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:49.336 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099215+0000", 2024-09-14T23:57:49.336 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.337 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.337 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:49.337 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099215+0000", 2024-09-14T23:57:49.337 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.337 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.337 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:49.337 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100672+0000", 2024-09-14T23:57:49.337 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00145689 2024-09-14T23:57:49.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.337 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.337 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:49.337 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100677+0000", 2024-09-14T23:57:49.337 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0139999999999998e-06 2024-09-14T23:57:49.338 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.338 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.338 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:49.338 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100689+0000", 2024-09-14T23:57:49.338 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.183e-05 2024-09-14T23:57:49.338 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.338 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.338 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:49.338 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100696+0000", 2024-09-14T23:57:49.338 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2599999999999999e-06 2024-09-14T23:57:49.338 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.338 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.338 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:49.338 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143877+0000", 2024-09-14T23:57:49.339 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043181279000000003 2024-09-14T23:57:49.339 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.339 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.339 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:49.339 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143919+0000", 2024-09-14T23:57:49.339 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1783000000000003e-05 2024-09-14T23:57:49.339 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.339 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.339 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:49.339 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143961+0000", 2024-09-14T23:57:49.339 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1603000000000001e-05 2024-09-14T23:57:49.339 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.339 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:49.340 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.340 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.340 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.340 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4070 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 2015232~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:49.340 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.100903+0000", 2024-09-14T23:57:49.340 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 70.197005771999997, 2024-09-14T23:57:49.340 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044103815999999997, 2024-09-14T23:57:49.340 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:49.340 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:49.340 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:49.340 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:49.340 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:49.340 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:49.340 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4070 2024-09-14T23:57:49.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.341 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:49.341 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.341 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:49.341 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100903+0000", 2024-09-14T23:57:49.341 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.341 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.341 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:49.341 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100903+0000", 2024-09-14T23:57:49.341 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.341 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.341 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:49.341 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.102671+0000", 2024-09-14T23:57:49.342 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001768279 2024-09-14T23:57:49.342 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.343 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.343 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:49.343 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.102675+0000", 2024-09-14T23:57:49.343 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9609999999999999e-06 2024-09-14T23:57:49.343 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.343 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.343 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:49.343 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.102684+0000", 2024-09-14T23:57:49.343 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.329e-06 2024-09-14T23:57:49.343 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.343 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.344 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:49.344 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.102689+0000", 2024-09-14T23:57:49.344 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7079999999999996e-06 2024-09-14T23:57:49.344 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.344 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.344 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:49.344 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.144912+0000", 2024-09-14T23:57:49.344 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.042223305000000003 2024-09-14T23:57:49.344 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.345 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.345 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:49.345 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.144960+0000", 2024-09-14T23:57:49.345 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7670000000000003e-05 2024-09-14T23:57:49.345 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.345 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.345 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:49.345 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.145006+0000", 2024-09-14T23:57:49.345 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6563999999999997e-05 2024-09-14T23:57:49.345 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.345 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:49.345 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.346 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.346 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.346 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4096 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1282048~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:49.346 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.104957+0000", 2024-09-14T23:57:49.346 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 70.192951598999997, 2024-09-14T23:57:49.346 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044093016999999998, 2024-09-14T23:57:49.346 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:49.346 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:49.346 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:49.346 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:49.346 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:49.346 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:49.346 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4096 2024-09-14T23:57:49.347 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.347 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:49.347 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.347 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:49.347 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104957+0000", 2024-09-14T23:57:49.347 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.347 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.347 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.347 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:49.347 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104957+0000", 2024-09-14T23:57:49.347 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.347 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.347 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.347 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:49.348 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104957+0000", 2024-09-14T23:57:49.348 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7400000000000003e-07 2024-09-14T23:57:49.348 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.348 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.348 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:49.348 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104960+0000", 2024-09-14T23:57:49.348 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3800000000000001e-06 2024-09-14T23:57:49.348 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.348 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.348 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:49.348 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104967+0000", 2024-09-14T23:57:49.349 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4030000000000001e-06 2024-09-14T23:57:49.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.349 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.349 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:49.349 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104973+0000", 2024-09-14T23:57:49.349 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7370000000000003e-06 2024-09-14T23:57:49.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.349 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.349 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:49.349 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.148883+0000", 2024-09-14T23:57:49.349 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043910124000000002 2024-09-14T23:57:49.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.350 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.350 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:49.350 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.148966+0000", 2024-09-14T23:57:49.350 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2868999999999996e-05 2024-09-14T23:57:49.350 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.350 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.350 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:49.350 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149050+0000", 2024-09-14T23:57:49.350 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3930000000000004e-05 2024-09-14T23:57:49.350 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.350 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:49.350 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.350 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.350 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.351 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4097 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1343488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:49.351 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.104983+0000", 2024-09-14T23:57:49.351 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 70.192925810000006, 2024-09-14T23:57:49.351 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044182104999999999, 2024-09-14T23:57:49.351 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:49.351 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:49.351 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:49.351 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:49.351 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:49.351 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:49.351 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4097 2024-09-14T23:57:49.351 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.351 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:49.351 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.352 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:49.352 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104983+0000", 2024-09-14T23:57:49.352 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.352 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.352 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.352 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:49.352 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104983+0000", 2024-09-14T23:57:49.352 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.352 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.352 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.352 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:49.352 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105207+0000", 2024-09-14T23:57:49.352 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000224687 2024-09-14T23:57:49.353 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.353 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.353 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:49.353 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105211+0000", 2024-09-14T23:57:49.353 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3050000000000001e-06 2024-09-14T23:57:49.353 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.353 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.353 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:49.353 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105217+0000", 2024-09-14T23:57:49.353 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8170000000000001e-06 2024-09-14T23:57:49.353 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.353 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.353 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:49.354 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105222+0000", 2024-09-14T23:57:49.354 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.267e-06 2024-09-14T23:57:49.354 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.354 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.354 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:49.354 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149079+0000", 2024-09-14T23:57:49.354 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043857581999999999 2024-09-14T23:57:49.354 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.354 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.354 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:49.354 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149122+0000", 2024-09-14T23:57:49.354 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2899999999999999e-05 2024-09-14T23:57:49.354 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.354 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.355 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:49.355 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149165+0000", 2024-09-14T23:57:49.355 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2546999999999998e-05 2024-09-14T23:57:49.355 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.355 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:49.355 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.355 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.355 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.355 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4098 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1404928~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:49.355 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.105242+0000", 2024-09-14T23:57:49.355 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 70.192665997999995, 2024-09-14T23:57:49.355 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044121104000000001, 2024-09-14T23:57:49.355 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:49.355 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:49.356 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:49.356 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:49.356 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:49.356 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:49.356 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4098 2024-09-14T23:57:49.356 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.356 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:49.356 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.356 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:49.356 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105242+0000", 2024-09-14T23:57:49.356 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.356 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.357 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.357 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:49.357 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105242+0000", 2024-09-14T23:57:49.357 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.357 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.357 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.357 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:49.357 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105243+0000", 2024-09-14T23:57:49.357 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0399999999999995e-07 2024-09-14T23:57:49.357 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.357 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.357 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:49.357 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105246+0000", 2024-09-14T23:57:49.358 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6309999999999999e-06 2024-09-14T23:57:49.358 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.358 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.358 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:49.358 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105253+0000", 2024-09-14T23:57:49.358 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0940000000000002e-06 2024-09-14T23:57:49.358 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.358 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.358 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:49.358 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105260+0000", 2024-09-14T23:57:49.358 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1799999999999999e-06 2024-09-14T23:57:49.358 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.358 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.359 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:49.359 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149201+0000", 2024-09-14T23:57:49.359 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043940629000000002 2024-09-14T23:57:49.359 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.359 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.359 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:49.359 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149283+0000", 2024-09-14T23:57:49.359 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2360000000000004e-05 2024-09-14T23:57:49.359 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.359 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.359 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:49.359 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149363+0000", 2024-09-14T23:57:49.359 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0506000000000005e-05 2024-09-14T23:57:49.359 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.360 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:49.360 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.360 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.360 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.360 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4099 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1466368~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:49.360 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.105270+0000", 2024-09-14T23:57:49.360 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 70.192638403000004, 2024-09-14T23:57:49.360 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044195433999999999, 2024-09-14T23:57:49.360 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:49.360 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:49.360 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:49.360 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:49.360 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:49.361 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:49.361 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4099 2024-09-14T23:57:49.361 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.361 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:49.361 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.361 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:49.361 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105270+0000", 2024-09-14T23:57:49.361 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.361 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.361 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.361 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:49.361 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105270+0000", 2024-09-14T23:57:49.362 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.362 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.362 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.362 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:49.362 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105429+0000", 2024-09-14T23:57:49.362 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015940299999999999 2024-09-14T23:57:49.362 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.362 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.362 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:49.362 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105433+0000", 2024-09-14T23:57:49.362 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3579999999999999e-06 2024-09-14T23:57:49.363 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.363 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.363 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:49.363 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105439+0000", 2024-09-14T23:57:49.363 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1759999999999998e-06 2024-09-14T23:57:49.363 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.363 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.363 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:49.363 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105445+0000", 2024-09-14T23:57:49.363 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9689999999999999e-06 2024-09-14T23:57:49.363 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.363 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.364 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:49.364 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149381+0000", 2024-09-14T23:57:49.364 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043936389999999999 2024-09-14T23:57:49.364 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.364 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.364 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:49.364 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149422+0000", 2024-09-14T23:57:49.364 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0865000000000003e-05 2024-09-14T23:57:49.364 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.364 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.364 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:49.364 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149465+0000", 2024-09-14T23:57:49.364 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3272999999999997e-05 2024-09-14T23:57:49.365 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.365 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:49.365 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.365 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.365 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.365 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4100 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1527808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:49.365 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.105454+0000", 2024-09-14T23:57:49.365 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 70.192453932000006, 2024-09-14T23:57:49.365 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044217411999999998, 2024-09-14T23:57:49.365 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:49.365 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:49.365 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:49.365 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:49.366 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:49.366 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:49.366 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4100 2024-09-14T23:57:49.366 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.366 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:49.366 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.366 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:49.366 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105454+0000", 2024-09-14T23:57:49.366 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.366 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.366 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.366 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:49.366 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105454+0000", 2024-09-14T23:57:49.367 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.367 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.367 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.367 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:49.367 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105578+0000", 2024-09-14T23:57:49.367 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012401799999999999 2024-09-14T23:57:49.367 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.367 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.367 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:49.367 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105583+0000", 2024-09-14T23:57:49.367 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1779999999999996e-06 2024-09-14T23:57:49.367 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.367 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.367 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:49.368 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105590+0000", 2024-09-14T23:57:49.368 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4279999999999999e-06 2024-09-14T23:57:49.368 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.368 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.368 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:49.368 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105596+0000", 2024-09-14T23:57:49.368 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0120000000000002e-06 2024-09-14T23:57:49.368 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.368 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.368 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:49.368 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149503+0000", 2024-09-14T23:57:49.368 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043906580000000001 2024-09-14T23:57:49.368 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.369 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.369 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:49.369 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149585+0000", 2024-09-14T23:57:49.369 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2356999999999998e-05 2024-09-14T23:57:49.369 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.369 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.369 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:49.369 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149672+0000", 2024-09-14T23:57:49.369 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6839e-05 2024-09-14T23:57:49.369 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.369 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:49.369 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.369 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.370 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.370 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4101 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1589248~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:49.370 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.105609+0000", 2024-09-14T23:57:49.370 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 70.192299392999999, 2024-09-14T23:57:49.370 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044175987999999999, 2024-09-14T23:57:49.370 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:49.370 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:49.370 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:49.370 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:49.370 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:49.370 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:49.370 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4101 2024-09-14T23:57:49.370 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.371 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:49.371 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.371 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:49.371 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105609+0000", 2024-09-14T23:57:49.371 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.371 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.371 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.371 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:49.371 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105609+0000", 2024-09-14T23:57:49.371 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.371 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.371 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.372 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:49.372 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.106671+0000", 2024-09-14T23:57:49.372 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010619270000000001 2024-09-14T23:57:49.372 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.372 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.372 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:49.372 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.106675+0000", 2024-09-14T23:57:49.372 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4429999999999996e-06 2024-09-14T23:57:49.372 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.372 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.372 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:49.372 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.106685+0000", 2024-09-14T23:57:49.372 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7429999999999997e-06 2024-09-14T23:57:49.372 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.373 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.373 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:49.373 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.106690+0000", 2024-09-14T23:57:49.373 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.293e-06 2024-09-14T23:57:49.373 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.373 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.373 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:49.373 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149700+0000", 2024-09-14T23:57:49.373 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043009512 2024-09-14T23:57:49.373 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.373 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.373 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:49.373 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149742+0000", 2024-09-14T23:57:49.374 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2083999999999999e-05 2024-09-14T23:57:49.374 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.374 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.374 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:49.374 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149785+0000", 2024-09-14T23:57:49.374 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2985999999999998e-05 2024-09-14T23:57:49.374 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.374 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:49.374 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.374 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.374 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.374 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4116 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 180224~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:49.374 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.107575+0000", 2024-09-14T23:57:49.374 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 70.190332925000007, 2024-09-14T23:57:49.375 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044157522999999997, 2024-09-14T23:57:49.375 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:49.375 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:49.375 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:49.375 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:49.375 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:49.375 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:49.375 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4116 2024-09-14T23:57:49.375 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.375 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:49.375 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.375 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:49.375 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.107575+0000", 2024-09-14T23:57:49.375 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.376 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.376 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.376 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:49.376 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.107575+0000", 2024-09-14T23:57:49.376 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.376 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.376 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.376 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:49.376 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108671+0000", 2024-09-14T23:57:49.376 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010954510000000001 2024-09-14T23:57:49.376 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.376 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.377 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:49.377 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108676+0000", 2024-09-14T23:57:49.377 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6639999999999997e-06 2024-09-14T23:57:49.377 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.377 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.377 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:49.377 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108685+0000", 2024-09-14T23:57:49.377 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1130000000000007e-06 2024-09-14T23:57:49.377 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.377 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.377 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:49.378 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108691+0000", 2024-09-14T23:57:49.378 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6019999999999996e-06 2024-09-14T23:57:49.378 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.378 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.378 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:49.378 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.151530+0000", 2024-09-14T23:57:49.378 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.042838345999999999 2024-09-14T23:57:49.378 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.378 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.378 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:49.378 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.151619+0000", 2024-09-14T23:57:49.378 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9302999999999995e-05 2024-09-14T23:57:49.379 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.379 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.379 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:49.379 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.151733+0000", 2024-09-14T23:57:49.379 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011404399999999999 2024-09-14T23:57:49.379 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.379 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:49.379 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.379 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.379 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.379 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4122 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 548864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:49.379 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.108842+0000", 2024-09-14T23:57:49.379 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 70.189066087, 2024-09-14T23:57:49.380 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044103631999999997, 2024-09-14T23:57:49.380 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:49.380 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:49.380 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:49.380 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:49.380 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:49.380 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:49.380 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4122 2024-09-14T23:57:49.380 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.380 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:49.380 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.380 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:49.380 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108842+0000", 2024-09-14T23:57:49.380 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.381 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.381 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.381 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:49.381 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108842+0000", 2024-09-14T23:57:49.381 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.381 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.381 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.381 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:49.381 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108843+0000", 2024-09-14T23:57:49.381 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7000000000000001e-07 2024-09-14T23:57:49.381 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.381 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.381 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:49.382 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108844+0000", 2024-09-14T23:57:49.382 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6479999999999999e-06 2024-09-14T23:57:49.382 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.382 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.382 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:49.382 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108848+0000", 2024-09-14T23:57:49.382 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8029999999999998e-06 2024-09-14T23:57:49.382 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.382 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.382 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:49.382 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108851+0000", 2024-09-14T23:57:49.382 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7939999999999998e-06 2024-09-14T23:57:49.382 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.383 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.383 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:49.383 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.152838+0000", 2024-09-14T23:57:49.383 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043987351000000001 2024-09-14T23:57:49.383 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.383 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.383 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:49.383 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.152885+0000", 2024-09-14T23:57:49.383 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7237000000000002e-05 2024-09-14T23:57:49.383 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.383 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.383 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:49.383 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.152946+0000", 2024-09-14T23:57:49.383 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0529000000000003e-05 2024-09-14T23:57:49.383 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.384 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:49.384 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.384 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.384 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.384 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4123 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 610304~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:49.384 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.108855+0000", 2024-09-14T23:57:49.384 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 70.189053758, 2024-09-14T23:57:49.384 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044321610999999997, 2024-09-14T23:57:49.384 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:49.384 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:49.384 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:49.384 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:49.384 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:49.384 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:49.385 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4123 2024-09-14T23:57:49.385 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.385 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:49.385 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.385 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:49.385 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108855+0000", 2024-09-14T23:57:49.385 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.385 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.385 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.385 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:49.385 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108855+0000", 2024-09-14T23:57:49.385 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.385 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.386 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.386 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:49.386 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.110671+0000", 2024-09-14T23:57:49.386 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018159719999999999 2024-09-14T23:57:49.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.386 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.386 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:49.386 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.110676+0000", 2024-09-14T23:57:49.386 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9450000000000001e-06 2024-09-14T23:57:49.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.386 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.386 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:49.386 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.110685+0000", 2024-09-14T23:57:49.386 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5359999999999998e-06 2024-09-14T23:57:49.387 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.387 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.387 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:49.387 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.110692+0000", 2024-09-14T23:57:49.387 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9280000000000004e-06 2024-09-14T23:57:49.387 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.387 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.387 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:49.387 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.152984+0000", 2024-09-14T23:57:49.387 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.042291677999999999 2024-09-14T23:57:49.387 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.387 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.387 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:49.387 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.153066+0000", 2024-09-14T23:57:49.388 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2657999999999994e-05 2024-09-14T23:57:49.388 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.388 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.388 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:49.388 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.153176+0000", 2024-09-14T23:57:49.388 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000109894 2024-09-14T23:57:49.388 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.388 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:49.388 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.388 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.388 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.388 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4136 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:49.388 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.111056+0000", 2024-09-14T23:57:49.388 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 70.186852610000003, 2024-09-14T23:57:49.389 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044097428000000001, 2024-09-14T23:57:49.389 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:49.389 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:49.389 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:49.389 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:49.389 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:49.389 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:49.389 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4136 2024-09-14T23:57:49.389 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.389 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:49.389 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.389 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:49.389 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.111056+0000", 2024-09-14T23:57:49.390 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.390 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.390 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.390 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:49.390 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.111056+0000", 2024-09-14T23:57:49.390 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.390 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.390 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.390 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:49.390 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.112671+0000", 2024-09-14T23:57:49.390 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016153999999999999 2024-09-14T23:57:49.390 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.390 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.390 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:49.391 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.112675+0000", 2024-09-14T23:57:49.391 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2830000000000003e-06 2024-09-14T23:57:49.391 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.391 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.391 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:49.391 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.112684+0000", 2024-09-14T23:57:49.391 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7800000000000006e-06 2024-09-14T23:57:49.391 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.391 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.391 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:49.391 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.112689+0000", 2024-09-14T23:57:49.391 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0000000000000004e-06 2024-09-14T23:57:49.391 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.391 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.392 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:49.392 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.155064+0000", 2024-09-14T23:57:49.392 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.042374423000000001 2024-09-14T23:57:49.392 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.392 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.392 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:49.392 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.155111+0000", 2024-09-14T23:57:49.392 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7367000000000002e-05 2024-09-14T23:57:49.392 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.392 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.392 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:49.392 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.155153+0000", 2024-09-14T23:57:49.392 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2175000000000002e-05 2024-09-14T23:57:49.392 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.393 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:49.393 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.393 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.393 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:46572 3.1d 3:b9e1f524:::rbd_header.1118cee06433:head [watch reconnect cookie 140670782499824 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-14T23:57:49.393 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:57.046868+0000", 2024-09-14T23:57:49.393 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 52.251040803999999, 2024-09-14T23:57:49.393 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0243412620000001, 2024-09-14T23:57:49.393 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:49.393 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:49.393 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:49.393 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:49.394 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:49.394 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:49.394 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 46572 2024-09-14T23:57:49.394 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.394 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:49.394 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.394 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:49.394 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046868+0000", 2024-09-14T23:57:49.394 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.394 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.395 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.395 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:49.395 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046868+0000", 2024-09-14T23:57:49.395 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.395 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.395 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.395 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:49.395 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046870+0000", 2024-09-14T23:57:49.395 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.204e-06 2024-09-14T23:57:49.395 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.395 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.395 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:49.395 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046877+0000", 2024-09-14T23:57:49.396 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5100000000000001e-06 2024-09-14T23:57:49.396 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.396 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.396 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:49.396 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046898+0000", 2024-09-14T23:57:49.396 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1149999999999999e-05 2024-09-14T23:57:49.396 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.396 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.396 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:49.396 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046911+0000", 2024-09-14T23:57:49.396 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2234e-05 2024-09-14T23:57:49.397 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.397 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.397 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:49.397 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046988+0000", 2024-09-14T23:57:49.397 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7445000000000001e-05 2024-09-14T23:57:49.397 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.397 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.397 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:57:49.397 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.047001+0000", 2024-09-14T23:57:49.397 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2951e-05 2024-09-14T23:57:49.397 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.397 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.397 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:49.398 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.069287+0000", 2024-09-14T23:57:49.398 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.022285748 2024-09-14T23:57:49.398 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.398 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.398 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for rw locks", 2024-09-14T23:57:49.398 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.069397+0000", 2024-09-14T23:57:49.398 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000110326 2024-09-14T23:57:49.398 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.398 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.398 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:49.398 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.069997+0000", 2024-09-14T23:57:49.398 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00059984399999999998 2024-09-14T23:57:49.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.399 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.399 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:49.399 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.070109+0000", 2024-09-14T23:57:49.399 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000112367 2024-09-14T23:57:49.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.399 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.399 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:49.399 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.071209+0000", 2024-09-14T23:57:49.399 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010994830000000001 2024-09-14T23:57:49.399 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.399 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:49.400 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.400 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.400 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:49.400 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:49.401 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T23:57:49.402 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:49.399+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:49.405 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #37, image image_client.0-clone37 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent38 2024-09-14T23:57:49.410 INFO:teuthology.orchestra.run.smithi042.stdout:4455 punch from 0x1c7ba6a to 0x1c82cd9, (0x726f bytes) 2024-09-14T23:57:49.416 INFO:teuthology.orchestra.run.smithi042.stdout:4456 read 0x1fb94c thru 0x203154 (0x7809 bytes) 2024-09-14T23:57:49.416 INFO:teuthology.orchestra.run.smithi042.stdout:4457 read 0x23bb9e0 thru 0x23c52da (0x98fb bytes) 2024-09-14T23:57:49.417 INFO:teuthology.orchestra.run.smithi042.stdout:4461 punch from 0x7672d8 to 0x77559c, (0xe2c4 bytes) 2024-09-14T23:57:49.417 INFO:teuthology.orchestra.run.smithi042.stdout:4462 read 0x1db7e70 thru 0x1dc7b63 (0xfcf4 bytes) 2024-09-14T23:57:49.417 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:49.426 INFO:teuthology.orchestra.run.smithi042.stdout:4465 writesame 0x6cff75d thru 0x6d025d1 data_size 0x1(0x2e75 bytes) 2024-09-14T23:57:49.437 INFO:teuthology.orchestra.run.smithi042.stdout:4466 write 0xbd1da0a thru 0xbd21a2b (0x4022 bytes) 2024-09-14T23:57:49.441 INFO:teuthology.orchestra.run.smithi042.stdout:4467 read 0xad57117 thru 0xad62968 (0xb852 bytes) 2024-09-14T23:57:49.442 INFO:teuthology.orchestra.run.smithi042.stdout:4468 punch from 0x4dece1 to 0x4e9776, (0xaa95 bytes) 2024-09-14T23:57:49.442 INFO:teuthology.orchestra.run.smithi042.stdout:4469 read 0x91e7d8c thru 0x91f786b (0xfae0 bytes) 2024-09-14T23:57:49.442 INFO:teuthology.orchestra.run.smithi042.stdout:4471 write 0x932b912 thru 0x9336f85 (0xb674 bytes) 2024-09-14T23:57:49.446 INFO:teuthology.orchestra.run.smithi042.stdout:4474 punch from 0x168d3e to 0x1756a5, (0xc967 bytes) 2024-09-14T23:57:49.446 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:49.446 INFO:teuthology.orchestra.run.smithi042.stdout:4476 write 0xb4b80f4 thru 0xb4c40e1 (0xbfee bytes) 2024-09-14T23:57:49.451 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:49.452 INFO:teuthology.orchestra.run.smithi042.stdout:4479 trunc from 0xbd21a2c to 0xd2fa09a 2024-09-14T23:57:49.458 INFO:teuthology.orchestra.run.smithi042.stdout:4480 writesame 0xa8b6da2 thru 0xa8b9fdd data_size 0xa(0x323c bytes) 2024-09-14T23:57:49.468 INFO:teuthology.orchestra.run.smithi042.stdout:4481 trunc from 0xd2fa09a to 0x7afdf5e 2024-09-14T23:57:49.479 INFO:teuthology.orchestra.run.smithi042.stdout:4483 write 0x25c1853 thru 0x25c7dc4 (0x6572 bytes) 2024-09-14T23:57:49.483 INFO:teuthology.orchestra.run.smithi042.stdout:4484 read 0xc7f61a thru 0xc8309e (0x3a85 bytes) 2024-09-14T23:57:49.483 INFO:teuthology.orchestra.run.smithi042.stdout:4485 write 0x67ff185 thru 0x680436c (0x51e8 bytes) 2024-09-14T23:57:49.487 INFO:teuthology.orchestra.run.smithi042.stdout:4486 punch from 0x5d175c4 to 0x5d26b4d, (0xf589 bytes) 2024-09-14T23:57:49.487 INFO:teuthology.orchestra.run.smithi042.stdout:4487 trunc from 0x7afdf5e to 0x74204c9 2024-09-14T23:57:49.502 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:49.500+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:49.508 INFO:teuthology.orchestra.run.smithi042.stdout:4488 writesame 0xe958384 thru 0xe961a37 data_size 0x1e(0x96b4 bytes) 2024-09-14T23:57:49.515 INFO:teuthology.orchestra.run.smithi042.stdout:4489 trunc from 0xe961a38 to 0x8a49e4f 2024-09-14T23:57:49.522 INFO:teuthology.orchestra.run.smithi042.stdout:4492 write 0x1642f81 thru 0x165218a (0xf20a bytes) 2024-09-14T23:57:49.528 INFO:teuthology.orchestra.run.smithi042.stdout:4493 read 0x8667e8c thru 0x8670100 (0x8275 bytes) 2024-09-14T23:57:49.528 INFO:teuthology.orchestra.run.smithi042.stdout:4494 punch from 0x4fc56e8 to 0x4fd4d75, (0xf68d bytes) 2024-09-14T23:57:49.530 INFO:teuthology.orchestra.run.smithi042.stdout:4495 trunc from 0x8a49e4f to 0x77da96a 2024-09-14T23:57:49.535 INFO:teuthology.orchestra.run.smithi042.stdout:4496 read 0x73568e4 thru 0x735aa80 (0x419d bytes) 2024-09-14T23:57:49.535 INFO:teuthology.orchestra.run.smithi042.stdout:4497 read 0x3538971 thru 0x353d91c (0x4fac bytes) 2024-09-14T23:57:49.535 INFO:teuthology.orchestra.run.smithi042.stdout:4498 trunc from 0x77da96a to 0x761eee9 2024-09-14T23:57:49.541 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:49.541 INFO:teuthology.orchestra.run.smithi042.stdout:4500 punch from 0x4e9c2c2 to 0x4e9e227, (0x1f65 bytes) 2024-09-14T23:57:49.541 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:49.549 INFO:teuthology.orchestra.run.smithi042.stdout:4502 write 0xa2c7292 thru 0xa2cab21 (0x3890 bytes) 2024-09-14T23:57:49.552 INFO:teuthology.orchestra.run.smithi042.stdout:4503 write 0x92e6ee9 thru 0x92f476b (0xd883 bytes) 2024-09-14T23:57:49.557 INFO:teuthology.orchestra.run.smithi042.stdout:4504 punch from 0x73642fd to 0x73664f1, (0x21f4 bytes) 2024-09-14T23:57:49.557 INFO:teuthology.orchestra.run.smithi042.stdout:4505 read 0xfc1a70 thru 0xfcbf58 (0xa4e9 bytes) 2024-09-14T23:57:49.557 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:49.558 INFO:teuthology.orchestra.run.smithi042.stdout:4508 punch from 0x892950e to 0x892cac8, (0x35ba bytes) 2024-09-14T23:57:49.558 INFO:teuthology.orchestra.run.smithi042.stdout:4509 read 0x52aa08e thru 0x52b8b57 (0xeaca bytes) 2024-09-14T23:57:49.558 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:49.558 INFO:teuthology.orchestra.run.smithi042.stdout:4511 punch from 0x52fd429 to 0x530a006, (0xcbdd bytes) 2024-09-14T23:57:49.558 INFO:teuthology.orchestra.run.smithi042.stdout:4513 writesame 0x9bee20f thru 0x9bf40f5 data_size 0x2b(0x5ee7 bytes) 2024-09-14T23:57:49.561 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:49.561 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:49.561 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:57:49.561 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:49.561 INFO:teuthology.orchestra.run.smithi042.stdout:4514 write 0x5ae1cf3 thru 0x5af04c5 (0xe7d3 bytes) 2024-09-14T23:57:49.564 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:49.565 INFO:teuthology.orchestra.run.smithi042.stdout:4516 write 0x184cd7f thru 0x185821d (0xb49f bytes) 2024-09-14T23:57:49.571 INFO:teuthology.orchestra.run.smithi042.stdout:4517 read 0x3d29350 thru 0x3d2df15 (0x4bc6 bytes) 2024-09-14T23:57:49.571 INFO:teuthology.orchestra.run.smithi042.stdout:4518 writesame 0x9f0ec51 thru 0x9f16b5f data_size 0x1(0x7f0f bytes) 2024-09-14T23:57:49.572 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T23:57:49.585 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:49.588 INFO:teuthology.orchestra.run.smithi042.stdout:4521 trunc from 0xa2cab22 to 0xc84a3df 2024-09-14T23:57:49.594 INFO:teuthology.orchestra.run.smithi042.stdout:4522 write 0x53defd3 thru 0x53e244e (0x347c bytes) 2024-09-14T23:57:49.598 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:49.598 INFO:teuthology.orchestra.run.smithi042.stdout:4524 trunc from 0xc84a3df to 0xc3b4790 2024-09-14T23:57:49.602 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:49.601+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:49.604 INFO:teuthology.orchestra.run.smithi042.stdout:4526 read 0x937513d thru 0x9378435 (0x32f9 bytes) 2024-09-14T23:57:49.605 INFO:teuthology.orchestra.run.smithi042.stdout:4528 trunc from 0xc3b4790 to 0xb64c21d 2024-09-14T23:57:49.610 INFO:teuthology.orchestra.run.smithi042.stdout:4529 write 0x4d7733d thru 0x4d7e9d3 (0x7697 bytes) 2024-09-14T23:57:49.621 INFO:teuthology.orchestra.run.smithi042.stdout:4532 trunc from 0xb64c21d to 0x39c3f31 2024-09-14T23:57:49.650 INFO:teuthology.orchestra.run.smithi042.stdout:4533 write 0xb4c55c6 thru 0xb4d0ea6 (0xb8e1 bytes) 2024-09-14T23:57:49.655 INFO:teuthology.orchestra.run.smithi042.stdout:4534 punch from 0x47bec37 to 0x47c86c7, (0x9a90 bytes) 2024-09-14T23:57:49.655 INFO:teuthology.orchestra.run.smithi042.stdout:4535 read 0x6d3019a thru 0x6d3e06b (0xded2 bytes) 2024-09-14T23:57:49.664 INFO:teuthology.orchestra.run.smithi042.stdout:4536 write 0xc64014c thru 0xc64a220 (0xa0d5 bytes) 2024-09-14T23:57:49.674 INFO:teuthology.orchestra.run.smithi042.stdout:4537 write 0xd178f8d thru 0xd17ad9b (0x1e0f bytes) 2024-09-14T23:57:49.678 INFO:teuthology.orchestra.run.smithi042.stdout:4538 punch from 0x8cf099e to 0x8cf2a07, (0x2069 bytes) 2024-09-14T23:57:49.678 INFO:teuthology.orchestra.run.smithi042.stdout:4539 read 0x9e63d9a thru 0x9e66e74 (0x30db bytes) 2024-09-14T23:57:49.678 INFO:teuthology.orchestra.run.smithi042.stdout:4540 write 0xb386bd thru 0xb3d885 (0x51c9 bytes) 2024-09-14T23:57:49.680 INFO:teuthology.orchestra.run.smithi042.stdout:4542 flatten 2024-09-14T23:57:49.697 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:49.697 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:49.697 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:57:49.697 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:57:49.697 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:49.703 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:49.702+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:49.706 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T23:57:49.709 INFO:teuthology.orchestra.run.smithi042.stdout:4544 punch from 0xbd8ba37 to 0xbd9928f, (0xd858 bytes) 2024-09-14T23:57:49.718 INFO:teuthology.orchestra.run.smithi042.stdout:4545 trunc from 0xd17ad9c to 0x4d063f3 2024-09-14T23:57:49.734 INFO:teuthology.orchestra.run.smithi042.stdout:4546 writesame 0x2561158 thru 0x25703bc data_size 0x1(0xf265 bytes) 2024-09-14T23:57:49.765 INFO:teuthology.orchestra.run.smithi042.stdout:4547 read 0xf55158 thru 0xf55b2b (0x9d4 bytes) 2024-09-14T23:57:49.766 INFO:teuthology.orchestra.run.smithi042.stdout:4548 read 0x4082699 thru 0x408d0af (0xaa17 bytes) 2024-09-14T23:57:49.766 INFO:teuthology.orchestra.run.smithi042.stdout:4549 writesame 0xc1332f thru 0xc1daaf data_size 0x1(0xa781 bytes) 2024-09-14T23:57:49.785 INFO:teuthology.orchestra.run.smithi042.stdout:4551 punch from 0x24a45ec to 0x24ab486, (0x6e9a bytes) 2024-09-14T23:57:49.785 INFO:teuthology.orchestra.run.smithi042.stdout:4552 read 0x30cda56 thru 0x30dd49c (0xfa47 bytes) 2024-09-14T23:57:49.785 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:49.785 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:49.785 INFO:teuthology.orchestra.run.smithi042.stdout:4557 read 0x348f576 thru 0x3496c90 (0x771b bytes) 2024-09-14T23:57:49.786 INFO:teuthology.orchestra.run.smithi042.stdout:4558 punch from 0xb054d2 to 0xb0ed8c, (0x98ba bytes) 2024-09-14T23:57:49.796 INFO:teuthology.orchestra.run.smithi042.stdout:4559 write 0x94e8c9f thru 0x94f0305 (0x7667 bytes) 2024-09-14T23:57:49.804 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:49.803+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:49.806 INFO:teuthology.orchestra.run.smithi042.stdout:4563 trunc from 0x94f0306 to 0xe47e1e1 2024-09-14T23:57:49.813 INFO:teuthology.orchestra.run.smithi042.stdout:4564 read 0x69a4869 thru 0x69b17ef (0xcf87 bytes) 2024-09-14T23:57:49.819 INFO:teuthology.orchestra.run.smithi042.stdout:4565 writesame 0xeacd4cc thru 0xeadc5e7 data_size 0x1a(0xf11c bytes) 2024-09-14T23:57:49.820 INFO:teuthology.orchestra.run.smithi042.stdout:4566 write 0x1a41b82 thru 0x1a494f6 (0x7975 bytes) 2024-09-14T23:57:49.824 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:49.824 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:57:49.824 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:57:49.824 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:57:49.824 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.824 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4284.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-14T23:57:49.824 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:57:49.824 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 87.297243442999999, 2024-09-14T23:57:49.824 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.004942086, 2024-09-14T23:57:49.824 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:49.824 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:49.825 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:49.825 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:49.825 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4284", 2024-09-14T23:57:49.825 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/4278347926", 2024-09-14T23:57:49.825 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1 2024-09-14T23:57:49.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.825 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:49.825 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.825 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:49.825 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:57:49.825 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.825 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.826 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:49.826 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:57:49.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.826 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.826 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:49.826 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524331+0000", 2024-09-14T23:57:49.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.383e-05 2024-09-14T23:57:49.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.826 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.826 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:49.826 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524339+0000", 2024-09-14T23:57:49.827 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8290000000000008e-06 2024-09-14T23:57:49.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.827 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.827 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:49.827 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524359+0000", 2024-09-14T23:57:49.827 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9998000000000001e-05 2024-09-14T23:57:49.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.828 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.828 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:49.828 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524373+0000", 2024-09-14T23:57:49.828 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4304000000000001e-05 2024-09-14T23:57:49.828 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.828 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.828 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:49.828 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524488+0000", 2024-09-14T23:57:49.828 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114663 2024-09-14T23:57:49.828 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.828 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.828 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:57:49.828 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524501+0000", 2024-09-14T23:57:49.828 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3349999999999999e-05 2024-09-14T23:57:49.829 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.829 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.829 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:49.829 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.526868+0000", 2024-09-14T23:57:49.829 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023670090000000002 2024-09-14T23:57:49.829 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.829 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.829 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:49.829 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.527041+0000", 2024-09-14T23:57:49.829 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000172597 2024-09-14T23:57:49.829 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.829 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.829 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-14T23:57:49.829 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.527384+0000", 2024-09-14T23:57:49.830 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00034309500000000002 2024-09-14T23:57:49.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.830 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.830 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:49.830 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529103+0000", 2024-09-14T23:57:49.830 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017185220000000001 2024-09-14T23:57:49.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.830 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.830 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:57:49.830 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529188+0000", 2024-09-14T23:57:49.830 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5591000000000001e-05 2024-09-14T23:57:49.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.830 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.830 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:49.831 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529227+0000", 2024-09-14T23:57:49.831 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8563000000000003e-05 2024-09-14T23:57:49.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.831 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.831 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:49.831 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529259+0000", 2024-09-14T23:57:49.831 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2734999999999997e-05 2024-09-14T23:57:49.831 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.831 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:49.831 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.831 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.831 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4284.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-14T23:57:49.831 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:23.530683+0000", 2024-09-14T23:57:49.832 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 86.290877777999995, 2024-09-14T23:57:49.832 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0036133110000000001, 2024-09-14T23:57:49.832 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:49.832 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:49.832 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:49.832 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:49.832 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4284", 2024-09-14T23:57:49.832 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/4278347926", 2024-09-14T23:57:49.832 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5 2024-09-14T23:57:49.832 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.832 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:49.832 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.832 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:49.833 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530683+0000", 2024-09-14T23:57:49.833 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.833 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.833 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.833 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:49.833 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530683+0000", 2024-09-14T23:57:49.833 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.833 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.833 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.833 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:49.833 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530685+0000", 2024-09-14T23:57:49.833 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.075e-06 2024-09-14T23:57:49.833 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.834 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.834 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:49.834 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530695+0000", 2024-09-14T23:57:49.834 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0261999999999999e-05 2024-09-14T23:57:49.834 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.834 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.834 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:49.834 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530715+0000", 2024-09-14T23:57:49.834 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9930000000000001e-05 2024-09-14T23:57:49.834 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.834 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.834 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:49.834 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530724+0000", 2024-09-14T23:57:49.834 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.901e-06 2024-09-14T23:57:49.835 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.835 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.835 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:49.835 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530800+0000", 2024-09-14T23:57:49.835 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5873999999999999e-05 2024-09-14T23:57:49.835 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.835 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.835 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-14T23:57:49.835 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530807+0000", 2024-09-14T23:57:49.835 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2389999999999996e-06 2024-09-14T23:57:49.835 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.835 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.835 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:49.835 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.532494+0000", 2024-09-14T23:57:49.836 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016871830000000001 2024-09-14T23:57:49.836 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.836 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.836 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:49.836 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.532599+0000", 2024-09-14T23:57:49.836 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010472 2024-09-14T23:57:49.836 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.836 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.836 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1", 2024-09-14T23:57:49.836 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.532715+0000", 2024-09-14T23:57:49.836 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011568199999999999 2024-09-14T23:57:49.836 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.836 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.836 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:57:49.837 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.533460+0000", 2024-09-14T23:57:49.837 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00074494999999999997 2024-09-14T23:57:49.837 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.837 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.837 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:49.837 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534230+0000", 2024-09-14T23:57:49.837 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00077036599999999998 2024-09-14T23:57:49.837 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.837 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.837 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:49.837 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534266+0000", 2024-09-14T23:57:49.837 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5725000000000001e-05 2024-09-14T23:57:49.838 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.838 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.838 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:49.838 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534296+0000", 2024-09-14T23:57:49.838 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0403999999999999e-05 2024-09-14T23:57:49.838 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.838 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:49.838 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.838 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.838 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.838 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4284.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-14T23:57:49.838 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:24.535160+0000", 2024-09-14T23:57:49.838 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 85.286401136999999, 2024-09-14T23:57:49.838 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033473650000000001, 2024-09-14T23:57:49.839 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:49.839 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:49.839 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:49.839 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:49.839 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4284", 2024-09-14T23:57:49.839 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/4278347926", 2024-09-14T23:57:49.839 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 7 2024-09-14T23:57:49.839 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.839 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:49.839 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.839 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:49.839 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535160+0000", 2024-09-14T23:57:49.839 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.840 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.840 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.840 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:49.840 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535160+0000", 2024-09-14T23:57:49.840 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.840 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.840 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.840 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:49.840 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535162+0000", 2024-09-14T23:57:49.840 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3369999999999998e-06 2024-09-14T23:57:49.840 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.840 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.840 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:49.840 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535171+0000", 2024-09-14T23:57:49.840 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7700000000000007e-06 2024-09-14T23:57:49.841 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.841 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.841 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:49.841 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535189+0000", 2024-09-14T23:57:49.841 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7938e-05 2024-09-14T23:57:49.841 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.841 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.841 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:49.841 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535197+0000", 2024-09-14T23:57:49.841 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5639999999999997e-06 2024-09-14T23:57:49.841 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.841 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.841 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:49.841 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535266+0000", 2024-09-14T23:57:49.842 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8926999999999999e-05 2024-09-14T23:57:49.842 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.842 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.842 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-14T23:57:49.842 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535271+0000", 2024-09-14T23:57:49.842 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0329999999999999e-06 2024-09-14T23:57:49.842 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.842 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.842 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:49.842 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.537064+0000", 2024-09-14T23:57:49.842 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001792929 2024-09-14T23:57:49.842 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.842 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.842 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:49.843 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.537131+0000", 2024-09-14T23:57:49.843 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6605000000000004e-05 2024-09-14T23:57:49.843 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.843 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.843 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1", 2024-09-14T23:57:49.843 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.537271+0000", 2024-09-14T23:57:49.843 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013992400000000001 2024-09-14T23:57:49.843 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.843 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.843 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:57:49.843 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.538093+0000", 2024-09-14T23:57:49.843 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00082210200000000001 2024-09-14T23:57:49.843 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.843 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.844 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:49.844 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.538439+0000", 2024-09-14T23:57:49.844 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00034595999999999997 2024-09-14T23:57:49.844 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.844 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.844 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:49.844 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.538476+0000", 2024-09-14T23:57:49.844 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6770000000000002e-05 2024-09-14T23:57:49.844 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.844 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.844 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:49.844 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.538507+0000", 2024-09-14T23:57:49.844 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1506e-05 2024-09-14T23:57:49.844 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.845 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:49.845 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.845 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.845 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.845 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T23:57:49.845 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:34.406877+0000", 2024-09-14T23:57:49.845 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 75.414684132000005, 2024-09-14T23:57:49.845 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030351169999999999, 2024-09-14T23:57:49.845 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:49.845 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:49.845 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:49.845 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:49.845 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:49.845 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:49.846 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 10 2024-09-14T23:57:49.846 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.846 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:49.846 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.846 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:49.846 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406877+0000", 2024-09-14T23:57:49.846 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.846 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.846 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.846 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:49.846 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406877+0000", 2024-09-14T23:57:49.846 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.846 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.846 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.846 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:49.847 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406878+0000", 2024-09-14T23:57:49.847 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.291e-06 2024-09-14T23:57:49.847 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.847 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.847 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:49.847 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406883+0000", 2024-09-14T23:57:49.847 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2399999999999998e-06 2024-09-14T23:57:49.847 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.847 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.847 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:49.847 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406891+0000", 2024-09-14T23:57:49.847 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5889999999999997e-06 2024-09-14T23:57:49.847 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.847 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.847 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:49.848 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406898+0000", 2024-09-14T23:57:49.848 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3599999999999998e-06 2024-09-14T23:57:49.848 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.848 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.848 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:49.848 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406948+0000", 2024-09-14T23:57:49.848 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9435999999999999e-05 2024-09-14T23:57:49.848 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.848 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.848 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-14T23:57:49.848 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406953+0000", 2024-09-14T23:57:49.848 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2809999999999999e-06 2024-09-14T23:57:49.848 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.848 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.849 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:49.849 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.408559+0000", 2024-09-14T23:57:49.849 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00160607 2024-09-14T23:57:49.849 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.849 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.849 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:49.849 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.408637+0000", 2024-09-14T23:57:49.849 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8504000000000006e-05 2024-09-14T23:57:49.849 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.849 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.849 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 4", 2024-09-14T23:57:49.849 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.408781+0000", 2024-09-14T23:57:49.849 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014374799999999999 2024-09-14T23:57:49.849 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.849 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.850 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:57:49.850 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.409659+0000", 2024-09-14T23:57:49.850 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000877781 2024-09-14T23:57:49.850 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.850 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.850 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:49.850 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.409822+0000", 2024-09-14T23:57:49.850 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016295400000000001 2024-09-14T23:57:49.850 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.850 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.850 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:49.850 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.409872+0000", 2024-09-14T23:57:49.850 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0022000000000003e-05 2024-09-14T23:57:49.850 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.850 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.851 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:49.851 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.409912+0000", 2024-09-14T23:57:49.851 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9841e-05 2024-09-14T23:57:49.851 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.851 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:49.851 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.851 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.851 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.851 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:50626 3.d e51/46 3:b5154cfe:::rbd_data.11183b9aac34.0000000000000004:head v 51'63, mlcod=45'61)", 2024-09-14T23:57:49.851 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:01.140129+0000", 2024-09-14T23:57:49.851 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 48.681431531999998, 2024-09-14T23:57:49.851 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0029883850000000001, 2024-09-14T23:57:49.851 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:49.851 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:49.852 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:49.852 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:49.852 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.852 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:49.852 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.140129+0000", 2024-09-14T23:57:49.852 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.852 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.852 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.852 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:49.852 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.140129+0000", 2024-09-14T23:57:49.852 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.852 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.852 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.852 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:49.853 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.140130+0000", 2024-09-14T23:57:49.853 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0529999999999999e-06 2024-09-14T23:57:49.853 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.853 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.853 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:49.853 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.140311+0000", 2024-09-14T23:57:49.853 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018054200000000001 2024-09-14T23:57:49.853 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.853 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.853 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:49.853 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.140325+0000", 2024-09-14T23:57:49.853 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.382e-05 2024-09-14T23:57:49.853 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.853 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.854 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:49.854 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.140332+0000", 2024-09-14T23:57:49.854 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5700000000000004e-06 2024-09-14T23:57:49.854 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.854 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.854 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:49.854 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.140392+0000", 2024-09-14T23:57:49.854 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9296000000000003e-05 2024-09-14T23:57:49.854 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.854 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.854 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:49.854 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.140436+0000", 2024-09-14T23:57:49.854 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4515999999999999e-05 2024-09-14T23:57:49.854 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.855 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.855 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:49.855 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.143075+0000", 2024-09-14T23:57:49.855 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0026385369999999998 2024-09-14T23:57:49.855 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.855 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.855 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:49.855 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.143118+0000", 2024-09-14T23:57:49.855 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3050999999999998e-05 2024-09-14T23:57:49.855 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.855 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:49.855 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.855 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.855 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.856 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56383 3.c e56/49 3:36b698b0:::rbd_data.1118c4b3aebe.0000000000000000:head v 56'194, mlcod=56'192)", 2024-09-14T23:57:49.856 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.270483+0000", 2024-09-14T23:57:49.856 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 43.551077978000002, 2024-09-14T23:57:49.856 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0079685880000000004, 2024-09-14T23:57:49.856 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:49.856 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:49.856 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:49.856 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:49.856 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.856 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:49.856 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.270483+0000", 2024-09-14T23:57:49.856 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.856 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.857 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.857 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:49.857 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.270483+0000", 2024-09-14T23:57:49.857 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.857 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.857 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.857 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:49.857 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.270484+0000", 2024-09-14T23:57:49.857 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7400000000000002e-07 2024-09-14T23:57:49.857 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.857 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.857 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:49.857 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273680+0000", 2024-09-14T23:57:49.857 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031966680000000002 2024-09-14T23:57:49.858 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.858 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.858 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:49.858 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273697+0000", 2024-09-14T23:57:49.858 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6595e-05 2024-09-14T23:57:49.858 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.858 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.858 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:49.858 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273705+0000", 2024-09-14T23:57:49.858 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7530000000000001e-06 2024-09-14T23:57:49.858 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.858 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.858 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:49.858 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273763+0000", 2024-09-14T23:57:49.859 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8238e-05 2024-09-14T23:57:49.859 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.859 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.859 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:49.859 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273781+0000", 2024-09-14T23:57:49.859 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8281e-05 2024-09-14T23:57:49.859 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.859 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.859 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:49.859 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.278408+0000", 2024-09-14T23:57:49.859 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00462716 2024-09-14T23:57:49.859 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.859 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.859 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:49.859 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.278451+0000", 2024-09-14T23:57:49.860 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3019000000000003e-05 2024-09-14T23:57:49.860 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.860 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:49.860 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.860 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.860 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.860 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56384 3.c e56/49 3:36b698b0:::rbd_data.1118c4b3aebe.0000000000000000:head v 56'196, mlcod=56'192)", 2024-09-14T23:57:49.860 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:57:49.860 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 43.547045306000001, 2024-09-14T23:57:49.860 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0046707529999999997, 2024-09-14T23:57:49.860 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:49.860 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:49.860 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:49.861 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:49.861 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.861 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:49.861 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:57:49.861 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.861 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.861 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.861 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:49.861 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:57:49.861 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.861 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.861 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.861 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:49.862 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:57:49.862 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1699999999999997e-07 2024-09-14T23:57:49.862 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.862 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.862 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:49.862 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274523+0000", 2024-09-14T23:57:49.862 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9129999999999997e-06 2024-09-14T23:57:49.862 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.862 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.862 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:49.862 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274535+0000", 2024-09-14T23:57:49.862 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1788e-05 2024-09-14T23:57:49.862 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.862 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.863 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:49.863 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274542+0000", 2024-09-14T23:57:49.863 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0539999999999997e-06 2024-09-14T23:57:49.863 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.863 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.863 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:49.863 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275889+0000", 2024-09-14T23:57:49.863 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00134669 2024-09-14T23:57:49.863 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.863 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.863 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:49.863 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275917+0000", 2024-09-14T23:57:49.863 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7664000000000001e-05 2024-09-14T23:57:49.864 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.864 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.864 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:49.864 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279134+0000", 2024-09-14T23:57:49.864 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0032178459999999999 2024-09-14T23:57:49.864 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.864 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.864 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:49.864 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279186+0000", 2024-09-14T23:57:49.864 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1881e-05 2024-09-14T23:57:49.864 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.864 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:49.864 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.864 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.865 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.865 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56381 3.1d e56/49 3:bcf9a8cf:::rbd_data.1118c4b3aebe.0000000000000006:head v 56'210, mlcod=56'208)", 2024-09-14T23:57:49.865 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.275462+0000", 2024-09-14T23:57:49.865 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 43.546098976000003, 2024-09-14T23:57:49.865 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0071310710000000001, 2024-09-14T23:57:49.865 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:49.865 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:49.865 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:49.865 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:49.865 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.865 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:49.865 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275462+0000", 2024-09-14T23:57:49.865 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.865 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.866 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.866 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:49.866 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275462+0000", 2024-09-14T23:57:49.866 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.866 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.866 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.866 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:49.866 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275463+0000", 2024-09-14T23:57:49.866 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2699999999999999e-07 2024-09-14T23:57:49.866 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.866 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.866 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:49.866 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276779+0000", 2024-09-14T23:57:49.866 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0013163649999999999 2024-09-14T23:57:49.867 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.867 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.867 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:49.867 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276795+0000", 2024-09-14T23:57:49.867 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.59e-05 2024-09-14T23:57:49.867 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.867 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.867 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:49.867 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276806+0000", 2024-09-14T23:57:49.867 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1106e-05 2024-09-14T23:57:49.867 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.867 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.867 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:49.867 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276941+0000", 2024-09-14T23:57:49.868 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013477500000000001 2024-09-14T23:57:49.868 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.868 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.868 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:49.868 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276963+0000", 2024-09-14T23:57:49.868 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2419999999999999e-05 2024-09-14T23:57:49.868 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.868 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.868 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:49.868 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282546+0000", 2024-09-14T23:57:49.868 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0055827100000000003 2024-09-14T23:57:49.868 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.868 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.869 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:49.869 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282593+0000", 2024-09-14T23:57:49.869 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7067999999999997e-05 2024-09-14T23:57:49.869 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.869 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:49.869 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.869 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.869 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.869 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56387 3.c e56/49 3:30ee4ba0:::rbd_data.1118c4b3aebe.0000000000000004:head v 56'197, mlcod=56'192)", 2024-09-14T23:57:49.869 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.276498+0000", 2024-09-14T23:57:49.869 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 43.545062881, 2024-09-14T23:57:49.869 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0092192639999999996, 2024-09-14T23:57:49.869 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:49.870 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:49.870 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:49.870 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:49.870 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.870 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:49.870 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276498+0000", 2024-09-14T23:57:49.870 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.870 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.870 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.870 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:49.870 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276498+0000", 2024-09-14T23:57:49.870 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.870 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.870 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.871 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:49.871 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276499+0000", 2024-09-14T23:57:49.871 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1100000000000004e-07 2024-09-14T23:57:49.871 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.871 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.871 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:49.871 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279646+0000", 2024-09-14T23:57:49.871 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031473220000000001 2024-09-14T23:57:49.871 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.871 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.871 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:49.871 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279672+0000", 2024-09-14T23:57:49.871 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6129999999999999e-05 2024-09-14T23:57:49.872 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.872 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.872 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:49.872 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279682+0000", 2024-09-14T23:57:49.872 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7170000000000003e-06 2024-09-14T23:57:49.872 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.872 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.872 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:49.872 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279742+0000", 2024-09-14T23:57:49.872 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9926999999999997e-05 2024-09-14T23:57:49.872 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.873 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.873 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:49.873 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279761+0000", 2024-09-14T23:57:49.873 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9402000000000002e-05 2024-09-14T23:57:49.873 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.873 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.873 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:49.873 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.285680+0000", 2024-09-14T23:57:49.873 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.005918881 2024-09-14T23:57:49.873 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.873 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.873 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:49.874 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.285717+0000", 2024-09-14T23:57:49.874 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6974000000000002e-05 2024-09-14T23:57:49.874 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.874 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:49.874 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.874 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.874 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.874 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56388 3.c e56/49 3:30ee4ba0:::rbd_data.1118c4b3aebe.0000000000000004:head v 56'199, mlcod=56'192)", 2024-09-14T23:57:49.874 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:57:49.874 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 43.541571292999997, 2024-09-14T23:57:49.874 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0060596740000000001, 2024-09-14T23:57:49.874 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:49.874 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:49.875 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:49.875 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:49.875 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.875 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:49.875 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:57:49.875 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.875 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.875 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.875 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:49.875 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:57:49.875 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.875 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.875 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.875 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:49.876 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:57:49.876 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2200000000000003e-07 2024-09-14T23:57:49.876 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.876 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.876 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:49.876 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279995+0000", 2024-09-14T23:57:49.876 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5569999999999997e-06 2024-09-14T23:57:49.876 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.876 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.876 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:49.876 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280004+0000", 2024-09-14T23:57:49.876 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0080000000000008e-06 2024-09-14T23:57:49.876 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.877 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.877 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:49.877 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280010+0000", 2024-09-14T23:57:49.877 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5749999999999997e-06 2024-09-14T23:57:49.877 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.877 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.877 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:49.877 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282968+0000", 2024-09-14T23:57:49.877 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0029579390000000001 2024-09-14T23:57:49.877 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.877 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.877 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:49.877 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282998+0000", 2024-09-14T23:57:49.877 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9947999999999999e-05 2024-09-14T23:57:49.877 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.878 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.878 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:49.878 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.286012+0000", 2024-09-14T23:57:49.878 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030132900000000001 2024-09-14T23:57:49.878 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.878 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.878 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:49.878 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.286049+0000", 2024-09-14T23:57:49.878 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7534999999999999e-05 2024-09-14T23:57:49.878 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.878 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:49.878 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.878 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.879 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.879 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56382 3.1d e56/49 3:bcf9a8cf:::rbd_data.1118c4b3aebe.0000000000000006:head v 56'212, mlcod=56'208)", 2024-09-14T23:57:49.879 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.280285+0000", 2024-09-14T23:57:49.879 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 43.54127561, 2024-09-14T23:57:49.879 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035393489999999998, 2024-09-14T23:57:49.879 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:49.879 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:49.879 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:49.879 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:49.879 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.879 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:49.879 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280285+0000", 2024-09-14T23:57:49.879 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.879 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.879 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.880 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:49.880 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280285+0000", 2024-09-14T23:57:49.880 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.880 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.880 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.880 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:49.880 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280286+0000", 2024-09-14T23:57:49.880 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5000000000000001e-07 2024-09-14T23:57:49.880 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.880 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.880 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:49.880 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280299+0000", 2024-09-14T23:57:49.880 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3312000000000001e-05 2024-09-14T23:57:49.881 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.881 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.881 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:49.881 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280311+0000", 2024-09-14T23:57:49.881 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1677e-05 2024-09-14T23:57:49.881 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.881 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.881 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:49.881 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280318+0000", 2024-09-14T23:57:49.881 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.742e-06 2024-09-14T23:57:49.881 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.881 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.881 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:49.881 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280374+0000", 2024-09-14T23:57:49.882 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.592e-05 2024-09-14T23:57:49.882 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.882 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.882 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:49.882 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280392+0000", 2024-09-14T23:57:49.882 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8490000000000001e-05 2024-09-14T23:57:49.882 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.882 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.882 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:49.882 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.283693+0000", 2024-09-14T23:57:49.882 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033012050000000002 2024-09-14T23:57:49.882 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.882 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.883 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:49.883 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.283825+0000", 2024-09-14T23:57:49.883 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013105300000000001 2024-09-14T23:57:49.883 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.883 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:49.883 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.883 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.883 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.883 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "pg_update_log_missing(3.c epoch 56/49 rep_tid 1493 entries 56'200 (0'0) error 3:370203a3:::rbd_data.1118c4b3aebe.000000000000000d:head by client.4376.0:56396 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 56'192)", 2024-09-14T23:57:49.883 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:57:49.883 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 43.541051152999998, 2024-09-14T23:57:49.883 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057370099999999999, 2024-09-14T23:57:49.883 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:49.884 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:49.884 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:57:49.884 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:49.884 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.884 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:49.884 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:57:49.884 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.884 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.884 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.884 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:49.884 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:57:49.884 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.884 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.884 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.885 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:49.885 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:57:49.885 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0699999999999997e-07 2024-09-14T23:57:49.885 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.885 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.885 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:49.885 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280519+0000", 2024-09-14T23:57:49.885 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6260000000000001e-06 2024-09-14T23:57:49.885 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.885 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.885 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:49.885 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280541+0000", 2024-09-14T23:57:49.885 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1976000000000001e-05 2024-09-14T23:57:49.885 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.886 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.886 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:49.886 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280556+0000", 2024-09-14T23:57:49.886 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5075000000000001e-05 2024-09-14T23:57:49.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.886 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.886 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:49.886 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.285494+0000", 2024-09-14T23:57:49.886 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0049378690000000001 2024-09-14T23:57:49.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.886 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.886 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:49.886 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.286247+0000", 2024-09-14T23:57:49.886 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00075285700000000001 2024-09-14T23:57:49.887 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.887 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:49.887 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.887 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.887 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.887 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56400 3.1c e56/49 3:3f5b81ba:::rbd_data.1118c4b3aebe.000000000000000a:head v 56'236, mlcod=56'234)", 2024-09-14T23:57:49.887 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.295064+0000", 2024-09-14T23:57:49.887 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 43.526496361, 2024-09-14T23:57:49.887 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0054331589999999999, 2024-09-14T23:57:49.887 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:49.887 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:49.887 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:49.887 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:49.888 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.888 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:49.888 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.295064+0000", 2024-09-14T23:57:49.888 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.888 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.888 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:49.888 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.295064+0000", 2024-09-14T23:57:49.888 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.888 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.888 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:49.888 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.295066+0000", 2024-09-14T23:57:49.888 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.502e-06 2024-09-14T23:57:49.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.889 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.889 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:49.889 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296403+0000", 2024-09-14T23:57:49.889 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001336554 2024-09-14T23:57:49.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.889 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.889 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:49.889 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296428+0000", 2024-09-14T23:57:49.889 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5179000000000001e-05 2024-09-14T23:57:49.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.889 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.889 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:49.889 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296440+0000", 2024-09-14T23:57:49.889 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2387e-05 2024-09-14T23:57:49.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.890 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.890 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:49.890 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296554+0000", 2024-09-14T23:57:49.890 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114269 2024-09-14T23:57:49.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.890 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.890 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:49.890 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296581+0000", 2024-09-14T23:57:49.890 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6673999999999999e-05 2024-09-14T23:57:49.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.890 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.890 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:49.890 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.300418+0000", 2024-09-14T23:57:49.891 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0038371249999999998 2024-09-14T23:57:49.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.891 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.891 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:49.891 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.300498+0000", 2024-09-14T23:57:49.891 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9468999999999994e-05 2024-09-14T23:57:49.891 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.891 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:49.891 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.891 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.891 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56401 3.1c e56/49 3:3f5b81ba:::rbd_data.1118c4b3aebe.000000000000000a:head v 56'238, mlcod=56'234)", 2024-09-14T23:57:49.891 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.297640+0000", 2024-09-14T23:57:49.891 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 43.523920521999997, 2024-09-14T23:57:49.892 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031491919999999999, 2024-09-14T23:57:49.892 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:49.892 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:49.892 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:49.892 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:49.892 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.892 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:49.892 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.297640+0000", 2024-09-14T23:57:49.892 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.892 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.892 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.892 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:49.892 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.297640+0000", 2024-09-14T23:57:49.892 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.893 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.893 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.893 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:49.893 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.297641+0000", 2024-09-14T23:57:49.893 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9000000000000005e-07 2024-09-14T23:57:49.893 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.893 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.893 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:49.893 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.297649+0000", 2024-09-14T23:57:49.893 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4510000000000003e-06 2024-09-14T23:57:49.893 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.893 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.893 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:49.893 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.297663+0000", 2024-09-14T23:57:49.894 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4262e-05 2024-09-14T23:57:49.894 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.894 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.894 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:49.894 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.297679+0000", 2024-09-14T23:57:49.894 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5798e-05 2024-09-14T23:57:49.894 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.894 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.894 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:49.894 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.298408+0000", 2024-09-14T23:57:49.894 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00072923699999999998 2024-09-14T23:57:49.894 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.894 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.894 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:49.895 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.298430+0000", 2024-09-14T23:57:49.895 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1753999999999999e-05 2024-09-14T23:57:49.895 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.895 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.895 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:49.895 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.300749+0000", 2024-09-14T23:57:49.895 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023190590000000001 2024-09-14T23:57:49.895 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.895 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.895 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:49.895 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.300789+0000", 2024-09-14T23:57:49.895 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0640999999999999e-05 2024-09-14T23:57:49.896 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.896 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:49.896 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.896 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.896 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.896 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:61921 3.5 e57/49 3:a05d004f:::rbd_data.11188d356229.000000000000000b:head v 57'279, mlcod=57'277)", 2024-09-14T23:57:49.896 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:07.560389+0000", 2024-09-14T23:57:49.896 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 42.261171763, 2024-09-14T23:57:49.896 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0034956520000000001, 2024-09-14T23:57:49.896 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:49.896 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:49.896 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:49.896 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:49.896 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.896 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:49.897 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.560389+0000", 2024-09-14T23:57:49.897 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.897 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.897 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:49.897 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.560389+0000", 2024-09-14T23:57:49.897 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.897 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.897 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:49.897 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.560390+0000", 2024-09-14T23:57:49.897 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4109999999999999e-06 2024-09-14T23:57:49.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.898 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.898 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:49.898 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.561839+0000", 2024-09-14T23:57:49.898 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001448406 2024-09-14T23:57:49.898 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.898 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.898 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:49.898 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.561854+0000", 2024-09-14T23:57:49.898 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5426000000000001e-05 2024-09-14T23:57:49.898 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.898 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.898 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:49.898 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.561863+0000", 2024-09-14T23:57:49.898 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7509999999999997e-06 2024-09-14T23:57:49.898 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.899 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.899 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:49.899 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.561917+0000", 2024-09-14T23:57:49.899 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3531e-05 2024-09-14T23:57:49.899 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.899 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.899 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:49.899 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.561944+0000", 2024-09-14T23:57:49.899 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7528000000000001e-05 2024-09-14T23:57:49.899 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.899 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.899 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:49.899 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563843+0000", 2024-09-14T23:57:49.899 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001899361 2024-09-14T23:57:49.900 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.900 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.900 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:49.900 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563885+0000", 2024-09-14T23:57:49.900 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1238000000000001e-05 2024-09-14T23:57:49.900 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.900 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:49.900 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.900 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.900 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.900 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:61922 3.5 e57/49 3:a11ebf8e:::rbd_data.11188d356229.000000000000000c:head v 57'280, mlcod=57'277)", 2024-09-14T23:57:49.900 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:07.563144+0000", 2024-09-14T23:57:49.900 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 42.258416709000002, 2024-09-14T23:57:49.900 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003977848, 2024-09-14T23:57:49.901 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:49.901 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:49.901 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:49.901 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:49.901 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.901 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:49.901 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563144+0000", 2024-09-14T23:57:49.901 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.901 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.901 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.901 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:49.901 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563144+0000", 2024-09-14T23:57:49.901 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.901 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.901 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.902 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:49.902 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563145+0000", 2024-09-14T23:57:49.902 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5400000000000001e-07 2024-09-14T23:57:49.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.902 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.902 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:49.902 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565056+0000", 2024-09-14T23:57:49.902 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0019108160000000001 2024-09-14T23:57:49.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.902 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.902 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:49.902 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565088+0000", 2024-09-14T23:57:49.902 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2586999999999998e-05 2024-09-14T23:57:49.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.903 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.903 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:49.903 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565100+0000", 2024-09-14T23:57:49.903 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1917e-05 2024-09-14T23:57:49.903 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.903 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.903 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:49.903 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565166+0000", 2024-09-14T23:57:49.903 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6245e-05 2024-09-14T23:57:49.903 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.903 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.903 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:49.903 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565190+0000", 2024-09-14T23:57:49.904 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3825999999999999e-05 2024-09-14T23:57:49.904 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.904 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.904 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:49.904 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.567080+0000", 2024-09-14T23:57:49.904 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001889541 2024-09-14T23:57:49.904 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.904 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.904 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:49.904 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.567122+0000", 2024-09-14T23:57:49.904 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2262000000000003e-05 2024-09-14T23:57:49.904 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.904 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:49.904 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.905 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.905 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.905 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:66564 3.c e60/49 3:33faa7ca:::rbd_data.111873792d9b.0000000000000003:head v 60'236, mlcod=57'233)", 2024-09-14T23:57:49.905 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:11.271596+0000", 2024-09-14T23:57:49.905 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 38.549965131, 2024-09-14T23:57:49.905 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0060396099999999999, 2024-09-14T23:57:49.905 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:49.905 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:49.905 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:49.905 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:49.905 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.905 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:49.905 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.271596+0000", 2024-09-14T23:57:49.905 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.906 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.906 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:49.906 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.271596+0000", 2024-09-14T23:57:49.906 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.906 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.906 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:49.906 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.271597+0000", 2024-09-14T23:57:49.906 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2559999999999999e-06 2024-09-14T23:57:49.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.906 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.906 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:49.906 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.274854+0000", 2024-09-14T23:57:49.907 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0032574470000000001 2024-09-14T23:57:49.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.907 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.907 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:49.907 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.274874+0000", 2024-09-14T23:57:49.907 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9449999999999998e-05 2024-09-14T23:57:49.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.907 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.907 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:49.907 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.274883+0000", 2024-09-14T23:57:49.907 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5980000000000003e-06 2024-09-14T23:57:49.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.907 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.907 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:49.908 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.274977+0000", 2024-09-14T23:57:49.908 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3270999999999995e-05 2024-09-14T23:57:49.908 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.908 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.908 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:49.908 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.275000+0000", 2024-09-14T23:57:49.908 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3088e-05 2024-09-14T23:57:49.908 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.908 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.908 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:49.908 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.277590+0000", 2024-09-14T23:57:49.908 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0025898660000000001 2024-09-14T23:57:49.908 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.908 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.909 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:49.909 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.277635+0000", 2024-09-14T23:57:49.909 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5633999999999997e-05 2024-09-14T23:57:49.909 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.909 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:49.909 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.909 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.909 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.909 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:130509 3.2 e88/75 3:4432e346:::rbd_data.111847077d5e.0000000000000005:head v 88'468, mlcod=87'466)", 2024-09-14T23:57:49.909 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:40.669710+0000", 2024-09-14T23:57:49.909 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 9.1518512839999993, 2024-09-14T23:57:49.909 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031769279999999999, 2024-09-14T23:57:49.909 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:49.910 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:49.910 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:49.910 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:49.910 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.910 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:49.910 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:40.669710+0000", 2024-09-14T23:57:49.910 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.910 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.910 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.910 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:49.910 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:40.669710+0000", 2024-09-14T23:57:49.910 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.910 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.910 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.911 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:49.911 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:40.669710+0000", 2024-09-14T23:57:49.911 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2699999999999999e-07 2024-09-14T23:57:49.911 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.911 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.911 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:49.911 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:40.671376+0000", 2024-09-14T23:57:49.911 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016661480000000001 2024-09-14T23:57:49.911 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.911 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.911 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:49.911 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:40.671389+0000", 2024-09-14T23:57:49.911 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2311e-05 2024-09-14T23:57:49.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.912 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.912 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:49.912 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:40.671395+0000", 2024-09-14T23:57:49.912 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6869999999999997e-06 2024-09-14T23:57:49.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.912 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.912 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:49.912 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:40.671451+0000", 2024-09-14T23:57:49.912 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5865000000000001e-05 2024-09-14T23:57:49.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.912 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.912 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:49.912 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:40.671475+0000", 2024-09-14T23:57:49.913 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3266999999999999e-05 2024-09-14T23:57:49.913 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.913 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.913 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:49.913 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:40.672858+0000", 2024-09-14T23:57:49.913 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0013835340000000001 2024-09-14T23:57:49.913 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.913 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.913 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:49.913 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:40.672886+0000", 2024-09-14T23:57:49.913 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8388999999999999e-05 2024-09-14T23:57:49.913 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.913 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:49.914 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.914 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.914 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.914 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:132869 3.16 e92/75 3:681b4799:::rbd_data.11186132a8b6.0000000000000001:head v 92'262, mlcod=87'260)", 2024-09-14T23:57:49.914 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:45.195492+0000", 2024-09-14T23:57:49.914 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.6260683780000003, 2024-09-14T23:57:49.914 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0034654880000000001, 2024-09-14T23:57:49.914 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:49.914 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:49.914 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:49.914 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:49.914 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.914 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:49.915 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.195492+0000", 2024-09-14T23:57:49.915 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.915 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.915 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:49.915 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.195492+0000", 2024-09-14T23:57:49.915 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.915 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.915 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:49.915 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.195494+0000", 2024-09-14T23:57:49.915 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.905e-06 2024-09-14T23:57:49.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.915 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.916 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:49.916 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.196549+0000", 2024-09-14T23:57:49.916 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010545489999999999 2024-09-14T23:57:49.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.916 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.916 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:49.916 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.196572+0000", 2024-09-14T23:57:49.916 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3603000000000001e-05 2024-09-14T23:57:49.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.916 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.916 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:49.916 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.196584+0000", 2024-09-14T23:57:49.916 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.169e-05 2024-09-14T23:57:49.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.917 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.917 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:49.917 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.196676+0000", 2024-09-14T23:57:49.917 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2088000000000003e-05 2024-09-14T23:57:49.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.917 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.917 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:49.917 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.196699+0000", 2024-09-14T23:57:49.917 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2365999999999999e-05 2024-09-14T23:57:49.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.917 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.917 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:49.917 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.198910+0000", 2024-09-14T23:57:49.917 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0022118609999999999 2024-09-14T23:57:49.918 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.918 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.918 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:49.918 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.198958+0000", 2024-09-14T23:57:49.918 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7426000000000003e-05 2024-09-14T23:57:49.918 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.918 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:49.918 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.918 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.918 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.918 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:133384 3.1f e92/49 3:fce5e529:::rbd_data.11186132a8b6.0000000000000002:head v 92'807, mlcod=89'804)", 2024-09-14T23:57:49.918 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:45.293642+0000", 2024-09-14T23:57:49.918 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.5279192479999999, 2024-09-14T23:57:49.918 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003760186, 2024-09-14T23:57:49.919 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:49.919 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:49.919 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:49.919 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:49.919 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.919 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:49.919 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.293642+0000", 2024-09-14T23:57:49.919 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.919 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.919 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:49.919 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.293642+0000", 2024-09-14T23:57:49.919 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:49.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.919 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.920 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:49.920 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.293643+0000", 2024-09-14T23:57:49.920 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2500000000000001e-06 2024-09-14T23:57:49.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.920 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.920 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:49.920 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.295402+0000", 2024-09-14T23:57:49.920 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017595410000000001 2024-09-14T23:57:49.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.920 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.920 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:49.920 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.295422+0000", 2024-09-14T23:57:49.920 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9763999999999999e-05 2024-09-14T23:57:49.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.921 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.921 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:49.921 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.295436+0000", 2024-09-14T23:57:49.921 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.364e-05 2024-09-14T23:57:49.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.921 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.921 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:49.921 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.295499+0000", 2024-09-14T23:57:49.921 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2770999999999999e-05 2024-09-14T23:57:49.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.921 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.921 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:49.921 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.295516+0000", 2024-09-14T23:57:49.922 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7240000000000001e-05 2024-09-14T23:57:49.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.922 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.922 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:49.922 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.297324+0000", 2024-09-14T23:57:49.922 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001807857 2024-09-14T23:57:49.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:49.922 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:49.922 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:49.922 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.297402+0000", 2024-09-14T23:57:49.922 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8122999999999997e-05 2024-09-14T23:57:49.922 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.922 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:49.922 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.923 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:49.923 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:49.923 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:49.925 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T23:57:49.925 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:49.922+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:49.927 INFO:teuthology.orchestra.run.smithi042.stdout:4568 trunc from 0xeadc5e8 to 0x8e53dfb 2024-09-14T23:57:49.927 INFO:teuthology.orchestra.run.smithi042.stdout:4569 read 0x4e6f519 thru 0x4e78476 (0x8f5e bytes) 2024-09-14T23:57:49.927 INFO:teuthology.orchestra.run.smithi042.stdout:4573 read 0x3074dd4 thru 0x308359e (0xe7cb bytes) 2024-09-14T23:57:49.928 INFO:teuthology.orchestra.run.smithi042.stdout:4574 write 0xa172bdb thru 0xa17e467 (0xb88d bytes) 2024-09-14T23:57:49.928 INFO:teuthology.orchestra.run.smithi042.stdout:4575 writesame 0xdd1a21d thru 0xdd1d39e data_size 0x1(0x3182 bytes) 2024-09-14T23:57:49.928 INFO:teuthology.orchestra.run.smithi042.stdout:4576 read 0x2ba8365 thru 0x2bb6c12 (0xe8ae bytes) 2024-09-14T23:57:49.928 INFO:teuthology.orchestra.run.smithi042.stdout:4577 writesame 0xd3a7d71 thru 0xd3b022d data_size 0x2f(0x84bd bytes) 2024-09-14T23:57:49.928 INFO:teuthology.orchestra.run.smithi042.stdout:4578 write 0x6e675ee thru 0x6e6779c (0x1af bytes) 2024-09-14T23:57:49.928 INFO:teuthology.orchestra.run.smithi042.stdout:4580 write 0xc272050 thru 0xc27e395 (0xc346 bytes) 2024-09-14T23:57:49.928 INFO:teuthology.orchestra.run.smithi042.stdout:4581 read 0xa7535a thru 0xa81603 (0xc2aa bytes) 2024-09-14T23:57:49.928 INFO:teuthology.orchestra.run.smithi042.stdout:4582 write 0x1bdb1c3 thru 0x1bdb315 (0x153 bytes) 2024-09-14T23:57:49.928 INFO:teuthology.orchestra.run.smithi042.stdout:4583 write 0xa3fa364 thru 0xa4099fa (0xf697 bytes) 2024-09-14T23:57:49.928 INFO:teuthology.orchestra.run.smithi042.stdout:4584 write 0xb7f54b2 thru 0xb7f7e42 (0x2991 bytes) 2024-09-14T23:57:49.928 INFO:teuthology.orchestra.run.smithi042.stdout:4586 read 0xa56c8c6 thru 0xa56d4e8 (0xc23 bytes) 2024-09-14T23:57:49.928 INFO:teuthology.orchestra.run.smithi042.stdout:4590 write 0xbddf976 thru 0xbde4fc8 (0x5653 bytes) 2024-09-14T23:57:49.928 INFO:teuthology.orchestra.run.smithi042.stdout:4592 trunc from 0xdd1d39f to 0xe78d6e8 2024-09-14T23:57:49.928 INFO:teuthology.orchestra.run.smithi042.stdout:4593 write 0xa8d6c26 thru 0xa8dc04e (0x5429 bytes) 2024-09-14T23:57:49.929 INFO:teuthology.orchestra.run.smithi042.stdout:4595 write 0x4c54de5 thru 0x4c59d26 (0x4f42 bytes) 2024-09-14T23:57:49.929 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:49.929 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:49.929 INFO:teuthology.orchestra.run.smithi042.stdout:4599 read 0x3bf9e6 thru 0x3c6b45 (0x7160 bytes) 2024-09-14T23:57:49.929 INFO:teuthology.orchestra.run.smithi042.stdout:4600 writesame 0xcd15aec thru 0xcd17da7 data_size 0x27(0x22bc bytes) 2024-09-14T23:57:49.929 INFO:teuthology.orchestra.run.smithi042.stdout:4602 writesame 0x7754d01 thru 0x775e811 data_size 0x35(0x9b11 bytes) 2024-09-14T23:57:49.929 INFO:teuthology.orchestra.run.smithi042.stdout:4603 read 0xbe791ab thru 0xbe8362b (0xa481 bytes) 2024-09-14T23:57:49.929 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:49.929 INFO:teuthology.orchestra.run.smithi042.stdout:4606 read 0x1ba4b16 thru 0x1ba75b5 (0x2aa0 bytes) 2024-09-14T23:57:49.929 INFO:teuthology.orchestra.run.smithi042.stdout:4607 writesame 0x249ef2c thru 0x24a1384 data_size 0x1(0x2459 bytes) 2024-09-14T23:57:49.929 INFO:teuthology.orchestra.run.smithi042.stdout:4608 read 0xdece12e thru 0xded971c (0xb5ef bytes) 2024-09-14T23:57:49.929 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:49.929 INFO:teuthology.orchestra.run.smithi042.stdout:4610 read 0x62e0663 thru 0x62e967c (0x901a bytes) 2024-09-14T23:57:49.930 INFO:teuthology.orchestra.run.smithi042.stdout:4611 writesame 0x8fe2362 thru 0x8febb06 data_size 0x1(0x97a5 bytes) 2024-09-14T23:57:49.930 INFO:teuthology.orchestra.run.smithi042.stdout:4612 writesame 0xa8eda7d thru 0xa8fac1b data_size 0x1(0xd19f bytes) 2024-09-14T23:57:49.938 INFO:teuthology.orchestra.run.smithi042.stdout:4613 writesame 0x7a9ba56 thru 0x7a9d9f9 data_size 0x3c(0x1fa4 bytes) 2024-09-14T23:57:49.942 INFO:teuthology.orchestra.run.smithi042.stdout:4616 read 0x618d66 thru 0x61aca3 (0x1f3e bytes) 2024-09-14T23:57:49.942 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:49.942 INFO:teuthology.orchestra.run.smithi042.stdout:4619 punch from 0x2e2d62b to 0x2e3627e, (0x8c53 bytes) 2024-09-14T23:57:49.942 INFO:teuthology.orchestra.run.smithi042.stdout:4620 writesame 0xc54db33 thru 0xc5533c4 data_size 0x3(0x5892 bytes) 2024-09-14T23:57:49.947 INFO:teuthology.orchestra.run.smithi042.stdout:4621 read 0x73e1dee thru 0x73e26f2 (0x905 bytes) 2024-09-14T23:57:49.947 INFO:teuthology.orchestra.run.smithi042.stdout:4623 read 0xbfc0dca thru 0xbfc71e3 (0x641a bytes) 2024-09-14T23:57:49.947 INFO:teuthology.orchestra.run.smithi042.stdout:4624 write 0xc074922 thru 0xc0809f2 (0xc0d1 bytes) 2024-09-14T23:57:49.949 INFO:teuthology.orchestra.run.smithi042.stdout:4625 trunc from 0xe78d6e8 to 0xe472c05 2024-09-14T23:57:49.954 INFO:teuthology.orchestra.run.smithi042.stdout:4626 writesame 0x36618aa thru 0x3662209 data_size 0x3(0x960 bytes) 2024-09-14T23:57:49.956 INFO:teuthology.orchestra.run.smithi042.stdout:4628 read 0xc5e1519 thru 0xc5ecc08 (0xb6f0 bytes) 2024-09-14T23:57:49.956 INFO:teuthology.orchestra.run.smithi042.stdout:4629 write 0x5b4f520 thru 0x5b5e4b6 (0xef97 bytes) 2024-09-14T23:57:49.963 INFO:teuthology.orchestra.run.smithi042.stdout:4631 trunc from 0xe472c05 to 0xc30b6ef 2024-09-14T23:57:49.975 INFO:teuthology.orchestra.run.smithi042.stdout:4632 punch from 0x684e121 to 0x6852f92, (0x4e71 bytes) 2024-09-14T23:57:49.975 INFO:teuthology.orchestra.run.smithi042.stdout:4633 read 0x16b5603 thru 0x16be59a (0x8f98 bytes) 2024-09-14T23:57:49.975 INFO:teuthology.orchestra.run.smithi042.stdout:4634 write 0x23b71ce thru 0x23be496 (0x72c9 bytes) 2024-09-14T23:57:49.977 INFO:teuthology.orchestra.run.smithi042.stdout:4635 read 0xb0fd5a9 thru 0xb10d30c (0xfd64 bytes) 2024-09-14T23:57:49.977 INFO:teuthology.orchestra.run.smithi042.stdout:4636 read 0x567587b thru 0x567aa52 (0x51d8 bytes) 2024-09-14T23:57:49.977 INFO:teuthology.orchestra.run.smithi042.stdout:4637 read 0xa53971c thru 0xa53bb9c (0x2481 bytes) 2024-09-14T23:57:49.977 INFO:teuthology.orchestra.run.smithi042.stdout:4639 punch from 0x612e1f6 to 0x6134f9b, (0x6da5 bytes) 2024-09-14T23:57:49.977 INFO:teuthology.orchestra.run.smithi042.stdout:4640 writesame 0x57325ce thru 0x57407d7 data_size 0x1(0xe20a bytes) 2024-09-14T23:57:50.004 INFO:teuthology.orchestra.run.smithi042.stdout:4641 read 0x3077323 thru 0x30858ab (0xe589 bytes) 2024-09-14T23:57:50.004 INFO:teuthology.orchestra.run.smithi042.stdout:4643 writesame 0xb2e4b23 thru 0xb2ee666 data_size 0x26(0x9b44 bytes) 2024-09-14T23:57:50.014 INFO:teuthology.orchestra.run.smithi042.stdout:4644 trunc from 0xc30b6ef to 0x7789026 2024-09-14T23:57:50.025 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:50.023+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:50.027 INFO:teuthology.orchestra.run.smithi042.stdout:4645 read 0x1d05793 thru 0x1d0cb94 (0x7402 bytes) 2024-09-14T23:57:50.027 INFO:teuthology.orchestra.run.smithi042.stdout:4646 clone 40 order 19 su 8192 sc 7 2024-09-14T23:57:50.082 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:50.082 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:50.082 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:57:50.082 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:50.091 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T23:57:50.134 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:50.132+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:50.202 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:50.203 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:50.203 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:57:50.203 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:57:50.203 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:50.209 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #38, image image_client.0-clone38 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent39 2024-09-14T23:57:50.210 INFO:teuthology.orchestra.run.smithi042.stdout:4647 write 0x4f31da8 thru 0x4f3fc77 (0xded0 bytes) 2024-09-14T23:57:50.212 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T23:57:50.226 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:50.225+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:50.236 INFO:teuthology.orchestra.run.smithi042.stdout:4648 write 0xac978dc thru 0xaca0d0d (0x9432 bytes) 2024-09-14T23:57:50.244 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T23:57:50.244 INFO:teuthology.orchestra.run.smithi042.stdout:4649 punch from 0xcd707d to 0xcd835a, (0x12dd bytes) 2024-09-14T23:57:50.251 INFO:teuthology.orchestra.run.smithi042.stdout:4650 write 0xc9ea6be thru 0xc9f7188 (0xcacb bytes) 2024-09-14T23:57:50.261 INFO:teuthology.orchestra.run.smithi042.stdout:4652 punch from 0x4839bbb to 0x483ac57, (0x109c bytes) 2024-09-14T23:57:50.262 INFO:teuthology.orchestra.run.smithi042.stdout:4653 read 0x8aca010 thru 0x8acc2e9 (0x22da bytes) 2024-09-14T23:57:50.262 INFO:teuthology.orchestra.run.smithi042.stdout:4654 punch from 0x40fc5a6 to 0x4106383, (0x9ddd bytes) 2024-09-14T23:57:50.262 INFO:teuthology.orchestra.run.smithi042.stdout:4656 punch from 0xa1465e5 to 0xa148eb0, (0x28cb bytes) 2024-09-14T23:57:50.262 INFO:teuthology.orchestra.run.smithi042.stdout:4657 read 0xb58a814 thru 0xb59362a (0x8e17 bytes) 2024-09-14T23:57:50.262 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:50.262 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:50.262 INFO:teuthology.orchestra.run.smithi042.stdout:4660 writesame 0x944b994 thru 0x9450b3b data_size 0x8(0x51a8 bytes) 2024-09-14T23:57:50.274 INFO:teuthology.orchestra.run.smithi042.stdout:4661 trunc from 0xc9f7189 to 0x6a8288d 2024-09-14T23:57:50.317 INFO:teuthology.orchestra.run.smithi042.stdout:4662 write 0xed7c6dd thru 0xed83f58 (0x787c bytes) 2024-09-14T23:57:50.323 INFO:teuthology.orchestra.run.smithi042.stdout:4663 punch from 0xe45291e to 0xe458532, (0x5c14 bytes) 2024-09-14T23:57:50.323 INFO:teuthology.orchestra.run.smithi042.stdout:4665 writesame 0x7edd0db thru 0x7edd3e5 data_size 0x1(0x30b bytes) 2024-09-14T23:57:50.326 INFO:teuthology.orchestra.run.smithi042.stdout:4667 read 0xdb97c2 thru 0xdc59ce (0xc20d bytes) 2024-09-14T23:57:50.327 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:50.325+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:50.327 INFO:teuthology.orchestra.run.smithi042.stdout:4670 write 0x55aa0ed thru 0x55ad917 (0x382b bytes) 2024-09-14T23:57:50.336 INFO:teuthology.orchestra.run.smithi042.stdout:4674 punch from 0xa4a3f74 to 0xa4ae805, (0xa891 bytes) 2024-09-14T23:57:50.336 INFO:teuthology.orchestra.run.smithi042.stdout:4675 writesame 0x99eb46c thru 0x99f5d24 data_size 0x1(0xa8b9 bytes) 2024-09-14T23:57:50.348 INFO:teuthology.orchestra.run.smithi042.stdout:4676 write 0x45c664d thru 0x45d4b45 (0xe4f9 bytes) 2024-09-14T23:57:50.362 INFO:teuthology.orchestra.run.smithi042.stdout:4677 write 0x35c7c9d thru 0x35c8b7b (0xedf bytes) 2024-09-14T23:57:50.369 INFO:teuthology.orchestra.run.smithi042.stdout:4679 trunc from 0xed83f59 to 0xeabf79d 2024-09-14T23:57:50.376 INFO:teuthology.orchestra.run.smithi042.stdout:4681 write 0xae6486f thru 0xae6fd37 (0xb4c9 bytes) 2024-09-14T23:57:50.379 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:50.379 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:57:50.380 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:57:50.380 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:57:50.380 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.380 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:107472 3.1e 3:7bf21588:::rbd_data.11187de8b0a5.000000000000005d:head [set-alloc-hint object_size 2097152 write_size 2097152,write 317788~1491] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-14T23:57:50.380 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:25.198493+0000", 2024-09-14T23:57:50.380 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 25.178817364, 2024-09-14T23:57:50.380 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.78175149300000002, 2024-09-14T23:57:50.380 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:50.380 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:50.380 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:50.380 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:50.380 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:50.381 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:50.381 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 107472 2024-09-14T23:57:50.381 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.381 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:50.381 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.381 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:50.381 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198493+0000", 2024-09-14T23:57:50.381 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:50.381 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.381 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.381 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:50.381 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198493+0000", 2024-09-14T23:57:50.381 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:50.381 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.382 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.382 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:50.382 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198495+0000", 2024-09-14T23:57:50.382 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.826e-06 2024-09-14T23:57:50.382 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.382 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.382 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:50.382 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198503+0000", 2024-09-14T23:57:50.382 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0150000000000003e-06 2024-09-14T23:57:50.382 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.382 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.382 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:50.382 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198521+0000", 2024-09-14T23:57:50.383 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8374000000000001e-05 2024-09-14T23:57:50.383 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.383 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.383 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:50.383 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198533+0000", 2024-09-14T23:57:50.383 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1593000000000001e-05 2024-09-14T23:57:50.383 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.383 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.383 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:50.383 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198603+0000", 2024-09-14T23:57:50.383 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0279999999999998e-05 2024-09-14T23:57:50.383 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.383 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.384 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:57:50.384 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198617+0000", 2024-09-14T23:57:50.384 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3517e-05 2024-09-14T23:57:50.384 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.384 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.384 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:50.384 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.977067+0000", 2024-09-14T23:57:50.384 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.77845065800000002 2024-09-14T23:57:50.384 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.384 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.384 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for missing object", 2024-09-14T23:57:50.384 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.977173+0000", 2024-09-14T23:57:50.384 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010543999999999999 2024-09-14T23:57:50.385 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.385 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.385 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:50.385 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.978737+0000", 2024-09-14T23:57:50.385 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001564078 2024-09-14T23:57:50.385 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.385 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.385 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:50.385 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.978845+0000", 2024-09-14T23:57:50.385 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000108643 2024-09-14T23:57:50.385 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.385 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.385 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1", 2024-09-14T23:57:50.385 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.978964+0000", 2024-09-14T23:57:50.386 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000119085 2024-09-14T23:57:50.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.386 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.386 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:50.386 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980109+0000", 2024-09-14T23:57:50.386 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0011442170000000001 2024-09-14T23:57:50.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.386 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.386 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:57:50.386 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980130+0000", 2024-09-14T23:57:50.386 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1766000000000001e-05 2024-09-14T23:57:50.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.386 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.386 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:50.386 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980171+0000", 2024-09-14T23:57:50.387 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0627999999999998e-05 2024-09-14T23:57:50.387 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.387 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.387 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:50.387 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980244+0000", 2024-09-14T23:57:50.387 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3373000000000004e-05 2024-09-14T23:57:50.387 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:50.387 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:50.387 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:50.387 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.387 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.387 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129149 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 6619136~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:50.387 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.925871+0000", 2024-09-14T23:57:50.387 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 11.451438921999999, 2024-09-14T23:57:50.388 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054291066999999998, 2024-09-14T23:57:50.388 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:50.388 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:50.388 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:50.388 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:50.388 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:50.388 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:50.388 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129149 2024-09-14T23:57:50.388 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.388 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:50.388 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.388 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:50.388 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.925871+0000", 2024-09-14T23:57:50.389 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:50.389 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.389 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.389 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:50.389 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.925871+0000", 2024-09-14T23:57:50.389 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:50.389 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.389 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.389 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:50.389 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.927669+0000", 2024-09-14T23:57:50.389 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001797838 2024-09-14T23:57:50.389 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.389 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.389 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:50.390 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.927672+0000", 2024-09-14T23:57:50.390 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5280000000000002e-06 2024-09-14T23:57:50.390 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.390 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.390 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:50.390 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.927678+0000", 2024-09-14T23:57:50.390 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0340000000000002e-06 2024-09-14T23:57:50.390 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.390 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.390 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:50.390 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.927681+0000", 2024-09-14T23:57:50.390 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.681e-06 2024-09-14T23:57:50.390 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.390 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.391 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:50.391 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.979979+0000", 2024-09-14T23:57:50.391 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052297950000000003 2024-09-14T23:57:50.391 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.391 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.391 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:50.391 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.980024+0000", 2024-09-14T23:57:50.391 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4693000000000001e-05 2024-09-14T23:57:50.391 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.391 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.391 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:50.391 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.980162+0000", 2024-09-14T23:57:50.391 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000138343 2024-09-14T23:57:50.391 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:50.392 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:50.392 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:50.392 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.392 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.392 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129159 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 5996544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:50.392 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.927783+0000", 2024-09-14T23:57:50.392 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 11.449527473, 2024-09-14T23:57:50.392 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054661122999999999, 2024-09-14T23:57:50.392 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:50.392 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:50.392 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:50.392 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:50.392 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:50.392 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:50.393 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129159 2024-09-14T23:57:50.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.393 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:50.393 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.393 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:50.393 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.927783+0000", 2024-09-14T23:57:50.393 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:50.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.393 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.393 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:50.393 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.927783+0000", 2024-09-14T23:57:50.393 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:50.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.393 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.394 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:50.394 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929684+0000", 2024-09-14T23:57:50.394 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0019011080000000001 2024-09-14T23:57:50.394 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.394 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.394 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:50.394 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929688+0000", 2024-09-14T23:57:50.394 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2610000000000004e-06 2024-09-14T23:57:50.394 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.394 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.394 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:50.394 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929697+0000", 2024-09-14T23:57:50.394 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0219999999999993e-06 2024-09-14T23:57:50.395 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.395 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.395 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:50.395 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929702+0000", 2024-09-14T23:57:50.395 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8840000000000002e-06 2024-09-14T23:57:50.395 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.395 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.395 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:50.395 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.982090+0000", 2024-09-14T23:57:50.395 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052388045000000001 2024-09-14T23:57:50.395 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.395 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.395 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:50.395 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.982208+0000", 2024-09-14T23:57:50.395 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118236 2024-09-14T23:57:50.396 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.396 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.396 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:50.396 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.982444+0000", 2024-09-14T23:57:50.396 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000235567 2024-09-14T23:57:50.396 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:50.396 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:50.396 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:50.396 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.396 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.396 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129166 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 8192000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:50.396 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.929794+0000", 2024-09-14T23:57:50.396 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 11.447516722, 2024-09-14T23:57:50.396 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054471973, 2024-09-14T23:57:50.397 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:50.397 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:50.397 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:50.397 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:50.397 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:50.397 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:50.397 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129166 2024-09-14T23:57:50.397 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.397 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:50.397 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.397 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:50.397 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929794+0000", 2024-09-14T23:57:50.397 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:50.398 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.398 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.398 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:50.398 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929794+0000", 2024-09-14T23:57:50.398 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:50.398 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.398 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.398 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:50.398 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929794+0000", 2024-09-14T23:57:50.398 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6899999999999997e-07 2024-09-14T23:57:50.398 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.398 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.398 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:50.399 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929797+0000", 2024-09-14T23:57:50.399 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.328e-06 2024-09-14T23:57:50.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.399 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.399 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:50.399 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929802+0000", 2024-09-14T23:57:50.399 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7309999999999998e-06 2024-09-14T23:57:50.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.399 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.399 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:50.399 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929807+0000", 2024-09-14T23:57:50.399 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4349999999999999e-06 2024-09-14T23:57:50.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.399 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.400 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:50.400 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984063+0000", 2024-09-14T23:57:50.400 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054256331999999997 2024-09-14T23:57:50.400 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.400 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.400 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:50.400 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984177+0000", 2024-09-14T23:57:50.400 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114225 2024-09-14T23:57:50.400 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.400 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.400 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:50.400 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984266+0000", 2024-09-14T23:57:50.400 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8352999999999999e-05 2024-09-14T23:57:50.401 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:50.401 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:50.401 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:50.401 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.401 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.401 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129167 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 6782976~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:50.401 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.929827+0000", 2024-09-14T23:57:50.401 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 11.447483312999999, 2024-09-14T23:57:50.401 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054637593999999998, 2024-09-14T23:57:50.401 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:50.401 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:50.401 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:50.401 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:50.401 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:50.401 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:50.402 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129167 2024-09-14T23:57:50.402 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.402 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:50.402 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.402 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:50.402 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929827+0000", 2024-09-14T23:57:50.402 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:50.402 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.402 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.402 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:50.402 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929827+0000", 2024-09-14T23:57:50.402 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:50.402 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.402 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.403 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:50.403 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929827+0000", 2024-09-14T23:57:50.403 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4000000000000002e-07 2024-09-14T23:57:50.403 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.403 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.403 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:50.403 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929830+0000", 2024-09-14T23:57:50.403 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.419e-06 2024-09-14T23:57:50.403 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.403 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.403 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:50.403 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929837+0000", 2024-09-14T23:57:50.403 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0450000000000003e-06 2024-09-14T23:57:50.403 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.404 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.404 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:50.404 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929842+0000", 2024-09-14T23:57:50.404 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.69e-06 2024-09-14T23:57:50.404 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.404 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.404 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:50.404 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984278+0000", 2024-09-14T23:57:50.404 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054436516999999997 2024-09-14T23:57:50.404 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.404 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.404 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:50.404 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984331+0000", 2024-09-14T23:57:50.405 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2590999999999998e-05 2024-09-14T23:57:50.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.405 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.405 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:50.405 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984465+0000", 2024-09-14T23:57:50.405 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013389200000000001 2024-09-14T23:57:50.405 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:50.405 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:50.405 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:50.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.405 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.405 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129168 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 8224768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:50.405 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.929849+0000", 2024-09-14T23:57:50.405 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 11.447461755000001, 2024-09-14T23:57:50.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054870368000000003, 2024-09-14T23:57:50.406 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:50.406 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:50.406 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:50.406 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:50.406 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:50.406 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:50.406 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129168 2024-09-14T23:57:50.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.406 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:50.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.406 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:50.406 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929849+0000", 2024-09-14T23:57:50.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:50.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.407 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:50.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929849+0000", 2024-09-14T23:57:50.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:50.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.407 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:50.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931681+0000", 2024-09-14T23:57:50.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018326989999999999 2024-09-14T23:57:50.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.407 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:50.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931686+0000", 2024-09-14T23:57:50.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7190000000000001e-06 2024-09-14T23:57:50.408 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.408 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.408 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:50.408 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931697+0000", 2024-09-14T23:57:50.408 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1287999999999999e-05 2024-09-14T23:57:50.408 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.408 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.408 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:50.408 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931704+0000", 2024-09-14T23:57:50.408 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3339999999999998e-06 2024-09-14T23:57:50.408 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.408 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.408 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:50.408 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984505+0000", 2024-09-14T23:57:50.408 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052801684000000002 2024-09-14T23:57:50.409 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.409 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.409 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:50.409 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984621+0000", 2024-09-14T23:57:50.409 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000116165 2024-09-14T23:57:50.409 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.409 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.409 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:50.409 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984719+0000", 2024-09-14T23:57:50.409 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7479000000000006e-05 2024-09-14T23:57:50.409 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:50.409 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:50.409 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:50.409 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.410 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.410 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129175 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 9011200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:50.410 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.931806+0000", 2024-09-14T23:57:50.410 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 11.445504736, 2024-09-14T23:57:50.410 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054322384000000001, 2024-09-14T23:57:50.410 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:50.410 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:50.410 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:50.410 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:50.410 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:50.410 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:50.410 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129175 2024-09-14T23:57:50.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.411 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:50.411 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.411 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:50.411 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931806+0000", 2024-09-14T23:57:50.411 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:50.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.411 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.411 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:50.411 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931806+0000", 2024-09-14T23:57:50.411 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:50.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.411 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.411 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:50.411 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931806+0000", 2024-09-14T23:57:50.411 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2300000000000001e-07 2024-09-14T23:57:50.412 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.412 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.412 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:50.412 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931808+0000", 2024-09-14T23:57:50.412 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1440000000000001e-06 2024-09-14T23:57:50.412 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.412 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.412 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:50.412 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931815+0000", 2024-09-14T23:57:50.412 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.776e-06 2024-09-14T23:57:50.412 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.412 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.412 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:50.413 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931821+0000", 2024-09-14T23:57:50.413 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.733e-06 2024-09-14T23:57:50.413 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.413 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.413 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:50.413 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.986046+0000", 2024-09-14T23:57:50.413 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054225284999999998 2024-09-14T23:57:50.413 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.413 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.413 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:50.413 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.986089+0000", 2024-09-14T23:57:50.413 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2895999999999997e-05 2024-09-14T23:57:50.413 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.413 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.414 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:50.414 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.986128+0000", 2024-09-14T23:57:50.414 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9026999999999998e-05 2024-09-14T23:57:50.414 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:50.414 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:50.414 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:50.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.414 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.414 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129176 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 10452992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:50.414 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.931830+0000", 2024-09-14T23:57:50.414 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 11.445479993999999, 2024-09-14T23:57:50.414 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054513292999999997, 2024-09-14T23:57:50.414 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:50.414 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:50.414 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:50.415 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:50.415 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:50.415 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:50.415 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129176 2024-09-14T23:57:50.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.415 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:50.415 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.415 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:50.415 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931830+0000", 2024-09-14T23:57:50.415 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:50.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.415 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.415 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:50.415 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931830+0000", 2024-09-14T23:57:50.416 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:50.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.416 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.416 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:50.416 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.933678+0000", 2024-09-14T23:57:50.416 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018472440000000001 2024-09-14T23:57:50.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.416 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.416 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:50.416 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.933686+0000", 2024-09-14T23:57:50.416 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0239999999999997e-06 2024-09-14T23:57:50.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.416 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.416 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:50.417 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.933694+0000", 2024-09-14T23:57:50.417 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8400000000000001e-06 2024-09-14T23:57:50.417 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.417 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.417 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:50.417 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.933701+0000", 2024-09-14T23:57:50.417 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2870000000000001e-06 2024-09-14T23:57:50.417 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.417 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.417 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:50.417 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.986165+0000", 2024-09-14T23:57:50.417 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052463900000000001 2024-09-14T23:57:50.417 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.418 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:50.418 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.986259+0000", 2024-09-14T23:57:50.418 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4435e-05 2024-09-14T23:57:50.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.418 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:50.418 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.986344+0000", 2024-09-14T23:57:50.418 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4562999999999997e-05 2024-09-14T23:57:50.418 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:50.418 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:50.418 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:50.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.419 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129201 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 10289152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:50.419 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.935960+0000", 2024-09-14T23:57:50.419 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 11.441350532, 2024-09-14T23:57:50.419 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054338487999999997, 2024-09-14T23:57:50.419 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:50.419 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:50.419 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:50.419 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:50.419 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:50.419 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:50.419 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129201 2024-09-14T23:57:50.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.419 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:50.419 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.420 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:50.420 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935960+0000", 2024-09-14T23:57:50.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:50.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.420 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.420 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:50.420 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935960+0000", 2024-09-14T23:57:50.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:50.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.420 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.420 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:50.420 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935960+0000", 2024-09-14T23:57:50.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.72e-07 2024-09-14T23:57:50.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.420 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.421 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:50.421 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935963+0000", 2024-09-14T23:57:50.421 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5390000000000002e-06 2024-09-14T23:57:50.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.421 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.421 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:50.421 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935971+0000", 2024-09-14T23:57:50.421 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4589999999999998e-06 2024-09-14T23:57:50.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.421 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.421 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:50.421 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935977+0000", 2024-09-14T23:57:50.421 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4110000000000004e-06 2024-09-14T23:57:50.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.422 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.422 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:50.422 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990213+0000", 2024-09-14T23:57:50.422 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054236642000000002 2024-09-14T23:57:50.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.422 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.422 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:50.422 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990257+0000", 2024-09-14T23:57:50.422 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3696000000000003e-05 2024-09-14T23:57:50.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.422 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.422 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:50.422 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990298+0000", 2024-09-14T23:57:50.422 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1369000000000003e-05 2024-09-14T23:57:50.423 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:50.423 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:50.423 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:50.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.423 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129202 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 11370496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:50.423 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.935986+0000", 2024-09-14T23:57:50.423 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 11.441323914, 2024-09-14T23:57:50.423 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054543700000000001, 2024-09-14T23:57:50.423 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:50.423 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:50.423 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:50.423 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:50.423 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:50.424 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:50.424 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129202 2024-09-14T23:57:50.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.424 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:50.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:50.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935986+0000", 2024-09-14T23:57:50.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:50.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:50.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935986+0000", 2024-09-14T23:57:50.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:50.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.425 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:50.425 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937678+0000", 2024-09-14T23:57:50.425 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016917849999999999 2024-09-14T23:57:50.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.425 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:50.425 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937682+0000", 2024-09-14T23:57:50.425 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2400000000000001e-06 2024-09-14T23:57:50.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.425 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:50.425 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937695+0000", 2024-09-14T23:57:50.426 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2999e-05 2024-09-14T23:57:50.426 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.426 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.426 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:50.426 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937700+0000", 2024-09-14T23:57:50.426 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8420000000000004e-06 2024-09-14T23:57:50.426 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.426 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.426 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:50.426 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990336+0000", 2024-09-14T23:57:50.426 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052636060999999998 2024-09-14T23:57:50.426 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.426 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.426 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:50.427 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990435+0000", 2024-09-14T23:57:50.427 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9015e-05 2024-09-14T23:57:50.427 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.427 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.427 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:50.427 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990530+0000", 2024-09-14T23:57:50.427 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4758000000000003e-05 2024-09-14T23:57:50.427 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:50.427 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:50.427 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:50.427 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.427 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.427 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129214 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 11468800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:50.427 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.937939+0000", 2024-09-14T23:57:50.428 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 11.439371, 2024-09-14T23:57:50.428 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054463733, 2024-09-14T23:57:50.428 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:50.428 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:50.428 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:50.428 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:50.428 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:50.428 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:50.428 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129214 2024-09-14T23:57:50.428 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.428 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:50.428 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.428 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:50.428 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937939+0000", 2024-09-14T23:57:50.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:50.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.429 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:50.429 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937939+0000", 2024-09-14T23:57:50.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:50.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.429 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:50.429 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939674+0000", 2024-09-14T23:57:50.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017342589999999999 2024-09-14T23:57:50.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.429 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:50.430 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939678+0000", 2024-09-14T23:57:50.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2590000000000002e-06 2024-09-14T23:57:50.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.430 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:50.430 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939699+0000", 2024-09-14T23:57:50.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1143000000000001e-05 2024-09-14T23:57:50.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.430 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:50.430 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939706+0000", 2024-09-14T23:57:50.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0269999999999998e-06 2024-09-14T23:57:50.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.431 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:50.431 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.992221+0000", 2024-09-14T23:57:50.431 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052514981000000002 2024-09-14T23:57:50.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.431 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.431 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:50.431 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.992321+0000", 2024-09-14T23:57:50.431 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9625999999999998e-05 2024-09-14T23:57:50.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.431 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.431 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:50.431 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.992403+0000", 2024-09-14T23:57:50.431 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2437999999999999e-05 2024-09-14T23:57:50.432 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:50.432 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:50.432 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:50.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.432 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129225 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 12648448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:50.432 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.939856+0000", 2024-09-14T23:57:50.432 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 11.437454444, 2024-09-14T23:57:50.432 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054305557999999997, 2024-09-14T23:57:50.432 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:50.432 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:50.432 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:50.432 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:50.432 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:50.433 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:50.433 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129225 2024-09-14T23:57:50.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.433 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:50.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.433 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:50.433 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939856+0000", 2024-09-14T23:57:50.433 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:50.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.433 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:50.433 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939856+0000", 2024-09-14T23:57:50.433 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:50.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.434 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:50.434 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939856+0000", 2024-09-14T23:57:50.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1199999999999999e-07 2024-09-14T23:57:50.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.434 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:50.434 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939857+0000", 2024-09-14T23:57:50.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.054e-06 2024-09-14T23:57:50.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.434 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:50.434 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939861+0000", 2024-09-14T23:57:50.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1459999999999998e-06 2024-09-14T23:57:50.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.435 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:50.435 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939864+0000", 2024-09-14T23:57:50.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7539999999999998e-06 2024-09-14T23:57:50.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.435 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:50.435 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.994078+0000", 2024-09-14T23:57:50.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054213483 2024-09-14T23:57:50.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.435 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:50.435 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.994119+0000", 2024-09-14T23:57:50.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1835000000000002e-05 2024-09-14T23:57:50.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.436 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:50.436 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.994161+0000", 2024-09-14T23:57:50.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2073999999999997e-05 2024-09-14T23:57:50.436 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:50.436 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:50.436 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:50.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.436 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129226 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 14090240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:50.436 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.939867+0000", 2024-09-14T23:57:50.437 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 11.437443072000001, 2024-09-14T23:57:50.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054523573999999998, 2024-09-14T23:57:50.437 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:50.437 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:50.437 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:50.437 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:50.437 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:50.437 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:50.437 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129226 2024-09-14T23:57:50.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.437 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:50.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.437 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:50.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939867+0000", 2024-09-14T23:57:50.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:50.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:50.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939867+0000", 2024-09-14T23:57:50.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:50.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:50.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941669+0000", 2024-09-14T23:57:50.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018021549999999999 2024-09-14T23:57:50.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.439 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:50.439 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941673+0000", 2024-09-14T23:57:50.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1269999999999999e-06 2024-09-14T23:57:50.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.439 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:50.439 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941679+0000", 2024-09-14T23:57:50.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1879999999999999e-06 2024-09-14T23:57:50.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.439 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:50.439 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941682+0000", 2024-09-14T23:57:50.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7110000000000001e-06 2024-09-14T23:57:50.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.440 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:50.440 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.994201+0000", 2024-09-14T23:57:50.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052518265000000001 2024-09-14T23:57:50.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.440 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:50.440 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.994302+0000", 2024-09-14T23:57:50.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101653 2024-09-14T23:57:50.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.440 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:50.440 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.994391+0000", 2024-09-14T23:57:50.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8474999999999996e-05 2024-09-14T23:57:50.441 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:50.441 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:50.441 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:50.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.441 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129237 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 12386304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:50.441 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.941825+0000", 2024-09-14T23:57:50.441 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 11.435485310000001, 2024-09-14T23:57:50.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054368076000000001, 2024-09-14T23:57:50.441 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:50.441 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:50.441 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:50.441 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:50.441 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:50.441 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:50.442 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129237 2024-09-14T23:57:50.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.442 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:50.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:50.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941825+0000", 2024-09-14T23:57:50.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:50.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:50.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941825+0000", 2024-09-14T23:57:50.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:50.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.443 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:50.443 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941825+0000", 2024-09-14T23:57:50.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.29e-07 2024-09-14T23:57:50.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.443 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:50.443 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941826+0000", 2024-09-14T23:57:50.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0999999999999997e-07 2024-09-14T23:57:50.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.443 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:50.443 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941834+0000", 2024-09-14T23:57:50.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5519999999999999e-06 2024-09-14T23:57:50.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:50.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941843+0000", 2024-09-14T23:57:50.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2960000000000004e-06 2024-09-14T23:57:50.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:50.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996108+0000", 2024-09-14T23:57:50.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054265200999999999 2024-09-14T23:57:50.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:50.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996152+0000", 2024-09-14T23:57:50.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4066999999999997e-05 2024-09-14T23:57:50.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:50.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996193+0000", 2024-09-14T23:57:50.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0821000000000001e-05 2024-09-14T23:57:50.445 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:50.445 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:50.445 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:50.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.445 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129238 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 13828096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:50.445 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.941851+0000", 2024-09-14T23:57:50.445 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 11.435458986, 2024-09-14T23:57:50.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.05458031, 2024-09-14T23:57:50.446 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:50.446 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:50.446 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:50.446 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:50.446 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:50.446 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:50.446 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129238 2024-09-14T23:57:50.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.446 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:50.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.446 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:50.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941851+0000", 2024-09-14T23:57:50.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:50.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:50.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941851+0000", 2024-09-14T23:57:50.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:50.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:50.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943670+0000", 2024-09-14T23:57:50.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018181930000000001 2024-09-14T23:57:50.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:50.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943673+0000", 2024-09-14T23:57:50.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3790000000000002e-06 2024-09-14T23:57:50.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:50.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943679+0000", 2024-09-14T23:57:50.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8719999999999999e-06 2024-09-14T23:57:50.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:50.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943683+0000", 2024-09-14T23:57:50.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8539999999999999e-06 2024-09-14T23:57:50.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:50.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996233+0000", 2024-09-14T23:57:50.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052550690999999997 2024-09-14T23:57:50.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:50.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996339+0000", 2024-09-14T23:57:50.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105989 2024-09-14T23:57:50.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:50.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996432+0000", 2024-09-14T23:57:50.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2331999999999996e-05 2024-09-14T23:57:50.449 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:50.449 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:50.450 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:50.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.450 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129249 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 12124160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:50.450 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.943826+0000", 2024-09-14T23:57:50.450 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 11.433484495, 2024-09-14T23:57:50.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054373266000000003, 2024-09-14T23:57:50.450 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:50.450 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:50.450 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:50.450 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:50.450 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:50.450 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:50.451 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129249 2024-09-14T23:57:50.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.451 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:50.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:50.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943826+0000", 2024-09-14T23:57:50.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:50.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:50.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943826+0000", 2024-09-14T23:57:50.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:50.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:50.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943826+0000", 2024-09-14T23:57:50.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.03e-07 2024-09-14T23:57:50.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:50.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943827+0000", 2024-09-14T23:57:50.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.08e-06 2024-09-14T23:57:50.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:50.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943834+0000", 2024-09-14T23:57:50.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4119999999999998e-06 2024-09-14T23:57:50.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:50.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943836+0000", 2024-09-14T23:57:50.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7369999999999998e-06 2024-09-14T23:57:50.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:50.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998115+0000", 2024-09-14T23:57:50.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054279040000000001 2024-09-14T23:57:50.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:50.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998159+0000", 2024-09-14T23:57:50.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3486999999999999e-05 2024-09-14T23:57:50.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:50.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998199+0000", 2024-09-14T23:57:50.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0306999999999999e-05 2024-09-14T23:57:50.454 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:50.454 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:50.454 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:50.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.454 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129250 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 13565952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:50.454 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.943840+0000", 2024-09-14T23:57:50.454 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 11.433470545, 2024-09-14T23:57:50.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054573875000000001, 2024-09-14T23:57:50.455 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:50.455 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:50.455 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:50.455 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:50.455 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:50.455 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:50.455 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129250 2024-09-14T23:57:50.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.455 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:50.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:50.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943840+0000", 2024-09-14T23:57:50.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:50.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.456 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:50.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943840+0000", 2024-09-14T23:57:50.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:50.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.456 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:50.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945669+0000", 2024-09-14T23:57:50.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018289720000000001 2024-09-14T23:57:50.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:50.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945671+0000", 2024-09-14T23:57:50.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6869999999999999e-06 2024-09-14T23:57:50.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:50.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945680+0000", 2024-09-14T23:57:50.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5259999999999993e-06 2024-09-14T23:57:50.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:50.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945685+0000", 2024-09-14T23:57:50.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8559999999999998e-06 2024-09-14T23:57:50.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:50.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998242+0000", 2024-09-14T23:57:50.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052557505999999997 2024-09-14T23:57:50.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:50.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998329+0000", 2024-09-14T23:57:50.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6410999999999996e-05 2024-09-14T23:57:50.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:50.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998414+0000", 2024-09-14T23:57:50.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4917e-05 2024-09-14T23:57:50.459 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:50.459 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:50.459 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:50.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.459 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129261 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 14385152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:50.459 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.945815+0000", 2024-09-14T23:57:50.459 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 11.431495251999999, 2024-09-14T23:57:50.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054331867999999998, 2024-09-14T23:57:50.459 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:50.459 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:50.459 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:50.460 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:50.460 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:50.460 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:50.460 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129261 2024-09-14T23:57:50.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.460 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:50.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:50.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945815+0000", 2024-09-14T23:57:50.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:50.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:50.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945815+0000", 2024-09-14T23:57:50.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:50.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:50.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945815+0000", 2024-09-14T23:57:50.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0599999999999999e-07 2024-09-14T23:57:50.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:50.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945816+0000", 2024-09-14T23:57:50.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0380000000000001e-06 2024-09-14T23:57:50.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:50.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945819+0000", 2024-09-14T23:57:50.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.005e-06 2024-09-14T23:57:50.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:50.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945822+0000", 2024-09-14T23:57:50.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.452e-06 2024-09-14T23:57:50.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:50.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000060+0000", 2024-09-14T23:57:50.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054238606000000002 2024-09-14T23:57:50.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:50.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000106+0000", 2024-09-14T23:57:50.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5188000000000002e-05 2024-09-14T23:57:50.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:50.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000147+0000", 2024-09-14T23:57:50.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1372999999999998e-05 2024-09-14T23:57:50.463 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:50.463 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:50.463 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:50.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.463 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129262 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 13303808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:50.463 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.945825+0000", 2024-09-14T23:57:50.464 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 11.431485604000001, 2024-09-14T23:57:50.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054515942999999997, 2024-09-14T23:57:50.464 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:50.464 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:50.464 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:50.464 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:50.464 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:50.464 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:50.464 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129262 2024-09-14T23:57:50.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.464 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:50.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:50.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945825+0000", 2024-09-14T23:57:50.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:50.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:50.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945825+0000", 2024-09-14T23:57:50.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:50.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:50.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.947670+0000", 2024-09-14T23:57:50.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018451229999999999 2024-09-14T23:57:50.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:50.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.947672+0000", 2024-09-14T23:57:50.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5289999999999998e-06 2024-09-14T23:57:50.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:50.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.947678+0000", 2024-09-14T23:57:50.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9880000000000001e-06 2024-09-14T23:57:50.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:50.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.947682+0000", 2024-09-14T23:57:50.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.63e-06 2024-09-14T23:57:50.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:50.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000183+0000", 2024-09-14T23:57:50.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052500770000000002 2024-09-14T23:57:50.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:50.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000266+0000", 2024-09-14T23:57:50.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3133000000000005e-05 2024-09-14T23:57:50.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:50.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000341+0000", 2024-09-14T23:57:50.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4770000000000004e-05 2024-09-14T23:57:50.467 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:50.468 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:50.468 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:50.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.468 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129299 3.10 3:0f1e06ee:::rbd_object_map.1118721be852:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:50.468 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.952330+0000", 2024-09-14T23:57:50.468 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 11.424980790999999, 2024-09-14T23:57:50.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054412818000000002, 2024-09-14T23:57:50.468 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:50.468 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:50.468 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:50.468 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:50.468 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:50.468 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:50.469 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129299 2024-09-14T23:57:50.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.469 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:50.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.469 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:50.469 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952330+0000", 2024-09-14T23:57:50.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:50.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.469 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:50.469 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952330+0000", 2024-09-14T23:57:50.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:50.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:50.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952330+0000", 2024-09-14T23:57:50.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4900000000000001e-07 2024-09-14T23:57:50.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:50.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952332+0000", 2024-09-14T23:57:50.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2879999999999999e-06 2024-09-14T23:57:50.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:50.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952340+0000", 2024-09-14T23:57:50.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9910000000000002e-06 2024-09-14T23:57:50.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:50.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952344+0000", 2024-09-14T23:57:50.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0280000000000002e-06 2024-09-14T23:57:50.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:50.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.004722+0000", 2024-09-14T23:57:50.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052377932000000002 2024-09-14T23:57:50.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:50.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.004778+0000", 2024-09-14T23:57:50.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5795999999999999e-05 2024-09-14T23:57:50.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:57:50.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.005016+0000", 2024-09-14T23:57:50.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000238006 2024-09-14T23:57:50.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:57:50.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.006366+0000", 2024-09-14T23:57:50.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001349786 2024-09-14T23:57:50.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:50.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.006685+0000", 2024-09-14T23:57:50.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00031970000000000002 2024-09-14T23:57:50.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:50.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.006718+0000", 2024-09-14T23:57:50.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2329000000000002e-05 2024-09-14T23:57:50.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:50.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:50.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:50.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.006742+0000", 2024-09-14T23:57:50.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4613e-05 2024-09-14T23:57:50.473 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:50.473 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:50.473 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:50.473 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:50.473 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:50.474 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:50.474 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:57:50.475 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:57:50.475 INFO:teuthology.orchestra.run.smithi042.stdout:4682 trunc from 0xeabf79d to 0x17e3ddf 2024-09-14T23:57:50.475 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:57:50.476 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:57:50.476 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T23:57:50.477 INFO:teuthology.orchestra.run.smithi042.stdout:4683 trunc from 0x17e3ddf to 0x2d46061 2024-09-14T23:57:50.477 INFO:teuthology.orchestra.run.smithi042.stdout:4684 read 0x65dc0f thru 0x65f12f (0x1521 bytes) 2024-09-14T23:57:50.477 INFO:teuthology.orchestra.run.smithi042.stdout:4686 punch from 0x254dd4a to 0x254fb94, (0x1e4a bytes) 2024-09-14T23:57:50.477 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:57:50.478 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:57:50.478 INFO:teuthology.orchestra.run.smithi042.stdout:4687 writesame 0x8597875 thru 0x859b825 data_size 0x5(0x3fb1 bytes) 2024-09-14T23:57:50.478 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:50.478 INFO:teuthology.orchestra.run.smithi042.stdout:4689 punch from 0x7b5c1bc to 0x7b62ee4, (0x6d28 bytes) 2024-09-14T23:57:50.478 INFO:teuthology.orchestra.run.smithi042.stdout:4692 write 0x3398d95 thru 0x33a5fa8 (0xd214 bytes) 2024-09-14T23:57:50.479 INFO:teuthology.orchestra.run.smithi042.stdout:4694 read 0x6f29fa7 thru 0x6f3212b (0x8185 bytes) 2024-09-14T23:57:50.479 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:57:50.479 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:57:50.479 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:50.473+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:50.479 INFO:teuthology.orchestra.run.smithi042.stdout:4695 write 0xb815fa8 thru 0xb82210f (0xc168 bytes) 2024-09-14T23:57:50.485 INFO:teuthology.orchestra.run.smithi042.stdout:4696 writesame 0x8f5dd92 thru 0x8f642a3 data_size 0x11(0x6512 bytes) 2024-09-14T23:57:50.491 INFO:teuthology.orchestra.run.smithi042.stdout:4697 write 0x154beaf thru 0x155a7a2 (0xe8f4 bytes) 2024-09-14T23:57:50.494 INFO:teuthology.orchestra.run.smithi042.stdout:4698 clone 41 order 21 su 524288 sc 5 2024-09-14T23:57:50.494 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:57:50.494 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:57:50.510 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:57:50.511 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:57:50.527 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:57:50.527 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:57:50.544 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:57:50.544 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:57:50.575 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:50.573+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:50.589 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:50.589 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:50.598 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T23:57:50.676 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:50.675+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:50.745 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:50.745 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:50.754 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T23:57:50.777 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:50.776+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:50.860 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:50.860 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:50.869 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T23:57:50.878 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:50.876+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:50.978 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:50.978 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:50.979 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:50.977+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:50.986 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T23:57:51.076 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T23:57:51.120 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:51.119+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:51.129 INFO:teuthology.orchestra.run.smithi042.stdout:leaving image image_client.0-clone40 intact 2024-09-14T23:57:51.141 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:51.141 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:51.152 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T23:57:51.192 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:51.191+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:51.257 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #39, image image_client.0-clone39 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent40 2024-09-14T23:57:51.282 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:51.280+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:51.309 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:51.310 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:51.319 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T23:57:51.393 INFO:teuthology.orchestra.run.smithi042.stdout:4699 read 0x7844362 thru 0x78483b4 (0x4053 bytes) 2024-09-14T23:57:51.393 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:51.391+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:51.393 INFO:teuthology.orchestra.run.smithi042.stdout:4700 punch from 0x9f51983 to 0x9f58a24, (0x70a1 bytes) 2024-09-14T23:57:51.396 INFO:teuthology.orchestra.run.smithi042.stdout:4701 write 0xa4b4a18 thru 0xa4bb5a2 (0x6b8b bytes) 2024-09-14T23:57:51.405 INFO:teuthology.orchestra.run.smithi042.stdout:4703 trunc from 0xb822110 to 0x7a6e1bf 2024-09-14T23:57:51.425 INFO:teuthology.orchestra.run.smithi042.stdout:4705 trunc from 0x7a6e1bf to 0xb91f7b9 2024-09-14T23:57:51.440 INFO:teuthology.orchestra.run.smithi042.stdout:4706 writesame 0xe2dec6d thru 0xe2e1594 data_size 0xc(0x2928 bytes) 2024-09-14T23:57:51.441 INFO:teuthology.orchestra.run.smithi042.stdout:4707 write 0x7dd7160 thru 0x7dd857f (0x1420 bytes) 2024-09-14T23:57:51.442 INFO:teuthology.orchestra.run.smithi042.stdout:4708 write 0x85fe432 thru 0x860d3a4 (0xef73 bytes) 2024-09-14T23:57:51.446 INFO:teuthology.orchestra.run.smithi042.stdout:4709 read 0x935e58d thru 0x9361034 (0x2aa8 bytes) 2024-09-14T23:57:51.453 INFO:teuthology.orchestra.run.smithi042.stdout:4710 trunc from 0xe2e1595 to 0x771302f 2024-09-14T23:57:51.469 INFO:teuthology.orchestra.run.smithi042.stdout:4712 read 0x455ce5f thru 0x456210c (0x52ae bytes) 2024-09-14T23:57:51.469 INFO:teuthology.orchestra.run.smithi042.stdout:4714 punch from 0x2b5584c to 0x2b6358d, (0xdd41 bytes) 2024-09-14T23:57:51.469 INFO:teuthology.orchestra.run.smithi042.stdout:4715 punch from 0x67e95a9 to 0x67e9864, (0x2bb bytes) 2024-09-14T23:57:51.469 INFO:teuthology.orchestra.run.smithi042.stdout:4716 punch from 0x2e7bff5 to 0x2e80f01, (0x4f0c bytes) 2024-09-14T23:57:51.472 INFO:teuthology.orchestra.run.smithi042.stdout:4717 trunc from 0x771302f to 0xb18f05e 2024-09-14T23:57:51.477 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:51.478 INFO:teuthology.orchestra.run.smithi042.stdout:4720 writesame 0xae0b775 thru 0xae12fe1 data_size 0x1(0x786d bytes) 2024-09-14T23:57:51.483 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:51.481+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:51.483 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:51.483 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:51.493 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T23:57:51.503 INFO:teuthology.orchestra.run.smithi042.stdout:4729 trunc from 0xb18f05e to 0x3be4d2b 2024-09-14T23:57:51.532 INFO:teuthology.orchestra.run.smithi042.stdout:4731 write 0xc9e0bbe thru 0xc9e7b4b (0x6f8e bytes) 2024-09-14T23:57:51.535 INFO:teuthology.orchestra.run.smithi042.stdout:4732 punch from 0x1addcbe to 0x1ae8449, (0xa78b bytes) 2024-09-14T23:57:51.535 INFO:teuthology.orchestra.run.smithi042.stdout:4733 write 0xd1cd65 thru 0xd2b4d5 (0xe771 bytes) 2024-09-14T23:57:51.550 INFO:teuthology.orchestra.run.smithi042.stdout:4735 read 0xb4811dc thru 0xb48a10b (0x8f30 bytes) 2024-09-14T23:57:51.557 INFO:teuthology.orchestra.run.smithi042.stdout:4736 write 0xdd57e69 thru 0xdd5bc01 (0x3d99 bytes) 2024-09-14T23:57:51.563 INFO:teuthology.orchestra.run.smithi042.stdout:4737 trunc from 0xdd5bc02 to 0xb3a168f 2024-09-14T23:57:51.574 INFO:teuthology.orchestra.run.smithi042.stdout:4738 write 0x69df248 thru 0x69eef2d (0xfce6 bytes) 2024-09-14T23:57:51.577 INFO:teuthology.orchestra.run.smithi042.stdout:4739 read 0x33d3323 thru 0x33de268 (0xaf46 bytes) 2024-09-14T23:57:51.577 INFO:teuthology.orchestra.run.smithi042.stdout:4740 read 0xd7a8b5 thru 0xd7bc36 (0x1382 bytes) 2024-09-14T23:57:51.578 INFO:teuthology.orchestra.run.smithi042.stdout:4741 writesame 0x2585a00 thru 0x2587dc4 data_size 0x1(0x23c5 bytes) 2024-09-14T23:57:51.584 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:51.582+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:51.585 INFO:teuthology.orchestra.run.smithi042.stdout:4742 write 0x8c4a36b thru 0x8c4beaf (0x1b45 bytes) 2024-09-14T23:57:51.588 INFO:teuthology.orchestra.run.smithi042.stdout:4744 punch from 0xa932abe to 0xa938755, (0x5c97 bytes) 2024-09-14T23:57:51.588 INFO:teuthology.orchestra.run.smithi042.stdout:4745 write 0x12232f6 thru 0x122effc (0xbd07 bytes) 2024-09-14T23:57:51.590 INFO:teuthology.orchestra.run.smithi042.stdout:4747 writesame 0x1bc902f thru 0x1bcbf1b data_size 0x1(0x2eed bytes) 2024-09-14T23:57:51.600 INFO:teuthology.orchestra.run.smithi042.stdout:4748 write 0x1a0577 thru 0x1a1a56 (0x14e0 bytes) 2024-09-14T23:57:51.621 INFO:teuthology.orchestra.run.smithi042.stdout:4751 punch from 0x2d8224b to 0x2d8f416, (0xd1cb bytes) 2024-09-14T23:57:51.621 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:51.622 INFO:teuthology.orchestra.run.smithi042.stdout:4754 write 0x47247f0 thru 0x472fffa (0xb80b bytes) 2024-09-14T23:57:51.625 INFO:teuthology.orchestra.run.smithi042.stdout:4756 read 0x74b61a8 thru 0x74c3220 (0xd079 bytes) 2024-09-14T23:57:51.625 INFO:teuthology.orchestra.run.smithi042.stdout:4757 read 0x9a636f thru 0x9abc3a (0x58cc bytes) 2024-09-14T23:57:51.625 INFO:teuthology.orchestra.run.smithi042.stdout:4758 punch from 0x45ab4a4 to 0x45b1bb1, (0x670d bytes) 2024-09-14T23:57:51.630 INFO:teuthology.orchestra.run.smithi042.stdout:4759 trunc from 0xb3a168f to 0x77da56b 2024-09-14T23:57:51.631 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:51.631 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:51.641 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T23:57:51.647 INFO:teuthology.orchestra.run.smithi042.stdout:4760 trunc from 0x77da56b to 0x295bffc 2024-09-14T23:57:51.662 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:51.677 INFO:teuthology.orchestra.run.smithi042.stdout:4762 write 0xabaf7c8 thru 0xabbd12c (0xd965 bytes) 2024-09-14T23:57:51.680 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:51.680 INFO:teuthology.orchestra.run.smithi042.stdout:4765 write 0x535517b thru 0x535ea24 (0x98aa bytes) 2024-09-14T23:57:51.683 INFO:teuthology.orchestra.run.smithi042.stdout:4766 read 0xacdec0 thru 0xaddd83 (0xfec4 bytes) 2024-09-14T23:57:51.685 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:51.683+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:51.694 INFO:teuthology.orchestra.run.smithi042.stdout:4767 trunc from 0xabbd12d to 0x2e7642c 2024-09-14T23:57:51.712 INFO:teuthology.orchestra.run.smithi042.stdout:4768 trunc from 0x2e7642c to 0x89a72b1 2024-09-14T23:57:51.718 INFO:teuthology.orchestra.run.smithi042.stdout:4769 read 0x7abf2ff thru 0x7ac8f83 (0x9c85 bytes) 2024-09-14T23:57:51.718 INFO:teuthology.orchestra.run.smithi042.stdout:4771 write 0x142d6f6 thru 0x143cd83 (0xf68e bytes) 2024-09-14T23:57:51.720 INFO:teuthology.orchestra.run.smithi042.stdout:4772 trunc from 0x89a72b1 to 0x8c180ab 2024-09-14T23:57:51.727 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:51.727 INFO:teuthology.orchestra.run.smithi042.stdout:4774 writesame 0x4ce2c25 thru 0x4ceff14 data_size 0x3c(0xd2f0 bytes) 2024-09-14T23:57:51.730 INFO:teuthology.orchestra.run.smithi042.stdout:4775 punch from 0x4d76ba7 to 0x4d83402, (0xc85b bytes) 2024-09-14T23:57:51.731 INFO:teuthology.orchestra.run.smithi042.stdout:4777 read 0x5c1967 thru 0x5c7b2c (0x61c6 bytes) 2024-09-14T23:57:51.740 INFO:teuthology.orchestra.run.smithi042.stdout:4778 write 0xc602ae9 thru 0xc609e3f (0x7357 bytes) 2024-09-14T23:57:51.743 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:51.756 INFO:teuthology.orchestra.run.smithi042.stdout:4780 trunc from 0xc609e40 to 0x14f8a22 2024-09-14T23:57:51.777 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:51.778 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:51.786 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:51.784+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:51.792 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T23:57:51.794 INFO:teuthology.orchestra.run.smithi042.stdout:4782 write 0xd763fd9 thru 0xd76f527 (0xb54f bytes) 2024-09-14T23:57:51.797 INFO:teuthology.orchestra.run.smithi042.stdout:4783 read 0xd2a8b0f thru 0xd2aa517 (0x1a09 bytes) 2024-09-14T23:57:51.811 INFO:teuthology.orchestra.run.smithi042.stdout:4784 trunc from 0xd76f528 to 0x12b43d9 2024-09-14T23:57:51.859 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:51.859 INFO:teuthology.orchestra.run.smithi042.stdout:4786 read 0xa630ba thru 0xa70d2b (0xdc72 bytes) 2024-09-14T23:57:51.859 INFO:teuthology.orchestra.run.smithi042.stdout:4787 punch from 0x1861a6 to 0x18c862, (0x66bc bytes) 2024-09-14T23:57:51.865 INFO:teuthology.orchestra.run.smithi042.stdout:4788 write 0x217f809 thru 0x2185a0b (0x6203 bytes) 2024-09-14T23:57:51.880 INFO:teuthology.orchestra.run.smithi042.stdout:4789 write 0x73e96a1 thru 0x73f5eb9 (0xc819 bytes) 2024-09-14T23:57:51.883 INFO:teuthology.orchestra.run.smithi042.stdout:4790 punch from 0x6973f42 to 0x6976ac8, (0x2b86 bytes) 2024-09-14T23:57:51.883 INFO:teuthology.orchestra.run.smithi042.stdout:4792 punch from 0x333f185 to 0x3342e0a, (0x3c85 bytes) 2024-09-14T23:57:51.883 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:51.884 INFO:teuthology.orchestra.run.smithi042.stdout:4795 read 0x31c3463 thru 0x31c4950 (0x14ee bytes) 2024-09-14T23:57:51.884 INFO:teuthology.orchestra.run.smithi042.stdout:4796 read 0x9f3c5c thru 0xa01b58 (0xdefd bytes) 2024-09-14T23:57:51.887 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:51.885+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:51.891 INFO:teuthology.orchestra.run.smithi042.stdout:4799 trunc from 0x73f5eba to 0x111d4de 2024-09-14T23:57:51.932 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:51.933 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:51.938 INFO:teuthology.orchestra.run.smithi042.stdout:4800 writesame 0xaf4a97e thru 0xaf4d50d data_size 0x44(0x2b90 bytes) 2024-09-14T23:57:51.938 INFO:teuthology.orchestra.run.smithi042.stdout:4802 read 0x751b3b thru 0x75eb47 (0xd00d bytes) 2024-09-14T23:57:51.939 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:51.939 INFO:teuthology.orchestra.run.smithi042.stdout:4807 read 0x72c076a thru 0x72c6066 (0x58fd bytes) 2024-09-14T23:57:51.939 INFO:teuthology.orchestra.run.smithi042.stdout:4808 read 0x5af560d thru 0x5b00d55 (0xb749 bytes) 2024-09-14T23:57:51.939 INFO:teuthology.orchestra.run.smithi042.stdout:4810 write 0x927c902 thru 0x928287d (0x5f7c bytes) 2024-09-14T23:57:51.942 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T23:57:51.944 INFO:teuthology.orchestra.run.smithi042.stdout:4811 punch from 0x1c43f59 to 0x1c53c40, (0xfce7 bytes) 2024-09-14T23:57:51.946 INFO:teuthology.orchestra.run.smithi042.stdout:4812 trunc from 0xaf4d50e to 0x8aa391c 2024-09-14T23:57:51.957 INFO:teuthology.orchestra.run.smithi042.stdout:4813 read 0x759fc47 thru 0x75a0c70 (0x102a bytes) 2024-09-14T23:57:51.957 INFO:teuthology.orchestra.run.smithi042.stdout:4814 read 0x52e1c8d thru 0x52eb9ad (0x9d21 bytes) 2024-09-14T23:57:51.968 INFO:teuthology.orchestra.run.smithi042.stdout:4816 write 0xccb9554 thru 0xccc4672 (0xb11f bytes) 2024-09-14T23:57:51.970 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:51.970 INFO:teuthology.orchestra.run.smithi042.stdout:4819 writesame 0xc62379 thru 0xc67e6a data_size 0x7(0x5af2 bytes) 2024-09-14T23:57:51.972 INFO:teuthology.orchestra.run.smithi042.stdout:4821 read 0x27b5556 thru 0x27ba594 (0x503f bytes) 2024-09-14T23:57:51.972 INFO:teuthology.orchestra.run.smithi042.stdout:4822 write 0x87e1272 thru 0x87e2ec0 (0x1c4f bytes) 2024-09-14T23:57:51.975 INFO:teuthology.orchestra.run.smithi042.stdout:4823 write 0xa25c3e5 thru 0xa267841 (0xb45d bytes) 2024-09-14T23:57:51.984 INFO:teuthology.orchestra.run.smithi042.stdout:4826 write 0xd1a44dc thru 0xd1a4aab (0x5d0 bytes) 2024-09-14T23:57:51.988 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:51.985+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:51.993 INFO:teuthology.orchestra.run.smithi042.stdout:4827 trunc from 0xd1a4aac to 0x632af6a 2024-09-14T23:57:52.003 INFO:teuthology.orchestra.run.smithi042.stdout:4828 punch from 0x2030771 to 0x2034f88, (0x4817 bytes) 2024-09-14T23:57:52.006 INFO:teuthology.orchestra.run.smithi042.stdout:4829 trunc from 0x632af6a to 0x8131112 2024-09-14T23:57:52.013 INFO:teuthology.orchestra.run.smithi042.stdout:4830 trunc from 0x8131112 to 0x6c8e0cc 2024-09-14T23:57:52.018 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:52.019 INFO:teuthology.orchestra.run.smithi042.stdout:4833 punch from 0x25587bd to 0x255ef85, (0x67c8 bytes) 2024-09-14T23:57:52.019 INFO:teuthology.orchestra.run.smithi042.stdout:4835 read 0x15f918d thru 0x15fdb6f (0x49e3 bytes) 2024-09-14T23:57:52.019 INFO:teuthology.orchestra.run.smithi042.stdout:4836 punch from 0x4b9631b to 0x4b9dc2d, (0x7912 bytes) 2024-09-14T23:57:52.032 INFO:teuthology.orchestra.run.smithi042.stdout:4837 write 0xe253673 thru 0xe2635c4 (0xff52 bytes) 2024-09-14T23:57:52.035 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:52.035 INFO:teuthology.orchestra.run.smithi042.stdout:4839 write 0x8eccc7f thru 0x8ed0a81 (0x3e03 bytes) 2024-09-14T23:57:52.038 INFO:teuthology.orchestra.run.smithi042.stdout:4841 punch from 0x7d2b4f5 to 0x7d33caa, (0x87b5 bytes) 2024-09-14T23:57:52.038 INFO:teuthology.orchestra.run.smithi042.stdout:4842 read 0xae51ad1 thru 0xae5731d (0x584d bytes) 2024-09-14T23:57:52.038 INFO:teuthology.orchestra.run.smithi042.stdout:4843 writesame 0x1247cca thru 0x12552bb data_size 0x5(0xd5f2 bytes) 2024-09-14T23:57:52.044 INFO:teuthology.orchestra.run.smithi042.stdout:4845 punch from 0x616b5e4 to 0x617573c, (0xa158 bytes) 2024-09-14T23:57:52.044 INFO:teuthology.orchestra.run.smithi042.stdout:4846 read 0xa6b63ef thru 0xa6bdd7c (0x798e bytes) 2024-09-14T23:57:52.044 INFO:teuthology.orchestra.run.smithi042.stdout:4847 writesame 0xa3b7f90 thru 0xa3c34c6 data_size 0x1(0xb537 bytes) 2024-09-14T23:57:52.064 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:52.065 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:52.072 INFO:teuthology.orchestra.run.smithi042.stdout:4848 trunc from 0xe2635c5 to 0xaa4f372 2024-09-14T23:57:52.077 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T23:57:52.079 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is unset 2024-09-14T23:57:52.087 INFO:teuthology.orchestra.run.smithi042.stdout:4851 write 0x1be89a0 thru 0x1bf39dc (0xb03d bytes) 2024-09-14T23:57:52.088 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:52.086+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:52.102 INFO:teuthology.orchestra.run.smithi042.stdout:4853 trunc from 0xaa4f372 to 0x1314f16 2024-09-14T23:57:52.113 INFO:teuthology.orchestra.run.smithi042.stdout:4854 read 0x103308f thru 0x1037f6e (0x4ee0 bytes) 2024-09-14T23:57:52.114 INFO:teuthology.orchestra.run.smithi042.stdout:4855 read 0x39184e thru 0x393b61 (0x2314 bytes) 2024-09-14T23:57:52.114 INFO:teuthology.orchestra.run.smithi042.stdout:4856 read 0x119774e thru 0x1199eed (0x27a0 bytes) 2024-09-14T23:57:52.114 INFO:teuthology.orchestra.run.smithi042.stdout:4858 read 0x113c7ea thru 0x1146df0 (0xa607 bytes) 2024-09-14T23:57:52.126 INFO:teuthology.orchestra.run.smithi042.stdout:4859 write 0x688a977 thru 0x6895589 (0xac13 bytes) 2024-09-14T23:57:52.130 INFO:teuthology.orchestra.run.smithi042.stdout:4863 read 0x2805a9e thru 0x280ecd9 (0x923c bytes) 2024-09-14T23:57:52.131 INFO:teuthology.orchestra.run.smithi042.stdout:4864 punch from 0x69570b to 0x697f02, (0x27f7 bytes) 2024-09-14T23:57:52.131 INFO:teuthology.orchestra.run.smithi042.stdout:4866 read 0x508d3ab thru 0x509431b (0x6f71 bytes) 2024-09-14T23:57:52.131 INFO:teuthology.orchestra.run.smithi042.stdout:4868 punch from 0x1352b6b to 0x1358884, (0x5d19 bytes) 2024-09-14T23:57:52.131 INFO:teuthology.orchestra.run.smithi042.stdout:4869 read 0x23de1ee thru 0x23e141f (0x3232 bytes) 2024-09-14T23:57:52.131 INFO:teuthology.orchestra.run.smithi042.stdout:4870 writesame 0x1a7df55 thru 0x1a8829d data_size 0x1(0xa349 bytes) 2024-09-14T23:57:52.160 INFO:teuthology.orchestra.run.smithi042.stdout:4871 write 0xbb7f718 thru 0xbb86538 (0x6e21 bytes) 2024-09-14T23:57:52.164 INFO:teuthology.orchestra.run.smithi042.stdout:4872 write 0x9d803ec thru 0x9d8723f (0x6e54 bytes) 2024-09-14T23:57:52.168 INFO:teuthology.orchestra.run.smithi042.stdout:4874 read 0xad76538 thru 0xad8236c (0xbe35 bytes) 2024-09-14T23:57:52.168 INFO:teuthology.orchestra.run.smithi042.stdout:4876 read 0x2010a8 thru 0x2052c3 (0x421c bytes) 2024-09-14T23:57:52.175 INFO:teuthology.orchestra.run.smithi042.stdout:4877 writesame 0xc5d5adc thru 0xc5d6b56 data_size 0x1(0x107b bytes) 2024-09-14T23:57:52.175 INFO:teuthology.orchestra.run.smithi042.stdout:4879 punch from 0x997e85 to 0x9981e0, (0x35b bytes) 2024-09-14T23:57:52.183 INFO:teuthology.orchestra.run.smithi042.stdout:4880 writesame 0xe14bfdf thru 0xe152063 data_size 0x1(0x6085 bytes) 2024-09-14T23:57:52.184 INFO:teuthology.orchestra.run.smithi042.stdout:4881 punch from 0xa9de294 to 0xa9e0dc2, (0x2b2e bytes) 2024-09-14T23:57:52.184 INFO:teuthology.orchestra.run.smithi042.stdout:4883 write 0x91ff5b0 thru 0x920c119 (0xcb6a bytes) 2024-09-14T23:57:52.188 INFO:teuthology.orchestra.run.smithi042.stdout:4885 read 0xa9042f5 thru 0xa908583 (0x428f bytes) 2024-09-14T23:57:52.188 INFO:teuthology.orchestra.run.smithi042.stdout:4887 punch from 0xe10480b to 0xe10dd91, (0x9586 bytes) 2024-09-14T23:57:52.188 INFO:teuthology.orchestra.run.smithi042.stdout:4888 write 0xb6ba2d9 thru 0xb6bee59 (0x4b81 bytes) 2024-09-14T23:57:52.189 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:52.187+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:52.189 INFO:teuthology.orchestra.run.smithi042.stdout:4889 write 0x53723c6 thru 0x5376713 (0x434e bytes) 2024-09-14T23:57:52.192 INFO:teuthology.orchestra.run.smithi042.stdout:4890 trunc from 0xe152064 to 0xe09a374 2024-09-14T23:57:52.198 INFO:teuthology.orchestra.run.smithi042.stdout:4891 punch from 0x648faaf to 0x649dfd0, (0xe521 bytes) 2024-09-14T23:57:52.198 INFO:teuthology.orchestra.run.smithi042.stdout:4892 punch from 0x892d468 to 0x892f3ff, (0x1f97 bytes) 2024-09-14T23:57:52.198 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:52.199 INFO:teuthology.orchestra.run.smithi042.stdout:4895 read 0xa80d324 thru 0xa80e844 (0x1521 bytes) 2024-09-14T23:57:52.199 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:52.199 INFO:teuthology.orchestra.run.smithi042.stdout:4897 write 0xd1ce2ba thru 0xd1d575d (0x74a4 bytes) 2024-09-14T23:57:52.201 INFO:teuthology.orchestra.run.smithi042.stdout:4898 punch from 0x2d88c43 to 0x2d8f33a, (0x66f7 bytes) 2024-09-14T23:57:52.201 INFO:teuthology.orchestra.run.smithi042.stdout:4900 read 0xae970bb thru 0xaea0b34 (0x9a7a bytes) 2024-09-14T23:57:52.201 INFO:teuthology.orchestra.run.smithi042.stdout:4902 read 0x7f36ddf thru 0x7f43bad (0xcdcf bytes) 2024-09-14T23:57:52.218 INFO:teuthology.orchestra.run.smithi042.stdout:4903 trunc from 0xe09a374 to 0x1138813 2024-09-14T23:57:52.219 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:52.219 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:52.231 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T23:57:52.246 INFO:teuthology.orchestra.run.smithi042.stdout:4904 write 0x89d9230 thru 0x89e4c44 (0xba15 bytes) 2024-09-14T23:57:52.249 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:52.254 INFO:teuthology.orchestra.run.smithi042.stdout:4908 trunc from 0x89e4c45 to 0xca733fa 2024-09-14T23:57:52.260 INFO:teuthology.orchestra.run.smithi042.stdout:4909 read 0xa9f5bf0 thru 0xaa042c8 (0xe6d9 bytes) 2024-09-14T23:57:52.260 INFO:teuthology.orchestra.run.smithi042.stdout:4910 read 0xa365037 thru 0xa36c3b7 (0x7381 bytes) 2024-09-14T23:57:52.261 INFO:teuthology.orchestra.run.smithi042.stdout:4911 write 0x9b38aa9 thru 0x9b47c25 (0xf17d bytes) 2024-09-14T23:57:52.264 INFO:teuthology.orchestra.run.smithi042.stdout:4913 punch from 0xb0637b7 to 0xb066d1b, (0x3564 bytes) 2024-09-14T23:57:52.265 INFO:teuthology.orchestra.run.smithi042.stdout:4914 read 0x58185f1 thru 0x5822e8c (0xa89c bytes) 2024-09-14T23:57:52.272 INFO:teuthology.orchestra.run.smithi042.stdout:4916 writesame 0xebb6715 thru 0xebc1acf data_size 0x31(0xb3bb bytes) 2024-09-14T23:57:52.278 INFO:teuthology.orchestra.run.smithi042.stdout:4917 trunc from 0xebc1ad0 to 0x97fd0a8 2024-09-14T23:57:52.285 INFO:teuthology.orchestra.run.smithi042.stdout:4918 punch from 0x2404f3c to 0x2409eb7, (0x4f7b bytes) 2024-09-14T23:57:52.285 INFO:teuthology.orchestra.run.smithi042.stdout:4919 write 0x1d49724 thru 0x1d55d14 (0xc5f1 bytes) 2024-09-14T23:57:52.288 INFO:teuthology.orchestra.run.smithi042.stdout:4920 write 0x152f226 thru 0x1531ef3 (0x2cce bytes) 2024-09-14T23:57:52.290 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:52.288+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:52.291 INFO:teuthology.orchestra.run.smithi042.stdout:4921 read 0x270f9ca thru 0x27124ea (0x2b21 bytes) 2024-09-14T23:57:52.297 INFO:teuthology.orchestra.run.smithi042.stdout:4922 write 0xa01ed25 thru 0xa02182e (0x2b0a bytes) 2024-09-14T23:57:52.300 INFO:teuthology.orchestra.run.smithi042.stdout:4923 write 0x71e63e8 thru 0x71f4346 (0xdf5f bytes) 2024-09-14T23:57:52.303 INFO:teuthology.orchestra.run.smithi042.stdout:4924 write 0x600f859 thru 0x60191a1 (0x9949 bytes) 2024-09-14T23:57:52.306 INFO:teuthology.orchestra.run.smithi042.stdout:4925 writesame 0x8d59bb9 thru 0x8d5fe3d data_size 0x7(0x6285 bytes) 2024-09-14T23:57:52.310 INFO:teuthology.orchestra.run.smithi042.stdout:4926 read 0x15c58fe thru 0x15cad68 (0x546b bytes) 2024-09-14T23:57:52.310 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:52.310 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:52.319 INFO:teuthology.orchestra.run.smithi042.stdout:4929 write 0xbf129e3 thru 0xbf21173 (0xe791 bytes) 2024-09-14T23:57:52.322 INFO:teuthology.orchestra.run.smithi042.stdout:4931 punch from 0x92491f9 to 0x9256698, (0xd49f bytes) 2024-09-14T23:57:52.322 INFO:teuthology.orchestra.run.smithi042.stdout:4932 punch from 0x981845c to 0x98190de, (0xc82 bytes) 2024-09-14T23:57:52.322 INFO:teuthology.orchestra.run.smithi042.stdout:4934 write 0xa570630 thru 0xa57b980 (0xb351 bytes) 2024-09-14T23:57:52.323 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:52.324 INFO:teuthology.orchestra.run.smithi042.stdout:4936 read 0xea996e thru 0xeaafc6 (0x1659 bytes) 2024-09-14T23:57:52.324 INFO:teuthology.orchestra.run.smithi042.stdout:4937 write 0x41f59c9 thru 0x41fe1eb (0x8823 bytes) 2024-09-14T23:57:52.328 INFO:teuthology.orchestra.run.smithi042.stdout:4939 write 0x47eb28d thru 0x47f50e9 (0x9e5d bytes) 2024-09-14T23:57:52.331 INFO:teuthology.orchestra.run.smithi042.stdout:4940 write 0x123c161 thru 0x123dd1e (0x1bbe bytes) 2024-09-14T23:57:52.344 INFO:teuthology.orchestra.run.smithi042.stdout:4941 trunc from 0xbf21174 to 0x207ab2d 2024-09-14T23:57:52.369 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:52.370 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:52.376 INFO:teuthology.orchestra.run.smithi042.stdout:4942 writesame 0xe80deff thru 0xe8108ee data_size 0x16(0x29f0 bytes) 2024-09-14T23:57:52.376 INFO:teuthology.orchestra.run.smithi042.stdout:4943 write 0x9adf48c thru 0x9ae44bf (0x5034 bytes) 2024-09-14T23:57:52.379 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T23:57:52.380 INFO:teuthology.orchestra.run.smithi042.stdout:4944 write 0x9f51829 thru 0x9f6145f (0xfc37 bytes) 2024-09-14T23:57:52.383 INFO:teuthology.orchestra.run.smithi042.stdout:4945 write 0x6a4bf59 thru 0x6a554cd (0x9575 bytes) 2024-09-14T23:57:52.387 INFO:teuthology.orchestra.run.smithi042.stdout:4946 read 0x646af16 thru 0x6474a1e (0x9b09 bytes) 2024-09-14T23:57:52.387 INFO:teuthology.orchestra.run.smithi042.stdout:4947 write 0xc31d525 thru 0xc323bc4 (0x66a0 bytes) 2024-09-14T23:57:52.391 INFO:teuthology.orchestra.run.smithi042.stdout:4948 read 0x9e29031 thru 0x9e2b879 (0x2849 bytes) 2024-09-14T23:57:52.391 INFO:teuthology.orchestra.run.smithi042.stdout:4949 writesame 0x772a4e9 thru 0x77355c8 data_size 0x50(0xb0e0 bytes) 2024-09-14T23:57:52.391 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:52.389+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:52.394 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:52.394 INFO:teuthology.orchestra.run.smithi042.stdout:4951 write 0x622597 thru 0x625025 (0x2a8f bytes) 2024-09-14T23:57:52.407 INFO:teuthology.orchestra.run.smithi042.stdout:4952 read 0x63b7b51 thru 0x63c20a3 (0xa553 bytes) 2024-09-14T23:57:52.408 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:52.408 INFO:teuthology.orchestra.run.smithi042.stdout:4954 punch from 0x213da20 to 0x21458e7, (0x7ec7 bytes) 2024-09-14T23:57:52.408 INFO:teuthology.orchestra.run.smithi042.stdout:4955 read 0xf220fc thru 0xf31ffd (0xff02 bytes) 2024-09-14T23:57:52.408 INFO:teuthology.orchestra.run.smithi042.stdout:4958 read 0x2193562 thru 0x219ed3e (0xb7dd bytes) 2024-09-14T23:57:52.408 INFO:teuthology.orchestra.run.smithi042.stdout:4960 read 0x8c3dfc thru 0x8cf0f2 (0xb2f7 bytes) 2024-09-14T23:57:52.408 INFO:teuthology.orchestra.run.smithi042.stdout:4963 write 0x986fefe thru 0x987fb32 (0xfc35 bytes) 2024-09-14T23:57:52.410 INFO:teuthology.orchestra.run.smithi042.stdout:4964 punch from 0x1dcdb7d to 0x1dd2465, (0x48e8 bytes) 2024-09-14T23:57:52.410 INFO:teuthology.orchestra.run.smithi042.stdout:4965 write 0x115f48d thru 0x11605b9 (0x112d bytes) 2024-09-14T23:57:52.411 INFO:teuthology.orchestra.run.smithi042.stdout:4966 write 0x363cdf0 thru 0x3642182 (0x5393 bytes) 2024-09-14T23:57:52.415 INFO:teuthology.orchestra.run.smithi042.stdout:4967 read 0x1690aa3 thru 0x169c9f6 (0xbf54 bytes) 2024-09-14T23:57:52.415 INFO:teuthology.orchestra.run.smithi042.stdout:4968 punch from 0x9d7966b to 0x9d8529d, (0xbc32 bytes) 2024-09-14T23:57:52.415 INFO:teuthology.orchestra.run.smithi042.stdout:4969 read 0x152f53a thru 0x1533526 (0x3fed bytes) 2024-09-14T23:57:52.416 INFO:teuthology.orchestra.run.smithi042.stdout:4970 write 0x7cfee53 thru 0x7d002ce (0x147c bytes) 2024-09-14T23:57:52.420 INFO:teuthology.orchestra.run.smithi042.stdout:4971 write 0x24ba7d thru 0x259b10 (0xe094 bytes) 2024-09-14T23:57:52.439 INFO:teuthology.orchestra.run.smithi042.stdout:4973 writesame 0xcd2eb2e thru 0xcd361dc data_size 0x1(0x76af bytes) 2024-09-14T23:57:52.456 INFO:teuthology.orchestra.run.smithi042.stdout:4974 punch from 0xd8abc76 to 0xd8b0025, (0x43af bytes) 2024-09-14T23:57:52.456 INFO:teuthology.orchestra.run.smithi042.stdout:4975 read 0x71fcaaf thru 0x7203ba5 (0x70f7 bytes) 2024-09-14T23:57:52.456 INFO:teuthology.orchestra.run.smithi042.stdout:4976 writesame 0x6de7b0 thru 0x6e9396 data_size 0x1(0xabe7 bytes) 2024-09-14T23:57:52.477 INFO:teuthology.orchestra.run.smithi042.stdout:4980 punch from 0x47e8b1a to 0x47ed509, (0x49ef bytes) 2024-09-14T23:57:52.477 INFO:teuthology.orchestra.run.smithi042.stdout:4981 read 0x5264ea7 thru 0x526ca34 (0x7b8e bytes) 2024-09-14T23:57:52.477 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:52.477 INFO:teuthology.orchestra.run.smithi042.stdout:4983 punch from 0xe4dff95 to 0xe4ebb91, (0xbbfc bytes) 2024-09-14T23:57:52.477 INFO:teuthology.orchestra.run.smithi042.stdout:4984 read 0x2af233 thru 0x2b0456 (0x1224 bytes) 2024-09-14T23:57:52.477 INFO:teuthology.orchestra.run.smithi042.stdout:4985 write 0x1099ad thru 0x10ba6c (0x20c0 bytes) 2024-09-14T23:57:52.479 INFO:teuthology.orchestra.run.smithi042.stdout:4986 read 0xe4c5be6 thru 0xe4c6d53 (0x116e bytes) 2024-09-14T23:57:52.482 INFO:teuthology.orchestra.run.smithi042.stdout:4987 trunc from 0xe8108ef to 0xb9a3979 2024-09-14T23:57:52.491 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:52.489+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:52.493 INFO:teuthology.orchestra.run.smithi042.stdout:4988 read 0x72d1dc thru 0x73260d (0x5432 bytes) 2024-09-14T23:57:52.494 INFO:teuthology.orchestra.run.smithi042.stdout:4989 punch from 0x4947dd to 0x499267, (0x4a8a bytes) 2024-09-14T23:57:52.494 INFO:teuthology.orchestra.run.smithi042.stdout:4990 writesame 0x544990a thru 0x5457f82 data_size 0x47(0xe679 bytes) 2024-09-14T23:57:52.497 INFO:teuthology.orchestra.run.smithi042.stdout:4991 read 0xacd5b1b thru 0xacdd4b1 (0x7997 bytes) 2024-09-14T23:57:52.497 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:52.497 INFO:teuthology.orchestra.run.smithi042.stdout:4994 read 0x7074e2 thru 0x708da7 (0x18c6 bytes) 2024-09-14T23:57:52.503 INFO:teuthology.orchestra.run.smithi042.stdout:4995 writesame 0xbe574a7 thru 0xbe58d67 data_size 0x1(0x18c1 bytes) 2024-09-14T23:57:52.503 INFO:teuthology.orchestra.run.smithi042.stdout:4997 read 0x98e24ab thru 0x98ee2dd (0xbe33 bytes) 2024-09-14T23:57:52.504 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:52.504 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:52.505 INFO:teuthology.orchestra.run.smithi042.stdout:4998 punch from 0xd08ad to 0xd2420, (0x1b73 bytes) 2024-09-14T23:57:52.505 INFO:teuthology.orchestra.run.smithi042.stdout:4999 read 0xa486e48 thru 0xa48cb43 (0x5cfc bytes) 2024-09-14T23:57:52.505 INFO:teuthology.orchestra.run.smithi042.stdout:5000 punch from 0x3c2f99c to 0x3c369de, (0x7042 bytes) 2024-09-14T23:57:52.505 INFO:teuthology.orchestra.run.smithi042.stdout:5002 read 0x5acf72a thru 0x5adc9b2 (0xd289 bytes) 2024-09-14T23:57:52.505 INFO:teuthology.orchestra.run.smithi042.stdout:5003 punch from 0x401a609 to 0x4021f1b, (0x7912 bytes) 2024-09-14T23:57:52.506 INFO:teuthology.orchestra.run.smithi042.stdout:5004 write 0x30251e5 thru 0x302aed5 (0x5cf1 bytes) 2024-09-14T23:57:52.507 INFO:teuthology.orchestra.run.smithi042.stdout:5005 read 0x9d8b4b2 thru 0x9d98f20 (0xda6f bytes) 2024-09-14T23:57:52.507 INFO:teuthology.orchestra.run.smithi042.stdout:5007 read 0x8896353 thru 0x88977da (0x1488 bytes) 2024-09-14T23:57:52.507 INFO:teuthology.orchestra.run.smithi042.stdout:5008 read 0x9f9b064 thru 0x9fa8b02 (0xda9f bytes) 2024-09-14T23:57:52.507 INFO:teuthology.orchestra.run.smithi042.stdout:5009 read 0x3a2cdce thru 0x3a3aa5f (0xdc92 bytes) 2024-09-14T23:57:52.507 INFO:teuthology.orchestra.run.smithi042.stdout:5010 write 0x428b5ae thru 0x4294625 (0x9078 bytes) 2024-09-14T23:57:52.510 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:52.510 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:52.510 INFO:teuthology.orchestra.run.smithi042.stdout:5014 writesame 0x2b9334b thru 0x2b96915 data_size 0x1(0x35cb bytes) 2024-09-14T23:57:52.514 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T23:57:52.520 INFO:teuthology.orchestra.run.smithi042.stdout:5015 write 0x325adfb thru 0x325ba8d (0xc93 bytes) 2024-09-14T23:57:52.523 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:52.523 INFO:teuthology.orchestra.run.smithi042.stdout:5017 read 0x2f76099 thru 0x2f79ffe (0x3f66 bytes) 2024-09-14T23:57:52.523 INFO:teuthology.orchestra.run.smithi042.stdout:5018 write 0x7d3b4ad thru 0x7d493f1 (0xdf45 bytes) 2024-09-14T23:57:52.525 INFO:teuthology.orchestra.run.smithi042.stdout:5019 read 0x34ce3c thru 0x34f4fe (0x26c3 bytes) 2024-09-14T23:57:52.531 INFO:teuthology.orchestra.run.smithi042.stdout:5020 writesame 0xc38a7f0 thru 0xc392e8b data_size 0xa(0x869c bytes) 2024-09-14T23:57:52.532 INFO:teuthology.orchestra.run.smithi042.stdout:5021 write 0xb6edfd9 thru 0xb6f6680 (0x86a8 bytes) 2024-09-14T23:57:52.535 INFO:teuthology.orchestra.run.smithi042.stdout:5022 write 0x41a654c thru 0x41abbb1 (0x5666 bytes) 2024-09-14T23:57:52.538 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:52.550 INFO:teuthology.orchestra.run.smithi042.stdout:5024 trunc from 0xc392e8c to 0x1e878ec 2024-09-14T23:57:52.557 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T23:57:52.563 INFO:teuthology.orchestra.run.smithi042.stdout:5025 punch from 0x1ad0ea3 to 0x1ad14d8, (0x635 bytes) 2024-09-14T23:57:52.580 INFO:teuthology.orchestra.run.smithi042.stdout:5026 write 0xc7ec6a3 thru 0xc7ed48a (0xde8 bytes) 2024-09-14T23:57:52.592 INFO:teuthology.orchestra.run.smithi042.stdout:5028 write 0xe32d08a thru 0xe334102 (0x7079 bytes) 2024-09-14T23:57:52.592 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:52.590+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:52.595 INFO:teuthology.orchestra.run.smithi042.stdout:5030 read 0x212bea6 thru 0x212dc09 (0x1d64 bytes) 2024-09-14T23:57:52.595 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:52.601 INFO:teuthology.orchestra.run.smithi042.stdout:5032 write 0xed9110f thru 0xeda0daf (0xfca1 bytes) 2024-09-14T23:57:52.605 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:52.610 INFO:teuthology.orchestra.run.smithi042.stdout:5036 write 0xede2c0e thru 0xedeebdf (0xbfd2 bytes) 2024-09-14T23:57:52.612 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:52.612 INFO:teuthology.orchestra.run.smithi042.stdout:5038 read 0x93ed14e thru 0x93f3d1b (0x6bce bytes) 2024-09-14T23:57:52.612 INFO:teuthology.orchestra.run.smithi042.stdout:5041 write 0xe3deaa6 thru 0xe3e6d16 (0x8271 bytes) 2024-09-14T23:57:52.615 INFO:teuthology.orchestra.run.smithi042.stdout:5042 read 0x1d4234d thru 0x1d43908 (0x15bc bytes) 2024-09-14T23:57:52.629 INFO:teuthology.orchestra.run.smithi042.stdout:5043 trunc from 0xedeebe0 to 0x36ef71f 2024-09-14T23:57:52.639 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:52.645 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:52.646 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:52.647 INFO:teuthology.orchestra.run.smithi042.stdout:5046 write 0x5af5f9a thru 0x5af61f0 (0x257 bytes) 2024-09-14T23:57:52.651 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:52.656 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T23:57:52.666 INFO:teuthology.orchestra.run.smithi042.stdout:5049 write 0xdb599da thru 0xdb64162 (0xa789 bytes) 2024-09-14T23:57:52.669 INFO:teuthology.orchestra.run.smithi042.stdout:5050 read 0x32da8cb thru 0x32e7a06 (0xd13c bytes) 2024-09-14T23:57:52.669 INFO:teuthology.orchestra.run.smithi042.stdout:5051 punch from 0xd8d863f to 0xd8e165a, (0x901b bytes) 2024-09-14T23:57:52.684 INFO:teuthology.orchestra.run.smithi042.stdout:5054 trunc from 0xdb64163 to 0xd41347 2024-09-14T23:57:52.696 INFO:teuthology.orchestra.run.smithi042.stdout:5055 clone 42 order 21 su 524288 sc 4 2024-09-14T23:57:52.700 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:52.698+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:52.703 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:57:52.703 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:57:52.720 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:57:52.720 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:57:52.736 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:57:52.736 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:57:52.752 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:57:52.752 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:57:52.767 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:52.767 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:52.768 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:57:52.769 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:57:52.776 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T23:57:52.785 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:57:52.785 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:57:52.802 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:57:52.802 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:57:52.817 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:52.815+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:52.818 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:57:52.818 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:57:52.895 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:52.893+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:52.924 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:52.924 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:52.924 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:57:52.924 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:57:52.924 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:52.933 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T23:57:53.017 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:53.015+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:53.084 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:53.085 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:57:53.085 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:57:53.085 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:57:53.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.085 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:3500 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T23:57:53.085 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:37.990193+0000", 2024-09-14T23:57:53.085 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 75.091112139000003, 2024-09-14T23:57:53.085 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99440750899999997, 2024-09-14T23:57:53.085 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:53.086 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:53.086 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:53.086 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:53.086 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:53.086 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:53.086 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3500 2024-09-14T23:57:53.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.087 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:53.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.087 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:53.087 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990193+0000", 2024-09-14T23:57:53.087 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.087 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:53.087 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990193+0000", 2024-09-14T23:57:53.088 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.088 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.088 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:53.088 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990194+0000", 2024-09-14T23:57:53.088 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5000000000000001e-07 2024-09-14T23:57:53.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.089 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.089 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:53.089 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990200+0000", 2024-09-14T23:57:53.089 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4389999999999997e-06 2024-09-14T23:57:53.089 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.090 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.090 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:53.090 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990215+0000", 2024-09-14T23:57:53.090 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4613999999999999e-05 2024-09-14T23:57:53.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:53.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990224+0000", 2024-09-14T23:57:53.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1740000000000006e-06 2024-09-14T23:57:53.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.092 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:53.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990283+0000", 2024-09-14T23:57:53.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9231999999999998e-05 2024-09-14T23:57:53.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.093 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.093 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:57:53.093 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:37.990291+0000", 2024-09-14T23:57:53.093 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5900000000000002e-06 2024-09-14T23:57:53.093 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.094 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.094 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:53.094 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:38.984249+0000", 2024-09-14T23:57:53.094 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99395822700000003 2024-09-14T23:57:53.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:53.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:38.984448+0000", 2024-09-14T23:57:53.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00019855900000000001 2024-09-14T23:57:53.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.096 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:53.096 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:38.984601+0000", 2024-09-14T23:57:53.096 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000152824 2024-09-14T23:57:53.097 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.097 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:53.097 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.097 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.097 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4055 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1093632~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:53.098 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099066+0000", 2024-09-14T23:57:53.098 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 73.982239250999996, 2024-09-14T23:57:53.098 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044174782000000003, 2024-09-14T23:57:53.098 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:53.098 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:53.098 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:53.099 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:53.099 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:53.099 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:53.099 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4055 2024-09-14T23:57:53.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.099 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:53.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.099 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:53.100 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099066+0000", 2024-09-14T23:57:53.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.100 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.100 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:53.100 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099066+0000", 2024-09-14T23:57:53.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.101 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:53.101 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099066+0000", 2024-09-14T23:57:53.101 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3700000000000001e-07 2024-09-14T23:57:53.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.102 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:53.102 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099067+0000", 2024-09-14T23:57:53.102 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1379999999999999e-06 2024-09-14T23:57:53.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.102 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.102 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:53.102 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099073+0000", 2024-09-14T23:57:53.103 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4790000000000004e-06 2024-09-14T23:57:53.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.103 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.103 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:53.103 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099075+0000", 2024-09-14T23:57:53.103 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5320000000000001e-06 2024-09-14T23:57:53.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.103 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.103 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:53.104 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143147+0000", 2024-09-14T23:57:53.104 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044071676999999997 2024-09-14T23:57:53.104 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.104 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.104 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:53.104 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143186+0000", 2024-09-14T23:57:53.104 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8821e-05 2024-09-14T23:57:53.104 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.104 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.104 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:53.104 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143241+0000", 2024-09-14T23:57:53.104 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4797999999999999e-05 2024-09-14T23:57:53.104 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.104 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:53.104 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.105 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4056 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1155072~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:53.105 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099079+0000", 2024-09-14T23:57:53.105 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 73.982226535999999, 2024-09-14T23:57:53.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044295020999999997, 2024-09-14T23:57:53.105 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:53.105 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:53.105 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:53.105 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:53.106 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:53.106 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:53.106 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4056 2024-09-14T23:57:53.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.106 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:53.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.106 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:53.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099079+0000", 2024-09-14T23:57:53.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.106 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:53.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099079+0000", 2024-09-14T23:57:53.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.107 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:53.107 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099079+0000", 2024-09-14T23:57:53.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1500000000000001e-07 2024-09-14T23:57:53.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.107 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:53.107 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099080+0000", 2024-09-14T23:57:53.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0210000000000001e-06 2024-09-14T23:57:53.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.107 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:53.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099083+0000", 2024-09-14T23:57:53.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.174e-06 2024-09-14T23:57:53.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:53.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099086+0000", 2024-09-14T23:57:53.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.446e-06 2024-09-14T23:57:53.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:53.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143269+0000", 2024-09-14T23:57:53.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044183489999999999 2024-09-14T23:57:53.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:53.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143314+0000", 2024-09-14T23:57:53.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5135000000000001e-05 2024-09-14T23:57:53.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:53.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143374+0000", 2024-09-14T23:57:53.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9540000000000003e-05 2024-09-14T23:57:53.110 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.110 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:53.110 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.110 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.111 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4057 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1216512~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:53.111 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099088+0000", 2024-09-14T23:57:53.111 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 73.982216749000003, 2024-09-14T23:57:53.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044406119000000001, 2024-09-14T23:57:53.111 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:53.111 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:53.111 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:53.111 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:53.111 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:53.111 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:53.111 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4057 2024-09-14T23:57:53.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.112 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:53.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:53.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099088+0000", 2024-09-14T23:57:53.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:53.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099088+0000", 2024-09-14T23:57:53.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.113 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:53.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099089+0000", 2024-09-14T23:57:53.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.04e-07 2024-09-14T23:57:53.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.113 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:53.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099090+0000", 2024-09-14T23:57:53.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.026e-06 2024-09-14T23:57:53.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.113 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:53.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099093+0000", 2024-09-14T23:57:53.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0819999999999999e-06 2024-09-14T23:57:53.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.114 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:53.114 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099095+0000", 2024-09-14T23:57:53.114 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4930000000000001e-06 2024-09-14T23:57:53.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.114 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:53.114 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143400+0000", 2024-09-14T23:57:53.114 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044304698000000003 2024-09-14T23:57:53.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.115 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:53.115 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143441+0000", 2024-09-14T23:57:53.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1100000000000003e-05 2024-09-14T23:57:53.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.115 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:53.115 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143495+0000", 2024-09-14T23:57:53.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3516000000000001e-05 2024-09-14T23:57:53.115 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.115 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:53.115 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.115 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4058 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1277952~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:53.116 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099099+0000", 2024-09-14T23:57:53.116 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 73.982206258000005, 2024-09-14T23:57:53.116 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044514583000000003, 2024-09-14T23:57:53.116 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:53.116 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:53.116 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:53.116 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:53.116 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:53.116 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:53.116 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4058 2024-09-14T23:57:53.116 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.116 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:53.116 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:53.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099099+0000", 2024-09-14T23:57:53.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:53.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099099+0000", 2024-09-14T23:57:53.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:53.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099099+0000", 2024-09-14T23:57:53.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9999999999999999e-07 2024-09-14T23:57:53.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.118 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.118 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:53.118 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099100+0000", 2024-09-14T23:57:53.118 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7199999999999997e-07 2024-09-14T23:57:53.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.118 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.118 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:53.118 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099103+0000", 2024-09-14T23:57:53.118 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.095e-06 2024-09-14T23:57:53.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.118 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.118 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:53.118 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099106+0000", 2024-09-14T23:57:53.118 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4229999999999999e-06 2024-09-14T23:57:53.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.119 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:53.119 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143524+0000", 2024-09-14T23:57:53.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044417891000000001 2024-09-14T23:57:53.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.119 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:53.119 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143568+0000", 2024-09-14T23:57:53.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4060000000000002e-05 2024-09-14T23:57:53.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.119 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:53.119 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143614+0000", 2024-09-14T23:57:53.120 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5942000000000002e-05 2024-09-14T23:57:53.120 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.120 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:53.120 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.120 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.120 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4059 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1339392~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:53.120 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099114+0000", 2024-09-14T23:57:53.120 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 73.982191718999999, 2024-09-14T23:57:53.120 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044617194999999998, 2024-09-14T23:57:53.120 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:53.120 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:53.120 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:53.120 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:53.121 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:53.121 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:53.121 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4059 2024-09-14T23:57:53.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.121 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:53.121 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.121 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:53.121 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099114+0000", 2024-09-14T23:57:53.121 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.122 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.122 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:53.122 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099114+0000", 2024-09-14T23:57:53.122 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.122 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.122 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:53.122 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099114+0000", 2024-09-14T23:57:53.122 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5899999999999998e-07 2024-09-14T23:57:53.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.123 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.123 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:53.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099115+0000", 2024-09-14T23:57:53.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3510000000000001e-06 2024-09-14T23:57:53.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.123 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.123 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:53.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099119+0000", 2024-09-14T23:57:53.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.16e-06 2024-09-14T23:57:53.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.123 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.123 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:53.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099123+0000", 2024-09-14T23:57:53.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1339999999999997e-06 2024-09-14T23:57:53.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.124 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:53.124 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143643+0000", 2024-09-14T23:57:53.124 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044519232999999998 2024-09-14T23:57:53.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.124 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:53.124 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143686+0000", 2024-09-14T23:57:53.124 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3349000000000001e-05 2024-09-14T23:57:53.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.124 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:53.124 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143731+0000", 2024-09-14T23:57:53.125 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4709000000000002e-05 2024-09-14T23:57:53.125 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.125 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:53.125 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.125 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.125 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4060 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1400832~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:53.125 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099129+0000", 2024-09-14T23:57:53.125 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 73.982176495999994, 2024-09-14T23:57:53.125 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044721271999999999, 2024-09-14T23:57:53.125 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:53.125 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:53.125 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:53.126 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:53.126 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:53.126 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:53.126 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4060 2024-09-14T23:57:53.126 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.126 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:53.126 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.126 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:53.126 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099129+0000", 2024-09-14T23:57:53.126 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.126 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.126 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.126 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:53.126 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099129+0000", 2024-09-14T23:57:53.127 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.127 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.127 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.127 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:53.127 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099191+0000", 2024-09-14T23:57:53.127 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1805999999999997e-05 2024-09-14T23:57:53.127 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.127 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.127 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:53.127 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099193+0000", 2024-09-14T23:57:53.127 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2330000000000001e-06 2024-09-14T23:57:53.127 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.127 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.127 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:53.127 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099200+0000", 2024-09-14T23:57:53.128 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4200000000000001e-06 2024-09-14T23:57:53.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.128 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.128 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:53.128 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099204+0000", 2024-09-14T23:57:53.128 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7500000000000001e-06 2024-09-14T23:57:53.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.128 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.128 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:53.128 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143760+0000", 2024-09-14T23:57:53.128 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044556182 2024-09-14T23:57:53.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.129 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.129 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:53.129 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143804+0000", 2024-09-14T23:57:53.129 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4239000000000001e-05 2024-09-14T23:57:53.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.129 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.129 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:53.129 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143850+0000", 2024-09-14T23:57:53.129 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5642000000000001e-05 2024-09-14T23:57:53.129 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.129 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:53.129 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.130 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.130 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4061 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1462272~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:53.130 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.099215+0000", 2024-09-14T23:57:53.130 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 73.982090370999998, 2024-09-14T23:57:53.130 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044745659, 2024-09-14T23:57:53.130 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:53.130 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:53.130 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:53.130 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:53.130 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:53.130 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:53.130 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4061 2024-09-14T23:57:53.130 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.130 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:53.131 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.131 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:53.131 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099215+0000", 2024-09-14T23:57:53.131 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.131 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.131 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:53.131 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.099215+0000", 2024-09-14T23:57:53.131 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.131 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.131 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:53.131 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100672+0000", 2024-09-14T23:57:53.131 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00145689 2024-09-14T23:57:53.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.132 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.132 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:53.132 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100677+0000", 2024-09-14T23:57:53.132 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0139999999999998e-06 2024-09-14T23:57:53.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.132 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.132 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:53.132 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100689+0000", 2024-09-14T23:57:53.132 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.183e-05 2024-09-14T23:57:53.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.132 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.132 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:53.132 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100696+0000", 2024-09-14T23:57:53.132 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2599999999999999e-06 2024-09-14T23:57:53.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.133 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:53.133 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143877+0000", 2024-09-14T23:57:53.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043181279000000003 2024-09-14T23:57:53.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.133 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:53.133 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143919+0000", 2024-09-14T23:57:53.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1783000000000003e-05 2024-09-14T23:57:53.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:53.134 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.143961+0000", 2024-09-14T23:57:53.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1603000000000001e-05 2024-09-14T23:57:53.134 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.134 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:53.134 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.134 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4070 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 2015232~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:53.134 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.100903+0000", 2024-09-14T23:57:53.134 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 73.980402663000007, 2024-09-14T23:57:53.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044103815999999997, 2024-09-14T23:57:53.134 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:53.134 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:53.135 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:53.135 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:53.135 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:53.135 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:53.135 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4070 2024-09-14T23:57:53.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.135 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:53.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.135 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:53.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100903+0000", 2024-09-14T23:57:53.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.135 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:53.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.100903+0000", 2024-09-14T23:57:53.136 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.136 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.136 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:53.136 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.102671+0000", 2024-09-14T23:57:53.136 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001768279 2024-09-14T23:57:53.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.136 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.136 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:53.136 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.102675+0000", 2024-09-14T23:57:53.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9609999999999999e-06 2024-09-14T23:57:53.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:53.137 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.102684+0000", 2024-09-14T23:57:53.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.329e-06 2024-09-14T23:57:53.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:53.137 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.102689+0000", 2024-09-14T23:57:53.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7079999999999996e-06 2024-09-14T23:57:53.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:53.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.144912+0000", 2024-09-14T23:57:53.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.042223305000000003 2024-09-14T23:57:53.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:53.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.144960+0000", 2024-09-14T23:57:53.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7670000000000003e-05 2024-09-14T23:57:53.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:53.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.145006+0000", 2024-09-14T23:57:53.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6563999999999997e-05 2024-09-14T23:57:53.138 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.138 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:53.138 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.139 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4096 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1282048~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:53.139 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.104957+0000", 2024-09-14T23:57:53.139 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 73.976348490000007, 2024-09-14T23:57:53.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044093016999999998, 2024-09-14T23:57:53.139 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:53.139 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:53.139 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:53.139 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:53.139 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:53.139 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:53.139 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4096 2024-09-14T23:57:53.140 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.140 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:53.140 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.140 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:53.140 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104957+0000", 2024-09-14T23:57:53.140 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.140 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.140 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.140 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:53.140 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104957+0000", 2024-09-14T23:57:53.140 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.140 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.140 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.140 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:53.141 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104957+0000", 2024-09-14T23:57:53.141 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7400000000000003e-07 2024-09-14T23:57:53.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.141 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:53.141 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104960+0000", 2024-09-14T23:57:53.141 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3800000000000001e-06 2024-09-14T23:57:53.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.141 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:53.141 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104967+0000", 2024-09-14T23:57:53.141 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4030000000000001e-06 2024-09-14T23:57:53.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.142 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:53.142 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104973+0000", 2024-09-14T23:57:53.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7370000000000003e-06 2024-09-14T23:57:53.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.142 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:53.142 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.148883+0000", 2024-09-14T23:57:53.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043910124000000002 2024-09-14T23:57:53.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.142 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:53.142 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.148966+0000", 2024-09-14T23:57:53.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2868999999999996e-05 2024-09-14T23:57:53.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:53.143 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149050+0000", 2024-09-14T23:57:53.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3930000000000004e-05 2024-09-14T23:57:53.143 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.143 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:53.143 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.143 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4097 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1343488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:53.143 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.104983+0000", 2024-09-14T23:57:53.143 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 73.976322701000001, 2024-09-14T23:57:53.144 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044182104999999999, 2024-09-14T23:57:53.144 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:53.144 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:53.144 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:53.144 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:53.144 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:53.144 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:53.144 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4097 2024-09-14T23:57:53.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.144 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:53.144 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.144 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:53.144 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104983+0000", 2024-09-14T23:57:53.145 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.145 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.145 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:53.145 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.104983+0000", 2024-09-14T23:57:53.145 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.145 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.145 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:53.145 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105207+0000", 2024-09-14T23:57:53.145 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000224687 2024-09-14T23:57:53.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.145 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.145 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:53.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105211+0000", 2024-09-14T23:57:53.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3050000000000001e-06 2024-09-14T23:57:53.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.146 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:53.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105217+0000", 2024-09-14T23:57:53.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8170000000000001e-06 2024-09-14T23:57:53.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.146 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:53.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105222+0000", 2024-09-14T23:57:53.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.267e-06 2024-09-14T23:57:53.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.146 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:53.147 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149079+0000", 2024-09-14T23:57:53.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043857581999999999 2024-09-14T23:57:53.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.147 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:53.147 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149122+0000", 2024-09-14T23:57:53.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2899999999999999e-05 2024-09-14T23:57:53.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.147 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:53.147 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149165+0000", 2024-09-14T23:57:53.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2546999999999998e-05 2024-09-14T23:57:53.147 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.147 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:53.148 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.148 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.148 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4098 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1404928~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:53.148 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.105242+0000", 2024-09-14T23:57:53.148 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 73.976062889000005, 2024-09-14T23:57:53.148 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044121104000000001, 2024-09-14T23:57:53.148 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:53.148 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:53.148 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:53.148 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:53.148 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:53.148 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:53.149 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4098 2024-09-14T23:57:53.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.149 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:53.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.149 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:53.149 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105242+0000", 2024-09-14T23:57:53.149 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.149 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:53.149 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105242+0000", 2024-09-14T23:57:53.149 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:53.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105243+0000", 2024-09-14T23:57:53.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0399999999999995e-07 2024-09-14T23:57:53.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:53.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105246+0000", 2024-09-14T23:57:53.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6309999999999999e-06 2024-09-14T23:57:53.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:53.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105253+0000", 2024-09-14T23:57:53.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0940000000000002e-06 2024-09-14T23:57:53.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:53.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105260+0000", 2024-09-14T23:57:53.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1799999999999999e-06 2024-09-14T23:57:53.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:53.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149201+0000", 2024-09-14T23:57:53.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043940629000000002 2024-09-14T23:57:53.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:53.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149283+0000", 2024-09-14T23:57:53.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2360000000000004e-05 2024-09-14T23:57:53.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.152 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:53.152 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149363+0000", 2024-09-14T23:57:53.152 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0506000000000005e-05 2024-09-14T23:57:53.152 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.152 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:53.152 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.152 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4099 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1466368~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:53.152 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.105270+0000", 2024-09-14T23:57:53.152 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 73.976035293999999, 2024-09-14T23:57:53.152 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044195433999999999, 2024-09-14T23:57:53.152 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:53.153 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:53.153 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:53.153 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:53.153 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:53.153 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:53.153 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4099 2024-09-14T23:57:53.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.153 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:53.153 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.153 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:53.153 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105270+0000", 2024-09-14T23:57:53.153 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.154 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.154 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:53.154 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105270+0000", 2024-09-14T23:57:53.154 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.154 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.154 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:53.154 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105429+0000", 2024-09-14T23:57:53.154 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015940299999999999 2024-09-14T23:57:53.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.154 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.154 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:53.154 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105433+0000", 2024-09-14T23:57:53.154 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3579999999999999e-06 2024-09-14T23:57:53.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:53.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105439+0000", 2024-09-14T23:57:53.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1759999999999998e-06 2024-09-14T23:57:53.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:53.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105445+0000", 2024-09-14T23:57:53.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9689999999999999e-06 2024-09-14T23:57:53.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:53.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149381+0000", 2024-09-14T23:57:53.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043936389999999999 2024-09-14T23:57:53.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.156 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:53.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149422+0000", 2024-09-14T23:57:53.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0865000000000003e-05 2024-09-14T23:57:53.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.156 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:53.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149465+0000", 2024-09-14T23:57:53.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3272999999999997e-05 2024-09-14T23:57:53.156 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.156 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:53.156 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.157 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.157 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4100 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1527808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:53.157 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.105454+0000", 2024-09-14T23:57:53.157 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 73.975850823000002, 2024-09-14T23:57:53.157 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044217411999999998, 2024-09-14T23:57:53.157 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:53.157 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:53.157 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:53.157 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:53.157 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:53.157 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:53.157 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4100 2024-09-14T23:57:53.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.157 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:53.158 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.158 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:53.158 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105454+0000", 2024-09-14T23:57:53.158 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.158 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.158 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:53.158 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105454+0000", 2024-09-14T23:57:53.158 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.158 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.158 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:53.158 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105578+0000", 2024-09-14T23:57:53.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012401799999999999 2024-09-14T23:57:53.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.159 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:53.159 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105583+0000", 2024-09-14T23:57:53.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1779999999999996e-06 2024-09-14T23:57:53.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.159 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:53.159 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105590+0000", 2024-09-14T23:57:53.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4279999999999999e-06 2024-09-14T23:57:53.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.160 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:53.160 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105596+0000", 2024-09-14T23:57:53.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0120000000000002e-06 2024-09-14T23:57:53.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.160 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.160 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:53.160 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149503+0000", 2024-09-14T23:57:53.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043906580000000001 2024-09-14T23:57:53.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.160 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.160 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:53.160 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149585+0000", 2024-09-14T23:57:53.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2356999999999998e-05 2024-09-14T23:57:53.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.161 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.161 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:53.161 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149672+0000", 2024-09-14T23:57:53.161 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6839e-05 2024-09-14T23:57:53.161 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.161 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:53.161 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.161 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.161 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4101 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1589248~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:53.161 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.105609+0000", 2024-09-14T23:57:53.161 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 73.975696283999994, 2024-09-14T23:57:53.161 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044175987999999999, 2024-09-14T23:57:53.162 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:53.162 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:53.162 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:53.162 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:53.162 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:53.162 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:53.162 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4101 2024-09-14T23:57:53.162 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.162 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:53.162 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.162 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:53.162 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105609+0000", 2024-09-14T23:57:53.162 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.162 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.163 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:53.163 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.105609+0000", 2024-09-14T23:57:53.163 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.163 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:53.163 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.106671+0000", 2024-09-14T23:57:53.163 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010619270000000001 2024-09-14T23:57:53.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.163 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:53.163 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.106675+0000", 2024-09-14T23:57:53.164 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4429999999999996e-06 2024-09-14T23:57:53.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.164 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.164 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:53.164 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.106685+0000", 2024-09-14T23:57:53.164 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7429999999999997e-06 2024-09-14T23:57:53.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.164 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.164 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:53.164 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.106690+0000", 2024-09-14T23:57:53.164 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.293e-06 2024-09-14T23:57:53.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.164 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.164 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:53.164 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149700+0000", 2024-09-14T23:57:53.165 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043009512 2024-09-14T23:57:53.165 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.165 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.165 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:53.165 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149742+0000", 2024-09-14T23:57:53.165 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2083999999999999e-05 2024-09-14T23:57:53.165 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.165 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.165 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:53.165 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.149785+0000", 2024-09-14T23:57:53.165 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2985999999999998e-05 2024-09-14T23:57:53.165 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.165 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:53.165 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.166 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.166 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4116 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 180224~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:53.166 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.107575+0000", 2024-09-14T23:57:53.166 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 73.973729816000002, 2024-09-14T23:57:53.166 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044157522999999997, 2024-09-14T23:57:53.166 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:53.166 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:53.166 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:53.166 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:53.166 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:53.166 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:53.166 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4116 2024-09-14T23:57:53.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.166 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:53.166 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.167 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:53.167 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.107575+0000", 2024-09-14T23:57:53.167 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.167 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.167 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.167 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:53.167 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.107575+0000", 2024-09-14T23:57:53.167 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.167 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.167 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.167 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:53.167 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108671+0000", 2024-09-14T23:57:53.167 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010954510000000001 2024-09-14T23:57:53.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.168 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.168 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:53.168 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108676+0000", 2024-09-14T23:57:53.168 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6639999999999997e-06 2024-09-14T23:57:53.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.168 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.168 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:53.168 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108685+0000", 2024-09-14T23:57:53.168 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1130000000000007e-06 2024-09-14T23:57:53.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.168 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.168 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:53.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108691+0000", 2024-09-14T23:57:53.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6019999999999996e-06 2024-09-14T23:57:53.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.169 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:53.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.151530+0000", 2024-09-14T23:57:53.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.042838345999999999 2024-09-14T23:57:53.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.169 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:53.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.151619+0000", 2024-09-14T23:57:53.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9302999999999995e-05 2024-09-14T23:57:53.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.170 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:53.170 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.151733+0000", 2024-09-14T23:57:53.170 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011404399999999999 2024-09-14T23:57:53.170 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.170 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:53.170 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.170 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.170 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4122 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 548864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:53.170 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.108842+0000", 2024-09-14T23:57:53.170 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 73.972462977999996, 2024-09-14T23:57:53.170 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044103631999999997, 2024-09-14T23:57:53.170 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:53.170 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:53.171 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:53.171 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:53.171 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:53.171 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:53.171 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4122 2024-09-14T23:57:53.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.171 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:53.171 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.171 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:53.171 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108842+0000", 2024-09-14T23:57:53.171 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.171 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.171 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:53.171 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108842+0000", 2024-09-14T23:57:53.171 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.172 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.172 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.172 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:53.172 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108843+0000", 2024-09-14T23:57:53.172 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7000000000000001e-07 2024-09-14T23:57:53.172 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.172 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.172 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:53.172 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108844+0000", 2024-09-14T23:57:53.172 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6479999999999999e-06 2024-09-14T23:57:53.172 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.172 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.172 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:53.172 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108848+0000", 2024-09-14T23:57:53.173 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8029999999999998e-06 2024-09-14T23:57:53.173 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.173 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.173 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:53.173 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108851+0000", 2024-09-14T23:57:53.173 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7939999999999998e-06 2024-09-14T23:57:53.173 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.173 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.173 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:53.173 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.152838+0000", 2024-09-14T23:57:53.173 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043987351000000001 2024-09-14T23:57:53.173 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.173 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.173 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:53.174 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.152885+0000", 2024-09-14T23:57:53.174 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7237000000000002e-05 2024-09-14T23:57:53.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.174 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.174 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:53.174 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.152946+0000", 2024-09-14T23:57:53.174 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0529000000000003e-05 2024-09-14T23:57:53.174 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.174 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:53.174 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.174 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.174 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4123 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 610304~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:53.174 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.108855+0000", 2024-09-14T23:57:53.174 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 73.972450648999995, 2024-09-14T23:57:53.175 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044321610999999997, 2024-09-14T23:57:53.175 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:53.175 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:53.175 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:53.175 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:53.175 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:53.175 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:53.175 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4123 2024-09-14T23:57:53.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.175 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:53.175 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.175 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:53.175 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108855+0000", 2024-09-14T23:57:53.175 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.176 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.176 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:53.176 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.108855+0000", 2024-09-14T23:57:53.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.176 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.176 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:53.176 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.110671+0000", 2024-09-14T23:57:53.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018159719999999999 2024-09-14T23:57:53.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.176 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.176 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:53.176 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.110676+0000", 2024-09-14T23:57:53.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9450000000000001e-06 2024-09-14T23:57:53.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.177 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:53.177 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.110685+0000", 2024-09-14T23:57:53.177 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5359999999999998e-06 2024-09-14T23:57:53.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.177 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:53.177 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.110692+0000", 2024-09-14T23:57:53.177 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9280000000000004e-06 2024-09-14T23:57:53.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.177 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:53.178 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.152984+0000", 2024-09-14T23:57:53.178 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.042291677999999999 2024-09-14T23:57:53.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.178 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:53.178 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.153066+0000", 2024-09-14T23:57:53.178 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2657999999999994e-05 2024-09-14T23:57:53.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.178 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:53.178 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.153176+0000", 2024-09-14T23:57:53.178 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000109894 2024-09-14T23:57:53.178 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.178 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:53.179 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.179 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.179 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:4136 3.1c 3:3ac759b3:::rbd_data.1118d8884d62.0000000000000007:3 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:57:53.179 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:39.111056+0000", 2024-09-14T23:57:53.179 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 73.970249500999998, 2024-09-14T23:57:53.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044097428000000001, 2024-09-14T23:57:53.179 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:53.179 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:53.179 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:53.179 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:53.179 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:53.179 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:53.179 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 4136 2024-09-14T23:57:53.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.180 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:53.180 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.180 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:53.180 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.111056+0000", 2024-09-14T23:57:53.180 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.180 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.180 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:53.180 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.111056+0000", 2024-09-14T23:57:53.180 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.180 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.181 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:53.181 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.112671+0000", 2024-09-14T23:57:53.181 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016153999999999999 2024-09-14T23:57:53.181 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.181 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.181 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:53.181 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.112675+0000", 2024-09-14T23:57:53.181 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2830000000000003e-06 2024-09-14T23:57:53.181 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.181 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.181 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:53.181 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.112684+0000", 2024-09-14T23:57:53.181 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7800000000000006e-06 2024-09-14T23:57:53.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.182 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.182 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:53.182 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.112689+0000", 2024-09-14T23:57:53.182 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0000000000000004e-06 2024-09-14T23:57:53.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.182 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.182 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:53.182 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.155064+0000", 2024-09-14T23:57:53.182 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.042374423000000001 2024-09-14T23:57:53.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.182 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.182 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:53.182 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.155111+0000", 2024-09-14T23:57:53.183 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7367000000000002e-05 2024-09-14T23:57:53.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.183 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.183 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:53.183 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:39.155153+0000", 2024-09-14T23:57:53.183 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2175000000000002e-05 2024-09-14T23:57:53.183 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.183 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:53.183 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.183 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.183 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:46572 3.1d 3:b9e1f524:::rbd_header.1118cee06433:head [watch reconnect cookie 140670782499824 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-14T23:57:53.183 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:57.046868+0000", 2024-09-14T23:57:53.184 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 56.034437695000001, 2024-09-14T23:57:53.184 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0243412620000001, 2024-09-14T23:57:53.184 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:53.184 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:53.184 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:53.184 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:53.184 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:53.184 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:53.184 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 46572 2024-09-14T23:57:53.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.184 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:53.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.184 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:53.185 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046868+0000", 2024-09-14T23:57:53.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.185 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.185 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:53.185 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046868+0000", 2024-09-14T23:57:53.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.185 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.185 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:53.185 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046870+0000", 2024-09-14T23:57:53.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.204e-06 2024-09-14T23:57:53.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.185 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.186 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:53.186 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046877+0000", 2024-09-14T23:57:53.186 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5100000000000001e-06 2024-09-14T23:57:53.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.186 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.186 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:53.186 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046898+0000", 2024-09-14T23:57:53.186 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1149999999999999e-05 2024-09-14T23:57:53.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.186 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.186 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:53.186 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046911+0000", 2024-09-14T23:57:53.186 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2234e-05 2024-09-14T23:57:53.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.187 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.187 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:53.187 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.046988+0000", 2024-09-14T23:57:53.187 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7445000000000001e-05 2024-09-14T23:57:53.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.187 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.187 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:57:53.187 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:57.047001+0000", 2024-09-14T23:57:53.187 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2951e-05 2024-09-14T23:57:53.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.187 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.187 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:53.187 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.069287+0000", 2024-09-14T23:57:53.187 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.022285748 2024-09-14T23:57:53.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.188 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.188 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for rw locks", 2024-09-14T23:57:53.188 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.069397+0000", 2024-09-14T23:57:53.188 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000110326 2024-09-14T23:57:53.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.188 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.188 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:53.188 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.069997+0000", 2024-09-14T23:57:53.188 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00059984399999999998 2024-09-14T23:57:53.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.188 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.188 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:53.188 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.070109+0000", 2024-09-14T23:57:53.189 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000112367 2024-09-14T23:57:53.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.189 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.189 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:53.189 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:58.071209+0000", 2024-09-14T23:57:53.189 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010994830000000001 2024-09-14T23:57:53.189 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.189 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:53.189 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.189 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.189 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:53.189 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:53.190 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T23:57:53.190 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T23:57:53.190 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-14T23:57:53.190 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T23:57:53.191 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone41 from 0xd41347 (overlap 0xd41347) to 0x2a6a41 2024-09-14T23:57:53.192 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T23:57:53.192 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:53.188+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:53.240 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #40, image image_client.0-clone40 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent41 2024-09-14T23:57:53.291 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:53.289+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:53.353 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:53.353 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:53.353 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:57:53.353 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:53.362 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T23:57:53.403 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:53.401+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:53.493 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:53.491+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:53.510 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:53.510 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:53.510 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:57:53.510 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:57:53.510 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:53.519 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T23:57:53.555 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:57:53.555 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:53.555 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 99, 2024-09-14T23:57:53.555 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "21916fcc-f95c-4bfa-8b8b-11ca45845613", 2024-09-14T23:57:53.555 INFO:teuthology.orchestra.run.smithi042.stdout: "created": "2024-09-14T23:56:03.167948+0000", 2024-09-14T23:57:53.555 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-09-14T23:57:53.075063+0000", 2024-09-14T23:57:53.556 INFO:teuthology.orchestra.run.smithi042.stdout: "last_up_change": "2024-09-14T23:56:14.322882+0000", 2024-09-14T23:57:53.556 INFO:teuthology.orchestra.run.smithi042.stdout: "last_in_change": "2024-09-14T23:56:56.035972+0000", 2024-09-14T23:57:53.556 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T23:57:53.556 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_num": 5799936, 2024-09-14T23:57:53.556 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_set": [ 2024-09-14T23:57:53.556 INFO:teuthology.orchestra.run.smithi042.stdout: "pglog_hardlimit", 2024-09-14T23:57:53.556 INFO:teuthology.orchestra.run.smithi042.stdout: "purged_snapdirs", 2024-09-14T23:57:53.556 INFO:teuthology.orchestra.run.smithi042.stdout: "recovery_deletes", 2024-09-14T23:57:53.556 INFO:teuthology.orchestra.run.smithi042.stdout: "sortbitwise" 2024-09-14T23:57:53.556 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-14T23:57:53.556 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_version": 6, 2024-09-14T23:57:53.556 INFO:teuthology.orchestra.run.smithi042.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T23:57:53.557 INFO:teuthology.orchestra.run.smithi042.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T23:57:53.557 INFO:teuthology.orchestra.run.smithi042.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T23:57:53.557 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_snapshot": "", 2024-09-14T23:57:53.557 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_max": 3, 2024-09-14T23:57:53.557 INFO:teuthology.orchestra.run.smithi042.stdout: "max_osd": 8, 2024-09-14T23:57:53.557 INFO:teuthology.orchestra.run.smithi042.stdout: "require_min_compat_client": "luminous", 2024-09-14T23:57:53.557 INFO:teuthology.orchestra.run.smithi042.stdout: "min_compat_client": "jewel", 2024-09-14T23:57:53.557 INFO:teuthology.orchestra.run.smithi042.stdout: "require_osd_release": "squid", 2024-09-14T23:57:53.557 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_crimson": false, 2024-09-14T23:57:53.557 INFO:teuthology.orchestra.run.smithi042.stdout: "pools": [ 2024-09-14T23:57:53.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.557 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 1, 2024-09-14T23:57:53.557 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": ".mgr", 2024-09-14T23:57:53.557 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-14T23:56:13.500197+0000", 2024-09-14T23:57:53.558 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-09-14T23:57:53.558 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool", 2024-09-14T23:57:53.558 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-14T23:57:53.558 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-14T23:57:53.558 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-14T23:57:53.558 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-14T23:57:53.558 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-14T23:57:53.558 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-14T23:57:53.558 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T23:57:53.558 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T23:57:53.558 INFO:teuthology.orchestra.run.smithi042.stdout: "is_stretch_pool": false, 2024-09-14T23:57:53.558 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-14T23:57:53.558 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-14T23:57:53.558 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 1, 2024-09-14T23:57:53.559 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 1, 2024-09-14T23:57:53.559 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 1, 2024-09-14T23:57:53.559 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 1, 2024-09-14T23:57:53.559 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 1, 2024-09-14T23:57:53.559 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-14T23:57:53.559 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-14T23:57:53.559 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-14T23:57:53.559 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-14T23:57:53.559 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-14T23:57:53.559 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-14T23:57:53.559 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-14T23:57:53.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.559 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "17", 2024-09-14T23:57:53.560 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "0", 2024-09-14T23:57:53.560 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T23:57:53.560 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T23:57:53.560 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-14T23:57:53.560 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-14T23:57:53.560 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 0, 2024-09-14T23:57:53.560 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 0, 2024-09-14T23:57:53.560 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-14T23:57:53.560 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-14T23:57:53.560 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-14T23:57:53.560 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-14T23:57:53.560 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-14T23:57:53.560 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-14T23:57:53.561 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-14T23:57:53.561 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-14T23:57:53.561 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-14T23:57:53.561 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-14T23:57:53.561 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-14T23:57:53.561 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T23:57:53.561 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T23:57:53.561 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T23:57:53.561 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-14T23:57:53.561 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-14T23:57:53.561 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-14T23:57:53.561 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-14T23:57:53.561 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-14T23:57:53.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.562 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-14T23:57:53.562 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-14T23:57:53.562 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-14T23:57:53.562 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-14T23:57:53.562 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-14T23:57:53.562 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T23:57:53.562 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-14T23:57:53.562 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-14T23:57:53.562 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-14T23:57:53.562 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-14T23:57:53.562 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-14T23:57:53.562 INFO:teuthology.orchestra.run.smithi042.stdout: "options": { 2024-09-14T23:57:53.563 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_max": 32, 2024-09-14T23:57:53.563 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_min": 1 2024-09-14T23:57:53.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.563 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-14T23:57:53.563 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": {} 2024-09-14T23:57:53.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.563 INFO:teuthology.orchestra.run.smithi042.stdout: "read_balance": { 2024-09-14T23:57:53.563 INFO:teuthology.orchestra.run.smithi042.stdout: "score_type": "Fair distribution", 2024-09-14T23:57:53.563 INFO:teuthology.orchestra.run.smithi042.stdout: "score_acting": 6.9000000953674316, 2024-09-14T23:57:53.563 INFO:teuthology.orchestra.run.smithi042.stdout: "score_stable": 6.9000000953674316, 2024-09-14T23:57:53.563 INFO:teuthology.orchestra.run.smithi042.stdout: "optimal_score": 0.28999999165534973, 2024-09-14T23:57:53.563 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_acting": 2, 2024-09-14T23:57:53.563 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_stable": 2, 2024-09-14T23:57:53.563 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity_weighted": 0.8712921142578125, 2024-09-14T23:57:53.564 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity": 1, 2024-09-14T23:57:53.564 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity_weighted": 0.87000000476837158 2024-09-14T23:57:53.564 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:53.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.564 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 2, 2024-09-14T23:57:53.564 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": "rbd", 2024-09-14T23:57:53.564 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-14T23:56:21.284705+0000", 2024-09-14T23:57:53.564 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 8193, 2024-09-14T23:57:53.564 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T23:57:53.564 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-14T23:57:53.564 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-14T23:57:53.565 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-14T23:57:53.565 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-14T23:57:53.565 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-14T23:57:53.565 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-14T23:57:53.565 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T23:57:53.565 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T23:57:53.565 INFO:teuthology.orchestra.run.smithi042.stdout: "is_stretch_pool": false, 2024-09-14T23:57:53.565 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-14T23:57:53.565 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-14T23:57:53.565 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 8, 2024-09-14T23:57:53.565 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 8, 2024-09-14T23:57:53.565 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 8, 2024-09-14T23:57:53.566 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 8, 2024-09-14T23:57:53.566 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 8, 2024-09-14T23:57:53.566 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-14T23:57:53.566 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-14T23:57:53.566 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-14T23:57:53.566 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-14T23:57:53.566 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-14T23:57:53.566 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-14T23:57:53.566 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-14T23:57:53.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.566 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "21", 2024-09-14T23:57:53.566 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "0", 2024-09-14T23:57:53.566 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T23:57:53.567 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T23:57:53.567 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-14T23:57:53.567 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-14T23:57:53.567 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 2, 2024-09-14T23:57:53.567 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 21, 2024-09-14T23:57:53.567 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-14T23:57:53.567 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-14T23:57:53.567 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-14T23:57:53.567 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-14T23:57:53.567 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-14T23:57:53.567 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-14T23:57:53.567 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-14T23:57:53.567 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-14T23:57:53.568 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-14T23:57:53.568 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-14T23:57:53.568 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-14T23:57:53.568 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T23:57:53.568 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T23:57:53.568 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T23:57:53.568 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-14T23:57:53.568 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-14T23:57:53.568 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-14T23:57:53.568 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-14T23:57:53.568 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-14T23:57:53.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.569 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-14T23:57:53.569 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-14T23:57:53.569 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-14T23:57:53.569 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-14T23:57:53.569 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-14T23:57:53.569 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T23:57:53.569 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-14T23:57:53.569 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-14T23:57:53.569 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-14T23:57:53.569 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-14T23:57:53.569 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-14T23:57:53.569 INFO:teuthology.orchestra.run.smithi042.stdout: "options": {}, 2024-09-14T23:57:53.569 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-14T23:57:53.569 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd": {} 2024-09-14T23:57:53.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.570 INFO:teuthology.orchestra.run.smithi042.stdout: "read_balance": { 2024-09-14T23:57:53.570 INFO:teuthology.orchestra.run.smithi042.stdout: "score_type": "Fair distribution", 2024-09-14T23:57:53.570 INFO:teuthology.orchestra.run.smithi042.stdout: "score_acting": 2.630000114440918, 2024-09-14T23:57:53.570 INFO:teuthology.orchestra.run.smithi042.stdout: "score_stable": 2.630000114440918, 2024-09-14T23:57:53.570 INFO:teuthology.orchestra.run.smithi042.stdout: "optimal_score": 1, 2024-09-14T23:57:53.570 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_acting": 2.630000114440918, 2024-09-14T23:57:53.570 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_stable": 2.630000114440918, 2024-09-14T23:57:53.570 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity_weighted": 0.8712921142578125, 2024-09-14T23:57:53.570 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity": 1, 2024-09-14T23:57:53.570 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity_weighted": 0.87000000476837158 2024-09-14T23:57:53.570 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:53.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.571 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 3, 2024-09-14T23:57:53.571 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": "pool_client.0", 2024-09-14T23:57:53.571 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-14T23:56:30.861504+0000", 2024-09-14T23:57:53.571 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 8193, 2024-09-14T23:57:53.571 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T23:57:53.571 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-14T23:57:53.571 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-14T23:57:53.571 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-14T23:57:53.571 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-14T23:57:53.571 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-14T23:57:53.571 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-14T23:57:53.571 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T23:57:53.572 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T23:57:53.572 INFO:teuthology.orchestra.run.smithi042.stdout: "is_stretch_pool": false, 2024-09-14T23:57:53.572 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-14T23:57:53.572 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-14T23:57:53.572 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 32, 2024-09-14T23:57:53.572 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 32, 2024-09-14T23:57:53.572 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 32, 2024-09-14T23:57:53.572 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 32, 2024-09-14T23:57:53.572 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 32, 2024-09-14T23:57:53.572 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-14T23:57:53.572 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-14T23:57:53.572 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-14T23:57:53.573 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-14T23:57:53.573 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-14T23:57:53.573 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-14T23:57:53.573 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-14T23:57:53.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.573 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "99", 2024-09-14T23:57:53.573 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "0", 2024-09-14T23:57:53.573 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T23:57:53.574 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T23:57:53.574 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-14T23:57:53.574 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-14T23:57:53.574 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 44, 2024-09-14T23:57:53.574 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 99, 2024-09-14T23:57:53.574 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-14T23:57:53.574 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-14T23:57:53.574 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-14T23:57:53.574 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-14T23:57:53.574 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-14T23:57:53.574 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-14T23:57:53.574 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-14T23:57:53.574 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-14T23:57:53.575 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-14T23:57:53.575 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-14T23:57:53.575 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-14T23:57:53.575 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T23:57:53.575 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T23:57:53.575 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T23:57:53.575 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-14T23:57:53.575 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-14T23:57:53.575 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-14T23:57:53.575 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-14T23:57:53.575 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-14T23:57:53.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.575 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-14T23:57:53.576 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-14T23:57:53.576 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-14T23:57:53.576 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-14T23:57:53.576 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-14T23:57:53.576 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T23:57:53.576 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-14T23:57:53.576 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-14T23:57:53.576 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-14T23:57:53.576 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-14T23:57:53.576 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-14T23:57:53.576 INFO:teuthology.orchestra.run.smithi042.stdout: "options": {}, 2024-09-14T23:57:53.576 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-14T23:57:53.576 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd": {} 2024-09-14T23:57:53.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.577 INFO:teuthology.orchestra.run.smithi042.stdout: "read_balance": { 2024-09-14T23:57:53.577 INFO:teuthology.orchestra.run.smithi042.stdout: "score_type": "Fair distribution", 2024-09-14T23:57:53.577 INFO:teuthology.orchestra.run.smithi042.stdout: "score_acting": 1.75, 2024-09-14T23:57:53.577 INFO:teuthology.orchestra.run.smithi042.stdout: "score_stable": 1.75, 2024-09-14T23:57:53.577 INFO:teuthology.orchestra.run.smithi042.stdout: "optimal_score": 1.1399999856948853, 2024-09-14T23:57:53.577 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_acting": 2, 2024-09-14T23:57:53.577 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_stable": 2, 2024-09-14T23:57:53.577 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity_weighted": 0.8712921142578125, 2024-09-14T23:57:53.577 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity": 1, 2024-09-14T23:57:53.577 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity_weighted": 0.87000000476837158 2024-09-14T23:57:53.577 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:53.578 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:53.578 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-14T23:57:53.578 INFO:teuthology.orchestra.run.smithi042.stdout: "osds": [ 2024-09-14T23:57:53.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.578 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 0, 2024-09-14T23:57:53.578 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "db9d0c67-2d60-4592-9ad3-f99e5294f93f", 2024-09-14T23:57:53.578 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-14T23:57:53.578 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-14T23:57:53.578 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-14T23:57:53.578 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-14T23:57:53.579 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-14T23:57:53.579 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-14T23:57:53.579 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 14, 2024-09-14T23:57:53.579 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 49, 2024-09-14T23:57:53.579 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-14T23:57:53.579 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-14T23:57:53.579 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-14T23:57:53.579 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:53.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.579 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:53.579 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6816", 2024-09-14T23:57:53.579 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 314865173 2024-09-14T23:57:53.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.580 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:53.580 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6817", 2024-09-14T23:57:53.580 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 314865173 2024-09-14T23:57:53.580 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:53.580 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:53.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.580 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-14T23:57:53.580 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:53.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.580 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:53.580 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6818", 2024-09-14T23:57:53.580 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 314865173 2024-09-14T23:57:53.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.581 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:53.581 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6819", 2024-09-14T23:57:53.581 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 314865173 2024-09-14T23:57:53.581 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:53.581 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:53.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.581 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-14T23:57:53.581 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:53.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.581 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:53.581 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6822", 2024-09-14T23:57:53.582 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 314865173 2024-09-14T23:57:53.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.582 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:53.582 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6823", 2024-09-14T23:57:53.582 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 314865173 2024-09-14T23:57:53.582 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:53.582 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:53.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.582 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-14T23:57:53.582 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:53.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.582 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:53.583 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6820", 2024-09-14T23:57:53.583 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 314865173 2024-09-14T23:57:53.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.583 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:53.583 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6821", 2024-09-14T23:57:53.583 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 314865173 2024-09-14T23:57:53.583 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:53.583 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:53.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.583 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6817/314865173", 2024-09-14T23:57:53.583 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6819/314865173", 2024-09-14T23:57:53.583 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6823/314865173", 2024-09-14T23:57:53.583 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6821/314865173", 2024-09-14T23:57:53.584 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-14T23:57:53.584 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-14T23:57:53.584 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-14T23:57:53.584 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:53.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.584 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 1, 2024-09-14T23:57:53.584 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "6c1a2765-c591-4c57-8ab3-ee6645970aa5", 2024-09-14T23:57:53.584 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-14T23:57:53.584 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-14T23:57:53.584 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-14T23:57:53.584 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-14T23:57:53.584 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-14T23:57:53.585 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-14T23:57:53.585 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 14, 2024-09-14T23:57:53.585 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 92, 2024-09-14T23:57:53.585 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-14T23:57:53.585 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-14T23:57:53.585 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-14T23:57:53.585 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:53.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.585 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:53.585 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6800", 2024-09-14T23:57:53.585 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3197912310 2024-09-14T23:57:53.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.586 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:53.586 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6801", 2024-09-14T23:57:53.586 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3197912310 2024-09-14T23:57:53.586 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:53.586 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:53.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.586 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-14T23:57:53.586 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:53.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.586 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:53.586 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6802", 2024-09-14T23:57:53.586 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3197912310 2024-09-14T23:57:53.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.587 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:53.587 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6803", 2024-09-14T23:57:53.587 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3197912310 2024-09-14T23:57:53.587 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:53.587 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:53.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.587 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-14T23:57:53.587 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:53.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.587 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:53.587 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6806", 2024-09-14T23:57:53.587 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3197912310 2024-09-14T23:57:53.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.588 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:53.588 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6807", 2024-09-14T23:57:53.588 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3197912310 2024-09-14T23:57:53.588 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:53.588 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:53.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.588 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-14T23:57:53.588 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:53.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.588 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:53.588 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6804", 2024-09-14T23:57:53.588 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3197912310 2024-09-14T23:57:53.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.589 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:53.589 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6805", 2024-09-14T23:57:53.589 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3197912310 2024-09-14T23:57:53.589 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:53.589 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:53.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.589 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6801/3197912310", 2024-09-14T23:57:53.589 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6803/3197912310", 2024-09-14T23:57:53.589 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6807/3197912310", 2024-09-14T23:57:53.589 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6805/3197912310", 2024-09-14T23:57:53.589 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-14T23:57:53.589 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-14T23:57:53.590 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-14T23:57:53.590 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:53.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.590 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 2, 2024-09-14T23:57:53.590 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "930db893-7be6-4645-a7f1-e043c5ec95eb", 2024-09-14T23:57:53.590 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-14T23:57:53.590 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-14T23:57:53.590 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-14T23:57:53.590 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-14T23:57:53.590 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-14T23:57:53.590 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-14T23:57:53.590 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 13, 2024-09-14T23:57:53.590 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 46, 2024-09-14T23:57:53.591 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-14T23:57:53.591 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-14T23:57:53.591 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-14T23:57:53.591 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:53.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.591 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:53.591 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6808", 2024-09-14T23:57:53.591 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3241153660 2024-09-14T23:57:53.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.591 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:53.591 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6809", 2024-09-14T23:57:53.591 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3241153660 2024-09-14T23:57:53.592 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:53.592 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:53.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.592 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-14T23:57:53.592 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:53.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.592 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:53.592 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6810", 2024-09-14T23:57:53.592 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3241153660 2024-09-14T23:57:53.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.592 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:53.592 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6811", 2024-09-14T23:57:53.593 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3241153660 2024-09-14T23:57:53.593 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:53.593 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:53.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.593 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-14T23:57:53.593 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:53.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.593 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:53.593 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6814", 2024-09-14T23:57:53.593 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3241153660 2024-09-14T23:57:53.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.593 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:53.593 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6815", 2024-09-14T23:57:53.594 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3241153660 2024-09-14T23:57:53.594 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:53.594 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:53.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.594 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-14T23:57:53.594 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:53.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.594 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:53.594 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6812", 2024-09-14T23:57:53.594 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3241153660 2024-09-14T23:57:53.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.594 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:53.594 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6813", 2024-09-14T23:57:53.595 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3241153660 2024-09-14T23:57:53.595 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:53.595 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:53.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.595 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6809/3241153660", 2024-09-14T23:57:53.595 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6811/3241153660", 2024-09-14T23:57:53.595 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6815/3241153660", 2024-09-14T23:57:53.595 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6813/3241153660", 2024-09-14T23:57:53.595 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-14T23:57:53.595 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-14T23:57:53.595 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-14T23:57:53.595 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:53.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.596 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 3, 2024-09-14T23:57:53.596 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "45bb3b84-b1c4-4f87-a7d1-0eb77b81741b", 2024-09-14T23:57:53.596 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-14T23:57:53.596 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-14T23:57:53.596 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-14T23:57:53.596 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-14T23:57:53.596 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-14T23:57:53.596 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-14T23:57:53.596 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-14T23:57:53.596 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 93, 2024-09-14T23:57:53.596 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-14T23:57:53.596 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-14T23:57:53.597 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-14T23:57:53.597 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:53.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.597 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:53.597 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6824", 2024-09-14T23:57:53.597 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1736416326 2024-09-14T23:57:53.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.597 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:53.597 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6825", 2024-09-14T23:57:53.597 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1736416326 2024-09-14T23:57:53.597 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:53.597 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:53.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.598 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-14T23:57:53.598 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:53.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.598 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:53.598 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6826", 2024-09-14T23:57:53.598 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1736416326 2024-09-14T23:57:53.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.598 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:53.598 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6827", 2024-09-14T23:57:53.598 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1736416326 2024-09-14T23:57:53.598 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:53.599 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:53.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.599 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-14T23:57:53.599 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:53.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.599 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:53.599 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6830", 2024-09-14T23:57:53.599 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1736416326 2024-09-14T23:57:53.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.599 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:53.599 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6831", 2024-09-14T23:57:53.599 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1736416326 2024-09-14T23:57:53.600 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:53.600 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:53.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.600 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-14T23:57:53.600 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:53.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.600 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:53.600 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6828", 2024-09-14T23:57:53.600 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1736416326 2024-09-14T23:57:53.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.601 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:53.601 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6829", 2024-09-14T23:57:53.601 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1736416326 2024-09-14T23:57:53.601 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:53.601 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:53.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.601 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6825/1736416326", 2024-09-14T23:57:53.601 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6827/1736416326", 2024-09-14T23:57:53.601 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6831/1736416326", 2024-09-14T23:57:53.601 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6829/1736416326", 2024-09-14T23:57:53.601 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-14T23:57:53.601 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-14T23:57:53.601 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-14T23:57:53.602 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:53.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.602 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 4, 2024-09-14T23:57:53.602 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "a812e38f-c0f9-4d33-9373-1910fb09103c", 2024-09-14T23:57:53.602 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-14T23:57:53.602 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-14T23:57:53.602 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-14T23:57:53.602 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 0.9703369140625, 2024-09-14T23:57:53.602 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-14T23:57:53.602 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-14T23:57:53.602 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-14T23:57:53.602 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 49, 2024-09-14T23:57:53.603 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-14T23:57:53.603 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-14T23:57:53.603 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-14T23:57:53.603 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:53.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.603 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:53.603 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6808", 2024-09-14T23:57:53.603 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1755154427 2024-09-14T23:57:53.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.603 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:53.603 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6809", 2024-09-14T23:57:53.603 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1755154427 2024-09-14T23:57:53.604 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:53.604 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:53.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.604 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-14T23:57:53.604 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:53.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.604 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:53.604 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6810", 2024-09-14T23:57:53.604 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1755154427 2024-09-14T23:57:53.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.604 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:53.604 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6811", 2024-09-14T23:57:53.605 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1755154427 2024-09-14T23:57:53.605 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:53.605 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:53.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.605 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-14T23:57:53.605 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:53.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.605 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:53.605 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6814", 2024-09-14T23:57:53.605 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1755154427 2024-09-14T23:57:53.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.605 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:53.605 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6815", 2024-09-14T23:57:53.606 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1755154427 2024-09-14T23:57:53.606 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:53.606 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:53.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.606 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-14T23:57:53.606 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:53.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.606 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:53.606 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6812", 2024-09-14T23:57:53.607 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1755154427 2024-09-14T23:57:53.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.607 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:53.607 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6813", 2024-09-14T23:57:53.607 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1755154427 2024-09-14T23:57:53.607 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:53.607 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:53.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.607 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.43:6809/1755154427", 2024-09-14T23:57:53.607 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.43:6811/1755154427", 2024-09-14T23:57:53.607 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.43:6815/1755154427", 2024-09-14T23:57:53.607 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.43:6813/1755154427", 2024-09-14T23:57:53.608 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-14T23:57:53.608 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-14T23:57:53.608 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-14T23:57:53.608 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:53.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.608 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 5, 2024-09-14T23:57:53.608 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "0de87e5b-7a21-4fb4-9543-d96c065786f8", 2024-09-14T23:57:53.608 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-14T23:57:53.608 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-14T23:57:53.608 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-14T23:57:53.608 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 0, 2024-09-14T23:57:53.608 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-14T23:57:53.608 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-14T23:57:53.609 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-14T23:57:53.609 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 23, 2024-09-14T23:57:53.609 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-14T23:57:53.609 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-14T23:57:53.609 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-14T23:57:53.609 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:53.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.609 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:53.609 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6816", 2024-09-14T23:57:53.609 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3033769028 2024-09-14T23:57:53.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.609 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:53.609 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6817", 2024-09-14T23:57:53.610 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3033769028 2024-09-14T23:57:53.610 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:53.610 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:53.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.610 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-14T23:57:53.610 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:53.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.610 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:53.610 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6818", 2024-09-14T23:57:53.610 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3033769028 2024-09-14T23:57:53.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.612 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:53.612 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6819", 2024-09-14T23:57:53.612 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3033769028 2024-09-14T23:57:53.612 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:53.612 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:53.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.612 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-14T23:57:53.612 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:53.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.613 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:53.613 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6822", 2024-09-14T23:57:53.613 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3033769028 2024-09-14T23:57:53.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.613 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:53.613 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6823", 2024-09-14T23:57:53.613 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3033769028 2024-09-14T23:57:53.613 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:53.613 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:53.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.613 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-14T23:57:53.614 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:53.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.614 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:53.614 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6820", 2024-09-14T23:57:53.614 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3033769028 2024-09-14T23:57:53.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.614 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:53.614 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6821", 2024-09-14T23:57:53.614 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3033769028 2024-09-14T23:57:53.614 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:53.614 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:53.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.615 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.43:6817/3033769028", 2024-09-14T23:57:53.615 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.43:6819/3033769028", 2024-09-14T23:57:53.615 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.43:6823/3033769028", 2024-09-14T23:57:53.615 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.43:6821/3033769028", 2024-09-14T23:57:53.615 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-14T23:57:53.615 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-14T23:57:53.615 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-14T23:57:53.615 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:53.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.615 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 6, 2024-09-14T23:57:53.615 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "c58c1d59-ad02-40f0-98e3-9078ef7e622b", 2024-09-14T23:57:53.615 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-14T23:57:53.615 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-14T23:57:53.616 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-14T23:57:53.616 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-14T23:57:53.616 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-14T23:57:53.616 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-14T23:57:53.616 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-14T23:57:53.616 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 76, 2024-09-14T23:57:53.616 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-14T23:57:53.616 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-14T23:57:53.616 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-14T23:57:53.616 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:53.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.616 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:53.616 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6800", 2024-09-14T23:57:53.617 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3314282416 2024-09-14T23:57:53.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.617 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:53.617 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6801", 2024-09-14T23:57:53.617 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3314282416 2024-09-14T23:57:53.617 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:53.617 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:53.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.617 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-14T23:57:53.617 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:53.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.617 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:53.617 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6802", 2024-09-14T23:57:53.618 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3314282416 2024-09-14T23:57:53.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.618 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:53.618 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6803", 2024-09-14T23:57:53.618 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3314282416 2024-09-14T23:57:53.618 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:53.618 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:53.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.618 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-14T23:57:53.618 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:53.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.618 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:53.618 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6806", 2024-09-14T23:57:53.619 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3314282416 2024-09-14T23:57:53.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.619 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:53.619 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6807", 2024-09-14T23:57:53.619 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3314282416 2024-09-14T23:57:53.619 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:53.619 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:53.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.619 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-14T23:57:53.619 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:53.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.619 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:53.620 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6804", 2024-09-14T23:57:53.620 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3314282416 2024-09-14T23:57:53.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.620 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:53.620 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6805", 2024-09-14T23:57:53.620 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3314282416 2024-09-14T23:57:53.620 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:53.620 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:53.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.620 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.43:6801/3314282416", 2024-09-14T23:57:53.620 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.43:6803/3314282416", 2024-09-14T23:57:53.620 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.43:6807/3314282416", 2024-09-14T23:57:53.621 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.43:6805/3314282416", 2024-09-14T23:57:53.621 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-14T23:57:53.621 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-14T23:57:53.621 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-14T23:57:53.621 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:53.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.621 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 7, 2024-09-14T23:57:53.621 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "389bdb04-20e3-4a7a-8504-687e034bd7dc", 2024-09-14T23:57:53.621 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-14T23:57:53.621 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-14T23:57:53.621 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-14T23:57:53.621 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-14T23:57:53.621 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-14T23:57:53.622 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-14T23:57:53.622 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-14T23:57:53.622 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 75, 2024-09-14T23:57:53.622 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-14T23:57:53.622 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-14T23:57:53.622 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-14T23:57:53.622 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:53.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.622 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:53.622 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6824", 2024-09-14T23:57:53.622 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1481350118 2024-09-14T23:57:53.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.623 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:53.623 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6825", 2024-09-14T23:57:53.623 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1481350118 2024-09-14T23:57:53.623 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:53.623 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:53.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.623 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-14T23:57:53.623 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:53.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.623 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:53.623 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6826", 2024-09-14T23:57:53.623 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1481350118 2024-09-14T23:57:53.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.624 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:53.624 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6827", 2024-09-14T23:57:53.624 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1481350118 2024-09-14T23:57:53.624 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:53.624 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:53.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.624 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-14T23:57:53.624 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:53.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.624 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:53.624 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6830", 2024-09-14T23:57:53.624 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1481350118 2024-09-14T23:57:53.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.625 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:53.625 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6831", 2024-09-14T23:57:53.625 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1481350118 2024-09-14T23:57:53.625 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:53.625 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:53.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.625 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-14T23:57:53.625 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:57:53.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.625 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:57:53.625 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6828", 2024-09-14T23:57:53.625 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1481350118 2024-09-14T23:57:53.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.626 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:57:53.626 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6829", 2024-09-14T23:57:53.626 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1481350118 2024-09-14T23:57:53.626 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:53.626 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:53.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.626 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.43:6825/1481350118", 2024-09-14T23:57:53.626 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.43:6827/1481350118", 2024-09-14T23:57:53.626 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.43:6831/1481350118", 2024-09-14T23:57:53.626 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.43:6829/1481350118", 2024-09-14T23:57:53.626 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-14T23:57:53.627 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-14T23:57:53.627 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-14T23:57:53.627 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:53.627 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:53.627 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-14T23:57:53.627 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_xinfo": [ 2024-09-14T23:57:53.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.627 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 0, 2024-09-14T23:57:53.627 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-14T23:57:53.627 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-14T23:57:53.627 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-14T23:57:53.627 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-14T23:57:53.627 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-14T23:57:53.627 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-14T23:56:11.077810+0000", 2024-09-14T23:57:53.628 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-14T23:57:53.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.628 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 1, 2024-09-14T23:57:53.628 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-14T23:57:53.628 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-14T23:57:53.628 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-14T23:57:53.628 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-14T23:57:53.628 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-14T23:57:53.628 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-14T23:56:11.578838+0000", 2024-09-14T23:57:53.628 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-14T23:57:53.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.628 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 2, 2024-09-14T23:57:53.629 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-14T23:57:53.629 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-14T23:57:53.629 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-14T23:57:53.629 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-14T23:57:53.629 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-14T23:57:53.629 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-14T23:56:10.895490+0000", 2024-09-14T23:57:53.629 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-14T23:57:53.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.629 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 3, 2024-09-14T23:57:53.629 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-14T23:57:53.629 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-14T23:57:53.629 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-14T23:57:53.630 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-14T23:57:53.630 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-14T23:57:53.630 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-14T23:56:12.290821+0000", 2024-09-14T23:57:53.630 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-14T23:57:53.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.630 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 4, 2024-09-14T23:57:53.630 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-14T23:57:53.630 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-14T23:57:53.630 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-14T23:57:53.630 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-14T23:57:53.630 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-14T23:57:53.630 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-14T23:56:11.800581+0000", 2024-09-14T23:57:53.630 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-14T23:57:53.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.631 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 5, 2024-09-14T23:57:53.631 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-14T23:57:53.631 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-14T23:57:53.631 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-14T23:57:53.631 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-14T23:57:53.631 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-14T23:57:53.631 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-14T23:56:11.883155+0000", 2024-09-14T23:57:53.631 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-14T23:57:53.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.631 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 6, 2024-09-14T23:57:53.631 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-14T23:57:53.632 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-14T23:57:53.632 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-14T23:57:53.632 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-14T23:57:53.632 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-14T23:57:53.632 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-14T23:56:11.604790+0000", 2024-09-14T23:57:53.632 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-14T23:57:53.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:53.632 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 7, 2024-09-14T23:57:53.632 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-14T23:57:53.632 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-14T23:57:53.632 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-14T23:57:53.632 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-14T23:57:53.632 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-14T23:57:53.633 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-14T23:56:12.067759+0000", 2024-09-14T23:57:53.633 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-14T23:57:53.633 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:53.633 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-14T23:57:53.633 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_upmap": [], 2024-09-14T23:57:53.633 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_upmap_items": [], 2024-09-14T23:57:53.633 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_upmap_primaries": [], 2024-09-14T23:57:53.633 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_temp": [], 2024-09-14T23:57:53.633 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_temp": [], 2024-09-14T23:57:53.633 INFO:teuthology.orchestra.run.smithi042.stdout: "blocklist": {}, 2024-09-14T23:57:53.633 INFO:teuthology.orchestra.run.smithi042.stdout: "range_blocklist": {}, 2024-09-14T23:57:53.633 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profiles": { 2024-09-14T23:57:53.633 INFO:teuthology.orchestra.run.smithi042.stdout: "default": { 2024-09-14T23:57:53.634 INFO:teuthology.orchestra.run.smithi042.stdout: "crush-failure-domain": "osd", 2024-09-14T23:57:53.634 INFO:teuthology.orchestra.run.smithi042.stdout: "k": "2", 2024-09-14T23:57:53.634 INFO:teuthology.orchestra.run.smithi042.stdout: "m": "1", 2024-09-14T23:57:53.634 INFO:teuthology.orchestra.run.smithi042.stdout: "plugin": "jerasure", 2024-09-14T23:57:53.634 INFO:teuthology.orchestra.run.smithi042.stdout: "technique": "reed_sol_van" 2024-09-14T23:57:53.634 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:53.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:53.634 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps_queue": [], 2024-09-14T23:57:53.634 INFO:teuthology.orchestra.run.smithi042.stdout: "new_removed_snaps": [], 2024-09-14T23:57:53.634 INFO:teuthology.orchestra.run.smithi042.stdout: "new_purged_snaps": [], 2024-09-14T23:57:53.634 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_node_flags": {}, 2024-09-14T23:57:53.634 INFO:teuthology.orchestra.run.smithi042.stdout: "device_class_flags": {}, 2024-09-14T23:57:53.635 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode": { 2024-09-14T23:57:53.635 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode_enabled": false, 2024-09-14T23:57:53.635 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_bucket_count": 0, 2024-09-14T23:57:53.635 INFO:teuthology.orchestra.run.smithi042.stdout: "degraded_stretch_mode": 0, 2024-09-14T23:57:53.635 INFO:teuthology.orchestra.run.smithi042.stdout: "recovering_stretch_mode": 0, 2024-09-14T23:57:53.635 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode_bucket": 0 2024-09-14T23:57:53.635 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:53.635 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:53.637 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:53.637 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:57:53.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:57:53.637 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:57:53.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.637 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4284.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-14T23:57:53.638 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:57:53.638 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 91.101143663000002, 2024-09-14T23:57:53.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.004942086, 2024-09-14T23:57:53.638 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:53.638 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:53.638 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:53.638 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:53.638 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4284", 2024-09-14T23:57:53.638 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/4278347926", 2024-09-14T23:57:53.638 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1 2024-09-14T23:57:53.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.638 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:53.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:53.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:57:53.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:53.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:57:53.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:53.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524331+0000", 2024-09-14T23:57:53.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.383e-05 2024-09-14T23:57:53.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:53.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524339+0000", 2024-09-14T23:57:53.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8290000000000008e-06 2024-09-14T23:57:53.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:53.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524359+0000", 2024-09-14T23:57:53.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9998000000000001e-05 2024-09-14T23:57:53.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:53.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524373+0000", 2024-09-14T23:57:53.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4304000000000001e-05 2024-09-14T23:57:53.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:53.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524488+0000", 2024-09-14T23:57:53.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114663 2024-09-14T23:57:53.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:57:53.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524501+0000", 2024-09-14T23:57:53.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3349999999999999e-05 2024-09-14T23:57:53.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:53.642 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.526868+0000", 2024-09-14T23:57:53.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023670090000000002 2024-09-14T23:57:53.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:53.642 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.527041+0000", 2024-09-14T23:57:53.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000172597 2024-09-14T23:57:53.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-14T23:57:53.642 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.527384+0000", 2024-09-14T23:57:53.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00034309500000000002 2024-09-14T23:57:53.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:53.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529103+0000", 2024-09-14T23:57:53.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017185220000000001 2024-09-14T23:57:53.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:57:53.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529188+0000", 2024-09-14T23:57:53.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5591000000000001e-05 2024-09-14T23:57:53.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:53.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529227+0000", 2024-09-14T23:57:53.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8563000000000003e-05 2024-09-14T23:57:53.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:53.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529259+0000", 2024-09-14T23:57:53.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2734999999999997e-05 2024-09-14T23:57:53.644 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.644 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:53.644 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.645 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4284.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-14T23:57:53.645 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:23.530683+0000", 2024-09-14T23:57:53.645 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 90.094777997999998, 2024-09-14T23:57:53.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0036133110000000001, 2024-09-14T23:57:53.645 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:53.645 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:53.645 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:53.645 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:53.645 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4284", 2024-09-14T23:57:53.645 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/4278347926", 2024-09-14T23:57:53.645 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5 2024-09-14T23:57:53.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.645 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:53.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:53.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530683+0000", 2024-09-14T23:57:53.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:53.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530683+0000", 2024-09-14T23:57:53.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:53.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530685+0000", 2024-09-14T23:57:53.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.075e-06 2024-09-14T23:57:53.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:53.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530695+0000", 2024-09-14T23:57:53.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0261999999999999e-05 2024-09-14T23:57:53.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:53.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530715+0000", 2024-09-14T23:57:53.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9930000000000001e-05 2024-09-14T23:57:53.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:53.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530724+0000", 2024-09-14T23:57:53.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.901e-06 2024-09-14T23:57:53.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:53.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530800+0000", 2024-09-14T23:57:53.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5873999999999999e-05 2024-09-14T23:57:53.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-14T23:57:53.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530807+0000", 2024-09-14T23:57:53.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2389999999999996e-06 2024-09-14T23:57:53.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:53.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.532494+0000", 2024-09-14T23:57:53.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016871830000000001 2024-09-14T23:57:53.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:53.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.532599+0000", 2024-09-14T23:57:53.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010472 2024-09-14T23:57:53.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1", 2024-09-14T23:57:53.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.532715+0000", 2024-09-14T23:57:53.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011568199999999999 2024-09-14T23:57:53.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:57:53.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.533460+0000", 2024-09-14T23:57:53.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00074494999999999997 2024-09-14T23:57:53.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:53.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534230+0000", 2024-09-14T23:57:53.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00077036599999999998 2024-09-14T23:57:53.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:53.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534266+0000", 2024-09-14T23:57:53.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5725000000000001e-05 2024-09-14T23:57:53.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.652 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:53.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534296+0000", 2024-09-14T23:57:53.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0403999999999999e-05 2024-09-14T23:57:53.652 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.652 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:53.652 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.652 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4284.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-14T23:57:53.652 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:24.535160+0000", 2024-09-14T23:57:53.652 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 89.090301357000001, 2024-09-14T23:57:53.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033473650000000001, 2024-09-14T23:57:53.653 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:53.653 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:53.653 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:53.653 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:53.653 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4284", 2024-09-14T23:57:53.653 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/4278347926", 2024-09-14T23:57:53.653 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 7 2024-09-14T23:57:53.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.653 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:53.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:53.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535160+0000", 2024-09-14T23:57:53.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:53.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535160+0000", 2024-09-14T23:57:53.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:53.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535162+0000", 2024-09-14T23:57:53.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3369999999999998e-06 2024-09-14T23:57:53.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:53.655 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535171+0000", 2024-09-14T23:57:53.655 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7700000000000007e-06 2024-09-14T23:57:53.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.655 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:53.655 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535189+0000", 2024-09-14T23:57:53.655 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7938e-05 2024-09-14T23:57:53.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.655 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:53.655 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535197+0000", 2024-09-14T23:57:53.655 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5639999999999997e-06 2024-09-14T23:57:53.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:53.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535266+0000", 2024-09-14T23:57:53.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8926999999999999e-05 2024-09-14T23:57:53.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-14T23:57:53.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535271+0000", 2024-09-14T23:57:53.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0329999999999999e-06 2024-09-14T23:57:53.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:53.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.537064+0000", 2024-09-14T23:57:53.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001792929 2024-09-14T23:57:53.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:53.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.537131+0000", 2024-09-14T23:57:53.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6605000000000004e-05 2024-09-14T23:57:53.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1", 2024-09-14T23:57:53.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.537271+0000", 2024-09-14T23:57:53.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013992400000000001 2024-09-14T23:57:53.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:57:53.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.538093+0000", 2024-09-14T23:57:53.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00082210200000000001 2024-09-14T23:57:53.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:53.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.538439+0000", 2024-09-14T23:57:53.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00034595999999999997 2024-09-14T23:57:53.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:53.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.538476+0000", 2024-09-14T23:57:53.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6770000000000002e-05 2024-09-14T23:57:53.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.659 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:53.659 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.538507+0000", 2024-09-14T23:57:53.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1506e-05 2024-09-14T23:57:53.659 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.659 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:53.659 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.659 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T23:57:53.659 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:34.406877+0000", 2024-09-14T23:57:53.659 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 79.218584351999993, 2024-09-14T23:57:53.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030351169999999999, 2024-09-14T23:57:53.660 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:53.660 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:53.660 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:53.660 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:53.660 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:53.660 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:53.660 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 10 2024-09-14T23:57:53.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.660 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:53.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:53.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406877+0000", 2024-09-14T23:57:53.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:53.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406877+0000", 2024-09-14T23:57:53.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:53.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406878+0000", 2024-09-14T23:57:53.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.291e-06 2024-09-14T23:57:53.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:53.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406883+0000", 2024-09-14T23:57:53.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2399999999999998e-06 2024-09-14T23:57:53.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:53.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406891+0000", 2024-09-14T23:57:53.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5889999999999997e-06 2024-09-14T23:57:53.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:53.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406898+0000", 2024-09-14T23:57:53.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3599999999999998e-06 2024-09-14T23:57:53.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.663 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:53.663 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406948+0000", 2024-09-14T23:57:53.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9435999999999999e-05 2024-09-14T23:57:53.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.663 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-14T23:57:53.663 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406953+0000", 2024-09-14T23:57:53.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2809999999999999e-06 2024-09-14T23:57:53.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.663 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:53.664 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.408559+0000", 2024-09-14T23:57:53.664 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00160607 2024-09-14T23:57:53.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.664 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.664 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:53.664 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.408637+0000", 2024-09-14T23:57:53.664 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8504000000000006e-05 2024-09-14T23:57:53.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.664 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.664 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 4", 2024-09-14T23:57:53.664 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.408781+0000", 2024-09-14T23:57:53.664 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014374799999999999 2024-09-14T23:57:53.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.664 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.665 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:57:53.665 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.409659+0000", 2024-09-14T23:57:53.665 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000877781 2024-09-14T23:57:53.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.665 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.665 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:53.665 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.409822+0000", 2024-09-14T23:57:53.665 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016295400000000001 2024-09-14T23:57:53.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.665 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.665 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:53.665 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.409872+0000", 2024-09-14T23:57:53.665 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0022000000000003e-05 2024-09-14T23:57:53.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:53.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.409912+0000", 2024-09-14T23:57:53.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9841e-05 2024-09-14T23:57:53.666 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.666 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:53.666 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.666 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:50626 3.d e51/46 3:b5154cfe:::rbd_data.11183b9aac34.0000000000000004:head v 51'63, mlcod=45'61)", 2024-09-14T23:57:53.666 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:01.140129+0000", 2024-09-14T23:57:53.666 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 52.485331752, 2024-09-14T23:57:53.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0029883850000000001, 2024-09-14T23:57:53.666 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:53.667 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:53.667 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:53.667 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:53.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:53.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.140129+0000", 2024-09-14T23:57:53.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:53.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.140129+0000", 2024-09-14T23:57:53.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.668 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.668 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:53.668 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.140130+0000", 2024-09-14T23:57:53.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0529999999999999e-06 2024-09-14T23:57:53.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.668 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.668 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:53.668 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.140311+0000", 2024-09-14T23:57:53.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018054200000000001 2024-09-14T23:57:53.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.668 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.668 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:53.668 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.140325+0000", 2024-09-14T23:57:53.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.382e-05 2024-09-14T23:57:53.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.669 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:53.669 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.140332+0000", 2024-09-14T23:57:53.669 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5700000000000004e-06 2024-09-14T23:57:53.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.669 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:53.669 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.140392+0000", 2024-09-14T23:57:53.669 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9296000000000003e-05 2024-09-14T23:57:53.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.669 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:53.669 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.140436+0000", 2024-09-14T23:57:53.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4515999999999999e-05 2024-09-14T23:57:53.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:53.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.143075+0000", 2024-09-14T23:57:53.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0026385369999999998 2024-09-14T23:57:53.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:53.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:01.143118+0000", 2024-09-14T23:57:53.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3050999999999998e-05 2024-09-14T23:57:53.670 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.670 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:53.671 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.671 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.671 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56383 3.c e56/49 3:36b698b0:::rbd_data.1118c4b3aebe.0000000000000000:head v 56'194, mlcod=56'192)", 2024-09-14T23:57:53.671 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.270483+0000", 2024-09-14T23:57:53.671 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.354978197999998, 2024-09-14T23:57:53.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0079685880000000004, 2024-09-14T23:57:53.671 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:53.671 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:53.671 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:53.671 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:53.671 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.671 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:53.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.270483+0000", 2024-09-14T23:57:53.672 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.672 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.672 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:53.672 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.270483+0000", 2024-09-14T23:57:53.672 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.672 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.672 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:53.672 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.270484+0000", 2024-09-14T23:57:53.672 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7400000000000002e-07 2024-09-14T23:57:53.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.672 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.673 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:53.673 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273680+0000", 2024-09-14T23:57:53.673 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031966680000000002 2024-09-14T23:57:53.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.673 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.673 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:53.673 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273697+0000", 2024-09-14T23:57:53.673 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6595e-05 2024-09-14T23:57:53.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.673 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.673 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:53.673 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273705+0000", 2024-09-14T23:57:53.673 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7530000000000001e-06 2024-09-14T23:57:53.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.674 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:53.674 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273763+0000", 2024-09-14T23:57:53.674 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8238e-05 2024-09-14T23:57:53.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.674 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:53.674 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273781+0000", 2024-09-14T23:57:53.674 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8281e-05 2024-09-14T23:57:53.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.674 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:53.675 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.278408+0000", 2024-09-14T23:57:53.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00462716 2024-09-14T23:57:53.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.675 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:53.675 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.278451+0000", 2024-09-14T23:57:53.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3019000000000003e-05 2024-09-14T23:57:53.675 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.675 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:53.675 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.675 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56384 3.c e56/49 3:36b698b0:::rbd_data.1118c4b3aebe.0000000000000000:head v 56'196, mlcod=56'192)", 2024-09-14T23:57:53.675 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:57:53.676 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.350945525999997, 2024-09-14T23:57:53.676 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0046707529999999997, 2024-09-14T23:57:53.676 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:53.676 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:53.676 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:53.676 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:53.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.676 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:53.676 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:57:53.676 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.676 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:53.676 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:57:53.677 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.677 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.677 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.677 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:53.677 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:57:53.677 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1699999999999997e-07 2024-09-14T23:57:53.677 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.677 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.677 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:53.677 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274523+0000", 2024-09-14T23:57:53.677 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9129999999999997e-06 2024-09-14T23:57:53.677 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.677 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.678 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:53.678 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274535+0000", 2024-09-14T23:57:53.678 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1788e-05 2024-09-14T23:57:53.678 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.678 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.678 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:53.678 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274542+0000", 2024-09-14T23:57:53.678 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0539999999999997e-06 2024-09-14T23:57:53.678 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.679 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:53.679 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275889+0000", 2024-09-14T23:57:53.679 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00134669 2024-09-14T23:57:53.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.679 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:53.679 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275917+0000", 2024-09-14T23:57:53.679 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7664000000000001e-05 2024-09-14T23:57:53.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.679 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:53.679 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279134+0000", 2024-09-14T23:57:53.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0032178459999999999 2024-09-14T23:57:53.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.680 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:53.680 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279186+0000", 2024-09-14T23:57:53.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1881e-05 2024-09-14T23:57:53.680 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.680 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:53.680 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.680 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56381 3.1d e56/49 3:bcf9a8cf:::rbd_data.1118c4b3aebe.0000000000000006:head v 56'210, mlcod=56'208)", 2024-09-14T23:57:53.680 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.275462+0000", 2024-09-14T23:57:53.681 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.349999195999999, 2024-09-14T23:57:53.681 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0071310710000000001, 2024-09-14T23:57:53.681 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:53.681 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:53.681 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:53.681 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:53.681 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.681 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:53.681 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275462+0000", 2024-09-14T23:57:53.681 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.681 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.681 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.681 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:53.682 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275462+0000", 2024-09-14T23:57:53.682 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.682 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.682 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.682 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:53.682 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275463+0000", 2024-09-14T23:57:53.682 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2699999999999999e-07 2024-09-14T23:57:53.682 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.682 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.682 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:53.682 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276779+0000", 2024-09-14T23:57:53.682 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0013163649999999999 2024-09-14T23:57:53.682 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.683 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.683 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:53.683 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276795+0000", 2024-09-14T23:57:53.683 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.59e-05 2024-09-14T23:57:53.683 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.683 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.683 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:53.683 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276806+0000", 2024-09-14T23:57:53.683 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1106e-05 2024-09-14T23:57:53.683 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.683 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.683 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:53.683 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276941+0000", 2024-09-14T23:57:53.684 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013477500000000001 2024-09-14T23:57:53.684 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.684 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.684 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:53.684 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276963+0000", 2024-09-14T23:57:53.684 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2419999999999999e-05 2024-09-14T23:57:53.684 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.684 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.684 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:53.684 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282546+0000", 2024-09-14T23:57:53.684 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0055827100000000003 2024-09-14T23:57:53.684 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.685 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.685 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:53.685 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282593+0000", 2024-09-14T23:57:53.685 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7067999999999997e-05 2024-09-14T23:57:53.685 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.685 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:53.685 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.685 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.685 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.685 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56387 3.c e56/49 3:30ee4ba0:::rbd_data.1118c4b3aebe.0000000000000004:head v 56'197, mlcod=56'192)", 2024-09-14T23:57:53.685 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.276498+0000", 2024-09-14T23:57:53.685 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.348963101000002, 2024-09-14T23:57:53.685 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0092192639999999996, 2024-09-14T23:57:53.685 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:53.686 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:53.686 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:53.686 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:53.686 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.686 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:53.686 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276498+0000", 2024-09-14T23:57:53.686 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.686 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.686 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.686 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:53.686 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276498+0000", 2024-09-14T23:57:53.686 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.686 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.687 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.687 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:53.687 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276499+0000", 2024-09-14T23:57:53.687 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1100000000000004e-07 2024-09-14T23:57:53.687 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.687 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.687 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:53.687 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279646+0000", 2024-09-14T23:57:53.687 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031473220000000001 2024-09-14T23:57:53.687 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.687 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.687 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:53.687 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279672+0000", 2024-09-14T23:57:53.688 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6129999999999999e-05 2024-09-14T23:57:53.688 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.688 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.688 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:53.688 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279682+0000", 2024-09-14T23:57:53.688 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7170000000000003e-06 2024-09-14T23:57:53.688 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.688 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.688 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:53.688 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279742+0000", 2024-09-14T23:57:53.688 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9926999999999997e-05 2024-09-14T23:57:53.688 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.688 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.689 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:53.689 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279761+0000", 2024-09-14T23:57:53.689 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9402000000000002e-05 2024-09-14T23:57:53.689 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.689 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.689 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:53.689 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.285680+0000", 2024-09-14T23:57:53.689 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.005918881 2024-09-14T23:57:53.689 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.689 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.689 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:53.689 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.285717+0000", 2024-09-14T23:57:53.689 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6974000000000002e-05 2024-09-14T23:57:53.689 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.690 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:53.690 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.690 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.690 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.690 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56388 3.c e56/49 3:30ee4ba0:::rbd_data.1118c4b3aebe.0000000000000004:head v 56'199, mlcod=56'192)", 2024-09-14T23:57:53.690 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:57:53.690 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.345471513, 2024-09-14T23:57:53.690 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0060596740000000001, 2024-09-14T23:57:53.690 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:53.690 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:53.690 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:53.690 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:53.691 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.691 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:53.691 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:57:53.691 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.691 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.691 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.691 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:53.691 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:57:53.691 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.691 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.691 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.691 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:53.691 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:57:53.692 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2200000000000003e-07 2024-09-14T23:57:53.692 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.692 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.692 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:53.692 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279995+0000", 2024-09-14T23:57:53.692 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5569999999999997e-06 2024-09-14T23:57:53.692 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.692 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.692 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:53.692 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280004+0000", 2024-09-14T23:57:53.692 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0080000000000008e-06 2024-09-14T23:57:53.692 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.692 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.692 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:53.693 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280010+0000", 2024-09-14T23:57:53.693 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5749999999999997e-06 2024-09-14T23:57:53.693 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.693 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.693 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:53.693 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282968+0000", 2024-09-14T23:57:53.693 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0029579390000000001 2024-09-14T23:57:53.693 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.693 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.693 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:53.693 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282998+0000", 2024-09-14T23:57:53.693 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9947999999999999e-05 2024-09-14T23:57:53.693 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.694 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.694 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:53.694 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.286012+0000", 2024-09-14T23:57:53.694 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030132900000000001 2024-09-14T23:57:53.694 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.694 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.694 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:53.694 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.286049+0000", 2024-09-14T23:57:53.694 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7534999999999999e-05 2024-09-14T23:57:53.694 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.694 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:53.694 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.694 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.694 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.695 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56382 3.1d e56/49 3:bcf9a8cf:::rbd_data.1118c4b3aebe.0000000000000006:head v 56'212, mlcod=56'208)", 2024-09-14T23:57:53.695 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.280285+0000", 2024-09-14T23:57:53.695 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.345175830000002, 2024-09-14T23:57:53.695 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035393489999999998, 2024-09-14T23:57:53.695 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:53.695 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:53.695 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:53.695 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:53.695 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.695 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:53.695 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280285+0000", 2024-09-14T23:57:53.695 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.696 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.696 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:53.696 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280285+0000", 2024-09-14T23:57:53.696 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.696 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.696 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:53.696 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280286+0000", 2024-09-14T23:57:53.696 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5000000000000001e-07 2024-09-14T23:57:53.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.696 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.696 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:53.696 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280299+0000", 2024-09-14T23:57:53.697 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3312000000000001e-05 2024-09-14T23:57:53.697 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.697 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.697 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:53.697 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280311+0000", 2024-09-14T23:57:53.697 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1677e-05 2024-09-14T23:57:53.697 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.697 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.697 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:53.697 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280318+0000", 2024-09-14T23:57:53.697 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.742e-06 2024-09-14T23:57:53.697 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.697 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.698 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:53.698 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280374+0000", 2024-09-14T23:57:53.698 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.592e-05 2024-09-14T23:57:53.698 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.698 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.698 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:53.698 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280392+0000", 2024-09-14T23:57:53.698 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8490000000000001e-05 2024-09-14T23:57:53.698 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.698 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.698 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:53.698 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.283693+0000", 2024-09-14T23:57:53.699 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033012050000000002 2024-09-14T23:57:53.699 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.699 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.699 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:53.699 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.283825+0000", 2024-09-14T23:57:53.699 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013105300000000001 2024-09-14T23:57:53.699 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.699 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:53.699 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.699 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.699 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.699 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "pg_update_log_missing(3.c epoch 56/49 rep_tid 1493 entries 56'200 (0'0) error 3:370203a3:::rbd_data.1118c4b3aebe.000000000000000d:head by client.4376.0:56396 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 56'192)", 2024-09-14T23:57:53.699 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:57:53.699 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.344951373000001, 2024-09-14T23:57:53.700 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057370099999999999, 2024-09-14T23:57:53.700 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:53.700 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:53.700 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:57:53.700 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:53.700 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.700 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:53.700 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:57:53.700 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.700 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.700 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.701 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:53.701 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:57:53.701 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.701 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.701 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.701 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:53.701 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:57:53.701 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0699999999999997e-07 2024-09-14T23:57:53.701 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.701 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.701 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:53.701 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280519+0000", 2024-09-14T23:57:53.701 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6260000000000001e-06 2024-09-14T23:57:53.702 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.702 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.702 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:53.702 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280541+0000", 2024-09-14T23:57:53.702 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1976000000000001e-05 2024-09-14T23:57:53.702 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.702 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.702 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:53.702 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280556+0000", 2024-09-14T23:57:53.702 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5075000000000001e-05 2024-09-14T23:57:53.702 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.702 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.702 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:53.703 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.285494+0000", 2024-09-14T23:57:53.703 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0049378690000000001 2024-09-14T23:57:53.703 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.703 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.703 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:53.703 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.286247+0000", 2024-09-14T23:57:53.703 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00075285700000000001 2024-09-14T23:57:53.703 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.703 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:53.703 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.703 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.703 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.703 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56400 3.1c e56/49 3:3f5b81ba:::rbd_data.1118c4b3aebe.000000000000000a:head v 56'236, mlcod=56'234)", 2024-09-14T23:57:53.703 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.295064+0000", 2024-09-14T23:57:53.704 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.330396581000002, 2024-09-14T23:57:53.704 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0054331589999999999, 2024-09-14T23:57:53.704 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:53.704 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:53.704 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:53.704 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:53.704 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.704 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:53.704 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.295064+0000", 2024-09-14T23:57:53.704 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.704 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.704 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.705 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:53.705 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.295064+0000", 2024-09-14T23:57:53.705 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.705 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.705 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.705 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:53.705 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.295066+0000", 2024-09-14T23:57:53.705 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.502e-06 2024-09-14T23:57:53.705 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.705 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.705 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:53.706 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296403+0000", 2024-09-14T23:57:53.706 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001336554 2024-09-14T23:57:53.706 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.706 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.706 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:53.706 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296428+0000", 2024-09-14T23:57:53.706 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5179000000000001e-05 2024-09-14T23:57:53.706 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.706 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.706 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:53.706 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296440+0000", 2024-09-14T23:57:53.706 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2387e-05 2024-09-14T23:57:53.706 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.706 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.707 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:53.707 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296554+0000", 2024-09-14T23:57:53.707 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114269 2024-09-14T23:57:53.707 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.707 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.707 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:53.707 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296581+0000", 2024-09-14T23:57:53.707 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6673999999999999e-05 2024-09-14T23:57:53.707 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.707 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.707 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:53.707 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.300418+0000", 2024-09-14T23:57:53.707 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0038371249999999998 2024-09-14T23:57:53.708 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.708 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.708 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:53.708 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.300498+0000", 2024-09-14T23:57:53.708 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9468999999999994e-05 2024-09-14T23:57:53.708 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.708 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:53.708 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.708 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.708 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.708 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56401 3.1c e56/49 3:3f5b81ba:::rbd_data.1118c4b3aebe.000000000000000a:head v 56'238, mlcod=56'234)", 2024-09-14T23:57:53.708 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.297640+0000", 2024-09-14T23:57:53.708 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.327820742, 2024-09-14T23:57:53.709 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031491919999999999, 2024-09-14T23:57:53.709 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:53.709 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:53.709 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:53.709 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:53.709 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.709 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:53.709 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.297640+0000", 2024-09-14T23:57:53.709 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.709 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.709 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.709 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:53.709 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.297640+0000", 2024-09-14T23:57:53.710 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.710 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.710 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.710 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:53.710 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.297641+0000", 2024-09-14T23:57:53.710 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9000000000000005e-07 2024-09-14T23:57:53.710 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.710 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.710 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:53.710 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.297649+0000", 2024-09-14T23:57:53.710 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4510000000000003e-06 2024-09-14T23:57:53.710 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.710 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.711 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:53.711 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.297663+0000", 2024-09-14T23:57:53.711 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4262e-05 2024-09-14T23:57:53.711 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.711 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.711 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:53.711 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.297679+0000", 2024-09-14T23:57:53.711 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5798e-05 2024-09-14T23:57:53.711 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.711 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.711 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:53.711 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.298408+0000", 2024-09-14T23:57:53.711 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00072923699999999998 2024-09-14T23:57:53.711 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.712 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.712 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:53.712 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.298430+0000", 2024-09-14T23:57:53.712 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1753999999999999e-05 2024-09-14T23:57:53.712 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.712 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.712 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:53.712 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.300749+0000", 2024-09-14T23:57:53.712 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023190590000000001 2024-09-14T23:57:53.712 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.712 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.712 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:53.713 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.300789+0000", 2024-09-14T23:57:53.713 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0640999999999999e-05 2024-09-14T23:57:53.713 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.713 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:53.713 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.713 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.713 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:61921 3.5 e57/49 3:a05d004f:::rbd_data.11188d356229.000000000000000b:head v 57'279, mlcod=57'277)", 2024-09-14T23:57:53.713 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:07.560389+0000", 2024-09-14T23:57:53.713 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 46.065071983000003, 2024-09-14T23:57:53.713 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0034956520000000001, 2024-09-14T23:57:53.713 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:53.713 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:53.713 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:53.714 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:53.714 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.714 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:53.714 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.560389+0000", 2024-09-14T23:57:53.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.714 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.714 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:53.714 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.560389+0000", 2024-09-14T23:57:53.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.714 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.714 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:53.714 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.560390+0000", 2024-09-14T23:57:53.715 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4109999999999999e-06 2024-09-14T23:57:53.715 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.715 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.715 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:53.715 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.561839+0000", 2024-09-14T23:57:53.715 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001448406 2024-09-14T23:57:53.715 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.715 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.715 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:53.715 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.561854+0000", 2024-09-14T23:57:53.715 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5426000000000001e-05 2024-09-14T23:57:53.715 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.716 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.716 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:53.716 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.561863+0000", 2024-09-14T23:57:53.716 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7509999999999997e-06 2024-09-14T23:57:53.716 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.716 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.716 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:53.716 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.561917+0000", 2024-09-14T23:57:53.716 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3531e-05 2024-09-14T23:57:53.716 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.716 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.716 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:53.716 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.561944+0000", 2024-09-14T23:57:53.717 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7528000000000001e-05 2024-09-14T23:57:53.717 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.717 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.717 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:53.717 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563843+0000", 2024-09-14T23:57:53.717 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001899361 2024-09-14T23:57:53.717 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.717 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.717 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:53.717 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563885+0000", 2024-09-14T23:57:53.717 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1238000000000001e-05 2024-09-14T23:57:53.717 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.717 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:53.718 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.718 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.718 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.718 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:61922 3.5 e57/49 3:a11ebf8e:::rbd_data.11188d356229.000000000000000c:head v 57'280, mlcod=57'277)", 2024-09-14T23:57:53.718 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:07.563144+0000", 2024-09-14T23:57:53.718 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 46.062316928999998, 2024-09-14T23:57:53.718 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003977848, 2024-09-14T23:57:53.718 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:53.718 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:53.718 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:53.718 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:53.718 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.718 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:53.719 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563144+0000", 2024-09-14T23:57:53.719 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.719 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.719 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:53.719 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563144+0000", 2024-09-14T23:57:53.719 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.719 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.719 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:53.719 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563145+0000", 2024-09-14T23:57:53.719 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5400000000000001e-07 2024-09-14T23:57:53.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.720 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.720 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:53.720 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565056+0000", 2024-09-14T23:57:53.720 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0019108160000000001 2024-09-14T23:57:53.720 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.720 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.720 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:53.720 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565088+0000", 2024-09-14T23:57:53.720 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2586999999999998e-05 2024-09-14T23:57:53.720 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.720 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.720 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:53.720 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565100+0000", 2024-09-14T23:57:53.721 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1917e-05 2024-09-14T23:57:53.721 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.721 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.721 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:53.721 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565166+0000", 2024-09-14T23:57:53.721 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6245e-05 2024-09-14T23:57:53.721 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.721 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.721 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:53.721 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565190+0000", 2024-09-14T23:57:53.721 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3825999999999999e-05 2024-09-14T23:57:53.721 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.721 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.722 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:53.722 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.567080+0000", 2024-09-14T23:57:53.722 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001889541 2024-09-14T23:57:53.722 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.722 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.722 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:53.722 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.567122+0000", 2024-09-14T23:57:53.722 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2262000000000003e-05 2024-09-14T23:57:53.722 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.722 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:53.722 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.722 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.722 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.723 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:66564 3.c e60/49 3:33faa7ca:::rbd_data.111873792d9b.0000000000000003:head v 60'236, mlcod=57'233)", 2024-09-14T23:57:53.723 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:11.271596+0000", 2024-09-14T23:57:53.723 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 42.353865351000003, 2024-09-14T23:57:53.723 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0060396099999999999, 2024-09-14T23:57:53.723 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:53.723 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:53.723 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:53.723 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:53.723 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.723 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:53.723 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.271596+0000", 2024-09-14T23:57:53.723 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.724 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.724 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.724 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:53.724 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.271596+0000", 2024-09-14T23:57:53.724 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.724 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.724 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.724 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:53.724 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.271597+0000", 2024-09-14T23:57:53.724 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2559999999999999e-06 2024-09-14T23:57:53.724 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.724 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.724 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:53.725 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.274854+0000", 2024-09-14T23:57:53.725 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0032574470000000001 2024-09-14T23:57:53.725 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.725 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.725 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:53.725 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.274874+0000", 2024-09-14T23:57:53.725 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9449999999999998e-05 2024-09-14T23:57:53.725 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.725 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.725 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:53.725 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.274883+0000", 2024-09-14T23:57:53.725 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5980000000000003e-06 2024-09-14T23:57:53.725 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.725 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.726 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:53.726 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.274977+0000", 2024-09-14T23:57:53.726 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3270999999999995e-05 2024-09-14T23:57:53.726 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.726 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.726 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:53.726 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.275000+0000", 2024-09-14T23:57:53.726 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3088e-05 2024-09-14T23:57:53.726 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.726 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.726 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:53.726 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.277590+0000", 2024-09-14T23:57:53.727 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0025898660000000001 2024-09-14T23:57:53.727 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.727 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.727 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:53.727 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.277635+0000", 2024-09-14T23:57:53.727 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5633999999999997e-05 2024-09-14T23:57:53.727 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.727 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:53.727 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.727 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.727 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.727 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:130509 3.2 e88/75 3:4432e346:::rbd_data.111847077d5e.0000000000000005:head v 88'468, mlcod=87'466)", 2024-09-14T23:57:53.727 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:40.669710+0000", 2024-09-14T23:57:53.727 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 12.955751504, 2024-09-14T23:57:53.728 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031769279999999999, 2024-09-14T23:57:53.728 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:53.728 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:53.728 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:53.728 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:53.728 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.728 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:53.728 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:40.669710+0000", 2024-09-14T23:57:53.728 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.728 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.728 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.728 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:53.728 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:40.669710+0000", 2024-09-14T23:57:53.729 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.729 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.729 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.729 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:53.729 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:40.669710+0000", 2024-09-14T23:57:53.729 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2699999999999999e-07 2024-09-14T23:57:53.729 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.729 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.729 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:53.729 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:40.671376+0000", 2024-09-14T23:57:53.729 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016661480000000001 2024-09-14T23:57:53.729 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.730 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.730 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:53.730 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:40.671389+0000", 2024-09-14T23:57:53.730 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2311e-05 2024-09-14T23:57:53.730 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.730 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.730 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:53.730 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:40.671395+0000", 2024-09-14T23:57:53.730 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6869999999999997e-06 2024-09-14T23:57:53.730 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.730 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.730 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:53.730 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:40.671451+0000", 2024-09-14T23:57:53.731 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5865000000000001e-05 2024-09-14T23:57:53.731 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.731 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.731 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:53.731 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:40.671475+0000", 2024-09-14T23:57:53.731 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3266999999999999e-05 2024-09-14T23:57:53.731 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.731 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.731 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:53.731 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:40.672858+0000", 2024-09-14T23:57:53.731 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0013835340000000001 2024-09-14T23:57:53.731 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.731 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.732 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:53.732 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:40.672886+0000", 2024-09-14T23:57:53.732 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8388999999999999e-05 2024-09-14T23:57:53.732 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.732 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:53.732 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.732 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.732 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.732 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:132869 3.16 e92/75 3:681b4799:::rbd_data.11186132a8b6.0000000000000001:head v 92'262, mlcod=87'260)", 2024-09-14T23:57:53.732 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:45.195492+0000", 2024-09-14T23:57:53.732 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.4299685980000003, 2024-09-14T23:57:53.732 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0034654880000000001, 2024-09-14T23:57:53.733 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:53.733 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:53.733 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:53.733 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:53.733 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.733 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:53.733 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.195492+0000", 2024-09-14T23:57:53.733 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.733 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.733 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.733 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:53.733 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.195492+0000", 2024-09-14T23:57:53.733 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.734 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.734 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.734 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:53.734 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.195494+0000", 2024-09-14T23:57:53.734 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.905e-06 2024-09-14T23:57:53.734 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.734 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.734 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:53.734 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.196549+0000", 2024-09-14T23:57:53.734 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010545489999999999 2024-09-14T23:57:53.734 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.734 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.734 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:53.734 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.196572+0000", 2024-09-14T23:57:53.735 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3603000000000001e-05 2024-09-14T23:57:53.735 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.735 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.735 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:53.735 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.196584+0000", 2024-09-14T23:57:53.735 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.169e-05 2024-09-14T23:57:53.735 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.735 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.735 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:53.735 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.196676+0000", 2024-09-14T23:57:53.735 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2088000000000003e-05 2024-09-14T23:57:53.735 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.736 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.736 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:53.736 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.196699+0000", 2024-09-14T23:57:53.736 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2365999999999999e-05 2024-09-14T23:57:53.736 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.736 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.736 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:53.736 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.198910+0000", 2024-09-14T23:57:53.736 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0022118609999999999 2024-09-14T23:57:53.736 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.736 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.736 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:53.736 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.198958+0000", 2024-09-14T23:57:53.737 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7426000000000003e-05 2024-09-14T23:57:53.737 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.737 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:53.737 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.737 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.737 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:133384 3.1f e92/49 3:fce5e529:::rbd_data.11186132a8b6.0000000000000002:head v 92'807, mlcod=89'804)", 2024-09-14T23:57:53.737 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:45.293642+0000", 2024-09-14T23:57:53.737 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 8.3318194680000008, 2024-09-14T23:57:53.737 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003760186, 2024-09-14T23:57:53.737 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:53.737 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:53.737 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:53.738 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:53.738 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.738 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:53.738 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.293642+0000", 2024-09-14T23:57:53.738 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.738 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.738 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.738 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:53.738 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.293642+0000", 2024-09-14T23:57:53.738 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:53.738 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.738 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.738 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:53.738 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.293643+0000", 2024-09-14T23:57:53.739 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2500000000000001e-06 2024-09-14T23:57:53.739 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.739 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.739 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:53.739 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.295402+0000", 2024-09-14T23:57:53.739 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017595410000000001 2024-09-14T23:57:53.739 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.739 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.739 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:53.739 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.295422+0000", 2024-09-14T23:57:53.739 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9763999999999999e-05 2024-09-14T23:57:53.739 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.739 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.740 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:53.740 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.295436+0000", 2024-09-14T23:57:53.740 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.364e-05 2024-09-14T23:57:53.740 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.740 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.740 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:53.740 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.295499+0000", 2024-09-14T23:57:53.740 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2770999999999999e-05 2024-09-14T23:57:53.740 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.740 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.740 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:53.740 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.295516+0000", 2024-09-14T23:57:53.740 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7240000000000001e-05 2024-09-14T23:57:53.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.741 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.741 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:53.741 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.297324+0000", 2024-09-14T23:57:53.741 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001807857 2024-09-14T23:57:53.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:53.741 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:53.741 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:53.741 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.297402+0000", 2024-09-14T23:57:53.741 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8122999999999997e-05 2024-09-14T23:57:53.741 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.741 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:53.741 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.741 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:53.742 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:53.742 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:53.742 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:53.634+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:53.743 INFO:tasks.thrashosds.thrasher:j is {'epoch': 99, 'fsid': '21916fcc-f95c-4bfa-8b8b-11ca45845613', 'created': '2024-09-14T23:56:03.167948+0000', 'modified': '2024-09-14T23:57:53.075063+0000', 'last_up_change': '2024-09-14T23:56:14.322882+0000', 'last_in_change': '2024-09-14T23:56:56.035972+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T23:56:13.500197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8712921142578125, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8700000047683716}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T23:56:21.284705+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.630000114440918, 'score_stable': 2.630000114440918, 'optimal_score': 1, 'raw_score_acting': 2.630000114440918, 'raw_score_stable': 2.630000114440918, 'primary_affinity_weighted': 0.8712921142578125, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8700000047683716}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T23:56:30.861504+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 44, 'snap_epoch': 99, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8712921142578125, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8700000047683716}}], 'osds': [{'osd': 0, 'uuid': 'db9d0c67-2d60-4592-9ad3-f99e5294f93f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 49, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6816', 'nonce': 314865173}, {'type': 'v1', 'addr': '172.21.15.42:6817', 'nonce': 314865173}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6818', 'nonce': 314865173}, {'type': 'v1', 'addr': '172.21.15.42:6819', 'nonce': 314865173}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6822', 'nonce': 314865173}, {'type': 'v1', 'addr': '172.21.15.42:6823', 'nonce': 314865173}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6820', 'nonce': 314865173}, {'type': 'v1', 'addr': '172.21.15.42:6821', 'nonce': 314865173}]}, 'public_addr': '172.21.15.42:6817/314865173', 'cluster_addr': '172.21.15.42:6819/314865173', 'heartbeat_back_addr': '172.21.15.42:6823/314865173', 'heartbeat_front_addr': '172.21.15.42:6821/314865173', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '6c1a2765-c591-4c57-8ab3-ee6645970aa5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 92, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6800', 'nonce': 3197912310}, {'type': 'v1', 'addr': '172.21.15.42:6801', 'nonce': 3197912310}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6802', 'nonce': 3197912310}, {'type': 'v1', 'addr': '172.21.15.42:6803', 'nonce': 3197912310}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6806', 'nonce': 3197912310}, {'type': 'v1', 'addr': '172.21.15.42:6807', 'nonce': 3197912310}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6804', 'nonce': 3197912310}, {'type': 'v1', 'addr': '172.21.15.42:6805', 'nonce': 3197912310}]}, 'public_addr': '172.21.15.42:6801/3197912310', 'cluster_addr': '172.21.15.42:6803/3197912310', 'heartbeat_back_addr': '172.21.15.42:6807/3197912310', 'heartbeat_front_addr': '172.21.15.42:6805/3197912310', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '930db893-7be6-4645-a7f1-e043c5ec95eb', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 46, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6808', 'nonce': 3241153660}, {'type': 'v1', 'addr': '172.21.15.42:6809', 'nonce': 3241153660}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6810', 'nonce': 3241153660}, {'type': 'v1', 'addr': '172.21.15.42:6811', 'nonce': 3241153660}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6814', 'nonce': 3241153660}, {'type': 'v1', 'addr': '172.21.15.42:6815', 'nonce': 3241153660}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6812', 'nonce': 3241153660}, {'type': 'v1', 'addr': '172.21.15.42:6813', 'nonce': 3241153660}]}, 'public_addr': '172.21.15.42:6809/3241153660', 'cluster_addr': '172.21.15.42:6811/3241153660', 'heartbeat_back_addr': '172.21.15.42:6815/3241153660', 'heartbeat_front_addr': '172.21.15.42:6813/3241153660', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '45bb3b84-b1c4-4f87-a7d1-0eb77b81741b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 93, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6824', 'nonce': 1736416326}, {'type': 'v1', 'addr': '172.21.15.42:6825', 'nonce': 1736416326}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6826', 'nonce': 1736416326}, {'type': 'v1', 'addr': '172.21.15.42:6827', 'nonce': 1736416326}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6830', 'nonce': 1736416326}, {'type': 'v1', 'addr': '172.21.15.42:6831', 'nonce': 1736416326}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6828', 'nonce': 1736416326}, {'type': 'v1', 'addr': '172.21.15.42:6829', 'nonce': 1736416326}]}, 'public_addr': '172.21.15.42:6825/1736416326', 'cluster_addr': '172.21.15.42:6827/1736416326', 'heartbeat_back_addr': '172.21.15.42:6831/1736416326', 'heartbeat_front_addr': '172.21.15.42:6829/1736416326', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'a812e38f-c0f9-4d33-9373-1910fb09103c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.9703369140625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 49, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6808', 'nonce': 1755154427}, {'type': 'v1', 'addr': '172.21.15.43:6809', 'nonce': 1755154427}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6810', 'nonce': 1755154427}, {'type': 'v1', 'addr': '172.21.15.43:6811', 'nonce': 1755154427}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6814', 'nonce': 1755154427}, {'type': 'v1', 'addr': '172.21.15.43:6815', 'nonce': 1755154427}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6812', 'nonce': 1755154427}, {'type': 'v1', 'addr': '172.21.15.43:6813', 'nonce': 1755154427}]}, 'public_addr': '172.21.15.43:6809/1755154427', 'cluster_addr': '172.21.15.43:6811/1755154427', 'heartbeat_back_addr': '172.21.15.43:6815/1755154427', 'heartbeat_front_addr': '172.21.15.43:6813/1755154427', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '0de87e5b-7a21-4fb4-9543-d96c065786f8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6816', 'nonce': 3033769028}, {'type': 'v1', 'addr': '172.21.15.43:6817', 'nonce': 3033769028}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6818', 'nonce': 3033769028}, {'type': 'v1', 'addr': '172.21.15.43:6819', 'nonce': 3033769028}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6822', 'nonce': 3033769028}, {'type': 'v1', 'addr': '172.21.15.43:6823', 'nonce': 3033769028}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6820', 'nonce': 3033769028}, {'type': 'v1', 'addr': '172.21.15.43:6821', 'nonce': 3033769028}]}, 'public_addr': '172.21.15.43:6817/3033769028', 'cluster_addr': '172.21.15.43:6819/3033769028', 'heartbeat_back_addr': '172.21.15.43:6823/3033769028', 'heartbeat_front_addr': '172.21.15.43:6821/3033769028', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'c58c1d59-ad02-40f0-98e3-9078ef7e622b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 76, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6800', 'nonce': 3314282416}, {'type': 'v1', 'addr': '172.21.15.43:6801', 'nonce': 3314282416}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6802', 'nonce': 3314282416}, {'type': 'v1', 'addr': '172.21.15.43:6803', 'nonce': 3314282416}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6806', 'nonce': 3314282416}, {'type': 'v1', 'addr': '172.21.15.43:6807', 'nonce': 3314282416}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6804', 'nonce': 3314282416}, {'type': 'v1', 'addr': '172.21.15.43:6805', 'nonce': 3314282416}]}, 'public_addr': '172.21.15.43:6801/3314282416', 'cluster_addr': '172.21.15.43:6803/3314282416', 'heartbeat_back_addr': '172.21.15.43:6807/3314282416', 'heartbeat_front_addr': '172.21.15.43:6805/3314282416', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '389bdb04-20e3-4a7a-8504-687e034bd7dc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 75, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6824', 'nonce': 1481350118}, {'type': 'v1', 'addr': '172.21.15.43:6825', 'nonce': 1481350118}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6826', 'nonce': 1481350118}, {'type': 'v1', 'addr': '172.21.15.43:6827', 'nonce': 1481350118}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6830', 'nonce': 1481350118}, {'type': 'v1', 'addr': '172.21.15.43:6831', 'nonce': 1481350118}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6828', 'nonce': 1481350118}, {'type': 'v1', 'addr': '172.21.15.43:6829', 'nonce': 1481350118}]}, 'public_addr': '172.21.15.43:6825/1481350118', 'cluster_addr': '172.21.15.43:6827/1481350118', 'heartbeat_back_addr': '172.21.15.43:6831/1481350118', 'heartbeat_front_addr': '172.21.15.43:6829/1481350118', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T23:56:11.077810+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T23:56:11.578838+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T23:56:10.895490+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T23:56:12.290821+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T23:56:11.800581+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T23:56:11.883155+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T23:56:11.604790+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T23:56:12.067759+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T23:57:53.743 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T23:57:53.745 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T23:57:53.747 INFO:teuthology.orchestra.run.smithi042.stdout:5056 read 0xb0b395 thru 0xb17e3e (0xcaaa bytes) 2024-09-14T23:57:53.747 INFO:teuthology.orchestra.run.smithi042.stdout:5057 write 0x61a580e thru 0x61b2402 (0xcbf5 bytes) 2024-09-14T23:57:53.748 INFO:teuthology.orchestra.run.smithi042.stdout:5059 trunc from 0x61b2403 to 0x422214b 2024-09-14T23:57:53.748 INFO:teuthology.orchestra.run.smithi042.stdout:5060 read 0x14e09b8 thru 0x14ed84b (0xce94 bytes) 2024-09-14T23:57:53.748 INFO:teuthology.orchestra.run.smithi042.stdout:5061 trunc from 0x422214b to 0xdf5e316 2024-09-14T23:57:53.748 INFO:teuthology.orchestra.run.smithi042.stdout:5062 write 0xe3f2348 thru 0xe401248 (0xef01 bytes) 2024-09-14T23:57:53.748 INFO:teuthology.orchestra.run.smithi042.stdout:5063 write 0x6821ad9 thru 0x682e0dc (0xc604 bytes) 2024-09-14T23:57:53.748 INFO:teuthology.orchestra.run.smithi042.stdout:5064 write 0x28abe2f thru 0x28b17b7 (0x5989 bytes) 2024-09-14T23:57:53.748 INFO:teuthology.orchestra.run.smithi042.stdout:truncating to largest ever: 0xed3ab25 2024-09-14T23:57:53.748 INFO:teuthology.orchestra.run.smithi042.stdout:5065 trunc from 0xe401249 to 0xed3ab25 2024-09-14T23:57:53.748 INFO:teuthology.orchestra.run.smithi042.stdout:5067 trunc from 0xed3ab25 to 0x9988b3e 2024-09-14T23:57:53.748 INFO:teuthology.orchestra.run.smithi042.stdout:5070 writesame 0x3c93e1e thru 0x3c9abd0 data_size 0x45(0x6db3 bytes) 2024-09-14T23:57:53.748 INFO:teuthology.orchestra.run.smithi042.stdout:5071 write 0x6944ee2 thru 0x694da6a (0x8b89 bytes) 2024-09-14T23:57:53.748 INFO:teuthology.orchestra.run.smithi042.stdout:5072 read 0x9949f25 thru 0x994f413 (0x54ef bytes) 2024-09-14T23:57:53.748 INFO:teuthology.orchestra.run.smithi042.stdout:5073 writesame 0xb981919 thru 0xb987782 data_size 0x5(0x5e6a bytes) 2024-09-14T23:57:53.749 INFO:teuthology.orchestra.run.smithi042.stdout:truncating to largest ever: 0xed85209 2024-09-14T23:57:53.749 INFO:teuthology.orchestra.run.smithi042.stdout:5075 trunc from 0xb987783 to 0xed85209 2024-09-14T23:57:53.749 INFO:teuthology.orchestra.run.smithi042.stdout:5076 read 0x68a79c1 thru 0x68b4c23 (0xd263 bytes) 2024-09-14T23:57:53.749 INFO:teuthology.orchestra.run.smithi042.stdout:5077 write 0xeb134a6 thru 0xeb161c6 (0x2d21 bytes) 2024-09-14T23:57:53.749 INFO:teuthology.orchestra.run.smithi042.stdout:5078 punch from 0x670f705 to 0x6711730, (0x202b bytes) 2024-09-14T23:57:53.749 INFO:teuthology.orchestra.run.smithi042.stdout:5079 write 0x68f600c thru 0x68fa835 (0x482a bytes) 2024-09-14T23:57:53.749 INFO:teuthology.orchestra.run.smithi042.stdout:5080 read 0x95bc4dd thru 0x95bf034 (0x2b58 bytes) 2024-09-14T23:57:53.749 INFO:teuthology.orchestra.run.smithi042.stdout:5082 write 0xa6b482a thru 0xa6c19ac (0xd183 bytes) 2024-09-14T23:57:53.749 INFO:teuthology.orchestra.run.smithi042.stdout:5083 write 0xa5ca160 thru 0xa5cd821 (0x36c2 bytes) 2024-09-14T23:57:53.749 INFO:teuthology.orchestra.run.smithi042.stdout:5084 write 0x58604ab thru 0x586d9dc (0xd532 bytes) 2024-09-14T23:57:53.749 INFO:teuthology.orchestra.run.smithi042.stdout:5085 clone 43 order 19 su 32768 sc 10 2024-09-14T23:57:53.786 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:53.784+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:53.847 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:53.846+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:53.893 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:53.893 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:53.893 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:57:53.893 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:53.902 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T23:57:53.948 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:53.947+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:54.009 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:54.009 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:54.009 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:57:54.009 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:57:54.010 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:54.018 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T23:57:54.036 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:57:54.037 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-14T23:57:54.049 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":143,"stamp":"2024-09-14T23:57:53.469325+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1147693738,"num_objects":2902,"num_object_clones":588,"num_object_copies":5804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2902,"num_whiteouts":1569,"num_read":178580,"num_read_kb":718502,"num_write":17929,"num_write_kb":399116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1295,"num_bytes_recovered":66002888,"num_keys_recovered":133,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15346,"ondisk_log_size":15346,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":441176,"kb_used_data":72228,"kb_used_omap":12,"kb_used_meta":368691,"kb_avail":754533544,"statfs":{"total":773094113280,"available":772642349056,"internally_reserved":0,"allocated":73961472,"data_stored":601726812,"data_compressed":45466261,"data_compressed_allocated":46465024,"data_compressed_original":580942039,"omap_allocated":12752,"internal_metadata":377540144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141343610,"num_objects":73,"num_object_clones":16,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":10,"num_read":10826,"num_read_kb":22992,"num_write":1726,"num_write_kb":16928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":4754342,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000701"},"pg_stats":[{"pgid":"3.1f","version":"99'883","reported_seq":4006,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-14T23:57:53.169591+0000","last_change":"2024-09-14T23:57:00.062727+0000","last_active":"2024-09-14T23:57:53.169591+0000","last_peered":"2024-09-14T23:57:53.169591+0000","last_clean":"2024-09-14T23:57:53.169591+0000","last_became_active":"2024-09-14T23:57:00.062116+0000","last_became_peered":"2024-09-14T23:57:00.062116+0000","last_unstale":"2024-09-14T23:57:53.169591+0000","last_undegraded":"2024-09-14T23:57:53.169591+0000","last_fullsized":"2024-09-14T23:57:53.169591+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:09:33.111322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015692659999999999,"stat_sum":{"num_bytes":51582047,"num_objects":107,"num_object_clones":23,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":51,"num_read":6666,"num_read_kb":18066,"num_write":1170,"num_write_kb":25027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":2327658,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"95'291","reported_seq":2477,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T23:57:50.067661+0000","last_change":"2024-09-14T23:57:47.057053+0000","last_active":"2024-09-14T23:57:50.067661+0000","last_peered":"2024-09-14T23:57:50.067661+0000","last_clean":"2024-09-14T23:57:50.067661+0000","last_became_active":"2024-09-14T23:57:47.055539+0000","last_became_peered":"2024-09-14T23:57:47.055539+0000","last_unstale":"2024-09-14T23:57:50.067661+0000","last_undegraded":"2024-09-14T23:57:50.067661+0000","last_fullsized":"2024-09-14T23:57:50.067661+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":291,"log_dups_size":0,"ondisk_log_size":291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:17:00.418423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010144430000000001,"stat_sum":{"num_bytes":25057664,"num_objects":96,"num_object_clones":25,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":46,"num_read":2006,"num_read_kb":10342,"num_write":259,"num_write_kb":9134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":9327506,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"96'351","reported_seq":2922,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T23:57:50.186561+0000","last_change":"2024-09-14T23:57:00.062960+0000","last_active":"2024-09-14T23:57:50.186561+0000","last_peered":"2024-09-14T23:57:50.186561+0000","last_clean":"2024-09-14T23:57:50.186561+0000","last_became_active":"2024-09-14T23:57:00.062242+0000","last_became_peered":"2024-09-14T23:57:00.062242+0000","last_unstale":"2024-09-14T23:57:50.186561+0000","last_undegraded":"2024-09-14T23:57:50.186561+0000","last_fullsized":"2024-09-14T23:57:50.186561+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:05:47.132518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001148175,"stat_sum":{"num_bytes":32029161,"num_objects":95,"num_object_clones":20,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":54,"num_read":3293,"num_read_kb":15949,"num_write":306,"num_write_kb":11370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":1249963,"num_keys_recovered":34,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"96'635","reported_seq":4568,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T23:57:50.196762+0000","last_change":"2024-09-14T23:57:00.062694+0000","last_active":"2024-09-14T23:57:50.196762+0000","last_peered":"2024-09-14T23:57:50.196762+0000","last_clean":"2024-09-14T23:57:50.196762+0000","last_became_active":"2024-09-14T23:57:00.062126+0000","last_became_peered":"2024-09-14T23:57:00.062126+0000","last_unstale":"2024-09-14T23:57:50.196762+0000","last_undegraded":"2024-09-14T23:57:50.196762+0000","last_fullsized":"2024-09-14T23:57:50.196762+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":635,"log_dups_size":0,"ondisk_log_size":635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:51:55.743612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095260199999999998,"stat_sum":{"num_bytes":19625316,"num_objects":82,"num_object_clones":12,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":49,"num_read":6247,"num_read_kb":17102,"num_write":837,"num_write_kb":6881,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":2675491,"num_keys_recovered":40,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"99'412","reported_seq":1508,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-14T23:57:53.078013+0000","last_change":"2024-09-14T23:56:34.401802+0000","last_active":"2024-09-14T23:57:53.078013+0000","last_peered":"2024-09-14T23:57:53.078013+0000","last_clean":"2024-09-14T23:57:53.078013+0000","last_became_active":"2024-09-14T23:56:32.398329+0000","last_became_peered":"2024-09-14T23:56:32.398329+0000","last_unstale":"2024-09-14T23:57:53.078013+0000","last_undegraded":"2024-09-14T23:57:53.078013+0000","last_fullsized":"2024-09-14T23:57:53.078013+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":412,"log_dups_size":0,"ondisk_log_size":412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:32.597901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018313229999999999,"stat_sum":{"num_bytes":36338711,"num_objects":78,"num_object_clones":19,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":38,"num_read":1686,"num_read_kb":10992,"num_write":480,"num_write_kb":9054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"95'402","reported_seq":4950,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T23:57:50.196470+0000","last_change":"2024-09-14T23:56:34.405093+0000","last_active":"2024-09-14T23:57:50.196470+0000","last_peered":"2024-09-14T23:57:50.196470+0000","last_clean":"2024-09-14T23:57:50.196470+0000","last_became_active":"2024-09-14T23:56:32.399655+0000","last_became_peered":"2024-09-14T23:56:32.399655+0000","last_unstale":"2024-09-14T23:57:50.196470+0000","last_undegraded":"2024-09-14T23:57:50.196470+0000","last_fullsized":"2024-09-14T23:57:50.196470+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:50:31.715252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096230199999999995,"stat_sum":{"num_bytes":89848192,"num_objects":99,"num_object_clones":22,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":59,"num_read":5122,"num_read_kb":34075,"num_write":374,"num_write_kb":19038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"98'517","reported_seq":2728,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-14T23:57:53.159573+0000","last_change":"2024-09-14T23:56:34.400617+0000","last_active":"2024-09-14T23:57:53.159573+0000","last_peered":"2024-09-14T23:57:53.159573+0000","last_clean":"2024-09-14T23:57:53.159573+0000","last_became_active":"2024-09-14T23:56:32.404364+0000","last_became_peered":"2024-09-14T23:56:32.404364+0000","last_unstale":"2024-09-14T23:57:53.159573+0000","last_undegraded":"2024-09-14T23:57:53.159573+0000","last_fullsized":"2024-09-14T23:57:53.159573+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":517,"log_dups_size":0,"ondisk_log_size":517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:58:07.475888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001003264,"stat_sum":{"num_bytes":49022121,"num_objects":97,"num_object_clones":15,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":53,"num_read":4392,"num_read_kb":13354,"num_write":546,"num_write_kb":5997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"95'532","reported_seq":4440,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T23:57:50.066950+0000","last_change":"2024-09-14T23:56:34.415021+0000","last_active":"2024-09-14T23:57:50.066950+0000","last_peered":"2024-09-14T23:57:50.066950+0000","last_clean":"2024-09-14T23:57:50.066950+0000","last_became_active":"2024-09-14T23:56:32.401799+0000","last_became_peered":"2024-09-14T23:56:32.401799+0000","last_unstale":"2024-09-14T23:57:50.066950+0000","last_undegraded":"2024-09-14T23:57:50.066950+0000","last_fullsized":"2024-09-14T23:57:50.066950+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:21:15.062747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062985000000000005,"stat_sum":{"num_bytes":45867492,"num_objects":121,"num_object_clones":29,"num_object_copies":242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":121,"num_whiteouts":66,"num_read":5316,"num_read_kb":25342,"num_write":547,"num_write_kb":9148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"99'371","reported_seq":2215,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-14T23:57:53.203892+0000","last_change":"2024-09-14T23:57:00.062824+0000","last_active":"2024-09-14T23:57:53.203892+0000","last_peered":"2024-09-14T23:57:53.203892+0000","last_clean":"2024-09-14T23:57:53.203892+0000","last_became_active":"2024-09-14T23:57:00.062304+0000","last_became_peered":"2024-09-14T23:57:00.062304+0000","last_unstale":"2024-09-14T23:57:53.203892+0000","last_undegraded":"2024-09-14T23:57:53.203892+0000","last_fullsized":"2024-09-14T23:57:53.203892+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:39.555350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014783859999999999,"stat_sum":{"num_bytes":11864925,"num_objects":81,"num_object_clones":18,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":42,"num_read":2464,"num_read_kb":8850,"num_write":351,"num_write_kb":5215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":3345543,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"95'295","reported_seq":6724,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T23:57:50.067103+0000","last_change":"2024-09-14T23:57:26.973933+0000","last_active":"2024-09-14T23:57:50.067103+0000","last_peered":"2024-09-14T23:57:50.067103+0000","last_clean":"2024-09-14T23:57:50.067103+0000","last_became_active":"2024-09-14T23:57:26.973212+0000","last_became_peered":"2024-09-14T23:57:26.973212+0000","last_unstale":"2024-09-14T23:57:50.067103+0000","last_undegraded":"2024-09-14T23:57:50.067103+0000","last_fullsized":"2024-09-14T23:57:50.067103+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":295,"log_dups_size":0,"ondisk_log_size":295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:40:18.932034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039416400000000002,"stat_sum":{"num_bytes":55048936,"num_objects":69,"num_object_clones":10,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":36,"num_read":6973,"num_read_kb":25395,"num_write":257,"num_write_kb":12825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":13520480,"num_keys_recovered":48,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.15","version":"95'464","reported_seq":7688,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T23:57:50.066965+0000","last_change":"2024-09-14T23:56:34.399382+0000","last_active":"2024-09-14T23:57:50.066965+0000","last_peered":"2024-09-14T23:57:50.066965+0000","last_clean":"2024-09-14T23:57:50.066965+0000","last_became_active":"2024-09-14T23:56:32.405387+0000","last_became_peered":"2024-09-14T23:56:32.405387+0000","last_unstale":"2024-09-14T23:57:50.066965+0000","last_undegraded":"2024-09-14T23:57:50.066965+0000","last_fullsized":"2024-09-14T23:57:50.066965+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:44:55.920068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046328199999999997,"stat_sum":{"num_bytes":39854225,"num_objects":85,"num_object_clones":14,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":48,"num_read":8885,"num_read_kb":33821,"num_write":516,"num_write_kb":23034,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"96'917","reported_seq":5753,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T23:57:50.074909+0000","last_change":"2024-09-14T23:56:34.404536+0000","last_active":"2024-09-14T23:57:50.074909+0000","last_peered":"2024-09-14T23:57:50.074909+0000","last_clean":"2024-09-14T23:57:50.074909+0000","last_became_active":"2024-09-14T23:56:32.403019+0000","last_became_peered":"2024-09-14T23:56:32.403019+0000","last_unstale":"2024-09-14T23:57:50.074909+0000","last_undegraded":"2024-09-14T23:57:50.074909+0000","last_fullsized":"2024-09-14T23:57:50.074909+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":917,"log_dups_size":0,"ondisk_log_size":917,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:41:47.331249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037505299999999997,"stat_sum":{"num_bytes":27458149,"num_objects":107,"num_object_clones":24,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":57,"num_read":8370,"num_read_kb":25812,"num_write":1446,"num_write_kb":14559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":169,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T23:57:50.109198+0000","last_change":"2024-09-14T23:56:24.727061+0000","last_active":"2024-09-14T23:57:50.109198+0000","last_peered":"2024-09-14T23:57:50.109198+0000","last_clean":"2024-09-14T23:57:50.109198+0000","last_became_active":"2024-09-14T23:56:22.526915+0000","last_became_peered":"2024-09-14T23:56:22.526915+0000","last_unstale":"2024-09-14T23:57:50.109198+0000","last_undegraded":"2024-09-14T23:57:50.109198+0000","last_fullsized":"2024-09-14T23:57:50.109198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:43:15.359266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045105900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"99'562","reported_seq":3877,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-14T23:57:53.188632+0000","last_change":"2024-09-14T23:56:34.402234+0000","last_active":"2024-09-14T23:57:53.188632+0000","last_peered":"2024-09-14T23:57:53.188632+0000","last_clean":"2024-09-14T23:57:53.188632+0000","last_became_active":"2024-09-14T23:56:32.398664+0000","last_became_peered":"2024-09-14T23:56:32.398664+0000","last_unstale":"2024-09-14T23:57:53.188632+0000","last_undegraded":"2024-09-14T23:57:53.188632+0000","last_fullsized":"2024-09-14T23:57:53.188632+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:31:54.225925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012179999999999999,"stat_sum":{"num_bytes":27047415,"num_objects":79,"num_object_clones":10,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":43,"num_read":5589,"num_read_kb":18540,"num_write":683,"num_write_kb":5702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":175,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-14T23:57:53.088116+0000","last_change":"2024-09-14T23:56:24.534632+0000","last_active":"2024-09-14T23:57:53.088116+0000","last_peered":"2024-09-14T23:57:53.088116+0000","last_clean":"2024-09-14T23:57:53.088116+0000","last_became_active":"2024-09-14T23:56:22.526355+0000","last_became_peered":"2024-09-14T23:56:22.526355+0000","last_unstale":"2024-09-14T23:57:53.088116+0000","last_undegraded":"2024-09-14T23:57:53.088116+0000","last_fullsized":"2024-09-14T23:57:53.088116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:02:16.059627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049910000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"95'371","reported_seq":6109,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T23:57:50.067774+0000","last_change":"2024-09-14T23:56:34.416134+0000","last_active":"2024-09-14T23:57:50.067774+0000","last_peered":"2024-09-14T23:57:50.067774+0000","last_clean":"2024-09-14T23:57:50.067774+0000","last_became_active":"2024-09-14T23:56:32.404281+0000","last_became_peered":"2024-09-14T23:56:32.404281+0000","last_unstale":"2024-09-14T23:57:50.067774+0000","last_undegraded":"2024-09-14T23:57:50.067774+0000","last_fullsized":"2024-09-14T23:57:50.067774+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:55:32.531337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013337620000000001,"stat_sum":{"num_bytes":33254849,"num_objects":103,"num_object_clones":24,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":59,"num_read":6877,"num_read_kb":41203,"num_write":353,"num_write_kb":16713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":169,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T23:57:50.067165+0000","last_change":"2024-09-14T23:56:25.093112+0000","last_active":"2024-09-14T23:57:50.067165+0000","last_peered":"2024-09-14T23:57:50.067165+0000","last_clean":"2024-09-14T23:57:50.067165+0000","last_became_active":"2024-09-14T23:56:22.528211+0000","last_became_peered":"2024-09-14T23:56:22.528211+0000","last_unstale":"2024-09-14T23:57:50.067165+0000","last_undegraded":"2024-09-14T23:57:50.067165+0000","last_fullsized":"2024-09-14T23:57:50.067165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:14:11.495286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000459686,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"99'441","reported_seq":4379,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-14T23:57:53.088076+0000","last_change":"2024-09-14T23:56:34.401111+0000","last_active":"2024-09-14T23:57:53.088076+0000","last_peered":"2024-09-14T23:57:53.088076+0000","last_clean":"2024-09-14T23:57:53.088076+0000","last_became_active":"2024-09-14T23:56:32.401089+0000","last_became_peered":"2024-09-14T23:56:32.401089+0000","last_unstale":"2024-09-14T23:57:53.088076+0000","last_undegraded":"2024-09-14T23:57:53.088076+0000","last_fullsized":"2024-09-14T23:57:53.088076+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:46:49.832090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013033610000000001,"stat_sum":{"num_bytes":29536425,"num_objects":109,"num_object_clones":22,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":57,"num_read":5561,"num_read_kb":17894,"num_write":430,"num_write_kb":12789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":175,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-14T23:57:53.088045+0000","last_change":"2024-09-14T23:56:24.534510+0000","last_active":"2024-09-14T23:57:53.088045+0000","last_peered":"2024-09-14T23:57:53.088045+0000","last_clean":"2024-09-14T23:57:53.088045+0000","last_became_active":"2024-09-14T23:56:22.527611+0000","last_became_peered":"2024-09-14T23:56:22.527611+0000","last_unstale":"2024-09-14T23:57:53.088045+0000","last_undegraded":"2024-09-14T23:57:53.088045+0000","last_fullsized":"2024-09-14T23:57:53.088045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:15:45.606026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049916299999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"96'503","reported_seq":3167,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T23:57:50.090379+0000","last_change":"2024-09-14T23:57:00.065551+0000","last_active":"2024-09-14T23:57:50.090379+0000","last_peered":"2024-09-14T23:57:50.090379+0000","last_clean":"2024-09-14T23:57:50.090379+0000","last_became_active":"2024-09-14T23:57:00.064678+0000","last_became_peered":"2024-09-14T23:57:00.064678+0000","last_unstale":"2024-09-14T23:57:50.090379+0000","last_undegraded":"2024-09-14T23:57:50.090379+0000","last_fullsized":"2024-09-14T23:57:50.090379+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":503,"log_dups_size":0,"ondisk_log_size":503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:28:37.441835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001272947,"stat_sum":{"num_bytes":13421205,"num_objects":107,"num_object_clones":21,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":60,"num_read":4808,"num_read_kb":13311,"num_write":554,"num_write_kb":9180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":3349587,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"95'387","reported_seq":3002,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T23:57:50.069397+0000","last_change":"2024-09-14T23:56:34.404751+0000","last_active":"2024-09-14T23:57:50.069397+0000","last_peered":"2024-09-14T23:57:50.069397+0000","last_clean":"2024-09-14T23:57:50.069397+0000","last_became_active":"2024-09-14T23:56:32.397883+0000","last_became_peered":"2024-09-14T23:56:32.397883+0000","last_unstale":"2024-09-14T23:57:50.069397+0000","last_undegraded":"2024-09-14T23:57:50.069397+0000","last_fullsized":"2024-09-14T23:57:50.069397+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:03:58.283214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048955799999999996,"stat_sum":{"num_bytes":59361942,"num_objects":101,"num_object_clones":22,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":51,"num_read":3773,"num_read_kb":22078,"num_write":337,"num_write_kb":15062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":186,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-14T23:57:53.087953+0000","last_change":"2024-09-14T23:56:57.052969+0000","last_active":"2024-09-14T23:57:53.087953+0000","last_peered":"2024-09-14T23:57:53.087953+0000","last_clean":"2024-09-14T23:57:53.087953+0000","last_became_active":"2024-09-14T23:56:57.050456+0000","last_became_peered":"2024-09-14T23:56:57.050456+0000","last_unstale":"2024-09-14T23:57:53.087953+0000","last_undegraded":"2024-09-14T23:57:53.087953+0000","last_fullsized":"2024-09-14T23:57:53.087953+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:57:07.644011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"98'417","reported_seq":8052,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-14T23:57:53.087978+0000","last_change":"2024-09-14T23:56:34.401159+0000","last_active":"2024-09-14T23:57:53.087978+0000","last_peered":"2024-09-14T23:57:53.087978+0000","last_clean":"2024-09-14T23:57:53.087978+0000","last_became_active":"2024-09-14T23:56:32.400278+0000","last_became_peered":"2024-09-14T23:56:32.400278+0000","last_unstale":"2024-09-14T23:57:53.087978+0000","last_undegraded":"2024-09-14T23:57:53.087978+0000","last_fullsized":"2024-09-14T23:57:53.087978+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":417,"log_dups_size":0,"ondisk_log_size":417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:49:16.088748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082507200000000004,"stat_sum":{"num_bytes":61091838,"num_objects":90,"num_object_clones":20,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":46,"num_read":7751,"num_read_kb":34715,"num_write":372,"num_write_kb":16251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":173,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-14T23:57:52.092778+0000","last_change":"2024-09-14T23:56:24.909260+0000","last_active":"2024-09-14T23:57:52.092778+0000","last_peered":"2024-09-14T23:57:52.092778+0000","last_clean":"2024-09-14T23:57:52.092778+0000","last_became_active":"2024-09-14T23:56:22.822598+0000","last_became_peered":"2024-09-14T23:56:22.822598+0000","last_unstale":"2024-09-14T23:57:52.092778+0000","last_undegraded":"2024-09-14T23:57:52.092778+0000","last_fullsized":"2024-09-14T23:57:52.092778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:32:32.568028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026863400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"98'584","reported_seq":4485,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-14T23:57:53.168971+0000","last_change":"2024-09-14T23:56:34.401366+0000","last_active":"2024-09-14T23:57:53.168971+0000","last_peered":"2024-09-14T23:57:53.168971+0000","last_clean":"2024-09-14T23:57:53.168971+0000","last_became_active":"2024-09-14T23:56:32.397551+0000","last_became_peered":"2024-09-14T23:56:32.397551+0000","last_unstale":"2024-09-14T23:57:53.168971+0000","last_undegraded":"2024-09-14T23:57:53.168971+0000","last_fullsized":"2024-09-14T23:57:53.168971+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":584,"log_dups_size":0,"ondisk_log_size":584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:17:41.521611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001418816,"stat_sum":{"num_bytes":16353889,"num_objects":91,"num_object_clones":14,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":52,"num_read":7019,"num_read_kb":16283,"num_write":742,"num_write_kb":4338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":169,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T23:57:50.066829+0000","last_change":"2024-09-14T23:56:25.093090+0000","last_active":"2024-09-14T23:57:50.066829+0000","last_peered":"2024-09-14T23:57:50.066829+0000","last_clean":"2024-09-14T23:57:50.066829+0000","last_became_active":"2024-09-14T23:56:22.528115+0000","last_became_peered":"2024-09-14T23:56:22.528115+0000","last_unstale":"2024-09-14T23:57:50.066829+0000","last_undegraded":"2024-09-14T23:57:50.066829+0000","last_fullsized":"2024-09-14T23:57:50.066829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:57:36.610730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036520999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"95'641","reported_seq":4207,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T23:57:50.067337+0000","last_change":"2024-09-14T23:57:26.975545+0000","last_active":"2024-09-14T23:57:50.067337+0000","last_peered":"2024-09-14T23:57:50.067337+0000","last_clean":"2024-09-14T23:57:50.067337+0000","last_became_active":"2024-09-14T23:57:26.974022+0000","last_became_peered":"2024-09-14T23:57:26.974022+0000","last_unstale":"2024-09-14T23:57:50.067337+0000","last_undegraded":"2024-09-14T23:57:50.067337+0000","last_fullsized":"2024-09-14T23:57:50.067337+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":641,"log_dups_size":0,"ondisk_log_size":641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:10:42.404143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073009599999999996,"stat_sum":{"num_bytes":19474348,"num_objects":104,"num_object_clones":25,"num_object_copies":208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":54,"num_read":5109,"num_read_kb":14815,"num_write":800,"num_write_kb":7903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":297,"num_bytes_recovered":12389647,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":171,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-14T23:57:52.092700+0000","last_change":"2024-09-14T23:56:57.054999+0000","last_active":"2024-09-14T23:57:52.092700+0000","last_peered":"2024-09-14T23:57:52.092700+0000","last_clean":"2024-09-14T23:57:52.092700+0000","last_became_active":"2024-09-14T23:56:57.050729+0000","last_became_peered":"2024-09-14T23:56:57.050729+0000","last_unstale":"2024-09-14T23:57:52.092700+0000","last_undegraded":"2024-09-14T23:57:52.092700+0000","last_fullsized":"2024-09-14T23:57:52.092700+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:23:11.727903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":320,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T23:57:50.066856+0000","last_change":"2024-09-14T23:56:15.332936+0000","last_active":"2024-09-14T23:57:50.066856+0000","last_peered":"2024-09-14T23:57:50.066856+0000","last_clean":"2024-09-14T23:57:50.066856+0000","last_became_active":"2024-09-14T23:56:15.331994+0000","last_became_peered":"2024-09-14T23:56:15.331994+0000","last_unstale":"2024-09-14T23:57:50.066856+0000","last_undegraded":"2024-09-14T23:57:50.066856+0000","last_fullsized":"2024-09-14T23:57:50.066856+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:14.322882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:14.322882+0000","last_clean_scrub_stamp":"2024-09-14T23:56:14.322882+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:36:12.639000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"94'370","reported_seq":4289,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T23:57:50.067164+0000","last_change":"2024-09-14T23:56:34.415037+0000","last_active":"2024-09-14T23:57:50.067164+0000","last_peered":"2024-09-14T23:57:50.067164+0000","last_clean":"2024-09-14T23:57:50.067164+0000","last_became_active":"2024-09-14T23:56:32.403572+0000","last_became_peered":"2024-09-14T23:56:32.403572+0000","last_unstale":"2024-09-14T23:57:50.067164+0000","last_undegraded":"2024-09-14T23:57:50.067164+0000","last_fullsized":"2024-09-14T23:57:50.067164+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:09:42.154721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052643700000000004,"stat_sum":{"num_bytes":32877576,"num_objects":78,"num_object_clones":16,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":45,"num_read":4672,"num_read_kb":16462,"num_write":330,"num_write_kb":10528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"95'250","reported_seq":2115,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T23:57:50.069247+0000","last_change":"2024-09-14T23:56:34.404977+0000","last_active":"2024-09-14T23:57:50.069247+0000","last_peered":"2024-09-14T23:57:50.069247+0000","last_clean":"2024-09-14T23:57:50.069247+0000","last_became_active":"2024-09-14T23:56:32.400017+0000","last_became_peered":"2024-09-14T23:56:32.400017+0000","last_unstale":"2024-09-14T23:57:50.069247+0000","last_undegraded":"2024-09-14T23:57:50.069247+0000","last_fullsized":"2024-09-14T23:57:50.069247+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":250,"log_dups_size":0,"ondisk_log_size":250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:15:01.032815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068884800000000004,"stat_sum":{"num_bytes":22078721,"num_objects":80,"num_object_clones":16,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":44,"num_read":1786,"num_read_kb":12130,"num_write":232,"num_write_kb":15435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"96'654","reported_seq":7080,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T23:57:50.109283+0000","last_change":"2024-09-14T23:56:34.498020+0000","last_active":"2024-09-14T23:57:50.109283+0000","last_peered":"2024-09-14T23:57:50.109283+0000","last_clean":"2024-09-14T23:57:50.109283+0000","last_became_active":"2024-09-14T23:56:32.402769+0000","last_became_peered":"2024-09-14T23:56:32.402769+0000","last_unstale":"2024-09-14T23:57:50.109283+0000","last_undegraded":"2024-09-14T23:57:50.109283+0000","last_fullsized":"2024-09-14T23:57:50.109283+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":654,"log_dups_size":0,"ondisk_log_size":654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:53:04.899302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035483999999999998,"stat_sum":{"num_bytes":26133279,"num_objects":80,"num_object_clones":12,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":48,"num_read":8484,"num_read_kb":29961,"num_write":953,"num_write_kb":9317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"95'506","reported_seq":13041,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T23:57:50.218143+0000","last_change":"2024-09-14T23:56:34.415414+0000","last_active":"2024-09-14T23:57:50.218143+0000","last_peered":"2024-09-14T23:57:50.218143+0000","last_clean":"2024-09-14T23:57:50.218143+0000","last_became_active":"2024-09-14T23:56:32.403231+0000","last_became_peered":"2024-09-14T23:56:32.403231+0000","last_unstale":"2024-09-14T23:57:50.218143+0000","last_undegraded":"2024-09-14T23:57:50.218143+0000","last_fullsized":"2024-09-14T23:57:50.218143+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:06:53.858796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072791000000000002,"stat_sum":{"num_bytes":38757501,"num_objects":75,"num_object_clones":17,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":39,"num_read":13547,"num_read_kb":56101,"num_write":626,"num_write_kb":8385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"96'433","reported_seq":13838,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T23:57:50.218673+0000","last_change":"2024-09-14T23:57:00.066551+0000","last_active":"2024-09-14T23:57:50.218673+0000","last_peered":"2024-09-14T23:57:50.218673+0000","last_clean":"2024-09-14T23:57:50.218673+0000","last_became_active":"2024-09-14T23:57:00.065790+0000","last_became_peered":"2024-09-14T23:57:00.065790+0000","last_unstale":"2024-09-14T23:57:50.218673+0000","last_undegraded":"2024-09-14T23:57:50.218673+0000","last_fullsized":"2024-09-14T23:57:50.218673+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:35:33.345953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092638099999999999,"stat_sum":{"num_bytes":74474757,"num_objects":94,"num_object_clones":22,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":48,"num_read":12934,"num_read_kb":58229,"num_write":549,"num_write_kb":33917,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":7912941,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"94'259","reported_seq":1590,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T23:57:50.067071+0000","last_change":"2024-09-14T23:57:26.973929+0000","last_active":"2024-09-14T23:57:50.067071+0000","last_peered":"2024-09-14T23:57:50.067071+0000","last_clean":"2024-09-14T23:57:50.067071+0000","last_became_active":"2024-09-14T23:57:26.973189+0000","last_became_peered":"2024-09-14T23:57:26.973189+0000","last_unstale":"2024-09-14T23:57:50.067071+0000","last_undegraded":"2024-09-14T23:57:50.067071+0000","last_fullsized":"2024-09-14T23:57:50.067071+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":259,"log_dups_size":0,"ondisk_log_size":259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:40:30.132106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057774300000000003,"stat_sum":{"num_bytes":10423348,"num_objects":69,"num_object_clones":12,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":40,"num_read":1006,"num_read_kb":7954,"num_write":258,"num_write_kb":5321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":9904034,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"95'379","reported_seq":4763,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T23:57:50.215217+0000","last_change":"2024-09-14T23:56:34.399894+0000","last_active":"2024-09-14T23:57:50.215217+0000","last_peered":"2024-09-14T23:57:50.215217+0000","last_clean":"2024-09-14T23:57:50.215217+0000","last_became_active":"2024-09-14T23:56:32.406251+0000","last_became_peered":"2024-09-14T23:56:32.406251+0000","last_unstale":"2024-09-14T23:57:50.215217+0000","last_undegraded":"2024-09-14T23:57:50.215217+0000","last_fullsized":"2024-09-14T23:57:50.215217+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:58:20.950436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073115799999999998,"stat_sum":{"num_bytes":29033791,"num_objects":85,"num_object_clones":17,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":42,"num_read":6820,"num_read_kb":20383,"num_write":365,"num_write_kb":11153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"94'380","reported_seq":3406,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T23:57:50.067230+0000","last_change":"2024-09-14T23:56:34.399966+0000","last_active":"2024-09-14T23:57:50.067230+0000","last_peered":"2024-09-14T23:57:50.067230+0000","last_clean":"2024-09-14T23:57:50.067230+0000","last_became_active":"2024-09-14T23:56:32.407109+0000","last_became_peered":"2024-09-14T23:56:32.407109+0000","last_unstale":"2024-09-14T23:57:50.067230+0000","last_undegraded":"2024-09-14T23:57:50.067230+0000","last_fullsized":"2024-09-14T23:57:50.067230+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":380,"log_dups_size":0,"ondisk_log_size":380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:53:59.807546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072766900000000002,"stat_sum":{"num_bytes":31857291,"num_objects":93,"num_object_clones":18,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":49,"num_read":4468,"num_read_kb":14080,"num_write":344,"num_write_kb":14935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"94'643","reported_seq":3394,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T23:57:50.109226+0000","last_change":"2024-09-14T23:56:34.498033+0000","last_active":"2024-09-14T23:57:50.109226+0000","last_peered":"2024-09-14T23:57:50.109226+0000","last_clean":"2024-09-14T23:57:50.109226+0000","last_became_active":"2024-09-14T23:56:32.402973+0000","last_became_peered":"2024-09-14T23:56:32.402973+0000","last_unstale":"2024-09-14T23:57:50.109226+0000","last_undegraded":"2024-09-14T23:57:50.109226+0000","last_fullsized":"2024-09-14T23:57:50.109226+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":643,"log_dups_size":0,"ondisk_log_size":643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:36:33.617969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035072199999999998,"stat_sum":{"num_bytes":49830135,"num_objects":98,"num_object_clones":17,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":58,"num_read":4378,"num_read_kb":27291,"num_write":1048,"num_write_kb":21244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"95'347","reported_seq":5618,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T23:57:50.066944+0000","last_change":"2024-09-14T23:56:34.399387+0000","last_active":"2024-09-14T23:57:50.066944+0000","last_peered":"2024-09-14T23:57:50.066944+0000","last_clean":"2024-09-14T23:57:50.066944+0000","last_became_active":"2024-09-14T23:56:32.405457+0000","last_became_peered":"2024-09-14T23:56:32.405457+0000","last_unstale":"2024-09-14T23:57:50.066944+0000","last_undegraded":"2024-09-14T23:57:50.066944+0000","last_fullsized":"2024-09-14T23:57:50.066944+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":347,"log_dups_size":0,"ondisk_log_size":347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:06:34.192957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048125700000000002,"stat_sum":{"num_bytes":36416126,"num_objects":87,"num_object_clones":16,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":46,"num_read":5830,"num_read_kb":25253,"num_write":360,"num_write_kb":6915,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"99'250","reported_seq":1860,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-14T23:57:53.170298+0000","last_change":"2024-09-14T23:56:34.402592+0000","last_active":"2024-09-14T23:57:53.170298+0000","last_peered":"2024-09-14T23:57:53.170298+0000","last_clean":"2024-09-14T23:57:53.170298+0000","last_became_active":"2024-09-14T23:56:32.402782+0000","last_became_peered":"2024-09-14T23:56:32.402782+0000","last_unstale":"2024-09-14T23:57:53.170298+0000","last_undegraded":"2024-09-14T23:57:53.170298+0000","last_fullsized":"2024-09-14T23:57:53.170298+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":250,"log_dups_size":0,"ondisk_log_size":250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:22:12.937342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001446697,"stat_sum":{"num_bytes":28356532,"num_objects":69,"num_object_clones":17,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":40,"num_read":1511,"num_read_kb":9519,"num_write":212,"num_write_kb":6245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"96'779","reported_seq":2839,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T23:57:50.215001+0000","last_change":"2024-09-14T23:56:34.399629+0000","last_active":"2024-09-14T23:57:50.215001+0000","last_peered":"2024-09-14T23:57:50.215001+0000","last_clean":"2024-09-14T23:57:50.215001+0000","last_became_active":"2024-09-14T23:56:32.407220+0000","last_became_peered":"2024-09-14T23:56:32.407220+0000","last_unstale":"2024-09-14T23:57:50.215001+0000","last_undegraded":"2024-09-14T23:57:50.215001+0000","last_fullsized":"2024-09-14T23:57:50.215001+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":779,"log_dups_size":0,"ondisk_log_size":779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:59:06.643858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062462200000000002,"stat_sum":{"num_bytes":23725444,"num_objects":89,"num_object_clones":19,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":49,"num_read":5121,"num_read_kb":23100,"num_write":1131,"num_write_kb":14083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1147103351,"num_objects":2898,"num_object_clones":588,"num_object_copies":5796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2898,"num_whiteouts":1569,"num_read":178464,"num_read_kb":718402,"num_write":17768,"num_write_kb":396698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1291,"num_bytes_recovered":66002850,"num_keys_recovered":133,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65527808,"data_stored":601583286,"data_compressed":46038484,"data_compressed_allocated":46825472,"data_compressed_original":584822330,"omap_allocated":0,"internal_metadata":0},"log_size":15226,"ondisk_log_size":15226,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64594,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509464,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60156,"kb_used_data":10924,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94311684,"statfs":{"total":96636764160,"available":96575164416,"internally_reserved":0,"allocated":11186176,"data_stored":99187771,"data_compressed":7690181,"data_compressed_allocated":7786496,"data_compressed_original":96734920,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509463,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44208,"kb_used_data":4696,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94327632,"statfs":{"total":96636764160,"available":96591495168,"internally_reserved":0,"allocated":4808704,"data_stored":28199716,"data_compressed":2264756,"data_compressed_allocated":2711552,"data_compressed_original":26809406,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509462,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58808,"kb_used_data":9540,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94313032,"statfs":{"total":96636764160,"available":96576544768,"internally_reserved":0,"allocated":9768960,"data_stored":83938541,"data_compressed":6286931,"data_compressed_allocated":6340608,"data_compressed_original":81463208,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509464,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":88112,"kb_used_data":12512,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94283728,"statfs":{"total":96636764160,"available":96546537472,"internally_reserved":0,"allocated":12812288,"data_stored":106834382,"data_compressed":7978169,"data_compressed_allocated":8458240,"data_compressed_original":103454422,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509462,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47232,"kb_used_data":11884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94324608,"statfs":{"total":96636764160,"available":96588398592,"internally_reserved":0,"allocated":12169216,"data_stored":110358514,"data_compressed":8313154,"data_compressed_allocated":8208384,"data_compressed_original":107275550,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542167,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34568,"kb_used_data":7484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337272,"statfs":{"total":96636764160,"available":96601366528,"internally_reserved":0,"allocated":7663616,"data_stored":51628668,"data_compressed":4105793,"data_compressed_allocated":3923968,"data_compressed_original":48597378,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542166,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63992,"kb_used_data":10600,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94307848,"statfs":{"total":96636764160,"available":96571236352,"internally_reserved":0,"allocated":10854400,"data_stored":89959171,"data_compressed":6437508,"data_compressed_allocated":6721536,"data_compressed_original":86744746,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44100,"kb_used_data":4588,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94327740,"statfs":{"total":96636764160,"available":96591605760,"internally_reserved":0,"allocated":4698112,"data_stored":31620049,"data_compressed":2389769,"data_compressed_allocated":2314240,"data_compressed_original":29862409,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32297,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32297,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":9654272,"data_stored":90805555,"data_compressed":6691477,"data_compressed_allocated":6819840,"data_compressed_original":88237069,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6688768,"data_stored":53686436,"data_compressed":4368619,"data_compressed_allocated":4136960,"data_compressed_original":51218818,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3588096,"data_stored":31116745,"data_compressed":2389769,"data_compressed_allocated":2314240,"data_compressed_original":29862409,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":11444224,"data_stored":110234063,"data_compressed":8346589,"data_compressed_allocated":8245248,"data_compressed_original":107377950,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":11706368,"data_stored":106320130,"data_compressed":7978169,"data_compressed_allocated":8458240,"data_compressed_original":103454422,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8536064,"data_stored":83311072,"data_compressed":6287315,"data_compressed_allocated":6340608,"data_compressed_original":81471400,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3821568,"data_stored":27817421,"data_compressed":2266552,"data_compressed_allocated":2715648,"data_compressed_original":26846270,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":10088448,"data_stored":98291864,"data_compressed":7709994,"data_compressed_allocated":7794688,"data_compressed_original":96353992,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T23:57:54.051 INFO:tasks.thrashosds.thrasher:Setting 2.2 to [6, 2, 1, 7] 2024-09-14T23:57:54.051 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '2.2', '6', '2', '1', '7'] 2024-09-14T23:57:54.051 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 2.2 6 2 1 7 2024-09-14T23:57:54.052 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:54.047+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:54.095 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T23:57:54.131 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:54.131 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:57:54.131 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:57:54.131 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:57:54.131 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.131 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:107472 3.1e 3:7bf21588:::rbd_data.11187de8b0a5.000000000000005d:head [set-alloc-hint object_size 2097152 write_size 2097152,write 317788~1491] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-14T23:57:54.132 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:25.198493+0000", 2024-09-14T23:57:54.132 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 28.930372338000002, 2024-09-14T23:57:54.132 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.78175149300000002, 2024-09-14T23:57:54.132 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:54.132 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:54.132 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:54.132 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:54.132 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:54.132 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:54.132 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 107472 2024-09-14T23:57:54.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.132 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:54.132 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.133 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:54.133 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198493+0000", 2024-09-14T23:57:54.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.133 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:54.133 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198493+0000", 2024-09-14T23:57:54.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:54.134 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198495+0000", 2024-09-14T23:57:54.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.826e-06 2024-09-14T23:57:54.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:54.134 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198503+0000", 2024-09-14T23:57:54.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0150000000000003e-06 2024-09-14T23:57:54.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:54.134 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198521+0000", 2024-09-14T23:57:54.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8374000000000001e-05 2024-09-14T23:57:54.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.135 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:54.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198533+0000", 2024-09-14T23:57:54.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1593000000000001e-05 2024-09-14T23:57:54.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.135 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:54.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198603+0000", 2024-09-14T23:57:54.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0279999999999998e-05 2024-09-14T23:57:54.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.135 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:57:54.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198617+0000", 2024-09-14T23:57:54.136 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3517e-05 2024-09-14T23:57:54.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.136 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.136 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:54.136 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.977067+0000", 2024-09-14T23:57:54.136 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.77845065800000002 2024-09-14T23:57:54.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.136 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.136 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for missing object", 2024-09-14T23:57:54.136 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.977173+0000", 2024-09-14T23:57:54.136 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010543999999999999 2024-09-14T23:57:54.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.136 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.136 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:54.137 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.978737+0000", 2024-09-14T23:57:54.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001564078 2024-09-14T23:57:54.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:54.137 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.978845+0000", 2024-09-14T23:57:54.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000108643 2024-09-14T23:57:54.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1", 2024-09-14T23:57:54.137 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.978964+0000", 2024-09-14T23:57:54.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000119085 2024-09-14T23:57:54.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:54.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980109+0000", 2024-09-14T23:57:54.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0011442170000000001 2024-09-14T23:57:54.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:57:54.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980130+0000", 2024-09-14T23:57:54.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1766000000000001e-05 2024-09-14T23:57:54.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.139 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:54.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980171+0000", 2024-09-14T23:57:54.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0627999999999998e-05 2024-09-14T23:57:54.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.139 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:54.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980244+0000", 2024-09-14T23:57:54.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3373000000000004e-05 2024-09-14T23:57:54.139 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.139 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:54.139 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.139 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129149 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 6619136~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:54.140 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.925871+0000", 2024-09-14T23:57:54.140 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 15.202993896000001, 2024-09-14T23:57:54.140 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054291066999999998, 2024-09-14T23:57:54.140 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:54.140 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:54.140 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:54.140 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:54.140 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:54.140 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:54.140 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129149 2024-09-14T23:57:54.140 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.140 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:54.140 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.140 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:54.141 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.925871+0000", 2024-09-14T23:57:54.141 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.141 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:54.141 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.925871+0000", 2024-09-14T23:57:54.141 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.141 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:54.141 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.927669+0000", 2024-09-14T23:57:54.141 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001797838 2024-09-14T23:57:54.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.142 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:54.142 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.927672+0000", 2024-09-14T23:57:54.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5280000000000002e-06 2024-09-14T23:57:54.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.142 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:54.142 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.927678+0000", 2024-09-14T23:57:54.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0340000000000002e-06 2024-09-14T23:57:54.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.142 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:54.142 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.927681+0000", 2024-09-14T23:57:54.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.681e-06 2024-09-14T23:57:54.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:54.143 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.979979+0000", 2024-09-14T23:57:54.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052297950000000003 2024-09-14T23:57:54.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:54.143 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.980024+0000", 2024-09-14T23:57:54.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4693000000000001e-05 2024-09-14T23:57:54.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:54.143 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.980162+0000", 2024-09-14T23:57:54.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000138343 2024-09-14T23:57:54.143 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.144 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:54.144 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.144 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.144 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129159 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 5996544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:54.144 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.927783+0000", 2024-09-14T23:57:54.144 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 15.201082446999999, 2024-09-14T23:57:54.144 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054661122999999999, 2024-09-14T23:57:54.144 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:54.144 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:54.144 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:54.144 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:54.144 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:54.144 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:54.145 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129159 2024-09-14T23:57:54.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.145 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:54.145 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.145 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:54.145 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.927783+0000", 2024-09-14T23:57:54.145 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.145 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.145 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:54.145 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.927783+0000", 2024-09-14T23:57:54.145 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.145 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.145 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:54.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929684+0000", 2024-09-14T23:57:54.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0019011080000000001 2024-09-14T23:57:54.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.146 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:54.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929688+0000", 2024-09-14T23:57:54.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2610000000000004e-06 2024-09-14T23:57:54.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.146 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:54.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929697+0000", 2024-09-14T23:57:54.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0219999999999993e-06 2024-09-14T23:57:54.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.147 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:54.147 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929702+0000", 2024-09-14T23:57:54.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8840000000000002e-06 2024-09-14T23:57:54.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.147 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:54.147 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.982090+0000", 2024-09-14T23:57:54.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052388045000000001 2024-09-14T23:57:54.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.147 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:54.148 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.982208+0000", 2024-09-14T23:57:54.148 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118236 2024-09-14T23:57:54.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.148 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.148 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:54.148 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.982444+0000", 2024-09-14T23:57:54.148 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000235567 2024-09-14T23:57:54.148 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.148 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:54.148 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.148 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.148 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129166 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 8192000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:54.148 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.929794+0000", 2024-09-14T23:57:54.149 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 15.199071696000001, 2024-09-14T23:57:54.149 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054471973, 2024-09-14T23:57:54.149 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:54.149 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:54.149 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:54.149 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:54.149 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:54.149 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:54.149 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129166 2024-09-14T23:57:54.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.149 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:54.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.149 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:54.149 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929794+0000", 2024-09-14T23:57:54.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:54.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929794+0000", 2024-09-14T23:57:54.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:54.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929794+0000", 2024-09-14T23:57:54.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6899999999999997e-07 2024-09-14T23:57:54.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:54.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929797+0000", 2024-09-14T23:57:54.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.328e-06 2024-09-14T23:57:54.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:54.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929802+0000", 2024-09-14T23:57:54.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7309999999999998e-06 2024-09-14T23:57:54.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:54.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929807+0000", 2024-09-14T23:57:54.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4349999999999999e-06 2024-09-14T23:57:54.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:54.152 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984063+0000", 2024-09-14T23:57:54.152 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054256331999999997 2024-09-14T23:57:54.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.152 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:54.152 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984177+0000", 2024-09-14T23:57:54.152 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114225 2024-09-14T23:57:54.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.152 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:54.152 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984266+0000", 2024-09-14T23:57:54.152 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8352999999999999e-05 2024-09-14T23:57:54.152 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.152 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:54.152 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.153 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.153 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129167 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 6782976~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:54.153 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.929827+0000", 2024-09-14T23:57:54.153 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 15.199038287, 2024-09-14T23:57:54.153 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054637593999999998, 2024-09-14T23:57:54.153 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:54.153 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:54.153 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:54.153 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:54.153 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:54.153 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:54.153 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129167 2024-09-14T23:57:54.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.154 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:54.154 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.154 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:54.154 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929827+0000", 2024-09-14T23:57:54.154 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.154 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.154 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:54.154 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929827+0000", 2024-09-14T23:57:54.154 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.154 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.154 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:54.154 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929827+0000", 2024-09-14T23:57:54.154 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4000000000000002e-07 2024-09-14T23:57:54.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:54.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929830+0000", 2024-09-14T23:57:54.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.419e-06 2024-09-14T23:57:54.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:54.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929837+0000", 2024-09-14T23:57:54.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0450000000000003e-06 2024-09-14T23:57:54.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:54.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929842+0000", 2024-09-14T23:57:54.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.69e-06 2024-09-14T23:57:54.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.156 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:54.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984278+0000", 2024-09-14T23:57:54.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054436516999999997 2024-09-14T23:57:54.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.156 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:54.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984331+0000", 2024-09-14T23:57:54.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2590999999999998e-05 2024-09-14T23:57:54.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.156 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:54.157 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984465+0000", 2024-09-14T23:57:54.157 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013389200000000001 2024-09-14T23:57:54.157 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.157 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:54.157 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.157 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.157 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129168 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 8224768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:54.157 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.929849+0000", 2024-09-14T23:57:54.157 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 15.199016729, 2024-09-14T23:57:54.157 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054870368000000003, 2024-09-14T23:57:54.157 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:54.157 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:54.157 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:54.157 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:54.158 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:54.158 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:54.158 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129168 2024-09-14T23:57:54.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.158 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:54.158 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.158 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:54.158 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929849+0000", 2024-09-14T23:57:54.158 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.158 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.158 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:54.158 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929849+0000", 2024-09-14T23:57:54.158 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.159 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:54.159 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931681+0000", 2024-09-14T23:57:54.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018326989999999999 2024-09-14T23:57:54.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.159 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:54.159 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931686+0000", 2024-09-14T23:57:54.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7190000000000001e-06 2024-09-14T23:57:54.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.159 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:54.159 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931697+0000", 2024-09-14T23:57:54.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1287999999999999e-05 2024-09-14T23:57:54.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.160 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.160 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:54.160 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931704+0000", 2024-09-14T23:57:54.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3339999999999998e-06 2024-09-14T23:57:54.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.160 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.160 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:54.160 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984505+0000", 2024-09-14T23:57:54.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052801684000000002 2024-09-14T23:57:54.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.160 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.160 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:54.161 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984621+0000", 2024-09-14T23:57:54.161 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000116165 2024-09-14T23:57:54.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.161 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.161 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:54.161 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984719+0000", 2024-09-14T23:57:54.161 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7479000000000006e-05 2024-09-14T23:57:54.161 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.161 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:54.161 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.161 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.161 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129175 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 9011200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:54.161 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.931806+0000", 2024-09-14T23:57:54.161 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 15.19705971, 2024-09-14T23:57:54.162 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054322384000000001, 2024-09-14T23:57:54.162 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:54.162 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:54.162 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:54.162 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:54.162 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:54.162 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:54.162 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129175 2024-09-14T23:57:54.162 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.162 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:54.162 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.162 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:54.162 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931806+0000", 2024-09-14T23:57:54.162 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.163 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:54.163 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931806+0000", 2024-09-14T23:57:54.163 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.163 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:54.163 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931806+0000", 2024-09-14T23:57:54.163 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2300000000000001e-07 2024-09-14T23:57:54.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.163 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:54.163 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931808+0000", 2024-09-14T23:57:54.164 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1440000000000001e-06 2024-09-14T23:57:54.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.164 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.164 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:54.164 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931815+0000", 2024-09-14T23:57:54.164 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.776e-06 2024-09-14T23:57:54.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.164 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.164 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:54.164 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931821+0000", 2024-09-14T23:57:54.164 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.733e-06 2024-09-14T23:57:54.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.164 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.164 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:54.165 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.986046+0000", 2024-09-14T23:57:54.165 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054225284999999998 2024-09-14T23:57:54.165 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.165 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.165 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:54.165 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.986089+0000", 2024-09-14T23:57:54.165 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2895999999999997e-05 2024-09-14T23:57:54.165 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.165 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.165 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:54.165 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.986128+0000", 2024-09-14T23:57:54.165 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9026999999999998e-05 2024-09-14T23:57:54.165 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.165 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:54.166 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.166 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.166 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129176 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 10452992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:54.166 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.931830+0000", 2024-09-14T23:57:54.166 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 15.197034968000001, 2024-09-14T23:57:54.166 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054513292999999997, 2024-09-14T23:57:54.166 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:54.166 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:54.166 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:54.166 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:54.166 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:54.166 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:54.166 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129176 2024-09-14T23:57:54.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.167 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:54.167 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.167 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:54.167 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931830+0000", 2024-09-14T23:57:54.167 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.167 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.167 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.167 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:54.167 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931830+0000", 2024-09-14T23:57:54.167 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.167 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.167 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.167 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:54.167 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.933678+0000", 2024-09-14T23:57:54.167 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018472440000000001 2024-09-14T23:57:54.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.168 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.168 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:54.168 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.933686+0000", 2024-09-14T23:57:54.168 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0239999999999997e-06 2024-09-14T23:57:54.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.168 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.168 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:54.168 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.933694+0000", 2024-09-14T23:57:54.168 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8400000000000001e-06 2024-09-14T23:57:54.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.168 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.168 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:54.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.933701+0000", 2024-09-14T23:57:54.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2870000000000001e-06 2024-09-14T23:57:54.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.169 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:54.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.986165+0000", 2024-09-14T23:57:54.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052463900000000001 2024-09-14T23:57:54.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.169 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:54.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.986259+0000", 2024-09-14T23:57:54.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4435e-05 2024-09-14T23:57:54.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.170 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:54.170 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.986344+0000", 2024-09-14T23:57:54.170 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4562999999999997e-05 2024-09-14T23:57:54.170 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.170 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:54.170 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.170 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.170 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129201 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 10289152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:54.170 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.935960+0000", 2024-09-14T23:57:54.170 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 15.192905506000001, 2024-09-14T23:57:54.170 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054338487999999997, 2024-09-14T23:57:54.170 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:54.170 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:54.170 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:54.171 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:54.171 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:54.171 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:54.171 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129201 2024-09-14T23:57:54.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.171 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:54.171 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.171 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:54.171 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935960+0000", 2024-09-14T23:57:54.171 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.171 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.171 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:54.171 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935960+0000", 2024-09-14T23:57:54.171 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.172 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.172 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.172 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:54.172 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935960+0000", 2024-09-14T23:57:54.172 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.72e-07 2024-09-14T23:57:54.172 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.172 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.172 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:54.172 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935963+0000", 2024-09-14T23:57:54.172 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5390000000000002e-06 2024-09-14T23:57:54.172 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.172 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.172 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:54.173 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935971+0000", 2024-09-14T23:57:54.173 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4589999999999998e-06 2024-09-14T23:57:54.173 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.173 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.173 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:54.173 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935977+0000", 2024-09-14T23:57:54.173 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4110000000000004e-06 2024-09-14T23:57:54.173 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.173 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.173 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:54.173 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990213+0000", 2024-09-14T23:57:54.174 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054236642000000002 2024-09-14T23:57:54.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.174 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.174 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:54.174 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990257+0000", 2024-09-14T23:57:54.174 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3696000000000003e-05 2024-09-14T23:57:54.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.174 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.174 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:54.174 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990298+0000", 2024-09-14T23:57:54.174 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1369000000000003e-05 2024-09-14T23:57:54.174 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.174 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:54.174 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.175 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.175 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129202 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 11370496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:54.175 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.935986+0000", 2024-09-14T23:57:54.175 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 15.192878887999999, 2024-09-14T23:57:54.175 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054543700000000001, 2024-09-14T23:57:54.175 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:54.175 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:54.175 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:54.175 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:54.175 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:54.176 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:54.176 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129202 2024-09-14T23:57:54.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.176 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:54.176 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.176 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:54.176 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935986+0000", 2024-09-14T23:57:54.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.176 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.176 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:54.176 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935986+0000", 2024-09-14T23:57:54.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.177 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:54.177 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937678+0000", 2024-09-14T23:57:54.177 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016917849999999999 2024-09-14T23:57:54.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.177 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:54.177 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937682+0000", 2024-09-14T23:57:54.177 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2400000000000001e-06 2024-09-14T23:57:54.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.177 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:54.177 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937695+0000", 2024-09-14T23:57:54.178 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2999e-05 2024-09-14T23:57:54.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.178 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:54.178 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937700+0000", 2024-09-14T23:57:54.178 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8420000000000004e-06 2024-09-14T23:57:54.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.178 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:54.178 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990336+0000", 2024-09-14T23:57:54.178 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052636060999999998 2024-09-14T23:57:54.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.178 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:54.179 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990435+0000", 2024-09-14T23:57:54.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9015e-05 2024-09-14T23:57:54.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.179 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.179 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:54.179 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990530+0000", 2024-09-14T23:57:54.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4758000000000003e-05 2024-09-14T23:57:54.179 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.179 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:54.179 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.179 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.180 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129214 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 11468800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:54.180 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.937939+0000", 2024-09-14T23:57:54.180 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 15.190925974000001, 2024-09-14T23:57:54.180 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054463733, 2024-09-14T23:57:54.180 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:54.180 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:54.180 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:54.180 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:54.180 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:54.180 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:54.180 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129214 2024-09-14T23:57:54.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.180 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:54.180 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.181 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:54.181 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937939+0000", 2024-09-14T23:57:54.181 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.181 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.181 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.181 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:54.181 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937939+0000", 2024-09-14T23:57:54.181 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.181 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.181 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.181 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:54.181 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939674+0000", 2024-09-14T23:57:54.181 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017342589999999999 2024-09-14T23:57:54.181 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.182 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.182 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:54.182 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939678+0000", 2024-09-14T23:57:54.182 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2590000000000002e-06 2024-09-14T23:57:54.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.182 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.182 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:54.182 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939699+0000", 2024-09-14T23:57:54.182 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1143000000000001e-05 2024-09-14T23:57:54.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.182 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.182 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:54.182 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939706+0000", 2024-09-14T23:57:54.182 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0269999999999998e-06 2024-09-14T23:57:54.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.183 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.183 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:54.183 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.992221+0000", 2024-09-14T23:57:54.183 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052514981000000002 2024-09-14T23:57:54.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.183 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.183 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:54.183 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.992321+0000", 2024-09-14T23:57:54.183 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9625999999999998e-05 2024-09-14T23:57:54.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.183 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.183 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:54.183 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.992403+0000", 2024-09-14T23:57:54.184 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2437999999999999e-05 2024-09-14T23:57:54.184 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.184 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:54.184 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.184 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129225 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 12648448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:54.184 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.939856+0000", 2024-09-14T23:57:54.184 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 15.189009417999999, 2024-09-14T23:57:54.184 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054305557999999997, 2024-09-14T23:57:54.185 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:54.185 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:54.185 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:54.185 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:54.185 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:54.185 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:54.185 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129225 2024-09-14T23:57:54.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.185 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:54.185 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.185 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:54.185 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939856+0000", 2024-09-14T23:57:54.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.185 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.186 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:54.186 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939856+0000", 2024-09-14T23:57:54.186 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.186 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.186 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:54.186 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939856+0000", 2024-09-14T23:57:54.186 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1199999999999999e-07 2024-09-14T23:57:54.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.186 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.186 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:54.186 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939857+0000", 2024-09-14T23:57:54.186 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.054e-06 2024-09-14T23:57:54.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.186 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.187 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:54.187 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939861+0000", 2024-09-14T23:57:54.187 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1459999999999998e-06 2024-09-14T23:57:54.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.187 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.187 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:54.187 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939864+0000", 2024-09-14T23:57:54.187 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7539999999999998e-06 2024-09-14T23:57:54.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.187 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.187 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:54.187 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.994078+0000", 2024-09-14T23:57:54.187 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054213483 2024-09-14T23:57:54.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.187 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.188 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:54.188 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.994119+0000", 2024-09-14T23:57:54.188 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1835000000000002e-05 2024-09-14T23:57:54.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.188 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.188 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:54.188 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.994161+0000", 2024-09-14T23:57:54.188 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2073999999999997e-05 2024-09-14T23:57:54.188 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.188 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:54.188 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.189 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.189 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129226 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 14090240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:54.189 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.939867+0000", 2024-09-14T23:57:54.189 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 15.188998046, 2024-09-14T23:57:54.189 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054523573999999998, 2024-09-14T23:57:54.189 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:54.189 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:54.189 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:54.189 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:54.189 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:54.189 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:54.189 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129226 2024-09-14T23:57:54.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.190 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:54.190 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.190 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:54.190 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939867+0000", 2024-09-14T23:57:54.190 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.190 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.190 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:54.190 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939867+0000", 2024-09-14T23:57:54.190 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.190 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.190 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:54.190 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941669+0000", 2024-09-14T23:57:54.191 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018021549999999999 2024-09-14T23:57:54.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.191 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.191 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:54.191 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941673+0000", 2024-09-14T23:57:54.191 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1269999999999999e-06 2024-09-14T23:57:54.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.191 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.191 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:54.191 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941679+0000", 2024-09-14T23:57:54.191 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1879999999999999e-06 2024-09-14T23:57:54.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.191 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.191 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:54.191 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941682+0000", 2024-09-14T23:57:54.192 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7110000000000001e-06 2024-09-14T23:57:54.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.192 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.192 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:54.192 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.994201+0000", 2024-09-14T23:57:54.192 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052518265000000001 2024-09-14T23:57:54.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.192 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.192 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:54.192 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.994302+0000", 2024-09-14T23:57:54.192 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101653 2024-09-14T23:57:54.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.192 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.192 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:54.193 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.994391+0000", 2024-09-14T23:57:54.193 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8474999999999996e-05 2024-09-14T23:57:54.193 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.193 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:54.193 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.193 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.193 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.193 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129237 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 12386304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:54.193 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.941825+0000", 2024-09-14T23:57:54.193 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 15.187040284, 2024-09-14T23:57:54.193 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054368076000000001, 2024-09-14T23:57:54.193 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:54.193 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:54.194 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:54.194 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:54.194 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:54.194 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:54.194 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129237 2024-09-14T23:57:54.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.194 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:54.194 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.194 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:54.194 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941825+0000", 2024-09-14T23:57:54.194 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.194 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.194 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:54.194 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941825+0000", 2024-09-14T23:57:54.195 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.195 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.195 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:54.195 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941825+0000", 2024-09-14T23:57:54.195 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.29e-07 2024-09-14T23:57:54.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.195 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.195 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:54.195 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941826+0000", 2024-09-14T23:57:54.195 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0999999999999997e-07 2024-09-14T23:57:54.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.195 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.196 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:54.196 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941834+0000", 2024-09-14T23:57:54.196 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5519999999999999e-06 2024-09-14T23:57:54.196 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.196 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.196 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:54.196 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941843+0000", 2024-09-14T23:57:54.196 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2960000000000004e-06 2024-09-14T23:57:54.196 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.196 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.196 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:54.197 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996108+0000", 2024-09-14T23:57:54.197 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054265200999999999 2024-09-14T23:57:54.197 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.197 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.197 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:54.197 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996152+0000", 2024-09-14T23:57:54.197 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4066999999999997e-05 2024-09-14T23:57:54.197 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.197 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.197 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:54.197 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996193+0000", 2024-09-14T23:57:54.197 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0821000000000001e-05 2024-09-14T23:57:54.197 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.197 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:54.198 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.198 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.198 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.198 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129238 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 13828096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:54.198 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.941851+0000", 2024-09-14T23:57:54.198 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 15.18701396, 2024-09-14T23:57:54.198 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.05458031, 2024-09-14T23:57:54.198 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:54.198 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:54.198 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:54.198 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:54.198 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:54.198 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:54.198 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129238 2024-09-14T23:57:54.198 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.199 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:54.199 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.199 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:54.199 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941851+0000", 2024-09-14T23:57:54.199 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.199 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.199 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:54.199 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941851+0000", 2024-09-14T23:57:54.199 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.199 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.199 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:54.199 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943670+0000", 2024-09-14T23:57:54.199 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018181930000000001 2024-09-14T23:57:54.200 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.200 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.200 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:54.200 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943673+0000", 2024-09-14T23:57:54.200 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3790000000000002e-06 2024-09-14T23:57:54.200 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.200 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.200 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:54.200 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943679+0000", 2024-09-14T23:57:54.200 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8719999999999999e-06 2024-09-14T23:57:54.200 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.200 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.200 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:54.201 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943683+0000", 2024-09-14T23:57:54.201 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8539999999999999e-06 2024-09-14T23:57:54.201 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.201 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.201 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:54.201 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996233+0000", 2024-09-14T23:57:54.201 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052550690999999997 2024-09-14T23:57:54.202 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.202 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.202 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:54.202 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996339+0000", 2024-09-14T23:57:54.202 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105989 2024-09-14T23:57:54.202 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.203 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.203 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:54.203 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996432+0000", 2024-09-14T23:57:54.203 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2331999999999996e-05 2024-09-14T23:57:54.203 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.203 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:54.204 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.204 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.204 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.204 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129249 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 12124160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:54.204 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.943826+0000", 2024-09-14T23:57:54.204 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 15.185039468999999, 2024-09-14T23:57:54.205 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054373266000000003, 2024-09-14T23:57:54.205 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:54.205 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:54.205 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:54.205 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:54.205 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:54.206 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:54.206 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129249 2024-09-14T23:57:54.206 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.206 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:54.206 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.206 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:54.207 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943826+0000", 2024-09-14T23:57:54.207 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.207 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.207 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:54.207 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943826+0000", 2024-09-14T23:57:54.208 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.208 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.208 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.208 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:54.208 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943826+0000", 2024-09-14T23:57:54.208 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.03e-07 2024-09-14T23:57:54.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.209 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.209 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:54.209 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943827+0000", 2024-09-14T23:57:54.209 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.08e-06 2024-09-14T23:57:54.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.210 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:54.210 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943834+0000", 2024-09-14T23:57:54.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4119999999999998e-06 2024-09-14T23:57:54.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.211 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:54.211 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943836+0000", 2024-09-14T23:57:54.211 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7369999999999998e-06 2024-09-14T23:57:54.211 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.211 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.211 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:54.212 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998115+0000", 2024-09-14T23:57:54.212 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054279040000000001 2024-09-14T23:57:54.212 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.212 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.212 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:54.212 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998159+0000", 2024-09-14T23:57:54.213 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3486999999999999e-05 2024-09-14T23:57:54.213 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.213 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.213 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:54.213 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998199+0000", 2024-09-14T23:57:54.213 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0306999999999999e-05 2024-09-14T23:57:54.214 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.214 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:54.214 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.214 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.214 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.214 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129250 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 13565952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:54.215 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.943840+0000", 2024-09-14T23:57:54.215 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 15.185025519, 2024-09-14T23:57:54.215 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054573875000000001, 2024-09-14T23:57:54.215 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:54.215 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:54.215 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:54.216 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:54.216 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:54.216 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:54.216 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129250 2024-09-14T23:57:54.216 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.217 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:54.217 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.217 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:54.217 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943840+0000", 2024-09-14T23:57:54.217 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.217 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.218 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.218 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:54.218 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943840+0000", 2024-09-14T23:57:54.218 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.218 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.218 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.219 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:54.219 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945669+0000", 2024-09-14T23:57:54.219 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018289720000000001 2024-09-14T23:57:54.219 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.219 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.219 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:54.220 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945671+0000", 2024-09-14T23:57:54.220 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6869999999999999e-06 2024-09-14T23:57:54.220 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.220 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.220 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:54.220 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945680+0000", 2024-09-14T23:57:54.221 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5259999999999993e-06 2024-09-14T23:57:54.221 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.221 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.221 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:54.221 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945685+0000", 2024-09-14T23:57:54.221 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8559999999999998e-06 2024-09-14T23:57:54.222 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.222 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.222 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:54.222 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998242+0000", 2024-09-14T23:57:54.222 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052557505999999997 2024-09-14T23:57:54.222 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.223 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.223 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:54.223 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998329+0000", 2024-09-14T23:57:54.223 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6410999999999996e-05 2024-09-14T23:57:54.223 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.224 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.224 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:54.224 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998414+0000", 2024-09-14T23:57:54.224 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4917e-05 2024-09-14T23:57:54.224 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.224 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:54.225 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.225 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.228 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.229 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129261 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 14385152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:54.229 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.945815+0000", 2024-09-14T23:57:54.229 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 15.183050226000001, 2024-09-14T23:57:54.229 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054331867999999998, 2024-09-14T23:57:54.229 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:54.229 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:54.230 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:54.230 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:54.230 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:54.230 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:54.230 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129261 2024-09-14T23:57:54.230 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.231 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:54.231 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.231 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:54.231 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945815+0000", 2024-09-14T23:57:54.231 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.231 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.232 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.232 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:54.232 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945815+0000", 2024-09-14T23:57:54.232 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.232 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.233 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.233 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:54.233 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945815+0000", 2024-09-14T23:57:54.233 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0599999999999999e-07 2024-09-14T23:57:54.233 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.233 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.234 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:54.234 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945816+0000", 2024-09-14T23:57:54.234 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0380000000000001e-06 2024-09-14T23:57:54.234 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.234 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.234 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:54.235 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945819+0000", 2024-09-14T23:57:54.235 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.005e-06 2024-09-14T23:57:54.235 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.235 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.235 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:54.235 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945822+0000", 2024-09-14T23:57:54.236 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.452e-06 2024-09-14T23:57:54.236 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.236 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.236 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:54.236 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000060+0000", 2024-09-14T23:57:54.236 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054238606000000002 2024-09-14T23:57:54.237 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.237 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.237 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:54.237 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000106+0000", 2024-09-14T23:57:54.237 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5188000000000002e-05 2024-09-14T23:57:54.237 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.238 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.238 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:54.238 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000147+0000", 2024-09-14T23:57:54.238 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1372999999999998e-05 2024-09-14T23:57:54.238 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.239 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:54.239 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.239 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.239 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.239 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129262 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 13303808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:54.239 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.945825+0000", 2024-09-14T23:57:54.240 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 15.183040578, 2024-09-14T23:57:54.240 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054515942999999997, 2024-09-14T23:57:54.240 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:54.240 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:54.240 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:54.240 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:54.241 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:54.241 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:54.241 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129262 2024-09-14T23:57:54.241 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.241 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:54.241 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.242 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:54.242 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945825+0000", 2024-09-14T23:57:54.242 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.242 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.242 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.242 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:54.243 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945825+0000", 2024-09-14T23:57:54.243 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.243 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.243 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.243 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:54.243 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.947670+0000", 2024-09-14T23:57:54.244 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018451229999999999 2024-09-14T23:57:54.244 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.244 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.244 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:54.244 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.947672+0000", 2024-09-14T23:57:54.244 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5289999999999998e-06 2024-09-14T23:57:54.245 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.245 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.245 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:54.245 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.947678+0000", 2024-09-14T23:57:54.245 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9880000000000001e-06 2024-09-14T23:57:54.245 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.246 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.246 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:54.246 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.947682+0000", 2024-09-14T23:57:54.246 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.63e-06 2024-09-14T23:57:54.246 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.246 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.247 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:54.247 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000183+0000", 2024-09-14T23:57:54.247 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052500770000000002 2024-09-14T23:57:54.247 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.247 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.248 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:54.248 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000266+0000", 2024-09-14T23:57:54.248 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3133000000000005e-05 2024-09-14T23:57:54.248 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.248 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.249 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:54.249 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000341+0000", 2024-09-14T23:57:54.249 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4770000000000004e-05 2024-09-14T23:57:54.249 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.249 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:54.250 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.250 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.250 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129299 3.10 3:0f1e06ee:::rbd_object_map.1118721be852:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:54.250 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.952330+0000", 2024-09-14T23:57:54.250 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 15.176535765000001, 2024-09-14T23:57:54.251 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054412818000000002, 2024-09-14T23:57:54.251 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:54.251 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:54.251 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:54.251 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:54.251 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:54.252 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:54.252 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129299 2024-09-14T23:57:54.252 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.252 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:54.252 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.252 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:54.253 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952330+0000", 2024-09-14T23:57:54.253 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.253 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.253 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.253 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:54.254 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952330+0000", 2024-09-14T23:57:54.254 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.254 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.254 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.254 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:54.254 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952330+0000", 2024-09-14T23:57:54.255 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4900000000000001e-07 2024-09-14T23:57:54.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.255 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:54.255 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952332+0000", 2024-09-14T23:57:54.256 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2879999999999999e-06 2024-09-14T23:57:54.256 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.256 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.256 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:54.256 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952340+0000", 2024-09-14T23:57:54.256 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9910000000000002e-06 2024-09-14T23:57:54.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.257 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.257 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:54.257 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952344+0000", 2024-09-14T23:57:54.257 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0280000000000002e-06 2024-09-14T23:57:54.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.258 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.258 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:54.258 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.004722+0000", 2024-09-14T23:57:54.258 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052377932000000002 2024-09-14T23:57:54.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.258 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.259 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:54.259 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.004778+0000", 2024-09-14T23:57:54.259 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5795999999999999e-05 2024-09-14T23:57:54.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.259 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.259 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:57:54.260 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.005016+0000", 2024-09-14T23:57:54.260 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000238006 2024-09-14T23:57:54.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.260 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.260 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:57:54.261 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.006366+0000", 2024-09-14T23:57:54.261 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001349786 2024-09-14T23:57:54.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.261 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.261 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:54.261 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.006685+0000", 2024-09-14T23:57:54.262 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00031970000000000002 2024-09-14T23:57:54.262 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.262 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.262 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:54.262 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.006718+0000", 2024-09-14T23:57:54.263 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2329000000000002e-05 2024-09-14T23:57:54.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.263 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:54.263 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.006742+0000", 2024-09-14T23:57:54.263 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4613e-05 2024-09-14T23:57:54.264 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.264 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:54.264 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.264 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.264 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:54.265 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:54.268 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone42 from 0xed85209 (overlap 0xd41347) to 0xbede26 2024-09-14T23:57:54.268 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #41, image image_client.0-clone41 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent42 2024-09-14T23:57:54.269 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:54.269 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:54.269 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:54.269 INFO:teuthology.orchestra.run.smithi042.stdout:5090 write 0x7cc2bf7 thru 0x7ccef9c (0xc3a6 bytes) 2024-09-14T23:57:54.269 INFO:teuthology.orchestra.run.smithi042.stdout:5092 write 0xaa5d50a thru 0xaa6ac1c (0xd713 bytes) 2024-09-14T23:57:54.269 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:54.265+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:54.270 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T23:57:54.270 INFO:teuthology.orchestra.run.smithi042.stdout:5093 trunc from 0xed85209 to 0xdf3adf2 2024-09-14T23:57:54.284 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:54.284 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:54.284 INFO:teuthology.orchestra.run.smithi042.stdout:5097 read 0xc900e9a thru 0xc900efb (0x62 bytes) 2024-09-14T23:57:54.284 INFO:teuthology.orchestra.run.smithi042.stdout:5098 read 0xcbc6411 thru 0xcbd5cd4 (0xf8c4 bytes) 2024-09-14T23:57:54.284 INFO:teuthology.orchestra.run.smithi042.stdout:5099 read 0x5d2e211 thru 0x5d35664 (0x7454 bytes) 2024-09-14T23:57:54.284 INFO:teuthology.orchestra.run.smithi042.stdout:5100 write 0x5321cb thru 0x536579 (0x43af bytes) 2024-09-14T23:57:54.291 INFO:teuthology.orchestra.run.smithi042.stdout:5101 punch from 0x634ece5 to 0x635ce14, (0xe12f bytes) 2024-09-14T23:57:54.291 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:54.291 INFO:teuthology.orchestra.run.smithi042.stdout:5105 read 0x765aef thru 0x76cdf0 (0x7302 bytes) 2024-09-14T23:57:54.291 INFO:teuthology.orchestra.run.smithi042.stdout:5106 read 0xa399b03 thru 0xa3a3fb7 (0xa4b5 bytes) 2024-09-14T23:57:54.292 INFO:teuthology.orchestra.run.smithi042.stdout:5107 read 0x37aa6c6 thru 0x37b9b02 (0xf43d bytes) 2024-09-14T23:57:54.292 INFO:teuthology.orchestra.run.smithi042.stdout:5108 writesame 0xbc2f2b8 thru 0xbc36fa9 data_size 0x3(0x7cf2 bytes) 2024-09-14T23:57:54.298 INFO:teuthology.orchestra.run.smithi042.stdout:5111 writesame 0xc1b4ba thru 0xc1df95 data_size 0x1a(0x2adc bytes) 2024-09-14T23:57:54.304 INFO:teuthology.orchestra.run.smithi042.stdout:5112 write 0x119d089 thru 0x11a64e7 (0x945f bytes) 2024-09-14T23:57:54.309 INFO:teuthology.orchestra.run.smithi042.stdout:5113 read 0xc744719 thru 0xc748ee3 (0x47cb bytes) 2024-09-14T23:57:54.309 INFO:teuthology.orchestra.run.smithi042.stdout:5114 read 0xb831a25 thru 0xb83b202 (0x97de bytes) 2024-09-14T23:57:54.309 INFO:teuthology.orchestra.run.smithi042.stdout:5115 writesame 0x9b39f63 thru 0x9b3e270 data_size 0x3(0x430e bytes) 2024-09-14T23:57:54.314 INFO:teuthology.orchestra.run.smithi042.stdout:5116 write 0x5b49f8c thru 0x5b4a9e0 (0xa55 bytes) 2024-09-14T23:57:54.319 INFO:teuthology.orchestra.run.smithi042.stdout:5117 punch from 0x773ed9a to 0x774c9e1, (0xdc47 bytes) 2024-09-14T23:57:54.319 INFO:teuthology.orchestra.run.smithi042.stdout:5118 read 0xdb11c5 thru 0xdbb0ed (0x9f29 bytes) 2024-09-14T23:57:54.319 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:54.319 INFO:teuthology.orchestra.run.smithi042.stdout:5120 punch from 0x6a033c4 to 0x6a1094a, (0xd586 bytes) 2024-09-14T23:57:54.319 INFO:teuthology.orchestra.run.smithi042.stdout:5121 read 0xb1e016d thru 0xb1ed0c4 (0xcf58 bytes) 2024-09-14T23:57:54.320 INFO:teuthology.orchestra.run.smithi042.stdout:5123 read 0x1420ce5 thru 0x142145c (0x778 bytes) 2024-09-14T23:57:54.328 INFO:teuthology.orchestra.run.smithi042.stdout:5124 trunc from 0xdf3adf2 to 0x60ab170 2024-09-14T23:57:54.353 INFO:teuthology.orchestra.run.smithi042.stdout:5125 punch from 0x3e7e5e7 to 0x3e8b02a, (0xca43 bytes) 2024-09-14T23:57:54.353 INFO:teuthology.orchestra.run.smithi042.stdout:5126 write 0x52d462e thru 0x52d4af3 (0x4c6 bytes) 2024-09-14T23:57:54.356 INFO:teuthology.orchestra.run.smithi042.stdout:5127 clone 44 order 25 su 2097152 sc 7 2024-09-14T23:57:54.367 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:54.365+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:54.392 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:54.392 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:54.392 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:57:54.392 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:54.401 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T23:57:54.485 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:54.483+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:54.548 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:54.548 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:54.548 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:57:54.549 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:57:54.549 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:54.557 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T23:57:54.598 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:54.596+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:54.669 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:54.668+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:54.706 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:54.706 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:57:54.706 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:57:54.706 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:57:54.706 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.707 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69262 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 4800512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:54.707 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300811+0000", 2024-09-14T23:57:54.707 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 39.403280752000001, 2024-09-14T23:57:54.707 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050403796000000001, 2024-09-14T23:57:54.707 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:54.707 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:54.707 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:54.707 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:54.707 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:54.707 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:54.707 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69262 2024-09-14T23:57:54.707 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.707 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:54.707 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.708 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:54.708 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300811+0000", 2024-09-14T23:57:54.708 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.708 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.708 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.708 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:54.708 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300811+0000", 2024-09-14T23:57:54.708 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.708 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.708 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.708 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:54.708 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300811+0000", 2024-09-14T23:57:54.708 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5700000000000002e-07 2024-09-14T23:57:54.709 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.709 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.709 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:54.709 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300815+0000", 2024-09-14T23:57:54.709 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.087e-06 2024-09-14T23:57:54.709 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.709 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.709 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:54.709 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300823+0000", 2024-09-14T23:57:54.709 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8720000000000002e-06 2024-09-14T23:57:54.709 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.709 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.709 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:54.709 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300830+0000", 2024-09-14T23:57:54.710 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0589999999999997e-06 2024-09-14T23:57:54.710 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.710 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.710 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:54.710 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351020+0000", 2024-09-14T23:57:54.710 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050189552999999998 2024-09-14T23:57:54.710 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.710 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.710 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:54.710 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351112+0000", 2024-09-14T23:57:54.710 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2615999999999996e-05 2024-09-14T23:57:54.710 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.710 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.710 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:54.711 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351214+0000", 2024-09-14T23:57:54.711 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101952 2024-09-14T23:57:54.711 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.711 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:54.711 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.711 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.711 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.711 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69263 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 4915200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:54.711 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300842+0000", 2024-09-14T23:57:54.711 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 39.403249248000002, 2024-09-14T23:57:54.711 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050590637000000001, 2024-09-14T23:57:54.711 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:54.711 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:54.711 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:54.712 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:54.712 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:54.712 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:54.712 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69263 2024-09-14T23:57:54.712 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.712 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:54.712 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.712 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:54.712 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300842+0000", 2024-09-14T23:57:54.712 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.712 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.712 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.712 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:54.713 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300842+0000", 2024-09-14T23:57:54.713 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.713 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.713 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:54.713 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300843+0000", 2024-09-14T23:57:54.713 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4000000000000002e-07 2024-09-14T23:57:54.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.713 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.713 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:54.713 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300846+0000", 2024-09-14T23:57:54.713 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8720000000000002e-06 2024-09-14T23:57:54.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.713 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.714 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:54.714 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300853+0000", 2024-09-14T23:57:54.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0450000000000003e-06 2024-09-14T23:57:54.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.714 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.714 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:54.714 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300859+0000", 2024-09-14T23:57:54.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9510000000000003e-06 2024-09-14T23:57:54.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.714 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.714 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:54.714 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351260+0000", 2024-09-14T23:57:54.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050401654999999997 2024-09-14T23:57:54.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.715 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.715 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:54.715 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351353+0000", 2024-09-14T23:57:54.715 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2785e-05 2024-09-14T23:57:54.715 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.715 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.715 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:54.715 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351433+0000", 2024-09-14T23:57:54.715 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9789000000000005e-05 2024-09-14T23:57:54.715 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.715 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:54.715 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.715 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.716 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.716 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69264 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5029888~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:54.716 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300866+0000", 2024-09-14T23:57:54.716 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 39.403225315, 2024-09-14T23:57:54.716 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050794063, 2024-09-14T23:57:54.716 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:54.716 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:54.716 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:54.716 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:54.716 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:54.716 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:54.716 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69264 2024-09-14T23:57:54.716 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.716 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:54.717 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.717 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:54.717 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300866+0000", 2024-09-14T23:57:54.717 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.717 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.717 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.717 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:54.717 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300866+0000", 2024-09-14T23:57:54.717 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.717 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.717 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.717 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:54.717 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300866+0000", 2024-09-14T23:57:54.717 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8799999999999998e-07 2024-09-14T23:57:54.718 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.718 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.718 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:54.718 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300869+0000", 2024-09-14T23:57:54.718 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0880000000000002e-06 2024-09-14T23:57:54.718 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.718 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.718 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:54.718 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300875+0000", 2024-09-14T23:57:54.718 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9280000000000004e-06 2024-09-14T23:57:54.718 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.718 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.718 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:54.718 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300882+0000", 2024-09-14T23:57:54.719 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4740000000000002e-06 2024-09-14T23:57:54.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.719 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.719 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:54.719 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351472+0000", 2024-09-14T23:57:54.719 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050590006999999999 2024-09-14T23:57:54.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.719 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.719 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:54.719 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351563+0000", 2024-09-14T23:57:54.719 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0762999999999998e-05 2024-09-14T23:57:54.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.719 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.720 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:54.720 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351660+0000", 2024-09-14T23:57:54.720 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7415000000000002e-05 2024-09-14T23:57:54.720 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.721 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:54.721 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.721 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.721 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.721 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69265 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5144576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:54.721 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300890+0000", 2024-09-14T23:57:54.721 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 39.403201811000002, 2024-09-14T23:57:54.721 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050921389999999997, 2024-09-14T23:57:54.721 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:54.721 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:54.721 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:54.721 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:54.722 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:54.722 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:54.722 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69265 2024-09-14T23:57:54.722 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.722 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:54.722 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.722 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:54.722 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300890+0000", 2024-09-14T23:57:54.722 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.722 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.722 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.722 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:54.722 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300890+0000", 2024-09-14T23:57:54.723 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.723 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.723 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.723 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:54.723 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300890+0000", 2024-09-14T23:57:54.723 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2500000000000001e-07 2024-09-14T23:57:54.723 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.723 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.723 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:54.723 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300892+0000", 2024-09-14T23:57:54.723 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2469999999999998e-06 2024-09-14T23:57:54.723 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.723 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.723 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:54.724 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300899+0000", 2024-09-14T23:57:54.724 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5509999999999996e-06 2024-09-14T23:57:54.724 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.724 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.724 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:54.724 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300904+0000", 2024-09-14T23:57:54.724 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3299999999999998e-06 2024-09-14T23:57:54.724 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.724 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.724 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:54.724 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351707+0000", 2024-09-14T23:57:54.724 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050802673999999999 2024-09-14T23:57:54.724 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.724 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.725 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:54.725 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351755+0000", 2024-09-14T23:57:54.725 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8362000000000002e-05 2024-09-14T23:57:54.725 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.725 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.725 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:54.725 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351811+0000", 2024-09-14T23:57:54.725 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5801000000000003e-05 2024-09-14T23:57:54.725 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.725 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:54.725 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.725 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.725 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.725 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69266 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5259264~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:54.726 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300911+0000", 2024-09-14T23:57:54.726 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 39.403180738000003, 2024-09-14T23:57:54.726 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051125089999999998, 2024-09-14T23:57:54.726 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:54.726 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:54.726 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:54.726 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:54.726 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:54.726 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:54.726 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69266 2024-09-14T23:57:54.726 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.726 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:54.726 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.726 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:54.727 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300911+0000", 2024-09-14T23:57:54.727 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.727 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.727 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.727 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:54.727 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300911+0000", 2024-09-14T23:57:54.727 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.727 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.727 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.727 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:54.727 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300911+0000", 2024-09-14T23:57:54.727 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-14T23:57:54.727 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.727 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.727 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:54.728 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300913+0000", 2024-09-14T23:57:54.728 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9300000000000002e-06 2024-09-14T23:57:54.728 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.728 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.728 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:54.728 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300919+0000", 2024-09-14T23:57:54.728 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.489e-06 2024-09-14T23:57:54.728 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.728 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.728 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:54.728 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300925+0000", 2024-09-14T23:57:54.728 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1719999999999998e-06 2024-09-14T23:57:54.728 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.728 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.729 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:54.729 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351847+0000", 2024-09-14T23:57:54.729 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050922621000000001 2024-09-14T23:57:54.729 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.729 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.729 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:54.729 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351938+0000", 2024-09-14T23:57:54.729 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0830999999999998e-05 2024-09-14T23:57:54.729 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.729 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.729 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:54.729 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352036+0000", 2024-09-14T23:57:54.729 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7676999999999993e-05 2024-09-14T23:57:54.729 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.730 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:54.730 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.730 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.730 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.730 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69267 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5373952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:54.730 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300945+0000", 2024-09-14T23:57:54.730 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 39.403145942999998, 2024-09-14T23:57:54.730 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051217934, 2024-09-14T23:57:54.730 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:54.730 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:54.730 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:54.730 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:54.730 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:54.730 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:54.730 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69267 2024-09-14T23:57:54.731 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.731 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:54.731 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.731 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:54.731 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300945+0000", 2024-09-14T23:57:54.731 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.731 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.731 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.731 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:54.731 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300945+0000", 2024-09-14T23:57:54.731 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.731 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.731 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.731 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:54.732 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300946+0000", 2024-09-14T23:57:54.732 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2600000000000002e-07 2024-09-14T23:57:54.732 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.732 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.732 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:54.732 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300948+0000", 2024-09-14T23:57:54.732 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4940000000000002e-06 2024-09-14T23:57:54.732 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.732 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.732 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:54.732 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300956+0000", 2024-09-14T23:57:54.732 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.622e-06 2024-09-14T23:57:54.732 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.732 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.733 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:54.733 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300962+0000", 2024-09-14T23:57:54.733 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5790000000000003e-06 2024-09-14T23:57:54.733 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.733 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.733 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:54.733 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352063+0000", 2024-09-14T23:57:54.733 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051101281999999998 2024-09-14T23:57:54.733 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.733 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.733 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:54.733 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352110+0000", 2024-09-14T23:57:54.733 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7401000000000002e-05 2024-09-14T23:57:54.734 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.734 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.734 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:54.734 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352163+0000", 2024-09-14T23:57:54.734 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3029999999999999e-05 2024-09-14T23:57:54.734 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.734 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:54.734 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.734 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.734 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.734 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69268 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5488640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:54.734 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300969+0000", 2024-09-14T23:57:54.734 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 39.403122271000001, 2024-09-14T23:57:54.735 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051413420000000001, 2024-09-14T23:57:54.735 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:54.735 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:54.735 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:54.735 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:54.735 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:54.735 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:54.735 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69268 2024-09-14T23:57:54.735 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.735 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:54.735 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.735 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:54.735 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300969+0000", 2024-09-14T23:57:54.735 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.736 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.736 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.736 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:54.736 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300969+0000", 2024-09-14T23:57:54.736 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.736 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.736 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.736 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:54.736 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300970+0000", 2024-09-14T23:57:54.736 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7899999999999999e-07 2024-09-14T23:57:54.736 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.736 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.736 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:54.736 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300982+0000", 2024-09-14T23:57:54.737 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2741e-05 2024-09-14T23:57:54.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.737 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.737 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:54.737 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300990+0000", 2024-09-14T23:57:54.737 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8169999999999999e-06 2024-09-14T23:57:54.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.737 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.737 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:54.737 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300995+0000", 2024-09-14T23:57:54.737 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2240000000000003e-06 2024-09-14T23:57:54.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.737 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.738 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:54.738 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352200+0000", 2024-09-14T23:57:54.738 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051204223 2024-09-14T23:57:54.738 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.738 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.738 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:54.738 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352285+0000", 2024-09-14T23:57:54.738 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5519999999999994e-05 2024-09-14T23:57:54.738 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.738 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.738 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:54.738 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352383+0000", 2024-09-14T23:57:54.738 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7516000000000006e-05 2024-09-14T23:57:54.739 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.739 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:54.739 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.739 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.739 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.739 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69269 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5603328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:54.739 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.301003+0000", 2024-09-14T23:57:54.739 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 39.403088089000001, 2024-09-14T23:57:54.739 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051497381000000002, 2024-09-14T23:57:54.739 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:54.739 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:54.739 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:54.739 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:54.739 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:54.740 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:54.740 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69269 2024-09-14T23:57:54.740 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.740 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:54.740 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.740 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:54.740 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.301003+0000", 2024-09-14T23:57:54.740 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.740 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.740 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.740 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:54.740 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.301003+0000", 2024-09-14T23:57:54.740 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.740 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.741 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.741 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:54.741 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302671+0000", 2024-09-14T23:57:54.741 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001667785 2024-09-14T23:57:54.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.741 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.741 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:54.741 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302676+0000", 2024-09-14T23:57:54.741 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6600000000000003e-06 2024-09-14T23:57:54.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.741 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.741 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:54.741 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302697+0000", 2024-09-14T23:57:54.741 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1288999999999999e-05 2024-09-14T23:57:54.742 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.742 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.742 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:54.742 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302716+0000", 2024-09-14T23:57:54.742 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9128999999999999e-05 2024-09-14T23:57:54.742 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.742 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.742 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:54.742 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352399+0000", 2024-09-14T23:57:54.742 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049682995000000001 2024-09-14T23:57:54.742 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.742 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.742 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:54.742 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352446+0000", 2024-09-14T23:57:54.743 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6341000000000002e-05 2024-09-14T23:57:54.743 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.743 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.743 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:54.743 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352501+0000", 2024-09-14T23:57:54.743 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5182000000000001e-05 2024-09-14T23:57:54.743 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.743 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:54.743 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.743 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.743 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.743 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69273 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6062080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:54.743 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302829+0000", 2024-09-14T23:57:54.743 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 39.401262523, 2024-09-14T23:57:54.744 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050384311000000001, 2024-09-14T23:57:54.744 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:54.744 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:54.744 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:54.744 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:54.744 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:54.744 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:54.744 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69273 2024-09-14T23:57:54.744 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.744 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:54.744 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.744 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:54.744 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302829+0000", 2024-09-14T23:57:54.745 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.745 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.745 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:54.745 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302829+0000", 2024-09-14T23:57:54.745 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.745 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.745 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:54.745 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302829+0000", 2024-09-14T23:57:54.745 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7700000000000005e-07 2024-09-14T23:57:54.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.745 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.745 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:54.746 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302832+0000", 2024-09-14T23:57:54.746 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6970000000000002e-06 2024-09-14T23:57:54.746 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.746 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.746 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:54.746 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302840+0000", 2024-09-14T23:57:54.746 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5719999999999997e-06 2024-09-14T23:57:54.746 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.746 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.746 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:54.746 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302845+0000", 2024-09-14T23:57:54.746 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7939999999999999e-06 2024-09-14T23:57:54.746 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.746 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.747 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:54.747 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353111+0000", 2024-09-14T23:57:54.747 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050265588 2024-09-14T23:57:54.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.747 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.747 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:54.747 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353160+0000", 2024-09-14T23:57:54.747 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8859000000000001e-05 2024-09-14T23:57:54.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.747 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.747 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:54.747 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353213+0000", 2024-09-14T23:57:54.747 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3324e-05 2024-09-14T23:57:54.747 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.748 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:54.748 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.748 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.748 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.748 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69274 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6176768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:54.748 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302854+0000", 2024-09-14T23:57:54.748 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 39.401237428999998, 2024-09-14T23:57:54.748 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050579526, 2024-09-14T23:57:54.748 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:54.748 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:54.748 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:54.748 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:54.748 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:54.748 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:54.749 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69274 2024-09-14T23:57:54.749 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.749 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:54.749 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.749 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:54.749 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302854+0000", 2024-09-14T23:57:54.749 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.749 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.749 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.749 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:54.749 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302854+0000", 2024-09-14T23:57:54.749 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.749 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.750 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.750 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:54.750 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302855+0000", 2024-09-14T23:57:54.750 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0200000000000001e-07 2024-09-14T23:57:54.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.750 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.750 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:54.750 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302857+0000", 2024-09-14T23:57:54.750 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.447e-06 2024-09-14T23:57:54.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.751 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.751 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:54.751 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302864+0000", 2024-09-14T23:57:54.751 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7730000000000002e-06 2024-09-14T23:57:54.751 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.751 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.751 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:54.751 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302869+0000", 2024-09-14T23:57:54.751 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3199999999999999e-06 2024-09-14T23:57:54.751 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.751 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.751 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:54.751 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353249+0000", 2024-09-14T23:57:54.751 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050379647999999999 2024-09-14T23:57:54.752 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.752 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.752 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:54.752 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353336+0000", 2024-09-14T23:57:54.752 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6767999999999993e-05 2024-09-14T23:57:54.752 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.752 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.752 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:54.752 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353433+0000", 2024-09-14T23:57:54.752 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7868000000000006e-05 2024-09-14T23:57:54.752 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.752 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:54.752 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.752 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.753 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.753 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69275 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6291456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:54.753 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302889+0000", 2024-09-14T23:57:54.753 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 39.401202810999997, 2024-09-14T23:57:54.753 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050663118, 2024-09-14T23:57:54.753 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:54.753 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:54.753 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:54.753 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:54.753 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:54.753 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:54.753 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69275 2024-09-14T23:57:54.753 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.753 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:54.754 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.754 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:54.754 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302889+0000", 2024-09-14T23:57:54.754 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.754 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.754 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:54.754 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302889+0000", 2024-09-14T23:57:54.754 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.754 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.754 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:54.754 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302889+0000", 2024-09-14T23:57:54.754 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6399999999999998e-07 2024-09-14T23:57:54.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.755 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:54.755 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302891+0000", 2024-09-14T23:57:54.755 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.339e-06 2024-09-14T23:57:54.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.755 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:54.755 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302898+0000", 2024-09-14T23:57:54.755 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0380000000000002e-06 2024-09-14T23:57:54.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.755 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:54.755 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302904+0000", 2024-09-14T23:57:54.755 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2159999999999997e-06 2024-09-14T23:57:54.756 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.756 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.756 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:54.756 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353449+0000", 2024-09-14T23:57:54.756 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050545386999999997 2024-09-14T23:57:54.756 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.756 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.756 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:54.756 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353494+0000", 2024-09-14T23:57:54.756 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4592999999999998e-05 2024-09-14T23:57:54.756 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.756 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.756 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:54.756 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353552+0000", 2024-09-14T23:57:54.757 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8180999999999997e-05 2024-09-14T23:57:54.757 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.757 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:54.757 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.757 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.757 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.757 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69276 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6406144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:54.757 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302911+0000", 2024-09-14T23:57:54.757 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 39.401180398000001, 2024-09-14T23:57:54.757 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050865753, 2024-09-14T23:57:54.757 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:54.757 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:54.757 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:54.757 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:54.758 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:54.758 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:54.758 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69276 2024-09-14T23:57:54.758 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.758 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:54.758 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.758 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:54.758 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302911+0000", 2024-09-14T23:57:54.758 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.758 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.758 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.758 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:54.758 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302911+0000", 2024-09-14T23:57:54.758 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.759 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.759 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.759 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:54.759 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302912+0000", 2024-09-14T23:57:54.759 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0800000000000004e-07 2024-09-14T23:57:54.759 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.759 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.759 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:54.759 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302914+0000", 2024-09-14T23:57:54.759 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9589999999999998e-06 2024-09-14T23:57:54.759 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.759 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.759 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:54.759 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302920+0000", 2024-09-14T23:57:54.760 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4579999999999998e-06 2024-09-14T23:57:54.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.760 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.760 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:54.760 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302925+0000", 2024-09-14T23:57:54.760 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9429999999999999e-06 2024-09-14T23:57:54.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.760 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.760 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:54.760 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353586+0000", 2024-09-14T23:57:54.760 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050660966000000002 2024-09-14T23:57:54.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.760 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.760 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:54.761 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353679+0000", 2024-09-14T23:57:54.761 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3163000000000002e-05 2024-09-14T23:57:54.761 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.761 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.761 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:54.761 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353777+0000", 2024-09-14T23:57:54.761 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7555999999999999e-05 2024-09-14T23:57:54.761 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.761 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:54.761 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.761 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.761 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.761 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69277 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6520832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:54.761 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302932+0000", 2024-09-14T23:57:54.762 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 39.401159311000001, 2024-09-14T23:57:54.762 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050977674000000001, 2024-09-14T23:57:54.762 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:54.762 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:54.762 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:54.762 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:54.762 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:54.762 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:54.762 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69277 2024-09-14T23:57:54.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.762 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:54.762 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.762 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:54.762 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302932+0000", 2024-09-14T23:57:54.763 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.763 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:54.763 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302932+0000", 2024-09-14T23:57:54.763 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.763 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:54.763 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302932+0000", 2024-09-14T23:57:54.763 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3299999999999998e-07 2024-09-14T23:57:54.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.764 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:54.764 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302935+0000", 2024-09-14T23:57:54.764 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2139999999999999e-06 2024-09-14T23:57:54.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.764 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.764 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:54.764 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302941+0000", 2024-09-14T23:57:54.764 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5039999999999999e-06 2024-09-14T23:57:54.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.764 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.764 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:54.764 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302946+0000", 2024-09-14T23:57:54.764 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2079999999999999e-06 2024-09-14T23:57:54.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.765 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.765 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:54.765 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353805+0000", 2024-09-14T23:57:54.765 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050858588000000003 2024-09-14T23:57:54.765 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.765 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.765 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:54.765 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353849+0000", 2024-09-14T23:57:54.765 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4511000000000001e-05 2024-09-14T23:57:54.765 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.765 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.765 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:54.766 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353910+0000", 2024-09-14T23:57:54.766 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0316000000000003e-05 2024-09-14T23:57:54.766 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.766 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:54.766 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.766 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.766 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.766 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69278 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6635520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:54.766 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302953+0000", 2024-09-14T23:57:54.766 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 39.401138183, 2024-09-14T23:57:54.766 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051174059000000001, 2024-09-14T23:57:54.766 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:54.766 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:54.766 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:54.767 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:54.767 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:54.767 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:54.767 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69278 2024-09-14T23:57:54.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.767 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:54.767 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.767 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:54.767 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302953+0000", 2024-09-14T23:57:54.767 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.767 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.767 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:54.768 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302953+0000", 2024-09-14T23:57:54.768 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.768 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.768 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:54.768 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302954+0000", 2024-09-14T23:57:54.768 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5100000000000001e-07 2024-09-14T23:57:54.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.768 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.768 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:54.768 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302964+0000", 2024-09-14T23:57:54.768 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0475000000000001e-05 2024-09-14T23:57:54.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.768 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.769 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:54.769 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302970+0000", 2024-09-14T23:57:54.769 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3879999999999997e-06 2024-09-14T23:57:54.769 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.769 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.769 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:54.769 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302975+0000", 2024-09-14T23:57:54.769 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8409999999999999e-06 2024-09-14T23:57:54.769 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.769 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.769 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:54.769 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353944+0000", 2024-09-14T23:57:54.769 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050968952999999997 2024-09-14T23:57:54.769 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.770 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.770 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:54.770 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354031+0000", 2024-09-14T23:57:54.770 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6537000000000002e-05 2024-09-14T23:57:54.770 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.770 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.770 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:54.770 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354127+0000", 2024-09-14T23:57:54.770 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6514000000000005e-05 2024-09-14T23:57:54.770 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.770 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:54.770 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.770 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.770 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.771 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69279 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6750208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:54.771 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302984+0000", 2024-09-14T23:57:54.771 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 39.401107832000001, 2024-09-14T23:57:54.771 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051258845999999997, 2024-09-14T23:57:54.771 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:54.771 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:54.771 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:54.771 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:54.771 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:54.771 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:54.771 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69279 2024-09-14T23:57:54.771 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.771 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:54.771 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.771 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:54.772 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302984+0000", 2024-09-14T23:57:54.772 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.772 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.772 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.772 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:54.772 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302984+0000", 2024-09-14T23:57:54.772 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.772 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.772 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.772 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:54.772 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303025+0000", 2024-09-14T23:57:54.772 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1690999999999998e-05 2024-09-14T23:57:54.772 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.772 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.773 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:54.773 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303029+0000", 2024-09-14T23:57:54.773 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6559999999999998e-06 2024-09-14T23:57:54.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.773 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.773 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:54.773 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303037+0000", 2024-09-14T23:57:54.773 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7950000000000008e-06 2024-09-14T23:57:54.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.773 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.773 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:54.773 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303043+0000", 2024-09-14T23:57:54.773 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9669999999999998e-06 2024-09-14T23:57:54.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.774 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.774 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:54.774 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354142+0000", 2024-09-14T23:57:54.774 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051099072000000002 2024-09-14T23:57:54.774 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.774 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.774 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:54.774 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354183+0000", 2024-09-14T23:57:54.774 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0834000000000002e-05 2024-09-14T23:57:54.774 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.774 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.774 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:54.774 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354242+0000", 2024-09-14T23:57:54.774 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9830999999999997e-05 2024-09-14T23:57:54.774 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.775 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:54.775 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.775 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.775 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.775 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69280 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6864896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:54.775 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.303052+0000", 2024-09-14T23:57:54.775 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 39.401039289000003, 2024-09-14T23:57:54.775 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051406463999999999, 2024-09-14T23:57:54.775 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:54.775 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:54.775 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:54.775 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:54.775 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:54.776 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:54.776 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69280 2024-09-14T23:57:54.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.776 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:54.776 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.776 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:54.776 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303052+0000", 2024-09-14T23:57:54.776 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.776 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.776 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:54.776 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303052+0000", 2024-09-14T23:57:54.776 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.777 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.777 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:54.777 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304673+0000", 2024-09-14T23:57:54.777 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001621389 2024-09-14T23:57:54.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.777 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.777 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:54.777 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304681+0000", 2024-09-14T23:57:54.777 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8439999999999998e-06 2024-09-14T23:57:54.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.777 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.777 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:54.777 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304691+0000", 2024-09-14T23:57:54.777 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8339999999999994e-06 2024-09-14T23:57:54.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.778 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.778 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:54.778 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304698+0000", 2024-09-14T23:57:54.778 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3629999999999999e-06 2024-09-14T23:57:54.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.778 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.778 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:54.778 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354278+0000", 2024-09-14T23:57:54.778 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049580618 2024-09-14T23:57:54.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.778 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.778 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:54.778 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354364+0000", 2024-09-14T23:57:54.779 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6059999999999999e-05 2024-09-14T23:57:54.779 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.779 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.779 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:54.779 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354459+0000", 2024-09-14T23:57:54.779 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4356000000000003e-05 2024-09-14T23:57:54.779 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.779 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:54.779 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.779 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.779 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.779 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69285 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 7438336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:54.779 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.304792+0000", 2024-09-14T23:57:54.779 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 39.399299880000001, 2024-09-14T23:57:54.780 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050398874000000003, 2024-09-14T23:57:54.780 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:54.780 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:54.780 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:54.780 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:54.780 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:54.780 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:54.780 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69285 2024-09-14T23:57:54.780 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.780 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:54.780 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.780 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:54.780 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304792+0000", 2024-09-14T23:57:54.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.781 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:54.781 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304792+0000", 2024-09-14T23:57:54.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.781 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:54.781 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304792+0000", 2024-09-14T23:57:54.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2800000000000003e-07 2024-09-14T23:57:54.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.781 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:54.782 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304794+0000", 2024-09-14T23:57:54.782 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8700000000000001e-06 2024-09-14T23:57:54.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.782 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:54.782 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304800+0000", 2024-09-14T23:57:54.782 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0889999999999996e-06 2024-09-14T23:57:54.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.782 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:54.782 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304805+0000", 2024-09-14T23:57:54.782 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7060000000000003e-06 2024-09-14T23:57:54.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.783 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:54.783 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355108+0000", 2024-09-14T23:57:54.783 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050303170000000001 2024-09-14T23:57:54.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.783 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.783 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:54.783 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355150+0000", 2024-09-14T23:57:54.783 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2271999999999998e-05 2024-09-14T23:57:54.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.783 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.783 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:54.783 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355190+0000", 2024-09-14T23:57:54.783 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0438999999999997e-05 2024-09-14T23:57:54.783 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.784 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:54.784 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.784 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.784 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.784 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69286 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 7553024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:54.784 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.304811+0000", 2024-09-14T23:57:54.784 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 39.399280517999998, 2024-09-14T23:57:54.784 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050585512999999999, 2024-09-14T23:57:54.784 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:54.784 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:54.784 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:54.784 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:54.784 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:54.784 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:54.784 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69286 2024-09-14T23:57:54.785 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.785 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:54.785 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.785 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:54.785 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304811+0000", 2024-09-14T23:57:54.785 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.785 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.785 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.785 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:54.785 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304811+0000", 2024-09-14T23:57:54.785 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.785 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.785 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.785 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:54.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304811+0000", 2024-09-14T23:57:54.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.89e-07 2024-09-14T23:57:54.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:54.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304814+0000", 2024-09-14T23:57:54.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3199999999999998e-06 2024-09-14T23:57:54.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:54.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304821+0000", 2024-09-14T23:57:54.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5689999999999999e-06 2024-09-14T23:57:54.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:54.787 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304826+0000", 2024-09-14T23:57:54.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1200000000000001e-06 2024-09-14T23:57:54.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.787 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:54.787 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355226+0000", 2024-09-14T23:57:54.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050399653000000003 2024-09-14T23:57:54.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.787 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:54.787 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355314+0000", 2024-09-14T23:57:54.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8389999999999999e-05 2024-09-14T23:57:54.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.787 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:54.788 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355396+0000", 2024-09-14T23:57:54.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2071999999999996e-05 2024-09-14T23:57:54.788 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.788 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:54.788 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.788 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69287 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 7667712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:54.788 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.304833+0000", 2024-09-14T23:57:54.788 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 39.399257919999997, 2024-09-14T23:57:54.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050664740999999999, 2024-09-14T23:57:54.788 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:54.788 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:54.788 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:54.789 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:54.789 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:54.789 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:54.789 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69287 2024-09-14T23:57:54.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.789 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:54.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.789 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:54.789 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304833+0000", 2024-09-14T23:57:54.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.789 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:54.789 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304833+0000", 2024-09-14T23:57:54.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:54.790 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304834+0000", 2024-09-14T23:57:54.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4999999999999998e-07 2024-09-14T23:57:54.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:54.790 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304836+0000", 2024-09-14T23:57:54.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0279999999999999e-06 2024-09-14T23:57:54.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:54.790 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304842+0000", 2024-09-14T23:57:54.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8919999999999997e-06 2024-09-14T23:57:54.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.791 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:54.791 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304847+0000", 2024-09-14T23:57:54.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6160000000000001e-06 2024-09-14T23:57:54.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.791 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:54.792 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355411+0000", 2024-09-14T23:57:54.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050563540999999997 2024-09-14T23:57:54.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.792 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:54.792 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355459+0000", 2024-09-14T23:57:54.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7979000000000003e-05 2024-09-14T23:57:54.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.792 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:54.792 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355498+0000", 2024-09-14T23:57:54.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9335000000000002e-05 2024-09-14T23:57:54.792 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.793 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:54.793 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.793 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69288 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 7782400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:54.793 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.304854+0000", 2024-09-14T23:57:54.793 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 39.399237481999997, 2024-09-14T23:57:54.793 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050855145999999997, 2024-09-14T23:57:54.793 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:54.793 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:54.793 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:54.793 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:54.793 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:54.793 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:54.794 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69288 2024-09-14T23:57:54.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.794 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:54.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.794 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:54.794 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304854+0000", 2024-09-14T23:57:54.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.794 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:54.794 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304854+0000", 2024-09-14T23:57:54.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:54.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:54.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.306670+0000", 2024-09-14T23:57:54.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00181631 2024-09-14T23:57:54.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:54.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.306675+0000", 2024-09-14T23:57:54.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0220000000000004e-06 2024-09-14T23:57:54.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:54.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.306684+0000", 2024-09-14T23:57:54.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7309999999999999e-06 2024-09-14T23:57:54.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.796 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:54.796 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.306689+0000", 2024-09-14T23:57:54.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6609999999999999e-06 2024-09-14T23:57:54.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.796 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:54.796 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355533+0000", 2024-09-14T23:57:54.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048844578 2024-09-14T23:57:54.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.796 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:54.796 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355622+0000", 2024-09-14T23:57:54.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9006999999999997e-05 2024-09-14T23:57:54.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:54.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:54.797 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:54.797 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355709+0000", 2024-09-14T23:57:54.797 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6836999999999995e-05 2024-09-14T23:57:54.797 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.797 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:54.797 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.797 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:54.797 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:54.797 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:54.799 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:54.797+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:54.799 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T23:57:54.830 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T23:57:54.899 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:54.898+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:54.908 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:54.908 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:57:54.908 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:57:54.908 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:54.917 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T23:57:54.969 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:57:54.969 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:57:54.987 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:57:54.987 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:57:55.000 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:54.999+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:55.003 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:57:55.004 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:57:55.020 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:57:55.020 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:57:55.038 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:57:55.038 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:57:55.054 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:57:55.055 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:57:55.067 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:55.067 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:55.071 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:57:55.072 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:57:55.076 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T23:57:55.084 INFO:teuthology.orchestra.run.smithi042.stderr:set 2.2 pg_upmap_items mapping to [6->2,1->7] 2024-09-14T23:57:55.085 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is set 2024-09-14T23:57:55.094 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:57:55.095 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:57:55.101 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:55.100+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:55.171 INFO:teuthology.orchestra.run.smithi042.stdout:leaving image image_client.0-clone43 intact 2024-09-14T23:57:55.202 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:55.200+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:55.203 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:55.203 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:55.214 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T23:57:55.299 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #42, image image_client.0-clone42 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent43 2024-09-14T23:57:55.303 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:55.300+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:55.327 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:55.327 INFO:teuthology.orchestra.run.smithi042.stdout:5129 write 0x24b9eee thru 0x24c3db7 (0x9eca bytes) 2024-09-14T23:57:55.375 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:55.375 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:55.385 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T23:57:55.388 INFO:teuthology.orchestra.run.smithi042.stdout:5130 punch from 0x1b795cd to 0x1b7ee85, (0x58b8 bytes) 2024-09-14T23:57:55.388 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:55.388 INFO:teuthology.orchestra.run.smithi042.stdout:5132 read 0x47d2f6a thru 0x47e2a8a (0xfb21 bytes) 2024-09-14T23:57:55.389 INFO:teuthology.orchestra.run.smithi042.stdout:5134 read 0x3b45b0a thru 0x3b4f448 (0x993f bytes) 2024-09-14T23:57:55.389 INFO:teuthology.orchestra.run.smithi042.stdout:5136 writesame 0x14c5a9f thru 0x14d2e8b data_size 0x1(0xd3ed bytes) 2024-09-14T23:57:55.429 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:55.428+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:55.471 INFO:teuthology.orchestra.run.smithi042.stdout:5138 punch from 0x441c82e to 0x4420737, (0x3f09 bytes) 2024-09-14T23:57:55.471 INFO:teuthology.orchestra.run.smithi042.stdout:5139 writesame 0x125d497 thru 0x1263875 data_size 0x1(0x63df bytes) 2024-09-14T23:57:55.502 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:55.502 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:55.504 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:55.503+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:55.514 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T23:57:55.524 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:55.524 INFO:teuthology.orchestra.run.smithi042.stdout:5141 read 0x26c7b74 thru 0x26cfd90 (0x821d bytes) 2024-09-14T23:57:55.534 INFO:teuthology.orchestra.run.smithi042.stdout:5142 write 0x9e95c1e thru 0x9e97afb (0x1ede bytes) 2024-09-14T23:57:55.536 INFO:teuthology.orchestra.run.smithi042.stdout:5143 punch from 0x3c14a26 to 0x3c1e2ca, (0x98a4 bytes) 2024-09-14T23:57:55.536 INFO:teuthology.orchestra.run.smithi042.stdout:5144 read 0x27a7d84 thru 0x27b6cab (0xef28 bytes) 2024-09-14T23:57:55.536 INFO:teuthology.orchestra.run.smithi042.stdout:5145 writesame 0x8ba61d6 thru 0x8bac711 data_size 0x2c(0x653c bytes) 2024-09-14T23:57:55.585 INFO:teuthology.orchestra.run.smithi042.stdout:5146 write 0xed533f4 thru 0xed59458 (0x6065 bytes) 2024-09-14T23:57:55.588 INFO:teuthology.orchestra.run.smithi042.stdout:5148 writesame 0xd3bab47 thru 0xd3c1000 data_size 0x1(0x64ba bytes) 2024-09-14T23:57:55.605 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:55.604+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:55.638 INFO:teuthology.orchestra.run.smithi042.stdout:5149 write 0x79230d9 thru 0x792396d (0x895 bytes) 2024-09-14T23:57:55.639 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:55.639 INFO:teuthology.orchestra.run.smithi042.stdout:5151 read 0xdb49aa thru 0xdb61b2 (0x1809 bytes) 2024-09-14T23:57:55.640 INFO:teuthology.orchestra.run.smithi042.stdout:5155 read 0xbd5620e thru 0xbd5cc24 (0x6a17 bytes) 2024-09-14T23:57:55.640 INFO:teuthology.orchestra.run.smithi042.stdout:5157 read 0x2f15943 thru 0x2f194d0 (0x3b8e bytes) 2024-09-14T23:57:55.641 INFO:teuthology.orchestra.run.smithi042.stdout:5158 punch from 0x6ae1cc6 to 0x6aee3e6, (0xc720 bytes) 2024-09-14T23:57:55.641 INFO:teuthology.orchestra.run.smithi042.stdout:5160 read 0xd7fca0b thru 0xd8086c3 (0xbcb9 bytes) 2024-09-14T23:57:55.642 INFO:teuthology.orchestra.run.smithi042.stdout:5161 write 0x6d37460 thru 0x6d3d06c (0x5c0d bytes) 2024-09-14T23:57:55.643 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:55.643 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:55.655 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T23:57:55.681 INFO:teuthology.orchestra.run.smithi042.stdout:5162 write 0x83e48a5 thru 0x83e95a5 (0x4d01 bytes) 2024-09-14T23:57:55.683 INFO:teuthology.orchestra.run.smithi042.stdout:5163 write 0x93bb0e4 thru 0x93ca917 (0xf834 bytes) 2024-09-14T23:57:55.684 INFO:teuthology.orchestra.run.smithi042.stdout:5165 punch from 0xdc33170 to 0xdc3bcaf, (0x8b3f bytes) 2024-09-14T23:57:55.689 INFO:teuthology.orchestra.run.smithi042.stdout:5166 trunc from 0xed59459 to 0xa9792d2 2024-09-14T23:57:55.707 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:55.705+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:55.749 INFO:teuthology.orchestra.run.smithi042.stdout:5167 write 0xb693c8f thru 0xb69ad4c (0x70be bytes) 2024-09-14T23:57:55.755 INFO:teuthology.orchestra.run.smithi042.stdout:5170 writesame 0xb7c98e9 thru 0xb7d449c data_size 0x42(0xabb4 bytes) 2024-09-14T23:57:55.757 INFO:teuthology.orchestra.run.smithi042.stdout:5172 read 0x13a0f55 thru 0x13a9c15 (0x8cc1 bytes) 2024-09-14T23:57:55.757 INFO:teuthology.orchestra.run.smithi042.stdout:5173 writesame 0x28b4efb thru 0x28bfc1e data_size 0xe(0xad24 bytes) 2024-09-14T23:57:55.760 INFO:teuthology.orchestra.run.smithi042.stdout:5174 writesame 0xac98071 thru 0xaca67ba data_size 0x3e(0xe74a bytes) 2024-09-14T23:57:55.769 INFO:teuthology.orchestra.run.smithi042.stdout:5175 write 0xc7998c4 thru 0xc7a84c7 (0xec04 bytes) 2024-09-14T23:57:55.770 INFO:teuthology.orchestra.run.smithi042.stdout:5176 writesame 0x5f05cdf thru 0x5f06b62 data_size 0x2(0xe84 bytes) 2024-09-14T23:57:55.772 INFO:teuthology.orchestra.run.smithi042.stdout:5177 read 0x9f1627a thru 0x9f19abe (0x3845 bytes) 2024-09-14T23:57:55.773 INFO:teuthology.orchestra.run.smithi042.stdout:5179 punch from 0x3dc2bee to 0x3dc8589, (0x599b bytes) 2024-09-14T23:57:55.773 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:55.773 INFO:teuthology.orchestra.run.smithi042.stdout:5181 write 0x88ea7ee thru 0x88f2e40 (0x8653 bytes) 2024-09-14T23:57:55.779 INFO:teuthology.orchestra.run.smithi042.stdout:5182 writesame 0xc7f142c thru 0xc7f7eb5 data_size 0xd(0x6a8a bytes) 2024-09-14T23:57:55.788 INFO:teuthology.orchestra.run.smithi042.stdout:5184 write 0xe201b8f thru 0xe209a46 (0x7eb8 bytes) 2024-09-14T23:57:55.791 INFO:teuthology.orchestra.run.smithi042.stdout:5185 writesame 0x2aa1d24 thru 0x2aa81fa data_size 0x5(0x64d7 bytes) 2024-09-14T23:57:55.794 INFO:teuthology.orchestra.run.smithi042.stdout:5186 write 0xb7a3fee thru 0xb7b3f9d (0xffb0 bytes) 2024-09-14T23:57:55.803 INFO:teuthology.orchestra.run.smithi042.stdout:5187 trunc from 0xe209a47 to 0x7aa868a 2024-09-14T23:57:55.807 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:55.805+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:55.808 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:55.808 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:55.815 INFO:teuthology.orchestra.run.smithi042.stdout:5189 read 0x54e8051 thru 0x54f14f9 (0x94a9 bytes) 2024-09-14T23:57:55.816 INFO:teuthology.orchestra.run.smithi042.stdout:5190 read 0x635b9b8 thru 0x635cf27 (0x1570 bytes) 2024-09-14T23:57:55.816 INFO:teuthology.orchestra.run.smithi042.stdout:5191 punch from 0x5aeb6e5 to 0x5af2bd2, (0x74ed bytes) 2024-09-14T23:57:55.818 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T23:57:55.821 INFO:teuthology.orchestra.run.smithi042.stdout:5192 trunc from 0x7aa868a to 0xc7c8c3a 2024-09-14T23:57:55.827 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:55.835 INFO:teuthology.orchestra.run.smithi042.stdout:5195 trunc from 0xc7c8c3a to 0x4dfa4fa 2024-09-14T23:57:55.844 INFO:teuthology.orchestra.run.smithi042.stdout:5197 writesame 0xef0df3 thru 0xef377e data_size 0x2(0x298c bytes) 2024-09-14T23:57:55.847 INFO:teuthology.orchestra.run.smithi042.stdout:5199 punch from 0x338d8c2 to 0x339a62d, (0xcd6b bytes) 2024-09-14T23:57:55.854 INFO:teuthology.orchestra.run.smithi042.stdout:5200 write 0x6926391 thru 0x6928b94 (0x2804 bytes) 2024-09-14T23:57:55.855 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:55.866 INFO:teuthology.orchestra.run.smithi042.stdout:5203 write 0xab71346 thru 0xab7e61d (0xd2d8 bytes) 2024-09-14T23:57:55.874 INFO:teuthology.orchestra.run.smithi042.stdout:5208 write 0xbcdb195 thru 0xbce2265 (0x70d1 bytes) 2024-09-14T23:57:55.876 INFO:teuthology.orchestra.run.smithi042.stdout:5210 write 0x838a6d3 thru 0x8394064 (0x9992 bytes) 2024-09-14T23:57:55.878 INFO:teuthology.orchestra.run.smithi042.stdout:5211 clone 45 order 23 su 32768 sc 9 2024-09-14T23:57:55.908 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:55.907+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:55.936 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:55.936 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:55.944 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T23:57:56.028 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:56.027+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:56.098 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:56.098 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:56.107 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T23:57:56.111 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:56.108+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:56.173 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone44 from 0xbce2266 (overlap 0x4dfa4fa) to 0x1764b17 2024-09-14T23:57:56.212 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:56.210+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:56.269 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:56.269 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:56.278 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T23:57:56.311 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #43, image image_client.0-clone43 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent44 2024-09-14T23:57:56.313 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:56.310+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:56.414 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:56.412+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:56.443 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:56.443 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:56.453 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T23:57:56.469 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:56.469 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:56.476 INFO:teuthology.orchestra.run.smithi042.stdout:5216 trunc from 0xbce2266 to 0x53901c2 2024-09-14T23:57:56.515 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:56.513+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:56.605 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:56.605 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:56.612 INFO:teuthology.orchestra.run.smithi042.stdout:5217 writesame 0x672cab8 thru 0x6736ca9 data_size 0x13(0xa1f2 bytes) 2024-09-14T23:57:56.616 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T23:57:56.616 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:56.614+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:56.624 INFO:teuthology.orchestra.run.smithi042.stdout:5218 write 0xa6d14c8 thru 0xa6e05e6 (0xf11f bytes) 2024-09-14T23:57:56.629 INFO:teuthology.orchestra.run.smithi042.stdout:5219 read 0x4c24b21 thru 0x4c265cf (0x1aaf bytes) 2024-09-14T23:57:56.629 INFO:teuthology.orchestra.run.smithi042.stdout:5220 write 0x517c335 thru 0x5182c15 (0x68e1 bytes) 2024-09-14T23:57:56.638 INFO:teuthology.orchestra.run.smithi042.stdout:5221 writesame 0xc01f540 thru 0xc02d80c data_size 0x1(0xe2cd bytes) 2024-09-14T23:57:56.640 INFO:teuthology.orchestra.run.smithi042.stdout:5222 read 0x20582c8 thru 0x2061b48 (0x9881 bytes) 2024-09-14T23:57:56.640 INFO:teuthology.orchestra.run.smithi042.stdout:5223 punch from 0x6048326 to 0x604b9e4, (0x36be bytes) 2024-09-14T23:57:56.641 INFO:teuthology.orchestra.run.smithi042.stdout:5224 write 0x1c12230 thru 0x1c20493 (0xe264 bytes) 2024-09-14T23:57:56.716 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:57:56.715+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:57:56.723 INFO:teuthology.orchestra.run.smithi042.stdout:5225 read 0x8b0f582 thru 0x8b12a1a (0x3499 bytes) 2024-09-14T23:57:56.723 INFO:teuthology.orchestra.run.smithi042.stdout:5228 writesame 0x916264f thru 0x9171fed data_size 0x77(0xf99f bytes) 2024-09-14T23:57:56.731 INFO:teuthology.orchestra.run.smithi042.stdout:5229 trunc from 0xc02d80d to 0x9a8fef1 2024-09-14T23:57:56.739 INFO:teuthology.orchestra.run.smithi042.stdout:5230 write 0x647174 thru 0x64d81c (0x66a9 bytes) 2024-09-14T23:57:56.770 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:56.770 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:56.782 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T23:57:56.799 INFO:teuthology.orchestra.run.smithi042.stdout:5231 punch from 0x4231e9e to 0x4236b17, (0x4c79 bytes) 2024-09-14T23:57:56.799 INFO:teuthology.orchestra.run.smithi042.stdout:5232 writesame 0x6c1cf92 thru 0x6c28d0a data_size 0x5(0xbd79 bytes) 2024-09-14T23:57:56.803 INFO:teuthology.orchestra.run.smithi042.stdout:5233 punch from 0x6844fcc to 0x6852185, (0xd1b9 bytes) 2024-09-14T23:57:56.808 INFO:teuthology.orchestra.run.smithi042.stdout:5234 trunc from 0x9a8fef1 to 0xda002dc 2024-09-14T23:57:56.815 INFO:teuthology.orchestra.run.smithi042.stdout:5235 write 0xbb03c8b thru 0xbb0af78 (0x72ee bytes) 2024-09-14T23:57:56.816 INFO:teuthology.orchestra.run.smithi042.stdout:5236 writesame 0xb866546 thru 0xb86a60c data_size 0x17(0x40c7 bytes) 2024-09-14T23:57:56.826 INFO:teuthology.orchestra.run.smithi042.stdout:5237 trunc from 0xda002dc to 0x89a9e06 2024-09-14T23:57:56.827 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:56.825+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:56.839 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:56.839 INFO:teuthology.orchestra.run.smithi042.stdout:5240 punch from 0x1ac7d9a to 0x1ad789c, (0xfb02 bytes) 2024-09-14T23:57:56.851 INFO:teuthology.orchestra.run.smithi042.stdout:5241 write 0xdf72a8b thru 0xdf7afa3 (0x8519 bytes) 2024-09-14T23:57:56.855 INFO:teuthology.orchestra.run.smithi042.stdout:5242 writesame 0xcac76aa thru 0xcad2ebf data_size 0x1(0xb816 bytes) 2024-09-14T23:57:56.875 INFO:teuthology.orchestra.run.smithi042.stdout:5244 writesame 0xfbaae3 thru 0xfbdbc0 data_size 0x2d(0x30de bytes) 2024-09-14T23:57:56.877 INFO:teuthology.orchestra.run.smithi042.stdout:5246 punch from 0x277f275 to 0x2783fbb, (0x4d46 bytes) 2024-09-14T23:57:56.883 INFO:teuthology.orchestra.run.smithi042.stdout:5247 write 0xe2302df thru 0xe231cd9 (0x19fb bytes) 2024-09-14T23:57:56.888 INFO:teuthology.orchestra.run.smithi042.stdout:5248 punch from 0xa26f2f1 to 0xa270391, (0x10a0 bytes) 2024-09-14T23:57:56.892 INFO:teuthology.orchestra.run.smithi042.stdout:5249 trunc from 0xe231cda to 0x9136a48 2024-09-14T23:57:56.910 INFO:teuthology.orchestra.run.smithi042.stdout:5250 trunc from 0x9136a48 to 0x42af812 2024-09-14T23:57:56.910 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:56.910 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:56.918 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:56.915+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:56.922 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T23:57:57.018 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:57.017+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:57.019 INFO:teuthology.orchestra.run.smithi042.stdout:5252 trunc from 0x42af812 to 0x5d7af3b 2024-09-14T23:57:57.025 INFO:teuthology.orchestra.run.smithi042.stdout:5254 read 0x56424a0 thru 0x564f9e3 (0xd544 bytes) 2024-09-14T23:57:57.025 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:57.025 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:57.025 INFO:teuthology.orchestra.run.smithi042.stdout:5257 read 0x42f18aa thru 0x42f97b1 (0x7f08 bytes) 2024-09-14T23:57:57.029 INFO:teuthology.orchestra.run.smithi042.stdout:5259 trunc from 0x5d7af3b to 0x922b4ca 2024-09-14T23:57:57.043 INFO:teuthology.orchestra.run.smithi042.stdout:5260 write 0x9efced5 thru 0x9efd12f (0x25b bytes) 2024-09-14T23:57:57.046 INFO:teuthology.orchestra.run.smithi042.stdout:5261 read 0x3799207 thru 0x37a5bbb (0xc9b5 bytes) 2024-09-14T23:57:57.047 INFO:teuthology.orchestra.run.smithi042.stdout:5262 punch from 0x585e676 to 0x586798b, (0x9315 bytes) 2024-09-14T23:57:57.057 INFO:teuthology.orchestra.run.smithi042.stdout:5263 write 0xd682f3a thru 0xd692e3e (0xff05 bytes) 2024-09-14T23:57:57.062 INFO:teuthology.orchestra.run.smithi042.stdout:5264 read 0xc9dd8f0 thru 0xc9df550 (0x1c61 bytes) 2024-09-14T23:57:57.063 INFO:teuthology.orchestra.run.smithi042.stdout:5266 punch from 0x314bdc to 0x322c22, (0xe046 bytes) 2024-09-14T23:57:57.063 INFO:teuthology.orchestra.run.smithi042.stdout:5268 writesame 0x8eb5683 thru 0x8ec43cd data_size 0x1(0xed4b bytes) 2024-09-14T23:57:57.087 INFO:teuthology.orchestra.run.smithi042.stdout:5269 write 0xa10ddde thru 0xa113e62 (0x6085 bytes) 2024-09-14T23:57:57.093 INFO:teuthology.orchestra.run.smithi042.stdout:5274 read 0x153811b thru 0x154394d (0xb833 bytes) 2024-09-14T23:57:57.093 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:57.093 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:57.098 INFO:teuthology.orchestra.run.smithi042.stdout:5275 trunc from 0xd692e3f to 0x7a1534f 2024-09-14T23:57:57.100 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T23:57:57.107 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T23:57:57.111 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T23:57:57.140 INFO:teuthology.orchestra.run.smithi042.stdout:5276 write 0x547f2ca thru 0x54881c1 (0x8ef8 bytes) 2024-09-14T23:57:57.141 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:57.139+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:57.143 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:57.143 INFO:teuthology.orchestra.run.smithi042.stdout:5279 read 0x42e5386 thru 0x42ea50d (0x5188 bytes) 2024-09-14T23:57:57.143 INFO:teuthology.orchestra.run.smithi042.stdout:5280 write 0x27c973b thru 0x27cc6b9 (0x2f7f bytes) 2024-09-14T23:57:57.144 INFO:teuthology.orchestra.run.smithi042.stdout:5282 punch from 0x6b0e414 to 0x6b1dbad, (0xf799 bytes) 2024-09-14T23:57:57.144 INFO:teuthology.orchestra.run.smithi042.stdout:5283 punch from 0x26c314e to 0x26c989f, (0x6751 bytes) 2024-09-14T23:57:57.148 INFO:teuthology.orchestra.run.smithi042.stdout:5284 trunc from 0x7a1534f to 0x3c84316 2024-09-14T23:57:57.179 INFO:teuthology.orchestra.run.smithi042.stdout:5287 write 0xbc6879e thru 0xbc687ac (0xf bytes) 2024-09-14T23:57:57.188 INFO:teuthology.orchestra.run.smithi042.stdout:5288 trunc from 0xbc687ad to 0x62d626c 2024-09-14T23:57:57.199 INFO:teuthology.orchestra.run.smithi042.stdout:5290 read 0x169cf1c thru 0x16a81c8 (0xb2ad bytes) 2024-09-14T23:57:57.205 INFO:teuthology.orchestra.run.smithi042.stdout:5291 trunc from 0x62d626c to 0x181acfe 2024-09-14T23:57:57.216 INFO:teuthology.orchestra.run.smithi042.stdout:5295 read 0x16e1231 thru 0x16e4de1 (0x3bb1 bytes) 2024-09-14T23:57:57.216 INFO:teuthology.orchestra.run.smithi042.stdout:5298 read 0xe3c679 thru 0xe47edb (0xb863 bytes) 2024-09-14T23:57:57.217 INFO:teuthology.orchestra.run.smithi042.stdout:5299 punch from 0x1289602 to 0x1291583, (0x7f81 bytes) 2024-09-14T23:57:57.217 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:57.217 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:57.219 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:57.217+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:57.226 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T23:57:57.227 INFO:teuthology.orchestra.run.smithi042.stdout:5300 trunc from 0x181acfe to 0xa61c1b0 2024-09-14T23:57:57.237 INFO:teuthology.orchestra.run.smithi042.stdout:5301 write 0xa751366 thru 0xa752a8d (0x1728 bytes) 2024-09-14T23:57:57.240 INFO:teuthology.orchestra.run.smithi042.stdout:5302 read 0x7d7af04 thru 0x7d7caf3 (0x1bf0 bytes) 2024-09-14T23:57:57.248 INFO:teuthology.orchestra.run.smithi042.stdout:5303 trunc from 0xa752a8e to 0x361a034 2024-09-14T23:57:57.275 INFO:teuthology.orchestra.run.smithi042.stdout:5304 write 0xbd623e3 thru 0xbd63d87 (0x19a5 bytes) 2024-09-14T23:57:57.279 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:57.279 INFO:teuthology.orchestra.run.smithi042.stdout:5306 read 0x6fc83b1 thru 0x6fcf6e0 (0x7330 bytes) 2024-09-14T23:57:57.279 INFO:teuthology.orchestra.run.smithi042.stdout:5308 writesame 0xac83cbd thru 0xac9325a data_size 0x95(0xf59e bytes) 2024-09-14T23:57:57.283 INFO:teuthology.orchestra.run.smithi042.stdout:5310 punch from 0x24a69a6 to 0x24b6469, (0xfac3 bytes) 2024-09-14T23:57:57.283 INFO:teuthology.orchestra.run.smithi042.stdout:5311 read 0x5ac3761 thru 0x5ac8630 (0x4ed0 bytes) 2024-09-14T23:57:57.283 INFO:teuthology.orchestra.run.smithi042.stdout:5312 punch from 0x8b9da17 to 0x8ba2b22, (0x510b bytes) 2024-09-14T23:57:57.284 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:57:57.285 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:57:57.285 INFO:teuthology.orchestra.run.smithi042.stdout:5314 trunc from 0xbd63d88 to 0x9c3cd52 2024-09-14T23:57:57.293 INFO:teuthology.orchestra.run.smithi042.stdout:5315 writesame 0x8ffa94a thru 0x900618d data_size 0x6(0xb844 bytes) 2024-09-14T23:57:57.299 INFO:teuthology.orchestra.run.smithi042.stdout:5317 punch from 0x1180b1 to 0x119949, (0x1898 bytes) 2024-09-14T23:57:57.299 INFO:teuthology.orchestra.run.smithi042.stdout:5318 punch from 0x284a9f8 to 0x28569ee, (0xbff6 bytes) 2024-09-14T23:57:57.304 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:57:57.305 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:57:57.306 INFO:teuthology.orchestra.run.smithi042.stdout:5319 writesame 0xa7b45ee thru 0xa7bfe37 data_size 0x9(0xb84a bytes) 2024-09-14T23:57:57.307 INFO:teuthology.orchestra.run.smithi042.stdout:5320 read 0x570288d thru 0x5706a3d (0x41b1 bytes) 2024-09-14T23:57:57.308 INFO:teuthology.orchestra.run.smithi042.stdout:5321 writesame 0x2dbb351 thru 0x2dbefbb data_size 0x1(0x3c6b bytes) 2024-09-14T23:57:57.316 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:57.316 INFO:teuthology.orchestra.run.smithi042.stdout:5323 punch from 0x1774d68 to 0x1780d51, (0xbfe9 bytes) 2024-09-14T23:57:57.316 INFO:teuthology.orchestra.run.smithi042.stdout:5324 writesame 0x57a50a4 thru 0x57ad728 data_size 0xd(0x8685 bytes) 2024-09-14T23:57:57.320 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:57.318+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:57.321 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:57.325 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:57:57.325 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:57:57.328 INFO:teuthology.orchestra.run.smithi042.stdout:5326 write 0xbe0ca28 thru 0xbe18695 (0xbc6e bytes) 2024-09-14T23:57:57.343 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:57:57.343 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:57:57.346 INFO:teuthology.orchestra.run.smithi042.stdout:5329 trunc from 0xbe18696 to 0x6ed26a 2024-09-14T23:57:57.363 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:57:57.363 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:57:57.364 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:57.364 INFO:teuthology.orchestra.run.smithi042.stdout:5331 read 0x346de7 thru 0x348b02 (0x1d1c bytes) 2024-09-14T23:57:57.364 INFO:teuthology.orchestra.run.smithi042.stdout:5332 read 0x259705 thru 0x261ba1 (0x849d bytes) 2024-09-14T23:57:57.365 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:57.365 INFO:teuthology.orchestra.run.smithi042.stdout:5334 trunc from 0x6ed26a to 0x7f5a3c 2024-09-14T23:57:57.369 INFO:teuthology.orchestra.run.smithi042.stdout:5335 read 0x139a5a thru 0x1422b6 (0x885d bytes) 2024-09-14T23:57:57.373 INFO:teuthology.orchestra.run.smithi042.stdout:5338 trunc from 0x7f5a3c to 0x3301a8f 2024-09-14T23:57:57.379 INFO:teuthology.orchestra.run.smithi042.stdout:5339 read 0x2240824 thru 0x224e376 (0xdb53 bytes) 2024-09-14T23:57:57.381 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:57:57.381 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:57:57.390 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:57.390 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:57.390 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:57:57.390 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:57:57.390 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:57.394 INFO:teuthology.orchestra.run.smithi042.stdout:5340 write 0xac1bf42 thru 0xac27718 (0xb7d7 bytes) 2024-09-14T23:57:57.399 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:57:57.399 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:57:57.401 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T23:57:57.402 INFO:teuthology.orchestra.run.smithi042.stdout:5341 trunc from 0xac27719 to 0xe814205 2024-09-14T23:57:57.407 INFO:teuthology.orchestra.run.smithi042.stdout:5342 read 0xd21b5a thru 0xd22f22 (0x13c9 bytes) 2024-09-14T23:57:57.408 INFO:teuthology.orchestra.run.smithi042.stdout:5343 write 0xd772752 thru 0xd7784ab (0x5d5a bytes) 2024-09-14T23:57:57.410 INFO:teuthology.orchestra.run.smithi042.stdout:5345 writesame 0xe1d6376 thru 0xe1deafd data_size 0x4(0x8788 bytes) 2024-09-14T23:57:57.417 INFO:teuthology.orchestra.run.smithi042.stdout:5346 read 0x94ba8d6 thru 0x94c83e9 (0xdb14 bytes) 2024-09-14T23:57:57.418 INFO:teuthology.orchestra.run.smithi042.stdout:5347 read 0x3ac3a7c thru 0x3ad2448 (0xe9cd bytes) 2024-09-14T23:57:57.418 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:57:57.418 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:57:57.418 INFO:teuthology.orchestra.run.smithi042.stdout:5348 write 0x856c47b thru 0x856fc76 (0x37fc bytes) 2024-09-14T23:57:57.422 INFO:teuthology.orchestra.run.smithi042.stdout:5349 punch from 0xc42a9ee to 0xc430258, (0x586a bytes) 2024-09-14T23:57:57.422 INFO:teuthology.orchestra.run.smithi042.stdout:5350 read 0x26802b7 thru 0x268279a (0x24e4 bytes) 2024-09-14T23:57:57.422 INFO:teuthology.orchestra.run.smithi042.stdout:5352 write 0xb47cd29 thru 0xb48527b (0x8553 bytes) 2024-09-14T23:57:57.426 INFO:teuthology.orchestra.run.smithi042.stdout:5353 read 0x48621df thru 0x4867aec (0x590e bytes) 2024-09-14T23:57:57.426 INFO:teuthology.orchestra.run.smithi042.stdout:5355 read 0x4c7cc94 thru 0x4c8bc8d (0xeffa bytes) 2024-09-14T23:57:57.427 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:57.427 INFO:teuthology.orchestra.run.smithi042.stdout:5357 punch from 0xc48052f to 0xc4816c8, (0x1199 bytes) 2024-09-14T23:57:57.427 INFO:teuthology.orchestra.run.smithi042.stdout:5359 write 0xe0dcaac thru 0xe0dfe7e (0x33d3 bytes) 2024-09-14T23:57:57.430 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:57.430 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:57.430 INFO:teuthology.orchestra.run.smithi042.stdout:5362 write 0xbb890d9 thru 0xbb90226 (0x714e bytes) 2024-09-14T23:57:57.431 INFO:teuthology.orchestra.run.smithi042.stdout:5363 write 0xaaad495 thru 0xaabb636 (0xe1a2 bytes) 2024-09-14T23:57:57.433 INFO:teuthology.orchestra.run.smithi042.stdout:5364 writesame 0x7732f63 thru 0x77385b3 data_size 0x1(0x5651 bytes) 2024-09-14T23:57:57.441 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:57.439+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:57.446 INFO:teuthology.orchestra.run.smithi042.stdout:5365 read 0x525b643 thru 0x5266f16 (0xb8d4 bytes) 2024-09-14T23:57:57.446 INFO:teuthology.orchestra.run.smithi042.stdout:5366 write 0xbcb557a thru 0xbcc0623 (0xb0aa bytes) 2024-09-14T23:57:57.448 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:57.448 INFO:teuthology.orchestra.run.smithi042.stdout:5368 read 0xc3cc25e thru 0xc3d1338 (0x50db bytes) 2024-09-14T23:57:57.449 INFO:teuthology.orchestra.run.smithi042.stdout:5369 write 0x1bf4281 thru 0x1c01341 (0xd0c1 bytes) 2024-09-14T23:57:57.451 INFO:teuthology.orchestra.run.smithi042.stdout:5370 punch from 0xca322e0 to 0xca35e37, (0x3b57 bytes) 2024-09-14T23:57:57.451 INFO:teuthology.orchestra.run.smithi042.stdout:5371 write 0x113260f thru 0x114217b (0xfb6d bytes) 2024-09-14T23:57:57.453 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:57.453 INFO:teuthology.orchestra.run.smithi042.stdout:5373 read 0x84a3769 thru 0x84aa0c1 (0x6959 bytes) 2024-09-14T23:57:57.453 INFO:teuthology.orchestra.run.smithi042.stdout:5375 clone 46 order 20 su 8192 sc 14 2024-09-14T23:57:57.522 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:57.521+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:57.551 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:57.551 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:57:57.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:57:57.551 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:57:57.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.551 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4284.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-14T23:57:57.551 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:57:57.551 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 95.024367493, 2024-09-14T23:57:57.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.004942086, 2024-09-14T23:57:57.552 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:57.552 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:57.552 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:57.552 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:57.552 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4284", 2024-09-14T23:57:57.552 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/4278347926", 2024-09-14T23:57:57.552 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1 2024-09-14T23:57:57.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.552 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:57.552 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.552 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:57.552 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:57:57.553 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:57.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.553 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:57.553 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:57:57.553 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:57.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.553 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:57.553 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524331+0000", 2024-09-14T23:57:57.553 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.383e-05 2024-09-14T23:57:57.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.553 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:57.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524339+0000", 2024-09-14T23:57:57.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8290000000000008e-06 2024-09-14T23:57:57.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:57.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524359+0000", 2024-09-14T23:57:57.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9998000000000001e-05 2024-09-14T23:57:57.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:57.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524373+0000", 2024-09-14T23:57:57.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4304000000000001e-05 2024-09-14T23:57:57.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:57.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524488+0000", 2024-09-14T23:57:57.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114663 2024-09-14T23:57:57.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:57:57.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524501+0000", 2024-09-14T23:57:57.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3349999999999999e-05 2024-09-14T23:57:57.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.556 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:57.556 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.526868+0000", 2024-09-14T23:57:57.556 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023670090000000002 2024-09-14T23:57:57.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.556 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.556 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:57.556 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.527041+0000", 2024-09-14T23:57:57.556 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000172597 2024-09-14T23:57:57.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.556 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.556 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-14T23:57:57.556 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.527384+0000", 2024-09-14T23:57:57.556 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00034309500000000002 2024-09-14T23:57:57.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.557 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.557 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:57.557 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529103+0000", 2024-09-14T23:57:57.557 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017185220000000001 2024-09-14T23:57:57.557 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.557 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.557 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:57:57.557 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529188+0000", 2024-09-14T23:57:57.557 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5591000000000001e-05 2024-09-14T23:57:57.557 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.558 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.558 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:57.558 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529227+0000", 2024-09-14T23:57:57.558 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8563000000000003e-05 2024-09-14T23:57:57.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.558 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.558 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:57.558 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529259+0000", 2024-09-14T23:57:57.558 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2734999999999997e-05 2024-09-14T23:57:57.558 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:57.558 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:57.558 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:57.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.559 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4284.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-14T23:57:57.559 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:23.530683+0000", 2024-09-14T23:57:57.559 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 94.018001827999996, 2024-09-14T23:57:57.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0036133110000000001, 2024-09-14T23:57:57.559 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:57.559 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:57.559 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:57.559 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:57.559 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4284", 2024-09-14T23:57:57.559 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/4278347926", 2024-09-14T23:57:57.559 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5 2024-09-14T23:57:57.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.559 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:57.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:57.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530683+0000", 2024-09-14T23:57:57.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:57.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:57.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530683+0000", 2024-09-14T23:57:57.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:57.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:57.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530685+0000", 2024-09-14T23:57:57.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.075e-06 2024-09-14T23:57:57.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.561 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:57.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530695+0000", 2024-09-14T23:57:57.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0261999999999999e-05 2024-09-14T23:57:57.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.561 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:57.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530715+0000", 2024-09-14T23:57:57.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9930000000000001e-05 2024-09-14T23:57:57.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.561 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:57.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530724+0000", 2024-09-14T23:57:57.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.901e-06 2024-09-14T23:57:57.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:57.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530800+0000", 2024-09-14T23:57:57.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5873999999999999e-05 2024-09-14T23:57:57.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-14T23:57:57.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530807+0000", 2024-09-14T23:57:57.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2389999999999996e-06 2024-09-14T23:57:57.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.563 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:57.563 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.532494+0000", 2024-09-14T23:57:57.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016871830000000001 2024-09-14T23:57:57.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.563 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:57.563 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.532599+0000", 2024-09-14T23:57:57.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010472 2024-09-14T23:57:57.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1", 2024-09-14T23:57:57.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.532715+0000", 2024-09-14T23:57:57.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011568199999999999 2024-09-14T23:57:57.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:57:57.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.533460+0000", 2024-09-14T23:57:57.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00074494999999999997 2024-09-14T23:57:57.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:57.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534230+0000", 2024-09-14T23:57:57.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00077036599999999998 2024-09-14T23:57:57.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.565 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:57.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534266+0000", 2024-09-14T23:57:57.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5725000000000001e-05 2024-09-14T23:57:57.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.565 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:57.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534296+0000", 2024-09-14T23:57:57.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0403999999999999e-05 2024-09-14T23:57:57.565 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:57.566 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:57.566 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:57.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.566 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4284.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-14T23:57:57.566 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:24.535160+0000", 2024-09-14T23:57:57.566 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 93.013525186999999, 2024-09-14T23:57:57.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033473650000000001, 2024-09-14T23:57:57.566 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:57.566 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:57.566 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:57.566 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:57.567 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4284", 2024-09-14T23:57:57.567 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/4278347926", 2024-09-14T23:57:57.567 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 7 2024-09-14T23:57:57.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.567 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:57.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.567 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:57.567 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535160+0000", 2024-09-14T23:57:57.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:57.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.567 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:57.567 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535160+0000", 2024-09-14T23:57:57.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:57.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.568 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:57.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535162+0000", 2024-09-14T23:57:57.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3369999999999998e-06 2024-09-14T23:57:57.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.568 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:57.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535171+0000", 2024-09-14T23:57:57.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7700000000000007e-06 2024-09-14T23:57:57.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.568 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:57.569 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535189+0000", 2024-09-14T23:57:57.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7938e-05 2024-09-14T23:57:57.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.569 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:57.569 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535197+0000", 2024-09-14T23:57:57.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5639999999999997e-06 2024-09-14T23:57:57.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.569 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:57.569 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535266+0000", 2024-09-14T23:57:57.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8926999999999999e-05 2024-09-14T23:57:57.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.570 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-14T23:57:57.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.535271+0000", 2024-09-14T23:57:57.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0329999999999999e-06 2024-09-14T23:57:57.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.570 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:57.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.537064+0000", 2024-09-14T23:57:57.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001792929 2024-09-14T23:57:57.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.570 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:57.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.537131+0000", 2024-09-14T23:57:57.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6605000000000004e-05 2024-09-14T23:57:57.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1", 2024-09-14T23:57:57.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.537271+0000", 2024-09-14T23:57:57.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013992400000000001 2024-09-14T23:57:57.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:57:57.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.538093+0000", 2024-09-14T23:57:57.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00082210200000000001 2024-09-14T23:57:57.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:57.572 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.538439+0000", 2024-09-14T23:57:57.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00034595999999999997 2024-09-14T23:57:57.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:57.572 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.538476+0000", 2024-09-14T23:57:57.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6770000000000002e-05 2024-09-14T23:57:57.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:57.573 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:24.538507+0000", 2024-09-14T23:57:57.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1506e-05 2024-09-14T23:57:57.573 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:57.573 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:57.573 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:57.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.573 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T23:57:57.573 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:34.406877+0000", 2024-09-14T23:57:57.573 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 83.141808182000005, 2024-09-14T23:57:57.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030351169999999999, 2024-09-14T23:57:57.574 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:57.574 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:57.574 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:57.574 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:57.574 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:57.574 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:57.574 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 10 2024-09-14T23:57:57.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.574 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:57.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.574 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:57.574 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406877+0000", 2024-09-14T23:57:57.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:57.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:57.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406877+0000", 2024-09-14T23:57:57.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:57.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:57.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406878+0000", 2024-09-14T23:57:57.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.291e-06 2024-09-14T23:57:57.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:57.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406883+0000", 2024-09-14T23:57:57.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2399999999999998e-06 2024-09-14T23:57:57.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:57.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406891+0000", 2024-09-14T23:57:57.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5889999999999997e-06 2024-09-14T23:57:57.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:57.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406898+0000", 2024-09-14T23:57:57.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3599999999999998e-06 2024-09-14T23:57:57.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:57.577 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406948+0000", 2024-09-14T23:57:57.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9435999999999999e-05 2024-09-14T23:57:57.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.577 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-14T23:57:57.577 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.406953+0000", 2024-09-14T23:57:57.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2809999999999999e-06 2024-09-14T23:57:57.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.577 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:57.577 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.408559+0000", 2024-09-14T23:57:57.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00160607 2024-09-14T23:57:57.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:57.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.408637+0000", 2024-09-14T23:57:57.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8504000000000006e-05 2024-09-14T23:57:57.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 4", 2024-09-14T23:57:57.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.408781+0000", 2024-09-14T23:57:57.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014374799999999999 2024-09-14T23:57:57.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:57:57.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.409659+0000", 2024-09-14T23:57:57.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000877781 2024-09-14T23:57:57.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:57.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.409822+0000", 2024-09-14T23:57:57.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016295400000000001 2024-09-14T23:57:57.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:57.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.409872+0000", 2024-09-14T23:57:57.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0022000000000003e-05 2024-09-14T23:57:57.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:57.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:34.409912+0000", 2024-09-14T23:57:57.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9841e-05 2024-09-14T23:57:57.580 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:57.580 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:57.580 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:57.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.580 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56383 3.c e56/49 3:36b698b0:::rbd_data.1118c4b3aebe.0000000000000000:head v 56'194, mlcod=56'192)", 2024-09-14T23:57:57.580 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.270483+0000", 2024-09-14T23:57:57.580 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.278202028000003, 2024-09-14T23:57:57.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0079685880000000004, 2024-09-14T23:57:57.580 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:57.580 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:57.581 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:57.581 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:57.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:57.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.270483+0000", 2024-09-14T23:57:57.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:57.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:57.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.270483+0000", 2024-09-14T23:57:57.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:57.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:57.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.270484+0000", 2024-09-14T23:57:57.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7400000000000002e-07 2024-09-14T23:57:57.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:57.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273680+0000", 2024-09-14T23:57:57.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031966680000000002 2024-09-14T23:57:57.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:57.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273697+0000", 2024-09-14T23:57:57.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6595e-05 2024-09-14T23:57:57.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:57.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273705+0000", 2024-09-14T23:57:57.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7530000000000001e-06 2024-09-14T23:57:57.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:57.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273763+0000", 2024-09-14T23:57:57.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8238e-05 2024-09-14T23:57:57.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:57.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273781+0000", 2024-09-14T23:57:57.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8281e-05 2024-09-14T23:57:57.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.584 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:57.584 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.278408+0000", 2024-09-14T23:57:57.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00462716 2024-09-14T23:57:57.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.584 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:57.584 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.278451+0000", 2024-09-14T23:57:57.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3019000000000003e-05 2024-09-14T23:57:57.584 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:57.584 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:57.584 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:57.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.585 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56384 3.c e56/49 3:36b698b0:::rbd_data.1118c4b3aebe.0000000000000000:head v 56'196, mlcod=56'192)", 2024-09-14T23:57:57.585 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:57:57.585 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.274169356000002, 2024-09-14T23:57:57.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0046707529999999997, 2024-09-14T23:57:57.585 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:57.585 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:57.585 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:57.585 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:57.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:57.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:57:57.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:57.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:57.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:57:57.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:57.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:57.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:57:57.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1699999999999997e-07 2024-09-14T23:57:57.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:57.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274523+0000", 2024-09-14T23:57:57.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9129999999999997e-06 2024-09-14T23:57:57.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:57.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274535+0000", 2024-09-14T23:57:57.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1788e-05 2024-09-14T23:57:57.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:57.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274542+0000", 2024-09-14T23:57:57.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0539999999999997e-06 2024-09-14T23:57:57.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:57.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275889+0000", 2024-09-14T23:57:57.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00134669 2024-09-14T23:57:57.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:57.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275917+0000", 2024-09-14T23:57:57.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7664000000000001e-05 2024-09-14T23:57:57.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:57.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279134+0000", 2024-09-14T23:57:57.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0032178459999999999 2024-09-14T23:57:57.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:57.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279186+0000", 2024-09-14T23:57:57.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1881e-05 2024-09-14T23:57:57.589 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:57.589 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:57.589 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:57.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.589 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56381 3.1d e56/49 3:bcf9a8cf:::rbd_data.1118c4b3aebe.0000000000000006:head v 56'210, mlcod=56'208)", 2024-09-14T23:57:57.589 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.275462+0000", 2024-09-14T23:57:57.589 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.273223025999997, 2024-09-14T23:57:57.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0071310710000000001, 2024-09-14T23:57:57.589 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:57.589 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:57.589 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:57.589 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:57.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:57.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275462+0000", 2024-09-14T23:57:57.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:57.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:57.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275462+0000", 2024-09-14T23:57:57.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:57.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:57.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275463+0000", 2024-09-14T23:57:57.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2699999999999999e-07 2024-09-14T23:57:57.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:57.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276779+0000", 2024-09-14T23:57:57.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0013163649999999999 2024-09-14T23:57:57.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:57.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276795+0000", 2024-09-14T23:57:57.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.59e-05 2024-09-14T23:57:57.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:57.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276806+0000", 2024-09-14T23:57:57.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1106e-05 2024-09-14T23:57:57.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:57.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276941+0000", 2024-09-14T23:57:57.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013477500000000001 2024-09-14T23:57:57.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:57.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276963+0000", 2024-09-14T23:57:57.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2419999999999999e-05 2024-09-14T23:57:57.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:57.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282546+0000", 2024-09-14T23:57:57.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0055827100000000003 2024-09-14T23:57:57.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:57.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282593+0000", 2024-09-14T23:57:57.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7067999999999997e-05 2024-09-14T23:57:57.593 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:57.593 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:57.593 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:57.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.593 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56387 3.c e56/49 3:30ee4ba0:::rbd_data.1118c4b3aebe.0000000000000004:head v 56'197, mlcod=56'192)", 2024-09-14T23:57:57.593 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.276498+0000", 2024-09-14T23:57:57.593 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.272186931, 2024-09-14T23:57:57.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0092192639999999996, 2024-09-14T23:57:57.593 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:57.593 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:57.593 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:57.593 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:57.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.594 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:57.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276498+0000", 2024-09-14T23:57:57.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:57.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.594 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:57.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276498+0000", 2024-09-14T23:57:57.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:57.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.594 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:57.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276499+0000", 2024-09-14T23:57:57.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1100000000000004e-07 2024-09-14T23:57:57.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:57.595 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279646+0000", 2024-09-14T23:57:57.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031473220000000001 2024-09-14T23:57:57.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:57.595 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279672+0000", 2024-09-14T23:57:57.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6129999999999999e-05 2024-09-14T23:57:57.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:57.595 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279682+0000", 2024-09-14T23:57:57.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7170000000000003e-06 2024-09-14T23:57:57.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.596 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:57.596 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279742+0000", 2024-09-14T23:57:57.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9926999999999997e-05 2024-09-14T23:57:57.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.596 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:57.596 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279761+0000", 2024-09-14T23:57:57.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9402000000000002e-05 2024-09-14T23:57:57.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.596 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:57.596 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.285680+0000", 2024-09-14T23:57:57.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.005918881 2024-09-14T23:57:57.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.597 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:57.597 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.285717+0000", 2024-09-14T23:57:57.597 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6974000000000002e-05 2024-09-14T23:57:57.597 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:57.597 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:57.597 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:57.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.597 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56388 3.c e56/49 3:30ee4ba0:::rbd_data.1118c4b3aebe.0000000000000004:head v 56'199, mlcod=56'192)", 2024-09-14T23:57:57.597 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:57:57.597 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.268695342999997, 2024-09-14T23:57:57.597 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0060596740000000001, 2024-09-14T23:57:57.598 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:57.598 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:57.598 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:57.598 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:57.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.598 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:57.598 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:57:57.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:57.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.598 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:57.598 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:57:57.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:57.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.599 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:57.599 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:57:57.599 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2200000000000003e-07 2024-09-14T23:57:57.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.599 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:57.599 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279995+0000", 2024-09-14T23:57:57.599 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5569999999999997e-06 2024-09-14T23:57:57.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.599 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:57.599 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280004+0000", 2024-09-14T23:57:57.599 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0080000000000008e-06 2024-09-14T23:57:57.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.600 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:57.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280010+0000", 2024-09-14T23:57:57.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5749999999999997e-06 2024-09-14T23:57:57.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.600 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:57.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282968+0000", 2024-09-14T23:57:57.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0029579390000000001 2024-09-14T23:57:57.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.600 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:57.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282998+0000", 2024-09-14T23:57:57.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9947999999999999e-05 2024-09-14T23:57:57.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:57.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.286012+0000", 2024-09-14T23:57:57.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030132900000000001 2024-09-14T23:57:57.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:57.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.286049+0000", 2024-09-14T23:57:57.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7534999999999999e-05 2024-09-14T23:57:57.601 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:57.601 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:57.601 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:57.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.602 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56382 3.1d e56/49 3:bcf9a8cf:::rbd_data.1118c4b3aebe.0000000000000006:head v 56'212, mlcod=56'208)", 2024-09-14T23:57:57.602 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.280285+0000", 2024-09-14T23:57:57.602 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.26839966, 2024-09-14T23:57:57.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035393489999999998, 2024-09-14T23:57:57.602 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:57.602 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:57.602 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:57.602 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:57.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.602 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:57.602 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280285+0000", 2024-09-14T23:57:57.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:57.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:57.603 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280285+0000", 2024-09-14T23:57:57.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:57.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:57.603 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280286+0000", 2024-09-14T23:57:57.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5000000000000001e-07 2024-09-14T23:57:57.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:57.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280299+0000", 2024-09-14T23:57:57.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3312000000000001e-05 2024-09-14T23:57:57.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:57.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280311+0000", 2024-09-14T23:57:57.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1677e-05 2024-09-14T23:57:57.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:57.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280318+0000", 2024-09-14T23:57:57.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.742e-06 2024-09-14T23:57:57.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:57.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280374+0000", 2024-09-14T23:57:57.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.592e-05 2024-09-14T23:57:57.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:57.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280392+0000", 2024-09-14T23:57:57.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8490000000000001e-05 2024-09-14T23:57:57.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:57.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.283693+0000", 2024-09-14T23:57:57.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033012050000000002 2024-09-14T23:57:57.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.606 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:57.606 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.283825+0000", 2024-09-14T23:57:57.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013105300000000001 2024-09-14T23:57:57.606 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:57.606 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:57.606 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:57.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.606 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "pg_update_log_missing(3.c epoch 56/49 rep_tid 1493 entries 56'200 (0'0) error 3:370203a3:::rbd_data.1118c4b3aebe.000000000000000d:head by client.4376.0:56396 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 56'192)", 2024-09-14T23:57:57.606 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:57:57.606 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.268175202999998, 2024-09-14T23:57:57.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057370099999999999, 2024-09-14T23:57:57.606 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:57.607 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:57.607 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:57:57.607 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:57.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.607 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:57.607 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:57:57.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:57.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.607 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:57.607 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:57:57.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:57.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.607 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:57.607 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:57:57.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0699999999999997e-07 2024-09-14T23:57:57.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:57.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280519+0000", 2024-09-14T23:57:57.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6260000000000001e-06 2024-09-14T23:57:57.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:57.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280541+0000", 2024-09-14T23:57:57.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1976000000000001e-05 2024-09-14T23:57:57.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:57.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280556+0000", 2024-09-14T23:57:57.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5075000000000001e-05 2024-09-14T23:57:57.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:57.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.285494+0000", 2024-09-14T23:57:57.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0049378690000000001 2024-09-14T23:57:57.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:57.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.286247+0000", 2024-09-14T23:57:57.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00075285700000000001 2024-09-14T23:57:57.609 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:57.610 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:57.610 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:57.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.610 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56400 3.1c e56/49 3:3f5b81ba:::rbd_data.1118c4b3aebe.000000000000000a:head v 56'236, mlcod=56'234)", 2024-09-14T23:57:57.610 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.295064+0000", 2024-09-14T23:57:57.610 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.253620411, 2024-09-14T23:57:57.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0054331589999999999, 2024-09-14T23:57:57.610 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:57.610 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:57.610 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:57.610 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:57.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:57.611 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.295064+0000", 2024-09-14T23:57:57.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:57.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.611 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:57.611 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.295064+0000", 2024-09-14T23:57:57.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:57.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.611 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:57.611 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.295066+0000", 2024-09-14T23:57:57.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.502e-06 2024-09-14T23:57:57.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.611 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:57.612 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296403+0000", 2024-09-14T23:57:57.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001336554 2024-09-14T23:57:57.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.612 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:57.612 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296428+0000", 2024-09-14T23:57:57.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5179000000000001e-05 2024-09-14T23:57:57.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.612 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:57.612 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296440+0000", 2024-09-14T23:57:57.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2387e-05 2024-09-14T23:57:57.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:57.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296554+0000", 2024-09-14T23:57:57.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114269 2024-09-14T23:57:57.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:57.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296581+0000", 2024-09-14T23:57:57.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6673999999999999e-05 2024-09-14T23:57:57.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:57.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.300418+0000", 2024-09-14T23:57:57.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0038371249999999998 2024-09-14T23:57:57.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.614 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:57.614 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.300498+0000", 2024-09-14T23:57:57.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9468999999999994e-05 2024-09-14T23:57:57.614 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:57.614 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:57.614 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:57.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.614 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56401 3.1c e56/49 3:3f5b81ba:::rbd_data.1118c4b3aebe.000000000000000a:head v 56'238, mlcod=56'234)", 2024-09-14T23:57:57.614 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.297640+0000", 2024-09-14T23:57:57.614 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.251044571999998, 2024-09-14T23:57:57.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031491919999999999, 2024-09-14T23:57:57.614 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:57.615 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:57.615 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:57.615 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:57.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.615 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:57.615 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.297640+0000", 2024-09-14T23:57:57.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:57.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.615 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:57.615 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.297640+0000", 2024-09-14T23:57:57.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:57.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:57.616 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.297641+0000", 2024-09-14T23:57:57.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9000000000000005e-07 2024-09-14T23:57:57.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:57.616 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.297649+0000", 2024-09-14T23:57:57.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4510000000000003e-06 2024-09-14T23:57:57.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:57.616 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.297663+0000", 2024-09-14T23:57:57.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4262e-05 2024-09-14T23:57:57.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:57.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.297679+0000", 2024-09-14T23:57:57.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5798e-05 2024-09-14T23:57:57.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:57.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.298408+0000", 2024-09-14T23:57:57.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00072923699999999998 2024-09-14T23:57:57.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:57.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.298430+0000", 2024-09-14T23:57:57.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1753999999999999e-05 2024-09-14T23:57:57.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:57.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.300749+0000", 2024-09-14T23:57:57.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023190590000000001 2024-09-14T23:57:57.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:57.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.300789+0000", 2024-09-14T23:57:57.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0640999999999999e-05 2024-09-14T23:57:57.618 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:57.618 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:57.618 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:57.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.618 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:61921 3.5 e57/49 3:a05d004f:::rbd_data.11188d356229.000000000000000b:head v 57'279, mlcod=57'277)", 2024-09-14T23:57:57.619 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:07.560389+0000", 2024-09-14T23:57:57.619 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 49.988295813000001, 2024-09-14T23:57:57.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0034956520000000001, 2024-09-14T23:57:57.619 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:57.619 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:57.619 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:57.619 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:57.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.619 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:57.619 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.560389+0000", 2024-09-14T23:57:57.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:57.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.619 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:57.619 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.560389+0000", 2024-09-14T23:57:57.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:57.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:57.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.560390+0000", 2024-09-14T23:57:57.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4109999999999999e-06 2024-09-14T23:57:57.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:57.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.561839+0000", 2024-09-14T23:57:57.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001448406 2024-09-14T23:57:57.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:57.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.561854+0000", 2024-09-14T23:57:57.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5426000000000001e-05 2024-09-14T23:57:57.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:57.621 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.561863+0000", 2024-09-14T23:57:57.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7509999999999997e-06 2024-09-14T23:57:57.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:57.621 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.561917+0000", 2024-09-14T23:57:57.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3531e-05 2024-09-14T23:57:57.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:57.621 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.561944+0000", 2024-09-14T23:57:57.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7528000000000001e-05 2024-09-14T23:57:57.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:57.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563843+0000", 2024-09-14T23:57:57.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001899361 2024-09-14T23:57:57.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:57.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563885+0000", 2024-09-14T23:57:57.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1238000000000001e-05 2024-09-14T23:57:57.622 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:57.622 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:57.622 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:57.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.623 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:61922 3.5 e57/49 3:a11ebf8e:::rbd_data.11188d356229.000000000000000c:head v 57'280, mlcod=57'277)", 2024-09-14T23:57:57.623 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:07.563144+0000", 2024-09-14T23:57:57.623 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 49.985540759000003, 2024-09-14T23:57:57.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003977848, 2024-09-14T23:57:57.623 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:57.623 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:57.623 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:57.623 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:57.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:57.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563144+0000", 2024-09-14T23:57:57.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:57.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.624 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:57.624 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563144+0000", 2024-09-14T23:57:57.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:57.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.624 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:57.624 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563145+0000", 2024-09-14T23:57:57.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5400000000000001e-07 2024-09-14T23:57:57.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.624 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:57.624 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565056+0000", 2024-09-14T23:57:57.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0019108160000000001 2024-09-14T23:57:57.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.625 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:57.625 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565088+0000", 2024-09-14T23:57:57.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2586999999999998e-05 2024-09-14T23:57:57.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.625 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:57.625 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565100+0000", 2024-09-14T23:57:57.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1917e-05 2024-09-14T23:57:57.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:57.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565166+0000", 2024-09-14T23:57:57.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6245e-05 2024-09-14T23:57:57.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:57.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565190+0000", 2024-09-14T23:57:57.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3825999999999999e-05 2024-09-14T23:57:57.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:57.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.567080+0000", 2024-09-14T23:57:57.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001889541 2024-09-14T23:57:57.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:57.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.567122+0000", 2024-09-14T23:57:57.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2262000000000003e-05 2024-09-14T23:57:57.627 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:57.627 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:57.627 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:57.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.627 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:66564 3.c e60/49 3:33faa7ca:::rbd_data.111873792d9b.0000000000000003:head v 60'236, mlcod=57'233)", 2024-09-14T23:57:57.627 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:11.271596+0000", 2024-09-14T23:57:57.627 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 46.277089181000001, 2024-09-14T23:57:57.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0060396099999999999, 2024-09-14T23:57:57.627 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:57.627 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:57.627 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:57.628 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:57.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:57.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.271596+0000", 2024-09-14T23:57:57.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:57.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:57.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.271596+0000", 2024-09-14T23:57:57.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:57.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:57.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.271597+0000", 2024-09-14T23:57:57.629 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2559999999999999e-06 2024-09-14T23:57:57.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.629 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:57.629 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.274854+0000", 2024-09-14T23:57:57.629 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0032574470000000001 2024-09-14T23:57:57.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.629 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:57.629 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.274874+0000", 2024-09-14T23:57:57.629 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9449999999999998e-05 2024-09-14T23:57:57.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:57.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.274883+0000", 2024-09-14T23:57:57.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5980000000000003e-06 2024-09-14T23:57:57.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:57.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.274977+0000", 2024-09-14T23:57:57.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3270999999999995e-05 2024-09-14T23:57:57.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:57.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.275000+0000", 2024-09-14T23:57:57.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3088e-05 2024-09-14T23:57:57.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:57.631 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.277590+0000", 2024-09-14T23:57:57.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0025898660000000001 2024-09-14T23:57:57.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.631 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:57.631 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.277635+0000", 2024-09-14T23:57:57.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5633999999999997e-05 2024-09-14T23:57:57.631 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:57.631 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:57.631 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:57.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.631 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:130509 3.2 e88/75 3:4432e346:::rbd_data.111847077d5e.0000000000000005:head v 88'468, mlcod=87'466)", 2024-09-14T23:57:57.631 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:40.669710+0000", 2024-09-14T23:57:57.632 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 16.878975334, 2024-09-14T23:57:57.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031769279999999999, 2024-09-14T23:57:57.632 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:57.632 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:57.632 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:57.632 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:57.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.632 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:57.632 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:40.669710+0000", 2024-09-14T23:57:57.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:57.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.632 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:57.632 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:40.669710+0000", 2024-09-14T23:57:57.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:57.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.633 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:57.633 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:40.669710+0000", 2024-09-14T23:57:57.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2699999999999999e-07 2024-09-14T23:57:57.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.633 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:57.633 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:40.671376+0000", 2024-09-14T23:57:57.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016661480000000001 2024-09-14T23:57:57.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.633 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:57.633 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:40.671389+0000", 2024-09-14T23:57:57.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2311e-05 2024-09-14T23:57:57.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:57.634 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:40.671395+0000", 2024-09-14T23:57:57.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6869999999999997e-06 2024-09-14T23:57:57.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:57.634 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:40.671451+0000", 2024-09-14T23:57:57.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5865000000000001e-05 2024-09-14T23:57:57.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:57.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:40.671475+0000", 2024-09-14T23:57:57.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3266999999999999e-05 2024-09-14T23:57:57.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:57.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:40.672858+0000", 2024-09-14T23:57:57.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0013835340000000001 2024-09-14T23:57:57.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:57.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:40.672886+0000", 2024-09-14T23:57:57.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8388999999999999e-05 2024-09-14T23:57:57.635 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:57.635 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:57.635 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:57.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.636 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:132869 3.16 e92/75 3:681b4799:::rbd_data.11186132a8b6.0000000000000001:head v 92'262, mlcod=87'260)", 2024-09-14T23:57:57.636 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:45.195492+0000", 2024-09-14T23:57:57.636 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 12.353192428, 2024-09-14T23:57:57.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0034654880000000001, 2024-09-14T23:57:57.636 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:57.636 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:57.636 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:57.636 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:57.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:57.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.195492+0000", 2024-09-14T23:57:57.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:57.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.637 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:57.637 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.195492+0000", 2024-09-14T23:57:57.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:57.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.637 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:57.637 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.195494+0000", 2024-09-14T23:57:57.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.905e-06 2024-09-14T23:57:57.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.637 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:57.637 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.196549+0000", 2024-09-14T23:57:57.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010545489999999999 2024-09-14T23:57:57.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.638 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:57.638 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.196572+0000", 2024-09-14T23:57:57.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3603000000000001e-05 2024-09-14T23:57:57.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.638 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:57.638 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.196584+0000", 2024-09-14T23:57:57.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.169e-05 2024-09-14T23:57:57.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.638 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:57.638 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.196676+0000", 2024-09-14T23:57:57.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2088000000000003e-05 2024-09-14T23:57:57.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:57.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.196699+0000", 2024-09-14T23:57:57.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2365999999999999e-05 2024-09-14T23:57:57.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:57.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.198910+0000", 2024-09-14T23:57:57.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0022118609999999999 2024-09-14T23:57:57.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:57.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.198958+0000", 2024-09-14T23:57:57.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7426000000000003e-05 2024-09-14T23:57:57.640 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:57.640 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:57.640 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:57.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.640 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:133384 3.1f e92/49 3:fce5e529:::rbd_data.11186132a8b6.0000000000000002:head v 92'807, mlcod=89'804)", 2024-09-14T23:57:57.640 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:45.293642+0000", 2024-09-14T23:57:57.640 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 12.255043298, 2024-09-14T23:57:57.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003760186, 2024-09-14T23:57:57.640 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:57.640 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:57.640 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:57.640 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:57.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:57.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.293642+0000", 2024-09-14T23:57:57.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:57.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:57.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.293642+0000", 2024-09-14T23:57:57.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:57.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:57.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.293643+0000", 2024-09-14T23:57:57.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2500000000000001e-06 2024-09-14T23:57:57.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:57.642 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.295402+0000", 2024-09-14T23:57:57.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017595410000000001 2024-09-14T23:57:57.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:57.642 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.295422+0000", 2024-09-14T23:57:57.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9763999999999999e-05 2024-09-14T23:57:57.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:57.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.295436+0000", 2024-09-14T23:57:57.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.364e-05 2024-09-14T23:57:57.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:57.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.295499+0000", 2024-09-14T23:57:57.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2770999999999999e-05 2024-09-14T23:57:57.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:57.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.295516+0000", 2024-09-14T23:57:57.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7240000000000001e-05 2024-09-14T23:57:57.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:57.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.297324+0000", 2024-09-14T23:57:57.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001807857 2024-09-14T23:57:57.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:57.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.297402+0000", 2024-09-14T23:57:57.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8122999999999997e-05 2024-09-14T23:57:57.644 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:57.644 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:57.644 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:57.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.644 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:145282 3.1c e100/49 3:3dcd75bf:::rbd_data.1118584371b4.0000000000000004:head v 100'647, mlcod=100'644)", 2024-09-14T23:57:57.645 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:54.171268+0000", 2024-09-14T23:57:57.645 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.377417211, 2024-09-14T23:57:57.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003527353, 2024-09-14T23:57:57.645 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:57.645 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:57.645 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:57.645 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:57.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:57.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:54.171268+0000", 2024-09-14T23:57:57.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:57.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:57.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:54.171268+0000", 2024-09-14T23:57:57.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:57.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:57.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:54.171268+0000", 2024-09-14T23:57:57.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4900000000000005e-07 2024-09-14T23:57:57.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:57.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:54.172187+0000", 2024-09-14T23:57:57.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00091850799999999995 2024-09-14T23:57:57.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:57.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:54.172199+0000", 2024-09-14T23:57:57.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2544000000000001e-05 2024-09-14T23:57:57.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:57.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:54.172205+0000", 2024-09-14T23:57:57.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1490000000000002e-06 2024-09-14T23:57:57.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:57.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:54.172267+0000", 2024-09-14T23:57:57.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2617e-05 2024-09-14T23:57:57.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:57.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:54.172283+0000", 2024-09-14T23:57:57.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5758000000000001e-05 2024-09-14T23:57:57.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:57.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:54.174754+0000", 2024-09-14T23:57:57.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0024711400000000001 2024-09-14T23:57:57.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:57.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:57.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:57.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:54.174795+0000", 2024-09-14T23:57:57.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0887999999999999e-05 2024-09-14T23:57:57.649 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:57.649 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:57.649 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:57.649 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:57.649 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:57.649 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:57.650 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T23:57:57.651 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:57.648+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:57.751 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:57.749+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:57.798 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:57.798 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:57.798 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:57:57.798 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:57.807 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T23:57:57.852 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:57.851+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:57.913 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:57.913 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:57.914 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:57:57.914 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:57:57.914 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:57.922 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T23:57:57.965 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:57.963+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:58.030 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:58.031 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:57:58.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:57:58.031 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:57:58.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.031 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:107472 3.1e 3:7bf21588:::rbd_data.11187de8b0a5.000000000000005d:head [set-alloc-hint object_size 2097152 write_size 2097152,write 317788~1491] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-14T23:57:58.031 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:25.198493+0000", 2024-09-14T23:57:58.031 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 32.829804383999999, 2024-09-14T23:57:58.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.78175149300000002, 2024-09-14T23:57:58.031 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:58.031 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:58.031 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:58.031 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:58.031 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:58.031 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:58.032 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 107472 2024-09-14T23:57:58.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.032 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:58.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:58.032 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198493+0000", 2024-09-14T23:57:58.032 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:58.032 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198493+0000", 2024-09-14T23:57:58.032 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.033 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:58.033 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198495+0000", 2024-09-14T23:57:58.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.826e-06 2024-09-14T23:57:58.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.033 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:58.033 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198503+0000", 2024-09-14T23:57:58.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0150000000000003e-06 2024-09-14T23:57:58.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.033 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:58.033 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198521+0000", 2024-09-14T23:57:58.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8374000000000001e-05 2024-09-14T23:57:58.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.034 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:58.034 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198533+0000", 2024-09-14T23:57:58.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1593000000000001e-05 2024-09-14T23:57:58.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.034 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:58.034 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198603+0000", 2024-09-14T23:57:58.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0279999999999998e-05 2024-09-14T23:57:58.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.034 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:57:58.034 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198617+0000", 2024-09-14T23:57:58.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3517e-05 2024-09-14T23:57:58.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:58.035 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.977067+0000", 2024-09-14T23:57:58.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.77845065800000002 2024-09-14T23:57:58.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for missing object", 2024-09-14T23:57:58.035 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.977173+0000", 2024-09-14T23:57:58.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010543999999999999 2024-09-14T23:57:58.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:58.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.978737+0000", 2024-09-14T23:57:58.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001564078 2024-09-14T23:57:58.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:58.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.978845+0000", 2024-09-14T23:57:58.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000108643 2024-09-14T23:57:58.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1", 2024-09-14T23:57:58.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.978964+0000", 2024-09-14T23:57:58.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000119085 2024-09-14T23:57:58.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.037 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:58.037 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980109+0000", 2024-09-14T23:57:58.037 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0011442170000000001 2024-09-14T23:57:58.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.037 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.037 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:57:58.037 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980130+0000", 2024-09-14T23:57:58.037 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1766000000000001e-05 2024-09-14T23:57:58.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.037 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.037 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:58.037 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980171+0000", 2024-09-14T23:57:58.037 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0627999999999998e-05 2024-09-14T23:57:58.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.038 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.038 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:58.038 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980244+0000", 2024-09-14T23:57:58.038 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3373000000000004e-05 2024-09-14T23:57:58.038 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.038 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:58.038 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.038 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.038 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129149 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 6619136~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:58.038 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.925871+0000", 2024-09-14T23:57:58.038 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.102425942, 2024-09-14T23:57:58.038 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054291066999999998, 2024-09-14T23:57:58.038 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:58.038 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:58.039 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:58.039 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:58.039 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:58.039 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:58.039 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129149 2024-09-14T23:57:58.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.039 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:58.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:58.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.925871+0000", 2024-09-14T23:57:58.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:58.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.925871+0000", 2024-09-14T23:57:58.040 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.040 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:58.040 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.927669+0000", 2024-09-14T23:57:58.040 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001797838 2024-09-14T23:57:58.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.040 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:58.040 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.927672+0000", 2024-09-14T23:57:58.040 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5280000000000002e-06 2024-09-14T23:57:58.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.040 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:58.041 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.927678+0000", 2024-09-14T23:57:58.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0340000000000002e-06 2024-09-14T23:57:58.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.041 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.041 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:58.041 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.927681+0000", 2024-09-14T23:57:58.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.681e-06 2024-09-14T23:57:58.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.041 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.041 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:58.041 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.979979+0000", 2024-09-14T23:57:58.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052297950000000003 2024-09-14T23:57:58.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.041 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.042 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:58.042 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.980024+0000", 2024-09-14T23:57:58.042 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4693000000000001e-05 2024-09-14T23:57:58.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.042 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.042 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:58.042 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.980162+0000", 2024-09-14T23:57:58.042 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000138343 2024-09-14T23:57:58.042 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.042 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:58.042 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.042 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.042 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129159 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 5996544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:58.042 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.927783+0000", 2024-09-14T23:57:58.043 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.100514492999999, 2024-09-14T23:57:58.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054661122999999999, 2024-09-14T23:57:58.043 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:58.043 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:58.043 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:58.043 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:58.043 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:58.043 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:58.043 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129159 2024-09-14T23:57:58.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.043 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:58.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.043 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:58.043 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.927783+0000", 2024-09-14T23:57:58.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.044 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:58.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.927783+0000", 2024-09-14T23:57:58.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.044 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:58.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929684+0000", 2024-09-14T23:57:58.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0019011080000000001 2024-09-14T23:57:58.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.044 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:58.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929688+0000", 2024-09-14T23:57:58.045 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2610000000000004e-06 2024-09-14T23:57:58.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.045 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.045 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:58.045 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929697+0000", 2024-09-14T23:57:58.045 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0219999999999993e-06 2024-09-14T23:57:58.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.045 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.045 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:58.045 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929702+0000", 2024-09-14T23:57:58.045 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8840000000000002e-06 2024-09-14T23:57:58.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.045 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.045 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:58.046 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.982090+0000", 2024-09-14T23:57:58.046 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052388045000000001 2024-09-14T23:57:58.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.046 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.046 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:58.046 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.982208+0000", 2024-09-14T23:57:58.046 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118236 2024-09-14T23:57:58.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.046 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.046 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:58.046 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.982444+0000", 2024-09-14T23:57:58.046 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000235567 2024-09-14T23:57:58.046 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.046 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:58.047 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.047 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129166 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 8192000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:58.047 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.929794+0000", 2024-09-14T23:57:58.047 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.098503741999998, 2024-09-14T23:57:58.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054471973, 2024-09-14T23:57:58.047 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:58.047 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:58.047 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:58.047 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:58.047 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:58.047 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:58.047 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129166 2024-09-14T23:57:58.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.047 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:58.048 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.048 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:58.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929794+0000", 2024-09-14T23:57:58.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.048 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.048 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:58.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929794+0000", 2024-09-14T23:57:58.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.048 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.048 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:58.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929794+0000", 2024-09-14T23:57:58.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6899999999999997e-07 2024-09-14T23:57:58.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.049 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.049 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:58.049 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929797+0000", 2024-09-14T23:57:58.049 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.328e-06 2024-09-14T23:57:58.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.049 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.049 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:58.049 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929802+0000", 2024-09-14T23:57:58.049 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7309999999999998e-06 2024-09-14T23:57:58.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.049 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.049 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:58.049 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929807+0000", 2024-09-14T23:57:58.050 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4349999999999999e-06 2024-09-14T23:57:58.050 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.050 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.050 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:58.050 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984063+0000", 2024-09-14T23:57:58.050 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054256331999999997 2024-09-14T23:57:58.050 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.050 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.050 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:58.050 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984177+0000", 2024-09-14T23:57:58.050 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114225 2024-09-14T23:57:58.050 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.050 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.050 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:58.050 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984266+0000", 2024-09-14T23:57:58.050 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8352999999999999e-05 2024-09-14T23:57:58.051 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.051 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:58.051 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.051 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.051 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129167 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 6782976~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:58.051 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.929827+0000", 2024-09-14T23:57:58.051 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.098470333000002, 2024-09-14T23:57:58.051 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054637593999999998, 2024-09-14T23:57:58.051 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:58.051 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:58.051 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:58.051 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:58.051 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:58.051 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:58.052 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129167 2024-09-14T23:57:58.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.052 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:58.052 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.052 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:58.052 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929827+0000", 2024-09-14T23:57:58.052 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.052 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.052 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:58.052 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929827+0000", 2024-09-14T23:57:58.052 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.053 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:58.053 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929827+0000", 2024-09-14T23:57:58.053 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4000000000000002e-07 2024-09-14T23:57:58.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.053 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:58.053 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929830+0000", 2024-09-14T23:57:58.053 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.419e-06 2024-09-14T23:57:58.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.053 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:58.053 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929837+0000", 2024-09-14T23:57:58.053 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0450000000000003e-06 2024-09-14T23:57:58.054 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.054 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.054 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:58.054 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929842+0000", 2024-09-14T23:57:58.054 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.69e-06 2024-09-14T23:57:58.054 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.054 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.054 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:58.054 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984278+0000", 2024-09-14T23:57:58.054 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054436516999999997 2024-09-14T23:57:58.054 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.054 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.054 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:58.055 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984331+0000", 2024-09-14T23:57:58.055 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2590999999999998e-05 2024-09-14T23:57:58.055 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.055 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.055 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:58.055 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984465+0000", 2024-09-14T23:57:58.055 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013389200000000001 2024-09-14T23:57:58.055 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.055 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:58.055 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.055 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.055 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.055 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129168 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 8224768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:58.055 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.929849+0000", 2024-09-14T23:57:58.056 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.098448775000001, 2024-09-14T23:57:58.056 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054870368000000003, 2024-09-14T23:57:58.056 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:58.056 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:58.056 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:58.056 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:58.056 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:58.056 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:58.056 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129168 2024-09-14T23:57:58.056 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.056 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:58.056 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.056 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:58.056 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929849+0000", 2024-09-14T23:57:58.057 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.057 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.057 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:58.057 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929849+0000", 2024-09-14T23:57:58.057 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.057 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.057 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:58.057 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931681+0000", 2024-09-14T23:57:58.057 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018326989999999999 2024-09-14T23:57:58.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.057 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.057 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:58.058 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931686+0000", 2024-09-14T23:57:58.058 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7190000000000001e-06 2024-09-14T23:57:58.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.058 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.058 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:58.058 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931697+0000", 2024-09-14T23:57:58.058 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1287999999999999e-05 2024-09-14T23:57:58.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.058 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.058 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:58.058 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931704+0000", 2024-09-14T23:57:58.058 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3339999999999998e-06 2024-09-14T23:57:58.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.059 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:58.059 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984505+0000", 2024-09-14T23:57:58.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052801684000000002 2024-09-14T23:57:58.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.059 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:58.059 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984621+0000", 2024-09-14T23:57:58.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000116165 2024-09-14T23:57:58.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.059 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:58.059 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984719+0000", 2024-09-14T23:57:58.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7479000000000006e-05 2024-09-14T23:57:58.060 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.060 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:58.060 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.060 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.060 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.060 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129175 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 9011200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:58.060 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.931806+0000", 2024-09-14T23:57:58.060 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.096491755999999, 2024-09-14T23:57:58.060 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054322384000000001, 2024-09-14T23:57:58.060 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:58.060 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:58.060 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:58.060 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:58.060 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:58.060 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:58.061 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129175 2024-09-14T23:57:58.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.061 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:58.061 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.061 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:58.061 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931806+0000", 2024-09-14T23:57:58.061 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.061 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.061 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:58.061 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931806+0000", 2024-09-14T23:57:58.061 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.061 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.062 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:58.062 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931806+0000", 2024-09-14T23:57:58.062 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2300000000000001e-07 2024-09-14T23:57:58.062 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.062 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.062 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:58.062 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931808+0000", 2024-09-14T23:57:58.062 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1440000000000001e-06 2024-09-14T23:57:58.062 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.062 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.062 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:58.062 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931815+0000", 2024-09-14T23:57:58.062 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.776e-06 2024-09-14T23:57:58.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:58.063 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931821+0000", 2024-09-14T23:57:58.063 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.733e-06 2024-09-14T23:57:58.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:58.063 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.986046+0000", 2024-09-14T23:57:58.063 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054225284999999998 2024-09-14T23:57:58.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:58.063 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.986089+0000", 2024-09-14T23:57:58.063 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2895999999999997e-05 2024-09-14T23:57:58.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:58.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.986128+0000", 2024-09-14T23:57:58.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9026999999999998e-05 2024-09-14T23:57:58.064 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.064 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:58.064 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.064 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129176 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 10452992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:58.064 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.931830+0000", 2024-09-14T23:57:58.065 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.096467014000002, 2024-09-14T23:57:58.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054513292999999997, 2024-09-14T23:57:58.065 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:58.065 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:58.065 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:58.065 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:58.065 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:58.065 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:58.065 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129176 2024-09-14T23:57:58.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.065 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:58.065 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.065 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:58.065 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931830+0000", 2024-09-14T23:57:58.066 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.066 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.066 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.066 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:58.066 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931830+0000", 2024-09-14T23:57:58.066 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.066 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.066 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.066 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:58.066 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.933678+0000", 2024-09-14T23:57:58.066 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018472440000000001 2024-09-14T23:57:58.066 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.066 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.066 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:58.066 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.933686+0000", 2024-09-14T23:57:58.066 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0239999999999997e-06 2024-09-14T23:57:58.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.067 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.067 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:58.067 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.933694+0000", 2024-09-14T23:57:58.067 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8400000000000001e-06 2024-09-14T23:57:58.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.067 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.067 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:58.067 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.933701+0000", 2024-09-14T23:57:58.067 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2870000000000001e-06 2024-09-14T23:57:58.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.067 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.067 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:58.067 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.986165+0000", 2024-09-14T23:57:58.067 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052463900000000001 2024-09-14T23:57:58.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.068 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.068 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:58.068 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.986259+0000", 2024-09-14T23:57:58.068 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4435e-05 2024-09-14T23:57:58.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.068 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.068 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:58.068 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.986344+0000", 2024-09-14T23:57:58.068 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4562999999999997e-05 2024-09-14T23:57:58.068 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.068 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:58.068 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.068 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.068 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129201 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 10289152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:58.069 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.935960+0000", 2024-09-14T23:57:58.069 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.092337552, 2024-09-14T23:57:58.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054338487999999997, 2024-09-14T23:57:58.069 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:58.069 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:58.069 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:58.069 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:58.069 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:58.069 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:58.069 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129201 2024-09-14T23:57:58.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.069 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:58.069 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.069 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:58.070 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935960+0000", 2024-09-14T23:57:58.070 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.070 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:58.070 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935960+0000", 2024-09-14T23:57:58.070 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.070 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:58.070 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935960+0000", 2024-09-14T23:57:58.070 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.72e-07 2024-09-14T23:57:58.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.070 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:58.071 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935963+0000", 2024-09-14T23:57:58.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5390000000000002e-06 2024-09-14T23:57:58.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.071 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.071 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:58.071 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935971+0000", 2024-09-14T23:57:58.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4589999999999998e-06 2024-09-14T23:57:58.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.071 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.071 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:58.071 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935977+0000", 2024-09-14T23:57:58.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4110000000000004e-06 2024-09-14T23:57:58.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.071 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.071 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:58.071 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990213+0000", 2024-09-14T23:57:58.072 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054236642000000002 2024-09-14T23:57:58.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.072 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.072 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:58.072 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990257+0000", 2024-09-14T23:57:58.072 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3696000000000003e-05 2024-09-14T23:57:58.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.072 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.072 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:58.072 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990298+0000", 2024-09-14T23:57:58.072 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1369000000000003e-05 2024-09-14T23:57:58.072 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.072 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:58.072 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.073 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.073 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129202 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 11370496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:58.073 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.935986+0000", 2024-09-14T23:57:58.073 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.092310934, 2024-09-14T23:57:58.073 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054543700000000001, 2024-09-14T23:57:58.073 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:58.073 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:58.073 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:58.073 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:58.073 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:58.073 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:58.073 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129202 2024-09-14T23:57:58.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.073 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:58.074 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.074 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:58.074 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935986+0000", 2024-09-14T23:57:58.074 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.074 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.074 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:58.074 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935986+0000", 2024-09-14T23:57:58.074 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.074 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.074 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:58.074 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937678+0000", 2024-09-14T23:57:58.074 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016917849999999999 2024-09-14T23:57:58.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.075 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.075 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:58.075 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937682+0000", 2024-09-14T23:57:58.075 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2400000000000001e-06 2024-09-14T23:57:58.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.075 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.075 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:58.075 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937695+0000", 2024-09-14T23:57:58.075 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2999e-05 2024-09-14T23:57:58.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.075 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.075 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:58.075 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937700+0000", 2024-09-14T23:57:58.076 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8420000000000004e-06 2024-09-14T23:57:58.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.076 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.076 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:58.076 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990336+0000", 2024-09-14T23:57:58.076 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052636060999999998 2024-09-14T23:57:58.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.076 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.076 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:58.076 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990435+0000", 2024-09-14T23:57:58.076 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9015e-05 2024-09-14T23:57:58.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.076 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.076 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:58.077 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990530+0000", 2024-09-14T23:57:58.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4758000000000003e-05 2024-09-14T23:57:58.077 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.077 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:58.077 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.077 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129214 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 11468800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:58.077 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.937939+0000", 2024-09-14T23:57:58.077 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.09035802, 2024-09-14T23:57:58.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054463733, 2024-09-14T23:57:58.077 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:58.077 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:58.078 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:58.078 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:58.078 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:58.078 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:58.078 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129214 2024-09-14T23:57:58.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.078 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:58.078 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.078 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:58.078 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937939+0000", 2024-09-14T23:57:58.078 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.078 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.078 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:58.078 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937939+0000", 2024-09-14T23:57:58.078 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.079 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.079 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.079 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:58.079 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939674+0000", 2024-09-14T23:57:58.079 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017342589999999999 2024-09-14T23:57:58.079 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.079 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.079 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:58.079 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939678+0000", 2024-09-14T23:57:58.079 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2590000000000002e-06 2024-09-14T23:57:58.079 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.079 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.079 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:58.079 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939699+0000", 2024-09-14T23:57:58.079 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1143000000000001e-05 2024-09-14T23:57:58.079 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.080 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.080 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:58.080 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939706+0000", 2024-09-14T23:57:58.080 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0269999999999998e-06 2024-09-14T23:57:58.080 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.080 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.080 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:58.080 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.992221+0000", 2024-09-14T23:57:58.080 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052514981000000002 2024-09-14T23:57:58.080 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.080 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.080 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:58.080 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.992321+0000", 2024-09-14T23:57:58.080 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9625999999999998e-05 2024-09-14T23:57:58.080 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.081 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.081 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:58.081 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.992403+0000", 2024-09-14T23:57:58.081 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2437999999999999e-05 2024-09-14T23:57:58.081 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.081 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:58.081 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.081 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.081 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129225 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 12648448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:58.081 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.939856+0000", 2024-09-14T23:57:58.081 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.088441463999999, 2024-09-14T23:57:58.081 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054305557999999997, 2024-09-14T23:57:58.081 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:58.081 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:58.082 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:58.082 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:58.082 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:58.082 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:58.082 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129225 2024-09-14T23:57:58.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.082 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:58.082 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.082 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:58.082 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939856+0000", 2024-09-14T23:57:58.082 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.082 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.082 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:58.083 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939856+0000", 2024-09-14T23:57:58.083 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.083 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.083 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:58.083 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939856+0000", 2024-09-14T23:57:58.083 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1199999999999999e-07 2024-09-14T23:57:58.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.083 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.083 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:58.083 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939857+0000", 2024-09-14T23:57:58.083 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.054e-06 2024-09-14T23:57:58.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.083 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.083 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:58.083 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939861+0000", 2024-09-14T23:57:58.084 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1459999999999998e-06 2024-09-14T23:57:58.084 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.084 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.084 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:58.084 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939864+0000", 2024-09-14T23:57:58.084 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7539999999999998e-06 2024-09-14T23:57:58.084 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.084 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.084 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:58.084 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.994078+0000", 2024-09-14T23:57:58.084 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054213483 2024-09-14T23:57:58.084 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.084 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.084 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:58.084 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.994119+0000", 2024-09-14T23:57:58.085 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1835000000000002e-05 2024-09-14T23:57:58.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.085 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:58.085 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.994161+0000", 2024-09-14T23:57:58.085 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2073999999999997e-05 2024-09-14T23:57:58.085 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.085 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:58.085 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.085 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129226 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 14090240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:58.085 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.939867+0000", 2024-09-14T23:57:58.085 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.088430091999999, 2024-09-14T23:57:58.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054523573999999998, 2024-09-14T23:57:58.086 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:58.086 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:58.086 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:58.086 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:58.086 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:58.086 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:58.086 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129226 2024-09-14T23:57:58.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.086 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:58.086 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.086 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:58.086 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939867+0000", 2024-09-14T23:57:58.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.087 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:58.087 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939867+0000", 2024-09-14T23:57:58.087 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.087 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:58.087 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941669+0000", 2024-09-14T23:57:58.087 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018021549999999999 2024-09-14T23:57:58.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.087 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:58.087 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941673+0000", 2024-09-14T23:57:58.087 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1269999999999999e-06 2024-09-14T23:57:58.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.088 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:58.088 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941679+0000", 2024-09-14T23:57:58.088 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1879999999999999e-06 2024-09-14T23:57:58.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.088 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.088 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:58.088 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941682+0000", 2024-09-14T23:57:58.088 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7110000000000001e-06 2024-09-14T23:57:58.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.088 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.088 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:58.088 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.994201+0000", 2024-09-14T23:57:58.088 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052518265000000001 2024-09-14T23:57:58.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.088 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.089 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:58.089 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.994302+0000", 2024-09-14T23:57:58.089 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101653 2024-09-14T23:57:58.089 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.089 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.089 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:58.089 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.994391+0000", 2024-09-14T23:57:58.089 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8474999999999996e-05 2024-09-14T23:57:58.089 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.089 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:58.089 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.089 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.089 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.089 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129237 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 12386304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:58.090 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.941825+0000", 2024-09-14T23:57:58.090 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.086472329999999, 2024-09-14T23:57:58.090 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054368076000000001, 2024-09-14T23:57:58.090 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:58.090 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:58.090 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:58.090 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:58.090 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:58.090 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:58.090 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129237 2024-09-14T23:57:58.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.090 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:58.090 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.090 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:58.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941825+0000", 2024-09-14T23:57:58.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:58.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941825+0000", 2024-09-14T23:57:58.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:58.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941825+0000", 2024-09-14T23:57:58.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.29e-07 2024-09-14T23:57:58.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.092 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:58.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941826+0000", 2024-09-14T23:57:58.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0999999999999997e-07 2024-09-14T23:57:58.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.092 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.092 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:58.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941834+0000", 2024-09-14T23:57:58.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5519999999999999e-06 2024-09-14T23:57:58.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.092 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.092 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:58.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941843+0000", 2024-09-14T23:57:58.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2960000000000004e-06 2024-09-14T23:57:58.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.093 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.093 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:58.093 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996108+0000", 2024-09-14T23:57:58.093 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054265200999999999 2024-09-14T23:57:58.093 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.093 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.093 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:58.093 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996152+0000", 2024-09-14T23:57:58.093 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4066999999999997e-05 2024-09-14T23:57:58.093 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.093 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.093 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:58.093 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996193+0000", 2024-09-14T23:57:58.094 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0821000000000001e-05 2024-09-14T23:57:58.094 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.094 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:58.094 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.094 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.094 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.094 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129238 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 13828096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:58.094 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.941851+0000", 2024-09-14T23:57:58.094 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.086446005999999, 2024-09-14T23:57:58.094 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.05458031, 2024-09-14T23:57:58.094 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:58.094 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:58.094 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:58.094 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:58.095 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:58.095 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:58.095 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129238 2024-09-14T23:57:58.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.095 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:58.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:58.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941851+0000", 2024-09-14T23:57:58.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:58.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941851+0000", 2024-09-14T23:57:58.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.096 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:58.096 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943670+0000", 2024-09-14T23:57:58.096 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018181930000000001 2024-09-14T23:57:58.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.096 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:58.096 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943673+0000", 2024-09-14T23:57:58.096 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3790000000000002e-06 2024-09-14T23:57:58.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.096 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:58.096 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943679+0000", 2024-09-14T23:57:58.097 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8719999999999999e-06 2024-09-14T23:57:58.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.097 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.097 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:58.097 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943683+0000", 2024-09-14T23:57:58.097 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8539999999999999e-06 2024-09-14T23:57:58.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.097 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.097 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:58.097 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996233+0000", 2024-09-14T23:57:58.097 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052550690999999997 2024-09-14T23:57:58.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.097 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.097 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:58.098 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996339+0000", 2024-09-14T23:57:58.098 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105989 2024-09-14T23:57:58.098 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.098 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.098 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:58.098 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996432+0000", 2024-09-14T23:57:58.098 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2331999999999996e-05 2024-09-14T23:57:58.098 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.098 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:58.098 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.098 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.098 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.098 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129249 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 12124160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:58.098 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.943826+0000", 2024-09-14T23:57:58.099 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.084471515000001, 2024-09-14T23:57:58.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054373266000000003, 2024-09-14T23:57:58.099 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:58.099 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:58.099 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:58.099 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:58.099 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:58.099 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:58.099 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129249 2024-09-14T23:57:58.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.099 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:58.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.099 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:58.099 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943826+0000", 2024-09-14T23:57:58.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.100 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.100 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:58.100 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943826+0000", 2024-09-14T23:57:58.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.100 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.100 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:58.100 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943826+0000", 2024-09-14T23:57:58.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.03e-07 2024-09-14T23:57:58.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.100 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.100 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:58.101 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943827+0000", 2024-09-14T23:57:58.101 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.08e-06 2024-09-14T23:57:58.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.101 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:58.101 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943834+0000", 2024-09-14T23:57:58.101 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4119999999999998e-06 2024-09-14T23:57:58.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.101 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:58.101 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943836+0000", 2024-09-14T23:57:58.101 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7369999999999998e-06 2024-09-14T23:57:58.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.102 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:58.102 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998115+0000", 2024-09-14T23:57:58.102 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054279040000000001 2024-09-14T23:57:58.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.102 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.102 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:58.102 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998159+0000", 2024-09-14T23:57:58.102 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3486999999999999e-05 2024-09-14T23:57:58.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.102 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.102 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:58.102 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998199+0000", 2024-09-14T23:57:58.102 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0306999999999999e-05 2024-09-14T23:57:58.103 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.103 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:58.103 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.103 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.103 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129250 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 13565952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:58.103 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.943840+0000", 2024-09-14T23:57:58.103 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.084457565000001, 2024-09-14T23:57:58.103 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054573875000000001, 2024-09-14T23:57:58.103 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:58.103 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:58.103 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:58.103 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:58.103 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:58.103 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:58.104 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129250 2024-09-14T23:57:58.104 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.104 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:58.104 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.104 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:58.104 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943840+0000", 2024-09-14T23:57:58.104 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.104 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.104 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.104 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:58.104 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943840+0000", 2024-09-14T23:57:58.104 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.104 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.104 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.104 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:58.105 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945669+0000", 2024-09-14T23:57:58.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018289720000000001 2024-09-14T23:57:58.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.105 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:58.105 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945671+0000", 2024-09-14T23:57:58.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6869999999999999e-06 2024-09-14T23:57:58.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.105 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:58.105 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945680+0000", 2024-09-14T23:57:58.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5259999999999993e-06 2024-09-14T23:57:58.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.106 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:58.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945685+0000", 2024-09-14T23:57:58.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8559999999999998e-06 2024-09-14T23:57:58.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.106 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:58.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998242+0000", 2024-09-14T23:57:58.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052557505999999997 2024-09-14T23:57:58.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.106 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:58.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998329+0000", 2024-09-14T23:57:58.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6410999999999996e-05 2024-09-14T23:57:58.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.107 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:58.107 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998414+0000", 2024-09-14T23:57:58.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4917e-05 2024-09-14T23:57:58.107 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.107 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:58.107 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.107 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129261 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 14385152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:58.107 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.945815+0000", 2024-09-14T23:57:58.107 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.082482272, 2024-09-14T23:57:58.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054331867999999998, 2024-09-14T23:57:58.108 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:58.108 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:58.108 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:58.108 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:58.108 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:58.108 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:58.108 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129261 2024-09-14T23:57:58.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.108 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:58.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:58.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945815+0000", 2024-09-14T23:57:58.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:58.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945815+0000", 2024-09-14T23:57:58.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:58.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945815+0000", 2024-09-14T23:57:58.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0599999999999999e-07 2024-09-14T23:57:58.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:58.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945816+0000", 2024-09-14T23:57:58.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0380000000000001e-06 2024-09-14T23:57:58.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.110 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.110 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:58.110 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945819+0000", 2024-09-14T23:57:58.110 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.005e-06 2024-09-14T23:57:58.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.110 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.110 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:58.110 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945822+0000", 2024-09-14T23:57:58.110 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.452e-06 2024-09-14T23:57:58.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.110 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:58.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000060+0000", 2024-09-14T23:57:58.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054238606000000002 2024-09-14T23:57:58.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:58.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000106+0000", 2024-09-14T23:57:58.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5188000000000002e-05 2024-09-14T23:57:58.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.113 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:58.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000147+0000", 2024-09-14T23:57:58.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1372999999999998e-05 2024-09-14T23:57:58.113 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.113 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:58.113 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.113 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129262 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 13303808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:58.113 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.945825+0000", 2024-09-14T23:57:58.113 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.082472624000001, 2024-09-14T23:57:58.114 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054515942999999997, 2024-09-14T23:57:58.114 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:58.114 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:58.114 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:58.114 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:58.114 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:58.114 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:58.114 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129262 2024-09-14T23:57:58.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.114 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:58.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.114 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:58.114 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945825+0000", 2024-09-14T23:57:58.114 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.115 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:58.115 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945825+0000", 2024-09-14T23:57:58.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.115 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:58.115 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.947670+0000", 2024-09-14T23:57:58.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018451229999999999 2024-09-14T23:57:58.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.115 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:58.115 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.947672+0000", 2024-09-14T23:57:58.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5289999999999998e-06 2024-09-14T23:57:58.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.116 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.116 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:58.116 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.947678+0000", 2024-09-14T23:57:58.116 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9880000000000001e-06 2024-09-14T23:57:58.116 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.116 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.116 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:58.116 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.947682+0000", 2024-09-14T23:57:58.116 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.63e-06 2024-09-14T23:57:58.116 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.116 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.116 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:58.116 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000183+0000", 2024-09-14T23:57:58.116 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052500770000000002 2024-09-14T23:57:58.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:58.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000266+0000", 2024-09-14T23:57:58.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3133000000000005e-05 2024-09-14T23:57:58.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:58.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000341+0000", 2024-09-14T23:57:58.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4770000000000004e-05 2024-09-14T23:57:58.117 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.117 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:58.117 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.118 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.118 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129299 3.10 3:0f1e06ee:::rbd_object_map.1118721be852:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:57:58.118 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.952330+0000", 2024-09-14T23:57:58.118 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.075967811000002, 2024-09-14T23:57:58.118 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054412818000000002, 2024-09-14T23:57:58.118 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:58.118 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:58.118 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:58.118 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:58.118 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:58.118 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:58.118 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129299 2024-09-14T23:57:58.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.118 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:58.118 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.118 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:58.119 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952330+0000", 2024-09-14T23:57:58.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.119 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:58.119 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952330+0000", 2024-09-14T23:57:58.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.119 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:58.119 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952330+0000", 2024-09-14T23:57:58.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4900000000000001e-07 2024-09-14T23:57:58.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.119 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:58.120 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952332+0000", 2024-09-14T23:57:58.120 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2879999999999999e-06 2024-09-14T23:57:58.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.120 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.120 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:58.120 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952340+0000", 2024-09-14T23:57:58.120 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9910000000000002e-06 2024-09-14T23:57:58.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.120 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.120 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:58.120 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952344+0000", 2024-09-14T23:57:58.120 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0280000000000002e-06 2024-09-14T23:57:58.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.120 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.120 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:58.121 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.004722+0000", 2024-09-14T23:57:58.121 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052377932000000002 2024-09-14T23:57:58.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.121 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.121 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:58.121 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.004778+0000", 2024-09-14T23:57:58.121 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5795999999999999e-05 2024-09-14T23:57:58.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.121 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.121 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:57:58.121 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.005016+0000", 2024-09-14T23:57:58.121 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000238006 2024-09-14T23:57:58.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.121 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.121 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:57:58.122 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.006366+0000", 2024-09-14T23:57:58.122 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001349786 2024-09-14T23:57:58.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.122 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.122 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:58.122 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.006685+0000", 2024-09-14T23:57:58.122 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00031970000000000002 2024-09-14T23:57:58.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.122 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.122 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:57:58.122 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.006718+0000", 2024-09-14T23:57:58.122 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2329000000000002e-05 2024-09-14T23:57:58.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.122 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.122 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:58.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.006742+0000", 2024-09-14T23:57:58.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4613e-05 2024-09-14T23:57:58.123 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.123 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:58.123 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.123 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.123 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:58.123 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:58.125 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is set 2024-09-14T23:57:58.125 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T23:57:58.126 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:58.123+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:58.201 INFO:teuthology.orchestra.run.smithi042.stdout:leaving image image_client.0-clone45 intact 2024-09-14T23:57:58.225 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:57:58.222+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:57:58.275 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:58.275 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:58.275 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:57:58.275 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:58.283 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T23:57:58.325 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:58.324+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:58.426 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:58.424+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:58.436 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:58.436 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:57:58.437 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:57:58.437 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:57:58.437 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:58.437 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #44, image image_client.0-clone44 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent45 2024-09-14T23:57:58.447 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T23:57:58.481 INFO:teuthology.orchestra.run.smithi042.stdout:5377 write 0x2aa1637 thru 0x2aae5bf (0xcf89 bytes) 2024-09-14T23:57:58.529 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:58.529 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:58.529 INFO:teuthology.orchestra.run.smithi042.stdout:5380 writesame 0xdc67d68 thru 0xdc690be data_size 0x1(0x1357 bytes) 2024-09-14T23:57:58.534 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:58.532+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:58.538 INFO:teuthology.orchestra.run.smithi042.stdout:5382 punch from 0x64498ad to 0x645146c, (0x7bbf bytes) 2024-09-14T23:57:58.539 INFO:teuthology.orchestra.run.smithi042.stdout:5383 read 0xce7487f thru 0xce772b5 (0x2a37 bytes) 2024-09-14T23:57:58.539 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:58.552 INFO:teuthology.orchestra.run.smithi042.stdout:5386 trunc from 0xe814205 to 0x2b4b61d 2024-09-14T23:57:58.598 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:58.598 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:58.599 INFO:teuthology.orchestra.run.smithi042.stdout:5390 read 0x242258 thru 0x2476d6 (0x547f bytes) 2024-09-14T23:57:58.609 INFO:teuthology.orchestra.run.smithi042.stdout:5391 trunc from 0x2b4b61d to 0xb67ffde 2024-09-14T23:57:58.616 INFO:teuthology.orchestra.run.smithi042.stdout:5393 write 0xaa0ed89 thru 0xaa1726f (0x84e7 bytes) 2024-09-14T23:57:58.623 INFO:teuthology.orchestra.run.smithi042.stdout:5394 read 0x6dd49fc thru 0x6dd8d99 (0x439e bytes) 2024-09-14T23:57:58.623 INFO:teuthology.orchestra.run.smithi042.stdout:5395 write 0x14c2cf3 thru 0x14ce244 (0xb552 bytes) 2024-09-14T23:57:58.628 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:58.625+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:58.633 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:57:58.634 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:57:58.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:57:58.634 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:57:58.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.634 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69262 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 4800512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:58.634 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300811+0000", 2024-09-14T23:57:58.634 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 43.329803767000001, 2024-09-14T23:57:58.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050403796000000001, 2024-09-14T23:57:58.634 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:58.634 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:58.634 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:58.634 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:58.634 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:58.635 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:58.635 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69262 2024-09-14T23:57:58.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.635 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:58.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:58.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300811+0000", 2024-09-14T23:57:58.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:58.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300811+0000", 2024-09-14T23:57:58.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:58.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300811+0000", 2024-09-14T23:57:58.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5700000000000002e-07 2024-09-14T23:57:58.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:58.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300815+0000", 2024-09-14T23:57:58.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.087e-06 2024-09-14T23:57:58.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:58.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300823+0000", 2024-09-14T23:57:58.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8720000000000002e-06 2024-09-14T23:57:58.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.637 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:58.637 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300830+0000", 2024-09-14T23:57:58.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0589999999999997e-06 2024-09-14T23:57:58.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.637 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:58.637 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351020+0000", 2024-09-14T23:57:58.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050189552999999998 2024-09-14T23:57:58.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.637 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:58.637 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351112+0000", 2024-09-14T23:57:58.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2615999999999996e-05 2024-09-14T23:57:58.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.638 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:58.638 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351214+0000", 2024-09-14T23:57:58.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101952 2024-09-14T23:57:58.638 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.638 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:58.638 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.638 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69263 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 4915200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:58.638 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300842+0000", 2024-09-14T23:57:58.638 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 43.329772263000002, 2024-09-14T23:57:58.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050590637000000001, 2024-09-14T23:57:58.639 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:58.639 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:58.639 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:58.639 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:58.639 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:58.639 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:58.639 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69263 2024-09-14T23:57:58.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.639 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:58.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:58.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300842+0000", 2024-09-14T23:57:58.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:58.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300842+0000", 2024-09-14T23:57:58.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:58.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300843+0000", 2024-09-14T23:57:58.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4000000000000002e-07 2024-09-14T23:57:58.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:58.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300846+0000", 2024-09-14T23:57:58.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8720000000000002e-06 2024-09-14T23:57:58.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:58.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300853+0000", 2024-09-14T23:57:58.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0450000000000003e-06 2024-09-14T23:57:58.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:58.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300859+0000", 2024-09-14T23:57:58.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9510000000000003e-06 2024-09-14T23:57:58.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:58.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351260+0000", 2024-09-14T23:57:58.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050401654999999997 2024-09-14T23:57:58.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:58.642 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351353+0000", 2024-09-14T23:57:58.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2785e-05 2024-09-14T23:57:58.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:58.642 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351433+0000", 2024-09-14T23:57:58.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9789000000000005e-05 2024-09-14T23:57:58.642 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.642 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:58.642 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.642 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69264 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5029888~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:58.643 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300866+0000", 2024-09-14T23:57:58.643 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 43.329748330000001, 2024-09-14T23:57:58.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050794063, 2024-09-14T23:57:58.643 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:58.643 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:58.643 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:58.643 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:58.643 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:58.643 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:58.643 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69264 2024-09-14T23:57:58.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.643 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:58.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:58.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300866+0000", 2024-09-14T23:57:58.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:58.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300866+0000", 2024-09-14T23:57:58.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:58.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300866+0000", 2024-09-14T23:57:58.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8799999999999998e-07 2024-09-14T23:57:58.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:58.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300869+0000", 2024-09-14T23:57:58.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0880000000000002e-06 2024-09-14T23:57:58.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:58.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300875+0000", 2024-09-14T23:57:58.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9280000000000004e-06 2024-09-14T23:57:58.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:58.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300882+0000", 2024-09-14T23:57:58.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4740000000000002e-06 2024-09-14T23:57:58.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:58.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351472+0000", 2024-09-14T23:57:58.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050590006999999999 2024-09-14T23:57:58.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:58.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351563+0000", 2024-09-14T23:57:58.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0762999999999998e-05 2024-09-14T23:57:58.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:58.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351660+0000", 2024-09-14T23:57:58.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7415000000000002e-05 2024-09-14T23:57:58.646 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.646 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:58.647 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.647 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69265 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5144576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:58.647 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300890+0000", 2024-09-14T23:57:58.647 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 43.329724826000003, 2024-09-14T23:57:58.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050921389999999997, 2024-09-14T23:57:58.647 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:58.647 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:58.647 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:58.647 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:58.647 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:58.647 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:58.647 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69265 2024-09-14T23:57:58.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.648 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:58.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:58.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300890+0000", 2024-09-14T23:57:58.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:58.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300890+0000", 2024-09-14T23:57:58.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:58.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300890+0000", 2024-09-14T23:57:58.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2500000000000001e-07 2024-09-14T23:57:58.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:58.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300892+0000", 2024-09-14T23:57:58.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2469999999999998e-06 2024-09-14T23:57:58.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:58.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300899+0000", 2024-09-14T23:57:58.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5509999999999996e-06 2024-09-14T23:57:58.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:58.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300904+0000", 2024-09-14T23:57:58.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3299999999999998e-06 2024-09-14T23:57:58.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:58.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351707+0000", 2024-09-14T23:57:58.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050802673999999999 2024-09-14T23:57:58.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:58.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351755+0000", 2024-09-14T23:57:58.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8362000000000002e-05 2024-09-14T23:57:58.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:58.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351811+0000", 2024-09-14T23:57:58.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5801000000000003e-05 2024-09-14T23:57:58.651 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.651 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:58.651 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.651 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69266 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5259264~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:58.651 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300911+0000", 2024-09-14T23:57:58.651 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 43.329703752999997, 2024-09-14T23:57:58.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051125089999999998, 2024-09-14T23:57:58.651 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:58.651 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:58.652 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:58.652 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:58.652 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:58.652 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:58.652 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69266 2024-09-14T23:57:58.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.652 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:58.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.652 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:58.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300911+0000", 2024-09-14T23:57:58.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.652 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:58.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300911+0000", 2024-09-14T23:57:58.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:58.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300911+0000", 2024-09-14T23:57:58.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-14T23:57:58.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:58.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300913+0000", 2024-09-14T23:57:58.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9300000000000002e-06 2024-09-14T23:57:58.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:58.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300919+0000", 2024-09-14T23:57:58.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.489e-06 2024-09-14T23:57:58.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:58.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300925+0000", 2024-09-14T23:57:58.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1719999999999998e-06 2024-09-14T23:57:58.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:58.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351847+0000", 2024-09-14T23:57:58.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050922621000000001 2024-09-14T23:57:58.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.655 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:58.655 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351938+0000", 2024-09-14T23:57:58.655 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0830999999999998e-05 2024-09-14T23:57:58.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.655 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:58.655 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352036+0000", 2024-09-14T23:57:58.655 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7676999999999993e-05 2024-09-14T23:57:58.655 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.655 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:58.655 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.655 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69267 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5373952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:58.656 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300945+0000", 2024-09-14T23:57:58.656 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 43.329668957999999, 2024-09-14T23:57:58.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051217934, 2024-09-14T23:57:58.656 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:58.656 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:58.656 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:58.656 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:58.656 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:58.656 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:58.656 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69267 2024-09-14T23:57:58.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.656 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:58.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:58.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300945+0000", 2024-09-14T23:57:58.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:58.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300945+0000", 2024-09-14T23:57:58.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:58.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300946+0000", 2024-09-14T23:57:58.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2600000000000002e-07 2024-09-14T23:57:58.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:58.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300948+0000", 2024-09-14T23:57:58.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4940000000000002e-06 2024-09-14T23:57:58.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:58.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300956+0000", 2024-09-14T23:57:58.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.622e-06 2024-09-14T23:57:58.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:58.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300962+0000", 2024-09-14T23:57:58.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5790000000000003e-06 2024-09-14T23:57:58.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.659 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:58.659 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352063+0000", 2024-09-14T23:57:58.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051101281999999998 2024-09-14T23:57:58.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.659 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:58.659 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352110+0000", 2024-09-14T23:57:58.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7401000000000002e-05 2024-09-14T23:57:58.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.659 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:58.659 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352163+0000", 2024-09-14T23:57:58.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3029999999999999e-05 2024-09-14T23:57:58.660 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.660 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:58.660 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.660 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69268 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5488640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:58.660 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300969+0000", 2024-09-14T23:57:58.660 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 43.329645286000002, 2024-09-14T23:57:58.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051413420000000001, 2024-09-14T23:57:58.660 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:58.660 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:58.660 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:58.660 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:58.660 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:58.661 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:58.661 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69268 2024-09-14T23:57:58.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.661 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:58.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:58.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300969+0000", 2024-09-14T23:57:58.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:58.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300969+0000", 2024-09-14T23:57:58.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:58.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300970+0000", 2024-09-14T23:57:58.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7899999999999999e-07 2024-09-14T23:57:58.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:58.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300982+0000", 2024-09-14T23:57:58.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2741e-05 2024-09-14T23:57:58.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:58.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300990+0000", 2024-09-14T23:57:58.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8169999999999999e-06 2024-09-14T23:57:58.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.663 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:58.663 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300995+0000", 2024-09-14T23:57:58.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2240000000000003e-06 2024-09-14T23:57:58.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.663 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:58.663 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352200+0000", 2024-09-14T23:57:58.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051204223 2024-09-14T23:57:58.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.663 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:58.663 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352285+0000", 2024-09-14T23:57:58.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5519999999999994e-05 2024-09-14T23:57:58.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.664 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.664 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:58.664 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352383+0000", 2024-09-14T23:57:58.664 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7516000000000006e-05 2024-09-14T23:57:58.664 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.664 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:58.664 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.664 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.664 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69269 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5603328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:58.664 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.301003+0000", 2024-09-14T23:57:58.664 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 43.329611104000001, 2024-09-14T23:57:58.665 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051497381000000002, 2024-09-14T23:57:58.665 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:58.665 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:58.665 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:58.665 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:58.665 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:58.665 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:58.665 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69269 2024-09-14T23:57:58.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.665 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:58.665 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.665 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:58.665 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.301003+0000", 2024-09-14T23:57:58.665 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:58.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.301003+0000", 2024-09-14T23:57:58.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:58.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302671+0000", 2024-09-14T23:57:58.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001667785 2024-09-14T23:57:58.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:58.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302676+0000", 2024-09-14T23:57:58.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6600000000000003e-06 2024-09-14T23:57:58.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:58.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302697+0000", 2024-09-14T23:57:58.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1288999999999999e-05 2024-09-14T23:57:58.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:58.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302716+0000", 2024-09-14T23:57:58.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9128999999999999e-05 2024-09-14T23:57:58.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:58.668 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352399+0000", 2024-09-14T23:57:58.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049682995000000001 2024-09-14T23:57:58.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.668 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.668 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:58.668 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352446+0000", 2024-09-14T23:57:58.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6341000000000002e-05 2024-09-14T23:57:58.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.668 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.668 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:58.668 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352501+0000", 2024-09-14T23:57:58.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5182000000000001e-05 2024-09-14T23:57:58.668 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.668 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:58.669 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.669 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69273 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6062080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:58.669 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302829+0000", 2024-09-14T23:57:58.669 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 43.327785538000001, 2024-09-14T23:57:58.669 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050384311000000001, 2024-09-14T23:57:58.669 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:58.669 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:58.669 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:58.669 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:58.669 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:58.669 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:58.670 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69273 2024-09-14T23:57:58.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.670 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:58.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:58.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302829+0000", 2024-09-14T23:57:58.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:58.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302829+0000", 2024-09-14T23:57:58.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:58.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302829+0000", 2024-09-14T23:57:58.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7700000000000005e-07 2024-09-14T23:57:58.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.671 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.671 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:58.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302832+0000", 2024-09-14T23:57:58.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6970000000000002e-06 2024-09-14T23:57:58.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.671 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.671 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:58.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302840+0000", 2024-09-14T23:57:58.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5719999999999997e-06 2024-09-14T23:57:58.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.671 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.671 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:58.672 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302845+0000", 2024-09-14T23:57:58.672 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7939999999999999e-06 2024-09-14T23:57:58.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.672 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.672 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:58.672 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353111+0000", 2024-09-14T23:57:58.672 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050265588 2024-09-14T23:57:58.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.672 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.672 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:58.672 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353160+0000", 2024-09-14T23:57:58.672 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8859000000000001e-05 2024-09-14T23:57:58.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.673 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.673 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:58.673 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353213+0000", 2024-09-14T23:57:58.673 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3324e-05 2024-09-14T23:57:58.673 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.673 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:58.673 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.673 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.673 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69274 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6176768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:58.673 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302854+0000", 2024-09-14T23:57:58.673 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 43.327760443999999, 2024-09-14T23:57:58.673 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050579526, 2024-09-14T23:57:58.673 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:58.674 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:58.674 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:58.674 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:58.674 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:58.674 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:58.674 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69274 2024-09-14T23:57:58.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.674 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:58.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.674 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:58.674 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302854+0000", 2024-09-14T23:57:58.674 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.674 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:58.675 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302854+0000", 2024-09-14T23:57:58.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.675 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:58.675 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302855+0000", 2024-09-14T23:57:58.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0200000000000001e-07 2024-09-14T23:57:58.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.675 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:58.675 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302857+0000", 2024-09-14T23:57:58.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.447e-06 2024-09-14T23:57:58.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.676 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:58.676 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302864+0000", 2024-09-14T23:57:58.676 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7730000000000002e-06 2024-09-14T23:57:58.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.676 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:58.676 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302869+0000", 2024-09-14T23:57:58.676 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3199999999999999e-06 2024-09-14T23:57:58.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.676 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:58.676 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353249+0000", 2024-09-14T23:57:58.676 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050379647999999999 2024-09-14T23:57:58.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.677 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.677 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:58.677 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353336+0000", 2024-09-14T23:57:58.677 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6767999999999993e-05 2024-09-14T23:57:58.677 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.677 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.677 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:58.677 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353433+0000", 2024-09-14T23:57:58.677 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7868000000000006e-05 2024-09-14T23:57:58.677 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.677 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:58.677 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.677 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.677 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.678 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69275 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6291456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:58.678 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302889+0000", 2024-09-14T23:57:58.678 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 43.327725825999998, 2024-09-14T23:57:58.678 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050663118, 2024-09-14T23:57:58.678 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:58.678 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:58.678 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:58.678 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:58.678 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:58.678 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:58.678 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69275 2024-09-14T23:57:58.678 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.678 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:58.678 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.679 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:58.679 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302889+0000", 2024-09-14T23:57:58.679 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.679 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:58.679 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302889+0000", 2024-09-14T23:57:58.679 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.679 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:58.679 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302889+0000", 2024-09-14T23:57:58.679 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6399999999999998e-07 2024-09-14T23:57:58.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.680 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:58.680 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302891+0000", 2024-09-14T23:57:58.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.339e-06 2024-09-14T23:57:58.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.680 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:58.680 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302898+0000", 2024-09-14T23:57:58.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0380000000000002e-06 2024-09-14T23:57:58.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.680 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:58.680 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302904+0000", 2024-09-14T23:57:58.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2159999999999997e-06 2024-09-14T23:57:58.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.681 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:58.681 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353449+0000", 2024-09-14T23:57:58.681 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050545386999999997 2024-09-14T23:57:58.681 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.681 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.681 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:58.681 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353494+0000", 2024-09-14T23:57:58.681 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4592999999999998e-05 2024-09-14T23:57:58.681 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.681 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.681 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:58.681 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353552+0000", 2024-09-14T23:57:58.681 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8180999999999997e-05 2024-09-14T23:57:58.682 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.682 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:58.682 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.682 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.682 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.682 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69276 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6406144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:58.682 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302911+0000", 2024-09-14T23:57:58.682 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 43.327703413000002, 2024-09-14T23:57:58.682 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050865753, 2024-09-14T23:57:58.682 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:58.682 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:58.682 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:58.682 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:58.682 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:58.682 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:58.683 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69276 2024-09-14T23:57:58.683 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.683 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:58.683 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.683 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:58.683 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302911+0000", 2024-09-14T23:57:58.683 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.683 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.683 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.683 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:58.683 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302911+0000", 2024-09-14T23:57:58.683 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.683 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.683 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.684 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:58.684 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302912+0000", 2024-09-14T23:57:58.684 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0800000000000004e-07 2024-09-14T23:57:58.684 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.684 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.684 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:58.684 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302914+0000", 2024-09-14T23:57:58.684 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9589999999999998e-06 2024-09-14T23:57:58.684 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.684 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.684 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:58.684 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302920+0000", 2024-09-14T23:57:58.684 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4579999999999998e-06 2024-09-14T23:57:58.684 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.685 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.685 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:58.685 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302925+0000", 2024-09-14T23:57:58.685 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9429999999999999e-06 2024-09-14T23:57:58.685 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.685 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.685 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:58.685 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353586+0000", 2024-09-14T23:57:58.685 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050660966000000002 2024-09-14T23:57:58.685 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.685 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.685 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:58.685 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353679+0000", 2024-09-14T23:57:58.685 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3163000000000002e-05 2024-09-14T23:57:58.686 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.686 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.686 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:58.686 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353777+0000", 2024-09-14T23:57:58.686 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7555999999999999e-05 2024-09-14T23:57:58.686 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.686 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:58.686 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.686 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.686 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.686 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69277 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6520832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:58.686 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302932+0000", 2024-09-14T23:57:58.686 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 43.327682326000001, 2024-09-14T23:57:58.686 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050977674000000001, 2024-09-14T23:57:58.687 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:58.687 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:58.687 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:58.687 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:58.687 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:58.687 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:58.687 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69277 2024-09-14T23:57:58.687 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.687 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:58.687 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.687 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:58.687 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302932+0000", 2024-09-14T23:57:58.687 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.687 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.687 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.688 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:58.688 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302932+0000", 2024-09-14T23:57:58.688 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.688 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.688 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.688 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:58.688 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302932+0000", 2024-09-14T23:57:58.688 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3299999999999998e-07 2024-09-14T23:57:58.688 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.688 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.688 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:58.688 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302935+0000", 2024-09-14T23:57:58.688 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2139999999999999e-06 2024-09-14T23:57:58.689 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.689 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.689 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:58.689 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302941+0000", 2024-09-14T23:57:58.689 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5039999999999999e-06 2024-09-14T23:57:58.689 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.689 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.689 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:58.689 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302946+0000", 2024-09-14T23:57:58.689 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2079999999999999e-06 2024-09-14T23:57:58.689 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.689 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.689 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:58.689 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353805+0000", 2024-09-14T23:57:58.689 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050858588000000003 2024-09-14T23:57:58.689 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.690 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.690 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:58.690 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353849+0000", 2024-09-14T23:57:58.690 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4511000000000001e-05 2024-09-14T23:57:58.690 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.690 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.690 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:58.690 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353910+0000", 2024-09-14T23:57:58.690 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0316000000000003e-05 2024-09-14T23:57:58.690 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.690 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:58.690 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.690 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.690 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.690 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69278 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6635520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:58.690 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302953+0000", 2024-09-14T23:57:58.691 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 43.327661198000001, 2024-09-14T23:57:58.691 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051174059000000001, 2024-09-14T23:57:58.691 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:58.691 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:58.691 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:58.691 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:58.691 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:58.691 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:58.691 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69278 2024-09-14T23:57:58.691 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.691 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:58.691 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.691 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:58.691 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302953+0000", 2024-09-14T23:57:58.691 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.692 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.692 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.692 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:58.692 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302953+0000", 2024-09-14T23:57:58.692 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.692 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.692 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.692 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:58.692 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302954+0000", 2024-09-14T23:57:58.692 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5100000000000001e-07 2024-09-14T23:57:58.692 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.692 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.692 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:58.693 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302964+0000", 2024-09-14T23:57:58.693 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0475000000000001e-05 2024-09-14T23:57:58.693 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.693 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.693 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:58.693 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302970+0000", 2024-09-14T23:57:58.693 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3879999999999997e-06 2024-09-14T23:57:58.693 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.693 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.693 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:58.693 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302975+0000", 2024-09-14T23:57:58.693 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8409999999999999e-06 2024-09-14T23:57:58.693 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.694 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.694 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:58.694 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353944+0000", 2024-09-14T23:57:58.694 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050968952999999997 2024-09-14T23:57:58.694 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.694 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.694 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:58.694 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354031+0000", 2024-09-14T23:57:58.694 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6537000000000002e-05 2024-09-14T23:57:58.694 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.694 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.694 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:58.694 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354127+0000", 2024-09-14T23:57:58.694 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6514000000000005e-05 2024-09-14T23:57:58.694 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.695 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:58.695 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.695 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.695 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.695 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69279 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6750208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:58.695 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302984+0000", 2024-09-14T23:57:58.695 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 43.327630847000002, 2024-09-14T23:57:58.695 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051258845999999997, 2024-09-14T23:57:58.695 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:58.695 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:58.695 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:58.695 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:58.695 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:58.695 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:58.695 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69279 2024-09-14T23:57:58.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.696 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:58.696 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.696 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:58.696 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302984+0000", 2024-09-14T23:57:58.696 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.696 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.696 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:58.696 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302984+0000", 2024-09-14T23:57:58.696 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.696 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.696 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:58.697 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303025+0000", 2024-09-14T23:57:58.697 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1690999999999998e-05 2024-09-14T23:57:58.697 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.697 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.697 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:58.697 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303029+0000", 2024-09-14T23:57:58.697 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6559999999999998e-06 2024-09-14T23:57:58.697 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.697 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.697 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:58.697 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303037+0000", 2024-09-14T23:57:58.697 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7950000000000008e-06 2024-09-14T23:57:58.697 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.697 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.698 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:58.698 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303043+0000", 2024-09-14T23:57:58.698 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9669999999999998e-06 2024-09-14T23:57:58.698 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.698 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.698 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:58.698 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354142+0000", 2024-09-14T23:57:58.698 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051099072000000002 2024-09-14T23:57:58.698 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.698 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.698 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:58.698 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354183+0000", 2024-09-14T23:57:58.698 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0834000000000002e-05 2024-09-14T23:57:58.698 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.699 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.699 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:58.699 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354242+0000", 2024-09-14T23:57:58.699 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9830999999999997e-05 2024-09-14T23:57:58.699 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.699 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:58.699 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.699 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.699 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.699 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69280 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6864896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:58.699 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.303052+0000", 2024-09-14T23:57:58.699 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 43.327562303999997, 2024-09-14T23:57:58.699 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051406463999999999, 2024-09-14T23:57:58.699 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:58.700 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:58.700 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:58.700 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:58.700 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:58.700 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:58.700 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69280 2024-09-14T23:57:58.700 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.700 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:58.700 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.700 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:58.700 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303052+0000", 2024-09-14T23:57:58.700 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.700 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.700 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.701 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:58.701 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303052+0000", 2024-09-14T23:57:58.701 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.701 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.701 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.701 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:58.701 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304673+0000", 2024-09-14T23:57:58.701 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001621389 2024-09-14T23:57:58.701 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.701 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.701 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:58.701 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304681+0000", 2024-09-14T23:57:58.701 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8439999999999998e-06 2024-09-14T23:57:58.701 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.702 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.702 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:58.702 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304691+0000", 2024-09-14T23:57:58.702 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8339999999999994e-06 2024-09-14T23:57:58.702 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.702 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.702 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:58.702 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304698+0000", 2024-09-14T23:57:58.702 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3629999999999999e-06 2024-09-14T23:57:58.702 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.702 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.702 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:58.702 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354278+0000", 2024-09-14T23:57:58.702 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049580618 2024-09-14T23:57:58.703 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.703 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.703 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:58.703 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354364+0000", 2024-09-14T23:57:58.703 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6059999999999999e-05 2024-09-14T23:57:58.703 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.703 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.703 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:58.703 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354459+0000", 2024-09-14T23:57:58.703 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4356000000000003e-05 2024-09-14T23:57:58.703 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.703 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:58.703 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.703 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.703 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.704 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69285 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 7438336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:58.704 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.304792+0000", 2024-09-14T23:57:58.704 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 43.325822895000002, 2024-09-14T23:57:58.704 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050398874000000003, 2024-09-14T23:57:58.704 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:58.704 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:58.704 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:58.704 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:58.704 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:58.704 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:58.704 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69285 2024-09-14T23:57:58.704 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.704 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:58.704 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.705 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:58.705 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304792+0000", 2024-09-14T23:57:58.705 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.705 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.705 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.705 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:58.705 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304792+0000", 2024-09-14T23:57:58.705 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.705 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.705 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.705 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:58.705 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304792+0000", 2024-09-14T23:57:58.705 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2800000000000003e-07 2024-09-14T23:57:58.705 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.706 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.706 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:58.706 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304794+0000", 2024-09-14T23:57:58.706 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8700000000000001e-06 2024-09-14T23:57:58.706 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.706 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.706 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:58.706 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304800+0000", 2024-09-14T23:57:58.706 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0889999999999996e-06 2024-09-14T23:57:58.706 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.706 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.706 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:58.706 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304805+0000", 2024-09-14T23:57:58.706 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7060000000000003e-06 2024-09-14T23:57:58.707 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.707 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.707 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:58.707 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355108+0000", 2024-09-14T23:57:58.707 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050303170000000001 2024-09-14T23:57:58.707 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.707 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.707 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:58.707 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355150+0000", 2024-09-14T23:57:58.707 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2271999999999998e-05 2024-09-14T23:57:58.707 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.707 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.707 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:58.707 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355190+0000", 2024-09-14T23:57:58.708 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0438999999999997e-05 2024-09-14T23:57:58.708 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.708 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:58.708 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.708 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.708 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.708 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69286 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 7553024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:58.708 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.304811+0000", 2024-09-14T23:57:58.708 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 43.325803532999998, 2024-09-14T23:57:58.708 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050585512999999999, 2024-09-14T23:57:58.708 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:58.708 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:58.708 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:58.708 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:58.708 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:58.709 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:58.709 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69286 2024-09-14T23:57:58.709 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.709 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:58.709 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.709 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:58.709 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304811+0000", 2024-09-14T23:57:58.709 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.709 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.709 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.709 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:58.709 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304811+0000", 2024-09-14T23:57:58.709 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.709 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.710 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.710 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:58.710 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304811+0000", 2024-09-14T23:57:58.710 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.89e-07 2024-09-14T23:57:58.710 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.710 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.710 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:58.710 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304814+0000", 2024-09-14T23:57:58.710 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3199999999999998e-06 2024-09-14T23:57:58.710 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.710 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.710 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:58.710 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304821+0000", 2024-09-14T23:57:58.710 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5689999999999999e-06 2024-09-14T23:57:58.711 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.711 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.711 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:58.711 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304826+0000", 2024-09-14T23:57:58.711 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1200000000000001e-06 2024-09-14T23:57:58.711 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.711 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.711 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:58.711 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355226+0000", 2024-09-14T23:57:58.711 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050399653000000003 2024-09-14T23:57:58.711 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.711 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.711 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:58.711 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355314+0000", 2024-09-14T23:57:58.711 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8389999999999999e-05 2024-09-14T23:57:58.712 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.712 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.712 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:58.712 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355396+0000", 2024-09-14T23:57:58.712 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2071999999999996e-05 2024-09-14T23:57:58.712 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.712 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:58.712 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.712 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.712 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.712 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69287 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 7667712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:58.712 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.304833+0000", 2024-09-14T23:57:58.712 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 43.325780934999997, 2024-09-14T23:57:58.712 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050664740999999999, 2024-09-14T23:57:58.713 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:58.713 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:58.713 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:58.713 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:58.713 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:58.713 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:58.713 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69287 2024-09-14T23:57:58.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.713 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:58.713 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.713 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:58.713 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304833+0000", 2024-09-14T23:57:58.713 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.713 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.714 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:58.714 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304833+0000", 2024-09-14T23:57:58.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.714 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.714 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:58.714 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304834+0000", 2024-09-14T23:57:58.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4999999999999998e-07 2024-09-14T23:57:58.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.714 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.714 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:58.714 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304836+0000", 2024-09-14T23:57:58.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0279999999999999e-06 2024-09-14T23:57:58.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.715 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.715 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:58.715 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304842+0000", 2024-09-14T23:57:58.715 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8919999999999997e-06 2024-09-14T23:57:58.715 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.715 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.715 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:58.715 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304847+0000", 2024-09-14T23:57:58.715 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6160000000000001e-06 2024-09-14T23:57:58.715 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.715 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.715 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:58.715 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355411+0000", 2024-09-14T23:57:58.715 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050563540999999997 2024-09-14T23:57:58.715 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.716 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.716 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:58.716 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355459+0000", 2024-09-14T23:57:58.716 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7979000000000003e-05 2024-09-14T23:57:58.716 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.716 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.716 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:58.716 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355498+0000", 2024-09-14T23:57:58.716 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9335000000000002e-05 2024-09-14T23:57:58.716 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.716 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:58.716 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.716 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.716 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.717 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69288 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 7782400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:57:58.717 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.304854+0000", 2024-09-14T23:57:58.717 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 43.325760496999997, 2024-09-14T23:57:58.717 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050855145999999997, 2024-09-14T23:57:58.717 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:57:58.717 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:57:58.717 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:57:58.717 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:57:58.717 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:57:58.717 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:58.717 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69288 2024-09-14T23:57:58.717 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.717 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:57:58.717 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.718 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:57:58.718 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304854+0000", 2024-09-14T23:57:58.718 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.718 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.718 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.718 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:57:58.718 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304854+0000", 2024-09-14T23:57:58.718 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:57:58.718 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.718 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.718 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:57:58.718 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.306670+0000", 2024-09-14T23:57:58.718 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00181631 2024-09-14T23:57:58.718 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.719 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.719 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:57:58.719 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.306675+0000", 2024-09-14T23:57:58.719 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0220000000000004e-06 2024-09-14T23:57:58.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.719 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.719 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:57:58.719 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.306684+0000", 2024-09-14T23:57:58.719 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7309999999999999e-06 2024-09-14T23:57:58.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.719 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.719 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:57:58.719 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.306689+0000", 2024-09-14T23:57:58.719 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6609999999999999e-06 2024-09-14T23:57:58.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.720 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.720 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:57:58.720 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355533+0000", 2024-09-14T23:57:58.720 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048844578 2024-09-14T23:57:58.720 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.720 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.720 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:57:58.720 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355622+0000", 2024-09-14T23:57:58.720 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9006999999999997e-05 2024-09-14T23:57:58.720 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:57:58.720 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:57:58.720 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:57:58.720 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355709+0000", 2024-09-14T23:57:58.720 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6836999999999995e-05 2024-09-14T23:57:58.721 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.721 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:58.721 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.721 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:57:58.721 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:57:58.721 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:57:58.722 INFO:teuthology.orchestra.run.smithi042.stdout:5396 write 0x8d7d264 thru 0x8d86c68 (0x9a05 bytes) 2024-09-14T23:57:58.723 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:58.723 INFO:teuthology.orchestra.run.smithi042.stdout:5401 trunc from 0xb67ffde to 0x565b494 2024-09-14T23:57:58.723 INFO:teuthology.orchestra.run.smithi042.stdout:5403 read 0x21c4f81 thru 0x21ce45b (0x94db bytes) 2024-09-14T23:57:58.723 INFO:teuthology.orchestra.run.smithi042.stdout:5404 read 0x36d8759 thru 0x36da03e (0x18e6 bytes) 2024-09-14T23:57:58.723 INFO:teuthology.orchestra.run.smithi042.stdout:5405 write 0x25fcf79 thru 0x2609574 (0xc5fc bytes) 2024-09-14T23:57:58.723 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T23:57:58.743 INFO:teuthology.orchestra.run.smithi042.stdout:5408 writesame 0x2506e59 thru 0x250b701 data_size 0x13(0x48a9 bytes) 2024-09-14T23:57:58.743 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:58.742+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:58.745 INFO:teuthology.orchestra.run.smithi042.stdout:5409 writesame 0x2112d6 thru 0x215f22 data_size 0x11(0x4c4d bytes) 2024-09-14T23:57:58.773 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:58.786 INFO:teuthology.orchestra.run.smithi042.stdout:5411 writesame 0xbe2997b thru 0xbe2a5bb data_size 0x1(0xc41 bytes) 2024-09-14T23:57:58.786 INFO:teuthology.orchestra.run.smithi042.stdout:5412 punch from 0x4e76b4 to 0x4ec9fe, (0x534a bytes) 2024-09-14T23:57:58.786 INFO:teuthology.orchestra.run.smithi042.stdout:5413 read 0xaf61e34 thru 0xaf6804d (0x621a bytes) 2024-09-14T23:57:58.786 INFO:teuthology.orchestra.run.smithi042.stdout:5414 write 0x760c76e thru 0x76113ee (0x4c81 bytes) 2024-09-14T23:57:58.791 INFO:teuthology.orchestra.run.smithi042.stdout:5416 read 0x6539c71 thru 0x6545567 (0xb8f7 bytes) 2024-09-14T23:57:58.791 INFO:teuthology.orchestra.run.smithi042.stdout:5417 writesame 0x1c5787f thru 0x1c630c9 data_size 0x1(0xb84b bytes) 2024-09-14T23:57:58.821 INFO:teuthology.orchestra.run.smithi042.stdout:5419 write 0xddc7332 thru 0xddcaed2 (0x3ba1 bytes) 2024-09-14T23:57:58.825 INFO:teuthology.orchestra.run.smithi042.stdout:5420 write 0xa633b3 thru 0xa67be5 (0x4833 bytes) 2024-09-14T23:57:58.828 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:58.827+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:58.838 INFO:teuthology.orchestra.run.smithi042.stdout:5421 read 0x26dcea8 thru 0x26e3184 (0x62dd bytes) 2024-09-14T23:57:58.838 INFO:teuthology.orchestra.run.smithi042.stdout:5422 write 0xd921c61 thru 0xd92b9b6 (0x9d56 bytes) 2024-09-14T23:57:58.846 INFO:teuthology.orchestra.run.smithi042.stdout:5423 punch from 0x2fdca6a to 0x2fec44a, (0xf9e0 bytes) 2024-09-14T23:57:58.846 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:58.846 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:58.846 INFO:teuthology.orchestra.run.smithi042.stdout:5426 write 0xcdcac0a thru 0xcdccf72 (0x2369 bytes) 2024-09-14T23:57:58.854 INFO:teuthology.orchestra.run.smithi042.stdout:5427 trunc from 0xddcaed3 to 0x89714ff 2024-09-14T23:57:58.859 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:58.859 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:57:58.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:58.860 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop_reply(client.4376.0:158528 3.12 e103/23)", 2024-09-14T23:57:58.860 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:58.858373+0000", 2024-09-14T23:57:58.860 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.000104351, 2024-09-14T23:57:58.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012224100000000001, 2024-09-14T23:57:58.860 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:58.860 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:58.860 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:58.860 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:58.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:58.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:58.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.858373+0000", 2024-09-14T23:57:58.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:58.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:58.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:58.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:58.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.858373+0000", 2024-09-14T23:57:58.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:58.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:58.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:58.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:58.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.858374+0000", 2024-09-14T23:57:58.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4399999999999999e-07 2024-09-14T23:57:58.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:58.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:58.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:58.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.858383+0000", 2024-09-14T23:57:58.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6859999999999995e-06 2024-09-14T23:57:58.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:58.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:58.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:58.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.858394+0000", 2024-09-14T23:57:58.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1083e-05 2024-09-14T23:57:58.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:58.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:58.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:58.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.858401+0000", 2024-09-14T23:57:58.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.289e-06 2024-09-14T23:57:58.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:58.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:58.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:58.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.858478+0000", 2024-09-14T23:57:58.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6423999999999999e-05 2024-09-14T23:57:58.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:58.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:58.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:58.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.858487+0000", 2024-09-14T23:57:58.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2259999999999994e-06 2024-09-14T23:57:58.863 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:58.863 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:58.863 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:58.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:58.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:58.863 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:158528 3.12 3:4fbe8597:::rbd_data.1118b2f495e6.00000000000000cc:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-14T23:57:58.863 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:58.856671+0000", 2024-09-14T23:57:58.863 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.0018066759999999999, 2024-09-14T23:57:58.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001902794, 2024-09-14T23:57:58.863 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:58.864 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:58.864 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:58.864 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:58.864 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:58.864 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:58.864 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 158528 2024-09-14T23:57:58.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:58.864 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:58.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:58.864 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:58.864 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.856671+0000", 2024-09-14T23:57:58.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:58.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:58.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:58.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:58.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.856671+0000", 2024-09-14T23:57:58.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:58.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:58.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:58.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:58.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.856673+0000", 2024-09-14T23:57:58.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.637e-06 2024-09-14T23:57:58.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:58.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:58.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:58.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.856679+0000", 2024-09-14T23:57:58.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6209999999999998e-06 2024-09-14T23:57:58.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:58.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:58.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:58.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.856697+0000", 2024-09-14T23:57:58.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7195999999999999e-05 2024-09-14T23:57:58.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:58.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:58.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:58.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.856703+0000", 2024-09-14T23:57:58.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9110000000000004e-06 2024-09-14T23:57:58.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:58.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:58.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:58.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.856772+0000", 2024-09-14T23:57:58.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8961000000000006e-05 2024-09-14T23:57:58.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:58.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:58.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:58.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.856869+0000", 2024-09-14T23:57:58.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6512e-05 2024-09-14T23:57:58.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:58.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:58.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 7", 2024-09-14T23:57:58.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.857591+0000", 2024-09-14T23:57:58.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00072240900000000003 2024-09-14T23:57:58.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:58.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:58.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:58.868 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.858494+0000", 2024-09-14T23:57:58.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00090215499999999997 2024-09-14T23:57:58.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:58.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:58.868 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:57:58.868 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.858513+0000", 2024-09-14T23:57:58.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.91e-05 2024-09-14T23:57:58.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:58.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:58.868 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:58.868 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.858561+0000", 2024-09-14T23:57:58.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8559e-05 2024-09-14T23:57:58.868 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:58.868 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:58.869 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:58.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:58.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:58.869 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:158529 3.1b 3:d815cc2b:::rbd_data.1118b2f495e6.00000000000000d4:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-14T23:57:58.869 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:58.856723+0000", 2024-09-14T23:57:58.869 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.001754469, 2024-09-14T23:57:58.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019502969999999999, 2024-09-14T23:57:58.869 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:58.869 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:58.869 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "waiting for sub ops", 2024-09-14T23:57:58.869 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:58.869 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:58.869 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:58.869 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 158529 2024-09-14T23:57:58.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:58.870 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:58.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:58.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:58.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.856723+0000", 2024-09-14T23:57:58.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:58.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:58.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:58.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:58.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.856723+0000", 2024-09-14T23:57:58.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:58.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:58.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:58.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:58.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.856724+0000", 2024-09-14T23:57:58.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6800000000000001e-07 2024-09-14T23:57:58.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:58.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:58.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:58.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.856728+0000", 2024-09-14T23:57:58.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7479999999999999e-06 2024-09-14T23:57:58.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:58.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:58.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:58.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.856743+0000", 2024-09-14T23:57:58.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5313999999999999e-05 2024-09-14T23:57:58.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:58.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:58.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:58.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.856749+0000", 2024-09-14T23:57:58.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7289999999999997e-06 2024-09-14T23:57:58.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:58.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:58.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:58.872 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.856806+0000", 2024-09-14T23:57:58.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7423000000000002e-05 2024-09-14T23:57:58.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:58.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:58.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:58.872 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.857006+0000", 2024-09-14T23:57:58.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019973300000000001 2024-09-14T23:57:58.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:58.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:58.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 4", 2024-09-14T23:57:58.872 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.857142+0000", 2024-09-14T23:57:58.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013595 2024-09-14T23:57:58.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:58.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:58.873 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:57:58.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.858576+0000", 2024-09-14T23:57:58.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001434154 2024-09-14T23:57:58.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:58.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:58.873 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:58.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.858680+0000", 2024-09-14T23:57:58.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010401 2024-09-14T23:57:58.873 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:58.873 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:58.873 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:58.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:58.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:58.873 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:158535 3.3 e103/23 3:c1fc423e:::rbd_data.1118b2f495e6.00000000000000db:head v 103'439, mlcod=103'437)", 2024-09-14T23:57:58.874 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:58.857578+0000", 2024-09-14T23:57:58.874 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.00089983299999999995, 2024-09-14T23:57:58.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001170581, 2024-09-14T23:57:58.874 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:58.874 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:58.874 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:58.874 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:58.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:58.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:58.874 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.857578+0000", 2024-09-14T23:57:58.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:58.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:58.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:58.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:58.874 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.857578+0000", 2024-09-14T23:57:58.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:58.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:58.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:58.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:58.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.857579+0000", 2024-09-14T23:57:58.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8899999999999998e-07 2024-09-14T23:57:58.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:58.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:58.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:58.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.857586+0000", 2024-09-14T23:57:58.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5469999999999999e-06 2024-09-14T23:57:58.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:58.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:58.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:58.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.857596+0000", 2024-09-14T23:57:58.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0020000000000001e-05 2024-09-14T23:57:58.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:58.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:58.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:58.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.857603+0000", 2024-09-14T23:57:58.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5289999999999997e-06 2024-09-14T23:57:58.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:58.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:58.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:58.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.857706+0000", 2024-09-14T23:57:58.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010310300000000001 2024-09-14T23:57:58.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:58.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:58.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:58.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.857725+0000", 2024-09-14T23:57:58.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9332000000000001e-05 2024-09-14T23:57:58.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:58.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:58.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:58.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.858709+0000", 2024-09-14T23:57:58.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00098324099999999993 2024-09-14T23:57:58.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:58.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:58.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:58.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.858749+0000", 2024-09-14T23:57:58.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.002e-05 2024-09-14T23:57:58.878 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:58.878 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:58.878 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:58.878 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:58.878 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-14T23:57:58.878 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 4 2024-09-14T23:57:58.878 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:58.879 INFO:teuthology.orchestra.run.smithi042.stdout:5428 write 0xad014a thru 0xad17d5 (0x168c bytes) 2024-09-14T23:57:58.879 INFO:teuthology.orchestra.run.smithi042.stdout:5429 read 0x35f14f9 thru 0x35f5002 (0x3b0a bytes) 2024-09-14T23:57:58.879 INFO:teuthology.orchestra.run.smithi042.stdout:5430 read 0x5c0397e thru 0x5c0f5b9 (0xbc3c bytes) 2024-09-14T23:57:58.880 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T23:57:58.880 INFO:teuthology.orchestra.run.smithi042.stdout:5431 writesame 0xab948f9 thru 0xaba16d2 data_size 0x3(0xcdda bytes) 2024-09-14T23:57:58.880 INFO:teuthology.orchestra.run.smithi042.stdout:5432 read 0x4be841f thru 0x4bebbb2 (0x3794 bytes) 2024-09-14T23:57:58.880 INFO:teuthology.orchestra.run.smithi042.stdout:5433 read 0x302e9e3 thru 0x30358eb (0x6f09 bytes) 2024-09-14T23:57:58.881 INFO:teuthology.orchestra.run.smithi042.stdout:5434 write 0x8bdc079 thru 0x8be6a52 (0xa9da bytes) 2024-09-14T23:57:58.885 INFO:teuthology.orchestra.run.smithi042.stdout:5435 read 0x60dd63a thru 0x60de62c (0xff3 bytes) 2024-09-14T23:57:58.885 INFO:teuthology.orchestra.run.smithi042.stdout:5436 punch from 0x33c44d6 to 0x33cf268, (0xad92 bytes) 2024-09-14T23:57:58.885 INFO:teuthology.orchestra.run.smithi042.stdout:5437 write 0x782bc3e thru 0x7830bd4 (0x4f97 bytes) 2024-09-14T23:57:58.890 INFO:teuthology.orchestra.run.smithi042.stdout:5439 writesame 0x39b20ef thru 0x39b918b data_size 0x1(0x709d bytes) 2024-09-14T23:57:58.900 INFO:teuthology.orchestra.run.smithi042.stdout:5440 write 0x79eb253 thru 0x79f1df6 (0x6ba4 bytes) 2024-09-14T23:57:58.902 INFO:teuthology.orchestra.run.smithi042.stdout:5441 writesame 0x14eb1a1 thru 0x14f0303 data_size 0xf(0x5163 bytes) 2024-09-14T23:57:58.929 INFO:teuthology.orchestra.run.smithi042.stdout:5444 writesame 0x1dcc2c3 thru 0x1dd29f3 data_size 0x1(0x6731 bytes) 2024-09-14T23:57:58.930 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:58.928+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:58.934 INFO:teuthology.orchestra.run.smithi042.stdout:5448 writesame 0x18ffb6f thru 0x190b617 data_size 0x1(0xbaa9 bytes) 2024-09-14T23:57:58.975 INFO:teuthology.orchestra.run.smithi042.stdout:5449 trunc from 0xaba16d3 to 0x44503f4 2024-09-14T23:57:58.989 INFO:teuthology.orchestra.run.smithi042.stdout:5450 punch from 0xbb83f9 to 0xbb8db4, (0x9bb bytes) 2024-09-14T23:57:58.989 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:58.999 INFO:teuthology.orchestra.run.smithi042.stdout:5452 write 0x884241c thru 0x884412b (0x1d10 bytes) 2024-09-14T23:57:59.003 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:59.003 INFO:teuthology.orchestra.run.smithi042.stdout:5455 read 0x84893f8 thru 0x849764e (0xe257 bytes) 2024-09-14T23:57:59.004 INFO:teuthology.orchestra.run.smithi042.stdout:5456 read 0x40111c3 thru 0x401789d (0x66db bytes) 2024-09-14T23:57:59.004 INFO:teuthology.orchestra.run.smithi042.stdout:5457 writesame 0x3b88312 thru 0x3b9741e data_size 0x1(0xf10d bytes) 2024-09-14T23:57:59.017 INFO:teuthology.orchestra.run.smithi042.stdout:5460 trunc from 0x884412c to 0xa54fbb8 2024-09-14T23:57:59.022 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:59.022 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:57:59.023 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:57:59.023 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:57:59.023 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:59.024 INFO:teuthology.orchestra.run.smithi042.stdout:5461 trunc from 0xa54fbb8 to 0xc56b7e9 2024-09-14T23:57:59.030 INFO:teuthology.orchestra.run.smithi042.stdout:5463 write 0x576ded thru 0x5818ee (0xab02 bytes) 2024-09-14T23:57:59.030 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:57:59.029+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:57:59.034 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T23:57:59.066 INFO:teuthology.orchestra.run.smithi042.stdout:5464 trunc from 0xc56b7e9 to 0x3b14dc7 2024-09-14T23:57:59.079 INFO:teuthology.orchestra.run.smithi042.stdout:5466 read 0x242e1a9 thru 0x243daac (0xf904 bytes) 2024-09-14T23:57:59.079 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:59.087 INFO:teuthology.orchestra.run.smithi042.stdout:5469 trunc from 0x3b14dc7 to 0x9fa72fe 2024-09-14T23:57:59.092 INFO:teuthology.orchestra.run.smithi042.stdout:5470 write 0xff58e8 thru 0xffa40b (0x4b24 bytes) 2024-09-14T23:57:59.094 INFO:teuthology.orchestra.run.smithi042.stdout:5471 write 0x1135c18 thru 0x113d27a (0x7663 bytes) 2024-09-14T23:57:59.103 INFO:teuthology.orchestra.run.smithi042.stdout:5472 write 0xa02f139 thru 0xa036169 (0x7031 bytes) 2024-09-14T23:57:59.118 INFO:teuthology.orchestra.run.smithi042.stdout:5473 write 0xc4969c5 thru 0xc498233 (0x186f bytes) 2024-09-14T23:57:59.121 INFO:teuthology.orchestra.run.smithi042.stdout:5475 read 0x232feaa thru 0x2334418 (0x456f bytes) 2024-09-14T23:57:59.122 INFO:teuthology.orchestra.run.smithi042.stdout:5477 read 0xa34f8cd thru 0xa359ea8 (0xa5dc bytes) 2024-09-14T23:57:59.122 INFO:teuthology.orchestra.run.smithi042.stdout:5479 read 0xa945ccd thru 0xa94f706 (0x9a3a bytes) 2024-09-14T23:57:59.122 INFO:teuthology.orchestra.run.smithi042.stdout:5480 writesame 0x4d78a1 thru 0x4dae0a data_size 0x6(0x356a bytes) 2024-09-14T23:57:59.131 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:57:59.130+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:57:59.136 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:59.136 INFO:teuthology.orchestra.run.smithi042.stdout:5483 punch from 0x28e1265 to 0x28edd2b, (0xcac6 bytes) 2024-09-14T23:57:59.136 INFO:teuthology.orchestra.run.smithi042.stdout:5485 punch from 0x9ad16b9 to 0x9ad6d73, (0x56ba bytes) 2024-09-14T23:57:59.137 INFO:teuthology.orchestra.run.smithi042.stdout:5486 read 0x3df10a4 thru 0x3df652b (0x5488 bytes) 2024-09-14T23:57:59.137 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:59.144 INFO:teuthology.orchestra.run.smithi042.stdout:5489 trunc from 0xc498234 to 0x5820c42 2024-09-14T23:57:59.162 INFO:teuthology.orchestra.run.smithi042.stdout:5490 write 0x830e429 thru 0x8318f65 (0xab3d bytes) 2024-09-14T23:57:59.170 INFO:teuthology.orchestra.run.smithi042.stdout:5492 read 0x773eb1a thru 0x77453f2 (0x68d9 bytes) 2024-09-14T23:57:59.178 INFO:teuthology.orchestra.run.smithi042.stdout:5494 writesame 0xb04f62e thru 0xb05317f data_size 0x3(0x3b52 bytes) 2024-09-14T23:57:59.178 INFO:teuthology.orchestra.run.smithi042.stdout:5495 read 0x8426550 thru 0x84287ef (0x22a0 bytes) 2024-09-14T23:57:59.179 INFO:teuthology.orchestra.run.smithi042.stdout:5496 read 0x4a37b39 thru 0x4a45d85 (0xe24d bytes) 2024-09-14T23:57:59.179 INFO:teuthology.orchestra.run.smithi042.stdout:5497 punch from 0x482f138 to 0x483df79, (0xee41 bytes) 2024-09-14T23:57:59.179 INFO:teuthology.orchestra.run.smithi042.stdout:5499 write 0x2f3eeac thru 0x2f43be2 (0x4d37 bytes) 2024-09-14T23:57:59.180 INFO:teuthology.orchestra.run.smithi042.stdout:5500 read 0x9b4da66 thru 0x9b56bcb (0x9166 bytes) 2024-09-14T23:57:59.181 INFO:teuthology.orchestra.run.smithi042.stdout:5502 read 0x59b9919 thru 0x59c44dc (0xabc4 bytes) 2024-09-14T23:57:59.181 INFO:teuthology.orchestra.run.smithi042.stdout:5507 write 0x774d4a7 thru 0x7755310 (0x7e6a bytes) 2024-09-14T23:57:59.187 INFO:teuthology.orchestra.run.smithi042.stdout:5508 read 0x3f874c3 thru 0x3f87c67 (0x7a5 bytes) 2024-09-14T23:57:59.187 INFO:teuthology.orchestra.run.smithi042.stdout:5510 punch from 0x523aaee to 0x5246f92, (0xc4a4 bytes) 2024-09-14T23:57:59.196 INFO:teuthology.orchestra.run.smithi042.stdout:5511 write 0xda8d5ad thru 0xda93f31 (0x6985 bytes) 2024-09-14T23:57:59.198 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:59.198 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:57:59.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:57:59.199 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:57:59.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.199 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:43054 3.1f 3:ffbfe596:::rbd_data.1118350d5ae5.00000000000002eb:head [set-alloc-hint object_size 262144 write_size 262144,write 196608~22770] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T23:57:59.199 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:57:59.199 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 66.229519517, 2024-09-14T23:57:59.199 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.99973334400000002, 2024-09-14T23:57:59.199 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:59.199 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:59.199 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:57:59.199 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:59.199 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:59.199 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:59.199 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 43054 2024-09-14T23:57:59.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.200 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:59.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.200 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:59.200 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:57:59.200 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:59.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.200 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:59.200 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:57:59.200 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:59.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.200 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:59.201 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966552+0000", 2024-09-14T23:57:59.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3790000000000001e-06 2024-09-14T23:57:59.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.201 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:59.201 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966565+0000", 2024-09-14T23:57:59.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2459999999999999e-05 2024-09-14T23:57:59.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.201 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:59.201 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966576+0000", 2024-09-14T23:57:59.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1039000000000001e-05 2024-09-14T23:57:59.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.202 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:59.202 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966582+0000", 2024-09-14T23:57:59.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9660000000000001e-06 2024-09-14T23:57:59.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.202 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:59.202 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966625+0000", 2024-09-14T23:57:59.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3529999999999998e-05 2024-09-14T23:57:59.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.202 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:57:59.202 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966632+0000", 2024-09-14T23:57:59.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9809999999999997e-06 2024-09-14T23:57:59.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.203 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:59.203 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.964460+0000", 2024-09-14T23:57:59.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.99782768300000002 2024-09-14T23:57:59.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.203 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:59.203 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.964641+0000", 2024-09-14T23:57:59.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000181089 2024-09-14T23:57:59.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.203 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-14T23:57:59.203 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.964809+0000", 2024-09-14T23:57:59.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000168505 2024-09-14T23:57:59.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.204 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:57:59.204 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966030+0000", 2024-09-14T23:57:59.204 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012206929999999999 2024-09-14T23:57:59.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.204 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:57:59.204 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966181+0000", 2024-09-14T23:57:59.204 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000150569 2024-09-14T23:57:59.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.204 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:57:59.204 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966243+0000", 2024-09-14T23:57:59.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2331999999999999e-05 2024-09-14T23:57:59.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:59.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966284+0000", 2024-09-14T23:57:59.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1118000000000002e-05 2024-09-14T23:57:59.205 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:59.205 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:59.205 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:59.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.205 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:144877 3.6 3:65baca07:::rbd_data.11189263ca38.000000000000000a:2a [read 909312~16384] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-14T23:57:59.205 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:53.415702+0000", 2024-09-14T23:57:59.205 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.7803678710000002, 2024-09-14T23:57:59.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.036982196000000002, 2024-09-14T23:57:59.206 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:59.206 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:59.206 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:59.206 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:59.206 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:59.206 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:59.206 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 144877 2024-09-14T23:57:59.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.206 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:59.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:59.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.415702+0000", 2024-09-14T23:57:59.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:59.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.207 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:59.207 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.415702+0000", 2024-09-14T23:57:59.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:59.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.207 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:59.207 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.415703+0000", 2024-09-14T23:57:59.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0499999999999999e-07 2024-09-14T23:57:59.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.207 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:59.207 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.415705+0000", 2024-09-14T23:57:59.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9819999999999998e-06 2024-09-14T23:57:59.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:59.208 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.415710+0000", 2024-09-14T23:57:59.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6779999999999997e-06 2024-09-14T23:57:59.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:59.208 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.415717+0000", 2024-09-14T23:57:59.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8589999999999999e-06 2024-09-14T23:57:59.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:59.208 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.452575+0000", 2024-09-14T23:57:59.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.036857525000000002 2024-09-14T23:57:59.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.209 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:59.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.452624+0000", 2024-09-14T23:57:59.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9397999999999997e-05 2024-09-14T23:57:59.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.209 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:59.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.452685+0000", 2024-09-14T23:57:59.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0349000000000001e-05 2024-09-14T23:57:59.209 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:59.209 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:59.209 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:59.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.210 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:144878 3.6 3:65baca07:::rbd_data.11189263ca38.000000000000000a:2a [read 974848~16384] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-14T23:57:59.210 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:53.415935+0000", 2024-09-14T23:57:59.210 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.7801356520000002, 2024-09-14T23:57:59.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.036967765, 2024-09-14T23:57:59.210 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:59.210 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:59.210 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:59.210 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:59.210 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:59.210 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:59.210 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 144878 2024-09-14T23:57:59.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.210 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:59.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:59.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.415935+0000", 2024-09-14T23:57:59.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:59.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:59.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.415935+0000", 2024-09-14T23:57:59.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:59.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:59.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.415935+0000", 2024-09-14T23:57:59.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8100000000000003e-07 2024-09-14T23:57:59.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:59.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.415939+0000", 2024-09-14T23:57:59.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5949999999999999e-06 2024-09-14T23:57:59.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.212 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:59.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.415946+0000", 2024-09-14T23:57:59.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4900000000000003e-06 2024-09-14T23:57:59.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.212 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:59.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.415952+0000", 2024-09-14T23:57:59.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2530000000000001e-06 2024-09-14T23:57:59.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:59.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.452724+0000", 2024-09-14T23:57:59.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.036771525999999999 2024-09-14T23:57:59.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:59.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.452803+0000", 2024-09-14T23:57:59.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8885000000000001e-05 2024-09-14T23:57:59.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:59.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.452902+0000", 2024-09-14T23:57:59.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9434999999999999e-05 2024-09-14T23:57:59.213 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:59.214 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:59.214 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:59.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.214 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:144879 3.6 3:65baca07:::rbd_data.11189263ca38.000000000000000a:2a [read 1040384~8192] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-14T23:57:59.214 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:53.415963+0000", 2024-09-14T23:57:59.214 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.7801072080000004, 2024-09-14T23:57:59.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037080130000000003, 2024-09-14T23:57:59.214 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:59.214 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:59.214 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:59.214 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:59.214 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:59.214 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:59.215 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 144879 2024-09-14T23:57:59.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.215 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:59.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:59.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.415963+0000", 2024-09-14T23:57:59.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:59.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:59.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.415963+0000", 2024-09-14T23:57:59.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:59.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:59.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.415964+0000", 2024-09-14T23:57:59.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4000000000000002e-07 2024-09-14T23:57:59.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:59.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.415966+0000", 2024-09-14T23:57:59.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3089999999999998e-06 2024-09-14T23:57:59.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:59.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.415972+0000", 2024-09-14T23:57:59.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5509999999999996e-06 2024-09-14T23:57:59.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:59.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.415978+0000", 2024-09-14T23:57:59.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6949999999999997e-06 2024-09-14T23:57:59.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:59.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.452935+0000", 2024-09-14T23:57:59.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.036956646000000003 2024-09-14T23:57:59.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:59.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.452984+0000", 2024-09-14T23:57:59.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9688000000000003e-05 2024-09-14T23:57:59.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.218 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:59.218 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.453043+0000", 2024-09-14T23:57:59.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8700999999999999e-05 2024-09-14T23:57:59.218 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:59.218 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:59.218 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:59.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.218 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:144912 3.6 3:6629293c:::rbd_data.11189263ca38.0000000000000009:2a [read 1359872~0] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-14T23:57:59.218 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:53.416279+0000", 2024-09-14T23:57:59.218 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.7797909540000001, 2024-09-14T23:57:59.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.036959829999999999, 2024-09-14T23:57:59.219 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:59.219 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:59.219 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:59.219 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:59.219 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:59.219 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:59.219 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 144912 2024-09-14T23:57:59.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.219 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:59.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:59.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416279+0000", 2024-09-14T23:57:59.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:59.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:59.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416279+0000", 2024-09-14T23:57:59.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:59.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:59.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416280+0000", 2024-09-14T23:57:59.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8400000000000005e-07 2024-09-14T23:57:59.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:59.221 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416282+0000", 2024-09-14T23:57:59.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.486e-06 2024-09-14T23:57:59.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:59.221 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416289+0000", 2024-09-14T23:57:59.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.63e-06 2024-09-14T23:57:59.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:59.222 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416294+0000", 2024-09-14T23:57:59.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.95e-06 2024-09-14T23:57:59.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.222 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:59.222 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.453081+0000", 2024-09-14T23:57:59.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.036787577000000002 2024-09-14T23:57:59.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.222 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:59.222 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.453161+0000", 2024-09-14T23:57:59.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0031000000000007e-05 2024-09-14T23:57:59.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:59.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.453239+0000", 2024-09-14T23:57:59.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7671999999999998e-05 2024-09-14T23:57:59.223 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:59.223 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:59.223 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:59.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.223 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:144913 3.6 3:6629293c:::rbd_data.11189263ca38.0000000000000009:2a [read 1425408~0] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-14T23:57:59.223 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:53.416302+0000", 2024-09-14T23:57:59.223 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.7797681030000003, 2024-09-14T23:57:59.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037067357000000002, 2024-09-14T23:57:59.223 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:59.223 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:59.224 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:59.224 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:59.224 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:59.224 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:59.224 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 144913 2024-09-14T23:57:59.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.224 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:59.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.224 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:59.224 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416302+0000", 2024-09-14T23:57:59.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:59.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.224 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:59.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416302+0000", 2024-09-14T23:57:59.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:59.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.225 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:59.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416303+0000", 2024-09-14T23:57:59.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7300000000000001e-07 2024-09-14T23:57:59.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.225 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:59.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416305+0000", 2024-09-14T23:57:59.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2900000000000001e-06 2024-09-14T23:57:59.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:59.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416311+0000", 2024-09-14T23:57:59.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0310000000000004e-06 2024-09-14T23:57:59.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:59.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416317+0000", 2024-09-14T23:57:59.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9499999999999998e-06 2024-09-14T23:57:59.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:59.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.453278+0000", 2024-09-14T23:57:59.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.036961234000000003 2024-09-14T23:57:59.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:59.227 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.453323+0000", 2024-09-14T23:57:59.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5022999999999999e-05 2024-09-14T23:57:59.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:59.227 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.453369+0000", 2024-09-14T23:57:59.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6356000000000001e-05 2024-09-14T23:57:59.227 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:59.227 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:59.227 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:59.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.227 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:144914 3.6 3:6629293c:::rbd_data.11189263ca38.0000000000000009:2a [read 1490944~0] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-14T23:57:59.228 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:53.416324+0000", 2024-09-14T23:57:59.228 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.7797465629999998, 2024-09-14T23:57:59.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037241315999999997, 2024-09-14T23:57:59.228 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:59.228 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:59.228 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:59.228 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:59.228 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:59.228 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:59.228 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 144914 2024-09-14T23:57:59.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.228 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:59.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.229 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:59.229 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416324+0000", 2024-09-14T23:57:59.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:59.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.229 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:59.229 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416324+0000", 2024-09-14T23:57:59.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:59.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.229 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:59.229 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416324+0000", 2024-09-14T23:57:59.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6400000000000003e-07 2024-09-14T23:57:59.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:59.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416326+0000", 2024-09-14T23:57:59.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1799999999999999e-06 2024-09-14T23:57:59.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:59.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416334+0000", 2024-09-14T23:57:59.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.345e-06 2024-09-14T23:57:59.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:59.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416339+0000", 2024-09-14T23:57:59.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1699999999999996e-06 2024-09-14T23:57:59.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.231 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:59.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.453406+0000", 2024-09-14T23:57:59.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037067584000000001 2024-09-14T23:57:59.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.231 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:59.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.453485+0000", 2024-09-14T23:57:59.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8740999999999997e-05 2024-09-14T23:57:59.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.231 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:59.232 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.453565+0000", 2024-09-14T23:57:59.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9832000000000004e-05 2024-09-14T23:57:59.232 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:59.232 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:59.232 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:59.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.232 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:144915 3.6 3:6629293c:::rbd_data.11189263ca38.0000000000000009:2a [read 1556480~0] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-14T23:57:59.232 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:53.416345+0000", 2024-09-14T23:57:59.232 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.7797248840000002, 2024-09-14T23:57:59.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037345237000000003, 2024-09-14T23:57:59.232 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:59.232 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:59.232 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:59.232 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:59.233 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:59.233 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:59.233 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 144915 2024-09-14T23:57:59.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.233 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:59.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.233 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:59.233 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416345+0000", 2024-09-14T23:57:59.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:59.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.233 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:59.233 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416345+0000", 2024-09-14T23:57:59.234 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:59.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.234 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:59.234 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416346+0000", 2024-09-14T23:57:59.234 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2600000000000002e-07 2024-09-14T23:57:59.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.234 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:59.234 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416348+0000", 2024-09-14T23:57:59.234 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1050000000000002e-06 2024-09-14T23:57:59.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.234 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:59.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416355+0000", 2024-09-14T23:57:59.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7349999999999999e-06 2024-09-14T23:57:59.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.235 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:59.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416360+0000", 2024-09-14T23:57:59.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2939999999999996e-06 2024-09-14T23:57:59.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.235 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:59.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.453602+0000", 2024-09-14T23:57:59.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037242453000000002 2024-09-14T23:57:59.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.236 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:59.236 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.453647+0000", 2024-09-14T23:57:59.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4113000000000003e-05 2024-09-14T23:57:59.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.236 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:59.236 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.453691+0000", 2024-09-14T23:57:59.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4011000000000003e-05 2024-09-14T23:57:59.236 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:59.236 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:59.236 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:59.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.236 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:144916 3.6 3:65baca07:::rbd_data.11189263ca38.000000000000000a:2a [read 1351680~8192] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-14T23:57:59.236 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:53.416368+0000", 2024-09-14T23:57:59.237 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.7797018050000002, 2024-09-14T23:57:59.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037549193000000002, 2024-09-14T23:57:59.237 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:59.237 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:59.237 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:59.237 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:59.237 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:59.237 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:59.237 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 144916 2024-09-14T23:57:59.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.237 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:59.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:59.238 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416368+0000", 2024-09-14T23:57:59.238 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:59.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.238 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:59.238 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416368+0000", 2024-09-14T23:57:59.238 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:59.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.238 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:59.238 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416369+0000", 2024-09-14T23:57:59.238 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3300000000000003e-07 2024-09-14T23:57:59.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.238 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:59.239 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416375+0000", 2024-09-14T23:57:59.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8379999999999998e-06 2024-09-14T23:57:59.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.239 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.239 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:59.239 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416382+0000", 2024-09-14T23:57:59.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2949999999999996e-06 2024-09-14T23:57:59.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.239 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.239 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:59.239 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416387+0000", 2024-09-14T23:57:59.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3890000000000004e-06 2024-09-14T23:57:59.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.239 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.240 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:59.240 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.453730+0000", 2024-09-14T23:57:59.240 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037342594999999999 2024-09-14T23:57:59.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.240 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:59.240 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.453813+0000", 2024-09-14T23:57:59.240 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3484000000000002e-05 2024-09-14T23:57:59.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.240 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:59.240 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.453918+0000", 2024-09-14T23:57:59.240 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000104159 2024-09-14T23:57:59.240 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:59.241 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:59.241 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:59.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.241 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.241 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:144917 3.6 3:65baca07:::rbd_data.11189263ca38.000000000000000a:2a [read 1417216~8192] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-14T23:57:59.241 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:53.416395+0000", 2024-09-14T23:57:59.241 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.7796749040000002, 2024-09-14T23:57:59.241 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037661671000000001, 2024-09-14T23:57:59.241 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:59.241 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:59.241 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:59.241 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:59.241 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:59.241 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:59.242 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 144917 2024-09-14T23:57:59.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.242 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:59.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.242 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:59.242 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416395+0000", 2024-09-14T23:57:59.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:59.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.242 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:59.242 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416395+0000", 2024-09-14T23:57:59.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:59.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.242 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:59.243 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416396+0000", 2024-09-14T23:57:59.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7899999999999999e-07 2024-09-14T23:57:59.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.243 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.243 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:59.243 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416398+0000", 2024-09-14T23:57:59.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1859999999999999e-06 2024-09-14T23:57:59.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.243 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.243 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:59.243 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416404+0000", 2024-09-14T23:57:59.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2249999999999997e-06 2024-09-14T23:57:59.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:59.244 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416409+0000", 2024-09-14T23:57:59.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9679999999999997e-06 2024-09-14T23:57:59.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:59.244 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.453950+0000", 2024-09-14T23:57:59.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037540967000000001 2024-09-14T23:57:59.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:59.244 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.453998+0000", 2024-09-14T23:57:59.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8284000000000001e-05 2024-09-14T23:57:59.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.245 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:59.245 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.454057+0000", 2024-09-14T23:57:59.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8561999999999999e-05 2024-09-14T23:57:59.245 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:59.245 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:59.245 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:59.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.245 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:144918 3.6 3:65baca07:::rbd_data.11189263ca38.000000000000000a:2a [read 1482752~8192] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-14T23:57:59.245 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:53.416416+0000", 2024-09-14T23:57:59.245 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.7796544110000001, 2024-09-14T23:57:59.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037857809999999999, 2024-09-14T23:57:59.245 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:59.245 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:59.246 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:59.246 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:59.246 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:59.246 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:59.246 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 144918 2024-09-14T23:57:59.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.246 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:59.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:59.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416416+0000", 2024-09-14T23:57:59.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:59.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:59.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416416+0000", 2024-09-14T23:57:59.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:59.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:59.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416416+0000", 2024-09-14T23:57:59.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.06e-07 2024-09-14T23:57:59.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:59.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416418+0000", 2024-09-14T23:57:59.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.063e-06 2024-09-14T23:57:59.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:59.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416424+0000", 2024-09-14T23:57:59.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9030000000000001e-06 2024-09-14T23:57:59.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.248 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:59.248 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416431+0000", 2024-09-14T23:57:59.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7469999999999999e-06 2024-09-14T23:57:59.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.248 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:59.248 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.454098+0000", 2024-09-14T23:57:59.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037666408999999998 2024-09-14T23:57:59.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.248 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:59.248 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.454176+0000", 2024-09-14T23:57:59.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8857999999999996e-05 2024-09-14T23:57:59.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.249 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:59.249 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.454274+0000", 2024-09-14T23:57:59.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7224000000000003e-05 2024-09-14T23:57:59.249 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:59.249 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:59.249 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:59.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.249 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:144919 3.6 3:65baca07:::rbd_data.11189263ca38.000000000000000a:2a [read 1548288~8192] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-14T23:57:59.250 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:53.416438+0000", 2024-09-14T23:57:59.250 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.7796318859999998, 2024-09-14T23:57:59.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037973489999999999, 2024-09-14T23:57:59.250 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:59.250 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:59.250 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:59.250 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:59.250 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:59.250 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:59.250 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 144919 2024-09-14T23:57:59.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.250 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:59.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.250 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:59.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416438+0000", 2024-09-14T23:57:59.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:59.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:59.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416438+0000", 2024-09-14T23:57:59.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:59.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:59.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416439+0000", 2024-09-14T23:57:59.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6499999999999999e-07 2024-09-14T23:57:59.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:59.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416441+0000", 2024-09-14T23:57:59.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.255e-06 2024-09-14T23:57:59.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:59.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416448+0000", 2024-09-14T23:57:59.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7830000000000001e-06 2024-09-14T23:57:59.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:59.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416453+0000", 2024-09-14T23:57:59.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1000000000000003e-06 2024-09-14T23:57:59.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.253 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.253 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:59.253 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.454305+0000", 2024-09-14T23:57:59.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037851535999999998 2024-09-14T23:57:59.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.253 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.253 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:59.253 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.454355+0000", 2024-09-14T23:57:59.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0652999999999998e-05 2024-09-14T23:57:59.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.253 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.253 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:59.253 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.454412+0000", 2024-09-14T23:57:59.254 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6598000000000003e-05 2024-09-14T23:57:59.254 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:59.254 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:59.254 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:59.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.254 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:144953 3.6 3:6629293c:::rbd_data.11189263ca38.0000000000000009:2a [read 1335296~0] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-14T23:57:59.254 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:53.416731+0000", 2024-09-14T23:57:59.254 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.7793393039999996, 2024-09-14T23:57:59.254 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037873579999999997, 2024-09-14T23:57:59.254 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:59.254 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:59.254 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:59.254 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:59.255 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:59.255 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:59.255 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 144953 2024-09-14T23:57:59.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.255 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:59.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:59.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416731+0000", 2024-09-14T23:57:59.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:59.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:59.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416731+0000", 2024-09-14T23:57:59.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:59.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:59.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416731+0000", 2024-09-14T23:57:59.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.32e-07 2024-09-14T23:57:59.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:59.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416733+0000", 2024-09-14T23:57:59.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.058e-06 2024-09-14T23:57:59.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:59.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416740+0000", 2024-09-14T23:57:59.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3729999999999998e-06 2024-09-14T23:57:59.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:59.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416745+0000", 2024-09-14T23:57:59.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0799999999999996e-06 2024-09-14T23:57:59.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:59.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.454452+0000", 2024-09-14T23:57:59.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037706867999999998 2024-09-14T23:57:59.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:59.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.454528+0000", 2024-09-14T23:57:59.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6614999999999997e-05 2024-09-14T23:57:59.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:59.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.454604+0000", 2024-09-14T23:57:59.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6154000000000003e-05 2024-09-14T23:57:59.258 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:59.258 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:59.258 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:59.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.258 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:144954 3.6 3:6629293c:::rbd_data.11189263ca38.0000000000000009:2a [read 1400832~0] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-14T23:57:59.259 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:53.416751+0000", 2024-09-14T23:57:59.259 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.7793191159999999, 2024-09-14T23:57:59.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037978535000000001, 2024-09-14T23:57:59.259 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:59.259 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:59.259 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:59.259 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:59.259 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:59.259 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:59.259 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 144954 2024-09-14T23:57:59.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.259 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:59.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.259 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:59.259 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416751+0000", 2024-09-14T23:57:59.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:59.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:59.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416751+0000", 2024-09-14T23:57:59.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:59.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:59.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416752+0000", 2024-09-14T23:57:59.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1200000000000003e-07 2024-09-14T23:57:59.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:59.261 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416754+0000", 2024-09-14T23:57:59.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9589999999999998e-06 2024-09-14T23:57:59.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:59.261 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416760+0000", 2024-09-14T23:57:59.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9730000000000002e-06 2024-09-14T23:57:59.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:59.261 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416764+0000", 2024-09-14T23:57:59.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9099999999999996e-06 2024-09-14T23:57:59.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:59.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.454641+0000", 2024-09-14T23:57:59.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037876716999999997 2024-09-14T23:57:59.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:59.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.454685+0000", 2024-09-14T23:57:59.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3927999999999997e-05 2024-09-14T23:57:59.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:59.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.454730+0000", 2024-09-14T23:57:59.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4536000000000002e-05 2024-09-14T23:57:59.263 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:59.263 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:59.263 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:59.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.263 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:144955 3.6 3:6629293c:::rbd_data.11189263ca38.0000000000000009:2a [read 1466368~0] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-14T23:57:59.263 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:53.416771+0000", 2024-09-14T23:57:59.263 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.779299355, 2024-09-14T23:57:59.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.038156383000000002, 2024-09-14T23:57:59.263 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:59.263 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:59.263 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:59.263 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:59.263 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:59.264 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:59.264 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 144955 2024-09-14T23:57:59.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.264 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:59.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:59.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416771+0000", 2024-09-14T23:57:59.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:59.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:59.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416771+0000", 2024-09-14T23:57:59.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:59.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.265 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:59.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416771+0000", 2024-09-14T23:57:59.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5200000000000002e-07 2024-09-14T23:57:59.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.265 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:59.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416773+0000", 2024-09-14T23:57:59.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.137e-06 2024-09-14T23:57:59.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.265 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:59.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416780+0000", 2024-09-14T23:57:59.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3820000000000001e-06 2024-09-14T23:57:59.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:59.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416785+0000", 2024-09-14T23:57:59.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8400000000000002e-06 2024-09-14T23:57:59.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:59.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.454770+0000", 2024-09-14T23:57:59.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037985473999999998 2024-09-14T23:57:59.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:59.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.454851+0000", 2024-09-14T23:57:59.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0519999999999995e-05 2024-09-14T23:57:59.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.267 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:59.267 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.454927+0000", 2024-09-14T23:57:59.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6577999999999998e-05 2024-09-14T23:57:59.267 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:59.267 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:59.267 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:59.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.267 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:144956 3.6 3:6629293c:::rbd_data.11189263ca38.0000000000000009:2a [read 1531904~0] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-14T23:57:59.267 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:53.416808+0000", 2024-09-14T23:57:59.268 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.7792626660000002, 2024-09-14T23:57:59.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.038244848999999997, 2024-09-14T23:57:59.268 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:59.268 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:59.268 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:59.268 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:59.268 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:59.268 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:59.268 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 144956 2024-09-14T23:57:59.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.268 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:59.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:59.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416808+0000", 2024-09-14T23:57:59.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:59.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:59.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416808+0000", 2024-09-14T23:57:59.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:59.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:59.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416808+0000", 2024-09-14T23:57:59.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5300000000000004e-07 2024-09-14T23:57:59.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:59.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416810+0000", 2024-09-14T23:57:59.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.379e-06 2024-09-14T23:57:59.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:59.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416817+0000", 2024-09-14T23:57:59.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3380000000000001e-06 2024-09-14T23:57:59.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:59.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416822+0000", 2024-09-14T23:57:59.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4129999999999997e-06 2024-09-14T23:57:59.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:59.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.454964+0000", 2024-09-14T23:57:59.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.038142084 2024-09-14T23:57:59.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:59.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.455007+0000", 2024-09-14T23:57:59.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3099000000000002e-05 2024-09-14T23:57:59.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:59.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.455052+0000", 2024-09-14T23:57:59.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4982999999999999e-05 2024-09-14T23:57:59.272 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:59.272 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:59.272 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:59.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.272 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:144957 3.6 3:65baca07:::rbd_data.11189263ca38.000000000000000a:2a [read 1327104~8192] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-14T23:57:59.272 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:53.416830+0000", 2024-09-14T23:57:59.272 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.779240315, 2024-09-14T23:57:59.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.038439040000000001, 2024-09-14T23:57:59.272 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:59.272 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:59.272 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:59.273 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:59.273 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:59.273 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:59.273 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 144957 2024-09-14T23:57:59.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.273 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:59.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:59.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416830+0000", 2024-09-14T23:57:59.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:59.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:59.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416830+0000", 2024-09-14T23:57:59.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:59.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:59.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416830+0000", 2024-09-14T23:57:59.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3300000000000002e-07 2024-09-14T23:57:59.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:59.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416833+0000", 2024-09-14T23:57:59.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1689999999999999e-06 2024-09-14T23:57:59.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:59.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416839+0000", 2024-09-14T23:57:59.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.285e-06 2024-09-14T23:57:59.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:59.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416844+0000", 2024-09-14T23:57:59.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8359999999999999e-06 2024-09-14T23:57:59.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:59.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.455090+0000", 2024-09-14T23:57:59.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.038246629999999997 2024-09-14T23:57:59.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.276 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:59.276 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.455173+0000", 2024-09-14T23:57:59.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2194999999999995e-05 2024-09-14T23:57:59.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.276 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:59.276 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.455269+0000", 2024-09-14T23:57:59.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6391999999999995e-05 2024-09-14T23:57:59.276 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:59.276 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:59.276 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:59.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.277 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:144958 3.6 3:65baca07:::rbd_data.11189263ca38.000000000000000a:2a [read 1392640~8192] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-14T23:57:59.277 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:53.416852+0000", 2024-09-14T23:57:59.277 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.7792181349999998, 2024-09-14T23:57:59.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.038556741999999998, 2024-09-14T23:57:59.277 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:59.277 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:59.277 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:59.277 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:59.277 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:59.277 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:59.277 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 144958 2024-09-14T23:57:59.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.277 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:59.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:59.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416852+0000", 2024-09-14T23:57:59.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:59.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:59.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416852+0000", 2024-09-14T23:57:59.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:59.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:59.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416852+0000", 2024-09-14T23:57:59.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2599999999999998e-07 2024-09-14T23:57:59.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:59.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416855+0000", 2024-09-14T23:57:59.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2050000000000001e-06 2024-09-14T23:57:59.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:59.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416861+0000", 2024-09-14T23:57:59.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2299999999999996e-06 2024-09-14T23:57:59.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:59.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416866+0000", 2024-09-14T23:57:59.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8860000000000003e-06 2024-09-14T23:57:59.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.280 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:59.280 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.455301+0000", 2024-09-14T23:57:59.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.038434838999999998 2024-09-14T23:57:59.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.280 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:59.280 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.455349+0000", 2024-09-14T23:57:59.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8795999999999998e-05 2024-09-14T23:57:59.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.280 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:59.280 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.455409+0000", 2024-09-14T23:57:59.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9360000000000001e-05 2024-09-14T23:57:59.280 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:59.280 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:59.281 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:59.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.281 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:144959 3.6 3:65baca07:::rbd_data.11189263ca38.000000000000000a:2a [read 1458176~8192] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-14T23:57:59.281 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:53.416872+0000", 2024-09-14T23:57:59.281 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.7791978329999996, 2024-09-14T23:57:59.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.038754086, 2024-09-14T23:57:59.281 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:59.281 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:59.281 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:59.281 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:59.281 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:59.282 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:59.282 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 144959 2024-09-14T23:57:59.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.282 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:59.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:59.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416872+0000", 2024-09-14T23:57:59.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:59.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:59.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416872+0000", 2024-09-14T23:57:59.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:59.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:59.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416873+0000", 2024-09-14T23:57:59.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4400000000000001e-07 2024-09-14T23:57:59.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:59.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416875+0000", 2024-09-14T23:57:59.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0839999999999999e-06 2024-09-14T23:57:59.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:59.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416881+0000", 2024-09-14T23:57:59.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.229e-06 2024-09-14T23:57:59.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:59.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416886+0000", 2024-09-14T23:57:59.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9439999999999996e-06 2024-09-14T23:57:59.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:59.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.455448+0000", 2024-09-14T23:57:59.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.038562328 2024-09-14T23:57:59.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:59.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.455528+0000", 2024-09-14T23:57:59.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9729999999999997e-05 2024-09-14T23:57:59.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.285 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:59.285 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.455626+0000", 2024-09-14T23:57:59.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8426999999999998e-05 2024-09-14T23:57:59.285 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:59.285 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:59.285 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:59.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.285 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:144960 3.6 3:65baca07:::rbd_data.11189263ca38.000000000000000a:2a [read 1523712~8192] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-14T23:57:59.285 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:53.416893+0000", 2024-09-14T23:57:59.285 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.7791776659999998, 2024-09-14T23:57:59.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.038862725000000001, 2024-09-14T23:57:59.285 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:57:59.286 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:57:59.286 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:57:59.286 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:57:59.286 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:57:59.286 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:57:59.286 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 144960 2024-09-14T23:57:59.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.286 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:57:59.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:57:59.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416893+0000", 2024-09-14T23:57:59.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:59.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:57:59.287 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416893+0000", 2024-09-14T23:57:59.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:57:59.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.287 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:57:59.287 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416893+0000", 2024-09-14T23:57:59.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3299999999999998e-07 2024-09-14T23:57:59.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.287 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:57:59.287 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416907+0000", 2024-09-14T23:57:59.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4569000000000001e-05 2024-09-14T23:57:59.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.287 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:57:59.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416915+0000", 2024-09-14T23:57:59.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7910000000000005e-06 2024-09-14T23:57:59.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:57:59.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416923+0000", 2024-09-14T23:57:59.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3459999999999996e-06 2024-09-14T23:57:59.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:57:59.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.455648+0000", 2024-09-14T23:57:59.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.03872573 2024-09-14T23:57:59.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:57:59.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.455699+0000", 2024-09-14T23:57:59.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0929e-05 2024-09-14T23:57:59.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:57:59.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:57:59.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:57:59.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.455755+0000", 2024-09-14T23:57:59.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6026999999999997e-05 2024-09-14T23:57:59.289 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:59.289 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:59.289 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:59.289 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:57:59.289 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:57:59.290 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:59.290 INFO:teuthology.orchestra.run.smithi042.stdout:5512 read 0x9ab7652 thru 0x9ac1e32 (0xa7e1 bytes) 2024-09-14T23:57:59.291 INFO:teuthology.orchestra.run.smithi042.stdout:5513 writesame 0xbfdab56 thru 0xbfe1d46 data_size 0x15(0x71f1 bytes) 2024-09-14T23:57:59.292 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T23:57:59.294 INFO:teuthology.orchestra.run.smithi042.stdout:5517 punch from 0x92de270 to 0x92edf8c, (0xfd1c bytes) 2024-09-14T23:57:59.294 INFO:teuthology.orchestra.run.smithi042.stdout:5518 read 0xcba9a26 thru 0xcba9dce (0x3a9 bytes) 2024-09-14T23:57:59.294 INFO:teuthology.orchestra.run.smithi042.stdout:5520 punch from 0xe66ad9 to 0xe6702e, (0x555 bytes) 2024-09-14T23:57:59.294 INFO:teuthology.orchestra.run.smithi042.stdout:5521 write 0xbc60bb8 thru 0xbc69466 (0x88af bytes) 2024-09-14T23:57:59.295 INFO:teuthology.orchestra.run.smithi042.stdout:5522 punch from 0x4e4f448 to 0x4e52a0e, (0x35c6 bytes) 2024-09-14T23:57:59.295 INFO:teuthology.orchestra.run.smithi042.stdout:5524 write 0x93a3bb8 thru 0x93a574c (0x1b95 bytes) 2024-09-14T23:57:59.295 INFO:teuthology.orchestra.run.smithi042.stdout:5525 trunc from 0xda93f32 to 0xe444b2c 2024-09-14T23:57:59.295 INFO:teuthology.orchestra.run.smithi042.stdout:5526 writesame 0x8d0438a thru 0x8d09257 data_size 0x16(0x4ece bytes) 2024-09-14T23:57:59.295 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:59.295 INFO:teuthology.orchestra.run.smithi042.stdout:5528 write 0x703fb6 thru 0x705d0a (0x1d55 bytes) 2024-09-14T23:57:59.295 INFO:teuthology.orchestra.run.smithi042.stdout:5530 read 0xbc6875b thru 0xbc74f02 (0xc7a8 bytes) 2024-09-14T23:57:59.295 INFO:teuthology.orchestra.run.smithi042.stdout:5531 read 0xd3eb085 thru 0xd3f8144 (0xd0c0 bytes) 2024-09-14T23:57:59.295 INFO:teuthology.orchestra.run.smithi042.stdout:5532 write 0xbbda599 thru 0xbbe5664 (0xb0cc bytes) 2024-09-14T23:57:59.295 INFO:teuthology.orchestra.run.smithi042.stdout:5533 write 0xe42bcf6 thru 0xe438561 (0xc86c bytes) 2024-09-14T23:57:59.295 INFO:teuthology.orchestra.run.smithi042.stdout:5535 writesame 0x6dab0d4 thru 0x6db44ec data_size 0x1(0x9419 bytes) 2024-09-14T23:57:59.295 INFO:teuthology.orchestra.run.smithi042.stdout:5536 write 0xe015697 thru 0xe01e872 (0x91dc bytes) 2024-09-14T23:57:59.295 INFO:teuthology.orchestra.run.smithi042.stdout:5537 write 0x8e02bbd thru 0x8e099a6 (0x6dea bytes) 2024-09-14T23:57:59.296 INFO:teuthology.orchestra.run.smithi042.stdout:5538 read 0x2d0184 thru 0x2d9a69 (0x98e6 bytes) 2024-09-14T23:57:59.296 INFO:teuthology.orchestra.run.smithi042.stdout:5539 read 0x1a95eb2 thru 0x1a98e5c (0x2fab bytes) 2024-09-14T23:57:59.296 INFO:teuthology.orchestra.run.smithi042.stdout:5540 writesame 0xb610d05 thru 0xb61be83 data_size 0x1(0xb17f bytes) 2024-09-14T23:57:59.296 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:59.296 INFO:teuthology.orchestra.run.smithi042.stdout:5543 read 0x8958b22 thru 0x895e9ab (0x5e8a bytes) 2024-09-14T23:57:59.296 INFO:teuthology.orchestra.run.smithi042.stdout:5546 read 0x25e40d8 thru 0x25e8be4 (0x4b0d bytes) 2024-09-14T23:57:59.296 INFO:teuthology.orchestra.run.smithi042.stdout:5547 trunc from 0xe444b2c to 0x7485721 2024-09-14T23:57:59.296 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:57:59.289+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:57:59.322 INFO:teuthology.orchestra.run.smithi042.stdout:5548 writesame 0xe13fe75 thru 0xe14c48b data_size 0x13(0xc617 bytes) 2024-09-14T23:57:59.324 INFO:teuthology.orchestra.run.smithi042.stdout:5549 trunc from 0xe14c48c to 0xc3f5b6a 2024-09-14T23:57:59.330 INFO:teuthology.orchestra.run.smithi042.stdout:5551 punch from 0x87da850 to 0x87ea2cd, (0xfa7d bytes) 2024-09-14T23:57:59.330 INFO:teuthology.orchestra.run.smithi042.stdout:5552 write 0x33aa83e thru 0x33b7223 (0xc9e6 bytes) 2024-09-14T23:57:59.332 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:59.332 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:59.332 INFO:teuthology.orchestra.run.smithi042.stdout:5555 write 0x72eb3b0 thru 0x72f8dac (0xd9fd bytes) 2024-09-14T23:57:59.337 INFO:teuthology.orchestra.run.smithi042.stdout:5558 write 0x8267311 thru 0x8267f77 (0xc67 bytes) 2024-09-14T23:57:59.350 INFO:teuthology.orchestra.run.smithi042.stdout:5559 trunc from 0xc3f5b6a to 0x38c60f2 2024-09-14T23:57:59.382 INFO:teuthology.orchestra.run.smithi042.stdout:5561 write 0xe7cca53 thru 0xe7d703c (0xa5ea bytes) 2024-09-14T23:57:59.391 INFO:teuthology.orchestra.run.smithi042.stdout:5563 read 0x128a8c8 thru 0x128b95c (0x1095 bytes) 2024-09-14T23:57:59.392 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:59.389+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:59.392 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:59.392 INFO:teuthology.orchestra.run.smithi042.stdout:5565 read 0x8ea20ff thru 0x8eaf14e (0xd050 bytes) 2024-09-14T23:57:59.392 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:59.392 INFO:teuthology.orchestra.run.smithi042.stdout:5568 write 0x65ef04c thru 0x65f4c78 (0x5c2d bytes) 2024-09-14T23:57:59.397 INFO:teuthology.orchestra.run.smithi042.stdout:5571 writesame 0xa02f11f thru 0xa035dd9 data_size 0x13(0x6cbb bytes) 2024-09-14T23:57:59.403 INFO:teuthology.orchestra.run.smithi042.stdout:5572 writesame 0x1489b2b thru 0x14905a9 data_size 0x1(0x6a7f bytes) 2024-09-14T23:57:59.408 INFO:teuthology.orchestra.run.smithi042.stdout:5574 write 0x2ec3f38 thru 0x2ec4e35 (0xefe bytes) 2024-09-14T23:57:59.410 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:59.410 INFO:teuthology.orchestra.run.smithi042.stdout:5576 write 0xd3fef73 thru 0xd40d8e6 (0xe974 bytes) 2024-09-14T23:57:59.419 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:59.420 INFO:teuthology.orchestra.run.smithi042.stdout:5580 punch from 0x24a931e to 0x24b672d, (0xd40f bytes) 2024-09-14T23:57:59.420 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:57:59.420 INFO:teuthology.orchestra.run.smithi042.stdout:5583 read 0xcf5a36c thru 0xcf5c2f8 (0x1f8d bytes) 2024-09-14T23:57:59.420 INFO:teuthology.orchestra.run.smithi042.stdout:5584 read 0x2f3bae4 thru 0x2f4b2c3 (0xf7e0 bytes) 2024-09-14T23:57:59.420 INFO:teuthology.orchestra.run.smithi042.stdout:5586 read 0x226457 thru 0x22f8b6 (0x9460 bytes) 2024-09-14T23:57:59.431 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T23:57:59.435 INFO:teuthology.orchestra.run.smithi042.stdout:5587 trunc from 0xe7d703d to 0x209bedd 2024-09-14T23:57:59.461 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:57:59.461 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:57:59.461 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:57:59.461 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:57:59.474 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T23:57:59.479 INFO:teuthology.orchestra.run.smithi042.stdout:5588 clone 47 order 22 su 131072 sc 7 2024-09-14T23:57:59.492 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:59.490+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:59.572 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:57:59.572 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:57:59.589 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:57:59.590 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:57:59.590 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:59.590 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:59.593 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:57:59.592+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:57:59.598 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T23:57:59.606 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:57:59.606 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:57:59.622 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:57:59.623 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:57:59.639 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:57:59.639 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:57:59.656 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:57:59.656 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:57:59.673 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:57:59.673 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:57:59.690 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:57:59.690 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:57:59.694 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:59.692+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:59.706 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:59.707 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:59.716 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T23:57:59.795 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:57:59.794+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:57:59.864 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:59.864 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:59.873 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T23:57:59.896 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:57:59.895+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:57:59.979 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:57:59.980 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:57:59.988 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T23:58:00.031 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:00.030+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:00.095 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:00.095 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:00.098 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:58:00.096+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:58:00.106 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T23:58:00.106 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T23:58:00.106 INFO:tasks.thrashosds.thrasher:Reweighting osd 6 to 0.9245371628386688 2024-09-14T23:58:00.106 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 0.9245371628386688 2024-09-14T23:58:00.107 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T23:58:00.127 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T23:58:00.199 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:00.198+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:00.214 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone46 from 0x209bedd (overlap 0x209bedd) to 0x104df6e 2024-09-14T23:58:00.285 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:00.285 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:00.296 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T23:58:00.337 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:00.335+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:00.401 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:00.400+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:00.410 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #45, image image_client.0-clone45 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent46 2024-09-14T23:58:00.453 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:00.453 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:00.467 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T23:58:00.502 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:00.500+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:00.603 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:00.602+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:00.617 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:00.618 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:00.627 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T23:58:00.668 INFO:teuthology.orchestra.run.smithi042.stdout:5589 read 0x175491f thru 0x175ff02 (0xb5e4 bytes) 2024-09-14T23:58:00.683 INFO:teuthology.orchestra.run.smithi042.stdout:5590 write 0x5b3dac9 thru 0x5b3e393 (0x8cb bytes) 2024-09-14T23:58:00.687 INFO:teuthology.orchestra.run.smithi042.stdout:5591 read 0x1f8e18 thru 0x1fbd1c (0x2f05 bytes) 2024-09-14T23:58:00.699 INFO:teuthology.orchestra.run.smithi042.stdout:5592 write 0xa34afd0 thru 0xa35a99a (0xf9cb bytes) 2024-09-14T23:58:00.702 INFO:teuthology.orchestra.run.smithi042.stdout:5593 write 0x21f1e2d thru 0x21f38dc (0x1ab0 bytes) 2024-09-14T23:58:00.704 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:00.703+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:00.708 INFO:teuthology.orchestra.run.smithi042.stdout:5596 writesame 0x66f2e5a thru 0x66fcb5a data_size 0x1(0x9d01 bytes) 2024-09-14T23:58:00.729 INFO:teuthology.orchestra.run.smithi042.stdout:5597 write 0x5e1bbb5 thru 0x5e21c41 (0x608d bytes) 2024-09-14T23:58:00.732 INFO:teuthology.orchestra.run.smithi042.stdout:5598 read 0x240bf45 thru 0x24126d8 (0x6794 bytes) 2024-09-14T23:58:00.732 INFO:teuthology.orchestra.run.smithi042.stdout:5599 write 0x29b0c5f thru 0x29c09ac (0xfd4e bytes) 2024-09-14T23:58:00.739 INFO:teuthology.orchestra.run.smithi042.stdout:5600 read 0x216222a thru 0x216447b (0x2252 bytes) 2024-09-14T23:58:00.740 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:00.740 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:00.744 INFO:teuthology.orchestra.run.smithi042.stdout:5601 trunc from 0xa35a99b to 0x5c37612 2024-09-14T23:58:00.749 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T23:58:00.756 INFO:teuthology.orchestra.run.smithi042.stdout:5603 read 0x5b33c74 thru 0x5b41767 (0xdaf4 bytes) 2024-09-14T23:58:00.771 INFO:teuthology.orchestra.run.smithi042.stdout:5604 write 0xc528677 thru 0xc537ff1 (0xf97b bytes) 2024-09-14T23:58:00.774 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:58:00.774 INFO:teuthology.orchestra.run.smithi042.stdout:5609 read 0x3ea177f thru 0x3ea5333 (0x3bb5 bytes) 2024-09-14T23:58:00.775 INFO:teuthology.orchestra.run.smithi042.stdout:5610 write 0xad4d6f7 thru 0xad5c362 (0xec6c bytes) 2024-09-14T23:58:00.778 INFO:teuthology.orchestra.run.smithi042.stdout:5611 writesame 0xbd0778e thru 0xbd0b374 data_size 0x1(0x3be7 bytes) 2024-09-14T23:58:00.795 INFO:teuthology.orchestra.run.smithi042.stdout:5612 trunc from 0xc537ff2 to 0x4772737 2024-09-14T23:58:00.805 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:58:00.805 INFO:teuthology.orchestra.run.smithi042.stdout:5614 write 0x13621 thru 0x1e3ca (0xadaa bytes) 2024-09-14T23:58:00.805 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:00.803+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:00.839 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:58:00.852 INFO:teuthology.orchestra.run.smithi042.stdout:5616 write 0xb0bfed7 thru 0xb0cf97e (0xfaa8 bytes) 2024-09-14T23:58:00.857 INFO:teuthology.orchestra.run.smithi042.stdout:5617 write 0x20d2d4a thru 0x20e0e56 (0xe10d bytes) 2024-09-14T23:58:00.864 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:58:00.865 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:58:00.865 INFO:teuthology.orchestra.run.smithi042.stdout:5620 write 0x113efca thru 0x1142901 (0x3938 bytes) 2024-09-14T23:58:00.878 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:00.878 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:00.891 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T23:58:00.905 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:00.904+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:00.913 INFO:teuthology.orchestra.run.smithi042.stdout:5621 read 0x19bb75 thru 0x1a28cf (0x6d5b bytes) 2024-09-14T23:58:00.913 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:58:00.913 INFO:teuthology.orchestra.run.smithi042.stdout:5624 write 0x6dd69a9 thru 0x6de2338 (0xb990 bytes) 2024-09-14T23:58:00.919 INFO:teuthology.orchestra.run.smithi042.stdout:5625 trunc from 0xb0cf97f to 0x910b6fb 2024-09-14T23:58:00.930 INFO:teuthology.orchestra.run.smithi042.stdout:5626 read 0x3e37ce thru 0x3e8457 (0x4c8a bytes) 2024-09-14T23:58:00.934 INFO:teuthology.orchestra.run.smithi042.stdout:5627 trunc from 0x910b6fb to 0xae69def 2024-09-14T23:58:00.939 INFO:teuthology.orchestra.run.smithi042.stdout:5628 punch from 0x8daa3df to 0x8dae494, (0x40b5 bytes) 2024-09-14T23:58:00.942 INFO:teuthology.orchestra.run.smithi042.stdout:5629 trunc from 0xae69def to 0x89070dc 2024-09-14T23:58:00.948 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:58:00.948 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:58:00.948 INFO:teuthology.orchestra.run.smithi042.stdout:5634 read 0x797e86b thru 0x7987f98 (0x972e bytes) 2024-09-14T23:58:00.948 INFO:teuthology.orchestra.run.smithi042.stdout:5635 punch from 0x593de7d to 0x594793d, (0x9ac0 bytes) 2024-09-14T23:58:00.948 INFO:teuthology.orchestra.run.smithi042.stdout:5636 read 0x1d96885 thru 0x1d9b115 (0x4891 bytes) 2024-09-14T23:58:00.948 INFO:teuthology.orchestra.run.smithi042.stdout:5637 punch from 0x824d1bb to 0x8259449, (0xc28e bytes) 2024-09-14T23:58:00.948 INFO:teuthology.orchestra.run.smithi042.stdout:5638 read 0x4848d67 thru 0x4852b97 (0x9e31 bytes) 2024-09-14T23:58:00.949 INFO:teuthology.orchestra.run.smithi042.stdout:5641 trunc from 0x89070dc to 0x91e9b63 2024-09-14T23:58:00.953 INFO:teuthology.orchestra.run.smithi042.stdout:5642 read 0x88cf73f thru 0x88dc9bd (0xd27f bytes) 2024-09-14T23:58:00.953 INFO:teuthology.orchestra.run.smithi042.stdout:5645 read 0x86951ff thru 0x869b869 (0x666b bytes) 2024-09-14T23:58:00.962 INFO:teuthology.orchestra.run.smithi042.stdout:5647 write 0xbd3cab3 thru 0xbd49f58 (0xd4a6 bytes) 2024-09-14T23:58:00.970 INFO:teuthology.orchestra.run.smithi042.stdout:5648 writesame 0xbe4a386 thru 0xbe508a7 data_size 0xf(0x6522 bytes) 2024-09-14T23:58:00.970 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:58:00.970 INFO:teuthology.orchestra.run.smithi042.stdout:5650 punch from 0x131c19b to 0x13235bb, (0x7420 bytes) 2024-09-14T23:58:00.980 INFO:teuthology.orchestra.run.smithi042.stdout:5651 writesame 0xea2bb83 thru 0xea38f0c data_size 0x1(0xd38a bytes) 2024-09-14T23:58:00.980 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:58:00.980 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:58:00.980 INFO:teuthology.orchestra.run.smithi042.stdout:5658 write 0x98882be thru 0x988e649 (0x638c bytes) 2024-09-14T23:58:00.983 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:58:00.983 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:58:00.993 INFO:teuthology.orchestra.run.smithi042.stdout:5662 trunc from 0xea38f0d to 0x63f0f92 2024-09-14T23:58:01.006 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:01.004+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:01.008 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:01.008 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:01.008 INFO:teuthology.orchestra.run.smithi042.stdout:5663 trunc from 0x63f0f92 to 0x1b850fe 2024-09-14T23:58:01.020 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T23:58:01.084 INFO:teuthology.orchestra.run.smithi042.stdout:5664 read 0x124f3dd thru 0x125c7aa (0xd3ce bytes) 2024-09-14T23:58:01.084 INFO:teuthology.orchestra.run.smithi042.stdout:5665 read 0x4325c8 thru 0x43726f (0x4ca8 bytes) 2024-09-14T23:58:01.085 INFO:teuthology.orchestra.run.smithi042.stdout:5666 punch from 0x145d4c5 to 0x146949f, (0xbfda bytes) 2024-09-14T23:58:01.097 INFO:teuthology.orchestra.run.smithi042.stdout:5667 trunc from 0x1b850fe to 0xc846bcf 2024-09-14T23:58:01.102 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.6 to 0.924537 (ecae) 2024-09-14T23:58:01.103 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is unset 2024-09-14T23:58:01.104 INFO:teuthology.orchestra.run.smithi042.stdout:5668 read 0xba54de thru 0xbb325e (0xdd81 bytes) 2024-09-14T23:58:01.108 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:01.106+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:01.109 INFO:teuthology.orchestra.run.smithi042.stdout:5669 read 0xa5567c thru 0xa5cd0c (0x7691 bytes) 2024-09-14T23:58:01.110 INFO:teuthology.orchestra.run.smithi042.stdout:5671 write 0x6b568f3 thru 0x6b6133b (0xaa49 bytes) 2024-09-14T23:58:01.115 INFO:teuthology.orchestra.run.smithi042.stdout:5672 read 0xb41e8b3 thru 0xb423323 (0x4a71 bytes) 2024-09-14T23:58:01.115 INFO:teuthology.orchestra.run.smithi042.stdout:5673 read 0xaf6be37 thru 0xaf74432 (0x85fc bytes) 2024-09-14T23:58:01.123 INFO:teuthology.orchestra.run.smithi042.stdout:5674 write 0xe7beebe thru 0xe7c68bd (0x7a00 bytes) 2024-09-14T23:58:01.127 INFO:teuthology.orchestra.run.smithi042.stdout:5676 punch from 0xaed6852 to 0xaee09ea, (0xa198 bytes) 2024-09-14T23:58:01.143 INFO:teuthology.orchestra.run.smithi042.stdout:5677 trunc from 0xe7c68be to 0xeed197 2024-09-14T23:58:01.186 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:01.186 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:01.195 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T23:58:01.197 INFO:teuthology.orchestra.run.smithi042.stdout:5678 read 0xe0deb8 thru 0xe121b4 (0x42fd bytes) 2024-09-14T23:58:01.197 INFO:teuthology.orchestra.run.smithi042.stdout:5679 punch from 0x702791 to 0x708a1e, (0x628d bytes) 2024-09-14T23:58:01.197 INFO:teuthology.orchestra.run.smithi042.stdout:5682 read 0xa11ed1 thru 0xa1f794 (0xd8c4 bytes) 2024-09-14T23:58:01.197 INFO:teuthology.orchestra.run.smithi042.stdout:5683 read 0xa09db3 thru 0xa168ae (0xcafc bytes) 2024-09-14T23:58:01.209 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:01.207+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:01.211 INFO:teuthology.orchestra.run.smithi042.stdout:5684 write 0x88fdd49 thru 0x8907dd6 (0xa08e bytes) 2024-09-14T23:58:01.216 INFO:teuthology.orchestra.run.smithi042.stdout:5686 punch from 0x188a569 to 0x1891bfc, (0x7693 bytes) 2024-09-14T23:58:01.216 INFO:teuthology.orchestra.run.smithi042.stdout:5687 read 0x7a8313f thru 0x7a8d16c (0xa02e bytes) 2024-09-14T23:58:01.216 INFO:teuthology.orchestra.run.smithi042.stdout:5688 write 0x6125cd4 thru 0x612c878 (0x6ba5 bytes) 2024-09-14T23:58:01.227 INFO:teuthology.orchestra.run.smithi042.stdout:5691 write 0xb99dabc thru 0xb99ea09 (0xf4e bytes) 2024-09-14T23:58:01.230 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:58:01.230 INFO:teuthology.orchestra.run.smithi042.stdout:5693 read 0xaf2268e thru 0xaf246e6 (0x2059 bytes) 2024-09-14T23:58:01.233 INFO:teuthology.orchestra.run.smithi042.stdout:5694 trunc from 0xb99ea0a to 0xde0f58a 2024-09-14T23:58:01.239 INFO:teuthology.orchestra.run.smithi042.stdout:5695 writesame 0xcaaec28 thru 0xcab98bd data_size 0x1(0xac96 bytes) 2024-09-14T23:58:01.258 INFO:teuthology.orchestra.run.smithi042.stdout:5696 write 0xb5b2ed thru 0xb678c2 (0xc5d6 bytes) 2024-09-14T23:58:01.260 INFO:teuthology.orchestra.run.smithi042.stdout:5698 writesame 0x963125f thru 0x963cc37 data_size 0x1(0xb9d9 bytes) 2024-09-14T23:58:01.281 INFO:teuthology.orchestra.run.smithi042.stdout:5700 trunc from 0xde0f58a to 0xe750806 2024-09-14T23:58:01.460 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:01.460 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:01.460 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T23:58:01.461 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:01.456+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:01.462 INFO:teuthology.orchestra.run.smithi042.stdout:5703 trunc from 0xe750806 to 0xafef319 2024-09-14T23:58:01.462 INFO:teuthology.orchestra.run.smithi042.stdout:5704 writesame 0x674e055 thru 0x675d2fe data_size 0x59(0xf2aa bytes) 2024-09-14T23:58:01.462 INFO:teuthology.orchestra.run.smithi042.stdout:5705 write 0xc3113ef thru 0xc3190e1 (0x7cf3 bytes) 2024-09-14T23:58:01.462 INFO:teuthology.orchestra.run.smithi042.stdout:5706 writesame 0x555f2ae thru 0x5561d8a data_size 0x1(0x2add bytes) 2024-09-14T23:58:01.463 INFO:teuthology.orchestra.run.smithi042.stdout:5708 trunc from 0xc3190e2 to 0x3a576cf 2024-09-14T23:58:01.463 INFO:teuthology.orchestra.run.smithi042.stdout:5709 write 0x3eb7c60 thru 0x3ec6eb8 (0xf259 bytes) 2024-09-14T23:58:01.463 INFO:teuthology.orchestra.run.smithi042.stdout:5710 trunc from 0x3ec6eb9 to 0x50c07d3 2024-09-14T23:58:01.463 INFO:teuthology.orchestra.run.smithi042.stdout:5711 write 0x1a1d96b thru 0x1a28e4d (0xb4e3 bytes) 2024-09-14T23:58:01.463 INFO:teuthology.orchestra.run.smithi042.stdout:5712 trunc from 0x50c07d3 to 0x7d28d92 2024-09-14T23:58:01.463 INFO:teuthology.orchestra.run.smithi042.stdout:5713 write 0x6cc4d34 thru 0x6cce3bd (0x968a bytes) 2024-09-14T23:58:01.463 INFO:teuthology.orchestra.run.smithi042.stdout:5714 read 0x3d7a106 thru 0x3d7cd2a (0x2c25 bytes) 2024-09-14T23:58:01.463 INFO:teuthology.orchestra.run.smithi042.stdout:5715 write 0xad5280c thru 0xad62708 (0xfefd bytes) 2024-09-14T23:58:01.463 INFO:teuthology.orchestra.run.smithi042.stdout:5716 trunc from 0xad62709 to 0x85fbe98 2024-09-14T23:58:01.463 INFO:teuthology.orchestra.run.smithi042.stdout:5717 read 0x6817561 thru 0x681de31 (0x68d1 bytes) 2024-09-14T23:58:01.463 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:58:01.463 INFO:teuthology.orchestra.run.smithi042.stdout:5719 trunc from 0x85fbe98 to 0x489276d 2024-09-14T23:58:01.463 INFO:teuthology.orchestra.run.smithi042.stdout:5721 read 0xbbf3c1 thru 0xbc0e51 (0x1a91 bytes) 2024-09-14T23:58:01.463 INFO:teuthology.orchestra.run.smithi042.stdout:5722 read 0x194f34e thru 0x1955555 (0x6208 bytes) 2024-09-14T23:58:01.464 INFO:teuthology.orchestra.run.smithi042.stdout:5723 write 0xc3086d4 thru 0xc31506e (0xc99b bytes) 2024-09-14T23:58:01.464 INFO:teuthology.orchestra.run.smithi042.stdout:5724 writesame 0x7c2a123 thru 0x7c344fc data_size 0x3(0xa3da bytes) 2024-09-14T23:58:01.464 INFO:teuthology.orchestra.run.smithi042.stdout:5725 write 0x38073bf thru 0x381598d (0xe5cf bytes) 2024-09-14T23:58:01.464 INFO:teuthology.orchestra.run.smithi042.stdout:5727 writesame 0xb06523 thru 0xb121db data_size 0x1(0xbcb9 bytes) 2024-09-14T23:58:01.465 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:58:01.466 INFO:teuthology.orchestra.run.smithi042.stdout:5729 trunc from 0xc31506f to 0xd16cec3 2024-09-14T23:58:01.472 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:58:01.472 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:58:01.472 INFO:teuthology.orchestra.run.smithi042.stdout:5733 read 0x7f6473f thru 0x7f708e6 (0xc1a8 bytes) 2024-09-14T23:58:01.472 INFO:teuthology.orchestra.run.smithi042.stdout:5734 write 0x9252d2c thru 0x92619a7 (0xec7c bytes) 2024-09-14T23:58:01.476 INFO:teuthology.orchestra.run.smithi042.stdout:5736 write 0x8f5fe7b thru 0x8f6ca44 (0xcbca bytes) 2024-09-14T23:58:01.480 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:58:01.480 INFO:teuthology.orchestra.run.smithi042.stdout:5738 punch from 0x594a17 to 0x59bb83, (0x716c bytes) 2024-09-14T23:58:01.480 INFO:teuthology.orchestra.run.smithi042.stdout:5739 read 0x9f1d0b0 thru 0x9f27b67 (0xaab8 bytes) 2024-09-14T23:58:01.481 INFO:teuthology.orchestra.run.smithi042.stdout:5740 write 0x897cc4e thru 0x8986e3a (0xa1ed bytes) 2024-09-14T23:58:01.484 INFO:teuthology.orchestra.run.smithi042.stdout:5741 write 0xa8a6f81 thru 0xa8afa56 (0x8ad6 bytes) 2024-09-14T23:58:01.487 INFO:teuthology.orchestra.run.smithi042.stdout:5742 write 0x87c458b thru 0x87c884e (0x42c4 bytes) 2024-09-14T23:58:01.488 INFO:teuthology.orchestra.run.smithi042.stdout:5743 write 0x430098a thru 0x4303d1a (0x3391 bytes) 2024-09-14T23:58:01.497 INFO:teuthology.orchestra.run.smithi042.stdout:5744 write 0xe1a734b thru 0xe1b2703 (0xb3b9 bytes) 2024-09-14T23:58:01.500 INFO:teuthology.orchestra.run.smithi042.stdout:5745 read 0x6c6484c thru 0x6c67a2e (0x31e3 bytes) 2024-09-14T23:58:01.500 INFO:teuthology.orchestra.run.smithi042.stdout:5746 read 0x60a0de1 thru 0x60a3ca5 (0x2ec5 bytes) 2024-09-14T23:58:01.500 INFO:teuthology.orchestra.run.smithi042.stdout:5747 read 0x593e6d1 thru 0x594ac69 (0xc599 bytes) 2024-09-14T23:58:01.500 INFO:teuthology.orchestra.run.smithi042.stdout:5749 write 0xe0a8660 thru 0xe0b4ddf (0xc780 bytes) 2024-09-14T23:58:01.503 INFO:teuthology.orchestra.run.smithi042.stdout:5750 writesame 0xafc2bc8 thru 0xafcf00c data_size 0xd(0xc445 bytes) 2024-09-14T23:58:01.505 INFO:teuthology.orchestra.run.smithi042.stdout:5751 read 0xca93dbe thru 0xca9911b (0x535e bytes) 2024-09-14T23:58:01.505 INFO:teuthology.orchestra.run.smithi042.stdout:5752 read 0x1662087 thru 0x166f636 (0xd5b0 bytes) 2024-09-14T23:58:01.506 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:58:01.506 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:58:01.506 INFO:teuthology.orchestra.run.smithi042.stdout:5756 writesame 0x72490b4 thru 0x72513d9 data_size 0x1(0x8326 bytes) 2024-09-14T23:58:01.523 INFO:teuthology.orchestra.run.smithi042.stdout:5759 write 0x5a2cd7e thru 0x5a336b7 (0x693a bytes) 2024-09-14T23:58:01.526 INFO:teuthology.orchestra.run.smithi042.stdout:5760 write 0xdbc0c64 thru 0xdbca4e2 (0x987f bytes) 2024-09-14T23:58:01.530 INFO:teuthology.orchestra.run.smithi042.stdout:5761 write 0x6051a83 thru 0x605f7f3 (0xdd71 bytes) 2024-09-14T23:58:01.531 INFO:teuthology.orchestra.run.smithi042.stdout:5762 read 0x4835465 thru 0x484526c (0xfe08 bytes) 2024-09-14T23:58:01.532 INFO:teuthology.orchestra.run.smithi042.stdout:5766 write 0x77f9335 thru 0x77fdf81 (0x4c4d bytes) 2024-09-14T23:58:01.534 INFO:teuthology.orchestra.run.smithi042.stdout:5767 read 0x80ff98a thru 0x810c74f (0xcdc6 bytes) 2024-09-14T23:58:01.534 INFO:teuthology.orchestra.run.smithi042.stdout:5769 punch from 0x2e9ab43 to 0x2ea0da1, (0x625e bytes) 2024-09-14T23:58:01.534 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:58:01.534 INFO:teuthology.orchestra.run.smithi042.stdout:5774 write 0x937d103 thru 0x938424c (0x714a bytes) 2024-09-14T23:58:01.544 INFO:teuthology.orchestra.run.smithi042.stdout:5775 trunc from 0xe1b2704 to 0x8631e30 2024-09-14T23:58:01.557 INFO:teuthology.orchestra.run.smithi042.stdout:5776 trunc from 0x8631e30 to 0x81f3dd9 2024-09-14T23:58:01.560 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:01.558+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:01.563 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:58:01.563 INFO:teuthology.orchestra.run.smithi042.stdout:5780 writesame 0x1784ea2 thru 0x1791192 data_size 0xf(0xc2f1 bytes) 2024-09-14T23:58:01.575 INFO:teuthology.orchestra.run.smithi042.stdout:5781 writesame 0xc25daac thru 0xc26a97b data_size 0x18(0xced0 bytes) 2024-09-14T23:58:01.575 INFO:teuthology.orchestra.run.smithi042.stdout:5782 write 0x91bc2e1 thru 0x91c8851 (0xc571 bytes) 2024-09-14T23:58:01.579 INFO:teuthology.orchestra.run.smithi042.stdout:5783 punch from 0x8e5a44 to 0x8ed84e, (0x7e0a bytes) 2024-09-14T23:58:01.579 INFO:teuthology.orchestra.run.smithi042.stdout:5785 read 0x275cc56 thru 0x276bf9f (0xf34a bytes) 2024-09-14T23:58:01.583 INFO:teuthology.orchestra.run.smithi042.stdout:5787 trunc from 0xc26a97c to 0x8007010 2024-09-14T23:58:01.601 INFO:teuthology.orchestra.run.smithi042.stdout:5788 trunc from 0x8007010 to 0x2ee17ce 2024-09-14T23:58:01.616 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:01.616 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:01.625 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T23:58:01.626 INFO:teuthology.orchestra.run.smithi042.stdout:5789 writesame 0xc118aea thru 0xc121860 data_size 0x1(0x8d77 bytes) 2024-09-14T23:58:01.626 INFO:teuthology.orchestra.run.smithi042.stdout:5790 write 0x1933ae8 thru 0x193a23e (0x6757 bytes) 2024-09-14T23:58:01.628 INFO:teuthology.orchestra.run.smithi042.stdout:5792 read 0x59360ac thru 0x594410a (0xe05f bytes) 2024-09-14T23:58:01.629 INFO:teuthology.orchestra.run.smithi042.stdout:5793 trunc from 0xc121861 to 0xb24740d 2024-09-14T23:58:01.635 INFO:teuthology.orchestra.run.smithi042.stdout:5795 read 0x24e7cdf thru 0x24f16bb (0x99dd bytes) 2024-09-14T23:58:01.635 INFO:teuthology.orchestra.run.smithi042.stdout:5797 writesame 0x5882b2 thru 0x592afd data_size 0x2(0xa84c bytes) 2024-09-14T23:58:01.658 INFO:teuthology.orchestra.run.smithi042.stdout:5798 trunc from 0xb24740d to 0x9c903f 2024-09-14T23:58:01.669 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:01.667+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:01.682 INFO:teuthology.orchestra.run.smithi042.stdout:5799 trunc from 0x9c903f to 0xe5d71bb 2024-09-14T23:58:01.688 INFO:teuthology.orchestra.run.smithi042.stdout:5800 write 0x569f658 thru 0x56abfae (0xc957 bytes) 2024-09-14T23:58:01.692 INFO:teuthology.orchestra.run.smithi042.stdout:5802 write 0x3c71b11 thru 0x3c72af8 (0xfe8 bytes) 2024-09-14T23:58:01.694 INFO:teuthology.orchestra.run.smithi042.stdout:5803 writesame 0xeba076 thru 0xebe544 data_size 0xd(0x44cf bytes) 2024-09-14T23:58:01.696 INFO:teuthology.orchestra.run.smithi042.stdout:5805 read 0xd0825bb thru 0xd083a01 (0x1447 bytes) 2024-09-14T23:58:01.697 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:58:01.697 INFO:teuthology.orchestra.run.smithi042.stdout:5807 write 0x2ffa942 thru 0x2ffcbfa (0x22b9 bytes) 2024-09-14T23:58:01.702 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T23:58:01.707 INFO:teuthology.orchestra.run.smithi042.stdout:5808 trunc from 0xe5d71bb to 0x7674e1b 2024-09-14T23:58:01.713 INFO:teuthology.orchestra.run.smithi042.stdout:5811 read 0x5d296b4 thru 0x5d37f36 (0xe883 bytes) 2024-09-14T23:58:01.713 INFO:teuthology.orchestra.run.smithi042.stdout:5813 punch from 0x379aec1 to 0x37aa98e, (0xfacd bytes) 2024-09-14T23:58:01.718 INFO:teuthology.orchestra.run.smithi042.stdout:5814 write 0x7cc3976 thru 0x7cc96e9 (0x5d74 bytes) 2024-09-14T23:58:01.722 INFO:teuthology.orchestra.run.smithi042.stdout:5816 read 0x286272 thru 0x288b2e (0x28bd bytes) 2024-09-14T23:58:01.726 INFO:teuthology.orchestra.run.smithi042.stdout:5818 trunc from 0x7cc96ea to 0x4d050a6 2024-09-14T23:58:01.736 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:01.737 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:01.737 INFO:teuthology.orchestra.run.smithi042.stdout:5819 punch from 0x126a89b to 0x126dcaa, (0x340f bytes) 2024-09-14T23:58:01.737 INFO:teuthology.orchestra.run.smithi042.stdout:5820 read 0x3b9e047 thru 0x3ba17d3 (0x378d bytes) 2024-09-14T23:58:01.738 INFO:teuthology.orchestra.run.smithi042.stdout:5821 punch from 0x1a24552 to 0x1a2ed90, (0xa83e bytes) 2024-09-14T23:58:01.738 INFO:teuthology.orchestra.run.smithi042.stdout:5822 read 0x1629ed1 thru 0x162d42e (0x355e bytes) 2024-09-14T23:58:01.738 INFO:teuthology.orchestra.run.smithi042.stdout:5824 read 0xf470f0 thru 0xf505d5 (0x94e6 bytes) 2024-09-14T23:58:01.741 INFO:teuthology.orchestra.run.smithi042.stdout:5825 trunc from 0x4d050a6 to 0x205aa27 2024-09-14T23:58:01.745 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T23:58:01.762 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:01.760+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:01.765 INFO:teuthology.orchestra.run.smithi042.stdout:5826 writesame 0x8e3411b thru 0x8e3b31b data_size 0xd(0x7201 bytes) 2024-09-14T23:58:01.775 INFO:teuthology.orchestra.run.smithi042.stdout:5827 write 0xcab615d thru 0xcac22fa (0xc19e bytes) 2024-09-14T23:58:01.778 INFO:teuthology.orchestra.run.smithi042.stdout:5828 punch from 0x69eff5a to 0x69fe724, (0xe7ca bytes) 2024-09-14T23:58:01.778 INFO:teuthology.orchestra.run.smithi042.stdout:5829 write 0x72bd8e3 thru 0x72be35c (0xa7a bytes) 2024-09-14T23:58:01.792 INFO:teuthology.orchestra.run.smithi042.stdout:5830 trunc from 0xcac22fb to 0x5c29484 2024-09-14T23:58:01.799 INFO:teuthology.orchestra.run.smithi042.stdout:5832 read 0x377cc8a thru 0x377f1a9 (0x2520 bytes) 2024-09-14T23:58:01.799 INFO:teuthology.orchestra.run.smithi042.stdout:5833 read 0x83e21c thru 0x84615b (0x7f40 bytes) 2024-09-14T23:58:01.800 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:58:01.800 INFO:teuthology.orchestra.run.smithi042.stdout:5835 read 0x3d9ef7e thru 0x3da2c9e (0x3d21 bytes) 2024-09-14T23:58:01.800 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:58:01.811 INFO:teuthology.orchestra.run.smithi042.stdout:5837 writesame 0xac2b956 thru 0xac39039 data_size 0x22(0xd6e4 bytes) 2024-09-14T23:58:01.811 INFO:teuthology.orchestra.run.smithi042.stdout:5838 writesame 0x21c5a90 thru 0x21d0b7e data_size 0x1(0xb0ef bytes) 2024-09-14T23:58:01.834 INFO:teuthology.orchestra.run.smithi042.stdout:5839 write 0x44de98b thru 0x44e9a90 (0xb106 bytes) 2024-09-14T23:58:01.838 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:58:01.838 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:58:01.848 INFO:teuthology.orchestra.run.smithi042.stdout:5840 write 0xeb72547 thru 0xeb73894 (0x134e bytes) 2024-09-14T23:58:01.852 INFO:teuthology.orchestra.run.smithi042.stdout:5841 read 0x8ed664d thru 0x8ee40cd (0xda81 bytes) 2024-09-14T23:58:01.852 INFO:teuthology.orchestra.run.smithi042.stdout:5842 read 0x7608008 thru 0x7616d0d (0xed06 bytes) 2024-09-14T23:58:01.852 INFO:teuthology.orchestra.run.smithi042.stdout:5843 writesame 0x86b4fea thru 0x86bdd9f data_size 0x22(0x8db6 bytes) 2024-09-14T23:58:01.855 INFO:teuthology.orchestra.run.smithi042.stdout:5844 write 0x1a8587f thru 0x1a8fefb (0xa67d bytes) 2024-09-14T23:58:01.857 INFO:teuthology.orchestra.run.smithi042.stdout:5845 read 0xb677e9a thru 0xb67cab0 (0x4c17 bytes) 2024-09-14T23:58:01.857 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:58:01.857 INFO:teuthology.orchestra.run.smithi042.stdout:5848 read 0xd60716d thru 0xd60d022 (0x5eb6 bytes) 2024-09-14T23:58:01.857 INFO:teuthology.orchestra.run.smithi042.stdout:5849 writesame 0x571f864 thru 0x57202f2 data_size 0x11(0xa8f bytes) 2024-09-14T23:58:01.858 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:58:01.858 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:58:01.861 INFO:teuthology.orchestra.run.smithi042.stdout:5850 write 0xde5188c thru 0xde604b9 (0xec2e bytes) 2024-09-14T23:58:01.863 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:01.860+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:01.865 INFO:teuthology.orchestra.run.smithi042.stdout:5851 read 0xe8c1513 thru 0xe8d01df (0xeccd bytes) 2024-09-14T23:58:01.865 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:58:01.865 INFO:teuthology.orchestra.run.smithi042.stdout:5853 punch from 0xa7251ab to 0xa72b88d, (0x66e2 bytes) 2024-09-14T23:58:01.865 INFO:teuthology.orchestra.run.smithi042.stdout:5854 write 0x3dc9a7a thru 0x3dd270b (0x8c92 bytes) 2024-09-14T23:58:01.866 INFO:teuthology.orchestra.run.smithi042.stdout:5855 writesame 0x8b2cfa4 thru 0x8b39c21 data_size 0xf(0xcc7e bytes) 2024-09-14T23:58:01.871 INFO:teuthology.orchestra.run.smithi042.stdout:5856 punch from 0x11964cf to 0x11a2b44, (0xc675 bytes) 2024-09-14T23:58:01.871 INFO:teuthology.orchestra.run.smithi042.stdout:5857 read 0xa3711f2 thru 0xa3730b8 (0x1ec7 bytes) 2024-09-14T23:58:01.871 INFO:teuthology.orchestra.run.smithi042.stdout:5858 write 0x104ffc6 thru 0x105740b (0x7446 bytes) 2024-09-14T23:58:01.872 INFO:teuthology.orchestra.run.smithi042.stdout:5859 read 0x814af99 thru 0x81552b9 (0xa321 bytes) 2024-09-14T23:58:01.872 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:58:01.877 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:58:01.877 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:58:01.888 INFO:teuthology.orchestra.run.smithi042.stdout:5861 trunc from 0xeb73895 to 0x284e8c 2024-09-14T23:58:01.896 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:58:01.896 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:58:01.903 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:01.903 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:01.912 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T23:58:01.915 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:58:01.915 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:58:01.918 INFO:teuthology.orchestra.run.smithi042.stdout:5862 write 0x6e5c6bc thru 0x6e5d445 (0xd8a bytes) 2024-09-14T23:58:01.927 INFO:teuthology.orchestra.run.smithi042.stdout:5863 writesame 0x7dd2649 thru 0x7ddad78 data_size 0x38(0x8730 bytes) 2024-09-14T23:58:01.933 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:58:01.933 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:58:01.938 INFO:teuthology.orchestra.run.smithi042.stdout:5864 writesame 0xd900893 thru 0xd909913 data_size 0x3b(0x9081 bytes) 2024-09-14T23:58:01.939 INFO:teuthology.orchestra.run.smithi042.stdout:5865 write 0x17c6d17 thru 0x17cf657 (0x8941 bytes) 2024-09-14T23:58:01.940 INFO:teuthology.orchestra.run.smithi042.stdout:5866 writesame 0x84e6a7a thru 0x84ef733 data_size 0x1(0x8cba bytes) 2024-09-14T23:58:01.950 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:58:01.950 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:58:01.959 INFO:teuthology.orchestra.run.smithi042.stdout:5867 writesame 0x8fcf774 thru 0x8fdbf68 data_size 0x3(0xc7f5 bytes) 2024-09-14T23:58:01.963 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:01.962+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:01.967 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:58:01.968 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:58:01.970 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:58:01.977 INFO:teuthology.orchestra.run.smithi042.stdout:5869 write 0xd9f11d2 thru 0xd9fa6f3 (0x9522 bytes) 2024-09-14T23:58:01.980 INFO:teuthology.orchestra.run.smithi042.stdout:5873 write 0x313b992 thru 0x313cad8 (0x1147 bytes) 2024-09-14T23:58:01.983 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:58:01.983 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:58:01.983 INFO:teuthology.orchestra.run.smithi042.stdout:5879 write 0x8a057af thru 0x8a0ad35 (0x5587 bytes) 2024-09-14T23:58:01.991 INFO:teuthology.orchestra.run.smithi042.stdout:5881 writesame 0xdcb70de thru 0xdcbf6cb data_size 0xe(0x85ee bytes) 2024-09-14T23:58:01.991 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:58:02.004 INFO:teuthology.orchestra.run.smithi042.stdout:5883 trunc from 0xdcbf6cc to 0x1d8cc53 2024-09-14T23:58:02.027 INFO:teuthology.orchestra.run.smithi042.stdout:5884 writesame 0x42bf5fb thru 0x42c5cec data_size 0x1(0x66f2 bytes) 2024-09-14T23:58:02.043 INFO:teuthology.orchestra.run.smithi042.stdout:5885 write 0xc925e5b thru 0xc92c025 (0x61cb bytes) 2024-09-14T23:58:02.047 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:58:02.047 INFO:teuthology.orchestra.run.smithi042.stdout:5888 write 0xaa9dd0d thru 0xaaa0fda (0x32ce bytes) 2024-09-14T23:58:02.051 INFO:teuthology.orchestra.run.smithi042.stdout:5889 read 0x5d54f53 thru 0x5d56d7f (0x1e2d bytes) 2024-09-14T23:58:02.051 INFO:teuthology.orchestra.run.smithi042.stdout:5890 write 0x61b0765 thru 0x61b3f49 (0x37e5 bytes) 2024-09-14T23:58:02.053 INFO:teuthology.orchestra.run.smithi042.stdout:5891 read 0x82e22ec thru 0x82e7f43 (0x5c58 bytes) 2024-09-14T23:58:02.060 INFO:teuthology.orchestra.run.smithi042.stdout:5892 writesame 0xe5693f0 thru 0xe5706c7 data_size 0x69(0x72d8 bytes) 2024-09-14T23:58:02.061 INFO:teuthology.orchestra.run.smithi042.stdout:5893 read 0x2e6c355 thru 0x2e75a3e (0x96ea bytes) 2024-09-14T23:58:02.061 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:58:02.061 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:58:02.064 INFO:teuthology.orchestra.run.smithi042.stdout:5896 trunc from 0xe5706c8 to 0xb4558f7 2024-09-14T23:58:02.064 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:02.062+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:02.070 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:02.070 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:58:02.070 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:58:02.070 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:58:02.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.070 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:107472 3.1e 3:7bf21588:::rbd_data.11187de8b0a5.000000000000005d:head [set-alloc-hint object_size 2097152 write_size 2097152,write 317788~1491] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-14T23:58:02.070 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:25.198493+0000", 2024-09-14T23:58:02.070 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.868863802999996, 2024-09-14T23:58:02.070 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.78175149300000002, 2024-09-14T23:58:02.070 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:02.071 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:02.071 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:02.071 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:02.071 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:02.071 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:02.071 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 107472 2024-09-14T23:58:02.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.071 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:02.071 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.071 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:02.071 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198493+0000", 2024-09-14T23:58:02.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.071 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.072 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:02.072 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198493+0000", 2024-09-14T23:58:02.072 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.072 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.072 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:02.072 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198495+0000", 2024-09-14T23:58:02.072 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.826e-06 2024-09-14T23:58:02.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.072 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.072 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:02.072 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198503+0000", 2024-09-14T23:58:02.072 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0150000000000003e-06 2024-09-14T23:58:02.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.073 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.073 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:02.073 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198521+0000", 2024-09-14T23:58:02.073 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8374000000000001e-05 2024-09-14T23:58:02.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.073 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.073 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:02.073 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198533+0000", 2024-09-14T23:58:02.073 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1593000000000001e-05 2024-09-14T23:58:02.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.073 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.073 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:02.073 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198603+0000", 2024-09-14T23:58:02.073 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0279999999999998e-05 2024-09-14T23:58:02.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.074 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.074 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:02.074 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198617+0000", 2024-09-14T23:58:02.074 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3517e-05 2024-09-14T23:58:02.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.074 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.074 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:02.074 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.977067+0000", 2024-09-14T23:58:02.074 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.77845065800000002 2024-09-14T23:58:02.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.074 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.074 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for missing object", 2024-09-14T23:58:02.074 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.977173+0000", 2024-09-14T23:58:02.075 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010543999999999999 2024-09-14T23:58:02.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.075 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.075 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:02.075 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.978737+0000", 2024-09-14T23:58:02.075 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001564078 2024-09-14T23:58:02.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.075 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.075 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:02.075 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.978845+0000", 2024-09-14T23:58:02.076 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000108643 2024-09-14T23:58:02.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.076 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.076 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1", 2024-09-14T23:58:02.076 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.978964+0000", 2024-09-14T23:58:02.076 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000119085 2024-09-14T23:58:02.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.076 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.076 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:02.076 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980109+0000", 2024-09-14T23:58:02.076 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0011442170000000001 2024-09-14T23:58:02.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.076 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.077 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:58:02.077 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980130+0000", 2024-09-14T23:58:02.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1766000000000001e-05 2024-09-14T23:58:02.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.077 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:02.077 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980171+0000", 2024-09-14T23:58:02.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0627999999999998e-05 2024-09-14T23:58:02.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.077 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:02.078 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980244+0000", 2024-09-14T23:58:02.078 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3373000000000004e-05 2024-09-14T23:58:02.078 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.078 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:02.078 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.078 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.078 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129149 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 6619136~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:02.078 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.925871+0000", 2024-09-14T23:58:02.078 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.141485361000001, 2024-09-14T23:58:02.078 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054291066999999998, 2024-09-14T23:58:02.079 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:02.079 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:02.079 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:02.079 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:02.079 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:02.079 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:02.079 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129149 2024-09-14T23:58:02.079 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.079 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:02.080 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.080 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:02.080 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.925871+0000", 2024-09-14T23:58:02.080 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.080 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.080 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.080 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:02.080 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.925871+0000", 2024-09-14T23:58:02.080 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.080 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.080 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.080 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:02.080 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.927669+0000", 2024-09-14T23:58:02.081 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001797838 2024-09-14T23:58:02.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.081 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.081 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:02.081 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.927672+0000", 2024-09-14T23:58:02.081 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5280000000000002e-06 2024-09-14T23:58:02.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.081 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.081 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:02.081 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.927678+0000", 2024-09-14T23:58:02.081 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0340000000000002e-06 2024-09-14T23:58:02.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.081 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.081 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:02.082 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.927681+0000", 2024-09-14T23:58:02.082 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.681e-06 2024-09-14T23:58:02.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.082 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.082 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:02.082 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.979979+0000", 2024-09-14T23:58:02.082 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052297950000000003 2024-09-14T23:58:02.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.082 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.082 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:02.082 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.980024+0000", 2024-09-14T23:58:02.082 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4693000000000001e-05 2024-09-14T23:58:02.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.082 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.082 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:02.083 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.980162+0000", 2024-09-14T23:58:02.083 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000138343 2024-09-14T23:58:02.083 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.083 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:02.083 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.083 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.083 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129159 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 5996544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:02.083 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.927783+0000", 2024-09-14T23:58:02.083 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.139573911999999, 2024-09-14T23:58:02.084 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054661122999999999, 2024-09-14T23:58:02.084 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:02.084 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:02.084 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:02.084 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:02.084 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:02.084 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:02.084 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129159 2024-09-14T23:58:02.084 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.084 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:02.084 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.084 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:02.084 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.927783+0000", 2024-09-14T23:58:02.085 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.085 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:02.085 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.927783+0000", 2024-09-14T23:58:02.085 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.085 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:02.085 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929684+0000", 2024-09-14T23:58:02.085 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0019011080000000001 2024-09-14T23:58:02.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.085 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:02.086 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929688+0000", 2024-09-14T23:58:02.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2610000000000004e-06 2024-09-14T23:58:02.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.086 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.086 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:02.086 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929697+0000", 2024-09-14T23:58:02.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0219999999999993e-06 2024-09-14T23:58:02.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.086 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.086 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:02.086 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929702+0000", 2024-09-14T23:58:02.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8840000000000002e-06 2024-09-14T23:58:02.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.087 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:02.087 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.982090+0000", 2024-09-14T23:58:02.087 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052388045000000001 2024-09-14T23:58:02.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.087 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:02.087 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.982208+0000", 2024-09-14T23:58:02.087 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118236 2024-09-14T23:58:02.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.088 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.088 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:02.088 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.982444+0000", 2024-09-14T23:58:02.088 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000235567 2024-09-14T23:58:02.088 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.088 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:02.088 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.088 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.088 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129166 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 8192000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:02.088 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.929794+0000", 2024-09-14T23:58:02.088 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.137563160999999, 2024-09-14T23:58:02.088 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054471973, 2024-09-14T23:58:02.088 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:02.089 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:02.089 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:02.089 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:02.089 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:02.089 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:02.089 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129166 2024-09-14T23:58:02.089 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.089 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:02.089 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.089 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:02.089 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929794+0000", 2024-09-14T23:58:02.090 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.090 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.090 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:02.090 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929794+0000", 2024-09-14T23:58:02.090 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.090 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.090 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:02.090 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929794+0000", 2024-09-14T23:58:02.090 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6899999999999997e-07 2024-09-14T23:58:02.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.090 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.090 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:02.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929797+0000", 2024-09-14T23:58:02.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.328e-06 2024-09-14T23:58:02.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:02.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929802+0000", 2024-09-14T23:58:02.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7309999999999998e-06 2024-09-14T23:58:02.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:02.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929807+0000", 2024-09-14T23:58:02.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4349999999999999e-06 2024-09-14T23:58:02.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.092 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:02.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984063+0000", 2024-09-14T23:58:02.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054256331999999997 2024-09-14T23:58:02.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.092 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.092 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:02.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984177+0000", 2024-09-14T23:58:02.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114225 2024-09-14T23:58:02.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.092 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.092 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:02.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984266+0000", 2024-09-14T23:58:02.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8352999999999999e-05 2024-09-14T23:58:02.092 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.093 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:02.093 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.093 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.093 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.093 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129167 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 6782976~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:02.093 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.929827+0000", 2024-09-14T23:58:02.093 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.137529751999999, 2024-09-14T23:58:02.093 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054637593999999998, 2024-09-14T23:58:02.093 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:02.093 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:02.093 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:02.093 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:02.093 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:02.093 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:02.094 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129167 2024-09-14T23:58:02.094 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.094 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:02.094 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.094 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:02.094 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929827+0000", 2024-09-14T23:58:02.094 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.094 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.094 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.094 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:02.094 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929827+0000", 2024-09-14T23:58:02.094 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.094 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.094 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.094 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:02.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929827+0000", 2024-09-14T23:58:02.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4000000000000002e-07 2024-09-14T23:58:02.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:02.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929830+0000", 2024-09-14T23:58:02.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.419e-06 2024-09-14T23:58:02.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:02.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929837+0000", 2024-09-14T23:58:02.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0450000000000003e-06 2024-09-14T23:58:02.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:02.096 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929842+0000", 2024-09-14T23:58:02.096 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.69e-06 2024-09-14T23:58:02.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.096 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:02.096 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984278+0000", 2024-09-14T23:58:02.096 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054436516999999997 2024-09-14T23:58:02.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.096 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:02.096 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984331+0000", 2024-09-14T23:58:02.096 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2590999999999998e-05 2024-09-14T23:58:02.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.097 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:02.097 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984465+0000", 2024-09-14T23:58:02.097 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013389200000000001 2024-09-14T23:58:02.097 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.097 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:02.097 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.097 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.097 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129168 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 8224768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:02.097 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.929849+0000", 2024-09-14T23:58:02.097 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.137508193999999, 2024-09-14T23:58:02.097 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054870368000000003, 2024-09-14T23:58:02.097 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:02.097 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:02.098 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:02.098 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:02.098 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:02.098 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:02.098 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129168 2024-09-14T23:58:02.098 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.098 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:02.098 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.098 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:02.098 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929849+0000", 2024-09-14T23:58:02.098 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.098 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.098 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.098 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:02.098 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929849+0000", 2024-09-14T23:58:02.098 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.099 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:02.099 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931681+0000", 2024-09-14T23:58:02.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018326989999999999 2024-09-14T23:58:02.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.099 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:02.099 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931686+0000", 2024-09-14T23:58:02.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7190000000000001e-06 2024-09-14T23:58:02.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.099 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:02.099 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931697+0000", 2024-09-14T23:58:02.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1287999999999999e-05 2024-09-14T23:58:02.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.100 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.100 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:02.100 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931704+0000", 2024-09-14T23:58:02.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3339999999999998e-06 2024-09-14T23:58:02.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.100 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.100 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:02.100 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984505+0000", 2024-09-14T23:58:02.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052801684000000002 2024-09-14T23:58:02.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.100 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.100 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:02.101 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984621+0000", 2024-09-14T23:58:02.101 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000116165 2024-09-14T23:58:02.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.101 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:02.101 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984719+0000", 2024-09-14T23:58:02.101 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7479000000000006e-05 2024-09-14T23:58:02.101 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.101 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:02.101 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.101 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129175 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 9011200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:02.101 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.931806+0000", 2024-09-14T23:58:02.101 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.135551175, 2024-09-14T23:58:02.102 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054322384000000001, 2024-09-14T23:58:02.102 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:02.102 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:02.102 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:02.102 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:02.102 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:02.102 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:02.102 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129175 2024-09-14T23:58:02.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.102 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:02.102 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.102 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:02.102 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931806+0000", 2024-09-14T23:58:02.102 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.102 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.103 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:02.103 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931806+0000", 2024-09-14T23:58:02.103 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.103 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.103 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:02.103 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931806+0000", 2024-09-14T23:58:02.103 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2300000000000001e-07 2024-09-14T23:58:02.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.103 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.103 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:02.103 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931808+0000", 2024-09-14T23:58:02.103 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1440000000000001e-06 2024-09-14T23:58:02.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.103 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.104 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:02.104 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931815+0000", 2024-09-14T23:58:02.104 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.776e-06 2024-09-14T23:58:02.104 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.104 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.104 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:02.104 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931821+0000", 2024-09-14T23:58:02.104 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.733e-06 2024-09-14T23:58:02.104 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.104 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.104 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:02.104 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.986046+0000", 2024-09-14T23:58:02.104 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054225284999999998 2024-09-14T23:58:02.104 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.105 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:02.105 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.986089+0000", 2024-09-14T23:58:02.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2895999999999997e-05 2024-09-14T23:58:02.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.105 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:02.105 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.986128+0000", 2024-09-14T23:58:02.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9026999999999998e-05 2024-09-14T23:58:02.105 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.105 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:02.105 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.106 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129176 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 10452992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:02.106 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.931830+0000", 2024-09-14T23:58:02.106 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.135526432999999, 2024-09-14T23:58:02.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054513292999999997, 2024-09-14T23:58:02.106 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:02.106 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:02.106 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:02.106 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:02.106 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:02.106 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:02.106 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129176 2024-09-14T23:58:02.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.106 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:02.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.106 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:02.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931830+0000", 2024-09-14T23:58:02.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.107 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:02.107 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931830+0000", 2024-09-14T23:58:02.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.107 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:02.107 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.933678+0000", 2024-09-14T23:58:02.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018472440000000001 2024-09-14T23:58:02.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.107 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:02.107 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.933686+0000", 2024-09-14T23:58:02.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0239999999999997e-06 2024-09-14T23:58:02.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:02.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.933694+0000", 2024-09-14T23:58:02.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8400000000000001e-06 2024-09-14T23:58:02.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:02.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.933701+0000", 2024-09-14T23:58:02.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2870000000000001e-06 2024-09-14T23:58:02.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:02.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.986165+0000", 2024-09-14T23:58:02.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052463900000000001 2024-09-14T23:58:02.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:02.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.986259+0000", 2024-09-14T23:58:02.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4435e-05 2024-09-14T23:58:02.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:02.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.986344+0000", 2024-09-14T23:58:02.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4562999999999997e-05 2024-09-14T23:58:02.109 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.109 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:02.109 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.110 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.110 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129201 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 10289152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:02.110 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.935960+0000", 2024-09-14T23:58:02.110 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.131396971000001, 2024-09-14T23:58:02.110 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054338487999999997, 2024-09-14T23:58:02.110 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:02.110 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:02.110 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:02.110 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:02.110 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:02.110 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:02.110 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129201 2024-09-14T23:58:02.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.110 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:02.110 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.111 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:02.111 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935960+0000", 2024-09-14T23:58:02.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.111 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.111 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:02.111 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935960+0000", 2024-09-14T23:58:02.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.111 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.111 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:02.111 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935960+0000", 2024-09-14T23:58:02.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.72e-07 2024-09-14T23:58:02.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.111 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:02.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935963+0000", 2024-09-14T23:58:02.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5390000000000002e-06 2024-09-14T23:58:02.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:02.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935971+0000", 2024-09-14T23:58:02.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4589999999999998e-06 2024-09-14T23:58:02.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:02.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935977+0000", 2024-09-14T23:58:02.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4110000000000004e-06 2024-09-14T23:58:02.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.113 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:02.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990213+0000", 2024-09-14T23:58:02.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054236642000000002 2024-09-14T23:58:02.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.113 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:02.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990257+0000", 2024-09-14T23:58:02.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3696000000000003e-05 2024-09-14T23:58:02.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.113 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:02.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990298+0000", 2024-09-14T23:58:02.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1369000000000003e-05 2024-09-14T23:58:02.114 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.114 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:02.114 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.114 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129202 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 11370496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:02.114 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.935986+0000", 2024-09-14T23:58:02.114 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.131370353000001, 2024-09-14T23:58:02.114 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054543700000000001, 2024-09-14T23:58:02.114 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:02.114 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:02.114 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:02.114 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:02.114 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:02.114 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:02.115 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129202 2024-09-14T23:58:02.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.115 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:02.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.115 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:02.115 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935986+0000", 2024-09-14T23:58:02.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.115 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:02.115 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935986+0000", 2024-09-14T23:58:02.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.115 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:02.116 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937678+0000", 2024-09-14T23:58:02.116 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016917849999999999 2024-09-14T23:58:02.116 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.116 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.116 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:02.116 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937682+0000", 2024-09-14T23:58:02.116 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2400000000000001e-06 2024-09-14T23:58:02.116 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.116 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.116 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:02.116 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937695+0000", 2024-09-14T23:58:02.116 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2999e-05 2024-09-14T23:58:02.116 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.116 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.116 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:02.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937700+0000", 2024-09-14T23:58:02.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8420000000000004e-06 2024-09-14T23:58:02.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:02.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990336+0000", 2024-09-14T23:58:02.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052636060999999998 2024-09-14T23:58:02.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:02.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990435+0000", 2024-09-14T23:58:02.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9015e-05 2024-09-14T23:58:02.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:02.118 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990530+0000", 2024-09-14T23:58:02.118 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4758000000000003e-05 2024-09-14T23:58:02.118 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.118 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:02.118 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.118 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.118 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129214 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 11468800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:02.118 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.937939+0000", 2024-09-14T23:58:02.118 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.129417439000001, 2024-09-14T23:58:02.118 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054463733, 2024-09-14T23:58:02.118 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:02.118 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:02.118 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:02.119 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:02.119 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:02.119 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:02.119 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129214 2024-09-14T23:58:02.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.119 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:02.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.119 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:02.119 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937939+0000", 2024-09-14T23:58:02.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.119 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:02.119 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937939+0000", 2024-09-14T23:58:02.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.120 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.120 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:02.120 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939674+0000", 2024-09-14T23:58:02.120 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017342589999999999 2024-09-14T23:58:02.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.120 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.120 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:02.120 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939678+0000", 2024-09-14T23:58:02.120 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2590000000000002e-06 2024-09-14T23:58:02.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.120 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.120 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:02.120 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939699+0000", 2024-09-14T23:58:02.120 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1143000000000001e-05 2024-09-14T23:58:02.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.121 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.121 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:02.121 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939706+0000", 2024-09-14T23:58:02.121 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0269999999999998e-06 2024-09-14T23:58:02.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.121 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.121 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:02.121 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.992221+0000", 2024-09-14T23:58:02.121 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052514981000000002 2024-09-14T23:58:02.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.121 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.121 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:02.121 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.992321+0000", 2024-09-14T23:58:02.122 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9625999999999998e-05 2024-09-14T23:58:02.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.122 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.122 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:02.122 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.992403+0000", 2024-09-14T23:58:02.122 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2437999999999999e-05 2024-09-14T23:58:02.122 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.122 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:02.122 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.122 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.122 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129225 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 12648448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:02.122 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.939856+0000", 2024-09-14T23:58:02.122 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.127500883, 2024-09-14T23:58:02.122 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054305557999999997, 2024-09-14T23:58:02.123 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:02.123 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:02.123 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:02.123 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:02.123 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:02.123 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:02.123 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129225 2024-09-14T23:58:02.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.123 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:02.123 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.123 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:02.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939856+0000", 2024-09-14T23:58:02.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.123 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.123 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:02.124 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939856+0000", 2024-09-14T23:58:02.124 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.124 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:02.124 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939856+0000", 2024-09-14T23:58:02.124 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1199999999999999e-07 2024-09-14T23:58:02.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.124 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:02.124 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939857+0000", 2024-09-14T23:58:02.124 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.054e-06 2024-09-14T23:58:02.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.124 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:02.124 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939861+0000", 2024-09-14T23:58:02.125 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1459999999999998e-06 2024-09-14T23:58:02.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.125 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.125 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:02.125 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939864+0000", 2024-09-14T23:58:02.125 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7539999999999998e-06 2024-09-14T23:58:02.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.125 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.125 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:02.125 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.994078+0000", 2024-09-14T23:58:02.125 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054213483 2024-09-14T23:58:02.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.125 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.125 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:02.126 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.994119+0000", 2024-09-14T23:58:02.126 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1835000000000002e-05 2024-09-14T23:58:02.126 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.126 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.126 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:02.126 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.994161+0000", 2024-09-14T23:58:02.126 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2073999999999997e-05 2024-09-14T23:58:02.126 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.126 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:02.126 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.126 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.126 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.126 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129226 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 14090240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:02.126 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.939867+0000", 2024-09-14T23:58:02.126 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.127489511, 2024-09-14T23:58:02.127 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054523573999999998, 2024-09-14T23:58:02.127 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:02.127 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:02.127 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:02.127 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:02.127 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:02.127 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:02.127 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129226 2024-09-14T23:58:02.127 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.127 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:02.128 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.128 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:02.128 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939867+0000", 2024-09-14T23:58:02.128 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.128 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.128 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:02.128 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939867+0000", 2024-09-14T23:58:02.128 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.128 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.128 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:02.128 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941669+0000", 2024-09-14T23:58:02.128 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018021549999999999 2024-09-14T23:58:02.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.129 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.129 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:02.129 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941673+0000", 2024-09-14T23:58:02.129 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1269999999999999e-06 2024-09-14T23:58:02.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.129 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.129 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:02.129 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941679+0000", 2024-09-14T23:58:02.129 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1879999999999999e-06 2024-09-14T23:58:02.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.129 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.129 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:02.130 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941682+0000", 2024-09-14T23:58:02.130 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7110000000000001e-06 2024-09-14T23:58:02.130 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.130 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.130 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:02.130 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.994201+0000", 2024-09-14T23:58:02.130 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052518265000000001 2024-09-14T23:58:02.130 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.130 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.130 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:02.130 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.994302+0000", 2024-09-14T23:58:02.130 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101653 2024-09-14T23:58:02.130 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.130 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.131 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:02.131 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.994391+0000", 2024-09-14T23:58:02.131 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8474999999999996e-05 2024-09-14T23:58:02.131 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.131 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:02.131 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.131 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.131 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129237 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 12386304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:02.131 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.941825+0000", 2024-09-14T23:58:02.131 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.125531749, 2024-09-14T23:58:02.131 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054368076000000001, 2024-09-14T23:58:02.131 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:02.131 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:02.132 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:02.132 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:02.132 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:02.132 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:02.132 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129237 2024-09-14T23:58:02.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.132 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:02.132 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.132 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:02.132 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941825+0000", 2024-09-14T23:58:02.132 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.132 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.132 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:02.132 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941825+0000", 2024-09-14T23:58:02.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.133 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:02.133 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941825+0000", 2024-09-14T23:58:02.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.29e-07 2024-09-14T23:58:02.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.133 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:02.133 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941826+0000", 2024-09-14T23:58:02.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0999999999999997e-07 2024-09-14T23:58:02.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.133 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:02.134 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941834+0000", 2024-09-14T23:58:02.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5519999999999999e-06 2024-09-14T23:58:02.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:02.134 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941843+0000", 2024-09-14T23:58:02.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2960000000000004e-06 2024-09-14T23:58:02.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:02.134 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996108+0000", 2024-09-14T23:58:02.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054265200999999999 2024-09-14T23:58:02.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:02.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996152+0000", 2024-09-14T23:58:02.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4066999999999997e-05 2024-09-14T23:58:02.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.135 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:02.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996193+0000", 2024-09-14T23:58:02.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0821000000000001e-05 2024-09-14T23:58:02.135 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.135 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:02.135 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.135 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129238 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 13828096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:02.135 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.941851+0000", 2024-09-14T23:58:02.136 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.125505425, 2024-09-14T23:58:02.136 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.05458031, 2024-09-14T23:58:02.136 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:02.136 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:02.136 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:02.136 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:02.136 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:02.136 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:02.136 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129238 2024-09-14T23:58:02.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.136 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:02.136 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.136 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:02.136 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941851+0000", 2024-09-14T23:58:02.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:02.137 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941851+0000", 2024-09-14T23:58:02.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:02.137 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943670+0000", 2024-09-14T23:58:02.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018181930000000001 2024-09-14T23:58:02.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:02.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943673+0000", 2024-09-14T23:58:02.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3790000000000002e-06 2024-09-14T23:58:02.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:02.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943679+0000", 2024-09-14T23:58:02.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8719999999999999e-06 2024-09-14T23:58:02.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:02.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943683+0000", 2024-09-14T23:58:02.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8539999999999999e-06 2024-09-14T23:58:02.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.139 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:02.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996233+0000", 2024-09-14T23:58:02.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052550690999999997 2024-09-14T23:58:02.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.139 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:02.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996339+0000", 2024-09-14T23:58:02.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105989 2024-09-14T23:58:02.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.139 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:02.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996432+0000", 2024-09-14T23:58:02.140 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2331999999999996e-05 2024-09-14T23:58:02.140 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.140 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:02.140 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.140 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.140 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.140 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129249 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 12124160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:02.140 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.943826+0000", 2024-09-14T23:58:02.140 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.123530934000001, 2024-09-14T23:58:02.140 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054373266000000003, 2024-09-14T23:58:02.140 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:02.140 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:02.140 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:02.140 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:02.141 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:02.141 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:02.141 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129249 2024-09-14T23:58:02.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.141 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:02.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.141 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:02.141 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943826+0000", 2024-09-14T23:58:02.141 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.141 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:02.141 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943826+0000", 2024-09-14T23:58:02.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.142 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:02.142 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943826+0000", 2024-09-14T23:58:02.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.03e-07 2024-09-14T23:58:02.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.142 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:02.142 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943827+0000", 2024-09-14T23:58:02.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.08e-06 2024-09-14T23:58:02.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.142 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:02.143 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943834+0000", 2024-09-14T23:58:02.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4119999999999998e-06 2024-09-14T23:58:02.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:02.143 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943836+0000", 2024-09-14T23:58:02.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7369999999999998e-06 2024-09-14T23:58:02.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:02.143 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998115+0000", 2024-09-14T23:58:02.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054279040000000001 2024-09-14T23:58:02.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.144 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:02.144 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998159+0000", 2024-09-14T23:58:02.144 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3486999999999999e-05 2024-09-14T23:58:02.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.144 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.144 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:02.144 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998199+0000", 2024-09-14T23:58:02.144 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0306999999999999e-05 2024-09-14T23:58:02.144 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.144 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:02.144 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.144 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.144 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129250 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 13565952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:02.145 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.943840+0000", 2024-09-14T23:58:02.145 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.123516983999998, 2024-09-14T23:58:02.145 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054573875000000001, 2024-09-14T23:58:02.145 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:02.145 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:02.145 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:02.145 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:02.145 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:02.145 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:02.145 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129250 2024-09-14T23:58:02.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.145 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:02.145 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.145 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:02.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943840+0000", 2024-09-14T23:58:02.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.146 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:02.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943840+0000", 2024-09-14T23:58:02.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.146 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:02.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945669+0000", 2024-09-14T23:58:02.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018289720000000001 2024-09-14T23:58:02.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.146 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:02.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945671+0000", 2024-09-14T23:58:02.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6869999999999999e-06 2024-09-14T23:58:02.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.147 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:02.147 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945680+0000", 2024-09-14T23:58:02.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5259999999999993e-06 2024-09-14T23:58:02.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.147 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:02.147 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945685+0000", 2024-09-14T23:58:02.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8559999999999998e-06 2024-09-14T23:58:02.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.147 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:02.147 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998242+0000", 2024-09-14T23:58:02.148 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052557505999999997 2024-09-14T23:58:02.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.148 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.148 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:02.148 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998329+0000", 2024-09-14T23:58:02.148 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6410999999999996e-05 2024-09-14T23:58:02.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.148 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.148 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:02.148 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998414+0000", 2024-09-14T23:58:02.148 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4917e-05 2024-09-14T23:58:02.148 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.148 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:02.148 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.149 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129261 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 14385152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:02.149 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.945815+0000", 2024-09-14T23:58:02.149 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.121541691000001, 2024-09-14T23:58:02.149 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054331867999999998, 2024-09-14T23:58:02.149 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:02.149 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:02.149 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:02.149 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:02.149 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:02.149 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:02.149 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129261 2024-09-14T23:58:02.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.149 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:02.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:02.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945815+0000", 2024-09-14T23:58:02.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:02.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945815+0000", 2024-09-14T23:58:02.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:02.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945815+0000", 2024-09-14T23:58:02.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0599999999999999e-07 2024-09-14T23:58:02.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:02.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945816+0000", 2024-09-14T23:58:02.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0380000000000001e-06 2024-09-14T23:58:02.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:02.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945819+0000", 2024-09-14T23:58:02.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.005e-06 2024-09-14T23:58:02.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:02.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945822+0000", 2024-09-14T23:58:02.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.452e-06 2024-09-14T23:58:02.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.152 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:02.152 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000060+0000", 2024-09-14T23:58:02.152 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054238606000000002 2024-09-14T23:58:02.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.152 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:02.152 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000106+0000", 2024-09-14T23:58:02.152 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5188000000000002e-05 2024-09-14T23:58:02.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.152 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:02.152 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000147+0000", 2024-09-14T23:58:02.152 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1372999999999998e-05 2024-09-14T23:58:02.153 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.153 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:02.153 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.153 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.153 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129262 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 13303808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:02.153 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.945825+0000", 2024-09-14T23:58:02.153 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.121532042999998, 2024-09-14T23:58:02.153 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054515942999999997, 2024-09-14T23:58:02.153 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:02.153 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:02.153 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:02.153 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:02.153 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:02.154 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:02.154 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129262 2024-09-14T23:58:02.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.154 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:02.154 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.154 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:02.154 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945825+0000", 2024-09-14T23:58:02.154 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.154 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.154 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:02.154 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945825+0000", 2024-09-14T23:58:02.154 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.154 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.154 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:02.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.947670+0000", 2024-09-14T23:58:02.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018451229999999999 2024-09-14T23:58:02.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:02.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.947672+0000", 2024-09-14T23:58:02.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5289999999999998e-06 2024-09-14T23:58:02.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:02.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.947678+0000", 2024-09-14T23:58:02.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9880000000000001e-06 2024-09-14T23:58:02.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:02.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.947682+0000", 2024-09-14T23:58:02.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.63e-06 2024-09-14T23:58:02.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.156 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:02.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000183+0000", 2024-09-14T23:58:02.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052500770000000002 2024-09-14T23:58:02.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.156 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:02.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000266+0000", 2024-09-14T23:58:02.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3133000000000005e-05 2024-09-14T23:58:02.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.157 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:02.157 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000341+0000", 2024-09-14T23:58:02.157 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4770000000000004e-05 2024-09-14T23:58:02.157 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.157 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:02.157 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.157 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.157 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129299 3.10 3:0f1e06ee:::rbd_object_map.1118721be852:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:02.157 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.952330+0000", 2024-09-14T23:58:02.157 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.115027229999999, 2024-09-14T23:58:02.157 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054412818000000002, 2024-09-14T23:58:02.157 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:02.157 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:02.157 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:02.158 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:02.158 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:02.158 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:02.158 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129299 2024-09-14T23:58:02.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.158 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:02.158 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.158 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:02.158 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952330+0000", 2024-09-14T23:58:02.158 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.158 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.158 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:02.158 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952330+0000", 2024-09-14T23:58:02.158 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.159 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:02.159 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952330+0000", 2024-09-14T23:58:02.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4900000000000001e-07 2024-09-14T23:58:02.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.159 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:02.159 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952332+0000", 2024-09-14T23:58:02.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2879999999999999e-06 2024-09-14T23:58:02.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.159 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:02.159 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952340+0000", 2024-09-14T23:58:02.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9910000000000002e-06 2024-09-14T23:58:02.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.160 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.160 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:02.160 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952344+0000", 2024-09-14T23:58:02.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0280000000000002e-06 2024-09-14T23:58:02.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.160 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.160 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:02.160 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.004722+0000", 2024-09-14T23:58:02.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052377932000000002 2024-09-14T23:58:02.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.160 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.160 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:02.160 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.004778+0000", 2024-09-14T23:58:02.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5795999999999999e-05 2024-09-14T23:58:02.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.161 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.161 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:58:02.161 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.005016+0000", 2024-09-14T23:58:02.161 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000238006 2024-09-14T23:58:02.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.161 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.161 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:58:02.161 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.006366+0000", 2024-09-14T23:58:02.161 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001349786 2024-09-14T23:58:02.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.161 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.161 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:02.161 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.006685+0000", 2024-09-14T23:58:02.161 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00031970000000000002 2024-09-14T23:58:02.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.162 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.162 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:02.162 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.006718+0000", 2024-09-14T23:58:02.162 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2329000000000002e-05 2024-09-14T23:58:02.162 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.162 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.162 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:02.162 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.006742+0000", 2024-09-14T23:58:02.162 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4613e-05 2024-09-14T23:58:02.162 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.162 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:02.162 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.162 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.162 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:02.162 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:02.164 INFO:teuthology.orchestra.run.smithi042.stdout:5898 read 0x978cb18 thru 0x97939db (0x6ec4 bytes) 2024-09-14T23:58:02.164 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:58:02.164 INFO:teuthology.orchestra.run.smithi042.stdout:5903 read 0x8a98b1e thru 0x8aa0bab (0x808e bytes) 2024-09-14T23:58:02.164 INFO:teuthology.orchestra.run.smithi042.stdout:5904 clone 48 order 18 su 1024 sc 4 2024-09-14T23:58:02.164 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T23:58:02.166 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:02.164+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:02.217 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone47 from 0xb4558f7 (overlap 0x284e8c) to 0x2446b1 2024-09-14T23:58:02.267 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:02.265+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:02.320 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:02.320 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:58:02.320 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:58:02.320 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:02.329 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T23:58:02.372 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:02.370+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:02.386 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #46, image image_client.0-clone46 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent47 2024-09-14T23:58:02.452 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:02.452 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:58:02.452 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:58:02.452 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:58:02.452 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:02.453 INFO:teuthology.orchestra.run.smithi042.stdout:5906 write 0x5ece673 thru 0x5ed8c5c (0xa5ea bytes) 2024-09-14T23:58:02.461 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T23:58:02.468 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:02.466+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:02.481 INFO:teuthology.orchestra.run.smithi042.stdout:5907 read 0x46b61d9 thru 0x46c457c (0xe3a4 bytes) 2024-09-14T23:58:02.481 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:58:02.487 INFO:teuthology.orchestra.run.smithi042.stdout:5910 write 0xba58878 thru 0xba5d3fc (0x4b85 bytes) 2024-09-14T23:58:02.493 INFO:teuthology.orchestra.run.smithi042.stdout:5911 trunc from 0xba5d3fd to 0xc25b2e2 2024-09-14T23:58:02.498 INFO:teuthology.orchestra.run.smithi042.stdout:5913 writesame 0xb129c41 thru 0xb12ce63 data_size 0x11(0x3223 bytes) 2024-09-14T23:58:02.527 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:58:02.528 INFO:teuthology.orchestra.run.smithi042.stdout:5917 clone 49 order 24 su 8388608 sc 13 2024-09-14T23:58:02.569 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:02.567+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:02.611 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:02.611 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:58:02.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:58:02.611 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:58:02.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.611 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69262 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 4800512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:02.611 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300811+0000", 2024-09-14T23:58:02.611 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.308018910000001, 2024-09-14T23:58:02.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050403796000000001, 2024-09-14T23:58:02.612 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:02.612 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:02.612 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:02.612 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:02.612 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:02.612 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:02.612 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69262 2024-09-14T23:58:02.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.612 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:02.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.612 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:02.612 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300811+0000", 2024-09-14T23:58:02.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:02.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300811+0000", 2024-09-14T23:58:02.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:02.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300811+0000", 2024-09-14T23:58:02.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5700000000000002e-07 2024-09-14T23:58:02.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:02.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300815+0000", 2024-09-14T23:58:02.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.087e-06 2024-09-14T23:58:02.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.614 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:02.614 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300823+0000", 2024-09-14T23:58:02.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8720000000000002e-06 2024-09-14T23:58:02.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.614 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:02.614 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300830+0000", 2024-09-14T23:58:02.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0589999999999997e-06 2024-09-14T23:58:02.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.614 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:02.615 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351020+0000", 2024-09-14T23:58:02.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050189552999999998 2024-09-14T23:58:02.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.615 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:02.615 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351112+0000", 2024-09-14T23:58:02.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2615999999999996e-05 2024-09-14T23:58:02.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.615 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:02.615 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351214+0000", 2024-09-14T23:58:02.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101952 2024-09-14T23:58:02.615 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.615 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:02.615 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.616 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69263 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 4915200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:02.616 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300842+0000", 2024-09-14T23:58:02.616 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.307987406000002, 2024-09-14T23:58:02.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050590637000000001, 2024-09-14T23:58:02.616 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:02.616 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:02.616 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:02.616 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:02.616 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:02.616 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:02.616 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69263 2024-09-14T23:58:02.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.617 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:02.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:02.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300842+0000", 2024-09-14T23:58:02.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:02.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300842+0000", 2024-09-14T23:58:02.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:02.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300843+0000", 2024-09-14T23:58:02.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4000000000000002e-07 2024-09-14T23:58:02.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:02.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300846+0000", 2024-09-14T23:58:02.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8720000000000002e-06 2024-09-14T23:58:02.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:02.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300853+0000", 2024-09-14T23:58:02.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0450000000000003e-06 2024-09-14T23:58:02.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.619 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:02.619 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300859+0000", 2024-09-14T23:58:02.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9510000000000003e-06 2024-09-14T23:58:02.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.619 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:02.619 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351260+0000", 2024-09-14T23:58:02.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050401654999999997 2024-09-14T23:58:02.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.619 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:02.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351353+0000", 2024-09-14T23:58:02.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2785e-05 2024-09-14T23:58:02.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:02.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351433+0000", 2024-09-14T23:58:02.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9789000000000005e-05 2024-09-14T23:58:02.620 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.620 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:02.620 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.620 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69264 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5029888~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:02.620 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300866+0000", 2024-09-14T23:58:02.621 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.307963473000001, 2024-09-14T23:58:02.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050794063, 2024-09-14T23:58:02.621 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:02.621 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:02.621 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:02.621 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:02.621 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:02.621 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:02.621 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69264 2024-09-14T23:58:02.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.621 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:02.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:02.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300866+0000", 2024-09-14T23:58:02.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:02.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300866+0000", 2024-09-14T23:58:02.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:02.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300866+0000", 2024-09-14T23:58:02.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8799999999999998e-07 2024-09-14T23:58:02.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:02.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300869+0000", 2024-09-14T23:58:02.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0880000000000002e-06 2024-09-14T23:58:02.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:02.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300875+0000", 2024-09-14T23:58:02.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9280000000000004e-06 2024-09-14T23:58:02.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:02.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300882+0000", 2024-09-14T23:58:02.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4740000000000002e-06 2024-09-14T23:58:02.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:02.624 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351472+0000", 2024-09-14T23:58:02.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050590006999999999 2024-09-14T23:58:02.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.624 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:02.624 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351563+0000", 2024-09-14T23:58:02.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0762999999999998e-05 2024-09-14T23:58:02.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.624 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:02.624 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351660+0000", 2024-09-14T23:58:02.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7415000000000002e-05 2024-09-14T23:58:02.624 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.624 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:02.625 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.625 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69265 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5144576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:02.625 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300890+0000", 2024-09-14T23:58:02.625 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.307939969000003, 2024-09-14T23:58:02.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050921389999999997, 2024-09-14T23:58:02.625 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:02.625 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:02.625 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:02.625 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:02.625 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:02.625 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:02.626 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69265 2024-09-14T23:58:02.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.626 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:02.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:02.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300890+0000", 2024-09-14T23:58:02.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:02.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300890+0000", 2024-09-14T23:58:02.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:02.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300890+0000", 2024-09-14T23:58:02.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2500000000000001e-07 2024-09-14T23:58:02.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:02.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300892+0000", 2024-09-14T23:58:02.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2469999999999998e-06 2024-09-14T23:58:02.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:02.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300899+0000", 2024-09-14T23:58:02.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5509999999999996e-06 2024-09-14T23:58:02.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:02.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300904+0000", 2024-09-14T23:58:02.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3299999999999998e-06 2024-09-14T23:58:02.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:02.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351707+0000", 2024-09-14T23:58:02.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050802673999999999 2024-09-14T23:58:02.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:02.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351755+0000", 2024-09-14T23:58:02.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8362000000000002e-05 2024-09-14T23:58:02.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:02.629 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351811+0000", 2024-09-14T23:58:02.629 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5801000000000003e-05 2024-09-14T23:58:02.629 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.629 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:02.629 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.629 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69266 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5259264~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:02.629 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300911+0000", 2024-09-14T23:58:02.629 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.307918895999997, 2024-09-14T23:58:02.629 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051125089999999998, 2024-09-14T23:58:02.629 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:02.629 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:02.629 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:02.630 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:02.630 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:02.630 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:02.630 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69266 2024-09-14T23:58:02.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.630 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:02.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:02.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300911+0000", 2024-09-14T23:58:02.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:02.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300911+0000", 2024-09-14T23:58:02.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.631 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:02.631 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300911+0000", 2024-09-14T23:58:02.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-14T23:58:02.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.631 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:02.631 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300913+0000", 2024-09-14T23:58:02.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9300000000000002e-06 2024-09-14T23:58:02.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.631 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:02.631 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300919+0000", 2024-09-14T23:58:02.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.489e-06 2024-09-14T23:58:02.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.632 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:02.632 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300925+0000", 2024-09-14T23:58:02.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1719999999999998e-06 2024-09-14T23:58:02.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.632 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:02.632 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351847+0000", 2024-09-14T23:58:02.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050922621000000001 2024-09-14T23:58:02.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.633 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:02.633 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351938+0000", 2024-09-14T23:58:02.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0830999999999998e-05 2024-09-14T23:58:02.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.633 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:02.633 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352036+0000", 2024-09-14T23:58:02.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7676999999999993e-05 2024-09-14T23:58:02.633 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.633 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:02.633 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.634 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69267 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5373952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:02.634 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300945+0000", 2024-09-14T23:58:02.634 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.307884100999999, 2024-09-14T23:58:02.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051217934, 2024-09-14T23:58:02.634 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:02.634 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:02.634 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:02.634 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:02.634 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:02.634 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:02.634 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69267 2024-09-14T23:58:02.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.634 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:02.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:02.634 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300945+0000", 2024-09-14T23:58:02.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:02.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300945+0000", 2024-09-14T23:58:02.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:02.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300946+0000", 2024-09-14T23:58:02.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2600000000000002e-07 2024-09-14T23:58:02.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:02.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300948+0000", 2024-09-14T23:58:02.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4940000000000002e-06 2024-09-14T23:58:02.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:02.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300956+0000", 2024-09-14T23:58:02.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.622e-06 2024-09-14T23:58:02.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:02.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300962+0000", 2024-09-14T23:58:02.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5790000000000003e-06 2024-09-14T23:58:02.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:02.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352063+0000", 2024-09-14T23:58:02.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051101281999999998 2024-09-14T23:58:02.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.637 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:02.637 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352110+0000", 2024-09-14T23:58:02.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7401000000000002e-05 2024-09-14T23:58:02.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.637 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:02.637 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352163+0000", 2024-09-14T23:58:02.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3029999999999999e-05 2024-09-14T23:58:02.637 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.637 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:02.637 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.638 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69268 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5488640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:02.638 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300969+0000", 2024-09-14T23:58:02.638 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.307860429000002, 2024-09-14T23:58:02.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051413420000000001, 2024-09-14T23:58:02.638 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:02.638 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:02.638 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:02.638 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:02.638 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:02.638 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:02.638 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69268 2024-09-14T23:58:02.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.638 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:02.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.638 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:02.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300969+0000", 2024-09-14T23:58:02.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:02.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300969+0000", 2024-09-14T23:58:02.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:02.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300970+0000", 2024-09-14T23:58:02.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7899999999999999e-07 2024-09-14T23:58:02.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:02.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300982+0000", 2024-09-14T23:58:02.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2741e-05 2024-09-14T23:58:02.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:02.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300990+0000", 2024-09-14T23:58:02.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8169999999999999e-06 2024-09-14T23:58:02.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:02.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300995+0000", 2024-09-14T23:58:02.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2240000000000003e-06 2024-09-14T23:58:02.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:02.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352200+0000", 2024-09-14T23:58:02.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051204223 2024-09-14T23:58:02.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:02.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352285+0000", 2024-09-14T23:58:02.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5519999999999994e-05 2024-09-14T23:58:02.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:02.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352383+0000", 2024-09-14T23:58:02.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7516000000000006e-05 2024-09-14T23:58:02.641 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.641 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:02.642 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.642 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69269 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5603328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:02.642 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.301003+0000", 2024-09-14T23:58:02.642 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.307826247000001, 2024-09-14T23:58:02.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051497381000000002, 2024-09-14T23:58:02.642 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:02.642 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:02.642 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:02.642 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:02.642 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:02.642 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:02.642 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69269 2024-09-14T23:58:02.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.643 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:02.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:02.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.301003+0000", 2024-09-14T23:58:02.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:02.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.301003+0000", 2024-09-14T23:58:02.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:02.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302671+0000", 2024-09-14T23:58:02.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001667785 2024-09-14T23:58:02.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:02.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302676+0000", 2024-09-14T23:58:02.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6600000000000003e-06 2024-09-14T23:58:02.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:02.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302697+0000", 2024-09-14T23:58:02.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1288999999999999e-05 2024-09-14T23:58:02.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:02.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302716+0000", 2024-09-14T23:58:02.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9128999999999999e-05 2024-09-14T23:58:02.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:02.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352399+0000", 2024-09-14T23:58:02.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049682995000000001 2024-09-14T23:58:02.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:02.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352446+0000", 2024-09-14T23:58:02.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6341000000000002e-05 2024-09-14T23:58:02.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:02.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352501+0000", 2024-09-14T23:58:02.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5182000000000001e-05 2024-09-14T23:58:02.646 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.646 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:02.646 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.646 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69273 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6062080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:02.646 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302829+0000", 2024-09-14T23:58:02.646 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.306000681, 2024-09-14T23:58:02.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050384311000000001, 2024-09-14T23:58:02.646 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:02.646 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:02.646 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:02.647 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:02.647 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:02.647 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:02.647 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69273 2024-09-14T23:58:02.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.647 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:02.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:02.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302829+0000", 2024-09-14T23:58:02.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:02.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302829+0000", 2024-09-14T23:58:02.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:02.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302829+0000", 2024-09-14T23:58:02.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7700000000000005e-07 2024-09-14T23:58:02.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:02.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302832+0000", 2024-09-14T23:58:02.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6970000000000002e-06 2024-09-14T23:58:02.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:02.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302840+0000", 2024-09-14T23:58:02.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5719999999999997e-06 2024-09-14T23:58:02.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:02.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302845+0000", 2024-09-14T23:58:02.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7939999999999999e-06 2024-09-14T23:58:02.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:02.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353111+0000", 2024-09-14T23:58:02.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050265588 2024-09-14T23:58:02.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:02.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353160+0000", 2024-09-14T23:58:02.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8859000000000001e-05 2024-09-14T23:58:02.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:02.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353213+0000", 2024-09-14T23:58:02.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3324e-05 2024-09-14T23:58:02.650 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.650 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:02.650 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.650 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69274 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6176768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:02.650 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302854+0000", 2024-09-14T23:58:02.650 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.305975586999999, 2024-09-14T23:58:02.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050579526, 2024-09-14T23:58:02.650 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:02.651 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:02.651 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:02.651 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:02.651 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:02.651 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:02.651 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69274 2024-09-14T23:58:02.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.651 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:02.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:02.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302854+0000", 2024-09-14T23:58:02.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.652 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:02.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302854+0000", 2024-09-14T23:58:02.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.652 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:02.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302855+0000", 2024-09-14T23:58:02.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0200000000000001e-07 2024-09-14T23:58:02.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.652 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:02.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302857+0000", 2024-09-14T23:58:02.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.447e-06 2024-09-14T23:58:02.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:02.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302864+0000", 2024-09-14T23:58:02.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7730000000000002e-06 2024-09-14T23:58:02.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:02.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302869+0000", 2024-09-14T23:58:02.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3199999999999999e-06 2024-09-14T23:58:02.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:02.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353249+0000", 2024-09-14T23:58:02.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050379647999999999 2024-09-14T23:58:02.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:02.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353336+0000", 2024-09-14T23:58:02.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6767999999999993e-05 2024-09-14T23:58:02.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:02.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353433+0000", 2024-09-14T23:58:02.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7868000000000006e-05 2024-09-14T23:58:02.654 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.654 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:02.654 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.655 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69275 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6291456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:02.655 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302889+0000", 2024-09-14T23:58:02.655 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.305940968999998, 2024-09-14T23:58:02.655 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050663118, 2024-09-14T23:58:02.655 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:02.655 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:02.655 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:02.655 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:02.655 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:02.655 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:02.655 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69275 2024-09-14T23:58:02.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.655 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:02.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:02.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302889+0000", 2024-09-14T23:58:02.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:02.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302889+0000", 2024-09-14T23:58:02.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:02.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302889+0000", 2024-09-14T23:58:02.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6399999999999998e-07 2024-09-14T23:58:02.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:02.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302891+0000", 2024-09-14T23:58:02.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.339e-06 2024-09-14T23:58:02.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:02.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302898+0000", 2024-09-14T23:58:02.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0380000000000002e-06 2024-09-14T23:58:02.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:02.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302904+0000", 2024-09-14T23:58:02.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2159999999999997e-06 2024-09-14T23:58:02.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:02.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353449+0000", 2024-09-14T23:58:02.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050545386999999997 2024-09-14T23:58:02.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:02.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353494+0000", 2024-09-14T23:58:02.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4592999999999998e-05 2024-09-14T23:58:02.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:02.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353552+0000", 2024-09-14T23:58:02.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8180999999999997e-05 2024-09-14T23:58:02.658 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.658 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:02.659 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.659 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69276 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6406144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:02.659 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302911+0000", 2024-09-14T23:58:02.659 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.305918556000002, 2024-09-14T23:58:02.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050865753, 2024-09-14T23:58:02.659 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:02.659 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:02.659 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:02.659 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:02.659 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:02.659 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:02.659 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69276 2024-09-14T23:58:02.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.660 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:02.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:02.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302911+0000", 2024-09-14T23:58:02.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:02.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302911+0000", 2024-09-14T23:58:02.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:02.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302912+0000", 2024-09-14T23:58:02.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0800000000000004e-07 2024-09-14T23:58:02.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:02.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302914+0000", 2024-09-14T23:58:02.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9589999999999998e-06 2024-09-14T23:58:02.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:02.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302920+0000", 2024-09-14T23:58:02.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4579999999999998e-06 2024-09-14T23:58:02.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:02.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302925+0000", 2024-09-14T23:58:02.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9429999999999999e-06 2024-09-14T23:58:02.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:02.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353586+0000", 2024-09-14T23:58:02.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050660966000000002 2024-09-14T23:58:02.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:02.663 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353679+0000", 2024-09-14T23:58:02.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3163000000000002e-05 2024-09-14T23:58:02.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.663 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:02.663 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353777+0000", 2024-09-14T23:58:02.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7555999999999999e-05 2024-09-14T23:58:02.663 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.663 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:02.663 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.663 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69277 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6520832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:02.663 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302932+0000", 2024-09-14T23:58:02.663 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.305897469000001, 2024-09-14T23:58:02.664 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050977674000000001, 2024-09-14T23:58:02.664 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:02.664 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:02.664 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:02.664 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:02.664 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:02.664 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:02.664 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69277 2024-09-14T23:58:02.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.664 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:02.664 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.664 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:02.664 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302932+0000", 2024-09-14T23:58:02.664 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.665 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.665 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:02.665 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302932+0000", 2024-09-14T23:58:02.665 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.665 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.665 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:02.665 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302932+0000", 2024-09-14T23:58:02.665 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3299999999999998e-07 2024-09-14T23:58:02.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.665 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.665 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:02.665 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302935+0000", 2024-09-14T23:58:02.665 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2139999999999999e-06 2024-09-14T23:58:02.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:02.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302941+0000", 2024-09-14T23:58:02.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5039999999999999e-06 2024-09-14T23:58:02.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:02.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302946+0000", 2024-09-14T23:58:02.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2079999999999999e-06 2024-09-14T23:58:02.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:02.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353805+0000", 2024-09-14T23:58:02.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050858588000000003 2024-09-14T23:58:02.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:02.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353849+0000", 2024-09-14T23:58:02.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4511000000000001e-05 2024-09-14T23:58:02.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:02.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353910+0000", 2024-09-14T23:58:02.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0316000000000003e-05 2024-09-14T23:58:02.667 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.667 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:02.667 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.667 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69278 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6635520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:02.668 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302953+0000", 2024-09-14T23:58:02.668 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.305876341000001, 2024-09-14T23:58:02.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051174059000000001, 2024-09-14T23:58:02.668 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:02.668 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:02.668 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:02.668 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:02.668 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:02.668 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:02.668 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69278 2024-09-14T23:58:02.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.668 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:02.668 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.668 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:02.669 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302953+0000", 2024-09-14T23:58:02.669 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.669 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:02.669 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302953+0000", 2024-09-14T23:58:02.669 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.669 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:02.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302954+0000", 2024-09-14T23:58:02.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5100000000000001e-07 2024-09-14T23:58:02.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:02.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302964+0000", 2024-09-14T23:58:02.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0475000000000001e-05 2024-09-14T23:58:02.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:02.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302970+0000", 2024-09-14T23:58:02.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3879999999999997e-06 2024-09-14T23:58:02.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.671 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:02.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302975+0000", 2024-09-14T23:58:02.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8409999999999999e-06 2024-09-14T23:58:02.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.671 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.671 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:02.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353944+0000", 2024-09-14T23:58:02.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050968952999999997 2024-09-14T23:58:02.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.671 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.671 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:02.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354031+0000", 2024-09-14T23:58:02.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6537000000000002e-05 2024-09-14T23:58:02.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.671 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.672 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:02.672 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354127+0000", 2024-09-14T23:58:02.672 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6514000000000005e-05 2024-09-14T23:58:02.672 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.672 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:02.672 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.672 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.672 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69279 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6750208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:02.672 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302984+0000", 2024-09-14T23:58:02.672 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.305845990000002, 2024-09-14T23:58:02.672 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051258845999999997, 2024-09-14T23:58:02.672 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:02.672 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:02.673 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:02.673 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:02.673 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:02.673 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:02.673 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69279 2024-09-14T23:58:02.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.673 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:02.673 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.673 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:02.673 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302984+0000", 2024-09-14T23:58:02.673 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.673 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.673 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:02.674 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302984+0000", 2024-09-14T23:58:02.674 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.674 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:02.674 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303025+0000", 2024-09-14T23:58:02.674 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1690999999999998e-05 2024-09-14T23:58:02.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.674 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:02.674 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303029+0000", 2024-09-14T23:58:02.674 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6559999999999998e-06 2024-09-14T23:58:02.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.674 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:02.674 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303037+0000", 2024-09-14T23:58:02.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7950000000000008e-06 2024-09-14T23:58:02.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.675 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:02.675 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303043+0000", 2024-09-14T23:58:02.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9669999999999998e-06 2024-09-14T23:58:02.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.675 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:02.675 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354142+0000", 2024-09-14T23:58:02.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051099072000000002 2024-09-14T23:58:02.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.675 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:02.675 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354183+0000", 2024-09-14T23:58:02.676 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0834000000000002e-05 2024-09-14T23:58:02.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.676 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:02.676 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354242+0000", 2024-09-14T23:58:02.676 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9830999999999997e-05 2024-09-14T23:58:02.676 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.676 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:02.676 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.676 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69280 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6864896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:02.676 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.303052+0000", 2024-09-14T23:58:02.676 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.305777446999997, 2024-09-14T23:58:02.677 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051406463999999999, 2024-09-14T23:58:02.677 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:02.677 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:02.677 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:02.677 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:02.677 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:02.677 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:02.677 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69280 2024-09-14T23:58:02.677 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.677 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:02.677 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.677 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:02.677 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303052+0000", 2024-09-14T23:58:02.678 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.678 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.678 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.678 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:02.678 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303052+0000", 2024-09-14T23:58:02.678 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.678 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.678 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.678 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:02.678 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304673+0000", 2024-09-14T23:58:02.678 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001621389 2024-09-14T23:58:02.678 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.678 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.678 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:02.678 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304681+0000", 2024-09-14T23:58:02.679 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8439999999999998e-06 2024-09-14T23:58:02.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.679 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:02.679 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304691+0000", 2024-09-14T23:58:02.679 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8339999999999994e-06 2024-09-14T23:58:02.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.679 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:02.679 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304698+0000", 2024-09-14T23:58:02.679 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3629999999999999e-06 2024-09-14T23:58:02.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.679 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:02.679 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354278+0000", 2024-09-14T23:58:02.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049580618 2024-09-14T23:58:02.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.680 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:02.680 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354364+0000", 2024-09-14T23:58:02.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6059999999999999e-05 2024-09-14T23:58:02.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.680 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:02.680 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354459+0000", 2024-09-14T23:58:02.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4356000000000003e-05 2024-09-14T23:58:02.680 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.680 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:02.681 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.681 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.681 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.681 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69285 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 7438336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:02.681 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.304792+0000", 2024-09-14T23:58:02.681 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.304038038000002, 2024-09-14T23:58:02.681 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050398874000000003, 2024-09-14T23:58:02.681 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:02.681 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:02.681 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:02.681 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:02.681 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:02.681 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:02.681 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69285 2024-09-14T23:58:02.682 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.682 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:02.682 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.682 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:02.682 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304792+0000", 2024-09-14T23:58:02.682 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.682 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.682 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.682 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:02.682 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304792+0000", 2024-09-14T23:58:02.682 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.682 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.682 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.682 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:02.683 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304792+0000", 2024-09-14T23:58:02.683 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2800000000000003e-07 2024-09-14T23:58:02.683 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.683 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.683 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:02.683 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304794+0000", 2024-09-14T23:58:02.683 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8700000000000001e-06 2024-09-14T23:58:02.683 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.683 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.683 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:02.683 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304800+0000", 2024-09-14T23:58:02.683 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0889999999999996e-06 2024-09-14T23:58:02.683 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.684 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.684 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:02.684 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304805+0000", 2024-09-14T23:58:02.684 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7060000000000003e-06 2024-09-14T23:58:02.684 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.684 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.684 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:02.684 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355108+0000", 2024-09-14T23:58:02.684 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050303170000000001 2024-09-14T23:58:02.684 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.684 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.684 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:02.684 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355150+0000", 2024-09-14T23:58:02.684 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2271999999999998e-05 2024-09-14T23:58:02.685 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.685 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.685 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:02.685 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355190+0000", 2024-09-14T23:58:02.685 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0438999999999997e-05 2024-09-14T23:58:02.685 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.685 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:02.685 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.685 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.685 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.685 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69286 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 7553024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:02.685 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.304811+0000", 2024-09-14T23:58:02.685 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.304018675999998, 2024-09-14T23:58:02.685 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050585512999999999, 2024-09-14T23:58:02.686 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:02.686 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:02.686 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:02.686 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:02.686 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:02.686 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:02.686 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69286 2024-09-14T23:58:02.686 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.686 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:02.686 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.686 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:02.686 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304811+0000", 2024-09-14T23:58:02.686 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.686 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.686 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.687 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:02.687 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304811+0000", 2024-09-14T23:58:02.687 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.687 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.687 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.687 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:02.687 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304811+0000", 2024-09-14T23:58:02.687 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.89e-07 2024-09-14T23:58:02.687 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.687 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.687 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:02.687 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304814+0000", 2024-09-14T23:58:02.687 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3199999999999998e-06 2024-09-14T23:58:02.687 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.687 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.688 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:02.688 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304821+0000", 2024-09-14T23:58:02.688 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5689999999999999e-06 2024-09-14T23:58:02.688 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.688 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.688 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:02.688 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304826+0000", 2024-09-14T23:58:02.688 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1200000000000001e-06 2024-09-14T23:58:02.688 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.688 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.688 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:02.688 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355226+0000", 2024-09-14T23:58:02.688 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050399653000000003 2024-09-14T23:58:02.688 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.688 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.689 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:02.689 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355314+0000", 2024-09-14T23:58:02.689 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8389999999999999e-05 2024-09-14T23:58:02.689 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.689 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.689 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:02.689 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355396+0000", 2024-09-14T23:58:02.689 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2071999999999996e-05 2024-09-14T23:58:02.689 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.689 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:02.689 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.689 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.689 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.689 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69287 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 7667712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:02.689 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.304833+0000", 2024-09-14T23:58:02.689 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.303996077999997, 2024-09-14T23:58:02.690 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050664740999999999, 2024-09-14T23:58:02.690 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:02.690 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:02.690 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:02.690 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:02.690 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:02.690 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:02.690 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69287 2024-09-14T23:58:02.690 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.690 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:02.690 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.690 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:02.690 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304833+0000", 2024-09-14T23:58:02.690 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.691 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.691 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.691 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:02.691 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304833+0000", 2024-09-14T23:58:02.691 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.691 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.691 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.691 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:02.691 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304834+0000", 2024-09-14T23:58:02.691 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4999999999999998e-07 2024-09-14T23:58:02.691 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.691 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.691 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:02.691 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304836+0000", 2024-09-14T23:58:02.691 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0279999999999999e-06 2024-09-14T23:58:02.692 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.692 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.692 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:02.692 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304842+0000", 2024-09-14T23:58:02.692 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8919999999999997e-06 2024-09-14T23:58:02.692 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.692 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.692 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:02.692 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304847+0000", 2024-09-14T23:58:02.692 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6160000000000001e-06 2024-09-14T23:58:02.692 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.692 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.692 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:02.692 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355411+0000", 2024-09-14T23:58:02.693 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050563540999999997 2024-09-14T23:58:02.693 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.693 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.693 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:02.693 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355459+0000", 2024-09-14T23:58:02.693 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7979000000000003e-05 2024-09-14T23:58:02.693 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.693 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.693 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:02.693 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355498+0000", 2024-09-14T23:58:02.693 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9335000000000002e-05 2024-09-14T23:58:02.693 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.693 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:02.693 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.693 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.694 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.694 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69288 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 7782400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:02.694 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.304854+0000", 2024-09-14T23:58:02.694 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.303975639999997, 2024-09-14T23:58:02.694 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050855145999999997, 2024-09-14T23:58:02.694 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:02.694 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:02.694 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:02.694 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:02.694 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:02.694 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:02.694 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69288 2024-09-14T23:58:02.694 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.694 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:02.694 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.695 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:02.695 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304854+0000", 2024-09-14T23:58:02.695 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.695 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.695 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.695 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:02.695 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304854+0000", 2024-09-14T23:58:02.695 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:02.695 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.695 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.695 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:02.695 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.306670+0000", 2024-09-14T23:58:02.695 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00181631 2024-09-14T23:58:02.695 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.695 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.695 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:02.695 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.306675+0000", 2024-09-14T23:58:02.696 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0220000000000004e-06 2024-09-14T23:58:02.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.696 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.696 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:02.696 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.306684+0000", 2024-09-14T23:58:02.696 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7309999999999999e-06 2024-09-14T23:58:02.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.696 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.696 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:02.696 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.306689+0000", 2024-09-14T23:58:02.696 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6609999999999999e-06 2024-09-14T23:58:02.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.696 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.696 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:02.696 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355533+0000", 2024-09-14T23:58:02.696 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048844578 2024-09-14T23:58:02.697 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.697 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.697 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:02.697 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355622+0000", 2024-09-14T23:58:02.697 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9006999999999997e-05 2024-09-14T23:58:02.697 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:02.697 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:02.697 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:02.697 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355709+0000", 2024-09-14T23:58:02.697 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6836999999999995e-05 2024-09-14T23:58:02.697 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.697 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:02.697 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.697 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:02.698 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:02.698 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:02.699 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:02.697+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:02.699 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T23:58:02.799 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:02.798+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:02.809 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:02.809 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:02.809 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:58:02.809 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:02.817 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T23:58:02.901 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:02.900+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:02.965 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:02.966 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:02.966 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:58:02.966 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:58:02.966 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:02.974 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T23:58:03.001 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:02.999+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:03.102 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:03.101+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:03.121 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T23:58:03.127 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:03.127 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:58:03.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:58:03.127 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:58:03.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.127 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:43054 3.1f 3:ffbfe596:::rbd_data.1118350d5ae5.00000000000002eb:head [set-alloc-hint object_size 262144 write_size 262144,write 196608~22770] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T23:58:03.127 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:58:03.127 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 70.158331578000002, 2024-09-14T23:58:03.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.99973334400000002, 2024-09-14T23:58:03.128 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:03.128 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:03.128 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:03.128 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:03.128 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:03.128 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:03.128 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 43054 2024-09-14T23:58:03.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.128 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:03.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:03.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:58:03.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:03.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:58:03.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:03.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966552+0000", 2024-09-14T23:58:03.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3790000000000001e-06 2024-09-14T23:58:03.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:03.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966565+0000", 2024-09-14T23:58:03.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2459999999999999e-05 2024-09-14T23:58:03.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:03.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966576+0000", 2024-09-14T23:58:03.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1039000000000001e-05 2024-09-14T23:58:03.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:03.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966582+0000", 2024-09-14T23:58:03.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9660000000000001e-06 2024-09-14T23:58:03.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:03.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966625+0000", 2024-09-14T23:58:03.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3529999999999998e-05 2024-09-14T23:58:03.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:58:03.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966632+0000", 2024-09-14T23:58:03.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9809999999999997e-06 2024-09-14T23:58:03.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:03.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.964460+0000", 2024-09-14T23:58:03.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.99782768300000002 2024-09-14T23:58:03.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:03.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.964641+0000", 2024-09-14T23:58:03.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000181089 2024-09-14T23:58:03.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-14T23:58:03.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.964809+0000", 2024-09-14T23:58:03.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000168505 2024-09-14T23:58:03.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:58:03.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966030+0000", 2024-09-14T23:58:03.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012206929999999999 2024-09-14T23:58:03.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:03.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966181+0000", 2024-09-14T23:58:03.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000150569 2024-09-14T23:58:03.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:03.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966243+0000", 2024-09-14T23:58:03.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2331999999999999e-05 2024-09-14T23:58:03.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:03.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966284+0000", 2024-09-14T23:58:03.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1118000000000002e-05 2024-09-14T23:58:03.134 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.134 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:03.134 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.134 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:144877 3.6 3:65baca07:::rbd_data.11189263ca38.000000000000000a:2a [read 909312~16384] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-14T23:58:03.134 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:53.415702+0000", 2024-09-14T23:58:03.134 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7091799319999996, 2024-09-14T23:58:03.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.036982196000000002, 2024-09-14T23:58:03.134 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:03.134 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:03.134 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:03.135 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:03.135 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:03.135 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:03.135 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 144877 2024-09-14T23:58:03.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.135 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:03.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:03.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.415702+0000", 2024-09-14T23:58:03.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:03.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.415702+0000", 2024-09-14T23:58:03.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:03.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.415703+0000", 2024-09-14T23:58:03.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0499999999999999e-07 2024-09-14T23:58:03.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:03.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.415705+0000", 2024-09-14T23:58:03.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9819999999999998e-06 2024-09-14T23:58:03.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:03.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.415710+0000", 2024-09-14T23:58:03.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6779999999999997e-06 2024-09-14T23:58:03.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:03.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.415717+0000", 2024-09-14T23:58:03.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8589999999999999e-06 2024-09-14T23:58:03.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:03.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.452575+0000", 2024-09-14T23:58:03.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.036857525000000002 2024-09-14T23:58:03.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:03.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.452624+0000", 2024-09-14T23:58:03.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9397999999999997e-05 2024-09-14T23:58:03.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:03.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.452685+0000", 2024-09-14T23:58:03.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0349000000000001e-05 2024-09-14T23:58:03.138 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.138 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:03.138 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.139 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:144878 3.6 3:65baca07:::rbd_data.11189263ca38.000000000000000a:2a [read 974848~16384] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-14T23:58:03.139 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:53.415935+0000", 2024-09-14T23:58:03.139 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7089477130000006, 2024-09-14T23:58:03.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.036967765, 2024-09-14T23:58:03.139 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:03.139 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:03.139 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:03.139 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:03.139 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:03.139 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:03.139 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 144878 2024-09-14T23:58:03.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.139 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:03.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:03.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.415935+0000", 2024-09-14T23:58:03.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:03.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.415935+0000", 2024-09-14T23:58:03.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:03.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.415935+0000", 2024-09-14T23:58:03.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8100000000000003e-07 2024-09-14T23:58:03.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:03.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.415939+0000", 2024-09-14T23:58:03.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5949999999999999e-06 2024-09-14T23:58:03.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:03.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.415946+0000", 2024-09-14T23:58:03.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4900000000000003e-06 2024-09-14T23:58:03.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:03.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.415952+0000", 2024-09-14T23:58:03.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2530000000000001e-06 2024-09-14T23:58:03.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:03.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.452724+0000", 2024-09-14T23:58:03.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.036771525999999999 2024-09-14T23:58:03.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:03.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.452803+0000", 2024-09-14T23:58:03.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8885000000000001e-05 2024-09-14T23:58:03.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:03.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.452902+0000", 2024-09-14T23:58:03.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9434999999999999e-05 2024-09-14T23:58:03.143 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.143 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:03.143 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.143 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:144879 3.6 3:65baca07:::rbd_data.11189263ca38.000000000000000a:2a [read 1040384~8192] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-14T23:58:03.143 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:53.415963+0000", 2024-09-14T23:58:03.143 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7089192690000008, 2024-09-14T23:58:03.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037080130000000003, 2024-09-14T23:58:03.143 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:03.143 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:03.143 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:03.143 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:03.143 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:03.143 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:03.144 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 144879 2024-09-14T23:58:03.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.144 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:03.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:03.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.415963+0000", 2024-09-14T23:58:03.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:03.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.415963+0000", 2024-09-14T23:58:03.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:03.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.415964+0000", 2024-09-14T23:58:03.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4000000000000002e-07 2024-09-14T23:58:03.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:03.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.415966+0000", 2024-09-14T23:58:03.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3089999999999998e-06 2024-09-14T23:58:03.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.145 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:03.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.415972+0000", 2024-09-14T23:58:03.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5509999999999996e-06 2024-09-14T23:58:03.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:03.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.415978+0000", 2024-09-14T23:58:03.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6949999999999997e-06 2024-09-14T23:58:03.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:03.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.452935+0000", 2024-09-14T23:58:03.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.036956646000000003 2024-09-14T23:58:03.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.146 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.146 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:03.146 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.452984+0000", 2024-09-14T23:58:03.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9688000000000003e-05 2024-09-14T23:58:03.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:03.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.453043+0000", 2024-09-14T23:58:03.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8700999999999999e-05 2024-09-14T23:58:03.147 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.147 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:03.147 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.147 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:144912 3.6 3:6629293c:::rbd_data.11189263ca38.0000000000000009:2a [read 1359872~0] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-14T23:58:03.147 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:53.416279+0000", 2024-09-14T23:58:03.147 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7086030149999996, 2024-09-14T23:58:03.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.036959829999999999, 2024-09-14T23:58:03.148 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:03.148 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:03.148 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:03.148 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:03.148 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:03.148 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:03.148 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 144912 2024-09-14T23:58:03.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.148 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:03.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:03.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416279+0000", 2024-09-14T23:58:03.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:03.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416279+0000", 2024-09-14T23:58:03.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:03.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416280+0000", 2024-09-14T23:58:03.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8400000000000005e-07 2024-09-14T23:58:03.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:03.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416282+0000", 2024-09-14T23:58:03.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.486e-06 2024-09-14T23:58:03.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:03.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416289+0000", 2024-09-14T23:58:03.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.63e-06 2024-09-14T23:58:03.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:03.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416294+0000", 2024-09-14T23:58:03.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.95e-06 2024-09-14T23:58:03.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:03.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.453081+0000", 2024-09-14T23:58:03.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.036787577000000002 2024-09-14T23:58:03.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:03.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.453161+0000", 2024-09-14T23:58:03.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0031000000000007e-05 2024-09-14T23:58:03.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.151 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:03.151 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.453239+0000", 2024-09-14T23:58:03.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7671999999999998e-05 2024-09-14T23:58:03.151 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.151 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:03.151 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.151 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.151 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:144913 3.6 3:6629293c:::rbd_data.11189263ca38.0000000000000009:2a [read 1425408~0] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-14T23:58:03.151 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:53.416302+0000", 2024-09-14T23:58:03.152 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7085801640000007, 2024-09-14T23:58:03.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037067357000000002, 2024-09-14T23:58:03.152 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:03.152 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:03.152 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:03.152 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:03.152 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:03.152 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:03.152 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 144913 2024-09-14T23:58:03.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.152 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:03.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.152 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:03.152 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416302+0000", 2024-09-14T23:58:03.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:03.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416302+0000", 2024-09-14T23:58:03.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:03.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416303+0000", 2024-09-14T23:58:03.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7300000000000001e-07 2024-09-14T23:58:03.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:03.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416305+0000", 2024-09-14T23:58:03.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2900000000000001e-06 2024-09-14T23:58:03.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:03.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416311+0000", 2024-09-14T23:58:03.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0310000000000004e-06 2024-09-14T23:58:03.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:03.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416317+0000", 2024-09-14T23:58:03.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9499999999999998e-06 2024-09-14T23:58:03.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:03.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.453278+0000", 2024-09-14T23:58:03.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.036961234000000003 2024-09-14T23:58:03.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:03.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.453323+0000", 2024-09-14T23:58:03.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5022999999999999e-05 2024-09-14T23:58:03.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:03.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.453369+0000", 2024-09-14T23:58:03.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6356000000000001e-05 2024-09-14T23:58:03.155 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.155 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:03.155 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.156 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:144914 3.6 3:6629293c:::rbd_data.11189263ca38.0000000000000009:2a [read 1490944~0] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-14T23:58:03.156 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:53.416324+0000", 2024-09-14T23:58:03.156 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7085586240000001, 2024-09-14T23:58:03.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037241315999999997, 2024-09-14T23:58:03.156 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:03.156 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:03.156 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:03.156 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:03.156 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:03.156 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:03.156 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 144914 2024-09-14T23:58:03.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.156 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:03.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.156 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:03.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416324+0000", 2024-09-14T23:58:03.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.157 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:03.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416324+0000", 2024-09-14T23:58:03.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.157 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:03.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416324+0000", 2024-09-14T23:58:03.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6400000000000003e-07 2024-09-14T23:58:03.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.157 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:03.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416326+0000", 2024-09-14T23:58:03.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1799999999999999e-06 2024-09-14T23:58:03.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:03.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416334+0000", 2024-09-14T23:58:03.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.345e-06 2024-09-14T23:58:03.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:03.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416339+0000", 2024-09-14T23:58:03.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1699999999999996e-06 2024-09-14T23:58:03.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:03.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.453406+0000", 2024-09-14T23:58:03.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037067584000000001 2024-09-14T23:58:03.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:03.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.453485+0000", 2024-09-14T23:58:03.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8740999999999997e-05 2024-09-14T23:58:03.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:03.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.453565+0000", 2024-09-14T23:58:03.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9832000000000004e-05 2024-09-14T23:58:03.159 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.159 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:03.159 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.160 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:144915 3.6 3:6629293c:::rbd_data.11189263ca38.0000000000000009:2a [read 1556480~0] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-14T23:58:03.160 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:53.416345+0000", 2024-09-14T23:58:03.160 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7085369450000005, 2024-09-14T23:58:03.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037345237000000003, 2024-09-14T23:58:03.160 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:03.160 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:03.160 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:03.160 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:03.160 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:03.160 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:03.160 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 144915 2024-09-14T23:58:03.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.160 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:03.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.161 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:03.161 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416345+0000", 2024-09-14T23:58:03.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.161 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:03.161 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416345+0000", 2024-09-14T23:58:03.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.161 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:03.161 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416346+0000", 2024-09-14T23:58:03.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2600000000000002e-07 2024-09-14T23:58:03.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:03.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416348+0000", 2024-09-14T23:58:03.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1050000000000002e-06 2024-09-14T23:58:03.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:03.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416355+0000", 2024-09-14T23:58:03.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7349999999999999e-06 2024-09-14T23:58:03.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:03.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416360+0000", 2024-09-14T23:58:03.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2939999999999996e-06 2024-09-14T23:58:03.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:03.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.453602+0000", 2024-09-14T23:58:03.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037242453000000002 2024-09-14T23:58:03.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:03.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.453647+0000", 2024-09-14T23:58:03.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4113000000000003e-05 2024-09-14T23:58:03.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:03.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.453691+0000", 2024-09-14T23:58:03.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4011000000000003e-05 2024-09-14T23:58:03.164 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.164 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:03.164 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.164 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:144916 3.6 3:65baca07:::rbd_data.11189263ca38.000000000000000a:2a [read 1351680~8192] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-14T23:58:03.164 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:53.416368+0000", 2024-09-14T23:58:03.164 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7085138660000005, 2024-09-14T23:58:03.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037549193000000002, 2024-09-14T23:58:03.164 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:03.164 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:03.164 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:03.164 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:03.164 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:03.165 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:03.165 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 144916 2024-09-14T23:58:03.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.165 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:03.165 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.165 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:03.165 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416368+0000", 2024-09-14T23:58:03.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.165 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.165 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:03.165 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416368+0000", 2024-09-14T23:58:03.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:03.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416369+0000", 2024-09-14T23:58:03.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3300000000000003e-07 2024-09-14T23:58:03.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:03.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416375+0000", 2024-09-14T23:58:03.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8379999999999998e-06 2024-09-14T23:58:03.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:03.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416382+0000", 2024-09-14T23:58:03.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2949999999999996e-06 2024-09-14T23:58:03.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:03.167 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416387+0000", 2024-09-14T23:58:03.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3890000000000004e-06 2024-09-14T23:58:03.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:03.167 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.453730+0000", 2024-09-14T23:58:03.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037342594999999999 2024-09-14T23:58:03.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:03.167 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.453813+0000", 2024-09-14T23:58:03.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3484000000000002e-05 2024-09-14T23:58:03.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.168 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:03.168 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.453918+0000", 2024-09-14T23:58:03.168 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000104159 2024-09-14T23:58:03.168 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.168 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:03.168 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.168 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:144917 3.6 3:65baca07:::rbd_data.11189263ca38.000000000000000a:2a [read 1417216~8192] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-14T23:58:03.168 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:53.416395+0000", 2024-09-14T23:58:03.168 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7084869650000005, 2024-09-14T23:58:03.168 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037661671000000001, 2024-09-14T23:58:03.168 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:03.169 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:03.169 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:03.169 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:03.169 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:03.169 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:03.169 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 144917 2024-09-14T23:58:03.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.169 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:03.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.169 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:03.169 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416395+0000", 2024-09-14T23:58:03.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.170 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:03.170 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416395+0000", 2024-09-14T23:58:03.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.170 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.170 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:03.170 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416396+0000", 2024-09-14T23:58:03.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7899999999999999e-07 2024-09-14T23:58:03.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.170 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.170 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:03.170 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416398+0000", 2024-09-14T23:58:03.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1859999999999999e-06 2024-09-14T23:58:03.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:03.171 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416404+0000", 2024-09-14T23:58:03.171 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2249999999999997e-06 2024-09-14T23:58:03.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:03.171 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416409+0000", 2024-09-14T23:58:03.171 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9679999999999997e-06 2024-09-14T23:58:03.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:03.171 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.453950+0000", 2024-09-14T23:58:03.171 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037540967000000001 2024-09-14T23:58:03.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.172 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:03.172 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.453998+0000", 2024-09-14T23:58:03.172 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8284000000000001e-05 2024-09-14T23:58:03.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.172 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:03.172 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.454057+0000", 2024-09-14T23:58:03.172 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8561999999999999e-05 2024-09-14T23:58:03.172 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.172 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:03.172 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.173 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.173 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:144918 3.6 3:65baca07:::rbd_data.11189263ca38.000000000000000a:2a [read 1482752~8192] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-14T23:58:03.173 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:53.416416+0000", 2024-09-14T23:58:03.173 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7084664719999996, 2024-09-14T23:58:03.173 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037857809999999999, 2024-09-14T23:58:03.173 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:03.173 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:03.173 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:03.173 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:03.173 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:03.173 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:03.173 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 144918 2024-09-14T23:58:03.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.174 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:03.174 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.174 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:03.174 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416416+0000", 2024-09-14T23:58:03.174 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.174 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.174 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:03.174 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416416+0000", 2024-09-14T23:58:03.174 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.174 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.174 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:03.175 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416416+0000", 2024-09-14T23:58:03.175 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.06e-07 2024-09-14T23:58:03.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.175 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:03.175 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416418+0000", 2024-09-14T23:58:03.175 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.063e-06 2024-09-14T23:58:03.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.175 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:03.175 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416424+0000", 2024-09-14T23:58:03.175 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9030000000000001e-06 2024-09-14T23:58:03.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.176 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:03.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416431+0000", 2024-09-14T23:58:03.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7469999999999999e-06 2024-09-14T23:58:03.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.176 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:03.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.454098+0000", 2024-09-14T23:58:03.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037666408999999998 2024-09-14T23:58:03.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.176 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:03.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.454176+0000", 2024-09-14T23:58:03.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8857999999999996e-05 2024-09-14T23:58:03.177 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.177 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.177 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:03.177 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.454274+0000", 2024-09-14T23:58:03.177 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7224000000000003e-05 2024-09-14T23:58:03.177 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.177 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:03.177 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.177 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.177 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.177 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:144919 3.6 3:65baca07:::rbd_data.11189263ca38.000000000000000a:2a [read 1548288~8192] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-14T23:58:03.177 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:53.416438+0000", 2024-09-14T23:58:03.177 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7084439469999992, 2024-09-14T23:58:03.178 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037973489999999999, 2024-09-14T23:58:03.178 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:03.178 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:03.178 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:03.178 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:03.178 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:03.178 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:03.178 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 144919 2024-09-14T23:58:03.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.178 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:03.178 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.178 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:03.178 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416438+0000", 2024-09-14T23:58:03.178 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.178 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.179 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:03.179 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416438+0000", 2024-09-14T23:58:03.179 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.179 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.179 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:03.179 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416439+0000", 2024-09-14T23:58:03.179 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6499999999999999e-07 2024-09-14T23:58:03.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.179 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.179 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:03.179 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416441+0000", 2024-09-14T23:58:03.179 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.255e-06 2024-09-14T23:58:03.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:03.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416448+0000", 2024-09-14T23:58:03.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7830000000000001e-06 2024-09-14T23:58:03.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:03.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416453+0000", 2024-09-14T23:58:03.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1000000000000003e-06 2024-09-14T23:58:03.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:03.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.454305+0000", 2024-09-14T23:58:03.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037851535999999998 2024-09-14T23:58:03.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.181 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.181 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:03.181 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.454355+0000", 2024-09-14T23:58:03.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0652999999999998e-05 2024-09-14T23:58:03.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.181 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.181 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:03.181 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.454412+0000", 2024-09-14T23:58:03.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6598000000000003e-05 2024-09-14T23:58:03.181 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.181 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:03.181 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.182 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:144953 3.6 3:6629293c:::rbd_data.11189263ca38.0000000000000009:2a [read 1335296~0] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-14T23:58:03.182 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:53.416731+0000", 2024-09-14T23:58:03.182 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7081513649999991, 2024-09-14T23:58:03.182 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037873579999999997, 2024-09-14T23:58:03.182 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:03.182 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:03.182 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:03.182 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:03.182 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:03.182 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:03.182 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 144953 2024-09-14T23:58:03.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.182 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:03.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:03.183 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416731+0000", 2024-09-14T23:58:03.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:03.183 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416731+0000", 2024-09-14T23:58:03.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:03.183 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416731+0000", 2024-09-14T23:58:03.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.32e-07 2024-09-14T23:58:03.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.184 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.184 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:03.184 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416733+0000", 2024-09-14T23:58:03.184 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.058e-06 2024-09-14T23:58:03.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.184 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.184 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:03.184 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416740+0000", 2024-09-14T23:58:03.184 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3729999999999998e-06 2024-09-14T23:58:03.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.184 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.184 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:03.184 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416745+0000", 2024-09-14T23:58:03.184 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0799999999999996e-06 2024-09-14T23:58:03.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.185 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:03.185 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.454452+0000", 2024-09-14T23:58:03.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037706867999999998 2024-09-14T23:58:03.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.185 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:03.185 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.454528+0000", 2024-09-14T23:58:03.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6614999999999997e-05 2024-09-14T23:58:03.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.185 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:03.186 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.454604+0000", 2024-09-14T23:58:03.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6154000000000003e-05 2024-09-14T23:58:03.186 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.186 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:03.186 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.186 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:144954 3.6 3:6629293c:::rbd_data.11189263ca38.0000000000000009:2a [read 1400832~0] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-14T23:58:03.186 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:53.416751+0000", 2024-09-14T23:58:03.186 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7081311770000003, 2024-09-14T23:58:03.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037978535000000001, 2024-09-14T23:58:03.186 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:03.186 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:03.186 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:03.186 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:03.187 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:03.187 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:03.187 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 144954 2024-09-14T23:58:03.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.187 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:03.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:03.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416751+0000", 2024-09-14T23:58:03.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:03.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416751+0000", 2024-09-14T23:58:03.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.188 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:03.188 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416752+0000", 2024-09-14T23:58:03.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1200000000000003e-07 2024-09-14T23:58:03.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.188 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:03.188 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416754+0000", 2024-09-14T23:58:03.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9589999999999998e-06 2024-09-14T23:58:03.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.188 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:03.188 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416760+0000", 2024-09-14T23:58:03.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9730000000000002e-06 2024-09-14T23:58:03.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:03.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416764+0000", 2024-09-14T23:58:03.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9099999999999996e-06 2024-09-14T23:58:03.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:03.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.454641+0000", 2024-09-14T23:58:03.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037876716999999997 2024-09-14T23:58:03.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:03.190 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.454685+0000", 2024-09-14T23:58:03.190 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3927999999999997e-05 2024-09-14T23:58:03.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.190 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.190 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:03.190 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.454730+0000", 2024-09-14T23:58:03.190 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4536000000000002e-05 2024-09-14T23:58:03.190 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.190 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:03.190 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.190 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.190 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:144955 3.6 3:6629293c:::rbd_data.11189263ca38.0000000000000009:2a [read 1466368~0] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-14T23:58:03.190 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:53.416771+0000", 2024-09-14T23:58:03.190 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7081114159999995, 2024-09-14T23:58:03.191 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.038156383000000002, 2024-09-14T23:58:03.191 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:03.191 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:03.191 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:03.191 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:03.191 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:03.191 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:03.191 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 144955 2024-09-14T23:58:03.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.191 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:03.191 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.191 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:03.191 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416771+0000", 2024-09-14T23:58:03.191 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.192 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:03.192 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416771+0000", 2024-09-14T23:58:03.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.192 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:03.192 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416771+0000", 2024-09-14T23:58:03.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5200000000000002e-07 2024-09-14T23:58:03.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.192 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:03.192 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416773+0000", 2024-09-14T23:58:03.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.137e-06 2024-09-14T23:58:03.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.193 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.193 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:03.193 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416780+0000", 2024-09-14T23:58:03.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3820000000000001e-06 2024-09-14T23:58:03.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.193 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.193 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:03.193 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416785+0000", 2024-09-14T23:58:03.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8400000000000002e-06 2024-09-14T23:58:03.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.193 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.193 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:03.193 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.454770+0000", 2024-09-14T23:58:03.194 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.037985473999999998 2024-09-14T23:58:03.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.194 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:03.194 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.454851+0000", 2024-09-14T23:58:03.194 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0519999999999995e-05 2024-09-14T23:58:03.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.194 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:03.194 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.454927+0000", 2024-09-14T23:58:03.194 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6577999999999998e-05 2024-09-14T23:58:03.194 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.194 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:03.194 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.195 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.195 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:144956 3.6 3:6629293c:::rbd_data.11189263ca38.0000000000000009:2a [read 1531904~0] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-14T23:58:03.195 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:53.416808+0000", 2024-09-14T23:58:03.195 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7080747269999996, 2024-09-14T23:58:03.195 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.038244848999999997, 2024-09-14T23:58:03.195 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:03.195 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:03.195 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:03.195 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:03.195 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:03.195 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:03.195 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 144956 2024-09-14T23:58:03.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.196 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:03.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.196 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:03.196 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416808+0000", 2024-09-14T23:58:03.196 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.196 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:03.196 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416808+0000", 2024-09-14T23:58:03.196 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.196 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:03.196 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416808+0000", 2024-09-14T23:58:03.196 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5300000000000004e-07 2024-09-14T23:58:03.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.197 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:03.197 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416810+0000", 2024-09-14T23:58:03.197 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.379e-06 2024-09-14T23:58:03.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.197 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:03.197 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416817+0000", 2024-09-14T23:58:03.197 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3380000000000001e-06 2024-09-14T23:58:03.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.197 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:03.197 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416822+0000", 2024-09-14T23:58:03.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4129999999999997e-06 2024-09-14T23:58:03.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.198 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:03.198 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.454964+0000", 2024-09-14T23:58:03.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.038142084 2024-09-14T23:58:03.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.198 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:03.198 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.455007+0000", 2024-09-14T23:58:03.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3099000000000002e-05 2024-09-14T23:58:03.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.198 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:03.198 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.455052+0000", 2024-09-14T23:58:03.199 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4982999999999999e-05 2024-09-14T23:58:03.199 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.199 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:03.199 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.199 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:144957 3.6 3:65baca07:::rbd_data.11189263ca38.000000000000000a:2a [read 1327104~8192] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-14T23:58:03.199 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:53.416830+0000", 2024-09-14T23:58:03.199 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7080523759999995, 2024-09-14T23:58:03.199 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.038439040000000001, 2024-09-14T23:58:03.199 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:03.199 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:03.199 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:03.199 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:03.200 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:03.200 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:03.200 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 144957 2024-09-14T23:58:03.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.200 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:03.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.200 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:03.200 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416830+0000", 2024-09-14T23:58:03.200 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.200 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:03.200 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416830+0000", 2024-09-14T23:58:03.200 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.201 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:03.201 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416830+0000", 2024-09-14T23:58:03.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3300000000000002e-07 2024-09-14T23:58:03.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.201 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:03.201 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416833+0000", 2024-09-14T23:58:03.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1689999999999999e-06 2024-09-14T23:58:03.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.201 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:03.201 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416839+0000", 2024-09-14T23:58:03.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.285e-06 2024-09-14T23:58:03.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.202 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:03.202 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416844+0000", 2024-09-14T23:58:03.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8359999999999999e-06 2024-09-14T23:58:03.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.202 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:03.202 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.455090+0000", 2024-09-14T23:58:03.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.038246629999999997 2024-09-14T23:58:03.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.202 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:03.202 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.455173+0000", 2024-09-14T23:58:03.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2194999999999995e-05 2024-09-14T23:58:03.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.203 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:03.203 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.455269+0000", 2024-09-14T23:58:03.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6391999999999995e-05 2024-09-14T23:58:03.203 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.203 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:03.203 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.203 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:144958 3.6 3:65baca07:::rbd_data.11189263ca38.000000000000000a:2a [read 1392640~8192] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-14T23:58:03.203 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:53.416852+0000", 2024-09-14T23:58:03.204 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7080301959999993, 2024-09-14T23:58:03.204 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.038556741999999998, 2024-09-14T23:58:03.204 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:03.204 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:03.204 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:03.204 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:03.204 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:03.204 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:03.204 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 144958 2024-09-14T23:58:03.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.204 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:03.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.204 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:03.204 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416852+0000", 2024-09-14T23:58:03.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:03.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416852+0000", 2024-09-14T23:58:03.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:03.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416852+0000", 2024-09-14T23:58:03.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2599999999999998e-07 2024-09-14T23:58:03.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:03.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416855+0000", 2024-09-14T23:58:03.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2050000000000001e-06 2024-09-14T23:58:03.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:03.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416861+0000", 2024-09-14T23:58:03.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2299999999999996e-06 2024-09-14T23:58:03.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:03.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416866+0000", 2024-09-14T23:58:03.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8860000000000003e-06 2024-09-14T23:58:03.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:03.207 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.455301+0000", 2024-09-14T23:58:03.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.038434838999999998 2024-09-14T23:58:03.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.207 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:03.207 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.455349+0000", 2024-09-14T23:58:03.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8795999999999998e-05 2024-09-14T23:58:03.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.207 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:03.207 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.455409+0000", 2024-09-14T23:58:03.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9360000000000001e-05 2024-09-14T23:58:03.207 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.207 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:03.208 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.208 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:144959 3.6 3:65baca07:::rbd_data.11189263ca38.000000000000000a:2a [read 1458176~8192] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-14T23:58:03.208 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:53.416872+0000", 2024-09-14T23:58:03.208 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7080098939999999, 2024-09-14T23:58:03.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.038754086, 2024-09-14T23:58:03.208 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:03.208 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:03.208 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:03.208 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:03.208 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:03.208 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:03.208 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 144959 2024-09-14T23:58:03.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.209 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:03.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.209 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:03.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416872+0000", 2024-09-14T23:58:03.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.209 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:03.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416872+0000", 2024-09-14T23:58:03.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:03.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416873+0000", 2024-09-14T23:58:03.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4400000000000001e-07 2024-09-14T23:58:03.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:03.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416875+0000", 2024-09-14T23:58:03.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0839999999999999e-06 2024-09-14T23:58:03.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:03.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416881+0000", 2024-09-14T23:58:03.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.229e-06 2024-09-14T23:58:03.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:03.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416886+0000", 2024-09-14T23:58:03.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9439999999999996e-06 2024-09-14T23:58:03.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:03.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.455448+0000", 2024-09-14T23:58:03.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.038562328 2024-09-14T23:58:03.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:03.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.455528+0000", 2024-09-14T23:58:03.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9729999999999997e-05 2024-09-14T23:58:03.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:03.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.455626+0000", 2024-09-14T23:58:03.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8426999999999998e-05 2024-09-14T23:58:03.212 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.212 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:03.212 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.212 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:144960 3.6 3:65baca07:::rbd_data.11189263ca38.000000000000000a:2a [read 1523712~8192] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-14T23:58:03.212 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:53.416893+0000", 2024-09-14T23:58:03.212 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7079897269999993, 2024-09-14T23:58:03.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.038862725000000001, 2024-09-14T23:58:03.212 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:03.212 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:03.212 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:03.212 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:03.213 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:03.213 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:03.213 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 144960 2024-09-14T23:58:03.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.213 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:03.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:03.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416893+0000", 2024-09-14T23:58:03.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:03.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416893+0000", 2024-09-14T23:58:03.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:03.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416893+0000", 2024-09-14T23:58:03.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3299999999999998e-07 2024-09-14T23:58:03.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:03.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416907+0000", 2024-09-14T23:58:03.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4569000000000001e-05 2024-09-14T23:58:03.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:03.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416915+0000", 2024-09-14T23:58:03.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7910000000000005e-06 2024-09-14T23:58:03.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:03.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.416923+0000", 2024-09-14T23:58:03.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3459999999999996e-06 2024-09-14T23:58:03.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:03.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.455648+0000", 2024-09-14T23:58:03.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.03872573 2024-09-14T23:58:03.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:03.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.455699+0000", 2024-09-14T23:58:03.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0929e-05 2024-09-14T23:58:03.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:03.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.455755+0000", 2024-09-14T23:58:03.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6026999999999997e-05 2024-09-14T23:58:03.216 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.216 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:03.216 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.216 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.216 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:03.216 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:03.217 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T23:58:03.218 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:03.215+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:03.235 INFO:teuthology.orchestra.run.smithi042.stdout:truncating image image_client.0-clone48 from 0xc25b2e2 (overlap 0xb4558f7) to 0xa24d011 2024-09-14T23:58:03.318 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:03.316+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:03.409 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:03.409 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:03.409 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:58:03.409 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:03.420 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:58:03.417+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:58:03.422 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T23:58:03.522 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:03.521+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:03.601 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:03.601 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:03.601 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:58:03.601 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:58:03.601 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:03.613 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T23:58:03.656 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:03.654+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:03.722 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:03.721+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:03.772 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:03.773 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:58:03.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:58:03.773 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:58:03.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.773 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5881 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:03.773 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375830+0000", 2024-09-14T23:58:03.773 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 84.394769242999999, 2024-09-14T23:58:03.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050523831999999998, 2024-09-14T23:58:03.773 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:03.773 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:03.773 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:03.773 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:03.773 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:03.774 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:03.774 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5881 2024-09-14T23:58:03.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.774 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:03.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:03.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375830+0000", 2024-09-14T23:58:03.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:03.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375830+0000", 2024-09-14T23:58:03.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:03.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375830+0000", 2024-09-14T23:58:03.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.34e-07 2024-09-14T23:58:03.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:03.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375831+0000", 2024-09-14T23:58:03.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9599999999999998e-07 2024-09-14T23:58:03.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:03.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375834+0000", 2024-09-14T23:58:03.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0489999999999999e-06 2024-09-14T23:58:03.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:03.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375838+0000", 2024-09-14T23:58:03.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5319999999999998e-06 2024-09-14T23:58:03.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:03.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426342+0000", 2024-09-14T23:58:03.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050504391000000003 2024-09-14T23:58:03.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:03.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426354+0000", 2024-09-14T23:58:03.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.173e-05 2024-09-14T23:58:03.776 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.776 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:03.777 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.777 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5882 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:03.777 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:58:03.777 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 84.394758639000003, 2024-09-14T23:58:03.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050549666, 2024-09-14T23:58:03.777 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:03.777 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:03.777 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:03.777 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:03.777 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:03.777 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:03.777 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5882 2024-09-14T23:58:03.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.778 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:03.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:03.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:58:03.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:03.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:58:03.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:03.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:58:03.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8099999999999998e-07 2024-09-14T23:58:03.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:03.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375842+0000", 2024-09-14T23:58:03.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9900000000000009e-07 2024-09-14T23:58:03.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:03.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375845+0000", 2024-09-14T23:58:03.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.055e-06 2024-09-14T23:58:03.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:03.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375848+0000", 2024-09-14T23:58:03.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7609999999999999e-06 2024-09-14T23:58:03.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:03.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426381+0000", 2024-09-14T23:58:03.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050533392000000003 2024-09-14T23:58:03.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:03.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426390+0000", 2024-09-14T23:58:03.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0780000000000002e-06 2024-09-14T23:58:03.780 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.780 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:03.780 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.781 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5884 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:03.781 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:58:03.781 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 84.394748499000002, 2024-09-14T23:58:03.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050717439000000003, 2024-09-14T23:58:03.781 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:03.781 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:03.781 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:03.781 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:03.781 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:03.781 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:03.781 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5884 2024-09-14T23:58:03.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.782 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:03.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:03.782 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:58:03.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:03.782 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:58:03.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:03.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:58:03.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0800000000000001e-07 2024-09-14T23:58:03.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:03.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375852+0000", 2024-09-14T23:58:03.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.02e-06 2024-09-14T23:58:03.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:03.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375855+0000", 2024-09-14T23:58:03.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9900000000000002e-06 2024-09-14T23:58:03.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:03.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375857+0000", 2024-09-14T23:58:03.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3499999999999999e-06 2024-09-14T23:58:03.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:03.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426556+0000", 2024-09-14T23:58:03.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050698139000000003 2024-09-14T23:58:03.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:03.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426568+0000", 2024-09-14T23:58:03.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2731999999999999e-05 2024-09-14T23:58:03.785 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.785 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:03.785 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.785 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5885 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:03.785 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:58:03.786 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 84.394729474000002, 2024-09-14T23:58:03.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050737333000000003, 2024-09-14T23:58:03.786 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:03.786 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:03.786 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:03.786 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:03.786 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:03.786 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:03.786 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5885 2024-09-14T23:58:03.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.786 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:03.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:03.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:58:03.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:03.787 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:58:03.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:03.787 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:58:03.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5899999999999998e-07 2024-09-14T23:58:03.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:03.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375872+0000", 2024-09-14T23:58:03.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4219999999999999e-06 2024-09-14T23:58:03.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:03.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375878+0000", 2024-09-14T23:58:03.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4980000000000003e-06 2024-09-14T23:58:03.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:03.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375881+0000", 2024-09-14T23:58:03.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6110000000000001e-06 2024-09-14T23:58:03.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:03.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426596+0000", 2024-09-14T23:58:03.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050715320000000001 2024-09-14T23:58:03.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:03.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426607+0000", 2024-09-14T23:58:03.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1223000000000001e-05 2024-09-14T23:58:03.789 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.789 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:03.789 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.789 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5886 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:03.790 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:58:03.790 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 84.394715360999996, 2024-09-14T23:58:03.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050803395000000001, 2024-09-14T23:58:03.790 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:03.790 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:03.790 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:03.790 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:03.790 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:03.790 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:03.790 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5886 2024-09-14T23:58:03.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.790 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:03.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:03.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:58:03.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:03.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:58:03.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:03.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:58:03.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2700000000000001e-07 2024-09-14T23:58:03.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:03.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375886+0000", 2024-09-14T23:58:03.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7060000000000001e-06 2024-09-14T23:58:03.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:03.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375890+0000", 2024-09-14T23:58:03.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2259999999999999e-06 2024-09-14T23:58:03.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:03.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375893+0000", 2024-09-14T23:58:03.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0819999999999999e-06 2024-09-14T23:58:03.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:03.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426677+0000", 2024-09-14T23:58:03.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050784158000000003 2024-09-14T23:58:03.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:03.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426687+0000", 2024-09-14T23:58:03.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9960000000000005e-06 2024-09-14T23:58:03.794 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.794 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:03.794 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.794 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5887 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:03.794 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:58:03.794 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 84.394693274999995, 2024-09-14T23:58:03.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050833070000000001, 2024-09-14T23:58:03.794 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:03.794 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:03.794 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:03.794 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:03.794 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:03.795 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:03.795 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5887 2024-09-14T23:58:03.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.795 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:03.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:03.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:58:03.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:03.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:58:03.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:03.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:58:03.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3299999999999998e-07 2024-09-14T23:58:03.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.796 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:03.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375908+0000", 2024-09-14T23:58:03.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7689999999999999e-06 2024-09-14T23:58:03.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.796 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:03.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375912+0000", 2024-09-14T23:58:03.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.535e-06 2024-09-14T23:58:03.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.796 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:03.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375916+0000", 2024-09-14T23:58:03.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8979999999999998e-06 2024-09-14T23:58:03.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:03.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426729+0000", 2024-09-14T23:58:03.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050813453000000001 2024-09-14T23:58:03.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:03.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426739+0000", 2024-09-14T23:58:03.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0081999999999999e-05 2024-09-14T23:58:03.797 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.797 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:03.798 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.798 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5888 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:03.798 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:58:03.798 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 84.394680198000003, 2024-09-14T23:58:03.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050854652, 2024-09-14T23:58:03.798 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:03.798 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:03.798 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:03.798 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:03.798 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:03.798 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:03.799 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5888 2024-09-14T23:58:03.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.799 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:03.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:03.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:58:03.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:03.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:58:03.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:03.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:58:03.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1400000000000001e-07 2024-09-14T23:58:03.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:03.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375921+0000", 2024-09-14T23:58:03.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.119e-06 2024-09-14T23:58:03.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:03.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375928+0000", 2024-09-14T23:58:03.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3170000000000004e-06 2024-09-14T23:58:03.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:03.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375931+0000", 2024-09-14T23:58:03.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6970000000000002e-06 2024-09-14T23:58:03.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:03.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426768+0000", 2024-09-14T23:58:03.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050837355000000001 2024-09-14T23:58:03.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:03.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426774+0000", 2024-09-14T23:58:03.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9499999999999998e-06 2024-09-14T23:58:03.801 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.802 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:03.802 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.802 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5889 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:03.802 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:58:03.802 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 84.394663273999996, 2024-09-14T23:58:03.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050885446000000001, 2024-09-14T23:58:03.802 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:03.802 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:03.802 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:03.802 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:03.802 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:03.803 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:03.803 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5889 2024-09-14T23:58:03.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.803 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:03.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:03.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:58:03.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:03.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:58:03.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:03.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:58:03.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.36e-07 2024-09-14T23:58:03.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:03.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375940+0000", 2024-09-14T23:58:03.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.844e-06 2024-09-14T23:58:03.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:03.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375944+0000", 2024-09-14T23:58:03.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6720000000000002e-06 2024-09-14T23:58:03.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:03.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375946+0000", 2024-09-14T23:58:03.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5629999999999999e-06 2024-09-14T23:58:03.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:03.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426810+0000", 2024-09-14T23:58:03.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050863255000000003 2024-09-14T23:58:03.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:03.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426822+0000", 2024-09-14T23:58:03.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1875999999999999e-05 2024-09-14T23:58:03.806 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.806 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:03.806 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.806 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5890 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:03.806 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:58:03.806 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 84.394649462000004, 2024-09-14T23:58:03.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050917549999999999, 2024-09-14T23:58:03.806 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:03.806 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:03.807 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:03.807 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:03.807 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:03.807 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:03.807 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5890 2024-09-14T23:58:03.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.807 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:03.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:03.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:58:03.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:03.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:58:03.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:03.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:58:03.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4299999999999999e-07 2024-09-14T23:58:03.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:03.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375951+0000", 2024-09-14T23:58:03.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1000000000000001e-06 2024-09-14T23:58:03.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:03.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375956+0000", 2024-09-14T23:58:03.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3370000000000001e-06 2024-09-14T23:58:03.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:03.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375958+0000", 2024-09-14T23:58:03.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4820000000000001e-06 2024-09-14T23:58:03.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:03.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426858+0000", 2024-09-14T23:58:03.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050899666000000003 2024-09-14T23:58:03.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:03.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426867+0000", 2024-09-14T23:58:03.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7219999999999994e-06 2024-09-14T23:58:03.810 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.810 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:03.810 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.810 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5891 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:03.810 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:58:03.810 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 84.394638139999998, 2024-09-14T23:58:03.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050948067, 2024-09-14T23:58:03.810 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:03.810 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:03.811 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:03.811 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:03.811 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:03.811 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:03.811 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5891 2024-09-14T23:58:03.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.811 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:03.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:03.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:58:03.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:03.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:58:03.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:03.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:58:03.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0800000000000001e-07 2024-09-14T23:58:03.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:03.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375962+0000", 2024-09-14T23:58:03.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9600000000000008e-07 2024-09-14T23:58:03.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:03.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375970+0000", 2024-09-14T23:58:03.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1600000000000001e-06 2024-09-14T23:58:03.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:03.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375972+0000", 2024-09-14T23:58:03.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4849999999999999e-06 2024-09-14T23:58:03.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:03.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426902+0000", 2024-09-14T23:58:03.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050929813999999997 2024-09-14T23:58:03.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:03.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426909+0000", 2024-09-14T23:58:03.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4039999999999998e-06 2024-09-14T23:58:03.814 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.814 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:03.814 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.814 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5892 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:03.814 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:58:03.814 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 84.394623283000001, 2024-09-14T23:58:03.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050965036999999998, 2024-09-14T23:58:03.814 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:03.814 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:03.814 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:03.815 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:03.815 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:03.815 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:03.815 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5892 2024-09-14T23:58:03.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.815 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:03.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:03.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:58:03.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:03.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:58:03.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:03.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:58:03.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.34e-07 2024-09-14T23:58:03.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:03.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375978+0000", 2024-09-14T23:58:03.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1880000000000001e-06 2024-09-14T23:58:03.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:03.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375981+0000", 2024-09-14T23:58:03.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2399999999999999e-06 2024-09-14T23:58:03.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:03.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375984+0000", 2024-09-14T23:58:03.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9110000000000002e-06 2024-09-14T23:58:03.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:03.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426934+0000", 2024-09-14T23:58:03.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050949912 2024-09-14T23:58:03.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:03.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426941+0000", 2024-09-14T23:58:03.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5519999999999999e-06 2024-09-14T23:58:03.818 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.818 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:03.818 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.818 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5893 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:03.818 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:58:03.818 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 84.394612686000002, 2024-09-14T23:58:03.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050975131999999999, 2024-09-14T23:58:03.818 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:03.818 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:03.819 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:03.819 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:03.819 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:03.819 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:03.819 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5893 2024-09-14T23:58:03.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.819 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:03.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:03.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:58:03.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:03.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:58:03.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:03.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:58:03.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2999999999999999e-07 2024-09-14T23:58:03.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:03.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375988+0000", 2024-09-14T23:58:03.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0410000000000001e-06 2024-09-14T23:58:03.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:03.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375992+0000", 2024-09-14T23:58:03.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.104e-06 2024-09-14T23:58:03.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:03.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375995+0000", 2024-09-14T23:58:03.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4310000000000001e-06 2024-09-14T23:58:03.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:03.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426958+0000", 2024-09-14T23:58:03.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050963642000000003 2024-09-14T23:58:03.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:03.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426962+0000", 2024-09-14T23:58:03.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6839999999999998e-06 2024-09-14T23:58:03.822 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.822 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:03.822 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.822 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5894 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:03.822 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:58:03.822 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 84.394601285999997, 2024-09-14T23:58:03.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050978333000000001, 2024-09-14T23:58:03.822 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:03.823 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:03.823 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:03.823 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:03.823 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:03.823 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:03.823 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5894 2024-09-14T23:58:03.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.823 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:03.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:03.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:58:03.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:03.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:58:03.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:03.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:58:03.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.23e-07 2024-09-14T23:58:03.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:03.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375999+0000", 2024-09-14T23:58:03.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9800000000000002e-07 2024-09-14T23:58:03.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:03.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376007+0000", 2024-09-14T23:58:03.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8539999999999997e-06 2024-09-14T23:58:03.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:03.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376011+0000", 2024-09-14T23:58:03.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.399e-06 2024-09-14T23:58:03.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:03.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426973+0000", 2024-09-14T23:58:03.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050962454999999997 2024-09-14T23:58:03.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:03.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426976+0000", 2024-09-14T23:58:03.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4039999999999999e-06 2024-09-14T23:58:03.826 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.826 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:03.826 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.826 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5895 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:03.826 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376014+0000", 2024-09-14T23:58:03.826 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 84.394584941999994, 2024-09-14T23:58:03.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050975951999999998, 2024-09-14T23:58:03.826 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:03.826 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:03.826 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:03.826 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:03.826 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:03.826 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:03.827 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5895 2024-09-14T23:58:03.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.827 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:03.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:03.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376014+0000", 2024-09-14T23:58:03.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:03.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376014+0000", 2024-09-14T23:58:03.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:03.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376015+0000", 2024-09-14T23:58:03.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2999999999999999e-07 2024-09-14T23:58:03.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:03.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376016+0000", 2024-09-14T23:58:03.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1659999999999999e-06 2024-09-14T23:58:03.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:03.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376024+0000", 2024-09-14T23:58:03.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8930000000000005e-06 2024-09-14T23:58:03.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:03.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376026+0000", 2024-09-14T23:58:03.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6869999999999999e-06 2024-09-14T23:58:03.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:03.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426987+0000", 2024-09-14T23:58:03.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050960601000000001 2024-09-14T23:58:03.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:03.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426990+0000", 2024-09-14T23:58:03.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3749999999999999e-06 2024-09-14T23:58:03.829 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.829 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:03.829 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.830 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5896 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:03.830 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:58:03.830 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 84.394569204999996, 2024-09-14T23:58:03.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050975656000000001, 2024-09-14T23:58:03.830 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:03.830 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:03.830 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:03.830 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:03.830 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:03.830 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:03.830 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5896 2024-09-14T23:58:03.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.830 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:03.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:03.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:58:03.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:03.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:58:03.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:03.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:58:03.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5100000000000001e-07 2024-09-14T23:58:03.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:03.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376032+0000", 2024-09-14T23:58:03.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.141e-06 2024-09-14T23:58:03.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:03.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376036+0000", 2024-09-14T23:58:03.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.25e-06 2024-09-14T23:58:03.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:03.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376038+0000", 2024-09-14T23:58:03.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5469999999999999e-06 2024-09-14T23:58:03.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:03.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427002+0000", 2024-09-14T23:58:03.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050964029000000001 2024-09-14T23:58:03.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:03.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427006+0000", 2024-09-14T23:58:03.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4379999999999999e-06 2024-09-14T23:58:03.833 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.833 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:03.834 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.836 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5897 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:03.836 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:58:03.836 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 84.394557405, 2024-09-14T23:58:03.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050978001000000002, 2024-09-14T23:58:03.836 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:03.836 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:03.836 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:03.836 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:03.837 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:03.837 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:03.837 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5897 2024-09-14T23:58:03.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.837 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:03.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:03.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:58:03.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:03.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:58:03.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:03.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:58:03.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2600000000000001e-07 2024-09-14T23:58:03.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:03.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376043+0000", 2024-09-14T23:58:03.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.054e-06 2024-09-14T23:58:03.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:03.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376047+0000", 2024-09-14T23:58:03.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0659999999999997e-06 2024-09-14T23:58:03.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:03.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376050+0000", 2024-09-14T23:58:03.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.88e-06 2024-09-14T23:58:03.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:03.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427017+0000", 2024-09-14T23:58:03.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050966453000000002 2024-09-14T23:58:03.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:03.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427020+0000", 2024-09-14T23:58:03.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3220000000000001e-06 2024-09-14T23:58:03.840 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.840 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:03.840 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.840 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5898 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:03.840 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376053+0000", 2024-09-14T23:58:03.840 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 84.394545919999999, 2024-09-14T23:58:03.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050982024000000001, 2024-09-14T23:58:03.841 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:03.841 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:03.841 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:03.841 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:03.841 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:03.841 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:03.841 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5898 2024-09-14T23:58:03.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.841 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:03.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:03.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376053+0000", 2024-09-14T23:58:03.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:03.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376053+0000", 2024-09-14T23:58:03.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:03.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376278+0000", 2024-09-14T23:58:03.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022426400000000001 2024-09-14T23:58:03.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:03.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376282+0000", 2024-09-14T23:58:03.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4739999999999999e-06 2024-09-14T23:58:03.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:03.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376296+0000", 2024-09-14T23:58:03.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4229999999999999e-05 2024-09-14T23:58:03.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:03.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376303+0000", 2024-09-14T23:58:03.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2199999999999997e-06 2024-09-14T23:58:03.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:03.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427031+0000", 2024-09-14T23:58:03.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050728154999999997 2024-09-14T23:58:03.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:03.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427036+0000", 2024-09-14T23:58:03.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6809999999999997e-06 2024-09-14T23:58:03.844 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.844 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:03.844 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.844 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5899 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:03.844 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376338+0000", 2024-09-14T23:58:03.845 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 84.394261196000002, 2024-09-14T23:58:03.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050712100000000003, 2024-09-14T23:58:03.845 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:03.845 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:03.845 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:03.845 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:03.845 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:03.845 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:03.845 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5899 2024-09-14T23:58:03.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.845 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:03.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:03.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376338+0000", 2024-09-14T23:58:03.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:03.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376338+0000", 2024-09-14T23:58:03.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:03.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376339+0000", 2024-09-14T23:58:03.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9100000000000003e-07 2024-09-14T23:58:03.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:03.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376342+0000", 2024-09-14T23:58:03.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7e-06 2024-09-14T23:58:03.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:03.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376350+0000", 2024-09-14T23:58:03.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5280000000000003e-06 2024-09-14T23:58:03.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:03.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376356+0000", 2024-09-14T23:58:03.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6010000000000002e-06 2024-09-14T23:58:03.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:03.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427047+0000", 2024-09-14T23:58:03.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050690962999999999 2024-09-14T23:58:03.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:03.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427050+0000", 2024-09-14T23:58:03.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.517e-06 2024-09-14T23:58:03.848 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.848 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:03.848 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.848 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5900 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:03.849 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376366+0000", 2024-09-14T23:58:03.849 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 84.394232936999998, 2024-09-14T23:58:03.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050697858999999998, 2024-09-14T23:58:03.849 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:03.849 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:03.849 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:03.849 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:03.849 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:03.849 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:03.849 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5900 2024-09-14T23:58:03.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.849 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:03.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:03.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376366+0000", 2024-09-14T23:58:03.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:03.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376366+0000", 2024-09-14T23:58:03.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:03.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377650+0000", 2024-09-14T23:58:03.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001283834 2024-09-14T23:58:03.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:03.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377655+0000", 2024-09-14T23:58:03.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7230000000000004e-06 2024-09-14T23:58:03.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:03.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377667+0000", 2024-09-14T23:58:03.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1746000000000001e-05 2024-09-14T23:58:03.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:03.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377676+0000", 2024-09-14T23:58:03.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3619999999999995e-06 2024-09-14T23:58:03.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:03.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427061+0000", 2024-09-14T23:58:03.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049384755000000002 2024-09-14T23:58:03.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:03.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427064+0000", 2024-09-14T23:58:03.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.439e-06 2024-09-14T23:58:03.853 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.853 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:03.853 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.853 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:132622 3.1e 3:79d0a5e0:::rbd_data.111847077d5e.0000000000000007:head [sparse-read 0~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T23:58:03.853 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:44.037901+0000", 2024-09-14T23:58:03.853 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.732698328000001, 2024-09-14T23:58:03.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1083420859999999, 2024-09-14T23:58:03.853 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:03.853 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:03.853 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:03.853 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:03.854 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:03.854 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:03.854 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 132622 2024-09-14T23:58:03.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.854 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:03.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:03.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037901+0000", 2024-09-14T23:58:03.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:03.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037901+0000", 2024-09-14T23:58:03.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:03.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:03.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037903+0000", 2024-09-14T23:58:03.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4619999999999999e-06 2024-09-14T23:58:03.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:03.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037910+0000", 2024-09-14T23:58:03.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7449999999999995e-06 2024-09-14T23:58:03.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:03.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037928+0000", 2024-09-14T23:58:03.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7305000000000001e-05 2024-09-14T23:58:03.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:03.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037939+0000", 2024-09-14T23:58:03.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1599e-05 2024-09-14T23:58:03.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:03.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.038049+0000", 2024-09-14T23:58:03.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000109577 2024-09-14T23:58:03.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op must wait for map", 2024-09-14T23:58:03.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.038057+0000", 2024-09-14T23:58:03.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1450000000000008e-06 2024-09-14T23:58:03.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:03.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.039937+0000", 2024-09-14T23:58:03.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00188024 2024-09-14T23:58:03.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:58:03.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.040039+0000", 2024-09-14T23:58:03.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000101934 2024-09-14T23:58:03.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:03.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:45.145805+0000", 2024-09-14T23:58:03.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.105765651 2024-09-14T23:58:03.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:03.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:45.145997+0000", 2024-09-14T23:58:03.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000192084 2024-09-14T23:58:03.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:03.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:03.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:03.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:45.146243+0000", 2024-09-14T23:58:03.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024634400000000001 2024-09-14T23:58:03.858 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.858 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:03.859 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.859 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:03.859 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:03.859 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:03.860 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T23:58:03.861 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:03.858+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:03.922 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #47, image image_client.0-clone47 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent48 2024-09-14T23:58:03.931 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:58:03.931 INFO:teuthology.orchestra.run.smithi042.stdout:5920 read 0xd1deb4 thru 0xd26085 (0x81d2 bytes) 2024-09-14T23:58:03.936 INFO:teuthology.orchestra.run.smithi042.stdout:5922 trunc from 0xc25b2e2 to 0xc8db3bd 2024-09-14T23:58:03.946 INFO:teuthology.orchestra.run.smithi042.stdout:5923 write 0x4eb5f28 thru 0x4ec589b (0xf974 bytes) 2024-09-14T23:58:03.961 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:03.960+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:03.980 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T23:58:04.057 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:04.058 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:04.058 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:58:04.058 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:04.062 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:04.061+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:04.070 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T23:58:04.116 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is unset 2024-09-14T23:58:04.158 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:58:04.158 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:58:04.163 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:04.160+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:04.177 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:58:04.177 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:58:04.194 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:58:04.194 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:58:04.211 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:58:04.211 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:58:04.238 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:04.238 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:04.244 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:58:04.245 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:58:04.247 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T23:58:04.262 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:58:04.262 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:58:04.264 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:04.262+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:04.279 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:58:04.279 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:58:04.297 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:58:04.297 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:58:04.364 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:04.362+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:04.370 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:04.371 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:04.379 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T23:58:04.465 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:04.463+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:04.486 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:04.486 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:04.495 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T23:58:04.579 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:04.578+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:04.643 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:04.643 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:04.651 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T23:58:04.693 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:04.691+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:04.732 INFO:teuthology.orchestra.run.smithi042.stdout:5925 write 0xa8de727 thru 0xa8e67e3 (0x80bd bytes) 2024-09-14T23:58:04.767 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:04.765+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:04.845 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:04.845 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:04.859 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T23:58:04.906 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:04.904+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:04.969 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:04.967+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:04.974 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:04.974 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:04.983 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T23:58:05.070 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:05.068+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:05.131 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:05.131 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:05.140 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T23:58:05.170 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:05.169+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:05.271 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:05.269+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:05.287 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:05.288 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:05.296 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T23:58:05.372 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:05.370+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:05.403 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:05.404 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:05.412 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T23:58:05.473 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:05.471+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:05.519 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:05.519 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:05.528 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T23:58:05.574 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:05.571+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:05.635 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:05.635 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:05.643 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T23:58:05.685 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:05.682+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:05.776 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:05.774+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:05.791 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:05.791 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:05.800 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T23:58:05.877 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:05.875+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:05.948 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:05.948 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:05.957 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T23:58:06.001 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:05.999+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:06.064 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:06.065 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:06.073 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T23:58:06.114 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:06.112+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:06.121 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T23:58:06.121 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T23:58:06.121 INFO:tasks.thrashosds.thrasher:Removing osd 7, in_osds are: [0, 1, 2, 3, 4, 6, 7, 5] 2024-09-14T23:58:06.121 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 7 2024-09-14T23:58:06.134 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T23:58:06.179 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:06.177+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:06.221 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:06.221 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:06.230 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T23:58:06.308 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T23:58:06.314 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:06.312+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:06.326 INFO:teuthology.orchestra.run.smithi042.stdout:5926 trunc from 0xc8db3bd to 0x6057759 2024-09-14T23:58:06.381 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:06.379+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:06.419 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:06.419 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:06.429 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T23:58:06.496 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:58:06.497 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:58:06.513 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:58:06.511+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:58:06.522 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:58:06.522 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:58:06.550 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:58:06.551 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:58:06.581 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:58:06.581 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:58:06.582 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:06.581+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:06.589 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:06.589 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:06.599 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T23:58:06.612 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:58:06.612 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:58:06.625 INFO:tasks.ceph.mon.a.smithi042.stderr:2024-09-14T23:58:06.623+0000 7f1713676640 -1 mon.a@0(leader).osd e108 definitely_dead 0 2024-09-14T23:58:06.645 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:58:06.645 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:58:06.678 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:58:06.678 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:58:06.683 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:06.681+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:06.710 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:58:06.710 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:58:06.763 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:06.763 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:06.764 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:58:06.764 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:06.776 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T23:58:06.784 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:06.782+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:06.886 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:06.884+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:06.980 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:06.980 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:06.980 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:58:06.980 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:58:06.980 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:06.986 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:58:06.985+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:58:06.993 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T23:58:07.088 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:07.086+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:07.127 INFO:teuthology.orchestra.run.smithi042.stderr:marked out osd.7. 2024-09-14T23:58:07.129 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is set 2024-09-14T23:58:07.188 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:58:07.186+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:58:07.200 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:07.200 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:58:07.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:58:07.201 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:58:07.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.201 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:43054 3.1f 3:ffbfe596:::rbd_data.1118350d5ae5.00000000000002eb:head [set-alloc-hint object_size 262144 write_size 262144,write 196608~22770] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T23:58:07.201 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:58:07.201 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 74.230517277000004, 2024-09-14T23:58:07.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.99973334400000002, 2024-09-14T23:58:07.201 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:07.201 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:07.201 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:07.201 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:07.201 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:07.202 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:07.202 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 43054 2024-09-14T23:58:07.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.202 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:07.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.202 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:07.202 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:58:07.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.202 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:07.202 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:58:07.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.203 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:07.203 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966552+0000", 2024-09-14T23:58:07.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3790000000000001e-06 2024-09-14T23:58:07.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.203 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:07.203 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966565+0000", 2024-09-14T23:58:07.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2459999999999999e-05 2024-09-14T23:58:07.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.203 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:07.203 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966576+0000", 2024-09-14T23:58:07.204 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1039000000000001e-05 2024-09-14T23:58:07.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.204 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:07.204 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966582+0000", 2024-09-14T23:58:07.204 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9660000000000001e-06 2024-09-14T23:58:07.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.204 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:07.204 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966625+0000", 2024-09-14T23:58:07.204 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3529999999999998e-05 2024-09-14T23:58:07.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.204 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:58:07.204 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966632+0000", 2024-09-14T23:58:07.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9809999999999997e-06 2024-09-14T23:58:07.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:07.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.964460+0000", 2024-09-14T23:58:07.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.99782768300000002 2024-09-14T23:58:07.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:07.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.964641+0000", 2024-09-14T23:58:07.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000181089 2024-09-14T23:58:07.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-14T23:58:07.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.964809+0000", 2024-09-14T23:58:07.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000168505 2024-09-14T23:58:07.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:58:07.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966030+0000", 2024-09-14T23:58:07.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012206929999999999 2024-09-14T23:58:07.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:07.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966181+0000", 2024-09-14T23:58:07.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000150569 2024-09-14T23:58:07.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.207 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:07.207 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966243+0000", 2024-09-14T23:58:07.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2331999999999999e-05 2024-09-14T23:58:07.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.207 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:07.207 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966284+0000", 2024-09-14T23:58:07.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1118000000000002e-05 2024-09-14T23:58:07.207 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.207 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:07.207 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.208 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184314 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4162560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:07.208 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.077247+0000", 2024-09-14T23:58:07.208 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.11982060899999999, 2024-09-14T23:58:07.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058772256000000002, 2024-09-14T23:58:07.208 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:07.208 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:07.208 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:07.208 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:07.208 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:07.208 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:07.208 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184314 2024-09-14T23:58:07.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.208 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:07.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.209 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:07.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.077247+0000", 2024-09-14T23:58:07.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.209 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:07.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.077247+0000", 2024-09-14T23:58:07.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.209 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:07.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.078906+0000", 2024-09-14T23:58:07.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016583279999999999 2024-09-14T23:58:07.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:07.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.078911+0000", 2024-09-14T23:58:07.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0420000000000002e-06 2024-09-14T23:58:07.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:07.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.078922+0000", 2024-09-14T23:58:07.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1354e-05 2024-09-14T23:58:07.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:07.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.078930+0000", 2024-09-14T23:58:07.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5730000000000002e-06 2024-09-14T23:58:07.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:07.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.135870+0000", 2024-09-14T23:58:07.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056940651000000002 2024-09-14T23:58:07.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:07.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.135951+0000", 2024-09-14T23:58:07.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0890999999999995e-05 2024-09-14T23:58:07.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:07.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.136020+0000", 2024-09-14T23:58:07.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8417000000000006e-05 2024-09-14T23:58:07.212 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.212 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:07.212 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.212 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184323 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4183040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:07.212 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.079228+0000", 2024-09-14T23:58:07.212 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.117839535, 2024-09-14T23:58:07.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058454487999999999, 2024-09-14T23:58:07.212 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:07.212 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:07.212 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:07.212 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:07.213 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:07.213 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:07.213 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184323 2024-09-14T23:58:07.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.213 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:07.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:07.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079228+0000", 2024-09-14T23:58:07.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:07.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079228+0000", 2024-09-14T23:58:07.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:07.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079229+0000", 2024-09-14T23:58:07.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5299999999999999e-07 2024-09-14T23:58:07.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:07.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079232+0000", 2024-09-14T23:58:07.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8839999999999998e-06 2024-09-14T23:58:07.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:07.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079238+0000", 2024-09-14T23:58:07.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2299999999999996e-06 2024-09-14T23:58:07.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:07.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079243+0000", 2024-09-14T23:58:07.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3029999999999999e-06 2024-09-14T23:58:07.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:07.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.137533+0000", 2024-09-14T23:58:07.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058289674999999999 2024-09-14T23:58:07.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:07.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.137615+0000", 2024-09-14T23:58:07.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2288999999999998e-05 2024-09-14T23:58:07.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:07.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.137683+0000", 2024-09-14T23:58:07.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7654e-05 2024-09-14T23:58:07.216 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.216 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:07.216 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.217 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184324 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4073472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:07.217 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.079251+0000", 2024-09-14T23:58:07.217 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.117817068, 2024-09-14T23:58:07.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058609383000000001, 2024-09-14T23:58:07.217 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:07.217 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:07.217 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:07.217 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:07.217 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:07.217 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:07.217 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184324 2024-09-14T23:58:07.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.217 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:07.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.218 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:07.218 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079251+0000", 2024-09-14T23:58:07.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.218 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:07.218 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079251+0000", 2024-09-14T23:58:07.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.218 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:07.218 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.080920+0000", 2024-09-14T23:58:07.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016687430000000001 2024-09-14T23:58:07.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:07.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.080924+0000", 2024-09-14T23:58:07.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3869999999999997e-06 2024-09-14T23:58:07.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:07.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.080935+0000", 2024-09-14T23:58:07.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0515e-05 2024-09-14T23:58:07.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:07.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.080942+0000", 2024-09-14T23:58:07.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0990000000000001e-06 2024-09-14T23:58:07.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:07.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.137713+0000", 2024-09-14T23:58:07.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056771288000000003 2024-09-14T23:58:07.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:07.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.137794+0000", 2024-09-14T23:58:07.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0613999999999998e-05 2024-09-14T23:58:07.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:07.221 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.137860+0000", 2024-09-14T23:58:07.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6736999999999995e-05 2024-09-14T23:58:07.221 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.221 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:07.221 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.221 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184346 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4099072~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:07.221 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.083616+0000", 2024-09-14T23:58:07.221 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.113451883, 2024-09-14T23:58:07.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058557419999999999, 2024-09-14T23:58:07.221 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:07.221 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:07.222 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:07.222 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:07.222 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:07.222 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:07.222 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184346 2024-09-14T23:58:07.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.222 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:07.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.222 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:07.222 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083616+0000", 2024-09-14T23:58:07.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.222 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:07.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083616+0000", 2024-09-14T23:58:07.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:07.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083620+0000", 2024-09-14T23:58:07.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0119999999999999e-06 2024-09-14T23:58:07.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:07.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083623+0000", 2024-09-14T23:58:07.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9059999999999998e-06 2024-09-14T23:58:07.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:07.224 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083631+0000", 2024-09-14T23:58:07.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2679999999999994e-06 2024-09-14T23:58:07.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.224 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:07.224 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083637+0000", 2024-09-14T23:58:07.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1210000000000003e-06 2024-09-14T23:58:07.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.224 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:07.224 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142018+0000", 2024-09-14T23:58:07.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058381111999999999 2024-09-14T23:58:07.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.224 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:07.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142100+0000", 2024-09-14T23:58:07.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1308000000000001e-05 2024-09-14T23:58:07.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.225 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:07.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142173+0000", 2024-09-14T23:58:07.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3693000000000001e-05 2024-09-14T23:58:07.225 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.225 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:07.225 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.225 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184347 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4122624~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:07.225 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.083647+0000", 2024-09-14T23:58:07.226 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.11342105299999999, 2024-09-14T23:58:07.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058706388999999998, 2024-09-14T23:58:07.226 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:07.226 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:07.226 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:07.226 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:07.226 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:07.226 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:07.226 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184347 2024-09-14T23:58:07.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.226 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:07.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:07.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083647+0000", 2024-09-14T23:58:07.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:07.227 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083647+0000", 2024-09-14T23:58:07.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:07.227 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083647+0000", 2024-09-14T23:58:07.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3099999999999998e-07 2024-09-14T23:58:07.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:07.227 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083650+0000", 2024-09-14T23:58:07.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2900000000000001e-06 2024-09-14T23:58:07.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.228 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:07.228 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083657+0000", 2024-09-14T23:58:07.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.486e-06 2024-09-14T23:58:07.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.228 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:07.228 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083663+0000", 2024-09-14T23:58:07.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4330000000000003e-06 2024-09-14T23:58:07.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.229 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:07.229 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142206+0000", 2024-09-14T23:58:07.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058543237999999997 2024-09-14T23:58:07.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.229 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:07.229 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142284+0000", 2024-09-14T23:58:07.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8487999999999998e-05 2024-09-14T23:58:07.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.229 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:07.229 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142353+0000", 2024-09-14T23:58:07.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9023000000000006e-05 2024-09-14T23:58:07.229 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.230 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:07.230 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.230 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184348 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4103168~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:07.230 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.083669+0000", 2024-09-14T23:58:07.230 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.113398636, 2024-09-14T23:58:07.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058864232000000002, 2024-09-14T23:58:07.230 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:07.230 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:07.230 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:07.230 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:07.230 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:07.230 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:07.230 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184348 2024-09-14T23:58:07.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.231 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:07.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.231 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:07.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083669+0000", 2024-09-14T23:58:07.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.231 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:07.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083669+0000", 2024-09-14T23:58:07.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.231 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:07.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083670+0000", 2024-09-14T23:58:07.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9999999999999998e-07 2024-09-14T23:58:07.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.232 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:07.232 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083672+0000", 2024-09-14T23:58:07.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1430000000000001e-06 2024-09-14T23:58:07.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.232 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:07.232 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083678+0000", 2024-09-14T23:58:07.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5200000000000003e-06 2024-09-14T23:58:07.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.232 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:07.233 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083683+0000", 2024-09-14T23:58:07.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9359999999999998e-06 2024-09-14T23:58:07.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.233 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:07.233 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142384+0000", 2024-09-14T23:58:07.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058701159000000003 2024-09-14T23:58:07.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.233 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:07.233 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142466+0000", 2024-09-14T23:58:07.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1024000000000002e-05 2024-09-14T23:58:07.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.234 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:07.234 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142534+0000", 2024-09-14T23:58:07.234 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8050000000000001e-05 2024-09-14T23:58:07.234 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.234 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:07.234 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.234 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184349 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4126720~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:07.234 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.083690+0000", 2024-09-14T23:58:07.234 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.113378101, 2024-09-14T23:58:07.234 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059019911000000001, 2024-09-14T23:58:07.234 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:07.235 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:07.235 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:07.235 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:07.235 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:07.235 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:07.235 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184349 2024-09-14T23:58:07.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.235 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:07.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.235 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:07.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083690+0000", 2024-09-14T23:58:07.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.235 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:07.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083690+0000", 2024-09-14T23:58:07.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.236 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:07.236 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.084900+0000", 2024-09-14T23:58:07.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012099770000000001 2024-09-14T23:58:07.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.236 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:07.236 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.084905+0000", 2024-09-14T23:58:07.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.028e-06 2024-09-14T23:58:07.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.236 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:07.236 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.084914+0000", 2024-09-14T23:58:07.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6619999999999994e-06 2024-09-14T23:58:07.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:07.237 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.084922+0000", 2024-09-14T23:58:07.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5720000000000003e-06 2024-09-14T23:58:07.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:07.237 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142565+0000", 2024-09-14T23:58:07.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057642605999999999 2024-09-14T23:58:07.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:07.238 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142644+0000", 2024-09-14T23:58:07.238 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9370999999999996e-05 2024-09-14T23:58:07.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.238 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:07.238 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142710+0000", 2024-09-14T23:58:07.238 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5695e-05 2024-09-14T23:58:07.238 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.238 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:07.238 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.238 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184354 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4115456~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:07.238 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.085029+0000", 2024-09-14T23:58:07.238 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.112039451, 2024-09-14T23:58:07.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058626327999999998, 2024-09-14T23:58:07.239 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:07.239 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:07.239 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:07.239 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:07.239 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:07.239 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:07.239 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184354 2024-09-14T23:58:07.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.239 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:07.239 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.239 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:07.239 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085029+0000", 2024-09-14T23:58:07.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.240 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:07.240 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085029+0000", 2024-09-14T23:58:07.240 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.240 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:07.240 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085029+0000", 2024-09-14T23:58:07.240 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0800000000000001e-07 2024-09-14T23:58:07.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.240 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:07.240 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085030+0000", 2024-09-14T23:58:07.240 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.226e-06 2024-09-14T23:58:07.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.241 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.241 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:07.241 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085033+0000", 2024-09-14T23:58:07.241 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9569999999999998e-06 2024-09-14T23:58:07.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.241 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.241 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:07.241 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085035+0000", 2024-09-14T23:58:07.241 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.587e-06 2024-09-14T23:58:07.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.241 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.241 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:07.242 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143491+0000", 2024-09-14T23:58:07.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058455337000000003 2024-09-14T23:58:07.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.242 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:07.242 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143575+0000", 2024-09-14T23:58:07.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3850000000000005e-05 2024-09-14T23:58:07.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.242 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:07.242 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143655+0000", 2024-09-14T23:58:07.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0162999999999998e-05 2024-09-14T23:58:07.242 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.242 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:07.243 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.243 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.243 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184355 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4139008~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:07.243 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.085039+0000", 2024-09-14T23:58:07.243 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.11202904700000001, 2024-09-14T23:58:07.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058801369999999999, 2024-09-14T23:58:07.243 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:07.243 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:07.243 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:07.243 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:07.243 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:07.243 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:07.243 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184355 2024-09-14T23:58:07.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.244 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:07.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:07.244 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085039+0000", 2024-09-14T23:58:07.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:07.244 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085039+0000", 2024-09-14T23:58:07.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:07.245 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085039+0000", 2024-09-14T23:58:07.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.03e-07 2024-09-14T23:58:07.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.245 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:07.245 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085040+0000", 2024-09-14T23:58:07.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.158e-06 2024-09-14T23:58:07.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.245 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:07.245 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085043+0000", 2024-09-14T23:58:07.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0240000000000002e-06 2024-09-14T23:58:07.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:07.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085046+0000", 2024-09-14T23:58:07.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4600000000000002e-06 2024-09-14T23:58:07.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:07.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143688+0000", 2024-09-14T23:58:07.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058642356 2024-09-14T23:58:07.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:07.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143766+0000", 2024-09-14T23:58:07.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8219000000000004e-05 2024-09-14T23:58:07.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:07.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143840+0000", 2024-09-14T23:58:07.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3949999999999995e-05 2024-09-14T23:58:07.247 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.247 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:07.247 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.247 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184356 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4119552~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:07.247 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.085049+0000", 2024-09-14T23:58:07.247 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.112018876, 2024-09-14T23:58:07.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058977992, 2024-09-14T23:58:07.247 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:07.247 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:07.248 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:07.248 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:07.248 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:07.248 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:07.248 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184356 2024-09-14T23:58:07.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.248 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:07.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.248 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:07.248 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085049+0000", 2024-09-14T23:58:07.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.248 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:07.249 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085049+0000", 2024-09-14T23:58:07.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.249 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:07.249 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.086934+0000", 2024-09-14T23:58:07.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018854189999999999 2024-09-14T23:58:07.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.249 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:07.249 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.086939+0000", 2024-09-14T23:58:07.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.493e-06 2024-09-14T23:58:07.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.250 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:07.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.086951+0000", 2024-09-14T23:58:07.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1969e-05 2024-09-14T23:58:07.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.250 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:07.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.086959+0000", 2024-09-14T23:58:07.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5780000000000001e-06 2024-09-14T23:58:07.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.250 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:07.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143871+0000", 2024-09-14T23:58:07.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056912114 2024-09-14T23:58:07.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:07.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143951+0000", 2024-09-14T23:58:07.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9938000000000006e-05 2024-09-14T23:58:07.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:07.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.144027+0000", 2024-09-14T23:58:07.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6481000000000002e-05 2024-09-14T23:58:07.251 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.251 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:07.251 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.251 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184364 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4135936~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:07.251 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.087207+0000", 2024-09-14T23:58:07.252 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.109860596, 2024-09-14T23:58:07.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058466420999999998, 2024-09-14T23:58:07.252 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:07.252 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:07.252 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:07.252 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:07.252 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:07.252 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:07.252 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184364 2024-09-14T23:58:07.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.252 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:07.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:07.253 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087207+0000", 2024-09-14T23:58:07.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.253 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.253 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:07.253 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087207+0000", 2024-09-14T23:58:07.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.253 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.253 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:07.253 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087208+0000", 2024-09-14T23:58:07.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.35e-07 2024-09-14T23:58:07.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.253 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.253 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:07.254 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087209+0000", 2024-09-14T23:58:07.254 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.573e-06 2024-09-14T23:58:07.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.254 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:07.254 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087212+0000", 2024-09-14T23:58:07.254 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2770000000000001e-06 2024-09-14T23:58:07.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.254 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:07.254 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087215+0000", 2024-09-14T23:58:07.254 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5500000000000001e-06 2024-09-14T23:58:07.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:07.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145520+0000", 2024-09-14T23:58:07.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058304835999999999 2024-09-14T23:58:07.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:07.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145602+0000", 2024-09-14T23:58:07.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1714999999999999e-05 2024-09-14T23:58:07.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:07.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145674+0000", 2024-09-14T23:58:07.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2235000000000002e-05 2024-09-14T23:58:07.255 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.256 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:07.256 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.256 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184365 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4159488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:07.256 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.087219+0000", 2024-09-14T23:58:07.256 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.109848565, 2024-09-14T23:58:07.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058643533999999997, 2024-09-14T23:58:07.256 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:07.256 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:07.256 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:07.256 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:07.256 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:07.257 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:07.257 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184365 2024-09-14T23:58:07.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.257 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:07.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:07.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087219+0000", 2024-09-14T23:58:07.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:07.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087219+0000", 2024-09-14T23:58:07.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:07.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087220+0000", 2024-09-14T23:58:07.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.39e-07 2024-09-14T23:58:07.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:07.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087222+0000", 2024-09-14T23:58:07.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0609999999999998e-06 2024-09-14T23:58:07.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:07.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087228+0000", 2024-09-14T23:58:07.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8590000000000001e-06 2024-09-14T23:58:07.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.259 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:07.259 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087232+0000", 2024-09-14T23:58:07.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8149999999999996e-06 2024-09-14T23:58:07.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.259 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:07.259 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145705+0000", 2024-09-14T23:58:07.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058472532000000001 2024-09-14T23:58:07.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.259 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:07.259 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145784+0000", 2024-09-14T23:58:07.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9382000000000007e-05 2024-09-14T23:58:07.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:07.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145863+0000", 2024-09-14T23:58:07.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8546000000000003e-05 2024-09-14T23:58:07.260 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.260 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:07.260 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.260 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184366 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4140032~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:07.260 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.087240+0000", 2024-09-14T23:58:07.261 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.109828117, 2024-09-14T23:58:07.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058818068000000001, 2024-09-14T23:58:07.261 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:07.261 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:07.261 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:07.261 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:07.261 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:07.261 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:07.261 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184366 2024-09-14T23:58:07.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.261 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:07.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:07.261 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087240+0000", 2024-09-14T23:58:07.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:07.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087240+0000", 2024-09-14T23:58:07.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:07.263 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.088904+0000", 2024-09-14T23:58:07.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001664315 2024-09-14T23:58:07.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.263 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:07.263 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.088909+0000", 2024-09-14T23:58:07.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7709999999999997e-06 2024-09-14T23:58:07.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.263 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:07.263 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.088921+0000", 2024-09-14T23:58:07.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1661999999999999e-05 2024-09-14T23:58:07.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:07.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.088928+0000", 2024-09-14T23:58:07.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8189999999999992e-06 2024-09-14T23:58:07.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:07.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145901+0000", 2024-09-14T23:58:07.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056972478 2024-09-14T23:58:07.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:07.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145977+0000", 2024-09-14T23:58:07.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5763e-05 2024-09-14T23:58:07.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.265 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:07.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.146058+0000", 2024-09-14T23:58:07.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1260000000000005e-05 2024-09-14T23:58:07.265 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.265 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:07.265 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.265 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184374 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4156416~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:07.265 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.089161+0000", 2024-09-14T23:58:07.265 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.107907408, 2024-09-14T23:58:07.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058445777999999997, 2024-09-14T23:58:07.265 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:07.266 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:07.266 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:07.266 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:07.266 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:07.266 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:07.266 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184374 2024-09-14T23:58:07.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.266 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:07.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:07.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089161+0000", 2024-09-14T23:58:07.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:07.267 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089161+0000", 2024-09-14T23:58:07.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.267 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:07.267 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089161+0000", 2024-09-14T23:58:07.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.96e-07 2024-09-14T23:58:07.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.267 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:07.267 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089168+0000", 2024-09-14T23:58:07.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8589999999999999e-06 2024-09-14T23:58:07.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:07.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089183+0000", 2024-09-14T23:58:07.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4878999999999999e-05 2024-09-14T23:58:07.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:07.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089190+0000", 2024-09-14T23:58:07.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.576e-06 2024-09-14T23:58:07.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:07.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147449+0000", 2024-09-14T23:58:07.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058258786999999999 2024-09-14T23:58:07.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:07.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147535+0000", 2024-09-14T23:58:07.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6223000000000004e-05 2024-09-14T23:58:07.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:07.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147606+0000", 2024-09-14T23:58:07.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1057999999999996e-05 2024-09-14T23:58:07.269 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.269 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:07.269 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.270 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184375 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4179968~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:07.270 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.089199+0000", 2024-09-14T23:58:07.270 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.10786878, 2024-09-14T23:58:07.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058597445999999997, 2024-09-14T23:58:07.270 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:07.270 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:07.270 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:07.270 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:07.270 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:07.270 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:07.270 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184375 2024-09-14T23:58:07.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.270 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:07.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:07.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089199+0000", 2024-09-14T23:58:07.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:07.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089199+0000", 2024-09-14T23:58:07.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:07.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089200+0000", 2024-09-14T23:58:07.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8100000000000003e-07 2024-09-14T23:58:07.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:07.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089202+0000", 2024-09-14T23:58:07.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7599999999999998e-06 2024-09-14T23:58:07.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:07.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089209+0000", 2024-09-14T23:58:07.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7279999999999998e-06 2024-09-14T23:58:07.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:07.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089215+0000", 2024-09-14T23:58:07.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5090000000000001e-06 2024-09-14T23:58:07.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:07.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147638+0000", 2024-09-14T23:58:07.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058423105000000003 2024-09-14T23:58:07.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:07.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147719+0000", 2024-09-14T23:58:07.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1217000000000005e-05 2024-09-14T23:58:07.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:07.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147797+0000", 2024-09-14T23:58:07.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7645999999999995e-05 2024-09-14T23:58:07.274 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.274 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:07.274 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.274 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184376 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4160512~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:07.274 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.089232+0000", 2024-09-14T23:58:07.274 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.10783629, 2024-09-14T23:58:07.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058751338, 2024-09-14T23:58:07.274 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:07.274 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:07.274 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:07.274 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:07.275 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:07.275 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:07.275 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184376 2024-09-14T23:58:07.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.275 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:07.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:07.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089232+0000", 2024-09-14T23:58:07.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:07.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089232+0000", 2024-09-14T23:58:07.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.276 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:07.276 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089232+0000", 2024-09-14T23:58:07.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5600000000000001e-07 2024-09-14T23:58:07.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.276 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:07.276 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089235+0000", 2024-09-14T23:58:07.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5639999999999999e-06 2024-09-14T23:58:07.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.276 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:07.276 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089242+0000", 2024-09-14T23:58:07.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0369999999999997e-06 2024-09-14T23:58:07.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:07.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089247+0000", 2024-09-14T23:58:07.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2079999999999999e-06 2024-09-14T23:58:07.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:07.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147831+0000", 2024-09-14T23:58:07.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058583853999999998 2024-09-14T23:58:07.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:07.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147915+0000", 2024-09-14T23:58:07.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4208000000000004e-05 2024-09-14T23:58:07.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:07.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147983+0000", 2024-09-14T23:58:07.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8010999999999996e-05 2024-09-14T23:58:07.278 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.278 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:07.278 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.278 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184377 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4184064~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:07.279 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.089255+0000", 2024-09-14T23:58:07.279 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.10781273399999999, 2024-09-14T23:58:07.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058906100000000003, 2024-09-14T23:58:07.279 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:07.279 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:07.279 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:07.279 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:07.279 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:07.279 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:07.279 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184377 2024-09-14T23:58:07.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.279 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:07.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:07.280 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089255+0000", 2024-09-14T23:58:07.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.280 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:07.280 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089255+0000", 2024-09-14T23:58:07.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.280 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:07.280 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.090903+0000", 2024-09-14T23:58:07.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001647871 2024-09-14T23:58:07.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:07.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.090908+0000", 2024-09-14T23:58:07.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7199999999999997e-06 2024-09-14T23:58:07.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:07.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.090919+0000", 2024-09-14T23:58:07.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1472e-05 2024-09-14T23:58:07.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:07.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.090927+0000", 2024-09-14T23:58:07.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0390000000000005e-06 2024-09-14T23:58:07.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:07.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.148013+0000", 2024-09-14T23:58:07.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057085662000000002 2024-09-14T23:58:07.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:07.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.148090+0000", 2024-09-14T23:58:07.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6778000000000002e-05 2024-09-14T23:58:07.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:07.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.148161+0000", 2024-09-14T23:58:07.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1557999999999995e-05 2024-09-14T23:58:07.283 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.283 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:07.283 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.283 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184386 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4185088~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:07.283 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.091063+0000", 2024-09-14T23:58:07.283 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.10600507000000001, 2024-09-14T23:58:07.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058603404999999997, 2024-09-14T23:58:07.283 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:07.283 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:07.283 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:07.283 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:07.283 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:07.284 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:07.284 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184386 2024-09-14T23:58:07.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.284 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:07.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:07.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091063+0000", 2024-09-14T23:58:07.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:07.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091063+0000", 2024-09-14T23:58:07.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.285 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:07.285 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091063+0000", 2024-09-14T23:58:07.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1899999999999999e-07 2024-09-14T23:58:07.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.285 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:07.285 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091064+0000", 2024-09-14T23:58:07.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2699999999999999e-06 2024-09-14T23:58:07.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.285 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:07.285 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091067+0000", 2024-09-14T23:58:07.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.971e-06 2024-09-14T23:58:07.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:07.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091070+0000", 2024-09-14T23:58:07.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.525e-06 2024-09-14T23:58:07.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:07.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.149510+0000", 2024-09-14T23:58:07.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058439758000000001 2024-09-14T23:58:07.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:07.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.149595+0000", 2024-09-14T23:58:07.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5146000000000001e-05 2024-09-14T23:58:07.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.287 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:07.287 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.149666+0000", 2024-09-14T23:58:07.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1515999999999998e-05 2024-09-14T23:58:07.287 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.287 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:07.287 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.287 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184388 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4189184~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:07.287 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.091073+0000", 2024-09-14T23:58:07.288 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.105994795, 2024-09-14T23:58:07.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058781609999999998, 2024-09-14T23:58:07.288 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:07.288 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:07.288 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:07.288 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:07.288 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:07.288 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:07.288 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184388 2024-09-14T23:58:07.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.288 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:07.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:07.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091073+0000", 2024-09-14T23:58:07.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:07.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091073+0000", 2024-09-14T23:58:07.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:07.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.092903+0000", 2024-09-14T23:58:07.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018296289999999999 2024-09-14T23:58:07.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:07.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.092910+0000", 2024-09-14T23:58:07.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7510000000000002e-06 2024-09-14T23:58:07.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:07.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.092918+0000", 2024-09-14T23:58:07.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7420000000000003e-06 2024-09-14T23:58:07.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:07.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.092925+0000", 2024-09-14T23:58:07.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0360000000000001e-06 2024-09-14T23:58:07.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:07.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.149700+0000", 2024-09-14T23:58:07.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056774340999999999 2024-09-14T23:58:07.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:07.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.149784+0000", 2024-09-14T23:58:07.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4393000000000003e-05 2024-09-14T23:58:07.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:07.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.149855+0000", 2024-09-14T23:58:07.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0717999999999996e-05 2024-09-14T23:58:07.291 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.292 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:07.292 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.292 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.292 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:07.292 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:07.293 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T23:58:07.294 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:58:07.291+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:58:07.394 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:07.392+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:07.461 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:07.461 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:07.461 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:58:07.461 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:07.470 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T23:58:07.513 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:07.512+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:07.577 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:07.577 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:07.577 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:58:07.577 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:58:07.578 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:07.586 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T23:58:07.627 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:07.625+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:07.697 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:07.695+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:07.734 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:07.734 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:58:07.734 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:58:07.735 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:58:07.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.735 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5881 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:07.735 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375830+0000", 2024-09-14T23:58:07.735 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 88.356602801999998, 2024-09-14T23:58:07.735 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050523831999999998, 2024-09-14T23:58:07.735 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:07.735 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:07.735 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:07.735 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:07.735 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:07.735 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:07.735 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5881 2024-09-14T23:58:07.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.736 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:07.736 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.736 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:07.736 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375830+0000", 2024-09-14T23:58:07.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.736 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.736 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:07.736 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375830+0000", 2024-09-14T23:58:07.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.736 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.736 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:07.737 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375830+0000", 2024-09-14T23:58:07.737 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.34e-07 2024-09-14T23:58:07.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.737 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:07.737 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375831+0000", 2024-09-14T23:58:07.737 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.9599999999999998e-07 2024-09-14T23:58:07.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.737 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:07.737 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375834+0000", 2024-09-14T23:58:07.737 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0489999999999999e-06 2024-09-14T23:58:07.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.737 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:07.738 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375838+0000", 2024-09-14T23:58:07.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5319999999999998e-06 2024-09-14T23:58:07.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.738 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.738 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:07.738 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426342+0000", 2024-09-14T23:58:07.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050504391000000003 2024-09-14T23:58:07.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.738 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.738 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:07.738 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426354+0000", 2024-09-14T23:58:07.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.173e-05 2024-09-14T23:58:07.738 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.738 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:07.739 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.739 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5882 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:07.739 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:58:07.739 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 88.356592198000001, 2024-09-14T23:58:07.739 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050549666, 2024-09-14T23:58:07.739 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:07.739 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:07.739 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:07.739 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:07.739 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:07.739 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:07.739 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5882 2024-09-14T23:58:07.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.740 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:07.740 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.740 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:07.740 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:58:07.740 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.741 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:07.741 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:58:07.741 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.741 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:07.741 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:58:07.741 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8099999999999998e-07 2024-09-14T23:58:07.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.741 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:07.741 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375842+0000", 2024-09-14T23:58:07.742 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9900000000000009e-07 2024-09-14T23:58:07.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.742 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:07.742 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375845+0000", 2024-09-14T23:58:07.742 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.055e-06 2024-09-14T23:58:07.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.742 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:07.742 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375848+0000", 2024-09-14T23:58:07.742 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7609999999999999e-06 2024-09-14T23:58:07.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.742 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:07.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426381+0000", 2024-09-14T23:58:07.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050533392000000003 2024-09-14T23:58:07.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:07.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426390+0000", 2024-09-14T23:58:07.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0780000000000002e-06 2024-09-14T23:58:07.743 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.743 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:07.744 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.744 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5884 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:07.744 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:58:07.744 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 88.356582058000001, 2024-09-14T23:58:07.744 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050717439000000003, 2024-09-14T23:58:07.744 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:07.744 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:07.744 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:07.744 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:07.744 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:07.744 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:07.744 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5884 2024-09-14T23:58:07.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.745 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:07.745 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.745 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:07.745 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:58:07.745 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.745 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.745 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:07.745 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:58:07.745 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.745 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.745 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:07.746 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:58:07.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0800000000000001e-07 2024-09-14T23:58:07.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.746 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:07.746 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375852+0000", 2024-09-14T23:58:07.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.02e-06 2024-09-14T23:58:07.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.746 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:07.746 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375855+0000", 2024-09-14T23:58:07.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9900000000000002e-06 2024-09-14T23:58:07.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.746 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:07.747 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375857+0000", 2024-09-14T23:58:07.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3499999999999999e-06 2024-09-14T23:58:07.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.747 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:07.747 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426556+0000", 2024-09-14T23:58:07.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050698139000000003 2024-09-14T23:58:07.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.747 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:07.747 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426568+0000", 2024-09-14T23:58:07.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2731999999999999e-05 2024-09-14T23:58:07.747 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.747 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:07.747 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.748 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5885 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:07.748 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:58:07.748 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 88.356563033, 2024-09-14T23:58:07.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050737333000000003, 2024-09-14T23:58:07.748 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:07.748 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:07.748 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:07.748 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:07.748 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:07.748 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:07.748 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5885 2024-09-14T23:58:07.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.749 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:07.749 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.749 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:07.749 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:58:07.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.749 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.749 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:07.749 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:58:07.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.749 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.749 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:07.749 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:58:07.750 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5899999999999998e-07 2024-09-14T23:58:07.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.750 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.750 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:07.750 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375872+0000", 2024-09-14T23:58:07.750 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4219999999999999e-06 2024-09-14T23:58:07.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.750 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.750 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:07.750 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375878+0000", 2024-09-14T23:58:07.750 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4980000000000003e-06 2024-09-14T23:58:07.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.750 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.750 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:07.751 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375881+0000", 2024-09-14T23:58:07.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6110000000000001e-06 2024-09-14T23:58:07.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.751 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:07.751 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426596+0000", 2024-09-14T23:58:07.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050715320000000001 2024-09-14T23:58:07.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.751 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:07.751 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426607+0000", 2024-09-14T23:58:07.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1223000000000001e-05 2024-09-14T23:58:07.751 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.751 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:07.751 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.752 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.752 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5886 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:07.752 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:58:07.752 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 88.356548919999994, 2024-09-14T23:58:07.752 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050803395000000001, 2024-09-14T23:58:07.752 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:07.752 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:07.752 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:07.752 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:07.752 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:07.752 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:07.752 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5886 2024-09-14T23:58:07.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.752 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:07.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.753 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:07.753 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:58:07.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.753 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:07.753 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:58:07.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.754 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:07.754 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:58:07.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2700000000000001e-07 2024-09-14T23:58:07.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.754 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:07.754 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375886+0000", 2024-09-14T23:58:07.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7060000000000001e-06 2024-09-14T23:58:07.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.755 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:07.755 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375890+0000", 2024-09-14T23:58:07.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2259999999999999e-06 2024-09-14T23:58:07.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.756 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:07.756 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375893+0000", 2024-09-14T23:58:07.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0819999999999999e-06 2024-09-14T23:58:07.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.756 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:07.756 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426677+0000", 2024-09-14T23:58:07.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050784158000000003 2024-09-14T23:58:07.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.756 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:07.756 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426687+0000", 2024-09-14T23:58:07.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9960000000000005e-06 2024-09-14T23:58:07.756 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.757 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:07.757 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.757 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5887 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:07.757 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:58:07.757 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 88.356526833999993, 2024-09-14T23:58:07.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050833070000000001, 2024-09-14T23:58:07.757 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:07.757 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:07.757 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:07.757 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:07.757 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:07.757 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:07.758 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5887 2024-09-14T23:58:07.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.758 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:07.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.758 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:07.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:58:07.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.758 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:07.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:58:07.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.758 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:07.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:58:07.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3299999999999998e-07 2024-09-14T23:58:07.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:07.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375908+0000", 2024-09-14T23:58:07.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7689999999999999e-06 2024-09-14T23:58:07.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:07.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375912+0000", 2024-09-14T23:58:07.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.535e-06 2024-09-14T23:58:07.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:07.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375916+0000", 2024-09-14T23:58:07.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8979999999999998e-06 2024-09-14T23:58:07.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.760 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:07.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426729+0000", 2024-09-14T23:58:07.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050813453000000001 2024-09-14T23:58:07.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.760 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:07.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426739+0000", 2024-09-14T23:58:07.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0081999999999999e-05 2024-09-14T23:58:07.760 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.760 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:07.761 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.761 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5888 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:07.761 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:58:07.761 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 88.356513757000002, 2024-09-14T23:58:07.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050854652, 2024-09-14T23:58:07.761 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:07.761 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:07.761 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:07.761 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:07.762 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:07.762 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:07.762 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5888 2024-09-14T23:58:07.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.762 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:07.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:07.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:58:07.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:07.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:58:07.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.764 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:07.765 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:58:07.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1400000000000001e-07 2024-09-14T23:58:07.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.765 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:07.765 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375921+0000", 2024-09-14T23:58:07.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.119e-06 2024-09-14T23:58:07.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:07.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375928+0000", 2024-09-14T23:58:07.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3170000000000004e-06 2024-09-14T23:58:07.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:07.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375931+0000", 2024-09-14T23:58:07.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6970000000000002e-06 2024-09-14T23:58:07.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:07.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426768+0000", 2024-09-14T23:58:07.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050837355000000001 2024-09-14T23:58:07.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:07.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426774+0000", 2024-09-14T23:58:07.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9499999999999998e-06 2024-09-14T23:58:07.769 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.769 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:07.769 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.770 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5889 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:07.770 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:58:07.770 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 88.356496832999994, 2024-09-14T23:58:07.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050885446000000001, 2024-09-14T23:58:07.770 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:07.770 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:07.771 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:07.771 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:07.771 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:07.771 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:07.771 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5889 2024-09-14T23:58:07.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.772 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:07.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:07.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:58:07.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:07.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:58:07.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:07.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:58:07.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.36e-07 2024-09-14T23:58:07.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:07.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375940+0000", 2024-09-14T23:58:07.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.844e-06 2024-09-14T23:58:07.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:07.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375944+0000", 2024-09-14T23:58:07.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6720000000000002e-06 2024-09-14T23:58:07.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:07.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375946+0000", 2024-09-14T23:58:07.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5629999999999999e-06 2024-09-14T23:58:07.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.777 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:07.777 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426810+0000", 2024-09-14T23:58:07.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050863255000000003 2024-09-14T23:58:07.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:07.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426822+0000", 2024-09-14T23:58:07.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1875999999999999e-05 2024-09-14T23:58:07.778 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.778 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:07.779 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.779 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5890 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:07.779 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:58:07.779 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 88.356483021000003, 2024-09-14T23:58:07.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050917549999999999, 2024-09-14T23:58:07.780 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:07.780 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:07.780 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:07.780 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:07.781 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:07.781 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:07.781 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5890 2024-09-14T23:58:07.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.781 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:07.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:07.782 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:58:07.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:07.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:58:07.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:07.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:58:07.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4299999999999999e-07 2024-09-14T23:58:07.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:07.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375951+0000", 2024-09-14T23:58:07.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1000000000000001e-06 2024-09-14T23:58:07.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:07.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375956+0000", 2024-09-14T23:58:07.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3370000000000001e-06 2024-09-14T23:58:07.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:07.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375958+0000", 2024-09-14T23:58:07.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4820000000000001e-06 2024-09-14T23:58:07.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:07.787 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426858+0000", 2024-09-14T23:58:07.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050899666000000003 2024-09-14T23:58:07.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:07.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426867+0000", 2024-09-14T23:58:07.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7219999999999994e-06 2024-09-14T23:58:07.788 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.788 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:07.788 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.789 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5891 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:07.789 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:58:07.789 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 88.356471698999997, 2024-09-14T23:58:07.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050948067, 2024-09-14T23:58:07.789 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:07.790 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:07.790 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:07.790 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:07.790 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:07.790 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:07.790 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5891 2024-09-14T23:58:07.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.791 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:07.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:07.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:58:07.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:07.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:58:07.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:07.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:58:07.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0800000000000001e-07 2024-09-14T23:58:07.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:07.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375962+0000", 2024-09-14T23:58:07.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9600000000000008e-07 2024-09-14T23:58:07.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:07.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375970+0000", 2024-09-14T23:58:07.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1600000000000001e-06 2024-09-14T23:58:07.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:07.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375972+0000", 2024-09-14T23:58:07.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4849999999999999e-06 2024-09-14T23:58:07.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:07.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426902+0000", 2024-09-14T23:58:07.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050929813999999997 2024-09-14T23:58:07.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:07.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426909+0000", 2024-09-14T23:58:07.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4039999999999998e-06 2024-09-14T23:58:07.795 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.796 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:07.796 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.796 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5892 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:07.796 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:58:07.796 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 88.356456842, 2024-09-14T23:58:07.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050965036999999998, 2024-09-14T23:58:07.796 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:07.796 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:07.796 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:07.796 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:07.797 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:07.797 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:07.797 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5892 2024-09-14T23:58:07.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.797 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:07.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:07.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:58:07.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:07.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:58:07.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:07.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:58:07.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.34e-07 2024-09-14T23:58:07.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:07.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375978+0000", 2024-09-14T23:58:07.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1880000000000001e-06 2024-09-14T23:58:07.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:07.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375981+0000", 2024-09-14T23:58:07.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2399999999999999e-06 2024-09-14T23:58:07.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:07.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375984+0000", 2024-09-14T23:58:07.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9110000000000002e-06 2024-09-14T23:58:07.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:07.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426934+0000", 2024-09-14T23:58:07.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050949912 2024-09-14T23:58:07.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:07.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426941+0000", 2024-09-14T23:58:07.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5519999999999999e-06 2024-09-14T23:58:07.800 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.800 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:07.800 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.800 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5893 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:07.800 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:58:07.800 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 88.356446245000001, 2024-09-14T23:58:07.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050975131999999999, 2024-09-14T23:58:07.800 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:07.800 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:07.800 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:07.800 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:07.801 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:07.801 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:07.801 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5893 2024-09-14T23:58:07.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.801 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:07.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:07.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:58:07.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:07.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:58:07.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:07.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:58:07.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2999999999999999e-07 2024-09-14T23:58:07.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:07.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375988+0000", 2024-09-14T23:58:07.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0410000000000001e-06 2024-09-14T23:58:07.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:07.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375992+0000", 2024-09-14T23:58:07.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.104e-06 2024-09-14T23:58:07.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:07.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375995+0000", 2024-09-14T23:58:07.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4310000000000001e-06 2024-09-14T23:58:07.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:07.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426958+0000", 2024-09-14T23:58:07.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050963642000000003 2024-09-14T23:58:07.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:07.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426962+0000", 2024-09-14T23:58:07.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6839999999999998e-06 2024-09-14T23:58:07.804 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.804 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:07.804 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.804 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5894 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:07.804 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:58:07.804 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 88.356434844999995, 2024-09-14T23:58:07.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050978333000000001, 2024-09-14T23:58:07.804 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:07.804 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:07.804 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:07.804 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:07.805 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:07.805 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:07.805 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5894 2024-09-14T23:58:07.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.805 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:07.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:07.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:58:07.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:07.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:58:07.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:07.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:58:07.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.23e-07 2024-09-14T23:58:07.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:07.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375999+0000", 2024-09-14T23:58:07.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9800000000000002e-07 2024-09-14T23:58:07.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:07.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376007+0000", 2024-09-14T23:58:07.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8539999999999997e-06 2024-09-14T23:58:07.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:07.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376011+0000", 2024-09-14T23:58:07.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.399e-06 2024-09-14T23:58:07.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:07.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426973+0000", 2024-09-14T23:58:07.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050962454999999997 2024-09-14T23:58:07.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:07.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426976+0000", 2024-09-14T23:58:07.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4039999999999999e-06 2024-09-14T23:58:07.808 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.808 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:07.808 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.808 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5895 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:07.808 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376014+0000", 2024-09-14T23:58:07.808 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 88.356418500999993, 2024-09-14T23:58:07.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050975951999999998, 2024-09-14T23:58:07.808 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:07.808 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:07.808 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:07.809 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:07.809 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:07.809 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:07.809 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5895 2024-09-14T23:58:07.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.809 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:07.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:07.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376014+0000", 2024-09-14T23:58:07.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:07.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376014+0000", 2024-09-14T23:58:07.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:07.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376015+0000", 2024-09-14T23:58:07.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2999999999999999e-07 2024-09-14T23:58:07.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:07.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376016+0000", 2024-09-14T23:58:07.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1659999999999999e-06 2024-09-14T23:58:07.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:07.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376024+0000", 2024-09-14T23:58:07.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8930000000000005e-06 2024-09-14T23:58:07.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:07.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376026+0000", 2024-09-14T23:58:07.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6869999999999999e-06 2024-09-14T23:58:07.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:07.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426987+0000", 2024-09-14T23:58:07.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050960601000000001 2024-09-14T23:58:07.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:07.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426990+0000", 2024-09-14T23:58:07.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3749999999999999e-06 2024-09-14T23:58:07.812 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.812 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:07.812 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.812 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5896 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:07.812 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:58:07.812 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 88.356402763999995, 2024-09-14T23:58:07.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050975656000000001, 2024-09-14T23:58:07.812 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:07.812 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:07.813 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:07.813 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:07.813 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:07.813 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:07.813 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5896 2024-09-14T23:58:07.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.813 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:07.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:07.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:58:07.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:07.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:58:07.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:07.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:58:07.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5100000000000001e-07 2024-09-14T23:58:07.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:07.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376032+0000", 2024-09-14T23:58:07.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.141e-06 2024-09-14T23:58:07.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:07.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376036+0000", 2024-09-14T23:58:07.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.25e-06 2024-09-14T23:58:07.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:07.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376038+0000", 2024-09-14T23:58:07.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5469999999999999e-06 2024-09-14T23:58:07.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:07.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427002+0000", 2024-09-14T23:58:07.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050964029000000001 2024-09-14T23:58:07.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:07.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427006+0000", 2024-09-14T23:58:07.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4379999999999999e-06 2024-09-14T23:58:07.816 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.816 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:07.816 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.816 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5897 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:07.816 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:58:07.816 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 88.356390963999999, 2024-09-14T23:58:07.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050978001000000002, 2024-09-14T23:58:07.816 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:07.816 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:07.817 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:07.817 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:07.817 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:07.817 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:07.817 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5897 2024-09-14T23:58:07.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.817 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:07.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:07.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:58:07.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:07.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:58:07.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:07.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:58:07.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2600000000000001e-07 2024-09-14T23:58:07.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:07.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376043+0000", 2024-09-14T23:58:07.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.054e-06 2024-09-14T23:58:07.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:07.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376047+0000", 2024-09-14T23:58:07.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0659999999999997e-06 2024-09-14T23:58:07.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:07.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376050+0000", 2024-09-14T23:58:07.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.88e-06 2024-09-14T23:58:07.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:07.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427017+0000", 2024-09-14T23:58:07.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050966453000000002 2024-09-14T23:58:07.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:07.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427020+0000", 2024-09-14T23:58:07.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3220000000000001e-06 2024-09-14T23:58:07.820 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.820 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:07.820 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.820 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5898 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:07.820 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376053+0000", 2024-09-14T23:58:07.820 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 88.356379478999997, 2024-09-14T23:58:07.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050982024000000001, 2024-09-14T23:58:07.820 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:07.820 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:07.821 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:07.821 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:07.821 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:07.821 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:07.821 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5898 2024-09-14T23:58:07.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.821 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:07.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:07.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376053+0000", 2024-09-14T23:58:07.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:07.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376053+0000", 2024-09-14T23:58:07.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:07.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376278+0000", 2024-09-14T23:58:07.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022426400000000001 2024-09-14T23:58:07.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:07.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376282+0000", 2024-09-14T23:58:07.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4739999999999999e-06 2024-09-14T23:58:07.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:07.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376296+0000", 2024-09-14T23:58:07.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4229999999999999e-05 2024-09-14T23:58:07.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:07.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376303+0000", 2024-09-14T23:58:07.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2199999999999997e-06 2024-09-14T23:58:07.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:07.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427031+0000", 2024-09-14T23:58:07.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050728154999999997 2024-09-14T23:58:07.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:07.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427036+0000", 2024-09-14T23:58:07.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6809999999999997e-06 2024-09-14T23:58:07.824 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.824 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:07.824 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.824 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5899 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:07.824 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376338+0000", 2024-09-14T23:58:07.824 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 88.356094755000001, 2024-09-14T23:58:07.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050712100000000003, 2024-09-14T23:58:07.824 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:07.824 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:07.824 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:07.825 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:07.825 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:07.825 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:07.825 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5899 2024-09-14T23:58:07.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.825 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:07.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:07.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376338+0000", 2024-09-14T23:58:07.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:07.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376338+0000", 2024-09-14T23:58:07.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:07.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376339+0000", 2024-09-14T23:58:07.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9100000000000003e-07 2024-09-14T23:58:07.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:07.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376342+0000", 2024-09-14T23:58:07.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7e-06 2024-09-14T23:58:07.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:07.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376350+0000", 2024-09-14T23:58:07.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5280000000000003e-06 2024-09-14T23:58:07.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:07.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376356+0000", 2024-09-14T23:58:07.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6010000000000002e-06 2024-09-14T23:58:07.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:07.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427047+0000", 2024-09-14T23:58:07.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050690962999999999 2024-09-14T23:58:07.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:07.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427050+0000", 2024-09-14T23:58:07.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.517e-06 2024-09-14T23:58:07.828 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.828 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:07.828 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.828 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5900 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:07.828 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376366+0000", 2024-09-14T23:58:07.828 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 88.356066495999997, 2024-09-14T23:58:07.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050697858999999998, 2024-09-14T23:58:07.828 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:07.828 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:07.828 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:07.828 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:07.829 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:07.829 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:07.829 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5900 2024-09-14T23:58:07.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.829 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:07.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:07.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376366+0000", 2024-09-14T23:58:07.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:07.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376366+0000", 2024-09-14T23:58:07.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:07.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377650+0000", 2024-09-14T23:58:07.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001283834 2024-09-14T23:58:07.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:07.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377655+0000", 2024-09-14T23:58:07.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7230000000000004e-06 2024-09-14T23:58:07.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:07.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377667+0000", 2024-09-14T23:58:07.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1746000000000001e-05 2024-09-14T23:58:07.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:07.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377676+0000", 2024-09-14T23:58:07.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3619999999999995e-06 2024-09-14T23:58:07.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:07.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427061+0000", 2024-09-14T23:58:07.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049384755000000002 2024-09-14T23:58:07.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:07.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427064+0000", 2024-09-14T23:58:07.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.439e-06 2024-09-14T23:58:07.832 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.832 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:07.832 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.832 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:132622 3.1e 3:79d0a5e0:::rbd_data.111847077d5e.0000000000000007:head [sparse-read 0~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T23:58:07.832 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:44.037901+0000", 2024-09-14T23:58:07.832 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.694531887, 2024-09-14T23:58:07.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1083420859999999, 2024-09-14T23:58:07.832 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:07.832 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:07.832 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:07.832 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:07.832 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:07.833 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:07.833 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 132622 2024-09-14T23:58:07.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.833 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:07.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:07.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037901+0000", 2024-09-14T23:58:07.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:07.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037901+0000", 2024-09-14T23:58:07.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:07.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:07.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037903+0000", 2024-09-14T23:58:07.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4619999999999999e-06 2024-09-14T23:58:07.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:07.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037910+0000", 2024-09-14T23:58:07.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7449999999999995e-06 2024-09-14T23:58:07.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:07.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037928+0000", 2024-09-14T23:58:07.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7305000000000001e-05 2024-09-14T23:58:07.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:07.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037939+0000", 2024-09-14T23:58:07.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1599e-05 2024-09-14T23:58:07.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:07.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.038049+0000", 2024-09-14T23:58:07.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000109577 2024-09-14T23:58:07.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op must wait for map", 2024-09-14T23:58:07.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.038057+0000", 2024-09-14T23:58:07.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1450000000000008e-06 2024-09-14T23:58:07.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:07.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.039937+0000", 2024-09-14T23:58:07.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00188024 2024-09-14T23:58:07.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:58:07.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.040039+0000", 2024-09-14T23:58:07.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000101934 2024-09-14T23:58:07.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:07.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:45.145805+0000", 2024-09-14T23:58:07.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.105765651 2024-09-14T23:58:07.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:07.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:45.145997+0000", 2024-09-14T23:58:07.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000192084 2024-09-14T23:58:07.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:07.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:07.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:07.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:45.146243+0000", 2024-09-14T23:58:07.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024634400000000001 2024-09-14T23:58:07.837 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.837 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:07.837 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.837 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:07.838 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:07.838 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:07.839 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T23:58:07.840 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:07.836+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:07.940 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:07.938+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:07.993 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:07.993 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:07.993 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:58:07.993 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:08.002 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T23:58:08.042 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:08.041+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:08.141 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:08.140+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:08.151 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:08.151 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:08.151 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:58:08.151 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:58:08.151 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:08.160 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T23:58:08.242 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:08.240+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:08.269 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:08.269 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:58:08.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:58:08.269 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:58:08.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.269 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:3578 3.4 e29/23 3:2298d860:::rbd_data.1118de1168b.0000000000000030:head v 29'15, mlcod=29'13)", 2024-09-14T23:58:08.270 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.036455+0000", 2024-09-14T23:58:08.270 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 89.230683562999999, 2024-09-14T23:58:08.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019449440000000001, 2024-09-14T23:58:08.270 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:08.270 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:08.270 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:08.270 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:08.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:08.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036455+0000", 2024-09-14T23:58:08.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:08.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036455+0000", 2024-09-14T23:58:08.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:08.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036456+0000", 2024-09-14T23:58:08.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0999999999999997e-07 2024-09-14T23:58:08.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:08.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036463+0000", 2024-09-14T23:58:08.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2200000000000003e-06 2024-09-14T23:58:08.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:08.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036478+0000", 2024-09-14T23:58:08.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4977000000000001e-05 2024-09-14T23:58:08.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:08.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036485+0000", 2024-09-14T23:58:08.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0559999999999999e-06 2024-09-14T23:58:08.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:08.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036693+0000", 2024-09-14T23:58:08.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000208051 2024-09-14T23:58:08.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:08.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036715+0000", 2024-09-14T23:58:08.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1330000000000001e-05 2024-09-14T23:58:08.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:08.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.038356+0000", 2024-09-14T23:58:08.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001641586 2024-09-14T23:58:08.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:08.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.038400+0000", 2024-09-14T23:58:08.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3914e-05 2024-09-14T23:58:08.274 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.274 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:08.274 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.274 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:3579 3.4 e29/23 3:27290fe6:::rbd_data.1118de1168b.0000000000000032:head v 29'16, mlcod=29'13)", 2024-09-14T23:58:08.274 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.037085+0000", 2024-09-14T23:58:08.274 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 89.230053972999997, 2024-09-14T23:58:08.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0020019220000000002, 2024-09-14T23:58:08.275 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:08.275 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:08.275 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:08.275 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:08.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:08.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037085+0000", 2024-09-14T23:58:08.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:08.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037085+0000", 2024-09-14T23:58:08.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.276 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:08.276 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037085+0000", 2024-09-14T23:58:08.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5499999999999997e-07 2024-09-14T23:58:08.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.276 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:08.276 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037091+0000", 2024-09-14T23:58:08.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7869999999999998e-06 2024-09-14T23:58:08.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.276 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:08.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037102+0000", 2024-09-14T23:58:08.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.031e-05 2024-09-14T23:58:08.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:08.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037116+0000", 2024-09-14T23:58:08.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4119e-05 2024-09-14T23:58:08.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:08.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037219+0000", 2024-09-14T23:58:08.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010372799999999999 2024-09-14T23:58:08.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:08.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037239+0000", 2024-09-14T23:58:08.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9993e-05 2024-09-14T23:58:08.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:08.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.039046+0000", 2024-09-14T23:58:08.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001806666 2024-09-14T23:58:08.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:08.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.039087+0000", 2024-09-14T23:58:08.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0564e-05 2024-09-14T23:58:08.278 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.278 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:08.278 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.279 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:32024 3.9 e39/23 3:94dc260b:::rbd_data.111814b482f8.0000000000000009:head v 39'45, mlcod=38'43)", 2024-09-14T23:58:08.279 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:49.061018+0000", 2024-09-14T23:58:08.279 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 79.206120634000001, 2024-09-14T23:58:08.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018482240000000001, 2024-09-14T23:58:08.280 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:08.280 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:08.280 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:08.280 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:08.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.280 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:08.280 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061018+0000", 2024-09-14T23:58:08.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.280 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:08.280 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061018+0000", 2024-09-14T23:58:08.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:08.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061018+0000", 2024-09-14T23:58:08.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9999999999999998e-07 2024-09-14T23:58:08.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:08.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061211+0000", 2024-09-14T23:58:08.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019206 2024-09-14T23:58:08.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:08.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061221+0000", 2024-09-14T23:58:08.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0659000000000001e-05 2024-09-14T23:58:08.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:08.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061230+0000", 2024-09-14T23:58:08.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4610000000000008e-06 2024-09-14T23:58:08.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:08.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061317+0000", 2024-09-14T23:58:08.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7514000000000003e-05 2024-09-14T23:58:08.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:08.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061338+0000", 2024-09-14T23:58:08.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0395999999999999e-05 2024-09-14T23:58:08.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:08.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.062830+0000", 2024-09-14T23:58:08.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0014921940000000001 2024-09-14T23:58:08.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:08.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.062866+0000", 2024-09-14T23:58:08.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6539999999999999e-05 2024-09-14T23:58:08.284 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.284 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:08.284 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.284 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:32420 3.9 e39/23 3:91dc34e7:::rbd_data.111814b482f8.000000000000001e:head v 39'46, mlcod=38'44)", 2024-09-14T23:58:08.284 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:49.086124+0000", 2024-09-14T23:58:08.284 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 79.181015134999996, 2024-09-14T23:58:08.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001686819, 2024-09-14T23:58:08.284 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:08.284 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:08.285 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:08.285 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:08.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.285 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:08.285 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086124+0000", 2024-09-14T23:58:08.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.285 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:08.285 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086124+0000", 2024-09-14T23:58:08.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:08.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086125+0000", 2024-09-14T23:58:08.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.361e-06 2024-09-14T23:58:08.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:08.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086605+0000", 2024-09-14T23:58:08.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00048026600000000003 2024-09-14T23:58:08.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.287 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:08.287 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086626+0000", 2024-09-14T23:58:08.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.037e-05 2024-09-14T23:58:08.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.287 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:08.287 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086635+0000", 2024-09-14T23:58:08.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5880000000000004e-06 2024-09-14T23:58:08.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.287 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:08.287 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086701+0000", 2024-09-14T23:58:08.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5905e-05 2024-09-14T23:58:08.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:08.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086727+0000", 2024-09-14T23:58:08.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5573000000000001e-05 2024-09-14T23:58:08.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:08.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.087767+0000", 2024-09-14T23:58:08.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001040537 2024-09-14T23:58:08.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:08.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.087810+0000", 2024-09-14T23:58:08.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3219000000000001e-05 2024-09-14T23:58:08.289 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.289 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:08.289 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.289 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:50582 3.9 e51/23 3:92198e7a:::rbd_data.11183b9aac34.000000000000000a:head v 51'63, mlcod=50'60)", 2024-09-14T23:58:08.289 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:01.131541+0000", 2024-09-14T23:58:08.289 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 67.135597943999997, 2024-09-14T23:58:08.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001795547, 2024-09-14T23:58:08.289 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:08.290 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:08.290 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:08.290 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:08.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:08.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131541+0000", 2024-09-14T23:58:08.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:08.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131541+0000", 2024-09-14T23:58:08.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:08.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131541+0000", 2024-09-14T23:58:08.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7099999999999997e-07 2024-09-14T23:58:08.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:08.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131548+0000", 2024-09-14T23:58:08.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.708e-06 2024-09-14T23:58:08.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:08.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131570+0000", 2024-09-14T23:58:08.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1815999999999999e-05 2024-09-14T23:58:08.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.292 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:08.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131581+0000", 2024-09-14T23:58:08.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0933e-05 2024-09-14T23:58:08.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.292 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:08.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131660+0000", 2024-09-14T23:58:08.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9071000000000002e-05 2024-09-14T23:58:08.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.293 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:08.293 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131682+0000", 2024-09-14T23:58:08.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2534000000000001e-05 2024-09-14T23:58:08.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.293 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:08.293 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.133287+0000", 2024-09-14T23:58:08.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016045510000000001 2024-09-14T23:58:08.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.293 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:08.293 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.133336+0000", 2024-09-14T23:58:08.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9563e-05 2024-09-14T23:58:08.294 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.294 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:08.294 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.294 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:51281 3.0 e52/23 3:04e60ec2:::rbd_data.11181975a1de.0000000000000003:head v 52'113, mlcod=44'111)", 2024-09-14T23:58:08.294 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:02.158822+0000", 2024-09-14T23:58:08.294 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 66.108316447999997, 2024-09-14T23:58:08.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019860609999999999, 2024-09-14T23:58:08.294 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:08.294 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:08.294 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:08.294 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:08.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.295 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:08.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.158822+0000", 2024-09-14T23:58:08.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.295 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:08.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.158822+0000", 2024-09-14T23:58:08.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.295 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:08.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.158823+0000", 2024-09-14T23:58:08.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6800000000000001e-07 2024-09-14T23:58:08.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.296 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:08.296 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.159132+0000", 2024-09-14T23:58:08.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030974900000000001 2024-09-14T23:58:08.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.296 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:08.296 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.159146+0000", 2024-09-14T23:58:08.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3261000000000001e-05 2024-09-14T23:58:08.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:08.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.159155+0000", 2024-09-14T23:58:08.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7410000000000004e-06 2024-09-14T23:58:08.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:08.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.159221+0000", 2024-09-14T23:58:08.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5698999999999996e-05 2024-09-14T23:58:08.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:08.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.159244+0000", 2024-09-14T23:58:08.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3108e-05 2024-09-14T23:58:08.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.298 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.298 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:08.298 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.160767+0000", 2024-09-14T23:58:08.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015231120000000001 2024-09-14T23:58:08.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.298 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.298 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:08.298 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.160808+0000", 2024-09-14T23:58:08.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1022999999999997e-05 2024-09-14T23:58:08.298 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.298 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:08.298 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.299 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:56107 3.0 e56/23 3:00c47890:::rbd_data.1118c4b3aebe.0000000000000009:head v 56'125, mlcod=52'123)", 2024-09-14T23:58:08.299 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.236123+0000", 2024-09-14T23:58:08.299 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 62.031015248999999, 2024-09-14T23:58:08.299 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0061273220000000002, 2024-09-14T23:58:08.299 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:08.299 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:08.299 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:08.299 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:08.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.299 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:08.299 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.236123+0000", 2024-09-14T23:58:08.299 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.300 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.300 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:08.300 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.236123+0000", 2024-09-14T23:58:08.300 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.300 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.300 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.300 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:08.300 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.236125+0000", 2024-09-14T23:58:08.300 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.313e-06 2024-09-14T23:58:08.300 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.300 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.300 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:08.301 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237428+0000", 2024-09-14T23:58:08.301 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013029890000000001 2024-09-14T23:58:08.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.301 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:08.301 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237452+0000", 2024-09-14T23:58:08.301 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4090000000000001e-05 2024-09-14T23:58:08.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.301 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:08.301 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237462+0000", 2024-09-14T23:58:08.301 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0421999999999999e-05 2024-09-14T23:58:08.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.302 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.302 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:08.302 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237559+0000", 2024-09-14T23:58:08.302 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6378000000000005e-05 2024-09-14T23:58:08.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.302 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.302 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:08.302 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237583+0000", 2024-09-14T23:58:08.302 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4182e-05 2024-09-14T23:58:08.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.302 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.302 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:08.303 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.242205+0000", 2024-09-14T23:58:08.303 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0046217289999999998 2024-09-14T23:58:08.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.303 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.303 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:08.303 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.242251+0000", 2024-09-14T23:58:08.303 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6218999999999999e-05 2024-09-14T23:58:08.303 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.303 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:08.303 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.303 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.303 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:56108 3.0 e56/23 3:00c47890:::rbd_data.1118c4b3aebe.0000000000000009:head v 56'127, mlcod=52'123)", 2024-09-14T23:58:08.303 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.239693+0000", 2024-09-14T23:58:08.304 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 62.027445778999997, 2024-09-14T23:58:08.304 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0043496209999999997, 2024-09-14T23:58:08.304 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:08.304 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:08.304 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:08.304 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:08.304 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.304 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:08.304 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239693+0000", 2024-09-14T23:58:08.304 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.304 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.304 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:08.305 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239693+0000", 2024-09-14T23:58:08.305 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.305 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.305 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:08.305 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239694+0000", 2024-09-14T23:58:08.305 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0499999999999999e-06 2024-09-14T23:58:08.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.305 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.305 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:08.305 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239702+0000", 2024-09-14T23:58:08.305 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1170000000000004e-06 2024-09-14T23:58:08.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.306 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.306 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:08.306 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239716+0000", 2024-09-14T23:58:08.306 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4306e-05 2024-09-14T23:58:08.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.306 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.306 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:08.306 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239724+0000", 2024-09-14T23:58:08.306 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0399999999999993e-06 2024-09-14T23:58:08.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.306 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.306 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:08.306 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239778+0000", 2024-09-14T23:58:08.307 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4032e-05 2024-09-14T23:58:08.307 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.307 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.307 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:08.307 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239801+0000", 2024-09-14T23:58:08.307 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2770000000000001e-05 2024-09-14T23:58:08.307 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.307 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.307 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:08.307 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.243990+0000", 2024-09-14T23:58:08.307 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0041890909999999998 2024-09-14T23:58:08.307 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.307 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.307 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:08.308 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.244043+0000", 2024-09-14T23:58:08.308 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2215000000000001e-05 2024-09-14T23:58:08.308 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.308 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:08.308 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.308 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.308 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:56385 3.9 e56/23 3:963d31c7:::rbd_data.1118c4b3aebe.0000000000000003:head v 56'87, mlcod=53'85)", 2024-09-14T23:58:08.308 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.283936+0000", 2024-09-14T23:58:08.308 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 61.983202869000003, 2024-09-14T23:58:08.308 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0063042080000000004, 2024-09-14T23:58:08.308 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:08.308 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:08.308 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:08.309 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:08.309 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.309 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:08.309 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.283936+0000", 2024-09-14T23:58:08.309 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.309 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.309 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.309 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:08.309 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.283936+0000", 2024-09-14T23:58:08.309 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.309 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.309 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.309 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:08.309 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.283937+0000", 2024-09-14T23:58:08.310 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1179999999999999e-06 2024-09-14T23:58:08.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.310 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:08.310 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286338+0000", 2024-09-14T23:58:08.310 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024010149999999998 2024-09-14T23:58:08.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.310 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:08.310 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286360+0000", 2024-09-14T23:58:08.310 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2212e-05 2024-09-14T23:58:08.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.310 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:08.311 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286370+0000", 2024-09-14T23:58:08.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5850000000000006e-06 2024-09-14T23:58:08.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.311 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:08.311 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286443+0000", 2024-09-14T23:58:08.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2757999999999997e-05 2024-09-14T23:58:08.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.311 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:08.311 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286466+0000", 2024-09-14T23:58:08.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3844000000000001e-05 2024-09-14T23:58:08.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.312 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:08.312 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.290197+0000", 2024-09-14T23:58:08.312 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003730176 2024-09-14T23:58:08.312 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.312 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.312 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:08.312 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.290240+0000", 2024-09-14T23:58:08.312 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.35e-05 2024-09-14T23:58:08.312 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.312 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:08.312 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.312 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.312 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.312 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:56386 3.9 e56/23 3:963d31c7:::rbd_data.1118c4b3aebe.0000000000000003:head v 56'89, mlcod=53'85)", 2024-09-14T23:58:08.313 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.287578+0000", 2024-09-14T23:58:08.313 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 61.979561142999998, 2024-09-14T23:58:08.313 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033580250000000002, 2024-09-14T23:58:08.313 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:08.313 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:08.313 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:08.313 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:08.313 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.313 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:08.313 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287578+0000", 2024-09-14T23:58:08.313 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.313 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.313 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.313 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:08.314 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287578+0000", 2024-09-14T23:58:08.314 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.314 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.314 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.314 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:08.314 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287579+0000", 2024-09-14T23:58:08.314 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.64e-07 2024-09-14T23:58:08.314 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.314 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.314 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:08.314 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287586+0000", 2024-09-14T23:58:08.314 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5310000000000004e-06 2024-09-14T23:58:08.314 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.314 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.315 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:08.315 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287597+0000", 2024-09-14T23:58:08.315 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0903999999999999e-05 2024-09-14T23:58:08.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.315 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:08.315 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287605+0000", 2024-09-14T23:58:08.315 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7889999999999995e-06 2024-09-14T23:58:08.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.315 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:08.315 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.288437+0000", 2024-09-14T23:58:08.315 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000831914 2024-09-14T23:58:08.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.316 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.316 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:08.316 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.288460+0000", 2024-09-14T23:58:08.316 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3410000000000001e-05 2024-09-14T23:58:08.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.316 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.316 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:08.316 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.290883+0000", 2024-09-14T23:58:08.316 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024230599999999999 2024-09-14T23:58:08.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.316 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.316 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:08.316 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.290936+0000", 2024-09-14T23:58:08.317 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2453000000000001e-05 2024-09-14T23:58:08.317 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.317 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:08.317 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.317 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.317 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:60299 3.9 e57/23 3:96f26b6f:::rbd_data.11188d356229.0000000000000009:head v 57'98, mlcod=56'96)", 2024-09-14T23:58:08.317 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:07.418336+0000", 2024-09-14T23:58:08.317 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 60.848802650000003, 2024-09-14T23:58:08.317 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022211459999999998, 2024-09-14T23:58:08.317 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:08.317 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:08.317 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:08.317 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:08.318 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.318 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:08.318 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.418336+0000", 2024-09-14T23:58:08.318 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.318 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.318 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:08.318 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.418336+0000", 2024-09-14T23:58:08.318 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.318 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.318 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:08.318 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.418337+0000", 2024-09-14T23:58:08.318 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7199999999999999e-07 2024-09-14T23:58:08.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.319 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.319 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:08.319 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.419396+0000", 2024-09-14T23:58:08.319 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010598319999999999 2024-09-14T23:58:08.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.319 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.319 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:08.319 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.419407+0000", 2024-09-14T23:58:08.319 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0956e-05 2024-09-14T23:58:08.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.319 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.319 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:08.319 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.419417+0000", 2024-09-14T23:58:08.319 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0040999999999999e-05 2024-09-14T23:58:08.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.320 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.320 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:08.320 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.419468+0000", 2024-09-14T23:58:08.320 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0497000000000001e-05 2024-09-14T23:58:08.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.320 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.320 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:08.320 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.419485+0000", 2024-09-14T23:58:08.320 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6966e-05 2024-09-14T23:58:08.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.320 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.320 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:08.321 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.420527+0000", 2024-09-14T23:58:08.321 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001042133 2024-09-14T23:58:08.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.321 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.321 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:08.321 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.420557+0000", 2024-09-14T23:58:08.321 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0249000000000001e-05 2024-09-14T23:58:08.321 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.321 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:08.321 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.321 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.321 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:64263 3.4 e57/23 3:20c6a768:::rbd_data.11188d356229.0000000000000001:head v 57'165, mlcod=57'163)", 2024-09-14T23:58:08.321 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:07.999031+0000", 2024-09-14T23:58:08.322 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 60.268107995000001, 2024-09-14T23:58:08.322 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023815210000000002, 2024-09-14T23:58:08.322 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:08.322 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:08.322 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:08.322 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:08.322 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.322 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:08.322 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999031+0000", 2024-09-14T23:58:08.322 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.322 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.322 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:08.322 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999031+0000", 2024-09-14T23:58:08.322 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.323 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.323 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:08.323 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999032+0000", 2024-09-14T23:58:08.323 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.176e-06 2024-09-14T23:58:08.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.323 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.323 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:08.323 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999660+0000", 2024-09-14T23:58:08.323 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00062817099999999996 2024-09-14T23:58:08.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.323 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.323 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:08.323 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999683+0000", 2024-09-14T23:58:08.324 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2816999999999999e-05 2024-09-14T23:58:08.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.324 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:08.324 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999693+0000", 2024-09-14T23:58:08.324 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0088999999999999e-05 2024-09-14T23:58:08.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.324 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:08.324 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999767+0000", 2024-09-14T23:58:08.324 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4056999999999999e-05 2024-09-14T23:58:08.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.325 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:08.325 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999793+0000", 2024-09-14T23:58:08.325 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5573000000000001e-05 2024-09-14T23:58:08.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.325 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:08.325 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:08.001379+0000", 2024-09-14T23:58:08.325 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015859019999999999 2024-09-14T23:58:08.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.325 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:08.325 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:08.001412+0000", 2024-09-14T23:58:08.325 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3735999999999997e-05 2024-09-14T23:58:08.326 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.326 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:08.326 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.326 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.326 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:66570 3.0 e60/23 3:055350b3:::rbd_data.111873792d9b.0000000000000005:head v 60'162, mlcod=57'159)", 2024-09-14T23:58:08.326 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:11.277517+0000", 2024-09-14T23:58:08.326 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 56.989621839000002, 2024-09-14T23:58:08.326 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003472199, 2024-09-14T23:58:08.326 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:08.326 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:08.326 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:08.326 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:08.326 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.327 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:08.327 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.277517+0000", 2024-09-14T23:58:08.327 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.327 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.327 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:08.327 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.277517+0000", 2024-09-14T23:58:08.327 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.327 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.327 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:08.327 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.277518+0000", 2024-09-14T23:58:08.327 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1379999999999999e-06 2024-09-14T23:58:08.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.328 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.328 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:08.328 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278504+0000", 2024-09-14T23:58:08.328 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00098617499999999994 2024-09-14T23:58:08.328 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.328 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.328 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:08.328 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278525+0000", 2024-09-14T23:58:08.328 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.09e-05 2024-09-14T23:58:08.328 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.328 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.328 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:08.328 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278534+0000", 2024-09-14T23:58:08.329 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6209999999999993e-06 2024-09-14T23:58:08.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.329 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.329 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:08.329 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278611+0000", 2024-09-14T23:58:08.329 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6926000000000002e-05 2024-09-14T23:58:08.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.329 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.329 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:08.329 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278641+0000", 2024-09-14T23:58:08.329 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0861999999999997e-05 2024-09-14T23:58:08.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.329 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.329 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:08.330 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.280953+0000", 2024-09-14T23:58:08.330 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023114239999999999 2024-09-14T23:58:08.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.330 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.330 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:08.330 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.280989+0000", 2024-09-14T23:58:08.330 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6152999999999998e-05 2024-09-14T23:58:08.330 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.330 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:08.330 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.330 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.330 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:70526 3.4 e64/23 3:2649db36:::rbd_data.111831f32e80.0000000000000058:head v 64'183, mlcod=64'181)", 2024-09-14T23:58:08.330 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:15.442503+0000", 2024-09-14T23:58:08.330 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 52.824635946999997, 2024-09-14T23:58:08.331 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001769837, 2024-09-14T23:58:08.331 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:08.331 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:08.331 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:08.331 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:08.331 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.331 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:08.331 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.442503+0000", 2024-09-14T23:58:08.331 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.331 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.331 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:08.331 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.442503+0000", 2024-09-14T23:58:08.332 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.332 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.332 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.332 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:08.332 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.442503+0000", 2024-09-14T23:58:08.332 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1099999999999995e-07 2024-09-14T23:58:08.332 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.332 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.332 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:08.332 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.442620+0000", 2024-09-14T23:58:08.332 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000116618 2024-09-14T23:58:08.332 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.332 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.332 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:08.333 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.442634+0000", 2024-09-14T23:58:08.333 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3984e-05 2024-09-14T23:58:08.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.333 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.333 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:08.333 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.442641+0000", 2024-09-14T23:58:08.333 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2540000000000003e-06 2024-09-14T23:58:08.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.333 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.333 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:08.333 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.442732+0000", 2024-09-14T23:58:08.333 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0480000000000001e-05 2024-09-14T23:58:08.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.333 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.334 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:08.334 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.442753+0000", 2024-09-14T23:58:08.334 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1571999999999999e-05 2024-09-14T23:58:08.334 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.334 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.334 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:08.334 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.444236+0000", 2024-09-14T23:58:08.334 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0014825439999999999 2024-09-14T23:58:08.334 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.334 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.334 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:08.334 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.444273+0000", 2024-09-14T23:58:08.334 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6674000000000002e-05 2024-09-14T23:58:08.334 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.335 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:08.335 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.335 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.335 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.335 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:106629 3.4 e70/23 3:240d85c6:::rbd_data.11187de8b0a5.0000000000000009:head v 70'233, mlcod=69'231)", 2024-09-14T23:58:08.335 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:21.489517+0000", 2024-09-14T23:58:08.335 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.777621314000001, 2024-09-14T23:58:08.335 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016279840000000001, 2024-09-14T23:58:08.335 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:08.335 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:08.335 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:08.335 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:08.335 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.336 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:08.336 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.489517+0000", 2024-09-14T23:58:08.336 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.336 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:08.336 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.489517+0000", 2024-09-14T23:58:08.336 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.336 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:08.336 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.489518+0000", 2024-09-14T23:58:08.336 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.029e-06 2024-09-14T23:58:08.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.337 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.337 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:08.337 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.489840+0000", 2024-09-14T23:58:08.337 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00032164100000000001 2024-09-14T23:58:08.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.337 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.337 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:08.337 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.489859+0000", 2024-09-14T23:58:08.337 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8828999999999999e-05 2024-09-14T23:58:08.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.337 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.337 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:08.337 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.489869+0000", 2024-09-14T23:58:08.338 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6600000000000007e-06 2024-09-14T23:58:08.338 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.338 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.338 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:08.338 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.489934+0000", 2024-09-14T23:58:08.338 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5357000000000005e-05 2024-09-14T23:58:08.338 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.338 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.338 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:08.338 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.489951+0000", 2024-09-14T23:58:08.338 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6951000000000001e-05 2024-09-14T23:58:08.338 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.338 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.338 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:08.338 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.491108+0000", 2024-09-14T23:58:08.339 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011571940000000001 2024-09-14T23:58:08.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.339 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.339 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:08.339 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.491145+0000", 2024-09-14T23:58:08.339 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7323000000000002e-05 2024-09-14T23:58:08.339 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.339 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:08.339 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.339 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.339 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:108739 3.9 e80/23 3:9372b409:::rbd_data.111867e26b91.0000000000000005:head v 80'172, mlcod=78'170)", 2024-09-14T23:58:08.339 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:31.064099+0000", 2024-09-14T23:58:08.340 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.203040061000003, 2024-09-14T23:58:08.340 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001688174, 2024-09-14T23:58:08.340 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:08.340 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:08.340 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:08.340 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:08.340 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.340 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:08.340 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:31.064099+0000", 2024-09-14T23:58:08.340 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.340 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.340 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:08.340 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:31.064099+0000", 2024-09-14T23:58:08.340 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.341 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.341 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:08.341 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:31.064099+0000", 2024-09-14T23:58:08.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.44e-07 2024-09-14T23:58:08.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.341 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.341 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:08.341 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:31.064453+0000", 2024-09-14T23:58:08.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00035374899999999999 2024-09-14T23:58:08.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.342 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.342 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:08.342 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:31.064468+0000", 2024-09-14T23:58:08.342 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4476e-05 2024-09-14T23:58:08.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.342 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.342 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:08.342 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:31.064475+0000", 2024-09-14T23:58:08.342 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9289999999999998e-06 2024-09-14T23:58:08.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.342 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.342 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:08.342 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:31.064530+0000", 2024-09-14T23:58:08.342 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4401000000000003e-05 2024-09-14T23:58:08.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.343 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:08.343 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:31.064551+0000", 2024-09-14T23:58:08.343 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0996e-05 2024-09-14T23:58:08.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.343 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:08.343 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:31.065748+0000", 2024-09-14T23:58:08.343 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011967449999999999 2024-09-14T23:58:08.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.343 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:08.344 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:31.065787+0000", 2024-09-14T23:58:08.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9233999999999997e-05 2024-09-14T23:58:08.344 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.344 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:08.344 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.344 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "pg_update_log_missing(3.9 epoch 84/23 rep_tid 2234 entries 84'200 (0'0) error 3:93b58b1d:::rbd_data.1118417a1b0b.000000000000001b:head by client.4376.0:118637 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 83'197)", 2024-09-14T23:58:08.344 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:35.103156+0000", 2024-09-14T23:58:08.344 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 33.163982566000001, 2024-09-14T23:58:08.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019852009999999998, 2024-09-14T23:58:08.344 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:08.344 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:08.344 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:08.345 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:08.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:08.345 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:35.103156+0000", 2024-09-14T23:58:08.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:08.345 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:35.103156+0000", 2024-09-14T23:58:08.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:08.345 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:35.103157+0000", 2024-09-14T23:58:08.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1969999999999999e-06 2024-09-14T23:58:08.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.346 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:08.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:35.103170+0000", 2024-09-14T23:58:08.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2921e-05 2024-09-14T23:58:08.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.346 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:08.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:35.103202+0000", 2024-09-14T23:58:08.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1992000000000001e-05 2024-09-14T23:58:08.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.346 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:08.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:35.103220+0000", 2024-09-14T23:58:08.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7312999999999998e-05 2024-09-14T23:58:08.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.347 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:08.347 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:35.103305+0000", 2024-09-14T23:58:08.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4975000000000005e-05 2024-09-14T23:58:08.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.347 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:08.347 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:35.105141+0000", 2024-09-14T23:58:08.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018368029999999999 2024-09-14T23:58:08.347 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.347 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:08.347 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.348 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:142751 3.4 e99/23 3:265f7763:::rbd_data.1118f9c96a7.0000000000000004:head v 99'441, mlcod=98'438)", 2024-09-14T23:58:08.348 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:53.165301+0000", 2024-09-14T23:58:08.348 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 15.101837416, 2024-09-14T23:58:08.348 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0021754449999999998, 2024-09-14T23:58:08.348 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:08.348 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:08.348 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:08.348 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:08.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.348 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:08.348 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.165301+0000", 2024-09-14T23:58:08.348 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.349 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:08.349 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.165301+0000", 2024-09-14T23:58:08.349 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.349 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:08.352 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.165302+0000", 2024-09-14T23:58:08.352 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0899999999999997e-07 2024-09-14T23:58:08.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.352 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:08.353 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.165310+0000", 2024-09-14T23:58:08.353 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7520000000000002e-06 2024-09-14T23:58:08.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:08.353 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.165321+0000", 2024-09-14T23:58:08.353 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0668e-05 2024-09-14T23:58:08.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:08.353 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.165330+0000", 2024-09-14T23:58:08.353 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.234e-06 2024-09-14T23:58:08.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.354 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:08.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.165393+0000", 2024-09-14T23:58:08.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2716000000000001e-05 2024-09-14T23:58:08.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.354 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:08.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.165418+0000", 2024-09-14T23:58:08.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4544000000000001e-05 2024-09-14T23:58:08.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.354 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:08.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.167436+0000", 2024-09-14T23:58:08.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002018597 2024-09-14T23:58:08.355 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.355 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.355 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:08.355 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.167477+0000", 2024-09-14T23:58:08.355 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0624999999999998e-05 2024-09-14T23:58:08.355 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.355 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:08.355 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.355 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.356 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.356 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:158983 3.4 e103/23 3:20e43b29:::rbd_data.1118b2f495e6.0000000000000012:head v 103'454, mlcod=103'452)", 2024-09-14T23:58:08.356 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:58.917699+0000", 2024-09-14T23:58:08.356 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.3494397889999998, 2024-09-14T23:58:08.356 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037933799999999998, 2024-09-14T23:58:08.356 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:08.356 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:08.356 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:08.356 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:08.356 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.356 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:08.356 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917699+0000", 2024-09-14T23:58:08.356 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.356 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.357 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.357 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:08.357 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917699+0000", 2024-09-14T23:58:08.357 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.357 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.357 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.357 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:08.357 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917700+0000", 2024-09-14T23:58:08.357 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3850000000000001e-06 2024-09-14T23:58:08.357 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.357 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.357 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:08.357 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917939+0000", 2024-09-14T23:58:08.358 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000238249 2024-09-14T23:58:08.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.358 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:08.358 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917961+0000", 2024-09-14T23:58:08.358 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2894000000000001e-05 2024-09-14T23:58:08.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.358 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:08.358 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917972+0000", 2024-09-14T23:58:08.358 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0261000000000001e-05 2024-09-14T23:58:08.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.358 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:08.359 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.918038+0000", 2024-09-14T23:58:08.359 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6223000000000006e-05 2024-09-14T23:58:08.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.359 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.359 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:08.359 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.918063+0000", 2024-09-14T23:58:08.359 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5252000000000001e-05 2024-09-14T23:58:08.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.359 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.359 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:08.359 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.921433+0000", 2024-09-14T23:58:08.359 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033699960000000001 2024-09-14T23:58:08.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.360 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.360 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:08.360 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.921492+0000", 2024-09-14T23:58:08.360 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9120000000000003e-05 2024-09-14T23:58:08.360 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.360 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:08.360 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.360 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.360 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.360 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4172.0:6 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T23:58:08.360 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.479840+0000", 2024-09-14T23:58:08.360 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.78729850899999998, 2024-09-14T23:58:08.360 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022238340000000001, 2024-09-14T23:58:08.361 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:08.361 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:08.361 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:08.361 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:08.361 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4172", 2024-09-14T23:58:08.361 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/2542269203", 2024-09-14T23:58:08.361 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 6 2024-09-14T23:58:08.361 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.361 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:08.361 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.361 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:08.361 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.479840+0000", 2024-09-14T23:58:08.361 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.361 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.362 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.362 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:08.362 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.479840+0000", 2024-09-14T23:58:08.362 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.362 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.362 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.362 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:08.362 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.479843+0000", 2024-09-14T23:58:08.362 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6680000000000001e-06 2024-09-14T23:58:08.362 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.362 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.362 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:08.362 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.479862+0000", 2024-09-14T23:58:08.363 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8762000000000001e-05 2024-09-14T23:58:08.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.363 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.363 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:08.363 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.479881+0000", 2024-09-14T23:58:08.363 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9860999999999998e-05 2024-09-14T23:58:08.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.363 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.363 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:08.363 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.479899+0000", 2024-09-14T23:58:08.363 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7127000000000001e-05 2024-09-14T23:58:08.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.363 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.363 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:08.364 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.481969+0000", 2024-09-14T23:58:08.364 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002070166 2024-09-14T23:58:08.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.364 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.364 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:08.364 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482064+0000", 2024-09-14T23:58:08.364 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5249999999999998e-05 2024-09-14T23:58:08.364 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.364 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:08.364 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.364 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.364 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:08.364 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:08.366 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T23:58:08.367 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:08.364+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:08.467 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:08.465+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:08.514 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:08.515 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:08.515 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:58:08.515 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:08.523 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T23:58:08.608 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:08.607+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:08.668 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:08.666+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:08.670 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:08.671 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:08.671 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:58:08.671 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:58:08.671 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:08.679 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T23:58:08.728 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T23:58:08.769 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:08.767+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:08.830 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:08.831 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:58:08.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:58:08.831 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:58:08.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.831 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5200 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2170880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:08.831 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253394+0000", 2024-09-14T23:58:08.831 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 89.574017202999997, 2024-09-14T23:58:08.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057428154000000002, 2024-09-14T23:58:08.831 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:08.831 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:08.831 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:08.831 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:08.831 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:08.831 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:08.832 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5200 2024-09-14T23:58:08.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.832 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:08.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:08.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253394+0000", 2024-09-14T23:58:08.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:08.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253394+0000", 2024-09-14T23:58:08.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:08.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253395+0000", 2024-09-14T23:58:08.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9100000000000004e-07 2024-09-14T23:58:08.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:08.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253397+0000", 2024-09-14T23:58:08.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2790000000000001e-06 2024-09-14T23:58:08.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:08.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253404+0000", 2024-09-14T23:58:08.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7959999999999998e-06 2024-09-14T23:58:08.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:08.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253409+0000", 2024-09-14T23:58:08.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1460000000000004e-06 2024-09-14T23:58:08.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:08.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310627+0000", 2024-09-14T23:58:08.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057218160999999997 2024-09-14T23:58:08.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:08.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310710+0000", 2024-09-14T23:58:08.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3102999999999994e-05 2024-09-14T23:58:08.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:08.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310822+0000", 2024-09-14T23:58:08.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000112178 2024-09-14T23:58:08.835 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.836 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:08.836 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.836 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5201 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2232320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:08.836 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:58:08.836 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 89.573992531000002, 2024-09-14T23:58:08.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057643228999999997, 2024-09-14T23:58:08.836 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:08.836 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:08.836 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:08.836 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:08.837 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:08.837 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:08.837 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5201 2024-09-14T23:58:08.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.837 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:08.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:08.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:58:08.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:08.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:58:08.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:08.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:58:08.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0699999999999998e-07 2024-09-14T23:58:08.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:08.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253422+0000", 2024-09-14T23:58:08.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.514e-06 2024-09-14T23:58:08.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:08.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253431+0000", 2024-09-14T23:58:08.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1919999999999994e-06 2024-09-14T23:58:08.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:08.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253436+0000", 2024-09-14T23:58:08.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5350000000000004e-06 2024-09-14T23:58:08.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:08.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310856+0000", 2024-09-14T23:58:08.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057419381999999998 2024-09-14T23:58:08.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:08.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310953+0000", 2024-09-14T23:58:08.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6774000000000006e-05 2024-09-14T23:58:08.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:08.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311062+0000", 2024-09-14T23:58:08.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000109425 2024-09-14T23:58:08.841 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.841 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:08.841 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.841 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5202 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2293760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:08.841 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:58:08.841 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 89.573967783000001, 2024-09-14T23:58:08.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057818255999999998, 2024-09-14T23:58:08.842 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:08.842 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:08.842 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:08.842 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:08.842 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:08.842 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:08.842 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5202 2024-09-14T23:58:08.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.842 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:08.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:08.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:58:08.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:08.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:58:08.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:08.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253573+0000", 2024-09-14T23:58:08.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012934000000000001 2024-09-14T23:58:08.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:08.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253576+0000", 2024-09-14T23:58:08.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5310000000000002e-06 2024-09-14T23:58:08.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:08.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253587+0000", 2024-09-14T23:58:08.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0522e-05 2024-09-14T23:58:08.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:08.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253595+0000", 2024-09-14T23:58:08.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8290000000000008e-06 2024-09-14T23:58:08.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:08.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311093+0000", 2024-09-14T23:58:08.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057498312000000003 2024-09-14T23:58:08.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:08.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311167+0000", 2024-09-14T23:58:08.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3582000000000001e-05 2024-09-14T23:58:08.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:08.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311262+0000", 2024-09-14T23:58:08.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5140000000000001e-05 2024-09-14T23:58:08.846 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.846 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:08.846 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.846 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5203 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2355200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:08.846 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:58:08.846 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 89.573805041, 2024-09-14T23:58:08.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057903938000000002, 2024-09-14T23:58:08.846 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:08.846 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:08.847 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:08.847 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:08.847 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:08.847 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:08.847 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5203 2024-09-14T23:58:08.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.847 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:08.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:08.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:58:08.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:08.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:58:08.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:08.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253902+0000", 2024-09-14T23:58:08.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029615400000000002 2024-09-14T23:58:08.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:08.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253906+0000", 2024-09-14T23:58:08.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9249999999999997e-06 2024-09-14T23:58:08.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:08.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253913+0000", 2024-09-14T23:58:08.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9099999999999999e-06 2024-09-14T23:58:08.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:08.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253917+0000", 2024-09-14T23:58:08.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9210000000000002e-06 2024-09-14T23:58:08.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:08.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311295+0000", 2024-09-14T23:58:08.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057378268000000003 2024-09-14T23:58:08.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:08.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311402+0000", 2024-09-14T23:58:08.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010644100000000001 2024-09-14T23:58:08.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:08.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311510+0000", 2024-09-14T23:58:08.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108319 2024-09-14T23:58:08.850 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.850 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:08.850 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.851 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5204 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2416640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:08.851 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:58:08.851 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 89.573486844000001, 2024-09-14T23:58:08.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057774860999999997, 2024-09-14T23:58:08.851 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:08.851 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:08.851 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:08.851 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:08.851 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:08.851 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:08.851 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5204 2024-09-14T23:58:08.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.851 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:08.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:08.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:58:08.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:08.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:58:08.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:08.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255903+0000", 2024-09-14T23:58:08.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001978807 2024-09-14T23:58:08.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:08.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255907+0000", 2024-09-14T23:58:08.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3610000000000001e-06 2024-09-14T23:58:08.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:08.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255914+0000", 2024-09-14T23:58:08.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.357e-06 2024-09-14T23:58:08.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:08.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255918+0000", 2024-09-14T23:58:08.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1779999999999996e-06 2024-09-14T23:58:08.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:08.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311541+0000", 2024-09-14T23:58:08.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055622697999999998 2024-09-14T23:58:08.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:08.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311614+0000", 2024-09-14T23:58:08.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3102999999999994e-05 2024-09-14T23:58:08.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:08.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311699+0000", 2024-09-14T23:58:08.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5357000000000003e-05 2024-09-14T23:58:08.855 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.855 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:08.855 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.855 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5211 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2846720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:08.855 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:58:08.855 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 89.571388630000001, 2024-09-14T23:58:08.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057528151, 2024-09-14T23:58:08.856 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:08.856 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:08.856 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:08.856 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:08.856 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:08.856 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:08.856 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5211 2024-09-14T23:58:08.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.856 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:08.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:08.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:58:08.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:08.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:58:08.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:08.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:58:08.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1399999999999998e-07 2024-09-14T23:58:08.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:08.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256024+0000", 2024-09-14T23:58:08.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.114e-06 2024-09-14T23:58:08.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:08.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256028+0000", 2024-09-14T23:58:08.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0840000000000002e-06 2024-09-14T23:58:08.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:08.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256031+0000", 2024-09-14T23:58:08.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6819999999999999e-06 2024-09-14T23:58:08.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:08.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313317+0000", 2024-09-14T23:58:08.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057286295000000001 2024-09-14T23:58:08.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:08.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313403+0000", 2024-09-14T23:58:08.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6287999999999998e-05 2024-09-14T23:58:08.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:08.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313551+0000", 2024-09-14T23:58:08.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000147374 2024-09-14T23:58:08.860 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.860 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:08.860 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.860 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5212 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2908160~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:08.860 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:58:08.860 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 89.571377436000006, 2024-09-14T23:58:08.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057735679999999998, 2024-09-14T23:58:08.860 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:08.860 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:08.861 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:08.861 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:08.861 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:08.861 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:08.861 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5212 2024-09-14T23:58:08.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.861 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:08.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:08.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:58:08.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:08.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:58:08.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:08.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:58:08.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0900000000000001e-07 2024-09-14T23:58:08.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:08.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256035+0000", 2024-09-14T23:58:08.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1200000000000001e-06 2024-09-14T23:58:08.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:08.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256038+0000", 2024-09-14T23:58:08.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.05e-06 2024-09-14T23:58:08.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:08.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256041+0000", 2024-09-14T23:58:08.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.667e-06 2024-09-14T23:58:08.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:08.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313584+0000", 2024-09-14T23:58:08.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057543484999999998 2024-09-14T23:58:08.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.864 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:08.864 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313668+0000", 2024-09-14T23:58:08.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3807000000000006e-05 2024-09-14T23:58:08.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.864 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:08.864 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313770+0000", 2024-09-14T23:58:08.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010134200000000001 2024-09-14T23:58:08.864 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.864 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:08.865 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.865 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5213 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2969600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:08.865 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:58:08.865 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 89.571367348999999, 2024-09-14T23:58:08.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058145467999999999, 2024-09-14T23:58:08.865 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:08.865 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:08.865 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:08.866 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:08.866 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:08.866 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:08.866 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5213 2024-09-14T23:58:08.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.866 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:08.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:08.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:58:08.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:08.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:58:08.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:08.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257901+0000", 2024-09-14T23:58:08.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018567519999999999 2024-09-14T23:58:08.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:08.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257904+0000", 2024-09-14T23:58:08.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4850000000000001e-06 2024-09-14T23:58:08.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.868 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:08.868 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257912+0000", 2024-09-14T23:58:08.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2090000000000005e-06 2024-09-14T23:58:08.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.868 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:08.869 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257918+0000", 2024-09-14T23:58:08.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6899999999999997e-06 2024-09-14T23:58:08.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.869 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:08.869 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313816+0000", 2024-09-14T23:58:08.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055898075999999998 2024-09-14T23:58:08.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.869 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:08.869 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.314082+0000", 2024-09-14T23:58:08.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000265386 2024-09-14T23:58:08.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:08.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.314189+0000", 2024-09-14T23:58:08.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010787 2024-09-14T23:58:08.870 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.870 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:08.870 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.870 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5218 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3276800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:08.870 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:58:08.870 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 89.56944034, 2024-09-14T23:58:08.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057497557999999997, 2024-09-14T23:58:08.871 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:08.871 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:08.871 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:08.871 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:08.871 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:08.871 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:08.871 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5218 2024-09-14T23:58:08.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.871 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:08.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:08.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:58:08.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:08.872 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:58:08.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:08.872 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:58:08.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1899999999999999e-07 2024-09-14T23:58:08.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.873 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:08.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257972+0000", 2024-09-14T23:58:08.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.217e-06 2024-09-14T23:58:08.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.873 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:08.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257975+0000", 2024-09-14T23:58:08.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9359999999999999e-06 2024-09-14T23:58:08.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.873 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:08.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257978+0000", 2024-09-14T23:58:08.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1049999999999999e-06 2024-09-14T23:58:08.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:08.874 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315331+0000", 2024-09-14T23:58:08.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057352093999999999 2024-09-14T23:58:08.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:08.874 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315384+0000", 2024-09-14T23:58:08.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3780999999999999e-05 2024-09-14T23:58:08.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:08.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315468+0000", 2024-09-14T23:58:08.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4206e-05 2024-09-14T23:58:08.875 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.875 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:08.875 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.875 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5219 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3338240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:08.875 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:58:08.875 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 89.569429663999998, 2024-09-14T23:58:08.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057637764000000001, 2024-09-14T23:58:08.875 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:08.875 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:08.876 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:08.876 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:08.876 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:08.876 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:08.876 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5219 2024-09-14T23:58:08.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.876 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:08.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:08.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:58:08.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:08.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:58:08.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:08.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:58:08.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.03e-07 2024-09-14T23:58:08.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:08.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257983+0000", 2024-09-14T23:58:08.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9199999999999999e-07 2024-09-14T23:58:08.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:08.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257986+0000", 2024-09-14T23:58:08.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0860000000000002e-06 2024-09-14T23:58:08.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:08.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257989+0000", 2024-09-14T23:58:08.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6589999999999999e-06 2024-09-14T23:58:08.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:08.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315509+0000", 2024-09-14T23:58:08.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057520262000000003 2024-09-14T23:58:08.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:08.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315554+0000", 2024-09-14T23:58:08.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5015999999999997e-05 2024-09-14T23:58:08.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:08.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315619+0000", 2024-09-14T23:58:08.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5545999999999999e-05 2024-09-14T23:58:08.879 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.879 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:08.880 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.880 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5220 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3399680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:08.880 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:58:08.880 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 89.569419707999998, 2024-09-14T23:58:08.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057792128999999998, 2024-09-14T23:58:08.880 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:08.880 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:08.880 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:08.880 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:08.880 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:08.880 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:08.880 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5220 2024-09-14T23:58:08.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.881 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:08.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:08.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:58:08.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:08.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:58:08.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.882 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:08.882 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:58:08.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1799999999999999e-07 2024-09-14T23:58:08.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.882 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:08.882 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257993+0000", 2024-09-14T23:58:08.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6200000000000006e-07 2024-09-14T23:58:08.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.882 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:08.882 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257996+0000", 2024-09-14T23:58:08.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9050000000000001e-06 2024-09-14T23:58:08.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.883 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:08.883 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257998+0000", 2024-09-14T23:58:08.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5950000000000001e-06 2024-09-14T23:58:08.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.883 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:08.883 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315672+0000", 2024-09-14T23:58:08.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057673637999999999 2024-09-14T23:58:08.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:08.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315734+0000", 2024-09-14T23:58:08.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2216000000000002e-05 2024-09-14T23:58:08.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:08.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315784+0000", 2024-09-14T23:58:08.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9595000000000002e-05 2024-09-14T23:58:08.884 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.884 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:08.884 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.885 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5221 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3461120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:08.885 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:58:08.885 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 89.569409989999997, 2024-09-14T23:58:08.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058003557999999997, 2024-09-14T23:58:08.885 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:08.885 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:08.885 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:08.885 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:08.885 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:08.885 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:08.885 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5221 2024-09-14T23:58:08.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.885 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:08.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:08.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:58:08.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:08.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:58:08.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:08.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259905+0000", 2024-09-14T23:58:08.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019035759999999999 2024-09-14T23:58:08.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:08.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259915+0000", 2024-09-14T23:58:08.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0518e-05 2024-09-14T23:58:08.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:08.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259925+0000", 2024-09-14T23:58:08.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7180000000000008e-06 2024-09-14T23:58:08.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:08.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259930+0000", 2024-09-14T23:58:08.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4769999999999997e-06 2024-09-14T23:58:08.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.888 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:08.888 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315820+0000", 2024-09-14T23:58:08.888 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055889960000000002 2024-09-14T23:58:08.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.888 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:08.888 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315914+0000", 2024-09-14T23:58:08.888 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4625999999999998e-05 2024-09-14T23:58:08.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.888 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:08.888 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.316005+0000", 2024-09-14T23:58:08.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0682999999999999e-05 2024-09-14T23:58:08.889 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.889 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:08.889 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.889 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5231 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 4075520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:08.889 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:58:08.889 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 89.567341256000006, 2024-09-14T23:58:08.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057524733000000002, 2024-09-14T23:58:08.889 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:08.889 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:08.889 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:08.890 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:08.890 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:08.890 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:08.890 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5231 2024-09-14T23:58:08.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.890 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:08.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:08.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:58:08.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:08.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:58:08.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:08.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261904+0000", 2024-09-14T23:58:08.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018338790000000001 2024-09-14T23:58:08.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:08.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261910+0000", 2024-09-14T23:58:08.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9689999999999999e-06 2024-09-14T23:58:08.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:08.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261923+0000", 2024-09-14T23:58:08.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3061000000000001e-05 2024-09-14T23:58:08.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:08.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261931+0000", 2024-09-14T23:58:08.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3159999999999997e-06 2024-09-14T23:58:08.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:08.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317456+0000", 2024-09-14T23:58:08.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055525219000000001 2024-09-14T23:58:08.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:08.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317520+0000", 2024-09-14T23:58:08.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4010000000000005e-05 2024-09-14T23:58:08.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:08.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317595+0000", 2024-09-14T23:58:08.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4278999999999998e-05 2024-09-14T23:58:08.893 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.893 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:08.893 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.894 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5236 3.8 3:1071668d:::rbd_data.1118de1168b.0000000000000092:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:08.894 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:58:08.894 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 89.565327792000005, 2024-09-14T23:58:08.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057499462000000001, 2024-09-14T23:58:08.894 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:08.894 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:08.894 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:08.894 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:08.894 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:08.894 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:08.894 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5236 2024-09-14T23:58:08.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.895 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:08.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:08.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:58:08.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:08.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:58:08.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:08.896 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262084+0000", 2024-09-14T23:58:08.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3399999999999999e-07 2024-09-14T23:58:08.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.896 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:08.896 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262087+0000", 2024-09-14T23:58:08.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2310000000000001e-06 2024-09-14T23:58:08.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.896 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:08.896 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262095+0000", 2024-09-14T23:58:08.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3799999999999996e-06 2024-09-14T23:58:08.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.897 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:08.897 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262101+0000", 2024-09-14T23:58:08.897 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0009999999999998e-06 2024-09-14T23:58:08.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.897 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:08.897 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317625+0000", 2024-09-14T23:58:08.897 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055524111000000001 2024-09-14T23:58:08.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.897 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:08.897 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317702+0000", 2024-09-14T23:58:08.898 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7533999999999993e-05 2024-09-14T23:58:08.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.898 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.898 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-14T23:58:08.898 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317892+0000", 2024-09-14T23:58:08.898 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018991400000000001 2024-09-14T23:58:08.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.898 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.898 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:58:08.898 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319407+0000", 2024-09-14T23:58:08.898 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015145250000000001 2024-09-14T23:58:08.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.898 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.899 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:08.899 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319481+0000", 2024-09-14T23:58:08.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3826999999999996e-05 2024-09-14T23:58:08.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.899 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:08.899 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319541+0000", 2024-09-14T23:58:08.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0470999999999998e-05 2024-09-14T23:58:08.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.899 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:08.899 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319583+0000", 2024-09-14T23:58:08.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1934000000000002e-05 2024-09-14T23:58:08.899 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.900 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:08.900 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.900 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:132622 3.1e 3.7a50b9e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-14T23:58:08.900 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:43.371285+0000", 2024-09-14T23:58:08.900 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 25.456125889999999, 2024-09-14T23:58:08.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.67037987200000004, 2024-09-14T23:58:08.900 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:08.900 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:08.900 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:08.900 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:08.901 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:08.901 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:08.901 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 132622 2024-09-14T23:58:08.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.901 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:08.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:08.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371285+0000", 2024-09-14T23:58:08.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:08.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371285+0000", 2024-09-14T23:58:08.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.902 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:08.902 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371286+0000", 2024-09-14T23:58:08.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6400000000000002e-07 2024-09-14T23:58:08.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.902 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:08.902 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371289+0000", 2024-09-14T23:58:08.903 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2059999999999999e-06 2024-09-14T23:58:08.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.903 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.903 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:08.903 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371299+0000", 2024-09-14T23:58:08.903 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0288e-05 2024-09-14T23:58:08.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.903 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.903 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:08.903 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371306+0000", 2024-09-14T23:58:08.903 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1650000000000003e-06 2024-09-14T23:58:08.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.903 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.904 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:08.904 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371617+0000", 2024-09-14T23:58:08.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000311043 2024-09-14T23:58:08.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.904 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:58:08.904 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371625+0000", 2024-09-14T23:58:08.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.833e-06 2024-09-14T23:58:08.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.904 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:08.904 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.041651+0000", 2024-09-14T23:58:08.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.67002580199999995 2024-09-14T23:58:08.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.905 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:08.905 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.041665+0000", 2024-09-14T23:58:08.905 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3971000000000001e-05 2024-09-14T23:58:08.905 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.905 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:08.905 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.905 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:169974 3.5 3:a704a2f0:::rbd_data.1118427720d0.0000000000000030:32 [read 965632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-14T23:58:08.905 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:03.433820+0000", 2024-09-14T23:58:08.905 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.3935912330000004, 2024-09-14T23:58:08.905 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057495521000000001, 2024-09-14T23:58:08.906 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:08.906 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:08.906 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:08.906 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:08.906 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:08.906 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:08.906 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 169974 2024-09-14T23:58:08.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.906 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:08.906 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.906 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:08.906 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.433820+0000", 2024-09-14T23:58:08.906 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.907 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:08.907 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.433820+0000", 2024-09-14T23:58:08.907 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.907 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:08.907 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.433925+0000", 2024-09-14T23:58:08.907 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000104621 2024-09-14T23:58:08.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.907 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:08.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.433929+0000", 2024-09-14T23:58:08.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2169999999999996e-06 2024-09-14T23:58:08.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.908 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:08.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.433939+0000", 2024-09-14T23:58:08.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0002e-05 2024-09-14T23:58:08.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.908 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:08.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.433946+0000", 2024-09-14T23:58:08.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7889999999999997e-06 2024-09-14T23:58:08.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:08.909 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.491144+0000", 2024-09-14T23:58:08.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057198221 2024-09-14T23:58:08.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:08.909 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.491221+0000", 2024-09-14T23:58:08.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6767000000000005e-05 2024-09-14T23:58:08.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:08.909 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.491316+0000", 2024-09-14T23:58:08.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4903999999999998e-05 2024-09-14T23:58:08.910 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.910 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:08.910 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.910 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.910 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:169975 3.5 3:a704a2f0:::rbd_data.1118427720d0.0000000000000030:32 [read 969728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-14T23:58:08.910 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:03.433958+0000", 2024-09-14T23:58:08.910 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.3934535339999998, 2024-09-14T23:58:08.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057564811, 2024-09-14T23:58:08.910 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:08.910 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:08.910 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:08.910 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:08.911 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:08.911 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:08.911 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 169975 2024-09-14T23:58:08.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.911 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:08.911 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.911 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:08.911 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.433958+0000", 2024-09-14T23:58:08.911 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.911 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.911 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:08.911 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.433958+0000", 2024-09-14T23:58:08.911 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.912 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.912 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.912 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:08.912 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.435096+0000", 2024-09-14T23:58:08.912 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001138708 2024-09-14T23:58:08.912 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.912 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.912 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:08.912 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.435101+0000", 2024-09-14T23:58:08.912 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5050000000000001e-06 2024-09-14T23:58:08.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.913 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:08.913 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.435111+0000", 2024-09-14T23:58:08.913 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5810000000000003e-06 2024-09-14T23:58:08.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.913 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:08.913 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.435120+0000", 2024-09-14T23:58:08.913 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0089999999999996e-06 2024-09-14T23:58:08.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.913 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:08.913 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.491345+0000", 2024-09-14T23:58:08.914 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056225479000000002 2024-09-14T23:58:08.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.914 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.914 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:08.914 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.491423+0000", 2024-09-14T23:58:08.914 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7990000000000004e-05 2024-09-14T23:58:08.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.914 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.914 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:08.914 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.491523+0000", 2024-09-14T23:58:08.914 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9538999999999997e-05 2024-09-14T23:58:08.914 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.914 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:08.915 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.915 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.915 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:172639 3.5 3:a704a2f0:::rbd_data.1118427720d0.0000000000000030:32 [read 1645568~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e107)", 2024-09-14T23:58:08.915 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:03.988365+0000", 2024-09-14T23:58:08.915 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.8390458140000003, 2024-09-14T23:58:08.915 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057556213000000002, 2024-09-14T23:58:08.915 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:08.915 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:08.915 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:08.915 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:08.915 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:08.915 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:08.915 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 172639 2024-09-14T23:58:08.916 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.916 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:08.916 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.916 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:08.916 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.988365+0000", 2024-09-14T23:58:08.916 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.916 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.916 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.916 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:08.916 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.988365+0000", 2024-09-14T23:58:08.916 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.916 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.916 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.916 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:08.917 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.989904+0000", 2024-09-14T23:58:08.917 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015380859999999999 2024-09-14T23:58:08.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.917 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.917 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:08.917 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.989909+0000", 2024-09-14T23:58:08.917 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2800000000000003e-06 2024-09-14T23:58:08.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.917 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.917 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:08.917 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.989925+0000", 2024-09-14T23:58:08.917 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5781999999999999e-05 2024-09-14T23:58:08.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.917 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.918 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:08.918 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.989933+0000", 2024-09-14T23:58:08.918 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0090000000000007e-06 2024-09-14T23:58:08.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.918 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.918 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:08.918 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:04.045741+0000", 2024-09-14T23:58:08.918 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055808112999999999 2024-09-14T23:58:08.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.918 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.918 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:08.918 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:04.045812+0000", 2024-09-14T23:58:08.918 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1531000000000003e-05 2024-09-14T23:58:08.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.919 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.919 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:08.919 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:04.045922+0000", 2024-09-14T23:58:08.919 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000109412 2024-09-14T23:58:08.919 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.919 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:08.919 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.919 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.919 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184571 3.2 3:46f8d78b:::rbd_data.1118427720d0.0000000000000001:32 [read 1405952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:08.919 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.086404+0000", 2024-09-14T23:58:08.919 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 1.741007457, 2024-09-14T23:58:08.919 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059148573000000003, 2024-09-14T23:58:08.920 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:08.920 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:08.920 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:08.920 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:08.920 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:08.920 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:08.920 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184571 2024-09-14T23:58:08.920 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.920 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:08.920 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.920 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:08.920 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.086404+0000", 2024-09-14T23:58:08.920 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.920 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.920 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.921 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:08.921 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.086404+0000", 2024-09-14T23:58:08.921 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.921 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.921 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.921 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:08.921 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087902+0000", 2024-09-14T23:58:08.921 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001498625 2024-09-14T23:58:08.921 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.921 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.921 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:08.921 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087907+0000", 2024-09-14T23:58:08.921 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9130000000000002e-06 2024-09-14T23:58:08.922 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.922 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.922 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:08.922 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087919+0000", 2024-09-14T23:58:08.922 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1719000000000001e-05 2024-09-14T23:58:08.922 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.922 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.922 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:08.922 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087927+0000", 2024-09-14T23:58:08.922 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3699999999999995e-06 2024-09-14T23:58:08.922 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.922 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.922 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:08.923 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143711+0000", 2024-09-14T23:58:08.923 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055783396999999998 2024-09-14T23:58:08.923 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.923 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.923 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:08.923 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143792+0000", 2024-09-14T23:58:08.923 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1292999999999996e-05 2024-09-14T23:58:08.923 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.923 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.924 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:08.924 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145552+0000", 2024-09-14T23:58:08.924 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001760256 2024-09-14T23:58:08.924 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.924 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:08.924 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.924 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.924 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.924 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184579 3.2 3:43badc91:::rbd_data.1118427720d0.0000000000000002:32 [read 1196032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:08.924 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.088145+0000", 2024-09-14T23:58:08.924 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 1.7392660530000001, 2024-09-14T23:58:08.924 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057643169000000001, 2024-09-14T23:58:08.924 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:08.925 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:08.925 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:08.925 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:08.925 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:08.925 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:08.925 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184579 2024-09-14T23:58:08.925 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.925 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:08.925 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.925 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:08.925 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.088145+0000", 2024-09-14T23:58:08.925 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.925 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.926 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.926 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:08.926 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.088145+0000", 2024-09-14T23:58:08.926 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:08.926 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.926 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.926 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:08.926 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089947+0000", 2024-09-14T23:58:08.926 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001801764 2024-09-14T23:58:08.926 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.926 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.926 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:08.927 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089952+0000", 2024-09-14T23:58:08.927 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8659999999999997e-06 2024-09-14T23:58:08.927 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.927 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.927 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:08.927 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089966+0000", 2024-09-14T23:58:08.927 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3929e-05 2024-09-14T23:58:08.927 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.927 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.927 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:08.927 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089974+0000", 2024-09-14T23:58:08.927 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1429999999999998e-06 2024-09-14T23:58:08.927 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.927 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.928 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:08.928 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145636+0000", 2024-09-14T23:58:08.928 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055662287999999997 2024-09-14T23:58:08.928 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.928 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.928 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:08.928 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145710+0000", 2024-09-14T23:58:08.928 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4166999999999996e-05 2024-09-14T23:58:08.928 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:08.928 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:08.928 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:08.928 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145788+0000", 2024-09-14T23:58:08.928 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8011999999999998e-05 2024-09-14T23:58:08.929 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.929 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:08.929 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.929 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:08.929 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:08.929 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:08.930 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:08.928+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:08.931 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T23:58:09.032 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:58:09.029+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:58:09.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-14T23:58:09.067+0000 7f2095d9b640 0 --2- 172.21.15.42:0/4283570152 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f20980b1920 0x7f2098071e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-14T23:58:09.108 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:58:09.109 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:58:09.116 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:09.116 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:58:09.116 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.116 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188164 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.116 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.030578+0000", 2024-09-14T23:58:09.116 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.063282959, 2024-09-14T23:58:09.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.063304851999999995, 2024-09-14T23:58:09.117 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.117 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.117 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.117 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.117 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.117 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.117 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188164 2024-09-14T23:58:09.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.117 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.030578+0000", 2024-09-14T23:58:09.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.118 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.118 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.118 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.030578+0000", 2024-09-14T23:58:09.118 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.118 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.118 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.118 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.030578+0000", 2024-09-14T23:58:09.118 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.39e-07 2024-09-14T23:58:09.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.118 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.118 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.118 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.030580+0000", 2024-09-14T23:58:09.118 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1009999999999999e-06 2024-09-14T23:58:09.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.119 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.119 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.030586+0000", 2024-09-14T23:58:09.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7760000000000003e-06 2024-09-14T23:58:09.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.119 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.119 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.030591+0000", 2024-09-14T23:58:09.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9200000000000003e-06 2024-09-14T23:58:09.119 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.119 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.119 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.120 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.120 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188199 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.120 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.032994+0000", 2024-09-14T23:58:09.120 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.060866423000000003, 2024-09-14T23:58:09.120 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.060956264000000003, 2024-09-14T23:58:09.120 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.120 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.120 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.120 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.120 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.120 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.120 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188199 2024-09-14T23:58:09.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.121 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.121 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.121 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.121 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032994+0000", 2024-09-14T23:58:09.121 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.121 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.121 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.121 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032994+0000", 2024-09-14T23:58:09.121 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.121 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.121 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.121 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032995+0000", 2024-09-14T23:58:09.122 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7599999999999997e-07 2024-09-14T23:58:09.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.122 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.122 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.122 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032997+0000", 2024-09-14T23:58:09.122 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5189999999999999e-06 2024-09-14T23:58:09.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.122 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.122 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.122 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033004+0000", 2024-09-14T23:58:09.122 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8800000000000002e-06 2024-09-14T23:58:09.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.122 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.122 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033010+0000", 2024-09-14T23:58:09.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7100000000000004e-06 2024-09-14T23:58:09.123 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.123 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.123 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.123 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.123 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188674 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.123 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.055972+0000", 2024-09-14T23:58:09.123 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.037888935999999998, 2024-09-14T23:58:09.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.038040414000000002, 2024-09-14T23:58:09.123 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.123 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.123 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.124 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.124 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.124 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.124 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188674 2024-09-14T23:58:09.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.124 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.124 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.124 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.055972+0000", 2024-09-14T23:58:09.124 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.124 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.124 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.055972+0000", 2024-09-14T23:58:09.125 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.125 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.125 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.125 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.055972+0000", 2024-09-14T23:58:09.125 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2599999999999998e-07 2024-09-14T23:58:09.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.125 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.125 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.125 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.055974+0000", 2024-09-14T23:58:09.125 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1610000000000001e-06 2024-09-14T23:58:09.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.126 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.126 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.126 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.055983+0000", 2024-09-14T23:58:09.126 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7679999999999997e-06 2024-09-14T23:58:09.126 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.126 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.126 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.126 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.055989+0000", 2024-09-14T23:58:09.126 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5500000000000002e-06 2024-09-14T23:58:09.126 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.126 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.126 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.126 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.126 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.127 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188711 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.127 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.058969+0000", 2024-09-14T23:58:09.127 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.034891577999999999, 2024-09-14T23:58:09.127 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.035101602000000003, 2024-09-14T23:58:09.127 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.127 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.127 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.127 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.127 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.127 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.127 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188711 2024-09-14T23:58:09.127 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.127 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.127 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.128 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.128 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058969+0000", 2024-09-14T23:58:09.128 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.128 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.128 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.128 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058969+0000", 2024-09-14T23:58:09.128 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.128 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.128 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.128 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058970+0000", 2024-09-14T23:58:09.128 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3799999999999998e-07 2024-09-14T23:58:09.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.129 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.129 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.129 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058972+0000", 2024-09-14T23:58:09.129 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2419999999999999e-06 2024-09-14T23:58:09.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.129 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.129 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.129 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058978+0000", 2024-09-14T23:58:09.129 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1510000000000001e-06 2024-09-14T23:58:09.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.129 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.129 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.129 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058983+0000", 2024-09-14T23:58:09.129 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8400000000000002e-06 2024-09-14T23:58:09.130 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.130 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.130 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.130 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.130 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.130 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189185 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.130 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.084106+0000", 2024-09-14T23:58:09.130 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0097543620000000008, 2024-09-14T23:58:09.130 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010024319, 2024-09-14T23:58:09.130 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.130 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.130 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.130 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.130 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.131 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.131 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189185 2024-09-14T23:58:09.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.131 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.131 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.131 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.131 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.084106+0000", 2024-09-14T23:58:09.131 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.131 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.131 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.131 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.084106+0000", 2024-09-14T23:58:09.131 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.132 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.132 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.132 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.084107+0000", 2024-09-14T23:58:09.132 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0900000000000001e-07 2024-09-14T23:58:09.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.132 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.132 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.132 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.084108+0000", 2024-09-14T23:58:09.132 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.105e-06 2024-09-14T23:58:09.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.132 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.132 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.132 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.084111+0000", 2024-09-14T23:58:09.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9569999999999998e-06 2024-09-14T23:58:09.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.133 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.133 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.084113+0000", 2024-09-14T23:58:09.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.396e-06 2024-09-14T23:58:09.133 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.133 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.133 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.133 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189224 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.133 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.087957+0000", 2024-09-14T23:58:09.133 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0059039280000000001, 2024-09-14T23:58:09.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0062358370000000002, 2024-09-14T23:58:09.134 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.134 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.134 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.134 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.134 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.134 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.134 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189224 2024-09-14T23:58:09.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.134 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.135 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087957+0000", 2024-09-14T23:58:09.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.135 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087957+0000", 2024-09-14T23:58:09.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.135 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087957+0000", 2024-09-14T23:58:09.136 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.82e-07 2024-09-14T23:58:09.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.136 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.136 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.136 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087960+0000", 2024-09-14T23:58:09.136 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4219999999999999e-06 2024-09-14T23:58:09.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.136 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.136 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.136 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087967+0000", 2024-09-14T23:58:09.136 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7819999999999996e-06 2024-09-14T23:58:09.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.136 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.136 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.137 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087972+0000", 2024-09-14T23:58:09.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6489999999999996e-06 2024-09-14T23:58:09.137 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.137 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.137 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.137 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189352 3.1d 3.e507d9bd (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.137 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.092089+0000", 2024-09-14T23:58:09.137 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.001771402, 2024-09-14T23:58:09.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0021615940000000002, 2024-09-14T23:58:09.137 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.137 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.138 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.138 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.138 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.138 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.138 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189352 2024-09-14T23:58:09.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.138 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.092089+0000", 2024-09-14T23:58:09.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.092089+0000", 2024-09-14T23:58:09.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.139 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.092090+0000", 2024-09-14T23:58:09.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5300000000000004e-07 2024-09-14T23:58:09.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.139 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.092092+0000", 2024-09-14T23:58:09.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.103e-06 2024-09-14T23:58:09.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.140 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.140 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.092098+0000", 2024-09-14T23:58:09.140 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8109999999999999e-06 2024-09-14T23:58:09.140 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.140 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.140 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.140 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.092104+0000", 2024-09-14T23:58:09.140 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1070000000000001e-06 2024-09-14T23:58:09.140 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.140 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.140 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.140 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.140 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.141 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188166 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.141 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.031979+0000", 2024-09-14T23:58:09.141 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.061881848000000003, 2024-09-14T23:58:09.141 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.062330036999999998, 2024-09-14T23:58:09.141 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.141 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.141 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.141 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.141 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.141 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.141 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188166 2024-09-14T23:58:09.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.141 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.142 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.142 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.031979+0000", 2024-09-14T23:58:09.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.142 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.142 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.031979+0000", 2024-09-14T23:58:09.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.142 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.142 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.031980+0000", 2024-09-14T23:58:09.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.054e-06 2024-09-14T23:58:09.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.143 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.031986+0000", 2024-09-14T23:58:09.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9920000000000004e-06 2024-09-14T23:58:09.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.143 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.031996+0000", 2024-09-14T23:58:09.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0299000000000001e-05 2024-09-14T23:58:09.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.143 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032003+0000", 2024-09-14T23:58:09.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1289999999999998e-06 2024-09-14T23:58:09.143 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.144 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.144 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.144 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.144 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188200 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.144 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.033017+0000", 2024-09-14T23:58:09.144 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.060843996999999997, 2024-09-14T23:58:09.144 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.061350596, 2024-09-14T23:58:09.144 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.144 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.144 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.144 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.144 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.144 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.145 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188200 2024-09-14T23:58:09.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.145 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.145 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.145 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.145 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033017+0000", 2024-09-14T23:58:09.145 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.145 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.145 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.145 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033017+0000", 2024-09-14T23:58:09.145 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.146 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033017+0000", 2024-09-14T23:58:09.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2300000000000002e-07 2024-09-14T23:58:09.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.146 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033019+0000", 2024-09-14T23:58:09.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9599999999999999e-06 2024-09-14T23:58:09.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.146 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033031+0000", 2024-09-14T23:58:09.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1568e-05 2024-09-14T23:58:09.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.147 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.147 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033043+0000", 2024-09-14T23:58:09.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2157999999999999e-05 2024-09-14T23:58:09.147 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.147 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.147 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.147 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188676 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.147 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.055996+0000", 2024-09-14T23:58:09.147 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.037864680999999997, 2024-09-14T23:58:09.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.038430201999999997, 2024-09-14T23:58:09.147 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.147 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.148 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.148 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.148 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.148 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.148 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188676 2024-09-14T23:58:09.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.148 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.148 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.148 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.148 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.055996+0000", 2024-09-14T23:58:09.148 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.148 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.148 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.149 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.055996+0000", 2024-09-14T23:58:09.149 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.149 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.149 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.055997+0000", 2024-09-14T23:58:09.149 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5400000000000002e-07 2024-09-14T23:58:09.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.149 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.149 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056004+0000", 2024-09-14T23:58:09.149 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9369999999999998e-06 2024-09-14T23:58:09.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056011+0000", 2024-09-14T23:58:09.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.875e-06 2024-09-14T23:58:09.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056017+0000", 2024-09-14T23:58:09.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2900000000000002e-06 2024-09-14T23:58:09.150 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.150 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.150 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.150 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188712 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.150 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.058990+0000", 2024-09-14T23:58:09.151 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.034871161999999997, 2024-09-14T23:58:09.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.035523849000000003, 2024-09-14T23:58:09.151 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.151 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.151 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.151 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.151 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.151 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.151 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188712 2024-09-14T23:58:09.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.151 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058990+0000", 2024-09-14T23:58:09.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.152 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.152 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058990+0000", 2024-09-14T23:58:09.152 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.152 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.152 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058990+0000", 2024-09-14T23:58:09.152 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6000000000000004e-07 2024-09-14T23:58:09.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.152 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.152 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058993+0000", 2024-09-14T23:58:09.153 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3350000000000001e-06 2024-09-14T23:58:09.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.153 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.153 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.153 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.059000+0000", 2024-09-14T23:58:09.153 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5109999999999997e-06 2024-09-14T23:58:09.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.153 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.153 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.153 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.059005+0000", 2024-09-14T23:58:09.153 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0950000000000003e-06 2024-09-14T23:58:09.153 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.153 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.153 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.154 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.154 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189187 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.154 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.084116+0000", 2024-09-14T23:58:09.154 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0097446479999999999, 2024-09-14T23:58:09.154 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010456983, 2024-09-14T23:58:09.154 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.154 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.154 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.154 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.154 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.154 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.154 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189187 2024-09-14T23:58:09.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.154 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.084116+0000", 2024-09-14T23:58:09.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.084116+0000", 2024-09-14T23:58:09.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.084116+0000", 2024-09-14T23:58:09.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2600000000000001e-07 2024-09-14T23:58:09.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.156 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.084117+0000", 2024-09-14T23:58:09.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8899999999999998e-07 2024-09-14T23:58:09.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.156 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.084120+0000", 2024-09-14T23:58:09.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0769999999999999e-06 2024-09-14T23:58:09.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.156 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.084123+0000", 2024-09-14T23:58:09.157 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4830000000000002e-06 2024-09-14T23:58:09.157 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.157 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.157 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.157 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.157 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189225 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.157 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.087980+0000", 2024-09-14T23:58:09.157 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0058812919999999998, 2024-09-14T23:58:09.157 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0066512309999999996, 2024-09-14T23:58:09.157 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.157 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.157 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.157 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.157 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.158 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.158 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189225 2024-09-14T23:58:09.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.158 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.158 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.158 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.158 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087980+0000", 2024-09-14T23:58:09.158 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.158 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.158 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.158 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087980+0000", 2024-09-14T23:58:09.158 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.158 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.159 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.159 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087980+0000", 2024-09-14T23:58:09.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0499999999999999e-07 2024-09-14T23:58:09.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.159 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.159 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087982+0000", 2024-09-14T23:58:09.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2850000000000001e-06 2024-09-14T23:58:09.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.159 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.159 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087989+0000", 2024-09-14T23:58:09.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3389999999999998e-06 2024-09-14T23:58:09.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.160 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.160 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087994+0000", 2024-09-14T23:58:09.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9849999999999997e-06 2024-09-14T23:58:09.160 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.160 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.160 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.160 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.160 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189353 3.1d 3.e507d9bd (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.160 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.092111+0000", 2024-09-14T23:58:09.160 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.001749918, 2024-09-14T23:58:09.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00258318, 2024-09-14T23:58:09.160 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.160 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.161 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.161 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.161 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.161 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.161 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189353 2024-09-14T23:58:09.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.161 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.161 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.161 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.161 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.092111+0000", 2024-09-14T23:58:09.161 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.161 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.161 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.162 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.092111+0000", 2024-09-14T23:58:09.162 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.162 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.162 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.162 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.162 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.092111+0000", 2024-09-14T23:58:09.162 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6300000000000005e-07 2024-09-14T23:58:09.162 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.162 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.162 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.162 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.092113+0000", 2024-09-14T23:58:09.162 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0169999999999999e-06 2024-09-14T23:58:09.162 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.162 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.163 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.163 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.092120+0000", 2024-09-14T23:58:09.163 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5370000000000003e-06 2024-09-14T23:58:09.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.163 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.163 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.092125+0000", 2024-09-14T23:58:09.163 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3349999999999997e-06 2024-09-14T23:58:09.163 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.163 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.163 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.164 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188168 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.164 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.032015+0000", 2024-09-14T23:58:09.164 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.061845323000000001, 2024-09-14T23:58:09.164 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.062741559000000002, 2024-09-14T23:58:09.164 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.164 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.164 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.164 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.164 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.164 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.164 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188168 2024-09-14T23:58:09.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.164 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.164 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.165 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.165 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032015+0000", 2024-09-14T23:58:09.165 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.165 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.165 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.165 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.165 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032015+0000", 2024-09-14T23:58:09.165 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.165 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.165 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.165 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.165 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032016+0000", 2024-09-14T23:58:09.165 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8999999999999997e-07 2024-09-14T23:58:09.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.166 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.166 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.166 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032025+0000", 2024-09-14T23:58:09.166 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1379999999999996e-06 2024-09-14T23:58:09.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.166 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.166 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.166 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032033+0000", 2024-09-14T23:58:09.166 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6380000000000004e-06 2024-09-14T23:58:09.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.166 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.166 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.166 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032045+0000", 2024-09-14T23:58:09.167 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2591e-05 2024-09-14T23:58:09.167 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.167 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.167 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.167 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.167 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.167 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188201 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.167 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.033052+0000", 2024-09-14T23:58:09.167 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.060809087999999997, 2024-09-14T23:58:09.167 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.061764670000000001, 2024-09-14T23:58:09.167 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.167 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.167 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.167 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.168 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.168 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.168 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188201 2024-09-14T23:58:09.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.168 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.168 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.168 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.168 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033052+0000", 2024-09-14T23:58:09.168 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.168 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.168 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.168 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033052+0000", 2024-09-14T23:58:09.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.169 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033052+0000", 2024-09-14T23:58:09.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2900000000000004e-07 2024-09-14T23:58:09.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.169 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.173 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033055+0000", 2024-09-14T23:58:09.173 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7209999999999999e-06 2024-09-14T23:58:09.173 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.173 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.173 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.173 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033063+0000", 2024-09-14T23:58:09.173 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7249999999999997e-06 2024-09-14T23:58:09.173 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.173 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.173 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.173 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033069+0000", 2024-09-14T23:58:09.173 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.835e-06 2024-09-14T23:58:09.173 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.174 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.174 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.174 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.174 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188678 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.174 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.056025+0000", 2024-09-14T23:58:09.174 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.037836294999999999, 2024-09-14T23:58:09.174 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.038849898000000001, 2024-09-14T23:58:09.174 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.174 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.174 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.174 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.174 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.174 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.174 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188678 2024-09-14T23:58:09.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.175 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.175 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.175 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.175 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056025+0000", 2024-09-14T23:58:09.175 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.175 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.175 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.175 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056025+0000", 2024-09-14T23:58:09.175 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.175 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.175 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.175 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056025+0000", 2024-09-14T23:58:09.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0900000000000002e-07 2024-09-14T23:58:09.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.176 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.176 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.176 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056027+0000", 2024-09-14T23:58:09.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5000000000000002e-06 2024-09-14T23:58:09.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.176 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.176 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.176 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056034+0000", 2024-09-14T23:58:09.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7560000000000002e-06 2024-09-14T23:58:09.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.176 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.176 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.177 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056040+0000", 2024-09-14T23:58:09.177 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4029999999999998e-06 2024-09-14T23:58:09.177 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.177 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.177 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.177 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188713 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.177 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.059012+0000", 2024-09-14T23:58:09.177 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.034849152000000001, 2024-09-14T23:58:09.177 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.035924101999999999, 2024-09-14T23:58:09.177 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.177 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.177 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.178 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.178 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.178 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.178 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188713 2024-09-14T23:58:09.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.178 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.178 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.178 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.059012+0000", 2024-09-14T23:58:09.178 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.178 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.178 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.059012+0000", 2024-09-14T23:58:09.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.179 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.179 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.179 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.059012+0000", 2024-09-14T23:58:09.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4499999999999997e-07 2024-09-14T23:58:09.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.179 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.179 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.179 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.059014+0000", 2024-09-14T23:58:09.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2989999999999999e-06 2024-09-14T23:58:09.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.179 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.179 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.179 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.059020+0000", 2024-09-14T23:58:09.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9769999999999997e-06 2024-09-14T23:58:09.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.180 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.180 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.180 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.059026+0000", 2024-09-14T23:58:09.180 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2020000000000003e-06 2024-09-14T23:58:09.180 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.180 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.180 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.180 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.180 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189189 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.180 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.084126+0000", 2024-09-14T23:58:09.180 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0097350479999999996, 2024-09-14T23:58:09.180 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010866948, 2024-09-14T23:58:09.181 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.181 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.181 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.181 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.181 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.181 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.181 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189189 2024-09-14T23:58:09.181 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.181 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.181 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.181 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.181 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.084126+0000", 2024-09-14T23:58:09.181 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.181 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.182 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.182 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.182 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.084126+0000", 2024-09-14T23:58:09.182 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.182 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.182 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.182 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.084238+0000", 2024-09-14T23:58:09.182 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011213 2024-09-14T23:58:09.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.182 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.182 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.182 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.084242+0000", 2024-09-14T23:58:09.182 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8809999999999998e-06 2024-09-14T23:58:09.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.183 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.183 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.183 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.084252+0000", 2024-09-14T23:58:09.183 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7739999999999999e-06 2024-09-14T23:58:09.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.183 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.183 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.183 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.084258+0000", 2024-09-14T23:58:09.183 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6850000000000004e-06 2024-09-14T23:58:09.183 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.183 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.183 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.184 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189226 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.184 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.088001+0000", 2024-09-14T23:58:09.184 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0058602130000000004, 2024-09-14T23:58:09.184 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0070476760000000001, 2024-09-14T23:58:09.184 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.184 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.184 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.184 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.184 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.184 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.184 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189226 2024-09-14T23:58:09.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.185 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.185 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.185 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.185 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088001+0000", 2024-09-14T23:58:09.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.185 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.185 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.185 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088001+0000", 2024-09-14T23:58:09.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.185 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.185 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.185 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088001+0000", 2024-09-14T23:58:09.186 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5999999999999999e-07 2024-09-14T23:58:09.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.186 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.186 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.186 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088003+0000", 2024-09-14T23:58:09.186 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1639999999999999e-06 2024-09-14T23:58:09.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.186 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.186 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.186 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088009+0000", 2024-09-14T23:58:09.186 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0209999999999996e-06 2024-09-14T23:58:09.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.186 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.186 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.187 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088014+0000", 2024-09-14T23:58:09.187 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8949999999999997e-06 2024-09-14T23:58:09.187 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.187 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.187 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.187 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.187 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189354 3.1d 3.e507d9bd (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.187 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.092132+0000", 2024-09-14T23:58:09.187 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0017290669999999999, 2024-09-14T23:58:09.187 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.002974389, 2024-09-14T23:58:09.187 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.187 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.187 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.188 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.188 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.188 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.188 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189354 2024-09-14T23:58:09.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.188 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.188 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.188 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.188 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.092132+0000", 2024-09-14T23:58:09.188 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.188 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.188 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.189 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.092132+0000", 2024-09-14T23:58:09.189 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.189 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.189 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.189 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.092132+0000", 2024-09-14T23:58:09.189 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1999999999999999e-07 2024-09-14T23:58:09.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.189 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.189 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.189 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.092135+0000", 2024-09-14T23:58:09.189 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.272e-06 2024-09-14T23:58:09.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.189 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.190 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.190 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.092141+0000", 2024-09-14T23:58:09.190 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2870000000000001e-06 2024-09-14T23:58:09.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.190 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.190 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.190 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.092146+0000", 2024-09-14T23:58:09.190 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1429999999999997e-06 2024-09-14T23:58:09.190 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.190 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.190 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.190 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.191 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188170 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.191 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.032060+0000", 2024-09-14T23:58:09.191 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.061800538000000002, 2024-09-14T23:58:09.191 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.063097295999999997, 2024-09-14T23:58:09.191 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.191 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.191 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.191 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.191 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.191 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.191 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188170 2024-09-14T23:58:09.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.191 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.192 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.192 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.192 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032060+0000", 2024-09-14T23:58:09.192 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.192 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.192 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.192 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032060+0000", 2024-09-14T23:58:09.192 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.192 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.192 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.192 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032061+0000", 2024-09-14T23:58:09.192 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4300000000000003e-07 2024-09-14T23:58:09.193 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.193 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.193 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.193 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032064+0000", 2024-09-14T23:58:09.193 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0299999999999998e-06 2024-09-14T23:58:09.193 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.193 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.193 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.193 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032073+0000", 2024-09-14T23:58:09.193 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7499999999999992e-06 2024-09-14T23:58:09.193 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.193 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.193 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.193 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032078+0000", 2024-09-14T23:58:09.194 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7810000000000002e-06 2024-09-14T23:58:09.194 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.194 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.194 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.194 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.194 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188202 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.194 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.033075+0000", 2024-09-14T23:58:09.194 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.060785374000000003, 2024-09-14T23:58:09.194 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.062141804000000002, 2024-09-14T23:58:09.194 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.194 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.194 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.194 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.195 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.195 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.195 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188202 2024-09-14T23:58:09.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.195 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.195 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.195 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.195 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033075+0000", 2024-09-14T23:58:09.195 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.195 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.195 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.195 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033075+0000", 2024-09-14T23:58:09.195 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.196 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.196 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.196 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033076+0000", 2024-09-14T23:58:09.196 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9900000000000001e-07 2024-09-14T23:58:09.196 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.196 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.196 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.196 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033078+0000", 2024-09-14T23:58:09.196 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2359999999999999e-06 2024-09-14T23:58:09.196 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.196 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.196 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.196 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033085+0000", 2024-09-14T23:58:09.197 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8140000000000003e-06 2024-09-14T23:58:09.197 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.197 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.197 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.197 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033090+0000", 2024-09-14T23:58:09.197 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3569999999999997e-06 2024-09-14T23:58:09.197 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.197 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.197 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.197 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.197 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.197 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188680 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.197 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.056046+0000", 2024-09-14T23:58:09.198 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.037814754999999999, 2024-09-14T23:58:09.198 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.039228720000000002, 2024-09-14T23:58:09.198 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.198 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.198 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.198 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.198 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.198 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.198 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188680 2024-09-14T23:58:09.198 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.198 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.198 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.198 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.199 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056046+0000", 2024-09-14T23:58:09.199 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.199 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.199 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.199 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056046+0000", 2024-09-14T23:58:09.199 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.199 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.199 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.199 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056046+0000", 2024-09-14T23:58:09.199 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.89e-07 2024-09-14T23:58:09.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.200 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.200 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.200 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056048+0000", 2024-09-14T23:58:09.200 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0470000000000001e-06 2024-09-14T23:58:09.200 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.200 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.200 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.201 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056056+0000", 2024-09-14T23:58:09.201 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3969999999999997e-06 2024-09-14T23:58:09.201 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.201 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.201 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.201 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056061+0000", 2024-09-14T23:58:09.201 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.553e-06 2024-09-14T23:58:09.201 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.201 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.201 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.201 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.201 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.202 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188714 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.202 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.059032+0000", 2024-09-14T23:58:09.202 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.034828333000000003, 2024-09-14T23:58:09.202 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.036301630000000001, 2024-09-14T23:58:09.202 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.202 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.202 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.202 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.202 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.202 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.202 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188714 2024-09-14T23:58:09.202 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.202 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.202 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.203 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.203 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.059032+0000", 2024-09-14T23:58:09.203 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.203 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.203 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.203 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.059032+0000", 2024-09-14T23:58:09.203 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.203 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.203 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.203 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.059033+0000", 2024-09-14T23:58:09.203 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.46e-07 2024-09-14T23:58:09.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.204 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.204 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.204 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.059035+0000", 2024-09-14T23:58:09.204 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2790000000000001e-06 2024-09-14T23:58:09.204 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.204 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.204 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.204 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.059042+0000", 2024-09-14T23:58:09.204 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4810000000000003e-06 2024-09-14T23:58:09.204 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.204 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.204 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.204 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.059046+0000", 2024-09-14T23:58:09.204 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7489999999999998e-06 2024-09-14T23:58:09.205 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.205 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.205 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.205 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.205 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189191 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.205 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.084269+0000", 2024-09-14T23:58:09.205 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0095919349999999994, 2024-09-14T23:58:09.205 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.011124009000000001, 2024-09-14T23:58:09.205 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.205 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.205 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.205 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.205 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.205 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.206 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189191 2024-09-14T23:58:09.206 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.206 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.206 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.206 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.206 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.084269+0000", 2024-09-14T23:58:09.206 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.206 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.206 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.206 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.206 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.084269+0000", 2024-09-14T23:58:09.206 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.207 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.207 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.207 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.084269+0000", 2024-09-14T23:58:09.207 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1900000000000002e-07 2024-09-14T23:58:09.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.207 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.207 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.207 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.084273+0000", 2024-09-14T23:58:09.207 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4410000000000002e-06 2024-09-14T23:58:09.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.207 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.207 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.208 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.084281+0000", 2024-09-14T23:58:09.208 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3210000000000005e-06 2024-09-14T23:58:09.208 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.208 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.208 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.208 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.084287+0000", 2024-09-14T23:58:09.208 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8050000000000003e-06 2024-09-14T23:58:09.208 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.208 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.208 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.208 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.208 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.208 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189227 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.208 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.088021+0000", 2024-09-14T23:58:09.209 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0058399250000000002, 2024-09-14T23:58:09.209 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0074319410000000001, 2024-09-14T23:58:09.209 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.209 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.209 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.209 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.209 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.209 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.209 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189227 2024-09-14T23:58:09.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.209 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.209 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.209 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.210 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088021+0000", 2024-09-14T23:58:09.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.210 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.210 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088021+0000", 2024-09-14T23:58:09.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.210 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.210 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088021+0000", 2024-09-14T23:58:09.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9900000000000001e-07 2024-09-14T23:58:09.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.211 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.211 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.211 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088023+0000", 2024-09-14T23:58:09.211 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.198e-06 2024-09-14T23:58:09.211 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.211 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.211 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.211 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088030+0000", 2024-09-14T23:58:09.211 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.139e-06 2024-09-14T23:58:09.211 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.211 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.211 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.211 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088034+0000", 2024-09-14T23:58:09.211 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7740000000000004e-06 2024-09-14T23:58:09.211 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.212 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.212 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.212 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.212 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.212 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189355 3.1d 3.e507d9bd (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.212 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.092153+0000", 2024-09-14T23:58:09.212 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0017082950000000001, 2024-09-14T23:58:09.212 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003359248, 2024-09-14T23:58:09.212 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.212 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.212 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.212 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.212 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.212 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.212 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189355 2024-09-14T23:58:09.213 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.213 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.213 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.213 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.213 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.092153+0000", 2024-09-14T23:58:09.213 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.213 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.213 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.213 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.213 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.092153+0000", 2024-09-14T23:58:09.213 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.213 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.213 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.213 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.214 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.093839+0000", 2024-09-14T23:58:09.214 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016859869999999999 2024-09-14T23:58:09.214 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.214 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.214 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.214 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.093844+0000", 2024-09-14T23:58:09.214 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0200000000000002e-06 2024-09-14T23:58:09.214 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.214 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.214 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.214 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.093856+0000", 2024-09-14T23:58:09.214 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2670000000000001e-05 2024-09-14T23:58:09.214 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.214 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.214 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.215 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.093863+0000", 2024-09-14T23:58:09.215 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2409999999999998e-06 2024-09-14T23:58:09.215 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.215 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.215 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.215 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.215 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.215 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188171 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.215 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.032086+0000", 2024-09-14T23:58:09.215 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.061774943999999998, 2024-09-14T23:58:09.215 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.063495686999999995, 2024-09-14T23:58:09.215 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.215 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.215 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.216 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.216 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.216 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.216 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188171 2024-09-14T23:58:09.216 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.216 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.216 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.216 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.216 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032086+0000", 2024-09-14T23:58:09.216 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.216 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.216 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.216 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.216 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032086+0000", 2024-09-14T23:58:09.217 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.217 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.217 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.217 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.217 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032086+0000", 2024-09-14T23:58:09.217 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7000000000000005e-07 2024-09-14T23:58:09.217 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.217 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.217 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.217 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032089+0000", 2024-09-14T23:58:09.217 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3599999999999999e-06 2024-09-14T23:58:09.217 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.217 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.217 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.217 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032095+0000", 2024-09-14T23:58:09.218 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4699999999999999e-06 2024-09-14T23:58:09.218 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.218 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.218 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.218 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032100+0000", 2024-09-14T23:58:09.218 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9960000000000001e-06 2024-09-14T23:58:09.218 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.218 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.218 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.218 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.218 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.218 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188203 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.218 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.033097+0000", 2024-09-14T23:58:09.218 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.060763658999999998, 2024-09-14T23:58:09.219 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.062544625000000006, 2024-09-14T23:58:09.219 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.219 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.219 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.219 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.219 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.219 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.219 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188203 2024-09-14T23:58:09.219 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.219 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.219 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.219 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.219 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033097+0000", 2024-09-14T23:58:09.219 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.219 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.220 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.220 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.220 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033097+0000", 2024-09-14T23:58:09.220 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.220 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.220 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.220 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.220 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033098+0000", 2024-09-14T23:58:09.220 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4200000000000001e-07 2024-09-14T23:58:09.220 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.220 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.220 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.220 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033100+0000", 2024-09-14T23:58:09.220 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.21e-06 2024-09-14T23:58:09.221 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.221 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.221 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.221 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033106+0000", 2024-09-14T23:58:09.221 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6370000000000002e-06 2024-09-14T23:58:09.221 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.221 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.221 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.221 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033112+0000", 2024-09-14T23:58:09.221 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3109999999999996e-06 2024-09-14T23:58:09.221 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.221 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.221 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.222 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.222 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.222 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188682 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.222 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.056068+0000", 2024-09-14T23:58:09.222 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.037792489999999998, 2024-09-14T23:58:09.222 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.039645673999999999, 2024-09-14T23:58:09.222 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.222 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.222 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.222 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.222 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.222 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.222 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188682 2024-09-14T23:58:09.223 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.223 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.223 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.223 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.223 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056068+0000", 2024-09-14T23:58:09.223 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.223 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.223 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.223 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.223 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056068+0000", 2024-09-14T23:58:09.224 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.224 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.224 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.224 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.224 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056069+0000", 2024-09-14T23:58:09.224 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1100000000000001e-07 2024-09-14T23:58:09.224 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.224 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.224 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.224 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056071+0000", 2024-09-14T23:58:09.224 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2330000000000001e-06 2024-09-14T23:58:09.224 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.224 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.224 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.224 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056078+0000", 2024-09-14T23:58:09.225 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5660000000000003e-06 2024-09-14T23:58:09.225 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.225 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.225 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.225 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056083+0000", 2024-09-14T23:58:09.225 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2889999999999997e-06 2024-09-14T23:58:09.225 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.225 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.225 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.225 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.225 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.225 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188715 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.225 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.059053+0000", 2024-09-14T23:58:09.225 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.034807658999999998, 2024-09-14T23:58:09.226 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.036719446000000003, 2024-09-14T23:58:09.226 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.226 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.226 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.226 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.226 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.226 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.226 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188715 2024-09-14T23:58:09.226 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.226 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.226 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.226 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.226 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.059053+0000", 2024-09-14T23:58:09.226 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.227 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.227 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.227 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.227 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.059053+0000", 2024-09-14T23:58:09.227 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.227 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.227 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.227 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.227 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.060672+0000", 2024-09-14T23:58:09.227 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016192610000000001 2024-09-14T23:58:09.227 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.227 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.227 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.228 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.060685+0000", 2024-09-14T23:58:09.228 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2624e-05 2024-09-14T23:58:09.228 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.228 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.228 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.228 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.060697+0000", 2024-09-14T23:58:09.228 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1977000000000001e-05 2024-09-14T23:58:09.228 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.228 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.228 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.228 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.060705+0000", 2024-09-14T23:58:09.228 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6890000000000004e-06 2024-09-14T23:58:09.228 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.228 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.229 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.229 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.229 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.229 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189193 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.229 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.084306+0000", 2024-09-14T23:58:09.229 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0095543250000000007, 2024-09-14T23:58:09.229 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.011536233999999999, 2024-09-14T23:58:09.229 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.229 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.229 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.229 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.229 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.229 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.230 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189193 2024-09-14T23:58:09.230 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.230 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.230 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.230 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.230 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.084306+0000", 2024-09-14T23:58:09.230 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.230 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.230 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.230 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.230 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.084306+0000", 2024-09-14T23:58:09.230 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.230 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.231 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.231 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.231 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085672+0000", 2024-09-14T23:58:09.231 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001365018 2024-09-14T23:58:09.231 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.231 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.231 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.231 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085676+0000", 2024-09-14T23:58:09.231 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8459999999999999e-06 2024-09-14T23:58:09.231 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.231 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.231 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.231 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085688+0000", 2024-09-14T23:58:09.231 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1661999999999999e-05 2024-09-14T23:58:09.232 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.232 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.232 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.232 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085696+0000", 2024-09-14T23:58:09.232 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9889999999999992e-06 2024-09-14T23:58:09.232 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.232 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.232 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.232 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.232 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.232 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189228 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.232 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.088041+0000", 2024-09-14T23:58:09.233 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0058198080000000001, 2024-09-14T23:58:09.233 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0078800510000000008, 2024-09-14T23:58:09.233 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.233 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.233 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.233 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.233 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.233 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.233 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189228 2024-09-14T23:58:09.233 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.233 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.233 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.233 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.233 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088041+0000", 2024-09-14T23:58:09.234 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.234 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.234 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.234 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.234 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088041+0000", 2024-09-14T23:58:09.234 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.234 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.234 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.234 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.234 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088042+0000", 2024-09-14T23:58:09.234 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5300000000000004e-07 2024-09-14T23:58:09.234 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.234 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.235 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.235 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088045+0000", 2024-09-14T23:58:09.235 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3730000000000001e-06 2024-09-14T23:58:09.235 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.235 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.235 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.235 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088051+0000", 2024-09-14T23:58:09.235 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3940000000000001e-06 2024-09-14T23:58:09.235 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.235 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.235 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.235 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088056+0000", 2024-09-14T23:58:09.235 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1340000000000003e-06 2024-09-14T23:58:09.236 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.236 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.236 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.236 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.236 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.236 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189356 3.1d 3.e507d9bd (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.236 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.093876+0000", 2024-09-14T23:58:09.236 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4294967295.9999852, 2024-09-14T23:58:09.236 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0021485499999999999, 2024-09-14T23:58:09.236 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.236 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.236 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.236 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.236 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.237 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.237 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189356 2024-09-14T23:58:09.237 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.237 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.237 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.237 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.237 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.093876+0000", 2024-09-14T23:58:09.237 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.237 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.237 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.237 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.237 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.093876+0000", 2024-09-14T23:58:09.238 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.238 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.238 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.238 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.093876+0000", 2024-09-14T23:58:09.238 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1799999999999995e-07 2024-09-14T23:58:09.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.238 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.238 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.238 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.093879+0000", 2024-09-14T23:58:09.238 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3740000000000001e-06 2024-09-14T23:58:09.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.238 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.238 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.238 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.093886+0000", 2024-09-14T23:58:09.239 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7259999999999996e-06 2024-09-14T23:58:09.239 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.239 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.239 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.239 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.093890+0000", 2024-09-14T23:58:09.239 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8169999999999998e-06 2024-09-14T23:58:09.239 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.239 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.239 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.239 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.239 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.239 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188172 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.239 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.032108+0000", 2024-09-14T23:58:09.240 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.061753183000000003, 2024-09-14T23:58:09.240 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.064012105999999999, 2024-09-14T23:58:09.240 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.240 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.240 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.240 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.240 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.240 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.240 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188172 2024-09-14T23:58:09.240 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.240 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.240 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.240 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.241 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032108+0000", 2024-09-14T23:58:09.241 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.241 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.241 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.241 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.241 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032108+0000", 2024-09-14T23:58:09.241 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.241 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.241 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.241 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.241 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032108+0000", 2024-09-14T23:58:09.241 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.34e-07 2024-09-14T23:58:09.241 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.241 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.241 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.242 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032110+0000", 2024-09-14T23:58:09.242 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3939999999999999e-06 2024-09-14T23:58:09.242 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.242 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.242 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.242 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032117+0000", 2024-09-14T23:58:09.242 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2090000000000002e-06 2024-09-14T23:58:09.242 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.242 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.242 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.242 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032122+0000", 2024-09-14T23:58:09.242 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0900000000000004e-06 2024-09-14T23:58:09.242 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.242 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.243 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.243 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.243 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.243 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188204 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.243 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.033118+0000", 2024-09-14T23:58:09.243 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.060742467000000001, 2024-09-14T23:58:09.243 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.063083325999999995, 2024-09-14T23:58:09.243 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.243 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.243 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.243 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.243 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.243 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.243 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188204 2024-09-14T23:58:09.243 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.244 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.244 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.244 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.244 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033118+0000", 2024-09-14T23:58:09.244 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.244 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.244 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.244 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.244 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033118+0000", 2024-09-14T23:58:09.244 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.244 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.244 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.244 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.244 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033119+0000", 2024-09-14T23:58:09.245 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1699999999999999e-07 2024-09-14T23:58:09.245 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.245 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.245 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.245 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033121+0000", 2024-09-14T23:58:09.245 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0229999999999999e-06 2024-09-14T23:58:09.245 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.245 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.245 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.245 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033127+0000", 2024-09-14T23:58:09.245 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4779999999999996e-06 2024-09-14T23:58:09.245 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.245 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.245 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.246 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033140+0000", 2024-09-14T23:58:09.246 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2913e-05 2024-09-14T23:58:09.246 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.246 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.246 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.246 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.246 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.246 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188684 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.246 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.056089+0000", 2024-09-14T23:58:09.246 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.037771616000000001, 2024-09-14T23:58:09.246 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.040173463999999999, 2024-09-14T23:58:09.246 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.246 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.246 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.246 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.246 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.247 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.247 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188684 2024-09-14T23:58:09.247 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.247 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.247 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.247 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.247 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056089+0000", 2024-09-14T23:58:09.247 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.247 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.247 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.247 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.247 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056089+0000", 2024-09-14T23:58:09.247 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.247 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.248 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.248 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.248 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056090+0000", 2024-09-14T23:58:09.248 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1699999999999999e-07 2024-09-14T23:58:09.248 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.248 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.248 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.248 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056092+0000", 2024-09-14T23:58:09.248 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0669999999999999e-06 2024-09-14T23:58:09.248 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.248 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.248 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.248 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056098+0000", 2024-09-14T23:58:09.248 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3609999999999997e-06 2024-09-14T23:58:09.248 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.249 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.249 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.249 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056103+0000", 2024-09-14T23:58:09.249 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2020000000000003e-06 2024-09-14T23:58:09.249 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.249 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.249 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.249 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.249 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.249 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188716 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.249 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.060833+0000", 2024-09-14T23:58:09.249 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.033028038000000003, 2024-09-14T23:58:09.249 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.035486657999999997, 2024-09-14T23:58:09.250 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.250 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.250 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.250 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.250 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.250 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.250 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188716 2024-09-14T23:58:09.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.250 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.250 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.250 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.250 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.060833+0000", 2024-09-14T23:58:09.250 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.251 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.251 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.251 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.251 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.060833+0000", 2024-09-14T23:58:09.251 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.251 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.251 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.251 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.251 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.060833+0000", 2024-09-14T23:58:09.251 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6899999999999997e-07 2024-09-14T23:58:09.251 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.251 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.251 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.251 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.060837+0000", 2024-09-14T23:58:09.252 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.298e-06 2024-09-14T23:58:09.252 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.252 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.252 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.252 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.060846+0000", 2024-09-14T23:58:09.252 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1819999999999995e-06 2024-09-14T23:58:09.252 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.252 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.252 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.252 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.060853+0000", 2024-09-14T23:58:09.252 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6800000000000004e-06 2024-09-14T23:58:09.252 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.252 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.252 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.253 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.253 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.253 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189196 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.253 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.085709+0000", 2024-09-14T23:58:09.253 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0081518440000000001, 2024-09-14T23:58:09.253 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010667357000000001, 2024-09-14T23:58:09.253 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.253 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.253 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.253 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.253 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.253 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.253 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189196 2024-09-14T23:58:09.253 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.253 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.254 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.254 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.254 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085709+0000", 2024-09-14T23:58:09.254 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.254 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.254 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.254 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.254 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085709+0000", 2024-09-14T23:58:09.254 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.254 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.254 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.254 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.254 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085710+0000", 2024-09-14T23:58:09.254 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0399999999999996e-07 2024-09-14T23:58:09.254 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.255 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.255 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085713+0000", 2024-09-14T23:58:09.255 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9500000000000001e-06 2024-09-14T23:58:09.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.255 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.255 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085720+0000", 2024-09-14T23:58:09.255 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4599999999999997e-06 2024-09-14T23:58:09.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.255 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.255 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085727+0000", 2024-09-14T23:58:09.255 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9110000000000004e-06 2024-09-14T23:58:09.255 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.255 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.256 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.256 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.256 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.256 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189229 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.256 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.088063+0000", 2024-09-14T23:58:09.256 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0057976030000000001, 2024-09-14T23:58:09.256 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0083697170000000005, 2024-09-14T23:58:09.256 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.256 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.256 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.256 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.256 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.256 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.256 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189229 2024-09-14T23:58:09.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.257 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.257 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.257 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.257 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088063+0000", 2024-09-14T23:58:09.257 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.257 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.257 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.257 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088063+0000", 2024-09-14T23:58:09.257 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.257 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.258 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.258 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088064+0000", 2024-09-14T23:58:09.258 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1799999999999995e-07 2024-09-14T23:58:09.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.258 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.258 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.258 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088066+0000", 2024-09-14T23:58:09.258 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2510000000000001e-06 2024-09-14T23:58:09.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.258 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.258 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.258 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088073+0000", 2024-09-14T23:58:09.258 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2289999999999997e-06 2024-09-14T23:58:09.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.259 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.259 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.259 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088078+0000", 2024-09-14T23:58:09.259 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0719999999999999e-06 2024-09-14T23:58:09.259 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.259 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.259 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.259 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.259 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189357 3.1d 3.e507d9bd (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.259 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.094298+0000", 2024-09-14T23:58:09.259 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4294967295.9995623, 2024-09-14T23:58:09.259 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0021923709999999998, 2024-09-14T23:58:09.259 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.260 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.260 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.260 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.260 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.260 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.260 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189357 2024-09-14T23:58:09.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.260 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.260 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.260 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.260 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094298+0000", 2024-09-14T23:58:09.260 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.260 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.261 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.261 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094298+0000", 2024-09-14T23:58:09.261 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.261 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.261 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.261 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094299+0000", 2024-09-14T23:58:09.261 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.54e-07 2024-09-14T23:58:09.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.261 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.261 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.261 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094303+0000", 2024-09-14T23:58:09.261 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2320000000000003e-06 2024-09-14T23:58:09.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.262 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.262 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.262 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094316+0000", 2024-09-14T23:58:09.262 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2101e-05 2024-09-14T23:58:09.262 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.262 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.262 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.262 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094324+0000", 2024-09-14T23:58:09.262 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9480000000000008e-06 2024-09-14T23:58:09.262 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.262 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.262 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.262 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.262 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.262 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188173 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.263 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.032130+0000", 2024-09-14T23:58:09.263 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.061730724000000001, 2024-09-14T23:58:09.263 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.064417198999999994, 2024-09-14T23:58:09.263 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.263 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.263 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.263 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.263 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.263 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.263 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188173 2024-09-14T23:58:09.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.263 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.263 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.264 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032130+0000", 2024-09-14T23:58:09.264 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.264 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.264 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.264 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032130+0000", 2024-09-14T23:58:09.264 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.264 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.264 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.264 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032130+0000", 2024-09-14T23:58:09.264 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9400000000000001e-07 2024-09-14T23:58:09.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.265 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.265 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032133+0000", 2024-09-14T23:58:09.265 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6350000000000002e-06 2024-09-14T23:58:09.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.265 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.265 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032140+0000", 2024-09-14T23:58:09.265 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.489e-06 2024-09-14T23:58:09.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.265 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.266 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032145+0000", 2024-09-14T23:58:09.266 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1560000000000003e-06 2024-09-14T23:58:09.266 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.266 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.266 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.266 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.266 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.266 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188205 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.266 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.033148+0000", 2024-09-14T23:58:09.266 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.060712620000000002, 2024-09-14T23:58:09.266 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.063455807000000003, 2024-09-14T23:58:09.266 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.266 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.266 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.267 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.267 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.267 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.267 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188205 2024-09-14T23:58:09.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.267 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.267 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.267 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.267 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033148+0000", 2024-09-14T23:58:09.267 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.267 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.267 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.267 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033148+0000", 2024-09-14T23:58:09.268 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.268 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.268 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.268 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033149+0000", 2024-09-14T23:58:09.268 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0200000000000002e-07 2024-09-14T23:58:09.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.268 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.268 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.268 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033156+0000", 2024-09-14T23:58:09.268 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3629999999999996e-06 2024-09-14T23:58:09.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.268 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.268 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.269 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033169+0000", 2024-09-14T23:58:09.269 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2649999999999999e-05 2024-09-14T23:58:09.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.269 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.269 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.269 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033176+0000", 2024-09-14T23:58:09.269 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2570000000000001e-06 2024-09-14T23:58:09.269 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.269 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.269 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.269 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.269 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188685 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.269 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.056110+0000", 2024-09-14T23:58:09.270 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.037751012, 2024-09-14T23:58:09.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.040550777000000003, 2024-09-14T23:58:09.270 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.270 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.270 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.270 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.270 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.270 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.270 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188685 2024-09-14T23:58:09.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.270 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.270 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.270 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.271 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056110+0000", 2024-09-14T23:58:09.271 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.271 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.271 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.271 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056110+0000", 2024-09-14T23:58:09.271 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.271 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.271 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.271 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056110+0000", 2024-09-14T23:58:09.271 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9499999999999998e-07 2024-09-14T23:58:09.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.271 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.271 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.272 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056112+0000", 2024-09-14T23:58:09.272 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2390000000000001e-06 2024-09-14T23:58:09.272 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.272 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.272 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.272 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056119+0000", 2024-09-14T23:58:09.272 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6579999999999996e-06 2024-09-14T23:58:09.272 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.272 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.272 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.272 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056125+0000", 2024-09-14T23:58:09.272 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5029999999999997e-06 2024-09-14T23:58:09.272 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.273 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.273 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.273 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.273 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.273 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188717 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.273 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.060940+0000", 2024-09-14T23:58:09.273 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.032920952000000003, 2024-09-14T23:58:09.273 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.035785544000000002, 2024-09-14T23:58:09.273 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.273 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.273 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.273 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.273 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.273 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.274 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188717 2024-09-14T23:58:09.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.274 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.274 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.274 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.060940+0000", 2024-09-14T23:58:09.274 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.274 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.274 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.060940+0000", 2024-09-14T23:58:09.274 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.275 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.275 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.275 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.060941+0000", 2024-09-14T23:58:09.275 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1900000000000002e-07 2024-09-14T23:58:09.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.275 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.275 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.275 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.060965+0000", 2024-09-14T23:58:09.275 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4725000000000001e-05 2024-09-14T23:58:09.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.275 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.275 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.275 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061052+0000", 2024-09-14T23:58:09.275 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6267999999999995e-05 2024-09-14T23:58:09.276 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.276 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.276 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.276 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061060+0000", 2024-09-14T23:58:09.276 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1960000000000008e-06 2024-09-14T23:58:09.276 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.276 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.276 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.276 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.276 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.276 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189198 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.276 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.085736+0000", 2024-09-14T23:58:09.276 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0081245910000000005, 2024-09-14T23:58:09.276 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.011046863000000001, 2024-09-14T23:58:09.277 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.277 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.277 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.277 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.277 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.277 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.277 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189198 2024-09-14T23:58:09.277 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.277 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.277 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.277 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.277 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085736+0000", 2024-09-14T23:58:09.277 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.277 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.278 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.278 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.278 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085736+0000", 2024-09-14T23:58:09.278 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.278 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.278 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.278 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.278 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085737+0000", 2024-09-14T23:58:09.278 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5600000000000001e-07 2024-09-14T23:58:09.278 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.278 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.278 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.278 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085739+0000", 2024-09-14T23:58:09.279 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4509999999999999e-06 2024-09-14T23:58:09.279 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.279 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.279 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.279 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085748+0000", 2024-09-14T23:58:09.279 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7029999999999995e-06 2024-09-14T23:58:09.279 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.279 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.279 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.279 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085754+0000", 2024-09-14T23:58:09.279 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3199999999999996e-06 2024-09-14T23:58:09.279 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.279 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.279 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.280 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.280 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.280 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189230 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.280 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.088085+0000", 2024-09-14T23:58:09.280 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0057757889999999999, 2024-09-14T23:58:09.280 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0087546340000000007, 2024-09-14T23:58:09.280 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.280 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.280 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.280 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.280 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.280 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.280 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189230 2024-09-14T23:58:09.281 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.281 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.281 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.281 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.281 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088085+0000", 2024-09-14T23:58:09.281 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.281 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.281 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.281 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.281 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088085+0000", 2024-09-14T23:58:09.281 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.281 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.281 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.281 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.282 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088085+0000", 2024-09-14T23:58:09.282 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5499999999999998e-07 2024-09-14T23:58:09.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.282 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.282 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.282 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088087+0000", 2024-09-14T23:58:09.282 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.951e-06 2024-09-14T23:58:09.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.282 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.282 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.282 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088094+0000", 2024-09-14T23:58:09.282 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0780000000000001e-06 2024-09-14T23:58:09.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.282 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.283 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.283 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088098+0000", 2024-09-14T23:58:09.283 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5410000000000002e-06 2024-09-14T23:58:09.283 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.283 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.283 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.283 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.283 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.283 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189358 3.1d 3.e507d9bd (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.283 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.094335+0000", 2024-09-14T23:58:09.283 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4294967295.9995255, 2024-09-14T23:58:09.283 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0025608139999999998, 2024-09-14T23:58:09.283 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.283 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.284 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.284 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.284 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.284 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.284 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189358 2024-09-14T23:58:09.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.284 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.284 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.284 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.284 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094335+0000", 2024-09-14T23:58:09.284 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.284 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.284 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.285 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094335+0000", 2024-09-14T23:58:09.285 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.285 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.285 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.285 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094336+0000", 2024-09-14T23:58:09.285 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7300000000000001e-07 2024-09-14T23:58:09.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.285 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.285 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.285 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094345+0000", 2024-09-14T23:58:09.285 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1220000000000001e-06 2024-09-14T23:58:09.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.285 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.286 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.286 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094353+0000", 2024-09-14T23:58:09.286 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.028e-06 2024-09-14T23:58:09.286 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.286 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.286 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.286 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094359+0000", 2024-09-14T23:58:09.286 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6210000000000001e-06 2024-09-14T23:58:09.286 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.286 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.286 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.286 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.286 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.286 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188174 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.287 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.032154+0000", 2024-09-14T23:58:09.287 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.061706418999999998, 2024-09-14T23:58:09.287 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.064802578999999999, 2024-09-14T23:58:09.287 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.287 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.287 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.287 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.287 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.287 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.287 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188174 2024-09-14T23:58:09.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.287 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.287 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.287 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.288 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032154+0000", 2024-09-14T23:58:09.288 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.288 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.288 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.288 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032154+0000", 2024-09-14T23:58:09.288 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.288 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.288 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.288 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032155+0000", 2024-09-14T23:58:09.288 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4900000000000001e-07 2024-09-14T23:58:09.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.288 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.289 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.289 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032157+0000", 2024-09-14T23:58:09.289 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6199999999999999e-06 2024-09-14T23:58:09.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.289 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.289 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.289 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032166+0000", 2024-09-14T23:58:09.289 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5050000000000007e-06 2024-09-14T23:58:09.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.289 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.289 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.289 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032172+0000", 2024-09-14T23:58:09.289 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6949999999999997e-06 2024-09-14T23:58:09.289 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.290 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.290 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.290 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.290 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188206 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.290 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.033186+0000", 2024-09-14T23:58:09.290 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.060675205000000003, 2024-09-14T23:58:09.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.063829304000000003, 2024-09-14T23:58:09.290 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.290 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.290 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.290 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.290 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.290 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.291 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188206 2024-09-14T23:58:09.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.291 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.291 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.291 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.291 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033186+0000", 2024-09-14T23:58:09.291 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.291 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.291 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.291 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033186+0000", 2024-09-14T23:58:09.291 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.291 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.292 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.292 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033186+0000", 2024-09-14T23:58:09.292 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7599999999999997e-07 2024-09-14T23:58:09.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.292 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.292 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.292 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033189+0000", 2024-09-14T23:58:09.292 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.492e-06 2024-09-14T23:58:09.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.292 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.292 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.292 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033197+0000", 2024-09-14T23:58:09.292 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.977e-06 2024-09-14T23:58:09.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.293 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.293 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.293 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033202+0000", 2024-09-14T23:58:09.293 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7420000000000003e-06 2024-09-14T23:58:09.293 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.293 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.293 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.293 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.293 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188686 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.293 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.056132+0000", 2024-09-14T23:58:09.293 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.037729088000000001, 2024-09-14T23:58:09.294 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.040935344999999998, 2024-09-14T23:58:09.294 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.294 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.294 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.294 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.294 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.294 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.294 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188686 2024-09-14T23:58:09.294 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.294 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.294 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.294 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.294 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056132+0000", 2024-09-14T23:58:09.294 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.295 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.295 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.295 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056132+0000", 2024-09-14T23:58:09.295 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.295 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.295 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.295 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056132+0000", 2024-09-14T23:58:09.295 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.08e-07 2024-09-14T23:58:09.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.295 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.295 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.296 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056135+0000", 2024-09-14T23:58:09.296 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3999999999999999e-06 2024-09-14T23:58:09.296 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.296 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.296 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.296 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056141+0000", 2024-09-14T23:58:09.296 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.939e-06 2024-09-14T23:58:09.296 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.296 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.296 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.296 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056147+0000", 2024-09-14T23:58:09.296 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4790000000000004e-06 2024-09-14T23:58:09.296 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.297 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.297 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.297 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.297 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188718 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.297 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.061308+0000", 2024-09-14T23:58:09.297 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.032552876000000001, 2024-09-14T23:58:09.297 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.035816093, 2024-09-14T23:58:09.297 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.297 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.297 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.297 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.297 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.298 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.298 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188718 2024-09-14T23:58:09.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.298 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.298 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.298 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.298 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061308+0000", 2024-09-14T23:58:09.298 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.298 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.298 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.298 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061308+0000", 2024-09-14T23:58:09.298 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.299 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.299 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.299 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061309+0000", 2024-09-14T23:58:09.299 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0299999999999998e-07 2024-09-14T23:58:09.299 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.299 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.299 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.299 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061312+0000", 2024-09-14T23:58:09.299 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.631e-06 2024-09-14T23:58:09.299 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.299 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.299 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.299 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061322+0000", 2024-09-14T23:58:09.300 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5459999999999997e-06 2024-09-14T23:58:09.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.300 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.300 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.300 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061329+0000", 2024-09-14T23:58:09.300 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7039999999999997e-06 2024-09-14T23:58:09.300 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.300 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.300 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.300 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.300 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189199 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.300 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.085762+0000", 2024-09-14T23:58:09.301 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0080988520000000001, 2024-09-14T23:58:09.301 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.011420969, 2024-09-14T23:58:09.301 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.301 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.301 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.301 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.301 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.301 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.301 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189199 2024-09-14T23:58:09.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.301 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.301 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.301 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.301 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085762+0000", 2024-09-14T23:58:09.302 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.302 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.302 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.302 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085762+0000", 2024-09-14T23:58:09.302 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.302 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.302 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.302 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085763+0000", 2024-09-14T23:58:09.302 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8999999999999996e-07 2024-09-14T23:58:09.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.302 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.303 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.303 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085770+0000", 2024-09-14T23:58:09.303 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1409999999999999e-06 2024-09-14T23:58:09.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.303 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.303 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085777+0000", 2024-09-14T23:58:09.303 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4830000000000002e-06 2024-09-14T23:58:09.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.303 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.303 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085782+0000", 2024-09-14T23:58:09.303 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0549999999999999e-06 2024-09-14T23:58:09.304 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.304 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.304 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.304 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.304 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189231 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.304 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.088105+0000", 2024-09-14T23:58:09.304 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0057559949999999999, 2024-09-14T23:58:09.304 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0091373459999999993, 2024-09-14T23:58:09.304 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.304 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.304 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.304 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.304 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.304 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.305 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189231 2024-09-14T23:58:09.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.305 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.305 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088105+0000", 2024-09-14T23:58:09.305 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.305 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088105+0000", 2024-09-14T23:58:09.305 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.305 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.306 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088105+0000", 2024-09-14T23:58:09.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1600000000000002e-07 2024-09-14T23:58:09.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.306 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.306 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.306 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088108+0000", 2024-09-14T23:58:09.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2740000000000002e-06 2024-09-14T23:58:09.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.306 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.306 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.306 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088114+0000", 2024-09-14T23:58:09.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.528e-06 2024-09-14T23:58:09.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.306 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.306 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.306 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088119+0000", 2024-09-14T23:58:09.307 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8509999999999998e-06 2024-09-14T23:58:09.307 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.307 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.307 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.307 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.307 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189359 3.1d 3.e507d9bd (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.307 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.094367+0000", 2024-09-14T23:58:09.307 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4294967295.9994941, 2024-09-14T23:58:09.307 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.002932315, 2024-09-14T23:58:09.307 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.307 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.307 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.307 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.307 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.308 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.308 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189359 2024-09-14T23:58:09.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.308 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.308 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.308 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.308 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094367+0000", 2024-09-14T23:58:09.308 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.308 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.308 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.308 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094367+0000", 2024-09-14T23:58:09.308 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.309 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.309 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.309 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094367+0000", 2024-09-14T23:58:09.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2199999999999999e-07 2024-09-14T23:58:09.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.309 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.309 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.309 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094370+0000", 2024-09-14T23:58:09.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2060000000000001e-06 2024-09-14T23:58:09.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.309 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.309 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.309 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094377+0000", 2024-09-14T23:58:09.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7919999999999993e-06 2024-09-14T23:58:09.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.310 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.310 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094382+0000", 2024-09-14T23:58:09.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1800000000000004e-06 2024-09-14T23:58:09.310 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.310 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.310 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.310 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188175 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.310 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.032179+0000", 2024-09-14T23:58:09.310 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.061681905000000002, 2024-09-14T23:58:09.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065180324999999997, 2024-09-14T23:58:09.311 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.311 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.311 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.311 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.311 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.311 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.311 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188175 2024-09-14T23:58:09.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.311 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.311 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.311 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.311 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032179+0000", 2024-09-14T23:58:09.311 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.312 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.312 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032179+0000", 2024-09-14T23:58:09.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.312 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.312 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032179+0000", 2024-09-14T23:58:09.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3099999999999998e-07 2024-09-14T23:58:09.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.312 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.312 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032182+0000", 2024-09-14T23:58:09.313 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3489999999999999e-06 2024-09-14T23:58:09.313 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.313 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.313 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.313 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032190+0000", 2024-09-14T23:58:09.313 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1289999999999996e-06 2024-09-14T23:58:09.313 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.313 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.313 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.313 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032196+0000", 2024-09-14T23:58:09.313 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7389999999999996e-06 2024-09-14T23:58:09.313 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.313 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.314 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.314 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.314 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.314 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188207 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.314 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.033209+0000", 2024-09-14T23:58:09.314 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.060651319000000002, 2024-09-14T23:58:09.314 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.064208045000000005, 2024-09-14T23:58:09.314 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.314 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.314 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.314 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.314 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.314 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.314 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188207 2024-09-14T23:58:09.315 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.315 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.315 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.315 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.315 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033209+0000", 2024-09-14T23:58:09.315 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.315 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.315 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.315 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.315 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033209+0000", 2024-09-14T23:58:09.315 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.315 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.315 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.315 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.316 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033210+0000", 2024-09-14T23:58:09.316 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1900000000000003e-07 2024-09-14T23:58:09.316 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.316 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.316 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.316 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033212+0000", 2024-09-14T23:58:09.316 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2390000000000001e-06 2024-09-14T23:58:09.316 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.316 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.316 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.316 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033219+0000", 2024-09-14T23:58:09.316 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9489999999999999e-06 2024-09-14T23:58:09.316 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.317 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.317 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.317 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033225+0000", 2024-09-14T23:58:09.317 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3920000000000002e-06 2024-09-14T23:58:09.317 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.317 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.317 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.317 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.317 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188687 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.317 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.056154+0000", 2024-09-14T23:58:09.317 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.037706532000000001, 2024-09-14T23:58:09.317 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.041321475000000003, 2024-09-14T23:58:09.317 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.318 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.318 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.318 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.318 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.318 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.318 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188687 2024-09-14T23:58:09.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.318 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.318 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.318 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.318 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056154+0000", 2024-09-14T23:58:09.318 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.319 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.319 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.319 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056154+0000", 2024-09-14T23:58:09.319 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.319 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.319 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.319 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056155+0000", 2024-09-14T23:58:09.319 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0200000000000003e-07 2024-09-14T23:58:09.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.319 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.319 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.319 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056157+0000", 2024-09-14T23:58:09.320 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3920000000000001e-06 2024-09-14T23:58:09.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.320 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.320 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.320 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056339+0000", 2024-09-14T23:58:09.320 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000181604 2024-09-14T23:58:09.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.320 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.320 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.320 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056523+0000", 2024-09-14T23:58:09.320 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000184778 2024-09-14T23:58:09.320 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.320 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.320 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.321 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.321 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188719 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.321 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.061339+0000", 2024-09-14T23:58:09.321 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.032521481999999997, 2024-09-14T23:58:09.321 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.036196925999999997, 2024-09-14T23:58:09.321 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.321 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.321 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.321 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.321 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.321 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.321 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188719 2024-09-14T23:58:09.321 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.321 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.322 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.322 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.322 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061339+0000", 2024-09-14T23:58:09.322 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.322 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.322 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.322 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061339+0000", 2024-09-14T23:58:09.322 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.322 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.322 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.322 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061340+0000", 2024-09-14T23:58:09.323 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8699999999999995e-07 2024-09-14T23:58:09.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.323 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.323 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.323 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061342+0000", 2024-09-14T23:58:09.323 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5919999999999999e-06 2024-09-14T23:58:09.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.323 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.323 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.323 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061349+0000", 2024-09-14T23:58:09.323 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7220000000000002e-06 2024-09-14T23:58:09.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.323 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.323 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.323 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061354+0000", 2024-09-14T23:58:09.324 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1900000000000003e-06 2024-09-14T23:58:09.324 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.324 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.324 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.324 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.324 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.324 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189200 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.324 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.085790+0000", 2024-09-14T23:58:09.324 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0080704030000000003, 2024-09-14T23:58:09.324 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.011802286, 2024-09-14T23:58:09.324 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.324 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.324 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.325 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.325 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.325 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.325 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189200 2024-09-14T23:58:09.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.325 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.325 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.325 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.325 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085790+0000", 2024-09-14T23:58:09.325 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.325 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.325 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.325 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085790+0000", 2024-09-14T23:58:09.326 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.326 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.326 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.326 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085791+0000", 2024-09-14T23:58:09.326 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5199999999999997e-07 2024-09-14T23:58:09.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.326 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.326 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.326 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085793+0000", 2024-09-14T23:58:09.326 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.216e-06 2024-09-14T23:58:09.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.326 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.326 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.327 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085800+0000", 2024-09-14T23:58:09.327 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9670000000000004e-06 2024-09-14T23:58:09.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.327 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.327 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.327 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085805+0000", 2024-09-14T23:58:09.327 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7759999999999997e-06 2024-09-14T23:58:09.327 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.327 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.327 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.327 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.328 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189232 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.328 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.088126+0000", 2024-09-14T23:58:09.328 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0057347680000000003, 2024-09-14T23:58:09.328 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0095239290000000004, 2024-09-14T23:58:09.328 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.328 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.328 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.328 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.328 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.328 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.328 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189232 2024-09-14T23:58:09.328 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.328 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.329 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.329 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.329 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088126+0000", 2024-09-14T23:58:09.329 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.329 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.329 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.329 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.329 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088126+0000", 2024-09-14T23:58:09.329 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.329 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.329 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.329 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.329 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088126+0000", 2024-09-14T23:58:09.330 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1899999999999998e-07 2024-09-14T23:58:09.330 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.330 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.330 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.330 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088128+0000", 2024-09-14T23:58:09.330 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.001e-06 2024-09-14T23:58:09.330 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.330 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.330 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.330 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088135+0000", 2024-09-14T23:58:09.330 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7499999999999997e-06 2024-09-14T23:58:09.330 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.330 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.330 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.331 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088141+0000", 2024-09-14T23:58:09.331 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7509999999999997e-06 2024-09-14T23:58:09.331 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.331 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.331 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.331 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.331 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.331 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189360 3.1d 3.e507d9bd (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.331 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.094390+0000", 2024-09-14T23:58:09.331 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4294967295.9994707, 2024-09-14T23:58:09.331 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033239480000000002, 2024-09-14T23:58:09.331 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.331 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.331 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.332 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.332 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.332 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.332 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189360 2024-09-14T23:58:09.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.332 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.332 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.332 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.332 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094390+0000", 2024-09-14T23:58:09.332 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.332 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.332 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.332 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094390+0000", 2024-09-14T23:58:09.333 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.333 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.333 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.333 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.333 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094391+0000", 2024-09-14T23:58:09.333 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2899999999999999e-07 2024-09-14T23:58:09.333 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.333 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.333 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.333 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094393+0000", 2024-09-14T23:58:09.333 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6139999999999999e-06 2024-09-14T23:58:09.333 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.333 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.333 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.334 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094400+0000", 2024-09-14T23:58:09.334 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4010000000000002e-06 2024-09-14T23:58:09.334 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.334 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.334 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.334 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094405+0000", 2024-09-14T23:58:09.334 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6409999999999999e-06 2024-09-14T23:58:09.334 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.334 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.334 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.334 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.334 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.334 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188176 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.334 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.032204+0000", 2024-09-14T23:58:09.335 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.061656945999999997, 2024-09-14T23:58:09.335 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065572000000000005, 2024-09-14T23:58:09.335 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.335 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.335 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.335 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.335 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.335 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.335 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188176 2024-09-14T23:58:09.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.335 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.335 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.335 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.336 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032204+0000", 2024-09-14T23:58:09.336 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.336 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.336 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.336 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032204+0000", 2024-09-14T23:58:09.336 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.336 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.336 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.336 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032204+0000", 2024-09-14T23:58:09.336 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0900000000000002e-07 2024-09-14T23:58:09.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.336 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.336 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.337 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032207+0000", 2024-09-14T23:58:09.337 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.711e-06 2024-09-14T23:58:09.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.337 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.337 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.337 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032215+0000", 2024-09-14T23:58:09.337 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4980000000000006e-06 2024-09-14T23:58:09.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.337 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.337 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.337 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032221+0000", 2024-09-14T23:58:09.337 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4530000000000001e-06 2024-09-14T23:58:09.337 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.338 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.338 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.338 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.338 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.338 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188208 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.338 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.033231+0000", 2024-09-14T23:58:09.338 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.060629526000000003, 2024-09-14T23:58:09.338 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.064602248000000001, 2024-09-14T23:58:09.338 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.338 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.338 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.338 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.338 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.338 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.338 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188208 2024-09-14T23:58:09.339 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.339 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.339 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.339 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.339 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033231+0000", 2024-09-14T23:58:09.339 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.339 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.339 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.339 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.339 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033231+0000", 2024-09-14T23:58:09.339 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.339 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.339 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.339 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.340 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033232+0000", 2024-09-14T23:58:09.340 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.27e-07 2024-09-14T23:58:09.340 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.340 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.340 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.340 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033234+0000", 2024-09-14T23:58:09.340 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.973e-06 2024-09-14T23:58:09.340 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.340 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.340 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.340 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033241+0000", 2024-09-14T23:58:09.340 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9959999999999996e-06 2024-09-14T23:58:09.340 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.341 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.341 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.341 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033246+0000", 2024-09-14T23:58:09.341 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3639999999999998e-06 2024-09-14T23:58:09.341 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.341 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.341 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.341 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.341 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188688 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.341 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.056535+0000", 2024-09-14T23:58:09.341 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.037325924000000003, 2024-09-14T23:58:09.341 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.041358484000000001, 2024-09-14T23:58:09.342 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.342 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.342 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.342 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.342 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.342 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.342 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188688 2024-09-14T23:58:09.342 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.342 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.342 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.342 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.342 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056535+0000", 2024-09-14T23:58:09.342 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.342 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.343 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.343 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.343 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056535+0000", 2024-09-14T23:58:09.343 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.343 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.343 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.343 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.343 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056536+0000", 2024-09-14T23:58:09.343 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8999999999999996e-07 2024-09-14T23:58:09.343 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.343 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.343 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.343 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056539+0000", 2024-09-14T23:58:09.343 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7469999999999999e-06 2024-09-14T23:58:09.344 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.344 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.344 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.344 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056548+0000", 2024-09-14T23:58:09.344 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1349999999999998e-06 2024-09-14T23:58:09.344 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.344 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.344 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.344 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056555+0000", 2024-09-14T23:58:09.344 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5359999999999998e-06 2024-09-14T23:58:09.344 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.344 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.344 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.345 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.345 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.345 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188720 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.345 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.061362+0000", 2024-09-14T23:58:09.345 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.032498788000000001, 2024-09-14T23:58:09.345 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.036589654999999999, 2024-09-14T23:58:09.345 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.345 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.345 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.345 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.345 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.345 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.345 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188720 2024-09-14T23:58:09.346 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.346 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.346 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.346 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.346 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061362+0000", 2024-09-14T23:58:09.346 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.346 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.346 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.346 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.346 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061362+0000", 2024-09-14T23:58:09.346 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.346 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.346 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.346 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.347 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061362+0000", 2024-09-14T23:58:09.347 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1100000000000001e-07 2024-09-14T23:58:09.347 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.347 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.347 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.347 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061365+0000", 2024-09-14T23:58:09.347 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3829999999999999e-06 2024-09-14T23:58:09.347 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.347 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.347 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.347 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061371+0000", 2024-09-14T23:58:09.347 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5429999999999999e-06 2024-09-14T23:58:09.347 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.347 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.348 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.348 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061378+0000", 2024-09-14T23:58:09.348 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1449999999999996e-06 2024-09-14T23:58:09.348 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.348 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.348 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.348 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.348 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.348 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189201 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.348 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.085812+0000", 2024-09-14T23:58:09.348 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0080490419999999993, 2024-09-14T23:58:09.348 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.012197583, 2024-09-14T23:58:09.348 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.348 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.348 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.349 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.349 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.349 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.349 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189201 2024-09-14T23:58:09.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.349 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.349 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.349 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.349 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085812+0000", 2024-09-14T23:58:09.349 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.349 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.349 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.349 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085812+0000", 2024-09-14T23:58:09.350 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.350 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.350 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.350 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.350 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085812+0000", 2024-09-14T23:58:09.350 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4200000000000001e-07 2024-09-14T23:58:09.350 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.350 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.350 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.350 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085814+0000", 2024-09-14T23:58:09.350 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.075e-06 2024-09-14T23:58:09.350 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.350 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.350 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.350 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085822+0000", 2024-09-14T23:58:09.351 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.892e-06 2024-09-14T23:58:09.351 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.351 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.351 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.351 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085827+0000", 2024-09-14T23:58:09.351 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1540000000000001e-06 2024-09-14T23:58:09.351 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.351 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.351 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.351 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.351 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.351 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189233 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.351 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.088148+0000", 2024-09-14T23:58:09.352 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0057131880000000001, 2024-09-14T23:58:09.352 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0099725330000000004, 2024-09-14T23:58:09.352 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.352 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.352 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.352 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.352 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.352 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.352 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189233 2024-09-14T23:58:09.352 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.352 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.352 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.352 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.352 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088148+0000", 2024-09-14T23:58:09.353 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.353 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.353 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.353 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.353 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088148+0000", 2024-09-14T23:58:09.353 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.353 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.353 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.353 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.353 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088148+0000", 2024-09-14T23:58:09.353 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9799999999999999e-07 2024-09-14T23:58:09.353 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.353 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.354 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.354 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088150+0000", 2024-09-14T23:58:09.354 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0320000000000002e-06 2024-09-14T23:58:09.354 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.354 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.354 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.354 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088157+0000", 2024-09-14T23:58:09.354 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.618e-06 2024-09-14T23:58:09.354 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.354 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.354 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.354 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088163+0000", 2024-09-14T23:58:09.354 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1399999999999997e-06 2024-09-14T23:58:09.354 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.355 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.355 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.355 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.355 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.355 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189361 3.1d 3.e507d9bd (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.355 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.094750+0000", 2024-09-14T23:58:09.355 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4294967295.9991102, 2024-09-14T23:58:09.355 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0034296639999999998, 2024-09-14T23:58:09.355 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.355 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.355 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.355 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.355 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.355 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.355 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189361 2024-09-14T23:58:09.356 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.356 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.356 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.356 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.356 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094750+0000", 2024-09-14T23:58:09.356 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.356 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.356 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.356 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.356 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094750+0000", 2024-09-14T23:58:09.356 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.356 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.356 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.357 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.357 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094751+0000", 2024-09-14T23:58:09.357 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7999999999999995e-07 2024-09-14T23:58:09.357 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.357 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.357 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.357 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094755+0000", 2024-09-14T23:58:09.357 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8240000000000001e-06 2024-09-14T23:58:09.357 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.357 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.357 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.357 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094767+0000", 2024-09-14T23:58:09.357 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2459e-05 2024-09-14T23:58:09.357 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.357 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.358 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.358 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094775+0000", 2024-09-14T23:58:09.358 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3470000000000001e-06 2024-09-14T23:58:09.358 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.358 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.358 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.358 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.358 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.358 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188177 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.358 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.032230+0000", 2024-09-14T23:58:09.358 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.061631035000000001, 2024-09-14T23:58:09.358 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066011374999999997, 2024-09-14T23:58:09.358 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.358 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.359 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.359 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.359 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.359 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.359 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188177 2024-09-14T23:58:09.359 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.359 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.359 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.359 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.359 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032230+0000", 2024-09-14T23:58:09.359 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.359 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.359 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.359 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.360 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032230+0000", 2024-09-14T23:58:09.360 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.360 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.360 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.360 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.360 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032230+0000", 2024-09-14T23:58:09.360 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.77e-07 2024-09-14T23:58:09.360 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.360 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.360 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.360 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032233+0000", 2024-09-14T23:58:09.360 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5179999999999999e-06 2024-09-14T23:58:09.360 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.360 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.360 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.361 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032241+0000", 2024-09-14T23:58:09.361 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2290000000000003e-06 2024-09-14T23:58:09.361 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.361 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.361 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.361 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032246+0000", 2024-09-14T23:58:09.361 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5140000000000001e-06 2024-09-14T23:58:09.361 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.361 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.361 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.361 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.361 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.361 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188209 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.362 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.033252+0000", 2024-09-14T23:58:09.362 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.060608425, 2024-09-14T23:58:09.362 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065045852000000001, 2024-09-14T23:58:09.362 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.362 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.362 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.362 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.362 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.362 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.362 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188209 2024-09-14T23:58:09.362 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.362 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.362 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.362 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.363 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033252+0000", 2024-09-14T23:58:09.363 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.363 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.363 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.363 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.363 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033252+0000", 2024-09-14T23:58:09.363 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.363 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.363 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.363 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.363 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033253+0000", 2024-09-14T23:58:09.363 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9299999999999999e-07 2024-09-14T23:58:09.363 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.363 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.364 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.364 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033255+0000", 2024-09-14T23:58:09.364 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9259999999999999e-06 2024-09-14T23:58:09.364 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.364 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.364 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.364 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033261+0000", 2024-09-14T23:58:09.364 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2910000000000004e-06 2024-09-14T23:58:09.364 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.364 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.364 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.364 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033266+0000", 2024-09-14T23:58:09.364 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.383e-06 2024-09-14T23:58:09.364 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.365 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.365 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.365 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.365 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.365 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188689 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.365 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.056565+0000", 2024-09-14T23:58:09.365 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.037295852999999997, 2024-09-14T23:58:09.365 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.041790226, 2024-09-14T23:58:09.365 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.365 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.365 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.365 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.365 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.365 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.366 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188689 2024-09-14T23:58:09.366 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.366 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.366 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.366 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.366 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056565+0000", 2024-09-14T23:58:09.366 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.366 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.366 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.366 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.366 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056565+0000", 2024-09-14T23:58:09.366 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.366 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.367 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.367 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.367 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056566+0000", 2024-09-14T23:58:09.367 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4899999999999995e-07 2024-09-14T23:58:09.367 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.367 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.367 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.367 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056568+0000", 2024-09-14T23:58:09.367 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6769999999999999e-06 2024-09-14T23:58:09.367 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.367 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.367 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.367 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056576+0000", 2024-09-14T23:58:09.367 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5259999999999996e-06 2024-09-14T23:58:09.368 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.368 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.368 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.368 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056582+0000", 2024-09-14T23:58:09.368 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8340000000000004e-06 2024-09-14T23:58:09.368 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.368 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.368 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.368 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.368 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.368 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188721 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.368 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.061385+0000", 2024-09-14T23:58:09.368 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.032475506000000001, 2024-09-14T23:58:09.369 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.037027799, 2024-09-14T23:58:09.369 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.369 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.369 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.369 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.369 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.369 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.369 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188721 2024-09-14T23:58:09.369 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.369 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.369 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.369 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.369 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061385+0000", 2024-09-14T23:58:09.369 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.370 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.370 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.370 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.370 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061385+0000", 2024-09-14T23:58:09.370 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.370 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.370 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.370 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.370 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061386+0000", 2024-09-14T23:58:09.370 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1800000000000001e-07 2024-09-14T23:58:09.370 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.370 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.370 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.370 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061388+0000", 2024-09-14T23:58:09.371 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4430000000000002e-06 2024-09-14T23:58:09.371 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.371 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.371 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.371 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061394+0000", 2024-09-14T23:58:09.371 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1340000000000001e-06 2024-09-14T23:58:09.371 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.371 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.371 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.371 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061399+0000", 2024-09-14T23:58:09.371 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7230000000000004e-06 2024-09-14T23:58:09.371 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.371 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.371 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.371 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.372 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.372 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189202 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.372 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.085834+0000", 2024-09-14T23:58:09.372 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0080267930000000008, 2024-09-14T23:58:09.372 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.012635884, 2024-09-14T23:58:09.372 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.372 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.372 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.372 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.372 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.372 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.372 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189202 2024-09-14T23:58:09.372 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.372 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.373 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.373 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.373 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085834+0000", 2024-09-14T23:58:09.373 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.373 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.373 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.373 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.373 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085834+0000", 2024-09-14T23:58:09.373 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.373 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.373 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.373 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.373 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085834+0000", 2024-09-14T23:58:09.373 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9099999999999999e-07 2024-09-14T23:58:09.374 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.374 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.374 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.374 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085836+0000", 2024-09-14T23:58:09.374 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9309999999999998e-06 2024-09-14T23:58:09.374 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.374 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.374 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.374 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085843+0000", 2024-09-14T23:58:09.374 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4409999999999998e-06 2024-09-14T23:58:09.374 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.374 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.374 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.375 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085848+0000", 2024-09-14T23:58:09.375 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9239999999999998e-06 2024-09-14T23:58:09.375 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.375 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.375 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.375 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.375 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.375 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189234 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.375 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.088169+0000", 2024-09-14T23:58:09.375 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.005691527, 2024-09-14T23:58:09.375 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010356552, 2024-09-14T23:58:09.375 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.375 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.375 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.375 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.376 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.376 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.376 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189234 2024-09-14T23:58:09.376 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.376 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.376 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.376 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.376 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088169+0000", 2024-09-14T23:58:09.376 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.376 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.376 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.376 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.376 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088169+0000", 2024-09-14T23:58:09.376 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.377 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.377 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.377 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.377 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088170+0000", 2024-09-14T23:58:09.377 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8700000000000001e-07 2024-09-14T23:58:09.380 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.380 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.380 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.380 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088181+0000", 2024-09-14T23:58:09.380 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1783999999999999e-05 2024-09-14T23:58:09.380 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.380 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.380 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.381 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088189+0000", 2024-09-14T23:58:09.381 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2320000000000004e-06 2024-09-14T23:58:09.381 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.381 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.381 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.381 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088195+0000", 2024-09-14T23:58:09.381 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4030000000000004e-06 2024-09-14T23:58:09.381 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.381 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.381 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.381 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.381 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.381 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189362 3.1d 3.e507d9bd (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.381 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.094787+0000", 2024-09-14T23:58:09.381 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4294967295.9990735, 2024-09-14T23:58:09.382 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0037953280000000002, 2024-09-14T23:58:09.382 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.382 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.382 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.382 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.382 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.382 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.382 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189362 2024-09-14T23:58:09.382 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.382 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.382 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.382 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.382 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094787+0000", 2024-09-14T23:58:09.382 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.382 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.383 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.383 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.383 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094787+0000", 2024-09-14T23:58:09.383 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.383 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.383 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.383 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.383 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094788+0000", 2024-09-14T23:58:09.383 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1099999999999995e-07 2024-09-14T23:58:09.383 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.383 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.383 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.383 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094790+0000", 2024-09-14T23:58:09.383 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7130000000000001e-06 2024-09-14T23:58:09.384 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.384 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.384 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.384 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094797+0000", 2024-09-14T23:58:09.384 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8970000000000002e-06 2024-09-14T23:58:09.384 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.384 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.384 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.384 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094802+0000", 2024-09-14T23:58:09.384 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9819999999999999e-06 2024-09-14T23:58:09.384 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.384 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.384 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.385 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.385 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.385 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188178 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.385 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.032255+0000", 2024-09-14T23:58:09.385 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.061605655000000002, 2024-09-14T23:58:09.385 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066388741000000001, 2024-09-14T23:58:09.385 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.385 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.385 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.385 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.385 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.385 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.385 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188178 2024-09-14T23:58:09.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.386 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.386 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.386 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.386 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032255+0000", 2024-09-14T23:58:09.386 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.386 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.386 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.386 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032255+0000", 2024-09-14T23:58:09.386 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.386 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.387 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.387 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032256+0000", 2024-09-14T23:58:09.387 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7500000000000001e-07 2024-09-14T23:58:09.387 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.387 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.387 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.387 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032258+0000", 2024-09-14T23:58:09.387 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4549999999999998e-06 2024-09-14T23:58:09.387 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.387 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.387 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.387 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032266+0000", 2024-09-14T23:58:09.387 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7389999999999999e-06 2024-09-14T23:58:09.387 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.387 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.388 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.388 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032271+0000", 2024-09-14T23:58:09.388 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1460000000000004e-06 2024-09-14T23:58:09.388 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.388 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.388 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.388 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.388 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.388 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188210 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.388 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.033273+0000", 2024-09-14T23:58:09.388 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.060587955999999998, 2024-09-14T23:58:09.388 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065440645000000006, 2024-09-14T23:58:09.388 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.388 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.389 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.389 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.389 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.389 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.389 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188210 2024-09-14T23:58:09.389 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.389 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.389 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.389 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.389 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033273+0000", 2024-09-14T23:58:09.389 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.389 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.389 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.389 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.390 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033273+0000", 2024-09-14T23:58:09.390 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.390 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.390 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.390 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.390 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033273+0000", 2024-09-14T23:58:09.390 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5600000000000001e-07 2024-09-14T23:58:09.390 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.390 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.390 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.390 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033275+0000", 2024-09-14T23:58:09.390 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1399999999999998e-06 2024-09-14T23:58:09.390 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.390 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.390 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.391 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033283+0000", 2024-09-14T23:58:09.391 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7409999999999992e-06 2024-09-14T23:58:09.391 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.391 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.391 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.391 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033289+0000", 2024-09-14T23:58:09.391 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3680000000000001e-06 2024-09-14T23:58:09.391 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.391 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.391 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.391 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.391 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.391 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188690 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.392 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.056589+0000", 2024-09-14T23:58:09.392 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.037272113000000003, 2024-09-14T23:58:09.392 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.042207860999999999, 2024-09-14T23:58:09.392 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.392 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.392 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.392 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.392 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.392 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.392 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188690 2024-09-14T23:58:09.392 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.392 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.392 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.392 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.393 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056589+0000", 2024-09-14T23:58:09.393 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.393 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.393 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.393 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056589+0000", 2024-09-14T23:58:09.393 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.393 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.393 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.393 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056589+0000", 2024-09-14T23:58:09.393 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8100000000000003e-07 2024-09-14T23:58:09.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.394 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.394 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.394 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056591+0000", 2024-09-14T23:58:09.394 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.024e-06 2024-09-14T23:58:09.394 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.394 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.394 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.394 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056601+0000", 2024-09-14T23:58:09.394 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.465e-06 2024-09-14T23:58:09.394 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.394 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.394 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.394 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056607+0000", 2024-09-14T23:58:09.394 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9789999999999998e-06 2024-09-14T23:58:09.395 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.395 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.395 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.395 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.395 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.395 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188722 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.395 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.061405+0000", 2024-09-14T23:58:09.395 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.032455393999999999, 2024-09-14T23:58:09.395 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.037449365999999998, 2024-09-14T23:58:09.395 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.395 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.395 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.395 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.395 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.395 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.396 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188722 2024-09-14T23:58:09.396 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.396 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.396 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.396 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.396 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061405+0000", 2024-09-14T23:58:09.396 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.396 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.396 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.396 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.396 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061405+0000", 2024-09-14T23:58:09.396 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.397 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.397 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.397 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.397 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061406+0000", 2024-09-14T23:58:09.397 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4299999999999999e-07 2024-09-14T23:58:09.397 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.397 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.397 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.397 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061408+0000", 2024-09-14T23:58:09.397 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1289999999999999e-06 2024-09-14T23:58:09.397 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.397 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.398 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.398 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061414+0000", 2024-09-14T23:58:09.398 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0159999999999997e-06 2024-09-14T23:58:09.398 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.398 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.398 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.398 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061419+0000", 2024-09-14T23:58:09.398 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0710000000000002e-06 2024-09-14T23:58:09.398 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.398 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.398 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.398 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.399 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.399 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189203 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.399 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.085855+0000", 2024-09-14T23:58:09.399 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0080060719999999995, 2024-09-14T23:58:09.399 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.013057602999999999, 2024-09-14T23:58:09.399 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.399 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.399 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.399 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.399 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.399 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.399 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189203 2024-09-14T23:58:09.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.400 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.400 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.400 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.400 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085855+0000", 2024-09-14T23:58:09.400 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.400 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.400 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.400 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.400 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085855+0000", 2024-09-14T23:58:09.400 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.400 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.400 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.400 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.400 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085855+0000", 2024-09-14T23:58:09.401 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1600000000000002e-07 2024-09-14T23:58:09.401 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.401 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.401 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.401 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085857+0000", 2024-09-14T23:58:09.401 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2139999999999999e-06 2024-09-14T23:58:09.401 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.401 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.401 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.401 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085864+0000", 2024-09-14T23:58:09.401 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3620000000000002e-06 2024-09-14T23:58:09.401 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.401 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.402 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.402 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085869+0000", 2024-09-14T23:58:09.402 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1569999999999999e-06 2024-09-14T23:58:09.402 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.402 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.402 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.402 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.402 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.402 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189235 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.402 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.088216+0000", 2024-09-14T23:58:09.402 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0056443980000000001, 2024-09-14T23:58:09.402 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010752106000000001, 2024-09-14T23:58:09.402 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.402 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.403 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.403 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.403 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.403 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.403 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189235 2024-09-14T23:58:09.403 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.403 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.403 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.403 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.403 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088216+0000", 2024-09-14T23:58:09.403 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.403 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.403 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.403 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.404 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088216+0000", 2024-09-14T23:58:09.404 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.404 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.404 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.404 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.404 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088217+0000", 2024-09-14T23:58:09.404 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6100000000000001e-07 2024-09-14T23:58:09.404 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.404 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.404 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.404 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088219+0000", 2024-09-14T23:58:09.404 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3999999999999999e-06 2024-09-14T23:58:09.404 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.404 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.405 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.405 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088227+0000", 2024-09-14T23:58:09.405 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.221e-06 2024-09-14T23:58:09.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.405 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.405 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.405 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088234+0000", 2024-09-14T23:58:09.405 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.311e-06 2024-09-14T23:58:09.405 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.405 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.405 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.405 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.405 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189363 3.1d 3.e507d9bd (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.406 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.094811+0000", 2024-09-14T23:58:09.406 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4294967295.9990501, 2024-09-14T23:58:09.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.004224546, 2024-09-14T23:58:09.406 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.406 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.406 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.406 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.406 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.406 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.406 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189363 2024-09-14T23:58:09.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.406 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.406 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094811+0000", 2024-09-14T23:58:09.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.407 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094811+0000", 2024-09-14T23:58:09.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.407 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094811+0000", 2024-09-14T23:58:09.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9999999999999998e-07 2024-09-14T23:58:09.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.408 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.408 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094813+0000", 2024-09-14T23:58:09.408 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.368e-06 2024-09-14T23:58:09.408 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.408 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.408 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.408 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094822+0000", 2024-09-14T23:58:09.408 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.473e-06 2024-09-14T23:58:09.408 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.408 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.408 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.408 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094828+0000", 2024-09-14T23:58:09.408 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6119999999999998e-06 2024-09-14T23:58:09.408 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.409 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.409 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.409 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.409 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.409 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188179 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.409 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.032281+0000", 2024-09-14T23:58:09.409 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.061580227000000001, 2024-09-14T23:58:09.409 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066816431999999995, 2024-09-14T23:58:09.409 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.409 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.409 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.409 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.409 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.409 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.410 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188179 2024-09-14T23:58:09.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.410 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.410 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.410 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.410 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032281+0000", 2024-09-14T23:58:09.410 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.410 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.410 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.410 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032281+0000", 2024-09-14T23:58:09.410 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.410 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.411 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.411 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032281+0000", 2024-09-14T23:58:09.411 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2600000000000002e-07 2024-09-14T23:58:09.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.411 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.411 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.411 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032284+0000", 2024-09-14T23:58:09.411 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5399999999999998e-06 2024-09-14T23:58:09.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.411 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.411 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.411 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032292+0000", 2024-09-14T23:58:09.411 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4149999999999999e-06 2024-09-14T23:58:09.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.412 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.412 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.412 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032297+0000", 2024-09-14T23:58:09.412 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2669999999999997e-06 2024-09-14T23:58:09.412 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.412 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.412 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.412 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.412 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.412 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188211 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.412 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.033295+0000", 2024-09-14T23:58:09.412 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.060565859, 2024-09-14T23:58:09.412 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065857053999999998, 2024-09-14T23:58:09.413 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.413 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.413 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.413 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.413 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.413 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.413 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188211 2024-09-14T23:58:09.413 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.413 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.413 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.413 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.413 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033295+0000", 2024-09-14T23:58:09.413 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.413 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.413 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.414 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.414 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033295+0000", 2024-09-14T23:58:09.414 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.414 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.414 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.414 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033685+0000", 2024-09-14T23:58:09.414 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00039020899999999998 2024-09-14T23:58:09.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.414 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.414 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.414 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033689+0000", 2024-09-14T23:58:09.414 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.233e-06 2024-09-14T23:58:09.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.415 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.415 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.415 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033700+0000", 2024-09-14T23:58:09.415 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1026000000000001e-05 2024-09-14T23:58:09.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.415 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.415 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.415 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033708+0000", 2024-09-14T23:58:09.415 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4390000000000003e-06 2024-09-14T23:58:09.415 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.415 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.415 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.415 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.415 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188691 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.416 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.056614+0000", 2024-09-14T23:58:09.416 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.037246658000000002, 2024-09-14T23:58:09.416 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.042595624999999998, 2024-09-14T23:58:09.416 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.416 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.416 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.416 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.416 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.416 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.416 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188691 2024-09-14T23:58:09.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.416 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.416 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.417 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.417 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056614+0000", 2024-09-14T23:58:09.417 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.417 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.417 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.417 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.417 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056614+0000", 2024-09-14T23:58:09.417 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.417 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.417 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.417 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.417 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056615+0000", 2024-09-14T23:58:09.417 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3599999999999999e-07 2024-09-14T23:58:09.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.418 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.418 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056617+0000", 2024-09-14T23:58:09.418 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2369999999999999e-06 2024-09-14T23:58:09.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.418 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.418 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056624+0000", 2024-09-14T23:58:09.418 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8800000000000002e-06 2024-09-14T23:58:09.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.418 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.418 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056629+0000", 2024-09-14T23:58:09.419 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2830000000000001e-06 2024-09-14T23:58:09.419 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.419 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.419 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.419 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.419 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188723 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.419 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.061426+0000", 2024-09-14T23:58:09.419 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.032435125000000002, 2024-09-14T23:58:09.419 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.037841473, 2024-09-14T23:58:09.419 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.419 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.419 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.419 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.420 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.420 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.420 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188723 2024-09-14T23:58:09.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.420 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.420 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.420 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.420 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061426+0000", 2024-09-14T23:58:09.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.420 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.420 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.420 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061426+0000", 2024-09-14T23:58:09.421 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.421 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.421 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.421 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061426+0000", 2024-09-14T23:58:09.421 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.53e-07 2024-09-14T23:58:09.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.421 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.421 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.421 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061428+0000", 2024-09-14T23:58:09.421 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0729999999999999e-06 2024-09-14T23:58:09.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.421 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.422 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.422 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061435+0000", 2024-09-14T23:58:09.422 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3929999999999996e-06 2024-09-14T23:58:09.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.422 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.422 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.422 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061440+0000", 2024-09-14T23:58:09.422 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1340000000000003e-06 2024-09-14T23:58:09.422 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.422 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.422 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.422 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.422 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189204 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.423 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.085875+0000", 2024-09-14T23:58:09.423 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0079853319999999995, 2024-09-14T23:58:09.423 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.013447855999999999, 2024-09-14T23:58:09.423 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.423 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.423 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.423 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.423 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.423 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.423 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189204 2024-09-14T23:58:09.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.423 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085875+0000", 2024-09-14T23:58:09.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085875+0000", 2024-09-14T23:58:09.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085876+0000", 2024-09-14T23:58:09.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9799999999999999e-07 2024-09-14T23:58:09.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.425 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.425 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085878+0000", 2024-09-14T23:58:09.425 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0509999999999999e-06 2024-09-14T23:58:09.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.425 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.425 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085884+0000", 2024-09-14T23:58:09.425 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4540000000000004e-06 2024-09-14T23:58:09.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.425 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.425 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085890+0000", 2024-09-14T23:58:09.426 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2240000000000003e-06 2024-09-14T23:58:09.426 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.426 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.426 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.426 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.426 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.426 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189236 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.426 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.088248+0000", 2024-09-14T23:58:09.426 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0056129190000000001, 2024-09-14T23:58:09.426 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.011130472000000001, 2024-09-14T23:58:09.426 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.426 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.426 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.426 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.427 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.427 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.427 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189236 2024-09-14T23:58:09.427 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.427 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.427 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.427 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.427 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088248+0000", 2024-09-14T23:58:09.427 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.427 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.427 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.427 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.427 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088248+0000", 2024-09-14T23:58:09.428 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.428 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.428 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.428 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.428 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088248+0000", 2024-09-14T23:58:09.428 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7e-07 2024-09-14T23:58:09.428 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.428 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.428 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.428 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088251+0000", 2024-09-14T23:58:09.428 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.745e-06 2024-09-14T23:58:09.428 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.428 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.428 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.429 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088258+0000", 2024-09-14T23:58:09.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1439999999999997e-06 2024-09-14T23:58:09.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.429 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.429 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088263+0000", 2024-09-14T23:58:09.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.78e-06 2024-09-14T23:58:09.429 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.429 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.429 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.429 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189364 3.1d 3.e507d9bd (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.430 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.094835+0000", 2024-09-14T23:58:09.430 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4294967295.9990263, 2024-09-14T23:58:09.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0046028199999999997, 2024-09-14T23:58:09.430 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.430 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.430 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.430 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.430 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.430 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.430 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189364 2024-09-14T23:58:09.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.430 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.430 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.431 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094835+0000", 2024-09-14T23:58:09.431 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.431 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.431 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.431 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094835+0000", 2024-09-14T23:58:09.431 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.431 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.431 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.431 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094835+0000", 2024-09-14T23:58:09.431 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4299999999999998e-07 2024-09-14T23:58:09.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.432 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.432 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094837+0000", 2024-09-14T23:58:09.432 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2230000000000001e-06 2024-09-14T23:58:09.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.432 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.432 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094844+0000", 2024-09-14T23:58:09.432 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3759999999999996e-06 2024-09-14T23:58:09.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.432 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.432 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094849+0000", 2024-09-14T23:58:09.433 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4099999999999999e-06 2024-09-14T23:58:09.433 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.433 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.433 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.433 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188180 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.433 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.032304+0000", 2024-09-14T23:58:09.433 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.061556592, 2024-09-14T23:58:09.433 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.067192718999999998, 2024-09-14T23:58:09.433 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.433 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.433 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.433 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.434 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.434 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.434 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188180 2024-09-14T23:58:09.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.434 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.434 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.434 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032304+0000", 2024-09-14T23:58:09.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.434 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.434 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032304+0000", 2024-09-14T23:58:09.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.435 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.435 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032305+0000", 2024-09-14T23:58:09.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1399999999999997e-07 2024-09-14T23:58:09.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.435 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.435 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032307+0000", 2024-09-14T23:58:09.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3989999999999998e-06 2024-09-14T23:58:09.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.435 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.435 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032315+0000", 2024-09-14T23:58:09.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3210000000000005e-06 2024-09-14T23:58:09.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.436 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.436 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032321+0000", 2024-09-14T23:58:09.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4859999999999997e-06 2024-09-14T23:58:09.436 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.436 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.436 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.436 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188212 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.436 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.033719+0000", 2024-09-14T23:58:09.436 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.060141496000000003, 2024-09-14T23:58:09.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.065834711000000004, 2024-09-14T23:58:09.437 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.437 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.437 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.437 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.437 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.437 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.437 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188212 2024-09-14T23:58:09.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.437 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.437 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033719+0000", 2024-09-14T23:58:09.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033719+0000", 2024-09-14T23:58:09.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033720+0000", 2024-09-14T23:58:09.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1900000000000002e-07 2024-09-14T23:58:09.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.439 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033722+0000", 2024-09-14T23:58:09.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5600000000000001e-06 2024-09-14T23:58:09.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.439 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.439 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033730+0000", 2024-09-14T23:58:09.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4660000000000002e-06 2024-09-14T23:58:09.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.439 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.439 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033736+0000", 2024-09-14T23:58:09.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6250000000000004e-06 2024-09-14T23:58:09.439 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.439 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.439 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.440 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188692 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.440 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.056636+0000", 2024-09-14T23:58:09.440 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.037224737000000001, 2024-09-14T23:58:09.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.04297463, 2024-09-14T23:58:09.440 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.440 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.440 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.440 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.440 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.440 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.440 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188692 2024-09-14T23:58:09.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.441 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056636+0000", 2024-09-14T23:58:09.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056636+0000", 2024-09-14T23:58:09.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056637+0000", 2024-09-14T23:58:09.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4900000000000001e-07 2024-09-14T23:58:09.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056644+0000", 2024-09-14T23:58:09.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4170000000000003e-06 2024-09-14T23:58:09.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056674+0000", 2024-09-14T23:58:09.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0057e-05 2024-09-14T23:58:09.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.443 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.443 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056686+0000", 2024-09-14T23:58:09.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1661000000000001e-05 2024-09-14T23:58:09.443 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.443 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.443 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.443 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188724 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.443 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.061446+0000", 2024-09-14T23:58:09.443 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.032414804999999998, 2024-09-14T23:58:09.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.038229383999999998, 2024-09-14T23:58:09.443 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.444 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.444 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.444 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.444 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.444 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.444 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188724 2024-09-14T23:58:09.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.444 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061446+0000", 2024-09-14T23:58:09.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061446+0000", 2024-09-14T23:58:09.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061446+0000", 2024-09-14T23:58:09.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7099999999999997e-07 2024-09-14T23:58:09.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061448+0000", 2024-09-14T23:58:09.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0590000000000001e-06 2024-09-14T23:58:09.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.446 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061454+0000", 2024-09-14T23:58:09.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0480000000000004e-06 2024-09-14T23:58:09.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.446 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061459+0000", 2024-09-14T23:58:09.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9099999999999996e-06 2024-09-14T23:58:09.446 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.446 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.446 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.447 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189205 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.447 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.085896+0000", 2024-09-14T23:58:09.447 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0079643950000000008, 2024-09-14T23:58:09.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.013837324999999999, 2024-09-14T23:58:09.447 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.447 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.447 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.447 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.447 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.447 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.447 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189205 2024-09-14T23:58:09.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.447 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085896+0000", 2024-09-14T23:58:09.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085896+0000", 2024-09-14T23:58:09.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085897+0000", 2024-09-14T23:58:09.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4200000000000001e-07 2024-09-14T23:58:09.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085899+0000", 2024-09-14T23:58:09.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.979e-06 2024-09-14T23:58:09.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085905+0000", 2024-09-14T23:58:09.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5819999999999998e-06 2024-09-14T23:58:09.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085910+0000", 2024-09-14T23:58:09.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9609999999999996e-06 2024-09-14T23:58:09.449 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.449 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.449 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.450 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189237 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.450 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.088271+0000", 2024-09-14T23:58:09.450 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.005590205, 2024-09-14T23:58:09.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.011520837000000001, 2024-09-14T23:58:09.450 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.450 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.450 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.450 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.450 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.450 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.450 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189237 2024-09-14T23:58:09.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.450 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088271+0000", 2024-09-14T23:58:09.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088271+0000", 2024-09-14T23:58:09.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088271+0000", 2024-09-14T23:58:09.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5600000000000001e-07 2024-09-14T23:58:09.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088273+0000", 2024-09-14T23:58:09.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3539999999999998e-06 2024-09-14T23:58:09.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088280+0000", 2024-09-14T23:58:09.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.246e-06 2024-09-14T23:58:09.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088285+0000", 2024-09-14T23:58:09.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9429999999999999e-06 2024-09-14T23:58:09.452 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.452 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.453 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.453 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189365 3.1d 3.e507d9bd (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.453 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.094856+0000", 2024-09-14T23:58:09.453 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4294967295.9990044, 2024-09-14T23:58:09.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0049923169999999996, 2024-09-14T23:58:09.453 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.453 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.453 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.453 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.453 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.453 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.453 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189365 2024-09-14T23:58:09.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.454 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094856+0000", 2024-09-14T23:58:09.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094856+0000", 2024-09-14T23:58:09.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094857+0000", 2024-09-14T23:58:09.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.89e-07 2024-09-14T23:58:09.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094859+0000", 2024-09-14T23:58:09.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0379999999999998e-06 2024-09-14T23:58:09.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094865+0000", 2024-09-14T23:58:09.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6959999999999999e-06 2024-09-14T23:58:09.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094871+0000", 2024-09-14T23:58:09.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3959999999999997e-06 2024-09-14T23:58:09.456 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.456 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.456 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.456 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188181 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.456 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.032328+0000", 2024-09-14T23:58:09.456 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.061532951000000002, 2024-09-14T23:58:09.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.067582640999999999, 2024-09-14T23:58:09.456 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.456 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.456 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.457 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.457 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.457 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.457 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188181 2024-09-14T23:58:09.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.457 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032328+0000", 2024-09-14T23:58:09.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032328+0000", 2024-09-14T23:58:09.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032328+0000", 2024-09-14T23:58:09.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5299999999999999e-07 2024-09-14T23:58:09.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032336+0000", 2024-09-14T23:58:09.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1999999999999997e-06 2024-09-14T23:58:09.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032344+0000", 2024-09-14T23:58:09.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4819999999999994e-06 2024-09-14T23:58:09.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032350+0000", 2024-09-14T23:58:09.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6250000000000004e-06 2024-09-14T23:58:09.459 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.459 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.459 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.459 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188213 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.459 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.033745+0000", 2024-09-14T23:58:09.460 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.060115900999999999, 2024-09-14T23:58:09.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066227173, 2024-09-14T23:58:09.460 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.460 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.460 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.460 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.460 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.460 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.460 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188213 2024-09-14T23:58:09.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.460 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033745+0000", 2024-09-14T23:58:09.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033745+0000", 2024-09-14T23:58:09.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033745+0000", 2024-09-14T23:58:09.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3500000000000002e-07 2024-09-14T23:58:09.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033748+0000", 2024-09-14T23:58:09.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.491e-06 2024-09-14T23:58:09.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033755+0000", 2024-09-14T23:58:09.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5739999999999998e-06 2024-09-14T23:58:09.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033761+0000", 2024-09-14T23:58:09.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6579999999999999e-06 2024-09-14T23:58:09.462 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.462 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.463 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.463 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188693 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.463 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.056701+0000", 2024-09-14T23:58:09.463 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.037159998, 2024-09-14T23:58:09.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043329126000000003, 2024-09-14T23:58:09.463 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.463 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.463 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.463 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.463 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.463 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.463 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188693 2024-09-14T23:58:09.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.464 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056701+0000", 2024-09-14T23:58:09.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056701+0000", 2024-09-14T23:58:09.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056702+0000", 2024-09-14T23:58:09.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5099999999999999e-07 2024-09-14T23:58:09.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056706+0000", 2024-09-14T23:58:09.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2649999999999998e-06 2024-09-14T23:58:09.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056716+0000", 2024-09-14T23:58:09.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8279999999999998e-06 2024-09-14T23:58:09.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056722+0000", 2024-09-14T23:58:09.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3720000000000001e-06 2024-09-14T23:58:09.466 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.466 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.466 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.466 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188725 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.466 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.061465+0000", 2024-09-14T23:58:09.466 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.032395535000000003, 2024-09-14T23:58:09.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.038622331000000003, 2024-09-14T23:58:09.466 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.467 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.467 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.467 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.467 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.467 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.467 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188725 2024-09-14T23:58:09.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.467 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061465+0000", 2024-09-14T23:58:09.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061465+0000", 2024-09-14T23:58:09.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061466+0000", 2024-09-14T23:58:09.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5199999999999998e-07 2024-09-14T23:58:09.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061468+0000", 2024-09-14T23:58:09.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9539999999999998e-06 2024-09-14T23:58:09.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.469 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.469 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061475+0000", 2024-09-14T23:58:09.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2640000000000002e-06 2024-09-14T23:58:09.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.469 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.469 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061482+0000", 2024-09-14T23:58:09.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7580000000000003e-06 2024-09-14T23:58:09.469 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.469 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.469 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.470 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189206 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.470 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.085917+0000", 2024-09-14T23:58:09.470 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0079442539999999996, 2024-09-14T23:58:09.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.014228051, 2024-09-14T23:58:09.470 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.470 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.470 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.470 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.470 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.470 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.470 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189206 2024-09-14T23:58:09.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.470 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085917+0000", 2024-09-14T23:58:09.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085917+0000", 2024-09-14T23:58:09.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085917+0000", 2024-09-14T23:58:09.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4300000000000003e-07 2024-09-14T23:58:09.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085920+0000", 2024-09-14T23:58:09.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1810000000000001e-06 2024-09-14T23:58:09.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085927+0000", 2024-09-14T23:58:09.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7630000000000003e-06 2024-09-14T23:58:09.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085933+0000", 2024-09-14T23:58:09.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6760000000000004e-06 2024-09-14T23:58:09.473 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.473 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.473 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.473 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189238 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.473 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.088292+0000", 2024-09-14T23:58:09.473 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.005568941, 2024-09-14T23:58:09.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.011908520000000001, 2024-09-14T23:58:09.473 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.473 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.474 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.474 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.474 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.474 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.474 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189238 2024-09-14T23:58:09.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.474 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088292+0000", 2024-09-14T23:58:09.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088292+0000", 2024-09-14T23:58:09.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088292+0000", 2024-09-14T23:58:09.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.96e-07 2024-09-14T23:58:09.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088295+0000", 2024-09-14T23:58:09.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3180000000000001e-06 2024-09-14T23:58:09.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088301+0000", 2024-09-14T23:58:09.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0900000000000001e-06 2024-09-14T23:58:09.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088306+0000", 2024-09-14T23:58:09.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9520000000000002e-06 2024-09-14T23:58:09.476 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.476 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.476 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.476 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189366 3.1d 3.e507d9bd (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.476 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.094878+0000", 2024-09-14T23:58:09.477 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4294967295.9989834, 2024-09-14T23:58:09.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0053814780000000003, 2024-09-14T23:58:09.477 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.477 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.477 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.477 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.477 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.477 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.477 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189366 2024-09-14T23:58:09.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.477 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094878+0000", 2024-09-14T23:58:09.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094878+0000", 2024-09-14T23:58:09.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094878+0000", 2024-09-14T23:58:09.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9299999999999999e-07 2024-09-14T23:58:09.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094880+0000", 2024-09-14T23:58:09.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.838e-06 2024-09-14T23:58:09.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094887+0000", 2024-09-14T23:58:09.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9110000000000004e-06 2024-09-14T23:58:09.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094892+0000", 2024-09-14T23:58:09.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3020000000000002e-06 2024-09-14T23:58:09.479 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.479 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.480 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.480 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188182 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.480 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.032357+0000", 2024-09-14T23:58:09.480 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.061503497999999997, 2024-09-14T23:58:09.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.067961635000000006, 2024-09-14T23:58:09.480 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.480 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.480 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.480 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.480 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.480 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.480 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188182 2024-09-14T23:58:09.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.481 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.481 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032357+0000", 2024-09-14T23:58:09.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.481 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032357+0000", 2024-09-14T23:58:09.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032358+0000", 2024-09-14T23:58:09.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2899999999999999e-07 2024-09-14T23:58:09.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032360+0000", 2024-09-14T23:58:09.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.311e-06 2024-09-14T23:58:09.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032368+0000", 2024-09-14T23:58:09.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1890000000000007e-06 2024-09-14T23:58:09.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032373+0000", 2024-09-14T23:58:09.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1030000000000001e-06 2024-09-14T23:58:09.483 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.483 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.483 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.483 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188214 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.483 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.033770+0000", 2024-09-14T23:58:09.483 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.060090434999999998, 2024-09-14T23:58:09.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066605559999999994, 2024-09-14T23:58:09.483 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.484 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.484 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.484 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.484 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.484 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.484 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188214 2024-09-14T23:58:09.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.484 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033770+0000", 2024-09-14T23:58:09.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.033770+0000", 2024-09-14T23:58:09.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034673+0000", 2024-09-14T23:58:09.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000902983 2024-09-14T23:58:09.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034678+0000", 2024-09-14T23:58:09.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5889999999999996e-06 2024-09-14T23:58:09.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034689+0000", 2024-09-14T23:58:09.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0601000000000001e-05 2024-09-14T23:58:09.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034696+0000", 2024-09-14T23:58:09.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7999999999999999e-06 2024-09-14T23:58:09.487 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.487 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.487 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.487 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188694 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.487 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.056731+0000", 2024-09-14T23:58:09.487 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.037129514000000002, 2024-09-14T23:58:09.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043702417, 2024-09-14T23:58:09.487 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.487 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.487 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.487 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.487 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.488 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.488 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188694 2024-09-14T23:58:09.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.488 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056731+0000", 2024-09-14T23:58:09.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056731+0000", 2024-09-14T23:58:09.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056732+0000", 2024-09-14T23:58:09.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2699999999999999e-07 2024-09-14T23:58:09.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056734+0000", 2024-09-14T23:58:09.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6110000000000001e-06 2024-09-14T23:58:09.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056742+0000", 2024-09-14T23:58:09.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1899999999999998e-06 2024-09-14T23:58:09.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056747+0000", 2024-09-14T23:58:09.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.322e-06 2024-09-14T23:58:09.490 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.490 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.490 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.490 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188726 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.490 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.061488+0000", 2024-09-14T23:58:09.490 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.032372604999999999, 2024-09-14T23:58:09.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.039001300000000003, 2024-09-14T23:58:09.491 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.491 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.491 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.491 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.491 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.491 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.491 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188726 2024-09-14T23:58:09.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.491 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061488+0000", 2024-09-14T23:58:09.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061488+0000", 2024-09-14T23:58:09.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061489+0000", 2024-09-14T23:58:09.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9000000000000002e-07 2024-09-14T23:58:09.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061491+0000", 2024-09-14T23:58:09.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2960000000000001e-06 2024-09-14T23:58:09.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061497+0000", 2024-09-14T23:58:09.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0599999999999996e-06 2024-09-14T23:58:09.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061502+0000", 2024-09-14T23:58:09.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9710000000000003e-06 2024-09-14T23:58:09.493 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.493 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.493 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.494 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189207 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.494 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.085959+0000", 2024-09-14T23:58:09.494 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0079017870000000004, 2024-09-14T23:58:09.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.014587055999999999, 2024-09-14T23:58:09.494 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.494 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.494 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.494 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.494 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.494 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.494 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189207 2024-09-14T23:58:09.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.495 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085959+0000", 2024-09-14T23:58:09.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085959+0000", 2024-09-14T23:58:09.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085960+0000", 2024-09-14T23:58:09.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9200000000000001e-07 2024-09-14T23:58:09.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085962+0000", 2024-09-14T23:58:09.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.441e-06 2024-09-14T23:58:09.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085971+0000", 2024-09-14T23:58:09.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8599999999999999e-06 2024-09-14T23:58:09.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085977+0000", 2024-09-14T23:58:09.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0120000000000002e-06 2024-09-14T23:58:09.497 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.497 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.497 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.497 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189239 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.497 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.088312+0000", 2024-09-14T23:58:09.497 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0055483579999999998, 2024-09-14T23:58:09.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.012290544, 2024-09-14T23:58:09.497 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.497 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.497 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.497 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.498 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.498 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.498 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189239 2024-09-14T23:58:09.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.498 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.498 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.498 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088312+0000", 2024-09-14T23:58:09.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.498 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.498 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088312+0000", 2024-09-14T23:58:09.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.499 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.499 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088313+0000", 2024-09-14T23:58:09.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6100000000000002e-07 2024-09-14T23:58:09.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.499 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.499 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088316+0000", 2024-09-14T23:58:09.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9050000000000001e-06 2024-09-14T23:58:09.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.499 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.499 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088322+0000", 2024-09-14T23:58:09.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3810000000000004e-06 2024-09-14T23:58:09.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088327+0000", 2024-09-14T23:58:09.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8330000000000001e-06 2024-09-14T23:58:09.500 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.500 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.500 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.500 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189367 3.1d 3.e507d9bd (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.501 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.094899+0000", 2024-09-14T23:58:09.501 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4294967295.9989619, 2024-09-14T23:58:09.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.005763181, 2024-09-14T23:58:09.501 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.501 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.501 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.501 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.501 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.501 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.501 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189367 2024-09-14T23:58:09.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.501 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094899+0000", 2024-09-14T23:58:09.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094899+0000", 2024-09-14T23:58:09.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094899+0000", 2024-09-14T23:58:09.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1600000000000002e-07 2024-09-14T23:58:09.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094901+0000", 2024-09-14T23:58:09.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1560000000000002e-06 2024-09-14T23:58:09.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094908+0000", 2024-09-14T23:58:09.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5710000000000003e-06 2024-09-14T23:58:09.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094913+0000", 2024-09-14T23:58:09.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.169e-06 2024-09-14T23:58:09.503 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.504 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.504 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.504 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188183 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.504 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.032380+0000", 2024-09-14T23:58:09.504 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.061480581999999999, 2024-09-14T23:58:09.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.068348994999999996, 2024-09-14T23:58:09.504 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.504 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.504 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.504 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.504 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.505 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.505 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188183 2024-09-14T23:58:09.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.505 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032380+0000", 2024-09-14T23:58:09.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032380+0000", 2024-09-14T23:58:09.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032381+0000", 2024-09-14T23:58:09.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1300000000000001e-07 2024-09-14T23:58:09.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032383+0000", 2024-09-14T23:58:09.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5179999999999999e-06 2024-09-14T23:58:09.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032392+0000", 2024-09-14T23:58:09.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5609999999999999e-06 2024-09-14T23:58:09.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.507 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.507 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032397+0000", 2024-09-14T23:58:09.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5720000000000002e-06 2024-09-14T23:58:09.507 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.507 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.507 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.507 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188215 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.507 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.034709+0000", 2024-09-14T23:58:09.508 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.059152060999999999, 2024-09-14T23:58:09.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066077545000000001, 2024-09-14T23:58:09.508 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.508 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.508 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.508 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.508 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.508 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.508 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188215 2024-09-14T23:58:09.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.508 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034709+0000", 2024-09-14T23:58:09.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034709+0000", 2024-09-14T23:58:09.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034709+0000", 2024-09-14T23:58:09.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0500000000000004e-07 2024-09-14T23:58:09.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034712+0000", 2024-09-14T23:58:09.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.61e-06 2024-09-14T23:58:09.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034719+0000", 2024-09-14T23:58:09.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5859999999999999e-06 2024-09-14T23:58:09.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034725+0000", 2024-09-14T23:58:09.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7479999999999999e-06 2024-09-14T23:58:09.510 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.511 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.511 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.511 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188695 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.511 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.056755+0000", 2024-09-14T23:58:09.511 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.037106205000000003, 2024-09-14T23:58:09.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044088971999999997, 2024-09-14T23:58:09.511 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.511 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.511 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.511 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.511 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.512 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.512 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188695 2024-09-14T23:58:09.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.512 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.512 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.512 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056755+0000", 2024-09-14T23:58:09.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.512 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.512 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056755+0000", 2024-09-14T23:58:09.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056755+0000", 2024-09-14T23:58:09.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.63e-07 2024-09-14T23:58:09.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056757+0000", 2024-09-14T23:58:09.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3369999999999998e-06 2024-09-14T23:58:09.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056764+0000", 2024-09-14T23:58:09.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.562e-06 2024-09-14T23:58:09.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056769+0000", 2024-09-14T23:58:09.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3170000000000001e-06 2024-09-14T23:58:09.514 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.514 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.514 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.514 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188727 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.514 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.061508+0000", 2024-09-14T23:58:09.514 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.032352888000000003, 2024-09-14T23:58:09.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.039391421000000003, 2024-09-14T23:58:09.515 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.515 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.515 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.515 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.515 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.515 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.515 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188727 2024-09-14T23:58:09.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.515 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061508+0000", 2024-09-14T23:58:09.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.516 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.516 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061508+0000", 2024-09-14T23:58:09.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.516 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.516 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061508+0000", 2024-09-14T23:58:09.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5999999999999999e-07 2024-09-14T23:58:09.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.516 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.516 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061520+0000", 2024-09-14T23:58:09.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1389999999999999e-05 2024-09-14T23:58:09.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.517 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061526+0000", 2024-09-14T23:58:09.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5599999999999999e-06 2024-09-14T23:58:09.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.517 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061532+0000", 2024-09-14T23:58:09.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0990000000000004e-06 2024-09-14T23:58:09.517 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.517 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.517 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.518 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189208 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.518 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.085985+0000", 2024-09-14T23:58:09.518 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0078756, 2024-09-14T23:58:09.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.014969341000000001, 2024-09-14T23:58:09.518 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.518 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.518 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.518 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.518 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.518 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.518 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189208 2024-09-14T23:58:09.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.518 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085985+0000", 2024-09-14T23:58:09.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085985+0000", 2024-09-14T23:58:09.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085986+0000", 2024-09-14T23:58:09.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9399999999999995e-07 2024-09-14T23:58:09.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085988+0000", 2024-09-14T23:58:09.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2259999999999999e-06 2024-09-14T23:58:09.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.085994+0000", 2024-09-14T23:58:09.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5570000000000001e-06 2024-09-14T23:58:09.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.521 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.521 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.086000+0000", 2024-09-14T23:58:09.521 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0300000000000001e-06 2024-09-14T23:58:09.521 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.521 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.521 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.521 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.521 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189240 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.521 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.088334+0000", 2024-09-14T23:58:09.521 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0055269500000000001, 2024-09-14T23:58:09.521 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.012675720999999999, 2024-09-14T23:58:09.521 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.521 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.522 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.522 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.522 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.522 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.522 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189240 2024-09-14T23:58:09.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.522 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.522 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088334+0000", 2024-09-14T23:58:09.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.522 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088334+0000", 2024-09-14T23:58:09.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088334+0000", 2024-09-14T23:58:09.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9999999999999998e-07 2024-09-14T23:58:09.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088336+0000", 2024-09-14T23:58:09.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1150000000000001e-06 2024-09-14T23:58:09.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088343+0000", 2024-09-14T23:58:09.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3249999999999996e-06 2024-09-14T23:58:09.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088347+0000", 2024-09-14T23:58:09.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.758e-06 2024-09-14T23:58:09.524 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.524 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.524 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.524 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189368 3.1d 3.e507d9bd (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.524 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.094919+0000", 2024-09-14T23:58:09.525 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4294967295.9989414, 2024-09-14T23:58:09.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0061459030000000003, 2024-09-14T23:58:09.525 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.525 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.525 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.525 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.525 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.525 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.525 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189368 2024-09-14T23:58:09.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.525 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.525 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.526 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094919+0000", 2024-09-14T23:58:09.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.526 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.526 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094919+0000", 2024-09-14T23:58:09.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.526 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.526 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094920+0000", 2024-09-14T23:58:09.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5900000000000003e-07 2024-09-14T23:58:09.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094922+0000", 2024-09-14T23:58:09.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.238e-06 2024-09-14T23:58:09.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094928+0000", 2024-09-14T23:58:09.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5400000000000001e-06 2024-09-14T23:58:09.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094933+0000", 2024-09-14T23:58:09.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9919999999999998e-06 2024-09-14T23:58:09.527 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.528 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.528 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.528 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188184 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.528 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.032404+0000", 2024-09-14T23:58:09.528 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.061456637000000001, 2024-09-14T23:58:09.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.068718874999999999, 2024-09-14T23:58:09.528 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.528 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.528 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.528 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.528 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.529 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.529 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188184 2024-09-14T23:58:09.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.529 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.529 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.529 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032404+0000", 2024-09-14T23:58:09.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.529 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.529 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032404+0000", 2024-09-14T23:58:09.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.530 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032405+0000", 2024-09-14T23:58:09.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4099999999999999e-07 2024-09-14T23:58:09.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.530 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032407+0000", 2024-09-14T23:58:09.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2089999999999999e-06 2024-09-14T23:58:09.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.530 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032415+0000", 2024-09-14T23:58:09.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.79e-06 2024-09-14T23:58:09.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032420+0000", 2024-09-14T23:58:09.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4489999999999998e-06 2024-09-14T23:58:09.531 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.531 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.531 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.531 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188216 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.531 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.034733+0000", 2024-09-14T23:58:09.531 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.059127657, 2024-09-14T23:58:09.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066446174999999996, 2024-09-14T23:58:09.532 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.532 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.532 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.532 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.532 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.532 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.532 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188216 2024-09-14T23:58:09.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.532 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.532 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034733+0000", 2024-09-14T23:58:09.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.533 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.533 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034733+0000", 2024-09-14T23:58:09.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.533 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.533 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034734+0000", 2024-09-14T23:58:09.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5599999999999995e-07 2024-09-14T23:58:09.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.533 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034736+0000", 2024-09-14T23:58:09.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4080000000000001e-06 2024-09-14T23:58:09.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034743+0000", 2024-09-14T23:58:09.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7209999999999997e-06 2024-09-14T23:58:09.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034748+0000", 2024-09-14T23:58:09.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5199999999999997e-06 2024-09-14T23:58:09.534 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.534 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.535 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.535 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188696 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.535 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.056777+0000", 2024-09-14T23:58:09.535 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.037083816999999998, 2024-09-14T23:58:09.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044460811000000003, 2024-09-14T23:58:09.535 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.535 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.535 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.535 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.535 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.535 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.535 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188696 2024-09-14T23:58:09.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.536 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056777+0000", 2024-09-14T23:58:09.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056777+0000", 2024-09-14T23:58:09.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056777+0000", 2024-09-14T23:58:09.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0299999999999999e-07 2024-09-14T23:58:09.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056780+0000", 2024-09-14T23:58:09.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.57e-06 2024-09-14T23:58:09.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056786+0000", 2024-09-14T23:58:09.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4189999999999999e-06 2024-09-14T23:58:09.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.538 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.538 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056792+0000", 2024-09-14T23:58:09.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3120000000000001e-06 2024-09-14T23:58:09.538 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.538 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.538 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.538 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188728 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.538 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.061541+0000", 2024-09-14T23:58:09.538 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.032319345999999999, 2024-09-14T23:58:09.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.039752783, 2024-09-14T23:58:09.538 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.538 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.538 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.539 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.539 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.539 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.539 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188728 2024-09-14T23:58:09.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.539 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.539 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.539 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061541+0000", 2024-09-14T23:58:09.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.539 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.539 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061541+0000", 2024-09-14T23:58:09.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061542+0000", 2024-09-14T23:58:09.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3700000000000001e-07 2024-09-14T23:58:09.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061544+0000", 2024-09-14T23:58:09.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2529999999999999e-06 2024-09-14T23:58:09.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061550+0000", 2024-09-14T23:58:09.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8669999999999999e-06 2024-09-14T23:58:09.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061555+0000", 2024-09-14T23:58:09.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.853e-06 2024-09-14T23:58:09.541 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.541 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.541 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.541 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189209 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.541 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.086007+0000", 2024-09-14T23:58:09.541 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.007853957, 2024-09-14T23:58:09.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015344353, 2024-09-14T23:58:09.542 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.542 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.542 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.542 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.542 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.542 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.542 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189209 2024-09-14T23:58:09.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.542 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.086007+0000", 2024-09-14T23:58:09.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.543 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.543 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.086007+0000", 2024-09-14T23:58:09.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.543 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.543 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.086007+0000", 2024-09-14T23:58:09.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4400000000000001e-07 2024-09-14T23:58:09.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.543 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.086010+0000", 2024-09-14T23:58:09.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.232e-06 2024-09-14T23:58:09.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.086016+0000", 2024-09-14T23:58:09.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6900000000000003e-06 2024-09-14T23:58:09.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.086021+0000", 2024-09-14T23:58:09.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1059999999999999e-06 2024-09-14T23:58:09.544 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.544 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.545 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.545 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189241 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.545 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.088354+0000", 2024-09-14T23:58:09.545 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.005506965, 2024-09-14T23:58:09.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.013052144999999999, 2024-09-14T23:58:09.545 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.545 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.545 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.545 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.545 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.545 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.545 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189241 2024-09-14T23:58:09.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.546 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088354+0000", 2024-09-14T23:58:09.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088354+0000", 2024-09-14T23:58:09.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088419+0000", 2024-09-14T23:58:09.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4943000000000005e-05 2024-09-14T23:58:09.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088422+0000", 2024-09-14T23:58:09.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.095e-06 2024-09-14T23:58:09.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088430+0000", 2024-09-14T23:58:09.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8970000000000008e-06 2024-09-14T23:58:09.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088435+0000", 2024-09-14T23:58:09.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6840000000000001e-06 2024-09-14T23:58:09.548 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.548 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.548 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.548 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.548 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189369 3.1d 3:bd9be0a7:::rbd_data.1118427720d0.0000000000000005:32 [read 1813504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.548 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.094941+0000", 2024-09-14T23:58:09.548 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4294967295.99892, 2024-09-14T23:58:09.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0065042800000000003, 2024-09-14T23:58:09.548 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.548 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.548 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:09.548 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.549 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.549 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.549 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189369 2024-09-14T23:58:09.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.549 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094941+0000", 2024-09-14T23:58:09.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094941+0000", 2024-09-14T23:58:09.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.550 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094941+0000", 2024-09-14T23:58:09.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9000000000000002e-07 2024-09-14T23:58:09.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.550 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094945+0000", 2024-09-14T23:58:09.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6040000000000001e-06 2024-09-14T23:58:09.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.550 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094952+0000", 2024-09-14T23:58:09.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.798e-06 2024-09-14T23:58:09.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094957+0000", 2024-09-14T23:58:09.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.592e-06 2024-09-14T23:58:09.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:09.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.101374+0000", 2024-09-14T23:58:09.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0064165480000000002 2024-09-14T23:58:09.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:09.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.101412+0000", 2024-09-14T23:58:09.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7540999999999999e-05 2024-09-14T23:58:09.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.552 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.552 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:09.552 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.101445+0000", 2024-09-14T23:58:09.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3806999999999997e-05 2024-09-14T23:58:09.552 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.552 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.552 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.552 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.552 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188185 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.552 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.032427+0000", 2024-09-14T23:58:09.552 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.061433356000000001, 2024-09-14T23:58:09.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.069114637000000007, 2024-09-14T23:58:09.553 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.553 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.553 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.553 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.553 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.553 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.553 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188185 2024-09-14T23:58:09.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.553 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.553 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.553 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032427+0000", 2024-09-14T23:58:09.553 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032427+0000", 2024-09-14T23:58:09.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032428+0000", 2024-09-14T23:58:09.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0499999999999999e-07 2024-09-14T23:58:09.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032430+0000", 2024-09-14T23:58:09.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1730000000000002e-06 2024-09-14T23:58:09.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032438+0000", 2024-09-14T23:58:09.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0970000000000006e-06 2024-09-14T23:58:09.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032443+0000", 2024-09-14T23:58:09.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3190000000000002e-06 2024-09-14T23:58:09.555 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.555 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.555 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.555 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188217 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.556 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.034756+0000", 2024-09-14T23:58:09.556 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.059105142999999999, 2024-09-14T23:58:09.556 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066845318000000001, 2024-09-14T23:58:09.556 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.556 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.556 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.556 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.556 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.556 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.556 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188217 2024-09-14T23:58:09.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.556 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.556 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.556 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.557 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034756+0000", 2024-09-14T23:58:09.557 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.557 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.557 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.557 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.557 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034756+0000", 2024-09-14T23:58:09.557 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.557 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.557 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.557 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.557 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034756+0000", 2024-09-14T23:58:09.557 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3000000000000001e-07 2024-09-14T23:58:09.557 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.558 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.558 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.558 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034770+0000", 2024-09-14T23:58:09.558 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3502e-05 2024-09-14T23:58:09.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.558 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.558 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.558 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034778+0000", 2024-09-14T23:58:09.558 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1429999999999998e-06 2024-09-14T23:58:09.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.558 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.558 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.558 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034784+0000", 2024-09-14T23:58:09.558 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0839999999999997e-06 2024-09-14T23:58:09.559 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.559 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.559 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.559 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188697 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.559 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.056798+0000", 2024-09-14T23:58:09.559 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.037062333000000003, 2024-09-14T23:58:09.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044860663000000002, 2024-09-14T23:58:09.559 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.559 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.559 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.559 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.559 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.560 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.560 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188697 2024-09-14T23:58:09.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.560 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056798+0000", 2024-09-14T23:58:09.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056798+0000", 2024-09-14T23:58:09.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.561 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056799+0000", 2024-09-14T23:58:09.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4900000000000001e-07 2024-09-14T23:58:09.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.561 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056801+0000", 2024-09-14T23:58:09.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2539999999999999e-06 2024-09-14T23:58:09.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.561 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056808+0000", 2024-09-14T23:58:09.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3500000000000002e-06 2024-09-14T23:58:09.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056812+0000", 2024-09-14T23:58:09.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7530000000000001e-06 2024-09-14T23:58:09.562 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.562 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.562 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.562 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188729 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.562 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.061562+0000", 2024-09-14T23:58:09.562 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.032298713, 2024-09-14T23:58:09.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.040159324000000003, 2024-09-14T23:58:09.563 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.563 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.563 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.563 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.563 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.563 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.563 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188729 2024-09-14T23:58:09.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.563 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.563 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.563 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061562+0000", 2024-09-14T23:58:09.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.563 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061562+0000", 2024-09-14T23:58:09.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061563+0000", 2024-09-14T23:58:09.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3500000000000002e-07 2024-09-14T23:58:09.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061565+0000", 2024-09-14T23:58:09.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.018e-06 2024-09-14T23:58:09.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.565 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061571+0000", 2024-09-14T23:58:09.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0360000000000003e-06 2024-09-14T23:58:09.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.565 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061577+0000", 2024-09-14T23:58:09.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2949999999999999e-06 2024-09-14T23:58:09.565 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.565 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.565 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.565 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189210 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.566 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.086030+0000", 2024-09-14T23:58:09.566 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0078305309999999999, 2024-09-14T23:58:09.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015748963000000001, 2024-09-14T23:58:09.566 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.566 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.566 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.566 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.566 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.566 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.566 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189210 2024-09-14T23:58:09.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.566 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.567 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.086030+0000", 2024-09-14T23:58:09.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.567 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.567 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.086030+0000", 2024-09-14T23:58:09.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.567 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.567 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.086031+0000", 2024-09-14T23:58:09.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4799999999999999e-07 2024-09-14T23:58:09.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.567 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.086033+0000", 2024-09-14T23:58:09.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.266e-06 2024-09-14T23:58:09.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.568 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.086040+0000", 2024-09-14T23:58:09.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4830000000000002e-06 2024-09-14T23:58:09.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.568 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.086046+0000", 2024-09-14T23:58:09.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3920000000000002e-06 2024-09-14T23:58:09.568 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.568 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.569 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.569 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189242 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.569 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.088444+0000", 2024-09-14T23:58:09.569 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0054163689999999999, 2024-09-14T23:58:09.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.013391297999999999, 2024-09-14T23:58:09.569 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.569 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.569 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.569 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.569 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.569 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.569 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189242 2024-09-14T23:58:09.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.570 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.570 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088444+0000", 2024-09-14T23:58:09.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.570 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088444+0000", 2024-09-14T23:58:09.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088445+0000", 2024-09-14T23:58:09.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6300000000000005e-07 2024-09-14T23:58:09.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088447+0000", 2024-09-14T23:58:09.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4660000000000002e-06 2024-09-14T23:58:09.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088454+0000", 2024-09-14T23:58:09.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6379999999999998e-06 2024-09-14T23:58:09.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.572 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088460+0000", 2024-09-14T23:58:09.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1530000000000002e-06 2024-09-14T23:58:09.572 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.572 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.572 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.572 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189370 3.1d 3:bd9be0a7:::rbd_data.1118427720d0.0000000000000005:32 [read 1800192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.572 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.094965+0000", 2024-09-14T23:58:09.572 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4294967295.9988961, 2024-09-14T23:58:09.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0067855729999999996, 2024-09-14T23:58:09.573 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.573 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.573 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:09.573 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.573 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.573 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.573 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189370 2024-09-14T23:58:09.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.573 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.573 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.573 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094965+0000", 2024-09-14T23:58:09.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.574 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.574 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094965+0000", 2024-09-14T23:58:09.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.574 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.574 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094965+0000", 2024-09-14T23:58:09.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5499999999999998e-07 2024-09-14T23:58:09.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.574 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.574 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094978+0000", 2024-09-14T23:58:09.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2972999999999999e-05 2024-09-14T23:58:09.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094988+0000", 2024-09-14T23:58:09.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3980000000000005e-06 2024-09-14T23:58:09.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.094995+0000", 2024-09-14T23:58:09.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3050000000000004e-06 2024-09-14T23:58:09.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:09.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.101593+0000", 2024-09-14T23:58:09.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0065980530000000004 2024-09-14T23:58:09.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:09.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.101680+0000", 2024-09-14T23:58:09.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7474999999999999e-05 2024-09-14T23:58:09.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:09.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.101750+0000", 2024-09-14T23:58:09.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9913999999999995e-05 2024-09-14T23:58:09.576 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.577 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.577 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.577 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188186 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.577 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.032451+0000", 2024-09-14T23:58:09.577 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.061409929000000002, 2024-09-14T23:58:09.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.069526457999999999, 2024-09-14T23:58:09.577 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.577 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.577 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.577 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.577 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.578 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.578 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188186 2024-09-14T23:58:09.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.578 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032451+0000", 2024-09-14T23:58:09.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032451+0000", 2024-09-14T23:58:09.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032451+0000", 2024-09-14T23:58:09.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1199999999999998e-07 2024-09-14T23:58:09.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032454+0000", 2024-09-14T23:58:09.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3800000000000001e-06 2024-09-14T23:58:09.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032462+0000", 2024-09-14T23:58:09.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2169999999999994e-06 2024-09-14T23:58:09.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032467+0000", 2024-09-14T23:58:09.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4979999999999997e-06 2024-09-14T23:58:09.580 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.580 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.580 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.580 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188218 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.580 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.034792+0000", 2024-09-14T23:58:09.580 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.059068505, 2024-09-14T23:58:09.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.067244765999999997, 2024-09-14T23:58:09.581 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.581 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.581 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.581 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.581 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.581 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.581 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188218 2024-09-14T23:58:09.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.581 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034792+0000", 2024-09-14T23:58:09.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034792+0000", 2024-09-14T23:58:09.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034793+0000", 2024-09-14T23:58:09.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5799999999999999e-07 2024-09-14T23:58:09.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034795+0000", 2024-09-14T23:58:09.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1909999999999999e-06 2024-09-14T23:58:09.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034802+0000", 2024-09-14T23:58:09.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1130000000000003e-06 2024-09-14T23:58:09.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034808+0000", 2024-09-14T23:58:09.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5600000000000001e-06 2024-09-14T23:58:09.583 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.583 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.583 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.584 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188698 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.584 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.056819+0000", 2024-09-14T23:58:09.584 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.037041519000000002, 2024-09-14T23:58:09.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.045275106000000002, 2024-09-14T23:58:09.584 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.584 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.584 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.584 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.584 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.584 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.584 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188698 2024-09-14T23:58:09.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.585 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056819+0000", 2024-09-14T23:58:09.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056819+0000", 2024-09-14T23:58:09.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056820+0000", 2024-09-14T23:58:09.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1100000000000001e-07 2024-09-14T23:58:09.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056822+0000", 2024-09-14T23:58:09.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.311e-06 2024-09-14T23:58:09.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056829+0000", 2024-09-14T23:58:09.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5540000000000003e-06 2024-09-14T23:58:09.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056834+0000", 2024-09-14T23:58:09.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1800000000000004e-06 2024-09-14T23:58:09.587 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.587 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.587 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.587 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188730 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.587 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.061584+0000", 2024-09-14T23:58:09.587 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.032277255999999997, 2024-09-14T23:58:09.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.040719732000000002, 2024-09-14T23:58:09.587 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.588 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.588 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.588 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.588 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.588 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.588 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188730 2024-09-14T23:58:09.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.588 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061584+0000", 2024-09-14T23:58:09.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061584+0000", 2024-09-14T23:58:09.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061584+0000", 2024-09-14T23:58:09.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.58e-07 2024-09-14T23:58:09.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061586+0000", 2024-09-14T23:58:09.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9379999999999999e-06 2024-09-14T23:58:09.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061593+0000", 2024-09-14T23:58:09.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.097e-06 2024-09-14T23:58:09.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061598+0000", 2024-09-14T23:58:09.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4530000000000001e-06 2024-09-14T23:58:09.590 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.590 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.590 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.591 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189211 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.591 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.086053+0000", 2024-09-14T23:58:09.591 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0078076170000000002, 2024-09-14T23:58:09.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.016308580999999999, 2024-09-14T23:58:09.591 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.591 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.591 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.591 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.591 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.591 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.591 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189211 2024-09-14T23:58:09.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.591 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.086053+0000", 2024-09-14T23:58:09.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.086053+0000", 2024-09-14T23:58:09.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.086054+0000", 2024-09-14T23:58:09.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8599999999999999e-07 2024-09-14T23:58:09.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.593 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.086060+0000", 2024-09-14T23:58:09.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7429999999999996e-06 2024-09-14T23:58:09.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.593 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.086067+0000", 2024-09-14T23:58:09.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9870000000000002e-06 2024-09-14T23:58:09.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.594 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.086073+0000", 2024-09-14T23:58:09.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4569999999999996e-06 2024-09-14T23:58:09.594 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.594 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.594 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.594 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189243 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.594 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.088469+0000", 2024-09-14T23:58:09.594 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0053922329999999997, 2024-09-14T23:58:09.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.013955303, 2024-09-14T23:58:09.594 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.594 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.595 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.595 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.595 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.595 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.595 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189243 2024-09-14T23:58:09.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.595 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.595 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088469+0000", 2024-09-14T23:58:09.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.596 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.088469+0000", 2024-09-14T23:58:09.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.596 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.596 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091003+0000", 2024-09-14T23:58:09.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0025341360000000002 2024-09-14T23:58:09.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.596 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.596 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091008+0000", 2024-09-14T23:58:09.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9439999999999996e-06 2024-09-14T23:58:09.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.596 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.597 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091016+0000", 2024-09-14T23:58:09.597 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2029999999999992e-06 2024-09-14T23:58:09.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.597 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.597 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091023+0000", 2024-09-14T23:58:09.597 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7569999999999998e-06 2024-09-14T23:58:09.597 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.597 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.597 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.597 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188187 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.597 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.032475+0000", 2024-09-14T23:58:09.598 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.061386188000000001, 2024-09-14T23:58:09.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.070008103000000002, 2024-09-14T23:58:09.598 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.598 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.598 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.598 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.598 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.598 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.598 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188187 2024-09-14T23:58:09.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.598 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.598 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.598 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032475+0000", 2024-09-14T23:58:09.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.599 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.599 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032475+0000", 2024-09-14T23:58:09.599 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.599 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.599 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032475+0000", 2024-09-14T23:58:09.599 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9400000000000001e-07 2024-09-14T23:58:09.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.599 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032477+0000", 2024-09-14T23:58:09.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2560000000000001e-06 2024-09-14T23:58:09.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.600 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032485+0000", 2024-09-14T23:58:09.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9140000000000008e-06 2024-09-14T23:58:09.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.600 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032490+0000", 2024-09-14T23:58:09.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2299999999999999e-06 2024-09-14T23:58:09.600 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.600 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.601 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.601 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188219 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.601 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.034815+0000", 2024-09-14T23:58:09.601 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.059046142000000003, 2024-09-14T23:58:09.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.067730871999999998, 2024-09-14T23:58:09.601 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.601 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.601 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.601 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.601 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.601 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.602 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188219 2024-09-14T23:58:09.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.602 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.602 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.602 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034815+0000", 2024-09-14T23:58:09.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.602 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.602 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034815+0000", 2024-09-14T23:58:09.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.603 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034815+0000", 2024-09-14T23:58:09.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0999999999999999e-07 2024-09-14T23:58:09.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.603 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034817+0000", 2024-09-14T23:58:09.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1399999999999998e-06 2024-09-14T23:58:09.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.603 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034824+0000", 2024-09-14T23:58:09.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4049999999999997e-06 2024-09-14T23:58:09.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034829+0000", 2024-09-14T23:58:09.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2830000000000001e-06 2024-09-14T23:58:09.604 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.604 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.604 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.604 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188699 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.604 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.056840+0000", 2024-09-14T23:58:09.604 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.037020503000000003, 2024-09-14T23:58:09.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.045763460999999998, 2024-09-14T23:58:09.604 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.605 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.605 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.605 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.605 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.605 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.605 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188699 2024-09-14T23:58:09.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.605 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056840+0000", 2024-09-14T23:58:09.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.606 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.606 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.056840+0000", 2024-09-14T23:58:09.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.606 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.606 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058673+0000", 2024-09-14T23:58:09.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018329150000000001 2024-09-14T23:58:09.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.606 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.606 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058678+0000", 2024-09-14T23:58:09.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6199999999999998e-06 2024-09-14T23:58:09.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.607 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.607 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058689+0000", 2024-09-14T23:58:09.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1541e-05 2024-09-14T23:58:09.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.607 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.607 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058697+0000", 2024-09-14T23:58:09.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9270000000000005e-06 2024-09-14T23:58:09.607 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.607 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.607 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.607 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188731 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.607 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.061605+0000", 2024-09-14T23:58:09.608 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.032255877000000002, 2024-09-14T23:58:09.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.041065563999999999, 2024-09-14T23:58:09.608 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.608 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.608 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.608 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.608 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.608 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.608 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188731 2024-09-14T23:58:09.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.608 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061605+0000", 2024-09-14T23:58:09.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061605+0000", 2024-09-14T23:58:09.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061605+0000", 2024-09-14T23:58:09.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.03e-07 2024-09-14T23:58:09.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061607+0000", 2024-09-14T23:58:09.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0870000000000001e-06 2024-09-14T23:58:09.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061614+0000", 2024-09-14T23:58:09.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1670000000000004e-06 2024-09-14T23:58:09.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061618+0000", 2024-09-14T23:58:09.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7129999999999996e-06 2024-09-14T23:58:09.610 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.610 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.610 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.611 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189212 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.611 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.086079+0000", 2024-09-14T23:58:09.611 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0077813159999999999, 2024-09-14T23:58:09.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.016651487, 2024-09-14T23:58:09.611 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.611 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.611 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.611 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.611 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.611 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.611 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189212 2024-09-14T23:58:09.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.612 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.612 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.612 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.086079+0000", 2024-09-14T23:58:09.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.612 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.612 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.086079+0000", 2024-09-14T23:58:09.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.612 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.086080+0000", 2024-09-14T23:58:09.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5300000000000004e-07 2024-09-14T23:58:09.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.086082+0000", 2024-09-14T23:58:09.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0049999999999999e-06 2024-09-14T23:58:09.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.086088+0000", 2024-09-14T23:58:09.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1410000000000002e-06 2024-09-14T23:58:09.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.614 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.614 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.086093+0000", 2024-09-14T23:58:09.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0340000000000004e-06 2024-09-14T23:58:09.614 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.614 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.614 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.614 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189244 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.614 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.091035+0000", 2024-09-14T23:58:09.614 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.002825564, 2024-09-14T23:58:09.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.011757452, 2024-09-14T23:58:09.614 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.615 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.615 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.615 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.615 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.615 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.615 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189244 2024-09-14T23:58:09.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.615 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.615 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.615 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091035+0000", 2024-09-14T23:58:09.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.615 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.616 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091035+0000", 2024-09-14T23:58:09.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.616 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091036+0000", 2024-09-14T23:58:09.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2600000000000002e-07 2024-09-14T23:58:09.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.616 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091039+0000", 2024-09-14T23:58:09.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9490000000000001e-06 2024-09-14T23:58:09.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091046+0000", 2024-09-14T23:58:09.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9979999999999997e-06 2024-09-14T23:58:09.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091051+0000", 2024-09-14T23:58:09.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6969999999999998e-06 2024-09-14T23:58:09.617 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.617 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.617 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.618 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188188 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.618 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.032498+0000", 2024-09-14T23:58:09.618 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.061363193000000003, 2024-09-14T23:58:09.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.070356108000000001, 2024-09-14T23:58:09.618 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.618 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.618 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.618 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.618 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.618 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.618 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188188 2024-09-14T23:58:09.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.619 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.619 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.619 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032498+0000", 2024-09-14T23:58:09.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.619 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.619 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032498+0000", 2024-09-14T23:58:09.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.619 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.619 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032498+0000", 2024-09-14T23:58:09.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5200000000000002e-07 2024-09-14T23:58:09.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032500+0000", 2024-09-14T23:58:09.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3640000000000002e-06 2024-09-14T23:58:09.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032507+0000", 2024-09-14T23:58:09.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8959999999999997e-06 2024-09-14T23:58:09.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.621 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032512+0000", 2024-09-14T23:58:09.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1229999999999999e-06 2024-09-14T23:58:09.621 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.621 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.621 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.621 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188220 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.621 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.034835+0000", 2024-09-14T23:58:09.621 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.059025382000000001, 2024-09-14T23:58:09.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.068081088999999997, 2024-09-14T23:58:09.621 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.621 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.621 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.622 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.622 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.622 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.622 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188220 2024-09-14T23:58:09.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.622 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034835+0000", 2024-09-14T23:58:09.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034835+0000", 2024-09-14T23:58:09.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034836+0000", 2024-09-14T23:58:09.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0400000000000002e-07 2024-09-14T23:58:09.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034838+0000", 2024-09-14T23:58:09.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9850000000000001e-06 2024-09-14T23:58:09.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.624 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034845+0000", 2024-09-14T23:58:09.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4189999999999996e-06 2024-09-14T23:58:09.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.624 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.624 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034851+0000", 2024-09-14T23:58:09.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7590000000000003e-06 2024-09-14T23:58:09.624 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.624 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.624 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.624 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188700 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.625 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.058715+0000", 2024-09-14T23:58:09.625 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.035145636000000001, 2024-09-14T23:58:09.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044259735000000001, 2024-09-14T23:58:09.625 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.625 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.625 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.625 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.625 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.625 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.625 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188700 2024-09-14T23:58:09.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.625 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058715+0000", 2024-09-14T23:58:09.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058715+0000", 2024-09-14T23:58:09.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058716+0000", 2024-09-14T23:58:09.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2e-07 2024-09-14T23:58:09.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058718+0000", 2024-09-14T23:58:09.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6170000000000001e-06 2024-09-14T23:58:09.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058726+0000", 2024-09-14T23:58:09.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8029999999999997e-06 2024-09-14T23:58:09.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058732+0000", 2024-09-14T23:58:09.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6370000000000004e-06 2024-09-14T23:58:09.628 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.628 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.628 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.628 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188732 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.628 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.061624+0000", 2024-09-14T23:58:09.628 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.032236315000000001, 2024-09-14T23:58:09.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.041411182999999997, 2024-09-14T23:58:09.628 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.628 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.628 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.628 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.629 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.629 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.629 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188732 2024-09-14T23:58:09.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.629 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.629 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.629 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061624+0000", 2024-09-14T23:58:09.629 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.629 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.629 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.061624+0000", 2024-09-14T23:58:09.629 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062673+0000", 2024-09-14T23:58:09.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001048469 2024-09-14T23:58:09.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062678+0000", 2024-09-14T23:58:09.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.921e-06 2024-09-14T23:58:09.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062688+0000", 2024-09-14T23:58:09.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0295000000000001e-05 2024-09-14T23:58:09.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.631 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.631 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062695+0000", 2024-09-14T23:58:09.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2130000000000002e-06 2024-09-14T23:58:09.631 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.631 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.631 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.631 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189213 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.631 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.086099+0000", 2024-09-14T23:58:09.631 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0077617160000000001, 2024-09-14T23:58:09.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.016994470000000001, 2024-09-14T23:58:09.632 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.632 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.632 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.632 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.632 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.632 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.632 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189213 2024-09-14T23:58:09.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.632 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.632 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.632 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.086099+0000", 2024-09-14T23:58:09.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.633 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.633 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.086099+0000", 2024-09-14T23:58:09.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.633 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.633 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087683+0000", 2024-09-14T23:58:09.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0015839420000000001 2024-09-14T23:58:09.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.633 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.633 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087688+0000", 2024-09-14T23:58:09.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6940000000000003e-06 2024-09-14T23:58:09.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.634 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087700+0000", 2024-09-14T23:58:09.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2099999999999999e-05 2024-09-14T23:58:09.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.634 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087708+0000", 2024-09-14T23:58:09.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9510000000000006e-06 2024-09-14T23:58:09.634 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.634 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.634 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.635 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189245 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.635 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.091059+0000", 2024-09-14T23:58:09.635 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0028015259999999999, 2024-09-14T23:58:09.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.012092729999999999, 2024-09-14T23:58:09.635 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.635 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.635 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.635 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.635 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.635 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.635 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189245 2024-09-14T23:58:09.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.635 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091059+0000", 2024-09-14T23:58:09.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091059+0000", 2024-09-14T23:58:09.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091060+0000", 2024-09-14T23:58:09.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9100000000000004e-07 2024-09-14T23:58:09.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.637 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.637 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091062+0000", 2024-09-14T23:58:09.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4310000000000001e-06 2024-09-14T23:58:09.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.637 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.637 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091069+0000", 2024-09-14T23:58:09.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3949999999999998e-06 2024-09-14T23:58:09.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.637 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.638 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091073+0000", 2024-09-14T23:58:09.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.843e-06 2024-09-14T23:58:09.638 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.638 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.638 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.638 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188189 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.638 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.032519+0000", 2024-09-14T23:58:09.638 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.061342002999999999, 2024-09-14T23:58:09.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.070698029999999995, 2024-09-14T23:58:09.638 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.638 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.638 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.638 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.638 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.639 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.639 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188189 2024-09-14T23:58:09.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.639 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032519+0000", 2024-09-14T23:58:09.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032519+0000", 2024-09-14T23:58:09.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032519+0000", 2024-09-14T23:58:09.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2500000000000001e-07 2024-09-14T23:58:09.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032521+0000", 2024-09-14T23:58:09.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0209999999999998e-06 2024-09-14T23:58:09.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032528+0000", 2024-09-14T23:58:09.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8430000000000004e-06 2024-09-14T23:58:09.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032533+0000", 2024-09-14T23:58:09.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0649999999999998e-06 2024-09-14T23:58:09.641 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.641 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.641 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.641 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188221 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.641 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.034858+0000", 2024-09-14T23:58:09.641 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.059002895999999999, 2024-09-14T23:58:09.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.068415727999999995, 2024-09-14T23:58:09.641 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.641 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.642 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.642 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.642 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.642 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.642 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188221 2024-09-14T23:58:09.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.642 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.642 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034858+0000", 2024-09-14T23:58:09.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.642 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034858+0000", 2024-09-14T23:58:09.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034858+0000", 2024-09-14T23:58:09.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.32e-07 2024-09-14T23:58:09.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034861+0000", 2024-09-14T23:58:09.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2000000000000001e-06 2024-09-14T23:58:09.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034867+0000", 2024-09-14T23:58:09.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6850000000000004e-06 2024-09-14T23:58:09.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034874+0000", 2024-09-14T23:58:09.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2599999999999999e-06 2024-09-14T23:58:09.644 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.644 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.644 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.644 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188701 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.644 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.058742+0000", 2024-09-14T23:58:09.644 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.035118973999999997, 2024-09-14T23:58:09.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044592645, 2024-09-14T23:58:09.645 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.645 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.645 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.645 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.645 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.645 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.645 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188701 2024-09-14T23:58:09.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.645 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058742+0000", 2024-09-14T23:58:09.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058742+0000", 2024-09-14T23:58:09.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058742+0000", 2024-09-14T23:58:09.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2399999999999999e-07 2024-09-14T23:58:09.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058745+0000", 2024-09-14T23:58:09.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3929999999999998e-06 2024-09-14T23:58:09.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058752+0000", 2024-09-14T23:58:09.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.187e-06 2024-09-14T23:58:09.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058757+0000", 2024-09-14T23:58:09.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6250000000000004e-06 2024-09-14T23:58:09.647 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.647 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.647 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.648 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188733 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.648 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.062708+0000", 2024-09-14T23:58:09.648 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.03115252, 2024-09-14T23:58:09.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.040682845000000002, 2024-09-14T23:58:09.648 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.648 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.648 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.648 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.648 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.648 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.648 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188733 2024-09-14T23:58:09.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.648 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062708+0000", 2024-09-14T23:58:09.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062708+0000", 2024-09-14T23:58:09.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062709+0000", 2024-09-14T23:58:09.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1799999999999995e-07 2024-09-14T23:58:09.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062712+0000", 2024-09-14T23:58:09.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6970000000000002e-06 2024-09-14T23:58:09.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062719+0000", 2024-09-14T23:58:09.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4900000000000003e-06 2024-09-14T23:58:09.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062724+0000", 2024-09-14T23:58:09.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3469999999999998e-06 2024-09-14T23:58:09.650 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.651 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.651 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.651 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189214 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.651 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.087722+0000", 2024-09-14T23:58:09.651 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0061392030000000002, 2024-09-14T23:58:09.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015730332999999999, 2024-09-14T23:58:09.651 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.651 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.651 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.651 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.651 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.651 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.652 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189214 2024-09-14T23:58:09.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.652 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.652 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087722+0000", 2024-09-14T23:58:09.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.652 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087722+0000", 2024-09-14T23:58:09.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087722+0000", 2024-09-14T23:58:09.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0999999999999998e-07 2024-09-14T23:58:09.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087725+0000", 2024-09-14T23:58:09.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.588e-06 2024-09-14T23:58:09.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087732+0000", 2024-09-14T23:58:09.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8940000000000004e-06 2024-09-14T23:58:09.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087737+0000", 2024-09-14T23:58:09.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2449999999999998e-06 2024-09-14T23:58:09.654 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.654 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.654 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.654 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189246 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.654 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.091082+0000", 2024-09-14T23:58:09.654 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0027789669999999998, 2024-09-14T23:58:09.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.012426641, 2024-09-14T23:58:09.654 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.654 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.655 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.655 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.655 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.655 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.655 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189246 2024-09-14T23:58:09.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.655 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.655 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.655 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091082+0000", 2024-09-14T23:58:09.655 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.655 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091082+0000", 2024-09-14T23:58:09.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091082+0000", 2024-09-14T23:58:09.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5200000000000002e-07 2024-09-14T23:58:09.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091090+0000", 2024-09-14T23:58:09.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8399999999999995e-06 2024-09-14T23:58:09.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091097+0000", 2024-09-14T23:58:09.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9369999999999998e-06 2024-09-14T23:58:09.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091102+0000", 2024-09-14T23:58:09.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1869999999999996e-06 2024-09-14T23:58:09.657 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.657 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.657 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.658 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188190 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.658 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.032539+0000", 2024-09-14T23:58:09.658 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.061321417000000003, 2024-09-14T23:58:09.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.071033414000000003, 2024-09-14T23:58:09.658 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.658 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.658 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.658 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.658 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.658 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.658 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188190 2024-09-14T23:58:09.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.658 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.659 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.659 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032539+0000", 2024-09-14T23:58:09.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.659 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.659 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032539+0000", 2024-09-14T23:58:09.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.659 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.659 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032540+0000", 2024-09-14T23:58:09.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8299999999999998e-07 2024-09-14T23:58:09.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032542+0000", 2024-09-14T23:58:09.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0109999999999999e-06 2024-09-14T23:58:09.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032550+0000", 2024-09-14T23:58:09.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1219999999999995e-06 2024-09-14T23:58:09.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032555+0000", 2024-09-14T23:58:09.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8729999999999998e-06 2024-09-14T23:58:09.661 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.661 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.661 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.661 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188222 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.661 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.034882+0000", 2024-09-14T23:58:09.661 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.058978445999999997, 2024-09-14T23:58:09.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.068747458999999997, 2024-09-14T23:58:09.661 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.661 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.662 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.662 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.662 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.662 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.662 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188222 2024-09-14T23:58:09.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.662 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034882+0000", 2024-09-14T23:58:09.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034882+0000", 2024-09-14T23:58:09.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.663 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.663 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034883+0000", 2024-09-14T23:58:09.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2100000000000002e-07 2024-09-14T23:58:09.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.663 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.663 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034885+0000", 2024-09-14T23:58:09.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5229999999999998e-06 2024-09-14T23:58:09.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.663 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.664 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034893+0000", 2024-09-14T23:58:09.664 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.328e-06 2024-09-14T23:58:09.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.664 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.664 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.664 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034898+0000", 2024-09-14T23:58:09.664 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7989999999999999e-06 2024-09-14T23:58:09.664 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.664 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.664 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.664 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.664 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188702 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.664 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.058766+0000", 2024-09-14T23:58:09.664 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.035094894000000001, 2024-09-14T23:58:09.665 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044929113, 2024-09-14T23:58:09.665 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.665 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.665 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.665 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.665 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.665 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.665 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188702 2024-09-14T23:58:09.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.665 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.665 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.665 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.665 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058766+0000", 2024-09-14T23:58:09.665 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058766+0000", 2024-09-14T23:58:09.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058766+0000", 2024-09-14T23:58:09.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1800000000000001e-07 2024-09-14T23:58:09.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058769+0000", 2024-09-14T23:58:09.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3070000000000001e-06 2024-09-14T23:58:09.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058775+0000", 2024-09-14T23:58:09.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.511e-06 2024-09-14T23:58:09.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058781+0000", 2024-09-14T23:58:09.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6540000000000004e-06 2024-09-14T23:58:09.667 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.667 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.667 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.668 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.668 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188734 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.668 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.062747+0000", 2024-09-14T23:58:09.668 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.031114096000000001, 2024-09-14T23:58:09.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.041001457999999998, 2024-09-14T23:58:09.668 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.668 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.668 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.668 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.668 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.668 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.668 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188734 2024-09-14T23:58:09.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.669 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.669 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.669 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062747+0000", 2024-09-14T23:58:09.669 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.669 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.669 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062747+0000", 2024-09-14T23:58:09.669 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.669 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.669 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062747+0000", 2024-09-14T23:58:09.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.96e-07 2024-09-14T23:58:09.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062749+0000", 2024-09-14T23:58:09.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.311e-06 2024-09-14T23:58:09.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062754+0000", 2024-09-14T23:58:09.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0030000000000002e-06 2024-09-14T23:58:09.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.671 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062758+0000", 2024-09-14T23:58:09.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.304e-06 2024-09-14T23:58:09.671 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.671 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.671 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.671 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.671 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189215 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.671 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.087745+0000", 2024-09-14T23:58:09.671 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0061159960000000003, 2024-09-14T23:58:09.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.016067624999999999, 2024-09-14T23:58:09.671 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.672 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.672 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.672 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.672 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.672 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.672 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189215 2024-09-14T23:58:09.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.672 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.672 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.672 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.672 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087745+0000", 2024-09-14T23:58:09.672 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.673 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.673 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.673 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087745+0000", 2024-09-14T23:58:09.673 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.673 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.673 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.673 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087745+0000", 2024-09-14T23:58:09.673 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5699999999999998e-07 2024-09-14T23:58:09.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.673 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.673 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.673 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087748+0000", 2024-09-14T23:58:09.673 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3539999999999998e-06 2024-09-14T23:58:09.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.674 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.674 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087754+0000", 2024-09-14T23:58:09.674 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.545e-06 2024-09-14T23:58:09.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.674 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.674 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087760+0000", 2024-09-14T23:58:09.674 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9370000000000001e-06 2024-09-14T23:58:09.674 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.674 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.674 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.675 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189247 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.675 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.091131+0000", 2024-09-14T23:58:09.675 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0027294950000000002, 2024-09-14T23:58:09.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.012738709000000001, 2024-09-14T23:58:09.675 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.675 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.675 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.675 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.675 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.675 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.675 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189247 2024-09-14T23:58:09.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.676 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.676 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.676 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091131+0000", 2024-09-14T23:58:09.676 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.676 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.676 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091131+0000", 2024-09-14T23:58:09.676 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.676 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.676 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091132+0000", 2024-09-14T23:58:09.677 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9599999999999999e-07 2024-09-14T23:58:09.677 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.677 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.677 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.677 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091135+0000", 2024-09-14T23:58:09.677 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8119999999999999e-06 2024-09-14T23:58:09.677 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.677 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.677 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.677 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091143+0000", 2024-09-14T23:58:09.677 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1000000000000004e-06 2024-09-14T23:58:09.677 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.677 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.678 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.678 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091149+0000", 2024-09-14T23:58:09.678 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9200000000000001e-06 2024-09-14T23:58:09.678 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.678 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.678 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.678 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.678 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.678 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188191 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.678 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.032562+0000", 2024-09-14T23:58:09.678 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.061299291999999998, 2024-09-14T23:58:09.678 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.071373846000000005, 2024-09-14T23:58:09.678 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.678 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.679 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.679 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.679 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.679 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.679 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188191 2024-09-14T23:58:09.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.679 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.679 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.679 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032562+0000", 2024-09-14T23:58:09.679 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.679 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.679 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032562+0000", 2024-09-14T23:58:09.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.680 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.680 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032562+0000", 2024-09-14T23:58:09.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1800000000000001e-07 2024-09-14T23:58:09.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.680 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.680 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032564+0000", 2024-09-14T23:58:09.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1849999999999998e-06 2024-09-14T23:58:09.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.680 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.681 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032571+0000", 2024-09-14T23:58:09.681 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7490000000000001e-06 2024-09-14T23:58:09.681 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.681 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.681 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.681 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032576+0000", 2024-09-14T23:58:09.681 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3410000000000002e-06 2024-09-14T23:58:09.681 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.681 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.681 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.681 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.681 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.681 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188223 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.681 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.034905+0000", 2024-09-14T23:58:09.682 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.058955525000000002, 2024-09-14T23:58:09.682 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.069087520999999999, 2024-09-14T23:58:09.682 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.682 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.682 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.682 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.682 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.682 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.682 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188223 2024-09-14T23:58:09.682 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.682 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.682 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.682 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.682 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034905+0000", 2024-09-14T23:58:09.683 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.683 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.683 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.683 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.683 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034905+0000", 2024-09-14T23:58:09.683 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.683 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.683 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.683 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.683 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034906+0000", 2024-09-14T23:58:09.683 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3599999999999999e-07 2024-09-14T23:58:09.683 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.683 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.683 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.683 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034908+0000", 2024-09-14T23:58:09.684 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9999999999999999e-06 2024-09-14T23:58:09.684 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.684 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.684 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.684 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034915+0000", 2024-09-14T23:58:09.684 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9199999999999998e-06 2024-09-14T23:58:09.684 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.684 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.684 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.684 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034920+0000", 2024-09-14T23:58:09.684 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5899999999999998e-06 2024-09-14T23:58:09.684 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.684 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.684 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.684 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.685 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.685 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188703 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.685 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.058788+0000", 2024-09-14T23:58:09.685 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.035072681000000001, 2024-09-14T23:58:09.685 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.045266455999999997, 2024-09-14T23:58:09.685 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.685 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.685 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.685 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.685 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.685 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.685 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188703 2024-09-14T23:58:09.685 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.686 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.686 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.686 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.686 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058788+0000", 2024-09-14T23:58:09.686 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.686 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.686 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.686 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.686 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058788+0000", 2024-09-14T23:58:09.686 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.686 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.686 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.686 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.686 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058788+0000", 2024-09-14T23:58:09.686 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6399999999999998e-07 2024-09-14T23:58:09.687 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.687 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.687 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.687 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058791+0000", 2024-09-14T23:58:09.687 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3970000000000001e-06 2024-09-14T23:58:09.687 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.687 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.687 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.687 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058797+0000", 2024-09-14T23:58:09.687 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9490000000000001e-06 2024-09-14T23:58:09.687 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.687 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.687 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.688 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058802+0000", 2024-09-14T23:58:09.688 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8930000000000004e-06 2024-09-14T23:58:09.688 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.688 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.688 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.688 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.688 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.688 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188735 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.688 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.062763+0000", 2024-09-14T23:58:09.688 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.031097633, 2024-09-14T23:58:09.688 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.041349258999999999, 2024-09-14T23:58:09.688 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.688 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.688 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.689 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.689 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.689 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.689 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188735 2024-09-14T23:58:09.689 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.689 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.689 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.689 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.689 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062763+0000", 2024-09-14T23:58:09.689 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.689 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.689 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.689 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.689 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062763+0000", 2024-09-14T23:58:09.690 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.690 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.690 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.690 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.690 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062763+0000", 2024-09-14T23:58:09.690 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1399999999999998e-07 2024-09-14T23:58:09.690 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.690 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.690 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.690 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062765+0000", 2024-09-14T23:58:09.690 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.311e-06 2024-09-14T23:58:09.690 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.690 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.690 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.691 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062768+0000", 2024-09-14T23:58:09.691 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2559999999999998e-06 2024-09-14T23:58:09.691 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.691 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.691 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.691 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062771+0000", 2024-09-14T23:58:09.691 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6139999999999999e-06 2024-09-14T23:58:09.691 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.691 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.691 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.691 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.691 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.691 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189216 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.691 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.087769+0000", 2024-09-14T23:58:09.691 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0060921940000000004, 2024-09-14T23:58:09.692 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.016399342000000001, 2024-09-14T23:58:09.692 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.692 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.692 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.692 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.692 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.692 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.692 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189216 2024-09-14T23:58:09.692 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.692 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.692 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.692 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.692 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087769+0000", 2024-09-14T23:58:09.693 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.693 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.693 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.693 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.693 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087769+0000", 2024-09-14T23:58:09.693 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.693 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.693 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.693 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.693 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087769+0000", 2024-09-14T23:58:09.693 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3000000000000001e-07 2024-09-14T23:58:09.693 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.693 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.694 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.694 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087772+0000", 2024-09-14T23:58:09.694 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.525e-06 2024-09-14T23:58:09.694 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.694 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.694 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.694 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087778+0000", 2024-09-14T23:58:09.694 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.725e-06 2024-09-14T23:58:09.694 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.694 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.694 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.694 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087785+0000", 2024-09-14T23:58:09.694 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4489999999999996e-06 2024-09-14T23:58:09.694 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.695 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.695 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.695 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.695 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.695 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189248 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.695 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.091158+0000", 2024-09-14T23:58:09.695 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.002703067, 2024-09-14T23:58:09.695 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.013070394000000001, 2024-09-14T23:58:09.695 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.695 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.695 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.695 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.695 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.695 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.696 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189248 2024-09-14T23:58:09.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.696 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.696 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.696 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.696 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091158+0000", 2024-09-14T23:58:09.696 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.696 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.696 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.696 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091158+0000", 2024-09-14T23:58:09.696 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.696 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.697 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.697 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091158+0000", 2024-09-14T23:58:09.697 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8699999999999995e-07 2024-09-14T23:58:09.697 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.697 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.697 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.697 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091161+0000", 2024-09-14T23:58:09.697 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2979999999999999e-06 2024-09-14T23:58:09.697 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.697 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.697 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.697 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091167+0000", 2024-09-14T23:58:09.697 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8859999999999998e-06 2024-09-14T23:58:09.698 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.698 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.698 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.698 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091173+0000", 2024-09-14T23:58:09.698 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3499999999999996e-06 2024-09-14T23:58:09.698 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.698 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.698 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.698 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.698 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.698 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188192 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.698 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.032582+0000", 2024-09-14T23:58:09.698 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.061278375000000003, 2024-09-14T23:58:09.698 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.071706308999999996, 2024-09-14T23:58:09.699 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.699 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.699 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.699 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.699 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.699 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.699 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188192 2024-09-14T23:58:09.699 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.699 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.699 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.699 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.699 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032582+0000", 2024-09-14T23:58:09.699 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.700 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.700 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.700 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.700 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032582+0000", 2024-09-14T23:58:09.700 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.700 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.700 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.700 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.700 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032583+0000", 2024-09-14T23:58:09.700 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7599999999999998e-07 2024-09-14T23:58:09.700 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.700 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.700 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.701 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032585+0000", 2024-09-14T23:58:09.701 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0590000000000001e-06 2024-09-14T23:58:09.701 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.701 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.701 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.701 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032591+0000", 2024-09-14T23:58:09.701 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5549999999999999e-06 2024-09-14T23:58:09.701 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.701 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.701 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.701 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032597+0000", 2024-09-14T23:58:09.701 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2650000000000004e-06 2024-09-14T23:58:09.701 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.701 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.702 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.702 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.702 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.702 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188224 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.702 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.034927+0000", 2024-09-14T23:58:09.702 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.058933884999999998, 2024-09-14T23:58:09.702 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.069421894999999997, 2024-09-14T23:58:09.702 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.702 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.702 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.702 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.702 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.702 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.702 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188224 2024-09-14T23:58:09.703 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.703 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.703 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.703 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.703 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034927+0000", 2024-09-14T23:58:09.703 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.703 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.703 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.703 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.703 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.034927+0000", 2024-09-14T23:58:09.703 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.703 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.703 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.703 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.704 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.036675+0000", 2024-09-14T23:58:09.704 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001748461 2024-09-14T23:58:09.704 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.704 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.704 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.704 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.036681+0000", 2024-09-14T23:58:09.704 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6779999999999997e-06 2024-09-14T23:58:09.704 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.704 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.704 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.704 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.036693+0000", 2024-09-14T23:58:09.704 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1669e-05 2024-09-14T23:58:09.704 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.704 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.704 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.705 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.036702+0000", 2024-09-14T23:58:09.705 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2010000000000005e-06 2024-09-14T23:58:09.705 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.705 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.705 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.705 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.705 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.705 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188704 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.705 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.058815+0000", 2024-09-14T23:58:09.705 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.035045856, 2024-09-14T23:58:09.705 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.045589446999999998, 2024-09-14T23:58:09.705 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.705 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.705 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.705 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.706 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.706 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.706 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188704 2024-09-14T23:58:09.706 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.706 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.706 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.706 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.706 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058815+0000", 2024-09-14T23:58:09.706 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.706 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.706 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.706 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.706 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058815+0000", 2024-09-14T23:58:09.706 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.706 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.707 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.707 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.707 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058816+0000", 2024-09-14T23:58:09.707 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.82e-07 2024-09-14T23:58:09.707 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.707 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.707 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.707 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058818+0000", 2024-09-14T23:58:09.707 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7130000000000001e-06 2024-09-14T23:58:09.707 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.707 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.707 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.707 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058825+0000", 2024-09-14T23:58:09.707 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5629999999999997e-06 2024-09-14T23:58:09.707 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.707 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.708 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.708 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058831+0000", 2024-09-14T23:58:09.708 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7039999999999999e-06 2024-09-14T23:58:09.708 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.708 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.708 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.708 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.708 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.708 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188736 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.708 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.062774+0000", 2024-09-14T23:58:09.708 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.031086949999999999, 2024-09-14T23:58:09.708 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.041689298999999999, 2024-09-14T23:58:09.708 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.709 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.709 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.709 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.709 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.709 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.709 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188736 2024-09-14T23:58:09.709 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.709 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.709 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.709 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.709 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062774+0000", 2024-09-14T23:58:09.709 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.709 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.710 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.710 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.710 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062774+0000", 2024-09-14T23:58:09.710 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.710 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.710 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.710 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.710 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062774+0000", 2024-09-14T23:58:09.710 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.04e-07 2024-09-14T23:58:09.710 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.710 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.710 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.710 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062775+0000", 2024-09-14T23:58:09.710 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1379999999999999e-06 2024-09-14T23:58:09.710 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.711 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.711 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.711 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062778+0000", 2024-09-14T23:58:09.711 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1200000000000002e-06 2024-09-14T23:58:09.711 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.711 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.711 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.711 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062781+0000", 2024-09-14T23:58:09.711 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4710000000000001e-06 2024-09-14T23:58:09.711 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.711 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.711 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.711 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.711 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.711 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189217 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.712 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.087792+0000", 2024-09-14T23:58:09.712 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0060683200000000003, 2024-09-14T23:58:09.712 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.016726622999999999, 2024-09-14T23:58:09.712 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.712 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.712 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.712 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.712 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.712 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.712 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189217 2024-09-14T23:58:09.712 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.712 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.712 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.712 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.713 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087792+0000", 2024-09-14T23:58:09.713 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.713 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.713 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.713 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087792+0000", 2024-09-14T23:58:09.713 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.713 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.713 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.713 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087793+0000", 2024-09-14T23:58:09.713 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6699999999999999e-07 2024-09-14T23:58:09.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.713 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.714 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.714 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087796+0000", 2024-09-14T23:58:09.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9830000000000001e-06 2024-09-14T23:58:09.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.714 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.714 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.714 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087801+0000", 2024-09-14T23:58:09.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4840000000000003e-06 2024-09-14T23:58:09.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.714 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.714 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.714 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087806+0000", 2024-09-14T23:58:09.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6249999999999998e-06 2024-09-14T23:58:09.714 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.714 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.714 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.715 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.715 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.715 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189249 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.715 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.091180+0000", 2024-09-14T23:58:09.715 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0026810570000000001, 2024-09-14T23:58:09.715 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.013398116999999999, 2024-09-14T23:58:09.715 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.715 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.715 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.715 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.715 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.715 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.715 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189249 2024-09-14T23:58:09.715 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.715 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.716 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.716 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.716 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091180+0000", 2024-09-14T23:58:09.716 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.716 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.716 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.716 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.716 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091180+0000", 2024-09-14T23:58:09.716 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.716 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.716 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.716 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.716 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091180+0000", 2024-09-14T23:58:09.716 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.58e-07 2024-09-14T23:58:09.717 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.717 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.717 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.717 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091183+0000", 2024-09-14T23:58:09.717 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2879999999999999e-06 2024-09-14T23:58:09.717 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.717 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.717 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.717 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091189+0000", 2024-09-14T23:58:09.717 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5440000000000004e-06 2024-09-14T23:58:09.717 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.717 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.717 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.717 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091194+0000", 2024-09-14T23:58:09.718 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8189999999999999e-06 2024-09-14T23:58:09.718 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.718 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.718 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.718 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.718 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.718 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188193 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.718 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.032603+0000", 2024-09-14T23:58:09.718 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.061257921, 2024-09-14T23:58:09.718 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.072034808000000006, 2024-09-14T23:58:09.718 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.718 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.718 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.718 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.719 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.719 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.719 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188193 2024-09-14T23:58:09.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.719 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.719 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.719 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.719 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032603+0000", 2024-09-14T23:58:09.719 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.719 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.719 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.719 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032603+0000", 2024-09-14T23:58:09.719 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.720 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.720 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.720 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032603+0000", 2024-09-14T23:58:09.720 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9799999999999999e-07 2024-09-14T23:58:09.720 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.720 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.720 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.720 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032610+0000", 2024-09-14T23:58:09.720 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7510000000000002e-06 2024-09-14T23:58:09.720 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.720 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.720 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.720 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032617+0000", 2024-09-14T23:58:09.720 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2899999999999997e-06 2024-09-14T23:58:09.721 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.721 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.721 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.721 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032623+0000", 2024-09-14T23:58:09.721 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2390000000000002e-06 2024-09-14T23:58:09.721 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.721 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.721 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.721 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.721 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.721 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188225 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.721 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.036716+0000", 2024-09-14T23:58:09.721 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.057144835999999997, 2024-09-14T23:58:09.721 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.067988357999999999, 2024-09-14T23:58:09.722 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.722 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.722 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.722 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.722 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.722 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.722 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188225 2024-09-14T23:58:09.722 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.722 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.722 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.722 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.722 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.036716+0000", 2024-09-14T23:58:09.722 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.722 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.723 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.723 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.723 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.036716+0000", 2024-09-14T23:58:09.723 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.723 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.723 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.723 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.723 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.036716+0000", 2024-09-14T23:58:09.723 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8500000000000002e-07 2024-09-14T23:58:09.723 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.723 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.723 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.723 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.036720+0000", 2024-09-14T23:58:09.723 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0960000000000001e-06 2024-09-14T23:58:09.724 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.724 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.724 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.724 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.036744+0000", 2024-09-14T23:58:09.724 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4437999999999999e-05 2024-09-14T23:58:09.724 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.724 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.724 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.724 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.036752+0000", 2024-09-14T23:58:09.724 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8809999999999996e-06 2024-09-14T23:58:09.724 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.724 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.724 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.724 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.725 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.725 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188705 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.725 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.058840+0000", 2024-09-14T23:58:09.725 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.035020867999999997, 2024-09-14T23:58:09.725 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.045922338, 2024-09-14T23:58:09.725 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.725 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.725 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.725 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.725 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.725 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.725 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188705 2024-09-14T23:58:09.725 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.726 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.726 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.726 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.726 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058840+0000", 2024-09-14T23:58:09.726 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.726 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.726 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.726 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.726 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058840+0000", 2024-09-14T23:58:09.726 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.726 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.726 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.726 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.726 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058840+0000", 2024-09-14T23:58:09.727 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.01e-07 2024-09-14T23:58:09.727 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.727 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.727 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.727 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058843+0000", 2024-09-14T23:58:09.727 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.351e-06 2024-09-14T23:58:09.727 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.727 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.727 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.727 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058849+0000", 2024-09-14T23:58:09.727 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7100000000000001e-06 2024-09-14T23:58:09.727 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.727 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.728 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.728 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058854+0000", 2024-09-14T23:58:09.728 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0499999999999999e-06 2024-09-14T23:58:09.728 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.728 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.728 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.728 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.728 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.728 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188737 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.728 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.062784+0000", 2024-09-14T23:58:09.728 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.031077119, 2024-09-14T23:58:09.728 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.042037599000000002, 2024-09-14T23:58:09.728 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.729 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.729 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.729 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.729 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.729 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.729 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188737 2024-09-14T23:58:09.729 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.729 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.729 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.729 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.729 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062784+0000", 2024-09-14T23:58:09.729 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.729 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.730 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.730 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.730 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062784+0000", 2024-09-14T23:58:09.730 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.730 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.730 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.730 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.730 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062784+0000", 2024-09-14T23:58:09.730 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1199999999999999e-07 2024-09-14T23:58:09.730 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.730 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.730 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.730 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062785+0000", 2024-09-14T23:58:09.730 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1489999999999999e-06 2024-09-14T23:58:09.730 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.731 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.731 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.731 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062790+0000", 2024-09-14T23:58:09.731 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5369999999999999e-06 2024-09-14T23:58:09.731 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.731 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.731 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.731 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062792+0000", 2024-09-14T23:58:09.731 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6919999999999998e-06 2024-09-14T23:58:09.731 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.731 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.731 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.731 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.731 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.731 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189218 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.732 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.087814+0000", 2024-09-14T23:58:09.732 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0060464730000000001, 2024-09-14T23:58:09.732 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.017064564000000001, 2024-09-14T23:58:09.732 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.732 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.732 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.732 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.732 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.732 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.732 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189218 2024-09-14T23:58:09.732 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.732 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.732 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.732 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.733 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087814+0000", 2024-09-14T23:58:09.733 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.733 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.733 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.733 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.733 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087814+0000", 2024-09-14T23:58:09.733 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.733 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.733 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.733 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.733 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087815+0000", 2024-09-14T23:58:09.733 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3099999999999998e-07 2024-09-14T23:58:09.733 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.733 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.734 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.734 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087817+0000", 2024-09-14T23:58:09.734 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3750000000000001e-06 2024-09-14T23:58:09.734 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.734 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.734 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.734 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087823+0000", 2024-09-14T23:58:09.734 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2550000000000003e-06 2024-09-14T23:58:09.734 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.734 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.734 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.734 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087828+0000", 2024-09-14T23:58:09.734 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8709999999999996e-06 2024-09-14T23:58:09.735 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.735 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.735 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.735 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.735 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.735 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189250 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.735 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.091201+0000", 2024-09-14T23:58:09.735 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0026598699999999999, 2024-09-14T23:58:09.735 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.013741216000000001, 2024-09-14T23:58:09.735 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.735 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.735 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.735 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.735 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.735 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.736 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189250 2024-09-14T23:58:09.736 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.736 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.736 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.736 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.736 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091201+0000", 2024-09-14T23:58:09.736 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.736 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.736 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.736 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.736 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091201+0000", 2024-09-14T23:58:09.736 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.736 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.736 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.737 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.737 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091202+0000", 2024-09-14T23:58:09.737 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3e-07 2024-09-14T23:58:09.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.737 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.737 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.737 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091205+0000", 2024-09-14T23:58:09.737 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5549999999999998e-06 2024-09-14T23:58:09.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.737 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.737 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.737 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091211+0000", 2024-09-14T23:58:09.737 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2040000000000002e-06 2024-09-14T23:58:09.738 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.738 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.738 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.738 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091217+0000", 2024-09-14T23:58:09.738 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2299999999999999e-06 2024-09-14T23:58:09.738 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.738 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.738 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.738 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.738 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.738 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188194 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.738 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.032861+0000", 2024-09-14T23:58:09.738 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.061000233000000001, 2024-09-14T23:58:09.739 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.072142987000000006, 2024-09-14T23:58:09.739 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.739 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.739 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.739 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.739 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.739 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.739 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188194 2024-09-14T23:58:09.739 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.739 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.739 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.739 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.739 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032861+0000", 2024-09-14T23:58:09.739 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.740 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.740 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.740 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.740 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032861+0000", 2024-09-14T23:58:09.740 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.740 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.740 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.740 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.740 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032861+0000", 2024-09-14T23:58:09.740 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1999999999999999e-07 2024-09-14T23:58:09.740 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.740 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.740 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.740 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032865+0000", 2024-09-14T23:58:09.740 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5489999999999998e-06 2024-09-14T23:58:09.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.741 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.741 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.741 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032873+0000", 2024-09-14T23:58:09.741 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5259999999999993e-06 2024-09-14T23:58:09.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.741 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.741 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.741 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032880+0000", 2024-09-14T23:58:09.741 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0119999999999999e-06 2024-09-14T23:58:09.741 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.741 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.741 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.742 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.742 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188226 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.742 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.036762+0000", 2024-09-14T23:58:09.742 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.057098397000000002, 2024-09-14T23:58:09.742 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.068286019000000003, 2024-09-14T23:58:09.742 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.742 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.742 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.742 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.742 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.742 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.742 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188226 2024-09-14T23:58:09.742 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.742 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.743 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.743 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.743 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.036762+0000", 2024-09-14T23:58:09.743 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.743 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.743 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.743 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.743 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.036762+0000", 2024-09-14T23:58:09.743 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.743 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.743 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.743 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.743 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.036763+0000", 2024-09-14T23:58:09.743 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4300000000000003e-07 2024-09-14T23:58:09.743 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.744 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.744 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.744 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.036766+0000", 2024-09-14T23:58:09.744 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.118e-06 2024-09-14T23:58:09.744 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.744 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.744 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.744 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.036776+0000", 2024-09-14T23:58:09.744 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6209999999999999e-06 2024-09-14T23:58:09.744 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.744 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.744 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.744 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.036783+0000", 2024-09-14T23:58:09.744 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2969999999999998e-06 2024-09-14T23:58:09.744 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.745 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.745 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.745 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.745 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188706 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.745 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.058862+0000", 2024-09-14T23:58:09.745 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.034998757999999998, 2024-09-14T23:58:09.745 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046244893000000002, 2024-09-14T23:58:09.745 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.745 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.745 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.745 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.745 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.746 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.746 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188706 2024-09-14T23:58:09.746 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.746 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.746 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.746 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.746 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058862+0000", 2024-09-14T23:58:09.746 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.746 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.746 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.746 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.746 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058862+0000", 2024-09-14T23:58:09.746 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.746 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.746 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.747 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.747 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058863+0000", 2024-09-14T23:58:09.747 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8800000000000002e-07 2024-09-14T23:58:09.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.747 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.747 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.747 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058865+0000", 2024-09-14T23:58:09.747 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5210000000000001e-06 2024-09-14T23:58:09.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.747 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.747 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.747 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058872+0000", 2024-09-14T23:58:09.747 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1069999999999999e-06 2024-09-14T23:58:09.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.747 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.748 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.748 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058877+0000", 2024-09-14T23:58:09.748 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9819999999999999e-06 2024-09-14T23:58:09.748 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.748 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.748 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.748 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.748 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.748 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188739 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.748 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.062795+0000", 2024-09-14T23:58:09.748 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.031065461999999999, 2024-09-14T23:58:09.748 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.042351465999999997, 2024-09-14T23:58:09.748 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.748 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.749 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.749 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.749 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.749 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.749 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188739 2024-09-14T23:58:09.749 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.749 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.749 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.749 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.749 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062795+0000", 2024-09-14T23:58:09.749 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.749 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.749 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.750 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.750 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062795+0000", 2024-09-14T23:58:09.750 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.750 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.750 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.750 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062796+0000", 2024-09-14T23:58:09.750 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0599999999999999e-07 2024-09-14T23:58:09.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.750 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.750 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.750 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062797+0000", 2024-09-14T23:58:09.750 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1090000000000001e-06 2024-09-14T23:58:09.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.750 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.751 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.751 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062800+0000", 2024-09-14T23:58:09.751 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1269999999999999e-06 2024-09-14T23:58:09.751 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.751 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.751 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.751 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062802+0000", 2024-09-14T23:58:09.751 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5019999999999999e-06 2024-09-14T23:58:09.751 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.751 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.751 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.751 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.751 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.751 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189219 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.752 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.087835+0000", 2024-09-14T23:58:09.752 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.006025692, 2024-09-14T23:58:09.752 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.017369938000000001, 2024-09-14T23:58:09.752 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.752 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.752 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.752 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.752 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.752 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.752 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189219 2024-09-14T23:58:09.752 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.752 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.752 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.752 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.752 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087835+0000", 2024-09-14T23:58:09.753 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.753 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.753 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.753 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.753 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087835+0000", 2024-09-14T23:58:09.753 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.753 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.753 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.753 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.753 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087836+0000", 2024-09-14T23:58:09.753 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1699999999999999e-07 2024-09-14T23:58:09.753 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.753 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.753 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.754 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087838+0000", 2024-09-14T23:58:09.754 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1840000000000002e-06 2024-09-14T23:58:09.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.754 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.754 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.754 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087844+0000", 2024-09-14T23:58:09.754 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3160000000000002e-06 2024-09-14T23:58:09.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.754 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.754 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.754 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087849+0000", 2024-09-14T23:58:09.754 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1229999999999999e-06 2024-09-14T23:58:09.754 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.754 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.754 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.755 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189251 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.755 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.091224+0000", 2024-09-14T23:58:09.755 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0026368020000000002, 2024-09-14T23:58:09.755 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.014038547, 2024-09-14T23:58:09.755 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.755 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.755 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.755 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.755 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.755 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.755 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189251 2024-09-14T23:58:09.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.756 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.756 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.756 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.756 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091224+0000", 2024-09-14T23:58:09.756 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.756 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.756 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.756 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.756 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091224+0000", 2024-09-14T23:58:09.756 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.756 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.756 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.756 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.757 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091224+0000", 2024-09-14T23:58:09.757 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4299999999999998e-07 2024-09-14T23:58:09.757 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.757 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.757 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.757 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091227+0000", 2024-09-14T23:58:09.757 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3319999999999999e-06 2024-09-14T23:58:09.757 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.757 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.757 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.757 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091233+0000", 2024-09-14T23:58:09.757 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9750000000000004e-06 2024-09-14T23:58:09.757 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.757 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.758 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.758 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091239+0000", 2024-09-14T23:58:09.758 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.032e-06 2024-09-14T23:58:09.758 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.758 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.758 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.758 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.758 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.758 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188195 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.758 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.032891+0000", 2024-09-14T23:58:09.758 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.060969996999999998, 2024-09-14T23:58:09.758 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.072436232000000003, 2024-09-14T23:58:09.758 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.758 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.759 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.759 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.759 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.759 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.759 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188195 2024-09-14T23:58:09.759 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.759 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.759 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.759 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.759 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032891+0000", 2024-09-14T23:58:09.759 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.759 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.759 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.759 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.760 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032891+0000", 2024-09-14T23:58:09.760 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.760 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.760 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.760 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032891+0000", 2024-09-14T23:58:09.760 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3200000000000005e-07 2024-09-14T23:58:09.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.760 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.760 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.760 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032894+0000", 2024-09-14T23:58:09.760 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.977e-06 2024-09-14T23:58:09.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.760 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.761 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.761 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032911+0000", 2024-09-14T23:58:09.761 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6976000000000001e-05 2024-09-14T23:58:09.761 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.761 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.761 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.761 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032918+0000", 2024-09-14T23:58:09.761 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1899999999999998e-06 2024-09-14T23:58:09.761 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.761 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.761 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.761 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.761 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.762 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188227 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.762 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.036791+0000", 2024-09-14T23:58:09.762 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.057069623999999999, 2024-09-14T23:58:09.762 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.068587508000000005, 2024-09-14T23:58:09.762 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.762 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.762 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.762 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.762 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.762 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.762 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188227 2024-09-14T23:58:09.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.762 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.762 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.762 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.763 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.036791+0000", 2024-09-14T23:58:09.763 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.763 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.763 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.036791+0000", 2024-09-14T23:58:09.763 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.763 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.763 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.036792+0000", 2024-09-14T23:58:09.763 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0900000000000002e-07 2024-09-14T23:58:09.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.763 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.763 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.036795+0000", 2024-09-14T23:58:09.764 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9629999999999998e-06 2024-09-14T23:58:09.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.764 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.764 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.764 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.036802+0000", 2024-09-14T23:58:09.764 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1439999999999997e-06 2024-09-14T23:58:09.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.764 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.764 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.764 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.036808+0000", 2024-09-14T23:58:09.764 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2790000000000004e-06 2024-09-14T23:58:09.764 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.764 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.764 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.765 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.765 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188707 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.765 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.058884+0000", 2024-09-14T23:58:09.765 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.034976341000000001, 2024-09-14T23:58:09.765 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046549285000000003, 2024-09-14T23:58:09.765 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.765 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.765 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.765 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.765 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.765 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.765 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188707 2024-09-14T23:58:09.765 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.765 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.766 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.766 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.766 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058884+0000", 2024-09-14T23:58:09.766 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.766 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.766 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.766 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.766 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058884+0000", 2024-09-14T23:58:09.766 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.766 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.766 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.766 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.766 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058885+0000", 2024-09-14T23:58:09.766 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.01e-07 2024-09-14T23:58:09.766 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.766 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.767 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.767 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058888+0000", 2024-09-14T23:58:09.767 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7819999999999998e-06 2024-09-14T23:58:09.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.767 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.767 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.767 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058894+0000", 2024-09-14T23:58:09.767 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6069999999999996e-06 2024-09-14T23:58:09.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.767 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.767 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.767 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058900+0000", 2024-09-14T23:58:09.767 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.259e-06 2024-09-14T23:58:09.767 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.767 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.768 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.768 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.768 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188741 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.768 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.062805+0000", 2024-09-14T23:58:09.768 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.031055625999999999, 2024-09-14T23:58:09.768 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.042684667000000003, 2024-09-14T23:58:09.768 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.768 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.768 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.768 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.768 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.768 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.768 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188741 2024-09-14T23:58:09.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.769 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.769 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.769 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.769 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062805+0000", 2024-09-14T23:58:09.769 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.769 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.769 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.769 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.769 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062805+0000", 2024-09-14T23:58:09.769 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.769 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.769 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.769 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.769 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.064673+0000", 2024-09-14T23:58:09.769 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00186739 2024-09-14T23:58:09.770 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.770 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.770 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.770 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.064682+0000", 2024-09-14T23:58:09.770 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5699999999999999e-06 2024-09-14T23:58:09.770 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.770 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.770 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.770 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.064694+0000", 2024-09-14T23:58:09.770 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2004999999999999e-05 2024-09-14T23:58:09.770 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.770 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.770 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.771 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.064703+0000", 2024-09-14T23:58:09.771 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4419999999999998e-06 2024-09-14T23:58:09.771 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.771 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.771 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.771 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.771 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.771 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189220 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.771 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.087856+0000", 2024-09-14T23:58:09.771 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.006004634, 2024-09-14T23:58:09.771 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.017690712000000001, 2024-09-14T23:58:09.771 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.771 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.771 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.772 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.772 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.772 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.772 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189220 2024-09-14T23:58:09.772 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.772 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.772 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.772 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.772 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087856+0000", 2024-09-14T23:58:09.772 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.772 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.772 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.772 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.772 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087856+0000", 2024-09-14T23:58:09.773 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.773 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.773 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.773 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087857+0000", 2024-09-14T23:58:09.773 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8699999999999995e-07 2024-09-14T23:58:09.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.773 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.773 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.773 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087859+0000", 2024-09-14T23:58:09.773 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3460000000000001e-06 2024-09-14T23:58:09.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.773 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.774 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.774 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087866+0000", 2024-09-14T23:58:09.774 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7610000000000002e-06 2024-09-14T23:58:09.774 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.774 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.774 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.774 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087871+0000", 2024-09-14T23:58:09.774 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9960000000000001e-06 2024-09-14T23:58:09.774 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.774 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.774 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.774 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.774 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.774 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189252 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.775 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.091246+0000", 2024-09-14T23:58:09.775 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0026150800000000001, 2024-09-14T23:58:09.775 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.014359946, 2024-09-14T23:58:09.775 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.775 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.775 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.775 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.775 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.775 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.775 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189252 2024-09-14T23:58:09.775 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.775 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.775 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.776 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.776 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091246+0000", 2024-09-14T23:58:09.776 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.776 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.776 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.776 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091246+0000", 2024-09-14T23:58:09.776 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.776 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.776 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.776 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091246+0000", 2024-09-14T23:58:09.776 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0800000000000004e-07 2024-09-14T23:58:09.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.777 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.777 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.777 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091248+0000", 2024-09-14T23:58:09.777 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9829999999999999e-06 2024-09-14T23:58:09.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.777 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.777 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.777 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091255+0000", 2024-09-14T23:58:09.777 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5320000000000003e-06 2024-09-14T23:58:09.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.777 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.777 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.778 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091262+0000", 2024-09-14T23:58:09.778 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4939999999999997e-06 2024-09-14T23:58:09.778 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.778 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.778 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.778 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.778 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188196 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.778 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.032927+0000", 2024-09-14T23:58:09.778 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.060933534999999997, 2024-09-14T23:58:09.778 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.072750839999999997, 2024-09-14T23:58:09.778 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.778 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.778 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.779 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.779 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.779 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.779 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188196 2024-09-14T23:58:09.779 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.779 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.779 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.779 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.779 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032927+0000", 2024-09-14T23:58:09.779 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.779 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.779 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.779 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.779 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032927+0000", 2024-09-14T23:58:09.779 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.780 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.780 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.780 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.780 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032928+0000", 2024-09-14T23:58:09.780 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2699999999999999e-07 2024-09-14T23:58:09.780 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.780 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.780 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.780 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032931+0000", 2024-09-14T23:58:09.780 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7599999999999998e-06 2024-09-14T23:58:09.780 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.780 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.780 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.781 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032938+0000", 2024-09-14T23:58:09.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.486e-06 2024-09-14T23:58:09.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.781 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.781 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032943+0000", 2024-09-14T23:58:09.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4369999999999998e-06 2024-09-14T23:58:09.781 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.781 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.781 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.781 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188228 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.782 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.036816+0000", 2024-09-14T23:58:09.782 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.057045137000000003, 2024-09-14T23:58:09.782 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.068921028999999995, 2024-09-14T23:58:09.782 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.782 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.782 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.782 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.782 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.782 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.782 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188228 2024-09-14T23:58:09.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.782 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.782 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.783 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.036816+0000", 2024-09-14T23:58:09.783 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.783 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.783 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.783 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.036816+0000", 2024-09-14T23:58:09.783 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.783 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.783 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.783 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.038674+0000", 2024-09-14T23:58:09.783 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001858508 2024-09-14T23:58:09.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.783 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.784 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.784 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.038680+0000", 2024-09-14T23:58:09.784 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.212e-06 2024-09-14T23:58:09.784 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.784 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.784 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.784 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.038691+0000", 2024-09-14T23:58:09.784 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0961000000000001e-05 2024-09-14T23:58:09.784 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.784 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.784 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.784 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.038699+0000", 2024-09-14T23:58:09.785 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7200000000000006e-06 2024-09-14T23:58:09.785 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.785 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.785 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.785 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.785 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.785 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188708 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.785 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.058907+0000", 2024-09-14T23:58:09.785 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.034953819999999997, 2024-09-14T23:58:09.785 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046890790000000002, 2024-09-14T23:58:09.785 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.785 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.785 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.785 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.786 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.786 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.786 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188708 2024-09-14T23:58:09.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.786 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058907+0000", 2024-09-14T23:58:09.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058907+0000", 2024-09-14T23:58:09.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.787 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.787 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058907+0000", 2024-09-14T23:58:09.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6199999999999998e-07 2024-09-14T23:58:09.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.787 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.787 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058910+0000", 2024-09-14T23:58:09.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2979999999999999e-06 2024-09-14T23:58:09.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.787 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.788 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058916+0000", 2024-09-14T23:58:09.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3980000000000004e-06 2024-09-14T23:58:09.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.788 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.788 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058921+0000", 2024-09-14T23:58:09.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2310000000000004e-06 2024-09-14T23:58:09.788 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.788 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.788 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.788 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188743 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.788 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.064716+0000", 2024-09-14T23:58:09.789 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.029145010999999998, 2024-09-14T23:58:09.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.041140585, 2024-09-14T23:58:09.789 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.789 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.789 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.789 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.789 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.789 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.789 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188743 2024-09-14T23:58:09.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.789 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.790 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.064716+0000", 2024-09-14T23:58:09.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.790 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.064716+0000", 2024-09-14T23:58:09.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.790 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.064716+0000", 2024-09-14T23:58:09.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7299999999999996e-07 2024-09-14T23:58:09.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.791 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.791 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.064719+0000", 2024-09-14T23:58:09.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0249999999999998e-06 2024-09-14T23:58:09.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.791 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.791 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.064727+0000", 2024-09-14T23:58:09.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1840000000000002e-06 2024-09-14T23:58:09.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.791 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.791 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.064732+0000", 2024-09-14T23:58:09.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6060000000000002e-06 2024-09-14T23:58:09.792 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.792 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.792 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.792 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189221 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.792 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.087879+0000", 2024-09-14T23:58:09.792 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0059822670000000003, 2024-09-14T23:58:09.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.018038332000000001, 2024-09-14T23:58:09.792 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.792 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.792 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.792 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.793 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.793 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.793 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189221 2024-09-14T23:58:09.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.793 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.793 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.793 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087879+0000", 2024-09-14T23:58:09.793 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.793 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.793 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087879+0000", 2024-09-14T23:58:09.793 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.794 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.794 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087879+0000", 2024-09-14T23:58:09.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.27e-07 2024-09-14T23:58:09.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.794 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.794 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087881+0000", 2024-09-14T23:58:09.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.03e-06 2024-09-14T23:58:09.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.794 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087888+0000", 2024-09-14T23:58:09.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7730000000000002e-06 2024-09-14T23:58:09.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087893+0000", 2024-09-14T23:58:09.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3249999999999998e-06 2024-09-14T23:58:09.795 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.795 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.795 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.795 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189253 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.795 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.091270+0000", 2024-09-14T23:58:09.796 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0025911770000000001, 2024-09-14T23:58:09.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.014706332000000001, 2024-09-14T23:58:09.796 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.796 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.796 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.796 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.796 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.796 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.796 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189253 2024-09-14T23:58:09.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.796 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.796 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.797 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091270+0000", 2024-09-14T23:58:09.797 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.797 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.797 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091270+0000", 2024-09-14T23:58:09.797 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.797 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.797 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091270+0000", 2024-09-14T23:58:09.797 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3799999999999997e-07 2024-09-14T23:58:09.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.798 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.798 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091272+0000", 2024-09-14T23:58:09.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2369999999999999e-06 2024-09-14T23:58:09.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.798 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.798 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091279+0000", 2024-09-14T23:58:09.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3640000000000004e-06 2024-09-14T23:58:09.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.798 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.798 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091284+0000", 2024-09-14T23:58:09.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0470000000000001e-06 2024-09-14T23:58:09.798 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.799 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.799 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.799 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.799 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188197 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.799 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.032951+0000", 2024-09-14T23:58:09.799 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.060910227999999997, 2024-09-14T23:58:09.799 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.073085786, 2024-09-14T23:58:09.799 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.799 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.799 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.799 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.799 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.800 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.800 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188197 2024-09-14T23:58:09.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.800 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032951+0000", 2024-09-14T23:58:09.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032951+0000", 2024-09-14T23:58:09.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.801 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.801 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.801 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.801 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032951+0000", 2024-09-14T23:58:09.801 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2399999999999999e-07 2024-09-14T23:58:09.801 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.801 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.801 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.801 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032954+0000", 2024-09-14T23:58:09.801 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5349999999999999e-06 2024-09-14T23:58:09.801 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.801 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.801 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.802 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032960+0000", 2024-09-14T23:58:09.802 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6150000000000002e-06 2024-09-14T23:58:09.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.802 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.802 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.802 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032965+0000", 2024-09-14T23:58:09.802 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2390000000000002e-06 2024-09-14T23:58:09.802 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.802 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.802 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.802 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.802 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188230 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.803 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.038713+0000", 2024-09-14T23:58:09.803 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.055147942999999998, 2024-09-14T23:58:09.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.067384889000000003, 2024-09-14T23:58:09.803 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.803 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.803 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.803 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.803 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.803 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.803 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188230 2024-09-14T23:58:09.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.803 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.803 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.038713+0000", 2024-09-14T23:58:09.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.038713+0000", 2024-09-14T23:58:09.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.038713+0000", 2024-09-14T23:58:09.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2e-07 2024-09-14T23:58:09.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.805 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.805 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.038729+0000", 2024-09-14T23:58:09.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5494000000000001e-05 2024-09-14T23:58:09.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.805 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.805 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.805 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.038737+0000", 2024-09-14T23:58:09.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0819999999999999e-06 2024-09-14T23:58:09.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.805 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.805 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.805 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.038743+0000", 2024-09-14T23:58:09.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0140000000000004e-06 2024-09-14T23:58:09.806 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.806 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.806 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.806 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.806 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188709 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.806 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.058929+0000", 2024-09-14T23:58:09.806 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.034932161000000003, 2024-09-14T23:58:09.806 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047226258, 2024-09-14T23:58:09.806 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.806 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.806 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.806 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.806 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.806 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.807 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188709 2024-09-14T23:58:09.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.807 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.807 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.807 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058929+0000", 2024-09-14T23:58:09.807 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.807 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.807 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058929+0000", 2024-09-14T23:58:09.807 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.808 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.808 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058929+0000", 2024-09-14T23:58:09.808 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2500000000000001e-07 2024-09-14T23:58:09.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.808 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.808 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.808 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058932+0000", 2024-09-14T23:58:09.808 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.638e-06 2024-09-14T23:58:09.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.808 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.808 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.808 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058939+0000", 2024-09-14T23:58:09.808 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4320000000000002e-06 2024-09-14T23:58:09.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.809 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.809 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058944+0000", 2024-09-14T23:58:09.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1290000000000004e-06 2024-09-14T23:58:09.809 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.809 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.809 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.809 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188745 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.809 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.064741+0000", 2024-09-14T23:58:09.809 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.02911969, 2024-09-14T23:58:09.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.041474311, 2024-09-14T23:58:09.810 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.810 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.810 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.810 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.810 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.810 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.810 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188745 2024-09-14T23:58:09.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.810 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.810 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.810 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.810 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.064741+0000", 2024-09-14T23:58:09.810 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.811 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.811 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.811 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.064741+0000", 2024-09-14T23:58:09.811 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.811 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.811 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.811 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.811 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.064742+0000", 2024-09-14T23:58:09.811 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2899999999999999e-07 2024-09-14T23:58:09.811 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.811 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.811 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.811 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.064744+0000", 2024-09-14T23:58:09.811 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6340000000000001e-06 2024-09-14T23:58:09.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.812 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.812 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.812 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.064751+0000", 2024-09-14T23:58:09.812 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9040000000000003e-06 2024-09-14T23:58:09.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.812 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.812 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.812 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.064756+0000", 2024-09-14T23:58:09.812 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0710000000000002e-06 2024-09-14T23:58:09.812 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.812 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.812 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.813 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.813 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189222 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.813 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.087900+0000", 2024-09-14T23:58:09.813 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0059609229999999999, 2024-09-14T23:58:09.813 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.018370428000000001, 2024-09-14T23:58:09.813 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.813 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.813 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.813 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.813 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.813 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.813 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189222 2024-09-14T23:58:09.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.814 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.814 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.814 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.814 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087900+0000", 2024-09-14T23:58:09.814 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.814 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.814 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.814 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087900+0000", 2024-09-14T23:58:09.814 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.814 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.815 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.815 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087900+0000", 2024-09-14T23:58:09.815 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8800000000000003e-07 2024-09-14T23:58:09.815 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.815 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.815 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.815 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087903+0000", 2024-09-14T23:58:09.815 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.357e-06 2024-09-14T23:58:09.815 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.815 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.815 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.815 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087909+0000", 2024-09-14T23:58:09.815 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3799999999999999e-06 2024-09-14T23:58:09.815 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.815 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.816 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.816 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087914+0000", 2024-09-14T23:58:09.816 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9300000000000002e-06 2024-09-14T23:58:09.816 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.816 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.816 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.816 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.816 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.816 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189254 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.816 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.091291+0000", 2024-09-14T23:58:09.816 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.002570236, 2024-09-14T23:58:09.816 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015037874999999999, 2024-09-14T23:58:09.816 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.817 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.817 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.817 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.817 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.817 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.817 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189254 2024-09-14T23:58:09.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.817 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.817 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.817 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.817 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091291+0000", 2024-09-14T23:58:09.817 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.817 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.817 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.818 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091291+0000", 2024-09-14T23:58:09.818 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.818 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.818 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.818 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.818 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091291+0000", 2024-09-14T23:58:09.818 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7399999999999998e-07 2024-09-14T23:58:09.818 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.818 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.818 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.818 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091293+0000", 2024-09-14T23:58:09.818 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3309999999999998e-06 2024-09-14T23:58:09.818 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.818 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.819 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.819 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091302+0000", 2024-09-14T23:58:09.819 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6810000000000004e-06 2024-09-14T23:58:09.819 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.819 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.819 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.819 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091307+0000", 2024-09-14T23:58:09.819 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9899999999999997e-06 2024-09-14T23:58:09.819 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.819 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.819 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.819 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.819 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.820 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188198 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.820 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.032972+0000", 2024-09-14T23:58:09.820 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.060888599000000002, 2024-09-14T23:58:09.820 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.073417404000000006, 2024-09-14T23:58:09.820 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.820 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.820 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.820 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.820 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.820 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.820 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188198 2024-09-14T23:58:09.820 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.820 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.820 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.821 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.821 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032972+0000", 2024-09-14T23:58:09.821 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.821 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.821 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.821 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.821 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032972+0000", 2024-09-14T23:58:09.821 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.821 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.821 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.821 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.821 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032973+0000", 2024-09-14T23:58:09.821 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2399999999999999e-07 2024-09-14T23:58:09.821 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.822 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.822 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.822 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032975+0000", 2024-09-14T23:58:09.822 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0880000000000002e-06 2024-09-14T23:58:09.822 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.822 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.822 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.822 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032981+0000", 2024-09-14T23:58:09.822 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6749999999999996e-06 2024-09-14T23:58:09.822 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.822 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.822 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.822 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.032987+0000", 2024-09-14T23:58:09.823 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3159999999999996e-06 2024-09-14T23:58:09.823 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.823 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.823 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.823 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.823 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.823 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188232 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.823 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.038752+0000", 2024-09-14T23:58:09.823 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.055108467000000001, 2024-09-14T23:58:09.823 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.067700634999999995, 2024-09-14T23:58:09.823 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.823 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.823 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.823 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.824 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.824 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.824 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188232 2024-09-14T23:58:09.824 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.824 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.824 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.824 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.824 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.038752+0000", 2024-09-14T23:58:09.824 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.824 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.824 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.824 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.824 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.038752+0000", 2024-09-14T23:58:09.825 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.825 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.825 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.825 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.038753+0000", 2024-09-14T23:58:09.825 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0900000000000002e-07 2024-09-14T23:58:09.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.825 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.825 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.825 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.038756+0000", 2024-09-14T23:58:09.825 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.723e-06 2024-09-14T23:58:09.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.825 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.826 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.826 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.038793+0000", 2024-09-14T23:58:09.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6974999999999998e-05 2024-09-14T23:58:09.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.826 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.826 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.826 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.038801+0000", 2024-09-14T23:58:09.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8650000000000007e-06 2024-09-14T23:58:09.826 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.826 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.826 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.826 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.826 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188710 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.827 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.058950+0000", 2024-09-14T23:58:09.827 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.034910373000000001, 2024-09-14T23:58:09.827 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047560926000000003, 2024-09-14T23:58:09.827 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.827 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.827 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.827 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.827 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.827 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.827 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188710 2024-09-14T23:58:09.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.827 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.827 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.827 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.827 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058950+0000", 2024-09-14T23:58:09.828 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.828 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.828 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.828 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.828 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058950+0000", 2024-09-14T23:58:09.828 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.828 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.828 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.828 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.828 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058951+0000", 2024-09-14T23:58:09.828 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4999999999999998e-07 2024-09-14T23:58:09.828 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.828 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.828 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.829 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058953+0000", 2024-09-14T23:58:09.829 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1600000000000001e-06 2024-09-14T23:58:09.829 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.829 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.829 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.829 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058959+0000", 2024-09-14T23:58:09.829 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9100000000000002e-06 2024-09-14T23:58:09.829 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.829 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.829 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.829 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.058963+0000", 2024-09-14T23:58:09.829 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6070000000000001e-06 2024-09-14T23:58:09.829 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.829 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.830 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.830 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.830 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188747 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:09.830 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.064764+0000", 2024-09-14T23:58:09.830 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.029097214, 2024-09-14T23:58:09.830 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.041808832999999997, 2024-09-14T23:58:09.830 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.830 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.830 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.830 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.830 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.830 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.830 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188747 2024-09-14T23:58:09.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.831 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.831 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.831 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.831 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.064764+0000", 2024-09-14T23:58:09.831 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.831 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.831 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.831 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.064764+0000", 2024-09-14T23:58:09.831 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.831 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.831 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.831 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.064764+0000", 2024-09-14T23:58:09.831 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0699999999999997e-07 2024-09-14T23:58:09.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.832 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.832 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.832 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.064766+0000", 2024-09-14T23:58:09.832 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.193e-06 2024-09-14T23:58:09.832 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.832 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.832 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.832 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.064773+0000", 2024-09-14T23:58:09.832 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3709999999999996e-06 2024-09-14T23:58:09.832 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.832 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.832 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.832 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.064778+0000", 2024-09-14T23:58:09.832 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.293e-06 2024-09-14T23:58:09.832 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.833 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.833 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.833 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.833 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.833 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189223 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.833 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.087932+0000", 2024-09-14T23:58:09.833 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.005929213, 2024-09-14T23:58:09.833 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.018699343, 2024-09-14T23:58:09.833 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.833 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.833 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.833 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.833 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.833 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.834 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189223 2024-09-14T23:58:09.834 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.834 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.834 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.834 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.834 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087932+0000", 2024-09-14T23:58:09.834 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.834 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.834 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.834 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.834 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087932+0000", 2024-09-14T23:58:09.834 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.834 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.834 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.834 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.835 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087932+0000", 2024-09-14T23:58:09.835 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6000000000000003e-07 2024-09-14T23:58:09.835 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.835 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.835 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.835 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087935+0000", 2024-09-14T23:58:09.835 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4600000000000002e-06 2024-09-14T23:58:09.835 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.835 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.835 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.835 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087943+0000", 2024-09-14T23:58:09.835 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0779999999999996e-06 2024-09-14T23:58:09.835 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.835 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.835 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.836 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.087949+0000", 2024-09-14T23:58:09.836 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7520000000000002e-06 2024-09-14T23:58:09.836 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.836 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.836 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.836 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.836 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.836 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:189255 3.f 3.5b6716ef (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T23:58:09.836 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:09.091314+0000", 2024-09-14T23:58:09.836 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0025472519999999998, 2024-09-14T23:58:09.836 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015383203999999999, 2024-09-14T23:58:09.836 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:09.836 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:09.836 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-14T23:58:09.837 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:09.837 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:09.837 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:09.837 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189255 2024-09-14T23:58:09.837 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.837 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:09.837 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.837 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:09.837 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091314+0000", 2024-09-14T23:58:09.837 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.837 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.837 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.837 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:09.838 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091314+0000", 2024-09-14T23:58:09.838 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:09.838 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.838 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.838 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:09.838 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091314+0000", 2024-09-14T23:58:09.838 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4600000000000004e-07 2024-09-14T23:58:09.838 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.838 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.838 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:09.838 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091317+0000", 2024-09-14T23:58:09.838 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3240000000000001e-06 2024-09-14T23:58:09.838 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.838 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.839 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:09.839 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091323+0000", 2024-09-14T23:58:09.839 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3929999999999996e-06 2024-09-14T23:58:09.839 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:09.839 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:09.839 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:09.839 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.091328+0000", 2024-09-14T23:58:09.839 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1819999999999997e-06 2024-09-14T23:58:09.839 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.839 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:09.839 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.839 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:09.839 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-14T23:58:09.839 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 211 2024-09-14T23:58:09.840 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:09.840 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T23:58:09.841 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:58:09.841 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:58:09.842 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:58:09.842 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:58:09.843 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T23:58:09.843 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:58:09.843 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:58:09.844 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:58:09.844 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:58:09.844 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:58:09.845 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:58:09.845 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:58:09.845 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:58:09.845 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:58:09.846 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:58:09.846 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:09.839+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:09.942 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:09.940+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:10.003 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:10.003 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:10.015 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T23:58:10.042 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:10.041+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:10.143 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:58:10.141+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:58:10.183 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:10.184 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:10.195 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T23:58:10.279 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:58:10.277+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:58:10.345 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:10.343+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:10.376 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:10.376 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:10.387 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T23:58:10.446 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:10.444+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:10.543 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:10.543 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:10.546 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:10.545+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:10.552 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T23:58:10.647 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:10.645+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:10.666 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:10.666 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:10.675 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T23:58:10.759 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:10.757+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:10.826 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:10.826 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:10.835 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T23:58:10.876 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:10.874+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:10.950 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:10.948+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:10.983 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:10.983 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:10.992 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T23:58:11.076 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:11.074+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:11.137 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is set 2024-09-14T23:58:11.147 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:11.147 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:11.151 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:11.150+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:11.160 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T23:58:11.253 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:11.251+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:11.321 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:11.321 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:11.330 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T23:58:11.354 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:11.353+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:11.456 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:11.453+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:11.500 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:11.501 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:11.512 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T23:58:11.557 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:11.555+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:11.658 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:11.656+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:11.668 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:11.668 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:11.677 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T23:58:11.759 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:58:11.758+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:58:11.843 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:11.843 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:11.848 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T23:58:11.857 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T23:58:11.860 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:11.859+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:11.962 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:11.959+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:12.027 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:58:12.027 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:58:12.047 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:58:12.048 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:58:12.062 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:12.060+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:12.062 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:12.062 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:12.062 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:58:12.062 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:58:12.062 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:12.065 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:58:12.065 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:58:12.071 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T23:58:12.083 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:58:12.084 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:58:12.101 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:58:12.101 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:58:12.118 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:58:12.118 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:58:12.137 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:58:12.138 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:58:12.157 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 5] out_osds: [7] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T23:58:12.157 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T23:58:12.157 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-14T23:58:12.157 INFO:tasks.thrashosds.thrasher:Setting osd 0 primary_affinity to 0.323135 2024-09-14T23:58:12.157 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 0.32313473012124205 2024-09-14T23:58:12.177 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:58:12.177 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:58:12.178 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:12.176+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:12.206 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:12.206 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:58:12.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:58:12.206 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:58:12.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.206 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:43054 3.1f 3:ffbfe596:::rbd_data.1118350d5ae5.00000000000002eb:head [set-alloc-hint object_size 262144 write_size 262144,write 196608~22770] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T23:58:12.206 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:58:12.206 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 79.235845480999998, 2024-09-14T23:58:12.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.99973334400000002, 2024-09-14T23:58:12.207 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:12.207 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:12.207 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:12.207 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:12.207 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:12.207 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:12.207 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 43054 2024-09-14T23:58:12.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.207 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:12.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:12.208 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:58:12.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:12.208 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:58:12.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:12.208 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966552+0000", 2024-09-14T23:58:12.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3790000000000001e-06 2024-09-14T23:58:12.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.209 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:12.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966565+0000", 2024-09-14T23:58:12.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2459999999999999e-05 2024-09-14T23:58:12.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.209 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:12.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966576+0000", 2024-09-14T23:58:12.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1039000000000001e-05 2024-09-14T23:58:12.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.209 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:12.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966582+0000", 2024-09-14T23:58:12.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9660000000000001e-06 2024-09-14T23:58:12.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:12.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966625+0000", 2024-09-14T23:58:12.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3529999999999998e-05 2024-09-14T23:58:12.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:58:12.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966632+0000", 2024-09-14T23:58:12.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9809999999999997e-06 2024-09-14T23:58:12.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:12.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.964460+0000", 2024-09-14T23:58:12.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.99782768300000002 2024-09-14T23:58:12.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:12.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.964641+0000", 2024-09-14T23:58:12.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000181089 2024-09-14T23:58:12.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-14T23:58:12.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.964809+0000", 2024-09-14T23:58:12.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000168505 2024-09-14T23:58:12.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:58:12.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966030+0000", 2024-09-14T23:58:12.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012206929999999999 2024-09-14T23:58:12.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.212 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:12.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966181+0000", 2024-09-14T23:58:12.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000150569 2024-09-14T23:58:12.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.212 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:12.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966243+0000", 2024-09-14T23:58:12.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2331999999999999e-05 2024-09-14T23:58:12.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:12.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966284+0000", 2024-09-14T23:58:12.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1118000000000002e-05 2024-09-14T23:58:12.213 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.213 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:12.213 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.213 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184314 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4162560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:12.213 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.077247+0000", 2024-09-14T23:58:12.213 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.125148813, 2024-09-14T23:58:12.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058772256000000002, 2024-09-14T23:58:12.213 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:12.213 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:12.213 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:12.214 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:12.214 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:12.214 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:12.214 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184314 2024-09-14T23:58:12.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.214 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:12.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:12.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.077247+0000", 2024-09-14T23:58:12.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:12.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.077247+0000", 2024-09-14T23:58:12.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:12.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.078906+0000", 2024-09-14T23:58:12.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016583279999999999 2024-09-14T23:58:12.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:12.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.078911+0000", 2024-09-14T23:58:12.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0420000000000002e-06 2024-09-14T23:58:12.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:12.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.078922+0000", 2024-09-14T23:58:12.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1354e-05 2024-09-14T23:58:12.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:12.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.078930+0000", 2024-09-14T23:58:12.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5730000000000002e-06 2024-09-14T23:58:12.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:12.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.135870+0000", 2024-09-14T23:58:12.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056940651000000002 2024-09-14T23:58:12.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:12.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.135951+0000", 2024-09-14T23:58:12.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0890999999999995e-05 2024-09-14T23:58:12.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:12.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.136020+0000", 2024-09-14T23:58:12.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8417000000000006e-05 2024-09-14T23:58:12.217 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.217 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:12.217 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.217 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184323 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4183040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:12.217 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.079228+0000", 2024-09-14T23:58:12.217 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.1231677390000003, 2024-09-14T23:58:12.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058454487999999999, 2024-09-14T23:58:12.218 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:12.218 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:12.218 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:12.218 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:12.218 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:12.218 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:12.218 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184323 2024-09-14T23:58:12.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.218 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:12.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:12.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079228+0000", 2024-09-14T23:58:12.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:12.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079228+0000", 2024-09-14T23:58:12.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:12.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079229+0000", 2024-09-14T23:58:12.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5299999999999999e-07 2024-09-14T23:58:12.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:12.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079232+0000", 2024-09-14T23:58:12.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8839999999999998e-06 2024-09-14T23:58:12.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:12.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079238+0000", 2024-09-14T23:58:12.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2299999999999996e-06 2024-09-14T23:58:12.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:12.221 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079243+0000", 2024-09-14T23:58:12.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3029999999999999e-06 2024-09-14T23:58:12.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:12.221 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.137533+0000", 2024-09-14T23:58:12.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058289674999999999 2024-09-14T23:58:12.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:12.222 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.137615+0000", 2024-09-14T23:58:12.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2288999999999998e-05 2024-09-14T23:58:12.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.222 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:12.222 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.137683+0000", 2024-09-14T23:58:12.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7654e-05 2024-09-14T23:58:12.222 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.222 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:12.222 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.222 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184324 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4073472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:12.222 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.079251+0000", 2024-09-14T23:58:12.223 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.1231452720000004, 2024-09-14T23:58:12.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058609383000000001, 2024-09-14T23:58:12.223 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:12.223 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:12.223 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:12.223 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:12.223 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:12.223 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:12.223 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184324 2024-09-14T23:58:12.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.223 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:12.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:12.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079251+0000", 2024-09-14T23:58:12.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.224 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:12.224 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079251+0000", 2024-09-14T23:58:12.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.224 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:12.224 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.080920+0000", 2024-09-14T23:58:12.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016687430000000001 2024-09-14T23:58:12.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.225 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:12.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.080924+0000", 2024-09-14T23:58:12.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3869999999999997e-06 2024-09-14T23:58:12.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.225 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:12.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.080935+0000", 2024-09-14T23:58:12.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0515e-05 2024-09-14T23:58:12.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.225 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:12.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.080942+0000", 2024-09-14T23:58:12.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0990000000000001e-06 2024-09-14T23:58:12.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:12.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.137713+0000", 2024-09-14T23:58:12.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056771288000000003 2024-09-14T23:58:12.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:12.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.137794+0000", 2024-09-14T23:58:12.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0613999999999998e-05 2024-09-14T23:58:12.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:12.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.137860+0000", 2024-09-14T23:58:12.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6736999999999995e-05 2024-09-14T23:58:12.227 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.227 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:12.227 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.227 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184346 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4099072~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:12.227 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.083616+0000", 2024-09-14T23:58:12.227 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.1187800870000002, 2024-09-14T23:58:12.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058557419999999999, 2024-09-14T23:58:12.227 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:12.227 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:12.227 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:12.227 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:12.228 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:12.228 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:12.228 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184346 2024-09-14T23:58:12.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.228 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:12.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.228 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:12.228 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083616+0000", 2024-09-14T23:58:12.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.229 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:12.229 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083616+0000", 2024-09-14T23:58:12.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.229 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:12.229 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083620+0000", 2024-09-14T23:58:12.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0119999999999999e-06 2024-09-14T23:58:12.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.229 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:12.229 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083623+0000", 2024-09-14T23:58:12.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9059999999999998e-06 2024-09-14T23:58:12.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:12.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083631+0000", 2024-09-14T23:58:12.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2679999999999994e-06 2024-09-14T23:58:12.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:12.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083637+0000", 2024-09-14T23:58:12.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1210000000000003e-06 2024-09-14T23:58:12.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:12.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142018+0000", 2024-09-14T23:58:12.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058381111999999999 2024-09-14T23:58:12.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.231 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:12.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142100+0000", 2024-09-14T23:58:12.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1308000000000001e-05 2024-09-14T23:58:12.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.231 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:12.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142173+0000", 2024-09-14T23:58:12.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3693000000000001e-05 2024-09-14T23:58:12.231 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.231 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:12.231 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.232 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184347 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4122624~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:12.232 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.083647+0000", 2024-09-14T23:58:12.232 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.1187492570000002, 2024-09-14T23:58:12.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058706388999999998, 2024-09-14T23:58:12.232 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:12.232 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:12.232 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:12.232 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:12.232 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:12.232 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:12.232 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184347 2024-09-14T23:58:12.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.232 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:12.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.233 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:12.233 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083647+0000", 2024-09-14T23:58:12.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.233 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:12.233 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083647+0000", 2024-09-14T23:58:12.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.233 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:12.233 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083647+0000", 2024-09-14T23:58:12.234 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3099999999999998e-07 2024-09-14T23:58:12.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.234 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:12.234 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083650+0000", 2024-09-14T23:58:12.234 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2900000000000001e-06 2024-09-14T23:58:12.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.234 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:12.234 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083657+0000", 2024-09-14T23:58:12.234 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.486e-06 2024-09-14T23:58:12.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.234 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:12.234 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083663+0000", 2024-09-14T23:58:12.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4330000000000003e-06 2024-09-14T23:58:12.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.235 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:12.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142206+0000", 2024-09-14T23:58:12.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058543237999999997 2024-09-14T23:58:12.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.235 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:12.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142284+0000", 2024-09-14T23:58:12.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8487999999999998e-05 2024-09-14T23:58:12.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.235 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:12.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142353+0000", 2024-09-14T23:58:12.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9023000000000006e-05 2024-09-14T23:58:12.236 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.236 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:12.236 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.236 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184348 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4103168~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:12.236 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.083669+0000", 2024-09-14T23:58:12.236 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.1187268399999999, 2024-09-14T23:58:12.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058864232000000002, 2024-09-14T23:58:12.236 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:12.236 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:12.236 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:12.236 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:12.236 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:12.236 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:12.237 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184348 2024-09-14T23:58:12.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.237 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:12.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:12.237 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083669+0000", 2024-09-14T23:58:12.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:12.237 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083669+0000", 2024-09-14T23:58:12.240 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.241 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:12.241 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083670+0000", 2024-09-14T23:58:12.241 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9999999999999998e-07 2024-09-14T23:58:12.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.241 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.241 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:12.241 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083672+0000", 2024-09-14T23:58:12.241 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1430000000000001e-06 2024-09-14T23:58:12.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.242 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:12.242 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083678+0000", 2024-09-14T23:58:12.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5200000000000003e-06 2024-09-14T23:58:12.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.242 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:12.242 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083683+0000", 2024-09-14T23:58:12.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9359999999999998e-06 2024-09-14T23:58:12.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.242 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:12.242 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142384+0000", 2024-09-14T23:58:12.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058701159000000003 2024-09-14T23:58:12.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.243 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:12.243 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142466+0000", 2024-09-14T23:58:12.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1024000000000002e-05 2024-09-14T23:58:12.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.243 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.243 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:12.243 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142534+0000", 2024-09-14T23:58:12.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8050000000000001e-05 2024-09-14T23:58:12.243 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.243 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:12.243 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.243 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.243 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184349 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4126720~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:12.244 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.083690+0000", 2024-09-14T23:58:12.244 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.1187063049999999, 2024-09-14T23:58:12.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059019911000000001, 2024-09-14T23:58:12.244 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:12.244 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:12.244 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:12.244 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:12.244 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:12.244 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:12.244 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184349 2024-09-14T23:58:12.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.245 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:12.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.245 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:12.245 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083690+0000", 2024-09-14T23:58:12.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.245 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:12.245 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083690+0000", 2024-09-14T23:58:12.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.245 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:12.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.084900+0000", 2024-09-14T23:58:12.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012099770000000001 2024-09-14T23:58:12.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:12.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.084905+0000", 2024-09-14T23:58:12.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.028e-06 2024-09-14T23:58:12.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:12.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.084914+0000", 2024-09-14T23:58:12.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6619999999999994e-06 2024-09-14T23:58:12.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:12.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.084922+0000", 2024-09-14T23:58:12.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5720000000000003e-06 2024-09-14T23:58:12.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:12.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142565+0000", 2024-09-14T23:58:12.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057642605999999999 2024-09-14T23:58:12.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:12.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142644+0000", 2024-09-14T23:58:12.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9370999999999996e-05 2024-09-14T23:58:12.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:12.248 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142710+0000", 2024-09-14T23:58:12.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5695e-05 2024-09-14T23:58:12.248 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.248 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:12.248 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.248 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184354 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4115456~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:12.248 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.085029+0000", 2024-09-14T23:58:12.248 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.1173676549999998, 2024-09-14T23:58:12.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058626327999999998, 2024-09-14T23:58:12.248 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:12.248 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:12.248 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:12.249 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:12.249 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:12.249 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:12.249 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184354 2024-09-14T23:58:12.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.249 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:12.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.249 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:12.249 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085029+0000", 2024-09-14T23:58:12.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.249 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:12.249 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085029+0000", 2024-09-14T23:58:12.249 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.250 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:12.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085029+0000", 2024-09-14T23:58:12.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0800000000000001e-07 2024-09-14T23:58:12.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.250 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:12.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085030+0000", 2024-09-14T23:58:12.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.226e-06 2024-09-14T23:58:12.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:12.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085033+0000", 2024-09-14T23:58:12.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9569999999999998e-06 2024-09-14T23:58:12.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:12.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085035+0000", 2024-09-14T23:58:12.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.587e-06 2024-09-14T23:58:12.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:12.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143491+0000", 2024-09-14T23:58:12.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058455337000000003 2024-09-14T23:58:12.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:12.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143575+0000", 2024-09-14T23:58:12.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3850000000000005e-05 2024-09-14T23:58:12.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:12.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143655+0000", 2024-09-14T23:58:12.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0162999999999998e-05 2024-09-14T23:58:12.252 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.253 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:12.253 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.253 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.253 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184355 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4139008~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:12.253 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.085039+0000", 2024-09-14T23:58:12.253 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.1173572509999996, 2024-09-14T23:58:12.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058801369999999999, 2024-09-14T23:58:12.253 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:12.253 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:12.253 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:12.253 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:12.253 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:12.253 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:12.254 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184355 2024-09-14T23:58:12.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.254 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:12.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.254 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:12.254 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085039+0000", 2024-09-14T23:58:12.254 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.254 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:12.254 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085039+0000", 2024-09-14T23:58:12.254 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.254 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:12.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085039+0000", 2024-09-14T23:58:12.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.03e-07 2024-09-14T23:58:12.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:12.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085040+0000", 2024-09-14T23:58:12.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.158e-06 2024-09-14T23:58:12.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:12.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085043+0000", 2024-09-14T23:58:12.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0240000000000002e-06 2024-09-14T23:58:12.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:12.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085046+0000", 2024-09-14T23:58:12.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4600000000000002e-06 2024-09-14T23:58:12.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:12.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143688+0000", 2024-09-14T23:58:12.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058642356 2024-09-14T23:58:12.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:12.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143766+0000", 2024-09-14T23:58:12.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8219000000000004e-05 2024-09-14T23:58:12.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:12.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143840+0000", 2024-09-14T23:58:12.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3949999999999995e-05 2024-09-14T23:58:12.257 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.257 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:12.257 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.257 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184356 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4119552~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:12.257 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.085049+0000", 2024-09-14T23:58:12.257 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.11734708, 2024-09-14T23:58:12.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058977992, 2024-09-14T23:58:12.257 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:12.257 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:12.258 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:12.258 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:12.258 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:12.258 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:12.258 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184356 2024-09-14T23:58:12.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.258 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:12.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:12.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085049+0000", 2024-09-14T23:58:12.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:12.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085049+0000", 2024-09-14T23:58:12.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.259 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:12.259 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.086934+0000", 2024-09-14T23:58:12.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018854189999999999 2024-09-14T23:58:12.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.259 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:12.259 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.086939+0000", 2024-09-14T23:58:12.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.493e-06 2024-09-14T23:58:12.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.259 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:12.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.086951+0000", 2024-09-14T23:58:12.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1969e-05 2024-09-14T23:58:12.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:12.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.086959+0000", 2024-09-14T23:58:12.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5780000000000001e-06 2024-09-14T23:58:12.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:12.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143871+0000", 2024-09-14T23:58:12.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056912114 2024-09-14T23:58:12.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:12.261 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143951+0000", 2024-09-14T23:58:12.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9938000000000006e-05 2024-09-14T23:58:12.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.261 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:12.261 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.144027+0000", 2024-09-14T23:58:12.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6481000000000002e-05 2024-09-14T23:58:12.261 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.261 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:12.261 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.261 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184364 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4135936~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:12.262 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.087207+0000", 2024-09-14T23:58:12.262 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.1151888000000003, 2024-09-14T23:58:12.262 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058466420999999998, 2024-09-14T23:58:12.262 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:12.262 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:12.262 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:12.262 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:12.262 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:12.262 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:12.262 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184364 2024-09-14T23:58:12.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.262 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:12.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:12.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087207+0000", 2024-09-14T23:58:12.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.263 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:12.263 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087207+0000", 2024-09-14T23:58:12.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.263 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:12.263 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087208+0000", 2024-09-14T23:58:12.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.35e-07 2024-09-14T23:58:12.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.263 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:12.263 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087209+0000", 2024-09-14T23:58:12.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.573e-06 2024-09-14T23:58:12.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:12.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087212+0000", 2024-09-14T23:58:12.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2770000000000001e-06 2024-09-14T23:58:12.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:12.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087215+0000", 2024-09-14T23:58:12.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5500000000000001e-06 2024-09-14T23:58:12.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:12.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145520+0000", 2024-09-14T23:58:12.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058304835999999999 2024-09-14T23:58:12.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.265 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:12.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145602+0000", 2024-09-14T23:58:12.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1714999999999999e-05 2024-09-14T23:58:12.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.265 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:12.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145674+0000", 2024-09-14T23:58:12.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2235000000000002e-05 2024-09-14T23:58:12.265 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.266 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:12.266 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.266 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184365 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4159488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:12.266 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.087219+0000", 2024-09-14T23:58:12.266 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.1151767689999996, 2024-09-14T23:58:12.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058643533999999997, 2024-09-14T23:58:12.266 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:12.266 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:12.266 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:12.266 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:12.266 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:12.266 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:12.266 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184365 2024-09-14T23:58:12.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.267 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:12.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.267 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:12.267 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087219+0000", 2024-09-14T23:58:12.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.267 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:12.267 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087219+0000", 2024-09-14T23:58:12.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.267 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:12.267 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087220+0000", 2024-09-14T23:58:12.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.39e-07 2024-09-14T23:58:12.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:12.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087222+0000", 2024-09-14T23:58:12.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0609999999999998e-06 2024-09-14T23:58:12.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:12.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087228+0000", 2024-09-14T23:58:12.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8590000000000001e-06 2024-09-14T23:58:12.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:12.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087232+0000", 2024-09-14T23:58:12.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8149999999999996e-06 2024-09-14T23:58:12.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:12.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145705+0000", 2024-09-14T23:58:12.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058472532000000001 2024-09-14T23:58:12.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:12.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145784+0000", 2024-09-14T23:58:12.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9382000000000007e-05 2024-09-14T23:58:12.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:12.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145863+0000", 2024-09-14T23:58:12.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8546000000000003e-05 2024-09-14T23:58:12.270 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.270 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:12.270 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.270 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184366 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4140032~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:12.270 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.087240+0000", 2024-09-14T23:58:12.270 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.1151563209999997, 2024-09-14T23:58:12.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058818068000000001, 2024-09-14T23:58:12.270 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:12.270 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:12.271 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:12.271 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:12.271 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:12.271 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:12.271 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184366 2024-09-14T23:58:12.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.271 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:12.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:12.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087240+0000", 2024-09-14T23:58:12.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:12.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087240+0000", 2024-09-14T23:58:12.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:12.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.088904+0000", 2024-09-14T23:58:12.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001664315 2024-09-14T23:58:12.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:12.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.088909+0000", 2024-09-14T23:58:12.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7709999999999997e-06 2024-09-14T23:58:12.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:12.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.088921+0000", 2024-09-14T23:58:12.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1661999999999999e-05 2024-09-14T23:58:12.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:12.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.088928+0000", 2024-09-14T23:58:12.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8189999999999992e-06 2024-09-14T23:58:12.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:12.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145901+0000", 2024-09-14T23:58:12.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056972478 2024-09-14T23:58:12.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:12.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145977+0000", 2024-09-14T23:58:12.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5763e-05 2024-09-14T23:58:12.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:12.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.146058+0000", 2024-09-14T23:58:12.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1260000000000005e-05 2024-09-14T23:58:12.274 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.274 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:12.274 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.275 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184375 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4179968~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:12.275 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.089199+0000", 2024-09-14T23:58:12.275 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.113196984, 2024-09-14T23:58:12.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058597445999999997, 2024-09-14T23:58:12.275 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:12.275 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:12.275 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:12.275 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:12.275 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:12.275 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:12.275 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184375 2024-09-14T23:58:12.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.275 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:12.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.276 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:12.276 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089199+0000", 2024-09-14T23:58:12.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.276 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:12.276 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089199+0000", 2024-09-14T23:58:12.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.276 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:12.276 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089200+0000", 2024-09-14T23:58:12.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8100000000000003e-07 2024-09-14T23:58:12.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:12.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089202+0000", 2024-09-14T23:58:12.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7599999999999998e-06 2024-09-14T23:58:12.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:12.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089209+0000", 2024-09-14T23:58:12.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7279999999999998e-06 2024-09-14T23:58:12.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:12.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089215+0000", 2024-09-14T23:58:12.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5090000000000001e-06 2024-09-14T23:58:12.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:12.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147638+0000", 2024-09-14T23:58:12.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058423105000000003 2024-09-14T23:58:12.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:12.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147719+0000", 2024-09-14T23:58:12.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1217000000000005e-05 2024-09-14T23:58:12.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:12.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147797+0000", 2024-09-14T23:58:12.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7645999999999995e-05 2024-09-14T23:58:12.278 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.279 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:12.279 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.279 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184376 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4160512~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:12.279 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.089232+0000", 2024-09-14T23:58:12.279 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.1131644940000003, 2024-09-14T23:58:12.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058751338, 2024-09-14T23:58:12.279 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:12.279 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:12.279 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:12.279 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:12.279 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:12.279 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:12.280 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184376 2024-09-14T23:58:12.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.280 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:12.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.280 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:12.280 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089232+0000", 2024-09-14T23:58:12.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.280 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:12.280 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089232+0000", 2024-09-14T23:58:12.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:12.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089232+0000", 2024-09-14T23:58:12.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5600000000000001e-07 2024-09-14T23:58:12.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:12.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089235+0000", 2024-09-14T23:58:12.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5639999999999999e-06 2024-09-14T23:58:12.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:12.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089242+0000", 2024-09-14T23:58:12.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0369999999999997e-06 2024-09-14T23:58:12.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:12.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089247+0000", 2024-09-14T23:58:12.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2079999999999999e-06 2024-09-14T23:58:12.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:12.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147831+0000", 2024-09-14T23:58:12.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058583853999999998 2024-09-14T23:58:12.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:12.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147915+0000", 2024-09-14T23:58:12.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4208000000000004e-05 2024-09-14T23:58:12.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:12.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147983+0000", 2024-09-14T23:58:12.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8010999999999996e-05 2024-09-14T23:58:12.283 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.283 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:12.283 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.283 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184377 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4184064~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:12.283 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.089255+0000", 2024-09-14T23:58:12.283 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.1131409379999999, 2024-09-14T23:58:12.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058906100000000003, 2024-09-14T23:58:12.283 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:12.283 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:12.283 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:12.284 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:12.284 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:12.284 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:12.284 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184377 2024-09-14T23:58:12.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.284 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:12.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:12.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089255+0000", 2024-09-14T23:58:12.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:12.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089255+0000", 2024-09-14T23:58:12.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.285 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:12.285 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.090903+0000", 2024-09-14T23:58:12.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001647871 2024-09-14T23:58:12.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.285 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:12.285 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.090908+0000", 2024-09-14T23:58:12.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7199999999999997e-06 2024-09-14T23:58:12.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.285 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:12.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.090919+0000", 2024-09-14T23:58:12.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1472e-05 2024-09-14T23:58:12.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:12.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.090927+0000", 2024-09-14T23:58:12.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0390000000000005e-06 2024-09-14T23:58:12.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:12.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.148013+0000", 2024-09-14T23:58:12.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057085662000000002 2024-09-14T23:58:12.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:12.287 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.148090+0000", 2024-09-14T23:58:12.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6778000000000002e-05 2024-09-14T23:58:12.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.287 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:12.287 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.148161+0000", 2024-09-14T23:58:12.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1557999999999995e-05 2024-09-14T23:58:12.287 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.287 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:12.287 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.287 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184386 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4185088~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:12.287 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.091063+0000", 2024-09-14T23:58:12.287 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.1113332739999997, 2024-09-14T23:58:12.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058603404999999997, 2024-09-14T23:58:12.288 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:12.288 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:12.288 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:12.288 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:12.288 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:12.288 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:12.288 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184386 2024-09-14T23:58:12.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.288 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:12.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:12.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091063+0000", 2024-09-14T23:58:12.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:12.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091063+0000", 2024-09-14T23:58:12.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:12.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091063+0000", 2024-09-14T23:58:12.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1899999999999999e-07 2024-09-14T23:58:12.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:12.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091064+0000", 2024-09-14T23:58:12.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2699999999999999e-06 2024-09-14T23:58:12.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:12.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091067+0000", 2024-09-14T23:58:12.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.971e-06 2024-09-14T23:58:12.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:12.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091070+0000", 2024-09-14T23:58:12.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.525e-06 2024-09-14T23:58:12.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:12.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.149510+0000", 2024-09-14T23:58:12.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058439758000000001 2024-09-14T23:58:12.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:12.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.149595+0000", 2024-09-14T23:58:12.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5146000000000001e-05 2024-09-14T23:58:12.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:12.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.149666+0000", 2024-09-14T23:58:12.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1515999999999998e-05 2024-09-14T23:58:12.291 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.291 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:12.291 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.291 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184388 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4189184~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:12.291 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.091073+0000", 2024-09-14T23:58:12.292 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.1113229990000004, 2024-09-14T23:58:12.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058781609999999998, 2024-09-14T23:58:12.292 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:12.292 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:12.292 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:12.292 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:12.292 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:12.292 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:12.292 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184388 2024-09-14T23:58:12.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.292 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:12.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.292 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:12.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091073+0000", 2024-09-14T23:58:12.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.293 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:12.293 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091073+0000", 2024-09-14T23:58:12.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.293 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:12.293 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.092903+0000", 2024-09-14T23:58:12.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018296289999999999 2024-09-14T23:58:12.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.293 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:12.294 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.092910+0000", 2024-09-14T23:58:12.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7510000000000002e-06 2024-09-14T23:58:12.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:12.294 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.092918+0000", 2024-09-14T23:58:12.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7420000000000003e-06 2024-09-14T23:58:12.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:12.294 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.092925+0000", 2024-09-14T23:58:12.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0360000000000001e-06 2024-09-14T23:58:12.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:12.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.149700+0000", 2024-09-14T23:58:12.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056774340999999999 2024-09-14T23:58:12.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.295 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:12.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.149784+0000", 2024-09-14T23:58:12.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4393000000000003e-05 2024-09-14T23:58:12.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.295 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:12.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.149855+0000", 2024-09-14T23:58:12.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0717999999999996e-05 2024-09-14T23:58:12.295 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.295 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:12.296 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.296 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4172.0:8 3.12 3:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-14T23:58:12.296 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.976999+0000", 2024-09-14T23:58:12.296 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.22539759, 2024-09-14T23:58:12.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0046036439999999, 2024-09-14T23:58:12.296 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:12.296 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:12.296 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:12.296 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:12.296 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4172", 2024-09-14T23:58:12.296 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/2542269203", 2024-09-14T23:58:12.297 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 8 2024-09-14T23:58:12.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.297 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:12.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:12.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.976999+0000", 2024-09-14T23:58:12.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:12.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.976999+0000", 2024-09-14T23:58:12.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.298 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:12.298 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.977000+0000", 2024-09-14T23:58:12.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5200000000000001e-06 2024-09-14T23:58:12.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.298 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.298 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:12.298 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.977005+0000", 2024-09-14T23:58:12.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.04e-06 2024-09-14T23:58:12.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.298 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.298 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:12.298 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.977013+0000", 2024-09-14T23:58:12.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2889999999999998e-06 2024-09-14T23:58:12.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.299 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:12.299 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.977024+0000", 2024-09-14T23:58:12.299 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0307e-05 2024-09-14T23:58:12.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.299 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:12.299 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.977086+0000", 2024-09-14T23:58:12.299 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2248000000000004e-05 2024-09-14T23:58:12.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.299 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:58:12.299 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.977097+0000", 2024-09-14T23:58:12.299 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1123e-05 2024-09-14T23:58:12.300 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.300 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.300 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:12.300 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:08.981470+0000", 2024-09-14T23:58:12.300 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.004373116 2024-09-14T23:58:12.300 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.300 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.300 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:12.300 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:08.981602+0000", 2024-09-14T23:58:12.300 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000132001 2024-09-14T23:58:12.300 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.300 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:12.300 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.300 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.300 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:12.301 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:12.302 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T23:58:12.303 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:12.300+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:12.403 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:12.401+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:12.455 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:12.455 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:12.455 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:58:12.455 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:12.464 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T23:58:12.504 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:12.502+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:12.572 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:12.572 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:12.572 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:58:12.573 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:58:12.573 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:12.581 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T23:58:12.624 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:12.623+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:12.690 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:12.690 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:58:12.690 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:58:12.690 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:58:12.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.690 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5882 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:12.690 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:58:12.690 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 93.311991496999994, 2024-09-14T23:58:12.691 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050549666, 2024-09-14T23:58:12.691 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:12.691 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:12.691 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:12.691 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:12.691 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:12.691 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:12.691 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5882 2024-09-14T23:58:12.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.691 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:12.692 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.692 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:12.692 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:58:12.692 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.692 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.692 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.692 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:12.692 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:58:12.692 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.692 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.692 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.693 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:12.693 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:58:12.693 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8099999999999998e-07 2024-09-14T23:58:12.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.693 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.693 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:12.693 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375842+0000", 2024-09-14T23:58:12.693 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9900000000000009e-07 2024-09-14T23:58:12.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:12.694 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375845+0000", 2024-09-14T23:58:12.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.055e-06 2024-09-14T23:58:12.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:12.694 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375848+0000", 2024-09-14T23:58:12.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7609999999999999e-06 2024-09-14T23:58:12.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.695 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:12.695 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426381+0000", 2024-09-14T23:58:12.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050533392000000003 2024-09-14T23:58:12.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.695 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:12.695 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426390+0000", 2024-09-14T23:58:12.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0780000000000002e-06 2024-09-14T23:58:12.695 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.695 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:12.695 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.696 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5884 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:12.696 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:58:12.696 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 93.311981356999993, 2024-09-14T23:58:12.696 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050717439000000003, 2024-09-14T23:58:12.696 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:12.696 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:12.696 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:12.696 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:12.696 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:12.696 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:12.696 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5884 2024-09-14T23:58:12.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.697 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:12.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.697 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:12.697 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:58:12.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.697 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:12.697 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:58:12.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.698 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:12.698 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:58:12.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0800000000000001e-07 2024-09-14T23:58:12.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.698 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:12.698 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375852+0000", 2024-09-14T23:58:12.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.02e-06 2024-09-14T23:58:12.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.698 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:12.699 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375855+0000", 2024-09-14T23:58:12.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9900000000000002e-06 2024-09-14T23:58:12.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.699 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:12.699 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375857+0000", 2024-09-14T23:58:12.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3499999999999999e-06 2024-09-14T23:58:12.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.699 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:12.699 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426556+0000", 2024-09-14T23:58:12.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050698139000000003 2024-09-14T23:58:12.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.700 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:12.700 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426568+0000", 2024-09-14T23:58:12.700 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2731999999999999e-05 2024-09-14T23:58:12.700 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.700 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:12.700 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.700 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5885 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:12.701 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:58:12.701 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 93.311962331999993, 2024-09-14T23:58:12.701 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050737333000000003, 2024-09-14T23:58:12.701 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:12.701 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:12.701 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:12.701 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:12.701 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:12.701 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:12.701 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5885 2024-09-14T23:58:12.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.701 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:12.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:12.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:58:12.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:12.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:58:12.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:12.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:58:12.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5899999999999998e-07 2024-09-14T23:58:12.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.703 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:12.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375872+0000", 2024-09-14T23:58:12.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4219999999999999e-06 2024-09-14T23:58:12.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.703 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:12.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375878+0000", 2024-09-14T23:58:12.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4980000000000003e-06 2024-09-14T23:58:12.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.704 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:12.704 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375881+0000", 2024-09-14T23:58:12.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6110000000000001e-06 2024-09-14T23:58:12.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.704 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:12.704 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426596+0000", 2024-09-14T23:58:12.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050715320000000001 2024-09-14T23:58:12.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.705 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:12.705 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426607+0000", 2024-09-14T23:58:12.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1223000000000001e-05 2024-09-14T23:58:12.705 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.705 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:12.705 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.705 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5886 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:12.705 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:58:12.705 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 93.311948219000001, 2024-09-14T23:58:12.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050803395000000001, 2024-09-14T23:58:12.706 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:12.706 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:12.706 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:12.706 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:12.706 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:12.706 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:12.706 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5886 2024-09-14T23:58:12.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.706 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:12.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:12.707 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:58:12.707 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.707 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:12.707 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:58:12.707 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.707 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:12.707 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:58:12.707 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2700000000000001e-07 2024-09-14T23:58:12.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:12.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375886+0000", 2024-09-14T23:58:12.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7060000000000001e-06 2024-09-14T23:58:12.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:12.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375890+0000", 2024-09-14T23:58:12.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2259999999999999e-06 2024-09-14T23:58:12.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:12.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375893+0000", 2024-09-14T23:58:12.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0819999999999999e-06 2024-09-14T23:58:12.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:12.709 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426677+0000", 2024-09-14T23:58:12.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050784158000000003 2024-09-14T23:58:12.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:12.709 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426687+0000", 2024-09-14T23:58:12.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9960000000000005e-06 2024-09-14T23:58:12.710 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.710 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:12.710 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.710 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5887 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:12.710 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:58:12.710 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 93.311926133, 2024-09-14T23:58:12.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050833070000000001, 2024-09-14T23:58:12.711 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:12.711 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:12.711 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:12.711 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:12.711 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:12.711 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:12.711 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5887 2024-09-14T23:58:12.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.711 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:12.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.712 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:12.712 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:58:12.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.712 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.712 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:12.712 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:58:12.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.712 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.712 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:12.713 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:58:12.713 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3299999999999998e-07 2024-09-14T23:58:12.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.713 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:12.713 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375908+0000", 2024-09-14T23:58:12.713 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7689999999999999e-06 2024-09-14T23:58:12.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.713 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:12.714 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375912+0000", 2024-09-14T23:58:12.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.535e-06 2024-09-14T23:58:12.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.714 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:12.714 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375916+0000", 2024-09-14T23:58:12.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8979999999999998e-06 2024-09-14T23:58:12.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.714 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:12.714 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426729+0000", 2024-09-14T23:58:12.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050813453000000001 2024-09-14T23:58:12.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:12.715 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426739+0000", 2024-09-14T23:58:12.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0081999999999999e-05 2024-09-14T23:58:12.715 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.715 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:12.715 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.715 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5888 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:12.715 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:58:12.716 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 93.311913055999995, 2024-09-14T23:58:12.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050854652, 2024-09-14T23:58:12.716 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:12.716 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:12.716 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:12.716 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:12.716 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:12.716 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:12.716 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5888 2024-09-14T23:58:12.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.716 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:12.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.716 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:12.717 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:58:12.717 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.717 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.717 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:12.717 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:58:12.717 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.717 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.717 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:12.717 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:58:12.717 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1400000000000001e-07 2024-09-14T23:58:12.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:12.718 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375921+0000", 2024-09-14T23:58:12.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.119e-06 2024-09-14T23:58:12.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:12.718 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375928+0000", 2024-09-14T23:58:12.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3170000000000004e-06 2024-09-14T23:58:12.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.718 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:12.718 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375931+0000", 2024-09-14T23:58:12.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6970000000000002e-06 2024-09-14T23:58:12.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:12.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426768+0000", 2024-09-14T23:58:12.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050837355000000001 2024-09-14T23:58:12.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:12.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426774+0000", 2024-09-14T23:58:12.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9499999999999998e-06 2024-09-14T23:58:12.719 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.719 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:12.720 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.720 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5889 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:12.720 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:58:12.720 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 93.311896132000001, 2024-09-14T23:58:12.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050885446000000001, 2024-09-14T23:58:12.720 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:12.720 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:12.720 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:12.720 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:12.721 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:12.721 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:12.721 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5889 2024-09-14T23:58:12.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.721 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:12.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.721 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:12.721 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:58:12.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.721 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:12.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:58:12.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.722 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:12.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:58:12.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.36e-07 2024-09-14T23:58:12.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.722 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:12.722 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375940+0000", 2024-09-14T23:58:12.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.844e-06 2024-09-14T23:58:12.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.723 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:12.723 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375944+0000", 2024-09-14T23:58:12.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6720000000000002e-06 2024-09-14T23:58:12.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.723 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:12.723 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375946+0000", 2024-09-14T23:58:12.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5629999999999999e-06 2024-09-14T23:58:12.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.723 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:12.724 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426810+0000", 2024-09-14T23:58:12.724 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050863255000000003 2024-09-14T23:58:12.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.724 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.724 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:12.724 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426822+0000", 2024-09-14T23:58:12.724 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1875999999999999e-05 2024-09-14T23:58:12.724 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.724 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:12.724 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.724 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.725 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5890 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:12.725 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:58:12.725 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 93.311882319999995, 2024-09-14T23:58:12.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050917549999999999, 2024-09-14T23:58:12.725 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:12.725 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:12.725 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:12.725 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:12.725 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:12.725 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:12.725 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5890 2024-09-14T23:58:12.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.726 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:12.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.726 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:12.726 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:58:12.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.726 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:12.726 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:58:12.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:12.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:58:12.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4299999999999999e-07 2024-09-14T23:58:12.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:12.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375951+0000", 2024-09-14T23:58:12.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1000000000000001e-06 2024-09-14T23:58:12.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.728 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:12.728 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375956+0000", 2024-09-14T23:58:12.728 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3370000000000001e-06 2024-09-14T23:58:12.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.728 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:12.728 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375958+0000", 2024-09-14T23:58:12.728 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4820000000000001e-06 2024-09-14T23:58:12.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.729 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.729 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:12.729 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426858+0000", 2024-09-14T23:58:12.729 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050899666000000003 2024-09-14T23:58:12.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.729 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.729 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:12.729 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426867+0000", 2024-09-14T23:58:12.729 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7219999999999994e-06 2024-09-14T23:58:12.729 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.729 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:12.729 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.730 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5891 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:12.730 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:58:12.730 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 93.311870998000003, 2024-09-14T23:58:12.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050948067, 2024-09-14T23:58:12.730 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:12.730 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:12.730 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:12.730 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:12.730 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:12.730 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:12.730 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5891 2024-09-14T23:58:12.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.731 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:12.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:12.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:58:12.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:12.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:58:12.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.732 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:12.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:58:12.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0800000000000001e-07 2024-09-14T23:58:12.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.732 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:12.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375962+0000", 2024-09-14T23:58:12.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9600000000000008e-07 2024-09-14T23:58:12.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.733 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:12.733 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375970+0000", 2024-09-14T23:58:12.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1600000000000001e-06 2024-09-14T23:58:12.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.733 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:12.733 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375972+0000", 2024-09-14T23:58:12.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4849999999999999e-06 2024-09-14T23:58:12.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.733 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:12.734 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426902+0000", 2024-09-14T23:58:12.734 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050929813999999997 2024-09-14T23:58:12.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.734 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.734 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:12.734 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426909+0000", 2024-09-14T23:58:12.734 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4039999999999998e-06 2024-09-14T23:58:12.734 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.734 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:12.734 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.734 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.735 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5892 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:12.735 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:58:12.735 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 93.311856141000007, 2024-09-14T23:58:12.735 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050965036999999998, 2024-09-14T23:58:12.735 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:12.735 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:12.735 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:12.735 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:12.735 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:12.735 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:12.735 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5892 2024-09-14T23:58:12.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.736 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:12.736 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.736 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:12.736 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:58:12.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.736 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.736 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:12.736 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:58:12.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.737 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:12.737 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:58:12.737 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.34e-07 2024-09-14T23:58:12.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.737 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:12.737 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375978+0000", 2024-09-14T23:58:12.737 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1880000000000001e-06 2024-09-14T23:58:12.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.737 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:12.738 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375981+0000", 2024-09-14T23:58:12.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2399999999999999e-06 2024-09-14T23:58:12.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.738 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.738 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:12.738 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375984+0000", 2024-09-14T23:58:12.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9110000000000002e-06 2024-09-14T23:58:12.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.738 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.738 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:12.738 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426934+0000", 2024-09-14T23:58:12.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050949912 2024-09-14T23:58:12.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.739 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:12.739 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426941+0000", 2024-09-14T23:58:12.739 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5519999999999999e-06 2024-09-14T23:58:12.739 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.739 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:12.739 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.739 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5893 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:12.739 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:58:12.739 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 93.311845543999993, 2024-09-14T23:58:12.739 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050975131999999999, 2024-09-14T23:58:12.739 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:12.740 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:12.740 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:12.740 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:12.740 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:12.740 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:12.740 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5893 2024-09-14T23:58:12.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.740 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:12.740 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.741 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:12.741 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:58:12.741 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.741 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:12.741 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:58:12.741 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.741 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:12.742 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:58:12.742 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2999999999999999e-07 2024-09-14T23:58:12.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.742 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:12.742 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375988+0000", 2024-09-14T23:58:12.742 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0410000000000001e-06 2024-09-14T23:58:12.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.742 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:12.742 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375992+0000", 2024-09-14T23:58:12.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.104e-06 2024-09-14T23:58:12.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:12.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375995+0000", 2024-09-14T23:58:12.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4310000000000001e-06 2024-09-14T23:58:12.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:12.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426958+0000", 2024-09-14T23:58:12.744 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050963642000000003 2024-09-14T23:58:12.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.744 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:12.744 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426962+0000", 2024-09-14T23:58:12.744 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6839999999999998e-06 2024-09-14T23:58:12.744 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.744 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:12.744 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.745 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5894 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:12.745 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:58:12.745 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 93.311834144000002, 2024-09-14T23:58:12.745 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050978333000000001, 2024-09-14T23:58:12.745 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:12.745 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:12.745 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:12.745 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:12.745 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:12.745 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:12.745 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5894 2024-09-14T23:58:12.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.746 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:12.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.746 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:12.746 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:58:12.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.746 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:12.746 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:58:12.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.747 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:12.747 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:58:12.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.23e-07 2024-09-14T23:58:12.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.747 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:12.747 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375999+0000", 2024-09-14T23:58:12.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9800000000000002e-07 2024-09-14T23:58:12.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.748 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:12.748 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376007+0000", 2024-09-14T23:58:12.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8539999999999997e-06 2024-09-14T23:58:12.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.748 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:12.748 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376011+0000", 2024-09-14T23:58:12.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.399e-06 2024-09-14T23:58:12.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.748 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:12.749 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426973+0000", 2024-09-14T23:58:12.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050962454999999997 2024-09-14T23:58:12.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.749 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.749 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:12.749 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426976+0000", 2024-09-14T23:58:12.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4039999999999999e-06 2024-09-14T23:58:12.749 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.749 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:12.749 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.749 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.749 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5895 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:12.750 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376014+0000", 2024-09-14T23:58:12.750 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 93.3118178, 2024-09-14T23:58:12.750 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050975951999999998, 2024-09-14T23:58:12.750 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:12.750 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:12.750 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:12.750 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:12.750 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:12.750 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:12.751 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5895 2024-09-14T23:58:12.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.751 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:12.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.751 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:12.751 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376014+0000", 2024-09-14T23:58:12.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.751 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:12.751 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376014+0000", 2024-09-14T23:58:12.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.752 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.752 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:12.752 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376015+0000", 2024-09-14T23:58:12.752 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2999999999999999e-07 2024-09-14T23:58:12.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.752 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.752 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:12.753 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376016+0000", 2024-09-14T23:58:12.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1659999999999999e-06 2024-09-14T23:58:12.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.753 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:12.753 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376024+0000", 2024-09-14T23:58:12.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8930000000000005e-06 2024-09-14T23:58:12.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.754 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:12.754 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376026+0000", 2024-09-14T23:58:12.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6869999999999999e-06 2024-09-14T23:58:12.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.754 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:12.754 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426987+0000", 2024-09-14T23:58:12.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050960601000000001 2024-09-14T23:58:12.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.755 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:12.755 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426990+0000", 2024-09-14T23:58:12.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3749999999999999e-06 2024-09-14T23:58:12.755 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.755 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:12.755 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.755 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5896 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:12.755 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:58:12.755 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 93.311802063000002, 2024-09-14T23:58:12.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050975656000000001, 2024-09-14T23:58:12.755 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:12.756 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:12.756 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:12.756 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:12.756 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:12.756 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:12.756 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5896 2024-09-14T23:58:12.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.756 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:12.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.756 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:12.756 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:58:12.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.757 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:12.757 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:58:12.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.757 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:12.757 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:58:12.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5100000000000001e-07 2024-09-14T23:58:12.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.758 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:12.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376032+0000", 2024-09-14T23:58:12.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.141e-06 2024-09-14T23:58:12.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.758 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:12.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376036+0000", 2024-09-14T23:58:12.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.25e-06 2024-09-14T23:58:12.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.758 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:12.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376038+0000", 2024-09-14T23:58:12.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5469999999999999e-06 2024-09-14T23:58:12.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:12.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427002+0000", 2024-09-14T23:58:12.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050964029000000001 2024-09-14T23:58:12.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:12.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427006+0000", 2024-09-14T23:58:12.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4379999999999999e-06 2024-09-14T23:58:12.760 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.760 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:12.760 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.760 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5897 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:12.760 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:58:12.760 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 93.311790263000006, 2024-09-14T23:58:12.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050978001000000002, 2024-09-14T23:58:12.760 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:12.760 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:12.760 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:12.761 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:12.761 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:12.761 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:12.761 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5897 2024-09-14T23:58:12.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.761 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:12.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.761 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:12.761 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:58:12.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:12.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:58:12.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:12.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:58:12.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2600000000000001e-07 2024-09-14T23:58:12.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:12.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376043+0000", 2024-09-14T23:58:12.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.054e-06 2024-09-14T23:58:12.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:12.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376047+0000", 2024-09-14T23:58:12.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0659999999999997e-06 2024-09-14T23:58:12.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:12.764 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376050+0000", 2024-09-14T23:58:12.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.88e-06 2024-09-14T23:58:12.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.764 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:12.764 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427017+0000", 2024-09-14T23:58:12.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050966453000000002 2024-09-14T23:58:12.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.764 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:12.764 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427020+0000", 2024-09-14T23:58:12.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3220000000000001e-06 2024-09-14T23:58:12.765 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.765 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:12.765 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.765 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5898 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:12.765 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376053+0000", 2024-09-14T23:58:12.765 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 93.311778778000004, 2024-09-14T23:58:12.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050982024000000001, 2024-09-14T23:58:12.765 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:12.765 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:12.766 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:12.766 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:12.766 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:12.766 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:12.766 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5898 2024-09-14T23:58:12.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.766 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:12.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:12.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376053+0000", 2024-09-14T23:58:12.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:12.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376053+0000", 2024-09-14T23:58:12.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:12.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376278+0000", 2024-09-14T23:58:12.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022426400000000001 2024-09-14T23:58:12.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:12.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376282+0000", 2024-09-14T23:58:12.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4739999999999999e-06 2024-09-14T23:58:12.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:12.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376296+0000", 2024-09-14T23:58:12.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4229999999999999e-05 2024-09-14T23:58:12.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:12.769 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376303+0000", 2024-09-14T23:58:12.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2199999999999997e-06 2024-09-14T23:58:12.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.769 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:12.769 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427031+0000", 2024-09-14T23:58:12.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050728154999999997 2024-09-14T23:58:12.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.769 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:12.769 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427036+0000", 2024-09-14T23:58:12.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6809999999999997e-06 2024-09-14T23:58:12.770 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.770 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:12.770 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.770 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5899 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:12.770 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376338+0000", 2024-09-14T23:58:12.770 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 93.311494053999994, 2024-09-14T23:58:12.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050712100000000003, 2024-09-14T23:58:12.770 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:12.771 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:12.771 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:12.771 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:12.771 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:12.771 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:12.771 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5899 2024-09-14T23:58:12.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.771 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:12.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:12.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376338+0000", 2024-09-14T23:58:12.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:12.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376338+0000", 2024-09-14T23:58:12.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:12.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376339+0000", 2024-09-14T23:58:12.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9100000000000003e-07 2024-09-14T23:58:12.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:12.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376342+0000", 2024-09-14T23:58:12.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7e-06 2024-09-14T23:58:12.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:12.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376350+0000", 2024-09-14T23:58:12.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5280000000000003e-06 2024-09-14T23:58:12.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:12.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376356+0000", 2024-09-14T23:58:12.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6010000000000002e-06 2024-09-14T23:58:12.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:12.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427047+0000", 2024-09-14T23:58:12.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050690962999999999 2024-09-14T23:58:12.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:12.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427050+0000", 2024-09-14T23:58:12.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.517e-06 2024-09-14T23:58:12.774 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.774 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:12.774 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.775 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5900 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:12.775 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376366+0000", 2024-09-14T23:58:12.775 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 93.311465795000004, 2024-09-14T23:58:12.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050697858999999998, 2024-09-14T23:58:12.775 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:12.775 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:12.775 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:12.775 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:12.775 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:12.775 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:12.776 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5900 2024-09-14T23:58:12.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.776 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:12.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:12.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376366+0000", 2024-09-14T23:58:12.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:12.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376366+0000", 2024-09-14T23:58:12.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.777 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:12.777 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377650+0000", 2024-09-14T23:58:12.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001283834 2024-09-14T23:58:12.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.777 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:12.777 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377655+0000", 2024-09-14T23:58:12.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7230000000000004e-06 2024-09-14T23:58:12.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:12.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377667+0000", 2024-09-14T23:58:12.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1746000000000001e-05 2024-09-14T23:58:12.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:12.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377676+0000", 2024-09-14T23:58:12.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3619999999999995e-06 2024-09-14T23:58:12.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:12.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427061+0000", 2024-09-14T23:58:12.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049384755000000002 2024-09-14T23:58:12.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:12.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427064+0000", 2024-09-14T23:58:12.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.439e-06 2024-09-14T23:58:12.779 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.779 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:12.779 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.780 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:132622 3.1e 3:79d0a5e0:::rbd_data.111847077d5e.0000000000000007:head [sparse-read 0~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T23:58:12.780 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:44.037901+0000", 2024-09-14T23:58:12.780 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.649931186, 2024-09-14T23:58:12.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1083420859999999, 2024-09-14T23:58:12.780 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:12.780 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:12.780 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:12.780 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:12.780 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:12.780 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:12.780 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 132622 2024-09-14T23:58:12.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.781 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:12.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.781 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:12.781 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037901+0000", 2024-09-14T23:58:12.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.781 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:12.781 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037901+0000", 2024-09-14T23:58:12.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:12.782 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037903+0000", 2024-09-14T23:58:12.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4619999999999999e-06 2024-09-14T23:58:12.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:12.782 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037910+0000", 2024-09-14T23:58:12.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7449999999999995e-06 2024-09-14T23:58:12.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:12.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037928+0000", 2024-09-14T23:58:12.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7305000000000001e-05 2024-09-14T23:58:12.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:12.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037939+0000", 2024-09-14T23:58:12.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1599e-05 2024-09-14T23:58:12.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:12.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.038049+0000", 2024-09-14T23:58:12.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000109577 2024-09-14T23:58:12.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op must wait for map", 2024-09-14T23:58:12.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.038057+0000", 2024-09-14T23:58:12.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1450000000000008e-06 2024-09-14T23:58:12.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:12.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.039937+0000", 2024-09-14T23:58:12.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00188024 2024-09-14T23:58:12.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:58:12.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.040039+0000", 2024-09-14T23:58:12.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000101934 2024-09-14T23:58:12.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:12.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:45.145805+0000", 2024-09-14T23:58:12.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.105765651 2024-09-14T23:58:12.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:12.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:45.145997+0000", 2024-09-14T23:58:12.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000192084 2024-09-14T23:58:12.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:12.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:45.146243+0000", 2024-09-14T23:58:12.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024634400000000001 2024-09-14T23:58:12.786 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.786 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:12.786 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.786 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4172.0:5 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:12.787 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.482606+0000", 2024-09-14T23:58:12.787 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.2052262049999998, 2024-09-14T23:58:12.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.49381641300000001, 2024-09-14T23:58:12.787 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:12.787 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:12.787 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:12.787 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:12.787 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4172", 2024-09-14T23:58:12.787 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/2542269203", 2024-09-14T23:58:12.787 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5 2024-09-14T23:58:12.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.787 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:12.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:12.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482606+0000", 2024-09-14T23:58:12.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:12.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482606+0000", 2024-09-14T23:58:12.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:12.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:12.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482607+0000", 2024-09-14T23:58:12.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3120000000000001e-06 2024-09-14T23:58:12.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:12.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482615+0000", 2024-09-14T23:58:12.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9300000000000003e-06 2024-09-14T23:58:12.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:12.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482632+0000", 2024-09-14T23:58:12.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6966999999999999e-05 2024-09-14T23:58:12.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:12.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482640+0000", 2024-09-14T23:58:12.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1820000000000006e-06 2024-09-14T23:58:12.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:12.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482703+0000", 2024-09-14T23:58:12.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2098000000000001e-05 2024-09-14T23:58:12.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:58:12.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482713+0000", 2024-09-14T23:58:12.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0064000000000001e-05 2024-09-14T23:58:12.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:12.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.976307+0000", 2024-09-14T23:58:12.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.49359442999999997 2024-09-14T23:58:12.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:12.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:12.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:12.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.976422+0000", 2024-09-14T23:58:12.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011543 2024-09-14T23:58:12.791 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.791 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:12.791 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.792 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:12.792 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:12.792 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:12.793 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T23:58:12.794 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:12.792+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:12.894 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:12.892+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:12.941 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:12.941 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:12.942 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:58:12.942 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:12.950 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T23:58:13.034 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:13.033+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:13.096 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:13.094+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:13.098 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:13.098 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:13.098 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:58:13.098 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:58:13.098 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:13.108 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T23:58:13.155 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T23:58:13.191 INFO:teuthology.orchestra.run.smithi042.stderr:set osd.0 primary-affinity to 0.323135 (8211762) 2024-09-14T23:58:13.210 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:13.208+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:13.266 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:13.267 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:58:13.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:58:13.267 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:58:13.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.267 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:3578 3.4 e29/23 3:2298d860:::rbd_data.1118de1168b.0000000000000030:head v 29'15, mlcod=29'13)", 2024-09-14T23:58:13.267 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.036455+0000", 2024-09-14T23:58:13.267 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 94.227169779999997, 2024-09-14T23:58:13.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019449440000000001, 2024-09-14T23:58:13.267 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:13.267 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:13.267 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:13.267 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:13.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:13.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036455+0000", 2024-09-14T23:58:13.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:13.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036455+0000", 2024-09-14T23:58:13.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:13.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036456+0000", 2024-09-14T23:58:13.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0999999999999997e-07 2024-09-14T23:58:13.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:13.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036463+0000", 2024-09-14T23:58:13.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2200000000000003e-06 2024-09-14T23:58:13.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:13.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036478+0000", 2024-09-14T23:58:13.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4977000000000001e-05 2024-09-14T23:58:13.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:13.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036485+0000", 2024-09-14T23:58:13.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0559999999999999e-06 2024-09-14T23:58:13.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:13.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036693+0000", 2024-09-14T23:58:13.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000208051 2024-09-14T23:58:13.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:13.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036715+0000", 2024-09-14T23:58:13.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1330000000000001e-05 2024-09-14T23:58:13.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:13.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.038356+0000", 2024-09-14T23:58:13.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001641586 2024-09-14T23:58:13.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.271 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:13.271 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.038400+0000", 2024-09-14T23:58:13.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3914e-05 2024-09-14T23:58:13.271 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.271 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:13.271 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.272 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:3579 3.4 e29/23 3:27290fe6:::rbd_data.1118de1168b.0000000000000032:head v 29'16, mlcod=29'13)", 2024-09-14T23:58:13.272 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.037085+0000", 2024-09-14T23:58:13.272 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 94.226540189999994, 2024-09-14T23:58:13.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0020019220000000002, 2024-09-14T23:58:13.272 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:13.272 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:13.272 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:13.272 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:13.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:13.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037085+0000", 2024-09-14T23:58:13.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:13.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037085+0000", 2024-09-14T23:58:13.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:13.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037085+0000", 2024-09-14T23:58:13.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5499999999999997e-07 2024-09-14T23:58:13.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:13.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037091+0000", 2024-09-14T23:58:13.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7869999999999998e-06 2024-09-14T23:58:13.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:13.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037102+0000", 2024-09-14T23:58:13.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.031e-05 2024-09-14T23:58:13.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:13.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037116+0000", 2024-09-14T23:58:13.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4119e-05 2024-09-14T23:58:13.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:13.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037219+0000", 2024-09-14T23:58:13.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010372799999999999 2024-09-14T23:58:13.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:13.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037239+0000", 2024-09-14T23:58:13.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9993e-05 2024-09-14T23:58:13.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:13.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.039046+0000", 2024-09-14T23:58:13.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001806666 2024-09-14T23:58:13.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:13.276 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.039087+0000", 2024-09-14T23:58:13.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0564e-05 2024-09-14T23:58:13.276 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.276 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:13.276 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.276 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.276 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:32024 3.9 e39/23 3:94dc260b:::rbd_data.111814b482f8.0000000000000009:head v 39'45, mlcod=38'43)", 2024-09-14T23:58:13.276 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:49.061018+0000", 2024-09-14T23:58:13.276 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 84.202606850999999, 2024-09-14T23:58:13.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018482240000000001, 2024-09-14T23:58:13.276 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:13.276 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:13.276 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:13.277 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:13.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:13.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061018+0000", 2024-09-14T23:58:13.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:13.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061018+0000", 2024-09-14T23:58:13.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:13.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061018+0000", 2024-09-14T23:58:13.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9999999999999998e-07 2024-09-14T23:58:13.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:13.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061211+0000", 2024-09-14T23:58:13.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019206 2024-09-14T23:58:13.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:13.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061221+0000", 2024-09-14T23:58:13.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0659000000000001e-05 2024-09-14T23:58:13.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:13.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061230+0000", 2024-09-14T23:58:13.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4610000000000008e-06 2024-09-14T23:58:13.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:13.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061317+0000", 2024-09-14T23:58:13.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7514000000000003e-05 2024-09-14T23:58:13.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:13.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061338+0000", 2024-09-14T23:58:13.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0395999999999999e-05 2024-09-14T23:58:13.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.280 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:13.280 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.062830+0000", 2024-09-14T23:58:13.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0014921940000000001 2024-09-14T23:58:13.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.280 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:13.280 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.062866+0000", 2024-09-14T23:58:13.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6539999999999999e-05 2024-09-14T23:58:13.280 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.280 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:13.280 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.281 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:32420 3.9 e39/23 3:91dc34e7:::rbd_data.111814b482f8.000000000000001e:head v 39'46, mlcod=38'44)", 2024-09-14T23:58:13.281 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:49.086124+0000", 2024-09-14T23:58:13.281 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 84.177501351999993, 2024-09-14T23:58:13.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001686819, 2024-09-14T23:58:13.281 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:13.281 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:13.281 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:13.281 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:13.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:13.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086124+0000", 2024-09-14T23:58:13.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:13.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086124+0000", 2024-09-14T23:58:13.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:13.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086125+0000", 2024-09-14T23:58:13.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.361e-06 2024-09-14T23:58:13.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:13.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086605+0000", 2024-09-14T23:58:13.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00048026600000000003 2024-09-14T23:58:13.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:13.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086626+0000", 2024-09-14T23:58:13.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.037e-05 2024-09-14T23:58:13.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:13.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086635+0000", 2024-09-14T23:58:13.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5880000000000004e-06 2024-09-14T23:58:13.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:13.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086701+0000", 2024-09-14T23:58:13.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5905e-05 2024-09-14T23:58:13.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:13.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086727+0000", 2024-09-14T23:58:13.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5573000000000001e-05 2024-09-14T23:58:13.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:13.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.087767+0000", 2024-09-14T23:58:13.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001040537 2024-09-14T23:58:13.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.285 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:13.285 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.087810+0000", 2024-09-14T23:58:13.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3219000000000001e-05 2024-09-14T23:58:13.285 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.285 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:13.285 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.285 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:50582 3.9 e51/23 3:92198e7a:::rbd_data.11183b9aac34.000000000000000a:head v 51'63, mlcod=50'60)", 2024-09-14T23:58:13.285 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:01.131541+0000", 2024-09-14T23:58:13.285 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 72.132084160999995, 2024-09-14T23:58:13.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001795547, 2024-09-14T23:58:13.285 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:13.285 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:13.285 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:13.286 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:13.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:13.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131541+0000", 2024-09-14T23:58:13.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:13.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131541+0000", 2024-09-14T23:58:13.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:13.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131541+0000", 2024-09-14T23:58:13.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7099999999999997e-07 2024-09-14T23:58:13.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.287 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:13.287 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131548+0000", 2024-09-14T23:58:13.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.708e-06 2024-09-14T23:58:13.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.287 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:13.287 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131570+0000", 2024-09-14T23:58:13.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1815999999999999e-05 2024-09-14T23:58:13.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.287 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:13.287 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131581+0000", 2024-09-14T23:58:13.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0933e-05 2024-09-14T23:58:13.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:13.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131660+0000", 2024-09-14T23:58:13.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9071000000000002e-05 2024-09-14T23:58:13.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:13.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131682+0000", 2024-09-14T23:58:13.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2534000000000001e-05 2024-09-14T23:58:13.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:13.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.133287+0000", 2024-09-14T23:58:13.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016045510000000001 2024-09-14T23:58:13.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:13.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.133336+0000", 2024-09-14T23:58:13.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9563e-05 2024-09-14T23:58:13.289 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.289 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:13.289 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.289 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:51281 3.0 e52/23 3:04e60ec2:::rbd_data.11181975a1de.0000000000000003:head v 52'113, mlcod=44'111)", 2024-09-14T23:58:13.289 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:02.158822+0000", 2024-09-14T23:58:13.290 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 71.104802664999994, 2024-09-14T23:58:13.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019860609999999999, 2024-09-14T23:58:13.290 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:13.290 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:13.290 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:13.290 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:13.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:13.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.158822+0000", 2024-09-14T23:58:13.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:13.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.158822+0000", 2024-09-14T23:58:13.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:13.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.158823+0000", 2024-09-14T23:58:13.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6800000000000001e-07 2024-09-14T23:58:13.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:13.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.159132+0000", 2024-09-14T23:58:13.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030974900000000001 2024-09-14T23:58:13.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:13.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.159146+0000", 2024-09-14T23:58:13.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3261000000000001e-05 2024-09-14T23:58:13.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.292 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:13.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.159155+0000", 2024-09-14T23:58:13.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7410000000000004e-06 2024-09-14T23:58:13.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.292 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:13.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.159221+0000", 2024-09-14T23:58:13.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5698999999999996e-05 2024-09-14T23:58:13.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.292 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:13.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.159244+0000", 2024-09-14T23:58:13.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3108e-05 2024-09-14T23:58:13.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.293 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:13.293 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.160767+0000", 2024-09-14T23:58:13.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015231120000000001 2024-09-14T23:58:13.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.293 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:13.293 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.160808+0000", 2024-09-14T23:58:13.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1022999999999997e-05 2024-09-14T23:58:13.293 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.293 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:13.293 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.294 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:56107 3.0 e56/23 3:00c47890:::rbd_data.1118c4b3aebe.0000000000000009:head v 56'125, mlcod=52'123)", 2024-09-14T23:58:13.294 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.236123+0000", 2024-09-14T23:58:13.294 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 67.027501466000004, 2024-09-14T23:58:13.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0061273220000000002, 2024-09-14T23:58:13.294 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:13.294 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:13.294 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:13.294 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:13.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:13.294 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.236123+0000", 2024-09-14T23:58:13.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:13.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.236123+0000", 2024-09-14T23:58:13.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.295 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:13.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.236125+0000", 2024-09-14T23:58:13.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.313e-06 2024-09-14T23:58:13.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.295 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:13.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237428+0000", 2024-09-14T23:58:13.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013029890000000001 2024-09-14T23:58:13.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.295 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:13.296 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237452+0000", 2024-09-14T23:58:13.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4090000000000001e-05 2024-09-14T23:58:13.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.296 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:13.296 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237462+0000", 2024-09-14T23:58:13.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0421999999999999e-05 2024-09-14T23:58:13.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.296 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:13.296 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237559+0000", 2024-09-14T23:58:13.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6378000000000005e-05 2024-09-14T23:58:13.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:13.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237583+0000", 2024-09-14T23:58:13.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4182e-05 2024-09-14T23:58:13.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:13.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.242205+0000", 2024-09-14T23:58:13.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0046217289999999998 2024-09-14T23:58:13.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.297 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:13.297 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.242251+0000", 2024-09-14T23:58:13.297 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6218999999999999e-05 2024-09-14T23:58:13.298 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.298 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:13.298 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.298 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.298 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:56108 3.0 e56/23 3:00c47890:::rbd_data.1118c4b3aebe.0000000000000009:head v 56'127, mlcod=52'123)", 2024-09-14T23:58:13.298 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.239693+0000", 2024-09-14T23:58:13.298 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 67.023931996000002, 2024-09-14T23:58:13.298 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0043496209999999997, 2024-09-14T23:58:13.298 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:13.298 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:13.298 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:13.298 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:13.298 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.299 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:13.299 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239693+0000", 2024-09-14T23:58:13.299 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.299 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:13.299 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239693+0000", 2024-09-14T23:58:13.299 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.299 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:13.299 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239694+0000", 2024-09-14T23:58:13.299 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0499999999999999e-06 2024-09-14T23:58:13.300 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.300 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.300 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:13.300 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239702+0000", 2024-09-14T23:58:13.300 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1170000000000004e-06 2024-09-14T23:58:13.300 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.300 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.300 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:13.300 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239716+0000", 2024-09-14T23:58:13.300 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4306e-05 2024-09-14T23:58:13.300 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.300 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.301 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:13.301 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239724+0000", 2024-09-14T23:58:13.301 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0399999999999993e-06 2024-09-14T23:58:13.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.301 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:13.301 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239778+0000", 2024-09-14T23:58:13.301 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4032e-05 2024-09-14T23:58:13.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.301 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:13.301 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239801+0000", 2024-09-14T23:58:13.301 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2770000000000001e-05 2024-09-14T23:58:13.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.302 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.302 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:13.302 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.243990+0000", 2024-09-14T23:58:13.302 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0041890909999999998 2024-09-14T23:58:13.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.302 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.302 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:13.302 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.244043+0000", 2024-09-14T23:58:13.302 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2215000000000001e-05 2024-09-14T23:58:13.302 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.302 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:13.302 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.303 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.303 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:56385 3.9 e56/23 3:963d31c7:::rbd_data.1118c4b3aebe.0000000000000003:head v 56'87, mlcod=53'85)", 2024-09-14T23:58:13.303 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.283936+0000", 2024-09-14T23:58:13.303 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 66.979689085999993, 2024-09-14T23:58:13.303 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0063042080000000004, 2024-09-14T23:58:13.303 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:13.303 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:13.303 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:13.303 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:13.303 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.303 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:13.303 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.283936+0000", 2024-09-14T23:58:13.303 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.304 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.304 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:13.304 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.283936+0000", 2024-09-14T23:58:13.304 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.304 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.304 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:13.304 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.283937+0000", 2024-09-14T23:58:13.304 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1179999999999999e-06 2024-09-14T23:58:13.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.304 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.304 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:13.305 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286338+0000", 2024-09-14T23:58:13.305 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024010149999999998 2024-09-14T23:58:13.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.305 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.305 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:13.305 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286360+0000", 2024-09-14T23:58:13.305 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2212e-05 2024-09-14T23:58:13.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.305 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.305 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:13.305 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286370+0000", 2024-09-14T23:58:13.305 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5850000000000006e-06 2024-09-14T23:58:13.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.305 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.306 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:13.306 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286443+0000", 2024-09-14T23:58:13.306 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2757999999999997e-05 2024-09-14T23:58:13.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.306 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.306 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:13.306 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286466+0000", 2024-09-14T23:58:13.306 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3844000000000001e-05 2024-09-14T23:58:13.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.306 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.306 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:13.306 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.290197+0000", 2024-09-14T23:58:13.306 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003730176 2024-09-14T23:58:13.307 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.307 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.307 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:13.307 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.290240+0000", 2024-09-14T23:58:13.307 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.35e-05 2024-09-14T23:58:13.307 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.307 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:13.307 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.307 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.307 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.307 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:56386 3.9 e56/23 3:963d31c7:::rbd_data.1118c4b3aebe.0000000000000003:head v 56'89, mlcod=53'85)", 2024-09-14T23:58:13.307 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.287578+0000", 2024-09-14T23:58:13.307 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 66.976047359999995, 2024-09-14T23:58:13.307 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033580250000000002, 2024-09-14T23:58:13.307 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:13.308 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:13.308 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:13.308 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:13.308 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.308 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:13.308 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287578+0000", 2024-09-14T23:58:13.308 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.308 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.308 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:13.308 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287578+0000", 2024-09-14T23:58:13.308 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.309 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.309 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.309 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:13.309 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287579+0000", 2024-09-14T23:58:13.309 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.64e-07 2024-09-14T23:58:13.309 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.309 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.309 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:13.309 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287586+0000", 2024-09-14T23:58:13.309 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5310000000000004e-06 2024-09-14T23:58:13.309 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.309 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.309 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:13.309 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287597+0000", 2024-09-14T23:58:13.310 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0903999999999999e-05 2024-09-14T23:58:13.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.310 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:13.310 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287605+0000", 2024-09-14T23:58:13.310 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7889999999999995e-06 2024-09-14T23:58:13.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.310 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:13.310 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.288437+0000", 2024-09-14T23:58:13.310 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000831914 2024-09-14T23:58:13.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.310 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:13.311 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.288460+0000", 2024-09-14T23:58:13.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3410000000000001e-05 2024-09-14T23:58:13.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.311 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:13.311 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.290883+0000", 2024-09-14T23:58:13.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024230599999999999 2024-09-14T23:58:13.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.311 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:13.311 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.290936+0000", 2024-09-14T23:58:13.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2453000000000001e-05 2024-09-14T23:58:13.312 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.312 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:13.312 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.312 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.312 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.312 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:60299 3.9 e57/23 3:96f26b6f:::rbd_data.11188d356229.0000000000000009:head v 57'98, mlcod=56'96)", 2024-09-14T23:58:13.312 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:07.418336+0000", 2024-09-14T23:58:13.312 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.845288866999994, 2024-09-14T23:58:13.312 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022211459999999998, 2024-09-14T23:58:13.312 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:13.312 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:13.312 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:13.312 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:13.313 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.313 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:13.313 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.418336+0000", 2024-09-14T23:58:13.313 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.313 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.313 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.313 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:13.313 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.418336+0000", 2024-09-14T23:58:13.313 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.313 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.313 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.313 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:13.313 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.418337+0000", 2024-09-14T23:58:13.314 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7199999999999999e-07 2024-09-14T23:58:13.314 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.314 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.314 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:13.314 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.419396+0000", 2024-09-14T23:58:13.314 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010598319999999999 2024-09-14T23:58:13.314 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.314 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.314 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:13.314 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.419407+0000", 2024-09-14T23:58:13.314 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0956e-05 2024-09-14T23:58:13.314 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.314 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.315 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:13.315 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.419417+0000", 2024-09-14T23:58:13.315 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0040999999999999e-05 2024-09-14T23:58:13.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.315 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:13.315 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.419468+0000", 2024-09-14T23:58:13.315 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0497000000000001e-05 2024-09-14T23:58:13.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.315 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:13.315 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.419485+0000", 2024-09-14T23:58:13.315 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6966e-05 2024-09-14T23:58:13.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.316 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.316 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:13.316 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.420527+0000", 2024-09-14T23:58:13.316 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001042133 2024-09-14T23:58:13.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.316 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.316 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:13.316 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.420557+0000", 2024-09-14T23:58:13.316 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0249000000000001e-05 2024-09-14T23:58:13.316 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.316 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:13.316 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.316 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.317 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:64263 3.4 e57/23 3:20c6a768:::rbd_data.11188d356229.0000000000000001:head v 57'165, mlcod=57'163)", 2024-09-14T23:58:13.317 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:07.999031+0000", 2024-09-14T23:58:13.317 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.264594212000006, 2024-09-14T23:58:13.317 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023815210000000002, 2024-09-14T23:58:13.317 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:13.317 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:13.317 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:13.317 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:13.317 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.317 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:13.317 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999031+0000", 2024-09-14T23:58:13.317 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.317 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.318 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:13.318 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999031+0000", 2024-09-14T23:58:13.318 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.318 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.318 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:13.318 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999032+0000", 2024-09-14T23:58:13.318 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.176e-06 2024-09-14T23:58:13.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.318 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.318 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:13.318 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999660+0000", 2024-09-14T23:58:13.318 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00062817099999999996 2024-09-14T23:58:13.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.318 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.319 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:13.319 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999683+0000", 2024-09-14T23:58:13.319 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2816999999999999e-05 2024-09-14T23:58:13.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.319 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.319 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:13.319 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999693+0000", 2024-09-14T23:58:13.319 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0088999999999999e-05 2024-09-14T23:58:13.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.319 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.319 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:13.319 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999767+0000", 2024-09-14T23:58:13.319 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4056999999999999e-05 2024-09-14T23:58:13.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.320 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.320 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:13.320 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999793+0000", 2024-09-14T23:58:13.320 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5573000000000001e-05 2024-09-14T23:58:13.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.320 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.320 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:13.320 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:08.001379+0000", 2024-09-14T23:58:13.320 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015859019999999999 2024-09-14T23:58:13.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.320 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.320 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:13.320 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:08.001412+0000", 2024-09-14T23:58:13.320 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3735999999999997e-05 2024-09-14T23:58:13.320 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.321 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:13.321 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.321 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.321 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:66570 3.0 e60/23 3:055350b3:::rbd_data.111873792d9b.0000000000000005:head v 60'162, mlcod=57'159)", 2024-09-14T23:58:13.321 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:11.277517+0000", 2024-09-14T23:58:13.321 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 61.986108055999999, 2024-09-14T23:58:13.321 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003472199, 2024-09-14T23:58:13.321 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:13.321 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:13.321 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:13.321 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:13.321 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.321 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:13.322 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.277517+0000", 2024-09-14T23:58:13.322 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.322 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.322 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:13.322 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.277517+0000", 2024-09-14T23:58:13.322 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.322 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.322 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:13.322 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.277518+0000", 2024-09-14T23:58:13.322 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1379999999999999e-06 2024-09-14T23:58:13.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.322 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.322 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:13.323 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278504+0000", 2024-09-14T23:58:13.323 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00098617499999999994 2024-09-14T23:58:13.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.323 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.323 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:13.323 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278525+0000", 2024-09-14T23:58:13.323 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.09e-05 2024-09-14T23:58:13.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.323 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.323 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:13.323 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278534+0000", 2024-09-14T23:58:13.323 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6209999999999993e-06 2024-09-14T23:58:13.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.324 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:13.324 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278611+0000", 2024-09-14T23:58:13.324 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6926000000000002e-05 2024-09-14T23:58:13.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.324 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:13.324 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278641+0000", 2024-09-14T23:58:13.324 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0861999999999997e-05 2024-09-14T23:58:13.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.324 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:13.324 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.280953+0000", 2024-09-14T23:58:13.324 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023114239999999999 2024-09-14T23:58:13.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.325 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:13.325 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.280989+0000", 2024-09-14T23:58:13.325 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6152999999999998e-05 2024-09-14T23:58:13.325 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.325 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:13.325 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.325 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:70526 3.4 e64/23 3:2649db36:::rbd_data.111831f32e80.0000000000000058:head v 64'183, mlcod=64'181)", 2024-09-14T23:58:13.325 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:15.442503+0000", 2024-09-14T23:58:13.325 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 57.821122164000002, 2024-09-14T23:58:13.325 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001769837, 2024-09-14T23:58:13.325 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:13.326 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:13.326 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:13.326 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:13.326 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.326 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:13.326 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.442503+0000", 2024-09-14T23:58:13.326 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.326 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.326 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:13.326 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.442503+0000", 2024-09-14T23:58:13.326 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.326 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.326 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:13.327 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.442503+0000", 2024-09-14T23:58:13.327 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1099999999999995e-07 2024-09-14T23:58:13.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.327 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.327 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:13.327 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.442620+0000", 2024-09-14T23:58:13.327 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000116618 2024-09-14T23:58:13.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.327 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.327 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:13.327 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.442634+0000", 2024-09-14T23:58:13.327 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3984e-05 2024-09-14T23:58:13.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.327 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.327 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:13.328 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.442641+0000", 2024-09-14T23:58:13.328 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2540000000000003e-06 2024-09-14T23:58:13.328 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.328 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.328 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:13.328 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.442732+0000", 2024-09-14T23:58:13.328 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0480000000000001e-05 2024-09-14T23:58:13.328 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.328 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.328 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:13.328 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.442753+0000", 2024-09-14T23:58:13.328 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1571999999999999e-05 2024-09-14T23:58:13.328 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.328 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.329 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:13.329 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.444236+0000", 2024-09-14T23:58:13.329 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0014825439999999999 2024-09-14T23:58:13.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.329 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.329 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:13.329 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.444273+0000", 2024-09-14T23:58:13.329 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6674000000000002e-05 2024-09-14T23:58:13.329 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.329 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:13.329 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.329 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.329 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:106629 3.4 e70/23 3:240d85c6:::rbd_data.11187de8b0a5.0000000000000009:head v 70'233, mlcod=69'231)", 2024-09-14T23:58:13.329 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:21.489517+0000", 2024-09-14T23:58:13.330 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 51.774107530999999, 2024-09-14T23:58:13.330 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016279840000000001, 2024-09-14T23:58:13.330 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:13.330 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:13.330 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:13.330 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:13.330 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.330 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:13.330 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.489517+0000", 2024-09-14T23:58:13.330 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.330 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.330 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:13.330 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.489517+0000", 2024-09-14T23:58:13.330 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.331 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.331 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:13.331 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.489518+0000", 2024-09-14T23:58:13.331 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.029e-06 2024-09-14T23:58:13.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.331 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.331 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:13.331 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.489840+0000", 2024-09-14T23:58:13.331 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00032164100000000001 2024-09-14T23:58:13.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.331 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.331 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:13.331 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.489859+0000", 2024-09-14T23:58:13.331 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8828999999999999e-05 2024-09-14T23:58:13.332 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.332 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.332 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:13.332 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.489869+0000", 2024-09-14T23:58:13.332 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6600000000000007e-06 2024-09-14T23:58:13.332 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.332 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.332 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:13.332 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.489934+0000", 2024-09-14T23:58:13.332 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5357000000000005e-05 2024-09-14T23:58:13.332 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.332 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.332 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:13.332 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.489951+0000", 2024-09-14T23:58:13.332 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6951000000000001e-05 2024-09-14T23:58:13.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.333 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.333 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:13.333 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.491108+0000", 2024-09-14T23:58:13.333 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011571940000000001 2024-09-14T23:58:13.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.333 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.333 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:13.333 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.491145+0000", 2024-09-14T23:58:13.333 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7323000000000002e-05 2024-09-14T23:58:13.333 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.333 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:13.333 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.334 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.334 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.334 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:108739 3.9 e80/23 3:9372b409:::rbd_data.111867e26b91.0000000000000005:head v 80'172, mlcod=78'170)", 2024-09-14T23:58:13.334 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:31.064099+0000", 2024-09-14T23:58:13.334 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.199526278, 2024-09-14T23:58:13.334 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001688174, 2024-09-14T23:58:13.334 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:13.334 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:13.334 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:13.334 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:13.334 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.334 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:13.334 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:31.064099+0000", 2024-09-14T23:58:13.334 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.335 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.335 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.335 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:13.335 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:31.064099+0000", 2024-09-14T23:58:13.335 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.335 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.335 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.335 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:13.335 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:31.064099+0000", 2024-09-14T23:58:13.335 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.44e-07 2024-09-14T23:58:13.335 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.335 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.335 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:13.335 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:31.064453+0000", 2024-09-14T23:58:13.336 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00035374899999999999 2024-09-14T23:58:13.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.336 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:13.336 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:31.064468+0000", 2024-09-14T23:58:13.336 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4476e-05 2024-09-14T23:58:13.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.336 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:13.336 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:31.064475+0000", 2024-09-14T23:58:13.336 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9289999999999998e-06 2024-09-14T23:58:13.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.336 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:13.337 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:31.064530+0000", 2024-09-14T23:58:13.337 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4401000000000003e-05 2024-09-14T23:58:13.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.337 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.337 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:13.337 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:31.064551+0000", 2024-09-14T23:58:13.337 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0996e-05 2024-09-14T23:58:13.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.337 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.337 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:13.337 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:31.065748+0000", 2024-09-14T23:58:13.337 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011967449999999999 2024-09-14T23:58:13.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.337 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.338 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:13.338 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:31.065787+0000", 2024-09-14T23:58:13.338 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9233999999999997e-05 2024-09-14T23:58:13.338 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.338 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:13.338 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.338 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.338 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.338 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "pg_update_log_missing(3.9 epoch 84/23 rep_tid 2234 entries 84'200 (0'0) error 3:93b58b1d:::rbd_data.1118417a1b0b.000000000000001b:head by client.4376.0:118637 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 83'197)", 2024-09-14T23:58:13.338 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:35.103156+0000", 2024-09-14T23:58:13.338 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 38.160468782999999, 2024-09-14T23:58:13.338 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019852009999999998, 2024-09-14T23:58:13.338 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:13.338 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:13.338 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:13.339 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:13.339 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.339 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:13.339 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:35.103156+0000", 2024-09-14T23:58:13.339 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.339 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.339 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:13.339 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:35.103156+0000", 2024-09-14T23:58:13.339 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.339 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.339 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:13.339 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:35.103157+0000", 2024-09-14T23:58:13.339 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1969999999999999e-06 2024-09-14T23:58:13.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.340 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.340 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:13.340 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:35.103170+0000", 2024-09-14T23:58:13.340 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2921e-05 2024-09-14T23:58:13.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.340 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.340 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:13.340 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:35.103202+0000", 2024-09-14T23:58:13.340 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1992000000000001e-05 2024-09-14T23:58:13.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.340 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.340 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:13.340 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:35.103220+0000", 2024-09-14T23:58:13.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7312999999999998e-05 2024-09-14T23:58:13.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.341 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.341 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:13.341 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:35.103305+0000", 2024-09-14T23:58:13.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4975000000000005e-05 2024-09-14T23:58:13.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.341 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.341 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:13.341 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:35.105141+0000", 2024-09-14T23:58:13.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018368029999999999 2024-09-14T23:58:13.341 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.341 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:13.341 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.342 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.342 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:142751 3.4 e99/23 3:265f7763:::rbd_data.1118f9c96a7.0000000000000004:head v 99'441, mlcod=98'438)", 2024-09-14T23:58:13.342 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:53.165301+0000", 2024-09-14T23:58:13.342 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.098323633, 2024-09-14T23:58:13.342 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0021754449999999998, 2024-09-14T23:58:13.342 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:13.342 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:13.342 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:13.342 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:13.342 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.342 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:13.342 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.165301+0000", 2024-09-14T23:58:13.343 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.343 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:13.343 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.165301+0000", 2024-09-14T23:58:13.343 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.343 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:13.343 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.165302+0000", 2024-09-14T23:58:13.343 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0899999999999997e-07 2024-09-14T23:58:13.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.344 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:13.344 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.165310+0000", 2024-09-14T23:58:13.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7520000000000002e-06 2024-09-14T23:58:13.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.344 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:13.344 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.165321+0000", 2024-09-14T23:58:13.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0668e-05 2024-09-14T23:58:13.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.344 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:13.344 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.165330+0000", 2024-09-14T23:58:13.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.234e-06 2024-09-14T23:58:13.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:13.345 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.165393+0000", 2024-09-14T23:58:13.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2716000000000001e-05 2024-09-14T23:58:13.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:13.345 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.165418+0000", 2024-09-14T23:58:13.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4544000000000001e-05 2024-09-14T23:58:13.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:13.345 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.167436+0000", 2024-09-14T23:58:13.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002018597 2024-09-14T23:58:13.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.346 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:13.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.167477+0000", 2024-09-14T23:58:13.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0624999999999998e-05 2024-09-14T23:58:13.346 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.346 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:13.346 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.346 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:158983 3.4 e103/23 3:20e43b29:::rbd_data.1118b2f495e6.0000000000000012:head v 103'454, mlcod=103'452)", 2024-09-14T23:58:13.346 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:58.917699+0000", 2024-09-14T23:58:13.346 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.345926005999999, 2024-09-14T23:58:13.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037933799999999998, 2024-09-14T23:58:13.346 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:13.347 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:13.347 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:13.347 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:13.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.347 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:13.347 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917699+0000", 2024-09-14T23:58:13.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.348 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:13.348 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917699+0000", 2024-09-14T23:58:13.348 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.348 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:13.348 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917700+0000", 2024-09-14T23:58:13.348 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3850000000000001e-06 2024-09-14T23:58:13.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.348 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:13.348 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917939+0000", 2024-09-14T23:58:13.349 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000238249 2024-09-14T23:58:13.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.349 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:13.349 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917961+0000", 2024-09-14T23:58:13.349 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2894000000000001e-05 2024-09-14T23:58:13.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.349 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:13.349 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917972+0000", 2024-09-14T23:58:13.349 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0261000000000001e-05 2024-09-14T23:58:13.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.349 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:13.350 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.918038+0000", 2024-09-14T23:58:13.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6223000000000006e-05 2024-09-14T23:58:13.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.350 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:13.350 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.918063+0000", 2024-09-14T23:58:13.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5252000000000001e-05 2024-09-14T23:58:13.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.350 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:13.350 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.921433+0000", 2024-09-14T23:58:13.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033699960000000001 2024-09-14T23:58:13.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.350 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:13.351 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.921492+0000", 2024-09-14T23:58:13.351 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9120000000000003e-05 2024-09-14T23:58:13.351 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.351 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:13.351 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.351 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4172.0:6 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T23:58:13.351 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.479840+0000", 2024-09-14T23:58:13.351 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.7837847260000004, 2024-09-14T23:58:13.351 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022238340000000001, 2024-09-14T23:58:13.351 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:13.351 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:13.351 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:13.352 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:13.352 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4172", 2024-09-14T23:58:13.352 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/2542269203", 2024-09-14T23:58:13.352 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 6 2024-09-14T23:58:13.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.352 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:13.352 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.352 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:13.352 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.479840+0000", 2024-09-14T23:58:13.352 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.352 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.352 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:13.352 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.479840+0000", 2024-09-14T23:58:13.353 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:13.353 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.479843+0000", 2024-09-14T23:58:13.353 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6680000000000001e-06 2024-09-14T23:58:13.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:13.353 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.479862+0000", 2024-09-14T23:58:13.353 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8762000000000001e-05 2024-09-14T23:58:13.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:13.353 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.479881+0000", 2024-09-14T23:58:13.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9860999999999998e-05 2024-09-14T23:58:13.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.354 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:13.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.479899+0000", 2024-09-14T23:58:13.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7127000000000001e-05 2024-09-14T23:58:13.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.354 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:13.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.481969+0000", 2024-09-14T23:58:13.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002070166 2024-09-14T23:58:13.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.355 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:13.355 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482064+0000", 2024-09-14T23:58:13.355 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5249999999999998e-05 2024-09-14T23:58:13.355 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.355 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:13.355 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.355 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.355 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:13.355 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:13.357 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T23:58:13.357 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:13.355+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:13.458 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:13.456+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:13.507 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:13.507 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:13.507 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:58:13.507 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:13.516 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T23:58:13.559 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:13.557+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:13.624 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:13.624 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:13.624 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:58:13.624 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:58:13.624 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:13.633 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T23:58:13.677 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:13.675+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:13.756 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:13.756 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:58:13.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:58:13.756 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:58:13.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.756 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5200 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2170880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:13.757 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253394+0000", 2024-09-14T23:58:13.757 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 94.500565140000006, 2024-09-14T23:58:13.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057428154000000002, 2024-09-14T23:58:13.757 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:13.757 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:13.757 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:13.757 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:13.757 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:13.757 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:13.757 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5200 2024-09-14T23:58:13.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.757 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:13.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.757 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:13.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253394+0000", 2024-09-14T23:58:13.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.758 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:13.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253394+0000", 2024-09-14T23:58:13.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.758 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:13.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253395+0000", 2024-09-14T23:58:13.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9100000000000004e-07 2024-09-14T23:58:13.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.758 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:13.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253397+0000", 2024-09-14T23:58:13.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2790000000000001e-06 2024-09-14T23:58:13.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:13.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253404+0000", 2024-09-14T23:58:13.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7959999999999998e-06 2024-09-14T23:58:13.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:13.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253409+0000", 2024-09-14T23:58:13.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1460000000000004e-06 2024-09-14T23:58:13.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:13.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310627+0000", 2024-09-14T23:58:13.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057218160999999997 2024-09-14T23:58:13.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.760 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:13.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310710+0000", 2024-09-14T23:58:13.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3102999999999994e-05 2024-09-14T23:58:13.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.760 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:13.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310822+0000", 2024-09-14T23:58:13.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000112178 2024-09-14T23:58:13.760 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.760 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:13.760 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.761 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5201 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2232320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:13.761 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:58:13.761 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 94.500540467999997, 2024-09-14T23:58:13.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057643228999999997, 2024-09-14T23:58:13.761 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:13.761 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:13.761 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:13.761 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:13.761 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:13.761 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:13.761 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5201 2024-09-14T23:58:13.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.761 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:13.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:13.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:58:13.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:13.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:58:13.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:13.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:58:13.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0699999999999998e-07 2024-09-14T23:58:13.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:13.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253422+0000", 2024-09-14T23:58:13.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.514e-06 2024-09-14T23:58:13.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:13.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253431+0000", 2024-09-14T23:58:13.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1919999999999994e-06 2024-09-14T23:58:13.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:13.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253436+0000", 2024-09-14T23:58:13.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5350000000000004e-06 2024-09-14T23:58:13.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.764 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:13.764 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310856+0000", 2024-09-14T23:58:13.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057419381999999998 2024-09-14T23:58:13.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.764 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:13.764 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310953+0000", 2024-09-14T23:58:13.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6774000000000006e-05 2024-09-14T23:58:13.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.764 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:13.765 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311062+0000", 2024-09-14T23:58:13.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000109425 2024-09-14T23:58:13.765 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.765 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:13.765 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.765 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5202 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2293760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:13.765 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:58:13.765 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 94.500515719999996, 2024-09-14T23:58:13.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057818255999999998, 2024-09-14T23:58:13.765 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:13.765 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:13.765 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:13.766 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:13.766 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:13.766 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:13.766 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5202 2024-09-14T23:58:13.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.766 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:13.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:13.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:58:13.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:13.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:58:13.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:13.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253573+0000", 2024-09-14T23:58:13.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012934000000000001 2024-09-14T23:58:13.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:13.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253576+0000", 2024-09-14T23:58:13.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5310000000000002e-06 2024-09-14T23:58:13.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:13.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253587+0000", 2024-09-14T23:58:13.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0522e-05 2024-09-14T23:58:13.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:13.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253595+0000", 2024-09-14T23:58:13.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8290000000000008e-06 2024-09-14T23:58:13.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:13.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311093+0000", 2024-09-14T23:58:13.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057498312000000003 2024-09-14T23:58:13.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:13.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311167+0000", 2024-09-14T23:58:13.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3582000000000001e-05 2024-09-14T23:58:13.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.769 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:13.769 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311262+0000", 2024-09-14T23:58:13.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5140000000000001e-05 2024-09-14T23:58:13.769 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.769 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:13.769 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.769 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5203 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2355200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:13.769 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:58:13.770 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 94.500352977999995, 2024-09-14T23:58:13.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057903938000000002, 2024-09-14T23:58:13.770 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:13.770 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:13.770 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:13.770 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:13.770 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:13.770 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:13.770 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5203 2024-09-14T23:58:13.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.770 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:13.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.770 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:13.770 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:58:13.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:13.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:58:13.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:13.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253902+0000", 2024-09-14T23:58:13.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029615400000000002 2024-09-14T23:58:13.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:13.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253906+0000", 2024-09-14T23:58:13.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9249999999999997e-06 2024-09-14T23:58:13.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:13.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253913+0000", 2024-09-14T23:58:13.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9099999999999999e-06 2024-09-14T23:58:13.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:13.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253917+0000", 2024-09-14T23:58:13.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9210000000000002e-06 2024-09-14T23:58:13.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:13.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311295+0000", 2024-09-14T23:58:13.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057378268000000003 2024-09-14T23:58:13.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:13.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311402+0000", 2024-09-14T23:58:13.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010644100000000001 2024-09-14T23:58:13.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:13.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311510+0000", 2024-09-14T23:58:13.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108319 2024-09-14T23:58:13.774 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.774 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:13.774 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.774 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5204 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2416640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:13.774 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:58:13.774 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 94.500034780999997, 2024-09-14T23:58:13.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057774860999999997, 2024-09-14T23:58:13.774 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:13.774 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:13.774 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:13.774 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:13.774 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:13.774 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:13.775 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5204 2024-09-14T23:58:13.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.775 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:13.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:13.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:58:13.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.775 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:13.775 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:58:13.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:13.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255903+0000", 2024-09-14T23:58:13.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001978807 2024-09-14T23:58:13.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.777 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:13.777 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255907+0000", 2024-09-14T23:58:13.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3610000000000001e-06 2024-09-14T23:58:13.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.777 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:13.777 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255914+0000", 2024-09-14T23:58:13.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.357e-06 2024-09-14T23:58:13.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:13.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255918+0000", 2024-09-14T23:58:13.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1779999999999996e-06 2024-09-14T23:58:13.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:13.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311541+0000", 2024-09-14T23:58:13.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055622697999999998 2024-09-14T23:58:13.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:13.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311614+0000", 2024-09-14T23:58:13.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3102999999999994e-05 2024-09-14T23:58:13.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:13.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311699+0000", 2024-09-14T23:58:13.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5357000000000003e-05 2024-09-14T23:58:13.779 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.779 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:13.779 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.779 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5211 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2846720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:13.779 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:58:13.779 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 94.497936566999996, 2024-09-14T23:58:13.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057528151, 2024-09-14T23:58:13.779 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:13.780 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:13.780 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:13.780 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:13.780 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:13.780 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:13.780 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5211 2024-09-14T23:58:13.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.780 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:13.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:13.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:58:13.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.781 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:13.781 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:58:13.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.781 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:13.781 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:58:13.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1399999999999998e-07 2024-09-14T23:58:13.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.781 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:13.781 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256024+0000", 2024-09-14T23:58:13.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.114e-06 2024-09-14T23:58:13.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:13.782 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256028+0000", 2024-09-14T23:58:13.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0840000000000002e-06 2024-09-14T23:58:13.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:13.782 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256031+0000", 2024-09-14T23:58:13.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6819999999999999e-06 2024-09-14T23:58:13.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:13.782 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313317+0000", 2024-09-14T23:58:13.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057286295000000001 2024-09-14T23:58:13.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:13.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313403+0000", 2024-09-14T23:58:13.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6287999999999998e-05 2024-09-14T23:58:13.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:13.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313551+0000", 2024-09-14T23:58:13.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000147374 2024-09-14T23:58:13.783 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.783 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:13.783 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.783 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5212 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2908160~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:13.784 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:58:13.784 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 94.497925373000001, 2024-09-14T23:58:13.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057735679999999998, 2024-09-14T23:58:13.784 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:13.784 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:13.784 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:13.784 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:13.784 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:13.784 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:13.784 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5212 2024-09-14T23:58:13.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.784 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:13.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:13.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:58:13.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:13.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:58:13.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:13.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:58:13.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0900000000000001e-07 2024-09-14T23:58:13.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:13.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256035+0000", 2024-09-14T23:58:13.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1200000000000001e-06 2024-09-14T23:58:13.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:13.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256038+0000", 2024-09-14T23:58:13.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.05e-06 2024-09-14T23:58:13.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:13.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256041+0000", 2024-09-14T23:58:13.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.667e-06 2024-09-14T23:58:13.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:13.787 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313584+0000", 2024-09-14T23:58:13.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057543484999999998 2024-09-14T23:58:13.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:13.787 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313668+0000", 2024-09-14T23:58:13.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3807000000000006e-05 2024-09-14T23:58:13.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:13.787 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313770+0000", 2024-09-14T23:58:13.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010134200000000001 2024-09-14T23:58:13.788 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.788 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:13.788 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.788 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5213 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2969600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:13.788 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:58:13.788 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 94.497915285999994, 2024-09-14T23:58:13.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058145467999999999, 2024-09-14T23:58:13.788 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:13.788 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:13.788 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:13.788 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:13.788 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:13.789 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:13.789 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5213 2024-09-14T23:58:13.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.789 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:13.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:13.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:58:13.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:13.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:58:13.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:13.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257901+0000", 2024-09-14T23:58:13.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018567519999999999 2024-09-14T23:58:13.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:13.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257904+0000", 2024-09-14T23:58:13.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4850000000000001e-06 2024-09-14T23:58:13.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:13.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257912+0000", 2024-09-14T23:58:13.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2090000000000005e-06 2024-09-14T23:58:13.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:13.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257918+0000", 2024-09-14T23:58:13.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6899999999999997e-06 2024-09-14T23:58:13.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:13.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313816+0000", 2024-09-14T23:58:13.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055898075999999998 2024-09-14T23:58:13.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:13.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.314082+0000", 2024-09-14T23:58:13.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000265386 2024-09-14T23:58:13.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:13.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.314189+0000", 2024-09-14T23:58:13.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010787 2024-09-14T23:58:13.792 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.792 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:13.792 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.792 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5218 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3276800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:13.792 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:58:13.793 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 94.495988276999995, 2024-09-14T23:58:13.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057497557999999997, 2024-09-14T23:58:13.793 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:13.793 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:13.793 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:13.793 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:13.793 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:13.793 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:13.793 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5218 2024-09-14T23:58:13.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.793 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:13.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:13.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:58:13.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:13.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:58:13.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:13.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:58:13.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1899999999999999e-07 2024-09-14T23:58:13.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:13.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257972+0000", 2024-09-14T23:58:13.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.217e-06 2024-09-14T23:58:13.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:13.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257975+0000", 2024-09-14T23:58:13.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9359999999999999e-06 2024-09-14T23:58:13.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:13.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257978+0000", 2024-09-14T23:58:13.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1049999999999999e-06 2024-09-14T23:58:13.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:13.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315331+0000", 2024-09-14T23:58:13.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057352093999999999 2024-09-14T23:58:13.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.796 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:13.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315384+0000", 2024-09-14T23:58:13.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3780999999999999e-05 2024-09-14T23:58:13.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.796 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:13.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315468+0000", 2024-09-14T23:58:13.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4206e-05 2024-09-14T23:58:13.796 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.796 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:13.797 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.797 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5219 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3338240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:13.797 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:58:13.797 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 94.495977601000007, 2024-09-14T23:58:13.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057637764000000001, 2024-09-14T23:58:13.797 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:13.797 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:13.797 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:13.797 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:13.797 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:13.797 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:13.798 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5219 2024-09-14T23:58:13.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.798 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:13.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:13.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:58:13.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:13.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:58:13.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:13.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:58:13.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.03e-07 2024-09-14T23:58:13.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:13.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257983+0000", 2024-09-14T23:58:13.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9199999999999999e-07 2024-09-14T23:58:13.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:13.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257986+0000", 2024-09-14T23:58:13.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0860000000000002e-06 2024-09-14T23:58:13.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:13.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257989+0000", 2024-09-14T23:58:13.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6589999999999999e-06 2024-09-14T23:58:13.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:13.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315509+0000", 2024-09-14T23:58:13.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057520262000000003 2024-09-14T23:58:13.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:13.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315554+0000", 2024-09-14T23:58:13.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5015999999999997e-05 2024-09-14T23:58:13.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:13.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315619+0000", 2024-09-14T23:58:13.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5545999999999999e-05 2024-09-14T23:58:13.801 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.801 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:13.801 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.801 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5220 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3399680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:13.801 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:58:13.801 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 94.495967644999993, 2024-09-14T23:58:13.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057792128999999998, 2024-09-14T23:58:13.802 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:13.802 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:13.802 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:13.802 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:13.802 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:13.802 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:13.802 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5220 2024-09-14T23:58:13.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.802 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:13.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:13.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:58:13.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:13.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:58:13.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:13.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:58:13.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1799999999999999e-07 2024-09-14T23:58:13.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:13.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257993+0000", 2024-09-14T23:58:13.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6200000000000006e-07 2024-09-14T23:58:13.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:13.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257996+0000", 2024-09-14T23:58:13.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9050000000000001e-06 2024-09-14T23:58:13.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:13.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257998+0000", 2024-09-14T23:58:13.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5950000000000001e-06 2024-09-14T23:58:13.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:13.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315672+0000", 2024-09-14T23:58:13.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057673637999999999 2024-09-14T23:58:13.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:13.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315734+0000", 2024-09-14T23:58:13.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2216000000000002e-05 2024-09-14T23:58:13.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:13.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315784+0000", 2024-09-14T23:58:13.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9595000000000002e-05 2024-09-14T23:58:13.805 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.805 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:13.805 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.806 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5221 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3461120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:13.806 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:58:13.806 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 94.495957927000006, 2024-09-14T23:58:13.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058003557999999997, 2024-09-14T23:58:13.806 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:13.806 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:13.806 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:13.806 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:13.806 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:13.806 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:13.806 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5221 2024-09-14T23:58:13.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.807 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:13.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:13.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:58:13.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:13.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:58:13.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:13.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259905+0000", 2024-09-14T23:58:13.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019035759999999999 2024-09-14T23:58:13.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:13.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259915+0000", 2024-09-14T23:58:13.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0518e-05 2024-09-14T23:58:13.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:13.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259925+0000", 2024-09-14T23:58:13.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7180000000000008e-06 2024-09-14T23:58:13.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:13.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259930+0000", 2024-09-14T23:58:13.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4769999999999997e-06 2024-09-14T23:58:13.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:13.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315820+0000", 2024-09-14T23:58:13.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055889960000000002 2024-09-14T23:58:13.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:13.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315914+0000", 2024-09-14T23:58:13.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4625999999999998e-05 2024-09-14T23:58:13.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:13.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.316005+0000", 2024-09-14T23:58:13.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0682999999999999e-05 2024-09-14T23:58:13.810 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.810 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:13.810 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.810 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5231 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 4075520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:13.810 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:58:13.810 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 94.493889193000001, 2024-09-14T23:58:13.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057524733000000002, 2024-09-14T23:58:13.810 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:13.810 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:13.810 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:13.811 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:13.811 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:13.811 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:13.811 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5231 2024-09-14T23:58:13.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.811 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:13.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:13.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:58:13.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:13.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:58:13.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:13.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261904+0000", 2024-09-14T23:58:13.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018338790000000001 2024-09-14T23:58:13.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:13.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261910+0000", 2024-09-14T23:58:13.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9689999999999999e-06 2024-09-14T23:58:13.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:13.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261923+0000", 2024-09-14T23:58:13.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3061000000000001e-05 2024-09-14T23:58:13.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:13.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261931+0000", 2024-09-14T23:58:13.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3159999999999997e-06 2024-09-14T23:58:13.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:13.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317456+0000", 2024-09-14T23:58:13.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055525219000000001 2024-09-14T23:58:13.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:13.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317520+0000", 2024-09-14T23:58:13.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4010000000000005e-05 2024-09-14T23:58:13.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:13.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317595+0000", 2024-09-14T23:58:13.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4278999999999998e-05 2024-09-14T23:58:13.814 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.814 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:13.814 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.815 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5236 3.8 3:1071668d:::rbd_data.1118de1168b.0000000000000092:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:13.815 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:58:13.815 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 94.491875729, 2024-09-14T23:58:13.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057499462000000001, 2024-09-14T23:58:13.815 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:13.815 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:13.815 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:13.815 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:13.815 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:13.815 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:13.815 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5236 2024-09-14T23:58:13.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.815 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:13.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:13.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:58:13.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:13.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:58:13.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:13.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262084+0000", 2024-09-14T23:58:13.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3399999999999999e-07 2024-09-14T23:58:13.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:13.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262087+0000", 2024-09-14T23:58:13.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2310000000000001e-06 2024-09-14T23:58:13.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:13.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262095+0000", 2024-09-14T23:58:13.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3799999999999996e-06 2024-09-14T23:58:13.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:13.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262101+0000", 2024-09-14T23:58:13.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0009999999999998e-06 2024-09-14T23:58:13.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:13.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317625+0000", 2024-09-14T23:58:13.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055524111000000001 2024-09-14T23:58:13.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:13.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317702+0000", 2024-09-14T23:58:13.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7533999999999993e-05 2024-09-14T23:58:13.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-14T23:58:13.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317892+0000", 2024-09-14T23:58:13.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018991400000000001 2024-09-14T23:58:13.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:58:13.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319407+0000", 2024-09-14T23:58:13.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015145250000000001 2024-09-14T23:58:13.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:13.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319481+0000", 2024-09-14T23:58:13.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3826999999999996e-05 2024-09-14T23:58:13.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:13.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319541+0000", 2024-09-14T23:58:13.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0470999999999998e-05 2024-09-14T23:58:13.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:13.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319583+0000", 2024-09-14T23:58:13.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1934000000000002e-05 2024-09-14T23:58:13.820 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.820 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:13.820 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.820 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:132622 3.1e 3.7a50b9e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-14T23:58:13.821 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:43.371285+0000", 2024-09-14T23:58:13.821 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 30.382673827000001, 2024-09-14T23:58:13.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.67037987200000004, 2024-09-14T23:58:13.821 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:13.821 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:13.821 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:13.821 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:13.821 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:13.821 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:13.821 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 132622 2024-09-14T23:58:13.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.821 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:13.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:13.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371285+0000", 2024-09-14T23:58:13.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:13.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371285+0000", 2024-09-14T23:58:13.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:13.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371286+0000", 2024-09-14T23:58:13.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6400000000000002e-07 2024-09-14T23:58:13.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:13.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371289+0000", 2024-09-14T23:58:13.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2059999999999999e-06 2024-09-14T23:58:13.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:13.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371299+0000", 2024-09-14T23:58:13.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0288e-05 2024-09-14T23:58:13.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:13.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371306+0000", 2024-09-14T23:58:13.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1650000000000003e-06 2024-09-14T23:58:13.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:13.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371617+0000", 2024-09-14T23:58:13.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000311043 2024-09-14T23:58:13.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:58:13.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371625+0000", 2024-09-14T23:58:13.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.833e-06 2024-09-14T23:58:13.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:13.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.041651+0000", 2024-09-14T23:58:13.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.67002580199999995 2024-09-14T23:58:13.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:13.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.041665+0000", 2024-09-14T23:58:13.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3971000000000001e-05 2024-09-14T23:58:13.825 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.825 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:13.825 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.825 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:169974 3.5 3:a704a2f0:::rbd_data.1118427720d0.0000000000000030:32 [read 965632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-14T23:58:13.825 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:03.433820+0000", 2024-09-14T23:58:13.825 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.320139169999999, 2024-09-14T23:58:13.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057495521000000001, 2024-09-14T23:58:13.826 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:13.826 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:13.826 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:13.826 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:13.826 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:13.826 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:13.826 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 169974 2024-09-14T23:58:13.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.826 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:13.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:13.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.433820+0000", 2024-09-14T23:58:13.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:13.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.433820+0000", 2024-09-14T23:58:13.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:13.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.433925+0000", 2024-09-14T23:58:13.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000104621 2024-09-14T23:58:13.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:13.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.433929+0000", 2024-09-14T23:58:13.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2169999999999996e-06 2024-09-14T23:58:13.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:13.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.433939+0000", 2024-09-14T23:58:13.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0002e-05 2024-09-14T23:58:13.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:13.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.433946+0000", 2024-09-14T23:58:13.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7889999999999997e-06 2024-09-14T23:58:13.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:13.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.491144+0000", 2024-09-14T23:58:13.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057198221 2024-09-14T23:58:13.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:13.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.491221+0000", 2024-09-14T23:58:13.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6767000000000005e-05 2024-09-14T23:58:13.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:13.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.491316+0000", 2024-09-14T23:58:13.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4903999999999998e-05 2024-09-14T23:58:13.829 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.829 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:13.830 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.830 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:169975 3.5 3:a704a2f0:::rbd_data.1118427720d0.0000000000000030:32 [read 969728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-14T23:58:13.830 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:03.433958+0000", 2024-09-14T23:58:13.830 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.320001470999999, 2024-09-14T23:58:13.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057564811, 2024-09-14T23:58:13.830 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:13.830 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:13.830 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:13.830 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:13.830 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:13.830 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:13.830 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 169975 2024-09-14T23:58:13.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.831 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:13.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:13.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.433958+0000", 2024-09-14T23:58:13.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:13.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.433958+0000", 2024-09-14T23:58:13.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:13.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.435096+0000", 2024-09-14T23:58:13.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001138708 2024-09-14T23:58:13.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:13.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.435101+0000", 2024-09-14T23:58:13.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5050000000000001e-06 2024-09-14T23:58:13.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:13.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.435111+0000", 2024-09-14T23:58:13.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5810000000000003e-06 2024-09-14T23:58:13.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:13.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.435120+0000", 2024-09-14T23:58:13.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0089999999999996e-06 2024-09-14T23:58:13.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:13.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.491345+0000", 2024-09-14T23:58:13.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056225479000000002 2024-09-14T23:58:13.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:13.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.491423+0000", 2024-09-14T23:58:13.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7990000000000004e-05 2024-09-14T23:58:13.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:13.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.491523+0000", 2024-09-14T23:58:13.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9538999999999997e-05 2024-09-14T23:58:13.834 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.834 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:13.834 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.834 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:172639 3.5 3:a704a2f0:::rbd_data.1118427720d0.0000000000000030:32 [read 1645568~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e107)", 2024-09-14T23:58:13.834 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:03.988365+0000", 2024-09-14T23:58:13.834 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7655937510000008, 2024-09-14T23:58:13.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057556213000000002, 2024-09-14T23:58:13.834 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:13.835 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:13.835 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:13.835 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:13.835 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:13.835 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:13.835 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 172639 2024-09-14T23:58:13.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.835 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:13.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:13.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.988365+0000", 2024-09-14T23:58:13.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:13.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.988365+0000", 2024-09-14T23:58:13.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:13.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.989904+0000", 2024-09-14T23:58:13.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015380859999999999 2024-09-14T23:58:13.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:13.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.989909+0000", 2024-09-14T23:58:13.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2800000000000003e-06 2024-09-14T23:58:13.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:13.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.989925+0000", 2024-09-14T23:58:13.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5781999999999999e-05 2024-09-14T23:58:13.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:13.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.989933+0000", 2024-09-14T23:58:13.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0090000000000007e-06 2024-09-14T23:58:13.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:13.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:04.045741+0000", 2024-09-14T23:58:13.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055808112999999999 2024-09-14T23:58:13.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:13.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:04.045812+0000", 2024-09-14T23:58:13.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1531000000000003e-05 2024-09-14T23:58:13.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:13.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:04.045922+0000", 2024-09-14T23:58:13.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000109412 2024-09-14T23:58:13.838 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.838 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:13.838 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.839 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184571 3.2 3:46f8d78b:::rbd_data.1118427720d0.0000000000000001:32 [read 1405952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:13.839 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.086404+0000", 2024-09-14T23:58:13.839 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.6675553939999999, 2024-09-14T23:58:13.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059148573000000003, 2024-09-14T23:58:13.839 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:13.839 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:13.839 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:13.839 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:13.839 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:13.839 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:13.839 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184571 2024-09-14T23:58:13.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.839 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:13.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:13.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.086404+0000", 2024-09-14T23:58:13.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:13.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.086404+0000", 2024-09-14T23:58:13.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:13.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087902+0000", 2024-09-14T23:58:13.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001498625 2024-09-14T23:58:13.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:13.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087907+0000", 2024-09-14T23:58:13.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9130000000000002e-06 2024-09-14T23:58:13.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:13.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087919+0000", 2024-09-14T23:58:13.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1719000000000001e-05 2024-09-14T23:58:13.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:13.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087927+0000", 2024-09-14T23:58:13.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3699999999999995e-06 2024-09-14T23:58:13.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:13.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143711+0000", 2024-09-14T23:58:13.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055783396999999998 2024-09-14T23:58:13.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:13.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143792+0000", 2024-09-14T23:58:13.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1292999999999996e-05 2024-09-14T23:58:13.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:13.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145552+0000", 2024-09-14T23:58:13.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001760256 2024-09-14T23:58:13.843 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.843 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:13.843 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.843 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184579 3.2 3:43badc91:::rbd_data.1118427720d0.0000000000000002:32 [read 1196032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:13.843 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.088145+0000", 2024-09-14T23:58:13.843 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 6.6658139900000002, 2024-09-14T23:58:13.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057643169000000001, 2024-09-14T23:58:13.843 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:13.843 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:13.843 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:13.844 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:13.844 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:13.844 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:13.844 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184579 2024-09-14T23:58:13.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.844 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:13.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:13.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.088145+0000", 2024-09-14T23:58:13.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:13.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.088145+0000", 2024-09-14T23:58:13.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:13.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:13.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089947+0000", 2024-09-14T23:58:13.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001801764 2024-09-14T23:58:13.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:13.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089952+0000", 2024-09-14T23:58:13.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8659999999999997e-06 2024-09-14T23:58:13.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:13.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089966+0000", 2024-09-14T23:58:13.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3929e-05 2024-09-14T23:58:13.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:13.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089974+0000", 2024-09-14T23:58:13.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1429999999999998e-06 2024-09-14T23:58:13.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:13.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145636+0000", 2024-09-14T23:58:13.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055662287999999997 2024-09-14T23:58:13.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:13.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145710+0000", 2024-09-14T23:58:13.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4166999999999996e-05 2024-09-14T23:58:13.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:13.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:13.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:13.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145788+0000", 2024-09-14T23:58:13.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8011999999999998e-05 2024-09-14T23:58:13.847 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.847 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:13.847 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.847 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:13.847 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:13.847 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:13.849 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T23:58:13.849 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:13.847+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:13.949 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:13.948+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:13.970 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:13.970 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:58:13.971 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:58:13.971 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:13.980 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T23:58:14.051 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:14.049+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:14.088 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:14.088 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:58:14.088 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:58:14.088 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:58:14.088 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:14.096 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T23:58:14.180 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:14.178+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:14.191 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is unset 2024-09-14T23:58:14.196 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T23:58:14.228 INFO:teuthology.orchestra.run.smithi042.stdout:5928 writesame 0x6dfef9a thru 0x6e07bb3 data_size 0x4f(0x8c1a bytes) 2024-09-14T23:58:14.230 INFO:teuthology.orchestra.run.smithi042.stdout:5929 read 0x1d76e89 thru 0x1d7d303 (0x647b bytes) 2024-09-14T23:58:14.230 INFO:teuthology.orchestra.run.smithi042.stdout:5931 read 0x673a781 thru 0x673bf07 (0x1787 bytes) 2024-09-14T23:58:14.230 INFO:teuthology.orchestra.run.smithi042.stdout:5932 read 0x5cb4951 thru 0x5cbe1b5 (0x9865 bytes) 2024-09-14T23:58:14.250 INFO:teuthology.orchestra.run.smithi042.stdout:5934 writesame 0xecc07bd thru 0xeccf12e data_size 0x1(0xe972 bytes) 2024-09-14T23:58:14.250 INFO:teuthology.orchestra.run.smithi042.stdout:5935 read 0xbede7f1 thru 0xbee054c (0x1d5c bytes) 2024-09-14T23:58:14.250 INFO:teuthology.orchestra.run.smithi042.stdout:5938 punch from 0xa7ea8fc to 0xa7ed260, (0x2964 bytes) 2024-09-14T23:58:14.250 INFO:teuthology.orchestra.run.smithi042.stdout:5939 read 0x5305c31 thru 0x5314fdd (0xf3ad bytes) 2024-09-14T23:58:14.250 INFO:teuthology.orchestra.run.smithi042.stdout:5941 writesame 0x98cc296 thru 0x98d56bb data_size 0x62(0x9426 bytes) 2024-09-14T23:58:14.251 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:14.251 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:58:14.251 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:58:14.252 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:58:14.252 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.252 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186203 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1329152~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:14.252 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.194279+0000", 2024-09-14T23:58:14.252 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.0542673640000002, 2024-09-14T23:58:14.252 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8011469710000001, 2024-09-14T23:58:14.252 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:14.252 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:14.252 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:14.252 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:14.252 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:14.252 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:14.252 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186203 2024-09-14T23:58:14.252 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.253 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:14.253 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.253 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:14.253 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194279+0000", 2024-09-14T23:58:14.253 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:14.253 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.253 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.253 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:14.253 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194279+0000", 2024-09-14T23:58:14.253 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:14.253 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.253 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.253 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:14.254 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194279+0000", 2024-09-14T23:58:14.254 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8100000000000001e-07 2024-09-14T23:58:14.254 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.254 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.254 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:14.254 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194291+0000", 2024-09-14T23:58:14.254 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1559999999999999e-05 2024-09-14T23:58:14.254 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.254 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.254 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:14.254 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194303+0000", 2024-09-14T23:58:14.254 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2130000000000001e-05 2024-09-14T23:58:14.254 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.254 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.255 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:14.255 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194311+0000", 2024-09-14T23:58:14.255 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8089999999999993e-06 2024-09-14T23:58:14.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.255 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:14.255 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194370+0000", 2024-09-14T23:58:14.255 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8739999999999997e-05 2024-09-14T23:58:14.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.255 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:14.255 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194376+0000", 2024-09-14T23:58:14.255 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2319999999999998e-06 2024-09-14T23:58:14.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.256 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:14.256 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986290+0000", 2024-09-14T23:58:14.256 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.791914334 2024-09-14T23:58:14.256 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.256 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.256 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:14.256 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986296+0000", 2024-09-14T23:58:14.256 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6300000000000003e-06 2024-09-14T23:58:14.256 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.256 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.256 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:14.256 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995193+0000", 2024-09-14T23:58:14.256 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0088975040000001 2024-09-14T23:58:14.256 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.256 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.257 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:14.257 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995307+0000", 2024-09-14T23:58:14.257 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000113656 2024-09-14T23:58:14.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.257 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.257 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:14.257 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995426+0000", 2024-09-14T23:58:14.257 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118495 2024-09-14T23:58:14.257 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:14.257 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:14.257 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:14.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.257 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.257 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186204 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1391616~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:14.258 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.194328+0000", 2024-09-14T23:58:14.258 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.0542180329999997, 2024-09-14T23:58:14.258 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8013834310000001, 2024-09-14T23:58:14.258 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:14.258 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:14.258 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:14.258 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:14.258 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:14.258 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:14.258 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186204 2024-09-14T23:58:14.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.258 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:14.258 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.258 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:14.259 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194328+0000", 2024-09-14T23:58:14.259 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:14.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.259 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.259 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:14.259 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194328+0000", 2024-09-14T23:58:14.259 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:14.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.259 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.259 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:14.259 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194328+0000", 2024-09-14T23:58:14.260 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7100000000000002e-07 2024-09-14T23:58:14.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.260 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.260 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:14.260 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194331+0000", 2024-09-14T23:58:14.260 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.807e-06 2024-09-14T23:58:14.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.260 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.260 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:14.260 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194339+0000", 2024-09-14T23:58:14.260 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9340000000000006e-06 2024-09-14T23:58:14.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.261 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.261 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:14.261 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194345+0000", 2024-09-14T23:58:14.261 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1090000000000003e-06 2024-09-14T23:58:14.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.261 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.261 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:14.261 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194412+0000", 2024-09-14T23:58:14.261 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6229000000000006e-05 2024-09-14T23:58:14.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.261 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.261 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:14.261 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194422+0000", 2024-09-14T23:58:14.261 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.078e-05 2024-09-14T23:58:14.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.262 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.262 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:14.262 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986322+0000", 2024-09-14T23:58:14.262 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.79189964899999998 2024-09-14T23:58:14.262 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.262 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.262 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:14.262 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986332+0000", 2024-09-14T23:58:14.262 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0095000000000001e-05 2024-09-14T23:58:14.262 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.262 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.262 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:14.262 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995554+0000", 2024-09-14T23:58:14.263 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0092218630000001 2024-09-14T23:58:14.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.263 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:14.263 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995610+0000", 2024-09-14T23:58:14.263 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6109000000000001e-05 2024-09-14T23:58:14.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.263 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:14.263 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995711+0000", 2024-09-14T23:58:14.263 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101285 2024-09-14T23:58:14.263 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:14.263 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:14.263 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:14.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.264 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.264 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186205 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1333248~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:14.264 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.194354+0000", 2024-09-14T23:58:14.264 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.0541921030000001, 2024-09-14T23:58:14.264 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8015471490000001, 2024-09-14T23:58:14.264 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:14.264 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:14.264 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:14.264 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:14.265 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:14.265 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:14.265 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186205 2024-09-14T23:58:14.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.265 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:14.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.265 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:14.265 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194354+0000", 2024-09-14T23:58:14.265 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:14.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.265 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:14.265 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194354+0000", 2024-09-14T23:58:14.266 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:14.266 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.266 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.266 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:14.266 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194354+0000", 2024-09-14T23:58:14.266 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.39e-07 2024-09-14T23:58:14.266 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.266 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.266 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:14.266 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194357+0000", 2024-09-14T23:58:14.267 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3479999999999998e-06 2024-09-14T23:58:14.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.267 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.267 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:14.267 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194364+0000", 2024-09-14T23:58:14.267 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0899999999999999e-06 2024-09-14T23:58:14.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.268 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.268 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:14.268 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194371+0000", 2024-09-14T23:58:14.268 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5530000000000004e-06 2024-09-14T23:58:14.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.268 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.268 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:14.268 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194469+0000", 2024-09-14T23:58:14.268 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8119e-05 2024-09-14T23:58:14.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.268 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.268 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:14.268 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194475+0000", 2024-09-14T23:58:14.269 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7910000000000001e-06 2024-09-14T23:58:14.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.269 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.269 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:14.269 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986367+0000", 2024-09-14T23:58:14.269 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.79189214699999999 2024-09-14T23:58:14.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.269 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.269 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:14.269 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986375+0000", 2024-09-14T23:58:14.269 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7800000000000001e-06 2024-09-14T23:58:14.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.269 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.270 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:14.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995746+0000", 2024-09-14T23:58:14.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0093706469999999 2024-09-14T23:58:14.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.270 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.270 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:14.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995810+0000", 2024-09-14T23:58:14.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4068999999999999e-05 2024-09-14T23:58:14.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.270 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.270 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:14.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995901+0000", 2024-09-14T23:58:14.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1166000000000001e-05 2024-09-14T23:58:14.270 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:14.270 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:14.271 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:14.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.271 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.271 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186206 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1395712~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:14.271 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.194380+0000", 2024-09-14T23:58:14.271 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.0541660820000001, 2024-09-14T23:58:14.271 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.801678347, 2024-09-14T23:58:14.271 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:14.271 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:14.271 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:14.271 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:14.271 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:14.271 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:14.272 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186206 2024-09-14T23:58:14.272 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.272 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:14.272 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.272 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:14.272 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194380+0000", 2024-09-14T23:58:14.272 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:14.272 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.272 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.272 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:14.272 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194380+0000", 2024-09-14T23:58:14.272 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:14.272 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.272 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.273 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:14.273 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194380+0000", 2024-09-14T23:58:14.273 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4400000000000001e-07 2024-09-14T23:58:14.273 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.273 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.273 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:14.273 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194383+0000", 2024-09-14T23:58:14.273 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3460000000000001e-06 2024-09-14T23:58:14.273 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.273 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.274 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:14.274 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194390+0000", 2024-09-14T23:58:14.274 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2189999999999998e-06 2024-09-14T23:58:14.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.274 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:14.274 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194403+0000", 2024-09-14T23:58:14.274 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2941e-05 2024-09-14T23:58:14.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.274 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:14.274 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194545+0000", 2024-09-14T23:58:14.274 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014232700000000001 2024-09-14T23:58:14.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.275 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:14.275 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194565+0000", 2024-09-14T23:58:14.275 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9409999999999999e-05 2024-09-14T23:58:14.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.275 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.275 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:14.275 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986409+0000", 2024-09-14T23:58:14.276 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.79184416000000002 2024-09-14T23:58:14.276 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.276 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.276 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:14.276 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986417+0000", 2024-09-14T23:58:14.276 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.551e-06 2024-09-14T23:58:14.276 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.276 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.276 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:14.276 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995921+0000", 2024-09-14T23:58:14.276 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.009504212 2024-09-14T23:58:14.276 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.276 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.277 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:14.277 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995974+0000", 2024-09-14T23:58:14.277 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2617999999999997e-05 2024-09-14T23:58:14.277 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.277 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.277 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:14.277 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996058+0000", 2024-09-14T23:58:14.277 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4118999999999999e-05 2024-09-14T23:58:14.277 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:14.277 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:14.277 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:14.277 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.277 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.277 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186207 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1337344~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:14.278 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.194412+0000", 2024-09-14T23:58:14.278 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.0541334840000003, 2024-09-14T23:58:14.278 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.801848141, 2024-09-14T23:58:14.278 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:14.278 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:14.278 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:14.278 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:14.278 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:14.278 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:14.278 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186207 2024-09-14T23:58:14.278 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.278 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:14.278 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.278 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:14.278 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194412+0000", 2024-09-14T23:58:14.279 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:14.279 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.279 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.279 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:14.279 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194412+0000", 2024-09-14T23:58:14.279 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:14.279 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.279 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.279 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:14.279 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194413+0000", 2024-09-14T23:58:14.279 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5600000000000001e-07 2024-09-14T23:58:14.279 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.279 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.279 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:14.280 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194418+0000", 2024-09-14T23:58:14.280 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3720000000000004e-06 2024-09-14T23:58:14.280 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.280 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.280 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:14.280 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194423+0000", 2024-09-14T23:58:14.280 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.763e-06 2024-09-14T23:58:14.280 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.280 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.280 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:14.280 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194426+0000", 2024-09-14T23:58:14.280 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1480000000000002e-06 2024-09-14T23:58:14.280 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.281 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.281 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:14.281 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194596+0000", 2024-09-14T23:58:14.281 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017009600000000001 2024-09-14T23:58:14.281 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.281 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.281 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:14.281 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194606+0000", 2024-09-14T23:58:14.281 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6639999999999993e-06 2024-09-14T23:58:14.281 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.281 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.281 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:14.282 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986460+0000", 2024-09-14T23:58:14.282 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.79185424400000004 2024-09-14T23:58:14.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.282 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.282 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:14.282 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986469+0000", 2024-09-14T23:58:14.282 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4419999999999998e-06 2024-09-14T23:58:14.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.282 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.283 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:14.283 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996123+0000", 2024-09-14T23:58:14.283 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0096546070000001 2024-09-14T23:58:14.283 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.283 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.283 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:14.283 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996179+0000", 2024-09-14T23:58:14.283 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5850000000000002e-05 2024-09-14T23:58:14.283 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.283 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.283 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:14.283 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996261+0000", 2024-09-14T23:58:14.283 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1599000000000003e-05 2024-09-14T23:58:14.284 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:14.284 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:14.284 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:14.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.284 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.284 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186208 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1399808~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:14.284 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.194445+0000", 2024-09-14T23:58:14.284 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.0541013140000004, 2024-09-14T23:58:14.284 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8019822269999999, 2024-09-14T23:58:14.284 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:14.284 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:14.284 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:14.284 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:14.285 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:14.285 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:14.285 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186208 2024-09-14T23:58:14.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.285 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:14.285 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.285 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:14.288 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194445+0000", 2024-09-14T23:58:14.289 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:14.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.289 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.289 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:14.289 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194445+0000", 2024-09-14T23:58:14.289 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:14.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.289 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.289 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:14.289 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978674+0000", 2024-09-14T23:58:14.289 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.784229124 2024-09-14T23:58:14.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.290 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.290 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:14.290 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978680+0000", 2024-09-14T23:58:14.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7969999999999997e-06 2024-09-14T23:58:14.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.290 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.290 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:14.290 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978689+0000", 2024-09-14T23:58:14.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6949999999999995e-06 2024-09-14T23:58:14.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.290 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.290 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:14.290 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978696+0000", 2024-09-14T23:58:14.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1300000000000003e-06 2024-09-14T23:58:14.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.291 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.291 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:14.291 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986509+0000", 2024-09-14T23:58:14.291 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0078130969999999998 2024-09-14T23:58:14.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.291 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.291 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:14.291 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986518+0000", 2024-09-14T23:58:14.292 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2810000000000008e-06 2024-09-14T23:58:14.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.292 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.292 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:14.292 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996280+0000", 2024-09-14T23:58:14.292 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0097623120000001 2024-09-14T23:58:14.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.292 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.292 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:14.292 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996337+0000", 2024-09-14T23:58:14.292 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7244000000000003e-05 2024-09-14T23:58:14.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.292 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.293 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:14.293 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996427+0000", 2024-09-14T23:58:14.293 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9547000000000002e-05 2024-09-14T23:58:14.293 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:14.293 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:14.293 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:14.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.293 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.293 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4172.0:7 3.11 3:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:14.293 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.483386+0000", 2024-09-14T23:58:14.293 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 6.7651598889999995, 2024-09-14T23:58:14.293 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.558667389, 2024-09-14T23:58:14.293 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:14.294 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:14.294 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:58:14.294 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:14.294 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4172", 2024-09-14T23:58:14.294 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/2542269203", 2024-09-14T23:58:14.294 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 7 2024-09-14T23:58:14.294 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.294 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:14.294 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.294 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:14.294 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.483386+0000", 2024-09-14T23:58:14.294 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:14.294 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.294 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.295 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:14.295 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.483386+0000", 2024-09-14T23:58:14.295 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:14.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.295 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.295 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:14.295 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978674+0000", 2024-09-14T23:58:14.295 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.49528781599999999 2024-09-14T23:58:14.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.295 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.295 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:14.295 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978687+0000", 2024-09-14T23:58:14.295 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3244999999999999e-05 2024-09-14T23:58:14.296 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.296 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.296 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:14.296 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978699+0000", 2024-09-14T23:58:14.296 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1832e-05 2024-09-14T23:58:14.296 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.296 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.296 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:14.296 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978707+0000", 2024-09-14T23:58:14.296 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4489999999999999e-06 2024-09-14T23:58:14.296 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.296 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.296 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:14.296 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978801+0000", 2024-09-14T23:58:14.297 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3511999999999995e-05 2024-09-14T23:58:14.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.297 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.297 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:14.297 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978814+0000", 2024-09-14T23:58:14.297 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2789000000000001e-05 2024-09-14T23:58:14.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.297 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.297 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:14.297 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.041948+0000", 2024-09-14T23:58:14.297 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0631338269999999 2024-09-14T23:58:14.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.297 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.298 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:14.298 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.042053+0000", 2024-09-14T23:58:14.298 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105919 2024-09-14T23:58:14.298 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:14.298 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:14.298 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:14.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.298 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.298 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187209 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1575936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:14.298 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.991888+0000", 2024-09-14T23:58:14.298 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 6.2566579180000002, 2024-09-14T23:58:14.298 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0712115470000001, 2024-09-14T23:58:14.298 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:14.298 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:14.299 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:14.299 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:14.299 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:14.299 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:14.299 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187209 2024-09-14T23:58:14.299 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.299 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:14.299 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.299 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:14.299 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991888+0000", 2024-09-14T23:58:14.299 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:14.299 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.299 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.300 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:14.300 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991888+0000", 2024-09-14T23:58:14.300 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:14.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.300 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.300 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:14.300 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991888+0000", 2024-09-14T23:58:14.300 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1199999999999998e-07 2024-09-14T23:58:14.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.300 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.300 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:14.300 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991890+0000", 2024-09-14T23:58:14.301 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9810000000000002e-06 2024-09-14T23:58:14.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.301 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.301 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:14.301 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991897+0000", 2024-09-14T23:58:14.301 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2609999999999999e-06 2024-09-14T23:58:14.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.301 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.301 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:14.301 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991902+0000", 2024-09-14T23:58:14.301 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0000000000000004e-06 2024-09-14T23:58:14.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.301 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.302 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:14.302 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995686+0000", 2024-09-14T23:58:14.302 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0037839290000000001 2024-09-14T23:58:14.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.302 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.302 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:14.302 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995693+0000", 2024-09-14T23:58:14.302 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7589999999999997e-06 2024-09-14T23:58:14.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.302 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.302 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:14.302 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062846+0000", 2024-09-14T23:58:14.302 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.067152417 2024-09-14T23:58:14.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.303 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:14.303 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062935+0000", 2024-09-14T23:58:14.303 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9080999999999996e-05 2024-09-14T23:58:14.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.303 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:14.303 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.063100+0000", 2024-09-14T23:58:14.303 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016470699999999999 2024-09-14T23:58:14.303 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:14.303 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:14.303 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:14.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.304 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.304 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187210 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1591296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:14.304 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.991908+0000", 2024-09-14T23:58:14.304 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 6.2566380050000001, 2024-09-14T23:58:14.304 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0713882530000001, 2024-09-14T23:58:14.304 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:14.304 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:14.304 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:14.304 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:14.304 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:14.304 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:14.304 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187210 2024-09-14T23:58:14.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.305 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:14.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.305 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:14.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991908+0000", 2024-09-14T23:58:14.305 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:14.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.305 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:14.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991908+0000", 2024-09-14T23:58:14.305 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:14.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.305 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:14.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993673+0000", 2024-09-14T23:58:14.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017651030000000001 2024-09-14T23:58:14.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.306 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.306 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:14.306 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993681+0000", 2024-09-14T23:58:14.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.858e-06 2024-09-14T23:58:14.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.306 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.306 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:14.306 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993691+0000", 2024-09-14T23:58:14.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9920000000000002e-06 2024-09-14T23:58:14.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.306 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.306 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:14.306 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993698+0000", 2024-09-14T23:58:14.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6490000000000002e-06 2024-09-14T23:58:14.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.307 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.307 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:14.307 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995731+0000", 2024-09-14T23:58:14.307 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0020338959999999999 2024-09-14T23:58:14.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.307 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.307 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:14.307 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995741+0000", 2024-09-14T23:58:14.307 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3079999999999997e-06 2024-09-14T23:58:14.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.307 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.307 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:14.307 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.063130+0000", 2024-09-14T23:58:14.308 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.067389159 2024-09-14T23:58:14.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.308 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.308 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:14.308 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.063206+0000", 2024-09-14T23:58:14.308 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6577999999999998e-05 2024-09-14T23:58:14.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.308 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.308 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:14.308 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.063296+0000", 2024-09-14T23:58:14.308 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9710000000000007e-05 2024-09-14T23:58:14.308 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:14.308 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:14.308 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:14.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.309 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.309 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187216 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1603584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:14.309 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.993831+0000", 2024-09-14T23:58:14.309 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 6.2547147990000003, 2024-09-14T23:58:14.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.071244627, 2024-09-14T23:58:14.309 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:14.309 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:14.309 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:14.309 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:14.309 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:14.309 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:14.309 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187216 2024-09-14T23:58:14.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.309 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:14.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.310 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:14.310 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993831+0000", 2024-09-14T23:58:14.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:14.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.310 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:14.310 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993831+0000", 2024-09-14T23:58:14.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:14.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.310 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:14.310 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993832+0000", 2024-09-14T23:58:14.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9700000000000002e-07 2024-09-14T23:58:14.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.311 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.311 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:14.311 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993834+0000", 2024-09-14T23:58:14.311 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6170000000000001e-06 2024-09-14T23:58:14.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.311 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.311 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:14.311 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993841+0000", 2024-09-14T23:58:14.311 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6980000000000001e-06 2024-09-14T23:58:14.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.311 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.311 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:14.311 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993846+0000", 2024-09-14T23:58:14.311 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4190000000000001e-06 2024-09-14T23:58:14.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.312 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:14.312 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995988+0000", 2024-09-14T23:58:14.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0021417390000000001 2024-09-14T23:58:14.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.312 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:14.312 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995998+0000", 2024-09-14T23:58:14.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5100000000000004e-06 2024-09-14T23:58:14.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.312 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:14.312 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.064883+0000", 2024-09-14T23:58:14.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.06888597 2024-09-14T23:58:14.313 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.313 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.313 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:14.313 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.064977+0000", 2024-09-14T23:58:14.313 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3817e-05 2024-09-14T23:58:14.313 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.313 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.313 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:14.313 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065076+0000", 2024-09-14T23:58:14.313 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8460000000000003e-05 2024-09-14T23:58:14.313 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:14.313 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:14.313 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:14.313 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.313 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.314 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187217 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1592320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:14.314 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.993855+0000", 2024-09-14T23:58:14.314 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 6.2546913660000003, 2024-09-14T23:58:14.314 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0714239349999999, 2024-09-14T23:58:14.314 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:14.314 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:14.314 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:14.314 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:14.314 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:14.314 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:14.314 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187217 2024-09-14T23:58:14.314 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.314 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:14.314 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.315 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:14.315 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993855+0000", 2024-09-14T23:58:14.315 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:14.315 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.315 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.315 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:14.315 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993855+0000", 2024-09-14T23:58:14.315 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:14.315 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.315 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.315 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:14.315 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993855+0000", 2024-09-14T23:58:14.315 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4899999999999995e-07 2024-09-14T23:58:14.315 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.316 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.316 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:14.316 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993857+0000", 2024-09-14T23:58:14.316 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3379999999999999e-06 2024-09-14T23:58:14.316 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.316 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.316 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:14.316 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993864+0000", 2024-09-14T23:58:14.316 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6000000000000003e-06 2024-09-14T23:58:14.316 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.316 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.316 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:14.316 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993869+0000", 2024-09-14T23:58:14.316 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3210000000000004e-06 2024-09-14T23:58:14.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.317 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.317 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:14.317 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996038+0000", 2024-09-14T23:58:14.317 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0021683050000000001 2024-09-14T23:58:14.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.317 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.317 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:14.317 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996048+0000", 2024-09-14T23:58:14.317 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0173999999999999e-05 2024-09-14T23:58:14.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.317 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.317 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:14.318 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065108+0000", 2024-09-14T23:58:14.318 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0690600619999999 2024-09-14T23:58:14.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.318 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.318 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:14.318 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065171+0000", 2024-09-14T23:58:14.318 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3423999999999994e-05 2024-09-14T23:58:14.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.318 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.318 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:14.318 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065279+0000", 2024-09-14T23:58:14.318 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000107162 2024-09-14T23:58:14.318 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:14.318 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:14.318 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:14.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.319 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.319 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187218 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1607680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:14.319 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.993877+0000", 2024-09-14T23:58:14.319 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 6.2546690859999998, 2024-09-14T23:58:14.319 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0716976810000001, 2024-09-14T23:58:14.319 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:14.319 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:14.319 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:14.319 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:14.319 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:14.319 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:14.319 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187218 2024-09-14T23:58:14.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.319 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:14.320 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.320 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:14.320 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993877+0000", 2024-09-14T23:58:14.320 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:14.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.320 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.320 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:14.320 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993877+0000", 2024-09-14T23:58:14.320 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:14.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.320 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.320 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:14.320 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993877+0000", 2024-09-14T23:58:14.320 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8299999999999998e-07 2024-09-14T23:58:14.321 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.321 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.321 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:14.321 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993879+0000", 2024-09-14T23:58:14.321 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5659999999999999e-06 2024-09-14T23:58:14.321 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.321 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.321 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:14.321 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993882+0000", 2024-09-14T23:58:14.321 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7280000000000001e-06 2024-09-14T23:58:14.321 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.321 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.321 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:14.321 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993885+0000", 2024-09-14T23:58:14.322 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.751e-06 2024-09-14T23:58:14.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.322 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.322 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:14.322 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996081+0000", 2024-09-14T23:58:14.322 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0021960590000000002 2024-09-14T23:58:14.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.322 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.322 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:14.322 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996091+0000", 2024-09-14T23:58:14.322 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4930000000000004e-06 2024-09-14T23:58:14.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.322 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.322 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:14.322 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065308+0000", 2024-09-14T23:58:14.323 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.069217407 2024-09-14T23:58:14.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.323 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.323 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:14.323 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065414+0000", 2024-09-14T23:58:14.323 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105438 2024-09-14T23:58:14.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.323 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.323 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:14.323 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065575+0000", 2024-09-14T23:58:14.323 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000160956 2024-09-14T23:58:14.323 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:14.323 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:14.323 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:14.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.324 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.324 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187219 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1596416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:14.324 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.993889+0000", 2024-09-14T23:58:14.324 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 6.2546572439999997, 2024-09-14T23:58:14.324 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0720031190000001, 2024-09-14T23:58:14.324 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:14.324 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:14.324 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:14.324 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:14.324 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:14.324 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:14.324 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187219 2024-09-14T23:58:14.324 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.324 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:14.324 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.325 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:14.325 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993889+0000", 2024-09-14T23:58:14.325 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:14.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.325 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.325 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:14.325 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993889+0000", 2024-09-14T23:58:14.325 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:14.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.325 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.325 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:14.325 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993889+0000", 2024-09-14T23:58:14.325 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2399999999999999e-07 2024-09-14T23:58:14.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.325 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.326 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:14.326 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993890+0000", 2024-09-14T23:58:14.326 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1939999999999999e-06 2024-09-14T23:58:14.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.326 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.326 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:14.326 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993893+0000", 2024-09-14T23:58:14.326 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1300000000000001e-06 2024-09-14T23:58:14.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.326 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.326 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:14.326 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993896+0000", 2024-09-14T23:58:14.326 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6879999999999999e-06 2024-09-14T23:58:14.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.326 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.327 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:14.327 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996119+0000", 2024-09-14T23:58:14.327 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0022226149999999998 2024-09-14T23:58:14.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.327 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.327 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:14.327 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996123+0000", 2024-09-14T23:58:14.327 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.211e-06 2024-09-14T23:58:14.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.327 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.327 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:14.327 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065616+0000", 2024-09-14T23:58:14.327 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0694935240000001 2024-09-14T23:58:14.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.327 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.328 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:14.328 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065733+0000", 2024-09-14T23:58:14.328 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000116492 2024-09-14T23:58:14.328 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.328 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.328 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:14.328 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065892+0000", 2024-09-14T23:58:14.328 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015894100000000001 2024-09-14T23:58:14.328 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:14.328 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:14.328 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:14.328 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.328 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.328 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187220 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1611776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:14.329 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.993899+0000", 2024-09-14T23:58:14.329 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 6.2546467860000003, 2024-09-14T23:58:14.329 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.072298041, 2024-09-14T23:58:14.329 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:14.329 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:14.329 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:14.329 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:14.329 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:14.329 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:14.329 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187220 2024-09-14T23:58:14.329 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.329 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:14.329 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.330 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:14.330 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993899+0000", 2024-09-14T23:58:14.330 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:14.330 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.330 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.330 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:14.330 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993899+0000", 2024-09-14T23:58:14.330 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:14.330 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.330 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.330 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:14.330 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993899+0000", 2024-09-14T23:58:14.330 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2100000000000001e-07 2024-09-14T23:58:14.330 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.330 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.330 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:14.331 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993901+0000", 2024-09-14T23:58:14.331 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.141e-06 2024-09-14T23:58:14.331 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.331 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.331 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:14.331 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993904+0000", 2024-09-14T23:58:14.331 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1470000000000001e-06 2024-09-14T23:58:14.331 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.331 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.331 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:14.331 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993906+0000", 2024-09-14T23:58:14.331 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6340000000000001e-06 2024-09-14T23:58:14.331 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.331 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.331 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:14.332 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996160+0000", 2024-09-14T23:58:14.332 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0022537659999999999 2024-09-14T23:58:14.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.332 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.332 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:14.332 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996169+0000", 2024-09-14T23:58:14.332 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2739999999999996e-06 2024-09-14T23:58:14.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.332 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.332 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:14.332 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065981+0000", 2024-09-14T23:58:14.332 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.069811742 2024-09-14T23:58:14.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.333 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.333 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:14.333 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.066092+0000", 2024-09-14T23:58:14.333 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000110701 2024-09-14T23:58:14.333 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.333 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.333 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:14.333 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.066197+0000", 2024-09-14T23:58:14.333 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010541500000000001 2024-09-14T23:58:14.333 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:14.333 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:14.333 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:14.333 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.333 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.333 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187221 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1600512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:14.334 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.993909+0000", 2024-09-14T23:58:14.334 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 6.2546365740000001, 2024-09-14T23:58:14.334 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.072643499, 2024-09-14T23:58:14.334 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:14.334 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:14.334 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:14.334 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:14.334 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:14.334 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:14.334 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187221 2024-09-14T23:58:14.334 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.334 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:14.334 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.334 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:14.334 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993909+0000", 2024-09-14T23:58:14.335 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:14.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.335 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.335 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:14.335 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993909+0000", 2024-09-14T23:58:14.335 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:14.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.335 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.335 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:14.335 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995673+0000", 2024-09-14T23:58:14.335 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001763141 2024-09-14T23:58:14.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.335 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.335 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:14.336 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995677+0000", 2024-09-14T23:58:14.336 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8709999999999996e-06 2024-09-14T23:58:14.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.336 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.336 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:14.336 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995686+0000", 2024-09-14T23:58:14.336 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6649999999999992e-06 2024-09-14T23:58:14.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.336 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.336 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:14.336 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995691+0000", 2024-09-14T23:58:14.336 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0289999999999996e-06 2024-09-14T23:58:14.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.336 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.336 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:14.337 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996199+0000", 2024-09-14T23:58:14.337 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00050799099999999995 2024-09-14T23:58:14.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.337 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.337 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:14.337 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996205+0000", 2024-09-14T23:58:14.337 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8259999999999998e-06 2024-09-14T23:58:14.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.337 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.337 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:14.337 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.066238+0000", 2024-09-14T23:58:14.337 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.070032928 2024-09-14T23:58:14.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.337 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.337 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:14.338 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.066393+0000", 2024-09-14T23:58:14.338 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015549800000000001 2024-09-14T23:58:14.338 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.338 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.338 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:14.338 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.066553+0000", 2024-09-14T23:58:14.338 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015955 2024-09-14T23:58:14.338 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:14.338 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:14.338 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:14.338 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.338 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.338 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187223 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1604608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:14.338 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.995728+0000", 2024-09-14T23:58:14.338 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 6.2528177579999999, 2024-09-14T23:58:14.339 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.071486478, 2024-09-14T23:58:14.339 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:14.339 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:14.339 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:14.339 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:14.339 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:14.339 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:14.339 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187223 2024-09-14T23:58:14.339 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.339 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:14.339 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.339 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:14.339 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995728+0000", 2024-09-14T23:58:14.339 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:14.339 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.340 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.340 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:14.340 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995728+0000", 2024-09-14T23:58:14.340 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:14.340 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.340 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.340 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:14.340 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995728+0000", 2024-09-14T23:58:14.340 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.65e-07 2024-09-14T23:58:14.340 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.340 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.340 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:14.340 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995730+0000", 2024-09-14T23:58:14.340 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.406e-06 2024-09-14T23:58:14.340 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.341 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.341 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:14.341 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995734+0000", 2024-09-14T23:58:14.341 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2640000000000002e-06 2024-09-14T23:58:14.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.341 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.341 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:14.341 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995737+0000", 2024-09-14T23:58:14.341 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0589999999999998e-06 2024-09-14T23:58:14.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.341 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.341 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:14.341 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996288+0000", 2024-09-14T23:58:14.341 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00055059599999999998 2024-09-14T23:58:14.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.342 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.342 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:14.342 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996294+0000", 2024-09-14T23:58:14.342 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0260000000000004e-06 2024-09-14T23:58:14.342 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.342 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.342 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:14.342 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.066938+0000", 2024-09-14T23:58:14.342 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0706439510000001 2024-09-14T23:58:14.342 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.342 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.342 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:14.342 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067050+0000", 2024-09-14T23:58:14.342 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011220100000000001 2024-09-14T23:58:14.343 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.343 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.343 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:14.343 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067215+0000", 2024-09-14T23:58:14.343 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016470999999999999 2024-09-14T23:58:14.343 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:14.343 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:14.343 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:14.343 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.343 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.343 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187224 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1619968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:14.343 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.995742+0000", 2024-09-14T23:58:14.343 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 6.252803943, 2024-09-14T23:58:14.343 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0717603609999999, 2024-09-14T23:58:14.343 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:14.344 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:14.344 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:14.344 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:14.344 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:14.344 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:14.344 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187224 2024-09-14T23:58:14.344 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.344 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:14.344 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.344 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:14.344 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995742+0000", 2024-09-14T23:58:14.344 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:14.344 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.344 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.345 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:14.345 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995742+0000", 2024-09-14T23:58:14.345 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:14.345 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.345 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.345 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:14.345 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995742+0000", 2024-09-14T23:58:14.345 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2399999999999999e-07 2024-09-14T23:58:14.345 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.345 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.345 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:14.345 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995744+0000", 2024-09-14T23:58:14.345 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.356e-06 2024-09-14T23:58:14.346 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.346 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.346 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:14.346 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995747+0000", 2024-09-14T23:58:14.346 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.304e-06 2024-09-14T23:58:14.346 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.346 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.346 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:14.346 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995750+0000", 2024-09-14T23:58:14.346 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6510000000000001e-06 2024-09-14T23:58:14.346 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.346 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.346 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:14.346 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996327+0000", 2024-09-14T23:58:14.347 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00057794300000000003 2024-09-14T23:58:14.347 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.347 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.347 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:14.347 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996337+0000", 2024-09-14T23:58:14.347 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3759999999999997e-06 2024-09-14T23:58:14.347 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.347 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.347 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:14.347 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067244+0000", 2024-09-14T23:58:14.347 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.070907367 2024-09-14T23:58:14.347 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.347 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.347 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:14.347 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067341+0000", 2024-09-14T23:58:14.347 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7277999999999999e-05 2024-09-14T23:58:14.348 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.348 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.348 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:14.348 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067502+0000", 2024-09-14T23:58:14.348 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016086200000000001 2024-09-14T23:58:14.348 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:14.348 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:14.348 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:14.348 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.348 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.348 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187225 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1608704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:14.348 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.995766+0000", 2024-09-14T23:58:14.348 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 6.252779769, 2024-09-14T23:58:14.348 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.072043649, 2024-09-14T23:58:14.348 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:14.349 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:14.349 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:14.349 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:14.349 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:14.349 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:14.349 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187225 2024-09-14T23:58:14.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.349 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:14.349 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.349 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:14.349 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995766+0000", 2024-09-14T23:58:14.349 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:14.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.349 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.350 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:14.350 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995766+0000", 2024-09-14T23:58:14.350 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:14.350 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.350 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.350 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:14.350 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995766+0000", 2024-09-14T23:58:14.350 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9200000000000002e-07 2024-09-14T23:58:14.350 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.350 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.350 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:14.350 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995768+0000", 2024-09-14T23:58:14.350 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8679999999999999e-06 2024-09-14T23:58:14.350 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.351 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.351 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:14.351 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995772+0000", 2024-09-14T23:58:14.351 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6339999999999999e-06 2024-09-14T23:58:14.351 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.351 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.351 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:14.351 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995775+0000", 2024-09-14T23:58:14.351 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8159999999999998e-06 2024-09-14T23:58:14.351 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.351 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.351 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:14.351 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996372+0000", 2024-09-14T23:58:14.351 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00059758800000000003 2024-09-14T23:58:14.352 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.352 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.352 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:14.352 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996377+0000", 2024-09-14T23:58:14.352 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7770000000000002e-06 2024-09-14T23:58:14.352 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.352 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.352 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:14.352 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067534+0000", 2024-09-14T23:58:14.352 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.071156821 2024-09-14T23:58:14.352 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.352 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.352 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:14.352 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067638+0000", 2024-09-14T23:58:14.353 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000103737 2024-09-14T23:58:14.353 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.353 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.353 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:14.353 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067810+0000", 2024-09-14T23:58:14.353 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017211599999999999 2024-09-14T23:58:14.353 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:14.353 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:14.353 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:14.353 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.353 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.353 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187226 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1624064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:14.353 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.995778+0000", 2024-09-14T23:58:14.353 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 6.2527675900000004, 2024-09-14T23:58:14.353 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.072339081, 2024-09-14T23:58:14.354 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:14.354 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:14.354 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:14.354 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:14.354 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:14.354 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:14.354 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187226 2024-09-14T23:58:14.354 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.354 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:14.354 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.354 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:14.354 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995778+0000", 2024-09-14T23:58:14.354 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:14.355 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.355 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.355 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:14.355 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995778+0000", 2024-09-14T23:58:14.355 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:14.355 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.355 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.355 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:14.355 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995779+0000", 2024-09-14T23:58:14.355 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0699999999999999e-07 2024-09-14T23:58:14.355 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.355 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.355 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:14.355 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995780+0000", 2024-09-14T23:58:14.355 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2139999999999999e-06 2024-09-14T23:58:14.356 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.356 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.356 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:14.356 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995783+0000", 2024-09-14T23:58:14.356 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1250000000000001e-06 2024-09-14T23:58:14.356 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.356 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.356 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:14.356 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995787+0000", 2024-09-14T23:58:14.356 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6679999999999999e-06 2024-09-14T23:58:14.356 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.356 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.356 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:14.356 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996404+0000", 2024-09-14T23:58:14.357 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00061777499999999999 2024-09-14T23:58:14.357 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.357 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.357 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:14.357 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996414+0000", 2024-09-14T23:58:14.357 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.431e-06 2024-09-14T23:58:14.357 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.357 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.357 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:14.357 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067844+0000", 2024-09-14T23:58:14.357 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0714302410000001 2024-09-14T23:58:14.357 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.357 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.357 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:14.357 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067952+0000", 2024-09-14T23:58:14.358 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000107778 2024-09-14T23:58:14.358 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.358 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.358 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:14.358 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.068117+0000", 2024-09-14T23:58:14.358 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000165642 2024-09-14T23:58:14.358 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:14.358 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:14.358 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:14.358 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.358 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.358 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187227 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1612800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:14.358 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.995806+0000", 2024-09-14T23:58:14.358 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 6.2527403530000001, 2024-09-14T23:58:14.359 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0726092810000001, 2024-09-14T23:58:14.359 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:14.359 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:14.359 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:14.359 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:14.359 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:14.359 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:14.359 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187227 2024-09-14T23:58:14.359 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.359 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:14.359 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.359 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:14.359 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995806+0000", 2024-09-14T23:58:14.360 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:14.360 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.360 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.360 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:14.360 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995806+0000", 2024-09-14T23:58:14.360 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:14.360 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.360 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.360 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:14.360 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995672+0000", 2024-09-14T23:58:14.360 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99986608099999996 2024-09-14T23:58:14.360 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.360 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.360 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:14.361 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995677+0000", 2024-09-14T23:58:14.361 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9520000000000002e-06 2024-09-14T23:58:14.361 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.361 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.361 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:14.361 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995686+0000", 2024-09-14T23:58:14.361 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8199999999999992e-06 2024-09-14T23:58:14.361 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.361 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.361 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:14.361 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995693+0000", 2024-09-14T23:58:14.361 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1630000000000001e-06 2024-09-14T23:58:14.361 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.361 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.362 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:14.362 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.068149+0000", 2024-09-14T23:58:14.362 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.072456692000000003 2024-09-14T23:58:14.362 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.362 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.362 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:14.362 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.068251+0000", 2024-09-14T23:58:14.362 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101963 2024-09-14T23:58:14.362 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:14.362 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:14.362 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:14.362 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.068415+0000", 2024-09-14T23:58:14.362 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016360999999999999 2024-09-14T23:58:14.362 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:14.363 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:14.363 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:14.363 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:14.363 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:14.363 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:14.365 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T23:58:14.367 INFO:teuthology.orchestra.run.smithi042.stdout:5943 write 0x66d3f07 thru 0x66dcbdd (0x8cd7 bytes) 2024-09-14T23:58:14.368 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:58:14.368 INFO:teuthology.orchestra.run.smithi042.stdout:5945 read 0x4d7b5df thru 0x4d843ac (0x8dce bytes) 2024-09-14T23:58:14.368 INFO:teuthology.orchestra.run.smithi042.stdout:5948 writesame 0xbaefb6e thru 0xbaf544c data_size 0x1(0x58df bytes) 2024-09-14T23:58:14.368 INFO:teuthology.orchestra.run.smithi042.stdout:5949 read 0x8d85b57 thru 0x8d8da14 (0x7ebe bytes) 2024-09-14T23:58:14.368 INFO:teuthology.orchestra.run.smithi042.stdout:5950 read 0x9ea77a8 thru 0x9eaf6ea (0x7f43 bytes) 2024-09-14T23:58:14.368 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:58:14.368 INFO:teuthology.orchestra.run.smithi042.stdout:5952 trunc from 0xeccf12f to 0xa60cb18 2024-09-14T23:58:14.368 INFO:teuthology.orchestra.run.smithi042.stdout:5953 writesame 0x882d18c thru 0x882e7af data_size 0xd(0x1624 bytes) 2024-09-14T23:58:14.368 INFO:teuthology.orchestra.run.smithi042.stdout:5954 writesame 0x23232b0 thru 0x2332d5f data_size 0xa8(0xfab0 bytes) 2024-09-14T23:58:14.368 INFO:teuthology.orchestra.run.smithi042.stdout:5955 write 0x3011f1a thru 0x3012769 (0x850 bytes) 2024-09-14T23:58:14.368 INFO:teuthology.orchestra.run.smithi042.stdout:5956 read 0x6d62bc1 thru 0x6d63466 (0x8a6 bytes) 2024-09-14T23:58:14.368 INFO:teuthology.orchestra.run.smithi042.stdout:5957 trunc from 0xa60cb18 to 0x9bf565b 2024-09-14T23:58:14.368 INFO:teuthology.orchestra.run.smithi042.stdout:5959 write 0x3c89cad thru 0x3c9157c (0x78d0 bytes) 2024-09-14T23:58:14.369 INFO:teuthology.orchestra.run.smithi042.stdout:5960 read 0x810ca57 thru 0x8118d45 (0xc2ef bytes) 2024-09-14T23:58:14.369 INFO:teuthology.orchestra.run.smithi042.stdout:5961 trunc from 0x9bf565b to 0xe569be9 2024-09-14T23:58:14.369 INFO:teuthology.orchestra.run.smithi042.stdout:5962 read 0x9ddeeda thru 0x9debb8d (0xccb4 bytes) 2024-09-14T23:58:14.369 INFO:teuthology.orchestra.run.smithi042.stdout:5963 read 0x12af2a4 thru 0x12b3c05 (0x4962 bytes) 2024-09-14T23:58:14.369 INFO:teuthology.orchestra.run.smithi042.stdout:5964 read 0x888486e thru 0x8888d0a (0x449d bytes) 2024-09-14T23:58:14.369 INFO:teuthology.orchestra.run.smithi042.stdout:5965 read 0xc39f7d0 thru 0xc3ad3a7 (0xdbd8 bytes) 2024-09-14T23:58:14.369 INFO:teuthology.orchestra.run.smithi042.stdout:5966 punch from 0x17dd760 to 0x17e3e51, (0x66f1 bytes) 2024-09-14T23:58:14.369 INFO:teuthology.orchestra.run.smithi042.stdout:5967 read 0xad506e1 thru 0xad5704f (0x696f bytes) 2024-09-14T23:58:14.369 INFO:teuthology.orchestra.run.smithi042.stdout:5968 trunc from 0xe569be9 to 0x7040182 2024-09-14T23:58:14.369 INFO:teuthology.orchestra.run.smithi042.stdout:5969 write 0x6b13adf thru 0x6b18cca (0x51ec bytes) 2024-09-14T23:58:14.369 INFO:teuthology.orchestra.run.smithi042.stdout:5970 write 0xd53cfc6 thru 0xd54331a (0x6355 bytes) 2024-09-14T23:58:14.369 INFO:teuthology.orchestra.run.smithi042.stdout:5971 write 0x96c58db thru 0x96d2f20 (0xd646 bytes) 2024-09-14T23:58:14.369 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:58:14.370 INFO:teuthology.orchestra.run.smithi042.stdout:5973 punch from 0x87b4d2d to 0x87c0fcd, (0xc2a0 bytes) 2024-09-14T23:58:14.370 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:58:14.370 INFO:teuthology.orchestra.run.smithi042.stdout:5975 punch from 0xb0c6b93 to 0xb0ccd1b, (0x6188 bytes) 2024-09-14T23:58:14.370 INFO:teuthology.orchestra.run.smithi042.stdout:5976 write 0xcda29b0 thru 0xcdac82f (0x9e80 bytes) 2024-09-14T23:58:14.370 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:58:14.370 INFO:teuthology.orchestra.run.smithi042.stdout:5983 read 0x206785d thru 0x206da57 (0x61fb bytes) 2024-09-14T23:58:14.370 INFO:teuthology.orchestra.run.smithi042.stdout:5985 read 0x22f390 thru 0x23c2e8 (0xcf59 bytes) 2024-09-14T23:58:14.370 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:58:14.371 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:58:14.372 INFO:teuthology.orchestra.run.smithi042.stdout:5986 writesame 0x47bd414 thru 0x47c4dc3 data_size 0x58(0x79b0 bytes) 2024-09-14T23:58:14.372 INFO:teuthology.orchestra.run.smithi042.stdout:5987 read 0x23a6844 thru 0x23b092c (0xa0e9 bytes) 2024-09-14T23:58:14.372 INFO:teuthology.orchestra.run.smithi042.stdout:5988 punch from 0xca79ad3 to 0xca82b26, (0x9053 bytes) 2024-09-14T23:58:14.372 INFO:teuthology.orchestra.run.smithi042.stdout:5990 read 0x3f660f4 thru 0x3f67da8 (0x1cb5 bytes) 2024-09-14T23:58:14.372 INFO:teuthology.orchestra.run.smithi042.stdout:5991 writesame 0xe5ae8f1 thru 0xe5bb544 data_size 0x12(0xcc54 bytes) 2024-09-14T23:58:14.372 INFO:teuthology.orchestra.run.smithi042.stdout:5994 read 0x14734ec thru 0x1476a34 (0x3549 bytes) 2024-09-14T23:58:14.372 INFO:teuthology.orchestra.run.smithi042.stdout:5996 punch from 0x4f77b0 to 0x4fadbf, (0x360f bytes) 2024-09-14T23:58:14.372 INFO:teuthology.orchestra.run.smithi042.stdout:5997 write 0xd0063ce thru 0xd011f87 (0xbbba bytes) 2024-09-14T23:58:14.372 INFO:teuthology.orchestra.run.smithi042.stdout:compare and write disabled 2024-09-14T23:58:14.372 INFO:teuthology.orchestra.run.smithi042.stdout:5999 writesame 0xee55520 thru 0xee62aa9 data_size 0x9(0xd58a bytes) 2024-09-14T23:58:14.372 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:14.362+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:14.373 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:58:14.373 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:58:14.373 INFO:teuthology.orchestra.run.smithi042.stdout:6000 trunc from 0xee62aaa to 0x7b1f986 2024-09-14T23:58:14.385 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:58:14.385 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:58:14.402 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:58:14.403 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:58:14.420 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:58:14.421 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:58:14.438 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:58:14.439 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:58:14.456 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:58:14.456 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:58:14.462 INFO:teuthology.orchestra.run.smithi042.stdout:checking clone #48, image image_client.0-clone48 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent49 2024-09-14T23:58:14.465 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:14.463+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:14.475 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:58:14.475 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:58:14.529 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:14.529 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:14.543 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T23:58:14.584 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:14.582+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:14.666 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:58:14.664+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:58:14.719 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:14.719 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:14.730 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T23:58:14.768 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:14.766+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:14.869 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:14.868+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:14.878 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:14.878 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:14.886 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T23:58:14.970 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:58:14.968+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:58:14.995 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:14.995 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:15.006 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T23:58:15.072 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:15.070+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:15.172 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:15.171+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:15.183 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:15.183 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:15.193 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T23:58:15.274 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:15.271+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:15.326 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:15.327 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:15.339 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T23:58:15.380 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:15.378+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:15.475 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:15.473+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:15.524 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:15.524 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:15.534 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T23:58:15.576 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:15.574+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:15.676 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:58:15.674+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:58:15.715 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:15.716 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:15.725 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T23:58:15.777 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:15.775+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:15.877 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:15.876+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:15.911 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:15.911 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:15.922 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T23:58:15.979 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:15.977+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:16.079 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:16.078+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:16.096 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:16.096 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:16.109 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T23:58:16.181 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:16.179+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:16.216 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T23:58:16.277 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:16.277 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:16.281 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:16.280+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:16.289 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T23:58:16.382 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:16.381+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:16.453 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:16.453 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:16.466 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T23:58:16.483 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:16.481+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:16.491 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T23:58:16.584 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:16.582+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:16.632 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:16.632 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:16.646 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T23:58:16.690 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:16.689+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:16.710 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:58:16.711 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:58:16.736 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:58:16.736 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:58:16.761 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:58:16.761 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:58:16.783 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:58:16.784 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:58:16.785 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:16.783+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:16.796 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:16.796 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:16.796 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:58:16.796 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:58:16.796 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:16.807 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T23:58:16.809 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:58:16.809 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:58:16.835 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:58:16.835 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:58:16.865 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:58:16.865 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:58:16.886 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:16.884+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:16.896 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:58:16.896 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:58:16.964 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:16.965 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:58:16.965 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:58:16.965 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:58:16.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.965 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5882 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:16.965 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:58:16.965 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 97.585812325999996, 2024-09-14T23:58:16.965 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050549666, 2024-09-14T23:58:16.965 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:16.965 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:16.965 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:16.965 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:16.965 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:16.965 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:16.966 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5882 2024-09-14T23:58:16.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.966 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:16.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.966 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:16.966 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:58:16.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:16.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.966 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:16.966 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:58:16.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:16.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.966 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:16.967 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:58:16.967 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8099999999999998e-07 2024-09-14T23:58:16.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.967 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:16.967 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375842+0000", 2024-09-14T23:58:16.967 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9900000000000009e-07 2024-09-14T23:58:16.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.967 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:16.967 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375845+0000", 2024-09-14T23:58:16.967 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.055e-06 2024-09-14T23:58:16.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.967 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:16.968 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375848+0000", 2024-09-14T23:58:16.968 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7609999999999999e-06 2024-09-14T23:58:16.968 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.968 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.968 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:16.968 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426381+0000", 2024-09-14T23:58:16.968 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050533392000000003 2024-09-14T23:58:16.968 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.968 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.968 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:16.968 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426390+0000", 2024-09-14T23:58:16.968 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0780000000000002e-06 2024-09-14T23:58:16.968 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:16.969 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:16.969 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:16.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.969 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.969 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5884 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:16.969 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:58:16.969 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 97.585802185999995, 2024-09-14T23:58:16.969 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050717439000000003, 2024-09-14T23:58:16.969 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:16.969 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:16.969 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:16.969 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:16.969 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:16.969 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:16.970 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5884 2024-09-14T23:58:16.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.970 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:16.970 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.970 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:16.970 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:58:16.970 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:16.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.970 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.970 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:16.970 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:58:16.970 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:16.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.970 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.970 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:16.971 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:58:16.971 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0800000000000001e-07 2024-09-14T23:58:16.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.971 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:16.971 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375852+0000", 2024-09-14T23:58:16.971 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.02e-06 2024-09-14T23:58:16.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.971 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:16.971 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375855+0000", 2024-09-14T23:58:16.971 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9900000000000002e-06 2024-09-14T23:58:16.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.972 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:16.972 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375857+0000", 2024-09-14T23:58:16.972 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3499999999999999e-06 2024-09-14T23:58:16.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.972 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.972 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:16.972 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426556+0000", 2024-09-14T23:58:16.972 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050698139000000003 2024-09-14T23:58:16.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.972 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.972 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:16.972 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426568+0000", 2024-09-14T23:58:16.972 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2731999999999999e-05 2024-09-14T23:58:16.973 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:16.973 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:16.973 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:16.973 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.973 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.973 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5885 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:16.973 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:58:16.973 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 97.585783160999995, 2024-09-14T23:58:16.973 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050737333000000003, 2024-09-14T23:58:16.973 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:16.973 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:16.973 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:16.973 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:16.973 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:16.974 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:16.974 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5885 2024-09-14T23:58:16.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.974 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:16.974 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.974 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:16.974 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:58:16.974 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:16.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.974 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.974 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:16.974 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:58:16.974 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:16.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.975 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.975 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:16.975 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:58:16.975 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5899999999999998e-07 2024-09-14T23:58:16.975 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.975 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.975 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:16.975 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375872+0000", 2024-09-14T23:58:16.975 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4219999999999999e-06 2024-09-14T23:58:16.975 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.975 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.975 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:16.975 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375878+0000", 2024-09-14T23:58:16.975 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4980000000000003e-06 2024-09-14T23:58:16.976 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.976 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.976 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:16.976 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375881+0000", 2024-09-14T23:58:16.976 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6110000000000001e-06 2024-09-14T23:58:16.976 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.976 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.976 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:16.976 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426596+0000", 2024-09-14T23:58:16.976 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050715320000000001 2024-09-14T23:58:16.976 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.976 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.976 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:16.977 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426607+0000", 2024-09-14T23:58:16.977 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1223000000000001e-05 2024-09-14T23:58:16.977 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:16.977 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:16.977 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:16.977 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.977 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.977 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5886 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:16.977 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:58:16.977 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 97.585769048000003, 2024-09-14T23:58:16.977 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050803395000000001, 2024-09-14T23:58:16.977 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:16.977 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:16.977 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:16.978 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:16.978 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:16.978 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:16.978 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5886 2024-09-14T23:58:16.978 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.978 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:16.978 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.978 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:16.978 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:58:16.978 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:16.978 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.978 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.978 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:16.978 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:58:16.979 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:16.979 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.979 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.979 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:16.979 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:58:16.979 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2700000000000001e-07 2024-09-14T23:58:16.979 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.979 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.979 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:16.979 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375886+0000", 2024-09-14T23:58:16.979 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7060000000000001e-06 2024-09-14T23:58:16.979 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.979 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.979 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:16.980 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375890+0000", 2024-09-14T23:58:16.980 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2259999999999999e-06 2024-09-14T23:58:16.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.980 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.980 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:16.980 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375893+0000", 2024-09-14T23:58:16.980 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0819999999999999e-06 2024-09-14T23:58:16.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.980 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.980 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:16.980 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426677+0000", 2024-09-14T23:58:16.980 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050784158000000003 2024-09-14T23:58:16.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.980 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.981 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:16.981 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426687+0000", 2024-09-14T23:58:16.981 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9960000000000005e-06 2024-09-14T23:58:16.981 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:16.981 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:16.981 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:16.981 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.981 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.981 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5887 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:16.981 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:58:16.981 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 97.585746962000002, 2024-09-14T23:58:16.981 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050833070000000001, 2024-09-14T23:58:16.981 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:16.982 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:16.982 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:16.982 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:16.982 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:16.982 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:16.982 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5887 2024-09-14T23:58:16.982 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.982 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:16.982 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.982 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:16.982 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:58:16.982 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:16.982 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.982 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.983 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:16.983 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:58:16.983 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:16.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.983 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.983 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:16.983 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:58:16.983 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3299999999999998e-07 2024-09-14T23:58:16.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.983 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.983 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:16.983 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375908+0000", 2024-09-14T23:58:16.983 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7689999999999999e-06 2024-09-14T23:58:16.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.984 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.984 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:16.984 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375912+0000", 2024-09-14T23:58:16.984 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.535e-06 2024-09-14T23:58:16.984 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.984 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.984 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:16.984 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375916+0000", 2024-09-14T23:58:16.984 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8979999999999998e-06 2024-09-14T23:58:16.984 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.984 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.984 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:16.984 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426729+0000", 2024-09-14T23:58:16.984 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050813453000000001 2024-09-14T23:58:16.985 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.985 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.985 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:16.985 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426739+0000", 2024-09-14T23:58:16.985 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0081999999999999e-05 2024-09-14T23:58:16.985 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:16.985 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:16.985 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:16.985 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.985 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.985 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5888 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:16.985 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:58:16.985 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 97.585733884999996, 2024-09-14T23:58:16.985 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050854652, 2024-09-14T23:58:16.985 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:16.986 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:16.986 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:16.986 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:16.986 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:16.986 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:16.986 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5888 2024-09-14T23:58:16.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.986 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:16.986 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.986 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:16.986 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:58:16.986 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:16.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.986 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.986 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:16.987 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:58:16.987 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:16.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.987 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.987 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:16.987 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:58:16.987 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1400000000000001e-07 2024-09-14T23:58:16.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.987 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.987 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:16.987 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375921+0000", 2024-09-14T23:58:16.987 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.119e-06 2024-09-14T23:58:16.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.987 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.987 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:16.987 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375928+0000", 2024-09-14T23:58:16.988 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3170000000000004e-06 2024-09-14T23:58:16.988 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.988 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.988 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:16.988 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375931+0000", 2024-09-14T23:58:16.988 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6970000000000002e-06 2024-09-14T23:58:16.988 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.988 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.988 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:16.988 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426768+0000", 2024-09-14T23:58:16.988 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050837355000000001 2024-09-14T23:58:16.988 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.988 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.988 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:16.988 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426774+0000", 2024-09-14T23:58:16.989 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9499999999999998e-06 2024-09-14T23:58:16.989 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:16.989 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:16.989 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:16.989 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.989 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.989 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5889 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:16.989 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:58:16.989 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 97.585716961000003, 2024-09-14T23:58:16.989 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050885446000000001, 2024-09-14T23:58:16.989 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:16.989 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:16.989 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:16.989 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:16.990 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:16.990 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:16.990 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5889 2024-09-14T23:58:16.990 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.990 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:16.990 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.990 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:16.990 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:58:16.990 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:16.990 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.990 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.990 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:16.990 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:58:16.990 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:16.991 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.991 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.991 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:16.991 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:58:16.991 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.36e-07 2024-09-14T23:58:16.991 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.991 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.991 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:16.991 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375940+0000", 2024-09-14T23:58:16.991 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.844e-06 2024-09-14T23:58:16.991 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.991 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.991 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:16.991 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375944+0000", 2024-09-14T23:58:16.992 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6720000000000002e-06 2024-09-14T23:58:16.992 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.992 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.992 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:16.992 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375946+0000", 2024-09-14T23:58:16.992 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5629999999999999e-06 2024-09-14T23:58:16.992 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.992 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.992 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:16.992 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426810+0000", 2024-09-14T23:58:16.992 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050863255000000003 2024-09-14T23:58:16.992 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.992 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.992 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:16.993 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426822+0000", 2024-09-14T23:58:16.993 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1875999999999999e-05 2024-09-14T23:58:16.993 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:16.993 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:16.993 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:16.993 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.993 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.993 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5890 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:16.993 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:58:16.993 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 97.585703148999997, 2024-09-14T23:58:16.993 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050917549999999999, 2024-09-14T23:58:16.993 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:16.993 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:16.994 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:16.994 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:16.994 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:16.994 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:16.994 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5890 2024-09-14T23:58:16.994 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.994 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:16.994 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.994 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:16.994 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:58:16.994 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:16.994 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.994 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.994 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:16.994 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:58:16.995 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:16.995 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.995 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.995 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:16.995 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:58:16.995 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4299999999999999e-07 2024-09-14T23:58:16.995 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.995 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.995 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:16.995 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375951+0000", 2024-09-14T23:58:16.995 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1000000000000001e-06 2024-09-14T23:58:16.995 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.995 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.995 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:16.996 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375956+0000", 2024-09-14T23:58:16.996 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3370000000000001e-06 2024-09-14T23:58:16.996 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.996 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.996 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:16.996 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375958+0000", 2024-09-14T23:58:16.996 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4820000000000001e-06 2024-09-14T23:58:16.996 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.996 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.996 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:16.996 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426858+0000", 2024-09-14T23:58:16.996 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050899666000000003 2024-09-14T23:58:16.996 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.997 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.997 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:16.997 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426867+0000", 2024-09-14T23:58:16.997 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7219999999999994e-06 2024-09-14T23:58:16.997 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:16.997 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:16.997 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:16.997 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.997 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.997 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5891 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:16.997 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:58:16.997 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 97.585691827000005, 2024-09-14T23:58:16.997 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050948067, 2024-09-14T23:58:16.997 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:16.998 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:16.998 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:16.998 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:16.998 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:16.998 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:16.998 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5891 2024-09-14T23:58:16.998 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.998 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:16.998 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.998 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:16.998 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:58:16.998 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:16.998 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.998 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.999 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:16.999 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:58:16.999 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:16.999 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.999 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.999 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:16.999 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:58:16.999 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0800000000000001e-07 2024-09-14T23:58:16.999 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:16.999 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:16.999 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:16.999 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375962+0000", 2024-09-14T23:58:16.999 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9600000000000008e-07 2024-09-14T23:58:17.000 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.000 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.000 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:17.000 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375970+0000", 2024-09-14T23:58:17.000 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1600000000000001e-06 2024-09-14T23:58:17.000 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.000 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.000 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:17.000 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375972+0000", 2024-09-14T23:58:17.000 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4849999999999999e-06 2024-09-14T23:58:17.000 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.000 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.000 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:17.000 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426902+0000", 2024-09-14T23:58:17.000 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050929813999999997 2024-09-14T23:58:17.001 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.001 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.001 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:17.001 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426909+0000", 2024-09-14T23:58:17.001 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4039999999999998e-06 2024-09-14T23:58:17.001 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.001 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:17.001 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.001 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.001 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.001 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5892 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:17.001 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:58:17.001 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 97.585676969999994, 2024-09-14T23:58:17.001 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050965036999999998, 2024-09-14T23:58:17.002 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:17.002 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:17.002 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:17.002 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:17.002 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:17.002 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:17.002 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5892 2024-09-14T23:58:17.002 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.002 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:17.002 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.002 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:17.002 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:58:17.002 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.002 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.003 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.003 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:17.003 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:58:17.003 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.003 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.003 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.003 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:17.003 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:58:17.003 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.34e-07 2024-09-14T23:58:17.003 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.003 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.003 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:17.004 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375978+0000", 2024-09-14T23:58:17.004 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1880000000000001e-06 2024-09-14T23:58:17.004 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.004 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.004 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:17.004 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375981+0000", 2024-09-14T23:58:17.004 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2399999999999999e-06 2024-09-14T23:58:17.004 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.004 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.004 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:17.004 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375984+0000", 2024-09-14T23:58:17.004 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9110000000000002e-06 2024-09-14T23:58:17.004 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.004 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.005 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:17.005 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426934+0000", 2024-09-14T23:58:17.005 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050949912 2024-09-14T23:58:17.005 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.005 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.005 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:17.005 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426941+0000", 2024-09-14T23:58:17.005 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5519999999999999e-06 2024-09-14T23:58:17.005 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.005 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:17.005 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.005 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.005 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.005 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5893 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:17.005 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:58:17.006 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 97.585666372999995, 2024-09-14T23:58:17.006 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050975131999999999, 2024-09-14T23:58:17.006 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:17.006 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:17.006 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:17.006 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:17.006 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:17.006 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:17.006 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5893 2024-09-14T23:58:17.006 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.006 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:17.006 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.006 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:17.006 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:58:17.006 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.007 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.007 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.007 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:17.007 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:58:17.007 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.007 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.007 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.007 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:17.007 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:58:17.007 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2999999999999999e-07 2024-09-14T23:58:17.007 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.007 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.007 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:17.007 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375988+0000", 2024-09-14T23:58:17.007 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0410000000000001e-06 2024-09-14T23:58:17.007 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.008 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.008 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:17.008 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375992+0000", 2024-09-14T23:58:17.008 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.104e-06 2024-09-14T23:58:17.008 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.008 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.008 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:17.008 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375995+0000", 2024-09-14T23:58:17.008 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4310000000000001e-06 2024-09-14T23:58:17.008 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.008 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.008 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:17.008 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426958+0000", 2024-09-14T23:58:17.008 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050963642000000003 2024-09-14T23:58:17.008 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.009 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.009 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:17.009 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426962+0000", 2024-09-14T23:58:17.009 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6839999999999998e-06 2024-09-14T23:58:17.009 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.009 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:17.009 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.009 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.009 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.009 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5894 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:17.009 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:58:17.009 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 97.585654973000004, 2024-09-14T23:58:17.009 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050978333000000001, 2024-09-14T23:58:17.009 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:17.010 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:17.010 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:17.010 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:17.010 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:17.010 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:17.010 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5894 2024-09-14T23:58:17.010 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.010 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:17.010 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.010 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:17.010 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:58:17.010 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.010 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.010 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.011 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:17.011 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:58:17.011 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.011 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.011 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.011 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:17.011 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:58:17.011 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.23e-07 2024-09-14T23:58:17.011 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.011 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.011 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:17.011 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375999+0000", 2024-09-14T23:58:17.011 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9800000000000002e-07 2024-09-14T23:58:17.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.012 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.012 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:17.012 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376007+0000", 2024-09-14T23:58:17.012 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8539999999999997e-06 2024-09-14T23:58:17.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.012 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.012 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:17.012 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376011+0000", 2024-09-14T23:58:17.012 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.399e-06 2024-09-14T23:58:17.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.012 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.012 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:17.012 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426973+0000", 2024-09-14T23:58:17.013 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050962454999999997 2024-09-14T23:58:17.013 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.013 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.013 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:17.013 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426976+0000", 2024-09-14T23:58:17.013 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4039999999999999e-06 2024-09-14T23:58:17.013 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.013 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:17.013 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.013 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.013 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.013 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5895 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:17.013 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376014+0000", 2024-09-14T23:58:17.013 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 97.585638629000002, 2024-09-14T23:58:17.014 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050975951999999998, 2024-09-14T23:58:17.014 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:17.014 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:17.014 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:17.014 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:17.014 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:17.014 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:17.014 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5895 2024-09-14T23:58:17.014 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.014 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:17.014 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.014 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:17.014 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376014+0000", 2024-09-14T23:58:17.014 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.015 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.015 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:17.015 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376014+0000", 2024-09-14T23:58:17.015 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.015 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.015 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:17.015 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376015+0000", 2024-09-14T23:58:17.015 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2999999999999999e-07 2024-09-14T23:58:17.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.015 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.015 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:17.015 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376016+0000", 2024-09-14T23:58:17.015 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1659999999999999e-06 2024-09-14T23:58:17.016 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.016 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.016 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:17.016 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376024+0000", 2024-09-14T23:58:17.016 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8930000000000005e-06 2024-09-14T23:58:17.016 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.016 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.016 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:17.016 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376026+0000", 2024-09-14T23:58:17.016 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6869999999999999e-06 2024-09-14T23:58:17.016 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.016 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.016 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:17.016 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426987+0000", 2024-09-14T23:58:17.017 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050960601000000001 2024-09-14T23:58:17.017 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.017 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.017 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:17.017 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426990+0000", 2024-09-14T23:58:17.017 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3749999999999999e-06 2024-09-14T23:58:17.017 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.017 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:17.017 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.017 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.017 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.017 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5896 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:17.017 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:58:17.017 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 97.585622892000004, 2024-09-14T23:58:17.018 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050975656000000001, 2024-09-14T23:58:17.018 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:17.018 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:17.018 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:17.018 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:17.018 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:17.018 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:17.018 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5896 2024-09-14T23:58:17.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.018 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:17.018 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.018 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:17.018 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:58:17.018 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.019 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.019 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.019 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:17.019 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:58:17.019 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.019 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.019 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.019 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:17.019 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:58:17.019 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5100000000000001e-07 2024-09-14T23:58:17.019 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.019 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.019 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:17.019 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376032+0000", 2024-09-14T23:58:17.019 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.141e-06 2024-09-14T23:58:17.020 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.020 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.020 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:17.020 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376036+0000", 2024-09-14T23:58:17.020 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.25e-06 2024-09-14T23:58:17.020 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.020 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.020 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:17.020 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376038+0000", 2024-09-14T23:58:17.020 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5469999999999999e-06 2024-09-14T23:58:17.020 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.020 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.021 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:17.021 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427002+0000", 2024-09-14T23:58:17.021 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050964029000000001 2024-09-14T23:58:17.021 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.021 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.021 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:17.021 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427006+0000", 2024-09-14T23:58:17.021 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4379999999999999e-06 2024-09-14T23:58:17.021 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.021 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:17.021 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.021 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.021 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.022 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5897 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:17.022 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:58:17.022 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 97.585611091999994, 2024-09-14T23:58:17.022 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050978001000000002, 2024-09-14T23:58:17.022 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:17.022 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:17.022 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:17.022 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:17.022 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:17.022 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:17.022 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5897 2024-09-14T23:58:17.022 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.022 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:17.022 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.023 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:17.023 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:58:17.023 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.023 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.023 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:17.023 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:58:17.023 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.023 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.023 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:17.023 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:58:17.023 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2600000000000001e-07 2024-09-14T23:58:17.024 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.024 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.024 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:17.024 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376043+0000", 2024-09-14T23:58:17.024 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.054e-06 2024-09-14T23:58:17.024 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.024 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.024 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:17.024 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376047+0000", 2024-09-14T23:58:17.025 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0659999999999997e-06 2024-09-14T23:58:17.025 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.025 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.025 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:17.025 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376050+0000", 2024-09-14T23:58:17.025 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.88e-06 2024-09-14T23:58:17.025 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.025 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.025 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:17.025 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427017+0000", 2024-09-14T23:58:17.025 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050966453000000002 2024-09-14T23:58:17.025 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.025 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.025 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:17.026 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427020+0000", 2024-09-14T23:58:17.026 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3220000000000001e-06 2024-09-14T23:58:17.026 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.026 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:17.026 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.026 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.026 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.026 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5898 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:17.026 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376053+0000", 2024-09-14T23:58:17.026 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 97.585599607000006, 2024-09-14T23:58:17.026 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050982024000000001, 2024-09-14T23:58:17.026 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:17.026 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:17.027 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:17.027 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:17.027 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:17.027 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:17.027 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5898 2024-09-14T23:58:17.027 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.027 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:17.027 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.027 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:17.027 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376053+0000", 2024-09-14T23:58:17.027 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.027 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.027 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.027 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:17.028 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376053+0000", 2024-09-14T23:58:17.028 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.028 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.028 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:17.028 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376278+0000", 2024-09-14T23:58:17.028 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022426400000000001 2024-09-14T23:58:17.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.028 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.028 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:17.028 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376282+0000", 2024-09-14T23:58:17.028 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4739999999999999e-06 2024-09-14T23:58:17.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.028 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.029 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:17.029 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376296+0000", 2024-09-14T23:58:17.029 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4229999999999999e-05 2024-09-14T23:58:17.029 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.029 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.029 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:17.029 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376303+0000", 2024-09-14T23:58:17.029 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2199999999999997e-06 2024-09-14T23:58:17.029 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.029 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.029 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:17.029 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427031+0000", 2024-09-14T23:58:17.029 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050728154999999997 2024-09-14T23:58:17.029 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.030 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.030 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:17.030 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427036+0000", 2024-09-14T23:58:17.030 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6809999999999997e-06 2024-09-14T23:58:17.030 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.030 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:17.030 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.030 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.030 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.030 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5899 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:17.030 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376338+0000", 2024-09-14T23:58:17.030 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 97.585314882999995, 2024-09-14T23:58:17.030 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050712100000000003, 2024-09-14T23:58:17.031 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:17.031 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:17.031 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:17.031 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:17.031 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:17.031 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:17.031 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5899 2024-09-14T23:58:17.031 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.031 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:17.031 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.031 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:17.031 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376338+0000", 2024-09-14T23:58:17.031 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.031 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.031 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.031 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:17.032 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376338+0000", 2024-09-14T23:58:17.032 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.032 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.032 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.032 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:17.032 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376339+0000", 2024-09-14T23:58:17.032 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9100000000000003e-07 2024-09-14T23:58:17.032 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.032 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.032 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:17.032 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376342+0000", 2024-09-14T23:58:17.032 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7e-06 2024-09-14T23:58:17.032 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.032 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.033 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:17.033 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376350+0000", 2024-09-14T23:58:17.033 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5280000000000003e-06 2024-09-14T23:58:17.033 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.033 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.033 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:17.033 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376356+0000", 2024-09-14T23:58:17.033 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6010000000000002e-06 2024-09-14T23:58:17.033 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.033 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.033 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:17.033 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427047+0000", 2024-09-14T23:58:17.033 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050690962999999999 2024-09-14T23:58:17.033 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.034 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.034 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:17.034 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427050+0000", 2024-09-14T23:58:17.034 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.517e-06 2024-09-14T23:58:17.034 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.034 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:17.034 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.034 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.034 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5900 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:17.034 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376366+0000", 2024-09-14T23:58:17.034 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 97.585286624000005, 2024-09-14T23:58:17.034 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050697858999999998, 2024-09-14T23:58:17.034 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:17.035 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:17.035 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:17.035 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:17.035 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:17.035 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:17.035 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5900 2024-09-14T23:58:17.035 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.035 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:17.035 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.035 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:17.035 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376366+0000", 2024-09-14T23:58:17.035 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.035 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.035 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.036 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:17.036 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376366+0000", 2024-09-14T23:58:17.036 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.036 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.036 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:17.036 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377650+0000", 2024-09-14T23:58:17.036 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001283834 2024-09-14T23:58:17.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.036 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.036 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:17.036 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377655+0000", 2024-09-14T23:58:17.036 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7230000000000004e-06 2024-09-14T23:58:17.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.037 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.037 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:17.037 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377667+0000", 2024-09-14T23:58:17.037 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1746000000000001e-05 2024-09-14T23:58:17.037 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.037 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.037 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:17.037 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377676+0000", 2024-09-14T23:58:17.037 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3619999999999995e-06 2024-09-14T23:58:17.037 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.037 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.037 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:17.037 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427061+0000", 2024-09-14T23:58:17.037 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049384755000000002 2024-09-14T23:58:17.038 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.038 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.038 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:17.038 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427064+0000", 2024-09-14T23:58:17.038 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.439e-06 2024-09-14T23:58:17.038 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.038 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:17.038 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.038 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.038 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.038 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:132622 3.1e 3:79d0a5e0:::rbd_data.111847077d5e.0000000000000007:head [sparse-read 0~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T23:58:17.038 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:44.037901+0000", 2024-09-14T23:58:17.038 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.923752014999998, 2024-09-14T23:58:17.038 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1083420859999999, 2024-09-14T23:58:17.039 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:17.039 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:17.039 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:17.039 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:17.039 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:17.039 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:17.039 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 132622 2024-09-14T23:58:17.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.039 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:17.039 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.039 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:17.039 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037901+0000", 2024-09-14T23:58:17.039 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.039 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.040 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:17.040 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037901+0000", 2024-09-14T23:58:17.040 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.040 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.040 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:17.040 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037903+0000", 2024-09-14T23:58:17.040 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4619999999999999e-06 2024-09-14T23:58:17.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.040 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.040 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:17.040 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037910+0000", 2024-09-14T23:58:17.040 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7449999999999995e-06 2024-09-14T23:58:17.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.041 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.041 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:17.041 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037928+0000", 2024-09-14T23:58:17.041 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7305000000000001e-05 2024-09-14T23:58:17.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.041 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.041 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:17.041 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037939+0000", 2024-09-14T23:58:17.041 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1599e-05 2024-09-14T23:58:17.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.041 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.041 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:17.041 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.038049+0000", 2024-09-14T23:58:17.041 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000109577 2024-09-14T23:58:17.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.042 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.042 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op must wait for map", 2024-09-14T23:58:17.042 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.038057+0000", 2024-09-14T23:58:17.042 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1450000000000008e-06 2024-09-14T23:58:17.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.042 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.042 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:17.042 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.039937+0000", 2024-09-14T23:58:17.042 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00188024 2024-09-14T23:58:17.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.042 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.042 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:58:17.043 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.040039+0000", 2024-09-14T23:58:17.043 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000101934 2024-09-14T23:58:17.043 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.043 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.043 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:17.043 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:45.145805+0000", 2024-09-14T23:58:17.043 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.105765651 2024-09-14T23:58:17.043 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.043 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.043 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:17.043 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:45.145997+0000", 2024-09-14T23:58:17.043 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000192084 2024-09-14T23:58:17.043 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.043 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.044 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:17.044 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:45.146243+0000", 2024-09-14T23:58:17.044 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024634400000000001 2024-09-14T23:58:17.044 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.044 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:17.044 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.044 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.044 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.044 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4172.0:5 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:17.044 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.482606+0000", 2024-09-14T23:58:17.044 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.4790470340000006, 2024-09-14T23:58:17.044 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.49381641300000001, 2024-09-14T23:58:17.044 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:17.044 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:17.044 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:17.045 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:17.045 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4172", 2024-09-14T23:58:17.045 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/2542269203", 2024-09-14T23:58:17.045 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5 2024-09-14T23:58:17.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.045 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:17.045 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.045 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:17.045 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482606+0000", 2024-09-14T23:58:17.045 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.045 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.045 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:17.045 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482606+0000", 2024-09-14T23:58:17.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.046 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:17.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482607+0000", 2024-09-14T23:58:17.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3120000000000001e-06 2024-09-14T23:58:17.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.046 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:17.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482615+0000", 2024-09-14T23:58:17.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9300000000000003e-06 2024-09-14T23:58:17.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.046 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:17.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482632+0000", 2024-09-14T23:58:17.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6966999999999999e-05 2024-09-14T23:58:17.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:17.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482640+0000", 2024-09-14T23:58:17.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1820000000000006e-06 2024-09-14T23:58:17.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:17.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482703+0000", 2024-09-14T23:58:17.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2098000000000001e-05 2024-09-14T23:58:17.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:58:17.048 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482713+0000", 2024-09-14T23:58:17.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0064000000000001e-05 2024-09-14T23:58:17.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.048 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:17.048 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.976307+0000", 2024-09-14T23:58:17.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.49359442999999997 2024-09-14T23:58:17.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.048 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:17.048 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.976422+0000", 2024-09-14T23:58:17.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011543 2024-09-14T23:58:17.048 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.048 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:17.048 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.049 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.049 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:17.049 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:17.050 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T23:58:17.051 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:58:17.049+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:58:17.151 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:17.150+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:17.230 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:17.230 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:17.230 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:58:17.230 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:17.242 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T23:58:17.259 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is unset 2024-09-14T23:58:17.260 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:17.259+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:17.353 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:58:17.351+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:58:17.392 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:17.392 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:17.392 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:58:17.393 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:58:17.393 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:17.406 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T23:58:17.453 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:17.452+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:17.554 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:17.552+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:17.555 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:17.555 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:58:17.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:58:17.555 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:58:17.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.556 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:3578 3.4 e29/23 3:2298d860:::rbd_data.1118de1168b.0000000000000030:head v 29'15, mlcod=29'13)", 2024-09-14T23:58:17.556 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.036455+0000", 2024-09-14T23:58:17.556 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 98.516777184000006, 2024-09-14T23:58:17.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019449440000000001, 2024-09-14T23:58:17.556 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:17.556 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:17.556 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:17.556 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:17.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:17.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036455+0000", 2024-09-14T23:58:17.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:17.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036455+0000", 2024-09-14T23:58:17.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:17.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036456+0000", 2024-09-14T23:58:17.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0999999999999997e-07 2024-09-14T23:58:17.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:17.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036463+0000", 2024-09-14T23:58:17.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2200000000000003e-06 2024-09-14T23:58:17.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:17.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036478+0000", 2024-09-14T23:58:17.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4977000000000001e-05 2024-09-14T23:58:17.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:17.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036485+0000", 2024-09-14T23:58:17.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0559999999999999e-06 2024-09-14T23:58:17.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:17.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036693+0000", 2024-09-14T23:58:17.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000208051 2024-09-14T23:58:17.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:17.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036715+0000", 2024-09-14T23:58:17.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1330000000000001e-05 2024-09-14T23:58:17.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:17.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.038356+0000", 2024-09-14T23:58:17.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001641586 2024-09-14T23:58:17.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:17.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.038400+0000", 2024-09-14T23:58:17.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3914e-05 2024-09-14T23:58:17.559 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.560 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:17.560 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.560 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:3579 3.4 e29/23 3:27290fe6:::rbd_data.1118de1168b.0000000000000032:head v 29'16, mlcod=29'13)", 2024-09-14T23:58:17.560 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.037085+0000", 2024-09-14T23:58:17.560 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 98.516147594000003, 2024-09-14T23:58:17.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0020019220000000002, 2024-09-14T23:58:17.560 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:17.560 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:17.560 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:17.560 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:17.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:17.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037085+0000", 2024-09-14T23:58:17.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:17.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037085+0000", 2024-09-14T23:58:17.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:17.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037085+0000", 2024-09-14T23:58:17.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5499999999999997e-07 2024-09-14T23:58:17.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:17.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037091+0000", 2024-09-14T23:58:17.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7869999999999998e-06 2024-09-14T23:58:17.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:17.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037102+0000", 2024-09-14T23:58:17.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.031e-05 2024-09-14T23:58:17.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:17.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037116+0000", 2024-09-14T23:58:17.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4119e-05 2024-09-14T23:58:17.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:17.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037219+0000", 2024-09-14T23:58:17.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010372799999999999 2024-09-14T23:58:17.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:17.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037239+0000", 2024-09-14T23:58:17.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9993e-05 2024-09-14T23:58:17.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:17.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.039046+0000", 2024-09-14T23:58:17.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001806666 2024-09-14T23:58:17.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:17.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.039087+0000", 2024-09-14T23:58:17.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0564e-05 2024-09-14T23:58:17.564 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.564 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:17.564 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.564 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:32024 3.9 e39/23 3:94dc260b:::rbd_data.111814b482f8.0000000000000009:head v 39'45, mlcod=38'43)", 2024-09-14T23:58:17.564 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:49.061018+0000", 2024-09-14T23:58:17.564 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 88.492214254999993, 2024-09-14T23:58:17.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018482240000000001, 2024-09-14T23:58:17.565 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:17.565 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:17.565 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:17.565 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:17.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:17.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061018+0000", 2024-09-14T23:58:17.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:17.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061018+0000", 2024-09-14T23:58:17.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:17.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061018+0000", 2024-09-14T23:58:17.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9999999999999998e-07 2024-09-14T23:58:17.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:17.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061211+0000", 2024-09-14T23:58:17.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019206 2024-09-14T23:58:17.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:17.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061221+0000", 2024-09-14T23:58:17.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0659000000000001e-05 2024-09-14T23:58:17.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:17.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061230+0000", 2024-09-14T23:58:17.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4610000000000008e-06 2024-09-14T23:58:17.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:17.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061317+0000", 2024-09-14T23:58:17.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7514000000000003e-05 2024-09-14T23:58:17.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:17.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061338+0000", 2024-09-14T23:58:17.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0395999999999999e-05 2024-09-14T23:58:17.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:17.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.062830+0000", 2024-09-14T23:58:17.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0014921940000000001 2024-09-14T23:58:17.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:17.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.062866+0000", 2024-09-14T23:58:17.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6539999999999999e-05 2024-09-14T23:58:17.568 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.569 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:17.569 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.569 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:32420 3.9 e39/23 3:91dc34e7:::rbd_data.111814b482f8.000000000000001e:head v 39'46, mlcod=38'44)", 2024-09-14T23:58:17.569 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:49.086124+0000", 2024-09-14T23:58:17.569 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 88.467108756000002, 2024-09-14T23:58:17.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001686819, 2024-09-14T23:58:17.569 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:17.569 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:17.569 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:17.569 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:17.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:17.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086124+0000", 2024-09-14T23:58:17.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:17.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086124+0000", 2024-09-14T23:58:17.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:17.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086125+0000", 2024-09-14T23:58:17.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.361e-06 2024-09-14T23:58:17.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:17.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086605+0000", 2024-09-14T23:58:17.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00048026600000000003 2024-09-14T23:58:17.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:17.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086626+0000", 2024-09-14T23:58:17.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.037e-05 2024-09-14T23:58:17.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:17.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086635+0000", 2024-09-14T23:58:17.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5880000000000004e-06 2024-09-14T23:58:17.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:17.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086701+0000", 2024-09-14T23:58:17.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5905e-05 2024-09-14T23:58:17.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:17.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.086727+0000", 2024-09-14T23:58:17.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5573000000000001e-05 2024-09-14T23:58:17.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:17.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.087767+0000", 2024-09-14T23:58:17.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001040537 2024-09-14T23:58:17.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:17.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.087810+0000", 2024-09-14T23:58:17.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3219000000000001e-05 2024-09-14T23:58:17.573 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.573 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:17.573 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.573 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:50582 3.9 e51/23 3:92198e7a:::rbd_data.11183b9aac34.000000000000000a:head v 51'63, mlcod=50'60)", 2024-09-14T23:58:17.574 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:01.131541+0000", 2024-09-14T23:58:17.574 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 76.421691565000003, 2024-09-14T23:58:17.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001795547, 2024-09-14T23:58:17.574 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:17.574 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:17.574 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:17.574 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:17.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:17.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131541+0000", 2024-09-14T23:58:17.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:17.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131541+0000", 2024-09-14T23:58:17.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:17.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131541+0000", 2024-09-14T23:58:17.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7099999999999997e-07 2024-09-14T23:58:17.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:17.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131548+0000", 2024-09-14T23:58:17.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.708e-06 2024-09-14T23:58:17.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:17.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131570+0000", 2024-09-14T23:58:17.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1815999999999999e-05 2024-09-14T23:58:17.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:17.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131581+0000", 2024-09-14T23:58:17.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0933e-05 2024-09-14T23:58:17.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:17.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131660+0000", 2024-09-14T23:58:17.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9071000000000002e-05 2024-09-14T23:58:17.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:17.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.131682+0000", 2024-09-14T23:58:17.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2534000000000001e-05 2024-09-14T23:58:17.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:17.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.133287+0000", 2024-09-14T23:58:17.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016045510000000001 2024-09-14T23:58:17.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:17.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:01.133336+0000", 2024-09-14T23:58:17.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9563e-05 2024-09-14T23:58:17.578 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.578 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:17.578 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.578 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:51281 3.0 e52/23 3:04e60ec2:::rbd_data.11181975a1de.0000000000000003:head v 52'113, mlcod=44'111)", 2024-09-14T23:58:17.578 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:02.158822+0000", 2024-09-14T23:58:17.578 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 75.394410069000003, 2024-09-14T23:58:17.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019860609999999999, 2024-09-14T23:58:17.579 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:17.579 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:17.579 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:17.579 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:17.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:17.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.158822+0000", 2024-09-14T23:58:17.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:17.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.158822+0000", 2024-09-14T23:58:17.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:17.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.158823+0000", 2024-09-14T23:58:17.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6800000000000001e-07 2024-09-14T23:58:17.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:17.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.159132+0000", 2024-09-14T23:58:17.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030974900000000001 2024-09-14T23:58:17.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:17.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.159146+0000", 2024-09-14T23:58:17.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3261000000000001e-05 2024-09-14T23:58:17.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:17.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.159155+0000", 2024-09-14T23:58:17.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7410000000000004e-06 2024-09-14T23:58:17.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:17.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.159221+0000", 2024-09-14T23:58:17.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5698999999999996e-05 2024-09-14T23:58:17.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:17.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.159244+0000", 2024-09-14T23:58:17.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3108e-05 2024-09-14T23:58:17.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:17.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.160767+0000", 2024-09-14T23:58:17.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015231120000000001 2024-09-14T23:58:17.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:17.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.160808+0000", 2024-09-14T23:58:17.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1022999999999997e-05 2024-09-14T23:58:17.582 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.582 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:17.582 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.583 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:56107 3.0 e56/23 3:00c47890:::rbd_data.1118c4b3aebe.0000000000000009:head v 56'125, mlcod=52'123)", 2024-09-14T23:58:17.583 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.236123+0000", 2024-09-14T23:58:17.583 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 71.317108869999998, 2024-09-14T23:58:17.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0061273220000000002, 2024-09-14T23:58:17.583 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:17.583 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:17.583 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:17.583 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:17.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:17.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.236123+0000", 2024-09-14T23:58:17.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:17.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.236123+0000", 2024-09-14T23:58:17.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:17.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.236125+0000", 2024-09-14T23:58:17.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.313e-06 2024-09-14T23:58:17.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:17.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237428+0000", 2024-09-14T23:58:17.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013029890000000001 2024-09-14T23:58:17.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:17.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237452+0000", 2024-09-14T23:58:17.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4090000000000001e-05 2024-09-14T23:58:17.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:17.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237462+0000", 2024-09-14T23:58:17.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0421999999999999e-05 2024-09-14T23:58:17.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:17.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237559+0000", 2024-09-14T23:58:17.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6378000000000005e-05 2024-09-14T23:58:17.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:17.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237583+0000", 2024-09-14T23:58:17.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4182e-05 2024-09-14T23:58:17.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:17.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.242205+0000", 2024-09-14T23:58:17.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0046217289999999998 2024-09-14T23:58:17.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:17.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.242251+0000", 2024-09-14T23:58:17.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6218999999999999e-05 2024-09-14T23:58:17.587 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.587 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:17.587 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.587 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:56108 3.0 e56/23 3:00c47890:::rbd_data.1118c4b3aebe.0000000000000009:head v 56'127, mlcod=52'123)", 2024-09-14T23:58:17.587 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.239693+0000", 2024-09-14T23:58:17.587 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 71.313539399999996, 2024-09-14T23:58:17.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0043496209999999997, 2024-09-14T23:58:17.587 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:17.587 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:17.587 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:17.588 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:17.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:17.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239693+0000", 2024-09-14T23:58:17.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:17.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239693+0000", 2024-09-14T23:58:17.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:17.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239694+0000", 2024-09-14T23:58:17.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0499999999999999e-06 2024-09-14T23:58:17.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:17.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239702+0000", 2024-09-14T23:58:17.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1170000000000004e-06 2024-09-14T23:58:17.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:17.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239716+0000", 2024-09-14T23:58:17.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4306e-05 2024-09-14T23:58:17.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:17.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239724+0000", 2024-09-14T23:58:17.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0399999999999993e-06 2024-09-14T23:58:17.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:17.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239778+0000", 2024-09-14T23:58:17.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4032e-05 2024-09-14T23:58:17.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:17.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239801+0000", 2024-09-14T23:58:17.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2770000000000001e-05 2024-09-14T23:58:17.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:17.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.243990+0000", 2024-09-14T23:58:17.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0041890909999999998 2024-09-14T23:58:17.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:17.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.244043+0000", 2024-09-14T23:58:17.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2215000000000001e-05 2024-09-14T23:58:17.591 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.591 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:17.591 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.592 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:56385 3.9 e56/23 3:963d31c7:::rbd_data.1118c4b3aebe.0000000000000003:head v 56'87, mlcod=53'85)", 2024-09-14T23:58:17.592 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.283936+0000", 2024-09-14T23:58:17.592 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 71.269296490000002, 2024-09-14T23:58:17.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0063042080000000004, 2024-09-14T23:58:17.592 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:17.592 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:17.592 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:17.592 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:17.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:17.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.283936+0000", 2024-09-14T23:58:17.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:17.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.283936+0000", 2024-09-14T23:58:17.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:17.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.283937+0000", 2024-09-14T23:58:17.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1179999999999999e-06 2024-09-14T23:58:17.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:17.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286338+0000", 2024-09-14T23:58:17.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024010149999999998 2024-09-14T23:58:17.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:17.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286360+0000", 2024-09-14T23:58:17.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2212e-05 2024-09-14T23:58:17.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:17.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286370+0000", 2024-09-14T23:58:17.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5850000000000006e-06 2024-09-14T23:58:17.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:17.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286443+0000", 2024-09-14T23:58:17.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2757999999999997e-05 2024-09-14T23:58:17.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:17.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286466+0000", 2024-09-14T23:58:17.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3844000000000001e-05 2024-09-14T23:58:17.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:17.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.290197+0000", 2024-09-14T23:58:17.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003730176 2024-09-14T23:58:17.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:17.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.290240+0000", 2024-09-14T23:58:17.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.35e-05 2024-09-14T23:58:17.596 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.596 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:17.596 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.596 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:56386 3.9 e56/23 3:963d31c7:::rbd_data.1118c4b3aebe.0000000000000003:head v 56'89, mlcod=53'85)", 2024-09-14T23:58:17.596 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.287578+0000", 2024-09-14T23:58:17.596 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 71.265654764000004, 2024-09-14T23:58:17.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033580250000000002, 2024-09-14T23:58:17.596 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:17.596 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:17.596 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:17.597 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:17.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:17.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287578+0000", 2024-09-14T23:58:17.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:17.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287578+0000", 2024-09-14T23:58:17.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:17.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287579+0000", 2024-09-14T23:58:17.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.64e-07 2024-09-14T23:58:17.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:17.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287586+0000", 2024-09-14T23:58:17.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5310000000000004e-06 2024-09-14T23:58:17.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:17.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287597+0000", 2024-09-14T23:58:17.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0903999999999999e-05 2024-09-14T23:58:17.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:17.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287605+0000", 2024-09-14T23:58:17.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7889999999999995e-06 2024-09-14T23:58:17.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:17.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.288437+0000", 2024-09-14T23:58:17.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000831914 2024-09-14T23:58:17.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:17.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.288460+0000", 2024-09-14T23:58:17.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3410000000000001e-05 2024-09-14T23:58:17.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:17.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.290883+0000", 2024-09-14T23:58:17.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024230599999999999 2024-09-14T23:58:17.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:17.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.290936+0000", 2024-09-14T23:58:17.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2453000000000001e-05 2024-09-14T23:58:17.600 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.600 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:17.600 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.600 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:60299 3.9 e57/23 3:96f26b6f:::rbd_data.11188d356229.0000000000000009:head v 57'98, mlcod=56'96)", 2024-09-14T23:58:17.600 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:07.418336+0000", 2024-09-14T23:58:17.600 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 70.134896271000002, 2024-09-14T23:58:17.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022211459999999998, 2024-09-14T23:58:17.601 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:17.601 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:17.601 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:17.601 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:17.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:17.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.418336+0000", 2024-09-14T23:58:17.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:17.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.418336+0000", 2024-09-14T23:58:17.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:17.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.418337+0000", 2024-09-14T23:58:17.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7199999999999999e-07 2024-09-14T23:58:17.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:17.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.419396+0000", 2024-09-14T23:58:17.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010598319999999999 2024-09-14T23:58:17.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:17.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.419407+0000", 2024-09-14T23:58:17.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0956e-05 2024-09-14T23:58:17.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:17.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.419417+0000", 2024-09-14T23:58:17.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0040999999999999e-05 2024-09-14T23:58:17.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:17.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.419468+0000", 2024-09-14T23:58:17.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0497000000000001e-05 2024-09-14T23:58:17.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:17.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.419485+0000", 2024-09-14T23:58:17.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6966e-05 2024-09-14T23:58:17.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:17.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.420527+0000", 2024-09-14T23:58:17.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001042133 2024-09-14T23:58:17.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:17.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.420557+0000", 2024-09-14T23:58:17.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0249000000000001e-05 2024-09-14T23:58:17.604 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.605 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:17.605 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.605 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:64263 3.4 e57/23 3:20c6a768:::rbd_data.11188d356229.0000000000000001:head v 57'165, mlcod=57'163)", 2024-09-14T23:58:17.605 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:07.999031+0000", 2024-09-14T23:58:17.605 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 69.554201616, 2024-09-14T23:58:17.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023815210000000002, 2024-09-14T23:58:17.605 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:17.605 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:17.605 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:17.605 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:17.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:17.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999031+0000", 2024-09-14T23:58:17.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:17.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999031+0000", 2024-09-14T23:58:17.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:17.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999032+0000", 2024-09-14T23:58:17.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.176e-06 2024-09-14T23:58:17.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:17.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999660+0000", 2024-09-14T23:58:17.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00062817099999999996 2024-09-14T23:58:17.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:17.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999683+0000", 2024-09-14T23:58:17.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2816999999999999e-05 2024-09-14T23:58:17.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:17.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999693+0000", 2024-09-14T23:58:17.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0088999999999999e-05 2024-09-14T23:58:17.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:17.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999767+0000", 2024-09-14T23:58:17.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4056999999999999e-05 2024-09-14T23:58:17.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:17.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999793+0000", 2024-09-14T23:58:17.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5573000000000001e-05 2024-09-14T23:58:17.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:17.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:08.001379+0000", 2024-09-14T23:58:17.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015859019999999999 2024-09-14T23:58:17.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:17.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:08.001412+0000", 2024-09-14T23:58:17.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3735999999999997e-05 2024-09-14T23:58:17.609 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.609 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:17.609 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.609 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:66570 3.0 e60/23 3:055350b3:::rbd_data.111873792d9b.0000000000000005:head v 60'162, mlcod=57'159)", 2024-09-14T23:58:17.609 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:11.277517+0000", 2024-09-14T23:58:17.609 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 66.275715460000001, 2024-09-14T23:58:17.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003472199, 2024-09-14T23:58:17.610 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:17.610 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:17.610 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:17.610 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:17.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:17.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.277517+0000", 2024-09-14T23:58:17.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:17.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.277517+0000", 2024-09-14T23:58:17.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:17.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.277518+0000", 2024-09-14T23:58:17.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1379999999999999e-06 2024-09-14T23:58:17.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:17.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278504+0000", 2024-09-14T23:58:17.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00098617499999999994 2024-09-14T23:58:17.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:17.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278525+0000", 2024-09-14T23:58:17.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.09e-05 2024-09-14T23:58:17.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:17.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278534+0000", 2024-09-14T23:58:17.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6209999999999993e-06 2024-09-14T23:58:17.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:17.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278611+0000", 2024-09-14T23:58:17.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6926000000000002e-05 2024-09-14T23:58:17.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:17.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278641+0000", 2024-09-14T23:58:17.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0861999999999997e-05 2024-09-14T23:58:17.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:17.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.280953+0000", 2024-09-14T23:58:17.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023114239999999999 2024-09-14T23:58:17.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:17.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.280989+0000", 2024-09-14T23:58:17.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6152999999999998e-05 2024-09-14T23:58:17.613 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.613 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:17.613 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.614 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:70526 3.4 e64/23 3:2649db36:::rbd_data.111831f32e80.0000000000000058:head v 64'183, mlcod=64'181)", 2024-09-14T23:58:17.614 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:15.442503+0000", 2024-09-14T23:58:17.614 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 62.110729567999996, 2024-09-14T23:58:17.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001769837, 2024-09-14T23:58:17.614 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:17.614 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:17.614 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:17.614 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:17.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:17.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.442503+0000", 2024-09-14T23:58:17.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:17.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.442503+0000", 2024-09-14T23:58:17.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:17.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.442503+0000", 2024-09-14T23:58:17.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1099999999999995e-07 2024-09-14T23:58:17.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:17.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.442620+0000", 2024-09-14T23:58:17.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000116618 2024-09-14T23:58:17.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:17.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.442634+0000", 2024-09-14T23:58:17.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3984e-05 2024-09-14T23:58:17.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:17.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.442641+0000", 2024-09-14T23:58:17.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2540000000000003e-06 2024-09-14T23:58:17.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:17.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.442732+0000", 2024-09-14T23:58:17.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0480000000000001e-05 2024-09-14T23:58:17.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:17.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.442753+0000", 2024-09-14T23:58:17.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1571999999999999e-05 2024-09-14T23:58:17.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:17.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.444236+0000", 2024-09-14T23:58:17.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0014825439999999999 2024-09-14T23:58:17.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:17.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:15.444273+0000", 2024-09-14T23:58:17.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6674000000000002e-05 2024-09-14T23:58:17.618 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.618 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:17.618 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.618 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:106629 3.4 e70/23 3:240d85c6:::rbd_data.11187de8b0a5.0000000000000009:head v 70'233, mlcod=69'231)", 2024-09-14T23:58:17.618 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:21.489517+0000", 2024-09-14T23:58:17.618 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 56.063714935, 2024-09-14T23:58:17.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016279840000000001, 2024-09-14T23:58:17.618 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:17.618 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:17.619 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:17.619 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:17.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:17.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.489517+0000", 2024-09-14T23:58:17.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:17.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.489517+0000", 2024-09-14T23:58:17.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:17.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.489518+0000", 2024-09-14T23:58:17.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.029e-06 2024-09-14T23:58:17.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:17.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.489840+0000", 2024-09-14T23:58:17.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00032164100000000001 2024-09-14T23:58:17.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:17.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.489859+0000", 2024-09-14T23:58:17.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8828999999999999e-05 2024-09-14T23:58:17.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:17.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.489869+0000", 2024-09-14T23:58:17.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6600000000000007e-06 2024-09-14T23:58:17.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:17.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.489934+0000", 2024-09-14T23:58:17.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5357000000000005e-05 2024-09-14T23:58:17.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:17.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.489951+0000", 2024-09-14T23:58:17.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6951000000000001e-05 2024-09-14T23:58:17.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:17.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.491108+0000", 2024-09-14T23:58:17.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011571940000000001 2024-09-14T23:58:17.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:17.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:21.491145+0000", 2024-09-14T23:58:17.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7323000000000002e-05 2024-09-14T23:58:17.622 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.622 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:17.622 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.623 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:108739 3.9 e80/23 3:9372b409:::rbd_data.111867e26b91.0000000000000005:head v 80'172, mlcod=78'170)", 2024-09-14T23:58:17.623 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:31.064099+0000", 2024-09-14T23:58:17.623 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.489133682000002, 2024-09-14T23:58:17.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001688174, 2024-09-14T23:58:17.623 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:17.623 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:17.623 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:17.623 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:17.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:17.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:31.064099+0000", 2024-09-14T23:58:17.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:17.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:31.064099+0000", 2024-09-14T23:58:17.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:17.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:31.064099+0000", 2024-09-14T23:58:17.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.44e-07 2024-09-14T23:58:17.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:17.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:31.064453+0000", 2024-09-14T23:58:17.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00035374899999999999 2024-09-14T23:58:17.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:17.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:31.064468+0000", 2024-09-14T23:58:17.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4476e-05 2024-09-14T23:58:17.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:17.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:31.064475+0000", 2024-09-14T23:58:17.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9289999999999998e-06 2024-09-14T23:58:17.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:17.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:31.064530+0000", 2024-09-14T23:58:17.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4401000000000003e-05 2024-09-14T23:58:17.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:17.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:31.064551+0000", 2024-09-14T23:58:17.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0996e-05 2024-09-14T23:58:17.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:17.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:31.065748+0000", 2024-09-14T23:58:17.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0011967449999999999 2024-09-14T23:58:17.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:17.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:31.065787+0000", 2024-09-14T23:58:17.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9233999999999997e-05 2024-09-14T23:58:17.627 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.627 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:17.627 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.627 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "pg_update_log_missing(3.9 epoch 84/23 rep_tid 2234 entries 84'200 (0'0) error 3:93b58b1d:::rbd_data.1118417a1b0b.000000000000001b:head by client.4376.0:118637 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 83'197)", 2024-09-14T23:58:17.627 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:35.103156+0000", 2024-09-14T23:58:17.627 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.450076187000001, 2024-09-14T23:58:17.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019852009999999998, 2024-09-14T23:58:17.627 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:17.628 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:17.628 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:17.628 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:17.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:17.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:35.103156+0000", 2024-09-14T23:58:17.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:17.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:35.103156+0000", 2024-09-14T23:58:17.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:17.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:35.103157+0000", 2024-09-14T23:58:17.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1969999999999999e-06 2024-09-14T23:58:17.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:17.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:35.103170+0000", 2024-09-14T23:58:17.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2921e-05 2024-09-14T23:58:17.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:17.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:35.103202+0000", 2024-09-14T23:58:17.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1992000000000001e-05 2024-09-14T23:58:17.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:17.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:35.103220+0000", 2024-09-14T23:58:17.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7312999999999998e-05 2024-09-14T23:58:17.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:17.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:35.103305+0000", 2024-09-14T23:58:17.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4975000000000005e-05 2024-09-14T23:58:17.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:17.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:35.105141+0000", 2024-09-14T23:58:17.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018368029999999999 2024-09-14T23:58:17.631 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.631 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:17.631 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.631 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:142751 3.4 e99/23 3:265f7763:::rbd_data.1118f9c96a7.0000000000000004:head v 99'441, mlcod=98'438)", 2024-09-14T23:58:17.631 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:53.165301+0000", 2024-09-14T23:58:17.631 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 24.387931037000001, 2024-09-14T23:58:17.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0021754449999999998, 2024-09-14T23:58:17.631 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:17.631 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:17.631 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:17.631 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:17.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:17.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.165301+0000", 2024-09-14T23:58:17.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:17.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.165301+0000", 2024-09-14T23:58:17.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:17.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.165302+0000", 2024-09-14T23:58:17.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0899999999999997e-07 2024-09-14T23:58:17.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:17.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.165310+0000", 2024-09-14T23:58:17.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7520000000000002e-06 2024-09-14T23:58:17.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:17.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.165321+0000", 2024-09-14T23:58:17.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0668e-05 2024-09-14T23:58:17.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:17.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.165330+0000", 2024-09-14T23:58:17.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.234e-06 2024-09-14T23:58:17.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:17.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.165393+0000", 2024-09-14T23:58:17.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2716000000000001e-05 2024-09-14T23:58:17.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:17.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.165418+0000", 2024-09-14T23:58:17.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4544000000000001e-05 2024-09-14T23:58:17.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:17.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.167436+0000", 2024-09-14T23:58:17.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002018597 2024-09-14T23:58:17.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:17.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.167477+0000", 2024-09-14T23:58:17.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0624999999999998e-05 2024-09-14T23:58:17.635 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.635 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:17.635 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.636 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:158983 3.4 e103/23 3:20e43b29:::rbd_data.1118b2f495e6.0000000000000012:head v 103'454, mlcod=103'452)", 2024-09-14T23:58:17.636 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:58.917699+0000", 2024-09-14T23:58:17.636 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.635533410000001, 2024-09-14T23:58:17.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037933799999999998, 2024-09-14T23:58:17.636 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:17.636 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:17.636 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:17.636 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:17.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:17.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917699+0000", 2024-09-14T23:58:17.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:17.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917699+0000", 2024-09-14T23:58:17.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:17.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917700+0000", 2024-09-14T23:58:17.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3850000000000001e-06 2024-09-14T23:58:17.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:17.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917939+0000", 2024-09-14T23:58:17.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000238249 2024-09-14T23:58:17.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:17.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917961+0000", 2024-09-14T23:58:17.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2894000000000001e-05 2024-09-14T23:58:17.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:17.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917972+0000", 2024-09-14T23:58:17.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0261000000000001e-05 2024-09-14T23:58:17.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:17.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.918038+0000", 2024-09-14T23:58:17.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6223000000000006e-05 2024-09-14T23:58:17.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:17.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.918063+0000", 2024-09-14T23:58:17.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5252000000000001e-05 2024-09-14T23:58:17.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:17.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.921433+0000", 2024-09-14T23:58:17.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033699960000000001 2024-09-14T23:58:17.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:17.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.921492+0000", 2024-09-14T23:58:17.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9120000000000003e-05 2024-09-14T23:58:17.640 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.640 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:17.640 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.640 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4172.0:6 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T23:58:17.640 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.479840+0000", 2024-09-14T23:58:17.640 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.07339213, 2024-09-14T23:58:17.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022238340000000001, 2024-09-14T23:58:17.640 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:17.640 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:17.640 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:17.641 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:17.641 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4172", 2024-09-14T23:58:17.641 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/2542269203", 2024-09-14T23:58:17.641 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 6 2024-09-14T23:58:17.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.641 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:17.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:17.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.479840+0000", 2024-09-14T23:58:17.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:17.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.479840+0000", 2024-09-14T23:58:17.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:17.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:17.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.479843+0000", 2024-09-14T23:58:17.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6680000000000001e-06 2024-09-14T23:58:17.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:17.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.479862+0000", 2024-09-14T23:58:17.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8762000000000001e-05 2024-09-14T23:58:17.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:17.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.479881+0000", 2024-09-14T23:58:17.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9860999999999998e-05 2024-09-14T23:58:17.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:17.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.479899+0000", 2024-09-14T23:58:17.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7127000000000001e-05 2024-09-14T23:58:17.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:17.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.481969+0000", 2024-09-14T23:58:17.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002070166 2024-09-14T23:58:17.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:17.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:17.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:17.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482064+0000", 2024-09-14T23:58:17.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5249999999999998e-05 2024-09-14T23:58:17.644 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.644 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:17.644 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.644 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:17.644 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:17.644 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:17.645 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T23:58:17.687 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:17.685+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:17.755 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:17.754+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:17.796 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:17.796 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:17.796 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:58:17.796 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:17.805 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T23:58:17.856 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:58:17.855+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:58:17.922 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:17.922 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:17.922 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:58:17.922 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:58:17.922 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:17.931 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T23:58:17.976 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:17.975+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:18.043 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:18.043 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:58:18.043 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:58:18.043 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:58:18.044 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.044 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5201 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2232320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:18.044 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:58:18.044 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 98.787867543000004, 2024-09-14T23:58:18.044 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057643228999999997, 2024-09-14T23:58:18.044 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:18.044 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:18.044 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:18.044 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:18.044 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:18.044 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:18.044 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5201 2024-09-14T23:58:18.044 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.044 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:18.045 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.045 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:18.045 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:58:18.045 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:18.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.045 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.045 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:18.045 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:58:18.045 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:18.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.045 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.045 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:18.045 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:58:18.045 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0699999999999998e-07 2024-09-14T23:58:18.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.046 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:18.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253422+0000", 2024-09-14T23:58:18.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.514e-06 2024-09-14T23:58:18.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.046 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:18.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253431+0000", 2024-09-14T23:58:18.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1919999999999994e-06 2024-09-14T23:58:18.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.046 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:18.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253436+0000", 2024-09-14T23:58:18.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5350000000000004e-06 2024-09-14T23:58:18.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:18.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310856+0000", 2024-09-14T23:58:18.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057419381999999998 2024-09-14T23:58:18.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:18.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310953+0000", 2024-09-14T23:58:18.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6774000000000006e-05 2024-09-14T23:58:18.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:18.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311062+0000", 2024-09-14T23:58:18.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000109425 2024-09-14T23:58:18.048 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:18.048 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:18.048 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:18.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.048 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5202 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2293760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:18.048 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:58:18.048 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 98.787842795000003, 2024-09-14T23:58:18.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057818255999999998, 2024-09-14T23:58:18.048 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:18.048 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:18.048 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:18.048 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:18.048 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:18.049 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:18.049 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5202 2024-09-14T23:58:18.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.049 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:18.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.049 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:18.049 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:58:18.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:18.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.049 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:18.049 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:58:18.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:18.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.050 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:18.050 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253573+0000", 2024-09-14T23:58:18.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012934000000000001 2024-09-14T23:58:18.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.050 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:18.050 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253576+0000", 2024-09-14T23:58:18.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5310000000000002e-06 2024-09-14T23:58:18.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.050 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:18.050 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253587+0000", 2024-09-14T23:58:18.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0522e-05 2024-09-14T23:58:18.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:18.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253595+0000", 2024-09-14T23:58:18.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8290000000000008e-06 2024-09-14T23:58:18.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:18.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311093+0000", 2024-09-14T23:58:18.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057498312000000003 2024-09-14T23:58:18.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:18.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311167+0000", 2024-09-14T23:58:18.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3582000000000001e-05 2024-09-14T23:58:18.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.052 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:18.052 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311262+0000", 2024-09-14T23:58:18.052 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5140000000000001e-05 2024-09-14T23:58:18.052 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:18.052 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:18.052 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:18.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.052 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.052 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5203 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2355200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:18.052 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:58:18.052 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 98.787680053000003, 2024-09-14T23:58:18.052 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057903938000000002, 2024-09-14T23:58:18.052 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:18.052 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:18.052 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:18.052 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:18.053 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:18.053 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:18.053 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5203 2024-09-14T23:58:18.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.053 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:18.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.053 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:18.053 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:58:18.053 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:18.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.053 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:18.053 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:58:18.053 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:18.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.054 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:18.054 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253902+0000", 2024-09-14T23:58:18.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029615400000000002 2024-09-14T23:58:18.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.054 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:18.054 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253906+0000", 2024-09-14T23:58:18.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9249999999999997e-06 2024-09-14T23:58:18.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.054 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:18.054 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253913+0000", 2024-09-14T23:58:18.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9099999999999999e-06 2024-09-14T23:58:18.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:18.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253917+0000", 2024-09-14T23:58:18.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9210000000000002e-06 2024-09-14T23:58:18.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:18.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311295+0000", 2024-09-14T23:58:18.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057378268000000003 2024-09-14T23:58:18.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:18.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311402+0000", 2024-09-14T23:58:18.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010644100000000001 2024-09-14T23:58:18.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:18.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311510+0000", 2024-09-14T23:58:18.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108319 2024-09-14T23:58:18.056 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:18.056 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:18.056 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:18.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.056 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5204 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2416640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:18.056 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:58:18.056 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 98.787361856000004, 2024-09-14T23:58:18.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057774860999999997, 2024-09-14T23:58:18.057 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:18.057 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:18.057 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:18.057 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:18.057 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:18.057 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:18.057 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5204 2024-09-14T23:58:18.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.057 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:18.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.057 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:18.057 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:58:18.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:18.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.058 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:18.058 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:58:18.058 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:18.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.058 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:18.058 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255903+0000", 2024-09-14T23:58:18.058 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001978807 2024-09-14T23:58:18.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.058 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:18.058 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255907+0000", 2024-09-14T23:58:18.058 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3610000000000001e-06 2024-09-14T23:58:18.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.058 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:18.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255914+0000", 2024-09-14T23:58:18.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.357e-06 2024-09-14T23:58:18.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:18.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255918+0000", 2024-09-14T23:58:18.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1779999999999996e-06 2024-09-14T23:58:18.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:18.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311541+0000", 2024-09-14T23:58:18.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055622697999999998 2024-09-14T23:58:18.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:18.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311614+0000", 2024-09-14T23:58:18.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3102999999999994e-05 2024-09-14T23:58:18.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:18.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311699+0000", 2024-09-14T23:58:18.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5357000000000003e-05 2024-09-14T23:58:18.060 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:18.060 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:18.060 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:18.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.060 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5211 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2846720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:18.060 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:58:18.060 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 98.785263642000004, 2024-09-14T23:58:18.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057528151, 2024-09-14T23:58:18.061 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:18.061 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:18.061 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:18.061 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:18.061 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:18.061 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:18.061 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5211 2024-09-14T23:58:18.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.061 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:18.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.061 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:18.061 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:58:18.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:18.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.062 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:18.062 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:58:18.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:18.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.062 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:18.062 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:58:18.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1399999999999998e-07 2024-09-14T23:58:18.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.062 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:18.062 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256024+0000", 2024-09-14T23:58:18.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.114e-06 2024-09-14T23:58:18.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:18.063 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256028+0000", 2024-09-14T23:58:18.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0840000000000002e-06 2024-09-14T23:58:18.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:18.063 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256031+0000", 2024-09-14T23:58:18.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6819999999999999e-06 2024-09-14T23:58:18.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:18.063 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313317+0000", 2024-09-14T23:58:18.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057286295000000001 2024-09-14T23:58:18.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.064 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:18.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313403+0000", 2024-09-14T23:58:18.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6287999999999998e-05 2024-09-14T23:58:18.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.064 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:18.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313551+0000", 2024-09-14T23:58:18.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000147374 2024-09-14T23:58:18.064 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:18.064 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:18.064 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:18.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.064 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5212 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2908160~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:18.064 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:58:18.065 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 98.785252447999994, 2024-09-14T23:58:18.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057735679999999998, 2024-09-14T23:58:18.065 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:18.065 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:18.065 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:18.065 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:18.065 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:18.065 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:18.065 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5212 2024-09-14T23:58:18.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.065 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:18.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:18.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:58:18.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:18.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:18.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:58:18.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:18.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:18.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:58:18.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0900000000000001e-07 2024-09-14T23:58:18.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:18.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256035+0000", 2024-09-14T23:58:18.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1200000000000001e-06 2024-09-14T23:58:18.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:18.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256038+0000", 2024-09-14T23:58:18.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.05e-06 2024-09-14T23:58:18.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:18.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256041+0000", 2024-09-14T23:58:18.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.667e-06 2024-09-14T23:58:18.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:18.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313584+0000", 2024-09-14T23:58:18.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057543484999999998 2024-09-14T23:58:18.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:18.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313668+0000", 2024-09-14T23:58:18.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3807000000000006e-05 2024-09-14T23:58:18.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:18.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313770+0000", 2024-09-14T23:58:18.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010134200000000001 2024-09-14T23:58:18.068 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:18.068 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:18.068 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:18.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.069 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5213 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2969600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:18.069 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:58:18.069 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 98.785242361000002, 2024-09-14T23:58:18.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058145467999999999, 2024-09-14T23:58:18.069 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:18.069 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:18.069 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:18.069 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:18.069 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:18.069 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:18.069 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5213 2024-09-14T23:58:18.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.070 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:18.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:18.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:58:18.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:18.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:18.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:58:18.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:18.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:18.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257901+0000", 2024-09-14T23:58:18.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018567519999999999 2024-09-14T23:58:18.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:18.071 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257904+0000", 2024-09-14T23:58:18.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4850000000000001e-06 2024-09-14T23:58:18.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:18.071 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257912+0000", 2024-09-14T23:58:18.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2090000000000005e-06 2024-09-14T23:58:18.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:18.071 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257918+0000", 2024-09-14T23:58:18.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6899999999999997e-06 2024-09-14T23:58:18.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:18.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313816+0000", 2024-09-14T23:58:18.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055898075999999998 2024-09-14T23:58:18.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:18.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.314082+0000", 2024-09-14T23:58:18.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000265386 2024-09-14T23:58:18.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:18.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.314189+0000", 2024-09-14T23:58:18.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010787 2024-09-14T23:58:18.073 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:18.073 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:18.073 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:18.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.073 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5218 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3276800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:18.073 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:58:18.073 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 98.783315352000002, 2024-09-14T23:58:18.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057497557999999997, 2024-09-14T23:58:18.073 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:18.073 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:18.073 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:18.074 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:18.074 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:18.074 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:18.074 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5218 2024-09-14T23:58:18.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.074 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:18.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:18.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:58:18.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:18.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:18.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:58:18.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:18.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:18.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:58:18.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1899999999999999e-07 2024-09-14T23:58:18.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:18.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257972+0000", 2024-09-14T23:58:18.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.217e-06 2024-09-14T23:58:18.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:18.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257975+0000", 2024-09-14T23:58:18.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9359999999999999e-06 2024-09-14T23:58:18.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:18.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257978+0000", 2024-09-14T23:58:18.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1049999999999999e-06 2024-09-14T23:58:18.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:18.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315331+0000", 2024-09-14T23:58:18.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057352093999999999 2024-09-14T23:58:18.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:18.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315384+0000", 2024-09-14T23:58:18.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3780999999999999e-05 2024-09-14T23:58:18.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:18.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315468+0000", 2024-09-14T23:58:18.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4206e-05 2024-09-14T23:58:18.077 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:18.077 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:18.077 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:18.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.077 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5219 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3338240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:18.077 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:58:18.077 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 98.783304676, 2024-09-14T23:58:18.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057637764000000001, 2024-09-14T23:58:18.078 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:18.078 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:18.078 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:18.078 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:18.078 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:18.078 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:18.078 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5219 2024-09-14T23:58:18.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.078 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:18.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:18.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:58:18.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:18.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:18.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:58:18.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:18.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:18.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:58:18.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.03e-07 2024-09-14T23:58:18.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:18.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257983+0000", 2024-09-14T23:58:18.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9199999999999999e-07 2024-09-14T23:58:18.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:18.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257986+0000", 2024-09-14T23:58:18.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0860000000000002e-06 2024-09-14T23:58:18.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:18.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257989+0000", 2024-09-14T23:58:18.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6589999999999999e-06 2024-09-14T23:58:18.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:18.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315509+0000", 2024-09-14T23:58:18.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057520262000000003 2024-09-14T23:58:18.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:18.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315554+0000", 2024-09-14T23:58:18.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5015999999999997e-05 2024-09-14T23:58:18.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:18.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315619+0000", 2024-09-14T23:58:18.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5545999999999999e-05 2024-09-14T23:58:18.081 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:18.081 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:18.081 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:18.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.082 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5220 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3399680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:18.082 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:58:18.082 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 98.783294720000001, 2024-09-14T23:58:18.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057792128999999998, 2024-09-14T23:58:18.082 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:18.082 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:18.082 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:18.082 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:18.082 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:18.082 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:18.082 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5220 2024-09-14T23:58:18.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.083 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:18.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:18.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:58:18.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:18.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:18.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:58:18.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:18.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:18.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:58:18.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1799999999999999e-07 2024-09-14T23:58:18.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:18.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257993+0000", 2024-09-14T23:58:18.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6200000000000006e-07 2024-09-14T23:58:18.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:18.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257996+0000", 2024-09-14T23:58:18.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9050000000000001e-06 2024-09-14T23:58:18.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:18.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257998+0000", 2024-09-14T23:58:18.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5950000000000001e-06 2024-09-14T23:58:18.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:18.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315672+0000", 2024-09-14T23:58:18.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057673637999999999 2024-09-14T23:58:18.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:18.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315734+0000", 2024-09-14T23:58:18.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2216000000000002e-05 2024-09-14T23:58:18.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:18.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315784+0000", 2024-09-14T23:58:18.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9595000000000002e-05 2024-09-14T23:58:18.086 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:18.086 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:18.086 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:18.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.086 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5221 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3461120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:18.086 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:58:18.086 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 98.783285002, 2024-09-14T23:58:18.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058003557999999997, 2024-09-14T23:58:18.086 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:18.086 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:18.087 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:18.087 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:18.087 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:18.087 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:18.087 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5221 2024-09-14T23:58:18.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.087 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:18.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:18.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:58:18.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:18.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:18.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:58:18.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:18.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:18.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259905+0000", 2024-09-14T23:58:18.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019035759999999999 2024-09-14T23:58:18.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:18.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259915+0000", 2024-09-14T23:58:18.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0518e-05 2024-09-14T23:58:18.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:18.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259925+0000", 2024-09-14T23:58:18.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7180000000000008e-06 2024-09-14T23:58:18.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:18.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259930+0000", 2024-09-14T23:58:18.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4769999999999997e-06 2024-09-14T23:58:18.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:18.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315820+0000", 2024-09-14T23:58:18.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055889960000000002 2024-09-14T23:58:18.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:18.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315914+0000", 2024-09-14T23:58:18.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4625999999999998e-05 2024-09-14T23:58:18.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:18.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.316005+0000", 2024-09-14T23:58:18.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0682999999999999e-05 2024-09-14T23:58:18.090 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:18.090 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:18.090 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:18.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.091 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5231 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 4075520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:18.091 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:58:18.091 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 98.781216267999994, 2024-09-14T23:58:18.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057524733000000002, 2024-09-14T23:58:18.091 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:18.091 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:18.091 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:18.091 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:18.091 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:18.091 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:18.091 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5231 2024-09-14T23:58:18.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.091 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:18.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:18.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:58:18.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:18.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:18.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:58:18.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:18.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:18.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261904+0000", 2024-09-14T23:58:18.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018338790000000001 2024-09-14T23:58:18.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:18.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261910+0000", 2024-09-14T23:58:18.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9689999999999999e-06 2024-09-14T23:58:18.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:18.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261923+0000", 2024-09-14T23:58:18.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3061000000000001e-05 2024-09-14T23:58:18.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:18.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261931+0000", 2024-09-14T23:58:18.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3159999999999997e-06 2024-09-14T23:58:18.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:18.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317456+0000", 2024-09-14T23:58:18.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055525219000000001 2024-09-14T23:58:18.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:18.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317520+0000", 2024-09-14T23:58:18.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4010000000000005e-05 2024-09-14T23:58:18.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:18.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317595+0000", 2024-09-14T23:58:18.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4278999999999998e-05 2024-09-14T23:58:18.095 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:18.095 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:18.095 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:18.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.095 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5236 3.8 3:1071668d:::rbd_data.1118de1168b.0000000000000092:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:18.095 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:58:18.095 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 98.779202803999993, 2024-09-14T23:58:18.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057499462000000001, 2024-09-14T23:58:18.095 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:18.095 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:18.095 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:18.096 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:18.096 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:18.096 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:18.096 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5236 2024-09-14T23:58:18.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.096 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:18.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:18.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:58:18.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:18.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:18.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:58:18.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:18.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:18.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262084+0000", 2024-09-14T23:58:18.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3399999999999999e-07 2024-09-14T23:58:18.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:18.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262087+0000", 2024-09-14T23:58:18.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2310000000000001e-06 2024-09-14T23:58:18.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:18.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262095+0000", 2024-09-14T23:58:18.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3799999999999996e-06 2024-09-14T23:58:18.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:18.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262101+0000", 2024-09-14T23:58:18.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0009999999999998e-06 2024-09-14T23:58:18.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:18.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317625+0000", 2024-09-14T23:58:18.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055524111000000001 2024-09-14T23:58:18.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:18.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317702+0000", 2024-09-14T23:58:18.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7533999999999993e-05 2024-09-14T23:58:18.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-14T23:58:18.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317892+0000", 2024-09-14T23:58:18.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018991400000000001 2024-09-14T23:58:18.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:58:18.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319407+0000", 2024-09-14T23:58:18.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015145250000000001 2024-09-14T23:58:18.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:18.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319481+0000", 2024-09-14T23:58:18.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3826999999999996e-05 2024-09-14T23:58:18.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:18.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319541+0000", 2024-09-14T23:58:18.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0470999999999998e-05 2024-09-14T23:58:18.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:18.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319583+0000", 2024-09-14T23:58:18.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1934000000000002e-05 2024-09-14T23:58:18.100 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:18.100 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:18.100 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:18.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.101 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:132622 3.1e 3.7a50b9e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-14T23:58:18.101 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:43.371285+0000", 2024-09-14T23:58:18.101 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 34.670000901999998, 2024-09-14T23:58:18.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.67037987200000004, 2024-09-14T23:58:18.101 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:18.101 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:18.101 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:18.101 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:18.101 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:18.101 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:18.101 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 132622 2024-09-14T23:58:18.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.102 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:18.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:18.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371285+0000", 2024-09-14T23:58:18.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:18.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:18.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371285+0000", 2024-09-14T23:58:18.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:18.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:18.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371286+0000", 2024-09-14T23:58:18.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6400000000000002e-07 2024-09-14T23:58:18.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:18.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371289+0000", 2024-09-14T23:58:18.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2059999999999999e-06 2024-09-14T23:58:18.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:18.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371299+0000", 2024-09-14T23:58:18.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0288e-05 2024-09-14T23:58:18.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:18.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371306+0000", 2024-09-14T23:58:18.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1650000000000003e-06 2024-09-14T23:58:18.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:18.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371617+0000", 2024-09-14T23:58:18.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000311043 2024-09-14T23:58:18.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:58:18.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371625+0000", 2024-09-14T23:58:18.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.833e-06 2024-09-14T23:58:18.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:18.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.041651+0000", 2024-09-14T23:58:18.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.67002580199999995 2024-09-14T23:58:18.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:18.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.041665+0000", 2024-09-14T23:58:18.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3971000000000001e-05 2024-09-14T23:58:18.105 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:18.105 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:18.105 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:18.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.106 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:169974 3.5 3:a704a2f0:::rbd_data.1118427720d0.0000000000000030:32 [read 965632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-14T23:58:18.106 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:03.433820+0000", 2024-09-14T23:58:18.106 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.607466244999999, 2024-09-14T23:58:18.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057495521000000001, 2024-09-14T23:58:18.106 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:18.106 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:18.106 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:18.106 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:18.106 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:18.106 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:18.106 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 169974 2024-09-14T23:58:18.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.107 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:18.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:18.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.433820+0000", 2024-09-14T23:58:18.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:18.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:18.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.433820+0000", 2024-09-14T23:58:18.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:18.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:18.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.433925+0000", 2024-09-14T23:58:18.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000104621 2024-09-14T23:58:18.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:18.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.433929+0000", 2024-09-14T23:58:18.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2169999999999996e-06 2024-09-14T23:58:18.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:18.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.433939+0000", 2024-09-14T23:58:18.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0002e-05 2024-09-14T23:58:18.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:18.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.433946+0000", 2024-09-14T23:58:18.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7889999999999997e-06 2024-09-14T23:58:18.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:18.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.491144+0000", 2024-09-14T23:58:18.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057198221 2024-09-14T23:58:18.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:18.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.491221+0000", 2024-09-14T23:58:18.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6767000000000005e-05 2024-09-14T23:58:18.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:18.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.491316+0000", 2024-09-14T23:58:18.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4903999999999998e-05 2024-09-14T23:58:18.110 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:18.110 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:18.110 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:18.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.110 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:169975 3.5 3:a704a2f0:::rbd_data.1118427720d0.0000000000000030:32 [read 969728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-14T23:58:18.110 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:03.433958+0000", 2024-09-14T23:58:18.110 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.607328546, 2024-09-14T23:58:18.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057564811, 2024-09-14T23:58:18.110 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:18.110 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:18.110 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:18.110 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:18.110 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:18.111 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:18.111 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 169975 2024-09-14T23:58:18.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.111 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:18.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:18.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.433958+0000", 2024-09-14T23:58:18.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:18.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:18.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.433958+0000", 2024-09-14T23:58:18.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:18.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:18.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.435096+0000", 2024-09-14T23:58:18.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001138708 2024-09-14T23:58:18.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:18.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.435101+0000", 2024-09-14T23:58:18.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5050000000000001e-06 2024-09-14T23:58:18.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:18.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.435111+0000", 2024-09-14T23:58:18.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5810000000000003e-06 2024-09-14T23:58:18.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:18.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.435120+0000", 2024-09-14T23:58:18.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0089999999999996e-06 2024-09-14T23:58:18.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:18.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.491345+0000", 2024-09-14T23:58:18.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056225479000000002 2024-09-14T23:58:18.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:18.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.491423+0000", 2024-09-14T23:58:18.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7990000000000004e-05 2024-09-14T23:58:18.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:18.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.491523+0000", 2024-09-14T23:58:18.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9538999999999997e-05 2024-09-14T23:58:18.114 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:18.114 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:18.114 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:18.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.114 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:172639 3.5 3:a704a2f0:::rbd_data.1118427720d0.0000000000000030:32 [read 1645568~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e107)", 2024-09-14T23:58:18.114 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:03.988365+0000", 2024-09-14T23:58:18.114 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.052920825999999, 2024-09-14T23:58:18.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057556213000000002, 2024-09-14T23:58:18.115 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:18.115 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:18.115 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:18.115 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:18.115 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:18.115 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:18.115 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 172639 2024-09-14T23:58:18.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.115 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:18.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:18.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.988365+0000", 2024-09-14T23:58:18.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:18.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:18.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.988365+0000", 2024-09-14T23:58:18.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:18.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:18.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.989904+0000", 2024-09-14T23:58:18.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015380859999999999 2024-09-14T23:58:18.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:18.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.989909+0000", 2024-09-14T23:58:18.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2800000000000003e-06 2024-09-14T23:58:18.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:18.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.989925+0000", 2024-09-14T23:58:18.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5781999999999999e-05 2024-09-14T23:58:18.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:18.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.989933+0000", 2024-09-14T23:58:18.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0090000000000007e-06 2024-09-14T23:58:18.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:18.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:04.045741+0000", 2024-09-14T23:58:18.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055808112999999999 2024-09-14T23:58:18.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:18.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:04.045812+0000", 2024-09-14T23:58:18.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1531000000000003e-05 2024-09-14T23:58:18.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:18.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:04.045922+0000", 2024-09-14T23:58:18.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000109412 2024-09-14T23:58:18.118 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:18.118 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:18.118 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:18.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.119 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184571 3.2 3:46f8d78b:::rbd_data.1118427720d0.0000000000000001:32 [read 1405952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:18.119 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.086404+0000", 2024-09-14T23:58:18.119 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.954882468999999, 2024-09-14T23:58:18.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059148573000000003, 2024-09-14T23:58:18.119 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:18.119 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:18.119 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:18.119 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:18.119 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:18.119 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:18.119 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184571 2024-09-14T23:58:18.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.120 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:18.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:18.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.086404+0000", 2024-09-14T23:58:18.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:18.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:18.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.086404+0000", 2024-09-14T23:58:18.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:18.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:18.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087902+0000", 2024-09-14T23:58:18.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001498625 2024-09-14T23:58:18.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:18.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087907+0000", 2024-09-14T23:58:18.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9130000000000002e-06 2024-09-14T23:58:18.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:18.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087919+0000", 2024-09-14T23:58:18.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1719000000000001e-05 2024-09-14T23:58:18.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:18.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087927+0000", 2024-09-14T23:58:18.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3699999999999995e-06 2024-09-14T23:58:18.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:18.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143711+0000", 2024-09-14T23:58:18.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055783396999999998 2024-09-14T23:58:18.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:18.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143792+0000", 2024-09-14T23:58:18.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1292999999999996e-05 2024-09-14T23:58:18.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:18.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145552+0000", 2024-09-14T23:58:18.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001760256 2024-09-14T23:58:18.123 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:18.123 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:18.123 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:18.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.123 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184579 3.2 3:43badc91:::rbd_data.1118427720d0.0000000000000002:32 [read 1196032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:18.123 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.088145+0000", 2024-09-14T23:58:18.123 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.953141065000001, 2024-09-14T23:58:18.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057643169000000001, 2024-09-14T23:58:18.123 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:18.123 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:18.124 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:18.124 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:18.124 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:18.124 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:18.124 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184579 2024-09-14T23:58:18.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.124 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:18.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:18.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.088145+0000", 2024-09-14T23:58:18.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:18.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:18.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.088145+0000", 2024-09-14T23:58:18.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:18.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:18.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089947+0000", 2024-09-14T23:58:18.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001801764 2024-09-14T23:58:18.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:18.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089952+0000", 2024-09-14T23:58:18.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8659999999999997e-06 2024-09-14T23:58:18.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:18.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089966+0000", 2024-09-14T23:58:18.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3929e-05 2024-09-14T23:58:18.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:18.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089974+0000", 2024-09-14T23:58:18.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1429999999999998e-06 2024-09-14T23:58:18.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:18.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145636+0000", 2024-09-14T23:58:18.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055662287999999997 2024-09-14T23:58:18.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:18.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145710+0000", 2024-09-14T23:58:18.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4166999999999996e-05 2024-09-14T23:58:18.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:18.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145788+0000", 2024-09-14T23:58:18.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8011999999999998e-05 2024-09-14T23:58:18.127 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:18.127 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:18.127 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:18.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.128 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:211800 3.12 3:493125e8:::rbd_data.1118c900944f.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-14T23:58:18.128 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:13.660130+0000", 2024-09-14T23:58:18.128 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.3811567389999997, 2024-09-14T23:58:18.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.55364332000000005, 2024-09-14T23:58:18.128 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:18.128 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:18.128 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:18.128 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:18.128 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:18.128 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:18.128 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 211800 2024-09-14T23:58:18.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.128 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:18.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:18.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.660130+0000", 2024-09-14T23:58:18.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:18.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:18.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.660130+0000", 2024-09-14T23:58:18.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:18.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:18.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.660133+0000", 2024-09-14T23:58:18.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.202e-06 2024-09-14T23:58:18.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:18.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.668135+0000", 2024-09-14T23:58:18.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0080027220000000003 2024-09-14T23:58:18.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:18.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.668172+0000", 2024-09-14T23:58:18.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6295999999999999e-05 2024-09-14T23:58:18.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:18.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.668187+0000", 2024-09-14T23:58:18.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5537e-05 2024-09-14T23:58:18.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:18.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.668284+0000", 2024-09-14T23:58:18.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6297000000000003e-05 2024-09-14T23:58:18.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:58:18.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.668299+0000", 2024-09-14T23:58:18.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5091e-05 2024-09-14T23:58:18.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:18.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:14.195910+0000", 2024-09-14T23:58:18.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.52761157999999997 2024-09-14T23:58:18.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:18.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:14.196106+0000", 2024-09-14T23:58:18.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019597399999999999 2024-09-14T23:58:18.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 0", 2024-09-14T23:58:18.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:14.197262+0000", 2024-09-14T23:58:18.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001155744 2024-09-14T23:58:18.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:58:18.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:14.207440+0000", 2024-09-14T23:58:18.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010178347000000001 2024-09-14T23:58:18.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:18.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:14.213691+0000", 2024-09-14T23:58:18.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0062504420000000002 2024-09-14T23:58:18.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:18.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:14.213735+0000", 2024-09-14T23:58:18.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4385999999999998e-05 2024-09-14T23:58:18.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:18.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:18.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:18.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:14.213773+0000", 2024-09-14T23:58:18.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7701999999999999e-05 2024-09-14T23:58:18.134 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:18.134 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:18.134 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:18.134 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:18.134 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:18.134 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:18.135 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T23:58:18.136 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:58:18.134+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:58:18.210 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 5] out_osds: [7] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T23:58:18.211 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T23:58:18.211 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-09-14T23:58:18.211 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-14T23:58:18.211 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-14T23:58:18.211 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2024-09-14T23:58:18.246 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:18.246 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:58:18.246 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:58:18.246 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:18.247 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:18.245+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:18.255 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T23:58:18.325 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:18.325 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-14T23:58:18.325 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:18.336 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:18.334+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:18.410 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:18.410 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:58:18.410 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:58:18.411 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:58:18.411 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:18.420 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T23:58:18.464 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:58:18.463+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:58:18.529 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:18.529 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:58:18.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:58:18.529 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:58:18.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.530 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186203 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1329152~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:18.530 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.194279+0000", 2024-09-14T23:58:18.530 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 11.332678737, 2024-09-14T23:58:18.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8011469710000001, 2024-09-14T23:58:18.530 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:18.530 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:18.530 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:18.530 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:18.530 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:18.530 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:18.530 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186203 2024-09-14T23:58:18.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.530 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:18.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.530 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:18.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194279+0000", 2024-09-14T23:58:18.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:18.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:18.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194279+0000", 2024-09-14T23:58:18.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:18.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:18.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194279+0000", 2024-09-14T23:58:18.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8100000000000001e-07 2024-09-14T23:58:18.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:18.532 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194291+0000", 2024-09-14T23:58:18.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1559999999999999e-05 2024-09-14T23:58:18.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:18.532 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194303+0000", 2024-09-14T23:58:18.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2130000000000001e-05 2024-09-14T23:58:18.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:18.532 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194311+0000", 2024-09-14T23:58:18.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8089999999999993e-06 2024-09-14T23:58:18.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.533 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:18.533 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194370+0000", 2024-09-14T23:58:18.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8739999999999997e-05 2024-09-14T23:58:18.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.533 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:18.533 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194376+0000", 2024-09-14T23:58:18.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2319999999999998e-06 2024-09-14T23:58:18.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.533 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:18.533 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986290+0000", 2024-09-14T23:58:18.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.791914334 2024-09-14T23:58:18.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:18.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986296+0000", 2024-09-14T23:58:18.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6300000000000003e-06 2024-09-14T23:58:18.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:18.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995193+0000", 2024-09-14T23:58:18.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0088975040000001 2024-09-14T23:58:18.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:18.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995307+0000", 2024-09-14T23:58:18.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000113656 2024-09-14T23:58:18.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:18.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995426+0000", 2024-09-14T23:58:18.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118495 2024-09-14T23:58:18.535 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:18.535 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:18.535 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:18.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.535 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186204 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1391616~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:18.535 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.194328+0000", 2024-09-14T23:58:18.535 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 11.332629406000001, 2024-09-14T23:58:18.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8013834310000001, 2024-09-14T23:58:18.535 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:18.536 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:18.536 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:18.536 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:18.536 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:18.536 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:18.536 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186204 2024-09-14T23:58:18.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.536 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:18.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:18.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194328+0000", 2024-09-14T23:58:18.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:18.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:18.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194328+0000", 2024-09-14T23:58:18.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:18.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:18.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194328+0000", 2024-09-14T23:58:18.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7100000000000002e-07 2024-09-14T23:58:18.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:18.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194331+0000", 2024-09-14T23:58:18.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.807e-06 2024-09-14T23:58:18.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:18.538 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194339+0000", 2024-09-14T23:58:18.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9340000000000006e-06 2024-09-14T23:58:18.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.538 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:18.538 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194345+0000", 2024-09-14T23:58:18.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1090000000000003e-06 2024-09-14T23:58:18.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.538 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:18.538 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194412+0000", 2024-09-14T23:58:18.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6229000000000006e-05 2024-09-14T23:58:18.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.538 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:18.539 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194422+0000", 2024-09-14T23:58:18.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.078e-05 2024-09-14T23:58:18.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.539 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:18.539 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986322+0000", 2024-09-14T23:58:18.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.79189964899999998 2024-09-14T23:58:18.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.539 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:18.539 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986332+0000", 2024-09-14T23:58:18.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0095000000000001e-05 2024-09-14T23:58:18.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:18.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995554+0000", 2024-09-14T23:58:18.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0092218630000001 2024-09-14T23:58:18.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:18.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995610+0000", 2024-09-14T23:58:18.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6109000000000001e-05 2024-09-14T23:58:18.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:18.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995711+0000", 2024-09-14T23:58:18.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101285 2024-09-14T23:58:18.540 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:18.540 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:18.540 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:18.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.541 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186205 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1333248~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:18.541 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.194354+0000", 2024-09-14T23:58:18.541 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 11.332603475999999, 2024-09-14T23:58:18.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8015471490000001, 2024-09-14T23:58:18.541 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:18.541 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:18.541 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:18.541 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:18.541 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:18.541 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:18.541 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186205 2024-09-14T23:58:18.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.541 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:18.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:18.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194354+0000", 2024-09-14T23:58:18.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:18.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:18.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194354+0000", 2024-09-14T23:58:18.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:18.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:18.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194354+0000", 2024-09-14T23:58:18.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.39e-07 2024-09-14T23:58:18.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.543 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:18.543 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194357+0000", 2024-09-14T23:58:18.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3479999999999998e-06 2024-09-14T23:58:18.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.543 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:18.543 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194364+0000", 2024-09-14T23:58:18.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0899999999999999e-06 2024-09-14T23:58:18.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.543 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:18.543 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194371+0000", 2024-09-14T23:58:18.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5530000000000004e-06 2024-09-14T23:58:18.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:18.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194469+0000", 2024-09-14T23:58:18.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8119e-05 2024-09-14T23:58:18.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:18.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194475+0000", 2024-09-14T23:58:18.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7910000000000001e-06 2024-09-14T23:58:18.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:18.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986367+0000", 2024-09-14T23:58:18.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.79189214699999999 2024-09-14T23:58:18.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:18.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986375+0000", 2024-09-14T23:58:18.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7800000000000001e-06 2024-09-14T23:58:18.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:18.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995746+0000", 2024-09-14T23:58:18.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0093706469999999 2024-09-14T23:58:18.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:18.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995810+0000", 2024-09-14T23:58:18.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4068999999999999e-05 2024-09-14T23:58:18.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:18.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995901+0000", 2024-09-14T23:58:18.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1166000000000001e-05 2024-09-14T23:58:18.546 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:18.546 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:18.546 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:18.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.546 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186206 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1395712~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:18.546 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.194380+0000", 2024-09-14T23:58:18.546 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 11.332577454999999, 2024-09-14T23:58:18.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.801678347, 2024-09-14T23:58:18.547 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:18.547 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:18.547 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:18.547 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:18.547 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:18.547 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:18.547 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186206 2024-09-14T23:58:18.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.547 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:18.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:18.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194380+0000", 2024-09-14T23:58:18.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:18.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.548 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.548 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:18.548 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194380+0000", 2024-09-14T23:58:18.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:18.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.548 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.548 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:18.548 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194380+0000", 2024-09-14T23:58:18.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4400000000000001e-07 2024-09-14T23:58:18.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.548 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.548 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:18.548 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194383+0000", 2024-09-14T23:58:18.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3460000000000001e-06 2024-09-14T23:58:18.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:18.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194390+0000", 2024-09-14T23:58:18.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2189999999999998e-06 2024-09-14T23:58:18.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:18.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194403+0000", 2024-09-14T23:58:18.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2941e-05 2024-09-14T23:58:18.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:18.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194545+0000", 2024-09-14T23:58:18.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014232700000000001 2024-09-14T23:58:18.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:18.550 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194565+0000", 2024-09-14T23:58:18.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9409999999999999e-05 2024-09-14T23:58:18.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:18.550 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986409+0000", 2024-09-14T23:58:18.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.79184416000000002 2024-09-14T23:58:18.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:18.550 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986417+0000", 2024-09-14T23:58:18.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.551e-06 2024-09-14T23:58:18.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:18.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995921+0000", 2024-09-14T23:58:18.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.009504212 2024-09-14T23:58:18.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:18.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995974+0000", 2024-09-14T23:58:18.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2617999999999997e-05 2024-09-14T23:58:18.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:18.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996058+0000", 2024-09-14T23:58:18.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4118999999999999e-05 2024-09-14T23:58:18.552 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:18.552 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:18.552 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:18.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.552 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.552 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186207 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1337344~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:18.552 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.194412+0000", 2024-09-14T23:58:18.552 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 11.332544857, 2024-09-14T23:58:18.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.801848141, 2024-09-14T23:58:18.552 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:18.552 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:18.552 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:18.552 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:18.552 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:18.552 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:18.553 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186207 2024-09-14T23:58:18.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.553 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:18.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.553 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:18.553 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194412+0000", 2024-09-14T23:58:18.553 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:18.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.553 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:18.553 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194412+0000", 2024-09-14T23:58:18.553 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:18.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.553 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:18.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194413+0000", 2024-09-14T23:58:18.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5600000000000001e-07 2024-09-14T23:58:18.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:18.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194418+0000", 2024-09-14T23:58:18.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3720000000000004e-06 2024-09-14T23:58:18.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:18.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194423+0000", 2024-09-14T23:58:18.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.763e-06 2024-09-14T23:58:18.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:18.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194426+0000", 2024-09-14T23:58:18.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1480000000000002e-06 2024-09-14T23:58:18.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:18.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194596+0000", 2024-09-14T23:58:18.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017009600000000001 2024-09-14T23:58:18.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:18.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194606+0000", 2024-09-14T23:58:18.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6639999999999993e-06 2024-09-14T23:58:18.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:18.556 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986460+0000", 2024-09-14T23:58:18.556 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.79185424400000004 2024-09-14T23:58:18.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.556 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.556 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:18.556 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986469+0000", 2024-09-14T23:58:18.556 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4419999999999998e-06 2024-09-14T23:58:18.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.556 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.556 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:18.556 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996123+0000", 2024-09-14T23:58:18.556 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0096546070000001 2024-09-14T23:58:18.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.557 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.557 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:18.557 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996179+0000", 2024-09-14T23:58:18.557 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5850000000000002e-05 2024-09-14T23:58:18.557 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.557 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.557 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:18.557 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996261+0000", 2024-09-14T23:58:18.557 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1599000000000003e-05 2024-09-14T23:58:18.557 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:18.557 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:18.557 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:18.557 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.557 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.558 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186208 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1399808~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:18.558 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.194445+0000", 2024-09-14T23:58:18.558 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 11.332512686999999, 2024-09-14T23:58:18.558 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8019822269999999, 2024-09-14T23:58:18.558 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:18.558 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:18.558 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:18.558 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:18.558 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:18.558 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:18.558 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186208 2024-09-14T23:58:18.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.558 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:18.558 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.558 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:18.558 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194445+0000", 2024-09-14T23:58:18.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:18.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:18.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194445+0000", 2024-09-14T23:58:18.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:18.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:18.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978674+0000", 2024-09-14T23:58:18.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.784229124 2024-09-14T23:58:18.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:18.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978680+0000", 2024-09-14T23:58:18.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7969999999999997e-06 2024-09-14T23:58:18.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:18.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978689+0000", 2024-09-14T23:58:18.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6949999999999995e-06 2024-09-14T23:58:18.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:18.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978696+0000", 2024-09-14T23:58:18.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1300000000000003e-06 2024-09-14T23:58:18.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.561 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:18.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986509+0000", 2024-09-14T23:58:18.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0078130969999999998 2024-09-14T23:58:18.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.561 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:18.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986518+0000", 2024-09-14T23:58:18.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2810000000000008e-06 2024-09-14T23:58:18.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.561 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:18.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996280+0000", 2024-09-14T23:58:18.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0097623120000001 2024-09-14T23:58:18.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:18.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996337+0000", 2024-09-14T23:58:18.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7244000000000003e-05 2024-09-14T23:58:18.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:18.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996427+0000", 2024-09-14T23:58:18.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9547000000000002e-05 2024-09-14T23:58:18.562 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:18.562 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:18.562 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:18.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.563 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4172.0:7 3.11 3:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:18.563 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.483386+0000", 2024-09-14T23:58:18.563 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 11.043571262, 2024-09-14T23:58:18.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.558667389, 2024-09-14T23:58:18.563 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:18.563 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:18.563 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:58:18.563 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:18.563 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4172", 2024-09-14T23:58:18.563 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/2542269203", 2024-09-14T23:58:18.563 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 7 2024-09-14T23:58:18.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.563 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:18.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.563 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:18.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.483386+0000", 2024-09-14T23:58:18.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:18.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:18.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.483386+0000", 2024-09-14T23:58:18.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:18.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:18.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978674+0000", 2024-09-14T23:58:18.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.49528781599999999 2024-09-14T23:58:18.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.565 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:18.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978687+0000", 2024-09-14T23:58:18.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3244999999999999e-05 2024-09-14T23:58:18.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.565 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:18.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978699+0000", 2024-09-14T23:58:18.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1832e-05 2024-09-14T23:58:18.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.565 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:18.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978707+0000", 2024-09-14T23:58:18.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4489999999999999e-06 2024-09-14T23:58:18.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:18.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978801+0000", 2024-09-14T23:58:18.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3511999999999995e-05 2024-09-14T23:58:18.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:18.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978814+0000", 2024-09-14T23:58:18.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2789000000000001e-05 2024-09-14T23:58:18.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:18.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.041948+0000", 2024-09-14T23:58:18.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0631338269999999 2024-09-14T23:58:18.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.567 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:18.567 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.042053+0000", 2024-09-14T23:58:18.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105919 2024-09-14T23:58:18.567 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:18.567 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:18.567 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:18.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.567 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187209 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1575936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:18.567 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.991888+0000", 2024-09-14T23:58:18.567 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 10.535069290999999, 2024-09-14T23:58:18.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0712115470000001, 2024-09-14T23:58:18.567 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:18.567 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:18.568 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:18.568 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:18.568 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:18.568 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:18.568 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187209 2024-09-14T23:58:18.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.568 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:18.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.568 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:18.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991888+0000", 2024-09-14T23:58:18.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:18.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.568 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:18.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991888+0000", 2024-09-14T23:58:18.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:18.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.569 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:18.569 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991888+0000", 2024-09-14T23:58:18.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1199999999999998e-07 2024-09-14T23:58:18.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.569 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:18.569 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991890+0000", 2024-09-14T23:58:18.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9810000000000002e-06 2024-09-14T23:58:18.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.569 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:18.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991897+0000", 2024-09-14T23:58:18.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2609999999999999e-06 2024-09-14T23:58:18.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.570 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:18.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991902+0000", 2024-09-14T23:58:18.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0000000000000004e-06 2024-09-14T23:58:18.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.570 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:18.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995686+0000", 2024-09-14T23:58:18.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0037839290000000001 2024-09-14T23:58:18.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:18.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995693+0000", 2024-09-14T23:58:18.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7589999999999997e-06 2024-09-14T23:58:18.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:18.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062846+0000", 2024-09-14T23:58:18.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.067152417 2024-09-14T23:58:18.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:18.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062935+0000", 2024-09-14T23:58:18.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9080999999999996e-05 2024-09-14T23:58:18.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:18.572 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.063100+0000", 2024-09-14T23:58:18.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016470699999999999 2024-09-14T23:58:18.572 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:18.572 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:18.572 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:18.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.572 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187210 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1591296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:18.572 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.991908+0000", 2024-09-14T23:58:18.572 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 10.535049378, 2024-09-14T23:58:18.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0713882530000001, 2024-09-14T23:58:18.572 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:18.572 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:18.573 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:18.573 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:18.573 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:18.573 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:18.573 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187210 2024-09-14T23:58:18.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.573 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:18.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.573 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:18.573 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991908+0000", 2024-09-14T23:58:18.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:18.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.574 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:18.574 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991908+0000", 2024-09-14T23:58:18.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:18.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.574 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:18.574 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993673+0000", 2024-09-14T23:58:18.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017651030000000001 2024-09-14T23:58:18.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.574 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:18.574 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993681+0000", 2024-09-14T23:58:18.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.858e-06 2024-09-14T23:58:18.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:18.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993691+0000", 2024-09-14T23:58:18.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9920000000000002e-06 2024-09-14T23:58:18.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:18.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993698+0000", 2024-09-14T23:58:18.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6490000000000002e-06 2024-09-14T23:58:18.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:18.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995731+0000", 2024-09-14T23:58:18.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0020338959999999999 2024-09-14T23:58:18.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:18.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995741+0000", 2024-09-14T23:58:18.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3079999999999997e-06 2024-09-14T23:58:18.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:18.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.063130+0000", 2024-09-14T23:58:18.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.067389159 2024-09-14T23:58:18.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:18.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.063206+0000", 2024-09-14T23:58:18.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6577999999999998e-05 2024-09-14T23:58:18.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.577 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:18.577 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.063296+0000", 2024-09-14T23:58:18.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9710000000000007e-05 2024-09-14T23:58:18.577 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:18.577 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:18.577 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:18.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.577 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187216 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1603584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:18.577 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.993831+0000", 2024-09-14T23:58:18.577 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 10.533126171999999, 2024-09-14T23:58:18.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.071244627, 2024-09-14T23:58:18.577 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:18.577 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:18.577 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:18.578 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:18.578 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:18.578 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:18.578 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187216 2024-09-14T23:58:18.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.578 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:18.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:18.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993831+0000", 2024-09-14T23:58:18.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:18.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:18.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993831+0000", 2024-09-14T23:58:18.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:18.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:18.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993832+0000", 2024-09-14T23:58:18.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9700000000000002e-07 2024-09-14T23:58:18.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:18.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993834+0000", 2024-09-14T23:58:18.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6170000000000001e-06 2024-09-14T23:58:18.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:18.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993841+0000", 2024-09-14T23:58:18.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6980000000000001e-06 2024-09-14T23:58:18.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:18.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993846+0000", 2024-09-14T23:58:18.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4190000000000001e-06 2024-09-14T23:58:18.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:18.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995988+0000", 2024-09-14T23:58:18.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0021417390000000001 2024-09-14T23:58:18.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:18.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995998+0000", 2024-09-14T23:58:18.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5100000000000004e-06 2024-09-14T23:58:18.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:18.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.064883+0000", 2024-09-14T23:58:18.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.06888597 2024-09-14T23:58:18.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:18.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.064977+0000", 2024-09-14T23:58:18.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3817e-05 2024-09-14T23:58:18.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:18.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065076+0000", 2024-09-14T23:58:18.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8460000000000003e-05 2024-09-14T23:58:18.582 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:18.582 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:18.582 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:18.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.582 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187217 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1592320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:18.582 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.993855+0000", 2024-09-14T23:58:18.582 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 10.533102739, 2024-09-14T23:58:18.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0714239349999999, 2024-09-14T23:58:18.583 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:18.583 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:18.583 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:18.583 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:18.583 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:18.583 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:18.583 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187217 2024-09-14T23:58:18.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.583 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:18.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:18.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993855+0000", 2024-09-14T23:58:18.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:18.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.584 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:18.584 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993855+0000", 2024-09-14T23:58:18.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:18.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.584 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:18.584 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993855+0000", 2024-09-14T23:58:18.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4899999999999995e-07 2024-09-14T23:58:18.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.584 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:18.584 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993857+0000", 2024-09-14T23:58:18.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3379999999999999e-06 2024-09-14T23:58:18.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:18.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993864+0000", 2024-09-14T23:58:18.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6000000000000003e-06 2024-09-14T23:58:18.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:18.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993869+0000", 2024-09-14T23:58:18.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3210000000000004e-06 2024-09-14T23:58:18.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:18.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996038+0000", 2024-09-14T23:58:18.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0021683050000000001 2024-09-14T23:58:18.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:18.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996048+0000", 2024-09-14T23:58:18.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0173999999999999e-05 2024-09-14T23:58:18.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:18.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065108+0000", 2024-09-14T23:58:18.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0690600619999999 2024-09-14T23:58:18.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:18.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065171+0000", 2024-09-14T23:58:18.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3423999999999994e-05 2024-09-14T23:58:18.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:18.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065279+0000", 2024-09-14T23:58:18.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000107162 2024-09-14T23:58:18.587 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:18.587 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:18.587 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:18.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.587 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187218 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1607680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:18.587 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.993877+0000", 2024-09-14T23:58:18.587 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 10.533080459000001, 2024-09-14T23:58:18.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0716976810000001, 2024-09-14T23:58:18.587 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:18.587 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:18.587 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:18.588 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:18.588 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:18.588 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:18.588 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187218 2024-09-14T23:58:18.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.588 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:18.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:18.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993877+0000", 2024-09-14T23:58:18.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:18.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:18.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993877+0000", 2024-09-14T23:58:18.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:18.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:18.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993877+0000", 2024-09-14T23:58:18.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8299999999999998e-07 2024-09-14T23:58:18.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:18.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993879+0000", 2024-09-14T23:58:18.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5659999999999999e-06 2024-09-14T23:58:18.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:18.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993882+0000", 2024-09-14T23:58:18.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7280000000000001e-06 2024-09-14T23:58:18.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:18.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993885+0000", 2024-09-14T23:58:18.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.751e-06 2024-09-14T23:58:18.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:18.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996081+0000", 2024-09-14T23:58:18.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0021960590000000002 2024-09-14T23:58:18.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:18.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996091+0000", 2024-09-14T23:58:18.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4930000000000004e-06 2024-09-14T23:58:18.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:18.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065308+0000", 2024-09-14T23:58:18.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.069217407 2024-09-14T23:58:18.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:18.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065414+0000", 2024-09-14T23:58:18.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105438 2024-09-14T23:58:18.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:18.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065575+0000", 2024-09-14T23:58:18.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000160956 2024-09-14T23:58:18.592 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:18.592 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:18.592 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:18.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.592 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187219 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1596416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:18.592 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.993889+0000", 2024-09-14T23:58:18.592 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 10.533068617, 2024-09-14T23:58:18.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0720031190000001, 2024-09-14T23:58:18.592 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:18.592 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:18.592 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:18.592 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:18.592 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:18.593 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:18.593 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187219 2024-09-14T23:58:18.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.593 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:18.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:18.593 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993889+0000", 2024-09-14T23:58:18.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:18.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:18.593 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993889+0000", 2024-09-14T23:58:18.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:18.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.594 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:18.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993889+0000", 2024-09-14T23:58:18.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2399999999999999e-07 2024-09-14T23:58:18.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.594 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:18.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993890+0000", 2024-09-14T23:58:18.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1939999999999999e-06 2024-09-14T23:58:18.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.594 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:18.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993893+0000", 2024-09-14T23:58:18.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1300000000000001e-06 2024-09-14T23:58:18.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.594 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:18.595 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993896+0000", 2024-09-14T23:58:18.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6879999999999999e-06 2024-09-14T23:58:18.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:18.595 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996119+0000", 2024-09-14T23:58:18.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0022226149999999998 2024-09-14T23:58:18.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:18.595 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996123+0000", 2024-09-14T23:58:18.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.211e-06 2024-09-14T23:58:18.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:18.596 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065616+0000", 2024-09-14T23:58:18.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0694935240000001 2024-09-14T23:58:18.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.596 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:18.596 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065733+0000", 2024-09-14T23:58:18.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000116492 2024-09-14T23:58:18.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.596 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:18.596 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065892+0000", 2024-09-14T23:58:18.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015894100000000001 2024-09-14T23:58:18.596 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:18.596 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:18.597 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:18.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.597 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187220 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1611776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:18.597 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.993899+0000", 2024-09-14T23:58:18.597 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 10.533058158999999, 2024-09-14T23:58:18.597 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.072298041, 2024-09-14T23:58:18.597 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:18.597 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:18.597 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:18.597 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:18.597 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:18.597 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:18.597 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187220 2024-09-14T23:58:18.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.598 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:18.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.598 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:18.598 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993899+0000", 2024-09-14T23:58:18.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:18.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.598 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:18.598 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993899+0000", 2024-09-14T23:58:18.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:18.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.598 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:18.599 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993899+0000", 2024-09-14T23:58:18.599 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2100000000000001e-07 2024-09-14T23:58:18.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.599 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:18.599 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993901+0000", 2024-09-14T23:58:18.599 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.141e-06 2024-09-14T23:58:18.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.599 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:18.599 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993904+0000", 2024-09-14T23:58:18.599 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1470000000000001e-06 2024-09-14T23:58:18.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.600 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:18.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993906+0000", 2024-09-14T23:58:18.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6340000000000001e-06 2024-09-14T23:58:18.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.600 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:18.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996160+0000", 2024-09-14T23:58:18.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0022537659999999999 2024-09-14T23:58:18.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.600 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:18.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996169+0000", 2024-09-14T23:58:18.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2739999999999996e-06 2024-09-14T23:58:18.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:18.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065981+0000", 2024-09-14T23:58:18.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.069811742 2024-09-14T23:58:18.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:18.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.066092+0000", 2024-09-14T23:58:18.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000110701 2024-09-14T23:58:18.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:18.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.066197+0000", 2024-09-14T23:58:18.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010541500000000001 2024-09-14T23:58:18.601 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:18.602 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:18.602 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:18.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.602 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187221 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1600512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:18.602 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.993909+0000", 2024-09-14T23:58:18.602 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 10.533047947, 2024-09-14T23:58:18.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.072643499, 2024-09-14T23:58:18.602 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:18.602 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:18.602 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:18.602 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:18.602 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:18.602 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:18.602 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187221 2024-09-14T23:58:18.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.603 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:18.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:18.603 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993909+0000", 2024-09-14T23:58:18.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:18.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:18.603 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993909+0000", 2024-09-14T23:58:18.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:18.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:18.603 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995673+0000", 2024-09-14T23:58:18.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001763141 2024-09-14T23:58:18.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:18.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995677+0000", 2024-09-14T23:58:18.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8709999999999996e-06 2024-09-14T23:58:18.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:18.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995686+0000", 2024-09-14T23:58:18.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6649999999999992e-06 2024-09-14T23:58:18.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:18.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995691+0000", 2024-09-14T23:58:18.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0289999999999996e-06 2024-09-14T23:58:18.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:18.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996199+0000", 2024-09-14T23:58:18.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00050799099999999995 2024-09-14T23:58:18.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:18.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996205+0000", 2024-09-14T23:58:18.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8259999999999998e-06 2024-09-14T23:58:18.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.606 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:18.606 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.066238+0000", 2024-09-14T23:58:18.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.070032928 2024-09-14T23:58:18.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.606 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:18.606 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.066393+0000", 2024-09-14T23:58:18.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015549800000000001 2024-09-14T23:58:18.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.606 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:18.606 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.066553+0000", 2024-09-14T23:58:18.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015955 2024-09-14T23:58:18.607 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:18.607 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:18.607 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:18.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.607 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187223 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1604608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:18.607 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.995728+0000", 2024-09-14T23:58:18.607 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 10.531229131, 2024-09-14T23:58:18.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.071486478, 2024-09-14T23:58:18.607 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:18.607 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:18.607 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:18.607 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:18.607 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:18.607 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:18.608 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187223 2024-09-14T23:58:18.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.608 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:18.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:18.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995728+0000", 2024-09-14T23:58:18.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:18.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:18.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995728+0000", 2024-09-14T23:58:18.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:18.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:18.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995728+0000", 2024-09-14T23:58:18.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.65e-07 2024-09-14T23:58:18.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:18.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995730+0000", 2024-09-14T23:58:18.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.406e-06 2024-09-14T23:58:18.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:18.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995734+0000", 2024-09-14T23:58:18.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2640000000000002e-06 2024-09-14T23:58:18.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:18.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995737+0000", 2024-09-14T23:58:18.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0589999999999998e-06 2024-09-14T23:58:18.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:18.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996288+0000", 2024-09-14T23:58:18.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00055059599999999998 2024-09-14T23:58:18.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:18.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996294+0000", 2024-09-14T23:58:18.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0260000000000004e-06 2024-09-14T23:58:18.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.611 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:18.611 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.066938+0000", 2024-09-14T23:58:18.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0706439510000001 2024-09-14T23:58:18.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.611 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:18.611 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067050+0000", 2024-09-14T23:58:18.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011220100000000001 2024-09-14T23:58:18.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.611 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:18.611 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067215+0000", 2024-09-14T23:58:18.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016470999999999999 2024-09-14T23:58:18.611 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:18.612 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:18.612 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:18.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.612 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187224 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1619968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:18.612 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.995742+0000", 2024-09-14T23:58:18.612 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 10.531215316000001, 2024-09-14T23:58:18.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0717603609999999, 2024-09-14T23:58:18.612 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:18.612 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:18.612 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:18.612 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:18.612 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:18.612 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:18.612 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187224 2024-09-14T23:58:18.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.613 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:18.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:18.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995742+0000", 2024-09-14T23:58:18.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:18.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:18.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995742+0000", 2024-09-14T23:58:18.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:18.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:18.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995742+0000", 2024-09-14T23:58:18.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2399999999999999e-07 2024-09-14T23:58:18.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.614 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:18.614 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995744+0000", 2024-09-14T23:58:18.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.356e-06 2024-09-14T23:58:18.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.614 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:18.614 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995747+0000", 2024-09-14T23:58:18.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.304e-06 2024-09-14T23:58:18.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.614 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:18.614 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995750+0000", 2024-09-14T23:58:18.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6510000000000001e-06 2024-09-14T23:58:18.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.615 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:18.615 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996327+0000", 2024-09-14T23:58:18.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00057794300000000003 2024-09-14T23:58:18.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.615 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:18.615 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996337+0000", 2024-09-14T23:58:18.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3759999999999997e-06 2024-09-14T23:58:18.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.615 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:18.615 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067244+0000", 2024-09-14T23:58:18.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.070907367 2024-09-14T23:58:18.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:18.616 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067341+0000", 2024-09-14T23:58:18.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7277999999999999e-05 2024-09-14T23:58:18.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:18.616 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067502+0000", 2024-09-14T23:58:18.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016086200000000001 2024-09-14T23:58:18.616 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:18.616 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:18.616 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:18.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.617 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187225 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1608704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:18.617 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.995766+0000", 2024-09-14T23:58:18.617 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 10.531191142000001, 2024-09-14T23:58:18.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.072043649, 2024-09-14T23:58:18.617 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:18.617 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:18.617 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:18.617 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:18.617 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:18.617 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:18.617 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187225 2024-09-14T23:58:18.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.617 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:18.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:18.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995766+0000", 2024-09-14T23:58:18.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:18.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:18.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995766+0000", 2024-09-14T23:58:18.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:18.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:18.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995766+0000", 2024-09-14T23:58:18.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9200000000000002e-07 2024-09-14T23:58:18.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:18.619 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995768+0000", 2024-09-14T23:58:18.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8679999999999999e-06 2024-09-14T23:58:18.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.619 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:18.619 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995772+0000", 2024-09-14T23:58:18.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6339999999999999e-06 2024-09-14T23:58:18.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.619 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:18.619 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995775+0000", 2024-09-14T23:58:18.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8159999999999998e-06 2024-09-14T23:58:18.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:18.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996372+0000", 2024-09-14T23:58:18.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00059758800000000003 2024-09-14T23:58:18.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:18.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996377+0000", 2024-09-14T23:58:18.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7770000000000002e-06 2024-09-14T23:58:18.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:18.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067534+0000", 2024-09-14T23:58:18.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.071156821 2024-09-14T23:58:18.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:18.621 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067638+0000", 2024-09-14T23:58:18.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000103737 2024-09-14T23:58:18.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:18.621 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067810+0000", 2024-09-14T23:58:18.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017211599999999999 2024-09-14T23:58:18.621 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:18.621 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:18.621 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:18.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.621 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187226 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1624064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:18.622 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.995778+0000", 2024-09-14T23:58:18.622 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 10.531178963, 2024-09-14T23:58:18.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.072339081, 2024-09-14T23:58:18.622 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:18.622 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:18.622 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:18.622 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:18.622 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:18.622 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:18.622 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187226 2024-09-14T23:58:18.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.622 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:18.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:18.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995778+0000", 2024-09-14T23:58:18.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:18.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:18.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995778+0000", 2024-09-14T23:58:18.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:18.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:18.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995779+0000", 2024-09-14T23:58:18.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0699999999999999e-07 2024-09-14T23:58:18.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:18.624 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995780+0000", 2024-09-14T23:58:18.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2139999999999999e-06 2024-09-14T23:58:18.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.624 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:18.624 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995783+0000", 2024-09-14T23:58:18.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1250000000000001e-06 2024-09-14T23:58:18.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.624 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:18.624 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995787+0000", 2024-09-14T23:58:18.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6679999999999999e-06 2024-09-14T23:58:18.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.625 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:18.625 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996404+0000", 2024-09-14T23:58:18.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00061777499999999999 2024-09-14T23:58:18.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.625 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:18.625 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996414+0000", 2024-09-14T23:58:18.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.431e-06 2024-09-14T23:58:18.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.625 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:18.625 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067844+0000", 2024-09-14T23:58:18.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0714302410000001 2024-09-14T23:58:18.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:18.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067952+0000", 2024-09-14T23:58:18.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000107778 2024-09-14T23:58:18.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:18.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.068117+0000", 2024-09-14T23:58:18.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000165642 2024-09-14T23:58:18.626 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:18.626 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:18.626 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:18.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.626 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187227 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1612800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:18.626 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.995806+0000", 2024-09-14T23:58:18.627 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 10.531151726000001, 2024-09-14T23:58:18.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0726092810000001, 2024-09-14T23:58:18.627 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:18.627 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:18.627 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:18.627 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:18.627 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:18.627 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:18.627 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187227 2024-09-14T23:58:18.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.627 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:18.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:18.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995806+0000", 2024-09-14T23:58:18.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:18.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:18.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995806+0000", 2024-09-14T23:58:18.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:18.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:18.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995672+0000", 2024-09-14T23:58:18.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99986608099999996 2024-09-14T23:58:18.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:18.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995677+0000", 2024-09-14T23:58:18.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9520000000000002e-06 2024-09-14T23:58:18.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.629 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:18.629 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995686+0000", 2024-09-14T23:58:18.629 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8199999999999992e-06 2024-09-14T23:58:18.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.629 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:18.629 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995693+0000", 2024-09-14T23:58:18.629 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1630000000000001e-06 2024-09-14T23:58:18.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.629 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:18.629 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.068149+0000", 2024-09-14T23:58:18.629 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.072456692000000003 2024-09-14T23:58:18.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:18.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.068251+0000", 2024-09-14T23:58:18.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101963 2024-09-14T23:58:18.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:18.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:18.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:18.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.068415+0000", 2024-09-14T23:58:18.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016360999999999999 2024-09-14T23:58:18.630 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:18.630 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:18.630 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:18.630 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:18.630 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:18.630 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:18.632 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T23:58:18.632 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:18.630+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:18.732 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:18.730+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:18.781 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:18.781 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:58:18.781 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:58:18.781 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:18.789 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T23:58:18.833 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:18.831+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:18.896 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:18.896 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:58:18.896 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:58:18.896 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:58:18.897 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:18.905 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T23:58:18.911 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T23:58:18.949 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:18.948+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:19.014 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:19.014 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:58:19.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:58:19.014 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:58:19.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.014 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4284.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-14T23:58:19.014 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:58:19.014 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 116.48729112300001, 2024-09-14T23:58:19.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.004942086, 2024-09-14T23:58:19.015 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:19.015 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:19.015 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:19.015 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:19.015 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4284", 2024-09-14T23:58:19.015 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/4278347926", 2024-09-14T23:58:19.015 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1 2024-09-14T23:58:19.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.015 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:19.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.015 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:19.015 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:58:19.015 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:19.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.016 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:19.016 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:58:19.016 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:19.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.016 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.016 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:19.016 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524331+0000", 2024-09-14T23:58:19.016 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.383e-05 2024-09-14T23:58:19.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.016 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.016 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:19.016 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524339+0000", 2024-09-14T23:58:19.017 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8290000000000008e-06 2024-09-14T23:58:19.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.017 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.017 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:19.017 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524359+0000", 2024-09-14T23:58:19.017 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9998000000000001e-05 2024-09-14T23:58:19.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.017 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.017 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:19.017 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524373+0000", 2024-09-14T23:58:19.017 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4304000000000001e-05 2024-09-14T23:58:19.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.017 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.018 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:19.018 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524488+0000", 2024-09-14T23:58:19.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114663 2024-09-14T23:58:19.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.018 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:19.018 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524501+0000", 2024-09-14T23:58:19.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3349999999999999e-05 2024-09-14T23:58:19.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.018 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:19.018 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.526868+0000", 2024-09-14T23:58:19.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023670090000000002 2024-09-14T23:58:19.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:19.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.527041+0000", 2024-09-14T23:58:19.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000172597 2024-09-14T23:58:19.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-14T23:58:19.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.527384+0000", 2024-09-14T23:58:19.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00034309500000000002 2024-09-14T23:58:19.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:19.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529103+0000", 2024-09-14T23:58:19.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017185220000000001 2024-09-14T23:58:19.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.020 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:58:19.020 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529188+0000", 2024-09-14T23:58:19.020 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5591000000000001e-05 2024-09-14T23:58:19.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.020 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:19.020 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529227+0000", 2024-09-14T23:58:19.020 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8563000000000003e-05 2024-09-14T23:58:19.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.020 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:19.020 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529259+0000", 2024-09-14T23:58:19.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2734999999999997e-05 2024-09-14T23:58:19.021 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:19.021 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:19.021 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:19.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.021 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4284.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-14T23:58:19.021 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:23.530683+0000", 2024-09-14T23:58:19.021 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 115.480925458, 2024-09-14T23:58:19.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0036133110000000001, 2024-09-14T23:58:19.021 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:19.021 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:19.021 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:19.022 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:19.022 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4284", 2024-09-14T23:58:19.022 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/4278347926", 2024-09-14T23:58:19.022 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5 2024-09-14T23:58:19.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.022 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:19.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.022 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:19.022 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530683+0000", 2024-09-14T23:58:19.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:19.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.022 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:19.022 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530683+0000", 2024-09-14T23:58:19.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:19.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.023 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:19.023 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530685+0000", 2024-09-14T23:58:19.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.075e-06 2024-09-14T23:58:19.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.023 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:19.023 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530695+0000", 2024-09-14T23:58:19.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0261999999999999e-05 2024-09-14T23:58:19.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.024 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:19.024 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530715+0000", 2024-09-14T23:58:19.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9930000000000001e-05 2024-09-14T23:58:19.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.024 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:19.024 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530724+0000", 2024-09-14T23:58:19.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.901e-06 2024-09-14T23:58:19.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.024 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:19.024 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530800+0000", 2024-09-14T23:58:19.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5873999999999999e-05 2024-09-14T23:58:19.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-14T23:58:19.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530807+0000", 2024-09-14T23:58:19.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2389999999999996e-06 2024-09-14T23:58:19.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:19.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.532494+0000", 2024-09-14T23:58:19.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016871830000000001 2024-09-14T23:58:19.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:19.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.532599+0000", 2024-09-14T23:58:19.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010472 2024-09-14T23:58:19.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1", 2024-09-14T23:58:19.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.532715+0000", 2024-09-14T23:58:19.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011568199999999999 2024-09-14T23:58:19.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:58:19.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.533460+0000", 2024-09-14T23:58:19.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00074494999999999997 2024-09-14T23:58:19.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:19.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534230+0000", 2024-09-14T23:58:19.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00077036599999999998 2024-09-14T23:58:19.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.027 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.027 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:19.027 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534266+0000", 2024-09-14T23:58:19.027 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5725000000000001e-05 2024-09-14T23:58:19.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.027 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.027 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:19.027 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534296+0000", 2024-09-14T23:58:19.027 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0403999999999999e-05 2024-09-14T23:58:19.027 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:19.027 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:19.027 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:19.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.028 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56383 3.c e56/49 3:36b698b0:::rbd_data.1118c4b3aebe.0000000000000000:head v 56'194, mlcod=56'192)", 2024-09-14T23:58:19.028 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.270483+0000", 2024-09-14T23:58:19.028 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 72.741125658000001, 2024-09-14T23:58:19.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0079685880000000004, 2024-09-14T23:58:19.028 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:19.028 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:19.028 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:19.028 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:19.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.028 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:19.028 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.270483+0000", 2024-09-14T23:58:19.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:19.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:19.029 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.270483+0000", 2024-09-14T23:58:19.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:19.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:19.029 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.270484+0000", 2024-09-14T23:58:19.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7400000000000002e-07 2024-09-14T23:58:19.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:19.029 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273680+0000", 2024-09-14T23:58:19.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031966680000000002 2024-09-14T23:58:19.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:19.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273697+0000", 2024-09-14T23:58:19.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6595e-05 2024-09-14T23:58:19.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:19.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273705+0000", 2024-09-14T23:58:19.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7530000000000001e-06 2024-09-14T23:58:19.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:19.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273763+0000", 2024-09-14T23:58:19.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8238e-05 2024-09-14T23:58:19.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:19.031 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273781+0000", 2024-09-14T23:58:19.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8281e-05 2024-09-14T23:58:19.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.031 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:19.031 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.278408+0000", 2024-09-14T23:58:19.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00462716 2024-09-14T23:58:19.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.031 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:19.031 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.278451+0000", 2024-09-14T23:58:19.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3019000000000003e-05 2024-09-14T23:58:19.031 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:19.031 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:19.032 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:19.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.032 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56384 3.c e56/49 3:36b698b0:::rbd_data.1118c4b3aebe.0000000000000000:head v 56'196, mlcod=56'192)", 2024-09-14T23:58:19.032 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:58:19.032 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 72.737092985999993, 2024-09-14T23:58:19.032 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0046707529999999997, 2024-09-14T23:58:19.032 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:19.032 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:19.032 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:19.032 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:19.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:19.033 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:58:19.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:19.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.033 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:19.033 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:58:19.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:19.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.033 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:19.033 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:58:19.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1699999999999997e-07 2024-09-14T23:58:19.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.033 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:19.034 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274523+0000", 2024-09-14T23:58:19.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9129999999999997e-06 2024-09-14T23:58:19.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.034 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:19.034 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274535+0000", 2024-09-14T23:58:19.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1788e-05 2024-09-14T23:58:19.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.034 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:19.034 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274542+0000", 2024-09-14T23:58:19.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0539999999999997e-06 2024-09-14T23:58:19.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:19.035 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275889+0000", 2024-09-14T23:58:19.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00134669 2024-09-14T23:58:19.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:19.035 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275917+0000", 2024-09-14T23:58:19.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7664000000000001e-05 2024-09-14T23:58:19.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:19.035 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279134+0000", 2024-09-14T23:58:19.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0032178459999999999 2024-09-14T23:58:19.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:19.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279186+0000", 2024-09-14T23:58:19.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1881e-05 2024-09-14T23:58:19.036 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:19.036 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:19.036 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:19.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.036 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56381 3.1d e56/49 3:bcf9a8cf:::rbd_data.1118c4b3aebe.0000000000000006:head v 56'210, mlcod=56'208)", 2024-09-14T23:58:19.036 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.275462+0000", 2024-09-14T23:58:19.036 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 72.736146656000003, 2024-09-14T23:58:19.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0071310710000000001, 2024-09-14T23:58:19.036 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:19.037 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:19.037 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:19.037 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:19.037 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.037 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:19.037 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275462+0000", 2024-09-14T23:58:19.037 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:19.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.037 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.037 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:19.037 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275462+0000", 2024-09-14T23:58:19.037 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:19.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.037 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.038 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:19.038 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275463+0000", 2024-09-14T23:58:19.038 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2699999999999999e-07 2024-09-14T23:58:19.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.038 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.038 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:19.038 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276779+0000", 2024-09-14T23:58:19.038 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0013163649999999999 2024-09-14T23:58:19.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.038 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.038 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:19.038 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276795+0000", 2024-09-14T23:58:19.038 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.59e-05 2024-09-14T23:58:19.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:19.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276806+0000", 2024-09-14T23:58:19.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1106e-05 2024-09-14T23:58:19.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:19.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276941+0000", 2024-09-14T23:58:19.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013477500000000001 2024-09-14T23:58:19.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:19.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276963+0000", 2024-09-14T23:58:19.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2419999999999999e-05 2024-09-14T23:58:19.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.040 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:19.040 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282546+0000", 2024-09-14T23:58:19.040 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0055827100000000003 2024-09-14T23:58:19.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.040 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:19.040 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282593+0000", 2024-09-14T23:58:19.040 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7067999999999997e-05 2024-09-14T23:58:19.040 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:19.040 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:19.040 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:19.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.041 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.041 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56387 3.c e56/49 3:30ee4ba0:::rbd_data.1118c4b3aebe.0000000000000004:head v 56'197, mlcod=56'192)", 2024-09-14T23:58:19.041 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.276498+0000", 2024-09-14T23:58:19.041 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 72.735110560999999, 2024-09-14T23:58:19.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0092192639999999996, 2024-09-14T23:58:19.041 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:19.041 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:19.041 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:19.041 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:19.041 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.041 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:19.041 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276498+0000", 2024-09-14T23:58:19.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:19.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.042 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.042 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:19.042 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276498+0000", 2024-09-14T23:58:19.042 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:19.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.042 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.042 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:19.042 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276499+0000", 2024-09-14T23:58:19.042 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1100000000000004e-07 2024-09-14T23:58:19.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.042 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.042 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:19.042 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279646+0000", 2024-09-14T23:58:19.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031473220000000001 2024-09-14T23:58:19.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.043 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:19.043 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279672+0000", 2024-09-14T23:58:19.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6129999999999999e-05 2024-09-14T23:58:19.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.043 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:19.043 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279682+0000", 2024-09-14T23:58:19.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7170000000000003e-06 2024-09-14T23:58:19.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.044 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:19.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279742+0000", 2024-09-14T23:58:19.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9926999999999997e-05 2024-09-14T23:58:19.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.044 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:19.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279761+0000", 2024-09-14T23:58:19.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9402000000000002e-05 2024-09-14T23:58:19.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.044 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:19.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.285680+0000", 2024-09-14T23:58:19.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.005918881 2024-09-14T23:58:19.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.045 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.045 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:19.045 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.285717+0000", 2024-09-14T23:58:19.045 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6974000000000002e-05 2024-09-14T23:58:19.045 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:19.045 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:19.045 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:19.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.045 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.045 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56388 3.c e56/49 3:30ee4ba0:::rbd_data.1118c4b3aebe.0000000000000004:head v 56'199, mlcod=56'192)", 2024-09-14T23:58:19.045 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:58:19.045 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 72.731618972999996, 2024-09-14T23:58:19.045 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0060596740000000001, 2024-09-14T23:58:19.045 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:19.046 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:19.046 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:19.046 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:19.046 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.046 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:19.046 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:58:19.046 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:19.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.046 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.046 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:19.046 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:58:19.046 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:19.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.046 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.047 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:19.047 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:58:19.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2200000000000003e-07 2024-09-14T23:58:19.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.047 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:19.047 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279995+0000", 2024-09-14T23:58:19.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5569999999999997e-06 2024-09-14T23:58:19.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.047 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:19.047 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280004+0000", 2024-09-14T23:58:19.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0080000000000008e-06 2024-09-14T23:58:19.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.048 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:19.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280010+0000", 2024-09-14T23:58:19.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5749999999999997e-06 2024-09-14T23:58:19.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.048 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.048 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:19.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282968+0000", 2024-09-14T23:58:19.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0029579390000000001 2024-09-14T23:58:19.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.048 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.048 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:19.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282998+0000", 2024-09-14T23:58:19.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9947999999999999e-05 2024-09-14T23:58:19.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.048 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.049 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:19.049 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.286012+0000", 2024-09-14T23:58:19.049 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030132900000000001 2024-09-14T23:58:19.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.049 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.049 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:19.049 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.286049+0000", 2024-09-14T23:58:19.049 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7534999999999999e-05 2024-09-14T23:58:19.049 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:19.049 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:19.049 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:19.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.049 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.049 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56382 3.1d e56/49 3:bcf9a8cf:::rbd_data.1118c4b3aebe.0000000000000006:head v 56'212, mlcod=56'208)", 2024-09-14T23:58:19.050 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.280285+0000", 2024-09-14T23:58:19.050 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 72.731323290000006, 2024-09-14T23:58:19.050 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035393489999999998, 2024-09-14T23:58:19.050 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:19.050 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:19.050 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:19.050 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:19.050 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.050 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:19.050 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280285+0000", 2024-09-14T23:58:19.050 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:19.050 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.050 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.050 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:19.050 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280285+0000", 2024-09-14T23:58:19.051 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:19.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.051 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.051 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:19.051 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280286+0000", 2024-09-14T23:58:19.051 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5000000000000001e-07 2024-09-14T23:58:19.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.051 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.051 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:19.051 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280299+0000", 2024-09-14T23:58:19.051 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3312000000000001e-05 2024-09-14T23:58:19.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.051 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.051 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:19.052 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280311+0000", 2024-09-14T23:58:19.052 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1677e-05 2024-09-14T23:58:19.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.052 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.052 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:19.052 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280318+0000", 2024-09-14T23:58:19.052 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.742e-06 2024-09-14T23:58:19.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.052 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.052 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:19.052 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280374+0000", 2024-09-14T23:58:19.052 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.592e-05 2024-09-14T23:58:19.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.052 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.053 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:19.053 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280392+0000", 2024-09-14T23:58:19.053 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8490000000000001e-05 2024-09-14T23:58:19.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.053 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:19.053 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.283693+0000", 2024-09-14T23:58:19.053 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033012050000000002 2024-09-14T23:58:19.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.053 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:19.053 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.283825+0000", 2024-09-14T23:58:19.053 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013105300000000001 2024-09-14T23:58:19.053 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:19.053 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:19.054 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:19.054 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.054 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.054 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "pg_update_log_missing(3.c epoch 56/49 rep_tid 1493 entries 56'200 (0'0) error 3:370203a3:::rbd_data.1118c4b3aebe.000000000000000d:head by client.4376.0:56396 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 56'192)", 2024-09-14T23:58:19.054 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:58:19.054 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 72.731098833000004, 2024-09-14T23:58:19.054 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057370099999999999, 2024-09-14T23:58:19.054 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:19.054 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:19.054 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:58:19.054 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:19.054 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.054 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:19.054 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:58:19.055 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:19.055 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.055 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.055 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:19.055 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:58:19.055 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:19.055 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.055 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.055 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:19.055 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:58:19.055 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0699999999999997e-07 2024-09-14T23:58:19.055 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.055 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.056 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:19.056 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280519+0000", 2024-09-14T23:58:19.056 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6260000000000001e-06 2024-09-14T23:58:19.056 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.056 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.056 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:19.056 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280541+0000", 2024-09-14T23:58:19.056 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1976000000000001e-05 2024-09-14T23:58:19.056 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.056 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.056 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:19.056 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280556+0000", 2024-09-14T23:58:19.056 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5075000000000001e-05 2024-09-14T23:58:19.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.057 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.057 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:19.057 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.285494+0000", 2024-09-14T23:58:19.057 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0049378690000000001 2024-09-14T23:58:19.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.057 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.057 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:19.057 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.286247+0000", 2024-09-14T23:58:19.057 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00075285700000000001 2024-09-14T23:58:19.057 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:19.057 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:19.057 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:19.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.058 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.058 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56400 3.1c e56/49 3:3f5b81ba:::rbd_data.1118c4b3aebe.000000000000000a:head v 56'236, mlcod=56'234)", 2024-09-14T23:58:19.058 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.295064+0000", 2024-09-14T23:58:19.058 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 72.716544041000006, 2024-09-14T23:58:19.058 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0054331589999999999, 2024-09-14T23:58:19.058 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:19.058 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:19.058 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:19.058 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:19.058 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.058 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:19.059 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.295064+0000", 2024-09-14T23:58:19.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:19.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.059 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:19.059 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.295064+0000", 2024-09-14T23:58:19.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:19.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.059 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:19.059 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.295066+0000", 2024-09-14T23:58:19.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.502e-06 2024-09-14T23:58:19.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.060 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.060 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:19.060 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296403+0000", 2024-09-14T23:58:19.060 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001336554 2024-09-14T23:58:19.060 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.060 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.060 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:19.060 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296428+0000", 2024-09-14T23:58:19.060 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5179000000000001e-05 2024-09-14T23:58:19.060 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.060 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.060 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:19.060 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296440+0000", 2024-09-14T23:58:19.060 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2387e-05 2024-09-14T23:58:19.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.061 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.061 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:19.061 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296554+0000", 2024-09-14T23:58:19.061 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114269 2024-09-14T23:58:19.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.061 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.061 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:19.061 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296581+0000", 2024-09-14T23:58:19.061 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6673999999999999e-05 2024-09-14T23:58:19.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.061 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.062 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:19.062 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.300418+0000", 2024-09-14T23:58:19.062 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0038371249999999998 2024-09-14T23:58:19.062 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.062 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.062 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:19.062 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.300498+0000", 2024-09-14T23:58:19.062 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9468999999999994e-05 2024-09-14T23:58:19.062 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:19.062 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:19.062 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:19.062 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.062 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.062 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:61921 3.5 e57/49 3:a05d004f:::rbd_data.11188d356229.000000000000000b:head v 57'279, mlcod=57'277)", 2024-09-14T23:58:19.063 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:07.560389+0000", 2024-09-14T23:58:19.063 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 71.451219442999999, 2024-09-14T23:58:19.063 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0034956520000000001, 2024-09-14T23:58:19.063 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:19.063 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:19.063 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:19.063 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:19.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:19.063 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.560389+0000", 2024-09-14T23:58:19.063 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:19.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:19.063 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.560389+0000", 2024-09-14T23:58:19.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:19.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:19.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.560390+0000", 2024-09-14T23:58:19.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4109999999999999e-06 2024-09-14T23:58:19.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:19.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.561839+0000", 2024-09-14T23:58:19.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001448406 2024-09-14T23:58:19.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.065 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:19.065 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.561854+0000", 2024-09-14T23:58:19.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5426000000000001e-05 2024-09-14T23:58:19.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.065 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.065 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:19.065 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.561863+0000", 2024-09-14T23:58:19.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7509999999999997e-06 2024-09-14T23:58:19.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.065 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.065 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:19.065 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.561917+0000", 2024-09-14T23:58:19.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3531e-05 2024-09-14T23:58:19.066 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.066 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.066 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:19.066 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.561944+0000", 2024-09-14T23:58:19.066 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7528000000000001e-05 2024-09-14T23:58:19.066 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.066 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.066 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:19.066 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563843+0000", 2024-09-14T23:58:19.066 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001899361 2024-09-14T23:58:19.066 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.066 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.066 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:19.066 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563885+0000", 2024-09-14T23:58:19.067 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1238000000000001e-05 2024-09-14T23:58:19.067 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:19.067 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:19.067 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:19.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.067 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.067 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:61922 3.5 e57/49 3:a11ebf8e:::rbd_data.11188d356229.000000000000000c:head v 57'280, mlcod=57'277)", 2024-09-14T23:58:19.067 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:07.563144+0000", 2024-09-14T23:58:19.067 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 71.448464388999994, 2024-09-14T23:58:19.067 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003977848, 2024-09-14T23:58:19.067 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:19.067 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:19.067 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:19.067 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:19.067 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.068 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:19.068 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563144+0000", 2024-09-14T23:58:19.068 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:19.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.068 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.068 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:19.068 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563144+0000", 2024-09-14T23:58:19.068 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:19.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.068 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.068 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:19.068 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563145+0000", 2024-09-14T23:58:19.068 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5400000000000001e-07 2024-09-14T23:58:19.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.069 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.069 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:19.069 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565056+0000", 2024-09-14T23:58:19.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0019108160000000001 2024-09-14T23:58:19.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.069 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.069 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:19.069 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565088+0000", 2024-09-14T23:58:19.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2586999999999998e-05 2024-09-14T23:58:19.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.069 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.069 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:19.069 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565100+0000", 2024-09-14T23:58:19.070 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1917e-05 2024-09-14T23:58:19.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.070 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:19.070 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565166+0000", 2024-09-14T23:58:19.070 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6245e-05 2024-09-14T23:58:19.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.070 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:19.070 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565190+0000", 2024-09-14T23:58:19.070 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3825999999999999e-05 2024-09-14T23:58:19.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.070 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:19.071 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.567080+0000", 2024-09-14T23:58:19.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001889541 2024-09-14T23:58:19.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.071 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.071 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:19.071 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.567122+0000", 2024-09-14T23:58:19.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2262000000000003e-05 2024-09-14T23:58:19.071 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:19.071 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:19.071 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:19.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.071 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.071 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:66564 3.c e60/49 3:33faa7ca:::rbd_data.111873792d9b.0000000000000003:head v 60'236, mlcod=57'233)", 2024-09-14T23:58:19.071 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:11.271596+0000", 2024-09-14T23:58:19.071 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 67.740012811, 2024-09-14T23:58:19.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0060396099999999999, 2024-09-14T23:58:19.072 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:19.072 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:19.072 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:19.072 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:19.072 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.072 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:19.072 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.271596+0000", 2024-09-14T23:58:19.072 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:19.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.072 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.072 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:19.072 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.271596+0000", 2024-09-14T23:58:19.072 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:19.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.073 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.073 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:19.073 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.271597+0000", 2024-09-14T23:58:19.073 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2559999999999999e-06 2024-09-14T23:58:19.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.073 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.073 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:19.073 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.274854+0000", 2024-09-14T23:58:19.073 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0032574470000000001 2024-09-14T23:58:19.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.073 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.073 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:19.073 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.274874+0000", 2024-09-14T23:58:19.074 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9449999999999998e-05 2024-09-14T23:58:19.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.074 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.074 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:19.074 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.274883+0000", 2024-09-14T23:58:19.074 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5980000000000003e-06 2024-09-14T23:58:19.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.074 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.074 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:19.074 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.274977+0000", 2024-09-14T23:58:19.074 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3270999999999995e-05 2024-09-14T23:58:19.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.074 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.074 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:19.075 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.275000+0000", 2024-09-14T23:58:19.075 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3088e-05 2024-09-14T23:58:19.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.075 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.075 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:19.075 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.277590+0000", 2024-09-14T23:58:19.075 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0025898660000000001 2024-09-14T23:58:19.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.075 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.075 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:19.075 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.277635+0000", 2024-09-14T23:58:19.075 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5633999999999997e-05 2024-09-14T23:58:19.075 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:19.076 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:19.076 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:19.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.076 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.076 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:132869 3.16 e92/75 3:681b4799:::rbd_data.11186132a8b6.0000000000000001:head v 92'262, mlcod=87'260)", 2024-09-14T23:58:19.076 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:45.195492+0000", 2024-09-14T23:58:19.076 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 33.816116057999999, 2024-09-14T23:58:19.076 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0034654880000000001, 2024-09-14T23:58:19.076 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:19.076 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:19.076 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:19.076 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:19.076 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.076 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.195492+0000", 2024-09-14T23:58:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.195492+0000", 2024-09-14T23:58:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.195494+0000", 2024-09-14T23:58:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.905e-06 2024-09-14T23:58:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.078 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:19.078 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.196549+0000", 2024-09-14T23:58:19.078 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010545489999999999 2024-09-14T23:58:19.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.078 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.078 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:19.078 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.196572+0000", 2024-09-14T23:58:19.078 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3603000000000001e-05 2024-09-14T23:58:19.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.078 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.078 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:19.078 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.196584+0000", 2024-09-14T23:58:19.078 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.169e-05 2024-09-14T23:58:19.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.079 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.079 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:19.079 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.196676+0000", 2024-09-14T23:58:19.079 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2088000000000003e-05 2024-09-14T23:58:19.079 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.079 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.079 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:19.079 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.196699+0000", 2024-09-14T23:58:19.079 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2365999999999999e-05 2024-09-14T23:58:19.079 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.079 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.079 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:19.079 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.198910+0000", 2024-09-14T23:58:19.079 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0022118609999999999 2024-09-14T23:58:19.080 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.080 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.080 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:19.080 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.198958+0000", 2024-09-14T23:58:19.080 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7426000000000003e-05 2024-09-14T23:58:19.080 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:19.080 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:19.080 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:19.080 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.080 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.080 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:133384 3.1f e92/49 3:fce5e529:::rbd_data.11186132a8b6.0000000000000002:head v 92'807, mlcod=89'804)", 2024-09-14T23:58:19.080 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:45.293642+0000", 2024-09-14T23:58:19.080 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 33.717966928000003, 2024-09-14T23:58:19.080 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003760186, 2024-09-14T23:58:19.080 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:19.081 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:19.081 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:19.081 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:19.081 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.081 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:19.081 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.293642+0000", 2024-09-14T23:58:19.081 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:19.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.081 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.081 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:19.081 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.293642+0000", 2024-09-14T23:58:19.081 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:19.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.081 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.081 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:19.082 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.293643+0000", 2024-09-14T23:58:19.082 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2500000000000001e-06 2024-09-14T23:58:19.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.082 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.082 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:19.082 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.295402+0000", 2024-09-14T23:58:19.082 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017595410000000001 2024-09-14T23:58:19.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.082 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.082 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:19.082 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.295422+0000", 2024-09-14T23:58:19.082 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9763999999999999e-05 2024-09-14T23:58:19.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.082 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.082 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:19.083 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.295436+0000", 2024-09-14T23:58:19.083 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.364e-05 2024-09-14T23:58:19.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.083 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.083 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:19.083 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.295499+0000", 2024-09-14T23:58:19.083 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2770999999999999e-05 2024-09-14T23:58:19.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.083 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.083 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:19.083 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.295516+0000", 2024-09-14T23:58:19.083 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7240000000000001e-05 2024-09-14T23:58:19.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.084 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.084 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:19.084 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.297324+0000", 2024-09-14T23:58:19.084 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001807857 2024-09-14T23:58:19.084 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.084 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.084 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:19.084 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.297402+0000", 2024-09-14T23:58:19.084 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8122999999999997e-05 2024-09-14T23:58:19.084 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:19.084 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:19.084 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:19.084 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.085 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:145282 3.1c e100/49 3:3dcd75bf:::rbd_data.1118584371b4.0000000000000004:head v 100'647, mlcod=100'644)", 2024-09-14T23:58:19.085 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:54.171268+0000", 2024-09-14T23:58:19.085 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.840340841, 2024-09-14T23:58:19.085 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003527353, 2024-09-14T23:58:19.085 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:19.085 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:19.085 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:19.085 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:19.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.085 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:19.085 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:54.171268+0000", 2024-09-14T23:58:19.085 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:19.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.086 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.086 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:19.086 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:54.171268+0000", 2024-09-14T23:58:19.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:19.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.086 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.086 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:19.086 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:54.171268+0000", 2024-09-14T23:58:19.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4900000000000005e-07 2024-09-14T23:58:19.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.086 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.086 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:19.086 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:54.172187+0000", 2024-09-14T23:58:19.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00091850799999999995 2024-09-14T23:58:19.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.087 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:19.087 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:54.172199+0000", 2024-09-14T23:58:19.087 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2544000000000001e-05 2024-09-14T23:58:19.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.087 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:19.087 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:54.172205+0000", 2024-09-14T23:58:19.087 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1490000000000002e-06 2024-09-14T23:58:19.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.087 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:19.088 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:54.172267+0000", 2024-09-14T23:58:19.088 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2617e-05 2024-09-14T23:58:19.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.088 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.088 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:19.088 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:54.172283+0000", 2024-09-14T23:58:19.088 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5758000000000001e-05 2024-09-14T23:58:19.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.088 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.088 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:19.088 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:54.174754+0000", 2024-09-14T23:58:19.088 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0024711400000000001 2024-09-14T23:58:19.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.088 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.089 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:19.089 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:54.174795+0000", 2024-09-14T23:58:19.089 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0887999999999999e-05 2024-09-14T23:58:19.089 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:19.089 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:19.089 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:19.089 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.089 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.089 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:156597 3.5 e103/49 3:a23c5924:::rbd_data.1118b2f495e6.0000000000000011:head v 103'518, mlcod=98'516)", 2024-09-14T23:58:19.089 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:58.656197+0000", 2024-09-14T23:58:19.089 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 20.355411871000001, 2024-09-14T23:58:19.089 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0040423029999999997, 2024-09-14T23:58:19.089 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:19.089 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:19.090 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:19.090 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:19.090 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.090 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:19.090 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656197+0000", 2024-09-14T23:58:19.090 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:19.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.090 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.090 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:19.090 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656197+0000", 2024-09-14T23:58:19.090 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:19.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.090 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:19.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656197+0000", 2024-09-14T23:58:19.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1900000000000002e-07 2024-09-14T23:58:19.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:19.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656676+0000", 2024-09-14T23:58:19.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00047921699999999999 2024-09-14T23:58:19.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:19.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656690+0000", 2024-09-14T23:58:19.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3916e-05 2024-09-14T23:58:19.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.092 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.092 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:19.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656698+0000", 2024-09-14T23:58:19.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6529999999999994e-06 2024-09-14T23:58:19.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.092 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.092 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:19.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656757+0000", 2024-09-14T23:58:19.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9237999999999998e-05 2024-09-14T23:58:19.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.092 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.093 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:19.093 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656780+0000", 2024-09-14T23:58:19.093 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2968000000000001e-05 2024-09-14T23:58:19.093 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.093 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.093 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:19.093 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.660188+0000", 2024-09-14T23:58:19.093 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0034078630000000001 2024-09-14T23:58:19.093 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.093 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.093 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:19.093 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.660239+0000", 2024-09-14T23:58:19.093 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0828999999999997e-05 2024-09-14T23:58:19.094 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:19.094 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:19.094 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:19.094 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.094 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.094 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:157947 3.1c e103/49 3:383374fe:::rbd_data.1118b2f495e6.000000000000000c:head v 103'771, mlcod=103'769)", 2024-09-14T23:58:19.094 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:58.765271+0000", 2024-09-14T23:58:19.094 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 20.24633699, 2024-09-14T23:58:19.094 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0040869829999999998, 2024-09-14T23:58:19.094 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:19.094 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:19.094 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:19.094 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:19.094 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:19.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765271+0000", 2024-09-14T23:58:19.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:19.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:19.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765271+0000", 2024-09-14T23:58:19.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:19.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:19.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765273+0000", 2024-09-14T23:58:19.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1149999999999999e-06 2024-09-14T23:58:19.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.096 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:19.096 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765854+0000", 2024-09-14T23:58:19.096 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00058119599999999997 2024-09-14T23:58:19.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.096 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:19.096 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765869+0000", 2024-09-14T23:58:19.096 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5641000000000002e-05 2024-09-14T23:58:19.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.096 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765878+0000", 2024-09-14T23:58:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4519999999999997e-06 2024-09-14T23:58:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765942+0000", 2024-09-14T23:58:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3727999999999997e-05 2024-09-14T23:58:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765968+0000", 2024-09-14T23:58:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6171999999999999e-05 2024-09-14T23:58:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:19.098 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.769319+0000", 2024-09-14T23:58:19.098 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033511629999999999 2024-09-14T23:58:19.098 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.098 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.098 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:19.098 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.769358+0000", 2024-09-14T23:58:19.098 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9515999999999999e-05 2024-09-14T23:58:19.098 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:19.098 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:19.098 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:19.098 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.098 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.098 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:160196 3.c e103/49 3:34623e68:::rbd_data.1118b2f495e6.0000000000000005:head v 103'551, mlcod=102'549)", 2024-09-14T23:58:19.098 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:59.131433+0000", 2024-09-14T23:58:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.880175681000001, 2024-09-14T23:58:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0036932530000000001, 2024-09-14T23:58:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.131433+0000", 2024-09-14T23:58:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.131433+0000", 2024-09-14T23:58:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:19.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.100 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.100 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:19.100 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.131433+0000", 2024-09-14T23:58:19.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8500000000000001e-07 2024-09-14T23:58:19.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.100 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.100 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:19.100 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.132144+0000", 2024-09-14T23:58:19.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00071022800000000003 2024-09-14T23:58:19.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.100 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.100 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:19.100 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.132154+0000", 2024-09-14T23:58:19.101 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0301e-05 2024-09-14T23:58:19.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.101 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:19.101 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.132159+0000", 2024-09-14T23:58:19.101 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6280000000000004e-06 2024-09-14T23:58:19.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.101 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:19.101 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.132208+0000", 2024-09-14T23:58:19.101 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9595000000000002e-05 2024-09-14T23:58:19.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.102 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:19.102 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.132231+0000", 2024-09-14T23:58:19.102 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3010000000000002e-05 2024-09-14T23:58:19.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.102 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.102 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:19.102 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.135087+0000", 2024-09-14T23:58:19.102 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0028553239999999998 2024-09-14T23:58:19.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.102 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.102 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:19.102 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.135126+0000", 2024-09-14T23:58:19.102 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9481999999999999e-05 2024-09-14T23:58:19.102 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:19.103 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:19.103 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:19.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.103 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.103 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188871 3.d 3:b35cff4d:::rbd_header.1118c900944f:head [watch reconnect cookie 140670715433840 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-14T23:58:19.103 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.970653+0000", 2024-09-14T23:58:19.103 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 11.040955948000001, 2024-09-14T23:58:19.103 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0727808270000001, 2024-09-14T23:58:19.103 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:19.103 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:19.103 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:19.103 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:19.103 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:19.103 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:19.103 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188871 2024-09-14T23:58:19.104 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.104 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:19.104 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.104 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:19.104 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970653+0000", 2024-09-14T23:58:19.104 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:19.104 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.104 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.104 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:19.104 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970653+0000", 2024-09-14T23:58:19.104 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:19.104 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.104 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.104 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970655+0000", 2024-09-14T23:58:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.277e-06 2024-09-14T23:58:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970662+0000", 2024-09-14T23:58:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9619999999999996e-06 2024-09-14T23:58:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970682+0000", 2024-09-14T23:58:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0279999999999999e-05 2024-09-14T23:58:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.106 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:19.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970691+0000", 2024-09-14T23:58:19.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2790000000000004e-06 2024-09-14T23:58:19.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.106 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:19.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970762+0000", 2024-09-14T23:58:19.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1019999999999994e-05 2024-09-14T23:58:19.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.106 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-14T23:58:19.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970770+0000", 2024-09-14T23:58:19.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7999999999999999e-06 2024-09-14T23:58:19.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.107 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:19.107 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.973516+0000", 2024-09-14T23:58:19.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0027456469999999999 2024-09-14T23:58:19.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.107 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:19.107 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.973525+0000", 2024-09-14T23:58:19.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4890000000000001e-06 2024-09-14T23:58:19.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.107 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:19.107 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.041451+0000", 2024-09-14T23:58:19.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.067926012 2024-09-14T23:58:19.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:19.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.041673+0000", 2024-09-14T23:58:19.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000222048 2024-09-14T23:58:19.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:19.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:19.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:19.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.043433+0000", 2024-09-14T23:58:19.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017600129999999999 2024-09-14T23:58:19.108 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:19.109 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:19.109 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:19.109 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:19.109 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:19.109 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:19.110 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:58:19.110 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:58:19.111 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:58:19.111 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:58:19.111 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T23:58:19.112 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:58:19.112 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:58:19.113 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:58:19.113 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:58:19.113 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:19.108+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:19.116 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:58:19.116 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:58:19.133 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:58:19.133 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:58:19.150 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:58:19.150 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:58:19.167 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:58:19.167 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:58:19.211 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:19.210+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:19.259 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:19.259 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:19.271 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T23:58:19.279 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T23:58:19.322 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:19.321+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:19.414 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:19.412+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:19.431 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:19.431 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:19.440 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T23:58:19.514 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:19.513+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:19.547 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:19.547 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:19.555 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T23:58:19.615 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:19.614+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:19.663 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:19.663 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:19.672 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T23:58:19.716 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:58:19.715+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:58:19.779 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:19.779 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:19.787 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T23:58:19.817 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:19.815+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:19.918 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:19.916+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:19.935 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:19.935 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:19.943 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T23:58:20.018 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:20.017+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:20.063 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:20.064 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:20.073 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T23:58:20.157 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:20.156+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:20.220 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:20.219+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:20.223 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:20.223 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:20.232 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T23:58:20.269 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is set 2024-09-14T23:58:20.336 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:20.335+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:20.399 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:20.399 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:20.409 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T23:58:20.421 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:20.420+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:20.522 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:20.521+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:20.555 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:20.555 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:20.564 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T23:58:20.648 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:20.647+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:20.712 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:20.712 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:20.721 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T23:58:20.724 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:20.723+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:20.825 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:58:20.823+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:58:20.868 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:20.868 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:20.877 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T23:58:20.925 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:20.924+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:20.984 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:20.985 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:20.993 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T23:58:21.027 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:21.025+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:21.128 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:21.126+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:21.142 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:21.142 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:21.151 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T23:58:21.179 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T23:58:21.229 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:21.228+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:21.307 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:21.307 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:21.317 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T23:58:21.324 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:58:21.324 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:58:21.329 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:21.328+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:21.344 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:58:21.344 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:58:21.364 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:58:21.364 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:58:21.384 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:58:21.384 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:58:21.403 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:58:21.403 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:58:21.419 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:58:21.420 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:58:21.431 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:21.429+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:21.434 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:21.434 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:58:21.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.434 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:250925 3.8 3:170d55b1:::rbd_object_map.11187b9f1032:head [call lock.assert_locked in=44b,call rbd.object_map_snap_remove in=49b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-14T23:58:21.434 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:21.432490+0000", 2024-09-14T23:58:21.434 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.00026093299999999998, 2024-09-14T23:58:21.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00027962799999999997, 2024-09-14T23:58:21.434 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:21.434 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:21.434 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:21.434 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:21.434 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:21.434 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:21.435 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 250925 2024-09-14T23:58:21.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.435 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:21.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.435 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:21.435 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:21.432490+0000", 2024-09-14T23:58:21.435 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:21.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.435 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:21.435 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:21.432490+0000", 2024-09-14T23:58:21.435 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:21.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.435 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:21.435 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:21.432490+0000", 2024-09-14T23:58:21.436 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3900000000000004e-07 2024-09-14T23:58:21.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.436 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:21.436 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:21.432496+0000", 2024-09-14T23:58:21.436 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0660000000000003e-06 2024-09-14T23:58:21.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.436 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:21.436 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:21.432507+0000", 2024-09-14T23:58:21.436 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1866e-05 2024-09-14T23:58:21.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.436 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:21.436 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:21.432515+0000", 2024-09-14T23:58:21.437 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1810000000000004e-06 2024-09-14T23:58:21.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.437 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:21.437 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:21.432554+0000", 2024-09-14T23:58:21.437 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9378000000000001e-05 2024-09-14T23:58:21.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.437 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:21.437 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:21.432606+0000", 2024-09-14T23:58:21.437 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2404999999999997e-05 2024-09-14T23:58:21.437 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:21.437 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:21.437 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:21.437 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:21.437 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-14T23:58:21.438 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 1 2024-09-14T23:58:21.438 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:21.440 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:58:21.440 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:58:21.443 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T23:58:21.457 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:58:21.458 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:58:21.532 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:21.531+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:21.565 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:21.565 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:21.565 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:58:21.565 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:58:21.565 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:21.574 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T23:58:21.632 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:21.631+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:21.683 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:21.683 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:58:21.683 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:58:21.683 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:58:21.683 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.683 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5201 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2232320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:21.683 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:58:21.683 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 102.427069381, 2024-09-14T23:58:21.683 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057643228999999997, 2024-09-14T23:58:21.683 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:21.684 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:21.684 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:21.684 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:21.684 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:21.684 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:21.684 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5201 2024-09-14T23:58:21.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.684 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:21.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.684 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:21.684 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:58:21.684 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:21.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.684 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:21.685 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:58:21.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:21.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.685 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:21.685 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:58:21.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0699999999999998e-07 2024-09-14T23:58:21.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.685 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:21.685 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253422+0000", 2024-09-14T23:58:21.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.514e-06 2024-09-14T23:58:21.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.686 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:21.686 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253431+0000", 2024-09-14T23:58:21.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1919999999999994e-06 2024-09-14T23:58:21.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.686 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:21.686 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253436+0000", 2024-09-14T23:58:21.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5350000000000004e-06 2024-09-14T23:58:21.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.686 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:21.686 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310856+0000", 2024-09-14T23:58:21.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057419381999999998 2024-09-14T23:58:21.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.686 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:21.687 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310953+0000", 2024-09-14T23:58:21.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6774000000000006e-05 2024-09-14T23:58:21.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.687 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:21.687 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311062+0000", 2024-09-14T23:58:21.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000109425 2024-09-14T23:58:21.687 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:21.687 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:21.687 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:21.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.687 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5202 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2293760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:21.687 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:58:21.687 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 102.42704463299999, 2024-09-14T23:58:21.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057818255999999998, 2024-09-14T23:58:21.688 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:21.688 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:21.688 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:21.688 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:21.688 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:21.688 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:21.688 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5202 2024-09-14T23:58:21.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.688 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:21.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.688 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:21.688 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:58:21.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:21.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.689 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:21.689 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:58:21.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:21.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.689 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:21.689 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253573+0000", 2024-09-14T23:58:21.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012934000000000001 2024-09-14T23:58:21.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.689 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:21.689 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253576+0000", 2024-09-14T23:58:21.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5310000000000002e-06 2024-09-14T23:58:21.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.690 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:21.690 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253587+0000", 2024-09-14T23:58:21.690 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0522e-05 2024-09-14T23:58:21.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.690 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:21.690 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253595+0000", 2024-09-14T23:58:21.690 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8290000000000008e-06 2024-09-14T23:58:21.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.690 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:21.690 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311093+0000", 2024-09-14T23:58:21.690 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057498312000000003 2024-09-14T23:58:21.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.691 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:21.691 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311167+0000", 2024-09-14T23:58:21.691 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3582000000000001e-05 2024-09-14T23:58:21.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.691 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:21.691 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311262+0000", 2024-09-14T23:58:21.691 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5140000000000001e-05 2024-09-14T23:58:21.691 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:21.691 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:21.691 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:21.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.691 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5203 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2355200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:21.691 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:58:21.692 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 102.42688189099999, 2024-09-14T23:58:21.692 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057903938000000002, 2024-09-14T23:58:21.692 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:21.692 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:21.692 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:21.692 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:21.692 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:21.692 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:21.692 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5203 2024-09-14T23:58:21.692 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.692 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:21.692 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.692 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:21.692 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:58:21.693 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:21.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.693 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.693 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:21.693 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:58:21.693 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:21.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.693 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.693 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:21.693 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253902+0000", 2024-09-14T23:58:21.693 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029615400000000002 2024-09-14T23:58:21.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.693 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.693 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:21.693 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253906+0000", 2024-09-14T23:58:21.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9249999999999997e-06 2024-09-14T23:58:21.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:21.694 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253913+0000", 2024-09-14T23:58:21.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9099999999999999e-06 2024-09-14T23:58:21.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:21.694 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253917+0000", 2024-09-14T23:58:21.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9210000000000002e-06 2024-09-14T23:58:21.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:21.694 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311295+0000", 2024-09-14T23:58:21.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057378268000000003 2024-09-14T23:58:21.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.695 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:21.695 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311402+0000", 2024-09-14T23:58:21.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010644100000000001 2024-09-14T23:58:21.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.695 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:21.695 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311510+0000", 2024-09-14T23:58:21.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108319 2024-09-14T23:58:21.695 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:21.695 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:21.695 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:21.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.696 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5204 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2416640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:21.696 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:58:21.696 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 102.426563694, 2024-09-14T23:58:21.696 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057774860999999997, 2024-09-14T23:58:21.696 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:21.696 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:21.696 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:21.696 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:21.696 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:21.696 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:21.696 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5204 2024-09-14T23:58:21.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.696 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:21.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.697 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:21.697 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:58:21.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:21.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.697 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:21.697 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:58:21.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:21.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.697 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:21.697 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255903+0000", 2024-09-14T23:58:21.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001978807 2024-09-14T23:58:21.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.698 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:21.698 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255907+0000", 2024-09-14T23:58:21.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3610000000000001e-06 2024-09-14T23:58:21.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.698 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:21.698 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255914+0000", 2024-09-14T23:58:21.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.357e-06 2024-09-14T23:58:21.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.698 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:21.698 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255918+0000", 2024-09-14T23:58:21.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1779999999999996e-06 2024-09-14T23:58:21.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.699 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:21.699 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311541+0000", 2024-09-14T23:58:21.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055622697999999998 2024-09-14T23:58:21.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.699 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:21.699 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311614+0000", 2024-09-14T23:58:21.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3102999999999994e-05 2024-09-14T23:58:21.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.699 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:21.699 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311699+0000", 2024-09-14T23:58:21.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5357000000000003e-05 2024-09-14T23:58:21.699 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:21.699 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:21.699 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:21.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.700 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5211 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2846720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:21.700 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:58:21.700 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 102.42446547999999, 2024-09-14T23:58:21.700 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057528151, 2024-09-14T23:58:21.700 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:21.700 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:21.700 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:21.700 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:21.700 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:21.700 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:21.700 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5211 2024-09-14T23:58:21.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.701 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:21.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.701 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:21.701 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:58:21.701 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:21.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.701 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:21.701 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:58:21.701 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:21.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.701 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:21.701 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:58:21.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1399999999999998e-07 2024-09-14T23:58:21.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:21.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256024+0000", 2024-09-14T23:58:21.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.114e-06 2024-09-14T23:58:21.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:21.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256028+0000", 2024-09-14T23:58:21.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0840000000000002e-06 2024-09-14T23:58:21.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:21.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256031+0000", 2024-09-14T23:58:21.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6819999999999999e-06 2024-09-14T23:58:21.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.703 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:21.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313317+0000", 2024-09-14T23:58:21.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057286295000000001 2024-09-14T23:58:21.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.703 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:21.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313403+0000", 2024-09-14T23:58:21.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6287999999999998e-05 2024-09-14T23:58:21.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.703 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:21.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313551+0000", 2024-09-14T23:58:21.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000147374 2024-09-14T23:58:21.704 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:21.704 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:21.704 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:21.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.704 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5212 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2908160~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:21.704 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:58:21.704 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 102.424454286, 2024-09-14T23:58:21.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057735679999999998, 2024-09-14T23:58:21.704 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:21.704 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:21.704 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:21.704 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:21.705 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:21.705 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:21.705 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5212 2024-09-14T23:58:21.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.705 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:21.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.705 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:21.705 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:58:21.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:21.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.705 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:21.705 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:58:21.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:21.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:21.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:58:21.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0900000000000001e-07 2024-09-14T23:58:21.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:21.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256035+0000", 2024-09-14T23:58:21.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1200000000000001e-06 2024-09-14T23:58:21.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:21.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256038+0000", 2024-09-14T23:58:21.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.05e-06 2024-09-14T23:58:21.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.707 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:21.707 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256041+0000", 2024-09-14T23:58:21.707 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.667e-06 2024-09-14T23:58:21.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.707 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:21.707 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313584+0000", 2024-09-14T23:58:21.707 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057543484999999998 2024-09-14T23:58:21.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.707 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:21.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313668+0000", 2024-09-14T23:58:21.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3807000000000006e-05 2024-09-14T23:58:21.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:21.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313770+0000", 2024-09-14T23:58:21.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010134200000000001 2024-09-14T23:58:21.708 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:21.708 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:21.708 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:21.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.708 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5213 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2969600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:21.708 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:58:21.709 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 102.42444419900001, 2024-09-14T23:58:21.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058145467999999999, 2024-09-14T23:58:21.709 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:21.709 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:21.709 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:21.709 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:21.709 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:21.709 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:21.709 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5213 2024-09-14T23:58:21.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.709 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:21.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:21.710 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:58:21.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:21.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.710 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:21.710 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:58:21.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:21.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.710 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:21.710 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257901+0000", 2024-09-14T23:58:21.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018567519999999999 2024-09-14T23:58:21.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:21.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257904+0000", 2024-09-14T23:58:21.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4850000000000001e-06 2024-09-14T23:58:21.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:21.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257912+0000", 2024-09-14T23:58:21.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2090000000000005e-06 2024-09-14T23:58:21.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:21.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257918+0000", 2024-09-14T23:58:21.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6899999999999997e-06 2024-09-14T23:58:21.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:21.712 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313816+0000", 2024-09-14T23:58:21.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055898075999999998 2024-09-14T23:58:21.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.712 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.712 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:21.712 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.314082+0000", 2024-09-14T23:58:21.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000265386 2024-09-14T23:58:21.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.712 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.712 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:21.712 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.314189+0000", 2024-09-14T23:58:21.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010787 2024-09-14T23:58:21.712 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:21.712 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:21.713 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:21.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.713 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5218 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3276800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:21.713 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:58:21.713 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 102.42251718999999, 2024-09-14T23:58:21.713 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057497557999999997, 2024-09-14T23:58:21.713 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:21.713 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:21.713 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:21.713 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:21.713 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:21.713 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:21.714 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5218 2024-09-14T23:58:21.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.714 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:21.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.714 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:21.714 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:58:21.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:21.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.714 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:21.714 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:58:21.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:21.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:21.715 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:58:21.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1899999999999999e-07 2024-09-14T23:58:21.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:21.715 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257972+0000", 2024-09-14T23:58:21.715 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.217e-06 2024-09-14T23:58:21.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.715 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:21.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257975+0000", 2024-09-14T23:58:21.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9359999999999999e-06 2024-09-14T23:58:21.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.716 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:21.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257978+0000", 2024-09-14T23:58:21.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1049999999999999e-06 2024-09-14T23:58:21.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.716 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:21.716 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315331+0000", 2024-09-14T23:58:21.716 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057352093999999999 2024-09-14T23:58:21.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.717 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:21.717 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315384+0000", 2024-09-14T23:58:21.717 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3780999999999999e-05 2024-09-14T23:58:21.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.717 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.717 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:21.717 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315468+0000", 2024-09-14T23:58:21.717 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4206e-05 2024-09-14T23:58:21.717 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:21.717 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:21.717 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:21.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.717 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.718 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5219 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3338240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:21.718 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:58:21.718 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 102.42250651400001, 2024-09-14T23:58:21.718 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057637764000000001, 2024-09-14T23:58:21.718 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:21.718 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:21.718 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:21.718 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:21.718 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:21.718 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:21.718 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5219 2024-09-14T23:58:21.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.718 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:21.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:21.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:58:21.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:21.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:21.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:58:21.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:21.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.719 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:21.719 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:58:21.719 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.03e-07 2024-09-14T23:58:21.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.719 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.720 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:21.720 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257983+0000", 2024-09-14T23:58:21.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9199999999999999e-07 2024-09-14T23:58:21.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.720 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:21.720 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257986+0000", 2024-09-14T23:58:21.720 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0860000000000002e-06 2024-09-14T23:58:21.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.720 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:21.720 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257989+0000", 2024-09-14T23:58:21.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6589999999999999e-06 2024-09-14T23:58:21.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.721 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:21.721 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315509+0000", 2024-09-14T23:58:21.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057520262000000003 2024-09-14T23:58:21.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.721 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:21.721 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315554+0000", 2024-09-14T23:58:21.721 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5015999999999997e-05 2024-09-14T23:58:21.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.721 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.721 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:21.721 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315619+0000", 2024-09-14T23:58:21.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5545999999999999e-05 2024-09-14T23:58:21.722 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:21.722 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:21.722 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:21.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.722 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.722 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5220 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3399680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:21.722 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:58:21.722 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 102.42249655800001, 2024-09-14T23:58:21.722 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057792128999999998, 2024-09-14T23:58:21.722 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:21.722 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:21.722 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:21.722 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:21.722 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:21.723 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:21.723 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5220 2024-09-14T23:58:21.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.723 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:21.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.723 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:21.723 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:58:21.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:21.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.723 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:21.723 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:58:21.723 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:21.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.724 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:21.724 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:58:21.724 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1799999999999999e-07 2024-09-14T23:58:21.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.724 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.724 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:21.724 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257993+0000", 2024-09-14T23:58:21.724 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6200000000000006e-07 2024-09-14T23:58:21.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.724 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.724 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:21.724 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257996+0000", 2024-09-14T23:58:21.724 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9050000000000001e-06 2024-09-14T23:58:21.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.724 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.725 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:21.725 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257998+0000", 2024-09-14T23:58:21.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5950000000000001e-06 2024-09-14T23:58:21.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.725 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:21.725 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315672+0000", 2024-09-14T23:58:21.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057673637999999999 2024-09-14T23:58:21.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.725 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:21.725 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315734+0000", 2024-09-14T23:58:21.725 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2216000000000002e-05 2024-09-14T23:58:21.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.725 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:21.726 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315784+0000", 2024-09-14T23:58:21.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9595000000000002e-05 2024-09-14T23:58:21.726 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:21.726 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:21.726 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:21.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.726 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5221 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3461120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:21.726 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:58:21.726 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 102.42248684, 2024-09-14T23:58:21.726 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058003557999999997, 2024-09-14T23:58:21.726 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:21.726 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:21.726 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:21.727 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:21.727 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:21.727 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:21.727 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5221 2024-09-14T23:58:21.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.727 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:21.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:21.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:58:21.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:21.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.727 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.727 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:21.727 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:58:21.727 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:21.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.728 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:21.728 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259905+0000", 2024-09-14T23:58:21.728 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019035759999999999 2024-09-14T23:58:21.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.728 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:21.728 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259915+0000", 2024-09-14T23:58:21.728 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0518e-05 2024-09-14T23:58:21.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.728 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.728 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:21.729 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259925+0000", 2024-09-14T23:58:21.729 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7180000000000008e-06 2024-09-14T23:58:21.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.729 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.729 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:21.729 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259930+0000", 2024-09-14T23:58:21.729 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4769999999999997e-06 2024-09-14T23:58:21.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.729 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.729 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:21.729 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315820+0000", 2024-09-14T23:58:21.729 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055889960000000002 2024-09-14T23:58:21.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.729 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.730 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:21.730 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315914+0000", 2024-09-14T23:58:21.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4625999999999998e-05 2024-09-14T23:58:21.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.730 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:21.730 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.316005+0000", 2024-09-14T23:58:21.730 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0682999999999999e-05 2024-09-14T23:58:21.730 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:21.730 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:21.730 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:21.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.730 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.730 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5231 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 4075520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:21.731 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:58:21.731 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 102.420418106, 2024-09-14T23:58:21.731 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057524733000000002, 2024-09-14T23:58:21.731 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:21.731 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:21.731 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:21.731 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:21.731 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:21.731 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:21.731 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5231 2024-09-14T23:58:21.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.731 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:21.731 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.731 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:21.731 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:58:21.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:21.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.732 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:21.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:58:21.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:21.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.732 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:21.732 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261904+0000", 2024-09-14T23:58:21.732 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018338790000000001 2024-09-14T23:58:21.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.733 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:21.733 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261910+0000", 2024-09-14T23:58:21.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9689999999999999e-06 2024-09-14T23:58:21.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.733 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:21.733 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261923+0000", 2024-09-14T23:58:21.733 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3061000000000001e-05 2024-09-14T23:58:21.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.733 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:21.733 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261931+0000", 2024-09-14T23:58:21.734 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3159999999999997e-06 2024-09-14T23:58:21.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.734 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.734 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:21.734 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317456+0000", 2024-09-14T23:58:21.734 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055525219000000001 2024-09-14T23:58:21.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.734 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.734 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:21.734 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317520+0000", 2024-09-14T23:58:21.734 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4010000000000005e-05 2024-09-14T23:58:21.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.734 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.734 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:21.734 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317595+0000", 2024-09-14T23:58:21.735 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4278999999999998e-05 2024-09-14T23:58:21.735 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:21.735 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:21.735 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:21.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.735 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5236 3.8 3:1071668d:::rbd_data.1118de1168b.0000000000000092:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:21.735 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:58:21.735 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 102.418404642, 2024-09-14T23:58:21.735 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057499462000000001, 2024-09-14T23:58:21.735 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:21.735 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:21.735 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:21.735 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:21.735 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:21.736 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:21.736 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5236 2024-09-14T23:58:21.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.736 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:21.736 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.736 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:21.736 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:58:21.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:21.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.736 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.736 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:21.736 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:58:21.736 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:21.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.737 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:21.737 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262084+0000", 2024-09-14T23:58:21.737 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3399999999999999e-07 2024-09-14T23:58:21.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.737 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:21.737 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262087+0000", 2024-09-14T23:58:21.737 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2310000000000001e-06 2024-09-14T23:58:21.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.737 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:21.737 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262095+0000", 2024-09-14T23:58:21.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3799999999999996e-06 2024-09-14T23:58:21.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.738 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.738 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:21.738 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262101+0000", 2024-09-14T23:58:21.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0009999999999998e-06 2024-09-14T23:58:21.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.738 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.738 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:21.738 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317625+0000", 2024-09-14T23:58:21.738 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055524111000000001 2024-09-14T23:58:21.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.738 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.739 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:21.739 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317702+0000", 2024-09-14T23:58:21.739 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7533999999999993e-05 2024-09-14T23:58:21.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.739 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-14T23:58:21.739 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317892+0000", 2024-09-14T23:58:21.739 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018991400000000001 2024-09-14T23:58:21.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.739 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:58:21.739 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319407+0000", 2024-09-14T23:58:21.739 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015145250000000001 2024-09-14T23:58:21.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.740 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.740 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:21.740 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319481+0000", 2024-09-14T23:58:21.740 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3826999999999996e-05 2024-09-14T23:58:21.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.740 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.740 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:21.740 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319541+0000", 2024-09-14T23:58:21.740 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0470999999999998e-05 2024-09-14T23:58:21.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.740 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.740 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:21.740 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319583+0000", 2024-09-14T23:58:21.740 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1934000000000002e-05 2024-09-14T23:58:21.741 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:21.741 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:21.741 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:21.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.741 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:132622 3.1e 3.7a50b9e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-14T23:58:21.741 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:43.371285+0000", 2024-09-14T23:58:21.741 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 38.309202740000003, 2024-09-14T23:58:21.741 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.67037987200000004, 2024-09-14T23:58:21.741 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:21.741 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:21.741 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:21.741 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:21.741 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:21.742 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:21.742 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 132622 2024-09-14T23:58:21.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.742 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:21.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.742 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:21.742 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371285+0000", 2024-09-14T23:58:21.742 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:21.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.742 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:21.742 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371285+0000", 2024-09-14T23:58:21.742 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:21.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:21.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371286+0000", 2024-09-14T23:58:21.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6400000000000002e-07 2024-09-14T23:58:21.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:21.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371289+0000", 2024-09-14T23:58:21.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2059999999999999e-06 2024-09-14T23:58:21.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.743 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:21.743 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371299+0000", 2024-09-14T23:58:21.743 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0288e-05 2024-09-14T23:58:21.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.744 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:21.744 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371306+0000", 2024-09-14T23:58:21.744 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1650000000000003e-06 2024-09-14T23:58:21.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.744 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:21.744 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371617+0000", 2024-09-14T23:58:21.744 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000311043 2024-09-14T23:58:21.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.744 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:58:21.744 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371625+0000", 2024-09-14T23:58:21.745 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.833e-06 2024-09-14T23:58:21.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.745 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.745 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:21.745 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.041651+0000", 2024-09-14T23:58:21.745 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.67002580199999995 2024-09-14T23:58:21.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.745 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.745 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:21.745 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.041665+0000", 2024-09-14T23:58:21.745 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3971000000000001e-05 2024-09-14T23:58:21.745 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:21.745 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:21.746 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:21.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.746 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.746 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:169974 3.5 3:a704a2f0:::rbd_data.1118427720d0.0000000000000030:32 [read 965632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-14T23:58:21.746 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:03.433820+0000", 2024-09-14T23:58:21.746 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.246668082999999, 2024-09-14T23:58:21.746 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057495521000000001, 2024-09-14T23:58:21.746 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:21.746 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:21.746 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:21.746 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:21.746 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:21.746 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:21.746 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 169974 2024-09-14T23:58:21.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.746 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:21.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.747 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:21.747 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.433820+0000", 2024-09-14T23:58:21.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:21.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.747 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:21.747 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.433820+0000", 2024-09-14T23:58:21.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:21.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.747 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.747 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:21.747 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.433925+0000", 2024-09-14T23:58:21.747 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000104621 2024-09-14T23:58:21.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.748 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:21.748 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.433929+0000", 2024-09-14T23:58:21.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2169999999999996e-06 2024-09-14T23:58:21.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.748 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:21.748 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.433939+0000", 2024-09-14T23:58:21.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0002e-05 2024-09-14T23:58:21.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.748 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:21.748 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.433946+0000", 2024-09-14T23:58:21.748 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7889999999999997e-06 2024-09-14T23:58:21.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.749 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.749 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:21.749 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.491144+0000", 2024-09-14T23:58:21.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057198221 2024-09-14T23:58:21.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.749 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.749 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:21.749 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.491221+0000", 2024-09-14T23:58:21.749 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6767000000000005e-05 2024-09-14T23:58:21.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.749 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.749 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:21.749 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.491316+0000", 2024-09-14T23:58:21.750 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4903999999999998e-05 2024-09-14T23:58:21.750 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:21.750 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:21.750 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:21.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.750 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.750 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:169975 3.5 3:a704a2f0:::rbd_data.1118427720d0.0000000000000030:32 [read 969728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-14T23:58:21.750 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:03.433958+0000", 2024-09-14T23:58:21.750 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.246530384, 2024-09-14T23:58:21.750 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057564811, 2024-09-14T23:58:21.750 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:21.750 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:21.750 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:21.751 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:21.751 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:21.751 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:21.751 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 169975 2024-09-14T23:58:21.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.751 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:21.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.751 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:21.751 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.433958+0000", 2024-09-14T23:58:21.751 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:21.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.751 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.751 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:21.751 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.433958+0000", 2024-09-14T23:58:21.752 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:21.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.752 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.752 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:21.752 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.435096+0000", 2024-09-14T23:58:21.752 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001138708 2024-09-14T23:58:21.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.752 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.752 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:21.752 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.435101+0000", 2024-09-14T23:58:21.752 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5050000000000001e-06 2024-09-14T23:58:21.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.752 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.752 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:21.752 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.435111+0000", 2024-09-14T23:58:21.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5810000000000003e-06 2024-09-14T23:58:21.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.753 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:21.753 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.435120+0000", 2024-09-14T23:58:21.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0089999999999996e-06 2024-09-14T23:58:21.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.753 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:21.753 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.491345+0000", 2024-09-14T23:58:21.753 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056225479000000002 2024-09-14T23:58:21.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.753 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.753 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:21.754 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.491423+0000", 2024-09-14T23:58:21.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7990000000000004e-05 2024-09-14T23:58:21.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.754 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:21.754 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.491523+0000", 2024-09-14T23:58:21.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9538999999999997e-05 2024-09-14T23:58:21.754 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:21.754 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:21.754 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:21.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.754 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:172639 3.5 3:a704a2f0:::rbd_data.1118427720d0.0000000000000030:32 [read 1645568~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e107)", 2024-09-14T23:58:21.754 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:03.988365+0000", 2024-09-14T23:58:21.754 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 17.692122663999999, 2024-09-14T23:58:21.754 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057556213000000002, 2024-09-14T23:58:21.755 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:21.755 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:21.755 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:21.755 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:21.755 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:21.755 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:21.755 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 172639 2024-09-14T23:58:21.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.755 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:21.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.755 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:21.755 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.988365+0000", 2024-09-14T23:58:21.755 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:21.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.756 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:21.756 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.988365+0000", 2024-09-14T23:58:21.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:21.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.756 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:21.756 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.989904+0000", 2024-09-14T23:58:21.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015380859999999999 2024-09-14T23:58:21.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.756 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:21.756 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.989909+0000", 2024-09-14T23:58:21.756 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2800000000000003e-06 2024-09-14T23:58:21.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.757 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:21.757 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.989925+0000", 2024-09-14T23:58:21.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5781999999999999e-05 2024-09-14T23:58:21.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.757 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:21.757 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.989933+0000", 2024-09-14T23:58:21.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0090000000000007e-06 2024-09-14T23:58:21.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.757 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:21.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:04.045741+0000", 2024-09-14T23:58:21.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055808112999999999 2024-09-14T23:58:21.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.758 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:21.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:04.045812+0000", 2024-09-14T23:58:21.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1531000000000003e-05 2024-09-14T23:58:21.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.758 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:21.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:04.045922+0000", 2024-09-14T23:58:21.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000109412 2024-09-14T23:58:21.758 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:21.758 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:21.759 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:21.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.759 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184571 3.2 3:46f8d78b:::rbd_data.1118427720d0.0000000000000001:32 [read 1405952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:21.759 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.086404+0000", 2024-09-14T23:58:21.759 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.594084306999999, 2024-09-14T23:58:21.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059148573000000003, 2024-09-14T23:58:21.759 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:21.759 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:21.759 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:21.759 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:21.759 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:21.759 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:21.759 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184571 2024-09-14T23:58:21.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.760 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:21.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.760 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:21.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.086404+0000", 2024-09-14T23:58:21.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:21.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.760 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:21.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.086404+0000", 2024-09-14T23:58:21.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:21.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.760 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:21.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087902+0000", 2024-09-14T23:58:21.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001498625 2024-09-14T23:58:21.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.761 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:21.761 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087907+0000", 2024-09-14T23:58:21.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9130000000000002e-06 2024-09-14T23:58:21.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.761 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:21.761 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087919+0000", 2024-09-14T23:58:21.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1719000000000001e-05 2024-09-14T23:58:21.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.761 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:21.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087927+0000", 2024-09-14T23:58:21.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3699999999999995e-06 2024-09-14T23:58:21.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:21.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143711+0000", 2024-09-14T23:58:21.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055783396999999998 2024-09-14T23:58:21.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:21.762 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143792+0000", 2024-09-14T23:58:21.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1292999999999996e-05 2024-09-14T23:58:21.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:21.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145552+0000", 2024-09-14T23:58:21.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001760256 2024-09-14T23:58:21.763 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:21.763 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:21.763 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:21.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.763 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184579 3.2 3:43badc91:::rbd_data.1118427720d0.0000000000000002:32 [read 1196032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:21.763 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.088145+0000", 2024-09-14T23:58:21.763 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.592342903, 2024-09-14T23:58:21.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057643169000000001, 2024-09-14T23:58:21.763 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:21.763 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:21.763 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:21.764 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:21.764 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:21.764 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:21.764 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184579 2024-09-14T23:58:21.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.764 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:21.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.764 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:21.764 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.088145+0000", 2024-09-14T23:58:21.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:21.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.764 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:21.764 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.088145+0000", 2024-09-14T23:58:21.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:21.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.765 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:21.765 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089947+0000", 2024-09-14T23:58:21.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001801764 2024-09-14T23:58:21.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.765 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:21.765 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089952+0000", 2024-09-14T23:58:21.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8659999999999997e-06 2024-09-14T23:58:21.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.765 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:21.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089966+0000", 2024-09-14T23:58:21.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3929e-05 2024-09-14T23:58:21.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:21.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089974+0000", 2024-09-14T23:58:21.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1429999999999998e-06 2024-09-14T23:58:21.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:21.766 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145636+0000", 2024-09-14T23:58:21.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055662287999999997 2024-09-14T23:58:21.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.766 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:21.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145710+0000", 2024-09-14T23:58:21.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4166999999999996e-05 2024-09-14T23:58:21.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:21.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145788+0000", 2024-09-14T23:58:21.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8011999999999998e-05 2024-09-14T23:58:21.767 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:21.767 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:21.767 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:21.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.767 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:211800 3.12 3:493125e8:::rbd_data.1118c900944f.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-14T23:58:21.767 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:13.660130+0000", 2024-09-14T23:58:21.767 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.0203585769999997, 2024-09-14T23:58:21.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.55364332000000005, 2024-09-14T23:58:21.768 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:21.768 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:21.768 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:21.768 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:21.768 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:21.768 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:21.768 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 211800 2024-09-14T23:58:21.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.768 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:21.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:21.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.660130+0000", 2024-09-14T23:58:21.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:21.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.769 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:21.769 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.660130+0000", 2024-09-14T23:58:21.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:21.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.769 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:21.769 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.660133+0000", 2024-09-14T23:58:21.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.202e-06 2024-09-14T23:58:21.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.769 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:21.769 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.668135+0000", 2024-09-14T23:58:21.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0080027220000000003 2024-09-14T23:58:21.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.770 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:21.770 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.668172+0000", 2024-09-14T23:58:21.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6295999999999999e-05 2024-09-14T23:58:21.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.770 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:21.770 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.668187+0000", 2024-09-14T23:58:21.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5537e-05 2024-09-14T23:58:21.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.770 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:21.770 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.668284+0000", 2024-09-14T23:58:21.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6297000000000003e-05 2024-09-14T23:58:21.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:58:21.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.668299+0000", 2024-09-14T23:58:21.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5091e-05 2024-09-14T23:58:21.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:21.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:14.195910+0000", 2024-09-14T23:58:21.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.52761157999999997 2024-09-14T23:58:21.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:21.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:14.196106+0000", 2024-09-14T23:58:21.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019597399999999999 2024-09-14T23:58:21.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 0", 2024-09-14T23:58:21.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:14.197262+0000", 2024-09-14T23:58:21.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001155744 2024-09-14T23:58:21.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:58:21.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:14.207440+0000", 2024-09-14T23:58:21.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010178347000000001 2024-09-14T23:58:21.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:21.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:14.213691+0000", 2024-09-14T23:58:21.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0062504420000000002 2024-09-14T23:58:21.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:21.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:14.213735+0000", 2024-09-14T23:58:21.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4385999999999998e-05 2024-09-14T23:58:21.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:21.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:21.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:21.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:14.213773+0000", 2024-09-14T23:58:21.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7701999999999999e-05 2024-09-14T23:58:21.773 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:21.774 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:21.774 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:21.774 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:21.774 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:21.774 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:21.775 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T23:58:21.776 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:21.774+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:21.876 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:21.875+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:21.890 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:21.890 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:58:21.890 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:58:21.890 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:21.899 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T23:58:21.983 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:21.982+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:22.047 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:22.047 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:58:22.047 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:58:22.047 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:58:22.047 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:22.056 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T23:58:22.078 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:22.077+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:22.179 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:22.177+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:22.205 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:22.205 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:58:22.205 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:58:22.205 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:58:22.206 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.206 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186203 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1329152~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:22.206 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.194279+0000", 2024-09-14T23:58:22.206 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 15.008552688, 2024-09-14T23:58:22.206 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8011469710000001, 2024-09-14T23:58:22.206 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:22.206 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:22.206 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:22.206 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:22.206 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:22.206 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:22.206 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186203 2024-09-14T23:58:22.206 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.207 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:22.207 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.207 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:22.207 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194279+0000", 2024-09-14T23:58:22.207 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.207 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.207 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:22.207 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194279+0000", 2024-09-14T23:58:22.207 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.207 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.207 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:22.207 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194279+0000", 2024-09-14T23:58:22.208 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8100000000000001e-07 2024-09-14T23:58:22.208 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.208 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.208 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:22.208 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194291+0000", 2024-09-14T23:58:22.208 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1559999999999999e-05 2024-09-14T23:58:22.208 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.208 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.208 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:22.208 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194303+0000", 2024-09-14T23:58:22.208 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2130000000000001e-05 2024-09-14T23:58:22.208 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.208 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.209 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:22.209 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194311+0000", 2024-09-14T23:58:22.209 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8089999999999993e-06 2024-09-14T23:58:22.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.209 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.209 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.209 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194370+0000", 2024-09-14T23:58:22.209 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8739999999999997e-05 2024-09-14T23:58:22.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.209 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.209 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:22.209 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194376+0000", 2024-09-14T23:58:22.209 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2319999999999998e-06 2024-09-14T23:58:22.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.210 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.210 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986290+0000", 2024-09-14T23:58:22.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.791914334 2024-09-14T23:58:22.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.210 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:22.210 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986296+0000", 2024-09-14T23:58:22.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6300000000000003e-06 2024-09-14T23:58:22.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.210 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.210 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995193+0000", 2024-09-14T23:58:22.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0088975040000001 2024-09-14T23:58:22.211 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.211 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.211 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:22.211 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995307+0000", 2024-09-14T23:58:22.211 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000113656 2024-09-14T23:58:22.211 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.211 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.211 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:22.211 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995426+0000", 2024-09-14T23:58:22.211 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118495 2024-09-14T23:58:22.211 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.211 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:22.211 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.212 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.212 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.212 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186204 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1391616~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:22.212 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.194328+0000", 2024-09-14T23:58:22.212 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 15.008503357, 2024-09-14T23:58:22.212 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8013834310000001, 2024-09-14T23:58:22.212 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:22.212 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:22.212 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:22.212 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:22.212 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:22.213 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:22.213 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186204 2024-09-14T23:58:22.213 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.213 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:22.213 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.213 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:22.213 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194328+0000", 2024-09-14T23:58:22.213 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.213 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.213 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.213 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:22.213 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194328+0000", 2024-09-14T23:58:22.213 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.213 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.214 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.214 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:22.214 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194328+0000", 2024-09-14T23:58:22.214 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7100000000000002e-07 2024-09-14T23:58:22.214 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.214 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.214 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:22.214 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194331+0000", 2024-09-14T23:58:22.214 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.807e-06 2024-09-14T23:58:22.214 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.214 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.214 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:22.214 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194339+0000", 2024-09-14T23:58:22.214 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9340000000000006e-06 2024-09-14T23:58:22.214 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.214 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.215 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:22.215 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194345+0000", 2024-09-14T23:58:22.215 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1090000000000003e-06 2024-09-14T23:58:22.215 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.215 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.215 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.215 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194412+0000", 2024-09-14T23:58:22.215 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6229000000000006e-05 2024-09-14T23:58:22.215 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.215 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.215 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:22.215 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194422+0000", 2024-09-14T23:58:22.215 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.078e-05 2024-09-14T23:58:22.215 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.216 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.216 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.216 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986322+0000", 2024-09-14T23:58:22.216 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.79189964899999998 2024-09-14T23:58:22.216 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.216 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.216 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:22.216 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986332+0000", 2024-09-14T23:58:22.216 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0095000000000001e-05 2024-09-14T23:58:22.216 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.216 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.216 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.216 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995554+0000", 2024-09-14T23:58:22.216 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0092218630000001 2024-09-14T23:58:22.217 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.217 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.217 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:22.217 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995610+0000", 2024-09-14T23:58:22.217 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6109000000000001e-05 2024-09-14T23:58:22.217 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.217 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.217 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:22.217 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995711+0000", 2024-09-14T23:58:22.217 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101285 2024-09-14T23:58:22.217 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.217 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:22.217 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.218 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.218 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.218 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186205 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1333248~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:22.218 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.194354+0000", 2024-09-14T23:58:22.218 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 15.008477427000001, 2024-09-14T23:58:22.218 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8015471490000001, 2024-09-14T23:58:22.218 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:22.218 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:22.218 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:22.218 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:22.218 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:22.218 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:22.218 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186205 2024-09-14T23:58:22.218 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.219 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:22.219 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.219 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:22.219 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194354+0000", 2024-09-14T23:58:22.219 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.219 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.219 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.219 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:22.219 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194354+0000", 2024-09-14T23:58:22.219 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.219 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.219 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.219 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:22.220 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194354+0000", 2024-09-14T23:58:22.220 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.39e-07 2024-09-14T23:58:22.220 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.220 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.220 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:22.220 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194357+0000", 2024-09-14T23:58:22.220 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3479999999999998e-06 2024-09-14T23:58:22.220 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.220 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.220 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:22.220 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194364+0000", 2024-09-14T23:58:22.220 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0899999999999999e-06 2024-09-14T23:58:22.220 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.221 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.221 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:22.221 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194371+0000", 2024-09-14T23:58:22.221 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5530000000000004e-06 2024-09-14T23:58:22.221 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.221 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.221 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.221 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194469+0000", 2024-09-14T23:58:22.221 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8119e-05 2024-09-14T23:58:22.221 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.221 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.221 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:22.221 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194475+0000", 2024-09-14T23:58:22.221 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7910000000000001e-06 2024-09-14T23:58:22.222 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.222 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.222 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.222 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986367+0000", 2024-09-14T23:58:22.222 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.79189214699999999 2024-09-14T23:58:22.222 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.222 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.222 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:22.222 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986375+0000", 2024-09-14T23:58:22.222 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7800000000000001e-06 2024-09-14T23:58:22.222 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.222 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.222 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.223 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995746+0000", 2024-09-14T23:58:22.223 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0093706469999999 2024-09-14T23:58:22.223 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.223 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.223 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:22.223 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995810+0000", 2024-09-14T23:58:22.223 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4068999999999999e-05 2024-09-14T23:58:22.223 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.223 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.223 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:22.223 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995901+0000", 2024-09-14T23:58:22.223 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1166000000000001e-05 2024-09-14T23:58:22.223 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.223 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:22.223 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.224 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.224 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.224 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186206 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1395712~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:22.224 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.194380+0000", 2024-09-14T23:58:22.224 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 15.008451406000001, 2024-09-14T23:58:22.224 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.801678347, 2024-09-14T23:58:22.224 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:22.224 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:22.224 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:22.224 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:22.224 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:22.224 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:22.224 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186206 2024-09-14T23:58:22.225 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.225 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:22.225 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.225 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:22.225 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194380+0000", 2024-09-14T23:58:22.225 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.225 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.225 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.225 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:22.225 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194380+0000", 2024-09-14T23:58:22.225 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.225 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.225 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.225 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:22.226 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194380+0000", 2024-09-14T23:58:22.226 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4400000000000001e-07 2024-09-14T23:58:22.226 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.226 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.226 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:22.226 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194383+0000", 2024-09-14T23:58:22.226 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3460000000000001e-06 2024-09-14T23:58:22.226 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.226 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.226 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:22.226 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194390+0000", 2024-09-14T23:58:22.226 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2189999999999998e-06 2024-09-14T23:58:22.226 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.226 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.226 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:22.227 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194403+0000", 2024-09-14T23:58:22.227 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2941e-05 2024-09-14T23:58:22.227 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.227 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.227 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.227 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194545+0000", 2024-09-14T23:58:22.227 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014232700000000001 2024-09-14T23:58:22.227 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.227 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.227 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:22.227 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194565+0000", 2024-09-14T23:58:22.227 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9409999999999999e-05 2024-09-14T23:58:22.227 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.228 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.228 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.228 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986409+0000", 2024-09-14T23:58:22.228 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.79184416000000002 2024-09-14T23:58:22.228 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.228 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.228 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:22.228 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986417+0000", 2024-09-14T23:58:22.228 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.551e-06 2024-09-14T23:58:22.228 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.228 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.228 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.228 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995921+0000", 2024-09-14T23:58:22.228 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.009504212 2024-09-14T23:58:22.229 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.229 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.229 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:22.229 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995974+0000", 2024-09-14T23:58:22.229 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2617999999999997e-05 2024-09-14T23:58:22.229 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.229 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.229 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:22.229 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996058+0000", 2024-09-14T23:58:22.229 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4118999999999999e-05 2024-09-14T23:58:22.229 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.229 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:22.229 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.230 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.230 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.230 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186207 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1337344~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:22.230 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.194412+0000", 2024-09-14T23:58:22.230 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 15.008418808, 2024-09-14T23:58:22.230 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.801848141, 2024-09-14T23:58:22.230 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:22.230 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:22.230 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:22.230 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:22.230 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:22.230 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:22.230 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186207 2024-09-14T23:58:22.230 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.231 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:22.231 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.231 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:22.231 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194412+0000", 2024-09-14T23:58:22.231 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.231 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.231 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.231 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:22.231 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194412+0000", 2024-09-14T23:58:22.231 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.231 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.231 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.231 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:22.231 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194413+0000", 2024-09-14T23:58:22.232 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5600000000000001e-07 2024-09-14T23:58:22.232 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.232 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.232 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:22.232 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194418+0000", 2024-09-14T23:58:22.232 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3720000000000004e-06 2024-09-14T23:58:22.232 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.232 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.232 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:22.232 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194423+0000", 2024-09-14T23:58:22.232 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.763e-06 2024-09-14T23:58:22.232 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.232 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.233 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:22.233 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194426+0000", 2024-09-14T23:58:22.233 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1480000000000002e-06 2024-09-14T23:58:22.233 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.233 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.233 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.233 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194596+0000", 2024-09-14T23:58:22.233 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017009600000000001 2024-09-14T23:58:22.233 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.233 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.233 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:22.233 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194606+0000", 2024-09-14T23:58:22.233 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6639999999999993e-06 2024-09-14T23:58:22.233 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.234 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.234 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.234 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986460+0000", 2024-09-14T23:58:22.234 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.79185424400000004 2024-09-14T23:58:22.234 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.234 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.234 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:22.234 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986469+0000", 2024-09-14T23:58:22.234 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4419999999999998e-06 2024-09-14T23:58:22.234 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.234 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.234 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.234 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996123+0000", 2024-09-14T23:58:22.234 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0096546070000001 2024-09-14T23:58:22.235 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.235 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.235 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:22.235 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996179+0000", 2024-09-14T23:58:22.235 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5850000000000002e-05 2024-09-14T23:58:22.235 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.235 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.235 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:22.235 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996261+0000", 2024-09-14T23:58:22.235 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1599000000000003e-05 2024-09-14T23:58:22.235 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.235 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:22.235 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.236 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.236 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.236 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186208 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1399808~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:22.236 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.194445+0000", 2024-09-14T23:58:22.236 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 15.008386637999999, 2024-09-14T23:58:22.236 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8019822269999999, 2024-09-14T23:58:22.236 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:22.236 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:22.236 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:22.236 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:22.236 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:22.236 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:22.236 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186208 2024-09-14T23:58:22.237 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.237 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:22.237 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.237 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:22.237 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194445+0000", 2024-09-14T23:58:22.237 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.237 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.237 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.237 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:22.237 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194445+0000", 2024-09-14T23:58:22.237 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.237 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.237 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.238 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:22.238 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978674+0000", 2024-09-14T23:58:22.238 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.784229124 2024-09-14T23:58:22.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.238 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.238 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:22.238 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978680+0000", 2024-09-14T23:58:22.238 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7969999999999997e-06 2024-09-14T23:58:22.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.238 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.238 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:22.238 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978689+0000", 2024-09-14T23:58:22.238 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6949999999999995e-06 2024-09-14T23:58:22.239 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.239 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.239 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:22.239 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978696+0000", 2024-09-14T23:58:22.239 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1300000000000003e-06 2024-09-14T23:58:22.239 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.239 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.239 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.239 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986509+0000", 2024-09-14T23:58:22.239 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0078130969999999998 2024-09-14T23:58:22.239 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.239 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.239 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:22.240 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986518+0000", 2024-09-14T23:58:22.240 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2810000000000008e-06 2024-09-14T23:58:22.240 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.240 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.240 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.240 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996280+0000", 2024-09-14T23:58:22.240 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0097623120000001 2024-09-14T23:58:22.240 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.240 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.240 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:22.240 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996337+0000", 2024-09-14T23:58:22.240 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7244000000000003e-05 2024-09-14T23:58:22.240 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.241 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.241 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:22.241 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996427+0000", 2024-09-14T23:58:22.241 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9547000000000002e-05 2024-09-14T23:58:22.241 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.241 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:22.241 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.241 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.241 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.241 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4172.0:7 3.11 3:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:22.241 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.483386+0000", 2024-09-14T23:58:22.241 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 14.719445213, 2024-09-14T23:58:22.241 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.558667389, 2024-09-14T23:58:22.242 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:22.242 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:22.242 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:58:22.242 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:22.242 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4172", 2024-09-14T23:58:22.242 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/2542269203", 2024-09-14T23:58:22.242 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 7 2024-09-14T23:58:22.242 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.242 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:22.242 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.242 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:22.242 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.483386+0000", 2024-09-14T23:58:22.242 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.242 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.243 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.243 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:22.243 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.483386+0000", 2024-09-14T23:58:22.243 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.243 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.243 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.243 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:22.243 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978674+0000", 2024-09-14T23:58:22.243 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.49528781599999999 2024-09-14T23:58:22.243 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.243 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.243 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:22.243 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978687+0000", 2024-09-14T23:58:22.244 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3244999999999999e-05 2024-09-14T23:58:22.244 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.244 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.244 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:22.244 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978699+0000", 2024-09-14T23:58:22.244 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1832e-05 2024-09-14T23:58:22.244 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.244 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.244 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:22.244 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978707+0000", 2024-09-14T23:58:22.244 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4489999999999999e-06 2024-09-14T23:58:22.244 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.244 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.244 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.245 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978801+0000", 2024-09-14T23:58:22.245 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3511999999999995e-05 2024-09-14T23:58:22.245 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.245 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.245 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:22.245 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978814+0000", 2024-09-14T23:58:22.245 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2789000000000001e-05 2024-09-14T23:58:22.245 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.245 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.245 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.245 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.041948+0000", 2024-09-14T23:58:22.245 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0631338269999999 2024-09-14T23:58:22.245 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.246 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.246 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:22.246 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.042053+0000", 2024-09-14T23:58:22.246 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105919 2024-09-14T23:58:22.246 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.246 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:22.246 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.246 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.246 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.246 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187209 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1575936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:22.246 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.991888+0000", 2024-09-14T23:58:22.246 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 14.210943242000001, 2024-09-14T23:58:22.246 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0712115470000001, 2024-09-14T23:58:22.246 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:22.247 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:22.247 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:22.247 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:22.247 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:22.247 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:22.247 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187209 2024-09-14T23:58:22.247 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.247 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:22.247 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.247 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:22.247 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991888+0000", 2024-09-14T23:58:22.247 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.247 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.247 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.247 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:22.248 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991888+0000", 2024-09-14T23:58:22.248 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.248 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.248 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.248 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:22.248 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991888+0000", 2024-09-14T23:58:22.248 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1199999999999998e-07 2024-09-14T23:58:22.248 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.248 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.248 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:22.248 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991890+0000", 2024-09-14T23:58:22.248 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9810000000000002e-06 2024-09-14T23:58:22.248 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.249 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.249 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:22.249 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991897+0000", 2024-09-14T23:58:22.249 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2609999999999999e-06 2024-09-14T23:58:22.249 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.249 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.249 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:22.249 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991902+0000", 2024-09-14T23:58:22.249 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0000000000000004e-06 2024-09-14T23:58:22.249 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.249 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.249 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.249 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995686+0000", 2024-09-14T23:58:22.249 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0037839290000000001 2024-09-14T23:58:22.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.250 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.250 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:22.250 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995693+0000", 2024-09-14T23:58:22.250 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7589999999999997e-06 2024-09-14T23:58:22.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.250 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.250 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.250 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062846+0000", 2024-09-14T23:58:22.250 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.067152417 2024-09-14T23:58:22.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.250 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.250 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:22.251 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062935+0000", 2024-09-14T23:58:22.251 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9080999999999996e-05 2024-09-14T23:58:22.251 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.251 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.251 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:22.251 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.063100+0000", 2024-09-14T23:58:22.251 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016470699999999999 2024-09-14T23:58:22.251 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.251 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:22.251 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.251 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.251 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.251 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187210 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1591296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:22.252 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.991908+0000", 2024-09-14T23:58:22.252 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 14.210923329, 2024-09-14T23:58:22.252 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0713882530000001, 2024-09-14T23:58:22.252 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:22.252 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:22.252 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:22.252 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:22.252 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:22.252 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:22.252 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187210 2024-09-14T23:58:22.252 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.252 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:22.252 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.253 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:22.253 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991908+0000", 2024-09-14T23:58:22.253 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.253 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.253 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.253 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:22.253 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991908+0000", 2024-09-14T23:58:22.253 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.253 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.253 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.253 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:22.253 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993673+0000", 2024-09-14T23:58:22.253 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017651030000000001 2024-09-14T23:58:22.253 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.254 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.254 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:22.254 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993681+0000", 2024-09-14T23:58:22.254 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.858e-06 2024-09-14T23:58:22.254 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.254 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.254 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:22.254 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993691+0000", 2024-09-14T23:58:22.254 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9920000000000002e-06 2024-09-14T23:58:22.254 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.254 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.254 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:22.254 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993698+0000", 2024-09-14T23:58:22.255 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6490000000000002e-06 2024-09-14T23:58:22.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.255 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.255 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995731+0000", 2024-09-14T23:58:22.255 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0020338959999999999 2024-09-14T23:58:22.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.255 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:22.255 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995741+0000", 2024-09-14T23:58:22.255 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3079999999999997e-06 2024-09-14T23:58:22.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.255 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.256 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.063130+0000", 2024-09-14T23:58:22.256 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.067389159 2024-09-14T23:58:22.256 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.256 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.256 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:22.256 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.063206+0000", 2024-09-14T23:58:22.256 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6577999999999998e-05 2024-09-14T23:58:22.256 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.256 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.256 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:22.256 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.063296+0000", 2024-09-14T23:58:22.256 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9710000000000007e-05 2024-09-14T23:58:22.256 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.256 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:22.257 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.257 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.257 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187216 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1603584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:22.257 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.993831+0000", 2024-09-14T23:58:22.257 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 14.209000122999999, 2024-09-14T23:58:22.257 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.071244627, 2024-09-14T23:58:22.257 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:22.257 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:22.257 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:22.257 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:22.257 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:22.257 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:22.258 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187216 2024-09-14T23:58:22.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.258 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:22.258 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.258 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:22.258 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993831+0000", 2024-09-14T23:58:22.258 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.258 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.258 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:22.258 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993831+0000", 2024-09-14T23:58:22.258 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.258 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.259 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:22.259 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993832+0000", 2024-09-14T23:58:22.259 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9700000000000002e-07 2024-09-14T23:58:22.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.259 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.259 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:22.259 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993834+0000", 2024-09-14T23:58:22.259 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6170000000000001e-06 2024-09-14T23:58:22.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.259 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.259 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:22.259 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993841+0000", 2024-09-14T23:58:22.259 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6980000000000001e-06 2024-09-14T23:58:22.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.260 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.260 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:22.260 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993846+0000", 2024-09-14T23:58:22.260 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4190000000000001e-06 2024-09-14T23:58:22.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.260 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.260 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.260 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995988+0000", 2024-09-14T23:58:22.260 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0021417390000000001 2024-09-14T23:58:22.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.260 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.260 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:22.260 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995998+0000", 2024-09-14T23:58:22.260 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5100000000000004e-06 2024-09-14T23:58:22.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.261 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.261 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.261 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.064883+0000", 2024-09-14T23:58:22.261 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.06888597 2024-09-14T23:58:22.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.261 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.261 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:22.261 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.064977+0000", 2024-09-14T23:58:22.261 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3817e-05 2024-09-14T23:58:22.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.261 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.261 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:22.262 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065076+0000", 2024-09-14T23:58:22.262 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8460000000000003e-05 2024-09-14T23:58:22.262 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.262 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:22.262 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.262 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.262 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.262 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187217 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1592320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:22.262 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.993855+0000", 2024-09-14T23:58:22.262 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 14.20897669, 2024-09-14T23:58:22.262 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0714239349999999, 2024-09-14T23:58:22.262 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:22.262 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:22.263 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:22.263 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:22.263 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:22.263 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:22.263 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187217 2024-09-14T23:58:22.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.263 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:22.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.263 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:22.263 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993855+0000", 2024-09-14T23:58:22.263 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.264 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:22.264 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993855+0000", 2024-09-14T23:58:22.264 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.264 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.264 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:22.264 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993855+0000", 2024-09-14T23:58:22.264 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4899999999999995e-07 2024-09-14T23:58:22.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.264 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.264 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:22.264 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993857+0000", 2024-09-14T23:58:22.264 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3379999999999999e-06 2024-09-14T23:58:22.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.265 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:22.265 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993864+0000", 2024-09-14T23:58:22.265 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6000000000000003e-06 2024-09-14T23:58:22.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.265 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:22.265 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993869+0000", 2024-09-14T23:58:22.265 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3210000000000004e-06 2024-09-14T23:58:22.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.265 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.265 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996038+0000", 2024-09-14T23:58:22.265 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0021683050000000001 2024-09-14T23:58:22.266 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.266 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.266 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:22.266 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996048+0000", 2024-09-14T23:58:22.266 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0173999999999999e-05 2024-09-14T23:58:22.266 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.266 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.266 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.266 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065108+0000", 2024-09-14T23:58:22.266 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0690600619999999 2024-09-14T23:58:22.266 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.266 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.266 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:22.267 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065171+0000", 2024-09-14T23:58:22.267 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3423999999999994e-05 2024-09-14T23:58:22.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.267 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.267 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:22.267 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065279+0000", 2024-09-14T23:58:22.267 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000107162 2024-09-14T23:58:22.267 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.267 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:22.267 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.267 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.267 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187218 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1607680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:22.268 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.993877+0000", 2024-09-14T23:58:22.268 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 14.20895441, 2024-09-14T23:58:22.268 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0716976810000001, 2024-09-14T23:58:22.268 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:22.268 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:22.268 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:22.268 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:22.268 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:22.268 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:22.268 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187218 2024-09-14T23:58:22.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.268 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:22.268 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.268 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:22.269 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993877+0000", 2024-09-14T23:58:22.269 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.269 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.269 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:22.269 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993877+0000", 2024-09-14T23:58:22.269 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.269 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.269 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:22.269 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993877+0000", 2024-09-14T23:58:22.269 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8299999999999998e-07 2024-09-14T23:58:22.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.270 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.270 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:22.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993879+0000", 2024-09-14T23:58:22.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5659999999999999e-06 2024-09-14T23:58:22.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.270 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.270 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:22.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993882+0000", 2024-09-14T23:58:22.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7280000000000001e-06 2024-09-14T23:58:22.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.270 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.270 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:22.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993885+0000", 2024-09-14T23:58:22.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.751e-06 2024-09-14T23:58:22.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.271 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.271 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.271 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996081+0000", 2024-09-14T23:58:22.271 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0021960590000000002 2024-09-14T23:58:22.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.271 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.271 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:22.271 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996091+0000", 2024-09-14T23:58:22.271 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4930000000000004e-06 2024-09-14T23:58:22.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.271 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.271 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.271 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065308+0000", 2024-09-14T23:58:22.271 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.069217407 2024-09-14T23:58:22.272 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.272 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.272 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:22.272 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065414+0000", 2024-09-14T23:58:22.272 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105438 2024-09-14T23:58:22.272 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.272 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.272 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:22.272 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065575+0000", 2024-09-14T23:58:22.272 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000160956 2024-09-14T23:58:22.272 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.272 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:22.272 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.273 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.273 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.273 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187219 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1596416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:22.273 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.993889+0000", 2024-09-14T23:58:22.273 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 14.208942567999999, 2024-09-14T23:58:22.273 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0720031190000001, 2024-09-14T23:58:22.273 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:22.273 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:22.273 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:22.273 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:22.273 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:22.273 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:22.273 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187219 2024-09-14T23:58:22.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.274 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:22.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.274 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:22.274 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993889+0000", 2024-09-14T23:58:22.274 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.274 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:22.274 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993889+0000", 2024-09-14T23:58:22.274 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.274 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:22.274 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993889+0000", 2024-09-14T23:58:22.275 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2399999999999999e-07 2024-09-14T23:58:22.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.275 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.275 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:22.275 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993890+0000", 2024-09-14T23:58:22.275 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1939999999999999e-06 2024-09-14T23:58:22.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.275 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.275 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:22.275 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993893+0000", 2024-09-14T23:58:22.275 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1300000000000001e-06 2024-09-14T23:58:22.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.275 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.275 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:22.276 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993896+0000", 2024-09-14T23:58:22.276 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6879999999999999e-06 2024-09-14T23:58:22.276 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.276 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.276 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.276 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996119+0000", 2024-09-14T23:58:22.276 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0022226149999999998 2024-09-14T23:58:22.276 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.276 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.276 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:22.276 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996123+0000", 2024-09-14T23:58:22.276 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.211e-06 2024-09-14T23:58:22.276 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.277 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.277 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.277 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065616+0000", 2024-09-14T23:58:22.277 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0694935240000001 2024-09-14T23:58:22.277 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.277 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.277 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:22.277 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065733+0000", 2024-09-14T23:58:22.277 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000116492 2024-09-14T23:58:22.277 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.277 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.277 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:22.278 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065892+0000", 2024-09-14T23:58:22.278 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015894100000000001 2024-09-14T23:58:22.278 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.278 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:22.278 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.278 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.278 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.278 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187220 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1611776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:22.278 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.993899+0000", 2024-09-14T23:58:22.278 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 14.208932109999999, 2024-09-14T23:58:22.278 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.072298041, 2024-09-14T23:58:22.278 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:22.278 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:22.278 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:22.279 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:22.279 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:22.279 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:22.279 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187220 2024-09-14T23:58:22.279 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.279 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:22.279 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.279 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:22.279 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993899+0000", 2024-09-14T23:58:22.279 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.279 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.279 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.279 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:22.279 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993899+0000", 2024-09-14T23:58:22.280 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.280 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.280 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.280 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:22.280 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993899+0000", 2024-09-14T23:58:22.280 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2100000000000001e-07 2024-09-14T23:58:22.280 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.280 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.280 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:22.280 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993901+0000", 2024-09-14T23:58:22.280 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.141e-06 2024-09-14T23:58:22.280 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.280 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.281 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:22.281 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993904+0000", 2024-09-14T23:58:22.281 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1470000000000001e-06 2024-09-14T23:58:22.281 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.281 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.281 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:22.281 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993906+0000", 2024-09-14T23:58:22.281 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6340000000000001e-06 2024-09-14T23:58:22.281 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.281 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.281 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.281 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996160+0000", 2024-09-14T23:58:22.281 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0022537659999999999 2024-09-14T23:58:22.281 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.282 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.282 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:22.282 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996169+0000", 2024-09-14T23:58:22.282 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2739999999999996e-06 2024-09-14T23:58:22.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.282 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.282 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.282 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065981+0000", 2024-09-14T23:58:22.282 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.069811742 2024-09-14T23:58:22.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.282 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.282 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:22.282 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.066092+0000", 2024-09-14T23:58:22.282 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000110701 2024-09-14T23:58:22.283 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.283 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.283 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:22.283 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.066197+0000", 2024-09-14T23:58:22.283 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010541500000000001 2024-09-14T23:58:22.283 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.283 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:22.283 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.283 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.283 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.283 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187221 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1600512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:22.283 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.993909+0000", 2024-09-14T23:58:22.283 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 14.208921898, 2024-09-14T23:58:22.283 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.072643499, 2024-09-14T23:58:22.284 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:22.284 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:22.284 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:22.284 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:22.284 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:22.284 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:22.284 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187221 2024-09-14T23:58:22.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.284 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:22.284 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.284 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:22.284 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993909+0000", 2024-09-14T23:58:22.284 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.285 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.285 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:22.285 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993909+0000", 2024-09-14T23:58:22.285 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.285 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.285 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:22.285 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995673+0000", 2024-09-14T23:58:22.285 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001763141 2024-09-14T23:58:22.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.285 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.285 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:22.285 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995677+0000", 2024-09-14T23:58:22.286 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8709999999999996e-06 2024-09-14T23:58:22.286 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.286 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.286 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:22.286 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995686+0000", 2024-09-14T23:58:22.286 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6649999999999992e-06 2024-09-14T23:58:22.286 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.286 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.286 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:22.286 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995691+0000", 2024-09-14T23:58:22.286 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0289999999999996e-06 2024-09-14T23:58:22.286 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.286 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.286 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.286 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996199+0000", 2024-09-14T23:58:22.287 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00050799099999999995 2024-09-14T23:58:22.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.287 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.287 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:22.287 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996205+0000", 2024-09-14T23:58:22.287 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8259999999999998e-06 2024-09-14T23:58:22.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.287 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.287 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.287 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.066238+0000", 2024-09-14T23:58:22.287 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.070032928 2024-09-14T23:58:22.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.288 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.288 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:22.288 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.066393+0000", 2024-09-14T23:58:22.288 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015549800000000001 2024-09-14T23:58:22.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.288 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.288 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:22.288 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.066553+0000", 2024-09-14T23:58:22.288 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015955 2024-09-14T23:58:22.288 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.288 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:22.288 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.289 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.289 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187223 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1604608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:22.289 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.995728+0000", 2024-09-14T23:58:22.289 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 14.207103082, 2024-09-14T23:58:22.289 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.071486478, 2024-09-14T23:58:22.289 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:22.289 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:22.289 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:22.289 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:22.289 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:22.289 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:22.289 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187223 2024-09-14T23:58:22.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.290 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:22.290 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.290 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:22.290 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995728+0000", 2024-09-14T23:58:22.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.290 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.290 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:22.290 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995728+0000", 2024-09-14T23:58:22.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.290 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.290 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:22.291 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995728+0000", 2024-09-14T23:58:22.291 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.65e-07 2024-09-14T23:58:22.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.291 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.291 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:22.291 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995730+0000", 2024-09-14T23:58:22.291 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.406e-06 2024-09-14T23:58:22.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.291 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.291 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:22.291 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995734+0000", 2024-09-14T23:58:22.291 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2640000000000002e-06 2024-09-14T23:58:22.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.291 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.292 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:22.292 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995737+0000", 2024-09-14T23:58:22.292 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0589999999999998e-06 2024-09-14T23:58:22.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.292 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.292 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.292 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996288+0000", 2024-09-14T23:58:22.292 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00055059599999999998 2024-09-14T23:58:22.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.292 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.292 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:22.292 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996294+0000", 2024-09-14T23:58:22.292 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0260000000000004e-06 2024-09-14T23:58:22.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.293 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.293 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.293 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.066938+0000", 2024-09-14T23:58:22.293 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0706439510000001 2024-09-14T23:58:22.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.293 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.293 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:22.293 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067050+0000", 2024-09-14T23:58:22.293 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011220100000000001 2024-09-14T23:58:22.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.293 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.293 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:22.294 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067215+0000", 2024-09-14T23:58:22.294 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016470999999999999 2024-09-14T23:58:22.294 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.294 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:22.294 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.294 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.294 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.294 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187224 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1619968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:22.294 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.995742+0000", 2024-09-14T23:58:22.294 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 14.207089267000001, 2024-09-14T23:58:22.294 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0717603609999999, 2024-09-14T23:58:22.294 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:22.294 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:22.295 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:22.295 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:22.295 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:22.295 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:22.295 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187224 2024-09-14T23:58:22.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.295 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:22.295 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.295 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:22.295 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995742+0000", 2024-09-14T23:58:22.295 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.295 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.295 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:22.296 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995742+0000", 2024-09-14T23:58:22.296 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.296 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.296 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.296 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:22.296 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995742+0000", 2024-09-14T23:58:22.296 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2399999999999999e-07 2024-09-14T23:58:22.296 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.296 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.296 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:22.296 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995744+0000", 2024-09-14T23:58:22.296 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.356e-06 2024-09-14T23:58:22.296 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.296 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.297 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:22.297 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995747+0000", 2024-09-14T23:58:22.297 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.304e-06 2024-09-14T23:58:22.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.297 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.297 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:22.297 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995750+0000", 2024-09-14T23:58:22.297 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6510000000000001e-06 2024-09-14T23:58:22.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.297 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.297 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.297 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996327+0000", 2024-09-14T23:58:22.297 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00057794300000000003 2024-09-14T23:58:22.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.298 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.298 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:22.298 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996337+0000", 2024-09-14T23:58:22.298 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3759999999999997e-06 2024-09-14T23:58:22.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.298 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.298 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.298 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067244+0000", 2024-09-14T23:58:22.298 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.070907367 2024-09-14T23:58:22.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.298 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.298 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:22.298 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067341+0000", 2024-09-14T23:58:22.298 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7277999999999999e-05 2024-09-14T23:58:22.299 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.299 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.299 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:22.299 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067502+0000", 2024-09-14T23:58:22.299 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016086200000000001 2024-09-14T23:58:22.299 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.299 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:22.299 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.299 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.299 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.299 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187225 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1608704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:22.299 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.995766+0000", 2024-09-14T23:58:22.299 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 14.207065093000001, 2024-09-14T23:58:22.300 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.072043649, 2024-09-14T23:58:22.300 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:22.300 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:22.300 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:22.300 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:22.300 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:22.300 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:22.300 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187225 2024-09-14T23:58:22.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.300 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:22.300 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.300 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:22.300 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995766+0000", 2024-09-14T23:58:22.300 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.301 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.301 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:22.301 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995766+0000", 2024-09-14T23:58:22.301 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.301 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.301 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:22.301 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995766+0000", 2024-09-14T23:58:22.301 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9200000000000002e-07 2024-09-14T23:58:22.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.301 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.301 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:22.301 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995768+0000", 2024-09-14T23:58:22.302 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8679999999999999e-06 2024-09-14T23:58:22.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.302 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.302 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:22.302 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995772+0000", 2024-09-14T23:58:22.302 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6339999999999999e-06 2024-09-14T23:58:22.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.302 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.302 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:22.302 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995775+0000", 2024-09-14T23:58:22.302 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8159999999999998e-06 2024-09-14T23:58:22.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.302 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.302 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.302 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996372+0000", 2024-09-14T23:58:22.302 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00059758800000000003 2024-09-14T23:58:22.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.303 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:22.303 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996377+0000", 2024-09-14T23:58:22.303 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7770000000000002e-06 2024-09-14T23:58:22.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.303 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.303 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067534+0000", 2024-09-14T23:58:22.303 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.071156821 2024-09-14T23:58:22.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.303 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:22.303 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067638+0000", 2024-09-14T23:58:22.304 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000103737 2024-09-14T23:58:22.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.304 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.304 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:22.304 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067810+0000", 2024-09-14T23:58:22.304 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017211599999999999 2024-09-14T23:58:22.304 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.304 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:22.304 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.304 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.304 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187226 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1624064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:22.304 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.995778+0000", 2024-09-14T23:58:22.304 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 14.207052914, 2024-09-14T23:58:22.305 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.072339081, 2024-09-14T23:58:22.305 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:22.305 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:22.305 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:22.305 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:22.305 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:22.305 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:22.305 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187226 2024-09-14T23:58:22.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.305 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:22.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.305 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:22.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995778+0000", 2024-09-14T23:58:22.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.306 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.306 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:22.306 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995778+0000", 2024-09-14T23:58:22.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.306 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.306 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:22.306 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995779+0000", 2024-09-14T23:58:22.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0699999999999999e-07 2024-09-14T23:58:22.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.306 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.307 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:22.307 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995780+0000", 2024-09-14T23:58:22.307 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2139999999999999e-06 2024-09-14T23:58:22.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.307 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.307 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:22.307 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995783+0000", 2024-09-14T23:58:22.307 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1250000000000001e-06 2024-09-14T23:58:22.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.307 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.307 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:22.307 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995787+0000", 2024-09-14T23:58:22.307 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6679999999999999e-06 2024-09-14T23:58:22.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.308 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.308 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.308 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996404+0000", 2024-09-14T23:58:22.308 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00061777499999999999 2024-09-14T23:58:22.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.308 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.308 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:22.308 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996414+0000", 2024-09-14T23:58:22.308 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.431e-06 2024-09-14T23:58:22.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.308 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.308 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.308 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067844+0000", 2024-09-14T23:58:22.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0714302410000001 2024-09-14T23:58:22.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.309 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.309 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:22.309 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067952+0000", 2024-09-14T23:58:22.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000107778 2024-09-14T23:58:22.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.309 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.309 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:22.309 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.068117+0000", 2024-09-14T23:58:22.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000165642 2024-09-14T23:58:22.309 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.309 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:22.309 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.310 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187227 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1612800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:22.310 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.995806+0000", 2024-09-14T23:58:22.310 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 14.207025677000001, 2024-09-14T23:58:22.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0726092810000001, 2024-09-14T23:58:22.310 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:22.310 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:22.310 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:22.310 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:22.310 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:22.310 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:22.310 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187227 2024-09-14T23:58:22.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.310 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:22.311 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.311 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:22.311 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995806+0000", 2024-09-14T23:58:22.311 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.311 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.311 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:22.311 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995806+0000", 2024-09-14T23:58:22.311 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.311 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.311 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:22.311 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995672+0000", 2024-09-14T23:58:22.311 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99986608099999996 2024-09-14T23:58:22.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.312 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:22.312 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995677+0000", 2024-09-14T23:58:22.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9520000000000002e-06 2024-09-14T23:58:22.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.312 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:22.312 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995686+0000", 2024-09-14T23:58:22.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8199999999999992e-06 2024-09-14T23:58:22.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.312 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:22.312 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995693+0000", 2024-09-14T23:58:22.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1630000000000001e-06 2024-09-14T23:58:22.313 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.313 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.313 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.313 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.068149+0000", 2024-09-14T23:58:22.313 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.072456692000000003 2024-09-14T23:58:22.313 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.313 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.313 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:22.313 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.068251+0000", 2024-09-14T23:58:22.313 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101963 2024-09-14T23:58:22.313 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.313 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.313 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:22.314 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.068415+0000", 2024-09-14T23:58:22.314 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016360999999999999 2024-09-14T23:58:22.314 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.314 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:22.314 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.314 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.314 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:22.314 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:22.314 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T23:58:22.316 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T23:58:22.317 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:22.314+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:22.416 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:22.414+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:22.470 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:22.470 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:58:22.470 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:58:22.470 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:22.479 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T23:58:22.517 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:22.516+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:22.585 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:22.586 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:58:22.586 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:58:22.586 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:58:22.586 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:22.594 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T23:58:22.618 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:22.617+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:22.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-14T23:58:22.637+0000 7f4ab4d22640 0 --2- 172.21.15.42:0/1350421453 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f4ab0070d40 0x7f4ab013bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-14T23:58:22.704 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:22.704 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:58:22.704 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:58:22.704 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:58:22.704 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.705 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4284.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-14T23:58:22.705 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:58:22.705 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 120.177145928, 2024-09-14T23:58:22.705 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.004942086, 2024-09-14T23:58:22.705 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:22.705 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:22.705 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:22.705 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:22.705 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4284", 2024-09-14T23:58:22.706 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/4278347926", 2024-09-14T23:58:22.706 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1 2024-09-14T23:58:22.706 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.706 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:22.706 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.706 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:22.706 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:58:22.706 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.706 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.707 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.707 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:22.707 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:58:22.707 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.707 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.707 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.707 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:22.707 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524331+0000", 2024-09-14T23:58:22.707 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.383e-05 2024-09-14T23:58:22.707 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.708 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.708 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:22.708 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524339+0000", 2024-09-14T23:58:22.708 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8290000000000008e-06 2024-09-14T23:58:22.708 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.708 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.708 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:22.708 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524359+0000", 2024-09-14T23:58:22.709 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9998000000000001e-05 2024-09-14T23:58:22.709 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.709 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.709 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:22.709 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524373+0000", 2024-09-14T23:58:22.709 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4304000000000001e-05 2024-09-14T23:58:22.709 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.709 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.709 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.710 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524488+0000", 2024-09-14T23:58:22.710 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114663 2024-09-14T23:58:22.710 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.710 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.710 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:22.710 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524501+0000", 2024-09-14T23:58:22.710 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3349999999999999e-05 2024-09-14T23:58:22.710 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.710 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.710 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.711 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.526868+0000", 2024-09-14T23:58:22.711 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023670090000000002 2024-09-14T23:58:22.711 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.711 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.711 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:22.711 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.527041+0000", 2024-09-14T23:58:22.711 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000172597 2024-09-14T23:58:22.711 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.711 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.712 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-14T23:58:22.712 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.527384+0000", 2024-09-14T23:58:22.712 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00034309500000000002 2024-09-14T23:58:22.712 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.712 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.712 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:22.712 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529103+0000", 2024-09-14T23:58:22.712 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017185220000000001 2024-09-14T23:58:22.712 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.713 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.713 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:58:22.713 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529188+0000", 2024-09-14T23:58:22.713 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5591000000000001e-05 2024-09-14T23:58:22.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.713 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.713 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:22.713 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529227+0000", 2024-09-14T23:58:22.713 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8563000000000003e-05 2024-09-14T23:58:22.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.714 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.714 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:22.714 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529259+0000", 2024-09-14T23:58:22.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2734999999999997e-05 2024-09-14T23:58:22.714 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.714 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:22.714 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.715 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.715 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4284.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-14T23:58:22.715 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:23.530683+0000", 2024-09-14T23:58:22.715 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 119.170780263, 2024-09-14T23:58:22.715 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0036133110000000001, 2024-09-14T23:58:22.715 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:22.715 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:22.715 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:22.715 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:22.715 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4284", 2024-09-14T23:58:22.716 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/4278347926", 2024-09-14T23:58:22.716 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5 2024-09-14T23:58:22.716 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.716 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:22.716 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.716 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:22.716 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530683+0000", 2024-09-14T23:58:22.716 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.716 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.717 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.717 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:22.717 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530683+0000", 2024-09-14T23:58:22.717 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.717 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.717 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.717 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:22.717 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530685+0000", 2024-09-14T23:58:22.718 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.075e-06 2024-09-14T23:58:22.718 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.718 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.718 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:22.718 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530695+0000", 2024-09-14T23:58:22.718 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0261999999999999e-05 2024-09-14T23:58:22.718 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.718 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.718 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:22.718 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530715+0000", 2024-09-14T23:58:22.719 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9930000000000001e-05 2024-09-14T23:58:22.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.719 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.719 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:22.719 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530724+0000", 2024-09-14T23:58:22.719 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.901e-06 2024-09-14T23:58:22.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.719 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.719 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.719 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530800+0000", 2024-09-14T23:58:22.720 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5873999999999999e-05 2024-09-14T23:58:22.720 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.720 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.720 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-14T23:58:22.720 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530807+0000", 2024-09-14T23:58:22.720 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2389999999999996e-06 2024-09-14T23:58:22.720 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.720 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.720 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.720 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.532494+0000", 2024-09-14T23:58:22.721 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016871830000000001 2024-09-14T23:58:22.721 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.721 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.721 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:22.721 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.532599+0000", 2024-09-14T23:58:22.721 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010472 2024-09-14T23:58:22.721 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.721 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.722 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1", 2024-09-14T23:58:22.722 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.532715+0000", 2024-09-14T23:58:22.722 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011568199999999999 2024-09-14T23:58:22.722 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.722 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.722 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:58:22.722 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.533460+0000", 2024-09-14T23:58:22.722 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00074494999999999997 2024-09-14T23:58:22.722 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.722 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.723 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:22.723 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534230+0000", 2024-09-14T23:58:22.723 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00077036599999999998 2024-09-14T23:58:22.723 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.723 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.723 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:22.723 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534266+0000", 2024-09-14T23:58:22.723 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5725000000000001e-05 2024-09-14T23:58:22.724 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.724 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.724 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:22.724 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534296+0000", 2024-09-14T23:58:22.724 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0403999999999999e-05 2024-09-14T23:58:22.724 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.724 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:22.724 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.724 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.724 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.724 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56383 3.c e56/49 3:36b698b0:::rbd_data.1118c4b3aebe.0000000000000000:head v 56'194, mlcod=56'192)", 2024-09-14T23:58:22.725 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.270483+0000", 2024-09-14T23:58:22.725 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 76.430980462999997, 2024-09-14T23:58:22.725 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0079685880000000004, 2024-09-14T23:58:22.725 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:22.725 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:22.725 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:22.725 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:22.725 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.725 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:22.725 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.270483+0000", 2024-09-14T23:58:22.725 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.725 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.725 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.726 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:22.726 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.270483+0000", 2024-09-14T23:58:22.726 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.726 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.726 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.726 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:22.726 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.270484+0000", 2024-09-14T23:58:22.726 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7400000000000002e-07 2024-09-14T23:58:22.726 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.726 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.726 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:22.726 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273680+0000", 2024-09-14T23:58:22.726 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031966680000000002 2024-09-14T23:58:22.727 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.727 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.727 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:22.727 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273697+0000", 2024-09-14T23:58:22.727 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6595e-05 2024-09-14T23:58:22.727 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.727 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.727 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:22.727 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273705+0000", 2024-09-14T23:58:22.727 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7530000000000001e-06 2024-09-14T23:58:22.727 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.727 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.727 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.728 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273763+0000", 2024-09-14T23:58:22.728 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8238e-05 2024-09-14T23:58:22.728 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.728 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.728 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:22.728 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273781+0000", 2024-09-14T23:58:22.728 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8281e-05 2024-09-14T23:58:22.728 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.728 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.728 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:22.728 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.278408+0000", 2024-09-14T23:58:22.728 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00462716 2024-09-14T23:58:22.728 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.728 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.728 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:22.729 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.278451+0000", 2024-09-14T23:58:22.729 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3019000000000003e-05 2024-09-14T23:58:22.729 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.729 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:22.729 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.729 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.729 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.729 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56384 3.c e56/49 3:36b698b0:::rbd_data.1118c4b3aebe.0000000000000000:head v 56'196, mlcod=56'192)", 2024-09-14T23:58:22.729 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:58:22.729 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 76.426947791000003, 2024-09-14T23:58:22.729 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0046707529999999997, 2024-09-14T23:58:22.729 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:22.730 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:22.730 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:22.730 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:22.730 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.730 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:22.730 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:58:22.730 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.730 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.730 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.730 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:22.730 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:58:22.730 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.730 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.730 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.730 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:22.731 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:58:22.731 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1699999999999997e-07 2024-09-14T23:58:22.731 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.731 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.731 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:22.731 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274523+0000", 2024-09-14T23:58:22.731 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9129999999999997e-06 2024-09-14T23:58:22.731 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.731 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.731 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:22.731 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274535+0000", 2024-09-14T23:58:22.731 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1788e-05 2024-09-14T23:58:22.731 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.732 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.732 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:22.732 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274542+0000", 2024-09-14T23:58:22.732 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0539999999999997e-06 2024-09-14T23:58:22.732 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.732 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.732 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.732 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275889+0000", 2024-09-14T23:58:22.732 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00134669 2024-09-14T23:58:22.732 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.732 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.732 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:22.733 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275917+0000", 2024-09-14T23:58:22.733 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7664000000000001e-05 2024-09-14T23:58:22.733 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.733 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.733 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:22.733 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279134+0000", 2024-09-14T23:58:22.733 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0032178459999999999 2024-09-14T23:58:22.733 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.733 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.733 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:22.733 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279186+0000", 2024-09-14T23:58:22.733 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1881e-05 2024-09-14T23:58:22.733 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.733 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:22.734 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.734 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.734 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.734 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56381 3.1d e56/49 3:bcf9a8cf:::rbd_data.1118c4b3aebe.0000000000000006:head v 56'210, mlcod=56'208)", 2024-09-14T23:58:22.734 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.275462+0000", 2024-09-14T23:58:22.734 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 76.426001460999998, 2024-09-14T23:58:22.734 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0071310710000000001, 2024-09-14T23:58:22.734 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:22.734 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:22.734 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:22.734 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:22.734 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.734 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:22.735 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275462+0000", 2024-09-14T23:58:22.735 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.735 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.735 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.735 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:22.735 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275462+0000", 2024-09-14T23:58:22.735 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.735 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.735 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.735 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:22.735 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275463+0000", 2024-09-14T23:58:22.735 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2699999999999999e-07 2024-09-14T23:58:22.735 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.736 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.736 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:22.736 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276779+0000", 2024-09-14T23:58:22.736 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0013163649999999999 2024-09-14T23:58:22.736 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.736 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.736 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:22.736 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276795+0000", 2024-09-14T23:58:22.736 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.59e-05 2024-09-14T23:58:22.736 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.736 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.736 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:22.736 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276806+0000", 2024-09-14T23:58:22.736 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1106e-05 2024-09-14T23:58:22.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.737 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.737 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.737 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276941+0000", 2024-09-14T23:58:22.737 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013477500000000001 2024-09-14T23:58:22.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.737 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.737 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:22.737 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276963+0000", 2024-09-14T23:58:22.737 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2419999999999999e-05 2024-09-14T23:58:22.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.737 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.737 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:22.738 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282546+0000", 2024-09-14T23:58:22.738 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0055827100000000003 2024-09-14T23:58:22.738 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.738 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.738 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:22.738 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282593+0000", 2024-09-14T23:58:22.738 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7067999999999997e-05 2024-09-14T23:58:22.738 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.738 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:22.738 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.738 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.738 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.738 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56387 3.c e56/49 3:30ee4ba0:::rbd_data.1118c4b3aebe.0000000000000004:head v 56'197, mlcod=56'192)", 2024-09-14T23:58:22.738 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.276498+0000", 2024-09-14T23:58:22.739 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 76.424965365999995, 2024-09-14T23:58:22.739 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0092192639999999996, 2024-09-14T23:58:22.739 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:22.739 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:22.739 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:22.739 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:22.739 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.739 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:22.739 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276498+0000", 2024-09-14T23:58:22.739 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.739 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.739 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.739 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:22.739 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276498+0000", 2024-09-14T23:58:22.740 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.740 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.740 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.740 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:22.740 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276499+0000", 2024-09-14T23:58:22.740 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1100000000000004e-07 2024-09-14T23:58:22.740 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.740 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.740 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:22.740 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279646+0000", 2024-09-14T23:58:22.740 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031473220000000001 2024-09-14T23:58:22.740 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.740 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.741 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:22.741 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279672+0000", 2024-09-14T23:58:22.741 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6129999999999999e-05 2024-09-14T23:58:22.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.741 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.741 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:22.741 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279682+0000", 2024-09-14T23:58:22.741 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7170000000000003e-06 2024-09-14T23:58:22.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.741 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.741 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.741 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279742+0000", 2024-09-14T23:58:22.741 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9926999999999997e-05 2024-09-14T23:58:22.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.742 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.742 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:22.742 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279761+0000", 2024-09-14T23:58:22.742 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9402000000000002e-05 2024-09-14T23:58:22.742 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.742 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.742 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:22.742 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.285680+0000", 2024-09-14T23:58:22.742 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.005918881 2024-09-14T23:58:22.742 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.742 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.742 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:22.742 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.285717+0000", 2024-09-14T23:58:22.743 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6974000000000002e-05 2024-09-14T23:58:22.743 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.743 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:22.743 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.743 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.743 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.743 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56388 3.c e56/49 3:30ee4ba0:::rbd_data.1118c4b3aebe.0000000000000004:head v 56'199, mlcod=56'192)", 2024-09-14T23:58:22.743 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:58:22.743 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 76.421473778000006, 2024-09-14T23:58:22.743 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0060596740000000001, 2024-09-14T23:58:22.743 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:22.743 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:22.743 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:22.743 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:22.744 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.744 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:22.744 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:58:22.744 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.744 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.744 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.744 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:22.744 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:58:22.744 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.744 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.744 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.744 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:22.744 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:58:22.745 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2200000000000003e-07 2024-09-14T23:58:22.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.745 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.745 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:22.745 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279995+0000", 2024-09-14T23:58:22.745 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5569999999999997e-06 2024-09-14T23:58:22.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.745 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.745 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:22.745 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280004+0000", 2024-09-14T23:58:22.745 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0080000000000008e-06 2024-09-14T23:58:22.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.746 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.746 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:22.746 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280010+0000", 2024-09-14T23:58:22.746 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5749999999999997e-06 2024-09-14T23:58:22.746 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.746 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.746 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.746 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282968+0000", 2024-09-14T23:58:22.746 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0029579390000000001 2024-09-14T23:58:22.746 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.746 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.746 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:22.746 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282998+0000", 2024-09-14T23:58:22.746 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9947999999999999e-05 2024-09-14T23:58:22.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.747 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.747 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:22.747 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.286012+0000", 2024-09-14T23:58:22.747 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030132900000000001 2024-09-14T23:58:22.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.747 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.747 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:22.747 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.286049+0000", 2024-09-14T23:58:22.747 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7534999999999999e-05 2024-09-14T23:58:22.747 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.747 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:22.747 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.747 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.748 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56382 3.1d e56/49 3:bcf9a8cf:::rbd_data.1118c4b3aebe.0000000000000006:head v 56'212, mlcod=56'208)", 2024-09-14T23:58:22.748 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.280285+0000", 2024-09-14T23:58:22.748 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 76.421178095000002, 2024-09-14T23:58:22.748 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035393489999999998, 2024-09-14T23:58:22.748 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:22.748 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:22.748 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:22.748 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:22.748 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.748 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:22.748 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280285+0000", 2024-09-14T23:58:22.748 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.748 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.748 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.748 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:22.748 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280285+0000", 2024-09-14T23:58:22.749 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.749 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.749 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.749 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:22.749 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280286+0000", 2024-09-14T23:58:22.749 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5000000000000001e-07 2024-09-14T23:58:22.749 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.749 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.749 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:22.749 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280299+0000", 2024-09-14T23:58:22.749 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3312000000000001e-05 2024-09-14T23:58:22.749 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.749 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.749 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:22.750 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280311+0000", 2024-09-14T23:58:22.750 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1677e-05 2024-09-14T23:58:22.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.750 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.750 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:22.750 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280318+0000", 2024-09-14T23:58:22.750 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.742e-06 2024-09-14T23:58:22.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.750 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.750 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.750 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280374+0000", 2024-09-14T23:58:22.750 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.592e-05 2024-09-14T23:58:22.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.750 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.751 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:22.751 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280392+0000", 2024-09-14T23:58:22.751 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8490000000000001e-05 2024-09-14T23:58:22.751 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.751 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.751 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:22.751 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.283693+0000", 2024-09-14T23:58:22.751 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033012050000000002 2024-09-14T23:58:22.751 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.751 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.751 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:22.751 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.283825+0000", 2024-09-14T23:58:22.751 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013105300000000001 2024-09-14T23:58:22.751 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.751 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:22.752 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.752 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.752 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.752 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "pg_update_log_missing(3.c epoch 56/49 rep_tid 1493 entries 56'200 (0'0) error 3:370203a3:::rbd_data.1118c4b3aebe.000000000000000d:head by client.4376.0:56396 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 56'192)", 2024-09-14T23:58:22.752 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:58:22.752 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 76.420953638, 2024-09-14T23:58:22.752 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057370099999999999, 2024-09-14T23:58:22.752 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:22.752 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:22.752 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:58:22.752 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:22.752 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.752 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:22.752 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:58:22.753 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.753 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.753 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.753 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:22.753 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:58:22.753 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.753 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.753 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.753 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:22.753 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:58:22.753 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0699999999999997e-07 2024-09-14T23:58:22.753 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.753 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.754 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:22.754 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280519+0000", 2024-09-14T23:58:22.754 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6260000000000001e-06 2024-09-14T23:58:22.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.754 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.754 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:22.754 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280541+0000", 2024-09-14T23:58:22.754 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1976000000000001e-05 2024-09-14T23:58:22.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.754 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.754 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:22.754 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280556+0000", 2024-09-14T23:58:22.754 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5075000000000001e-05 2024-09-14T23:58:22.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.755 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.755 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.285494+0000", 2024-09-14T23:58:22.755 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0049378690000000001 2024-09-14T23:58:22.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.755 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:22.755 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.286247+0000", 2024-09-14T23:58:22.755 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00075285700000000001 2024-09-14T23:58:22.755 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.755 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:22.755 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.756 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56400 3.1c e56/49 3:3f5b81ba:::rbd_data.1118c4b3aebe.000000000000000a:head v 56'236, mlcod=56'234)", 2024-09-14T23:58:22.756 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.295064+0000", 2024-09-14T23:58:22.756 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 76.406398846000002, 2024-09-14T23:58:22.756 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0054331589999999999, 2024-09-14T23:58:22.756 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:22.756 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:22.756 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:22.756 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:22.756 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.756 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:22.756 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.295064+0000", 2024-09-14T23:58:22.756 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.756 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.756 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.757 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:22.757 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.295064+0000", 2024-09-14T23:58:22.757 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.757 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.757 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.757 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:22.757 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.295066+0000", 2024-09-14T23:58:22.757 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.502e-06 2024-09-14T23:58:22.757 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.757 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.757 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:22.757 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296403+0000", 2024-09-14T23:58:22.757 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001336554 2024-09-14T23:58:22.757 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.757 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.758 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:22.758 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296428+0000", 2024-09-14T23:58:22.758 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5179000000000001e-05 2024-09-14T23:58:22.758 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.758 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.758 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:22.758 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296440+0000", 2024-09-14T23:58:22.758 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2387e-05 2024-09-14T23:58:22.758 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.758 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.758 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.758 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296554+0000", 2024-09-14T23:58:22.758 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114269 2024-09-14T23:58:22.758 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.758 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.759 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:22.759 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296581+0000", 2024-09-14T23:58:22.759 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6673999999999999e-05 2024-09-14T23:58:22.759 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.759 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.759 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:22.759 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.300418+0000", 2024-09-14T23:58:22.759 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0038371249999999998 2024-09-14T23:58:22.759 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.759 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.759 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:22.759 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.300498+0000", 2024-09-14T23:58:22.759 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9468999999999994e-05 2024-09-14T23:58:22.759 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.759 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:22.760 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.760 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.760 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:61921 3.5 e57/49 3:a05d004f:::rbd_data.11188d356229.000000000000000b:head v 57'279, mlcod=57'277)", 2024-09-14T23:58:22.760 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:07.560389+0000", 2024-09-14T23:58:22.760 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 75.141074247999995, 2024-09-14T23:58:22.760 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0034956520000000001, 2024-09-14T23:58:22.760 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:22.760 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:22.760 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:22.760 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:22.760 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.760 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:22.760 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.560389+0000", 2024-09-14T23:58:22.760 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.761 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.761 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.761 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:22.761 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.560389+0000", 2024-09-14T23:58:22.761 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.761 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.761 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.761 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:22.761 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.560390+0000", 2024-09-14T23:58:22.761 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4109999999999999e-06 2024-09-14T23:58:22.761 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.761 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.761 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:22.762 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.561839+0000", 2024-09-14T23:58:22.762 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001448406 2024-09-14T23:58:22.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.762 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.762 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:22.762 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.561854+0000", 2024-09-14T23:58:22.762 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5426000000000001e-05 2024-09-14T23:58:22.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.762 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.762 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:22.762 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.561863+0000", 2024-09-14T23:58:22.762 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7509999999999997e-06 2024-09-14T23:58:22.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.763 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.763 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.561917+0000", 2024-09-14T23:58:22.763 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3531e-05 2024-09-14T23:58:22.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.763 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:22.763 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.561944+0000", 2024-09-14T23:58:22.763 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7528000000000001e-05 2024-09-14T23:58:22.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.763 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:22.763 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563843+0000", 2024-09-14T23:58:22.763 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001899361 2024-09-14T23:58:22.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.764 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.764 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:22.764 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563885+0000", 2024-09-14T23:58:22.764 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1238000000000001e-05 2024-09-14T23:58:22.764 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.764 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:22.764 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.764 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.764 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:61922 3.5 e57/49 3:a11ebf8e:::rbd_data.11188d356229.000000000000000c:head v 57'280, mlcod=57'277)", 2024-09-14T23:58:22.764 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:07.563144+0000", 2024-09-14T23:58:22.764 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 75.138319194000005, 2024-09-14T23:58:22.764 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003977848, 2024-09-14T23:58:22.765 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:22.765 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:22.765 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:22.765 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:22.765 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.765 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:22.765 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563144+0000", 2024-09-14T23:58:22.765 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.765 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.765 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.765 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:22.765 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563144+0000", 2024-09-14T23:58:22.765 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.766 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.766 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.766 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:22.766 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563145+0000", 2024-09-14T23:58:22.766 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5400000000000001e-07 2024-09-14T23:58:22.766 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.766 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.766 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:22.766 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565056+0000", 2024-09-14T23:58:22.766 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0019108160000000001 2024-09-14T23:58:22.766 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.766 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.766 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:22.766 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565088+0000", 2024-09-14T23:58:22.767 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2586999999999998e-05 2024-09-14T23:58:22.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.767 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.767 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:22.767 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565100+0000", 2024-09-14T23:58:22.767 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1917e-05 2024-09-14T23:58:22.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.767 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.767 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.767 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565166+0000", 2024-09-14T23:58:22.767 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6245e-05 2024-09-14T23:58:22.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.767 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.768 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:22.768 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565190+0000", 2024-09-14T23:58:22.768 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3825999999999999e-05 2024-09-14T23:58:22.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.768 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.768 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:22.768 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.567080+0000", 2024-09-14T23:58:22.768 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001889541 2024-09-14T23:58:22.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.768 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.768 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:22.768 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.567122+0000", 2024-09-14T23:58:22.768 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2262000000000003e-05 2024-09-14T23:58:22.769 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.769 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:22.769 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.769 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.769 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.769 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:66564 3.c e60/49 3:33faa7ca:::rbd_data.111873792d9b.0000000000000003:head v 60'236, mlcod=57'233)", 2024-09-14T23:58:22.769 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:11.271596+0000", 2024-09-14T23:58:22.769 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 71.429867615999996, 2024-09-14T23:58:22.769 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0060396099999999999, 2024-09-14T23:58:22.769 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:22.769 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:22.769 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:22.769 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:22.769 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.770 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:22.770 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.271596+0000", 2024-09-14T23:58:22.770 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.770 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.770 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.770 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:22.770 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.271596+0000", 2024-09-14T23:58:22.770 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.770 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.770 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.770 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:22.770 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.271597+0000", 2024-09-14T23:58:22.770 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2559999999999999e-06 2024-09-14T23:58:22.770 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.770 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.771 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:22.771 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.274854+0000", 2024-09-14T23:58:22.771 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0032574470000000001 2024-09-14T23:58:22.771 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.771 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.771 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:22.771 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.274874+0000", 2024-09-14T23:58:22.771 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9449999999999998e-05 2024-09-14T23:58:22.771 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.771 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.771 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:22.771 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.274883+0000", 2024-09-14T23:58:22.771 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5980000000000003e-06 2024-09-14T23:58:22.771 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.771 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.772 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.772 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.274977+0000", 2024-09-14T23:58:22.772 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3270999999999995e-05 2024-09-14T23:58:22.772 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.772 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.772 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:22.772 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.275000+0000", 2024-09-14T23:58:22.772 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3088e-05 2024-09-14T23:58:22.772 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.772 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.772 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:22.772 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.277590+0000", 2024-09-14T23:58:22.772 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0025898660000000001 2024-09-14T23:58:22.772 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.773 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.773 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:22.773 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.277635+0000", 2024-09-14T23:58:22.773 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5633999999999997e-05 2024-09-14T23:58:22.773 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.773 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:22.773 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.773 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.773 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:133384 3.1f e92/49 3:fce5e529:::rbd_data.11186132a8b6.0000000000000002:head v 92'807, mlcod=89'804)", 2024-09-14T23:58:22.773 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:45.293642+0000", 2024-09-14T23:58:22.773 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 37.407821732999999, 2024-09-14T23:58:22.773 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003760186, 2024-09-14T23:58:22.773 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:22.773 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:22.774 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:22.774 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:22.774 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.774 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:22.774 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.293642+0000", 2024-09-14T23:58:22.774 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.774 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.774 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.774 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:22.774 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.293642+0000", 2024-09-14T23:58:22.774 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.774 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.774 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.774 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:22.775 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.293643+0000", 2024-09-14T23:58:22.775 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2500000000000001e-06 2024-09-14T23:58:22.775 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.775 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.775 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:22.775 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.295402+0000", 2024-09-14T23:58:22.775 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017595410000000001 2024-09-14T23:58:22.775 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.775 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.775 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:22.775 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.295422+0000", 2024-09-14T23:58:22.775 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9763999999999999e-05 2024-09-14T23:58:22.775 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.775 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.776 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:22.776 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.295436+0000", 2024-09-14T23:58:22.776 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.364e-05 2024-09-14T23:58:22.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.776 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.776 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.776 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.295499+0000", 2024-09-14T23:58:22.776 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2770999999999999e-05 2024-09-14T23:58:22.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.776 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.776 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:22.776 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.295516+0000", 2024-09-14T23:58:22.776 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7240000000000001e-05 2024-09-14T23:58:22.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.776 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.777 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:22.777 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.297324+0000", 2024-09-14T23:58:22.777 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001807857 2024-09-14T23:58:22.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.777 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.777 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:22.777 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.297402+0000", 2024-09-14T23:58:22.777 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8122999999999997e-05 2024-09-14T23:58:22.777 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.777 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:22.777 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.777 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.777 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:145282 3.1c e100/49 3:3dcd75bf:::rbd_data.1118584371b4.0000000000000004:head v 100'647, mlcod=100'644)", 2024-09-14T23:58:22.778 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:54.171268+0000", 2024-09-14T23:58:22.778 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 28.530195645999999, 2024-09-14T23:58:22.778 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003527353, 2024-09-14T23:58:22.778 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:22.778 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:22.778 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:22.778 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:22.778 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.778 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:22.778 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:54.171268+0000", 2024-09-14T23:58:22.778 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.778 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.778 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:22.779 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:54.171268+0000", 2024-09-14T23:58:22.779 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.779 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.779 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.779 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:22.779 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:54.171268+0000", 2024-09-14T23:58:22.779 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4900000000000005e-07 2024-09-14T23:58:22.779 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.779 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.779 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:22.779 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:54.172187+0000", 2024-09-14T23:58:22.779 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00091850799999999995 2024-09-14T23:58:22.779 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.779 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.780 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:22.780 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:54.172199+0000", 2024-09-14T23:58:22.780 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2544000000000001e-05 2024-09-14T23:58:22.780 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.780 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.780 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:22.780 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:54.172205+0000", 2024-09-14T23:58:22.780 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1490000000000002e-06 2024-09-14T23:58:22.780 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.780 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.780 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.780 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:54.172267+0000", 2024-09-14T23:58:22.780 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2617e-05 2024-09-14T23:58:22.780 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.780 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.781 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:22.781 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:54.172283+0000", 2024-09-14T23:58:22.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5758000000000001e-05 2024-09-14T23:58:22.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.781 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:22.781 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:54.174754+0000", 2024-09-14T23:58:22.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0024711400000000001 2024-09-14T23:58:22.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.781 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:22.781 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:54.174795+0000", 2024-09-14T23:58:22.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0887999999999999e-05 2024-09-14T23:58:22.781 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.782 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:22.782 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.782 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:156597 3.5 e103/49 3:a23c5924:::rbd_data.1118b2f495e6.0000000000000011:head v 103'518, mlcod=98'516)", 2024-09-14T23:58:22.782 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:58.656197+0000", 2024-09-14T23:58:22.782 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 24.045266676000001, 2024-09-14T23:58:22.782 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0040423029999999997, 2024-09-14T23:58:22.782 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:22.782 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:22.782 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:22.782 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:22.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.783 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:22.783 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656197+0000", 2024-09-14T23:58:22.783 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.783 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.783 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:22.783 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656197+0000", 2024-09-14T23:58:22.783 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.783 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.783 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:22.783 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656197+0000", 2024-09-14T23:58:22.783 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1900000000000002e-07 2024-09-14T23:58:22.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.784 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.784 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:22.784 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656676+0000", 2024-09-14T23:58:22.784 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00047921699999999999 2024-09-14T23:58:22.784 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.784 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.784 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:22.784 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656690+0000", 2024-09-14T23:58:22.784 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3916e-05 2024-09-14T23:58:22.784 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.784 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.784 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:22.784 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656698+0000", 2024-09-14T23:58:22.784 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6529999999999994e-06 2024-09-14T23:58:22.785 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.785 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.785 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.785 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656757+0000", 2024-09-14T23:58:22.785 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9237999999999998e-05 2024-09-14T23:58:22.785 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.785 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.785 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:22.785 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656780+0000", 2024-09-14T23:58:22.785 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2968000000000001e-05 2024-09-14T23:58:22.785 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.785 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.785 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:22.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.660188+0000", 2024-09-14T23:58:22.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0034078630000000001 2024-09-14T23:58:22.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:22.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.660239+0000", 2024-09-14T23:58:22.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0828999999999997e-05 2024-09-14T23:58:22.786 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.786 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:22.786 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.786 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:157947 3.1c e103/49 3:383374fe:::rbd_data.1118b2f495e6.000000000000000c:head v 103'771, mlcod=103'769)", 2024-09-14T23:58:22.786 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:58.765271+0000", 2024-09-14T23:58:22.787 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.936191794999999, 2024-09-14T23:58:22.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0040869829999999998, 2024-09-14T23:58:22.787 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:22.787 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:22.787 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:22.787 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:22.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.787 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:22.787 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765271+0000", 2024-09-14T23:58:22.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.787 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:22.787 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765271+0000", 2024-09-14T23:58:22.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.788 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:22.788 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765273+0000", 2024-09-14T23:58:22.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1149999999999999e-06 2024-09-14T23:58:22.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.788 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:22.788 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765854+0000", 2024-09-14T23:58:22.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00058119599999999997 2024-09-14T23:58:22.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.789 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:22.789 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765869+0000", 2024-09-14T23:58:22.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5641000000000002e-05 2024-09-14T23:58:22.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.789 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:22.789 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765878+0000", 2024-09-14T23:58:22.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4519999999999997e-06 2024-09-14T23:58:22.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.789 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.789 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765942+0000", 2024-09-14T23:58:22.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3727999999999997e-05 2024-09-14T23:58:22.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:22.790 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765968+0000", 2024-09-14T23:58:22.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6171999999999999e-05 2024-09-14T23:58:22.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:22.790 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.769319+0000", 2024-09-14T23:58:22.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033511629999999999 2024-09-14T23:58:22.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:22.790 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.769358+0000", 2024-09-14T23:58:22.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9515999999999999e-05 2024-09-14T23:58:22.791 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.791 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:22.791 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.791 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:160196 3.c e103/49 3:34623e68:::rbd_data.1118b2f495e6.0000000000000005:head v 103'551, mlcod=102'549)", 2024-09-14T23:58:22.791 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:59.131433+0000", 2024-09-14T23:58:22.791 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.570030486, 2024-09-14T23:58:22.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0036932530000000001, 2024-09-14T23:58:22.791 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:22.791 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:22.791 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:22.791 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:22.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.792 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:22.792 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.131433+0000", 2024-09-14T23:58:22.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.792 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:22.792 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.131433+0000", 2024-09-14T23:58:22.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.792 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:22.792 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.131433+0000", 2024-09-14T23:58:22.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8500000000000001e-07 2024-09-14T23:58:22.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.793 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:22.793 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.132144+0000", 2024-09-14T23:58:22.793 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00071022800000000003 2024-09-14T23:58:22.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.793 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:22.793 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.132154+0000", 2024-09-14T23:58:22.793 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0301e-05 2024-09-14T23:58:22.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.793 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:22.793 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.132159+0000", 2024-09-14T23:58:22.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6280000000000004e-06 2024-09-14T23:58:22.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.794 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.794 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.132208+0000", 2024-09-14T23:58:22.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9595000000000002e-05 2024-09-14T23:58:22.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.794 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:22.794 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.132231+0000", 2024-09-14T23:58:22.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3010000000000002e-05 2024-09-14T23:58:22.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.794 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:22.794 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.135087+0000", 2024-09-14T23:58:22.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0028553239999999998 2024-09-14T23:58:22.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:22.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.135126+0000", 2024-09-14T23:58:22.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9481999999999999e-05 2024-09-14T23:58:22.795 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.795 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:22.795 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.795 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188871 3.d 3:b35cff4d:::rbd_header.1118c900944f:head [watch reconnect cookie 140670715433840 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-14T23:58:22.795 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.970653+0000", 2024-09-14T23:58:22.796 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 14.730810753, 2024-09-14T23:58:22.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0727808270000001, 2024-09-14T23:58:22.796 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:22.796 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:22.796 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:22.796 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:22.796 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:22.796 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:22.796 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188871 2024-09-14T23:58:22.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.796 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:22.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.796 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:22.797 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970653+0000", 2024-09-14T23:58:22.797 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.797 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:22.797 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970653+0000", 2024-09-14T23:58:22.797 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.797 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:22.797 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970655+0000", 2024-09-14T23:58:22.797 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.277e-06 2024-09-14T23:58:22.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.798 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:22.798 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970662+0000", 2024-09-14T23:58:22.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9619999999999996e-06 2024-09-14T23:58:22.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.798 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:22.798 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970682+0000", 2024-09-14T23:58:22.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0279999999999999e-05 2024-09-14T23:58:22.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.798 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:22.798 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970691+0000", 2024-09-14T23:58:22.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2790000000000004e-06 2024-09-14T23:58:22.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.799 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.799 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.799 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970762+0000", 2024-09-14T23:58:22.799 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1019999999999994e-05 2024-09-14T23:58:22.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.799 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.799 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-14T23:58:22.799 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970770+0000", 2024-09-14T23:58:22.799 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7999999999999999e-06 2024-09-14T23:58:22.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.799 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.799 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.799 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.973516+0000", 2024-09-14T23:58:22.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0027456469999999999 2024-09-14T23:58:22.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:22.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.973525+0000", 2024-09-14T23:58:22.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4890000000000001e-06 2024-09-14T23:58:22.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.041451+0000", 2024-09-14T23:58:22.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.067926012 2024-09-14T23:58:22.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:22.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.041673+0000", 2024-09-14T23:58:22.801 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000222048 2024-09-14T23:58:22.801 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.801 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.801 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:22.801 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.043433+0000", 2024-09-14T23:58:22.801 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017600129999999999 2024-09-14T23:58:22.801 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.801 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:22.801 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.801 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.801 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.801 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(osd.0.0:3238 3.1f e122/49 3:f8a59e83:::rbd_data.11187b9f1032.0000000000000006:2f v 122'943, mlcod=120'940)", 2024-09-14T23:58:22.801 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:22.285283+0000", 2024-09-14T23:58:22.801 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.41617992999999998, 2024-09-14T23:58:22.801 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003756194, 2024-09-14T23:58:22.801 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:22.802 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:22.802 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:22.802 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:22.802 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.802 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:22.802 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.285283+0000", 2024-09-14T23:58:22.802 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.802 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.802 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:22.802 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.285283+0000", 2024-09-14T23:58:22.802 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:22.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.802 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.803 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:22.803 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.285284+0000", 2024-09-14T23:58:22.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5600000000000002e-07 2024-09-14T23:58:22.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.803 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:22.803 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.285289+0000", 2024-09-14T23:58:22.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9579999999999998e-06 2024-09-14T23:58:22.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.803 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:22.803 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.285300+0000", 2024-09-14T23:58:22.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1688999999999999e-05 2024-09-14T23:58:22.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.803 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:22.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.285309+0000", 2024-09-14T23:58:22.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.388e-06 2024-09-14T23:58:22.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:22.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.285367+0000", 2024-09-14T23:58:22.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8811000000000003e-05 2024-09-14T23:58:22.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:22.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.285388+0000", 2024-09-14T23:58:22.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0117000000000001e-05 2024-09-14T23:58:22.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:22.805 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.288994+0000", 2024-09-14T23:58:22.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0036064270000000002 2024-09-14T23:58:22.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:22.805 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:22.805 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:22.805 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.289040+0000", 2024-09-14T23:58:22.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5547999999999998e-05 2024-09-14T23:58:22.805 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.805 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:22.805 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.805 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:22.805 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:22.805 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:22.807 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T23:58:22.807 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:22.805+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:22.908 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:22.907+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:22.955 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:22.955 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:58:22.955 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:58:22.955 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:22.964 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T23:58:23.008 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:23.007+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:23.071 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:23.071 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:58:23.072 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:58:23.072 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:58:23.072 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:23.080 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T23:58:23.121 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:23.120+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:23.210 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:23.209+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:23.229 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:23.229 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:58:23.229 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:58:23.230 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:58:23.230 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.230 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:107472 3.1e 3:7bf21588:::rbd_data.11187de8b0a5.000000000000005d:head [set-alloc-hint object_size 2097152 write_size 2097152,write 317788~1491] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-14T23:58:23.230 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:25.198493+0000", 2024-09-14T23:58:23.230 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 58.028569040999997, 2024-09-14T23:58:23.230 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.78175149300000002, 2024-09-14T23:58:23.230 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:23.230 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:23.230 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:23.230 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:23.230 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:23.230 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:23.230 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 107472 2024-09-14T23:58:23.231 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.231 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:23.231 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.231 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:23.231 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198493+0000", 2024-09-14T23:58:23.231 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:23.231 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.231 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.231 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:23.231 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198493+0000", 2024-09-14T23:58:23.231 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:23.231 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.232 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.232 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:23.232 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198495+0000", 2024-09-14T23:58:23.232 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.826e-06 2024-09-14T23:58:23.232 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.232 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.232 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:23.232 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198503+0000", 2024-09-14T23:58:23.232 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0150000000000003e-06 2024-09-14T23:58:23.232 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.232 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.232 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:23.232 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198521+0000", 2024-09-14T23:58:23.232 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8374000000000001e-05 2024-09-14T23:58:23.233 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.233 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.233 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:23.233 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198533+0000", 2024-09-14T23:58:23.233 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1593000000000001e-05 2024-09-14T23:58:23.233 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.233 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.233 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:23.233 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198603+0000", 2024-09-14T23:58:23.233 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0279999999999998e-05 2024-09-14T23:58:23.233 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.234 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.234 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:23.234 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198617+0000", 2024-09-14T23:58:23.234 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3517e-05 2024-09-14T23:58:23.234 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.234 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.234 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:23.234 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.977067+0000", 2024-09-14T23:58:23.234 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.77845065800000002 2024-09-14T23:58:23.234 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.234 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.234 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for missing object", 2024-09-14T23:58:23.234 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.977173+0000", 2024-09-14T23:58:23.234 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010543999999999999 2024-09-14T23:58:23.235 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.235 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.235 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:23.235 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.978737+0000", 2024-09-14T23:58:23.235 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001564078 2024-09-14T23:58:23.235 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.235 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.235 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:23.235 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.978845+0000", 2024-09-14T23:58:23.235 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000108643 2024-09-14T23:58:23.235 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.235 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.235 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1", 2024-09-14T23:58:23.235 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.978964+0000", 2024-09-14T23:58:23.236 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000119085 2024-09-14T23:58:23.236 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.236 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.236 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:23.236 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980109+0000", 2024-09-14T23:58:23.236 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0011442170000000001 2024-09-14T23:58:23.236 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.236 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.236 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:58:23.236 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980130+0000", 2024-09-14T23:58:23.236 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1766000000000001e-05 2024-09-14T23:58:23.236 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.236 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.236 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:23.237 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980171+0000", 2024-09-14T23:58:23.237 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0627999999999998e-05 2024-09-14T23:58:23.237 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.237 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.237 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:23.237 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980244+0000", 2024-09-14T23:58:23.237 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3373000000000004e-05 2024-09-14T23:58:23.237 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:23.237 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:23.237 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:23.237 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.237 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.237 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129159 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 5996544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:23.237 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.927783+0000", 2024-09-14T23:58:23.238 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 44.299279149999997, 2024-09-14T23:58:23.238 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054661122999999999, 2024-09-14T23:58:23.238 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:23.238 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:23.238 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:23.238 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:23.238 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:23.238 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:23.238 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129159 2024-09-14T23:58:23.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.238 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:23.238 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.238 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:23.238 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.927783+0000", 2024-09-14T23:58:23.239 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:23.239 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.239 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.239 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:23.239 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.927783+0000", 2024-09-14T23:58:23.239 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:23.239 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.239 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.239 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:23.239 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929684+0000", 2024-09-14T23:58:23.239 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0019011080000000001 2024-09-14T23:58:23.239 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.239 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.239 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:23.240 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929688+0000", 2024-09-14T23:58:23.240 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2610000000000004e-06 2024-09-14T23:58:23.240 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.240 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.240 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:23.240 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929697+0000", 2024-09-14T23:58:23.240 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0219999999999993e-06 2024-09-14T23:58:23.240 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.240 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.240 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:23.240 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929702+0000", 2024-09-14T23:58:23.240 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8840000000000002e-06 2024-09-14T23:58:23.240 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.241 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.241 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:23.241 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.982090+0000", 2024-09-14T23:58:23.241 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052388045000000001 2024-09-14T23:58:23.241 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.241 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.241 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:23.241 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.982208+0000", 2024-09-14T23:58:23.241 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118236 2024-09-14T23:58:23.241 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.241 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.241 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:23.241 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.982444+0000", 2024-09-14T23:58:23.241 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000235567 2024-09-14T23:58:23.242 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:23.242 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:23.242 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:23.242 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.242 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.242 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129166 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 8192000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:23.242 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.929794+0000", 2024-09-14T23:58:23.242 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 44.297268399000004, 2024-09-14T23:58:23.242 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054471973, 2024-09-14T23:58:23.242 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:23.242 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:23.242 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:23.242 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:23.242 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:23.243 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:23.243 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129166 2024-09-14T23:58:23.243 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.243 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:23.243 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.243 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:23.243 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929794+0000", 2024-09-14T23:58:23.243 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:23.243 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.243 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.243 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:23.243 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929794+0000", 2024-09-14T23:58:23.243 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:23.243 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.244 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.244 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:23.244 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929794+0000", 2024-09-14T23:58:23.244 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6899999999999997e-07 2024-09-14T23:58:23.244 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.244 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.244 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:23.244 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929797+0000", 2024-09-14T23:58:23.244 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.328e-06 2024-09-14T23:58:23.244 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.244 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.244 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:23.244 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929802+0000", 2024-09-14T23:58:23.245 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7309999999999998e-06 2024-09-14T23:58:23.245 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.245 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.245 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:23.245 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929807+0000", 2024-09-14T23:58:23.245 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4349999999999999e-06 2024-09-14T23:58:23.245 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.245 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.245 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:23.245 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984063+0000", 2024-09-14T23:58:23.245 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054256331999999997 2024-09-14T23:58:23.245 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.245 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.246 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:23.246 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984177+0000", 2024-09-14T23:58:23.246 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114225 2024-09-14T23:58:23.246 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.246 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.246 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:23.246 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984266+0000", 2024-09-14T23:58:23.246 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8352999999999999e-05 2024-09-14T23:58:23.246 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:23.246 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:23.246 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:23.246 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.246 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.246 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129167 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 6782976~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:23.247 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.929827+0000", 2024-09-14T23:58:23.247 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 44.29723499, 2024-09-14T23:58:23.247 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054637593999999998, 2024-09-14T23:58:23.247 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:23.247 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:23.247 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:23.247 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:23.247 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:23.247 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:23.247 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129167 2024-09-14T23:58:23.248 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.248 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:23.248 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.248 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:23.248 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929827+0000", 2024-09-14T23:58:23.248 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:23.248 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.248 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.248 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:23.248 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929827+0000", 2024-09-14T23:58:23.248 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:23.248 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.249 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.249 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:23.249 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929827+0000", 2024-09-14T23:58:23.249 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4000000000000002e-07 2024-09-14T23:58:23.249 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.249 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.249 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:23.249 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929830+0000", 2024-09-14T23:58:23.249 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.419e-06 2024-09-14T23:58:23.249 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.249 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.249 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:23.249 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929837+0000", 2024-09-14T23:58:23.250 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0450000000000003e-06 2024-09-14T23:58:23.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.250 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.250 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:23.250 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929842+0000", 2024-09-14T23:58:23.250 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.69e-06 2024-09-14T23:58:23.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.250 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.250 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:23.250 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984278+0000", 2024-09-14T23:58:23.250 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054436516999999997 2024-09-14T23:58:23.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.250 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.250 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:23.251 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984331+0000", 2024-09-14T23:58:23.251 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2590999999999998e-05 2024-09-14T23:58:23.251 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.251 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.251 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:23.251 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984465+0000", 2024-09-14T23:58:23.251 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013389200000000001 2024-09-14T23:58:23.251 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:23.251 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:23.251 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:23.251 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.251 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.251 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129168 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 8224768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:23.252 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.929849+0000", 2024-09-14T23:58:23.252 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 44.297213432, 2024-09-14T23:58:23.252 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054870368000000003, 2024-09-14T23:58:23.252 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:23.252 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:23.252 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:23.252 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:23.252 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:23.252 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:23.252 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129168 2024-09-14T23:58:23.252 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.252 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:23.252 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.253 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:23.253 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929849+0000", 2024-09-14T23:58:23.253 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:23.253 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.253 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.253 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:23.253 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929849+0000", 2024-09-14T23:58:23.253 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:23.253 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.253 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.253 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:23.253 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931681+0000", 2024-09-14T23:58:23.253 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018326989999999999 2024-09-14T23:58:23.254 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.254 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.254 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:23.254 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931686+0000", 2024-09-14T23:58:23.254 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7190000000000001e-06 2024-09-14T23:58:23.254 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.254 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.254 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:23.254 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931697+0000", 2024-09-14T23:58:23.254 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1287999999999999e-05 2024-09-14T23:58:23.254 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.254 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.254 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:23.254 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931704+0000", 2024-09-14T23:58:23.255 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3339999999999998e-06 2024-09-14T23:58:23.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.255 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:23.255 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984505+0000", 2024-09-14T23:58:23.255 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052801684000000002 2024-09-14T23:58:23.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.255 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:23.255 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984621+0000", 2024-09-14T23:58:23.255 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000116165 2024-09-14T23:58:23.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.255 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:23.256 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984719+0000", 2024-09-14T23:58:23.256 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7479000000000006e-05 2024-09-14T23:58:23.256 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:23.256 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:23.256 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:23.256 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.256 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.256 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129176 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 10452992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:23.256 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.931830+0000", 2024-09-14T23:58:23.256 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 44.295231671000003, 2024-09-14T23:58:23.256 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054513292999999997, 2024-09-14T23:58:23.256 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:23.256 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:23.257 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:23.257 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:23.257 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:23.257 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:23.257 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129176 2024-09-14T23:58:23.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.257 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:23.257 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.257 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:23.257 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931830+0000", 2024-09-14T23:58:23.257 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:23.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.257 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.257 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:23.258 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931830+0000", 2024-09-14T23:58:23.258 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:23.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.258 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.258 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:23.258 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.933678+0000", 2024-09-14T23:58:23.258 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018472440000000001 2024-09-14T23:58:23.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.258 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.258 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:23.258 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.933686+0000", 2024-09-14T23:58:23.258 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0239999999999997e-06 2024-09-14T23:58:23.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.259 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.259 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:23.259 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.933694+0000", 2024-09-14T23:58:23.259 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8400000000000001e-06 2024-09-14T23:58:23.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.259 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.259 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:23.259 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.933701+0000", 2024-09-14T23:58:23.259 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2870000000000001e-06 2024-09-14T23:58:23.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.259 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.259 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:23.259 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.986165+0000", 2024-09-14T23:58:23.260 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052463900000000001 2024-09-14T23:58:23.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.260 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.260 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:23.260 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.986259+0000", 2024-09-14T23:58:23.260 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4435e-05 2024-09-14T23:58:23.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.260 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.260 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:23.260 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.986344+0000", 2024-09-14T23:58:23.260 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4562999999999997e-05 2024-09-14T23:58:23.260 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:23.260 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:23.261 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:23.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.261 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.261 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129201 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 10289152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:23.261 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.935960+0000", 2024-09-14T23:58:23.261 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 44.291102209000002, 2024-09-14T23:58:23.261 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054338487999999997, 2024-09-14T23:58:23.261 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:23.261 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:23.261 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:23.261 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:23.261 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:23.261 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:23.262 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129201 2024-09-14T23:58:23.262 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.262 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:23.262 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.262 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:23.262 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935960+0000", 2024-09-14T23:58:23.262 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:23.262 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.262 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.262 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:23.262 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935960+0000", 2024-09-14T23:58:23.262 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:23.262 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.262 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.263 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:23.263 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935960+0000", 2024-09-14T23:58:23.263 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.72e-07 2024-09-14T23:58:23.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.263 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:23.263 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935963+0000", 2024-09-14T23:58:23.263 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5390000000000002e-06 2024-09-14T23:58:23.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.263 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:23.263 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935971+0000", 2024-09-14T23:58:23.263 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4589999999999998e-06 2024-09-14T23:58:23.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.264 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.264 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:23.264 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935977+0000", 2024-09-14T23:58:23.264 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4110000000000004e-06 2024-09-14T23:58:23.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.264 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.264 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:23.264 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990213+0000", 2024-09-14T23:58:23.264 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054236642000000002 2024-09-14T23:58:23.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.264 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.265 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:23.265 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990257+0000", 2024-09-14T23:58:23.265 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3696000000000003e-05 2024-09-14T23:58:23.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.265 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:23.265 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990298+0000", 2024-09-14T23:58:23.265 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1369000000000003e-05 2024-09-14T23:58:23.265 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:23.265 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:23.265 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:23.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.266 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.266 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129202 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 11370496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:23.266 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.935986+0000", 2024-09-14T23:58:23.266 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 44.291075591000002, 2024-09-14T23:58:23.266 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054543700000000001, 2024-09-14T23:58:23.266 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:23.266 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:23.266 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:23.266 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:23.266 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:23.266 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:23.266 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129202 2024-09-14T23:58:23.266 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.267 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:23.267 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.267 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:23.267 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935986+0000", 2024-09-14T23:58:23.267 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:23.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.267 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.267 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:23.267 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935986+0000", 2024-09-14T23:58:23.267 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:23.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.267 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.268 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:23.268 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937678+0000", 2024-09-14T23:58:23.268 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016917849999999999 2024-09-14T23:58:23.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.268 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.268 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:23.268 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937682+0000", 2024-09-14T23:58:23.268 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2400000000000001e-06 2024-09-14T23:58:23.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.268 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.268 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:23.268 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937695+0000", 2024-09-14T23:58:23.269 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2999e-05 2024-09-14T23:58:23.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.269 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.269 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:23.269 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937700+0000", 2024-09-14T23:58:23.269 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8420000000000004e-06 2024-09-14T23:58:23.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.269 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.269 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:23.269 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990336+0000", 2024-09-14T23:58:23.269 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052636060999999998 2024-09-14T23:58:23.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.270 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.270 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:23.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990435+0000", 2024-09-14T23:58:23.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9015e-05 2024-09-14T23:58:23.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.270 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.270 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:23.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990530+0000", 2024-09-14T23:58:23.271 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4758000000000003e-05 2024-09-14T23:58:23.271 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:23.271 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:23.271 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:23.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.271 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.271 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129214 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 11468800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:23.271 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.937939+0000", 2024-09-14T23:58:23.271 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 44.289122677000002, 2024-09-14T23:58:23.271 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054463733, 2024-09-14T23:58:23.271 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:23.271 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:23.272 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:23.272 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:23.272 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:23.272 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:23.272 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129214 2024-09-14T23:58:23.272 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.272 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:23.272 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.272 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:23.272 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937939+0000", 2024-09-14T23:58:23.272 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:23.272 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.272 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.273 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:23.273 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937939+0000", 2024-09-14T23:58:23.273 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:23.273 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.273 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.273 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:23.273 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939674+0000", 2024-09-14T23:58:23.273 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017342589999999999 2024-09-14T23:58:23.273 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.273 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.273 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:23.273 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939678+0000", 2024-09-14T23:58:23.273 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2590000000000002e-06 2024-09-14T23:58:23.273 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.274 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:23.274 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939699+0000", 2024-09-14T23:58:23.274 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1143000000000001e-05 2024-09-14T23:58:23.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.274 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:23.274 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939706+0000", 2024-09-14T23:58:23.274 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0269999999999998e-06 2024-09-14T23:58:23.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.274 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:23.274 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.992221+0000", 2024-09-14T23:58:23.274 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052514981000000002 2024-09-14T23:58:23.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.275 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.275 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:23.275 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.992321+0000", 2024-09-14T23:58:23.275 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9625999999999998e-05 2024-09-14T23:58:23.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.275 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.275 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:23.275 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.992403+0000", 2024-09-14T23:58:23.275 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2437999999999999e-05 2024-09-14T23:58:23.275 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:23.275 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:23.275 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:23.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.276 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.276 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129226 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 14090240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:23.276 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.939867+0000", 2024-09-14T23:58:23.276 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 44.287194749000001, 2024-09-14T23:58:23.276 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054523573999999998, 2024-09-14T23:58:23.276 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:23.276 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:23.276 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:23.276 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:23.276 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:23.276 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:23.276 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129226 2024-09-14T23:58:23.276 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.277 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:23.277 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.277 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:23.277 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939867+0000", 2024-09-14T23:58:23.277 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:23.277 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.277 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.277 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:23.277 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939867+0000", 2024-09-14T23:58:23.277 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:23.277 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.277 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.277 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:23.278 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941669+0000", 2024-09-14T23:58:23.278 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018021549999999999 2024-09-14T23:58:23.278 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.278 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.278 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:23.278 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941673+0000", 2024-09-14T23:58:23.278 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1269999999999999e-06 2024-09-14T23:58:23.278 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.278 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.278 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:23.278 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941679+0000", 2024-09-14T23:58:23.278 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1879999999999999e-06 2024-09-14T23:58:23.278 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.278 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.279 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:23.279 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941682+0000", 2024-09-14T23:58:23.279 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7110000000000001e-06 2024-09-14T23:58:23.279 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.279 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.279 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:23.279 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.994201+0000", 2024-09-14T23:58:23.279 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052518265000000001 2024-09-14T23:58:23.279 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.279 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.279 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:23.279 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.994302+0000", 2024-09-14T23:58:23.279 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101653 2024-09-14T23:58:23.280 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.280 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.280 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:23.280 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.994391+0000", 2024-09-14T23:58:23.280 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8474999999999996e-05 2024-09-14T23:58:23.280 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:23.280 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:23.280 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:23.280 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.280 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.280 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129237 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 12386304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:23.280 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.941825+0000", 2024-09-14T23:58:23.280 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 44.285236986999998, 2024-09-14T23:58:23.280 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054368076000000001, 2024-09-14T23:58:23.281 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:23.281 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:23.281 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:23.281 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:23.281 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:23.281 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:23.281 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129237 2024-09-14T23:58:23.281 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.281 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:23.281 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.281 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:23.281 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941825+0000", 2024-09-14T23:58:23.281 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:23.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.282 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.282 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:23.282 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941825+0000", 2024-09-14T23:58:23.282 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:23.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.282 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.282 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:23.282 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941825+0000", 2024-09-14T23:58:23.282 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.29e-07 2024-09-14T23:58:23.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.282 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.282 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:23.283 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941826+0000", 2024-09-14T23:58:23.283 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0999999999999997e-07 2024-09-14T23:58:23.283 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.283 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.283 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:23.283 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941834+0000", 2024-09-14T23:58:23.283 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5519999999999999e-06 2024-09-14T23:58:23.283 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.283 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.283 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:23.283 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941843+0000", 2024-09-14T23:58:23.283 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2960000000000004e-06 2024-09-14T23:58:23.283 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.283 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.284 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:23.284 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996108+0000", 2024-09-14T23:58:23.284 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054265200999999999 2024-09-14T23:58:23.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.284 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.284 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:23.284 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996152+0000", 2024-09-14T23:58:23.284 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4066999999999997e-05 2024-09-14T23:58:23.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.284 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.284 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:23.284 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996193+0000", 2024-09-14T23:58:23.284 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0821000000000001e-05 2024-09-14T23:58:23.285 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:23.285 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:23.285 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:23.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.285 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.285 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129238 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 13828096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:23.285 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.941851+0000", 2024-09-14T23:58:23.285 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 44.285210663000001, 2024-09-14T23:58:23.285 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.05458031, 2024-09-14T23:58:23.285 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:23.285 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:23.285 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:23.286 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:23.286 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:23.286 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:23.286 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129238 2024-09-14T23:58:23.286 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.286 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:23.286 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.286 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:23.286 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941851+0000", 2024-09-14T23:58:23.286 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:23.286 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.287 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.287 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:23.287 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941851+0000", 2024-09-14T23:58:23.287 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:23.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.287 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.287 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:23.287 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943670+0000", 2024-09-14T23:58:23.287 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018181930000000001 2024-09-14T23:58:23.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.287 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.287 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:23.287 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943673+0000", 2024-09-14T23:58:23.287 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3790000000000002e-06 2024-09-14T23:58:23.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.288 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.288 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:23.288 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943679+0000", 2024-09-14T23:58:23.288 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8719999999999999e-06 2024-09-14T23:58:23.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.288 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.288 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:23.288 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943683+0000", 2024-09-14T23:58:23.288 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8539999999999999e-06 2024-09-14T23:58:23.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.288 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.288 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:23.288 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996233+0000", 2024-09-14T23:58:23.289 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052550690999999997 2024-09-14T23:58:23.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.289 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.289 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:23.289 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996339+0000", 2024-09-14T23:58:23.289 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105989 2024-09-14T23:58:23.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.289 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.289 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:23.289 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996432+0000", 2024-09-14T23:58:23.289 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2331999999999996e-05 2024-09-14T23:58:23.289 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:23.289 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:23.290 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:23.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.290 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.290 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129249 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 12124160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:23.290 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.943826+0000", 2024-09-14T23:58:23.290 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 44.283236172000002, 2024-09-14T23:58:23.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054373266000000003, 2024-09-14T23:58:23.290 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:23.290 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:23.290 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:23.290 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:23.290 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:23.290 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:23.291 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129249 2024-09-14T23:58:23.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.291 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:23.291 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.291 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:23.291 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943826+0000", 2024-09-14T23:58:23.291 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:23.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.291 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.291 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:23.291 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943826+0000", 2024-09-14T23:58:23.291 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:23.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.291 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.292 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:23.292 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943826+0000", 2024-09-14T23:58:23.292 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.03e-07 2024-09-14T23:58:23.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.292 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.292 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:23.292 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943827+0000", 2024-09-14T23:58:23.292 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.08e-06 2024-09-14T23:58:23.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.292 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.292 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:23.292 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943834+0000", 2024-09-14T23:58:23.292 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4119999999999998e-06 2024-09-14T23:58:23.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.293 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.293 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:23.293 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943836+0000", 2024-09-14T23:58:23.293 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7369999999999998e-06 2024-09-14T23:58:23.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.293 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.293 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:23.293 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998115+0000", 2024-09-14T23:58:23.293 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054279040000000001 2024-09-14T23:58:23.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.293 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.293 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:23.293 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998159+0000", 2024-09-14T23:58:23.294 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3486999999999999e-05 2024-09-14T23:58:23.294 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.294 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.294 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:23.294 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998199+0000", 2024-09-14T23:58:23.294 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0306999999999999e-05 2024-09-14T23:58:23.294 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:23.294 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:23.294 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:23.294 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.294 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.294 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129250 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 13565952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:23.295 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.943840+0000", 2024-09-14T23:58:23.295 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 44.283222221999999, 2024-09-14T23:58:23.295 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054573875000000001, 2024-09-14T23:58:23.295 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:23.295 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:23.295 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:23.295 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:23.295 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:23.295 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:23.295 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129250 2024-09-14T23:58:23.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.295 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:23.295 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.295 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:23.296 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943840+0000", 2024-09-14T23:58:23.296 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:23.296 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.296 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.296 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:23.296 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943840+0000", 2024-09-14T23:58:23.296 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:23.296 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.296 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.296 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:23.296 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945669+0000", 2024-09-14T23:58:23.296 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018289720000000001 2024-09-14T23:58:23.296 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.296 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.297 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:23.297 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945671+0000", 2024-09-14T23:58:23.297 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6869999999999999e-06 2024-09-14T23:58:23.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.297 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.297 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:23.297 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945680+0000", 2024-09-14T23:58:23.297 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5259999999999993e-06 2024-09-14T23:58:23.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.297 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.297 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:23.297 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945685+0000", 2024-09-14T23:58:23.297 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8559999999999998e-06 2024-09-14T23:58:23.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.298 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.298 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:23.298 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998242+0000", 2024-09-14T23:58:23.298 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052557505999999997 2024-09-14T23:58:23.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.298 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.298 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:23.298 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998329+0000", 2024-09-14T23:58:23.298 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6410999999999996e-05 2024-09-14T23:58:23.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.298 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.298 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:23.299 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998414+0000", 2024-09-14T23:58:23.299 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4917e-05 2024-09-14T23:58:23.299 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:23.299 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:23.299 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:23.299 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.299 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.299 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129261 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 14385152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:23.299 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.945815+0000", 2024-09-14T23:58:23.299 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 44.281246928999998, 2024-09-14T23:58:23.299 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054331867999999998, 2024-09-14T23:58:23.299 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:23.299 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:23.299 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:23.300 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:23.300 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:23.300 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:23.300 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129261 2024-09-14T23:58:23.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.300 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:23.300 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.300 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:23.300 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945815+0000", 2024-09-14T23:58:23.300 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:23.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.300 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.300 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:23.300 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945815+0000", 2024-09-14T23:58:23.301 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:23.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.301 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.301 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:23.301 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945815+0000", 2024-09-14T23:58:23.301 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0599999999999999e-07 2024-09-14T23:58:23.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.301 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.301 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:23.301 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945816+0000", 2024-09-14T23:58:23.301 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0380000000000001e-06 2024-09-14T23:58:23.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.301 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.301 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:23.302 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945819+0000", 2024-09-14T23:58:23.302 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.005e-06 2024-09-14T23:58:23.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.302 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.302 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:23.302 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945822+0000", 2024-09-14T23:58:23.302 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.452e-06 2024-09-14T23:58:23.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.302 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.302 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:23.302 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000060+0000", 2024-09-14T23:58:23.302 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054238606000000002 2024-09-14T23:58:23.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.302 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.303 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:23.303 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000106+0000", 2024-09-14T23:58:23.303 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5188000000000002e-05 2024-09-14T23:58:23.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.303 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:23.303 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000147+0000", 2024-09-14T23:58:23.303 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1372999999999998e-05 2024-09-14T23:58:23.303 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:23.303 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:23.303 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:23.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.303 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129262 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 13303808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:23.304 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.945825+0000", 2024-09-14T23:58:23.304 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 44.281237281000003, 2024-09-14T23:58:23.304 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054515942999999997, 2024-09-14T23:58:23.304 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:23.304 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:23.304 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:23.304 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:23.304 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:23.304 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:23.304 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129262 2024-09-14T23:58:23.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.304 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:23.304 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.304 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:23.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945825+0000", 2024-09-14T23:58:23.305 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:23.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.305 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:23.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945825+0000", 2024-09-14T23:58:23.305 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:23.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.305 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:23.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.947670+0000", 2024-09-14T23:58:23.305 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018451229999999999 2024-09-14T23:58:23.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.306 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:23.306 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.947672+0000", 2024-09-14T23:58:23.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5289999999999998e-06 2024-09-14T23:58:23.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.306 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.306 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:23.306 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.947678+0000", 2024-09-14T23:58:23.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9880000000000001e-06 2024-09-14T23:58:23.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.306 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.306 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:23.306 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.947682+0000", 2024-09-14T23:58:23.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.63e-06 2024-09-14T23:58:23.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.306 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.307 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:23.307 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000183+0000", 2024-09-14T23:58:23.307 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052500770000000002 2024-09-14T23:58:23.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.307 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.307 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:23.307 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000266+0000", 2024-09-14T23:58:23.307 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3133000000000005e-05 2024-09-14T23:58:23.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.307 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.307 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:23.307 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000341+0000", 2024-09-14T23:58:23.307 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4770000000000004e-05 2024-09-14T23:58:23.307 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:23.308 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:23.308 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:23.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.308 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.308 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129299 3.10 3:0f1e06ee:::rbd_object_map.1118721be852:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:23.308 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.952330+0000", 2024-09-14T23:58:23.308 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 44.274732468000003, 2024-09-14T23:58:23.308 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054412818000000002, 2024-09-14T23:58:23.308 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:23.308 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:23.308 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:23.308 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:23.308 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:23.308 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:23.308 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129299 2024-09-14T23:58:23.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.309 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:23.309 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.309 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:23.309 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952330+0000", 2024-09-14T23:58:23.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:23.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.309 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.309 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:23.309 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952330+0000", 2024-09-14T23:58:23.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:23.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.309 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.309 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:23.310 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952330+0000", 2024-09-14T23:58:23.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4900000000000001e-07 2024-09-14T23:58:23.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.310 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:23.310 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952332+0000", 2024-09-14T23:58:23.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2879999999999999e-06 2024-09-14T23:58:23.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.310 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:23.310 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952340+0000", 2024-09-14T23:58:23.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9910000000000002e-06 2024-09-14T23:58:23.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.310 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:23.311 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952344+0000", 2024-09-14T23:58:23.311 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0280000000000002e-06 2024-09-14T23:58:23.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.311 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.311 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:23.311 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.004722+0000", 2024-09-14T23:58:23.311 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052377932000000002 2024-09-14T23:58:23.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.311 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.311 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:23.311 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.004778+0000", 2024-09-14T23:58:23.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5795999999999999e-05 2024-09-14T23:58:23.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.312 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:58:23.312 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.005016+0000", 2024-09-14T23:58:23.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000238006 2024-09-14T23:58:23.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.312 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:58:23.312 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.006366+0000", 2024-09-14T23:58:23.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001349786 2024-09-14T23:58:23.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.313 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:23.313 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.006685+0000", 2024-09-14T23:58:23.313 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00031970000000000002 2024-09-14T23:58:23.313 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.313 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.313 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:23.313 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.006718+0000", 2024-09-14T23:58:23.313 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2329000000000002e-05 2024-09-14T23:58:23.313 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.313 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.313 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:23.313 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.006742+0000", 2024-09-14T23:58:23.314 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4613e-05 2024-09-14T23:58:23.314 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:23.314 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:23.314 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:23.314 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.314 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.314 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186210 3.d 3.b2ff3acd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:23.314 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.129044+0000", 2024-09-14T23:58:23.314 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 16.098018105000001, 2024-09-14T23:58:23.314 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.84555443900000005, 2024-09-14T23:58:23.314 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:23.314 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:23.314 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:58:23.315 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:23.315 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:23.315 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:23.315 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186210 2024-09-14T23:58:23.315 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.315 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:23.315 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.315 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:23.315 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129044+0000", 2024-09-14T23:58:23.315 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:23.315 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.315 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.316 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:23.316 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129044+0000", 2024-09-14T23:58:23.316 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:23.316 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.316 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.316 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:23.316 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129046+0000", 2024-09-14T23:58:23.316 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.742e-06 2024-09-14T23:58:23.316 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.316 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.316 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:23.316 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129052+0000", 2024-09-14T23:58:23.316 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7390000000000002e-06 2024-09-14T23:58:23.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.317 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.317 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:23.317 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129067+0000", 2024-09-14T23:58:23.317 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.417e-05 2024-09-14T23:58:23.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.317 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.317 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:23.317 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129076+0000", 2024-09-14T23:58:23.317 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2289999999999992e-06 2024-09-14T23:58:23.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.317 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.317 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:23.317 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.130947+0000", 2024-09-14T23:58:23.318 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001871075 2024-09-14T23:58:23.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.318 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.318 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:23.318 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.130955+0000", 2024-09-14T23:58:23.318 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4500000000000004e-06 2024-09-14T23:58:23.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.318 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.318 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:23.318 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974587+0000", 2024-09-14T23:58:23.318 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.84363205500000005 2024-09-14T23:58:23.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.319 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.319 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:23.319 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974598+0000", 2024-09-14T23:58:23.319 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0978999999999999e-05 2024-09-14T23:58:23.319 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:23.319 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:23.319 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:23.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.319 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.319 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187415 3.d 3.b2ff3acd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:23.319 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.254681+0000", 2024-09-14T23:58:23.319 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 15.972380584, 2024-09-14T23:58:23.319 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.71995530699999999, 2024-09-14T23:58:23.320 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:23.320 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:23.320 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:58:23.320 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:23.320 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:23.320 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:23.320 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187415 2024-09-14T23:58:23.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.320 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:23.320 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.320 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:23.320 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254681+0000", 2024-09-14T23:58:23.320 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:23.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.320 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.321 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:23.321 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254681+0000", 2024-09-14T23:58:23.321 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:23.321 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.321 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.321 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:23.321 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254682+0000", 2024-09-14T23:58:23.321 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2e-07 2024-09-14T23:58:23.321 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.321 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.321 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:23.321 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254684+0000", 2024-09-14T23:58:23.322 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2730000000000001e-06 2024-09-14T23:58:23.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.322 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.322 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:23.322 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254689+0000", 2024-09-14T23:58:23.322 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4769999999999997e-06 2024-09-14T23:58:23.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.322 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.322 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:23.322 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254692+0000", 2024-09-14T23:58:23.322 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9890000000000001e-06 2024-09-14T23:58:23.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.322 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.323 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:23.323 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254721+0000", 2024-09-14T23:58:23.323 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9762999999999999e-05 2024-09-14T23:58:23.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.323 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.323 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:23.323 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254725+0000", 2024-09-14T23:58:23.323 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9650000000000002e-06 2024-09-14T23:58:23.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.323 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.323 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:23.323 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974628+0000", 2024-09-14T23:58:23.323 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.71990235499999999 2024-09-14T23:58:23.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.323 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.324 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:23.324 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974637+0000", 2024-09-14T23:58:23.324 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0650000000000005e-06 2024-09-14T23:58:23.324 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:23.324 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:23.324 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:23.324 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.324 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.324 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187457 3.d 3.b2ff3acd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:23.324 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.257980+0000", 2024-09-14T23:58:23.324 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 15.969082009999999, 2024-09-14T23:58:23.324 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.71671711199999999, 2024-09-14T23:58:23.324 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:23.325 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:23.325 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:58:23.325 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:23.325 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:23.325 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:23.325 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187457 2024-09-14T23:58:23.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.325 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:23.325 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.325 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:23.325 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257980+0000", 2024-09-14T23:58:23.325 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:23.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.326 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.326 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:23.326 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257980+0000", 2024-09-14T23:58:23.326 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:23.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.326 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.326 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:23.326 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257980+0000", 2024-09-14T23:58:23.326 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.17e-07 2024-09-14T23:58:23.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.326 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.326 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:23.326 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257981+0000", 2024-09-14T23:58:23.326 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.172e-06 2024-09-14T23:58:23.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.327 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.327 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:23.327 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257984+0000", 2024-09-14T23:58:23.327 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9550000000000001e-06 2024-09-14T23:58:23.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.327 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.327 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:23.327 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257987+0000", 2024-09-14T23:58:23.327 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5799999999999999e-06 2024-09-14T23:58:23.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.327 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.327 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:23.327 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.258030+0000", 2024-09-14T23:58:23.328 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3118999999999998e-05 2024-09-14T23:58:23.328 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.328 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.328 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:23.328 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.258038+0000", 2024-09-14T23:58:23.328 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6850000000000001e-06 2024-09-14T23:58:23.328 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.328 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.328 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:23.328 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974688+0000", 2024-09-14T23:58:23.328 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.71665016000000004 2024-09-14T23:58:23.328 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:23.328 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:23.328 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:23.329 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974697+0000", 2024-09-14T23:58:23.329 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2240000000000001e-06 2024-09-14T23:58:23.329 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:23.329 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:23.329 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:23.329 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:23.329 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:23.329 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:23.331 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is set 2024-09-14T23:58:23.331 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T23:58:23.332 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:23.329+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:23.337 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 5] out_osds: [7] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T23:58:23.337 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T23:58:23.337 INFO:tasks.thrashosds.thrasher:Removing osd 3, in_osds are: [0, 1, 2, 3, 4, 6, 5] 2024-09-14T23:58:23.338 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 3 2024-09-14T23:58:23.431 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:58:23.430+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:58:23.469 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T23:58:23.481 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:23.481 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:58:23.481 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:58:23.481 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:23.489 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T23:58:23.532 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:23.531+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:23.595 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:58:23.595 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:58:23.612 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:58:23.612 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:58:23.628 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:58:23.629 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:58:23.633 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:23.632+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:23.637 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:23.637 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:58:23.637 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:58:23.637 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:58:23.637 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:23.646 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T23:58:23.646 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:58:23.646 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:58:23.662 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:58:23.662 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:58:23.679 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:58:23.679 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:58:23.681 INFO:tasks.ceph.mon.a.smithi042.stderr:2024-09-14T23:58:23.680+0000 7f1713676640 -1 mon.a@0(leader).osd e123 definitely_dead 0 2024-09-14T23:58:23.695 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:58:23.695 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:58:23.712 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:58:23.713 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:58:23.734 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:23.733+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:23.793 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:23.793 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:23.801 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T23:58:23.842 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:23.841+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:23.909 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:23.909 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:23.918 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T23:58:23.935 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:23.934+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:24.025 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:24.025 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:24.034 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T23:58:24.036 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:58:24.034+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:58:24.137 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:24.135+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:24.140 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:24.140 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:24.149 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T23:58:24.237 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:24.236+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:24.292 INFO:teuthology.orchestra.run.smithi042.stderr:marked out osd.3. 2024-09-14T23:58:24.305 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:24.305 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:24.317 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T23:58:24.338 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:24.337+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:24.426 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:24.426 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:24.435 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T23:58:24.439 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:24.437+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:24.539 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:24.538+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:24.542 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:24.542 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:24.551 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T23:58:24.640 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:24.639+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:24.698 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:24.699 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:24.707 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T23:58:24.750 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:24.749+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:24.814 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:24.814 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:24.823 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T23:58:24.863 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:24.862+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:24.943 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:24.942+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:24.970 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:24.970 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:24.978 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T23:58:25.044 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:25.043+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:25.086 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:25.086 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:25.095 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T23:58:25.145 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:25.144+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:25.202 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:25.202 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:25.211 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T23:58:25.247 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:25.245+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:25.333 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T23:58:25.364 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:25.365 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:25.365 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:25.363+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:25.374 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T23:58:25.448 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:25.446+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:25.494 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:25.494 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:25.503 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T23:58:25.549 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:25.548+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:25.650 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:25.649+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:25.652 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:25.652 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:25.661 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T23:58:25.724 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T23:58:25.751 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:58:25.750+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:58:25.809 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:25.809 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:25.818 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T23:58:25.849 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:58:25.849 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:58:25.859 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:58:25.858+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:58:25.866 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:58:25.866 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:58:25.882 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:58:25.882 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:58:25.898 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:58:25.898 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:58:25.915 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:58:25.915 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:58:25.932 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:58:25.932 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:58:25.948 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:58:25.949 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:58:25.953 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:58:25.952+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:58:25.965 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:58:25.965 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:58:25.966 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:25.967 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:58:25.967 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:58:25.967 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:25.975 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T23:58:26.059 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:26.058+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:26.123 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:26.123 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:58:26.123 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:58:26.123 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:58:26.123 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:26.131 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T23:58:26.154 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:26.153+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:26.255 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:26.253+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:26.280 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:26.280 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:58:26.281 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:58:26.281 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:58:26.281 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.281 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4284.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-14T23:58:26.281 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:58:26.281 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 123.753998637, 2024-09-14T23:58:26.281 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.004942086, 2024-09-14T23:58:26.281 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:26.281 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:26.281 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:26.281 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:26.281 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4284", 2024-09-14T23:58:26.281 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/4278347926", 2024-09-14T23:58:26.281 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1 2024-09-14T23:58:26.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.282 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:26.282 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.282 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:26.282 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:58:26.282 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.282 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.282 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:26.282 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:58:26.282 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.283 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.283 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:26.283 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524331+0000", 2024-09-14T23:58:26.283 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.383e-05 2024-09-14T23:58:26.283 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.283 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.283 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:26.283 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524339+0000", 2024-09-14T23:58:26.283 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8290000000000008e-06 2024-09-14T23:58:26.283 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.283 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.283 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:26.283 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524359+0000", 2024-09-14T23:58:26.283 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9998000000000001e-05 2024-09-14T23:58:26.283 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.284 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.284 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:26.284 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524373+0000", 2024-09-14T23:58:26.284 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4304000000000001e-05 2024-09-14T23:58:26.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.284 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.284 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:26.284 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524488+0000", 2024-09-14T23:58:26.284 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114663 2024-09-14T23:58:26.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.284 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.284 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:26.284 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524501+0000", 2024-09-14T23:58:26.284 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3349999999999999e-05 2024-09-14T23:58:26.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.285 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.285 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:26.285 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.526868+0000", 2024-09-14T23:58:26.285 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023670090000000002 2024-09-14T23:58:26.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.285 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.285 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:26.285 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.527041+0000", 2024-09-14T23:58:26.285 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000172597 2024-09-14T23:58:26.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.285 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.285 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-14T23:58:26.285 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.527384+0000", 2024-09-14T23:58:26.286 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00034309500000000002 2024-09-14T23:58:26.286 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.286 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.286 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:26.286 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529103+0000", 2024-09-14T23:58:26.286 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017185220000000001 2024-09-14T23:58:26.286 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.286 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.286 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:58:26.286 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529188+0000", 2024-09-14T23:58:26.286 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5591000000000001e-05 2024-09-14T23:58:26.286 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.286 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.287 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:26.287 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529227+0000", 2024-09-14T23:58:26.287 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8563000000000003e-05 2024-09-14T23:58:26.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.287 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.287 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:26.287 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529259+0000", 2024-09-14T23:58:26.287 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2734999999999997e-05 2024-09-14T23:58:26.287 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.287 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:26.287 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.287 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.287 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4284.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-14T23:58:26.287 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:23.530683+0000", 2024-09-14T23:58:26.288 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 122.74763297200001, 2024-09-14T23:58:26.288 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0036133110000000001, 2024-09-14T23:58:26.288 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:26.288 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:26.288 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:26.288 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:26.288 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4284", 2024-09-14T23:58:26.288 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/4278347926", 2024-09-14T23:58:26.288 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5 2024-09-14T23:58:26.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.288 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:26.288 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.288 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:26.288 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530683+0000", 2024-09-14T23:58:26.289 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.289 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.289 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:26.289 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530683+0000", 2024-09-14T23:58:26.289 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.289 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.289 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:26.289 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530685+0000", 2024-09-14T23:58:26.289 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.075e-06 2024-09-14T23:58:26.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.289 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.289 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:26.290 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530695+0000", 2024-09-14T23:58:26.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0261999999999999e-05 2024-09-14T23:58:26.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.290 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.290 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:26.290 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530715+0000", 2024-09-14T23:58:26.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9930000000000001e-05 2024-09-14T23:58:26.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.290 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.290 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:26.290 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530724+0000", 2024-09-14T23:58:26.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.901e-06 2024-09-14T23:58:26.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.290 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.290 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:26.291 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530800+0000", 2024-09-14T23:58:26.291 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5873999999999999e-05 2024-09-14T23:58:26.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.291 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.291 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-14T23:58:26.291 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530807+0000", 2024-09-14T23:58:26.291 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2389999999999996e-06 2024-09-14T23:58:26.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.291 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.291 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:26.291 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.532494+0000", 2024-09-14T23:58:26.291 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016871830000000001 2024-09-14T23:58:26.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.291 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.292 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:26.292 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.532599+0000", 2024-09-14T23:58:26.292 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010472 2024-09-14T23:58:26.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.292 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.292 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1", 2024-09-14T23:58:26.292 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.532715+0000", 2024-09-14T23:58:26.292 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011568199999999999 2024-09-14T23:58:26.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.293 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.293 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:58:26.293 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.533460+0000", 2024-09-14T23:58:26.293 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00074494999999999997 2024-09-14T23:58:26.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.293 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.293 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:26.293 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534230+0000", 2024-09-14T23:58:26.293 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00077036599999999998 2024-09-14T23:58:26.294 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.294 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.294 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:26.294 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534266+0000", 2024-09-14T23:58:26.294 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5725000000000001e-05 2024-09-14T23:58:26.294 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.294 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.295 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:26.295 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534296+0000", 2024-09-14T23:58:26.295 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0403999999999999e-05 2024-09-14T23:58:26.295 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.295 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:26.295 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.295 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.295 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56383 3.c e56/49 3:36b698b0:::rbd_data.1118c4b3aebe.0000000000000000:head v 56'194, mlcod=56'192)", 2024-09-14T23:58:26.295 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.270483+0000", 2024-09-14T23:58:26.295 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 80.007833172000005, 2024-09-14T23:58:26.295 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0079685880000000004, 2024-09-14T23:58:26.295 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:26.295 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:26.296 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:26.296 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:26.296 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.296 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:26.296 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.270483+0000", 2024-09-14T23:58:26.296 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.296 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.296 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.296 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:26.296 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.270483+0000", 2024-09-14T23:58:26.296 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.296 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.296 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.296 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:26.297 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.270484+0000", 2024-09-14T23:58:26.297 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7400000000000002e-07 2024-09-14T23:58:26.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.297 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.297 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:26.297 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273680+0000", 2024-09-14T23:58:26.297 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031966680000000002 2024-09-14T23:58:26.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.297 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.297 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:26.297 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273697+0000", 2024-09-14T23:58:26.297 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6595e-05 2024-09-14T23:58:26.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.298 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.298 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:26.298 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273705+0000", 2024-09-14T23:58:26.298 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7530000000000001e-06 2024-09-14T23:58:26.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.298 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.298 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:26.298 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273763+0000", 2024-09-14T23:58:26.298 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8238e-05 2024-09-14T23:58:26.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.299 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.299 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:26.299 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273781+0000", 2024-09-14T23:58:26.299 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8281e-05 2024-09-14T23:58:26.299 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.299 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.299 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:26.299 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.278408+0000", 2024-09-14T23:58:26.299 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00462716 2024-09-14T23:58:26.299 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.299 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.299 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:26.299 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.278451+0000", 2024-09-14T23:58:26.299 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3019000000000003e-05 2024-09-14T23:58:26.300 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.300 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:26.300 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.300 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.300 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56384 3.c e56/49 3:36b698b0:::rbd_data.1118c4b3aebe.0000000000000000:head v 56'196, mlcod=56'192)", 2024-09-14T23:58:26.300 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:58:26.300 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 80.003800499999997, 2024-09-14T23:58:26.300 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0046707529999999997, 2024-09-14T23:58:26.300 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:26.300 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:26.300 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:26.300 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:26.300 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.301 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:26.301 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:58:26.301 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.301 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.301 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:26.301 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:58:26.301 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.301 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.301 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:26.301 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:58:26.301 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1699999999999997e-07 2024-09-14T23:58:26.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.302 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.302 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:26.302 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274523+0000", 2024-09-14T23:58:26.302 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9129999999999997e-06 2024-09-14T23:58:26.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.302 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.302 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:26.302 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274535+0000", 2024-09-14T23:58:26.302 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1788e-05 2024-09-14T23:58:26.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.302 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.302 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:26.302 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274542+0000", 2024-09-14T23:58:26.302 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0539999999999997e-06 2024-09-14T23:58:26.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.303 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:26.303 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275889+0000", 2024-09-14T23:58:26.303 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00134669 2024-09-14T23:58:26.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.303 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:26.303 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275917+0000", 2024-09-14T23:58:26.303 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7664000000000001e-05 2024-09-14T23:58:26.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.303 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:26.303 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279134+0000", 2024-09-14T23:58:26.304 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0032178459999999999 2024-09-14T23:58:26.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.304 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.304 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:26.304 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279186+0000", 2024-09-14T23:58:26.304 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1881e-05 2024-09-14T23:58:26.304 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.304 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:26.304 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.304 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.304 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56381 3.1d e56/49 3:bcf9a8cf:::rbd_data.1118c4b3aebe.0000000000000006:head v 56'210, mlcod=56'208)", 2024-09-14T23:58:26.304 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.275462+0000", 2024-09-14T23:58:26.304 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 80.002854170000006, 2024-09-14T23:58:26.305 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0071310710000000001, 2024-09-14T23:58:26.305 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:26.305 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:26.305 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:26.305 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:26.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.305 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:26.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275462+0000", 2024-09-14T23:58:26.305 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.305 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:26.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275462+0000", 2024-09-14T23:58:26.305 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.306 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.306 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:26.306 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275463+0000", 2024-09-14T23:58:26.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2699999999999999e-07 2024-09-14T23:58:26.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.306 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.306 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:26.306 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276779+0000", 2024-09-14T23:58:26.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0013163649999999999 2024-09-14T23:58:26.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.306 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.306 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:26.306 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276795+0000", 2024-09-14T23:58:26.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.59e-05 2024-09-14T23:58:26.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.307 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.307 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:26.307 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276806+0000", 2024-09-14T23:58:26.307 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1106e-05 2024-09-14T23:58:26.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.307 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.307 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:26.307 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276941+0000", 2024-09-14T23:58:26.307 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013477500000000001 2024-09-14T23:58:26.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.307 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.307 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:26.307 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276963+0000", 2024-09-14T23:58:26.307 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2419999999999999e-05 2024-09-14T23:58:26.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.308 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.308 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:26.308 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282546+0000", 2024-09-14T23:58:26.308 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0055827100000000003 2024-09-14T23:58:26.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.308 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.308 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:26.308 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282593+0000", 2024-09-14T23:58:26.308 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7067999999999997e-05 2024-09-14T23:58:26.308 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.308 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:26.308 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.309 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.309 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56387 3.c e56/49 3:30ee4ba0:::rbd_data.1118c4b3aebe.0000000000000004:head v 56'197, mlcod=56'192)", 2024-09-14T23:58:26.309 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.276498+0000", 2024-09-14T23:58:26.309 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 80.001818075000003, 2024-09-14T23:58:26.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0092192639999999996, 2024-09-14T23:58:26.309 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:26.309 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:26.309 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:26.309 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:26.309 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.309 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:26.309 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276498+0000", 2024-09-14T23:58:26.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.310 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:26.310 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276498+0000", 2024-09-14T23:58:26.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.310 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:26.310 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276499+0000", 2024-09-14T23:58:26.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1100000000000004e-07 2024-09-14T23:58:26.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.310 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:26.310 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279646+0000", 2024-09-14T23:58:26.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031473220000000001 2024-09-14T23:58:26.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.311 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.311 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:26.311 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279672+0000", 2024-09-14T23:58:26.311 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6129999999999999e-05 2024-09-14T23:58:26.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.311 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.311 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:26.311 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279682+0000", 2024-09-14T23:58:26.311 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7170000000000003e-06 2024-09-14T23:58:26.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.311 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.311 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:26.311 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279742+0000", 2024-09-14T23:58:26.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9926999999999997e-05 2024-09-14T23:58:26.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.312 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:26.312 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279761+0000", 2024-09-14T23:58:26.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9402000000000002e-05 2024-09-14T23:58:26.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.312 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:26.312 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.285680+0000", 2024-09-14T23:58:26.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.005918881 2024-09-14T23:58:26.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.313 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.313 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:26.313 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.285717+0000", 2024-09-14T23:58:26.313 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6974000000000002e-05 2024-09-14T23:58:26.313 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.313 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:26.313 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.313 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.313 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.313 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56388 3.c e56/49 3:30ee4ba0:::rbd_data.1118c4b3aebe.0000000000000004:head v 56'199, mlcod=56'192)", 2024-09-14T23:58:26.313 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:58:26.313 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 79.998326487, 2024-09-14T23:58:26.313 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0060596740000000001, 2024-09-14T23:58:26.313 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:26.313 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:26.314 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:26.314 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:26.314 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.314 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:26.314 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:58:26.314 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.314 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.314 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.314 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:26.314 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:58:26.314 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.314 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.314 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.314 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:26.314 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:58:26.315 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2200000000000003e-07 2024-09-14T23:58:26.315 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.315 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.315 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:26.315 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279995+0000", 2024-09-14T23:58:26.315 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5569999999999997e-06 2024-09-14T23:58:26.315 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.315 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.315 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:26.315 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280004+0000", 2024-09-14T23:58:26.315 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0080000000000008e-06 2024-09-14T23:58:26.315 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.315 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.315 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:26.315 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280010+0000", 2024-09-14T23:58:26.316 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5749999999999997e-06 2024-09-14T23:58:26.316 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.316 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.316 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:26.316 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282968+0000", 2024-09-14T23:58:26.316 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0029579390000000001 2024-09-14T23:58:26.316 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.316 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.316 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:26.316 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282998+0000", 2024-09-14T23:58:26.316 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9947999999999999e-05 2024-09-14T23:58:26.316 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.316 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.316 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:26.317 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.286012+0000", 2024-09-14T23:58:26.317 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030132900000000001 2024-09-14T23:58:26.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.317 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.317 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:26.317 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.286049+0000", 2024-09-14T23:58:26.317 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7534999999999999e-05 2024-09-14T23:58:26.317 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.317 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:26.317 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.317 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.317 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56382 3.1d e56/49 3:bcf9a8cf:::rbd_data.1118c4b3aebe.0000000000000006:head v 56'212, mlcod=56'208)", 2024-09-14T23:58:26.317 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.280285+0000", 2024-09-14T23:58:26.318 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 79.998030803999995, 2024-09-14T23:58:26.318 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035393489999999998, 2024-09-14T23:58:26.318 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:26.318 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:26.318 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:26.318 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:26.318 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.318 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:26.318 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280285+0000", 2024-09-14T23:58:26.318 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.318 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.318 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:26.318 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280285+0000", 2024-09-14T23:58:26.319 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.319 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.319 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:26.319 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280286+0000", 2024-09-14T23:58:26.319 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5000000000000001e-07 2024-09-14T23:58:26.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.319 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.319 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:26.319 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280299+0000", 2024-09-14T23:58:26.319 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3312000000000001e-05 2024-09-14T23:58:26.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.319 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.319 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:26.319 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280311+0000", 2024-09-14T23:58:26.320 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1677e-05 2024-09-14T23:58:26.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.320 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.320 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:26.320 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280318+0000", 2024-09-14T23:58:26.320 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.742e-06 2024-09-14T23:58:26.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.320 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.320 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:26.320 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280374+0000", 2024-09-14T23:58:26.320 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.592e-05 2024-09-14T23:58:26.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.320 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.320 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:26.321 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280392+0000", 2024-09-14T23:58:26.321 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8490000000000001e-05 2024-09-14T23:58:26.321 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.321 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.321 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:26.321 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.283693+0000", 2024-09-14T23:58:26.321 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033012050000000002 2024-09-14T23:58:26.321 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.321 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.321 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:26.321 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.283825+0000", 2024-09-14T23:58:26.321 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013105300000000001 2024-09-14T23:58:26.321 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.321 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:26.321 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.322 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.322 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "pg_update_log_missing(3.c epoch 56/49 rep_tid 1493 entries 56'200 (0'0) error 3:370203a3:::rbd_data.1118c4b3aebe.000000000000000d:head by client.4376.0:56396 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 56'192)", 2024-09-14T23:58:26.322 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:58:26.322 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 79.997806346999994, 2024-09-14T23:58:26.322 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057370099999999999, 2024-09-14T23:58:26.322 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:26.322 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:26.322 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:58:26.322 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:26.322 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.322 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:26.322 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:58:26.322 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.323 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.323 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:26.323 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:58:26.323 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.323 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.323 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:26.323 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:58:26.323 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0699999999999997e-07 2024-09-14T23:58:26.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.323 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.323 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:26.323 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280519+0000", 2024-09-14T23:58:26.324 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6260000000000001e-06 2024-09-14T23:58:26.324 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.324 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.324 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:26.324 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280541+0000", 2024-09-14T23:58:26.324 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1976000000000001e-05 2024-09-14T23:58:26.324 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.324 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.324 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:26.324 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280556+0000", 2024-09-14T23:58:26.324 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5075000000000001e-05 2024-09-14T23:58:26.324 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.324 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.324 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:26.325 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.285494+0000", 2024-09-14T23:58:26.325 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0049378690000000001 2024-09-14T23:58:26.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.325 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.325 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:26.325 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.286247+0000", 2024-09-14T23:58:26.325 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00075285700000000001 2024-09-14T23:58:26.325 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.325 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:26.325 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.325 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.325 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56400 3.1c e56/49 3:3f5b81ba:::rbd_data.1118c4b3aebe.000000000000000a:head v 56'236, mlcod=56'234)", 2024-09-14T23:58:26.325 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.295064+0000", 2024-09-14T23:58:26.325 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 79.983251554999995, 2024-09-14T23:58:26.326 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0054331589999999999, 2024-09-14T23:58:26.326 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:26.326 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:26.326 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:26.326 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:26.326 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.326 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:26.326 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.295064+0000", 2024-09-14T23:58:26.326 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.326 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.326 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:26.326 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.295064+0000", 2024-09-14T23:58:26.326 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.327 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.327 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:26.327 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.295066+0000", 2024-09-14T23:58:26.327 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.502e-06 2024-09-14T23:58:26.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.327 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.327 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:26.327 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296403+0000", 2024-09-14T23:58:26.327 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001336554 2024-09-14T23:58:26.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.327 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.327 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:26.327 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296428+0000", 2024-09-14T23:58:26.328 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5179000000000001e-05 2024-09-14T23:58:26.328 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.328 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.328 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:26.328 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296440+0000", 2024-09-14T23:58:26.328 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2387e-05 2024-09-14T23:58:26.328 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.328 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.328 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:26.328 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296554+0000", 2024-09-14T23:58:26.328 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114269 2024-09-14T23:58:26.328 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.328 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.328 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:26.329 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296581+0000", 2024-09-14T23:58:26.329 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6673999999999999e-05 2024-09-14T23:58:26.329 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.329 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.329 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:26.329 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.300418+0000", 2024-09-14T23:58:26.329 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0038371249999999998 2024-09-14T23:58:26.329 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.329 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.329 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:26.329 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.300498+0000", 2024-09-14T23:58:26.329 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9468999999999994e-05 2024-09-14T23:58:26.329 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.329 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:26.329 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.330 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.330 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.330 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:61921 3.5 e57/49 3:a05d004f:::rbd_data.11188d356229.000000000000000b:head v 57'279, mlcod=57'277)", 2024-09-14T23:58:26.330 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:07.560389+0000", 2024-09-14T23:58:26.330 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 78.717926957000003, 2024-09-14T23:58:26.330 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0034956520000000001, 2024-09-14T23:58:26.330 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:26.330 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:26.330 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:26.330 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:26.330 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.330 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:26.330 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.560389+0000", 2024-09-14T23:58:26.330 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.330 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.331 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.331 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:26.331 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.560389+0000", 2024-09-14T23:58:26.331 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.331 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.331 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.331 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:26.331 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.560390+0000", 2024-09-14T23:58:26.331 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4109999999999999e-06 2024-09-14T23:58:26.331 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.331 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.331 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:26.331 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.561839+0000", 2024-09-14T23:58:26.331 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001448406 2024-09-14T23:58:26.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.332 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.332 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:26.332 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.561854+0000", 2024-09-14T23:58:26.332 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5426000000000001e-05 2024-09-14T23:58:26.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.332 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.332 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:26.332 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.561863+0000", 2024-09-14T23:58:26.332 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7509999999999997e-06 2024-09-14T23:58:26.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.332 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.332 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:26.332 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.561917+0000", 2024-09-14T23:58:26.332 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3531e-05 2024-09-14T23:58:26.333 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.333 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.333 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:26.333 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.561944+0000", 2024-09-14T23:58:26.333 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7528000000000001e-05 2024-09-14T23:58:26.333 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.333 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.333 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:26.333 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563843+0000", 2024-09-14T23:58:26.333 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001899361 2024-09-14T23:58:26.333 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.333 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.333 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:26.333 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563885+0000", 2024-09-14T23:58:26.334 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1238000000000001e-05 2024-09-14T23:58:26.334 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.334 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:26.334 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.334 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.334 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.334 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:61922 3.5 e57/49 3:a11ebf8e:::rbd_data.11188d356229.000000000000000c:head v 57'280, mlcod=57'277)", 2024-09-14T23:58:26.334 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:07.563144+0000", 2024-09-14T23:58:26.334 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 78.715171902999998, 2024-09-14T23:58:26.334 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003977848, 2024-09-14T23:58:26.334 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:26.334 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:26.334 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:26.334 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:26.334 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.335 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:26.335 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563144+0000", 2024-09-14T23:58:26.335 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.335 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.335 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:26.335 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563144+0000", 2024-09-14T23:58:26.335 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.335 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.335 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:26.335 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563145+0000", 2024-09-14T23:58:26.335 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5400000000000001e-07 2024-09-14T23:58:26.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.336 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.336 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:26.336 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565056+0000", 2024-09-14T23:58:26.336 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0019108160000000001 2024-09-14T23:58:26.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.336 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.336 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:26.336 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565088+0000", 2024-09-14T23:58:26.336 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2586999999999998e-05 2024-09-14T23:58:26.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.336 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.336 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:26.336 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565100+0000", 2024-09-14T23:58:26.336 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1917e-05 2024-09-14T23:58:26.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.337 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.337 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:26.337 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565166+0000", 2024-09-14T23:58:26.337 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6245e-05 2024-09-14T23:58:26.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.337 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.337 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:26.337 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565190+0000", 2024-09-14T23:58:26.337 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3825999999999999e-05 2024-09-14T23:58:26.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.337 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.337 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:26.337 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.567080+0000", 2024-09-14T23:58:26.338 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001889541 2024-09-14T23:58:26.338 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.338 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.338 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:26.338 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.567122+0000", 2024-09-14T23:58:26.338 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2262000000000003e-05 2024-09-14T23:58:26.338 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.338 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:26.338 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.338 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.338 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.338 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:66564 3.c e60/49 3:33faa7ca:::rbd_data.111873792d9b.0000000000000003:head v 60'236, mlcod=57'233)", 2024-09-14T23:58:26.338 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:11.271596+0000", 2024-09-14T23:58:26.338 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 75.006720325000003, 2024-09-14T23:58:26.338 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0060396099999999999, 2024-09-14T23:58:26.339 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:26.339 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:26.339 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:26.339 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:26.339 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.339 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:26.339 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.271596+0000", 2024-09-14T23:58:26.339 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.339 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.339 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.339 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:26.339 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.271596+0000", 2024-09-14T23:58:26.339 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.339 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.340 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.340 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:26.340 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.271597+0000", 2024-09-14T23:58:26.340 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2559999999999999e-06 2024-09-14T23:58:26.340 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.340 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.340 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:26.340 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.274854+0000", 2024-09-14T23:58:26.340 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0032574470000000001 2024-09-14T23:58:26.340 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.340 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.340 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:26.340 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.274874+0000", 2024-09-14T23:58:26.340 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9449999999999998e-05 2024-09-14T23:58:26.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.341 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.341 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:26.341 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.274883+0000", 2024-09-14T23:58:26.341 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5980000000000003e-06 2024-09-14T23:58:26.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.341 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.341 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:26.341 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.274977+0000", 2024-09-14T23:58:26.341 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3270999999999995e-05 2024-09-14T23:58:26.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.341 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.341 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:26.341 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.275000+0000", 2024-09-14T23:58:26.342 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3088e-05 2024-09-14T23:58:26.342 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.342 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.342 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:26.342 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.277590+0000", 2024-09-14T23:58:26.342 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0025898660000000001 2024-09-14T23:58:26.342 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.342 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.342 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:26.342 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.277635+0000", 2024-09-14T23:58:26.342 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5633999999999997e-05 2024-09-14T23:58:26.342 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.342 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:26.342 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.342 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.343 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.343 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:133384 3.1f e92/49 3:fce5e529:::rbd_data.11186132a8b6.0000000000000002:head v 92'807, mlcod=89'804)", 2024-09-14T23:58:26.343 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:45.293642+0000", 2024-09-14T23:58:26.343 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 40.984674441999999, 2024-09-14T23:58:26.343 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003760186, 2024-09-14T23:58:26.343 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:26.343 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:26.343 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:26.343 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:26.343 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.343 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:26.343 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.293642+0000", 2024-09-14T23:58:26.343 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.343 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.344 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.344 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:26.344 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.293642+0000", 2024-09-14T23:58:26.344 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.344 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.344 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.344 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:26.344 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.293643+0000", 2024-09-14T23:58:26.344 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2500000000000001e-06 2024-09-14T23:58:26.344 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.344 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.344 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:26.344 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.295402+0000", 2024-09-14T23:58:26.344 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017595410000000001 2024-09-14T23:58:26.345 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.345 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.345 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:26.345 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.295422+0000", 2024-09-14T23:58:26.345 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9763999999999999e-05 2024-09-14T23:58:26.345 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.345 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.345 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:26.345 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.295436+0000", 2024-09-14T23:58:26.345 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.364e-05 2024-09-14T23:58:26.345 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.345 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.345 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:26.345 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.295499+0000", 2024-09-14T23:58:26.345 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2770999999999999e-05 2024-09-14T23:58:26.346 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.346 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.346 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:26.346 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.295516+0000", 2024-09-14T23:58:26.346 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7240000000000001e-05 2024-09-14T23:58:26.346 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.346 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.346 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:26.346 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.297324+0000", 2024-09-14T23:58:26.346 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001807857 2024-09-14T23:58:26.346 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.346 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.346 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:26.346 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.297402+0000", 2024-09-14T23:58:26.347 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8122999999999997e-05 2024-09-14T23:58:26.347 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.347 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:26.347 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.347 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.347 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.347 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:145282 3.1c e100/49 3:3dcd75bf:::rbd_data.1118584371b4.0000000000000004:head v 100'647, mlcod=100'644)", 2024-09-14T23:58:26.347 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:54.171268+0000", 2024-09-14T23:58:26.347 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 32.107048355000003, 2024-09-14T23:58:26.347 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003527353, 2024-09-14T23:58:26.347 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:26.347 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:26.347 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:26.347 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:26.347 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.348 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:26.348 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:54.171268+0000", 2024-09-14T23:58:26.348 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.348 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.348 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.348 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:26.348 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:54.171268+0000", 2024-09-14T23:58:26.348 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.348 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.348 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.348 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:26.348 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:54.171268+0000", 2024-09-14T23:58:26.348 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4900000000000005e-07 2024-09-14T23:58:26.348 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.349 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.349 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:26.349 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:54.172187+0000", 2024-09-14T23:58:26.349 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00091850799999999995 2024-09-14T23:58:26.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.349 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.349 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:26.349 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:54.172199+0000", 2024-09-14T23:58:26.349 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2544000000000001e-05 2024-09-14T23:58:26.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.349 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.349 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:26.349 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:54.172205+0000", 2024-09-14T23:58:26.349 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1490000000000002e-06 2024-09-14T23:58:26.350 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.350 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.350 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:26.350 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:54.172267+0000", 2024-09-14T23:58:26.350 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2617e-05 2024-09-14T23:58:26.350 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.350 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.350 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:26.350 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:54.172283+0000", 2024-09-14T23:58:26.350 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5758000000000001e-05 2024-09-14T23:58:26.350 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.350 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.350 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:26.350 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:54.174754+0000", 2024-09-14T23:58:26.351 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0024711400000000001 2024-09-14T23:58:26.351 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.351 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.351 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:26.351 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:54.174795+0000", 2024-09-14T23:58:26.351 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0887999999999999e-05 2024-09-14T23:58:26.351 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.351 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:26.351 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.351 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.351 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.351 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:156597 3.5 e103/49 3:a23c5924:::rbd_data.1118b2f495e6.0000000000000011:head v 103'518, mlcod=98'516)", 2024-09-14T23:58:26.351 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:58.656197+0000", 2024-09-14T23:58:26.351 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.622119385000001, 2024-09-14T23:58:26.351 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0040423029999999997, 2024-09-14T23:58:26.352 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:26.352 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:26.352 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:26.352 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:26.352 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.352 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:26.352 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656197+0000", 2024-09-14T23:58:26.352 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.352 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.352 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.352 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:26.352 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656197+0000", 2024-09-14T23:58:26.352 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.352 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.353 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.353 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:26.353 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656197+0000", 2024-09-14T23:58:26.353 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1900000000000002e-07 2024-09-14T23:58:26.353 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.353 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.353 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:26.353 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656676+0000", 2024-09-14T23:58:26.353 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00047921699999999999 2024-09-14T23:58:26.353 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.353 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.353 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:26.353 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656690+0000", 2024-09-14T23:58:26.353 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3916e-05 2024-09-14T23:58:26.354 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.354 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.354 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:26.354 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656698+0000", 2024-09-14T23:58:26.354 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6529999999999994e-06 2024-09-14T23:58:26.354 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.354 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.354 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:26.354 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656757+0000", 2024-09-14T23:58:26.354 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9237999999999998e-05 2024-09-14T23:58:26.354 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.354 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.354 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:26.354 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656780+0000", 2024-09-14T23:58:26.355 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2968000000000001e-05 2024-09-14T23:58:26.355 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.355 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.355 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:26.355 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.660188+0000", 2024-09-14T23:58:26.355 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0034078630000000001 2024-09-14T23:58:26.355 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.355 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.355 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:26.355 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.660239+0000", 2024-09-14T23:58:26.355 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0828999999999997e-05 2024-09-14T23:58:26.355 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.355 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:26.355 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.355 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.356 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.356 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:157947 3.1c e103/49 3:383374fe:::rbd_data.1118b2f495e6.000000000000000c:head v 103'771, mlcod=103'769)", 2024-09-14T23:58:26.356 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:58.765271+0000", 2024-09-14T23:58:26.356 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.513044504, 2024-09-14T23:58:26.356 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0040869829999999998, 2024-09-14T23:58:26.356 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:26.356 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:26.356 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:26.356 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:26.356 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.356 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:26.356 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765271+0000", 2024-09-14T23:58:26.356 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.356 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.357 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.357 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:26.357 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765271+0000", 2024-09-14T23:58:26.357 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.357 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.357 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.357 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:26.357 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765273+0000", 2024-09-14T23:58:26.357 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1149999999999999e-06 2024-09-14T23:58:26.357 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.357 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.357 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:26.357 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765854+0000", 2024-09-14T23:58:26.357 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00058119599999999997 2024-09-14T23:58:26.357 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.358 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.358 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:26.358 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765869+0000", 2024-09-14T23:58:26.358 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5641000000000002e-05 2024-09-14T23:58:26.358 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.358 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.358 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:26.358 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765878+0000", 2024-09-14T23:58:26.358 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4519999999999997e-06 2024-09-14T23:58:26.358 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.358 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.358 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:26.358 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765942+0000", 2024-09-14T23:58:26.358 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3727999999999997e-05 2024-09-14T23:58:26.359 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.359 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.359 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:26.359 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765968+0000", 2024-09-14T23:58:26.359 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6171999999999999e-05 2024-09-14T23:58:26.359 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.359 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.359 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:26.359 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.769319+0000", 2024-09-14T23:58:26.359 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033511629999999999 2024-09-14T23:58:26.359 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.359 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.359 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:26.359 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.769358+0000", 2024-09-14T23:58:26.359 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9515999999999999e-05 2024-09-14T23:58:26.360 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.360 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:26.360 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.360 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.360 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.360 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:160196 3.c e103/49 3:34623e68:::rbd_data.1118b2f495e6.0000000000000005:head v 103'551, mlcod=102'549)", 2024-09-14T23:58:26.360 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:59.131433+0000", 2024-09-14T23:58:26.360 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.146883195000001, 2024-09-14T23:58:26.360 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0036932530000000001, 2024-09-14T23:58:26.360 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:26.360 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:26.360 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:26.360 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:26.360 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.360 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:26.361 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.131433+0000", 2024-09-14T23:58:26.361 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.361 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.361 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.361 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:26.361 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.131433+0000", 2024-09-14T23:58:26.361 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.361 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.361 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.361 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:26.361 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.131433+0000", 2024-09-14T23:58:26.361 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8500000000000001e-07 2024-09-14T23:58:26.361 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.361 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.362 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:26.362 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.132144+0000", 2024-09-14T23:58:26.362 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00071022800000000003 2024-09-14T23:58:26.362 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.362 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.362 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:26.362 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.132154+0000", 2024-09-14T23:58:26.362 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0301e-05 2024-09-14T23:58:26.362 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.362 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.362 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:26.362 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.132159+0000", 2024-09-14T23:58:26.362 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6280000000000004e-06 2024-09-14T23:58:26.362 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.362 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.363 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:26.363 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.132208+0000", 2024-09-14T23:58:26.363 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9595000000000002e-05 2024-09-14T23:58:26.363 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.363 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.363 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:26.363 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.132231+0000", 2024-09-14T23:58:26.363 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3010000000000002e-05 2024-09-14T23:58:26.363 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.363 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.363 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:26.363 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.135087+0000", 2024-09-14T23:58:26.363 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0028553239999999998 2024-09-14T23:58:26.363 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.364 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.364 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:26.364 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.135126+0000", 2024-09-14T23:58:26.364 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9481999999999999e-05 2024-09-14T23:58:26.364 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.364 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:26.364 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.364 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.364 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.364 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188871 3.d 3:b35cff4d:::rbd_header.1118c900944f:head [watch reconnect cookie 140670715433840 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-14T23:58:26.364 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.970653+0000", 2024-09-14T23:58:26.364 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.307663462000001, 2024-09-14T23:58:26.364 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0727808270000001, 2024-09-14T23:58:26.364 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:26.365 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:26.365 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:26.365 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:26.365 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:26.365 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:26.365 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188871 2024-09-14T23:58:26.365 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.365 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:26.365 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.365 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:26.365 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970653+0000", 2024-09-14T23:58:26.365 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.365 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.365 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.365 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:26.366 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970653+0000", 2024-09-14T23:58:26.366 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.366 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.366 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.366 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:26.366 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970655+0000", 2024-09-14T23:58:26.366 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.277e-06 2024-09-14T23:58:26.366 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.366 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.366 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:26.366 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970662+0000", 2024-09-14T23:58:26.366 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9619999999999996e-06 2024-09-14T23:58:26.366 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.366 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.366 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:26.367 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970682+0000", 2024-09-14T23:58:26.367 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0279999999999999e-05 2024-09-14T23:58:26.367 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.367 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.367 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:26.367 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970691+0000", 2024-09-14T23:58:26.367 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2790000000000004e-06 2024-09-14T23:58:26.367 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.367 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.367 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:26.367 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970762+0000", 2024-09-14T23:58:26.367 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1019999999999994e-05 2024-09-14T23:58:26.367 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.367 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.367 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-14T23:58:26.368 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970770+0000", 2024-09-14T23:58:26.368 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7999999999999999e-06 2024-09-14T23:58:26.368 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.368 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.368 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:26.368 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.973516+0000", 2024-09-14T23:58:26.368 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0027456469999999999 2024-09-14T23:58:26.368 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.368 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.368 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:26.368 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.973525+0000", 2024-09-14T23:58:26.368 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4890000000000001e-06 2024-09-14T23:58:26.368 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.368 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.369 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:26.369 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.041451+0000", 2024-09-14T23:58:26.369 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.067926012 2024-09-14T23:58:26.369 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.369 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.369 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:26.369 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.041673+0000", 2024-09-14T23:58:26.369 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000222048 2024-09-14T23:58:26.369 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.369 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.369 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:26.369 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.043433+0000", 2024-09-14T23:58:26.369 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017600129999999999 2024-09-14T23:58:26.369 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.370 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:26.370 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.370 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.370 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.370 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(osd.0.0:3238 3.1f e122/49 3:f8a59e83:::rbd_data.11187b9f1032.0000000000000006:2f v 122'943, mlcod=120'940)", 2024-09-14T23:58:26.370 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:22.285283+0000", 2024-09-14T23:58:26.370 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.9930326389999999, 2024-09-14T23:58:26.370 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003756194, 2024-09-14T23:58:26.370 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:26.370 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:26.370 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:26.370 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:26.370 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.370 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:26.371 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.285283+0000", 2024-09-14T23:58:26.371 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.371 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.371 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.371 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:26.371 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.285283+0000", 2024-09-14T23:58:26.371 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.371 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.371 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.371 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:26.371 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.285284+0000", 2024-09-14T23:58:26.371 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5600000000000002e-07 2024-09-14T23:58:26.371 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.371 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.371 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:26.372 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.285289+0000", 2024-09-14T23:58:26.372 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9579999999999998e-06 2024-09-14T23:58:26.372 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.372 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.372 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:26.372 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.285300+0000", 2024-09-14T23:58:26.372 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1688999999999999e-05 2024-09-14T23:58:26.372 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.372 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.372 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:26.372 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.285309+0000", 2024-09-14T23:58:26.372 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.388e-06 2024-09-14T23:58:26.372 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.372 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.373 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:26.373 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.285367+0000", 2024-09-14T23:58:26.373 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8811000000000003e-05 2024-09-14T23:58:26.373 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.373 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.373 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:26.373 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.285388+0000", 2024-09-14T23:58:26.373 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0117000000000001e-05 2024-09-14T23:58:26.373 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.373 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.373 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:26.373 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.288994+0000", 2024-09-14T23:58:26.373 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0036064270000000002 2024-09-14T23:58:26.373 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.374 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.374 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:26.374 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.289040+0000", 2024-09-14T23:58:26.374 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5547999999999998e-05 2024-09-14T23:58:26.374 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.374 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:26.374 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.374 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.374 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:26.374 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:26.376 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is unset 2024-09-14T23:58:26.376 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T23:58:26.377 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:26.374+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:26.476 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:26.474+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:26.565 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:26.565 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:58:26.565 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:58:26.565 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:26.576 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T23:58:26.577 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:26.576+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:26.678 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:26.677+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:26.749 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:26.749 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:58:26.749 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:58:26.749 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:58:26.749 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:26.760 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T23:58:26.780 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:26.779+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:26.881 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:26.879+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:26.930 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:26.931 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:58:26.931 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:58:26.931 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:58:26.931 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.931 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:107472 3.1e 3:7bf21588:::rbd_data.11187de8b0a5.000000000000005d:head [set-alloc-hint object_size 2097152 write_size 2097152,write 317788~1491] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-14T23:58:26.931 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:25.198493+0000", 2024-09-14T23:58:26.931 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 61.729718804000001, 2024-09-14T23:58:26.931 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.78175149300000002, 2024-09-14T23:58:26.931 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:26.931 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:26.931 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:26.931 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:26.931 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:26.932 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:26.932 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 107472 2024-09-14T23:58:26.932 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.932 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:26.932 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.932 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:26.932 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198493+0000", 2024-09-14T23:58:26.932 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.932 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.932 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.932 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:26.932 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198493+0000", 2024-09-14T23:58:26.932 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.932 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.933 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.933 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:26.933 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198495+0000", 2024-09-14T23:58:26.933 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.826e-06 2024-09-14T23:58:26.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.933 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.933 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:26.933 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198503+0000", 2024-09-14T23:58:26.933 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0150000000000003e-06 2024-09-14T23:58:26.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.933 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.933 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:26.933 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198521+0000", 2024-09-14T23:58:26.934 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8374000000000001e-05 2024-09-14T23:58:26.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.934 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.934 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:26.934 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198533+0000", 2024-09-14T23:58:26.934 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1593000000000001e-05 2024-09-14T23:58:26.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.934 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.934 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:26.934 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198603+0000", 2024-09-14T23:58:26.934 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0279999999999998e-05 2024-09-14T23:58:26.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.934 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.934 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:26.934 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198617+0000", 2024-09-14T23:58:26.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3517e-05 2024-09-14T23:58:26.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.935 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:26.935 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.977067+0000", 2024-09-14T23:58:26.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.77845065800000002 2024-09-14T23:58:26.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.935 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for missing object", 2024-09-14T23:58:26.935 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.977173+0000", 2024-09-14T23:58:26.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010543999999999999 2024-09-14T23:58:26.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.935 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:26.936 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.978737+0000", 2024-09-14T23:58:26.936 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001564078 2024-09-14T23:58:26.936 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.936 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.936 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:26.936 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.978845+0000", 2024-09-14T23:58:26.936 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000108643 2024-09-14T23:58:26.936 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.936 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.937 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1", 2024-09-14T23:58:26.937 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.978964+0000", 2024-09-14T23:58:26.937 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000119085 2024-09-14T23:58:26.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.937 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.937 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:26.937 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980109+0000", 2024-09-14T23:58:26.937 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0011442170000000001 2024-09-14T23:58:26.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.937 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.937 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:58:26.937 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980130+0000", 2024-09-14T23:58:26.937 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1766000000000001e-05 2024-09-14T23:58:26.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.938 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.938 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:26.938 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980171+0000", 2024-09-14T23:58:26.938 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0627999999999998e-05 2024-09-14T23:58:26.938 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.938 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.938 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:26.938 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980244+0000", 2024-09-14T23:58:26.938 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3373000000000004e-05 2024-09-14T23:58:26.938 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.938 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:26.938 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.938 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.938 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.938 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129159 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 5996544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:26.939 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.927783+0000", 2024-09-14T23:58:26.939 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 48.000428913, 2024-09-14T23:58:26.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054661122999999999, 2024-09-14T23:58:26.939 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:26.939 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:26.939 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:26.939 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:26.939 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:26.939 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:26.939 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129159 2024-09-14T23:58:26.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.939 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:26.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.939 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:26.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.927783+0000", 2024-09-14T23:58:26.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:26.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.927783+0000", 2024-09-14T23:58:26.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:26.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929684+0000", 2024-09-14T23:58:26.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0019011080000000001 2024-09-14T23:58:26.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.941 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:26.941 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929688+0000", 2024-09-14T23:58:26.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2610000000000004e-06 2024-09-14T23:58:26.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.941 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:26.941 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929697+0000", 2024-09-14T23:58:26.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0219999999999993e-06 2024-09-14T23:58:26.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.941 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:26.941 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929702+0000", 2024-09-14T23:58:26.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8840000000000002e-06 2024-09-14T23:58:26.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.942 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.942 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:26.942 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.982090+0000", 2024-09-14T23:58:26.942 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052388045000000001 2024-09-14T23:58:26.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.942 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.942 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:26.942 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.982208+0000", 2024-09-14T23:58:26.942 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118236 2024-09-14T23:58:26.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.942 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.942 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:26.942 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.982444+0000", 2024-09-14T23:58:26.942 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000235567 2024-09-14T23:58:26.943 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.943 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:26.943 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.943 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129166 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 8192000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:26.943 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.929794+0000", 2024-09-14T23:58:26.943 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.998418162, 2024-09-14T23:58:26.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054471973, 2024-09-14T23:58:26.943 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:26.943 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:26.943 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:26.943 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:26.943 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:26.944 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:26.944 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129166 2024-09-14T23:58:26.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.944 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:26.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:26.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929794+0000", 2024-09-14T23:58:26.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:26.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929794+0000", 2024-09-14T23:58:26.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:26.945 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929794+0000", 2024-09-14T23:58:26.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6899999999999997e-07 2024-09-14T23:58:26.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:26.945 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929797+0000", 2024-09-14T23:58:26.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.328e-06 2024-09-14T23:58:26.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:26.945 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929802+0000", 2024-09-14T23:58:26.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7309999999999998e-06 2024-09-14T23:58:26.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.946 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:26.946 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929807+0000", 2024-09-14T23:58:26.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4349999999999999e-06 2024-09-14T23:58:26.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.946 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:26.946 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984063+0000", 2024-09-14T23:58:26.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054256331999999997 2024-09-14T23:58:26.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.946 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:26.947 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984177+0000", 2024-09-14T23:58:26.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114225 2024-09-14T23:58:26.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.947 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:26.947 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984266+0000", 2024-09-14T23:58:26.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8352999999999999e-05 2024-09-14T23:58:26.947 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.947 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:26.947 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.947 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129167 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 6782976~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:26.947 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.929827+0000", 2024-09-14T23:58:26.947 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.998384753000003, 2024-09-14T23:58:26.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054637593999999998, 2024-09-14T23:58:26.948 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:26.948 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:26.948 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:26.948 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:26.948 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:26.948 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:26.948 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129167 2024-09-14T23:58:26.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.948 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:26.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.948 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:26.948 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929827+0000", 2024-09-14T23:58:26.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:26.949 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929827+0000", 2024-09-14T23:58:26.949 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:26.949 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929827+0000", 2024-09-14T23:58:26.949 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4000000000000002e-07 2024-09-14T23:58:26.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:26.949 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929830+0000", 2024-09-14T23:58:26.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.419e-06 2024-09-14T23:58:26.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.950 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:26.950 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929837+0000", 2024-09-14T23:58:26.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0450000000000003e-06 2024-09-14T23:58:26.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.950 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:26.950 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929842+0000", 2024-09-14T23:58:26.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.69e-06 2024-09-14T23:58:26.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.951 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:26.951 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984278+0000", 2024-09-14T23:58:26.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054436516999999997 2024-09-14T23:58:26.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.951 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:26.951 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984331+0000", 2024-09-14T23:58:26.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2590999999999998e-05 2024-09-14T23:58:26.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.951 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:26.951 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984465+0000", 2024-09-14T23:58:26.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013389200000000001 2024-09-14T23:58:26.951 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.952 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:26.952 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.952 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129168 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 8224768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:26.952 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.929849+0000", 2024-09-14T23:58:26.952 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.998363195000003, 2024-09-14T23:58:26.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054870368000000003, 2024-09-14T23:58:26.952 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:26.952 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:26.952 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:26.952 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:26.952 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:26.952 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:26.953 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129168 2024-09-14T23:58:26.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.953 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:26.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.953 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:26.953 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929849+0000", 2024-09-14T23:58:26.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.953 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:26.953 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929849+0000", 2024-09-14T23:58:26.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.954 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:26.954 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931681+0000", 2024-09-14T23:58:26.954 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018326989999999999 2024-09-14T23:58:26.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.954 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.954 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:26.954 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931686+0000", 2024-09-14T23:58:26.954 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7190000000000001e-06 2024-09-14T23:58:26.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.954 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.954 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:26.954 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931697+0000", 2024-09-14T23:58:26.954 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1287999999999999e-05 2024-09-14T23:58:26.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.955 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:26.955 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931704+0000", 2024-09-14T23:58:26.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3339999999999998e-06 2024-09-14T23:58:26.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.955 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:26.955 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984505+0000", 2024-09-14T23:58:26.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052801684000000002 2024-09-14T23:58:26.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.955 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:26.955 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984621+0000", 2024-09-14T23:58:26.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000116165 2024-09-14T23:58:26.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:26.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984719+0000", 2024-09-14T23:58:26.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7479000000000006e-05 2024-09-14T23:58:26.956 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.956 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:26.956 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.956 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129176 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 10452992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:26.956 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.931830+0000", 2024-09-14T23:58:26.956 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.996381434, 2024-09-14T23:58:26.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054513292999999997, 2024-09-14T23:58:26.956 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:26.957 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:26.957 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:26.957 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:26.957 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:26.957 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:26.957 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129176 2024-09-14T23:58:26.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.957 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:26.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:26.957 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931830+0000", 2024-09-14T23:58:26.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:26.958 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931830+0000", 2024-09-14T23:58:26.958 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.958 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.958 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:26.958 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.933678+0000", 2024-09-14T23:58:26.958 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018472440000000001 2024-09-14T23:58:26.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.958 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.958 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:26.958 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.933686+0000", 2024-09-14T23:58:26.958 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0239999999999997e-06 2024-09-14T23:58:26.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.958 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.959 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:26.959 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.933694+0000", 2024-09-14T23:58:26.959 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8400000000000001e-06 2024-09-14T23:58:26.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.959 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.959 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:26.959 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.933701+0000", 2024-09-14T23:58:26.959 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2870000000000001e-06 2024-09-14T23:58:26.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.959 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.959 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:26.959 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.986165+0000", 2024-09-14T23:58:26.959 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052463900000000001 2024-09-14T23:58:26.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.960 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:26.960 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.986259+0000", 2024-09-14T23:58:26.960 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4435e-05 2024-09-14T23:58:26.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.960 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:26.960 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.986344+0000", 2024-09-14T23:58:26.960 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4562999999999997e-05 2024-09-14T23:58:26.960 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.960 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:26.960 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.961 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129201 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 10289152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:26.961 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.935960+0000", 2024-09-14T23:58:26.961 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.992251971999998, 2024-09-14T23:58:26.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054338487999999997, 2024-09-14T23:58:26.961 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:26.961 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:26.961 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:26.961 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:26.961 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:26.961 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:26.961 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129201 2024-09-14T23:58:26.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.961 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:26.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:26.962 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935960+0000", 2024-09-14T23:58:26.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.962 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:26.962 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935960+0000", 2024-09-14T23:58:26.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.962 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:26.962 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935960+0000", 2024-09-14T23:58:26.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.72e-07 2024-09-14T23:58:26.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.963 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:26.963 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935963+0000", 2024-09-14T23:58:26.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5390000000000002e-06 2024-09-14T23:58:26.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.963 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.963 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:26.963 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935971+0000", 2024-09-14T23:58:26.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4589999999999998e-06 2024-09-14T23:58:26.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.963 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.963 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:26.963 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935977+0000", 2024-09-14T23:58:26.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4110000000000004e-06 2024-09-14T23:58:26.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.963 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.964 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:26.964 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990213+0000", 2024-09-14T23:58:26.964 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054236642000000002 2024-09-14T23:58:26.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.964 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:26.964 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990257+0000", 2024-09-14T23:58:26.964 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3696000000000003e-05 2024-09-14T23:58:26.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.964 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:26.964 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990298+0000", 2024-09-14T23:58:26.964 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1369000000000003e-05 2024-09-14T23:58:26.964 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.965 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:26.965 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.965 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.965 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129202 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 11370496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:26.965 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.935986+0000", 2024-09-14T23:58:26.965 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.992225353999999, 2024-09-14T23:58:26.965 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054543700000000001, 2024-09-14T23:58:26.965 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:26.965 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:26.965 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:26.965 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:26.965 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:26.965 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:26.966 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129202 2024-09-14T23:58:26.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.966 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:26.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:26.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935986+0000", 2024-09-14T23:58:26.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:26.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935986+0000", 2024-09-14T23:58:26.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.967 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:26.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937678+0000", 2024-09-14T23:58:26.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016917849999999999 2024-09-14T23:58:26.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.967 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:26.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937682+0000", 2024-09-14T23:58:26.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2400000000000001e-06 2024-09-14T23:58:26.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.967 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:26.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937695+0000", 2024-09-14T23:58:26.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2999e-05 2024-09-14T23:58:26.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.968 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.968 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:26.968 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937700+0000", 2024-09-14T23:58:26.968 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8420000000000004e-06 2024-09-14T23:58:26.968 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.968 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.968 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:26.968 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990336+0000", 2024-09-14T23:58:26.968 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052636060999999998 2024-09-14T23:58:26.968 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.968 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.968 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:26.968 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990435+0000", 2024-09-14T23:58:26.968 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9015e-05 2024-09-14T23:58:26.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.969 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.969 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:26.969 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990530+0000", 2024-09-14T23:58:26.969 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4758000000000003e-05 2024-09-14T23:58:26.969 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.969 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:26.969 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.969 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.969 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129214 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 11468800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:26.969 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.937939+0000", 2024-09-14T23:58:26.969 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.990272439999998, 2024-09-14T23:58:26.969 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054463733, 2024-09-14T23:58:26.970 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:26.970 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:26.970 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:26.970 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:26.970 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:26.970 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:26.970 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129214 2024-09-14T23:58:26.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.970 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:26.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.970 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:26.970 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937939+0000", 2024-09-14T23:58:26.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.971 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:26.971 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937939+0000", 2024-09-14T23:58:26.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.971 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:26.971 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939674+0000", 2024-09-14T23:58:26.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017342589999999999 2024-09-14T23:58:26.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.971 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:26.971 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939678+0000", 2024-09-14T23:58:26.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2590000000000002e-06 2024-09-14T23:58:26.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.972 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.972 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:26.972 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939699+0000", 2024-09-14T23:58:26.972 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1143000000000001e-05 2024-09-14T23:58:26.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.972 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.972 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:26.972 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939706+0000", 2024-09-14T23:58:26.972 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0269999999999998e-06 2024-09-14T23:58:26.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.972 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.972 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:26.972 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.992221+0000", 2024-09-14T23:58:26.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052514981000000002 2024-09-14T23:58:26.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.973 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:26.973 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.992321+0000", 2024-09-14T23:58:26.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9625999999999998e-05 2024-09-14T23:58:26.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.973 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:26.973 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.992403+0000", 2024-09-14T23:58:26.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2437999999999999e-05 2024-09-14T23:58:26.973 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.973 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:26.973 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.974 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129226 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 14090240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:26.974 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.939867+0000", 2024-09-14T23:58:26.974 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.988344511999998, 2024-09-14T23:58:26.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054523573999999998, 2024-09-14T23:58:26.974 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:26.974 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:26.974 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:26.974 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:26.974 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:26.974 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:26.974 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129226 2024-09-14T23:58:26.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.975 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:26.975 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.975 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:26.975 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939867+0000", 2024-09-14T23:58:26.975 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.975 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.975 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:26.975 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939867+0000", 2024-09-14T23:58:26.975 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.975 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.975 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:26.975 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941669+0000", 2024-09-14T23:58:26.975 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018021549999999999 2024-09-14T23:58:26.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.976 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:26.976 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941673+0000", 2024-09-14T23:58:26.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1269999999999999e-06 2024-09-14T23:58:26.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.976 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:26.976 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941679+0000", 2024-09-14T23:58:26.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1879999999999999e-06 2024-09-14T23:58:26.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.976 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:26.977 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941682+0000", 2024-09-14T23:58:26.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7110000000000001e-06 2024-09-14T23:58:26.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.977 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:26.977 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.994201+0000", 2024-09-14T23:58:26.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052518265000000001 2024-09-14T23:58:26.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.979 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.979 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:26.980 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.994302+0000", 2024-09-14T23:58:26.980 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101653 2024-09-14T23:58:26.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.980 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:26.980 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.994391+0000", 2024-09-14T23:58:26.980 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8474999999999996e-05 2024-09-14T23:58:26.980 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.980 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:26.980 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.980 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129237 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 12386304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:26.981 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.941825+0000", 2024-09-14T23:58:26.981 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.986386750000001, 2024-09-14T23:58:26.981 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054368076000000001, 2024-09-14T23:58:26.981 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:26.981 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:26.981 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:26.981 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:26.981 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:26.981 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:26.981 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129237 2024-09-14T23:58:26.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.981 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:26.981 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.981 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:26.981 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941825+0000", 2024-09-14T23:58:26.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.982 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:26.982 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941825+0000", 2024-09-14T23:58:26.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.982 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:26.982 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941825+0000", 2024-09-14T23:58:26.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.29e-07 2024-09-14T23:58:26.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.982 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:26.983 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941826+0000", 2024-09-14T23:58:26.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0999999999999997e-07 2024-09-14T23:58:26.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.983 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.983 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:26.983 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941834+0000", 2024-09-14T23:58:26.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5519999999999999e-06 2024-09-14T23:58:26.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.983 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.983 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:26.983 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941843+0000", 2024-09-14T23:58:26.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2960000000000004e-06 2024-09-14T23:58:26.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.983 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.983 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:26.984 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996108+0000", 2024-09-14T23:58:26.984 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054265200999999999 2024-09-14T23:58:26.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.984 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:26.984 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996152+0000", 2024-09-14T23:58:26.984 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4066999999999997e-05 2024-09-14T23:58:26.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.984 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:26.984 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996193+0000", 2024-09-14T23:58:26.984 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0821000000000001e-05 2024-09-14T23:58:26.984 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.984 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:26.985 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.985 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.985 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129238 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 13828096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:26.985 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.941851+0000", 2024-09-14T23:58:26.985 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.986360425999997, 2024-09-14T23:58:26.985 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.05458031, 2024-09-14T23:58:26.985 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:26.985 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:26.985 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:26.985 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:26.985 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:26.985 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:26.985 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129238 2024-09-14T23:58:26.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.986 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:26.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.986 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:26.986 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941851+0000", 2024-09-14T23:58:26.986 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.986 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:26.986 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941851+0000", 2024-09-14T23:58:26.986 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.986 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:26.986 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943670+0000", 2024-09-14T23:58:26.987 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018181930000000001 2024-09-14T23:58:26.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.987 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:26.987 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943673+0000", 2024-09-14T23:58:26.987 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3790000000000002e-06 2024-09-14T23:58:26.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.987 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:26.987 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943679+0000", 2024-09-14T23:58:26.987 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8719999999999999e-06 2024-09-14T23:58:26.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.987 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:26.988 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943683+0000", 2024-09-14T23:58:26.988 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8539999999999999e-06 2024-09-14T23:58:26.988 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.988 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.988 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:26.988 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996233+0000", 2024-09-14T23:58:26.988 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052550690999999997 2024-09-14T23:58:26.988 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.988 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.988 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:26.988 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996339+0000", 2024-09-14T23:58:26.988 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105989 2024-09-14T23:58:26.988 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.988 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.989 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:26.989 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996432+0000", 2024-09-14T23:58:26.989 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2331999999999996e-05 2024-09-14T23:58:26.989 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.989 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:26.989 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.989 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.989 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.989 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129249 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 12124160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:26.989 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.943826+0000", 2024-09-14T23:58:26.989 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.984385934999999, 2024-09-14T23:58:26.989 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054373266000000003, 2024-09-14T23:58:26.989 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:26.990 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:26.990 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:26.990 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:26.990 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:26.990 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:26.990 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129249 2024-09-14T23:58:26.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.990 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:26.990 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.990 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:26.990 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943826+0000", 2024-09-14T23:58:26.990 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.990 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.991 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:26.991 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943826+0000", 2024-09-14T23:58:26.991 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.991 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.991 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:26.991 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943826+0000", 2024-09-14T23:58:26.991 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.03e-07 2024-09-14T23:58:26.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.991 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.991 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:26.991 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943827+0000", 2024-09-14T23:58:26.991 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.08e-06 2024-09-14T23:58:26.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.992 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.992 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:26.992 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943834+0000", 2024-09-14T23:58:26.992 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4119999999999998e-06 2024-09-14T23:58:26.992 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.992 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.992 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:26.992 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943836+0000", 2024-09-14T23:58:26.992 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7369999999999998e-06 2024-09-14T23:58:26.992 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.992 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.992 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:26.992 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998115+0000", 2024-09-14T23:58:26.992 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054279040000000001 2024-09-14T23:58:26.993 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.993 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.993 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:26.993 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998159+0000", 2024-09-14T23:58:26.993 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3486999999999999e-05 2024-09-14T23:58:26.993 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.993 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.993 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:26.993 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998199+0000", 2024-09-14T23:58:26.993 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0306999999999999e-05 2024-09-14T23:58:26.993 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.993 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:26.993 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129250 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 13565952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.943840+0000", 2024-09-14T23:58:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.984371985000003, 2024-09-14T23:58:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054573875000000001, 2024-09-14T23:58:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129250 2024-09-14T23:58:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943840+0000", 2024-09-14T23:58:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943840+0000", 2024-09-14T23:58:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945669+0000", 2024-09-14T23:58:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018289720000000001 2024-09-14T23:58:26.996 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.996 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.996 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:26.996 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945671+0000", 2024-09-14T23:58:26.996 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6869999999999999e-06 2024-09-14T23:58:26.996 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.996 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.996 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:26.996 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945680+0000", 2024-09-14T23:58:26.996 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5259999999999993e-06 2024-09-14T23:58:26.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.997 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.997 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:26.997 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945685+0000", 2024-09-14T23:58:26.997 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8559999999999998e-06 2024-09-14T23:58:26.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.997 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.997 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:26.997 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998242+0000", 2024-09-14T23:58:26.997 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052557505999999997 2024-09-14T23:58:26.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.997 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.997 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:26.998 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998329+0000", 2024-09-14T23:58:26.998 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6410999999999996e-05 2024-09-14T23:58:26.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.998 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:26.998 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998414+0000", 2024-09-14T23:58:26.998 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4917e-05 2024-09-14T23:58:26.998 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.998 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:26.998 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:26.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.998 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129261 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 14385152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:26.998 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.945815+0000", 2024-09-14T23:58:26.998 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.982396692000002, 2024-09-14T23:58:26.999 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054331867999999998, 2024-09-14T23:58:26.999 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:26.999 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:26.999 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:26.999 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:26.999 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:26.999 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:26.999 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129261 2024-09-14T23:58:26.999 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:26.999 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:26.999 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:26.999 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:26.999 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945815+0000", 2024-09-14T23:58:26.999 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:27.000 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.000 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.000 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:27.000 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945815+0000", 2024-09-14T23:58:27.000 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:27.000 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.000 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.000 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:27.000 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945815+0000", 2024-09-14T23:58:27.000 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0599999999999999e-07 2024-09-14T23:58:27.000 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.000 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.000 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:27.001 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945816+0000", 2024-09-14T23:58:27.001 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0380000000000001e-06 2024-09-14T23:58:27.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:27.001 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945819+0000", 2024-09-14T23:58:27.001 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.005e-06 2024-09-14T23:58:27.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:27.001 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945822+0000", 2024-09-14T23:58:27.001 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.452e-06 2024-09-14T23:58:27.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.002 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:27.002 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000060+0000", 2024-09-14T23:58:27.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054238606000000002 2024-09-14T23:58:27.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.002 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:27.002 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000106+0000", 2024-09-14T23:58:27.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5188000000000002e-05 2024-09-14T23:58:27.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.002 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:27.002 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000147+0000", 2024-09-14T23:58:27.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1372999999999998e-05 2024-09-14T23:58:27.003 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:27.003 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:27.003 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:27.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.003 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129262 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 13303808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:27.003 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.945825+0000", 2024-09-14T23:58:27.003 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.982387043999999, 2024-09-14T23:58:27.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054515942999999997, 2024-09-14T23:58:27.003 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:27.003 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:27.003 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:27.004 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:27.004 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:27.004 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:27.004 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129262 2024-09-14T23:58:27.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.004 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:27.004 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:27.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945825+0000", 2024-09-14T23:58:27.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:27.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.004 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:27.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945825+0000", 2024-09-14T23:58:27.005 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:27.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.005 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.005 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:27.005 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.947670+0000", 2024-09-14T23:58:27.005 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018451229999999999 2024-09-14T23:58:27.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.005 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.005 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:27.005 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.947672+0000", 2024-09-14T23:58:27.005 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5289999999999998e-06 2024-09-14T23:58:27.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.005 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.005 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:27.006 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.947678+0000", 2024-09-14T23:58:27.006 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9880000000000001e-06 2024-09-14T23:58:27.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.006 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:27.006 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.947682+0000", 2024-09-14T23:58:27.006 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.63e-06 2024-09-14T23:58:27.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.006 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:27.006 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000183+0000", 2024-09-14T23:58:27.006 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052500770000000002 2024-09-14T23:58:27.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.006 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:27.007 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000266+0000", 2024-09-14T23:58:27.007 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3133000000000005e-05 2024-09-14T23:58:27.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.007 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.007 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:27.007 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000341+0000", 2024-09-14T23:58:27.007 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4770000000000004e-05 2024-09-14T23:58:27.007 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:27.007 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:27.007 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:27.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.007 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.007 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129299 3.10 3:0f1e06ee:::rbd_object_map.1118721be852:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:27.007 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.952330+0000", 2024-09-14T23:58:27.007 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.975882231, 2024-09-14T23:58:27.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054412818000000002, 2024-09-14T23:58:27.008 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:27.008 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:27.008 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:27.008 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:27.008 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:27.008 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:27.008 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129299 2024-09-14T23:58:27.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.008 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:27.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:27.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952330+0000", 2024-09-14T23:58:27.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:27.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.009 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.009 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:27.009 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952330+0000", 2024-09-14T23:58:27.009 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:27.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.009 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.009 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:27.009 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952330+0000", 2024-09-14T23:58:27.009 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4900000000000001e-07 2024-09-14T23:58:27.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.009 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.009 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:27.009 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952332+0000", 2024-09-14T23:58:27.010 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2879999999999999e-06 2024-09-14T23:58:27.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.010 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.010 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:27.010 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952340+0000", 2024-09-14T23:58:27.010 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9910000000000002e-06 2024-09-14T23:58:27.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.010 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.010 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:27.010 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952344+0000", 2024-09-14T23:58:27.010 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0280000000000002e-06 2024-09-14T23:58:27.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.010 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.010 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:27.010 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.004722+0000", 2024-09-14T23:58:27.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052377932000000002 2024-09-14T23:58:27.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.011 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.011 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:27.011 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.004778+0000", 2024-09-14T23:58:27.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5795999999999999e-05 2024-09-14T23:58:27.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.011 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.011 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:58:27.011 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.005016+0000", 2024-09-14T23:58:27.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000238006 2024-09-14T23:58:27.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.011 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.011 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:58:27.011 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.006366+0000", 2024-09-14T23:58:27.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001349786 2024-09-14T23:58:27.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.012 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:27.012 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.006685+0000", 2024-09-14T23:58:27.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00031970000000000002 2024-09-14T23:58:27.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.012 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:27.012 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.006718+0000", 2024-09-14T23:58:27.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2329000000000002e-05 2024-09-14T23:58:27.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.012 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:27.013 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.006742+0000", 2024-09-14T23:58:27.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4613e-05 2024-09-14T23:58:27.013 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:27.013 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:27.013 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:27.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.013 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186210 3.d 3.b2ff3acd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:27.013 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.129044+0000", 2024-09-14T23:58:27.013 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.799167868000001, 2024-09-14T23:58:27.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.84555443900000005, 2024-09-14T23:58:27.013 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:27.013 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:27.013 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:58:27.014 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:27.014 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:27.014 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:27.014 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186210 2024-09-14T23:58:27.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.014 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:27.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.014 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:27.014 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129044+0000", 2024-09-14T23:58:27.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:27.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.014 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:27.014 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129044+0000", 2024-09-14T23:58:27.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:27.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.015 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:27.015 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129046+0000", 2024-09-14T23:58:27.015 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.742e-06 2024-09-14T23:58:27.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.015 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:27.015 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129052+0000", 2024-09-14T23:58:27.015 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7390000000000002e-06 2024-09-14T23:58:27.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.015 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:27.015 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129067+0000", 2024-09-14T23:58:27.015 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.417e-05 2024-09-14T23:58:27.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.016 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.016 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:27.016 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129076+0000", 2024-09-14T23:58:27.016 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2289999999999992e-06 2024-09-14T23:58:27.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.016 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.016 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:27.016 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.130947+0000", 2024-09-14T23:58:27.016 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001871075 2024-09-14T23:58:27.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.016 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.016 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:27.016 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.130955+0000", 2024-09-14T23:58:27.016 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4500000000000004e-06 2024-09-14T23:58:27.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.017 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.017 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:27.017 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974587+0000", 2024-09-14T23:58:27.017 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.84363205500000005 2024-09-14T23:58:27.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.017 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.017 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:27.017 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974598+0000", 2024-09-14T23:58:27.017 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0978999999999999e-05 2024-09-14T23:58:27.017 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:27.017 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:27.017 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:27.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.017 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.017 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187415 3.d 3.b2ff3acd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:27.018 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.254681+0000", 2024-09-14T23:58:27.018 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.673530347, 2024-09-14T23:58:27.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.71995530699999999, 2024-09-14T23:58:27.018 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:27.018 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:27.018 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:58:27.018 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:27.018 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:27.018 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:27.018 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187415 2024-09-14T23:58:27.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.018 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:27.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.018 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:27.018 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254681+0000", 2024-09-14T23:58:27.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:27.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:27.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254681+0000", 2024-09-14T23:58:27.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:27.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:27.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254682+0000", 2024-09-14T23:58:27.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2e-07 2024-09-14T23:58:27.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:27.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254684+0000", 2024-09-14T23:58:27.020 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2730000000000001e-06 2024-09-14T23:58:27.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.020 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:27.020 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254689+0000", 2024-09-14T23:58:27.020 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4769999999999997e-06 2024-09-14T23:58:27.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.020 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:27.020 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254692+0000", 2024-09-14T23:58:27.020 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9890000000000001e-06 2024-09-14T23:58:27.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.020 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:27.021 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254721+0000", 2024-09-14T23:58:27.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9762999999999999e-05 2024-09-14T23:58:27.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.021 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:27.021 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254725+0000", 2024-09-14T23:58:27.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9650000000000002e-06 2024-09-14T23:58:27.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.021 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:27.021 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974628+0000", 2024-09-14T23:58:27.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.71990235499999999 2024-09-14T23:58:27.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.021 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:27.022 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974637+0000", 2024-09-14T23:58:27.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0650000000000005e-06 2024-09-14T23:58:27.022 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:27.022 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:27.022 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:27.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.022 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187457 3.d 3.b2ff3acd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:27.022 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.257980+0000", 2024-09-14T23:58:27.022 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.670231773000001, 2024-09-14T23:58:27.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.71671711199999999, 2024-09-14T23:58:27.022 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:27.022 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:27.023 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:58:27.023 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:27.023 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:27.023 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:27.023 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187457 2024-09-14T23:58:27.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.023 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:27.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.023 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:27.023 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257980+0000", 2024-09-14T23:58:27.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:27.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.023 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:27.024 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257980+0000", 2024-09-14T23:58:27.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:27.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.024 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:27.024 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257980+0000", 2024-09-14T23:58:27.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.17e-07 2024-09-14T23:58:27.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.024 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:27.024 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257981+0000", 2024-09-14T23:58:27.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.172e-06 2024-09-14T23:58:27.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:27.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257984+0000", 2024-09-14T23:58:27.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9550000000000001e-06 2024-09-14T23:58:27.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:27.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257987+0000", 2024-09-14T23:58:27.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5799999999999999e-06 2024-09-14T23:58:27.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:27.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.258030+0000", 2024-09-14T23:58:27.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3118999999999998e-05 2024-09-14T23:58:27.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:27.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.258038+0000", 2024-09-14T23:58:27.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6850000000000001e-06 2024-09-14T23:58:27.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:27.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974688+0000", 2024-09-14T23:58:27.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.71665016000000004 2024-09-14T23:58:27.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:27.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974697+0000", 2024-09-14T23:58:27.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2240000000000001e-06 2024-09-14T23:58:27.026 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:27.027 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:27.027 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:27.027 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:27.027 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:27.027 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:27.028 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T23:58:27.029 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:27.026+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:27.129 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:27.127+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:27.142 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:27.143 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:58:27.143 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:58:27.143 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:27.151 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T23:58:27.230 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:27.228+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:27.259 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:27.259 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:58:27.259 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:58:27.259 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:58:27.259 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:27.268 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T23:58:27.331 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:27.329+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:27.390 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:27.390 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:58:27.390 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:58:27.390 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:58:27.390 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.390 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69262 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 4800512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:27.390 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300811+0000", 2024-09-14T23:58:27.390 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 72.086922314000006, 2024-09-14T23:58:27.390 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050403796000000001, 2024-09-14T23:58:27.390 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:27.391 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:27.391 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:27.391 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:27.391 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:27.391 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:27.391 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69262 2024-09-14T23:58:27.391 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.391 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:27.391 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.391 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:27.391 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300811+0000", 2024-09-14T23:58:27.391 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:27.391 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.391 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.392 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:27.392 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300811+0000", 2024-09-14T23:58:27.392 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:27.392 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.392 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.392 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:27.392 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300811+0000", 2024-09-14T23:58:27.392 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5700000000000002e-07 2024-09-14T23:58:27.392 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.392 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.392 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:27.392 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300815+0000", 2024-09-14T23:58:27.392 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.087e-06 2024-09-14T23:58:27.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.393 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.393 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:27.393 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300823+0000", 2024-09-14T23:58:27.393 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8720000000000002e-06 2024-09-14T23:58:27.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.393 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.393 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:27.393 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300830+0000", 2024-09-14T23:58:27.393 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0589999999999997e-06 2024-09-14T23:58:27.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.393 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.393 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:27.393 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351020+0000", 2024-09-14T23:58:27.394 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050189552999999998 2024-09-14T23:58:27.394 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.394 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.394 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:27.394 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351112+0000", 2024-09-14T23:58:27.394 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2615999999999996e-05 2024-09-14T23:58:27.394 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.394 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.394 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:27.394 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351214+0000", 2024-09-14T23:58:27.394 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101952 2024-09-14T23:58:27.394 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:27.394 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:27.394 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:27.395 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.395 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.395 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69263 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 4915200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:27.395 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300842+0000", 2024-09-14T23:58:27.395 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 72.08689081, 2024-09-14T23:58:27.395 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050590637000000001, 2024-09-14T23:58:27.395 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:27.395 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:27.395 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:27.395 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:27.395 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:27.395 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:27.395 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69263 2024-09-14T23:58:27.395 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.396 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:27.396 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.396 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:27.396 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300842+0000", 2024-09-14T23:58:27.396 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:27.396 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.396 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.396 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:27.396 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300842+0000", 2024-09-14T23:58:27.396 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:27.396 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.396 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.396 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:27.396 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300843+0000", 2024-09-14T23:58:27.397 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4000000000000002e-07 2024-09-14T23:58:27.397 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.397 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.397 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:27.397 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300846+0000", 2024-09-14T23:58:27.397 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8720000000000002e-06 2024-09-14T23:58:27.397 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.397 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.397 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:27.397 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300853+0000", 2024-09-14T23:58:27.397 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0450000000000003e-06 2024-09-14T23:58:27.397 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.397 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.398 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:27.398 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300859+0000", 2024-09-14T23:58:27.398 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9510000000000003e-06 2024-09-14T23:58:27.398 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.398 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.398 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:27.398 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351260+0000", 2024-09-14T23:58:27.398 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050401654999999997 2024-09-14T23:58:27.398 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.398 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.398 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:27.398 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351353+0000", 2024-09-14T23:58:27.398 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2785e-05 2024-09-14T23:58:27.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.399 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.399 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:27.399 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351433+0000", 2024-09-14T23:58:27.399 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9789000000000005e-05 2024-09-14T23:58:27.399 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:27.399 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:27.399 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:27.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.399 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.399 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69264 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5029888~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:27.399 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300866+0000", 2024-09-14T23:58:27.399 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 72.086866877000006, 2024-09-14T23:58:27.399 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050794063, 2024-09-14T23:58:27.399 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:27.400 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:27.400 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:27.400 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:27.400 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:27.400 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:27.400 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69264 2024-09-14T23:58:27.400 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.400 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:27.400 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.400 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:27.400 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300866+0000", 2024-09-14T23:58:27.400 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:27.400 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.400 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.401 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:27.401 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300866+0000", 2024-09-14T23:58:27.401 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:27.401 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.401 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.401 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:27.401 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300866+0000", 2024-09-14T23:58:27.401 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8799999999999998e-07 2024-09-14T23:58:27.401 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.401 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.401 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:27.401 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300869+0000", 2024-09-14T23:58:27.401 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0880000000000002e-06 2024-09-14T23:58:27.401 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.402 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.402 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:27.402 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300875+0000", 2024-09-14T23:58:27.402 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9280000000000004e-06 2024-09-14T23:58:27.402 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.402 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.402 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:27.402 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300882+0000", 2024-09-14T23:58:27.402 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4740000000000002e-06 2024-09-14T23:58:27.402 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.402 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.402 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:27.402 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351472+0000", 2024-09-14T23:58:27.402 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050590006999999999 2024-09-14T23:58:27.403 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.403 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.403 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:27.403 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351563+0000", 2024-09-14T23:58:27.403 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0762999999999998e-05 2024-09-14T23:58:27.403 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.403 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.403 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:27.403 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351660+0000", 2024-09-14T23:58:27.403 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7415000000000002e-05 2024-09-14T23:58:27.403 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:27.403 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:27.403 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:27.403 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.404 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.404 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69265 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5144576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:27.404 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300890+0000", 2024-09-14T23:58:27.404 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 72.086843372999994, 2024-09-14T23:58:27.404 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050921389999999997, 2024-09-14T23:58:27.404 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:27.404 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:27.404 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:27.404 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:27.404 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:27.404 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:27.404 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69265 2024-09-14T23:58:27.404 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.404 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:27.404 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.405 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:27.405 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300890+0000", 2024-09-14T23:58:27.405 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:27.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.405 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.405 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:27.405 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300890+0000", 2024-09-14T23:58:27.405 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:27.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.405 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.405 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:27.405 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300890+0000", 2024-09-14T23:58:27.405 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2500000000000001e-07 2024-09-14T23:58:27.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.406 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:27.406 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300892+0000", 2024-09-14T23:58:27.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2469999999999998e-06 2024-09-14T23:58:27.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.406 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:27.406 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300899+0000", 2024-09-14T23:58:27.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5509999999999996e-06 2024-09-14T23:58:27.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.406 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:27.406 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300904+0000", 2024-09-14T23:58:27.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3299999999999998e-06 2024-09-14T23:58:27.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.407 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:27.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351707+0000", 2024-09-14T23:58:27.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050802673999999999 2024-09-14T23:58:27.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.407 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:27.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351755+0000", 2024-09-14T23:58:27.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8362000000000002e-05 2024-09-14T23:58:27.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.407 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:27.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351811+0000", 2024-09-14T23:58:27.408 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5801000000000003e-05 2024-09-14T23:58:27.408 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:27.408 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:27.408 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:27.408 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.408 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.408 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69266 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5259264~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:27.408 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300911+0000", 2024-09-14T23:58:27.408 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 72.086822299999994, 2024-09-14T23:58:27.408 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051125089999999998, 2024-09-14T23:58:27.408 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:27.408 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:27.408 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:27.409 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:27.409 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:27.409 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:27.409 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69266 2024-09-14T23:58:27.409 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.409 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:27.409 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.409 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:27.409 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300911+0000", 2024-09-14T23:58:27.409 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:27.409 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.409 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.409 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:27.409 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300911+0000", 2024-09-14T23:58:27.409 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:27.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.410 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.410 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:27.410 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300911+0000", 2024-09-14T23:58:27.410 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-14T23:58:27.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.410 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.410 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:27.410 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300913+0000", 2024-09-14T23:58:27.410 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9300000000000002e-06 2024-09-14T23:58:27.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.410 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.410 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:27.410 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300919+0000", 2024-09-14T23:58:27.411 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.489e-06 2024-09-14T23:58:27.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.411 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.411 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:27.411 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300925+0000", 2024-09-14T23:58:27.411 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1719999999999998e-06 2024-09-14T23:58:27.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.411 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.411 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:27.411 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351847+0000", 2024-09-14T23:58:27.411 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050922621000000001 2024-09-14T23:58:27.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.411 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.411 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:27.411 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351938+0000", 2024-09-14T23:58:27.412 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0830999999999998e-05 2024-09-14T23:58:27.412 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.412 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.412 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:27.412 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352036+0000", 2024-09-14T23:58:27.412 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7676999999999993e-05 2024-09-14T23:58:27.412 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:27.412 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:27.412 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:27.412 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.412 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.412 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69267 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5373952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:27.412 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300945+0000", 2024-09-14T23:58:27.413 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 72.086787505000004, 2024-09-14T23:58:27.413 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051217934, 2024-09-14T23:58:27.413 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:27.413 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:27.413 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:27.413 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:27.413 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:27.413 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:27.413 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69267 2024-09-14T23:58:27.413 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.413 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:27.413 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.413 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:27.413 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300945+0000", 2024-09-14T23:58:27.414 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:27.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.414 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.414 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:27.414 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300945+0000", 2024-09-14T23:58:27.414 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:27.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.414 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.414 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:27.414 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300946+0000", 2024-09-14T23:58:27.414 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2600000000000002e-07 2024-09-14T23:58:27.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.414 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.414 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:27.414 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300948+0000", 2024-09-14T23:58:27.415 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4940000000000002e-06 2024-09-14T23:58:27.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.415 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.415 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:27.415 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300956+0000", 2024-09-14T23:58:27.415 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.622e-06 2024-09-14T23:58:27.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.415 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.415 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:27.415 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300962+0000", 2024-09-14T23:58:27.415 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5790000000000003e-06 2024-09-14T23:58:27.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.415 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.415 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:27.415 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352063+0000", 2024-09-14T23:58:27.416 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051101281999999998 2024-09-14T23:58:27.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.416 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.416 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:27.416 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352110+0000", 2024-09-14T23:58:27.416 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7401000000000002e-05 2024-09-14T23:58:27.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.416 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.416 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:27.416 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352163+0000", 2024-09-14T23:58:27.416 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3029999999999999e-05 2024-09-14T23:58:27.416 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:27.416 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:27.417 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:27.417 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.417 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.417 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69268 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5488640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:27.417 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300969+0000", 2024-09-14T23:58:27.417 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 72.086763833000006, 2024-09-14T23:58:27.417 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051413420000000001, 2024-09-14T23:58:27.417 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:27.417 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:27.417 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:27.417 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:27.417 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:27.417 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:27.417 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69268 2024-09-14T23:58:27.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.418 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:27.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.418 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:27.418 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300969+0000", 2024-09-14T23:58:27.418 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:27.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.418 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:27.418 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300969+0000", 2024-09-14T23:58:27.418 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:27.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.419 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:27.419 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300970+0000", 2024-09-14T23:58:27.419 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7899999999999999e-07 2024-09-14T23:58:27.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.419 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.419 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:27.419 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300982+0000", 2024-09-14T23:58:27.419 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2741e-05 2024-09-14T23:58:27.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.419 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.419 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:27.419 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300990+0000", 2024-09-14T23:58:27.419 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8169999999999999e-06 2024-09-14T23:58:27.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.420 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.420 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:27.420 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300995+0000", 2024-09-14T23:58:27.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2240000000000003e-06 2024-09-14T23:58:27.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.420 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.420 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:27.420 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352200+0000", 2024-09-14T23:58:27.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051204223 2024-09-14T23:58:27.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.420 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.420 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:27.420 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352285+0000", 2024-09-14T23:58:27.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5519999999999994e-05 2024-09-14T23:58:27.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.421 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.421 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:27.421 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352383+0000", 2024-09-14T23:58:27.421 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7516000000000006e-05 2024-09-14T23:58:27.421 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:27.421 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:27.421 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:27.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.421 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.421 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69269 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5603328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:27.421 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.301003+0000", 2024-09-14T23:58:27.421 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 72.086729650999999, 2024-09-14T23:58:27.421 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051497381000000002, 2024-09-14T23:58:27.422 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:27.422 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:27.422 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:27.422 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:27.422 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:27.422 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:27.422 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69269 2024-09-14T23:58:27.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.422 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:27.422 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.422 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:27.422 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.301003+0000", 2024-09-14T23:58:27.422 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:27.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.423 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:27.423 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.301003+0000", 2024-09-14T23:58:27.423 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:27.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.423 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:27.423 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302671+0000", 2024-09-14T23:58:27.423 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001667785 2024-09-14T23:58:27.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.423 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:27.423 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302676+0000", 2024-09-14T23:58:27.423 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6600000000000003e-06 2024-09-14T23:58:27.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:27.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302697+0000", 2024-09-14T23:58:27.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1288999999999999e-05 2024-09-14T23:58:27.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:27.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302716+0000", 2024-09-14T23:58:27.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9128999999999999e-05 2024-09-14T23:58:27.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:27.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352399+0000", 2024-09-14T23:58:27.425 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049682995000000001 2024-09-14T23:58:27.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.425 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:27.425 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352446+0000", 2024-09-14T23:58:27.425 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6341000000000002e-05 2024-09-14T23:58:27.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.425 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:27.425 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352501+0000", 2024-09-14T23:58:27.425 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5182000000000001e-05 2024-09-14T23:58:27.425 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:27.425 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:27.426 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:27.426 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.426 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.426 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69273 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6062080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:27.426 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302829+0000", 2024-09-14T23:58:27.426 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 72.084904085000005, 2024-09-14T23:58:27.426 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050384311000000001, 2024-09-14T23:58:27.426 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:27.426 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:27.426 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:27.426 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:27.426 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:27.426 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:27.426 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69273 2024-09-14T23:58:27.426 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.427 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:27.427 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.427 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:27.427 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302829+0000", 2024-09-14T23:58:27.427 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:27.427 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.427 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.427 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:27.427 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302829+0000", 2024-09-14T23:58:27.427 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:27.427 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.427 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.427 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:27.427 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302829+0000", 2024-09-14T23:58:27.427 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7700000000000005e-07 2024-09-14T23:58:27.428 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.428 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.428 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:27.428 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302832+0000", 2024-09-14T23:58:27.428 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6970000000000002e-06 2024-09-14T23:58:27.428 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.428 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.428 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:27.428 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302840+0000", 2024-09-14T23:58:27.428 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5719999999999997e-06 2024-09-14T23:58:27.428 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.428 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.428 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:27.428 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302845+0000", 2024-09-14T23:58:27.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7939999999999999e-06 2024-09-14T23:58:27.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.429 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:27.429 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353111+0000", 2024-09-14T23:58:27.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050265588 2024-09-14T23:58:27.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.429 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:27.429 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353160+0000", 2024-09-14T23:58:27.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8859000000000001e-05 2024-09-14T23:58:27.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.429 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:27.429 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353213+0000", 2024-09-14T23:58:27.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3324e-05 2024-09-14T23:58:27.430 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:27.430 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:27.430 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:27.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.430 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69274 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6176768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:27.430 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302854+0000", 2024-09-14T23:58:27.430 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 72.084878990999997, 2024-09-14T23:58:27.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050579526, 2024-09-14T23:58:27.430 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:27.430 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:27.430 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:27.430 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:27.430 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:27.431 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:27.431 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69274 2024-09-14T23:58:27.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.431 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:27.431 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.431 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:27.431 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302854+0000", 2024-09-14T23:58:27.431 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:27.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.431 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.431 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:27.431 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302854+0000", 2024-09-14T23:58:27.431 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:27.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.432 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:27.432 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302855+0000", 2024-09-14T23:58:27.432 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0200000000000001e-07 2024-09-14T23:58:27.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.432 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:27.432 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302857+0000", 2024-09-14T23:58:27.432 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.447e-06 2024-09-14T23:58:27.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.432 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:27.432 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302864+0000", 2024-09-14T23:58:27.433 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7730000000000002e-06 2024-09-14T23:58:27.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.433 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:27.433 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302869+0000", 2024-09-14T23:58:27.433 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3199999999999999e-06 2024-09-14T23:58:27.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.433 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:27.433 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353249+0000", 2024-09-14T23:58:27.433 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050379647999999999 2024-09-14T23:58:27.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.434 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:27.434 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353336+0000", 2024-09-14T23:58:27.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6767999999999993e-05 2024-09-14T23:58:27.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.434 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:27.434 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353433+0000", 2024-09-14T23:58:27.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7868000000000006e-05 2024-09-14T23:58:27.434 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:27.434 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:27.434 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:27.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.434 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69275 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6291456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:27.434 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302889+0000", 2024-09-14T23:58:27.435 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 72.084844372999996, 2024-09-14T23:58:27.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050663118, 2024-09-14T23:58:27.435 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:27.435 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:27.435 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:27.435 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:27.435 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:27.435 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:27.435 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69275 2024-09-14T23:58:27.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.435 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:27.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.435 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:27.435 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302889+0000", 2024-09-14T23:58:27.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:27.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.436 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:27.436 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302889+0000", 2024-09-14T23:58:27.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:27.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.436 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:27.436 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302889+0000", 2024-09-14T23:58:27.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6399999999999998e-07 2024-09-14T23:58:27.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.436 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:27.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302891+0000", 2024-09-14T23:58:27.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.339e-06 2024-09-14T23:58:27.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.437 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:27.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302898+0000", 2024-09-14T23:58:27.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0380000000000002e-06 2024-09-14T23:58:27.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.437 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:27.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302904+0000", 2024-09-14T23:58:27.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2159999999999997e-06 2024-09-14T23:58:27.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:27.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353449+0000", 2024-09-14T23:58:27.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050545386999999997 2024-09-14T23:58:27.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:27.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353494+0000", 2024-09-14T23:58:27.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4592999999999998e-05 2024-09-14T23:58:27.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:27.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353552+0000", 2024-09-14T23:58:27.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8180999999999997e-05 2024-09-14T23:58:27.438 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:27.439 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:27.439 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:27.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.439 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69276 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6406144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:27.439 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302911+0000", 2024-09-14T23:58:27.439 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 72.084821959999999, 2024-09-14T23:58:27.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050865753, 2024-09-14T23:58:27.439 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:27.439 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:27.439 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:27.439 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:27.439 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:27.439 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:27.439 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69276 2024-09-14T23:58:27.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.440 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:27.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.440 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:27.440 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302911+0000", 2024-09-14T23:58:27.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:27.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.440 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:27.440 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302911+0000", 2024-09-14T23:58:27.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:27.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.440 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:27.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302912+0000", 2024-09-14T23:58:27.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0800000000000004e-07 2024-09-14T23:58:27.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:27.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302914+0000", 2024-09-14T23:58:27.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9589999999999998e-06 2024-09-14T23:58:27.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:27.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302920+0000", 2024-09-14T23:58:27.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4579999999999998e-06 2024-09-14T23:58:27.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:27.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302925+0000", 2024-09-14T23:58:27.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9429999999999999e-06 2024-09-14T23:58:27.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:27.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353586+0000", 2024-09-14T23:58:27.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050660966000000002 2024-09-14T23:58:27.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:27.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353679+0000", 2024-09-14T23:58:27.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3163000000000002e-05 2024-09-14T23:58:27.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:27.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353777+0000", 2024-09-14T23:58:27.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7555999999999999e-05 2024-09-14T23:58:27.443 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:27.443 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:27.443 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:27.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.443 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69277 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6520832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:27.443 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302932+0000", 2024-09-14T23:58:27.443 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 72.084800873000006, 2024-09-14T23:58:27.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050977674000000001, 2024-09-14T23:58:27.443 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:27.443 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:27.443 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:27.443 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:27.443 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:27.443 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:27.444 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69277 2024-09-14T23:58:27.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.444 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:27.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:27.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302932+0000", 2024-09-14T23:58:27.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:27.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:27.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302932+0000", 2024-09-14T23:58:27.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:27.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:27.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302932+0000", 2024-09-14T23:58:27.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3299999999999998e-07 2024-09-14T23:58:27.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:27.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302935+0000", 2024-09-14T23:58:27.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2139999999999999e-06 2024-09-14T23:58:27.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:27.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302941+0000", 2024-09-14T23:58:27.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5039999999999999e-06 2024-09-14T23:58:27.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.446 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:27.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302946+0000", 2024-09-14T23:58:27.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2079999999999999e-06 2024-09-14T23:58:27.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.446 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:27.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353805+0000", 2024-09-14T23:58:27.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050858588000000003 2024-09-14T23:58:27.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.446 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:27.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353849+0000", 2024-09-14T23:58:27.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4511000000000001e-05 2024-09-14T23:58:27.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:27.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353910+0000", 2024-09-14T23:58:27.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0316000000000003e-05 2024-09-14T23:58:27.447 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:27.447 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:27.447 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:27.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.447 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69278 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6635520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:27.447 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302953+0000", 2024-09-14T23:58:27.447 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 72.084779745000006, 2024-09-14T23:58:27.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051174059000000001, 2024-09-14T23:58:27.448 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:27.448 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:27.448 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:27.448 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:27.448 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:27.448 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:27.448 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69278 2024-09-14T23:58:27.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.448 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:27.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:27.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302953+0000", 2024-09-14T23:58:27.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:27.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:27.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302953+0000", 2024-09-14T23:58:27.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:27.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:27.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302954+0000", 2024-09-14T23:58:27.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5100000000000001e-07 2024-09-14T23:58:27.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:27.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302964+0000", 2024-09-14T23:58:27.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0475000000000001e-05 2024-09-14T23:58:27.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.450 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:27.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302970+0000", 2024-09-14T23:58:27.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3879999999999997e-06 2024-09-14T23:58:27.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.450 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:27.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302975+0000", 2024-09-14T23:58:27.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8409999999999999e-06 2024-09-14T23:58:27.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.450 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:27.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353944+0000", 2024-09-14T23:58:27.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050968952999999997 2024-09-14T23:58:27.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:27.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354031+0000", 2024-09-14T23:58:27.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6537000000000002e-05 2024-09-14T23:58:27.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:27.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354127+0000", 2024-09-14T23:58:27.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6514000000000005e-05 2024-09-14T23:58:27.451 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:27.451 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:27.451 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:27.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.452 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69279 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6750208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:27.452 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302984+0000", 2024-09-14T23:58:27.452 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 72.084749393999999, 2024-09-14T23:58:27.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051258845999999997, 2024-09-14T23:58:27.452 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:27.452 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:27.452 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:27.452 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:27.452 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:27.452 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:27.452 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69279 2024-09-14T23:58:27.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.452 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:27.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:27.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302984+0000", 2024-09-14T23:58:27.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:27.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:27.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302984+0000", 2024-09-14T23:58:27.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:27.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:27.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303025+0000", 2024-09-14T23:58:27.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1690999999999998e-05 2024-09-14T23:58:27.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:27.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303029+0000", 2024-09-14T23:58:27.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6559999999999998e-06 2024-09-14T23:58:27.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:27.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303037+0000", 2024-09-14T23:58:27.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7950000000000008e-06 2024-09-14T23:58:27.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:27.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303043+0000", 2024-09-14T23:58:27.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9669999999999998e-06 2024-09-14T23:58:27.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:27.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354142+0000", 2024-09-14T23:58:27.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051099072000000002 2024-09-14T23:58:27.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:27.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354183+0000", 2024-09-14T23:58:27.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0834000000000002e-05 2024-09-14T23:58:27.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:27.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354242+0000", 2024-09-14T23:58:27.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9830999999999997e-05 2024-09-14T23:58:27.456 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:27.456 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:27.456 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:27.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.456 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69280 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6864896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:27.456 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.303052+0000", 2024-09-14T23:58:27.456 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 72.084680851000002, 2024-09-14T23:58:27.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051406463999999999, 2024-09-14T23:58:27.456 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:27.457 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:27.457 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:27.457 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:27.458 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:27.458 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:27.458 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69280 2024-09-14T23:58:27.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.458 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:27.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:27.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303052+0000", 2024-09-14T23:58:27.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:27.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:27.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303052+0000", 2024-09-14T23:58:27.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:27.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:27.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304673+0000", 2024-09-14T23:58:27.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001621389 2024-09-14T23:58:27.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:27.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304681+0000", 2024-09-14T23:58:27.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8439999999999998e-06 2024-09-14T23:58:27.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:27.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304691+0000", 2024-09-14T23:58:27.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8339999999999994e-06 2024-09-14T23:58:27.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:27.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304698+0000", 2024-09-14T23:58:27.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3629999999999999e-06 2024-09-14T23:58:27.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:27.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354278+0000", 2024-09-14T23:58:27.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049580618 2024-09-14T23:58:27.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:27.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354364+0000", 2024-09-14T23:58:27.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6059999999999999e-05 2024-09-14T23:58:27.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:27.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354459+0000", 2024-09-14T23:58:27.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4356000000000003e-05 2024-09-14T23:58:27.462 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:27.462 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:27.462 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:27.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.462 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69285 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 7438336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:27.462 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.304792+0000", 2024-09-14T23:58:27.462 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 72.082941442000006, 2024-09-14T23:58:27.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050398874000000003, 2024-09-14T23:58:27.462 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:27.463 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:27.463 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:27.463 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:27.463 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:27.463 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:27.463 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69285 2024-09-14T23:58:27.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.463 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:27.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:27.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304792+0000", 2024-09-14T23:58:27.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:27.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:27.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304792+0000", 2024-09-14T23:58:27.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:27.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:27.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304792+0000", 2024-09-14T23:58:27.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2800000000000003e-07 2024-09-14T23:58:27.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:27.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304794+0000", 2024-09-14T23:58:27.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8700000000000001e-06 2024-09-14T23:58:27.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:27.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304800+0000", 2024-09-14T23:58:27.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0889999999999996e-06 2024-09-14T23:58:27.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:27.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304805+0000", 2024-09-14T23:58:27.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7060000000000003e-06 2024-09-14T23:58:27.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:27.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355108+0000", 2024-09-14T23:58:27.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050303170000000001 2024-09-14T23:58:27.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:27.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355150+0000", 2024-09-14T23:58:27.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2271999999999998e-05 2024-09-14T23:58:27.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:27.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355190+0000", 2024-09-14T23:58:27.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0438999999999997e-05 2024-09-14T23:58:27.467 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:27.467 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:27.467 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:27.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.467 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69286 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 7553024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:27.467 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.304811+0000", 2024-09-14T23:58:27.468 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 72.082922080000003, 2024-09-14T23:58:27.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050585512999999999, 2024-09-14T23:58:27.468 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:27.468 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:27.468 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:27.468 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:27.468 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:27.468 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:27.468 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69286 2024-09-14T23:58:27.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.468 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:27.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.469 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:27.469 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304811+0000", 2024-09-14T23:58:27.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:27.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.469 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:27.469 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304811+0000", 2024-09-14T23:58:27.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:27.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:27.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304811+0000", 2024-09-14T23:58:27.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.89e-07 2024-09-14T23:58:27.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:27.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304814+0000", 2024-09-14T23:58:27.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3199999999999998e-06 2024-09-14T23:58:27.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:27.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304821+0000", 2024-09-14T23:58:27.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5689999999999999e-06 2024-09-14T23:58:27.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:27.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304826+0000", 2024-09-14T23:58:27.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1200000000000001e-06 2024-09-14T23:58:27.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:27.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355226+0000", 2024-09-14T23:58:27.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050399653000000003 2024-09-14T23:58:27.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:27.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355314+0000", 2024-09-14T23:58:27.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8389999999999999e-05 2024-09-14T23:58:27.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:27.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355396+0000", 2024-09-14T23:58:27.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2071999999999996e-05 2024-09-14T23:58:27.473 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:27.473 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:27.473 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:27.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.473 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69287 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 7667712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:27.473 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.304833+0000", 2024-09-14T23:58:27.473 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 72.082899482000002, 2024-09-14T23:58:27.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050664740999999999, 2024-09-14T23:58:27.473 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:27.473 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:27.473 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:27.474 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:27.474 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:27.474 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:27.474 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69287 2024-09-14T23:58:27.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.474 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:27.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:27.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304833+0000", 2024-09-14T23:58:27.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:27.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:27.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304833+0000", 2024-09-14T23:58:27.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:27.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:27.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304834+0000", 2024-09-14T23:58:27.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4999999999999998e-07 2024-09-14T23:58:27.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:27.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304836+0000", 2024-09-14T23:58:27.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0279999999999999e-06 2024-09-14T23:58:27.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:27.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304842+0000", 2024-09-14T23:58:27.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8919999999999997e-06 2024-09-14T23:58:27.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:27.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304847+0000", 2024-09-14T23:58:27.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6160000000000001e-06 2024-09-14T23:58:27.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:27.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355411+0000", 2024-09-14T23:58:27.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050563540999999997 2024-09-14T23:58:27.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:27.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355459+0000", 2024-09-14T23:58:27.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7979000000000003e-05 2024-09-14T23:58:27.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:27.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355498+0000", 2024-09-14T23:58:27.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9335000000000002e-05 2024-09-14T23:58:27.478 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:27.478 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:27.478 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:27.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.478 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69288 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 7782400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:27.478 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.304854+0000", 2024-09-14T23:58:27.478 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 72.082879043999995, 2024-09-14T23:58:27.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050855145999999997, 2024-09-14T23:58:27.479 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:27.479 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:27.479 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:27.479 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:27.479 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:27.479 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:27.479 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69288 2024-09-14T23:58:27.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.479 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:27.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:27.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304854+0000", 2024-09-14T23:58:27.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:27.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:27.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304854+0000", 2024-09-14T23:58:27.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:27.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:27.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.306670+0000", 2024-09-14T23:58:27.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00181631 2024-09-14T23:58:27.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:27.481 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.306675+0000", 2024-09-14T23:58:27.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0220000000000004e-06 2024-09-14T23:58:27.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:27.481 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.306684+0000", 2024-09-14T23:58:27.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7309999999999999e-06 2024-09-14T23:58:27.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:27.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.306689+0000", 2024-09-14T23:58:27.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6609999999999999e-06 2024-09-14T23:58:27.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:27.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355533+0000", 2024-09-14T23:58:27.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048844578 2024-09-14T23:58:27.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:27.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355622+0000", 2024-09-14T23:58:27.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9006999999999997e-05 2024-09-14T23:58:27.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:27.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:27.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:27.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355709+0000", 2024-09-14T23:58:27.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6836999999999995e-05 2024-09-14T23:58:27.483 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:27.483 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:27.483 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:27.483 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:27.483 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:27.484 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:27.485 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T23:58:27.486 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:27.483+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:27.586 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:27.584+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:27.593 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:27.593 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:27.593 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:58:27.593 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:27.602 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T23:58:27.687 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:27.686+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:27.749 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:27.749 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:27.749 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:58:27.749 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:58:27.749 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:27.758 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T23:58:27.788 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:58:27.787+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:58:27.889 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:27.888+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:27.906 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:27.906 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:58:27.906 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:58:27.906 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:58:27.906 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.906 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:43054 3.1f 3:ffbfe596:::rbd_data.1118350d5ae5.00000000000002eb:head [set-alloc-hint object_size 262144 write_size 262144,write 196608~22770] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T23:58:27.906 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:58:27.906 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 94.937240520000003, 2024-09-14T23:58:27.907 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.99973334400000002, 2024-09-14T23:58:27.907 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:27.907 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:27.907 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:27.907 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:27.907 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:27.907 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:27.907 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 43054 2024-09-14T23:58:27.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.907 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:27.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.907 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:27.907 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:58:27.907 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:27.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.908 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:27.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:58:27.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:27.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.908 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:27.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966552+0000", 2024-09-14T23:58:27.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3790000000000001e-06 2024-09-14T23:58:27.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.908 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:27.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966565+0000", 2024-09-14T23:58:27.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2459999999999999e-05 2024-09-14T23:58:27.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:27.909 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966576+0000", 2024-09-14T23:58:27.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1039000000000001e-05 2024-09-14T23:58:27.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:27.909 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966582+0000", 2024-09-14T23:58:27.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9660000000000001e-06 2024-09-14T23:58:27.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:27.909 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966625+0000", 2024-09-14T23:58:27.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3529999999999998e-05 2024-09-14T23:58:27.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.910 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.910 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:58:27.910 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966632+0000", 2024-09-14T23:58:27.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9809999999999997e-06 2024-09-14T23:58:27.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.910 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.910 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:27.910 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.964460+0000", 2024-09-14T23:58:27.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.99782768300000002 2024-09-14T23:58:27.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.911 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.911 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:27.911 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.964641+0000", 2024-09-14T23:58:27.911 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000181089 2024-09-14T23:58:27.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.911 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.911 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-14T23:58:27.911 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.964809+0000", 2024-09-14T23:58:27.911 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000168505 2024-09-14T23:58:27.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.911 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.911 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:58:27.911 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966030+0000", 2024-09-14T23:58:27.912 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012206929999999999 2024-09-14T23:58:27.912 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.912 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.912 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:27.912 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966181+0000", 2024-09-14T23:58:27.912 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000150569 2024-09-14T23:58:27.912 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.912 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.912 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:27.912 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966243+0000", 2024-09-14T23:58:27.912 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2331999999999999e-05 2024-09-14T23:58:27.912 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.912 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.912 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:27.913 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966284+0000", 2024-09-14T23:58:27.913 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1118000000000002e-05 2024-09-14T23:58:27.913 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:27.913 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:27.913 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:27.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.913 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184314 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4162560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:27.913 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.077247+0000", 2024-09-14T23:58:27.913 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.826543852, 2024-09-14T23:58:27.913 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058772256000000002, 2024-09-14T23:58:27.913 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:27.913 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:27.913 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:27.914 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:27.914 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:27.914 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:27.914 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184314 2024-09-14T23:58:27.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.914 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:27.914 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.914 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:27.914 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.077247+0000", 2024-09-14T23:58:27.914 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:27.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.914 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.914 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:27.915 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.077247+0000", 2024-09-14T23:58:27.915 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:27.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.915 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.915 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:27.915 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.078906+0000", 2024-09-14T23:58:27.915 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016583279999999999 2024-09-14T23:58:27.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.915 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.915 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:27.915 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.078911+0000", 2024-09-14T23:58:27.915 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0420000000000002e-06 2024-09-14T23:58:27.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.915 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.915 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:27.915 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.078922+0000", 2024-09-14T23:58:27.916 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1354e-05 2024-09-14T23:58:27.916 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.916 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.916 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:27.916 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.078930+0000", 2024-09-14T23:58:27.916 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5730000000000002e-06 2024-09-14T23:58:27.916 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.916 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.916 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:27.916 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.135870+0000", 2024-09-14T23:58:27.916 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056940651000000002 2024-09-14T23:58:27.916 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.916 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.916 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:27.917 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.135951+0000", 2024-09-14T23:58:27.917 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0890999999999995e-05 2024-09-14T23:58:27.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.917 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.917 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:27.917 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.136020+0000", 2024-09-14T23:58:27.917 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8417000000000006e-05 2024-09-14T23:58:27.917 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:27.917 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:27.917 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:27.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.917 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.917 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184323 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4183040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:27.917 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.079228+0000", 2024-09-14T23:58:27.918 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.824562778000001, 2024-09-14T23:58:27.918 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058454487999999999, 2024-09-14T23:58:27.918 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:27.918 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:27.918 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:27.918 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:27.918 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:27.918 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:27.918 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184323 2024-09-14T23:58:27.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.918 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:27.918 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.918 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:27.918 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079228+0000", 2024-09-14T23:58:27.919 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:27.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.919 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.919 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:27.919 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079228+0000", 2024-09-14T23:58:27.919 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:27.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.919 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.919 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:27.919 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079229+0000", 2024-09-14T23:58:27.919 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5299999999999999e-07 2024-09-14T23:58:27.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.919 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.920 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:27.920 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079232+0000", 2024-09-14T23:58:27.920 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8839999999999998e-06 2024-09-14T23:58:27.920 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.920 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.920 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:27.920 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079238+0000", 2024-09-14T23:58:27.920 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2299999999999996e-06 2024-09-14T23:58:27.920 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.920 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.920 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:27.920 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079243+0000", 2024-09-14T23:58:27.920 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3029999999999999e-06 2024-09-14T23:58:27.921 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.921 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.921 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:27.921 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.137533+0000", 2024-09-14T23:58:27.921 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058289674999999999 2024-09-14T23:58:27.921 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.921 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.921 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:27.921 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.137615+0000", 2024-09-14T23:58:27.921 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2288999999999998e-05 2024-09-14T23:58:27.921 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.921 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.921 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:27.922 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.137683+0000", 2024-09-14T23:58:27.922 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7654e-05 2024-09-14T23:58:27.922 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:27.922 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:27.922 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:27.922 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.922 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.922 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184324 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4073472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:27.922 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.079251+0000", 2024-09-14T23:58:27.922 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.824540311, 2024-09-14T23:58:27.922 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058609383000000001, 2024-09-14T23:58:27.922 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:27.922 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:27.923 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:27.923 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:27.923 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:27.923 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:27.923 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184324 2024-09-14T23:58:27.923 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.923 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:27.923 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.923 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:27.923 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079251+0000", 2024-09-14T23:58:27.923 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:27.923 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.923 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.923 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:27.924 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079251+0000", 2024-09-14T23:58:27.924 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:27.924 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.924 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.924 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:27.924 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.080920+0000", 2024-09-14T23:58:27.924 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016687430000000001 2024-09-14T23:58:27.924 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.924 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.924 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:27.924 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.080924+0000", 2024-09-14T23:58:27.924 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3869999999999997e-06 2024-09-14T23:58:27.925 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.925 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.925 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:27.925 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.080935+0000", 2024-09-14T23:58:27.925 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0515e-05 2024-09-14T23:58:27.925 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.925 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.925 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:27.925 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.080942+0000", 2024-09-14T23:58:27.925 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0990000000000001e-06 2024-09-14T23:58:27.925 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.925 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.925 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:27.926 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.137713+0000", 2024-09-14T23:58:27.926 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056771288000000003 2024-09-14T23:58:27.926 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.926 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.926 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:27.926 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.137794+0000", 2024-09-14T23:58:27.926 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0613999999999998e-05 2024-09-14T23:58:27.926 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.926 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.926 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:27.926 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.137860+0000", 2024-09-14T23:58:27.926 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6736999999999995e-05 2024-09-14T23:58:27.926 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:27.926 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:27.927 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:27.927 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.927 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.927 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184346 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4099072~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:27.927 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.083616+0000", 2024-09-14T23:58:27.927 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.820175125999999, 2024-09-14T23:58:27.927 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058557419999999999, 2024-09-14T23:58:27.927 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:27.927 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:27.927 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:27.927 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:27.927 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:27.927 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:27.927 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184346 2024-09-14T23:58:27.927 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.928 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:27.928 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.928 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:27.928 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083616+0000", 2024-09-14T23:58:27.928 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:27.928 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.928 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.928 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:27.928 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083616+0000", 2024-09-14T23:58:27.928 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:27.928 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.928 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.928 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:27.928 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083620+0000", 2024-09-14T23:58:27.929 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0119999999999999e-06 2024-09-14T23:58:27.929 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.929 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.929 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:27.929 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083623+0000", 2024-09-14T23:58:27.929 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9059999999999998e-06 2024-09-14T23:58:27.929 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.929 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.929 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:27.929 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083631+0000", 2024-09-14T23:58:27.929 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2679999999999994e-06 2024-09-14T23:58:27.929 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.929 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.929 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:27.929 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083637+0000", 2024-09-14T23:58:27.930 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1210000000000003e-06 2024-09-14T23:58:27.930 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.930 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.930 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:27.930 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142018+0000", 2024-09-14T23:58:27.930 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058381111999999999 2024-09-14T23:58:27.930 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.930 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.930 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:27.930 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142100+0000", 2024-09-14T23:58:27.930 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1308000000000001e-05 2024-09-14T23:58:27.930 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.930 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.930 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:27.930 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142173+0000", 2024-09-14T23:58:27.931 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3693000000000001e-05 2024-09-14T23:58:27.931 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:27.931 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:27.931 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:27.931 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.931 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.931 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184347 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4122624~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:27.931 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.083647+0000", 2024-09-14T23:58:27.931 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.820144295999999, 2024-09-14T23:58:27.931 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058706388999999998, 2024-09-14T23:58:27.931 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:27.931 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:27.931 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:27.931 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:27.931 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:27.931 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:27.932 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184347 2024-09-14T23:58:27.932 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.932 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:27.932 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.932 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:27.932 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083647+0000", 2024-09-14T23:58:27.932 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:27.932 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.932 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.932 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:27.932 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083647+0000", 2024-09-14T23:58:27.932 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:27.932 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.932 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.933 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:27.933 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083647+0000", 2024-09-14T23:58:27.933 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3099999999999998e-07 2024-09-14T23:58:27.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.933 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.933 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:27.933 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083650+0000", 2024-09-14T23:58:27.933 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2900000000000001e-06 2024-09-14T23:58:27.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.933 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.933 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:27.933 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083657+0000", 2024-09-14T23:58:27.933 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.486e-06 2024-09-14T23:58:27.934 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.934 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.934 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:27.934 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083663+0000", 2024-09-14T23:58:27.934 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4330000000000003e-06 2024-09-14T23:58:27.934 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.934 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.934 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:27.934 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142206+0000", 2024-09-14T23:58:27.934 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058543237999999997 2024-09-14T23:58:27.934 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.934 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.934 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:27.935 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142284+0000", 2024-09-14T23:58:27.935 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8487999999999998e-05 2024-09-14T23:58:27.935 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.935 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.935 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:27.935 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142353+0000", 2024-09-14T23:58:27.935 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9023000000000006e-05 2024-09-14T23:58:27.935 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:27.935 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:27.935 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:27.935 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.935 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.935 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184348 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4103168~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:27.935 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.083669+0000", 2024-09-14T23:58:27.935 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.820121878999998, 2024-09-14T23:58:27.936 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058864232000000002, 2024-09-14T23:58:27.936 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:27.936 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:27.936 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:27.936 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:27.936 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:27.936 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:27.936 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184348 2024-09-14T23:58:27.936 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.936 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:27.936 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.936 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:27.936 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083669+0000", 2024-09-14T23:58:27.936 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:27.936 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.937 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.937 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:27.937 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083669+0000", 2024-09-14T23:58:27.937 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:27.937 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.937 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.937 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:27.937 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083670+0000", 2024-09-14T23:58:27.937 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9999999999999998e-07 2024-09-14T23:58:27.937 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.937 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.937 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:27.937 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083672+0000", 2024-09-14T23:58:27.937 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1430000000000001e-06 2024-09-14T23:58:27.938 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.938 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.938 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:27.938 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083678+0000", 2024-09-14T23:58:27.938 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5200000000000003e-06 2024-09-14T23:58:27.938 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.938 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.938 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:27.938 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083683+0000", 2024-09-14T23:58:27.938 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9359999999999998e-06 2024-09-14T23:58:27.938 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.938 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.938 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:27.938 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142384+0000", 2024-09-14T23:58:27.938 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058701159000000003 2024-09-14T23:58:27.939 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.939 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.939 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:27.939 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142466+0000", 2024-09-14T23:58:27.939 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1024000000000002e-05 2024-09-14T23:58:27.939 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.939 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.939 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:27.939 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142534+0000", 2024-09-14T23:58:27.939 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8050000000000001e-05 2024-09-14T23:58:27.939 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:27.939 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:27.939 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:27.940 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.940 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.940 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184349 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4126720~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:27.940 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.083690+0000", 2024-09-14T23:58:27.940 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.820101344000001, 2024-09-14T23:58:27.940 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059019911000000001, 2024-09-14T23:58:27.940 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:27.940 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:27.940 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:27.940 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:27.940 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:27.940 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:27.940 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184349 2024-09-14T23:58:27.940 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.941 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:27.941 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.941 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:27.941 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083690+0000", 2024-09-14T23:58:27.941 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:27.941 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.941 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.941 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:27.941 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083690+0000", 2024-09-14T23:58:27.941 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:27.941 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.941 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.942 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:27.942 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.084900+0000", 2024-09-14T23:58:27.942 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012099770000000001 2024-09-14T23:58:27.942 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.942 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.942 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:27.942 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.084905+0000", 2024-09-14T23:58:27.942 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.028e-06 2024-09-14T23:58:27.942 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.942 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.942 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:27.942 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.084914+0000", 2024-09-14T23:58:27.942 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6619999999999994e-06 2024-09-14T23:58:27.942 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.943 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.943 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:27.943 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.084922+0000", 2024-09-14T23:58:27.943 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5720000000000003e-06 2024-09-14T23:58:27.943 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.943 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.943 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:27.943 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142565+0000", 2024-09-14T23:58:27.943 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057642605999999999 2024-09-14T23:58:27.943 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.943 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.943 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:27.943 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142644+0000", 2024-09-14T23:58:27.943 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9370999999999996e-05 2024-09-14T23:58:27.944 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.944 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.944 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:27.944 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142710+0000", 2024-09-14T23:58:27.944 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5695e-05 2024-09-14T23:58:27.944 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:27.944 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:27.944 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:27.944 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.944 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.944 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184354 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4115456~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:27.944 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.085029+0000", 2024-09-14T23:58:27.944 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.818762694, 2024-09-14T23:58:27.944 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058626327999999998, 2024-09-14T23:58:27.945 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:27.945 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:27.945 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:27.945 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:27.945 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:27.945 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:27.945 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184354 2024-09-14T23:58:27.945 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.945 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:27.945 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.945 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:27.945 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085029+0000", 2024-09-14T23:58:27.945 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:27.945 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.945 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.946 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:27.946 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085029+0000", 2024-09-14T23:58:27.946 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:27.946 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.946 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.946 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:27.946 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085029+0000", 2024-09-14T23:58:27.946 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0800000000000001e-07 2024-09-14T23:58:27.946 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.946 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.946 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:27.946 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085030+0000", 2024-09-14T23:58:27.946 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.226e-06 2024-09-14T23:58:27.946 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.946 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.947 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:27.947 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085033+0000", 2024-09-14T23:58:27.947 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9569999999999998e-06 2024-09-14T23:58:27.947 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.947 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.947 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:27.947 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085035+0000", 2024-09-14T23:58:27.947 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.587e-06 2024-09-14T23:58:27.947 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.947 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.947 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:27.947 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143491+0000", 2024-09-14T23:58:27.948 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058455337000000003 2024-09-14T23:58:27.948 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.948 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.948 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:27.948 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143575+0000", 2024-09-14T23:58:27.948 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3850000000000005e-05 2024-09-14T23:58:27.948 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.948 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.948 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:27.948 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143655+0000", 2024-09-14T23:58:27.948 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0162999999999998e-05 2024-09-14T23:58:27.948 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:27.948 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:27.948 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:27.949 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.949 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.949 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184355 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4139008~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:27.949 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.085039+0000", 2024-09-14T23:58:27.949 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.818752289999999, 2024-09-14T23:58:27.949 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058801369999999999, 2024-09-14T23:58:27.949 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:27.949 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:27.949 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:27.949 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:27.949 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:27.949 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:27.949 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184355 2024-09-14T23:58:27.949 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.949 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:27.950 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.950 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:27.950 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085039+0000", 2024-09-14T23:58:27.950 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:27.950 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.950 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.950 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:27.950 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085039+0000", 2024-09-14T23:58:27.950 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:27.950 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.950 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.950 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:27.951 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085039+0000", 2024-09-14T23:58:27.951 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.03e-07 2024-09-14T23:58:27.951 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.951 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.951 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:27.951 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085040+0000", 2024-09-14T23:58:27.951 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.158e-06 2024-09-14T23:58:27.951 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.951 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.951 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:27.951 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085043+0000", 2024-09-14T23:58:27.951 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0240000000000002e-06 2024-09-14T23:58:27.951 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.951 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.951 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:27.952 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085046+0000", 2024-09-14T23:58:27.952 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4600000000000002e-06 2024-09-14T23:58:27.952 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.952 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.952 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:27.952 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143688+0000", 2024-09-14T23:58:27.952 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058642356 2024-09-14T23:58:27.952 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.952 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.952 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:27.952 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143766+0000", 2024-09-14T23:58:27.952 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8219000000000004e-05 2024-09-14T23:58:27.952 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.952 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.953 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:27.953 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143840+0000", 2024-09-14T23:58:27.953 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3949999999999995e-05 2024-09-14T23:58:27.953 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:27.953 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:27.953 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:27.953 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.953 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.953 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184356 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4119552~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:27.953 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.085049+0000", 2024-09-14T23:58:27.953 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.818742118999999, 2024-09-14T23:58:27.953 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058977992, 2024-09-14T23:58:27.953 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:27.954 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:27.954 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:27.954 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:27.954 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:27.954 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:27.954 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184356 2024-09-14T23:58:27.954 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.954 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:27.954 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.954 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:27.954 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085049+0000", 2024-09-14T23:58:27.954 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:27.954 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.954 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.954 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:27.955 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085049+0000", 2024-09-14T23:58:27.955 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:27.955 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.955 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.955 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:27.955 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.086934+0000", 2024-09-14T23:58:27.955 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018854189999999999 2024-09-14T23:58:27.955 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.955 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.955 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:27.955 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.086939+0000", 2024-09-14T23:58:27.955 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.493e-06 2024-09-14T23:58:27.955 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.955 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.955 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:27.956 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.086951+0000", 2024-09-14T23:58:27.956 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1969e-05 2024-09-14T23:58:27.956 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.956 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.956 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:27.956 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.086959+0000", 2024-09-14T23:58:27.956 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5780000000000001e-06 2024-09-14T23:58:27.956 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.956 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.956 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:27.956 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143871+0000", 2024-09-14T23:58:27.956 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056912114 2024-09-14T23:58:27.956 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.956 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.957 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:27.957 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143951+0000", 2024-09-14T23:58:27.957 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9938000000000006e-05 2024-09-14T23:58:27.957 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.957 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.957 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:27.957 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.144027+0000", 2024-09-14T23:58:27.957 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6481000000000002e-05 2024-09-14T23:58:27.957 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:27.957 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:27.957 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:27.957 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.957 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.957 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184364 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4135936~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:27.958 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.087207+0000", 2024-09-14T23:58:27.958 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.816583839, 2024-09-14T23:58:27.958 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058466420999999998, 2024-09-14T23:58:27.958 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:27.958 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:27.958 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:27.958 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:27.958 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:27.958 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:27.958 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184364 2024-09-14T23:58:27.958 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.958 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:27.958 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.958 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:27.958 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087207+0000", 2024-09-14T23:58:27.959 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:27.959 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.959 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.959 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:27.959 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087207+0000", 2024-09-14T23:58:27.959 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:27.959 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.959 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.959 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:27.959 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087208+0000", 2024-09-14T23:58:27.959 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.35e-07 2024-09-14T23:58:27.959 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.959 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.959 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:27.959 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087209+0000", 2024-09-14T23:58:27.959 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.573e-06 2024-09-14T23:58:27.960 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.960 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.960 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:27.960 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087212+0000", 2024-09-14T23:58:27.960 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2770000000000001e-06 2024-09-14T23:58:27.960 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.960 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.960 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:27.960 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087215+0000", 2024-09-14T23:58:27.960 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5500000000000001e-06 2024-09-14T23:58:27.960 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.960 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.960 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:27.960 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145520+0000", 2024-09-14T23:58:27.961 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058304835999999999 2024-09-14T23:58:27.961 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.961 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.961 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:27.961 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145602+0000", 2024-09-14T23:58:27.961 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1714999999999999e-05 2024-09-14T23:58:27.961 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.961 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.961 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:27.961 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145674+0000", 2024-09-14T23:58:27.961 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2235000000000002e-05 2024-09-14T23:58:27.961 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:27.961 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:27.961 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:27.962 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.962 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.962 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184365 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4159488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:27.962 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.087219+0000", 2024-09-14T23:58:27.962 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.816571807999999, 2024-09-14T23:58:27.962 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058643533999999997, 2024-09-14T23:58:27.962 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:27.962 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:27.962 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:27.962 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:27.962 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:27.962 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:27.962 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184365 2024-09-14T23:58:27.962 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.963 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:27.963 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.963 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:27.963 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087219+0000", 2024-09-14T23:58:27.963 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:27.963 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.963 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.963 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:27.963 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087219+0000", 2024-09-14T23:58:27.963 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:27.963 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.963 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.963 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:27.964 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087220+0000", 2024-09-14T23:58:27.964 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.39e-07 2024-09-14T23:58:27.964 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.964 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.964 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:27.964 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087222+0000", 2024-09-14T23:58:27.964 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0609999999999998e-06 2024-09-14T23:58:27.964 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.964 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.964 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:27.964 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087228+0000", 2024-09-14T23:58:27.964 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8590000000000001e-06 2024-09-14T23:58:27.964 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.965 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:27.965 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087232+0000", 2024-09-14T23:58:27.965 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8149999999999996e-06 2024-09-14T23:58:27.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.965 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:27.965 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145705+0000", 2024-09-14T23:58:27.965 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058472532000000001 2024-09-14T23:58:27.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.965 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:27.965 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145784+0000", 2024-09-14T23:58:27.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9382000000000007e-05 2024-09-14T23:58:27.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.966 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:27.966 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145863+0000", 2024-09-14T23:58:27.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8546000000000003e-05 2024-09-14T23:58:27.966 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:27.966 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:27.966 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:27.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.966 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184366 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4140032~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:27.966 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.087240+0000", 2024-09-14T23:58:27.966 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.816551359999998, 2024-09-14T23:58:27.967 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058818068000000001, 2024-09-14T23:58:27.967 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:27.967 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:27.967 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:27.967 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:27.967 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:27.967 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:27.967 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184366 2024-09-14T23:58:27.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.967 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:27.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.967 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:27.967 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087240+0000", 2024-09-14T23:58:27.967 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:27.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.968 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.968 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:27.968 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087240+0000", 2024-09-14T23:58:27.968 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:27.968 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.968 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.968 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:27.968 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.088904+0000", 2024-09-14T23:58:27.968 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001664315 2024-09-14T23:58:27.968 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.968 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.968 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:27.968 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.088909+0000", 2024-09-14T23:58:27.968 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7709999999999997e-06 2024-09-14T23:58:27.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.969 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.969 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:27.969 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.088921+0000", 2024-09-14T23:58:27.969 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1661999999999999e-05 2024-09-14T23:58:27.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.969 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.969 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:27.969 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.088928+0000", 2024-09-14T23:58:27.969 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8189999999999992e-06 2024-09-14T23:58:27.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.969 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.969 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:27.969 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145901+0000", 2024-09-14T23:58:27.970 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056972478 2024-09-14T23:58:27.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.970 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.970 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:27.970 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145977+0000", 2024-09-14T23:58:27.970 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5763e-05 2024-09-14T23:58:27.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.970 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.970 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:27.970 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.146058+0000", 2024-09-14T23:58:27.970 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1260000000000005e-05 2024-09-14T23:58:27.970 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:27.971 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:27.971 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:27.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.971 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184375 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4179968~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:27.971 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.089199+0000", 2024-09-14T23:58:27.971 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.814592022999999, 2024-09-14T23:58:27.971 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058597445999999997, 2024-09-14T23:58:27.971 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:27.971 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:27.971 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:27.971 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:27.971 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:27.971 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:27.972 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184375 2024-09-14T23:58:27.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.972 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:27.972 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.972 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:27.972 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089199+0000", 2024-09-14T23:58:27.972 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:27.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.972 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.972 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:27.972 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089199+0000", 2024-09-14T23:58:27.972 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:27.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.972 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.973 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:27.973 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089200+0000", 2024-09-14T23:58:27.973 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8100000000000003e-07 2024-09-14T23:58:27.973 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.973 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.973 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:27.973 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089202+0000", 2024-09-14T23:58:27.973 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7599999999999998e-06 2024-09-14T23:58:27.973 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.973 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.973 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:27.973 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089209+0000", 2024-09-14T23:58:27.973 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7279999999999998e-06 2024-09-14T23:58:27.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.974 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.974 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:27.974 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089215+0000", 2024-09-14T23:58:27.974 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5090000000000001e-06 2024-09-14T23:58:27.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.974 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.974 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:27.974 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147638+0000", 2024-09-14T23:58:27.974 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058423105000000003 2024-09-14T23:58:27.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.974 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.974 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:27.975 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147719+0000", 2024-09-14T23:58:27.975 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1217000000000005e-05 2024-09-14T23:58:27.975 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.975 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.975 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:27.975 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147797+0000", 2024-09-14T23:58:27.975 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7645999999999995e-05 2024-09-14T23:58:27.975 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:27.975 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:27.975 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:27.975 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.975 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.975 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184376 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4160512~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:27.976 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.089232+0000", 2024-09-14T23:58:27.976 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.814559533000001, 2024-09-14T23:58:27.976 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058751338, 2024-09-14T23:58:27.976 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:27.976 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:27.976 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:27.976 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:27.976 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:27.976 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:27.976 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184376 2024-09-14T23:58:27.976 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.976 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:27.977 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.977 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:27.977 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089232+0000", 2024-09-14T23:58:27.977 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:27.977 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.977 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.977 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:27.977 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089232+0000", 2024-09-14T23:58:27.977 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:27.977 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.977 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.977 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:27.977 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089232+0000", 2024-09-14T23:58:27.977 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5600000000000001e-07 2024-09-14T23:58:27.978 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.978 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.978 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:27.978 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089235+0000", 2024-09-14T23:58:27.978 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5639999999999999e-06 2024-09-14T23:58:27.978 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.978 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.978 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:27.978 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089242+0000", 2024-09-14T23:58:27.978 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0369999999999997e-06 2024-09-14T23:58:27.978 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.978 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.978 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:27.978 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089247+0000", 2024-09-14T23:58:27.979 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2079999999999999e-06 2024-09-14T23:58:27.979 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.979 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.979 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:27.979 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147831+0000", 2024-09-14T23:58:27.979 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058583853999999998 2024-09-14T23:58:27.979 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.979 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.979 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:27.979 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147915+0000", 2024-09-14T23:58:27.979 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4208000000000004e-05 2024-09-14T23:58:27.979 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.979 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.979 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:27.980 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147983+0000", 2024-09-14T23:58:27.980 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8010999999999996e-05 2024-09-14T23:58:27.980 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:27.980 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:27.980 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:27.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.980 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.980 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184377 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4184064~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:27.980 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.089255+0000", 2024-09-14T23:58:27.980 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.814535976999998, 2024-09-14T23:58:27.980 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058906100000000003, 2024-09-14T23:58:27.980 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:27.980 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:27.981 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:27.981 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:27.981 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:27.981 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:27.981 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184377 2024-09-14T23:58:27.981 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.981 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:27.981 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.981 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:27.981 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089255+0000", 2024-09-14T23:58:27.981 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:27.981 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.981 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.981 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:27.982 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089255+0000", 2024-09-14T23:58:27.982 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:27.982 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.982 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.982 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:27.982 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.090903+0000", 2024-09-14T23:58:27.982 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001647871 2024-09-14T23:58:27.982 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.982 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.982 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:27.982 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.090908+0000", 2024-09-14T23:58:27.982 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7199999999999997e-06 2024-09-14T23:58:27.982 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.982 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.983 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:27.983 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.090919+0000", 2024-09-14T23:58:27.983 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1472e-05 2024-09-14T23:58:27.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.983 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.983 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:27.983 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.090927+0000", 2024-09-14T23:58:27.983 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0390000000000005e-06 2024-09-14T23:58:27.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.983 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.983 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:27.983 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.148013+0000", 2024-09-14T23:58:27.983 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057085662000000002 2024-09-14T23:58:27.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.983 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.984 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:27.984 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.148090+0000", 2024-09-14T23:58:27.984 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6778000000000002e-05 2024-09-14T23:58:27.984 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.984 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.984 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:27.984 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.148161+0000", 2024-09-14T23:58:27.984 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1557999999999995e-05 2024-09-14T23:58:27.984 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:27.984 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:27.984 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:27.984 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.984 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.985 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184386 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4185088~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:27.985 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.091063+0000", 2024-09-14T23:58:27.985 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.812728313000001, 2024-09-14T23:58:27.985 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058603404999999997, 2024-09-14T23:58:27.985 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:27.985 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:27.985 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:27.985 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:27.985 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:27.985 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:27.985 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184386 2024-09-14T23:58:27.985 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.985 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:27.985 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.986 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:27.986 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091063+0000", 2024-09-14T23:58:27.986 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:27.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.986 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.986 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:27.986 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091063+0000", 2024-09-14T23:58:27.986 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:27.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.986 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.986 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:27.986 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091063+0000", 2024-09-14T23:58:27.986 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1899999999999999e-07 2024-09-14T23:58:27.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.986 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.987 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:27.987 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091064+0000", 2024-09-14T23:58:27.987 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2699999999999999e-06 2024-09-14T23:58:27.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.987 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.987 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:27.987 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091067+0000", 2024-09-14T23:58:27.987 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.971e-06 2024-09-14T23:58:27.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.987 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.987 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:27.987 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091070+0000", 2024-09-14T23:58:27.987 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.525e-06 2024-09-14T23:58:27.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.988 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.988 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:27.988 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.149510+0000", 2024-09-14T23:58:27.988 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058439758000000001 2024-09-14T23:58:27.988 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.988 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.988 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:27.988 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.149595+0000", 2024-09-14T23:58:27.988 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5146000000000001e-05 2024-09-14T23:58:27.988 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.988 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.988 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:27.988 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.149666+0000", 2024-09-14T23:58:27.988 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1515999999999998e-05 2024-09-14T23:58:27.989 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:27.989 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:27.989 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:27.989 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.989 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.989 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184388 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4189184~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:27.989 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.091073+0000", 2024-09-14T23:58:27.989 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 20.812718038, 2024-09-14T23:58:27.989 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058781609999999998, 2024-09-14T23:58:27.989 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:27.989 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:27.989 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:27.989 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:27.990 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:27.990 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:27.990 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184388 2024-09-14T23:58:27.990 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.990 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:27.990 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.990 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:27.990 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091073+0000", 2024-09-14T23:58:27.990 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:27.990 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.990 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.990 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:27.990 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091073+0000", 2024-09-14T23:58:27.990 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:27.991 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.991 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.991 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:27.991 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.092903+0000", 2024-09-14T23:58:27.991 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018296289999999999 2024-09-14T23:58:27.991 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.991 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.991 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:27.991 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.092910+0000", 2024-09-14T23:58:27.991 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7510000000000002e-06 2024-09-14T23:58:27.991 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.991 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.991 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:27.992 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.092918+0000", 2024-09-14T23:58:27.992 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7420000000000003e-06 2024-09-14T23:58:27.992 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.992 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.992 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:27.992 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.092925+0000", 2024-09-14T23:58:27.992 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0360000000000001e-06 2024-09-14T23:58:27.992 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.992 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.992 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:27.992 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.149700+0000", 2024-09-14T23:58:27.992 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056774340999999999 2024-09-14T23:58:27.992 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.993 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.993 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:27.993 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.149784+0000", 2024-09-14T23:58:27.993 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4393000000000003e-05 2024-09-14T23:58:27.993 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.993 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.993 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:27.993 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.149855+0000", 2024-09-14T23:58:27.993 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0717999999999996e-05 2024-09-14T23:58:27.993 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:27.993 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:27.993 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:27.993 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.994 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.994 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4172.0:8 3.12 3:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-14T23:58:27.994 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.976999+0000", 2024-09-14T23:58:27.994 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 19.926792629000001, 2024-09-14T23:58:27.994 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0046036439999999, 2024-09-14T23:58:27.994 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:27.994 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:27.994 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:27.994 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:27.994 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4172", 2024-09-14T23:58:27.994 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/2542269203", 2024-09-14T23:58:27.994 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 8 2024-09-14T23:58:27.994 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.994 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:27.995 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.995 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:27.995 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.976999+0000", 2024-09-14T23:58:27.995 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:27.995 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.995 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.995 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:27.995 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.976999+0000", 2024-09-14T23:58:27.995 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:27.995 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.995 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.995 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:27.995 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.977000+0000", 2024-09-14T23:58:27.995 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5200000000000001e-06 2024-09-14T23:58:27.995 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.996 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.996 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:27.996 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.977005+0000", 2024-09-14T23:58:27.996 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.04e-06 2024-09-14T23:58:27.996 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.996 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.996 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:27.996 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.977013+0000", 2024-09-14T23:58:27.996 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2889999999999998e-06 2024-09-14T23:58:27.996 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.996 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.996 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:27.996 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.977024+0000", 2024-09-14T23:58:27.996 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0307e-05 2024-09-14T23:58:27.997 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.997 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.997 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:27.997 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.977086+0000", 2024-09-14T23:58:27.997 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2248000000000004e-05 2024-09-14T23:58:27.997 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.997 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.997 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:58:27.997 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.977097+0000", 2024-09-14T23:58:27.997 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1123e-05 2024-09-14T23:58:27.997 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.997 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.997 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:27.997 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:08.981470+0000", 2024-09-14T23:58:27.998 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.004373116 2024-09-14T23:58:27.998 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:27.998 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:27.998 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:27.998 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:08.981602+0000", 2024-09-14T23:58:27.998 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000132001 2024-09-14T23:58:27.998 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:27.998 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:27.998 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:27.998 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:27.998 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:27.998 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:27.999 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T23:58:28.000 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T23:58:28.001 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:27.999+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:28.101 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:28.100+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:28.151 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:28.151 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:28.151 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:58:28.151 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:28.160 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T23:58:28.167 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:58:28.167 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:58:28.187 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:58:28.187 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:58:28.202 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:28.201+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:28.203 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:58:28.204 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:58:28.220 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:58:28.221 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:58:28.238 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:58:28.238 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:58:28.255 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:58:28.255 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:58:28.270 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:28.270 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:28.272 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:58:28.273 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:58:28.279 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T23:58:28.290 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:58:28.290 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:58:28.303 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:28.302+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:28.378 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T23:58:28.398 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:28.398 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:28.404 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:28.403+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:28.407 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T23:58:28.505 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:28.504+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:28.558 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:28.558 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:28.566 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T23:58:28.610 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:28.609+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:28.674 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:28.674 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:28.682 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T23:58:28.707 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:28.706+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:28.808 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:28.807+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:28.830 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:28.830 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:28.839 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T23:58:28.922 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:28.921+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:28.974 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is unset 2024-09-14T23:58:28.995 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:28.995 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:29.003 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T23:58:29.044 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:29.043+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:29.111 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:29.110+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:29.157 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:29.157 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:29.166 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T23:58:29.212 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:29.210+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:29.273 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:29.273 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:29.282 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T23:58:29.318 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 6, 5] out_osds: [7, 3] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T23:58:29.318 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T23:58:29.318 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-14T23:58:29.319 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T23:58:29.323 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:29.322+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:29.395 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:29.395 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:29.404 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T23:58:29.414 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:29.413+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:29.510 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:29.510 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:29.514 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:29.513+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:29.519 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T23:58:29.616 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:29.614+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:29.616 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:58:29.617 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:29.617 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 129, 2024-09-14T23:58:29.617 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "21916fcc-f95c-4bfa-8b8b-11ca45845613", 2024-09-14T23:58:29.617 INFO:teuthology.orchestra.run.smithi042.stdout: "created": "2024-09-14T23:56:03.167948+0000", 2024-09-14T23:58:29.617 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-09-14T23:58:28.971948+0000", 2024-09-14T23:58:29.617 INFO:teuthology.orchestra.run.smithi042.stdout: "last_up_change": "2024-09-14T23:56:14.322882+0000", 2024-09-14T23:58:29.617 INFO:teuthology.orchestra.run.smithi042.stdout: "last_in_change": "2024-09-14T23:58:24.287442+0000", 2024-09-14T23:58:29.617 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T23:58:29.617 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_num": 5799936, 2024-09-14T23:58:29.617 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_set": [ 2024-09-14T23:58:29.617 INFO:teuthology.orchestra.run.smithi042.stdout: "pglog_hardlimit", 2024-09-14T23:58:29.617 INFO:teuthology.orchestra.run.smithi042.stdout: "purged_snapdirs", 2024-09-14T23:58:29.617 INFO:teuthology.orchestra.run.smithi042.stdout: "recovery_deletes", 2024-09-14T23:58:29.618 INFO:teuthology.orchestra.run.smithi042.stdout: "sortbitwise" 2024-09-14T23:58:29.618 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-14T23:58:29.618 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_version": 6, 2024-09-14T23:58:29.618 INFO:teuthology.orchestra.run.smithi042.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T23:58:29.618 INFO:teuthology.orchestra.run.smithi042.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T23:58:29.618 INFO:teuthology.orchestra.run.smithi042.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T23:58:29.618 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_snapshot": "", 2024-09-14T23:58:29.618 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_max": 3, 2024-09-14T23:58:29.618 INFO:teuthology.orchestra.run.smithi042.stdout: "max_osd": 8, 2024-09-14T23:58:29.618 INFO:teuthology.orchestra.run.smithi042.stdout: "require_min_compat_client": "luminous", 2024-09-14T23:58:29.618 INFO:teuthology.orchestra.run.smithi042.stdout: "min_compat_client": "luminous", 2024-09-14T23:58:29.618 INFO:teuthology.orchestra.run.smithi042.stdout: "require_osd_release": "squid", 2024-09-14T23:58:29.618 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_crimson": false, 2024-09-14T23:58:29.619 INFO:teuthology.orchestra.run.smithi042.stdout: "pools": [ 2024-09-14T23:58:29.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.619 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 1, 2024-09-14T23:58:29.619 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": ".mgr", 2024-09-14T23:58:29.619 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-14T23:56:13.500197+0000", 2024-09-14T23:58:29.619 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-09-14T23:58:29.619 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool", 2024-09-14T23:58:29.619 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-14T23:58:29.619 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-14T23:58:29.619 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-14T23:58:29.619 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-14T23:58:29.619 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-14T23:58:29.619 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-14T23:58:29.619 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T23:58:29.620 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T23:58:29.620 INFO:teuthology.orchestra.run.smithi042.stdout: "is_stretch_pool": false, 2024-09-14T23:58:29.620 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-14T23:58:29.620 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-14T23:58:29.620 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 1, 2024-09-14T23:58:29.620 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 1, 2024-09-14T23:58:29.620 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 1, 2024-09-14T23:58:29.620 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 1, 2024-09-14T23:58:29.620 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 1, 2024-09-14T23:58:29.620 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-14T23:58:29.620 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-14T23:58:29.620 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-14T23:58:29.620 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-14T23:58:29.621 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-14T23:58:29.621 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-14T23:58:29.621 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-14T23:58:29.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.621 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "17", 2024-09-14T23:58:29.621 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "0", 2024-09-14T23:58:29.621 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T23:58:29.621 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T23:58:29.621 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-14T23:58:29.621 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-14T23:58:29.621 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 0, 2024-09-14T23:58:29.621 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 0, 2024-09-14T23:58:29.621 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-14T23:58:29.621 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-14T23:58:29.622 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-14T23:58:29.622 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-14T23:58:29.622 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-14T23:58:29.622 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-14T23:58:29.622 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-14T23:58:29.622 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-14T23:58:29.622 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-14T23:58:29.622 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-14T23:58:29.622 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-14T23:58:29.622 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T23:58:29.622 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T23:58:29.622 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T23:58:29.622 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-14T23:58:29.622 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-14T23:58:29.622 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-14T23:58:29.623 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-14T23:58:29.623 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-14T23:58:29.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.623 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-14T23:58:29.623 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-14T23:58:29.623 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-14T23:58:29.623 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-14T23:58:29.623 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-14T23:58:29.623 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T23:58:29.623 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-14T23:58:29.623 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-14T23:58:29.623 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-14T23:58:29.623 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-14T23:58:29.623 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-14T23:58:29.624 INFO:teuthology.orchestra.run.smithi042.stdout: "options": { 2024-09-14T23:58:29.624 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_max": 32, 2024-09-14T23:58:29.624 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_min": 1 2024-09-14T23:58:29.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.624 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-14T23:58:29.624 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": {} 2024-09-14T23:58:29.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.624 INFO:teuthology.orchestra.run.smithi042.stdout: "read_balance": { 2024-09-14T23:58:29.624 INFO:teuthology.orchestra.run.smithi042.stdout: "score_type": "Fair distribution", 2024-09-14T23:58:29.624 INFO:teuthology.orchestra.run.smithi042.stdout: "score_acting": 6.9000000953674316, 2024-09-14T23:58:29.624 INFO:teuthology.orchestra.run.smithi042.stdout: "score_stable": 6.9000000953674316, 2024-09-14T23:58:29.624 INFO:teuthology.orchestra.run.smithi042.stdout: "optimal_score": 0.28999999165534973, 2024-09-14T23:58:29.624 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_acting": 2, 2024-09-14T23:58:29.624 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_stable": 2, 2024-09-14T23:58:29.625 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity_weighted": 0.78668212890625, 2024-09-14T23:58:29.625 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity": 0.89999997615814209, 2024-09-14T23:58:29.625 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity_weighted": 0.79000002145767212 2024-09-14T23:58:29.625 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:29.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.625 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 2, 2024-09-14T23:58:29.625 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": "rbd", 2024-09-14T23:58:29.625 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-14T23:56:21.284705+0000", 2024-09-14T23:58:29.625 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 8193, 2024-09-14T23:58:29.625 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T23:58:29.625 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-14T23:58:29.625 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-14T23:58:29.626 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-14T23:58:29.626 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-14T23:58:29.626 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-14T23:58:29.626 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-14T23:58:29.626 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T23:58:29.626 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T23:58:29.626 INFO:teuthology.orchestra.run.smithi042.stdout: "is_stretch_pool": false, 2024-09-14T23:58:29.626 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-14T23:58:29.626 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-14T23:58:29.626 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 8, 2024-09-14T23:58:29.626 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 8, 2024-09-14T23:58:29.626 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 8, 2024-09-14T23:58:29.626 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 8, 2024-09-14T23:58:29.626 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 8, 2024-09-14T23:58:29.626 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-14T23:58:29.627 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-14T23:58:29.627 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-14T23:58:29.627 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-14T23:58:29.627 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-14T23:58:29.627 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-14T23:58:29.627 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-14T23:58:29.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.627 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "21", 2024-09-14T23:58:29.627 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "0", 2024-09-14T23:58:29.627 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T23:58:29.627 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T23:58:29.627 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-14T23:58:29.627 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-14T23:58:29.627 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 2, 2024-09-14T23:58:29.627 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 21, 2024-09-14T23:58:29.628 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-14T23:58:29.628 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-14T23:58:29.628 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-14T23:58:29.628 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-14T23:58:29.628 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-14T23:58:29.628 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-14T23:58:29.628 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-14T23:58:29.628 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-14T23:58:29.628 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-14T23:58:29.628 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-14T23:58:29.628 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-14T23:58:29.628 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T23:58:29.628 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T23:58:29.628 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T23:58:29.629 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-14T23:58:29.629 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-14T23:58:29.629 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-14T23:58:29.629 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-14T23:58:29.629 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-14T23:58:29.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.629 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-14T23:58:29.629 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-14T23:58:29.629 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-14T23:58:29.629 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-14T23:58:29.629 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-14T23:58:29.629 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T23:58:29.629 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-14T23:58:29.630 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-14T23:58:29.630 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-14T23:58:29.630 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-14T23:58:29.630 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-14T23:58:29.630 INFO:teuthology.orchestra.run.smithi042.stdout: "options": {}, 2024-09-14T23:58:29.630 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-14T23:58:29.630 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd": {} 2024-09-14T23:58:29.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.630 INFO:teuthology.orchestra.run.smithi042.stdout: "read_balance": { 2024-09-14T23:58:29.630 INFO:teuthology.orchestra.run.smithi042.stdout: "score_type": "Fair distribution", 2024-09-14T23:58:29.630 INFO:teuthology.orchestra.run.smithi042.stdout: "score_acting": 3.4900000095367432, 2024-09-14T23:58:29.630 INFO:teuthology.orchestra.run.smithi042.stdout: "score_stable": 3.4900000095367432, 2024-09-14T23:58:29.630 INFO:teuthology.orchestra.run.smithi042.stdout: "optimal_score": 0.86000001430511475, 2024-09-14T23:58:29.630 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_acting": 3, 2024-09-14T23:58:29.630 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_stable": 3, 2024-09-14T23:58:29.631 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity_weighted": 0.78668212890625, 2024-09-14T23:58:29.631 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity": 0.89999997615814209, 2024-09-14T23:58:29.631 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity_weighted": 0.79000002145767212 2024-09-14T23:58:29.631 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:29.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.631 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 3, 2024-09-14T23:58:29.631 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": "pool_client.0", 2024-09-14T23:58:29.631 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-14T23:56:30.861504+0000", 2024-09-14T23:58:29.631 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 8193, 2024-09-14T23:58:29.631 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T23:58:29.631 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-14T23:58:29.631 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-14T23:58:29.631 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-14T23:58:29.631 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-14T23:58:29.632 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-14T23:58:29.632 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-14T23:58:29.632 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T23:58:29.632 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T23:58:29.632 INFO:teuthology.orchestra.run.smithi042.stdout: "is_stretch_pool": false, 2024-09-14T23:58:29.632 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-14T23:58:29.632 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-14T23:58:29.632 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 32, 2024-09-14T23:58:29.632 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 32, 2024-09-14T23:58:29.632 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 32, 2024-09-14T23:58:29.632 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 32, 2024-09-14T23:58:29.632 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 32, 2024-09-14T23:58:29.632 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-14T23:58:29.632 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-14T23:58:29.633 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-14T23:58:29.633 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-14T23:58:29.633 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-14T23:58:29.633 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-14T23:58:29.633 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-14T23:58:29.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.633 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "129", 2024-09-14T23:58:29.633 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "0", 2024-09-14T23:58:29.633 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T23:58:29.633 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T23:58:29.633 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-14T23:58:29.633 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-14T23:58:29.634 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 51, 2024-09-14T23:58:29.634 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 129, 2024-09-14T23:58:29.634 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-14T23:58:29.634 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-14T23:58:29.634 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-14T23:58:29.634 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-14T23:58:29.634 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-14T23:58:29.635 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-14T23:58:29.635 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-14T23:58:29.635 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-14T23:58:29.635 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-14T23:58:29.636 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-14T23:58:29.636 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-14T23:58:29.636 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T23:58:29.636 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T23:58:29.636 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T23:58:29.636 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-14T23:58:29.636 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-14T23:58:29.636 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-14T23:58:29.636 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-14T23:58:29.636 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-14T23:58:29.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.636 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-14T23:58:29.636 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-14T23:58:29.636 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-14T23:58:29.636 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-14T23:58:29.637 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-14T23:58:29.637 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T23:58:29.637 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-14T23:58:29.637 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-14T23:58:29.637 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-14T23:58:29.637 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-14T23:58:29.637 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-14T23:58:29.637 INFO:teuthology.orchestra.run.smithi042.stdout: "options": {}, 2024-09-14T23:58:29.637 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-14T23:58:29.637 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd": {} 2024-09-14T23:58:29.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.637 INFO:teuthology.orchestra.run.smithi042.stdout: "read_balance": { 2024-09-14T23:58:29.637 INFO:teuthology.orchestra.run.smithi042.stdout: "score_type": "Fair distribution", 2024-09-14T23:58:29.637 INFO:teuthology.orchestra.run.smithi042.stdout: "score_acting": 2.619999885559082, 2024-09-14T23:58:29.637 INFO:teuthology.orchestra.run.smithi042.stdout: "score_stable": 2.619999885559082, 2024-09-14T23:58:29.638 INFO:teuthology.orchestra.run.smithi042.stdout: "optimal_score": 0.86000001430511475, 2024-09-14T23:58:29.638 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_acting": 2.25, 2024-09-14T23:58:29.638 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_stable": 2.25, 2024-09-14T23:58:29.638 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity_weighted": 0.78668212890625, 2024-09-14T23:58:29.638 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity": 0.89999997615814209, 2024-09-14T23:58:29.638 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity_weighted": 0.79000002145767212 2024-09-14T23:58:29.638 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:29.638 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:29.638 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-14T23:58:29.638 INFO:teuthology.orchestra.run.smithi042.stdout: "osds": [ 2024-09-14T23:58:29.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.638 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 0, 2024-09-14T23:58:29.638 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "db9d0c67-2d60-4592-9ad3-f99e5294f93f", 2024-09-14T23:58:29.638 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-14T23:58:29.639 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-14T23:58:29.639 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-14T23:58:29.639 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 0.3231201171875, 2024-09-14T23:58:29.639 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-14T23:58:29.639 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-14T23:58:29.639 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 14, 2024-09-14T23:58:29.639 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 127, 2024-09-14T23:58:29.639 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-14T23:58:29.639 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-14T23:58:29.639 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-14T23:58:29.639 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:29.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.639 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:29.639 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6816", 2024-09-14T23:58:29.639 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 314865173 2024-09-14T23:58:29.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.640 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:29.640 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6817", 2024-09-14T23:58:29.640 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 314865173 2024-09-14T23:58:29.640 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:29.640 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:29.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.640 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-14T23:58:29.640 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:29.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.640 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:29.640 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6818", 2024-09-14T23:58:29.640 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 314865173 2024-09-14T23:58:29.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.641 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:29.641 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6819", 2024-09-14T23:58:29.641 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 314865173 2024-09-14T23:58:29.641 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:29.641 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:29.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.641 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-14T23:58:29.641 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:29.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.641 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:29.641 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6822", 2024-09-14T23:58:29.641 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 314865173 2024-09-14T23:58:29.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.642 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:29.642 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6823", 2024-09-14T23:58:29.642 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 314865173 2024-09-14T23:58:29.642 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:29.642 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:29.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.642 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-14T23:58:29.642 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:29.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.642 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:29.642 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6820", 2024-09-14T23:58:29.642 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 314865173 2024-09-14T23:58:29.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.642 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:29.643 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6821", 2024-09-14T23:58:29.643 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 314865173 2024-09-14T23:58:29.643 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:29.643 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:29.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.643 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6817/314865173", 2024-09-14T23:58:29.643 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6819/314865173", 2024-09-14T23:58:29.643 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6823/314865173", 2024-09-14T23:58:29.643 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6821/314865173", 2024-09-14T23:58:29.643 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-14T23:58:29.643 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-14T23:58:29.643 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-14T23:58:29.643 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:29.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.644 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 1, 2024-09-14T23:58:29.644 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "6c1a2765-c591-4c57-8ab3-ee6645970aa5", 2024-09-14T23:58:29.644 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-14T23:58:29.644 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-14T23:58:29.644 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-14T23:58:29.644 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-14T23:58:29.644 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-14T23:58:29.644 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-14T23:58:29.644 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 14, 2024-09-14T23:58:29.644 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 128, 2024-09-14T23:58:29.644 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-14T23:58:29.644 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-14T23:58:29.644 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-14T23:58:29.644 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:29.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.644 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:29.645 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6800", 2024-09-14T23:58:29.645 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3197912310 2024-09-14T23:58:29.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.645 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:29.645 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6801", 2024-09-14T23:58:29.645 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3197912310 2024-09-14T23:58:29.645 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:29.645 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:29.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.645 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-14T23:58:29.645 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:29.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.646 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:29.646 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6802", 2024-09-14T23:58:29.646 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3197912310 2024-09-14T23:58:29.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.646 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:29.646 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6803", 2024-09-14T23:58:29.646 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3197912310 2024-09-14T23:58:29.646 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:29.646 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:29.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.646 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-14T23:58:29.646 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:29.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.647 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:29.647 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6806", 2024-09-14T23:58:29.647 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3197912310 2024-09-14T23:58:29.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.647 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:29.647 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6807", 2024-09-14T23:58:29.647 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3197912310 2024-09-14T23:58:29.647 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:29.647 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:29.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.647 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-14T23:58:29.647 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:29.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.648 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:29.648 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6804", 2024-09-14T23:58:29.648 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3197912310 2024-09-14T23:58:29.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.648 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:29.648 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6805", 2024-09-14T23:58:29.648 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3197912310 2024-09-14T23:58:29.648 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:29.648 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:29.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.648 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6801/3197912310", 2024-09-14T23:58:29.648 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6803/3197912310", 2024-09-14T23:58:29.649 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6807/3197912310", 2024-09-14T23:58:29.649 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6805/3197912310", 2024-09-14T23:58:29.649 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-14T23:58:29.649 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-14T23:58:29.649 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-14T23:58:29.649 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:29.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.649 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 2, 2024-09-14T23:58:29.649 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "930db893-7be6-4645-a7f1-e043c5ec95eb", 2024-09-14T23:58:29.649 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-14T23:58:29.649 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-14T23:58:29.649 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-14T23:58:29.649 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-14T23:58:29.649 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-14T23:58:29.650 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-14T23:58:29.650 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 13, 2024-09-14T23:58:29.650 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 128, 2024-09-14T23:58:29.650 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-14T23:58:29.650 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-14T23:58:29.650 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-14T23:58:29.650 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:29.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.650 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:29.650 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6808", 2024-09-14T23:58:29.650 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3241153660 2024-09-14T23:58:29.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.650 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:29.651 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6809", 2024-09-14T23:58:29.651 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3241153660 2024-09-14T23:58:29.651 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:29.651 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:29.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.651 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-14T23:58:29.651 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:29.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.651 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:29.651 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6810", 2024-09-14T23:58:29.651 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3241153660 2024-09-14T23:58:29.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.652 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:29.652 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6811", 2024-09-14T23:58:29.652 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3241153660 2024-09-14T23:58:29.652 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:29.652 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:29.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.652 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-14T23:58:29.652 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:29.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.652 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:29.652 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6814", 2024-09-14T23:58:29.652 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3241153660 2024-09-14T23:58:29.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.653 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:29.653 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6815", 2024-09-14T23:58:29.653 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3241153660 2024-09-14T23:58:29.653 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:29.653 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:29.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.653 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-14T23:58:29.653 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:29.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.653 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:29.653 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6812", 2024-09-14T23:58:29.653 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3241153660 2024-09-14T23:58:29.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.654 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:29.654 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6813", 2024-09-14T23:58:29.654 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3241153660 2024-09-14T23:58:29.654 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:29.654 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:29.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.654 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6809/3241153660", 2024-09-14T23:58:29.654 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6811/3241153660", 2024-09-14T23:58:29.654 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6815/3241153660", 2024-09-14T23:58:29.654 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6813/3241153660", 2024-09-14T23:58:29.654 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-14T23:58:29.655 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-14T23:58:29.655 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-14T23:58:29.655 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:29.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.655 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 3, 2024-09-14T23:58:29.655 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "45bb3b84-b1c4-4f87-a7d1-0eb77b81741b", 2024-09-14T23:58:29.655 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-14T23:58:29.655 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 0, 2024-09-14T23:58:29.655 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 0, 2024-09-14T23:58:29.655 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-14T23:58:29.655 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-14T23:58:29.655 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-14T23:58:29.656 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-14T23:58:29.656 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 127, 2024-09-14T23:58:29.656 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-14T23:58:29.656 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-14T23:58:29.656 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-14T23:58:29.656 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:29.656 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.656 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:29.656 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6824", 2024-09-14T23:58:29.656 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1736416326 2024-09-14T23:58:29.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.656 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.656 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:29.656 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6825", 2024-09-14T23:58:29.657 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1736416326 2024-09-14T23:58:29.657 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:29.657 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:29.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.657 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-14T23:58:29.657 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:29.657 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.657 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:29.657 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6826", 2024-09-14T23:58:29.657 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1736416326 2024-09-14T23:58:29.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.657 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.657 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:29.657 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6827", 2024-09-14T23:58:29.658 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1736416326 2024-09-14T23:58:29.658 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:29.658 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:29.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.658 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-14T23:58:29.658 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:29.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.658 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:29.658 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6830", 2024-09-14T23:58:29.658 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1736416326 2024-09-14T23:58:29.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.658 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:29.658 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6831", 2024-09-14T23:58:29.659 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1736416326 2024-09-14T23:58:29.659 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:29.659 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:29.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.659 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-14T23:58:29.659 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:29.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.659 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:29.659 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6828", 2024-09-14T23:58:29.659 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1736416326 2024-09-14T23:58:29.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.659 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:29.659 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6829", 2024-09-14T23:58:29.660 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1736416326 2024-09-14T23:58:29.660 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:29.660 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:29.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.660 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6825/1736416326", 2024-09-14T23:58:29.660 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6827/1736416326", 2024-09-14T23:58:29.660 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6831/1736416326", 2024-09-14T23:58:29.660 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6829/1736416326", 2024-09-14T23:58:29.660 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-14T23:58:29.660 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-14T23:58:29.660 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-14T23:58:29.660 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:29.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.660 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 4, 2024-09-14T23:58:29.661 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "a812e38f-c0f9-4d33-9373-1910fb09103c", 2024-09-14T23:58:29.661 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-14T23:58:29.661 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-14T23:58:29.661 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-14T23:58:29.661 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 0.9703369140625, 2024-09-14T23:58:29.661 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-14T23:58:29.661 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-14T23:58:29.661 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-14T23:58:29.661 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 127, 2024-09-14T23:58:29.661 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-14T23:58:29.661 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-14T23:58:29.661 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-14T23:58:29.661 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:29.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.662 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:29.662 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6808", 2024-09-14T23:58:29.662 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1755154427 2024-09-14T23:58:29.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.662 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:29.662 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6809", 2024-09-14T23:58:29.662 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1755154427 2024-09-14T23:58:29.662 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:29.662 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:29.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.662 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-14T23:58:29.662 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:29.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.663 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:29.663 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6810", 2024-09-14T23:58:29.663 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1755154427 2024-09-14T23:58:29.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.663 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:29.663 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6811", 2024-09-14T23:58:29.663 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1755154427 2024-09-14T23:58:29.663 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:29.663 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:29.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.663 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-14T23:58:29.663 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:29.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.664 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:29.664 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6814", 2024-09-14T23:58:29.664 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1755154427 2024-09-14T23:58:29.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.664 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:29.664 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6815", 2024-09-14T23:58:29.664 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1755154427 2024-09-14T23:58:29.664 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:29.664 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:29.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.664 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-14T23:58:29.664 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:29.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.664 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:29.665 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6812", 2024-09-14T23:58:29.665 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1755154427 2024-09-14T23:58:29.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.665 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:29.665 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6813", 2024-09-14T23:58:29.665 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1755154427 2024-09-14T23:58:29.665 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:29.665 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:29.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.665 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.43:6809/1755154427", 2024-09-14T23:58:29.665 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.43:6811/1755154427", 2024-09-14T23:58:29.665 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.43:6815/1755154427", 2024-09-14T23:58:29.665 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.43:6813/1755154427", 2024-09-14T23:58:29.666 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-14T23:58:29.666 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-14T23:58:29.666 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-14T23:58:29.666 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:29.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.666 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 5, 2024-09-14T23:58:29.666 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "0de87e5b-7a21-4fb4-9543-d96c065786f8", 2024-09-14T23:58:29.666 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-14T23:58:29.666 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-14T23:58:29.666 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-14T23:58:29.666 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 0, 2024-09-14T23:58:29.666 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-14T23:58:29.667 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-14T23:58:29.667 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-14T23:58:29.667 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 111, 2024-09-14T23:58:29.667 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-14T23:58:29.667 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-14T23:58:29.667 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-14T23:58:29.667 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:29.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.667 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:29.667 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6816", 2024-09-14T23:58:29.667 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3033769028 2024-09-14T23:58:29.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.667 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:29.667 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6817", 2024-09-14T23:58:29.668 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3033769028 2024-09-14T23:58:29.668 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:29.668 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:29.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.668 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-14T23:58:29.668 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:29.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.668 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:29.668 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6818", 2024-09-14T23:58:29.668 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3033769028 2024-09-14T23:58:29.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.668 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:29.669 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6819", 2024-09-14T23:58:29.669 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3033769028 2024-09-14T23:58:29.669 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:29.669 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:29.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.669 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-14T23:58:29.669 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:29.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.669 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:29.669 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6822", 2024-09-14T23:58:29.669 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3033769028 2024-09-14T23:58:29.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.670 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:29.670 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6823", 2024-09-14T23:58:29.670 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3033769028 2024-09-14T23:58:29.670 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:29.670 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:29.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.670 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-14T23:58:29.670 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:29.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.670 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:29.670 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6820", 2024-09-14T23:58:29.671 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3033769028 2024-09-14T23:58:29.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.671 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:29.671 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6821", 2024-09-14T23:58:29.671 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3033769028 2024-09-14T23:58:29.671 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:29.671 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:29.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.671 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.43:6817/3033769028", 2024-09-14T23:58:29.671 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.43:6819/3033769028", 2024-09-14T23:58:29.671 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.43:6823/3033769028", 2024-09-14T23:58:29.671 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.43:6821/3033769028", 2024-09-14T23:58:29.672 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-14T23:58:29.672 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-14T23:58:29.672 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-14T23:58:29.672 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:29.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.672 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 6, 2024-09-14T23:58:29.672 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "c58c1d59-ad02-40f0-98e3-9078ef7e622b", 2024-09-14T23:58:29.672 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-14T23:58:29.672 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-14T23:58:29.672 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 0.924530029296875, 2024-09-14T23:58:29.672 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-14T23:58:29.672 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-14T23:58:29.673 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-14T23:58:29.673 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-14T23:58:29.673 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 127, 2024-09-14T23:58:29.673 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-14T23:58:29.673 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-14T23:58:29.673 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-14T23:58:29.673 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:29.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.673 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:29.673 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6800", 2024-09-14T23:58:29.673 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3314282416 2024-09-14T23:58:29.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.673 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:29.674 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6801", 2024-09-14T23:58:29.674 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3314282416 2024-09-14T23:58:29.674 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:29.674 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:29.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.674 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-14T23:58:29.674 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:29.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.674 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:29.674 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6802", 2024-09-14T23:58:29.674 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3314282416 2024-09-14T23:58:29.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.674 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:29.674 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6803", 2024-09-14T23:58:29.674 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3314282416 2024-09-14T23:58:29.675 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:29.675 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:29.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.675 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-14T23:58:29.675 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:29.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.675 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:29.675 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6806", 2024-09-14T23:58:29.675 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3314282416 2024-09-14T23:58:29.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.675 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:29.675 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6807", 2024-09-14T23:58:29.675 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3314282416 2024-09-14T23:58:29.676 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:29.676 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:29.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.676 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-14T23:58:29.676 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:29.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.676 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:29.676 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6804", 2024-09-14T23:58:29.676 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3314282416 2024-09-14T23:58:29.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.676 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:29.676 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6805", 2024-09-14T23:58:29.676 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3314282416 2024-09-14T23:58:29.677 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:29.677 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:29.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.677 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.43:6801/3314282416", 2024-09-14T23:58:29.677 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.43:6803/3314282416", 2024-09-14T23:58:29.677 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.43:6807/3314282416", 2024-09-14T23:58:29.677 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.43:6805/3314282416", 2024-09-14T23:58:29.677 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-14T23:58:29.677 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-14T23:58:29.677 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-14T23:58:29.677 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:29.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.677 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.678 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 7, 2024-09-14T23:58:29.678 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "389bdb04-20e3-4a7a-8504-687e034bd7dc", 2024-09-14T23:58:29.678 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-14T23:58:29.678 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 0, 2024-09-14T23:58:29.678 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 0, 2024-09-14T23:58:29.678 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-14T23:58:29.678 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-14T23:58:29.678 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-14T23:58:29.678 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-14T23:58:29.678 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 101, 2024-09-14T23:58:29.678 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-14T23:58:29.678 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-14T23:58:29.678 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-14T23:58:29.678 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:29.678 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.679 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:29.679 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6824", 2024-09-14T23:58:29.679 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1481350118 2024-09-14T23:58:29.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.679 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:29.679 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6825", 2024-09-14T23:58:29.679 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1481350118 2024-09-14T23:58:29.679 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:29.679 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:29.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.679 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-14T23:58:29.679 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:29.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.680 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:29.680 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6826", 2024-09-14T23:58:29.680 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1481350118 2024-09-14T23:58:29.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.680 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:29.680 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6827", 2024-09-14T23:58:29.680 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1481350118 2024-09-14T23:58:29.680 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:29.680 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:29.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.680 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-14T23:58:29.680 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:29.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.681 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:29.681 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6830", 2024-09-14T23:58:29.681 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1481350118 2024-09-14T23:58:29.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.681 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:29.681 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6831", 2024-09-14T23:58:29.681 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1481350118 2024-09-14T23:58:29.681 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:29.681 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:29.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.681 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-14T23:58:29.681 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:29.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.682 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:29.682 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6828", 2024-09-14T23:58:29.682 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1481350118 2024-09-14T23:58:29.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.682 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:29.682 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6829", 2024-09-14T23:58:29.682 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1481350118 2024-09-14T23:58:29.682 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:29.682 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:29.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.682 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.43:6825/1481350118", 2024-09-14T23:58:29.682 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.43:6827/1481350118", 2024-09-14T23:58:29.682 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.43:6831/1481350118", 2024-09-14T23:58:29.683 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.43:6829/1481350118", 2024-09-14T23:58:29.683 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-14T23:58:29.683 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-14T23:58:29.683 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-14T23:58:29.683 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:29.683 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:29.683 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-14T23:58:29.683 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_xinfo": [ 2024-09-14T23:58:29.683 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.683 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 0, 2024-09-14T23:58:29.683 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-14T23:58:29.683 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-14T23:58:29.683 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-14T23:58:29.683 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-14T23:58:29.683 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-14T23:58:29.683 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-14T23:56:11.077810+0000", 2024-09-14T23:58:29.684 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-14T23:58:29.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.684 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 1, 2024-09-14T23:58:29.684 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-14T23:58:29.684 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-14T23:58:29.684 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-14T23:58:29.684 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-14T23:58:29.684 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-14T23:58:29.684 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-14T23:56:11.578838+0000", 2024-09-14T23:58:29.684 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-14T23:58:29.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.684 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 2, 2024-09-14T23:58:29.684 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-14T23:58:29.685 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-14T23:58:29.685 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-14T23:58:29.685 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-14T23:58:29.685 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-14T23:58:29.685 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-14T23:56:10.895490+0000", 2024-09-14T23:58:29.685 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-14T23:58:29.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.685 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 3, 2024-09-14T23:58:29.685 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-14T23:58:29.685 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-14T23:58:29.685 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-14T23:58:29.685 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-14T23:58:29.685 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 65536, 2024-09-14T23:58:29.685 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-14T23:56:12.290821+0000", 2024-09-14T23:58:29.686 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-14T23:58:29.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.686 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 4, 2024-09-14T23:58:29.686 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-14T23:58:29.686 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-14T23:58:29.686 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-14T23:58:29.686 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-14T23:58:29.686 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-14T23:58:29.686 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-14T23:56:11.800581+0000", 2024-09-14T23:58:29.686 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-14T23:58:29.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.686 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 5, 2024-09-14T23:58:29.686 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-14T23:58:29.687 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-14T23:58:29.687 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-14T23:58:29.687 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-14T23:58:29.687 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-14T23:58:29.687 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-14T23:56:11.883155+0000", 2024-09-14T23:58:29.687 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-14T23:58:29.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.687 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 6, 2024-09-14T23:58:29.687 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-14T23:58:29.687 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-14T23:58:29.687 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-14T23:58:29.687 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-14T23:58:29.687 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-14T23:58:29.687 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-14T23:56:11.604790+0000", 2024-09-14T23:58:29.688 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-14T23:58:29.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.688 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 7, 2024-09-14T23:58:29.688 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-14T23:58:29.688 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-14T23:58:29.688 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-14T23:58:29.688 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-14T23:58:29.688 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 65536, 2024-09-14T23:58:29.688 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-14T23:56:12.067759+0000", 2024-09-14T23:58:29.688 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-14T23:58:29.688 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:29.688 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-14T23:58:29.688 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_upmap": [], 2024-09-14T23:58:29.688 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_upmap_items": [ 2024-09-14T23:58:29.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.689 INFO:teuthology.orchestra.run.smithi042.stdout: "pgid": "3.e", 2024-09-14T23:58:29.689 INFO:teuthology.orchestra.run.smithi042.stdout: "mappings": [ 2024-09-14T23:58:29.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.689 INFO:teuthology.orchestra.run.smithi042.stdout: "from": 4, 2024-09-14T23:58:29.689 INFO:teuthology.orchestra.run.smithi042.stdout: "to": 2 2024-09-14T23:58:29.689 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:29.689 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:29.689 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:29.689 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-14T23:58:29.689 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_upmap_primaries": [], 2024-09-14T23:58:29.689 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_temp": [], 2024-09-14T23:58:29.689 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_temp": [], 2024-09-14T23:58:29.689 INFO:teuthology.orchestra.run.smithi042.stdout: "blocklist": {}, 2024-09-14T23:58:29.689 INFO:teuthology.orchestra.run.smithi042.stdout: "range_blocklist": {}, 2024-09-14T23:58:29.690 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profiles": { 2024-09-14T23:58:29.690 INFO:teuthology.orchestra.run.smithi042.stdout: "default": { 2024-09-14T23:58:29.690 INFO:teuthology.orchestra.run.smithi042.stdout: "crush-failure-domain": "osd", 2024-09-14T23:58:29.690 INFO:teuthology.orchestra.run.smithi042.stdout: "k": "2", 2024-09-14T23:58:29.690 INFO:teuthology.orchestra.run.smithi042.stdout: "m": "1", 2024-09-14T23:58:29.690 INFO:teuthology.orchestra.run.smithi042.stdout: "plugin": "jerasure", 2024-09-14T23:58:29.690 INFO:teuthology.orchestra.run.smithi042.stdout: "technique": "reed_sol_van" 2024-09-14T23:58:29.690 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:29.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:29.690 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps_queue": [ 2024-09-14T23:58:29.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.690 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 3, 2024-09-14T23:58:29.690 INFO:teuthology.orchestra.run.smithi042.stdout: "snaps": [ 2024-09-14T23:58:29.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.690 INFO:teuthology.orchestra.run.smithi042.stdout: "begin": 43, 2024-09-14T23:58:29.691 INFO:teuthology.orchestra.run.smithi042.stdout: "length": 9 2024-09-14T23:58:29.691 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:29.691 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:29.691 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:29.691 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-14T23:58:29.691 INFO:teuthology.orchestra.run.smithi042.stdout: "new_removed_snaps": [ 2024-09-14T23:58:29.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.691 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 3, 2024-09-14T23:58:29.691 INFO:teuthology.orchestra.run.smithi042.stdout: "snaps": [ 2024-09-14T23:58:29.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:29.691 INFO:teuthology.orchestra.run.smithi042.stdout: "begin": 43, 2024-09-14T23:58:29.691 INFO:teuthology.orchestra.run.smithi042.stdout: "length": 1 2024-09-14T23:58:29.691 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:29.691 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:29.691 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:29.692 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-14T23:58:29.692 INFO:teuthology.orchestra.run.smithi042.stdout: "new_purged_snaps": [], 2024-09-14T23:58:29.692 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_node_flags": {}, 2024-09-14T23:58:29.692 INFO:teuthology.orchestra.run.smithi042.stdout: "device_class_flags": {}, 2024-09-14T23:58:29.692 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode": { 2024-09-14T23:58:29.692 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode_enabled": false, 2024-09-14T23:58:29.692 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_bucket_count": 0, 2024-09-14T23:58:29.692 INFO:teuthology.orchestra.run.smithi042.stdout: "degraded_stretch_mode": 0, 2024-09-14T23:58:29.692 INFO:teuthology.orchestra.run.smithi042.stdout: "recovering_stretch_mode": 0, 2024-09-14T23:58:29.692 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode_bucket": 0 2024-09-14T23:58:29.692 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:29.692 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:29.694 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:29.694 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:29.695 INFO:tasks.thrashosds.thrasher:j is {'epoch': 129, 'fsid': '21916fcc-f95c-4bfa-8b8b-11ca45845613', 'created': '2024-09-14T23:56:03.167948+0000', 'modified': '2024-09-14T23:58:28.971948+0000', 'last_up_change': '2024-09-14T23:56:14.322882+0000', 'last_in_change': '2024-09-14T23:58:24.287442+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T23:56:13.500197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.78668212890625, 'average_primary_affinity': 0.8999999761581421, 'average_primary_affinity_weighted': 0.7900000214576721}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T23:56:21.284705+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3.490000009536743, 'score_stable': 3.490000009536743, 'optimal_score': 0.8600000143051147, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.78668212890625, 'average_primary_affinity': 0.8999999761581421, 'average_primary_affinity_weighted': 0.7900000214576721}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T23:56:30.861504+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '129', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 51, 'snap_epoch': 129, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.619999885559082, 'score_stable': 2.619999885559082, 'optimal_score': 0.8600000143051147, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.78668212890625, 'average_primary_affinity': 0.8999999761581421, 'average_primary_affinity_weighted': 0.7900000214576721}}], 'osds': [{'osd': 0, 'uuid': 'db9d0c67-2d60-4592-9ad3-f99e5294f93f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.3231201171875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 127, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6816', 'nonce': 314865173}, {'type': 'v1', 'addr': '172.21.15.42:6817', 'nonce': 314865173}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6818', 'nonce': 314865173}, {'type': 'v1', 'addr': '172.21.15.42:6819', 'nonce': 314865173}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6822', 'nonce': 314865173}, {'type': 'v1', 'addr': '172.21.15.42:6823', 'nonce': 314865173}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6820', 'nonce': 314865173}, {'type': 'v1', 'addr': '172.21.15.42:6821', 'nonce': 314865173}]}, 'public_addr': '172.21.15.42:6817/314865173', 'cluster_addr': '172.21.15.42:6819/314865173', 'heartbeat_back_addr': '172.21.15.42:6823/314865173', 'heartbeat_front_addr': '172.21.15.42:6821/314865173', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '6c1a2765-c591-4c57-8ab3-ee6645970aa5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 128, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6800', 'nonce': 3197912310}, {'type': 'v1', 'addr': '172.21.15.42:6801', 'nonce': 3197912310}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6802', 'nonce': 3197912310}, {'type': 'v1', 'addr': '172.21.15.42:6803', 'nonce': 3197912310}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6806', 'nonce': 3197912310}, {'type': 'v1', 'addr': '172.21.15.42:6807', 'nonce': 3197912310}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6804', 'nonce': 3197912310}, {'type': 'v1', 'addr': '172.21.15.42:6805', 'nonce': 3197912310}]}, 'public_addr': '172.21.15.42:6801/3197912310', 'cluster_addr': '172.21.15.42:6803/3197912310', 'heartbeat_back_addr': '172.21.15.42:6807/3197912310', 'heartbeat_front_addr': '172.21.15.42:6805/3197912310', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '930db893-7be6-4645-a7f1-e043c5ec95eb', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 128, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6808', 'nonce': 3241153660}, {'type': 'v1', 'addr': '172.21.15.42:6809', 'nonce': 3241153660}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6810', 'nonce': 3241153660}, {'type': 'v1', 'addr': '172.21.15.42:6811', 'nonce': 3241153660}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6814', 'nonce': 3241153660}, {'type': 'v1', 'addr': '172.21.15.42:6815', 'nonce': 3241153660}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6812', 'nonce': 3241153660}, {'type': 'v1', 'addr': '172.21.15.42:6813', 'nonce': 3241153660}]}, 'public_addr': '172.21.15.42:6809/3241153660', 'cluster_addr': '172.21.15.42:6811/3241153660', 'heartbeat_back_addr': '172.21.15.42:6815/3241153660', 'heartbeat_front_addr': '172.21.15.42:6813/3241153660', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '45bb3b84-b1c4-4f87-a7d1-0eb77b81741b', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 127, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6824', 'nonce': 1736416326}, {'type': 'v1', 'addr': '172.21.15.42:6825', 'nonce': 1736416326}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6826', 'nonce': 1736416326}, {'type': 'v1', 'addr': '172.21.15.42:6827', 'nonce': 1736416326}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6830', 'nonce': 1736416326}, {'type': 'v1', 'addr': '172.21.15.42:6831', 'nonce': 1736416326}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6828', 'nonce': 1736416326}, {'type': 'v1', 'addr': '172.21.15.42:6829', 'nonce': 1736416326}]}, 'public_addr': '172.21.15.42:6825/1736416326', 'cluster_addr': '172.21.15.42:6827/1736416326', 'heartbeat_back_addr': '172.21.15.42:6831/1736416326', 'heartbeat_front_addr': '172.21.15.42:6829/1736416326', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'a812e38f-c0f9-4d33-9373-1910fb09103c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.9703369140625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 127, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6808', 'nonce': 1755154427}, {'type': 'v1', 'addr': '172.21.15.43:6809', 'nonce': 1755154427}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6810', 'nonce': 1755154427}, {'type': 'v1', 'addr': '172.21.15.43:6811', 'nonce': 1755154427}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6814', 'nonce': 1755154427}, {'type': 'v1', 'addr': '172.21.15.43:6815', 'nonce': 1755154427}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6812', 'nonce': 1755154427}, {'type': 'v1', 'addr': '172.21.15.43:6813', 'nonce': 1755154427}]}, 'public_addr': '172.21.15.43:6809/1755154427', 'cluster_addr': '172.21.15.43:6811/1755154427', 'heartbeat_back_addr': '172.21.15.43:6815/1755154427', 'heartbeat_front_addr': '172.21.15.43:6813/1755154427', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '0de87e5b-7a21-4fb4-9543-d96c065786f8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 111, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6816', 'nonce': 3033769028}, {'type': 'v1', 'addr': '172.21.15.43:6817', 'nonce': 3033769028}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6818', 'nonce': 3033769028}, {'type': 'v1', 'addr': '172.21.15.43:6819', 'nonce': 3033769028}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6822', 'nonce': 3033769028}, {'type': 'v1', 'addr': '172.21.15.43:6823', 'nonce': 3033769028}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6820', 'nonce': 3033769028}, {'type': 'v1', 'addr': '172.21.15.43:6821', 'nonce': 3033769028}]}, 'public_addr': '172.21.15.43:6817/3033769028', 'cluster_addr': '172.21.15.43:6819/3033769028', 'heartbeat_back_addr': '172.21.15.43:6823/3033769028', 'heartbeat_front_addr': '172.21.15.43:6821/3033769028', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'c58c1d59-ad02-40f0-98e3-9078ef7e622b', 'up': 1, 'in': 1, 'weight': 0.924530029296875, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 127, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6800', 'nonce': 3314282416}, {'type': 'v1', 'addr': '172.21.15.43:6801', 'nonce': 3314282416}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6802', 'nonce': 3314282416}, {'type': 'v1', 'addr': '172.21.15.43:6803', 'nonce': 3314282416}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6806', 'nonce': 3314282416}, {'type': 'v1', 'addr': '172.21.15.43:6807', 'nonce': 3314282416}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6804', 'nonce': 3314282416}, {'type': 'v1', 'addr': '172.21.15.43:6805', 'nonce': 3314282416}]}, 'public_addr': '172.21.15.43:6801/3314282416', 'cluster_addr': '172.21.15.43:6803/3314282416', 'heartbeat_back_addr': '172.21.15.43:6807/3314282416', 'heartbeat_front_addr': '172.21.15.43:6805/3314282416', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '389bdb04-20e3-4a7a-8504-687e034bd7dc', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 101, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6824', 'nonce': 1481350118}, {'type': 'v1', 'addr': '172.21.15.43:6825', 'nonce': 1481350118}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6826', 'nonce': 1481350118}, {'type': 'v1', 'addr': '172.21.15.43:6827', 'nonce': 1481350118}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6830', 'nonce': 1481350118}, {'type': 'v1', 'addr': '172.21.15.43:6831', 'nonce': 1481350118}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6828', 'nonce': 1481350118}, {'type': 'v1', 'addr': '172.21.15.43:6829', 'nonce': 1481350118}]}, 'public_addr': '172.21.15.43:6825/1481350118', 'cluster_addr': '172.21.15.43:6827/1481350118', 'heartbeat_back_addr': '172.21.15.43:6831/1481350118', 'heartbeat_front_addr': '172.21.15.43:6829/1481350118', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T23:56:11.077810+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T23:56:11.578838+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T23:56:10.895490+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-14T23:56:12.290821+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T23:56:11.800581+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T23:56:11.883155+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T23:56:11.604790+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-14T23:56:12.067759+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.e', 'mappings': [{'from': 4, 'to': 2}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 43, 'length': 9}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 43, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T23:58:29.695 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T23:58:29.695 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T23:58:29.739 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:58:29.738+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:58:29.803 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:29.803 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:29.811 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T23:58:29.852 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:29.851+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:29.917 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:29.916+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:29.959 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:29.959 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:29.967 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T23:58:30.031 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:58:30.031 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-14T23:58:30.045 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":192,"stamp":"2024-09-14T23:58:29.974837+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1020971901,"num_objects":2702,"num_object_clones":523,"num_object_copies":5404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2702,"num_whiteouts":1483,"num_read":236761,"num_read_kb":868100,"num_write":19629,"num_write_kb":595795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2963,"num_bytes_recovered":196685224,"num_keys_recovered":436,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16600,"ondisk_log_size":16600,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":73,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":333596,"kb_used_data":65920,"kb_used_omap":9,"kb_used_meta":267510,"kb_avail":565897444,"statfs":{"total":579820584960,"available":579478982656,"internally_reserved":0,"allocated":67502080,"data_stored":535591139,"data_compressed":40858744,"data_compressed_allocated":42119168,"data_compressed_original":516676779,"omap_allocated":9569,"internal_metadata":273930911},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-38024495,"num_objects":-52,"num_object_clones":-18,"num_object_copies":-104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-52,"num_whiteouts":-9,"num_read":469,"num_read_kb":399,"num_write":81,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":755,"num_bytes_recovered":42486199,"num_keys_recovered":81,"num_objects_omap":-2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.680439"},"pg_stats":[{"pgid":"3.1f","version":"127'951","reported_seq":5395,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-14T23:58:27.974504+0000","last_change":"2024-09-14T23:58:27.329663+0000","last_active":"2024-09-14T23:58:27.974504+0000","last_peered":"2024-09-14T23:58:27.974504+0000","last_clean":"2024-09-14T23:58:27.974504+0000","last_became_active":"2024-09-14T23:57:00.062116+0000","last_became_peered":"2024-09-14T23:57:00.062116+0000","last_unstale":"2024-09-14T23:58:27.974504+0000","last_undegraded":"2024-09-14T23:58:27.974504+0000","last_fullsized":"2024-09-14T23:58:27.974504+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":951,"log_dups_size":0,"ondisk_log_size":951,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:27:36.277024+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.0079908219999999999,"stat_sum":{"num_bytes":49857593,"num_objects":104,"num_object_clones":21,"num_object_copies":208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":51,"num_read":7920,"num_read_kb":36826,"num_write":1237,"num_write_kb":38022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":2327658,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2c","length":"8"}]},{"pgid":"3.1e","version":"122'404","reported_seq":5934,"reported_epoch":122,"state":"peering","last_fresh":"2024-09-14T23:58:22.302319+0000","last_change":"2024-09-14T23:58:24.293882+0000","last_active":"2024-09-14T23:58:24.293882+0000","last_peered":"2024-09-14T23:58:22.302319+0000","last_clean":"2024-09-14T23:58:22.302319+0000","last_became_active":"2024-09-14T23:57:47.055539+0000","last_became_peered":"2024-09-14T23:57:47.055539+0000","last_unstale":"2024-09-14T23:58:22.302319+0000","last_undegraded":"2024-09-14T23:58:22.302319+0000","last_fullsized":"2024-09-14T23:58:22.302319+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.008426322,"stat_sum":{"num_bytes":25057664,"num_objects":98,"num_object_clones":25,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":48,"num_read":5321,"num_read_kb":37528,"num_write":447,"num_write_kb":39218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":9327506,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2e","length":"6"}]},{"pgid":"3.1d","version":"121'483","reported_seq":13003,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-14T23:58:24.307902+0000","last_change":"2024-09-14T23:58:24.308169+0000","last_active":"2024-09-14T23:58:24.307902+0000","last_peered":"2024-09-14T23:58:24.307902+0000","last_clean":"2024-09-14T23:58:24.307902+0000","last_became_active":"2024-09-14T23:57:00.062242+0000","last_became_peered":"2024-09-14T23:57:00.062242+0000","last_unstale":"2024-09-14T23:58:24.307902+0000","last_undegraded":"2024-09-14T23:58:24.307902+0000","last_fullsized":"2024-09-14T23:58:24.307902+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:58:26.846403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0088019959999999994,"stat_sum":{"num_bytes":32029325,"num_objects":97,"num_object_clones":20,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":54,"num_read":14005,"num_read_kb":22296,"num_write":465,"num_write_kb":24083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":1249963,"num_keys_recovered":34,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2d","length":"7"}]},{"pgid":"3.1c","version":"124'842","reported_seq":5755,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-14T23:58:24.316540+0000","last_change":"2024-09-14T23:58:24.307494+0000","last_active":"2024-09-14T23:58:24.316540+0000","last_peered":"2024-09-14T23:58:24.316540+0000","last_clean":"2024-09-14T23:58:24.316540+0000","last_became_active":"2024-09-14T23:57:00.062126+0000","last_became_peered":"2024-09-14T23:57:00.062126+0000","last_unstale":"2024-09-14T23:58:24.316540+0000","last_undegraded":"2024-09-14T23:58:24.316540+0000","last_fullsized":"2024-09-14T23:58:24.316540+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":842,"log_dups_size":0,"ondisk_log_size":842,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:37.034760+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.0088069150000000002,"stat_sum":{"num_bytes":21067108,"num_objects":89,"num_object_clones":12,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":54,"num_read":8690,"num_read_kb":24808,"num_write":1145,"num_write_kb":9713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":2675491,"num_keys_recovered":40,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2d","length":"7"}]},{"pgid":"3.1b","version":"127'457","reported_seq":1998,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-14T23:58:27.974375+0000","last_change":"2024-09-14T23:58:27.326759+0000","last_active":"2024-09-14T23:58:27.974375+0000","last_peered":"2024-09-14T23:58:27.974375+0000","last_clean":"2024-09-14T23:58:27.974375+0000","last_became_active":"2024-09-14T23:56:32.398329+0000","last_became_peered":"2024-09-14T23:56:32.398329+0000","last_unstale":"2024-09-14T23:58:27.974375+0000","last_undegraded":"2024-09-14T23:58:27.974375+0000","last_fullsized":"2024-09-14T23:58:27.974375+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:23:47.411535+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0072507509999999997,"stat_sum":{"num_bytes":34241559,"num_objects":77,"num_object_clones":18,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":39,"num_read":2042,"num_read_kb":19852,"num_write":591,"num_write_kb":10338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2c","length":"8"}]},{"pgid":"3.1a","version":"124'816","reported_seq":7589,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-14T23:58:24.320209+0000","last_change":"2024-09-14T23:58:24.303765+0000","last_active":"2024-09-14T23:58:24.320209+0000","last_peered":"2024-09-14T23:58:24.320209+0000","last_clean":"2024-09-14T23:58:24.320209+0000","last_became_active":"2024-09-14T23:56:32.399655+0000","last_became_peered":"2024-09-14T23:56:32.399655+0000","last_unstale":"2024-09-14T23:58:24.320209+0000","last_undegraded":"2024-09-14T23:58:24.320209+0000","last_fullsized":"2024-09-14T23:58:24.320209+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":816,"log_dups_size":0,"ondisk_log_size":816,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:48:42.165483+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0051902429999999998,"stat_sum":{"num_bytes":89848192,"num_objects":102,"num_object_clones":22,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":62,"num_read":9375,"num_read_kb":53195,"num_write":937,"num_write_kb":34819,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2d","length":"7"}]},{"pgid":"3.19","version":"128'572","reported_seq":3702,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-14T23:58:27.985007+0000","last_change":"2024-09-14T23:58:27.985225+0000","last_active":"2024-09-14T23:58:27.985007+0000","last_peered":"2024-09-14T23:58:27.985007+0000","last_clean":"2024-09-14T23:58:27.985007+0000","last_became_active":"2024-09-14T23:58:27.976964+0000","last_became_peered":"2024-09-14T23:58:27.976964+0000","last_unstale":"2024-09-14T23:58:27.985007+0000","last_undegraded":"2024-09-14T23:58:27.985007+0000","last_fullsized":"2024-09-14T23:58:27.985007+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:00:37.799842+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0070333569999999996,"stat_sum":{"num_bytes":49022121,"num_objects":98,"num_object_clones":15,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":54,"num_read":4905,"num_read_kb":20047,"num_write":632,"num_write_kb":15814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":159,"num_bytes_recovered":6182078,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2c","length":"8"}]},{"pgid":"3.18","version":"127'600","reported_seq":5917,"reported_epoch":128,"state":"creating+peering","last_fresh":"2024-09-14T23:58:27.393794+0000","last_change":"2024-09-14T23:58:28.209587+0000","last_active":"2024-09-14T23:58:28.209587+0000","last_peered":"2024-09-14T23:58:27.393794+0000","last_clean":"2024-09-14T23:58:22.304353+0000","last_became_active":"2024-09-14T23:58:26.325454+0000","last_became_peered":"2024-09-14T23:58:26.325454+0000","last_unstale":"2024-09-14T23:58:27.393794+0000","last_undegraded":"2024-09-14T23:58:27.393794+0000","last_fullsized":"2024-09-14T23:58:26.296407+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":600,"log_dups_size":0,"ondisk_log_size":600,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.010343115999999999,"stat_sum":{"num_bytes":48906724,"num_objects":126,"num_object_clones":30,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":126,"num_whiteouts":68,"num_read":6318,"num_read_kb":59012,"num_write":669,"num_write_kb":26079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":12559960,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2e","length":"6"}]},{"pgid":"3.17","version":"126'467","reported_seq":11260,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-14T23:58:27.974607+0000","last_change":"2024-09-14T23:58:27.322661+0000","last_active":"2024-09-14T23:58:27.974607+0000","last_peered":"2024-09-14T23:58:27.974607+0000","last_clean":"2024-09-14T23:58:27.974607+0000","last_became_active":"2024-09-14T23:57:00.062304+0000","last_became_peered":"2024-09-14T23:57:00.062304+0000","last_unstale":"2024-09-14T23:58:27.974607+0000","last_undegraded":"2024-09-14T23:58:27.974607+0000","last_fullsized":"2024-09-14T23:58:27.974607+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":467,"log_dups_size":0,"ondisk_log_size":467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:42:50.267002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001441681,"stat_sum":{"num_bytes":11864925,"num_objects":84,"num_object_clones":18,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":46,"num_read":12245,"num_read_kb":16702,"num_write":455,"num_write_kb":6288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":3345543,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2c","length":"8"}]},{"pgid":"3.16","version":"127'376","reported_seq":7002,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-14T23:58:27.973828+0000","last_change":"2024-09-14T23:58:27.323126+0000","last_active":"2024-09-14T23:58:27.973828+0000","last_peered":"2024-09-14T23:58:27.973828+0000","last_clean":"2024-09-14T23:58:27.973828+0000","last_became_active":"2024-09-14T23:58:11.139452+0000","last_became_peered":"2024-09-14T23:58:11.139452+0000","last_unstale":"2024-09-14T23:58:27.973828+0000","last_undegraded":"2024-09-14T23:58:27.973828+0000","last_fullsized":"2024-09-14T23:58:27.973828+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:35:50.838953+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0065442449999999997,"stat_sum":{"num_bytes":55556840,"num_objects":70,"num_object_clones":10,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":36,"num_read":7089,"num_read_kb":26909,"num_write":326,"num_write_kb":13356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":286,"num_bytes_recovered":24669349,"num_keys_recovered":72,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2c","length":"8"}]},{"pgid":"3.15","version":"123'507","reported_seq":8706,"reported_epoch":124,"state":"unknown","last_fresh":"2024-09-14T23:58:24.299247+0000","last_change":"2024-09-14T23:58:25.212585+0000","last_active":"2024-09-14T23:58:24.299247+0000","last_peered":"2024-09-14T23:58:23.286277+0000","last_clean":"2024-09-14T23:58:23.286277+0000","last_became_active":"2024-09-14T23:58:11.138649+0000","last_became_peered":"2024-09-14T23:58:11.138649+0000","last_unstale":"2024-09-14T23:58:24.299247+0000","last_undegraded":"2024-09-14T23:58:24.299247+0000","last_fullsized":"2024-09-14T23:58:24.299247+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001820475,"stat_sum":{"num_bytes":39854225,"num_objects":89,"num_object_clones":14,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":52,"num_read":9513,"num_read_kb":35763,"num_write":551,"num_write_kb":23678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":129,"num_bytes_recovered":14165217,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2e","length":"6"}]},{"pgid":"3.14","version":"122'977","reported_seq":7393,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-14T23:58:24.307569+0000","last_change":"2024-09-14T23:58:24.307834+0000","last_active":"2024-09-14T23:58:24.307569+0000","last_peered":"2024-09-14T23:58:24.307569+0000","last_clean":"2024-09-14T23:58:24.307569+0000","last_became_active":"2024-09-14T23:58:12.137872+0000","last_became_peered":"2024-09-14T23:58:12.137872+0000","last_unstale":"2024-09-14T23:58:24.307569+0000","last_undegraded":"2024-09-14T23:58:24.307569+0000","last_fullsized":"2024-09-14T23:58:24.307569+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":977,"log_dups_size":0,"ondisk_log_size":977,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:03:12.151457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0091736579999999995,"stat_sum":{"num_bytes":27458149,"num_objects":108,"num_object_clones":24,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":58,"num_read":9548,"num_read_kb":42229,"num_write":1511,"num_write_kb":28893,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":21529916,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2d","length":"7"}]},{"pgid":"2.7","version":"0'0","reported_seq":230,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-14T23:58:24.293658+0000","last_change":"2024-09-14T23:58:07.977363+0000","last_active":"2024-09-14T23:58:24.293658+0000","last_peered":"2024-09-14T23:58:24.293658+0000","last_clean":"2024-09-14T23:58:24.293658+0000","last_became_active":"2024-09-14T23:58:07.977047+0000","last_became_peered":"2024-09-14T23:58:07.977047+0000","last_unstale":"2024-09-14T23:58:24.293658+0000","last_undegraded":"2024-09-14T23:58:24.293658+0000","last_fullsized":"2024-09-14T23:58:24.293658+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:07:21.332155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045105900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"127'611","reported_seq":4780,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-14T23:58:27.974479+0000","last_change":"2024-09-14T23:58:27.319805+0000","last_active":"2024-09-14T23:58:27.974479+0000","last_peered":"2024-09-14T23:58:27.974479+0000","last_clean":"2024-09-14T23:58:27.974479+0000","last_became_active":"2024-09-14T23:56:32.398664+0000","last_became_peered":"2024-09-14T23:56:32.398664+0000","last_unstale":"2024-09-14T23:58:27.974479+0000","last_undegraded":"2024-09-14T23:58:27.974479+0000","last_fullsized":"2024-09-14T23:58:27.974479+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:25:53.492181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014270140000000001,"stat_sum":{"num_bytes":27047379,"num_objects":83,"num_object_clones":10,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":48,"num_read":6381,"num_read_kb":24139,"num_write":713,"num_write_kb":5727,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2c","length":"8"}]},{"pgid":"2.6","version":"0'0","reported_seq":233,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-14T23:58:27.973945+0000","last_change":"2024-09-14T23:56:24.534632+0000","last_active":"2024-09-14T23:58:27.973945+0000","last_peered":"2024-09-14T23:58:27.973945+0000","last_clean":"2024-09-14T23:58:27.973945+0000","last_became_active":"2024-09-14T23:56:22.526355+0000","last_became_peered":"2024-09-14T23:56:22.526355+0000","last_unstale":"2024-09-14T23:58:27.973945+0000","last_undegraded":"2024-09-14T23:58:27.973945+0000","last_fullsized":"2024-09-14T23:58:27.973945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:02:16.059627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049910000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"126'406","reported_seq":7062,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-14T23:58:27.984548+0000","last_change":"2024-09-14T23:58:27.984548+0000","last_active":"2024-09-14T23:58:27.984548+0000","last_peered":"2024-09-14T23:58:27.984548+0000","last_clean":"2024-09-14T23:58:27.984548+0000","last_became_active":"2024-09-14T23:58:27.979626+0000","last_became_peered":"2024-09-14T23:58:27.979626+0000","last_unstale":"2024-09-14T23:58:27.984548+0000","last_undegraded":"2024-09-14T23:58:27.984548+0000","last_fullsized":"2024-09-14T23:58:27.984548+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":406,"log_dups_size":0,"ondisk_log_size":406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:55:16.766799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012639509999999999,"stat_sum":{"num_bytes":33689025,"num_objects":106,"num_object_clones":24,"num_object_copies":212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":106,"num_whiteouts":61,"num_read":7067,"num_read_kb":42968,"num_write":433,"num_write_kb":17282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":320,"num_bytes_recovered":28694058,"num_keys_recovered":42,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2c","length":"8"}]},{"pgid":"2.5","version":"0'0","reported_seq":47,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-14T23:58:27.974449+0000","last_change":"2024-09-14T23:58:07.974964+0000","last_active":"2024-09-14T23:58:27.974449+0000","last_peered":"2024-09-14T23:58:27.974449+0000","last_clean":"2024-09-14T23:58:27.974449+0000","last_became_active":"2024-09-14T23:58:07.974307+0000","last_became_peered":"2024-09-14T23:58:07.974307+0000","last_unstale":"2024-09-14T23:58:27.974449+0000","last_undegraded":"2024-09-14T23:58:27.974449+0000","last_fullsized":"2024-09-14T23:58:27.974449+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:27:42.149000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"127'509","reported_seq":4807,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-14T23:58:27.973579+0000","last_change":"2024-09-14T23:58:27.323783+0000","last_active":"2024-09-14T23:58:27.973579+0000","last_peered":"2024-09-14T23:58:27.973579+0000","last_clean":"2024-09-14T23:58:27.973579+0000","last_became_active":"2024-09-14T23:56:32.401089+0000","last_became_peered":"2024-09-14T23:56:32.401089+0000","last_unstale":"2024-09-14T23:58:27.973579+0000","last_undegraded":"2024-09-14T23:58:27.973579+0000","last_fullsized":"2024-09-14T23:58:27.973579+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":509,"log_dups_size":0,"ondisk_log_size":509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:04:35.802914+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0077918909999999996,"stat_sum":{"num_bytes":27439273,"num_objects":108,"num_object_clones":21,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":58,"num_read":5835,"num_read_kb":25712,"num_write":535,"num_write_kb":13573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2c","length":"8"}]},{"pgid":"2.4","version":"0'0","reported_seq":234,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-14T23:58:27.973720+0000","last_change":"2024-09-14T23:56:24.534510+0000","last_active":"2024-09-14T23:58:27.973720+0000","last_peered":"2024-09-14T23:58:27.973720+0000","last_clean":"2024-09-14T23:58:27.973720+0000","last_became_active":"2024-09-14T23:56:22.527611+0000","last_became_peered":"2024-09-14T23:56:22.527611+0000","last_unstale":"2024-09-14T23:58:27.973720+0000","last_undegraded":"2024-09-14T23:58:27.973720+0000","last_fullsized":"2024-09-14T23:58:27.973720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:15:45.606026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049916299999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"119'603","reported_seq":6884,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T23:58:19.315379+0000","last_change":"2024-09-14T23:58:19.315379+0000","last_active":"2024-09-14T23:58:19.315379+0000","last_peered":"2024-09-14T23:58:19.315379+0000","last_clean":"2024-09-14T23:58:19.315379+0000","last_became_active":"2024-09-14T23:57:00.064678+0000","last_became_peered":"2024-09-14T23:57:00.064678+0000","last_unstale":"2024-09-14T23:58:19.315379+0000","last_undegraded":"2024-09-14T23:58:19.315379+0000","last_fullsized":"2024-09-14T23:58:19.315379+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:07:33.063089+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0030734669999999999,"stat_sum":{"num_bytes":15280789,"num_objects":116,"num_object_clones":22,"num_object_copies":232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":116,"num_whiteouts":65,"num_read":8573,"num_read_kb":18207,"num_write":702,"num_write_kb":10066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":3349587,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"2"}]},{"pgid":"3.3","version":"124'526","reported_seq":11814,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-14T23:58:24.342719+0000","last_change":"2024-09-14T23:58:24.307890+0000","last_active":"2024-09-14T23:58:24.342719+0000","last_peered":"2024-09-14T23:58:24.342719+0000","last_clean":"2024-09-14T23:58:24.342719+0000","last_became_active":"2024-09-14T23:56:32.397883+0000","last_became_peered":"2024-09-14T23:56:32.397883+0000","last_unstale":"2024-09-14T23:58:24.342719+0000","last_undegraded":"2024-09-14T23:58:24.342719+0000","last_fullsized":"2024-09-14T23:58:24.342719+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":526,"log_dups_size":0,"ondisk_log_size":526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:31:38.624288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0089058990000000001,"stat_sum":{"num_bytes":62327193,"num_objects":111,"num_object_clones":22,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111,"num_whiteouts":53,"num_read":12574,"num_read_kb":25067,"num_write":547,"num_write_kb":35299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2d","length":"7"}]},{"pgid":"2.2","version":"21'2","reported_seq":50,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-14T23:58:27.973639+0000","last_change":"2024-09-14T23:58:07.978936+0000","last_active":"2024-09-14T23:58:27.973639+0000","last_peered":"2024-09-14T23:58:27.973639+0000","last_clean":"2024-09-14T23:58:27.973639+0000","last_became_active":"2024-09-14T23:58:07.975815+0000","last_became_peered":"2024-09-14T23:58:07.975815+0000","last_unstale":"2024-09-14T23:58:27.973639+0000","last_undegraded":"2024-09-14T23:58:27.973639+0000","last_fullsized":"2024-09-14T23:58:27.973639+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:51:25.065497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"122'454","reported_seq":8961,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-14T23:58:27.974086+0000","last_change":"2024-09-14T23:58:27.324150+0000","last_active":"2024-09-14T23:58:27.974086+0000","last_peered":"2024-09-14T23:58:27.974086+0000","last_clean":"2024-09-14T23:58:27.974086+0000","last_became_active":"2024-09-14T23:56:32.400278+0000","last_became_peered":"2024-09-14T23:56:32.400278+0000","last_unstale":"2024-09-14T23:58:27.974086+0000","last_undegraded":"2024-09-14T23:58:27.974086+0000","last_fullsized":"2024-09-14T23:58:27.974086+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:24:42.130573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065869810000000004,"stat_sum":{"num_bytes":61091838,"num_objects":90,"num_object_clones":20,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":46,"num_read":8534,"num_read_kb":62558,"num_write":408,"num_write_kb":30360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2c","length":"8"}]},{"pgid":"2.1","version":"0'0","reported_seq":234,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-14T23:58:26.436877+0000","last_change":"2024-09-14T23:58:25.299874+0000","last_active":"2024-09-14T23:58:26.436877+0000","last_peered":"2024-09-14T23:58:26.436877+0000","last_clean":"2024-09-14T23:58:26.436877+0000","last_became_active":"2024-09-14T23:58:25.299570+0000","last_became_peered":"2024-09-14T23:58:25.299570+0000","last_unstale":"2024-09-14T23:58:26.436877+0000","last_undegraded":"2024-09-14T23:58:26.436877+0000","last_fullsized":"2024-09-14T23:58:26.436877+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:42:03.033617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026863400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"121'641","reported_seq":6381,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-14T23:58:24.303836+0000","last_change":"2024-09-14T23:58:24.304199+0000","last_active":"2024-09-14T23:58:24.303836+0000","last_peered":"2024-09-14T23:58:24.303836+0000","last_clean":"2024-09-14T23:58:24.303836+0000","last_became_active":"2024-09-14T23:58:14.192509+0000","last_became_peered":"2024-09-14T23:58:14.192509+0000","last_unstale":"2024-09-14T23:58:24.303836+0000","last_undegraded":"2024-09-14T23:58:24.303836+0000","last_fullsized":"2024-09-14T23:58:24.303836+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":641,"log_dups_size":0,"ondisk_log_size":641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:45:01.104124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0055414139999999997,"stat_sum":{"num_bytes":16353889,"num_objects":93,"num_object_clones":14,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":54,"num_read":8362,"num_read_kb":21222,"num_write":836,"num_write_kb":5090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2d","length":"7"}]},{"pgid":"2.0","version":"0'0","reported_seq":47,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-14T23:58:27.974559+0000","last_change":"2024-09-14T23:58:07.974790+0000","last_active":"2024-09-14T23:58:27.974559+0000","last_peered":"2024-09-14T23:58:27.974559+0000","last_clean":"2024-09-14T23:58:27.974559+0000","last_became_active":"2024-09-14T23:58:07.974482+0000","last_became_peered":"2024-09-14T23:58:07.974482+0000","last_unstale":"2024-09-14T23:58:27.974559+0000","last_undegraded":"2024-09-14T23:58:27.974559+0000","last_fullsized":"2024-09-14T23:58:27.974559+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:26:59.848225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"unknown","last_fresh":"2024-09-14T23:58:24.296622+0000","last_change":"2024-09-14T23:58:25.209240+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T23:58:24.296622+0000","last_undegraded":"2024-09-14T23:58:24.296622+0000","last_fullsized":"2024-09-14T23:58:24.296622+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":226,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-14T23:58:26.436974+0000","last_change":"2024-09-14T23:56:57.054999+0000","last_active":"2024-09-14T23:58:26.436974+0000","last_peered":"2024-09-14T23:58:26.436974+0000","last_clean":"2024-09-14T23:58:26.436974+0000","last_became_active":"2024-09-14T23:56:57.050729+0000","last_became_peered":"2024-09-14T23:56:57.050729+0000","last_unstale":"2024-09-14T23:58:26.436974+0000","last_undegraded":"2024-09-14T23:58:26.436974+0000","last_fullsized":"2024-09-14T23:58:26.436974+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:23:11.727903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":191,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-14T23:58:24.293287+0000","last_change":"2024-09-14T23:58:14.191995+0000","last_active":"2024-09-14T23:58:24.293287+0000","last_peered":"2024-09-14T23:58:24.293287+0000","last_clean":"2024-09-14T23:58:24.293287+0000","last_became_active":"2024-09-14T23:58:14.191597+0000","last_became_peered":"2024-09-14T23:58:14.191597+0000","last_unstale":"2024-09-14T23:58:24.293287+0000","last_undegraded":"2024-09-14T23:58:24.293287+0000","last_fullsized":"2024-09-14T23:58:24.293287+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:14.322882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:14.322882+0000","last_clean_scrub_stamp":"2024-09-14T23:56:14.322882+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:11:57.219870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.8","version":"122'527","reported_seq":5168,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-14T23:58:27.980473+0000","last_change":"2024-09-14T23:58:27.980473+0000","last_active":"2024-09-14T23:58:27.980473+0000","last_peered":"2024-09-14T23:58:27.980473+0000","last_clean":"2024-09-14T23:58:27.980473+0000","last_became_active":"2024-09-14T23:58:27.976555+0000","last_became_peered":"2024-09-14T23:58:27.976555+0000","last_unstale":"2024-09-14T23:58:27.980473+0000","last_undegraded":"2024-09-14T23:58:27.980473+0000","last_fullsized":"2024-09-14T23:58:27.980473+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:51:10.646673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029997489999999999,"stat_sum":{"num_bytes":33270792,"num_objects":82,"num_object_clones":16,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":48,"num_read":5557,"num_read_kb":17805,"num_write":481,"num_write_kb":10908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":4505829,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2c","length":"8"}]},{"pgid":"3.9","version":"120'290","reported_seq":10634,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-14T23:58:24.307298+0000","last_change":"2024-09-14T23:58:24.307560+0000","last_active":"2024-09-14T23:58:24.307298+0000","last_peered":"2024-09-14T23:58:24.307298+0000","last_clean":"2024-09-14T23:58:24.307298+0000","last_became_active":"2024-09-14T23:56:32.400017+0000","last_became_peered":"2024-09-14T23:56:32.400017+0000","last_unstale":"2024-09-14T23:58:24.307298+0000","last_undegraded":"2024-09-14T23:58:24.307298+0000","last_fullsized":"2024-09-14T23:58:24.307298+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":290,"log_dups_size":0,"ondisk_log_size":290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:17:49.726903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0086521889999999994,"stat_sum":{"num_bytes":22512897,"num_objects":83,"num_object_clones":16,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":46,"num_read":10180,"num_read_kb":13011,"num_write":276,"num_write_kb":15704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2d","length":"7"}]},{"pgid":"3.a","version":"123'734","reported_seq":15547,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-14T23:58:24.293518+0000","last_change":"2024-09-14T23:58:24.293739+0000","last_active":"2024-09-14T23:58:24.293518+0000","last_peered":"2024-09-14T23:58:24.293518+0000","last_clean":"2024-09-14T23:58:24.293518+0000","last_became_active":"2024-09-14T23:56:32.402769+0000","last_became_peered":"2024-09-14T23:56:32.402769+0000","last_unstale":"2024-09-14T23:58:24.293518+0000","last_undegraded":"2024-09-14T23:58:24.293518+0000","last_fullsized":"2024-09-14T23:58:24.293518+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":734,"log_dups_size":0,"ondisk_log_size":734,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:55:49.911946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032676,"stat_sum":{"num_bytes":26657567,"num_objects":84,"num_object_clones":13,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":51,"num_read":16818,"num_read_kb":32576,"num_write":1034,"num_write_kb":14954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2d","length":"7"}]},{"pgid":"3.b","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"unknown","last_fresh":"2024-09-14T23:58:24.298046+0000","last_change":"2024-09-14T23:58:25.212537+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T23:58:24.298046+0000","last_undegraded":"2024-09-14T23:58:24.298046+0000","last_fullsized":"2024-09-14T23:58:24.298046+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1,"reported_epoch":124,"state":"unknown","last_fresh":"2024-09-14T23:58:24.296684+0000","last_change":"2024-09-14T23:58:25.209276+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T23:58:24.296684+0000","last_undegraded":"2024-09-14T23:58:24.296684+0000","last_fullsized":"2024-09-14T23:58:24.296684+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"121'445","reported_seq":2888,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-14T23:58:24.293656+0000","last_change":"2024-09-14T23:58:24.293900+0000","last_active":"2024-09-14T23:58:24.293656+0000","last_peered":"2024-09-14T23:58:24.293656+0000","last_clean":"2024-09-14T23:58:24.293656+0000","last_became_active":"2024-09-14T23:58:11.139554+0000","last_became_peered":"2024-09-14T23:58:11.139554+0000","last_unstale":"2024-09-14T23:58:24.293656+0000","last_undegraded":"2024-09-14T23:58:24.293656+0000","last_fullsized":"2024-09-14T23:58:24.293656+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:27:33.481540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000436876,"stat_sum":{"num_bytes":10947636,"num_objects":74,"num_object_clones":13,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":43,"num_read":3755,"num_read_kb":12139,"num_write":491,"num_write_kb":8305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":318,"num_bytes_recovered":15479703,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2d","length":"7"}]},{"pgid":"3.e","version":"124'408","reported_seq":4950,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-14T23:58:26.446398+0000","last_change":"2024-09-14T23:58:24.304207+0000","last_active":"2024-09-14T23:58:26.446398+0000","last_peered":"2024-09-14T23:58:26.446398+0000","last_clean":"2024-09-14T23:58:26.446398+0000","last_became_active":"2024-09-14T23:58:12.139534+0000","last_became_peered":"2024-09-14T23:58:12.139534+0000","last_unstale":"2024-09-14T23:58:26.446398+0000","last_undegraded":"2024-09-14T23:58:26.446398+0000","last_fullsized":"2024-09-14T23:58:26.446398+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:08:34.513071+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0048908880000000004,"stat_sum":{"num_bytes":29033791,"num_objects":87,"num_object_clones":17,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":44,"num_read":6888,"num_read_kb":20647,"num_write":389,"num_write_kb":11338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":7127327,"num_keys_recovered":34,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2d","length":"7"}]},{"pgid":"3.f","version":"123'538","reported_seq":13348,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-14T23:58:24.306912+0000","last_change":"2024-09-14T23:58:24.307204+0000","last_active":"2024-09-14T23:58:24.306912+0000","last_peered":"2024-09-14T23:58:24.306912+0000","last_clean":"2024-09-14T23:58:24.306912+0000","last_became_active":"2024-09-14T23:58:11.137740+0000","last_became_peered":"2024-09-14T23:58:11.137740+0000","last_unstale":"2024-09-14T23:58:24.306912+0000","last_undegraded":"2024-09-14T23:58:24.306912+0000","last_fullsized":"2024-09-14T23:58:24.306912+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":538,"log_dups_size":0,"ondisk_log_size":538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:56:16.766239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0083430689999999998,"stat_sum":{"num_bytes":31857291,"num_objects":94,"num_object_clones":18,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":50,"num_read":14010,"num_read_kb":21096,"num_write":581,"num_write_kb":35760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":8450382,"num_keys_recovered":48,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2d","length":"7"}]},{"pgid":"3.10","version":"122'695","reported_seq":6038,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-14T23:58:24.294383+0000","last_change":"2024-09-14T23:58:24.294684+0000","last_active":"2024-09-14T23:58:24.294383+0000","last_peered":"2024-09-14T23:58:24.294383+0000","last_clean":"2024-09-14T23:58:24.294383+0000","last_became_active":"2024-09-14T23:56:32.402973+0000","last_became_peered":"2024-09-14T23:56:32.402973+0000","last_unstale":"2024-09-14T23:58:24.294383+0000","last_undegraded":"2024-09-14T23:58:24.294383+0000","last_fullsized":"2024-09-14T23:58:24.294383+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:40:23.007934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048284899999999998,"stat_sum":{"num_bytes":50338715,"num_objects":101,"num_object_clones":17,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":60,"num_read":6906,"num_read_kb":57881,"num_write":1108,"num_write_kb":51448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2d","length":"7"}]},{"pgid":"3.11","version":"124'478","reported_seq":7848,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-14T23:58:24.306646+0000","last_change":"2024-09-14T23:58:24.306848+0000","last_active":"2024-09-14T23:58:24.306646+0000","last_peered":"2024-09-14T23:58:24.306646+0000","last_clean":"2024-09-14T23:58:24.306646+0000","last_became_active":"2024-09-14T23:58:11.138513+0000","last_became_peered":"2024-09-14T23:58:11.138513+0000","last_unstale":"2024-09-14T23:58:24.306646+0000","last_undegraded":"2024-09-14T23:58:24.306646+0000","last_fullsized":"2024-09-14T23:58:24.306646+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":478,"log_dups_size":0,"ondisk_log_size":478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:32:40.338935+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0083340559999999994,"stat_sum":{"num_bytes":36735614,"num_objects":88,"num_object_clones":16,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":46,"num_read":8472,"num_read_kb":36032,"num_write":515,"num_write_kb":28053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":141,"num_bytes_recovered":16337558,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2d","length":"7"}]},{"pgid":"3.12","version":"128'332","reported_seq":2895,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-14T23:58:27.983229+0000","last_change":"2024-09-14T23:58:27.983229+0000","last_active":"2024-09-14T23:58:27.983229+0000","last_peered":"2024-09-14T23:58:27.983229+0000","last_clean":"2024-09-14T23:58:27.983229+0000","last_became_active":"2024-09-14T23:58:27.975890+0000","last_became_peered":"2024-09-14T23:58:27.975890+0000","last_unstale":"2024-09-14T23:58:27.983229+0000","last_undegraded":"2024-09-14T23:58:27.983229+0000","last_fullsized":"2024-09-14T23:58:27.983229+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:08:45.419877+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0053605609999999998,"stat_sum":{"num_bytes":27307956,"num_objects":67,"num_object_clones":16,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":39,"num_read":3164,"num_read_kb":16377,"num_write":275,"num_write_kb":14646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":5103606,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2c","length":"8"}]},{"pgid":"3.13","version":"124'831","reported_seq":3831,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-14T23:58:24.304395+0000","last_change":"2024-09-14T23:58:24.304911+0000","last_active":"2024-09-14T23:58:24.304395+0000","last_peered":"2024-09-14T23:58:24.304395+0000","last_clean":"2024-09-14T23:58:24.304395+0000","last_became_active":"2024-09-14T23:58:12.139670+0000","last_became_peered":"2024-09-14T23:58:12.139670+0000","last_unstale":"2024-09-14T23:58:24.304395+0000","last_undegraded":"2024-09-14T23:58:24.304395+0000","last_fullsized":"2024-09-14T23:58:24.304395+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":831,"log_dups_size":0,"ondisk_log_size":831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:01:00.291413+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.006177854,"stat_sum":{"num_bytes":23725444,"num_objects":89,"num_object_clones":19,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":49,"num_read":6598,"num_read_kb":25396,"num_write":1178,"num_write_kb":14563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":9604455,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2d","length":"7"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1020381514,"num_objects":2698,"num_object_clones":523,"num_object_copies":5396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2698,"num_whiteouts":1483,"num_read":236645,"num_read_kb":868000,"num_write":19468,"num_write_kb":593377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2959,"num_bytes_recovered":196685186,"num_keys_recovered":436,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":85512192,"data_stored":789821676,"data_compressed":60597624,"data_compressed_allocated":62283776,"data_compressed_original":769344227,"omap_allocated":0,"internal_metadata":0},"log_size":16480,"ondisk_log_size":16480,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":1771104,"data_compressed":98526,"data_compressed_allocated":135168,"data_compressed_original":1736704,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":0,"seq":64424509471,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509470,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34544,"kb_used_data":7512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337296,"statfs":{"total":96636764160,"available":96601391104,"internally_reserved":0,"allocated":7692288,"data_stored":47636986,"data_compressed":3682181,"data_compressed_allocated":4460544,"data_compressed_original":45469065,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509469,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67640,"kb_used_data":10080,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94304200,"statfs":{"total":96636764160,"available":96567500800,"internally_reserved":0,"allocated":10321920,"data_stored":86241207,"data_compressed":6419077,"data_compressed_allocated":6529024,"data_compressed_original":83602614,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509471,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77156,"kb_used_data":14020,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94294684,"statfs":{"total":96636764160,"available":96557756416,"internally_reserved":0,"allocated":14356480,"data_stored":120782158,"data_compressed":9201056,"data_compressed_allocated":9707520,"data_compressed_original":117287163,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":0,"seq":64424509469,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542174,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48060,"kb_used_data":12708,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94323780,"statfs":{"total":96636764160,"available":96587550720,"internally_reserved":0,"allocated":13012992,"data_stored":96988706,"data_compressed":7777993,"data_compressed_allocated":7409664,"data_compressed_original":92513967,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542173,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59588,"kb_used_data":14508,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94312252,"statfs":{"total":96636764160,"available":96575746048,"internally_reserved":0,"allocated":14856192,"data_stored":128713806,"data_compressed":9671122,"data_compressed_allocated":10043392,"data_compressed_original":125069497,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46608,"kb_used_data":7092,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94325232,"statfs":{"total":96636764160,"available":96589037568,"internally_reserved":0,"allocated":7262208,"data_stored":55228276,"data_compressed":4107315,"data_compressed_allocated":3969024,"data_compressed_original":52734473,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32297,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":33932,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32297,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":13180928,"data_stored":126340479,"data_compressed":9579678,"data_compressed_allocated":9920512,"data_compressed_original":123447481,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":11251712,"data_stored":94102359,"data_compressed":7649070,"data_compressed_allocated":7245824,"data_compressed_original":90416815,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":7069696,"data_stored":63898269,"data_compressed":4948394,"data_compressed_allocated":4751360,"data_compressed_original":61725193,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12812288,"data_stored":122065494,"data_compressed":9432023,"data_compressed_allocated":9170944,"data_compressed_original":118845279,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12881920,"data_stored":120085453,"data_compressed":9201056,"data_compressed_allocated":9707520,"data_compressed_original":117287163,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8843264,"data_stored":85544483,"data_compressed":6419077,"data_compressed_allocated":6529024,"data_compressed_original":83602614,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":6168576,"data_stored":46349913,"data_compressed":3648249,"data_compressed_allocated":4415488,"data_compressed_original":44879241,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":13303808,"data_stored":131435226,"data_compressed":9720077,"data_compressed_allocated":10543104,"data_compressed_original":129140441,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T23:58:30.047 INFO:tasks.thrashosds.thrasher:Setting 2.0 to [2, 0, 3, 1] 2024-09-14T23:58:30.047 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '2.0', '2', '0', '3', '1'] 2024-09-14T23:58:30.048 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 2.0 2 0 3 1 2024-09-14T23:58:30.052 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:30.051+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:30.118 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:30.117+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:30.123 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:30.123 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:30.132 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T23:58:30.219 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:30.217+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:30.280 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:30.280 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:30.290 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T23:58:30.302 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T23:58:30.333 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:30.332+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:30.397 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:30.397 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:30.406 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T23:58:30.421 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:30.420+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:30.427 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:58:30.427 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:58:30.444 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:58:30.444 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:58:30.460 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:58:30.460 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:58:30.476 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:58:30.476 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:58:30.493 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:58:30.493 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:58:30.511 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:58:30.512 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:58:30.522 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:30.520+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:30.528 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:58:30.528 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:58:30.557 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:30.557 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:58:30.557 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:58:30.558 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:58:30.558 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.558 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:107472 3.1e 3:7bf21588:::rbd_data.11187de8b0a5.000000000000005d:head [set-alloc-hint object_size 2097152 write_size 2097152,write 317788~1491] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-14T23:58:30.558 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:25.198493+0000", 2024-09-14T23:58:30.558 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 65.356394855000005, 2024-09-14T23:58:30.558 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.78175149300000002, 2024-09-14T23:58:30.558 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:30.558 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:30.558 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:30.558 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:30.558 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:30.558 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:30.558 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 107472 2024-09-14T23:58:30.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.559 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:30.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:30.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198493+0000", 2024-09-14T23:58:30.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:30.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:30.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198493+0000", 2024-09-14T23:58:30.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:30.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:30.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198495+0000", 2024-09-14T23:58:30.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.826e-06 2024-09-14T23:58:30.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:30.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198503+0000", 2024-09-14T23:58:30.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0150000000000003e-06 2024-09-14T23:58:30.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:30.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198521+0000", 2024-09-14T23:58:30.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8374000000000001e-05 2024-09-14T23:58:30.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:30.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198533+0000", 2024-09-14T23:58:30.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1593000000000001e-05 2024-09-14T23:58:30.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.561 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:30.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198603+0000", 2024-09-14T23:58:30.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0279999999999998e-05 2024-09-14T23:58:30.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.561 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:30.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198617+0000", 2024-09-14T23:58:30.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3517e-05 2024-09-14T23:58:30.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.561 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:30.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.977067+0000", 2024-09-14T23:58:30.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.77845065800000002 2024-09-14T23:58:30.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for missing object", 2024-09-14T23:58:30.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.977173+0000", 2024-09-14T23:58:30.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010543999999999999 2024-09-14T23:58:30.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:30.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.978737+0000", 2024-09-14T23:58:30.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001564078 2024-09-14T23:58:30.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:30.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.978845+0000", 2024-09-14T23:58:30.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000108643 2024-09-14T23:58:30.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.563 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1", 2024-09-14T23:58:30.563 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.978964+0000", 2024-09-14T23:58:30.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000119085 2024-09-14T23:58:30.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.563 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:30.563 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980109+0000", 2024-09-14T23:58:30.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0011442170000000001 2024-09-14T23:58:30.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.563 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:58:30.563 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980130+0000", 2024-09-14T23:58:30.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1766000000000001e-05 2024-09-14T23:58:30.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:30.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980171+0000", 2024-09-14T23:58:30.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0627999999999998e-05 2024-09-14T23:58:30.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:30.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980244+0000", 2024-09-14T23:58:30.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3373000000000004e-05 2024-09-14T23:58:30.564 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:30.564 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:30.564 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:30.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.565 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129159 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 5996544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:30.565 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.927783+0000", 2024-09-14T23:58:30.565 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.627104963999997, 2024-09-14T23:58:30.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054661122999999999, 2024-09-14T23:58:30.565 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:30.565 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:30.565 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:30.565 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:30.565 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:30.565 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:30.565 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129159 2024-09-14T23:58:30.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.566 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:30.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:30.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.927783+0000", 2024-09-14T23:58:30.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:30.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:30.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.927783+0000", 2024-09-14T23:58:30.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:30.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:30.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929684+0000", 2024-09-14T23:58:30.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0019011080000000001 2024-09-14T23:58:30.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.567 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:30.567 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929688+0000", 2024-09-14T23:58:30.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2610000000000004e-06 2024-09-14T23:58:30.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.567 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:30.567 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929697+0000", 2024-09-14T23:58:30.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0219999999999993e-06 2024-09-14T23:58:30.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.567 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:30.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929702+0000", 2024-09-14T23:58:30.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8840000000000002e-06 2024-09-14T23:58:30.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.568 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:30.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.982090+0000", 2024-09-14T23:58:30.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052388045000000001 2024-09-14T23:58:30.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.568 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:30.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.982208+0000", 2024-09-14T23:58:30.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118236 2024-09-14T23:58:30.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.569 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:30.569 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.982444+0000", 2024-09-14T23:58:30.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000235567 2024-09-14T23:58:30.569 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:30.569 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:30.569 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:30.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.569 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129166 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 8192000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:30.569 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.929794+0000", 2024-09-14T23:58:30.569 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.625094212999997, 2024-09-14T23:58:30.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054471973, 2024-09-14T23:58:30.569 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:30.569 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:30.570 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:30.570 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:30.570 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:30.570 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:30.570 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129166 2024-09-14T23:58:30.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.570 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:30.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.570 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:30.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929794+0000", 2024-09-14T23:58:30.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:30.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.570 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:30.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929794+0000", 2024-09-14T23:58:30.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:30.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:30.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929794+0000", 2024-09-14T23:58:30.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6899999999999997e-07 2024-09-14T23:58:30.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:30.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929797+0000", 2024-09-14T23:58:30.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.328e-06 2024-09-14T23:58:30.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:30.572 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929802+0000", 2024-09-14T23:58:30.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7309999999999998e-06 2024-09-14T23:58:30.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:30.572 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929807+0000", 2024-09-14T23:58:30.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4349999999999999e-06 2024-09-14T23:58:30.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:30.572 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984063+0000", 2024-09-14T23:58:30.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054256331999999997 2024-09-14T23:58:30.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.573 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:30.573 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984177+0000", 2024-09-14T23:58:30.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114225 2024-09-14T23:58:30.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.573 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:30.573 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984266+0000", 2024-09-14T23:58:30.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8352999999999999e-05 2024-09-14T23:58:30.573 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:30.573 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:30.573 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:30.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.573 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129167 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 6782976~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:30.574 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.929827+0000", 2024-09-14T23:58:30.574 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.625060804, 2024-09-14T23:58:30.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054637593999999998, 2024-09-14T23:58:30.574 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:30.574 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:30.574 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:30.574 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:30.574 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:30.574 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:30.574 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129167 2024-09-14T23:58:30.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.574 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:30.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.574 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:30.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929827+0000", 2024-09-14T23:58:30.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:30.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:30.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929827+0000", 2024-09-14T23:58:30.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:30.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:30.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929827+0000", 2024-09-14T23:58:30.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4000000000000002e-07 2024-09-14T23:58:30.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:30.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929830+0000", 2024-09-14T23:58:30.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.419e-06 2024-09-14T23:58:30.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:30.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929837+0000", 2024-09-14T23:58:30.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0450000000000003e-06 2024-09-14T23:58:30.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:30.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929842+0000", 2024-09-14T23:58:30.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.69e-06 2024-09-14T23:58:30.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:30.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984278+0000", 2024-09-14T23:58:30.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054436516999999997 2024-09-14T23:58:30.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.577 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:30.577 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984331+0000", 2024-09-14T23:58:30.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2590999999999998e-05 2024-09-14T23:58:30.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.577 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:30.577 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984465+0000", 2024-09-14T23:58:30.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013389200000000001 2024-09-14T23:58:30.577 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:30.577 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:30.577 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:30.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.578 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129168 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 8224768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:30.578 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.929849+0000", 2024-09-14T23:58:30.578 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.625039246, 2024-09-14T23:58:30.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054870368000000003, 2024-09-14T23:58:30.578 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:30.578 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:30.578 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:30.578 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:30.578 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:30.578 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:30.578 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129168 2024-09-14T23:58:30.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.578 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:30.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:30.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929849+0000", 2024-09-14T23:58:30.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:30.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:30.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929849+0000", 2024-09-14T23:58:30.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:30.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:30.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931681+0000", 2024-09-14T23:58:30.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018326989999999999 2024-09-14T23:58:30.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:30.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931686+0000", 2024-09-14T23:58:30.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7190000000000001e-06 2024-09-14T23:58:30.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:30.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931697+0000", 2024-09-14T23:58:30.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1287999999999999e-05 2024-09-14T23:58:30.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:30.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931704+0000", 2024-09-14T23:58:30.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3339999999999998e-06 2024-09-14T23:58:30.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:30.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984505+0000", 2024-09-14T23:58:30.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052801684000000002 2024-09-14T23:58:30.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:30.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984621+0000", 2024-09-14T23:58:30.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000116165 2024-09-14T23:58:30.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:30.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984719+0000", 2024-09-14T23:58:30.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7479000000000006e-05 2024-09-14T23:58:30.581 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:30.581 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:30.581 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:30.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.582 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129176 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 10452992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:30.582 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.931830+0000", 2024-09-14T23:58:30.582 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.623057484999997, 2024-09-14T23:58:30.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054513292999999997, 2024-09-14T23:58:30.582 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:30.582 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:30.582 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:30.582 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:30.582 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:30.582 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:30.582 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129176 2024-09-14T23:58:30.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.582 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:30.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:30.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931830+0000", 2024-09-14T23:58:30.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:30.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:30.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931830+0000", 2024-09-14T23:58:30.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:30.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:30.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.933678+0000", 2024-09-14T23:58:30.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018472440000000001 2024-09-14T23:58:30.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.584 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:30.584 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.933686+0000", 2024-09-14T23:58:30.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0239999999999997e-06 2024-09-14T23:58:30.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.584 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:30.584 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.933694+0000", 2024-09-14T23:58:30.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8400000000000001e-06 2024-09-14T23:58:30.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.584 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:30.584 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.933701+0000", 2024-09-14T23:58:30.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2870000000000001e-06 2024-09-14T23:58:30.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:30.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.986165+0000", 2024-09-14T23:58:30.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052463900000000001 2024-09-14T23:58:30.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:30.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.986259+0000", 2024-09-14T23:58:30.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4435e-05 2024-09-14T23:58:30.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:30.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.986344+0000", 2024-09-14T23:58:30.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4562999999999997e-05 2024-09-14T23:58:30.586 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:30.586 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:30.586 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:30.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.586 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129201 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 10289152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:30.586 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.935960+0000", 2024-09-14T23:58:30.586 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.618928023000002, 2024-09-14T23:58:30.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054338487999999997, 2024-09-14T23:58:30.586 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:30.586 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:30.586 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:30.586 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:30.586 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:30.586 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:30.587 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129201 2024-09-14T23:58:30.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.587 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:30.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:30.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935960+0000", 2024-09-14T23:58:30.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:30.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:30.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935960+0000", 2024-09-14T23:58:30.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:30.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:30.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935960+0000", 2024-09-14T23:58:30.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.72e-07 2024-09-14T23:58:30.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:30.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935963+0000", 2024-09-14T23:58:30.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5390000000000002e-06 2024-09-14T23:58:30.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:30.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935971+0000", 2024-09-14T23:58:30.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4589999999999998e-06 2024-09-14T23:58:30.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:30.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935977+0000", 2024-09-14T23:58:30.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4110000000000004e-06 2024-09-14T23:58:30.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:30.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990213+0000", 2024-09-14T23:58:30.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054236642000000002 2024-09-14T23:58:30.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:30.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990257+0000", 2024-09-14T23:58:30.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3696000000000003e-05 2024-09-14T23:58:30.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:30.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990298+0000", 2024-09-14T23:58:30.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1369000000000003e-05 2024-09-14T23:58:30.590 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:30.590 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:30.590 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:30.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.590 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129202 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 11370496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:30.590 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.935986+0000", 2024-09-14T23:58:30.590 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.618901405000003, 2024-09-14T23:58:30.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054543700000000001, 2024-09-14T23:58:30.590 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:30.590 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:30.590 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:30.590 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:30.590 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:30.591 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:30.591 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129202 2024-09-14T23:58:30.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.591 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:30.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:30.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935986+0000", 2024-09-14T23:58:30.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:30.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:30.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935986+0000", 2024-09-14T23:58:30.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:30.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:30.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937678+0000", 2024-09-14T23:58:30.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016917849999999999 2024-09-14T23:58:30.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:30.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937682+0000", 2024-09-14T23:58:30.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2400000000000001e-06 2024-09-14T23:58:30.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:30.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937695+0000", 2024-09-14T23:58:30.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2999e-05 2024-09-14T23:58:30.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:30.593 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937700+0000", 2024-09-14T23:58:30.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8420000000000004e-06 2024-09-14T23:58:30.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:30.593 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990336+0000", 2024-09-14T23:58:30.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052636060999999998 2024-09-14T23:58:30.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:30.593 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990435+0000", 2024-09-14T23:58:30.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9015e-05 2024-09-14T23:58:30.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.594 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:30.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990530+0000", 2024-09-14T23:58:30.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4758000000000003e-05 2024-09-14T23:58:30.594 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:30.594 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:30.594 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:30.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.594 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129214 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 11468800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:30.594 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.937939+0000", 2024-09-14T23:58:30.594 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.616948491000002, 2024-09-14T23:58:30.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054463733, 2024-09-14T23:58:30.594 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:30.594 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:30.594 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:30.595 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:30.595 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:30.595 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:30.595 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129214 2024-09-14T23:58:30.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.595 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:30.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:30.595 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937939+0000", 2024-09-14T23:58:30.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:30.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:30.596 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937939+0000", 2024-09-14T23:58:30.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:30.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.596 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:30.596 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939674+0000", 2024-09-14T23:58:30.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017342589999999999 2024-09-14T23:58:30.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.596 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:30.596 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939678+0000", 2024-09-14T23:58:30.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2590000000000002e-06 2024-09-14T23:58:30.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.596 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:30.597 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939699+0000", 2024-09-14T23:58:30.597 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1143000000000001e-05 2024-09-14T23:58:30.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.597 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:30.597 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939706+0000", 2024-09-14T23:58:30.597 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0269999999999998e-06 2024-09-14T23:58:30.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.597 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:30.597 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.992221+0000", 2024-09-14T23:58:30.597 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052514981000000002 2024-09-14T23:58:30.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.598 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:30.598 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.992321+0000", 2024-09-14T23:58:30.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9625999999999998e-05 2024-09-14T23:58:30.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.598 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:30.598 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.992403+0000", 2024-09-14T23:58:30.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2437999999999999e-05 2024-09-14T23:58:30.598 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:30.598 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:30.598 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:30.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.599 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129226 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 14090240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:30.599 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.939867+0000", 2024-09-14T23:58:30.599 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.615020563000002, 2024-09-14T23:58:30.599 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054523573999999998, 2024-09-14T23:58:30.599 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:30.599 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:30.599 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:30.599 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:30.599 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:30.599 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:30.599 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129226 2024-09-14T23:58:30.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.599 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:30.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.599 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:30.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939867+0000", 2024-09-14T23:58:30.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:30.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.600 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:30.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939867+0000", 2024-09-14T23:58:30.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:30.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.600 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:30.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941669+0000", 2024-09-14T23:58:30.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018021549999999999 2024-09-14T23:58:30.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:30.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941673+0000", 2024-09-14T23:58:30.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1269999999999999e-06 2024-09-14T23:58:30.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:30.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941679+0000", 2024-09-14T23:58:30.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1879999999999999e-06 2024-09-14T23:58:30.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:30.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941682+0000", 2024-09-14T23:58:30.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7110000000000001e-06 2024-09-14T23:58:30.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.602 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:30.602 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.994201+0000", 2024-09-14T23:58:30.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052518265000000001 2024-09-14T23:58:30.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.602 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:30.602 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.994302+0000", 2024-09-14T23:58:30.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101653 2024-09-14T23:58:30.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.602 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:30.602 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.994391+0000", 2024-09-14T23:58:30.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8474999999999996e-05 2024-09-14T23:58:30.603 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:30.603 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:30.603 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:30.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.603 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129237 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 12386304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:30.603 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.941825+0000", 2024-09-14T23:58:30.603 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.613062800999998, 2024-09-14T23:58:30.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054368076000000001, 2024-09-14T23:58:30.603 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:30.603 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:30.603 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:30.603 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:30.603 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:30.603 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:30.603 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129237 2024-09-14T23:58:30.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.604 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:30.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:30.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941825+0000", 2024-09-14T23:58:30.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:30.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:30.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941825+0000", 2024-09-14T23:58:30.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:30.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:30.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941825+0000", 2024-09-14T23:58:30.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.29e-07 2024-09-14T23:58:30.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:30.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941826+0000", 2024-09-14T23:58:30.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0999999999999997e-07 2024-09-14T23:58:30.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:30.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941834+0000", 2024-09-14T23:58:30.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5519999999999999e-06 2024-09-14T23:58:30.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:30.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941843+0000", 2024-09-14T23:58:30.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2960000000000004e-06 2024-09-14T23:58:30.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.606 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:30.606 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996108+0000", 2024-09-14T23:58:30.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054265200999999999 2024-09-14T23:58:30.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.606 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:30.606 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996152+0000", 2024-09-14T23:58:30.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4066999999999997e-05 2024-09-14T23:58:30.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.606 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:30.606 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996193+0000", 2024-09-14T23:58:30.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0821000000000001e-05 2024-09-14T23:58:30.606 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:30.607 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:30.607 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:30.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.607 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129238 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 13828096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:30.607 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.941851+0000", 2024-09-14T23:58:30.607 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.613036477000001, 2024-09-14T23:58:30.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.05458031, 2024-09-14T23:58:30.607 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:30.607 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:30.607 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:30.607 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:30.607 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:30.607 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:30.607 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129238 2024-09-14T23:58:30.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.607 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:30.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:30.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941851+0000", 2024-09-14T23:58:30.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:30.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:30.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941851+0000", 2024-09-14T23:58:30.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:30.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:30.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943670+0000", 2024-09-14T23:58:30.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018181930000000001 2024-09-14T23:58:30.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:30.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943673+0000", 2024-09-14T23:58:30.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3790000000000002e-06 2024-09-14T23:58:30.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:30.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943679+0000", 2024-09-14T23:58:30.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8719999999999999e-06 2024-09-14T23:58:30.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:30.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943683+0000", 2024-09-14T23:58:30.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8539999999999999e-06 2024-09-14T23:58:30.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:30.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996233+0000", 2024-09-14T23:58:30.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052550690999999997 2024-09-14T23:58:30.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:30.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996339+0000", 2024-09-14T23:58:30.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105989 2024-09-14T23:58:30.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:30.611 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996432+0000", 2024-09-14T23:58:30.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2331999999999996e-05 2024-09-14T23:58:30.611 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:30.611 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:30.611 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:30.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.611 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129249 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 12124160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:30.611 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.943826+0000", 2024-09-14T23:58:30.611 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.611061986000003, 2024-09-14T23:58:30.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054373266000000003, 2024-09-14T23:58:30.611 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:30.611 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:30.611 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:30.611 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:30.612 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:30.612 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:30.612 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129249 2024-09-14T23:58:30.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.612 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:30.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.612 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:30.612 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943826+0000", 2024-09-14T23:58:30.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:30.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.612 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:30.612 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943826+0000", 2024-09-14T23:58:30.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:30.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:30.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943826+0000", 2024-09-14T23:58:30.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.03e-07 2024-09-14T23:58:30.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:30.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943827+0000", 2024-09-14T23:58:30.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.08e-06 2024-09-14T23:58:30.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:30.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943834+0000", 2024-09-14T23:58:30.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4119999999999998e-06 2024-09-14T23:58:30.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.614 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:30.614 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943836+0000", 2024-09-14T23:58:30.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7369999999999998e-06 2024-09-14T23:58:30.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.614 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:30.614 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998115+0000", 2024-09-14T23:58:30.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054279040000000001 2024-09-14T23:58:30.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.614 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:30.614 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998159+0000", 2024-09-14T23:58:30.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3486999999999999e-05 2024-09-14T23:58:30.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.615 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:30.615 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998199+0000", 2024-09-14T23:58:30.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0306999999999999e-05 2024-09-14T23:58:30.615 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:30.615 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:30.615 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:30.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.615 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129250 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 13565952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:30.615 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.943840+0000", 2024-09-14T23:58:30.615 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.611048036, 2024-09-14T23:58:30.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054573875000000001, 2024-09-14T23:58:30.615 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:30.615 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:30.615 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:30.616 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:30.616 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:30.616 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:30.616 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129250 2024-09-14T23:58:30.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.616 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:30.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:30.616 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943840+0000", 2024-09-14T23:58:30.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:30.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:30.616 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943840+0000", 2024-09-14T23:58:30.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:30.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:30.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945669+0000", 2024-09-14T23:58:30.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018289720000000001 2024-09-14T23:58:30.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:30.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945671+0000", 2024-09-14T23:58:30.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6869999999999999e-06 2024-09-14T23:58:30.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:30.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945680+0000", 2024-09-14T23:58:30.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5259999999999993e-06 2024-09-14T23:58:30.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:30.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945685+0000", 2024-09-14T23:58:30.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8559999999999998e-06 2024-09-14T23:58:30.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:30.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998242+0000", 2024-09-14T23:58:30.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052557505999999997 2024-09-14T23:58:30.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:30.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998329+0000", 2024-09-14T23:58:30.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6410999999999996e-05 2024-09-14T23:58:30.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.619 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:30.619 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998414+0000", 2024-09-14T23:58:30.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4917e-05 2024-09-14T23:58:30.619 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:30.619 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:30.619 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:30.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.619 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129262 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 13303808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:30.619 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.945825+0000", 2024-09-14T23:58:30.619 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.609063095000003, 2024-09-14T23:58:30.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054515942999999997, 2024-09-14T23:58:30.619 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:30.619 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:30.619 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:30.620 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:30.620 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:30.620 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:30.620 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129262 2024-09-14T23:58:30.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.620 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:30.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:30.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945825+0000", 2024-09-14T23:58:30.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:30.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:30.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945825+0000", 2024-09-14T23:58:30.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:30.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:30.621 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.947670+0000", 2024-09-14T23:58:30.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018451229999999999 2024-09-14T23:58:30.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:30.621 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.947672+0000", 2024-09-14T23:58:30.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5289999999999998e-06 2024-09-14T23:58:30.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:30.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.947678+0000", 2024-09-14T23:58:30.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9880000000000001e-06 2024-09-14T23:58:30.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:30.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.947682+0000", 2024-09-14T23:58:30.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.63e-06 2024-09-14T23:58:30.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:30.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000183+0000", 2024-09-14T23:58:30.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052500770000000002 2024-09-14T23:58:30.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:30.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000266+0000", 2024-09-14T23:58:30.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3133000000000005e-05 2024-09-14T23:58:30.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:30.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000341+0000", 2024-09-14T23:58:30.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4770000000000004e-05 2024-09-14T23:58:30.623 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:30.623 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:30.623 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:30.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.623 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129299 3.10 3:0f1e06ee:::rbd_object_map.1118721be852:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:30.623 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.952330+0000", 2024-09-14T23:58:30.623 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.602558281999997, 2024-09-14T23:58:30.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054412818000000002, 2024-09-14T23:58:30.624 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:30.624 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:30.624 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:30.624 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:30.624 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:30.624 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:30.624 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129299 2024-09-14T23:58:30.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.624 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:30.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.624 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:30.624 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952330+0000", 2024-09-14T23:58:30.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:30.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.625 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:30.625 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952330+0000", 2024-09-14T23:58:30.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:30.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.625 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:30.625 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952330+0000", 2024-09-14T23:58:30.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4900000000000001e-07 2024-09-14T23:58:30.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.625 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:30.625 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952332+0000", 2024-09-14T23:58:30.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2879999999999999e-06 2024-09-14T23:58:30.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:30.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952340+0000", 2024-09-14T23:58:30.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9910000000000002e-06 2024-09-14T23:58:30.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:30.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952344+0000", 2024-09-14T23:58:30.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0280000000000002e-06 2024-09-14T23:58:30.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:30.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.004722+0000", 2024-09-14T23:58:30.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052377932000000002 2024-09-14T23:58:30.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:30.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.004778+0000", 2024-09-14T23:58:30.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5795999999999999e-05 2024-09-14T23:58:30.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:58:30.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.005016+0000", 2024-09-14T23:58:30.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000238006 2024-09-14T23:58:30.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:58:30.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.006366+0000", 2024-09-14T23:58:30.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001349786 2024-09-14T23:58:30.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:30.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.006685+0000", 2024-09-14T23:58:30.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00031970000000000002 2024-09-14T23:58:30.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:30.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.006718+0000", 2024-09-14T23:58:30.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2329000000000002e-05 2024-09-14T23:58:30.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:30.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.006742+0000", 2024-09-14T23:58:30.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4613e-05 2024-09-14T23:58:30.629 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:30.629 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:30.629 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:30.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.629 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186210 3.d 3.b2ff3acd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:30.629 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.129044+0000", 2024-09-14T23:58:30.629 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.425843918999998, 2024-09-14T23:58:30.629 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.84555443900000005, 2024-09-14T23:58:30.629 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:30.629 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:30.629 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:58:30.629 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:30.629 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:30.629 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:30.629 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186210 2024-09-14T23:58:30.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.630 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:30.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:30.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129044+0000", 2024-09-14T23:58:30.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:30.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:30.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129044+0000", 2024-09-14T23:58:30.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:30.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:30.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129046+0000", 2024-09-14T23:58:30.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.742e-06 2024-09-14T23:58:30.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.631 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:30.631 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129052+0000", 2024-09-14T23:58:30.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7390000000000002e-06 2024-09-14T23:58:30.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.631 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:30.631 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129067+0000", 2024-09-14T23:58:30.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.417e-05 2024-09-14T23:58:30.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.631 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:30.631 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129076+0000", 2024-09-14T23:58:30.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2289999999999992e-06 2024-09-14T23:58:30.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.632 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:30.632 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.130947+0000", 2024-09-14T23:58:30.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001871075 2024-09-14T23:58:30.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.632 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:30.632 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.130955+0000", 2024-09-14T23:58:30.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4500000000000004e-06 2024-09-14T23:58:30.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.632 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:30.633 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974587+0000", 2024-09-14T23:58:30.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.84363205500000005 2024-09-14T23:58:30.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.633 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:30.633 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974598+0000", 2024-09-14T23:58:30.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0978999999999999e-05 2024-09-14T23:58:30.633 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:30.633 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:30.633 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:30.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.633 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187415 3.d 3.b2ff3acd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:30.633 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.254681+0000", 2024-09-14T23:58:30.634 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.300206398, 2024-09-14T23:58:30.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.71995530699999999, 2024-09-14T23:58:30.634 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:30.634 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:30.634 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:58:30.634 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:30.634 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:30.634 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:30.634 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187415 2024-09-14T23:58:30.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.634 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:30.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:30.634 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254681+0000", 2024-09-14T23:58:30.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:30.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:30.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254681+0000", 2024-09-14T23:58:30.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:30.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:30.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254682+0000", 2024-09-14T23:58:30.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2e-07 2024-09-14T23:58:30.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:30.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254684+0000", 2024-09-14T23:58:30.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2730000000000001e-06 2024-09-14T23:58:30.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:30.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254689+0000", 2024-09-14T23:58:30.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4769999999999997e-06 2024-09-14T23:58:30.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:30.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254692+0000", 2024-09-14T23:58:30.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9890000000000001e-06 2024-09-14T23:58:30.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:30.637 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254721+0000", 2024-09-14T23:58:30.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9762999999999999e-05 2024-09-14T23:58:30.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.637 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:30.637 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254725+0000", 2024-09-14T23:58:30.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9650000000000002e-06 2024-09-14T23:58:30.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.637 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:30.637 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974628+0000", 2024-09-14T23:58:30.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.71990235499999999 2024-09-14T23:58:30.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.637 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:30.638 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974637+0000", 2024-09-14T23:58:30.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0650000000000005e-06 2024-09-14T23:58:30.638 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:30.638 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:30.638 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:30.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.638 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187457 3.d 3.b2ff3acd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:30.638 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.257980+0000", 2024-09-14T23:58:30.638 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.296907824000002, 2024-09-14T23:58:30.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.71671711199999999, 2024-09-14T23:58:30.638 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:30.638 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:30.638 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:58:30.638 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:30.639 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:30.639 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:30.639 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187457 2024-09-14T23:58:30.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.639 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:30.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:30.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257980+0000", 2024-09-14T23:58:30.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:30.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:30.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257980+0000", 2024-09-14T23:58:30.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:30.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:30.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257980+0000", 2024-09-14T23:58:30.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.17e-07 2024-09-14T23:58:30.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:30.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257981+0000", 2024-09-14T23:58:30.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.172e-06 2024-09-14T23:58:30.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:30.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257984+0000", 2024-09-14T23:58:30.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9550000000000001e-06 2024-09-14T23:58:30.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:30.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257987+0000", 2024-09-14T23:58:30.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5799999999999999e-06 2024-09-14T23:58:30.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:30.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.258030+0000", 2024-09-14T23:58:30.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3118999999999998e-05 2024-09-14T23:58:30.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:30.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.258038+0000", 2024-09-14T23:58:30.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6850000000000001e-06 2024-09-14T23:58:30.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:30.642 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974688+0000", 2024-09-14T23:58:30.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.71665016000000004 2024-09-14T23:58:30.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:30.642 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974697+0000", 2024-09-14T23:58:30.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2240000000000001e-06 2024-09-14T23:58:30.642 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:30.642 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:30.642 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:30.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.642 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:251656 3.c 3:32436721:::rbd_object_map.1118e09062ff:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-14T23:58:30.643 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:27.433419+0000", 2024-09-14T23:58:30.643 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.1214691499999998, 2024-09-14T23:58:30.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.55083879499999999, 2024-09-14T23:58:30.643 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:30.643 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:30.643 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:30.643 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:30.643 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:30.643 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:30.643 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 251656 2024-09-14T23:58:30.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.643 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:30.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:30.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433419+0000", 2024-09-14T23:58:30.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:30.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:30.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433419+0000", 2024-09-14T23:58:30.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:30.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:30.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433419+0000", 2024-09-14T23:58:30.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1500000000000003e-07 2024-09-14T23:58:30.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:30.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433424+0000", 2024-09-14T23:58:30.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2409999999999997e-06 2024-09-14T23:58:30.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:30.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433435+0000", 2024-09-14T23:58:30.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1009000000000001e-05 2024-09-14T23:58:30.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:30.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433440+0000", 2024-09-14T23:58:30.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1950000000000002e-06 2024-09-14T23:58:30.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:30.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433487+0000", 2024-09-14T23:58:30.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7184000000000001e-05 2024-09-14T23:58:30.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:30.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433494+0000", 2024-09-14T23:58:30.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5490000000000003e-06 2024-09-14T23:58:30.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:30.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.981923+0000", 2024-09-14T23:58:30.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.54842898600000001 2024-09-14T23:58:30.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:30.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.982097+0000", 2024-09-14T23:58:30.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017428699999999999 2024-09-14T23:58:30.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5", 2024-09-14T23:58:30.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.982334+0000", 2024-09-14T23:58:30.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00023700899999999999 2024-09-14T23:58:30.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:30.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.983927+0000", 2024-09-14T23:58:30.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0015932710000000001 2024-09-14T23:58:30.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:58:30.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.984128+0000", 2024-09-14T23:58:30.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020104199999999999 2024-09-14T23:58:30.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:30.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.984212+0000", 2024-09-14T23:58:30.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3709999999999996e-05 2024-09-14T23:58:30.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:30.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:30.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:30.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.984257+0000", 2024-09-14T23:58:30.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5497000000000002e-05 2024-09-14T23:58:30.648 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:30.648 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:30.648 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:30.648 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:30.648 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:30.649 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:30.650 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T23:58:30.650 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:30.648+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:30.746 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:58:30.746 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:58:30.750 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:30.749+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:30.799 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:30.799 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:58:30.799 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:58:30.799 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:30.807 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T23:58:30.851 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:30.850+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:30.915 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:30.915 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:58:30.915 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:58:30.916 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:58:30.916 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:30.924 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T23:58:30.968 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:58:30.967+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:58:30.976 INFO:teuthology.orchestra.run.smithi042.stderr:set 2.0 pg_upmap_items mapping to [2->0,3->1] 2024-09-14T23:58:30.995 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T23:58:31.040 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:31.040 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:58:31.040 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:58:31.040 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:58:31.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.040 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69262 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 4800512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:31.040 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300811+0000", 2024-09-14T23:58:31.041 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 75.737264945000007, 2024-09-14T23:58:31.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050403796000000001, 2024-09-14T23:58:31.041 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:31.041 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:31.041 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:31.041 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:31.041 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:31.041 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:31.041 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69262 2024-09-14T23:58:31.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.041 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:31.041 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.041 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:31.041 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300811+0000", 2024-09-14T23:58:31.042 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:31.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.042 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.042 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:31.042 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300811+0000", 2024-09-14T23:58:31.042 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:31.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.042 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.042 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:31.042 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300811+0000", 2024-09-14T23:58:31.042 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5700000000000002e-07 2024-09-14T23:58:31.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.042 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.042 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:31.043 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300815+0000", 2024-09-14T23:58:31.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.087e-06 2024-09-14T23:58:31.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.043 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:31.043 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300823+0000", 2024-09-14T23:58:31.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8720000000000002e-06 2024-09-14T23:58:31.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.043 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:31.043 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300830+0000", 2024-09-14T23:58:31.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0589999999999997e-06 2024-09-14T23:58:31.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.044 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:31.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351020+0000", 2024-09-14T23:58:31.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050189552999999998 2024-09-14T23:58:31.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.044 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:31.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351112+0000", 2024-09-14T23:58:31.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2615999999999996e-05 2024-09-14T23:58:31.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.044 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:31.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351214+0000", 2024-09-14T23:58:31.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101952 2024-09-14T23:58:31.044 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:31.045 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:31.045 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:31.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.045 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.045 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69263 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 4915200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:31.045 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300842+0000", 2024-09-14T23:58:31.045 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 75.737233441000001, 2024-09-14T23:58:31.045 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050590637000000001, 2024-09-14T23:58:31.045 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:31.045 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:31.045 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:31.045 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:31.045 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:31.045 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:31.046 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69263 2024-09-14T23:58:31.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.046 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:31.046 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.046 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:31.046 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300842+0000", 2024-09-14T23:58:31.046 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:31.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.046 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.046 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:31.046 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300842+0000", 2024-09-14T23:58:31.046 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:31.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.046 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.047 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:31.047 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300843+0000", 2024-09-14T23:58:31.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4000000000000002e-07 2024-09-14T23:58:31.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.047 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:31.047 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300846+0000", 2024-09-14T23:58:31.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8720000000000002e-06 2024-09-14T23:58:31.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.047 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:31.047 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300853+0000", 2024-09-14T23:58:31.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0450000000000003e-06 2024-09-14T23:58:31.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.048 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.048 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:31.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300859+0000", 2024-09-14T23:58:31.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9510000000000003e-06 2024-09-14T23:58:31.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.048 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.048 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:31.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351260+0000", 2024-09-14T23:58:31.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050401654999999997 2024-09-14T23:58:31.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.048 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.048 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:31.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351353+0000", 2024-09-14T23:58:31.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2785e-05 2024-09-14T23:58:31.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.049 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.049 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:31.049 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351433+0000", 2024-09-14T23:58:31.049 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9789000000000005e-05 2024-09-14T23:58:31.049 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:31.049 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:31.049 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:31.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.049 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.049 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69264 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5029888~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:31.049 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300866+0000", 2024-09-14T23:58:31.049 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 75.737209508000007, 2024-09-14T23:58:31.049 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050794063, 2024-09-14T23:58:31.050 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:31.050 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:31.050 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:31.050 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:31.050 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:31.050 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:31.050 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69264 2024-09-14T23:58:31.050 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.050 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:31.050 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.050 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:31.050 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300866+0000", 2024-09-14T23:58:31.050 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:31.050 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.050 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.051 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:31.051 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300866+0000", 2024-09-14T23:58:31.051 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:31.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.051 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.051 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:31.051 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300866+0000", 2024-09-14T23:58:31.051 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8799999999999998e-07 2024-09-14T23:58:31.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.051 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.051 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:31.051 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300869+0000", 2024-09-14T23:58:31.051 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0880000000000002e-06 2024-09-14T23:58:31.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.051 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.052 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:31.052 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300875+0000", 2024-09-14T23:58:31.052 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9280000000000004e-06 2024-09-14T23:58:31.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.052 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.052 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:31.052 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300882+0000", 2024-09-14T23:58:31.052 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4740000000000002e-06 2024-09-14T23:58:31.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.052 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.052 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:31.052 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351472+0000", 2024-09-14T23:58:31.052 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050590006999999999 2024-09-14T23:58:31.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.052 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.053 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:31.053 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351563+0000", 2024-09-14T23:58:31.053 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0762999999999998e-05 2024-09-14T23:58:31.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.053 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:31.053 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351660+0000", 2024-09-14T23:58:31.053 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7415000000000002e-05 2024-09-14T23:58:31.053 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:31.053 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:31.053 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:31.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.053 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69265 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5144576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:31.053 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300890+0000", 2024-09-14T23:58:31.054 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 75.737186003999994, 2024-09-14T23:58:31.054 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050921389999999997, 2024-09-14T23:58:31.054 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:31.054 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:31.054 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:31.054 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:31.054 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:31.054 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:31.054 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69265 2024-09-14T23:58:31.054 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.054 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:31.054 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.054 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:31.055 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300890+0000", 2024-09-14T23:58:31.055 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:31.055 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.055 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.055 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:31.055 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300890+0000", 2024-09-14T23:58:31.055 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:31.055 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.055 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.055 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:31.055 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300890+0000", 2024-09-14T23:58:31.055 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2500000000000001e-07 2024-09-14T23:58:31.055 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.055 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.055 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:31.056 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300892+0000", 2024-09-14T23:58:31.056 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2469999999999998e-06 2024-09-14T23:58:31.056 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.056 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.056 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:31.056 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300899+0000", 2024-09-14T23:58:31.056 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5509999999999996e-06 2024-09-14T23:58:31.056 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.056 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.056 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:31.056 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300904+0000", 2024-09-14T23:58:31.056 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3299999999999998e-06 2024-09-14T23:58:31.056 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.056 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.057 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:31.057 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351707+0000", 2024-09-14T23:58:31.057 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050802673999999999 2024-09-14T23:58:31.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.057 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.057 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:31.057 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351755+0000", 2024-09-14T23:58:31.057 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8362000000000002e-05 2024-09-14T23:58:31.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.057 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.057 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:31.057 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351811+0000", 2024-09-14T23:58:31.057 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5801000000000003e-05 2024-09-14T23:58:31.058 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:31.058 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:31.058 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:31.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.058 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.058 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69266 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5259264~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:31.058 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300911+0000", 2024-09-14T23:58:31.058 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 75.737164930999995, 2024-09-14T23:58:31.058 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051125089999999998, 2024-09-14T23:58:31.058 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:31.058 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:31.058 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:31.058 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:31.058 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:31.059 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:31.059 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69266 2024-09-14T23:58:31.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.059 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:31.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.059 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:31.059 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300911+0000", 2024-09-14T23:58:31.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:31.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.059 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:31.059 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300911+0000", 2024-09-14T23:58:31.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:31.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.060 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:31.060 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300911+0000", 2024-09-14T23:58:31.060 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-14T23:58:31.060 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.060 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.060 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:31.060 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300913+0000", 2024-09-14T23:58:31.060 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9300000000000002e-06 2024-09-14T23:58:31.060 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.060 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.060 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:31.060 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300919+0000", 2024-09-14T23:58:31.060 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.489e-06 2024-09-14T23:58:31.060 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.061 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.061 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:31.061 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300925+0000", 2024-09-14T23:58:31.061 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1719999999999998e-06 2024-09-14T23:58:31.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.061 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.061 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:31.061 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351847+0000", 2024-09-14T23:58:31.061 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050922621000000001 2024-09-14T23:58:31.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.061 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.061 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:31.061 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351938+0000", 2024-09-14T23:58:31.061 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0830999999999998e-05 2024-09-14T23:58:31.062 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.062 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.062 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:31.062 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352036+0000", 2024-09-14T23:58:31.062 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7676999999999993e-05 2024-09-14T23:58:31.062 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:31.062 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:31.062 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:31.062 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.062 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.062 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69267 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5373952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:31.062 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300945+0000", 2024-09-14T23:58:31.062 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 75.737130136000005, 2024-09-14T23:58:31.062 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051217934, 2024-09-14T23:58:31.062 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:31.063 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:31.063 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:31.063 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:31.063 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:31.063 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:31.063 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69267 2024-09-14T23:58:31.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.063 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:31.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:31.063 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300945+0000", 2024-09-14T23:58:31.063 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:31.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:31.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300945+0000", 2024-09-14T23:58:31.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:31.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:31.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300946+0000", 2024-09-14T23:58:31.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2600000000000002e-07 2024-09-14T23:58:31.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:31.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300948+0000", 2024-09-14T23:58:31.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4940000000000002e-06 2024-09-14T23:58:31.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:31.065 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300956+0000", 2024-09-14T23:58:31.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.622e-06 2024-09-14T23:58:31.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.065 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.065 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:31.065 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300962+0000", 2024-09-14T23:58:31.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5790000000000003e-06 2024-09-14T23:58:31.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.065 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.065 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:31.065 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352063+0000", 2024-09-14T23:58:31.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051101281999999998 2024-09-14T23:58:31.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.065 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.065 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:31.066 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352110+0000", 2024-09-14T23:58:31.066 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7401000000000002e-05 2024-09-14T23:58:31.066 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.066 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.066 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:31.066 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352163+0000", 2024-09-14T23:58:31.066 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3029999999999999e-05 2024-09-14T23:58:31.067 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:31.067 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:31.068 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:31.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.068 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.068 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69268 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5488640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:31.068 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300969+0000", 2024-09-14T23:58:31.068 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 75.737106463999993, 2024-09-14T23:58:31.068 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051413420000000001, 2024-09-14T23:58:31.068 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:31.068 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:31.068 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:31.068 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:31.068 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:31.068 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:31.068 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69268 2024-09-14T23:58:31.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.069 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:31.069 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.069 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:31.069 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300969+0000", 2024-09-14T23:58:31.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:31.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.069 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.069 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:31.069 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300969+0000", 2024-09-14T23:58:31.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:31.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.069 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.069 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:31.069 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300970+0000", 2024-09-14T23:58:31.070 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7899999999999999e-07 2024-09-14T23:58:31.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.070 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:31.070 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300982+0000", 2024-09-14T23:58:31.070 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2741e-05 2024-09-14T23:58:31.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.070 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:31.070 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300990+0000", 2024-09-14T23:58:31.070 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8169999999999999e-06 2024-09-14T23:58:31.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.070 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:31.071 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300995+0000", 2024-09-14T23:58:31.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2240000000000003e-06 2024-09-14T23:58:31.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.071 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.071 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:31.071 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352200+0000", 2024-09-14T23:58:31.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051204223 2024-09-14T23:58:31.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.071 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.071 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:31.071 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352285+0000", 2024-09-14T23:58:31.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5519999999999994e-05 2024-09-14T23:58:31.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.071 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.071 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:31.071 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352383+0000", 2024-09-14T23:58:31.072 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7516000000000006e-05 2024-09-14T23:58:31.072 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:31.072 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:31.072 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:31.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.072 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.072 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69269 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5603328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:31.072 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.301003+0000", 2024-09-14T23:58:31.072 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 75.737072282, 2024-09-14T23:58:31.072 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051497381000000002, 2024-09-14T23:58:31.072 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:31.072 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:31.072 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:31.072 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:31.073 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:31.073 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:31.073 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69269 2024-09-14T23:58:31.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.073 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:31.073 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.073 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:31.073 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.301003+0000", 2024-09-14T23:58:31.073 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:31.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.073 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.073 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:31.073 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.301003+0000", 2024-09-14T23:58:31.073 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:31.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.074 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.074 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:31.074 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302671+0000", 2024-09-14T23:58:31.074 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001667785 2024-09-14T23:58:31.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.074 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.074 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:31.074 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302676+0000", 2024-09-14T23:58:31.074 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6600000000000003e-06 2024-09-14T23:58:31.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.074 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.074 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:31.074 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302697+0000", 2024-09-14T23:58:31.075 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1288999999999999e-05 2024-09-14T23:58:31.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.075 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.075 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:31.075 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302716+0000", 2024-09-14T23:58:31.075 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9128999999999999e-05 2024-09-14T23:58:31.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.075 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.075 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:31.075 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352399+0000", 2024-09-14T23:58:31.075 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049682995000000001 2024-09-14T23:58:31.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.075 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.075 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:31.076 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352446+0000", 2024-09-14T23:58:31.076 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6341000000000002e-05 2024-09-14T23:58:31.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.076 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.076 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:31.076 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352501+0000", 2024-09-14T23:58:31.076 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5182000000000001e-05 2024-09-14T23:58:31.076 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:31.076 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:31.076 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:31.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.076 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.076 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69273 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6062080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:31.077 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302829+0000", 2024-09-14T23:58:31.077 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 75.735246716000006, 2024-09-14T23:58:31.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050384311000000001, 2024-09-14T23:58:31.077 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:31.077 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:31.077 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:31.077 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:31.077 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:31.077 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:31.077 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69273 2024-09-14T23:58:31.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.077 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:31.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.077 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:31.078 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302829+0000", 2024-09-14T23:58:31.078 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:31.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.078 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.078 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:31.078 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302829+0000", 2024-09-14T23:58:31.078 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:31.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.078 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.078 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:31.078 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302829+0000", 2024-09-14T23:58:31.078 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7700000000000005e-07 2024-09-14T23:58:31.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.078 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.079 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:31.079 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302832+0000", 2024-09-14T23:58:31.079 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6970000000000002e-06 2024-09-14T23:58:31.079 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.079 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.079 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:31.079 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302840+0000", 2024-09-14T23:58:31.079 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5719999999999997e-06 2024-09-14T23:58:31.079 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.079 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.079 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:31.079 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302845+0000", 2024-09-14T23:58:31.079 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7939999999999999e-06 2024-09-14T23:58:31.079 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.079 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.080 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:31.080 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353111+0000", 2024-09-14T23:58:31.080 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050265588 2024-09-14T23:58:31.080 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.080 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.080 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:31.080 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353160+0000", 2024-09-14T23:58:31.080 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8859000000000001e-05 2024-09-14T23:58:31.080 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.080 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.080 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:31.080 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353213+0000", 2024-09-14T23:58:31.080 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3324e-05 2024-09-14T23:58:31.080 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:31.081 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:31.081 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:31.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.081 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.081 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69274 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6176768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:31.081 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302854+0000", 2024-09-14T23:58:31.081 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 75.735221621999997, 2024-09-14T23:58:31.081 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050579526, 2024-09-14T23:58:31.081 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:31.081 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:31.081 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:31.081 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:31.081 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:31.081 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:31.082 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69274 2024-09-14T23:58:31.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.082 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:31.082 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.082 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:31.082 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302854+0000", 2024-09-14T23:58:31.082 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:31.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.082 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.082 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:31.082 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302854+0000", 2024-09-14T23:58:31.082 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:31.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.082 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.083 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:31.083 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302855+0000", 2024-09-14T23:58:31.083 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0200000000000001e-07 2024-09-14T23:58:31.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.083 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.083 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:31.083 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302857+0000", 2024-09-14T23:58:31.083 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.447e-06 2024-09-14T23:58:31.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.083 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.083 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:31.083 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302864+0000", 2024-09-14T23:58:31.083 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7730000000000002e-06 2024-09-14T23:58:31.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.084 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.084 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:31.084 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302869+0000", 2024-09-14T23:58:31.084 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3199999999999999e-06 2024-09-14T23:58:31.084 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.084 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.084 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:31.084 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353249+0000", 2024-09-14T23:58:31.084 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050379647999999999 2024-09-14T23:58:31.084 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.084 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.084 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:31.084 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353336+0000", 2024-09-14T23:58:31.084 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6767999999999993e-05 2024-09-14T23:58:31.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.085 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:31.085 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353433+0000", 2024-09-14T23:58:31.085 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7868000000000006e-05 2024-09-14T23:58:31.085 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:31.085 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:31.085 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:31.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.085 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69275 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6291456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:31.085 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302889+0000", 2024-09-14T23:58:31.085 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 75.735187003999997, 2024-09-14T23:58:31.085 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050663118, 2024-09-14T23:58:31.085 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:31.085 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:31.086 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:31.086 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:31.086 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:31.086 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:31.086 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69275 2024-09-14T23:58:31.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.086 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:31.086 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.086 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:31.086 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302889+0000", 2024-09-14T23:58:31.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:31.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.086 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.086 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:31.086 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302889+0000", 2024-09-14T23:58:31.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:31.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.087 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:31.087 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302889+0000", 2024-09-14T23:58:31.087 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6399999999999998e-07 2024-09-14T23:58:31.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.087 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:31.087 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302891+0000", 2024-09-14T23:58:31.087 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.339e-06 2024-09-14T23:58:31.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.087 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:31.087 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302898+0000", 2024-09-14T23:58:31.088 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0380000000000002e-06 2024-09-14T23:58:31.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.088 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.088 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:31.088 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302904+0000", 2024-09-14T23:58:31.088 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2159999999999997e-06 2024-09-14T23:58:31.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.088 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.088 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:31.088 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353449+0000", 2024-09-14T23:58:31.088 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050545386999999997 2024-09-14T23:58:31.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.088 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.088 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:31.088 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353494+0000", 2024-09-14T23:58:31.089 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4592999999999998e-05 2024-09-14T23:58:31.089 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.089 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.089 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:31.089 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353552+0000", 2024-09-14T23:58:31.089 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8180999999999997e-05 2024-09-14T23:58:31.089 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:31.089 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:31.089 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:31.089 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.089 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.089 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69276 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6406144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:31.089 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302911+0000", 2024-09-14T23:58:31.089 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 75.735164591, 2024-09-14T23:58:31.089 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050865753, 2024-09-14T23:58:31.090 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:31.090 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:31.090 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:31.090 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:31.090 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:31.090 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:31.090 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69276 2024-09-14T23:58:31.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.090 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:31.090 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.090 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:31.090 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302911+0000", 2024-09-14T23:58:31.090 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:31.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.090 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.090 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:31.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302911+0000", 2024-09-14T23:58:31.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:31.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:31.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302912+0000", 2024-09-14T23:58:31.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0800000000000004e-07 2024-09-14T23:58:31.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:31.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302914+0000", 2024-09-14T23:58:31.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9589999999999998e-06 2024-09-14T23:58:31.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.092 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:31.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302920+0000", 2024-09-14T23:58:31.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4579999999999998e-06 2024-09-14T23:58:31.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.092 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.092 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:31.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302925+0000", 2024-09-14T23:58:31.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9429999999999999e-06 2024-09-14T23:58:31.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.092 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.092 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:31.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353586+0000", 2024-09-14T23:58:31.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050660966000000002 2024-09-14T23:58:31.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.092 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.093 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:31.093 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353679+0000", 2024-09-14T23:58:31.093 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3163000000000002e-05 2024-09-14T23:58:31.093 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.093 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.093 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:31.093 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353777+0000", 2024-09-14T23:58:31.093 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7555999999999999e-05 2024-09-14T23:58:31.093 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:31.093 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:31.093 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:31.093 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.093 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.093 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69277 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6520832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:31.093 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302932+0000", 2024-09-14T23:58:31.094 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 75.735143504000007, 2024-09-14T23:58:31.094 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050977674000000001, 2024-09-14T23:58:31.094 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:31.094 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:31.094 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:31.094 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:31.094 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:31.094 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:31.094 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69277 2024-09-14T23:58:31.094 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.094 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:31.094 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.094 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:31.094 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302932+0000", 2024-09-14T23:58:31.094 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:31.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:31.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302932+0000", 2024-09-14T23:58:31.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:31.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:31.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302932+0000", 2024-09-14T23:58:31.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3299999999999998e-07 2024-09-14T23:58:31.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:31.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302935+0000", 2024-09-14T23:58:31.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2139999999999999e-06 2024-09-14T23:58:31.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.096 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:31.096 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302941+0000", 2024-09-14T23:58:31.096 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5039999999999999e-06 2024-09-14T23:58:31.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.096 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:31.096 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302946+0000", 2024-09-14T23:58:31.096 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2079999999999999e-06 2024-09-14T23:58:31.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.096 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:31.097 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353805+0000", 2024-09-14T23:58:31.097 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050858588000000003 2024-09-14T23:58:31.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.097 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.097 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:31.097 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353849+0000", 2024-09-14T23:58:31.097 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4511000000000001e-05 2024-09-14T23:58:31.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.097 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.097 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:31.097 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353910+0000", 2024-09-14T23:58:31.097 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0316000000000003e-05 2024-09-14T23:58:31.097 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:31.097 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:31.097 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:31.098 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.098 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.098 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69278 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6635520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:31.098 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302953+0000", 2024-09-14T23:58:31.098 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 75.735122376000007, 2024-09-14T23:58:31.098 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051174059000000001, 2024-09-14T23:58:31.098 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:31.098 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:31.098 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:31.098 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:31.098 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:31.098 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:31.098 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69278 2024-09-14T23:58:31.098 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.099 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:31.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.099 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:31.099 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302953+0000", 2024-09-14T23:58:31.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:31.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.099 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:31.099 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302953+0000", 2024-09-14T23:58:31.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:31.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.099 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:31.099 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302954+0000", 2024-09-14T23:58:31.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5100000000000001e-07 2024-09-14T23:58:31.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.100 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.100 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:31.100 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302964+0000", 2024-09-14T23:58:31.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0475000000000001e-05 2024-09-14T23:58:31.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.100 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.100 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:31.100 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302970+0000", 2024-09-14T23:58:31.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3879999999999997e-06 2024-09-14T23:58:31.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.100 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.100 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:31.101 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302975+0000", 2024-09-14T23:58:31.101 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8409999999999999e-06 2024-09-14T23:58:31.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.101 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:31.101 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353944+0000", 2024-09-14T23:58:31.101 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050968952999999997 2024-09-14T23:58:31.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.101 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:31.101 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354031+0000", 2024-09-14T23:58:31.101 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6537000000000002e-05 2024-09-14T23:58:31.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.102 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:31.102 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354127+0000", 2024-09-14T23:58:31.102 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6514000000000005e-05 2024-09-14T23:58:31.102 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:31.102 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:31.102 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:31.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.102 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.102 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69279 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6750208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:31.102 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302984+0000", 2024-09-14T23:58:31.102 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 75.735092025, 2024-09-14T23:58:31.102 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051258845999999997, 2024-09-14T23:58:31.102 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:31.102 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:31.102 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:31.103 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:31.103 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:31.103 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:31.103 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69279 2024-09-14T23:58:31.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.103 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:31.103 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.103 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:31.103 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302984+0000", 2024-09-14T23:58:31.103 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:31.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.103 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.103 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:31.103 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302984+0000", 2024-09-14T23:58:31.104 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:31.104 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.104 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.104 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:31.104 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303025+0000", 2024-09-14T23:58:31.104 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1690999999999998e-05 2024-09-14T23:58:31.104 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.104 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.104 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:31.104 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303029+0000", 2024-09-14T23:58:31.104 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6559999999999998e-06 2024-09-14T23:58:31.104 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.104 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.104 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:31.105 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303037+0000", 2024-09-14T23:58:31.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7950000000000008e-06 2024-09-14T23:58:31.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.105 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:31.105 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303043+0000", 2024-09-14T23:58:31.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9669999999999998e-06 2024-09-14T23:58:31.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.105 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:31.105 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354142+0000", 2024-09-14T23:58:31.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051099072000000002 2024-09-14T23:58:31.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.106 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:31.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354183+0000", 2024-09-14T23:58:31.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0834000000000002e-05 2024-09-14T23:58:31.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.106 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:31.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354242+0000", 2024-09-14T23:58:31.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9830999999999997e-05 2024-09-14T23:58:31.106 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:31.106 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:31.106 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:31.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.106 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69280 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6864896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:31.106 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.303052+0000", 2024-09-14T23:58:31.107 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 75.735023482000003, 2024-09-14T23:58:31.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051406463999999999, 2024-09-14T23:58:31.107 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:31.107 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:31.107 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:31.107 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:31.107 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:31.107 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:31.107 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69280 2024-09-14T23:58:31.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.107 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:31.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.107 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:31.107 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303052+0000", 2024-09-14T23:58:31.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:31.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:31.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303052+0000", 2024-09-14T23:58:31.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:31.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:31.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304673+0000", 2024-09-14T23:58:31.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001621389 2024-09-14T23:58:31.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:31.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304681+0000", 2024-09-14T23:58:31.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8439999999999998e-06 2024-09-14T23:58:31.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:31.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304691+0000", 2024-09-14T23:58:31.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8339999999999994e-06 2024-09-14T23:58:31.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:31.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304698+0000", 2024-09-14T23:58:31.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3629999999999999e-06 2024-09-14T23:58:31.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:31.110 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354278+0000", 2024-09-14T23:58:31.110 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049580618 2024-09-14T23:58:31.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.110 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.110 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:31.110 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354364+0000", 2024-09-14T23:58:31.110 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6059999999999999e-05 2024-09-14T23:58:31.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.110 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.110 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:31.110 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354459+0000", 2024-09-14T23:58:31.110 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4356000000000003e-05 2024-09-14T23:58:31.110 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:31.110 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:31.111 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:31.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.111 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.111 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69285 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 7438336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:31.111 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.304792+0000", 2024-09-14T23:58:31.111 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 75.733284072999993, 2024-09-14T23:58:31.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050398874000000003, 2024-09-14T23:58:31.111 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:31.111 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:31.111 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:31.111 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:31.111 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:31.111 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:31.111 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69285 2024-09-14T23:58:31.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.112 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:31.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:31.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304792+0000", 2024-09-14T23:58:31.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:31.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:31.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304792+0000", 2024-09-14T23:58:31.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:31.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:31.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304792+0000", 2024-09-14T23:58:31.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2800000000000003e-07 2024-09-14T23:58:31.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.113 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:31.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304794+0000", 2024-09-14T23:58:31.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8700000000000001e-06 2024-09-14T23:58:31.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.113 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:31.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304800+0000", 2024-09-14T23:58:31.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0889999999999996e-06 2024-09-14T23:58:31.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.113 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:31.114 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304805+0000", 2024-09-14T23:58:31.114 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7060000000000003e-06 2024-09-14T23:58:31.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.114 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:31.114 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355108+0000", 2024-09-14T23:58:31.114 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050303170000000001 2024-09-14T23:58:31.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.114 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:31.114 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355150+0000", 2024-09-14T23:58:31.114 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2271999999999998e-05 2024-09-14T23:58:31.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.114 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:31.115 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355190+0000", 2024-09-14T23:58:31.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0438999999999997e-05 2024-09-14T23:58:31.115 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:31.115 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:31.115 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:31.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.115 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69286 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 7553024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:31.115 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.304811+0000", 2024-09-14T23:58:31.115 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 75.733264711000004, 2024-09-14T23:58:31.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050585512999999999, 2024-09-14T23:58:31.115 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:31.115 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:31.115 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:31.116 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:31.116 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:31.116 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:31.116 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69286 2024-09-14T23:58:31.116 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.116 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:31.116 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.116 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:31.116 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304811+0000", 2024-09-14T23:58:31.116 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:31.116 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.116 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.116 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:31.116 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304811+0000", 2024-09-14T23:58:31.116 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:31.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:31.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304811+0000", 2024-09-14T23:58:31.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.89e-07 2024-09-14T23:58:31.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:31.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304814+0000", 2024-09-14T23:58:31.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3199999999999998e-06 2024-09-14T23:58:31.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:31.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304821+0000", 2024-09-14T23:58:31.118 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5689999999999999e-06 2024-09-14T23:58:31.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.118 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.118 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:31.118 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304826+0000", 2024-09-14T23:58:31.118 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1200000000000001e-06 2024-09-14T23:58:31.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.118 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.118 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:31.118 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355226+0000", 2024-09-14T23:58:31.118 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050399653000000003 2024-09-14T23:58:31.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.118 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.118 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:31.118 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355314+0000", 2024-09-14T23:58:31.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8389999999999999e-05 2024-09-14T23:58:31.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.119 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:31.119 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355396+0000", 2024-09-14T23:58:31.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2071999999999996e-05 2024-09-14T23:58:31.119 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:31.119 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:31.119 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:31.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.119 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69287 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 7667712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:31.119 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.304833+0000", 2024-09-14T23:58:31.119 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 75.733242113000003, 2024-09-14T23:58:31.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050664740999999999, 2024-09-14T23:58:31.120 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:31.120 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:31.120 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:31.120 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:31.120 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:31.120 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:31.120 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69287 2024-09-14T23:58:31.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.120 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:31.120 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.120 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:31.120 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304833+0000", 2024-09-14T23:58:31.120 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:31.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.121 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.121 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:31.121 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304833+0000", 2024-09-14T23:58:31.121 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:31.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.121 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.121 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:31.121 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304834+0000", 2024-09-14T23:58:31.121 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4999999999999998e-07 2024-09-14T23:58:31.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.121 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.121 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:31.121 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304836+0000", 2024-09-14T23:58:31.121 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0279999999999999e-06 2024-09-14T23:58:31.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.122 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.122 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:31.122 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304842+0000", 2024-09-14T23:58:31.122 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8919999999999997e-06 2024-09-14T23:58:31.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.122 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.122 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:31.122 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304847+0000", 2024-09-14T23:58:31.122 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6160000000000001e-06 2024-09-14T23:58:31.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.122 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.122 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:31.122 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355411+0000", 2024-09-14T23:58:31.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050563540999999997 2024-09-14T23:58:31.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.123 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.123 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:31.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355459+0000", 2024-09-14T23:58:31.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7979000000000003e-05 2024-09-14T23:58:31.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.123 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.123 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:31.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355498+0000", 2024-09-14T23:58:31.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9335000000000002e-05 2024-09-14T23:58:31.123 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:31.123 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:31.123 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:31.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.124 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69288 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 7782400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:31.124 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.304854+0000", 2024-09-14T23:58:31.124 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 75.733221674999996, 2024-09-14T23:58:31.124 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050855145999999997, 2024-09-14T23:58:31.124 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:31.124 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:31.124 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:31.124 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:31.124 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:31.124 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:31.124 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69288 2024-09-14T23:58:31.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.124 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:31.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.125 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:31.125 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304854+0000", 2024-09-14T23:58:31.125 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:31.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.125 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.125 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:31.125 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304854+0000", 2024-09-14T23:58:31.125 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:31.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.125 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.125 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:31.125 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.306670+0000", 2024-09-14T23:58:31.125 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00181631 2024-09-14T23:58:31.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.126 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.126 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:31.126 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.306675+0000", 2024-09-14T23:58:31.126 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0220000000000004e-06 2024-09-14T23:58:31.126 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.126 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.126 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:31.126 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.306684+0000", 2024-09-14T23:58:31.126 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7309999999999999e-06 2024-09-14T23:58:31.126 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.126 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.126 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:31.126 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.306689+0000", 2024-09-14T23:58:31.126 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6609999999999999e-06 2024-09-14T23:58:31.127 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.127 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.127 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:31.127 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355533+0000", 2024-09-14T23:58:31.127 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048844578 2024-09-14T23:58:31.127 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.127 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.127 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:31.127 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355622+0000", 2024-09-14T23:58:31.127 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9006999999999997e-05 2024-09-14T23:58:31.127 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:31.127 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:31.127 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:31.127 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355709+0000", 2024-09-14T23:58:31.127 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6836999999999995e-05 2024-09-14T23:58:31.128 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:31.128 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:31.128 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:31.128 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:31.128 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:31.128 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:31.129 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T23:58:31.130 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:31.127+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:31.230 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:31.229+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:31.240 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:31.240 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:31.240 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:58:31.240 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:31.249 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T23:58:31.331 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:58:31.329+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:58:31.356 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:31.356 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:31.356 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:58:31.356 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:58:31.356 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:31.365 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T23:58:31.432 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:58:31.430+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:58:31.473 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:31.473 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:58:31.473 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:58:31.474 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:58:31.474 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.474 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:43054 3.1f 3:ffbfe596:::rbd_data.1118350d5ae5.00000000000002eb:head [set-alloc-hint object_size 262144 write_size 262144,write 196608~22770] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T23:58:31.474 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:58:31.474 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 98.504648239999995, 2024-09-14T23:58:31.474 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.99973334400000002, 2024-09-14T23:58:31.474 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:31.474 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:31.474 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:31.474 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:31.474 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:31.474 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:31.474 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 43054 2024-09-14T23:58:31.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.475 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:31.475 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.475 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:31.475 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:58:31.475 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:31.475 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.475 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.475 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:31.475 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:58:31.475 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:31.475 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.475 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.475 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:31.475 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966552+0000", 2024-09-14T23:58:31.475 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3790000000000001e-06 2024-09-14T23:58:31.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.476 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:31.476 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966565+0000", 2024-09-14T23:58:31.476 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2459999999999999e-05 2024-09-14T23:58:31.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.476 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:31.476 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966576+0000", 2024-09-14T23:58:31.476 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1039000000000001e-05 2024-09-14T23:58:31.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.476 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:31.476 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966582+0000", 2024-09-14T23:58:31.477 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9660000000000001e-06 2024-09-14T23:58:31.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.477 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.477 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:31.477 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966625+0000", 2024-09-14T23:58:31.477 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3529999999999998e-05 2024-09-14T23:58:31.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.477 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.477 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:58:31.478 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966632+0000", 2024-09-14T23:58:31.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9809999999999997e-06 2024-09-14T23:58:31.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.478 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:31.478 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.964460+0000", 2024-09-14T23:58:31.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.99782768300000002 2024-09-14T23:58:31.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.478 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:31.478 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.964641+0000", 2024-09-14T23:58:31.478 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000181089 2024-09-14T23:58:31.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.479 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-14T23:58:31.479 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.964809+0000", 2024-09-14T23:58:31.479 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000168505 2024-09-14T23:58:31.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.479 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:58:31.479 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966030+0000", 2024-09-14T23:58:31.479 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012206929999999999 2024-09-14T23:58:31.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.480 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.480 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:31.480 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966181+0000", 2024-09-14T23:58:31.480 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000150569 2024-09-14T23:58:31.480 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.480 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.480 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:31.480 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966243+0000", 2024-09-14T23:58:31.480 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2331999999999999e-05 2024-09-14T23:58:31.480 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.480 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.480 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:31.480 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966284+0000", 2024-09-14T23:58:31.480 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1118000000000002e-05 2024-09-14T23:58:31.481 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:31.481 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:31.481 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:31.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.481 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.481 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184314 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4162560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:31.481 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.077247+0000", 2024-09-14T23:58:31.481 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 24.393951571999999, 2024-09-14T23:58:31.481 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058772256000000002, 2024-09-14T23:58:31.481 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:31.481 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:31.481 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:31.481 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:31.481 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:31.482 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:31.482 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184314 2024-09-14T23:58:31.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.482 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:31.482 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.482 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:31.482 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.077247+0000", 2024-09-14T23:58:31.482 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:31.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.482 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.482 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:31.482 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.077247+0000", 2024-09-14T23:58:31.482 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:31.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.482 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.483 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:31.483 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.078906+0000", 2024-09-14T23:58:31.483 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016583279999999999 2024-09-14T23:58:31.483 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.483 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.483 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:31.483 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.078911+0000", 2024-09-14T23:58:31.483 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0420000000000002e-06 2024-09-14T23:58:31.483 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.483 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.483 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:31.483 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.078922+0000", 2024-09-14T23:58:31.484 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1354e-05 2024-09-14T23:58:31.484 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.484 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.484 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:31.484 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.078930+0000", 2024-09-14T23:58:31.484 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5730000000000002e-06 2024-09-14T23:58:31.484 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.484 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.484 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:31.484 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.135870+0000", 2024-09-14T23:58:31.484 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056940651000000002 2024-09-14T23:58:31.484 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.484 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.484 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:31.485 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.135951+0000", 2024-09-14T23:58:31.485 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0890999999999995e-05 2024-09-14T23:58:31.485 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.485 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.485 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:31.485 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.136020+0000", 2024-09-14T23:58:31.485 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8417000000000006e-05 2024-09-14T23:58:31.485 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:31.485 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:31.485 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:31.485 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.485 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.485 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184323 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4183040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:31.485 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.079228+0000", 2024-09-14T23:58:31.486 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 24.391970497999999, 2024-09-14T23:58:31.486 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058454487999999999, 2024-09-14T23:58:31.486 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:31.486 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:31.486 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:31.486 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:31.486 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:31.486 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:31.486 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184323 2024-09-14T23:58:31.486 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.486 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:31.486 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.486 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:31.486 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079228+0000", 2024-09-14T23:58:31.486 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:31.487 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.487 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.487 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:31.487 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079228+0000", 2024-09-14T23:58:31.487 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:31.487 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.487 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.487 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:31.487 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079229+0000", 2024-09-14T23:58:31.487 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5299999999999999e-07 2024-09-14T23:58:31.487 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.487 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.487 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:31.487 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079232+0000", 2024-09-14T23:58:31.488 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8839999999999998e-06 2024-09-14T23:58:31.488 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.488 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.488 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:31.488 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079238+0000", 2024-09-14T23:58:31.488 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2299999999999996e-06 2024-09-14T23:58:31.488 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.488 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.488 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:31.488 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079243+0000", 2024-09-14T23:58:31.488 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3029999999999999e-06 2024-09-14T23:58:31.488 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.488 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.489 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:31.489 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.137533+0000", 2024-09-14T23:58:31.489 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058289674999999999 2024-09-14T23:58:31.489 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.489 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.489 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:31.489 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.137615+0000", 2024-09-14T23:58:31.489 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2288999999999998e-05 2024-09-14T23:58:31.489 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.489 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.489 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:31.489 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.137683+0000", 2024-09-14T23:58:31.489 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7654e-05 2024-09-14T23:58:31.489 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:31.490 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:31.490 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:31.490 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.490 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.490 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184324 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4073472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:31.490 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.079251+0000", 2024-09-14T23:58:31.490 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 24.391948030999998, 2024-09-14T23:58:31.490 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058609383000000001, 2024-09-14T23:58:31.490 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:31.490 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:31.490 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:31.490 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:31.490 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:31.490 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:31.491 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184324 2024-09-14T23:58:31.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.491 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:31.491 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.491 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:31.491 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079251+0000", 2024-09-14T23:58:31.491 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:31.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.491 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.491 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:31.491 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079251+0000", 2024-09-14T23:58:31.491 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:31.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.491 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.491 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:31.492 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.080920+0000", 2024-09-14T23:58:31.492 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016687430000000001 2024-09-14T23:58:31.492 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.492 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.492 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:31.492 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.080924+0000", 2024-09-14T23:58:31.492 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3869999999999997e-06 2024-09-14T23:58:31.492 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.492 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.492 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:31.492 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.080935+0000", 2024-09-14T23:58:31.492 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0515e-05 2024-09-14T23:58:31.492 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.492 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.493 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:31.493 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.080942+0000", 2024-09-14T23:58:31.493 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0990000000000001e-06 2024-09-14T23:58:31.493 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.493 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.493 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:31.493 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.137713+0000", 2024-09-14T23:58:31.493 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056771288000000003 2024-09-14T23:58:31.493 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.493 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.493 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:31.493 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.137794+0000", 2024-09-14T23:58:31.493 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0613999999999998e-05 2024-09-14T23:58:31.493 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.494 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.494 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:31.494 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.137860+0000", 2024-09-14T23:58:31.494 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6736999999999995e-05 2024-09-14T23:58:31.494 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:31.494 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:31.494 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:31.494 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.494 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.494 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184346 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4099072~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:31.494 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.083616+0000", 2024-09-14T23:58:31.494 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 24.387582846000001, 2024-09-14T23:58:31.494 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058557419999999999, 2024-09-14T23:58:31.494 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:31.494 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:31.495 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:31.495 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:31.495 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:31.495 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:31.495 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184346 2024-09-14T23:58:31.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.495 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:31.495 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.495 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:31.495 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083616+0000", 2024-09-14T23:58:31.495 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:31.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.495 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.495 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:31.495 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083616+0000", 2024-09-14T23:58:31.496 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:31.496 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.496 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.496 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:31.496 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083620+0000", 2024-09-14T23:58:31.496 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0119999999999999e-06 2024-09-14T23:58:31.496 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.496 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.496 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:31.496 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083623+0000", 2024-09-14T23:58:31.496 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9059999999999998e-06 2024-09-14T23:58:31.496 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.496 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.497 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:31.497 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083631+0000", 2024-09-14T23:58:31.497 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2679999999999994e-06 2024-09-14T23:58:31.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.497 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.497 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:31.497 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083637+0000", 2024-09-14T23:58:31.497 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1210000000000003e-06 2024-09-14T23:58:31.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.497 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.497 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:31.497 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142018+0000", 2024-09-14T23:58:31.498 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058381111999999999 2024-09-14T23:58:31.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.498 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:31.498 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142100+0000", 2024-09-14T23:58:31.498 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1308000000000001e-05 2024-09-14T23:58:31.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.498 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:31.498 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142173+0000", 2024-09-14T23:58:31.498 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3693000000000001e-05 2024-09-14T23:58:31.498 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:31.498 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:31.498 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:31.499 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.499 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.499 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184347 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4122624~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:31.499 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.083647+0000", 2024-09-14T23:58:31.499 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 24.387552016000001, 2024-09-14T23:58:31.499 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058706388999999998, 2024-09-14T23:58:31.499 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:31.499 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:31.499 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:31.499 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:31.499 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:31.500 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:31.500 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184347 2024-09-14T23:58:31.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.500 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:31.500 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.500 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:31.500 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083647+0000", 2024-09-14T23:58:31.500 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:31.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.500 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.500 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:31.500 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083647+0000", 2024-09-14T23:58:31.500 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:31.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.501 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.501 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:31.501 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083647+0000", 2024-09-14T23:58:31.501 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3099999999999998e-07 2024-09-14T23:58:31.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.501 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.501 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:31.501 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083650+0000", 2024-09-14T23:58:31.501 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2900000000000001e-06 2024-09-14T23:58:31.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.502 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.502 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:31.502 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083657+0000", 2024-09-14T23:58:31.502 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.486e-06 2024-09-14T23:58:31.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.502 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.502 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:31.502 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083663+0000", 2024-09-14T23:58:31.502 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4330000000000003e-06 2024-09-14T23:58:31.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.502 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.502 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:31.502 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142206+0000", 2024-09-14T23:58:31.502 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058543237999999997 2024-09-14T23:58:31.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.503 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.503 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:31.503 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142284+0000", 2024-09-14T23:58:31.503 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8487999999999998e-05 2024-09-14T23:58:31.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.503 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.503 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:31.503 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142353+0000", 2024-09-14T23:58:31.503 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9023000000000006e-05 2024-09-14T23:58:31.503 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:31.503 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:31.503 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:31.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.504 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.504 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184348 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4103168~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:31.504 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.083669+0000", 2024-09-14T23:58:31.504 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 24.387529599000001, 2024-09-14T23:58:31.504 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058864232000000002, 2024-09-14T23:58:31.504 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:31.504 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:31.504 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:31.504 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:31.504 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:31.504 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:31.504 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184348 2024-09-14T23:58:31.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.504 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:31.504 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.505 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:31.505 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083669+0000", 2024-09-14T23:58:31.505 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:31.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.505 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.505 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:31.505 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083669+0000", 2024-09-14T23:58:31.505 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:31.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.505 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.505 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:31.505 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083670+0000", 2024-09-14T23:58:31.505 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9999999999999998e-07 2024-09-14T23:58:31.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.506 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:31.506 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083672+0000", 2024-09-14T23:58:31.506 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1430000000000001e-06 2024-09-14T23:58:31.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.506 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:31.506 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083678+0000", 2024-09-14T23:58:31.506 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5200000000000003e-06 2024-09-14T23:58:31.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.506 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:31.506 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083683+0000", 2024-09-14T23:58:31.506 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9359999999999998e-06 2024-09-14T23:58:31.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.507 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.507 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:31.507 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142384+0000", 2024-09-14T23:58:31.507 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058701159000000003 2024-09-14T23:58:31.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.507 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.507 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:31.507 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142466+0000", 2024-09-14T23:58:31.507 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1024000000000002e-05 2024-09-14T23:58:31.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.508 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.508 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:31.508 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142534+0000", 2024-09-14T23:58:31.508 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8050000000000001e-05 2024-09-14T23:58:31.508 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:31.508 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:31.508 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:31.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.508 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.508 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184349 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4126720~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:31.508 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.083690+0000", 2024-09-14T23:58:31.508 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 24.387509064, 2024-09-14T23:58:31.509 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059019911000000001, 2024-09-14T23:58:31.509 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:31.509 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:31.509 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:31.509 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:31.509 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:31.509 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:31.509 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184349 2024-09-14T23:58:31.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.509 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:31.509 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.509 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:31.509 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083690+0000", 2024-09-14T23:58:31.509 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:31.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.510 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:31.510 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083690+0000", 2024-09-14T23:58:31.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:31.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.510 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:31.510 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.084900+0000", 2024-09-14T23:58:31.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012099770000000001 2024-09-14T23:58:31.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.510 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:31.510 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.084905+0000", 2024-09-14T23:58:31.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.028e-06 2024-09-14T23:58:31.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.511 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:31.511 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.084914+0000", 2024-09-14T23:58:31.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6619999999999994e-06 2024-09-14T23:58:31.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.511 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:31.511 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.084922+0000", 2024-09-14T23:58:31.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5720000000000003e-06 2024-09-14T23:58:31.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.512 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:31.512 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142565+0000", 2024-09-14T23:58:31.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057642605999999999 2024-09-14T23:58:31.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.512 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:31.512 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142644+0000", 2024-09-14T23:58:31.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9370999999999996e-05 2024-09-14T23:58:31.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.512 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:31.512 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142710+0000", 2024-09-14T23:58:31.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5695e-05 2024-09-14T23:58:31.512 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:31.513 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:31.513 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:31.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.513 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184354 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4115456~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:31.513 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.085029+0000", 2024-09-14T23:58:31.513 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 24.386170413999999, 2024-09-14T23:58:31.513 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058626327999999998, 2024-09-14T23:58:31.513 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:31.513 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:31.513 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:31.513 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:31.513 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:31.513 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:31.513 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184354 2024-09-14T23:58:31.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.514 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:31.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:31.514 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085029+0000", 2024-09-14T23:58:31.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:31.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:31.514 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085029+0000", 2024-09-14T23:58:31.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:31.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:31.515 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085029+0000", 2024-09-14T23:58:31.515 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0800000000000001e-07 2024-09-14T23:58:31.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.515 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:31.515 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085030+0000", 2024-09-14T23:58:31.515 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.226e-06 2024-09-14T23:58:31.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.515 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:31.515 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085033+0000", 2024-09-14T23:58:31.515 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9569999999999998e-06 2024-09-14T23:58:31.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.515 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:31.516 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085035+0000", 2024-09-14T23:58:31.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.587e-06 2024-09-14T23:58:31.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.516 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:31.516 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143491+0000", 2024-09-14T23:58:31.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058455337000000003 2024-09-14T23:58:31.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.516 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:31.516 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143575+0000", 2024-09-14T23:58:31.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3850000000000005e-05 2024-09-14T23:58:31.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.517 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:31.517 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143655+0000", 2024-09-14T23:58:31.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0162999999999998e-05 2024-09-14T23:58:31.517 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:31.517 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:31.517 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:31.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.517 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184355 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4139008~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:31.517 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.085039+0000", 2024-09-14T23:58:31.517 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 24.386160010000001, 2024-09-14T23:58:31.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058801369999999999, 2024-09-14T23:58:31.518 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:31.518 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:31.518 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:31.518 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:31.518 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:31.518 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:31.518 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184355 2024-09-14T23:58:31.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.518 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:31.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.518 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:31.518 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085039+0000", 2024-09-14T23:58:31.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:31.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.519 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:31.519 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085039+0000", 2024-09-14T23:58:31.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:31.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.519 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:31.519 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085039+0000", 2024-09-14T23:58:31.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.03e-07 2024-09-14T23:58:31.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.519 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:31.519 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085040+0000", 2024-09-14T23:58:31.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.158e-06 2024-09-14T23:58:31.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.520 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:31.520 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085043+0000", 2024-09-14T23:58:31.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0240000000000002e-06 2024-09-14T23:58:31.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.520 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:31.520 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085046+0000", 2024-09-14T23:58:31.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4600000000000002e-06 2024-09-14T23:58:31.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.520 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:31.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143688+0000", 2024-09-14T23:58:31.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058642356 2024-09-14T23:58:31.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.521 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:31.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143766+0000", 2024-09-14T23:58:31.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8219000000000004e-05 2024-09-14T23:58:31.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.521 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:31.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143840+0000", 2024-09-14T23:58:31.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3949999999999995e-05 2024-09-14T23:58:31.521 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:31.521 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:31.522 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:31.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.522 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184356 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4119552~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:31.522 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.085049+0000", 2024-09-14T23:58:31.522 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 24.386149839000002, 2024-09-14T23:58:31.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058977992, 2024-09-14T23:58:31.522 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:31.522 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:31.522 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:31.522 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:31.522 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:31.522 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:31.522 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184356 2024-09-14T23:58:31.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.522 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:31.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:31.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085049+0000", 2024-09-14T23:58:31.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:31.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:31.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085049+0000", 2024-09-14T23:58:31.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:31.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:31.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.086934+0000", 2024-09-14T23:58:31.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018854189999999999 2024-09-14T23:58:31.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:31.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.086939+0000", 2024-09-14T23:58:31.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.493e-06 2024-09-14T23:58:31.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:31.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.086951+0000", 2024-09-14T23:58:31.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1969e-05 2024-09-14T23:58:31.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:31.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.086959+0000", 2024-09-14T23:58:31.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5780000000000001e-06 2024-09-14T23:58:31.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:31.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143871+0000", 2024-09-14T23:58:31.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056912114 2024-09-14T23:58:31.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:31.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143951+0000", 2024-09-14T23:58:31.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9938000000000006e-05 2024-09-14T23:58:31.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.526 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:31.526 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.144027+0000", 2024-09-14T23:58:31.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6481000000000002e-05 2024-09-14T23:58:31.526 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:31.526 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:31.526 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:31.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.526 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184364 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4135936~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:31.526 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.087207+0000", 2024-09-14T23:58:31.526 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 24.383991558999998, 2024-09-14T23:58:31.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058466420999999998, 2024-09-14T23:58:31.526 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:31.527 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:31.527 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:31.527 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:31.527 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:31.527 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:31.527 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184364 2024-09-14T23:58:31.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.527 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:31.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:31.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087207+0000", 2024-09-14T23:58:31.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:31.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:31.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087207+0000", 2024-09-14T23:58:31.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:31.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:31.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087208+0000", 2024-09-14T23:58:31.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.35e-07 2024-09-14T23:58:31.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:31.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087209+0000", 2024-09-14T23:58:31.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.573e-06 2024-09-14T23:58:31.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:31.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087212+0000", 2024-09-14T23:58:31.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2770000000000001e-06 2024-09-14T23:58:31.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:31.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087215+0000", 2024-09-14T23:58:31.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5500000000000001e-06 2024-09-14T23:58:31.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:31.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145520+0000", 2024-09-14T23:58:31.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058304835999999999 2024-09-14T23:58:31.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:31.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145602+0000", 2024-09-14T23:58:31.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1714999999999999e-05 2024-09-14T23:58:31.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:31.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145674+0000", 2024-09-14T23:58:31.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2235000000000002e-05 2024-09-14T23:58:31.530 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:31.530 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:31.530 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:31.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.531 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184365 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4159488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:31.531 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.087219+0000", 2024-09-14T23:58:31.531 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 24.383979528000001, 2024-09-14T23:58:31.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058643533999999997, 2024-09-14T23:58:31.531 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:31.531 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:31.531 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:31.531 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:31.531 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:31.531 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:31.531 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184365 2024-09-14T23:58:31.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.532 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:31.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:31.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087219+0000", 2024-09-14T23:58:31.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:31.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:31.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087219+0000", 2024-09-14T23:58:31.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:31.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:31.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087220+0000", 2024-09-14T23:58:31.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.39e-07 2024-09-14T23:58:31.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:31.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087222+0000", 2024-09-14T23:58:31.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0609999999999998e-06 2024-09-14T23:58:31.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:31.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087228+0000", 2024-09-14T23:58:31.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8590000000000001e-06 2024-09-14T23:58:31.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:31.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087232+0000", 2024-09-14T23:58:31.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8149999999999996e-06 2024-09-14T23:58:31.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:31.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145705+0000", 2024-09-14T23:58:31.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058472532000000001 2024-09-14T23:58:31.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:31.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145784+0000", 2024-09-14T23:58:31.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9382000000000007e-05 2024-09-14T23:58:31.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:31.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145863+0000", 2024-09-14T23:58:31.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8546000000000003e-05 2024-09-14T23:58:31.535 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:31.535 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:31.535 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:31.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.535 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184366 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4140032~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:31.535 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.087240+0000", 2024-09-14T23:58:31.535 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 24.38395908, 2024-09-14T23:58:31.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058818068000000001, 2024-09-14T23:58:31.536 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:31.536 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:31.536 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:31.536 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:31.536 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:31.536 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:31.536 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184366 2024-09-14T23:58:31.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.536 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:31.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:31.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087240+0000", 2024-09-14T23:58:31.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:31.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:31.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087240+0000", 2024-09-14T23:58:31.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:31.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:31.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.088904+0000", 2024-09-14T23:58:31.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001664315 2024-09-14T23:58:31.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:31.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.088909+0000", 2024-09-14T23:58:31.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7709999999999997e-06 2024-09-14T23:58:31.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:31.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.088921+0000", 2024-09-14T23:58:31.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1661999999999999e-05 2024-09-14T23:58:31.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:31.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.088928+0000", 2024-09-14T23:58:31.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8189999999999992e-06 2024-09-14T23:58:31.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:31.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145901+0000", 2024-09-14T23:58:31.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056972478 2024-09-14T23:58:31.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:31.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145977+0000", 2024-09-14T23:58:31.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5763e-05 2024-09-14T23:58:31.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:31.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.146058+0000", 2024-09-14T23:58:31.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1260000000000005e-05 2024-09-14T23:58:31.539 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:31.539 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:31.540 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:31.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.540 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184375 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4179968~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:31.540 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.089199+0000", 2024-09-14T23:58:31.540 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 24.381999743000002, 2024-09-14T23:58:31.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058597445999999997, 2024-09-14T23:58:31.540 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:31.540 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:31.540 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:31.540 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:31.540 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:31.540 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:31.540 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184375 2024-09-14T23:58:31.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.541 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:31.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:31.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089199+0000", 2024-09-14T23:58:31.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:31.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:31.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089199+0000", 2024-09-14T23:58:31.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:31.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:31.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089200+0000", 2024-09-14T23:58:31.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8100000000000003e-07 2024-09-14T23:58:31.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:31.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089202+0000", 2024-09-14T23:58:31.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7599999999999998e-06 2024-09-14T23:58:31.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:31.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089209+0000", 2024-09-14T23:58:31.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7279999999999998e-06 2024-09-14T23:58:31.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:31.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089215+0000", 2024-09-14T23:58:31.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5090000000000001e-06 2024-09-14T23:58:31.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:31.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147638+0000", 2024-09-14T23:58:31.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058423105000000003 2024-09-14T23:58:31.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:31.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147719+0000", 2024-09-14T23:58:31.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1217000000000005e-05 2024-09-14T23:58:31.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:31.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147797+0000", 2024-09-14T23:58:31.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7645999999999995e-05 2024-09-14T23:58:31.544 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:31.544 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:31.544 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:31.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.544 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184376 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4160512~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:31.544 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.089232+0000", 2024-09-14T23:58:31.544 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 24.381967252999999, 2024-09-14T23:58:31.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058751338, 2024-09-14T23:58:31.544 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:31.544 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:31.544 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:31.544 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:31.545 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:31.545 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:31.545 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184376 2024-09-14T23:58:31.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.545 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:31.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:31.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089232+0000", 2024-09-14T23:58:31.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:31.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:31.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089232+0000", 2024-09-14T23:58:31.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:31.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:31.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089232+0000", 2024-09-14T23:58:31.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5600000000000001e-07 2024-09-14T23:58:31.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:31.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089235+0000", 2024-09-14T23:58:31.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5639999999999999e-06 2024-09-14T23:58:31.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:31.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089242+0000", 2024-09-14T23:58:31.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0369999999999997e-06 2024-09-14T23:58:31.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:31.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089247+0000", 2024-09-14T23:58:31.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2079999999999999e-06 2024-09-14T23:58:31.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:31.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147831+0000", 2024-09-14T23:58:31.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058583853999999998 2024-09-14T23:58:31.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:31.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147915+0000", 2024-09-14T23:58:31.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4208000000000004e-05 2024-09-14T23:58:31.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:31.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147983+0000", 2024-09-14T23:58:31.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8010999999999996e-05 2024-09-14T23:58:31.548 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:31.548 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:31.548 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:31.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.548 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184377 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4184064~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:31.549 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.089255+0000", 2024-09-14T23:58:31.549 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 24.381943697000001, 2024-09-14T23:58:31.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058906100000000003, 2024-09-14T23:58:31.549 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:31.549 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:31.549 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:31.549 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:31.549 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:31.549 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:31.549 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184377 2024-09-14T23:58:31.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.549 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:31.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:31.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089255+0000", 2024-09-14T23:58:31.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:31.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:31.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089255+0000", 2024-09-14T23:58:31.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:31.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:31.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.090903+0000", 2024-09-14T23:58:31.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001647871 2024-09-14T23:58:31.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:31.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.090908+0000", 2024-09-14T23:58:31.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7199999999999997e-06 2024-09-14T23:58:31.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:31.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.090919+0000", 2024-09-14T23:58:31.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1472e-05 2024-09-14T23:58:31.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:31.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.090927+0000", 2024-09-14T23:58:31.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0390000000000005e-06 2024-09-14T23:58:31.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:31.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.148013+0000", 2024-09-14T23:58:31.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057085662000000002 2024-09-14T23:58:31.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:31.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.148090+0000", 2024-09-14T23:58:31.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6778000000000002e-05 2024-09-14T23:58:31.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:31.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.148161+0000", 2024-09-14T23:58:31.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1557999999999995e-05 2024-09-14T23:58:31.553 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:31.553 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:31.553 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:31.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.553 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184386 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4185088~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:31.553 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.091063+0000", 2024-09-14T23:58:31.553 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 24.380136032999999, 2024-09-14T23:58:31.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058603404999999997, 2024-09-14T23:58:31.553 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:31.553 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:31.553 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:31.553 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:31.553 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:31.554 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:31.554 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184386 2024-09-14T23:58:31.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.554 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:31.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:31.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091063+0000", 2024-09-14T23:58:31.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:31.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:31.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091063+0000", 2024-09-14T23:58:31.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:31.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:31.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091063+0000", 2024-09-14T23:58:31.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1899999999999999e-07 2024-09-14T23:58:31.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:31.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091064+0000", 2024-09-14T23:58:31.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2699999999999999e-06 2024-09-14T23:58:31.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:31.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091067+0000", 2024-09-14T23:58:31.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.971e-06 2024-09-14T23:58:31.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:31.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091070+0000", 2024-09-14T23:58:31.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.525e-06 2024-09-14T23:58:31.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:31.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.149510+0000", 2024-09-14T23:58:31.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058439758000000001 2024-09-14T23:58:31.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:31.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.149595+0000", 2024-09-14T23:58:31.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5146000000000001e-05 2024-09-14T23:58:31.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:31.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.149666+0000", 2024-09-14T23:58:31.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1515999999999998e-05 2024-09-14T23:58:31.557 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:31.557 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:31.557 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:31.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.557 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184388 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4189184~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:31.557 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.091073+0000", 2024-09-14T23:58:31.557 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 24.380125757999998, 2024-09-14T23:58:31.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058781609999999998, 2024-09-14T23:58:31.557 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:31.558 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:31.558 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:31.558 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:31.558 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:31.558 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:31.558 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184388 2024-09-14T23:58:31.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.558 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:31.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:31.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091073+0000", 2024-09-14T23:58:31.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:31.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:31.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091073+0000", 2024-09-14T23:58:31.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:31.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:31.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.092903+0000", 2024-09-14T23:58:31.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018296289999999999 2024-09-14T23:58:31.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:31.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.092910+0000", 2024-09-14T23:58:31.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7510000000000002e-06 2024-09-14T23:58:31.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:31.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.092918+0000", 2024-09-14T23:58:31.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7420000000000003e-06 2024-09-14T23:58:31.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:31.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.092925+0000", 2024-09-14T23:58:31.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0360000000000001e-06 2024-09-14T23:58:31.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:31.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.149700+0000", 2024-09-14T23:58:31.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056774340999999999 2024-09-14T23:58:31.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:31.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.149784+0000", 2024-09-14T23:58:31.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4393000000000003e-05 2024-09-14T23:58:31.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:31.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.149855+0000", 2024-09-14T23:58:31.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0717999999999996e-05 2024-09-14T23:58:31.561 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:31.561 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:31.561 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:31.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.562 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4172.0:8 3.12 3:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-14T23:58:31.562 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.976999+0000", 2024-09-14T23:58:31.562 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.494200349, 2024-09-14T23:58:31.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0046036439999999, 2024-09-14T23:58:31.562 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:31.562 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:31.562 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:31.562 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:31.562 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4172", 2024-09-14T23:58:31.562 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/2542269203", 2024-09-14T23:58:31.562 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 8 2024-09-14T23:58:31.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.562 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:31.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:31.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.976999+0000", 2024-09-14T23:58:31.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:31.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:31.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.976999+0000", 2024-09-14T23:58:31.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:31.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:31.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.977000+0000", 2024-09-14T23:58:31.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5200000000000001e-06 2024-09-14T23:58:31.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:31.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.977005+0000", 2024-09-14T23:58:31.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.04e-06 2024-09-14T23:58:31.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:31.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.977013+0000", 2024-09-14T23:58:31.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2889999999999998e-06 2024-09-14T23:58:31.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:31.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.977024+0000", 2024-09-14T23:58:31.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0307e-05 2024-09-14T23:58:31.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:31.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.977086+0000", 2024-09-14T23:58:31.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2248000000000004e-05 2024-09-14T23:58:31.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:58:31.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.977097+0000", 2024-09-14T23:58:31.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1123e-05 2024-09-14T23:58:31.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:31.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:08.981470+0000", 2024-09-14T23:58:31.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.004373116 2024-09-14T23:58:31.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:31.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:08.981602+0000", 2024-09-14T23:58:31.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000132001 2024-09-14T23:58:31.566 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:31.566 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:31.566 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:31.566 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:31.566 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:31.566 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:31.568 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T23:58:31.568 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:31.566+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:31.668 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:31.667+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:31.715 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:31.715 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:31.715 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:58:31.716 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:31.724 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T23:58:31.769 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:31.768+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:31.870 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:31.869+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:31.871 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:31.871 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:31.871 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:58:31.871 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:58:31.871 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:31.880 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T23:58:31.971 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:31.969+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:31.977 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is set 2024-09-14T23:58:31.995 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:31.995 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:58:31.995 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:58:31.995 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:58:31.995 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.995 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5882 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:31.995 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:58:31.995 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.616862166, 2024-09-14T23:58:31.995 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050549666, 2024-09-14T23:58:31.995 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:31.995 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:31.996 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:31.996 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:31.996 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:31.996 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:31.996 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5882 2024-09-14T23:58:31.996 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.996 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:31.996 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.996 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:31.996 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:58:31.996 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:31.996 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.996 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.997 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:31.997 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:58:31.997 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:31.997 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.997 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.997 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:31.997 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:58:31.997 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8099999999999998e-07 2024-09-14T23:58:31.997 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.997 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.997 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:31.997 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375842+0000", 2024-09-14T23:58:31.997 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9900000000000009e-07 2024-09-14T23:58:31.998 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.998 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.998 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:31.998 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375845+0000", 2024-09-14T23:58:31.998 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.055e-06 2024-09-14T23:58:31.998 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.998 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.998 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:31.998 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375848+0000", 2024-09-14T23:58:31.998 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7609999999999999e-06 2024-09-14T23:58:31.998 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.998 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.998 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:31.998 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426381+0000", 2024-09-14T23:58:31.999 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050533392000000003 2024-09-14T23:58:31.999 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.999 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.999 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:31.999 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426390+0000", 2024-09-14T23:58:31.999 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0780000000000002e-06 2024-09-14T23:58:31.999 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:31.999 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:31.999 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:31.999 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:31.999 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:31.999 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5884 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:31.999 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:58:31.999 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.616852026, 2024-09-14T23:58:32.000 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050717439000000003, 2024-09-14T23:58:32.000 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:32.000 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:32.000 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:32.000 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:32.000 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:32.000 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:32.000 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5884 2024-09-14T23:58:32.000 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.000 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:32.000 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.000 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:32.000 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:58:32.001 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.001 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.001 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.001 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:32.001 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:58:32.001 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.001 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.001 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.001 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:32.001 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:58:32.001 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0800000000000001e-07 2024-09-14T23:58:32.001 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.001 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.001 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:32.001 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375852+0000", 2024-09-14T23:58:32.002 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.02e-06 2024-09-14T23:58:32.002 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.002 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.002 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:32.002 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375855+0000", 2024-09-14T23:58:32.002 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9900000000000002e-06 2024-09-14T23:58:32.002 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.002 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.002 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:32.002 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375857+0000", 2024-09-14T23:58:32.002 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3499999999999999e-06 2024-09-14T23:58:32.002 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.002 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.002 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:32.003 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426556+0000", 2024-09-14T23:58:32.003 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050698139000000003 2024-09-14T23:58:32.003 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.003 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.003 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:32.003 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426568+0000", 2024-09-14T23:58:32.003 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2731999999999999e-05 2024-09-14T23:58:32.003 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.003 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:32.003 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.003 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.003 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.003 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5885 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:32.003 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:58:32.003 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.616833001, 2024-09-14T23:58:32.004 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050737333000000003, 2024-09-14T23:58:32.004 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:32.004 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:32.004 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:32.004 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:32.004 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:32.004 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:32.004 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5885 2024-09-14T23:58:32.004 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.004 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:32.004 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.004 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:32.004 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:58:32.004 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.005 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.005 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.005 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:32.005 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:58:32.005 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.005 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.005 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.005 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:32.005 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:58:32.005 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5899999999999998e-07 2024-09-14T23:58:32.005 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.005 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.005 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:32.006 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375872+0000", 2024-09-14T23:58:32.006 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4219999999999999e-06 2024-09-14T23:58:32.006 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.006 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.006 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:32.006 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375878+0000", 2024-09-14T23:58:32.006 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4980000000000003e-06 2024-09-14T23:58:32.006 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.006 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.006 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:32.006 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375881+0000", 2024-09-14T23:58:32.006 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6110000000000001e-06 2024-09-14T23:58:32.006 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.006 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.006 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:32.007 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426596+0000", 2024-09-14T23:58:32.007 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050715320000000001 2024-09-14T23:58:32.007 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.007 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.007 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:32.007 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426607+0000", 2024-09-14T23:58:32.007 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1223000000000001e-05 2024-09-14T23:58:32.007 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.007 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:32.007 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.007 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.007 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.007 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5886 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:32.007 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:58:32.008 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.616818888, 2024-09-14T23:58:32.008 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050803395000000001, 2024-09-14T23:58:32.008 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:32.008 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:32.008 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:32.008 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:32.008 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:32.008 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:32.008 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5886 2024-09-14T23:58:32.008 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.008 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:32.008 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.008 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:32.008 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:58:32.009 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.009 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.009 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.009 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:32.009 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:58:32.009 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.009 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.009 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.009 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:32.009 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:58:32.009 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2700000000000001e-07 2024-09-14T23:58:32.009 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.009 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.009 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:32.010 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375886+0000", 2024-09-14T23:58:32.010 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7060000000000001e-06 2024-09-14T23:58:32.010 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.010 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.010 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:32.010 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375890+0000", 2024-09-14T23:58:32.010 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2259999999999999e-06 2024-09-14T23:58:32.010 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.010 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.010 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:32.010 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375893+0000", 2024-09-14T23:58:32.010 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0819999999999999e-06 2024-09-14T23:58:32.010 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.010 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.011 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:32.011 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426677+0000", 2024-09-14T23:58:32.011 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050784158000000003 2024-09-14T23:58:32.011 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.011 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.011 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:32.011 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426687+0000", 2024-09-14T23:58:32.011 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9960000000000005e-06 2024-09-14T23:58:32.011 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.011 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:32.011 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.011 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.011 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.011 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5887 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:32.012 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:58:32.012 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.616796802, 2024-09-14T23:58:32.012 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050833070000000001, 2024-09-14T23:58:32.012 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:32.012 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:32.012 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:32.012 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:32.012 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:32.012 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:32.012 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5887 2024-09-14T23:58:32.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.012 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:32.012 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.012 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:32.013 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:58:32.013 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.013 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.013 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.013 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:32.013 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:58:32.013 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.013 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.013 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.013 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:32.013 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:58:32.013 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3299999999999998e-07 2024-09-14T23:58:32.013 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.013 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.014 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:32.014 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375908+0000", 2024-09-14T23:58:32.014 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7689999999999999e-06 2024-09-14T23:58:32.014 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.014 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.014 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:32.014 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375912+0000", 2024-09-14T23:58:32.014 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.535e-06 2024-09-14T23:58:32.014 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.014 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.014 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:32.015 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375916+0000", 2024-09-14T23:58:32.015 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8979999999999998e-06 2024-09-14T23:58:32.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.015 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.015 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:32.015 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426729+0000", 2024-09-14T23:58:32.015 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050813453000000001 2024-09-14T23:58:32.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.015 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.015 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:32.015 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426739+0000", 2024-09-14T23:58:32.015 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0081999999999999e-05 2024-09-14T23:58:32.015 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.016 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:32.016 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.016 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.016 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.016 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5888 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:32.016 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:58:32.016 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.616783725, 2024-09-14T23:58:32.016 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050854652, 2024-09-14T23:58:32.016 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:32.016 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:32.017 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:32.017 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:32.017 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:32.017 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:32.017 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5888 2024-09-14T23:58:32.017 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.017 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:32.017 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.017 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:32.017 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:58:32.017 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.017 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.017 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.017 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:32.018 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:58:32.018 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.018 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.018 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:32.018 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:58:32.018 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1400000000000001e-07 2024-09-14T23:58:32.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.018 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.018 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:32.018 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375921+0000", 2024-09-14T23:58:32.018 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.119e-06 2024-09-14T23:58:32.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.018 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.018 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:32.019 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375928+0000", 2024-09-14T23:58:32.019 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3170000000000004e-06 2024-09-14T23:58:32.019 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.019 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.019 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:32.019 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375931+0000", 2024-09-14T23:58:32.019 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6970000000000002e-06 2024-09-14T23:58:32.019 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.019 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.019 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:32.019 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426768+0000", 2024-09-14T23:58:32.019 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050837355000000001 2024-09-14T23:58:32.019 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.019 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.020 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:32.020 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426774+0000", 2024-09-14T23:58:32.020 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9499999999999998e-06 2024-09-14T23:58:32.020 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.020 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:32.020 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.020 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.020 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.020 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5889 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:32.020 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:58:32.020 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.616766801, 2024-09-14T23:58:32.020 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050885446000000001, 2024-09-14T23:58:32.020 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:32.020 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:32.021 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:32.021 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:32.021 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:32.021 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:32.021 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5889 2024-09-14T23:58:32.021 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.021 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:32.021 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.021 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:32.021 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:58:32.021 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.021 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.021 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.021 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:32.022 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:58:32.022 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.022 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.022 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.022 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:32.022 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:58:32.022 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.36e-07 2024-09-14T23:58:32.022 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.022 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.022 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:32.022 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375940+0000", 2024-09-14T23:58:32.022 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.844e-06 2024-09-14T23:58:32.022 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.022 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.023 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:32.023 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375944+0000", 2024-09-14T23:58:32.023 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6720000000000002e-06 2024-09-14T23:58:32.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.023 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.023 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:32.023 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375946+0000", 2024-09-14T23:58:32.023 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5629999999999999e-06 2024-09-14T23:58:32.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.023 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.023 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:32.023 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426810+0000", 2024-09-14T23:58:32.023 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050863255000000003 2024-09-14T23:58:32.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.024 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.024 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:32.024 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426822+0000", 2024-09-14T23:58:32.024 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1875999999999999e-05 2024-09-14T23:58:32.024 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.024 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:32.024 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.024 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.024 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.024 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5890 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:32.024 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:58:32.024 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.61675298900001, 2024-09-14T23:58:32.024 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050917549999999999, 2024-09-14T23:58:32.025 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:32.025 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:32.025 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:32.025 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:32.025 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:32.025 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:32.025 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5890 2024-09-14T23:58:32.025 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.025 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:32.025 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.025 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:32.025 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:58:32.025 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.025 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.026 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.026 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:32.026 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:58:32.026 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.026 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.026 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.026 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:32.026 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:58:32.026 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4299999999999999e-07 2024-09-14T23:58:32.026 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.026 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.026 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:32.026 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375951+0000", 2024-09-14T23:58:32.026 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1000000000000001e-06 2024-09-14T23:58:32.027 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.027 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.027 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:32.027 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375956+0000", 2024-09-14T23:58:32.027 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3370000000000001e-06 2024-09-14T23:58:32.027 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.027 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.027 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:32.027 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375958+0000", 2024-09-14T23:58:32.027 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4820000000000001e-06 2024-09-14T23:58:32.027 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.027 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.027 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:32.027 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426858+0000", 2024-09-14T23:58:32.028 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050899666000000003 2024-09-14T23:58:32.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.028 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.028 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:32.028 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426867+0000", 2024-09-14T23:58:32.028 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7219999999999994e-06 2024-09-14T23:58:32.028 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.028 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:32.028 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.028 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.028 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5891 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:32.029 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:58:32.029 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.616741667, 2024-09-14T23:58:32.029 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050948067, 2024-09-14T23:58:32.029 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:32.029 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:32.029 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:32.029 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:32.029 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:32.029 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:32.029 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5891 2024-09-14T23:58:32.029 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.029 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:32.029 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.029 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:32.030 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:58:32.030 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.030 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.030 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.030 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:32.030 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:58:32.030 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.030 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.030 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.030 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:32.030 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:58:32.030 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0800000000000001e-07 2024-09-14T23:58:32.030 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.030 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.031 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:32.031 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375962+0000", 2024-09-14T23:58:32.031 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9600000000000008e-07 2024-09-14T23:58:32.031 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.031 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.031 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:32.031 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375970+0000", 2024-09-14T23:58:32.031 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1600000000000001e-06 2024-09-14T23:58:32.031 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.031 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.031 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:32.031 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375972+0000", 2024-09-14T23:58:32.031 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4849999999999999e-06 2024-09-14T23:58:32.031 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.032 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.032 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:32.032 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426902+0000", 2024-09-14T23:58:32.032 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050929813999999997 2024-09-14T23:58:32.032 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.032 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.032 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:32.032 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426909+0000", 2024-09-14T23:58:32.032 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4039999999999998e-06 2024-09-14T23:58:32.032 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.032 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:32.032 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.032 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.032 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.033 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5892 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:32.033 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:58:32.033 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.61672681, 2024-09-14T23:58:32.033 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050965036999999998, 2024-09-14T23:58:32.033 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:32.033 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:32.033 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:32.033 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:32.033 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:32.033 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:32.033 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5892 2024-09-14T23:58:32.033 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.033 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:32.034 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.034 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:32.034 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:58:32.034 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.034 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.034 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:32.034 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:58:32.034 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.034 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.034 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:32.034 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:58:32.034 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.34e-07 2024-09-14T23:58:32.035 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.035 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.035 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:32.035 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375978+0000", 2024-09-14T23:58:32.035 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1880000000000001e-06 2024-09-14T23:58:32.035 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.035 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.035 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:32.035 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375981+0000", 2024-09-14T23:58:32.035 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2399999999999999e-06 2024-09-14T23:58:32.035 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.035 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.035 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:32.035 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375984+0000", 2024-09-14T23:58:32.035 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9110000000000002e-06 2024-09-14T23:58:32.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.036 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.036 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:32.036 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426934+0000", 2024-09-14T23:58:32.036 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050949912 2024-09-14T23:58:32.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.036 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.036 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:32.036 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426941+0000", 2024-09-14T23:58:32.036 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5519999999999999e-06 2024-09-14T23:58:32.036 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.036 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:32.036 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.036 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.036 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5893 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:32.037 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:58:32.037 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.616716213, 2024-09-14T23:58:32.037 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050975131999999999, 2024-09-14T23:58:32.037 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:32.037 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:32.037 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:32.037 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:32.037 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:32.037 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:32.037 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5893 2024-09-14T23:58:32.037 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.037 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:32.037 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.037 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:32.038 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:58:32.038 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.038 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.038 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.038 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:32.038 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:58:32.038 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.038 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.038 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.038 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:32.038 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:58:32.038 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2999999999999999e-07 2024-09-14T23:58:32.038 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.038 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.038 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:32.038 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375988+0000", 2024-09-14T23:58:32.039 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0410000000000001e-06 2024-09-14T23:58:32.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.039 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.039 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:32.039 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375992+0000", 2024-09-14T23:58:32.039 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.104e-06 2024-09-14T23:58:32.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.039 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.039 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:32.039 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375995+0000", 2024-09-14T23:58:32.039 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4310000000000001e-06 2024-09-14T23:58:32.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.039 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.039 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:32.039 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426958+0000", 2024-09-14T23:58:32.040 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050963642000000003 2024-09-14T23:58:32.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.040 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.040 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:32.040 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426962+0000", 2024-09-14T23:58:32.040 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6839999999999998e-06 2024-09-14T23:58:32.040 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.040 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:32.040 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.040 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.040 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5894 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:32.040 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:58:32.041 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.616704813, 2024-09-14T23:58:32.041 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050978333000000001, 2024-09-14T23:58:32.041 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:32.041 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:32.041 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:32.041 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:32.041 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:32.041 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:32.041 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5894 2024-09-14T23:58:32.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.041 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:32.042 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.042 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:32.042 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:58:32.042 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.042 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.042 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:32.042 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:58:32.042 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.042 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.042 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:32.042 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:58:32.043 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.23e-07 2024-09-14T23:58:32.043 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.043 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.043 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:32.043 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375999+0000", 2024-09-14T23:58:32.043 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9800000000000002e-07 2024-09-14T23:58:32.043 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.043 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.043 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:32.043 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376007+0000", 2024-09-14T23:58:32.043 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8539999999999997e-06 2024-09-14T23:58:32.043 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.043 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.043 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:32.043 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376011+0000", 2024-09-14T23:58:32.044 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.399e-06 2024-09-14T23:58:32.044 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.044 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.044 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:32.044 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426973+0000", 2024-09-14T23:58:32.044 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050962454999999997 2024-09-14T23:58:32.044 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.044 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.044 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:32.044 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426976+0000", 2024-09-14T23:58:32.044 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4039999999999999e-06 2024-09-14T23:58:32.044 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.044 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:32.044 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.044 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.044 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.045 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5895 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:32.045 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376014+0000", 2024-09-14T23:58:32.045 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.616688469, 2024-09-14T23:58:32.045 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050975951999999998, 2024-09-14T23:58:32.045 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:32.045 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:32.045 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:32.045 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:32.045 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:32.045 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:32.045 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5895 2024-09-14T23:58:32.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.045 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:32.045 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.046 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:32.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376014+0000", 2024-09-14T23:58:32.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.046 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:32.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376014+0000", 2024-09-14T23:58:32.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.046 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:32.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376015+0000", 2024-09-14T23:58:32.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2999999999999999e-07 2024-09-14T23:58:32.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:32.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376016+0000", 2024-09-14T23:58:32.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1659999999999999e-06 2024-09-14T23:58:32.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:32.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376024+0000", 2024-09-14T23:58:32.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8930000000000005e-06 2024-09-14T23:58:32.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:32.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376026+0000", 2024-09-14T23:58:32.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6869999999999999e-06 2024-09-14T23:58:32.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.048 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:32.048 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426987+0000", 2024-09-14T23:58:32.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050960601000000001 2024-09-14T23:58:32.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.048 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:32.048 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426990+0000", 2024-09-14T23:58:32.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3749999999999999e-06 2024-09-14T23:58:32.048 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.048 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:32.048 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.049 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5896 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:32.049 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:58:32.049 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.616672732, 2024-09-14T23:58:32.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050975656000000001, 2024-09-14T23:58:32.049 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:32.049 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:32.049 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:32.049 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:32.049 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:32.049 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:32.049 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5896 2024-09-14T23:58:32.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.049 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:32.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.049 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:32.050 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:58:32.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.050 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:32.050 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:58:32.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.050 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:32.050 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:58:32.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5100000000000001e-07 2024-09-14T23:58:32.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.050 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:32.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376032+0000", 2024-09-14T23:58:32.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.141e-06 2024-09-14T23:58:32.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:32.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376036+0000", 2024-09-14T23:58:32.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.25e-06 2024-09-14T23:58:32.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:32.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376038+0000", 2024-09-14T23:58:32.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5469999999999999e-06 2024-09-14T23:58:32.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:32.052 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427002+0000", 2024-09-14T23:58:32.052 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050964029000000001 2024-09-14T23:58:32.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.052 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.052 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:32.052 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427006+0000", 2024-09-14T23:58:32.052 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4379999999999999e-06 2024-09-14T23:58:32.052 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.052 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:32.052 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.052 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.052 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5897 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:32.053 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:58:32.053 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.616660932, 2024-09-14T23:58:32.053 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050978001000000002, 2024-09-14T23:58:32.053 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:32.053 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:32.053 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:32.053 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:32.053 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:32.053 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:32.053 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5897 2024-09-14T23:58:32.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.053 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:32.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.054 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:32.054 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:58:32.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.054 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:32.054 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:58:32.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.054 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:32.054 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:58:32.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2600000000000001e-07 2024-09-14T23:58:32.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:32.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376043+0000", 2024-09-14T23:58:32.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.054e-06 2024-09-14T23:58:32.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:32.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376047+0000", 2024-09-14T23:58:32.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0659999999999997e-06 2024-09-14T23:58:32.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:32.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376050+0000", 2024-09-14T23:58:32.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.88e-06 2024-09-14T23:58:32.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:32.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427017+0000", 2024-09-14T23:58:32.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050966453000000002 2024-09-14T23:58:32.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:32.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427020+0000", 2024-09-14T23:58:32.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3220000000000001e-06 2024-09-14T23:58:32.056 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.056 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:32.056 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.057 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5898 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:32.057 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376053+0000", 2024-09-14T23:58:32.057 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.616649447, 2024-09-14T23:58:32.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050982024000000001, 2024-09-14T23:58:32.057 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:32.057 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:32.057 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:32.057 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:32.057 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:32.057 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:32.057 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5898 2024-09-14T23:58:32.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.058 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:32.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.058 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:32.058 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376053+0000", 2024-09-14T23:58:32.058 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.058 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:32.058 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376053+0000", 2024-09-14T23:58:32.058 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.058 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:32.058 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376278+0000", 2024-09-14T23:58:32.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022426400000000001 2024-09-14T23:58:32.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:32.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376282+0000", 2024-09-14T23:58:32.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4739999999999999e-06 2024-09-14T23:58:32.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:32.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376296+0000", 2024-09-14T23:58:32.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4229999999999999e-05 2024-09-14T23:58:32.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:32.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376303+0000", 2024-09-14T23:58:32.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2199999999999997e-06 2024-09-14T23:58:32.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:32.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427031+0000", 2024-09-14T23:58:32.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050728154999999997 2024-09-14T23:58:32.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:32.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427036+0000", 2024-09-14T23:58:32.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6809999999999997e-06 2024-09-14T23:58:32.060 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.060 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:32.061 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.061 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5899 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:32.061 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376338+0000", 2024-09-14T23:58:32.061 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.616364723, 2024-09-14T23:58:32.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050712100000000003, 2024-09-14T23:58:32.061 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:32.061 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:32.061 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:32.061 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:32.061 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:32.061 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:32.061 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5899 2024-09-14T23:58:32.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.062 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:32.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.062 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:32.062 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376338+0000", 2024-09-14T23:58:32.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.062 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:32.062 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376338+0000", 2024-09-14T23:58:32.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.062 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:32.063 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376339+0000", 2024-09-14T23:58:32.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9100000000000003e-07 2024-09-14T23:58:32.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:32.063 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376342+0000", 2024-09-14T23:58:32.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7e-06 2024-09-14T23:58:32.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:32.063 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376350+0000", 2024-09-14T23:58:32.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5280000000000003e-06 2024-09-14T23:58:32.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.064 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:32.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376356+0000", 2024-09-14T23:58:32.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6010000000000002e-06 2024-09-14T23:58:32.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.064 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:32.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427047+0000", 2024-09-14T23:58:32.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050690962999999999 2024-09-14T23:58:32.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.064 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:32.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427050+0000", 2024-09-14T23:58:32.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.517e-06 2024-09-14T23:58:32.064 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.064 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:32.065 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.065 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5900 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:32.065 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376366+0000", 2024-09-14T23:58:32.065 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.616336464, 2024-09-14T23:58:32.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050697858999999998, 2024-09-14T23:58:32.065 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:32.065 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:32.065 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:32.065 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:32.065 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:32.065 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:32.065 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5900 2024-09-14T23:58:32.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.066 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:32.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:32.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376366+0000", 2024-09-14T23:58:32.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:32.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376366+0000", 2024-09-14T23:58:32.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:32.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377650+0000", 2024-09-14T23:58:32.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001283834 2024-09-14T23:58:32.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:32.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377655+0000", 2024-09-14T23:58:32.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7230000000000004e-06 2024-09-14T23:58:32.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:32.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377667+0000", 2024-09-14T23:58:32.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1746000000000001e-05 2024-09-14T23:58:32.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:32.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377676+0000", 2024-09-14T23:58:32.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3619999999999995e-06 2024-09-14T23:58:32.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:32.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427061+0000", 2024-09-14T23:58:32.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049384755000000002 2024-09-14T23:58:32.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:32.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427064+0000", 2024-09-14T23:58:32.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.439e-06 2024-09-14T23:58:32.069 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.069 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:32.069 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.069 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:132622 3.1e 3:79d0a5e0:::rbd_data.111847077d5e.0000000000000007:head [sparse-read 0~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T23:58:32.069 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:44.037901+0000", 2024-09-14T23:58:32.069 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 47.954801854999999, 2024-09-14T23:58:32.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1083420859999999, 2024-09-14T23:58:32.069 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:32.069 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:32.069 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:32.069 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:32.070 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:32.070 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:32.070 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 132622 2024-09-14T23:58:32.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.070 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:32.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:32.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037901+0000", 2024-09-14T23:58:32.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:32.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037901+0000", 2024-09-14T23:58:32.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:32.071 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037903+0000", 2024-09-14T23:58:32.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4619999999999999e-06 2024-09-14T23:58:32.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:32.071 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037910+0000", 2024-09-14T23:58:32.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7449999999999995e-06 2024-09-14T23:58:32.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:32.071 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037928+0000", 2024-09-14T23:58:32.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7305000000000001e-05 2024-09-14T23:58:32.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:32.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037939+0000", 2024-09-14T23:58:32.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1599e-05 2024-09-14T23:58:32.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:32.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.038049+0000", 2024-09-14T23:58:32.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000109577 2024-09-14T23:58:32.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op must wait for map", 2024-09-14T23:58:32.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.038057+0000", 2024-09-14T23:58:32.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1450000000000008e-06 2024-09-14T23:58:32.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:32.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.039937+0000", 2024-09-14T23:58:32.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00188024 2024-09-14T23:58:32.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:58:32.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.040039+0000", 2024-09-14T23:58:32.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000101934 2024-09-14T23:58:32.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:32.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:45.145805+0000", 2024-09-14T23:58:32.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.105765651 2024-09-14T23:58:32.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:32.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:45.145997+0000", 2024-09-14T23:58:32.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000192084 2024-09-14T23:58:32.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:32.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:45.146243+0000", 2024-09-14T23:58:32.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024634400000000001 2024-09-14T23:58:32.075 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.075 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:32.075 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.075 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4172.0:5 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:32.075 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.482606+0000", 2024-09-14T23:58:32.075 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 24.510096873999998, 2024-09-14T23:58:32.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.49381641300000001, 2024-09-14T23:58:32.075 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:32.075 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:32.075 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:32.075 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:32.075 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4172", 2024-09-14T23:58:32.076 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/2542269203", 2024-09-14T23:58:32.076 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5 2024-09-14T23:58:32.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.076 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:32.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:32.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482606+0000", 2024-09-14T23:58:32.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:32.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482606+0000", 2024-09-14T23:58:32.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:32.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482607+0000", 2024-09-14T23:58:32.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3120000000000001e-06 2024-09-14T23:58:32.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:32.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482615+0000", 2024-09-14T23:58:32.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9300000000000003e-06 2024-09-14T23:58:32.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:32.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482632+0000", 2024-09-14T23:58:32.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6966999999999999e-05 2024-09-14T23:58:32.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:32.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482640+0000", 2024-09-14T23:58:32.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1820000000000006e-06 2024-09-14T23:58:32.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:32.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482703+0000", 2024-09-14T23:58:32.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2098000000000001e-05 2024-09-14T23:58:32.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:58:32.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482713+0000", 2024-09-14T23:58:32.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0064000000000001e-05 2024-09-14T23:58:32.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:32.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.976307+0000", 2024-09-14T23:58:32.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.49359442999999997 2024-09-14T23:58:32.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:32.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.976422+0000", 2024-09-14T23:58:32.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011543 2024-09-14T23:58:32.079 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.079 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:32.079 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.080 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.080 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:32.080 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:32.082 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T23:58:32.082 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:32.079+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:32.181 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:32.180+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:32.235 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:32.235 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:32.235 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:58:32.235 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:32.244 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T23:58:32.284 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:32.283+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:32.382 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:32.381+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:32.392 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:32.393 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:32.393 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:58:32.393 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:58:32.393 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:32.401 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T23:58:32.483 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:32.482+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:32.550 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:32.550 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:58:32.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:58:32.550 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:58:32.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.551 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:3578 3.4 e29/23 3:2298d860:::rbd_data.1118de1168b.0000000000000030:head v 29'15, mlcod=29'13)", 2024-09-14T23:58:32.551 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.036455+0000", 2024-09-14T23:58:32.551 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 113.51166853399999, 2024-09-14T23:58:32.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019449440000000001, 2024-09-14T23:58:32.551 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:32.551 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:32.551 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:32.551 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:32.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:32.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036455+0000", 2024-09-14T23:58:32.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:32.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036455+0000", 2024-09-14T23:58:32.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:32.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036456+0000", 2024-09-14T23:58:32.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0999999999999997e-07 2024-09-14T23:58:32.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:32.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036463+0000", 2024-09-14T23:58:32.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2200000000000003e-06 2024-09-14T23:58:32.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:32.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036478+0000", 2024-09-14T23:58:32.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4977000000000001e-05 2024-09-14T23:58:32.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:32.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036485+0000", 2024-09-14T23:58:32.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0559999999999999e-06 2024-09-14T23:58:32.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:32.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036693+0000", 2024-09-14T23:58:32.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000208051 2024-09-14T23:58:32.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:32.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036715+0000", 2024-09-14T23:58:32.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1330000000000001e-05 2024-09-14T23:58:32.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:32.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.038356+0000", 2024-09-14T23:58:32.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001641586 2024-09-14T23:58:32.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:32.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.038400+0000", 2024-09-14T23:58:32.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3914e-05 2024-09-14T23:58:32.555 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.555 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:32.555 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.555 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:3579 3.4 e29/23 3:27290fe6:::rbd_data.1118de1168b.0000000000000032:head v 29'16, mlcod=29'13)", 2024-09-14T23:58:32.555 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.037085+0000", 2024-09-14T23:58:32.555 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 113.51103894400001, 2024-09-14T23:58:32.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0020019220000000002, 2024-09-14T23:58:32.555 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:32.555 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:32.555 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:32.556 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:32.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:32.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037085+0000", 2024-09-14T23:58:32.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:32.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037085+0000", 2024-09-14T23:58:32.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:32.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037085+0000", 2024-09-14T23:58:32.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5499999999999997e-07 2024-09-14T23:58:32.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:32.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037091+0000", 2024-09-14T23:58:32.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7869999999999998e-06 2024-09-14T23:58:32.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:32.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037102+0000", 2024-09-14T23:58:32.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.031e-05 2024-09-14T23:58:32.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:32.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037116+0000", 2024-09-14T23:58:32.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4119e-05 2024-09-14T23:58:32.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:32.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037219+0000", 2024-09-14T23:58:32.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010372799999999999 2024-09-14T23:58:32.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:32.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037239+0000", 2024-09-14T23:58:32.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9993e-05 2024-09-14T23:58:32.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:32.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.039046+0000", 2024-09-14T23:58:32.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001806666 2024-09-14T23:58:32.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:32.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.039087+0000", 2024-09-14T23:58:32.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0564e-05 2024-09-14T23:58:32.559 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.559 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:32.559 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.560 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:32024 3.9 e39/23 3:94dc260b:::rbd_data.111814b482f8.0000000000000009:head v 39'45, mlcod=38'43)", 2024-09-14T23:58:32.560 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:49.061018+0000", 2024-09-14T23:58:32.560 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 103.487105605, 2024-09-14T23:58:32.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018482240000000001, 2024-09-14T23:58:32.560 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:32.560 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:32.560 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:32.560 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:32.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:32.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061018+0000", 2024-09-14T23:58:32.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:32.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061018+0000", 2024-09-14T23:58:32.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:32.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061018+0000", 2024-09-14T23:58:32.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9999999999999998e-07 2024-09-14T23:58:32.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:32.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061211+0000", 2024-09-14T23:58:32.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019206 2024-09-14T23:58:32.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:32.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061221+0000", 2024-09-14T23:58:32.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0659000000000001e-05 2024-09-14T23:58:32.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:32.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061230+0000", 2024-09-14T23:58:32.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4610000000000008e-06 2024-09-14T23:58:32.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:32.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061317+0000", 2024-09-14T23:58:32.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7514000000000003e-05 2024-09-14T23:58:32.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:32.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061338+0000", 2024-09-14T23:58:32.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0395999999999999e-05 2024-09-14T23:58:32.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:32.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.062830+0000", 2024-09-14T23:58:32.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0014921940000000001 2024-09-14T23:58:32.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:32.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.062866+0000", 2024-09-14T23:58:32.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6539999999999999e-05 2024-09-14T23:58:32.564 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.564 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:32.564 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.564 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:51281 3.0 e52/23 3:04e60ec2:::rbd_data.11181975a1de.0000000000000003:head v 52'113, mlcod=44'111)", 2024-09-14T23:58:32.564 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:02.158822+0000", 2024-09-14T23:58:32.565 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 90.389301419000006, 2024-09-14T23:58:32.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019860609999999999, 2024-09-14T23:58:32.565 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:32.565 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:32.565 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:32.565 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:32.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:32.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.158822+0000", 2024-09-14T23:58:32.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:32.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.158822+0000", 2024-09-14T23:58:32.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:32.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.158823+0000", 2024-09-14T23:58:32.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6800000000000001e-07 2024-09-14T23:58:32.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:32.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.159132+0000", 2024-09-14T23:58:32.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030974900000000001 2024-09-14T23:58:32.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:32.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.159146+0000", 2024-09-14T23:58:32.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3261000000000001e-05 2024-09-14T23:58:32.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:32.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.159155+0000", 2024-09-14T23:58:32.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7410000000000004e-06 2024-09-14T23:58:32.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:32.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.159221+0000", 2024-09-14T23:58:32.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5698999999999996e-05 2024-09-14T23:58:32.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:32.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.159244+0000", 2024-09-14T23:58:32.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3108e-05 2024-09-14T23:58:32.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:32.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.160767+0000", 2024-09-14T23:58:32.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015231120000000001 2024-09-14T23:58:32.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:32.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.160808+0000", 2024-09-14T23:58:32.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1022999999999997e-05 2024-09-14T23:58:32.569 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.569 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:32.569 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.569 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:56107 3.0 e56/23 3:00c47890:::rbd_data.1118c4b3aebe.0000000000000009:head v 56'125, mlcod=52'123)", 2024-09-14T23:58:32.569 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.236123+0000", 2024-09-14T23:58:32.569 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 86.312000220000002, 2024-09-14T23:58:32.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0061273220000000002, 2024-09-14T23:58:32.569 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:32.569 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:32.569 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:32.569 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:32.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:32.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.236123+0000", 2024-09-14T23:58:32.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:32.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.236123+0000", 2024-09-14T23:58:32.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:32.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.236125+0000", 2024-09-14T23:58:32.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.313e-06 2024-09-14T23:58:32.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:32.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237428+0000", 2024-09-14T23:58:32.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013029890000000001 2024-09-14T23:58:32.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:32.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237452+0000", 2024-09-14T23:58:32.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4090000000000001e-05 2024-09-14T23:58:32.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:32.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237462+0000", 2024-09-14T23:58:32.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0421999999999999e-05 2024-09-14T23:58:32.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:32.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237559+0000", 2024-09-14T23:58:32.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6378000000000005e-05 2024-09-14T23:58:32.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:32.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237583+0000", 2024-09-14T23:58:32.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4182e-05 2024-09-14T23:58:32.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:32.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.242205+0000", 2024-09-14T23:58:32.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0046217289999999998 2024-09-14T23:58:32.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:32.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.242251+0000", 2024-09-14T23:58:32.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6218999999999999e-05 2024-09-14T23:58:32.573 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.573 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:32.573 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.573 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:56108 3.0 e56/23 3:00c47890:::rbd_data.1118c4b3aebe.0000000000000009:head v 56'127, mlcod=52'123)", 2024-09-14T23:58:32.573 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.239693+0000", 2024-09-14T23:58:32.573 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 86.308430749999999, 2024-09-14T23:58:32.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0043496209999999997, 2024-09-14T23:58:32.574 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:32.574 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:32.574 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:32.574 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:32.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:32.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239693+0000", 2024-09-14T23:58:32.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:32.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239693+0000", 2024-09-14T23:58:32.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:32.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239694+0000", 2024-09-14T23:58:32.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0499999999999999e-06 2024-09-14T23:58:32.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:32.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239702+0000", 2024-09-14T23:58:32.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1170000000000004e-06 2024-09-14T23:58:32.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:32.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239716+0000", 2024-09-14T23:58:32.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4306e-05 2024-09-14T23:58:32.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:32.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239724+0000", 2024-09-14T23:58:32.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0399999999999993e-06 2024-09-14T23:58:32.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:32.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239778+0000", 2024-09-14T23:58:32.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4032e-05 2024-09-14T23:58:32.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:32.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239801+0000", 2024-09-14T23:58:32.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2770000000000001e-05 2024-09-14T23:58:32.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:32.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.243990+0000", 2024-09-14T23:58:32.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0041890909999999998 2024-09-14T23:58:32.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:32.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.244043+0000", 2024-09-14T23:58:32.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2215000000000001e-05 2024-09-14T23:58:32.577 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.577 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:32.577 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.578 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:56385 3.9 e56/23 3:963d31c7:::rbd_data.1118c4b3aebe.0000000000000003:head v 56'87, mlcod=53'85)", 2024-09-14T23:58:32.578 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.283936+0000", 2024-09-14T23:58:32.578 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 86.264187840000005, 2024-09-14T23:58:32.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0063042080000000004, 2024-09-14T23:58:32.578 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:32.578 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:32.578 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:32.578 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:32.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:32.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.283936+0000", 2024-09-14T23:58:32.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:32.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.283936+0000", 2024-09-14T23:58:32.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:32.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.283937+0000", 2024-09-14T23:58:32.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1179999999999999e-06 2024-09-14T23:58:32.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:32.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286338+0000", 2024-09-14T23:58:32.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024010149999999998 2024-09-14T23:58:32.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:32.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286360+0000", 2024-09-14T23:58:32.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2212e-05 2024-09-14T23:58:32.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:32.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286370+0000", 2024-09-14T23:58:32.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5850000000000006e-06 2024-09-14T23:58:32.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286443+0000", 2024-09-14T23:58:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2757999999999997e-05 2024-09-14T23:58:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286466+0000", 2024-09-14T23:58:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3844000000000001e-05 2024-09-14T23:58:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.290197+0000", 2024-09-14T23:58:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003730176 2024-09-14T23:58:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.290240+0000", 2024-09-14T23:58:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.35e-05 2024-09-14T23:58:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:56386 3.9 e56/23 3:963d31c7:::rbd_data.1118c4b3aebe.0000000000000003:head v 56'89, mlcod=53'85)", 2024-09-14T23:58:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.287578+0000", 2024-09-14T23:58:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 86.260546113999993, 2024-09-14T23:58:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033580250000000002, 2024-09-14T23:58:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:32.582 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287578+0000", 2024-09-14T23:58:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287578+0000", 2024-09-14T23:58:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287579+0000", 2024-09-14T23:58:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.64e-07 2024-09-14T23:58:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:32.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287586+0000", 2024-09-14T23:58:32.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5310000000000004e-06 2024-09-14T23:58:32.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:32.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287597+0000", 2024-09-14T23:58:32.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0903999999999999e-05 2024-09-14T23:58:32.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:32.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287605+0000", 2024-09-14T23:58:32.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7889999999999995e-06 2024-09-14T23:58:32.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:32.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.288437+0000", 2024-09-14T23:58:32.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000831914 2024-09-14T23:58:32.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:32.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.288460+0000", 2024-09-14T23:58:32.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3410000000000001e-05 2024-09-14T23:58:32.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.290883+0000", 2024-09-14T23:58:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024230599999999999 2024-09-14T23:58:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.290936+0000", 2024-09-14T23:58:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2453000000000001e-05 2024-09-14T23:58:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:60299 3.9 e57/23 3:96f26b6f:::rbd_data.11188d356229.0000000000000009:head v 57'98, mlcod=56'96)", 2024-09-14T23:58:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:07.418336+0000", 2024-09-14T23:58:32.586 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 85.129787621000006, 2024-09-14T23:58:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022211459999999998, 2024-09-14T23:58:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.418336+0000", 2024-09-14T23:58:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.418336+0000", 2024-09-14T23:58:32.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:32.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.418337+0000", 2024-09-14T23:58:32.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7199999999999999e-07 2024-09-14T23:58:32.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:32.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.419396+0000", 2024-09-14T23:58:32.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010598319999999999 2024-09-14T23:58:32.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:32.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.419407+0000", 2024-09-14T23:58:32.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0956e-05 2024-09-14T23:58:32.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:32.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.419417+0000", 2024-09-14T23:58:32.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0040999999999999e-05 2024-09-14T23:58:32.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:32.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.419468+0000", 2024-09-14T23:58:32.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0497000000000001e-05 2024-09-14T23:58:32.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.419485+0000", 2024-09-14T23:58:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6966e-05 2024-09-14T23:58:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.420527+0000", 2024-09-14T23:58:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001042133 2024-09-14T23:58:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.420557+0000", 2024-09-14T23:58:32.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0249000000000001e-05 2024-09-14T23:58:32.591 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.591 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:32.591 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.591 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:64263 3.4 e57/23 3:20c6a768:::rbd_data.11188d356229.0000000000000001:head v 57'165, mlcod=57'163)", 2024-09-14T23:58:32.591 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:07.999031+0000", 2024-09-14T23:58:32.591 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 84.549092966000003, 2024-09-14T23:58:32.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023815210000000002, 2024-09-14T23:58:32.591 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:32.591 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:32.591 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:32.591 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:32.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:32.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999031+0000", 2024-09-14T23:58:32.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:32.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999031+0000", 2024-09-14T23:58:32.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:32.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999032+0000", 2024-09-14T23:58:32.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.176e-06 2024-09-14T23:58:32.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:32.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999660+0000", 2024-09-14T23:58:32.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00062817099999999996 2024-09-14T23:58:32.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:32.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999683+0000", 2024-09-14T23:58:32.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2816999999999999e-05 2024-09-14T23:58:32.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:32.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999693+0000", 2024-09-14T23:58:32.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0088999999999999e-05 2024-09-14T23:58:32.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:32.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999767+0000", 2024-09-14T23:58:32.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4056999999999999e-05 2024-09-14T23:58:32.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:32.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999793+0000", 2024-09-14T23:58:32.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5573000000000001e-05 2024-09-14T23:58:32.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:32.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:08.001379+0000", 2024-09-14T23:58:32.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015859019999999999 2024-09-14T23:58:32.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:32.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:08.001412+0000", 2024-09-14T23:58:32.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3735999999999997e-05 2024-09-14T23:58:32.595 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.595 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:32.595 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.595 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:66570 3.0 e60/23 3:055350b3:::rbd_data.111873792d9b.0000000000000005:head v 60'162, mlcod=57'159)", 2024-09-14T23:58:32.595 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:11.277517+0000", 2024-09-14T23:58:32.595 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 81.270606810000004, 2024-09-14T23:58:32.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003472199, 2024-09-14T23:58:32.595 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:32.595 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:32.595 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:32.596 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:32.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:32.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.277517+0000", 2024-09-14T23:58:32.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:32.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.277517+0000", 2024-09-14T23:58:32.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:32.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.277518+0000", 2024-09-14T23:58:32.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1379999999999999e-06 2024-09-14T23:58:32.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:32.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278504+0000", 2024-09-14T23:58:32.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00098617499999999994 2024-09-14T23:58:32.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:32.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278525+0000", 2024-09-14T23:58:32.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.09e-05 2024-09-14T23:58:32.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:32.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278534+0000", 2024-09-14T23:58:32.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6209999999999993e-06 2024-09-14T23:58:32.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:32.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278611+0000", 2024-09-14T23:58:32.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6926000000000002e-05 2024-09-14T23:58:32.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:32.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278641+0000", 2024-09-14T23:58:32.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0861999999999997e-05 2024-09-14T23:58:32.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:32.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.280953+0000", 2024-09-14T23:58:32.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023114239999999999 2024-09-14T23:58:32.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:32.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.280989+0000", 2024-09-14T23:58:32.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6152999999999998e-05 2024-09-14T23:58:32.599 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.599 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:32.599 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.599 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "pg_update_log_missing(3.9 epoch 84/23 rep_tid 2234 entries 84'200 (0'0) error 3:93b58b1d:::rbd_data.1118417a1b0b.000000000000001b:head by client.4376.0:118637 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 83'197)", 2024-09-14T23:58:32.599 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:35.103156+0000", 2024-09-14T23:58:32.599 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 57.444967536999997, 2024-09-14T23:58:32.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019852009999999998, 2024-09-14T23:58:32.600 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:32.600 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:32.600 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:32.600 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:32.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:32.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:35.103156+0000", 2024-09-14T23:58:32.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:32.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:35.103156+0000", 2024-09-14T23:58:32.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:32.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:35.103157+0000", 2024-09-14T23:58:32.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1969999999999999e-06 2024-09-14T23:58:32.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:32.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:35.103170+0000", 2024-09-14T23:58:32.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2921e-05 2024-09-14T23:58:32.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:32.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:35.103202+0000", 2024-09-14T23:58:32.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1992000000000001e-05 2024-09-14T23:58:32.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:32.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:35.103220+0000", 2024-09-14T23:58:32.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7312999999999998e-05 2024-09-14T23:58:32.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:32.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:35.103305+0000", 2024-09-14T23:58:32.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4975000000000005e-05 2024-09-14T23:58:32.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:32.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:35.105141+0000", 2024-09-14T23:58:32.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018368029999999999 2024-09-14T23:58:32.603 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.603 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:32.603 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.603 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:142751 3.4 e99/23 3:265f7763:::rbd_data.1118f9c96a7.0000000000000004:head v 99'441, mlcod=98'438)", 2024-09-14T23:58:32.603 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:53.165301+0000", 2024-09-14T23:58:32.603 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 39.382822386999997, 2024-09-14T23:58:32.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0021754449999999998, 2024-09-14T23:58:32.603 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:32.603 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:32.603 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:32.604 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:32.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:32.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.165301+0000", 2024-09-14T23:58:32.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:32.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.165301+0000", 2024-09-14T23:58:32.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:32.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.165302+0000", 2024-09-14T23:58:32.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0899999999999997e-07 2024-09-14T23:58:32.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:32.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.165310+0000", 2024-09-14T23:58:32.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7520000000000002e-06 2024-09-14T23:58:32.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:32.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.165321+0000", 2024-09-14T23:58:32.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0668e-05 2024-09-14T23:58:32.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:32.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.165330+0000", 2024-09-14T23:58:32.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.234e-06 2024-09-14T23:58:32.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:32.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.165393+0000", 2024-09-14T23:58:32.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2716000000000001e-05 2024-09-14T23:58:32.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:32.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.165418+0000", 2024-09-14T23:58:32.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4544000000000001e-05 2024-09-14T23:58:32.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.167436+0000", 2024-09-14T23:58:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002018597 2024-09-14T23:58:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.167477+0000", 2024-09-14T23:58:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0624999999999998e-05 2024-09-14T23:58:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.607 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:158983 3.4 e103/23 3:20e43b29:::rbd_data.1118b2f495e6.0000000000000012:head v 103'454, mlcod=103'452)", 2024-09-14T23:58:32.608 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:58.917699+0000", 2024-09-14T23:58:32.608 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 33.630424759999997, 2024-09-14T23:58:32.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037933799999999998, 2024-09-14T23:58:32.608 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:32.608 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:32.608 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:32.608 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:32.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:32.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917699+0000", 2024-09-14T23:58:32.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:32.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917699+0000", 2024-09-14T23:58:32.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:32.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917700+0000", 2024-09-14T23:58:32.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3850000000000001e-06 2024-09-14T23:58:32.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:32.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917939+0000", 2024-09-14T23:58:32.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000238249 2024-09-14T23:58:32.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:32.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917961+0000", 2024-09-14T23:58:32.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2894000000000001e-05 2024-09-14T23:58:32.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:32.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917972+0000", 2024-09-14T23:58:32.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0261000000000001e-05 2024-09-14T23:58:32.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:32.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.918038+0000", 2024-09-14T23:58:32.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6223000000000006e-05 2024-09-14T23:58:32.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:32.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.918063+0000", 2024-09-14T23:58:32.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5252000000000001e-05 2024-09-14T23:58:32.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:32.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.921433+0000", 2024-09-14T23:58:32.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033699960000000001 2024-09-14T23:58:32.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:32.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.921492+0000", 2024-09-14T23:58:32.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9120000000000003e-05 2024-09-14T23:58:32.612 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.612 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:32.612 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.612 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4172.0:6 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T23:58:32.612 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.479840+0000", 2024-09-14T23:58:32.612 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 25.068283480000002, 2024-09-14T23:58:32.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022238340000000001, 2024-09-14T23:58:32.612 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:32.612 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:32.613 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:32.613 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:32.613 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4172", 2024-09-14T23:58:32.613 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/2542269203", 2024-09-14T23:58:32.613 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 6 2024-09-14T23:58:32.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.613 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:32.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:32.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.479840+0000", 2024-09-14T23:58:32.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:32.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.479840+0000", 2024-09-14T23:58:32.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:32.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.479843+0000", 2024-09-14T23:58:32.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6680000000000001e-06 2024-09-14T23:58:32.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:32.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.479862+0000", 2024-09-14T23:58:32.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8762000000000001e-05 2024-09-14T23:58:32.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:32.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.479881+0000", 2024-09-14T23:58:32.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9860999999999998e-05 2024-09-14T23:58:32.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:32.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.479899+0000", 2024-09-14T23:58:32.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7127000000000001e-05 2024-09-14T23:58:32.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:32.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.481969+0000", 2024-09-14T23:58:32.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002070166 2024-09-14T23:58:32.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:32.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482064+0000", 2024-09-14T23:58:32.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5249999999999998e-05 2024-09-14T23:58:32.616 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.616 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:32.616 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.616 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(osd.1.0:1460 3.0 e122/23 3:00cc1e32:::rbd_data.11187b9f1032.0000000000000001:2f v 122'453, mlcod=118'449)", 2024-09-14T23:58:32.616 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:22.287423+0000", 2024-09-14T23:58:32.616 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.260701103000001, 2024-09-14T23:58:32.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0046041659999999998, 2024-09-14T23:58:32.616 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:32.616 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:32.617 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:32.617 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:32.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:32.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.287423+0000", 2024-09-14T23:58:32.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:32.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.287423+0000", 2024-09-14T23:58:32.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:32.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.287423+0000", 2024-09-14T23:58:32.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7500000000000001e-07 2024-09-14T23:58:32.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:32.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.287435+0000", 2024-09-14T23:58:32.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2006e-05 2024-09-14T23:58:32.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:32.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.287444+0000", 2024-09-14T23:58:32.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.465e-06 2024-09-14T23:58:32.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:32.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.287453+0000", 2024-09-14T23:58:32.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0100000000000001e-06 2024-09-14T23:58:32.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:32.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.287519+0000", 2024-09-14T23:58:32.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6149999999999995e-05 2024-09-14T23:58:32.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:32.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.287543+0000", 2024-09-14T23:58:32.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3682000000000001e-05 2024-09-14T23:58:32.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:32.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.291977+0000", 2024-09-14T23:58:32.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0044333829999999999 2024-09-14T23:58:32.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:32.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.292027+0000", 2024-09-14T23:58:32.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0195000000000003e-05 2024-09-14T23:58:32.620 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.620 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:32.620 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.620 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDPGPush(3.1e 126/125 [PushOp(3:7f7265c3:::rbd_data.11188d356229.000000000000000d:16, version: 58'124, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:7f7265c3:::rbd_data.11188d356229.000000000000000d:16@58'124, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 16=[]:{16: [16]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T23:58:32.621 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:26.552687+0000", 2024-09-14T23:58:32.621 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.9954368389999999, 2024-09-14T23:58:32.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023212060000000001, 2024-09-14T23:58:32.621 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:32.621 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:32.621 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:32.621 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:32.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:32.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.552687+0000", 2024-09-14T23:58:32.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:32.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.552687+0000", 2024-09-14T23:58:32.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:32.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.552688+0000", 2024-09-14T23:58:32.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.47e-07 2024-09-14T23:58:32.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:32.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.553659+0000", 2024-09-14T23:58:32.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00097088899999999995 2024-09-14T23:58:32.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:32.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.553682+0000", 2024-09-14T23:58:32.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3862e-05 2024-09-14T23:58:32.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:32.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.553703+0000", 2024-09-14T23:58:32.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0310999999999999e-05 2024-09-14T23:58:32.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:32.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.553813+0000", 2024-09-14T23:58:32.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011051 2024-09-14T23:58:32.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:32.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.553828+0000", 2024-09-14T23:58:32.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4625e-05 2024-09-14T23:58:32.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:32.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.555008+0000", 2024-09-14T23:58:32.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001180162 2024-09-14T23:58:32.624 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.624 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:32.624 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.624 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDPGPush(3.5 126/125 [PushOp(3:a05d004f:::rbd_data.11188d356229.000000000000000b:16, version: 58'296, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:a05d004f:::rbd_data.11188d356229.000000000000000b:16@58'296, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 16=[]:{16: [16]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T23:58:32.625 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:27.212140+0000", 2024-09-14T23:58:32.625 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.3359833390000002, 2024-09-14T23:58:32.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001842773, 2024-09-14T23:58:32.625 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:32.625 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:32.625 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:32.625 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:32.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:32.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.212140+0000", 2024-09-14T23:58:32.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:32.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.212140+0000", 2024-09-14T23:58:32.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:32.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.212141+0000", 2024-09-14T23:58:32.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.68e-07 2024-09-14T23:58:32.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:32.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.212695+0000", 2024-09-14T23:58:32.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00055376799999999995 2024-09-14T23:58:32.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:32.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.212712+0000", 2024-09-14T23:58:32.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7258e-05 2024-09-14T23:58:32.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:32.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.212725+0000", 2024-09-14T23:58:32.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3247999999999999e-05 2024-09-14T23:58:32.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:32.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.212807+0000", 2024-09-14T23:58:32.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2191e-05 2024-09-14T23:58:32.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:32.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.212815+0000", 2024-09-14T23:58:32.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6029999999999999e-06 2024-09-14T23:58:32.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:32.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.213983+0000", 2024-09-14T23:58:32.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001168037 2024-09-14T23:58:32.628 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.628 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:32.628 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.629 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(osd.1.0:1465 3.4 e127/23 3:265f7763:::rbd_data.1118f9c96a7.0000000000000004:2c v 127'508, mlcod=120'505)", 2024-09-14T23:58:32.629 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:27.318056+0000", 2024-09-14T23:58:32.629 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.2300674159999998, 2024-09-14T23:58:32.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0049337449999999998, 2024-09-14T23:58:32.629 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:32.629 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:32.629 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:32.629 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:32.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:32.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.318056+0000", 2024-09-14T23:58:32.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:32.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.318056+0000", 2024-09-14T23:58:32.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:32.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.318057+0000", 2024-09-14T23:58:32.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9799999999999999e-07 2024-09-14T23:58:32.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:32.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.318061+0000", 2024-09-14T23:58:32.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7899999999999999e-06 2024-09-14T23:58:32.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:32.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.318074+0000", 2024-09-14T23:58:32.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2537999999999999e-05 2024-09-14T23:58:32.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:32.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.318083+0000", 2024-09-14T23:58:32.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3829999999999998e-06 2024-09-14T23:58:32.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:32.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.318165+0000", 2024-09-14T23:58:32.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1928999999999994e-05 2024-09-14T23:58:32.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:32.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.318199+0000", 2024-09-14T23:58:32.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3587000000000002e-05 2024-09-14T23:58:32.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:32.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.322939+0000", 2024-09-14T23:58:32.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0047397730000000001 2024-09-14T23:58:32.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:32.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.322990+0000", 2024-09-14T23:58:32.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1347000000000002e-05 2024-09-14T23:58:32.633 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.633 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:32.633 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.633 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:252476 3.e 3:725cbf00:::rbd_header.11186acd4a72:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e132)", 2024-09-14T23:58:32.633 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:32.121781+0000", 2024-09-14T23:58:32.633 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.426342579, 2024-09-14T23:58:32.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022671459999999998, 2024-09-14T23:58:32.633 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:32.633 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:32.633 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:32.634 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:32.634 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:32.634 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:32.634 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 252476 2024-09-14T23:58:32.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.634 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:32.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:32.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:32.121781+0000", 2024-09-14T23:58:32.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:32.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:32.121781+0000", 2024-09-14T23:58:32.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:32.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:32.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:32.121783+0000", 2024-09-14T23:58:32.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.531e-06 2024-09-14T23:58:32.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:32.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:32.121790+0000", 2024-09-14T23:58:32.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7670000000000003e-06 2024-09-14T23:58:32.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:32.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:32.121806+0000", 2024-09-14T23:58:32.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5291999999999999e-05 2024-09-14T23:58:32.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:32.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:32.121813+0000", 2024-09-14T23:58:32.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.418e-06 2024-09-14T23:58:32.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:32.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:32.121879+0000", 2024-09-14T23:58:32.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5811999999999999e-05 2024-09-14T23:58:32.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op must wait for map", 2024-09-14T23:58:32.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:32.121888+0000", 2024-09-14T23:58:32.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6600000000000001e-06 2024-09-14T23:58:32.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:32.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:32.123710+0000", 2024-09-14T23:58:32.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001822838 2024-09-14T23:58:32.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:32.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:32.123803+0000", 2024-09-14T23:58:32.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3077000000000003e-05 2024-09-14T23:58:32.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:32.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:32.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:32.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:32.124048+0000", 2024-09-14T23:58:32.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024475099999999997 2024-09-14T23:58:32.638 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.638 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:32.638 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.638 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:32.638 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:32.638 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:32.640 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T23:58:32.640 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:32.638+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:32.741 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:32.739+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:32.758 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T23:58:32.790 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:32.790 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:32.790 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:58:32.790 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:32.799 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T23:58:32.841 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:58:32.840+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:58:32.884 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:58:32.884 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:58:32.901 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:58:32.902 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:58:32.918 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:58:32.918 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:58:32.934 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:58:32.935 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:58:32.942 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:32.940+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:32.949 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:32.949 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:32.951 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:58:32.952 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:58:32.958 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T23:58:32.969 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:58:32.969 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:58:32.995 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:58:32.996 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:58:33.016 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:58:33.016 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:58:33.070 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:33.069+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:33.086 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:33.086 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:33.096 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T23:58:33.143 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:33.142+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:33.212 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:33.212 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:33.221 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T23:58:33.244 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:33.242+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:33.345 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:33.343+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:33.369 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:33.370 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:33.378 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T23:58:33.446 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:33.444+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:33.486 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:33.486 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:33.495 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T23:58:33.546 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:33.545+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:33.602 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:33.602 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:33.610 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T23:58:33.651 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:33.650+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:33.748 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:33.747+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:33.757 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:33.757 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:33.766 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T23:58:33.849 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:33.848+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:33.914 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:33.914 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:33.923 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T23:58:33.950 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:33.949+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:34.041 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:34.041 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:34.050 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T23:58:34.051 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:34.050+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:34.082 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T23:58:34.152 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:34.151+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:34.162 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:34.162 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:34.171 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T23:58:34.253 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:34.252+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:34.319 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:34.319 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:34.328 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T23:58:34.371 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:58:34.370+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:58:34.436 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:34.436 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:34.445 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T23:58:34.485 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:34.484+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:34.557 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:34.556+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:34.591 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:34.592 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:34.600 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T23:58:34.658 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:34.657+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:34.748 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:34.748 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:34.756 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T23:58:34.759 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:34.758+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:34.859 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:34.858+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:34.870 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:34.870 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:34.879 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T23:58:34.960 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:34.959+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:34.976 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is set 2024-09-14T23:58:34.993 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:34.993 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:35.004 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T23:58:35.030 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T23:58:35.061 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:35.059+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:35.160 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:35.160 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:35.161 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:35.160+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:35.171 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T23:58:35.181 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:58:35.181 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:58:35.199 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:58:35.200 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:58:35.216 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:58:35.217 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:58:35.234 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:58:35.234 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:58:35.251 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:58:35.252 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:58:35.262 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:35.260+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:35.268 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:58:35.269 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:58:35.285 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:58:35.285 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:58:35.291 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:35.291 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:35.291 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:58:35.291 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:35.300 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T23:58:35.302 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:58:35.302 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:58:35.364 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:35.361+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:35.408 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:35.409 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:35.409 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:58:35.409 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:58:35.409 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:35.417 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T23:58:35.463 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:35.461+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:35.526 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:35.526 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:58:35.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:58:35.526 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:58:35.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.526 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5882 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:35.527 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:58:35.527 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 116.14856365200001, 2024-09-14T23:58:35.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050549666, 2024-09-14T23:58:35.527 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:35.527 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:35.527 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:35.527 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:35.527 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:35.527 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:35.527 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5882 2024-09-14T23:58:35.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.527 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:35.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:35.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:58:35.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:35.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:35.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:58:35.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:35.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:35.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:58:35.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8099999999999998e-07 2024-09-14T23:58:35.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:35.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375842+0000", 2024-09-14T23:58:35.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9900000000000009e-07 2024-09-14T23:58:35.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:35.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375845+0000", 2024-09-14T23:58:35.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.055e-06 2024-09-14T23:58:35.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:35.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375848+0000", 2024-09-14T23:58:35.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7609999999999999e-06 2024-09-14T23:58:35.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:35.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426381+0000", 2024-09-14T23:58:35.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050533392000000003 2024-09-14T23:58:35.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:35.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426390+0000", 2024-09-14T23:58:35.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0780000000000002e-06 2024-09-14T23:58:35.530 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:35.530 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:35.530 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:35.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.530 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5884 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:35.530 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:58:35.530 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 116.14855351200001, 2024-09-14T23:58:35.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050717439000000003, 2024-09-14T23:58:35.531 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:35.531 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:35.531 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:35.531 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:35.531 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:35.531 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:35.531 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5884 2024-09-14T23:58:35.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.531 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:35.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:35.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:58:35.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:35.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:35.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:58:35.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:35.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:35.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:58:35.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0800000000000001e-07 2024-09-14T23:58:35.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:35.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375852+0000", 2024-09-14T23:58:35.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.02e-06 2024-09-14T23:58:35.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:35.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375855+0000", 2024-09-14T23:58:35.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9900000000000002e-06 2024-09-14T23:58:35.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:35.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375857+0000", 2024-09-14T23:58:35.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3499999999999999e-06 2024-09-14T23:58:35.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:35.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426556+0000", 2024-09-14T23:58:35.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050698139000000003 2024-09-14T23:58:35.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:35.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426568+0000", 2024-09-14T23:58:35.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2731999999999999e-05 2024-09-14T23:58:35.534 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:35.534 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:35.534 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:35.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.534 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5885 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:35.534 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:58:35.534 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 116.14853448700001, 2024-09-14T23:58:35.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050737333000000003, 2024-09-14T23:58:35.534 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:35.534 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:35.534 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:35.534 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:35.535 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:35.535 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:35.535 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5885 2024-09-14T23:58:35.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.535 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:35.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:35.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:58:35.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:35.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:35.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:58:35.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:35.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:35.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:58:35.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5899999999999998e-07 2024-09-14T23:58:35.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:35.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375872+0000", 2024-09-14T23:58:35.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4219999999999999e-06 2024-09-14T23:58:35.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:35.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375878+0000", 2024-09-14T23:58:35.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4980000000000003e-06 2024-09-14T23:58:35.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:35.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375881+0000", 2024-09-14T23:58:35.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6110000000000001e-06 2024-09-14T23:58:35.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:35.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426596+0000", 2024-09-14T23:58:35.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050715320000000001 2024-09-14T23:58:35.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:35.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426607+0000", 2024-09-14T23:58:35.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1223000000000001e-05 2024-09-14T23:58:35.537 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:35.537 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:35.537 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:35.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.538 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5886 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:35.538 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:58:35.538 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 116.148520374, 2024-09-14T23:58:35.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050803395000000001, 2024-09-14T23:58:35.538 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:35.538 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:35.538 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:35.538 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:35.538 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:35.538 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:35.538 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5886 2024-09-14T23:58:35.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.538 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:35.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:35.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:58:35.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:35.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:35.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:58:35.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:35.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:35.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:58:35.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2700000000000001e-07 2024-09-14T23:58:35.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:35.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375886+0000", 2024-09-14T23:58:35.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7060000000000001e-06 2024-09-14T23:58:35.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:35.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375890+0000", 2024-09-14T23:58:35.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2259999999999999e-06 2024-09-14T23:58:35.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:35.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375893+0000", 2024-09-14T23:58:35.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0819999999999999e-06 2024-09-14T23:58:35.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:35.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426677+0000", 2024-09-14T23:58:35.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050784158000000003 2024-09-14T23:58:35.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:35.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426687+0000", 2024-09-14T23:58:35.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9960000000000005e-06 2024-09-14T23:58:35.541 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:35.541 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:35.541 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:35.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.541 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5887 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:35.541 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:58:35.541 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 116.148498288, 2024-09-14T23:58:35.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050833070000000001, 2024-09-14T23:58:35.542 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:35.542 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:35.542 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:35.542 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:35.542 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:35.542 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:35.542 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5887 2024-09-14T23:58:35.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.542 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:35.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:35.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:58:35.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:35.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:35.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:58:35.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:35.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:35.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:58:35.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3299999999999998e-07 2024-09-14T23:58:35.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:35.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375908+0000", 2024-09-14T23:58:35.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7689999999999999e-06 2024-09-14T23:58:35.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:35.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375912+0000", 2024-09-14T23:58:35.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.535e-06 2024-09-14T23:58:35.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:35.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375916+0000", 2024-09-14T23:58:35.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8979999999999998e-06 2024-09-14T23:58:35.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:35.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426729+0000", 2024-09-14T23:58:35.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050813453000000001 2024-09-14T23:58:35.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:35.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426739+0000", 2024-09-14T23:58:35.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0081999999999999e-05 2024-09-14T23:58:35.545 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:35.545 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:35.545 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:35.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.545 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5888 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:35.545 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:58:35.545 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 116.14848521099999, 2024-09-14T23:58:35.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050854652, 2024-09-14T23:58:35.545 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:35.545 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:35.545 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:35.545 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:35.545 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:35.545 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:35.546 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5888 2024-09-14T23:58:35.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.546 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:35.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:35.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:58:35.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:35.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:35.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:58:35.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:35.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:35.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:58:35.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1400000000000001e-07 2024-09-14T23:58:35.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:35.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375921+0000", 2024-09-14T23:58:35.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.119e-06 2024-09-14T23:58:35.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:35.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375928+0000", 2024-09-14T23:58:35.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3170000000000004e-06 2024-09-14T23:58:35.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:35.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375931+0000", 2024-09-14T23:58:35.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6970000000000002e-06 2024-09-14T23:58:35.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:35.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426768+0000", 2024-09-14T23:58:35.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050837355000000001 2024-09-14T23:58:35.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:35.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426774+0000", 2024-09-14T23:58:35.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9499999999999998e-06 2024-09-14T23:58:35.548 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:35.548 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:35.548 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:35.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.549 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5889 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:35.549 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:58:35.549 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 116.148468287, 2024-09-14T23:58:35.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050885446000000001, 2024-09-14T23:58:35.549 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:35.549 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:35.549 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:35.549 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:35.549 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:35.549 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:35.549 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5889 2024-09-14T23:58:35.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.549 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:35.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:35.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:58:35.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:35.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:35.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:58:35.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:35.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:35.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:58:35.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.36e-07 2024-09-14T23:58:35.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:35.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375940+0000", 2024-09-14T23:58:35.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.844e-06 2024-09-14T23:58:35.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:35.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375944+0000", 2024-09-14T23:58:35.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6720000000000002e-06 2024-09-14T23:58:35.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:35.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375946+0000", 2024-09-14T23:58:35.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5629999999999999e-06 2024-09-14T23:58:35.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:35.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426810+0000", 2024-09-14T23:58:35.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050863255000000003 2024-09-14T23:58:35.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:35.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426822+0000", 2024-09-14T23:58:35.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1875999999999999e-05 2024-09-14T23:58:35.552 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:35.552 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:35.552 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:35.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.552 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5890 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:35.552 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:58:35.552 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 116.14845447499999, 2024-09-14T23:58:35.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050917549999999999, 2024-09-14T23:58:35.553 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:35.553 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:35.553 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:35.553 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:35.553 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:35.553 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:35.553 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5890 2024-09-14T23:58:35.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.553 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:35.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:35.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:58:35.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:35.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:35.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:58:35.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:35.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:35.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:58:35.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4299999999999999e-07 2024-09-14T23:58:35.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:35.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375951+0000", 2024-09-14T23:58:35.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1000000000000001e-06 2024-09-14T23:58:35.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:35.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375956+0000", 2024-09-14T23:58:35.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3370000000000001e-06 2024-09-14T23:58:35.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:35.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375958+0000", 2024-09-14T23:58:35.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4820000000000001e-06 2024-09-14T23:58:35.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:35.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426858+0000", 2024-09-14T23:58:35.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050899666000000003 2024-09-14T23:58:35.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:35.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426867+0000", 2024-09-14T23:58:35.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7219999999999994e-06 2024-09-14T23:58:35.556 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:35.556 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:35.556 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:35.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.556 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5891 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:35.556 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:58:35.556 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 116.148443153, 2024-09-14T23:58:35.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050948067, 2024-09-14T23:58:35.556 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:35.556 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:35.556 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:35.557 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:35.557 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:35.557 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:35.557 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5891 2024-09-14T23:58:35.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.557 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:35.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:35.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:58:35.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:35.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:35.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:58:35.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:35.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:35.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:58:35.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0800000000000001e-07 2024-09-14T23:58:35.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:35.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375962+0000", 2024-09-14T23:58:35.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9600000000000008e-07 2024-09-14T23:58:35.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:35.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375970+0000", 2024-09-14T23:58:35.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1600000000000001e-06 2024-09-14T23:58:35.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:35.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375972+0000", 2024-09-14T23:58:35.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4849999999999999e-06 2024-09-14T23:58:35.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:35.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426902+0000", 2024-09-14T23:58:35.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050929813999999997 2024-09-14T23:58:35.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:35.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426909+0000", 2024-09-14T23:58:35.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4039999999999998e-06 2024-09-14T23:58:35.559 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:35.559 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:35.559 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:35.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.560 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5892 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:35.560 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:58:35.560 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 116.14842829600001, 2024-09-14T23:58:35.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050965036999999998, 2024-09-14T23:58:35.560 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:35.560 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:35.560 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:35.560 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:35.560 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:35.560 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:35.560 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5892 2024-09-14T23:58:35.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.560 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:35.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:35.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:58:35.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:35.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:35.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:58:35.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:35.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:35.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:58:35.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.34e-07 2024-09-14T23:58:35.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:35.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375978+0000", 2024-09-14T23:58:35.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1880000000000001e-06 2024-09-14T23:58:35.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:35.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375981+0000", 2024-09-14T23:58:35.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2399999999999999e-06 2024-09-14T23:58:35.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:35.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375984+0000", 2024-09-14T23:58:35.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9110000000000002e-06 2024-09-14T23:58:35.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:35.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426934+0000", 2024-09-14T23:58:35.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050949912 2024-09-14T23:58:35.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:35.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426941+0000", 2024-09-14T23:58:35.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5519999999999999e-06 2024-09-14T23:58:35.563 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:35.563 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:35.563 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:35.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.563 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5893 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:35.563 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:58:35.563 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 116.14841769900001, 2024-09-14T23:58:35.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050975131999999999, 2024-09-14T23:58:35.563 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:35.564 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:35.564 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:35.564 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:35.564 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:35.564 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:35.564 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5893 2024-09-14T23:58:35.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.564 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:35.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:35.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:58:35.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:35.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:35.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:58:35.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:35.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:35.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:58:35.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2999999999999999e-07 2024-09-14T23:58:35.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:35.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375988+0000", 2024-09-14T23:58:35.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0410000000000001e-06 2024-09-14T23:58:35.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:35.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375992+0000", 2024-09-14T23:58:35.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.104e-06 2024-09-14T23:58:35.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:35.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375995+0000", 2024-09-14T23:58:35.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4310000000000001e-06 2024-09-14T23:58:35.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:35.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426958+0000", 2024-09-14T23:58:35.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050963642000000003 2024-09-14T23:58:35.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:35.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426962+0000", 2024-09-14T23:58:35.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6839999999999998e-06 2024-09-14T23:58:35.569 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:35.569 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:35.569 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:35.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.569 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5894 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:35.570 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:58:35.570 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 116.148406299, 2024-09-14T23:58:35.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050978333000000001, 2024-09-14T23:58:35.570 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:35.570 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:35.570 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:35.570 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:35.570 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:35.570 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:35.570 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5894 2024-09-14T23:58:35.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.570 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:35.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:35.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:58:35.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:35.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:35.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:58:35.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:35.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:35.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:58:35.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.23e-07 2024-09-14T23:58:35.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:35.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375999+0000", 2024-09-14T23:58:35.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9800000000000002e-07 2024-09-14T23:58:35.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:35.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376007+0000", 2024-09-14T23:58:35.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8539999999999997e-06 2024-09-14T23:58:35.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:35.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376011+0000", 2024-09-14T23:58:35.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.399e-06 2024-09-14T23:58:35.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:35.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426973+0000", 2024-09-14T23:58:35.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050962454999999997 2024-09-14T23:58:35.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:35.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426976+0000", 2024-09-14T23:58:35.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4039999999999999e-06 2024-09-14T23:58:35.573 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:35.573 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:35.573 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:35.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.573 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5895 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:35.573 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376014+0000", 2024-09-14T23:58:35.573 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 116.148389955, 2024-09-14T23:58:35.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050975951999999998, 2024-09-14T23:58:35.573 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:35.573 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:35.573 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:35.573 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:35.574 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:35.574 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:35.574 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5895 2024-09-14T23:58:35.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.574 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:35.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:35.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376014+0000", 2024-09-14T23:58:35.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:35.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:35.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376014+0000", 2024-09-14T23:58:35.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:35.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:35.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376015+0000", 2024-09-14T23:58:35.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2999999999999999e-07 2024-09-14T23:58:35.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:35.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376016+0000", 2024-09-14T23:58:35.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1659999999999999e-06 2024-09-14T23:58:35.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:35.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376024+0000", 2024-09-14T23:58:35.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8930000000000005e-06 2024-09-14T23:58:35.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:35.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376026+0000", 2024-09-14T23:58:35.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6869999999999999e-06 2024-09-14T23:58:35.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:35.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426987+0000", 2024-09-14T23:58:35.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050960601000000001 2024-09-14T23:58:35.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:35.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426990+0000", 2024-09-14T23:58:35.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3749999999999999e-06 2024-09-14T23:58:35.576 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:35.576 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:35.576 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:35.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.577 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5896 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:35.577 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:58:35.577 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 116.148374218, 2024-09-14T23:58:35.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050975656000000001, 2024-09-14T23:58:35.577 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:35.577 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:35.577 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:35.577 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:35.577 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:35.577 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:35.577 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5896 2024-09-14T23:58:35.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.577 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:35.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:35.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:58:35.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:35.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:35.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:58:35.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:35.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:35.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:58:35.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5100000000000001e-07 2024-09-14T23:58:35.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:35.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376032+0000", 2024-09-14T23:58:35.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.141e-06 2024-09-14T23:58:35.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:35.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376036+0000", 2024-09-14T23:58:35.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.25e-06 2024-09-14T23:58:35.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:35.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376038+0000", 2024-09-14T23:58:35.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5469999999999999e-06 2024-09-14T23:58:35.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:35.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427002+0000", 2024-09-14T23:58:35.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050964029000000001 2024-09-14T23:58:35.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:35.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427006+0000", 2024-09-14T23:58:35.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4379999999999999e-06 2024-09-14T23:58:35.580 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:35.580 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:35.580 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:35.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.580 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5897 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:35.580 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:58:35.580 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 116.148362418, 2024-09-14T23:58:35.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050978001000000002, 2024-09-14T23:58:35.581 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:35.581 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:35.581 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:35.581 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:35.581 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:35.581 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:35.581 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5897 2024-09-14T23:58:35.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.581 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:35.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:35.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:58:35.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:35.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:35.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:58:35.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:35.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:35.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:58:35.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2600000000000001e-07 2024-09-14T23:58:35.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:35.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376043+0000", 2024-09-14T23:58:35.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.054e-06 2024-09-14T23:58:35.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:35.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376047+0000", 2024-09-14T23:58:35.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0659999999999997e-06 2024-09-14T23:58:35.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:35.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376050+0000", 2024-09-14T23:58:35.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.88e-06 2024-09-14T23:58:35.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:35.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427017+0000", 2024-09-14T23:58:35.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050966453000000002 2024-09-14T23:58:35.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:35.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427020+0000", 2024-09-14T23:58:35.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3220000000000001e-06 2024-09-14T23:58:35.584 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:35.584 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:35.584 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:35.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.584 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5898 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:35.584 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376053+0000", 2024-09-14T23:58:35.584 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 116.148350933, 2024-09-14T23:58:35.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050982024000000001, 2024-09-14T23:58:35.584 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:35.584 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:35.584 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:35.585 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:35.585 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:35.585 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:35.585 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5898 2024-09-14T23:58:35.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.585 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:35.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:35.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376053+0000", 2024-09-14T23:58:35.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:35.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:35.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376053+0000", 2024-09-14T23:58:35.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:35.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:35.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376278+0000", 2024-09-14T23:58:35.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022426400000000001 2024-09-14T23:58:35.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:35.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376282+0000", 2024-09-14T23:58:35.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4739999999999999e-06 2024-09-14T23:58:35.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:35.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376296+0000", 2024-09-14T23:58:35.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4229999999999999e-05 2024-09-14T23:58:35.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:35.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376303+0000", 2024-09-14T23:58:35.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2199999999999997e-06 2024-09-14T23:58:35.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:35.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427031+0000", 2024-09-14T23:58:35.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050728154999999997 2024-09-14T23:58:35.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:35.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427036+0000", 2024-09-14T23:58:35.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6809999999999997e-06 2024-09-14T23:58:35.587 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:35.587 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:35.588 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:35.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.588 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5899 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:35.588 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376338+0000", 2024-09-14T23:58:35.588 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 116.14806620900001, 2024-09-14T23:58:35.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050712100000000003, 2024-09-14T23:58:35.588 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:35.588 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:35.588 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:35.588 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:35.588 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:35.588 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:35.588 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5899 2024-09-14T23:58:35.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.589 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:35.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:35.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376338+0000", 2024-09-14T23:58:35.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:35.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:35.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376338+0000", 2024-09-14T23:58:35.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:35.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:35.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376339+0000", 2024-09-14T23:58:35.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9100000000000003e-07 2024-09-14T23:58:35.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:35.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376342+0000", 2024-09-14T23:58:35.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7e-06 2024-09-14T23:58:35.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:35.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376350+0000", 2024-09-14T23:58:35.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5280000000000003e-06 2024-09-14T23:58:35.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:35.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376356+0000", 2024-09-14T23:58:35.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6010000000000002e-06 2024-09-14T23:58:35.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:35.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427047+0000", 2024-09-14T23:58:35.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050690962999999999 2024-09-14T23:58:35.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:35.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427050+0000", 2024-09-14T23:58:35.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.517e-06 2024-09-14T23:58:35.591 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:35.591 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:35.591 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:35.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.591 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5900 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:35.592 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376366+0000", 2024-09-14T23:58:35.592 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 116.14803795, 2024-09-14T23:58:35.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050697858999999998, 2024-09-14T23:58:35.592 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:35.592 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:35.592 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:35.592 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:35.592 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:35.592 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:35.592 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5900 2024-09-14T23:58:35.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.592 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:35.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:35.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376366+0000", 2024-09-14T23:58:35.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:35.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:35.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376366+0000", 2024-09-14T23:58:35.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:35.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:35.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377650+0000", 2024-09-14T23:58:35.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001283834 2024-09-14T23:58:35.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:35.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377655+0000", 2024-09-14T23:58:35.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7230000000000004e-06 2024-09-14T23:58:35.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:35.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377667+0000", 2024-09-14T23:58:35.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1746000000000001e-05 2024-09-14T23:58:35.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:35.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377676+0000", 2024-09-14T23:58:35.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3619999999999995e-06 2024-09-14T23:58:35.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:35.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427061+0000", 2024-09-14T23:58:35.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049384755000000002 2024-09-14T23:58:35.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:35.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427064+0000", 2024-09-14T23:58:35.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.439e-06 2024-09-14T23:58:35.595 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:35.595 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:35.595 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:35.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.595 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:132622 3.1e 3:79d0a5e0:::rbd_data.111847077d5e.0000000000000007:head [sparse-read 0~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T23:58:35.596 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:44.037901+0000", 2024-09-14T23:58:35.596 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 51.486503341000002, 2024-09-14T23:58:35.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1083420859999999, 2024-09-14T23:58:35.596 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:35.596 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:35.596 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:35.596 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:35.596 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:35.596 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:35.596 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 132622 2024-09-14T23:58:35.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.596 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:35.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:35.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037901+0000", 2024-09-14T23:58:35.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:35.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:35.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037901+0000", 2024-09-14T23:58:35.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:35.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:35.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037903+0000", 2024-09-14T23:58:35.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4619999999999999e-06 2024-09-14T23:58:35.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:35.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037910+0000", 2024-09-14T23:58:35.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7449999999999995e-06 2024-09-14T23:58:35.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:35.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037928+0000", 2024-09-14T23:58:35.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7305000000000001e-05 2024-09-14T23:58:35.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:35.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037939+0000", 2024-09-14T23:58:35.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1599e-05 2024-09-14T23:58:35.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:35.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.038049+0000", 2024-09-14T23:58:35.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000109577 2024-09-14T23:58:35.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op must wait for map", 2024-09-14T23:58:35.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.038057+0000", 2024-09-14T23:58:35.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1450000000000008e-06 2024-09-14T23:58:35.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:35.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.039937+0000", 2024-09-14T23:58:35.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00188024 2024-09-14T23:58:35.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:58:35.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.040039+0000", 2024-09-14T23:58:35.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000101934 2024-09-14T23:58:35.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:35.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:45.145805+0000", 2024-09-14T23:58:35.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.105765651 2024-09-14T23:58:35.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:35.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:45.145997+0000", 2024-09-14T23:58:35.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000192084 2024-09-14T23:58:35.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:35.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:45.146243+0000", 2024-09-14T23:58:35.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024634400000000001 2024-09-14T23:58:35.601 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:35.601 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:35.601 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:35.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.601 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4172.0:5 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:35.601 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.482606+0000", 2024-09-14T23:58:35.601 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.041798360000001, 2024-09-14T23:58:35.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.49381641300000001, 2024-09-14T23:58:35.601 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:35.601 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:35.601 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:35.601 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:35.601 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4172", 2024-09-14T23:58:35.601 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/2542269203", 2024-09-14T23:58:35.602 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5 2024-09-14T23:58:35.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.602 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:35.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:35.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482606+0000", 2024-09-14T23:58:35.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:35.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:35.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482606+0000", 2024-09-14T23:58:35.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:35.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:35.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482607+0000", 2024-09-14T23:58:35.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3120000000000001e-06 2024-09-14T23:58:35.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:35.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482615+0000", 2024-09-14T23:58:35.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9300000000000003e-06 2024-09-14T23:58:35.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:35.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482632+0000", 2024-09-14T23:58:35.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6966999999999999e-05 2024-09-14T23:58:35.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:35.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482640+0000", 2024-09-14T23:58:35.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1820000000000006e-06 2024-09-14T23:58:35.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:35.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482703+0000", 2024-09-14T23:58:35.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2098000000000001e-05 2024-09-14T23:58:35.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:58:35.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482713+0000", 2024-09-14T23:58:35.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0064000000000001e-05 2024-09-14T23:58:35.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:35.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.976307+0000", 2024-09-14T23:58:35.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.49359442999999997 2024-09-14T23:58:35.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:35.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.976422+0000", 2024-09-14T23:58:35.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011543 2024-09-14T23:58:35.605 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:35.605 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:35.605 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:35.605 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:35.605 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:35.605 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:35.606 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T23:58:35.607 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:35.605+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:35.707 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:58:35.706+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:58:35.755 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:35.755 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:35.755 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:58:35.755 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:35.764 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T23:58:35.808 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:35.807+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:35.871 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:35.871 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:35.871 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:58:35.871 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:58:35.871 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:35.880 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T23:58:35.908 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:58:35.907+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:58:35.991 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:35.991 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:58:35.991 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:58:35.991 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:58:35.991 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.991 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:3578 3.4 e29/23 3:2298d860:::rbd_data.1118de1168b.0000000000000030:head v 29'15, mlcod=29'13)", 2024-09-14T23:58:35.991 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.036455+0000", 2024-09-14T23:58:35.991 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 116.951568255, 2024-09-14T23:58:35.991 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019449440000000001, 2024-09-14T23:58:35.991 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:35.991 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:35.992 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:35.992 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:35.992 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.992 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:35.992 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036455+0000", 2024-09-14T23:58:35.992 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:35.992 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.992 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.992 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:35.992 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036455+0000", 2024-09-14T23:58:35.993 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:35.993 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.993 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.993 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:35.993 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036456+0000", 2024-09-14T23:58:35.993 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0999999999999997e-07 2024-09-14T23:58:35.993 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.993 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.993 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:35.993 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036463+0000", 2024-09-14T23:58:35.993 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2200000000000003e-06 2024-09-14T23:58:35.993 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.993 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.993 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:35.994 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036478+0000", 2024-09-14T23:58:35.994 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4977000000000001e-05 2024-09-14T23:58:35.994 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.994 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.994 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:35.994 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036485+0000", 2024-09-14T23:58:35.994 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0559999999999999e-06 2024-09-14T23:58:35.994 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.994 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.994 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:35.994 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036693+0000", 2024-09-14T23:58:35.995 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000208051 2024-09-14T23:58:35.995 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.995 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.995 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:35.995 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.036715+0000", 2024-09-14T23:58:35.995 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1330000000000001e-05 2024-09-14T23:58:35.995 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.995 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.995 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:35.995 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.038356+0000", 2024-09-14T23:58:35.995 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001641586 2024-09-14T23:58:35.995 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.995 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.995 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:35.995 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.038400+0000", 2024-09-14T23:58:35.996 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3914e-05 2024-09-14T23:58:35.996 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:35.996 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:35.996 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:35.996 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.996 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.996 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:3579 3.4 e29/23 3:27290fe6:::rbd_data.1118de1168b.0000000000000032:head v 29'16, mlcod=29'13)", 2024-09-14T23:58:35.996 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.037085+0000", 2024-09-14T23:58:35.996 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 116.950938665, 2024-09-14T23:58:35.996 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0020019220000000002, 2024-09-14T23:58:35.996 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:35.997 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:35.997 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:35.997 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:35.997 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.997 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:35.997 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037085+0000", 2024-09-14T23:58:35.997 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:35.997 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.997 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.997 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:35.997 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037085+0000", 2024-09-14T23:58:35.997 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:35.998 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.998 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.998 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:35.998 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037085+0000", 2024-09-14T23:58:35.998 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5499999999999997e-07 2024-09-14T23:58:35.998 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.998 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.998 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:35.998 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037091+0000", 2024-09-14T23:58:35.998 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.7869999999999998e-06 2024-09-14T23:58:35.998 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.998 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.998 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:35.998 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037102+0000", 2024-09-14T23:58:35.998 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.031e-05 2024-09-14T23:58:35.999 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.999 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.999 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:35.999 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037116+0000", 2024-09-14T23:58:35.999 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4119e-05 2024-09-14T23:58:35.999 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.999 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.999 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:35.999 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037219+0000", 2024-09-14T23:58:35.999 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010372799999999999 2024-09-14T23:58:35.999 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:35.999 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:35.999 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:35.999 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.037239+0000", 2024-09-14T23:58:35.999 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9993e-05 2024-09-14T23:58:36.000 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.000 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.000 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:36.000 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.039046+0000", 2024-09-14T23:58:36.000 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001806666 2024-09-14T23:58:36.000 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.000 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.000 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:36.000 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.039087+0000", 2024-09-14T23:58:36.000 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0564e-05 2024-09-14T23:58:36.000 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.000 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.000 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.000 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.000 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.001 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:32024 3.9 e39/23 3:94dc260b:::rbd_data.111814b482f8.0000000000000009:head v 39'45, mlcod=38'43)", 2024-09-14T23:58:36.001 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:49.061018+0000", 2024-09-14T23:58:36.001 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 106.927005326, 2024-09-14T23:58:36.001 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018482240000000001, 2024-09-14T23:58:36.001 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:36.001 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:36.001 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:36.001 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:36.001 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.001 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:36.001 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061018+0000", 2024-09-14T23:58:36.001 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.001 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.001 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.001 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:36.002 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061018+0000", 2024-09-14T23:58:36.002 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.002 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.002 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.002 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:36.002 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061018+0000", 2024-09-14T23:58:36.002 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9999999999999998e-07 2024-09-14T23:58:36.002 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.002 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.002 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:36.002 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061211+0000", 2024-09-14T23:58:36.002 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019206 2024-09-14T23:58:36.002 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.002 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.003 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:36.003 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061221+0000", 2024-09-14T23:58:36.003 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0659000000000001e-05 2024-09-14T23:58:36.003 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.003 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.003 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:36.003 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061230+0000", 2024-09-14T23:58:36.003 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4610000000000008e-06 2024-09-14T23:58:36.003 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.003 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.003 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:36.003 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061317+0000", 2024-09-14T23:58:36.003 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7514000000000003e-05 2024-09-14T23:58:36.003 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.003 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.004 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:36.004 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.061338+0000", 2024-09-14T23:58:36.004 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0395999999999999e-05 2024-09-14T23:58:36.004 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.004 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.004 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:36.004 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.062830+0000", 2024-09-14T23:58:36.004 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0014921940000000001 2024-09-14T23:58:36.004 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.005 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.005 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:36.005 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:49.062866+0000", 2024-09-14T23:58:36.005 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6539999999999999e-05 2024-09-14T23:58:36.005 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.005 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.005 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.005 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.005 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.005 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:51281 3.0 e52/23 3:04e60ec2:::rbd_data.11181975a1de.0000000000000003:head v 52'113, mlcod=44'111)", 2024-09-14T23:58:36.005 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:02.158822+0000", 2024-09-14T23:58:36.005 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 93.829201139999995, 2024-09-14T23:58:36.006 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019860609999999999, 2024-09-14T23:58:36.006 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:36.006 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:36.006 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:36.006 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:36.006 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.006 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:36.006 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.158822+0000", 2024-09-14T23:58:36.006 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.006 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.006 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.006 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:36.006 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.158822+0000", 2024-09-14T23:58:36.006 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.006 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.007 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.007 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:36.007 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.158823+0000", 2024-09-14T23:58:36.007 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6800000000000001e-07 2024-09-14T23:58:36.007 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.007 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.007 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:36.007 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.159132+0000", 2024-09-14T23:58:36.007 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00030974900000000001 2024-09-14T23:58:36.007 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.007 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.007 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:36.007 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.159146+0000", 2024-09-14T23:58:36.007 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3261000000000001e-05 2024-09-14T23:58:36.008 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.008 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.008 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:36.008 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.159155+0000", 2024-09-14T23:58:36.008 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7410000000000004e-06 2024-09-14T23:58:36.008 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.008 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.008 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:36.008 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.159221+0000", 2024-09-14T23:58:36.008 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5698999999999996e-05 2024-09-14T23:58:36.008 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.008 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.008 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:36.008 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.159244+0000", 2024-09-14T23:58:36.009 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3108e-05 2024-09-14T23:58:36.009 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.009 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.009 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:36.009 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.160767+0000", 2024-09-14T23:58:36.009 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015231120000000001 2024-09-14T23:58:36.009 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.009 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.009 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:36.009 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:02.160808+0000", 2024-09-14T23:58:36.009 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1022999999999997e-05 2024-09-14T23:58:36.009 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.009 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.009 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.010 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.010 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.010 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:56107 3.0 e56/23 3:00c47890:::rbd_data.1118c4b3aebe.0000000000000009:head v 56'125, mlcod=52'123)", 2024-09-14T23:58:36.010 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.236123+0000", 2024-09-14T23:58:36.010 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 89.751899941000005, 2024-09-14T23:58:36.010 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0061273220000000002, 2024-09-14T23:58:36.010 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:36.010 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:36.010 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:36.010 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:36.010 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.010 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:36.010 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.236123+0000", 2024-09-14T23:58:36.010 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.010 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.011 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.011 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:36.011 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.236123+0000", 2024-09-14T23:58:36.011 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.011 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.011 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.011 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:36.011 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.236125+0000", 2024-09-14T23:58:36.011 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.313e-06 2024-09-14T23:58:36.011 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.011 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.011 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:36.011 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237428+0000", 2024-09-14T23:58:36.011 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013029890000000001 2024-09-14T23:58:36.011 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.012 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.012 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:36.012 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237452+0000", 2024-09-14T23:58:36.012 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4090000000000001e-05 2024-09-14T23:58:36.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.012 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.012 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:36.012 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237462+0000", 2024-09-14T23:58:36.012 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0421999999999999e-05 2024-09-14T23:58:36.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.012 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.012 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:36.012 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237559+0000", 2024-09-14T23:58:36.012 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6378000000000005e-05 2024-09-14T23:58:36.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.013 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.013 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:36.013 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237583+0000", 2024-09-14T23:58:36.013 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4182e-05 2024-09-14T23:58:36.013 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.013 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.013 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:36.013 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.242205+0000", 2024-09-14T23:58:36.013 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0046217289999999998 2024-09-14T23:58:36.013 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.013 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.013 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:36.013 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.242251+0000", 2024-09-14T23:58:36.013 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6218999999999999e-05 2024-09-14T23:58:36.014 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.014 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.014 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.014 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.014 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.014 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:56108 3.0 e56/23 3:00c47890:::rbd_data.1118c4b3aebe.0000000000000009:head v 56'127, mlcod=52'123)", 2024-09-14T23:58:36.014 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.239693+0000", 2024-09-14T23:58:36.014 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 89.748330471000003, 2024-09-14T23:58:36.014 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0043496209999999997, 2024-09-14T23:58:36.014 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:36.015 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:36.015 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:36.015 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:36.015 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.015 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:36.015 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239693+0000", 2024-09-14T23:58:36.015 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.015 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.015 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:36.015 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239693+0000", 2024-09-14T23:58:36.015 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.015 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.016 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:36.016 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239694+0000", 2024-09-14T23:58:36.016 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0499999999999999e-06 2024-09-14T23:58:36.016 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.016 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.016 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:36.016 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239702+0000", 2024-09-14T23:58:36.016 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1170000000000004e-06 2024-09-14T23:58:36.016 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.016 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.017 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:36.017 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239716+0000", 2024-09-14T23:58:36.017 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4306e-05 2024-09-14T23:58:36.017 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.017 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.017 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:36.017 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239724+0000", 2024-09-14T23:58:36.017 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0399999999999993e-06 2024-09-14T23:58:36.017 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.017 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.018 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:36.018 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239778+0000", 2024-09-14T23:58:36.018 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4032e-05 2024-09-14T23:58:36.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.018 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.018 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:36.018 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239801+0000", 2024-09-14T23:58:36.018 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2770000000000001e-05 2024-09-14T23:58:36.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.018 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.018 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:36.018 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.243990+0000", 2024-09-14T23:58:36.018 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0041890909999999998 2024-09-14T23:58:36.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.019 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.019 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:36.019 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.244043+0000", 2024-09-14T23:58:36.019 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2215000000000001e-05 2024-09-14T23:58:36.019 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.019 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.019 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.019 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.019 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.019 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:56385 3.9 e56/23 3:963d31c7:::rbd_data.1118c4b3aebe.0000000000000003:head v 56'87, mlcod=53'85)", 2024-09-14T23:58:36.019 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.283936+0000", 2024-09-14T23:58:36.019 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 89.704087560999994, 2024-09-14T23:58:36.019 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0063042080000000004, 2024-09-14T23:58:36.020 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:36.020 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:36.020 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:36.020 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:36.020 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.020 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:36.020 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.283936+0000", 2024-09-14T23:58:36.020 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.020 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.020 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.020 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:36.020 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.283936+0000", 2024-09-14T23:58:36.021 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.021 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.021 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.021 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:36.021 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.283937+0000", 2024-09-14T23:58:36.021 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1179999999999999e-06 2024-09-14T23:58:36.022 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.022 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.022 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:36.022 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286338+0000", 2024-09-14T23:58:36.022 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024010149999999998 2024-09-14T23:58:36.022 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.022 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.023 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:36.023 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286360+0000", 2024-09-14T23:58:36.023 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2212e-05 2024-09-14T23:58:36.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.023 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.023 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:36.023 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286370+0000", 2024-09-14T23:58:36.023 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5850000000000006e-06 2024-09-14T23:58:36.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.023 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.023 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:36.023 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286443+0000", 2024-09-14T23:58:36.023 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2757999999999997e-05 2024-09-14T23:58:36.024 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.024 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.024 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:36.024 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286466+0000", 2024-09-14T23:58:36.024 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3844000000000001e-05 2024-09-14T23:58:36.024 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.024 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.024 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:36.024 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.290197+0000", 2024-09-14T23:58:36.024 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003730176 2024-09-14T23:58:36.024 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.024 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.024 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:36.024 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.290240+0000", 2024-09-14T23:58:36.025 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.35e-05 2024-09-14T23:58:36.025 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.025 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.025 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.025 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.025 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.025 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:56386 3.9 e56/23 3:963d31c7:::rbd_data.1118c4b3aebe.0000000000000003:head v 56'89, mlcod=53'85)", 2024-09-14T23:58:36.025 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.287578+0000", 2024-09-14T23:58:36.025 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 89.700445834999996, 2024-09-14T23:58:36.025 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033580250000000002, 2024-09-14T23:58:36.025 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:36.025 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:36.025 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:36.025 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:36.026 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.026 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:36.026 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287578+0000", 2024-09-14T23:58:36.026 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.026 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.026 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.026 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:36.026 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287578+0000", 2024-09-14T23:58:36.026 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.026 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.026 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.026 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:36.027 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287579+0000", 2024-09-14T23:58:36.027 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.64e-07 2024-09-14T23:58:36.027 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.027 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.027 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:36.027 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287586+0000", 2024-09-14T23:58:36.027 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5310000000000004e-06 2024-09-14T23:58:36.027 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.027 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.027 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:36.027 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287597+0000", 2024-09-14T23:58:36.027 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0903999999999999e-05 2024-09-14T23:58:36.027 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.027 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.027 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:36.028 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287605+0000", 2024-09-14T23:58:36.028 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7889999999999995e-06 2024-09-14T23:58:36.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.028 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.028 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:36.028 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.288437+0000", 2024-09-14T23:58:36.028 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000831914 2024-09-14T23:58:36.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.028 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.028 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:36.028 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.288460+0000", 2024-09-14T23:58:36.028 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3410000000000001e-05 2024-09-14T23:58:36.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.029 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.029 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:36.029 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.290883+0000", 2024-09-14T23:58:36.029 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024230599999999999 2024-09-14T23:58:36.029 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.029 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.030 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:36.030 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.290936+0000", 2024-09-14T23:58:36.030 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2453000000000001e-05 2024-09-14T23:58:36.030 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.030 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.030 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.030 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.030 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.030 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:60299 3.9 e57/23 3:96f26b6f:::rbd_data.11188d356229.0000000000000009:head v 57'98, mlcod=56'96)", 2024-09-14T23:58:36.030 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:07.418336+0000", 2024-09-14T23:58:36.030 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 88.569687341999995, 2024-09-14T23:58:36.030 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022211459999999998, 2024-09-14T23:58:36.030 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:36.031 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:36.031 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:36.031 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:36.031 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.031 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:36.031 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.418336+0000", 2024-09-14T23:58:36.031 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.031 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.031 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.031 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:36.031 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.418336+0000", 2024-09-14T23:58:36.031 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.031 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.032 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.032 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:36.032 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.418337+0000", 2024-09-14T23:58:36.032 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7199999999999999e-07 2024-09-14T23:58:36.032 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.032 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.032 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:36.032 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.419396+0000", 2024-09-14T23:58:36.032 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0010598319999999999 2024-09-14T23:58:36.032 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.032 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.032 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:36.032 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.419407+0000", 2024-09-14T23:58:36.033 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0956e-05 2024-09-14T23:58:36.033 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.033 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.033 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:36.033 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.419417+0000", 2024-09-14T23:58:36.033 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0040999999999999e-05 2024-09-14T23:58:36.033 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.033 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.033 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:36.033 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.419468+0000", 2024-09-14T23:58:36.033 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0497000000000001e-05 2024-09-14T23:58:36.033 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.033 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.034 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:36.034 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.419485+0000", 2024-09-14T23:58:36.034 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6966e-05 2024-09-14T23:58:36.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.034 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.034 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:36.034 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.420527+0000", 2024-09-14T23:58:36.034 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001042133 2024-09-14T23:58:36.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.034 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.034 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:36.034 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.420557+0000", 2024-09-14T23:58:36.034 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0249000000000001e-05 2024-09-14T23:58:36.034 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.034 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.035 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.035 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.035 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.035 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:64263 3.4 e57/23 3:20c6a768:::rbd_data.11188d356229.0000000000000001:head v 57'165, mlcod=57'163)", 2024-09-14T23:58:36.035 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:07.999031+0000", 2024-09-14T23:58:36.035 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 87.988992687000007, 2024-09-14T23:58:36.035 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023815210000000002, 2024-09-14T23:58:36.035 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:36.035 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:36.035 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:36.035 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:36.035 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.035 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:36.035 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999031+0000", 2024-09-14T23:58:36.036 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.036 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.036 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:36.036 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999031+0000", 2024-09-14T23:58:36.036 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.036 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.036 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:36.036 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999032+0000", 2024-09-14T23:58:36.036 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.176e-06 2024-09-14T23:58:36.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.036 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.037 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:36.037 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999660+0000", 2024-09-14T23:58:36.037 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00062817099999999996 2024-09-14T23:58:36.037 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.037 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.037 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:36.037 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999683+0000", 2024-09-14T23:58:36.037 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2816999999999999e-05 2024-09-14T23:58:36.037 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.037 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.037 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:36.037 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999693+0000", 2024-09-14T23:58:36.037 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0088999999999999e-05 2024-09-14T23:58:36.037 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.038 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.038 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:36.038 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999767+0000", 2024-09-14T23:58:36.038 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4056999999999999e-05 2024-09-14T23:58:36.038 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.038 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.038 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:36.038 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999793+0000", 2024-09-14T23:58:36.038 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5573000000000001e-05 2024-09-14T23:58:36.038 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.038 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.038 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:36.038 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:08.001379+0000", 2024-09-14T23:58:36.038 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015859019999999999 2024-09-14T23:58:36.038 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.038 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.039 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:36.039 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:08.001412+0000", 2024-09-14T23:58:36.039 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3735999999999997e-05 2024-09-14T23:58:36.039 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.039 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.039 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.039 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.039 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:66570 3.0 e60/23 3:055350b3:::rbd_data.111873792d9b.0000000000000005:head v 60'162, mlcod=57'159)", 2024-09-14T23:58:36.039 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:11.277517+0000", 2024-09-14T23:58:36.039 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 84.710506530999993, 2024-09-14T23:58:36.039 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003472199, 2024-09-14T23:58:36.040 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:36.040 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:36.040 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:36.040 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:36.040 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.040 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:36.040 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.277517+0000", 2024-09-14T23:58:36.040 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.040 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.040 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:36.040 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.277517+0000", 2024-09-14T23:58:36.040 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.041 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.041 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:36.041 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.277518+0000", 2024-09-14T23:58:36.041 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1379999999999999e-06 2024-09-14T23:58:36.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.041 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.041 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:36.041 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278504+0000", 2024-09-14T23:58:36.041 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00098617499999999994 2024-09-14T23:58:36.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.041 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.041 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:36.041 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278525+0000", 2024-09-14T23:58:36.042 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.09e-05 2024-09-14T23:58:36.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.042 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.042 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:36.042 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278534+0000", 2024-09-14T23:58:36.042 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6209999999999993e-06 2024-09-14T23:58:36.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.042 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.042 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:36.042 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278611+0000", 2024-09-14T23:58:36.042 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6926000000000002e-05 2024-09-14T23:58:36.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.042 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.043 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:36.043 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278641+0000", 2024-09-14T23:58:36.043 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0861999999999997e-05 2024-09-14T23:58:36.043 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.043 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.043 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:36.043 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.280953+0000", 2024-09-14T23:58:36.043 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023114239999999999 2024-09-14T23:58:36.043 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.043 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.043 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:36.044 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.280989+0000", 2024-09-14T23:58:36.044 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6152999999999998e-05 2024-09-14T23:58:36.044 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.044 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.044 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.044 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.044 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.044 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "pg_update_log_missing(3.9 epoch 84/23 rep_tid 2234 entries 84'200 (0'0) error 3:93b58b1d:::rbd_data.1118417a1b0b.000000000000001b:head by client.4376.0:118637 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 83'197)", 2024-09-14T23:58:36.044 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:35.103156+0000", 2024-09-14T23:58:36.044 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 60.884867258, 2024-09-14T23:58:36.044 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019852009999999998, 2024-09-14T23:58:36.045 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:36.045 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:36.045 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:36.045 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:36.045 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.045 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:36.045 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:35.103156+0000", 2024-09-14T23:58:36.045 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.045 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.045 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:36.045 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:35.103156+0000", 2024-09-14T23:58:36.045 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.046 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:36.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:35.103157+0000", 2024-09-14T23:58:36.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1969999999999999e-06 2024-09-14T23:58:36.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.046 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:36.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:35.103170+0000", 2024-09-14T23:58:36.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2921e-05 2024-09-14T23:58:36.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.046 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:36.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:35.103202+0000", 2024-09-14T23:58:36.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1992000000000001e-05 2024-09-14T23:58:36.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:36.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:35.103220+0000", 2024-09-14T23:58:36.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7312999999999998e-05 2024-09-14T23:58:36.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:36.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:35.103305+0000", 2024-09-14T23:58:36.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4975000000000005e-05 2024-09-14T23:58:36.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:36.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:35.105141+0000", 2024-09-14T23:58:36.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018368029999999999 2024-09-14T23:58:36.047 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.047 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.048 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.048 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:142751 3.4 e99/23 3:265f7763:::rbd_data.1118f9c96a7.0000000000000004:head v 99'441, mlcod=98'438)", 2024-09-14T23:58:36.048 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:53.165301+0000", 2024-09-14T23:58:36.048 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.822722108000001, 2024-09-14T23:58:36.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0021754449999999998, 2024-09-14T23:58:36.048 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:36.048 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:36.048 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:36.048 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:36.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.048 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:36.048 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.165301+0000", 2024-09-14T23:58:36.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.049 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:36.049 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.165301+0000", 2024-09-14T23:58:36.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.049 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:36.049 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.165302+0000", 2024-09-14T23:58:36.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0899999999999997e-07 2024-09-14T23:58:36.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.050 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:36.050 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.165310+0000", 2024-09-14T23:58:36.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7520000000000002e-06 2024-09-14T23:58:36.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.050 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:36.050 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.165321+0000", 2024-09-14T23:58:36.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0668e-05 2024-09-14T23:58:36.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.050 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:36.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.165330+0000", 2024-09-14T23:58:36.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.234e-06 2024-09-14T23:58:36.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:36.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.165393+0000", 2024-09-14T23:58:36.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2716000000000001e-05 2024-09-14T23:58:36.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:36.052 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.165418+0000", 2024-09-14T23:58:36.052 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4544000000000001e-05 2024-09-14T23:58:36.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.052 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.052 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:36.052 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.167436+0000", 2024-09-14T23:58:36.053 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002018597 2024-09-14T23:58:36.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.053 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:36.053 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:53.167477+0000", 2024-09-14T23:58:36.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0624999999999998e-05 2024-09-14T23:58:36.054 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.054 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.054 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.054 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:158983 3.4 e103/23 3:20e43b29:::rbd_data.1118b2f495e6.0000000000000012:head v 103'454, mlcod=103'452)", 2024-09-14T23:58:36.054 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:58.917699+0000", 2024-09-14T23:58:36.054 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.070324481, 2024-09-14T23:58:36.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037933799999999998, 2024-09-14T23:58:36.055 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:36.055 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:36.055 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:36.055 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:36.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:36.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917699+0000", 2024-09-14T23:58:36.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:36.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917699+0000", 2024-09-14T23:58:36.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:36.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917700+0000", 2024-09-14T23:58:36.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3850000000000001e-06 2024-09-14T23:58:36.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.057 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:36.057 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917939+0000", 2024-09-14T23:58:36.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000238249 2024-09-14T23:58:36.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.057 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:36.057 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917961+0000", 2024-09-14T23:58:36.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2894000000000001e-05 2024-09-14T23:58:36.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.058 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:36.058 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917972+0000", 2024-09-14T23:58:36.058 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0261000000000001e-05 2024-09-14T23:58:36.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.058 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:36.058 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.918038+0000", 2024-09-14T23:58:36.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6223000000000006e-05 2024-09-14T23:58:36.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:36.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.918063+0000", 2024-09-14T23:58:36.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5252000000000001e-05 2024-09-14T23:58:36.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:36.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.921433+0000", 2024-09-14T23:58:36.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033699960000000001 2024-09-14T23:58:36.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:36.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.921492+0000", 2024-09-14T23:58:36.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9120000000000003e-05 2024-09-14T23:58:36.060 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.060 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.060 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.060 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4172.0:6 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-14T23:58:36.060 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.479840+0000", 2024-09-14T23:58:36.060 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.508183201000001, 2024-09-14T23:58:36.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022238340000000001, 2024-09-14T23:58:36.060 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:36.060 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:36.060 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:36.060 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:36.060 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4172", 2024-09-14T23:58:36.061 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/2542269203", 2024-09-14T23:58:36.061 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 6 2024-09-14T23:58:36.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.061 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:36.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.061 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:36.061 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.479840+0000", 2024-09-14T23:58:36.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.061 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:36.061 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.479840+0000", 2024-09-14T23:58:36.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.062 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:36.062 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.479843+0000", 2024-09-14T23:58:36.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6680000000000001e-06 2024-09-14T23:58:36.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.062 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:36.062 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.479862+0000", 2024-09-14T23:58:36.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.8762000000000001e-05 2024-09-14T23:58:36.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.062 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:36.062 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.479881+0000", 2024-09-14T23:58:36.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9860999999999998e-05 2024-09-14T23:58:36.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:36.063 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.479899+0000", 2024-09-14T23:58:36.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7127000000000001e-05 2024-09-14T23:58:36.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:36.063 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.481969+0000", 2024-09-14T23:58:36.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.002070166 2024-09-14T23:58:36.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:36.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482064+0000", 2024-09-14T23:58:36.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5249999999999998e-05 2024-09-14T23:58:36.064 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.064 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.064 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.064 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(osd.1.0:1460 3.0 e122/23 3:00cc1e32:::rbd_data.11187b9f1032.0000000000000001:2f v 122'453, mlcod=118'449)", 2024-09-14T23:58:36.064 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:22.287423+0000", 2024-09-14T23:58:36.064 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.700600824, 2024-09-14T23:58:36.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0046041659999999998, 2024-09-14T23:58:36.064 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:36.064 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:36.065 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:36.065 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:36.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:36.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.287423+0000", 2024-09-14T23:58:36.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:36.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.287423+0000", 2024-09-14T23:58:36.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:36.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.287423+0000", 2024-09-14T23:58:36.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7500000000000001e-07 2024-09-14T23:58:36.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:36.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.287435+0000", 2024-09-14T23:58:36.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2006e-05 2024-09-14T23:58:36.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:36.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.287444+0000", 2024-09-14T23:58:36.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.465e-06 2024-09-14T23:58:36.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:36.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.287453+0000", 2024-09-14T23:58:36.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0100000000000001e-06 2024-09-14T23:58:36.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:36.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.287519+0000", 2024-09-14T23:58:36.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6149999999999995e-05 2024-09-14T23:58:36.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:36.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.287543+0000", 2024-09-14T23:58:36.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3682000000000001e-05 2024-09-14T23:58:36.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:36.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.291977+0000", 2024-09-14T23:58:36.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0044333829999999999 2024-09-14T23:58:36.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:36.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.292027+0000", 2024-09-14T23:58:36.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0195000000000003e-05 2024-09-14T23:58:36.070 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.070 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.070 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.070 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDPGPush(3.1e 126/125 [PushOp(3:7f7265c3:::rbd_data.11188d356229.000000000000000d:16, version: 58'124, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:7f7265c3:::rbd_data.11188d356229.000000000000000d:16@58'124, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 16=[]:{16: [16]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T23:58:36.070 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:26.552687+0000", 2024-09-14T23:58:36.071 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.4353365599999997, 2024-09-14T23:58:36.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023212060000000001, 2024-09-14T23:58:36.071 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:36.071 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:36.071 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:36.071 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:36.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:36.071 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.552687+0000", 2024-09-14T23:58:36.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:36.071 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.552687+0000", 2024-09-14T23:58:36.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:36.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.552688+0000", 2024-09-14T23:58:36.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.47e-07 2024-09-14T23:58:36.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:36.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.553659+0000", 2024-09-14T23:58:36.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00097088899999999995 2024-09-14T23:58:36.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:36.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.553682+0000", 2024-09-14T23:58:36.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3862e-05 2024-09-14T23:58:36.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:36.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.553703+0000", 2024-09-14T23:58:36.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0310999999999999e-05 2024-09-14T23:58:36.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:36.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.553813+0000", 2024-09-14T23:58:36.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011051 2024-09-14T23:58:36.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:36.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.553828+0000", 2024-09-14T23:58:36.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4625e-05 2024-09-14T23:58:36.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:36.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.555008+0000", 2024-09-14T23:58:36.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001180162 2024-09-14T23:58:36.074 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.074 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.074 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.074 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDPGPush(3.5 126/125 [PushOp(3:a05d004f:::rbd_data.11188d356229.000000000000000b:16, version: 58'296, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:a05d004f:::rbd_data.11188d356229.000000000000000b:16@58'296, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 16=[]:{16: [16]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T23:58:36.074 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:27.212140+0000", 2024-09-14T23:58:36.074 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.77588306, 2024-09-14T23:58:36.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001842773, 2024-09-14T23:58:36.075 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:36.075 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:36.075 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:36.075 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:36.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:36.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.212140+0000", 2024-09-14T23:58:36.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:36.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.212140+0000", 2024-09-14T23:58:36.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:36.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.212141+0000", 2024-09-14T23:58:36.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.68e-07 2024-09-14T23:58:36.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:36.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.212695+0000", 2024-09-14T23:58:36.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00055376799999999995 2024-09-14T23:58:36.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:36.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.212712+0000", 2024-09-14T23:58:36.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7258e-05 2024-09-14T23:58:36.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:36.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.212725+0000", 2024-09-14T23:58:36.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3247999999999999e-05 2024-09-14T23:58:36.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:36.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.212807+0000", 2024-09-14T23:58:36.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2191e-05 2024-09-14T23:58:36.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:36.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.212815+0000", 2024-09-14T23:58:36.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6029999999999999e-06 2024-09-14T23:58:36.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:36.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.213983+0000", 2024-09-14T23:58:36.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001168037 2024-09-14T23:58:36.078 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.078 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.078 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.078 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(osd.1.0:1465 3.4 e127/23 3:265f7763:::rbd_data.1118f9c96a7.0000000000000004:2c v 127'508, mlcod=120'505)", 2024-09-14T23:58:36.078 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:27.318056+0000", 2024-09-14T23:58:36.078 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.6699671370000004, 2024-09-14T23:58:36.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0049337449999999998, 2024-09-14T23:58:36.079 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:36.079 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:36.079 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:36.079 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:36.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:36.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.318056+0000", 2024-09-14T23:58:36.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:36.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.318056+0000", 2024-09-14T23:58:36.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:36.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.318057+0000", 2024-09-14T23:58:36.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9799999999999999e-07 2024-09-14T23:58:36.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:36.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.318061+0000", 2024-09-14T23:58:36.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7899999999999999e-06 2024-09-14T23:58:36.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:36.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.318074+0000", 2024-09-14T23:58:36.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2537999999999999e-05 2024-09-14T23:58:36.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:36.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.318083+0000", 2024-09-14T23:58:36.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3829999999999998e-06 2024-09-14T23:58:36.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:36.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.318165+0000", 2024-09-14T23:58:36.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1928999999999994e-05 2024-09-14T23:58:36.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:36.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.318199+0000", 2024-09-14T23:58:36.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3587000000000002e-05 2024-09-14T23:58:36.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:36.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.322939+0000", 2024-09-14T23:58:36.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0047397730000000001 2024-09-14T23:58:36.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:36.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.322990+0000", 2024-09-14T23:58:36.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1347000000000002e-05 2024-09-14T23:58:36.082 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.082 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.082 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.083 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:252476 3.e 3:725cbf00:::rbd_header.11186acd4a72:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e132)", 2024-09-14T23:58:36.083 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:32.121781+0000", 2024-09-14T23:58:36.083 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 3.8662423000000001, 2024-09-14T23:58:36.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022671459999999998, 2024-09-14T23:58:36.083 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:36.083 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:36.083 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:36.083 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:36.083 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:36.083 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:36.083 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 252476 2024-09-14T23:58:36.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.083 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:36.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:36.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:32.121781+0000", 2024-09-14T23:58:36.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:36.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:32.121781+0000", 2024-09-14T23:58:36.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:36.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:32.121783+0000", 2024-09-14T23:58:36.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.531e-06 2024-09-14T23:58:36.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:36.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:32.121790+0000", 2024-09-14T23:58:36.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7670000000000003e-06 2024-09-14T23:58:36.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:36.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:32.121806+0000", 2024-09-14T23:58:36.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5291999999999999e-05 2024-09-14T23:58:36.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:36.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:32.121813+0000", 2024-09-14T23:58:36.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.418e-06 2024-09-14T23:58:36.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:36.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:32.121879+0000", 2024-09-14T23:58:36.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5811999999999999e-05 2024-09-14T23:58:36.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op must wait for map", 2024-09-14T23:58:36.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:32.121888+0000", 2024-09-14T23:58:36.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6600000000000001e-06 2024-09-14T23:58:36.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:36.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:32.123710+0000", 2024-09-14T23:58:36.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001822838 2024-09-14T23:58:36.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:36.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:32.123803+0000", 2024-09-14T23:58:36.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3077000000000003e-05 2024-09-14T23:58:36.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:36.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:32.124048+0000", 2024-09-14T23:58:36.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024475099999999997 2024-09-14T23:58:36.087 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.087 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.087 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.087 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.087 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.087 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:36.088 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 6, 5] out_osds: [7, 3] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T23:58:36.088 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T23:58:36.088 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-14T23:58:36.088 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T23:58:36.090 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T23:58:36.090 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:36.088+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:36.190 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:58:36.189+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:58:36.242 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:36.242 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:36.242 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:58:36.242 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:36.252 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T23:58:36.295 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:36.294+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:36.359 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:36.360 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:36.360 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:58:36.360 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:58:36.360 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:36.369 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T23:58:36.393 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:36.391+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:36.428 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:58:36.428 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:36.428 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 136, 2024-09-14T23:58:36.428 INFO:teuthology.orchestra.run.smithi042.stdout: "fsid": "21916fcc-f95c-4bfa-8b8b-11ca45845613", 2024-09-14T23:58:36.428 INFO:teuthology.orchestra.run.smithi042.stdout: "created": "2024-09-14T23:56:03.167948+0000", 2024-09-14T23:58:36.428 INFO:teuthology.orchestra.run.smithi042.stdout: "modified": "2024-09-14T23:58:35.982436+0000", 2024-09-14T23:58:36.428 INFO:teuthology.orchestra.run.smithi042.stdout: "last_up_change": "2024-09-14T23:56:14.322882+0000", 2024-09-14T23:58:36.429 INFO:teuthology.orchestra.run.smithi042.stdout: "last_in_change": "2024-09-14T23:58:24.287442+0000", 2024-09-14T23:58:36.429 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T23:58:36.429 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_num": 5806080, 2024-09-14T23:58:36.429 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_set": [ 2024-09-14T23:58:36.429 INFO:teuthology.orchestra.run.smithi042.stdout: "nodeep-scrub", 2024-09-14T23:58:36.429 INFO:teuthology.orchestra.run.smithi042.stdout: "noscrub", 2024-09-14T23:58:36.429 INFO:teuthology.orchestra.run.smithi042.stdout: "pglog_hardlimit", 2024-09-14T23:58:36.429 INFO:teuthology.orchestra.run.smithi042.stdout: "purged_snapdirs", 2024-09-14T23:58:36.429 INFO:teuthology.orchestra.run.smithi042.stdout: "recovery_deletes", 2024-09-14T23:58:36.429 INFO:teuthology.orchestra.run.smithi042.stdout: "sortbitwise" 2024-09-14T23:58:36.429 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-14T23:58:36.429 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_version": 6, 2024-09-14T23:58:36.430 INFO:teuthology.orchestra.run.smithi042.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T23:58:36.430 INFO:teuthology.orchestra.run.smithi042.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T23:58:36.430 INFO:teuthology.orchestra.run.smithi042.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T23:58:36.430 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_snapshot": "", 2024-09-14T23:58:36.430 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_max": 3, 2024-09-14T23:58:36.430 INFO:teuthology.orchestra.run.smithi042.stdout: "max_osd": 8, 2024-09-14T23:58:36.430 INFO:teuthology.orchestra.run.smithi042.stdout: "require_min_compat_client": "luminous", 2024-09-14T23:58:36.430 INFO:teuthology.orchestra.run.smithi042.stdout: "min_compat_client": "luminous", 2024-09-14T23:58:36.430 INFO:teuthology.orchestra.run.smithi042.stdout: "require_osd_release": "squid", 2024-09-14T23:58:36.430 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_crimson": false, 2024-09-14T23:58:36.430 INFO:teuthology.orchestra.run.smithi042.stdout: "pools": [ 2024-09-14T23:58:36.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.430 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 1, 2024-09-14T23:58:36.431 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": ".mgr", 2024-09-14T23:58:36.431 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-14T23:56:13.500197+0000", 2024-09-14T23:58:36.431 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-09-14T23:58:36.431 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool", 2024-09-14T23:58:36.431 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-14T23:58:36.431 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-14T23:58:36.431 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-14T23:58:36.431 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-14T23:58:36.431 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-14T23:58:36.431 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-14T23:58:36.431 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T23:58:36.431 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T23:58:36.431 INFO:teuthology.orchestra.run.smithi042.stdout: "is_stretch_pool": false, 2024-09-14T23:58:36.431 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-14T23:58:36.431 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-14T23:58:36.432 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 1, 2024-09-14T23:58:36.432 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 1, 2024-09-14T23:58:36.432 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 1, 2024-09-14T23:58:36.432 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 1, 2024-09-14T23:58:36.432 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 1, 2024-09-14T23:58:36.432 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-14T23:58:36.432 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-14T23:58:36.432 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-14T23:58:36.432 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-14T23:58:36.432 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-14T23:58:36.432 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-14T23:58:36.432 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-14T23:58:36.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.433 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "17", 2024-09-14T23:58:36.433 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "0", 2024-09-14T23:58:36.433 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T23:58:36.433 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T23:58:36.433 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-14T23:58:36.433 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-14T23:58:36.433 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 0, 2024-09-14T23:58:36.433 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 0, 2024-09-14T23:58:36.433 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-14T23:58:36.433 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-14T23:58:36.433 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-14T23:58:36.433 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-14T23:58:36.433 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-14T23:58:36.434 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-14T23:58:36.434 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-14T23:58:36.434 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-14T23:58:36.434 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-14T23:58:36.434 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-14T23:58:36.434 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-14T23:58:36.434 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T23:58:36.434 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T23:58:36.434 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T23:58:36.434 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-14T23:58:36.434 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-14T23:58:36.434 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-14T23:58:36.434 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-14T23:58:36.434 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-14T23:58:36.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.435 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-14T23:58:36.435 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-14T23:58:36.435 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-14T23:58:36.435 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-14T23:58:36.435 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-14T23:58:36.435 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T23:58:36.435 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-14T23:58:36.435 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-14T23:58:36.435 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-14T23:58:36.435 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-14T23:58:36.435 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-14T23:58:36.435 INFO:teuthology.orchestra.run.smithi042.stdout: "options": { 2024-09-14T23:58:36.435 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_max": 32, 2024-09-14T23:58:36.435 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_min": 1 2024-09-14T23:58:36.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.436 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-14T23:58:36.436 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": {} 2024-09-14T23:58:36.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.436 INFO:teuthology.orchestra.run.smithi042.stdout: "read_balance": { 2024-09-14T23:58:36.436 INFO:teuthology.orchestra.run.smithi042.stdout: "score_type": "Fair distribution", 2024-09-14T23:58:36.436 INFO:teuthology.orchestra.run.smithi042.stdout: "score_acting": 6.9000000953674316, 2024-09-14T23:58:36.436 INFO:teuthology.orchestra.run.smithi042.stdout: "score_stable": 6.9000000953674316, 2024-09-14T23:58:36.436 INFO:teuthology.orchestra.run.smithi042.stdout: "optimal_score": 0.28999999165534973, 2024-09-14T23:58:36.436 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_acting": 2, 2024-09-14T23:58:36.436 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_stable": 2, 2024-09-14T23:58:36.436 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity_weighted": 0.78668212890625, 2024-09-14T23:58:36.436 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity": 0.89999997615814209, 2024-09-14T23:58:36.437 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity_weighted": 0.79000002145767212 2024-09-14T23:58:36.437 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.437 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 2, 2024-09-14T23:58:36.437 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": "rbd", 2024-09-14T23:58:36.437 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-14T23:56:21.284705+0000", 2024-09-14T23:58:36.437 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 8193, 2024-09-14T23:58:36.437 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T23:58:36.437 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-14T23:58:36.437 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-14T23:58:36.437 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-14T23:58:36.437 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-14T23:58:36.438 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-14T23:58:36.438 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-14T23:58:36.438 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T23:58:36.438 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T23:58:36.438 INFO:teuthology.orchestra.run.smithi042.stdout: "is_stretch_pool": false, 2024-09-14T23:58:36.438 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-14T23:58:36.438 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-14T23:58:36.438 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 8, 2024-09-14T23:58:36.438 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 8, 2024-09-14T23:58:36.438 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 8, 2024-09-14T23:58:36.438 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 8, 2024-09-14T23:58:36.438 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 8, 2024-09-14T23:58:36.438 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-14T23:58:36.438 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-14T23:58:36.439 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-14T23:58:36.439 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-14T23:58:36.439 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-14T23:58:36.439 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-14T23:58:36.439 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-14T23:58:36.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.439 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "21", 2024-09-14T23:58:36.439 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "0", 2024-09-14T23:58:36.439 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T23:58:36.439 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T23:58:36.439 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-14T23:58:36.439 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-14T23:58:36.439 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 2, 2024-09-14T23:58:36.439 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 21, 2024-09-14T23:58:36.440 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-14T23:58:36.440 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-14T23:58:36.440 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-14T23:58:36.440 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-14T23:58:36.440 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-14T23:58:36.440 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-14T23:58:36.440 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-14T23:58:36.440 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-14T23:58:36.440 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-14T23:58:36.440 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-14T23:58:36.440 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-14T23:58:36.441 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T23:58:36.441 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T23:58:36.441 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T23:58:36.441 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-14T23:58:36.441 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-14T23:58:36.441 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-14T23:58:36.441 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-14T23:58:36.441 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-14T23:58:36.441 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.441 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-14T23:58:36.441 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-14T23:58:36.441 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-14T23:58:36.442 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-14T23:58:36.442 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-14T23:58:36.442 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T23:58:36.442 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-14T23:58:36.442 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-14T23:58:36.442 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-14T23:58:36.442 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-14T23:58:36.442 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-14T23:58:36.442 INFO:teuthology.orchestra.run.smithi042.stdout: "options": {}, 2024-09-14T23:58:36.442 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-14T23:58:36.442 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd": {} 2024-09-14T23:58:36.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.443 INFO:teuthology.orchestra.run.smithi042.stdout: "read_balance": { 2024-09-14T23:58:36.443 INFO:teuthology.orchestra.run.smithi042.stdout: "score_type": "Fair distribution", 2024-09-14T23:58:36.443 INFO:teuthology.orchestra.run.smithi042.stdout: "score_acting": 3.4900000095367432, 2024-09-14T23:58:36.443 INFO:teuthology.orchestra.run.smithi042.stdout: "score_stable": 3.4900000095367432, 2024-09-14T23:58:36.443 INFO:teuthology.orchestra.run.smithi042.stdout: "optimal_score": 0.86000001430511475, 2024-09-14T23:58:36.443 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_acting": 3, 2024-09-14T23:58:36.443 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_stable": 3, 2024-09-14T23:58:36.443 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity_weighted": 0.78668212890625, 2024-09-14T23:58:36.443 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity": 0.89999997615814209, 2024-09-14T23:58:36.443 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity_weighted": 0.79000002145767212 2024-09-14T23:58:36.443 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.443 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.443 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 3, 2024-09-14T23:58:36.444 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": "pool_client.0", 2024-09-14T23:58:36.444 INFO:teuthology.orchestra.run.smithi042.stdout: "create_time": "2024-09-14T23:56:30.861504+0000", 2024-09-14T23:58:36.444 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 8193, 2024-09-14T23:58:36.444 INFO:teuthology.orchestra.run.smithi042.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T23:58:36.444 INFO:teuthology.orchestra.run.smithi042.stdout: "type": 1, 2024-09-14T23:58:36.444 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 2, 2024-09-14T23:58:36.444 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": 1, 2024-09-14T23:58:36.444 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_rule": 0, 2024-09-14T23:58:36.444 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_count": 0, 2024-09-14T23:58:36.445 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_target": 0, 2024-09-14T23:58:36.445 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T23:58:36.445 INFO:teuthology.orchestra.run.smithi042.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T23:58:36.445 INFO:teuthology.orchestra.run.smithi042.stdout: "is_stretch_pool": false, 2024-09-14T23:58:36.445 INFO:teuthology.orchestra.run.smithi042.stdout: "object_hash": 2, 2024-09-14T23:58:36.445 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscale_mode": "off", 2024-09-14T23:58:36.445 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": 32, 2024-09-14T23:58:36.445 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num": 32, 2024-09-14T23:58:36.445 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_placement_num_target": 32, 2024-09-14T23:58:36.445 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_target": 32, 2024-09-14T23:58:36.446 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num_pending": 32, 2024-09-14T23:58:36.446 INFO:teuthology.orchestra.run.smithi042.stdout: "last_pg_merge_meta": { 2024-09-14T23:58:36.446 INFO:teuthology.orchestra.run.smithi042.stdout: "source_pgid": "0.0", 2024-09-14T23:58:36.446 INFO:teuthology.orchestra.run.smithi042.stdout: "ready_epoch": 0, 2024-09-14T23:58:36.446 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_started": 0, 2024-09-14T23:58:36.446 INFO:teuthology.orchestra.run.smithi042.stdout: "last_epoch_clean": 0, 2024-09-14T23:58:36.446 INFO:teuthology.orchestra.run.smithi042.stdout: "source_version": "0'0", 2024-09-14T23:58:36.446 INFO:teuthology.orchestra.run.smithi042.stdout: "target_version": "0'0" 2024-09-14T23:58:36.446 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.446 INFO:teuthology.orchestra.run.smithi042.stdout: "last_change": "136", 2024-09-14T23:58:36.446 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend": "0", 2024-09-14T23:58:36.446 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T23:58:36.446 INFO:teuthology.orchestra.run.smithi042.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T23:58:36.446 INFO:teuthology.orchestra.run.smithi042.stdout: "auid": 0, 2024-09-14T23:58:36.447 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_mode": "selfmanaged", 2024-09-14T23:58:36.447 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_seq": 51, 2024-09-14T23:58:36.447 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_epoch": 136, 2024-09-14T23:58:36.447 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_snaps": [], 2024-09-14T23:58:36.447 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps": "[]", 2024-09-14T23:58:36.447 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_bytes": 0, 2024-09-14T23:58:36.447 INFO:teuthology.orchestra.run.smithi042.stdout: "quota_max_objects": 0, 2024-09-14T23:58:36.447 INFO:teuthology.orchestra.run.smithi042.stdout: "tiers": [], 2024-09-14T23:58:36.447 INFO:teuthology.orchestra.run.smithi042.stdout: "tier_of": -1, 2024-09-14T23:58:36.447 INFO:teuthology.orchestra.run.smithi042.stdout: "read_tier": -1, 2024-09-14T23:58:36.447 INFO:teuthology.orchestra.run.smithi042.stdout: "write_tier": -1, 2024-09-14T23:58:36.447 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_mode": "none", 2024-09-14T23:58:36.447 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_bytes": 0, 2024-09-14T23:58:36.448 INFO:teuthology.orchestra.run.smithi042.stdout: "target_max_objects": 0, 2024-09-14T23:58:36.448 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T23:58:36.448 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T23:58:36.448 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T23:58:36.448 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_flush_age": 0, 2024-09-14T23:58:36.448 INFO:teuthology.orchestra.run.smithi042.stdout: "cache_min_evict_age": 0, 2024-09-14T23:58:36.448 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profile": "", 2024-09-14T23:58:36.448 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_params": { 2024-09-14T23:58:36.448 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "none" 2024-09-14T23:58:36.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.448 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_period": 0, 2024-09-14T23:58:36.448 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_count": 0, 2024-09-14T23:58:36.448 INFO:teuthology.orchestra.run.smithi042.stdout: "use_gmt_hitset": true, 2024-09-14T23:58:36.448 INFO:teuthology.orchestra.run.smithi042.stdout: "min_read_recency_for_promote": 0, 2024-09-14T23:58:36.449 INFO:teuthology.orchestra.run.smithi042.stdout: "min_write_recency_for_promote": 0, 2024-09-14T23:58:36.449 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T23:58:36.449 INFO:teuthology.orchestra.run.smithi042.stdout: "hit_set_search_last_n": 0, 2024-09-14T23:58:36.449 INFO:teuthology.orchestra.run.smithi042.stdout: "grade_table": [], 2024-09-14T23:58:36.449 INFO:teuthology.orchestra.run.smithi042.stdout: "stripe_width": 0, 2024-09-14T23:58:36.449 INFO:teuthology.orchestra.run.smithi042.stdout: "expected_num_objects": 0, 2024-09-14T23:58:36.449 INFO:teuthology.orchestra.run.smithi042.stdout: "fast_read": false, 2024-09-14T23:58:36.449 INFO:teuthology.orchestra.run.smithi042.stdout: "options": {}, 2024-09-14T23:58:36.449 INFO:teuthology.orchestra.run.smithi042.stdout: "application_metadata": { 2024-09-14T23:58:36.449 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd": {} 2024-09-14T23:58:36.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.449 INFO:teuthology.orchestra.run.smithi042.stdout: "read_balance": { 2024-09-14T23:58:36.449 INFO:teuthology.orchestra.run.smithi042.stdout: "score_type": "Fair distribution", 2024-09-14T23:58:36.449 INFO:teuthology.orchestra.run.smithi042.stdout: "score_acting": 2.619999885559082, 2024-09-14T23:58:36.450 INFO:teuthology.orchestra.run.smithi042.stdout: "score_stable": 2.619999885559082, 2024-09-14T23:58:36.450 INFO:teuthology.orchestra.run.smithi042.stdout: "optimal_score": 0.86000001430511475, 2024-09-14T23:58:36.450 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_acting": 2.25, 2024-09-14T23:58:36.450 INFO:teuthology.orchestra.run.smithi042.stdout: "raw_score_stable": 2.25, 2024-09-14T23:58:36.450 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity_weighted": 0.78668212890625, 2024-09-14T23:58:36.450 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity": 0.89999997615814209, 2024-09-14T23:58:36.450 INFO:teuthology.orchestra.run.smithi042.stdout: "average_primary_affinity_weighted": 0.79000002145767212 2024-09-14T23:58:36.450 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.450 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.450 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-14T23:58:36.450 INFO:teuthology.orchestra.run.smithi042.stdout: "osds": [ 2024-09-14T23:58:36.450 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.450 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 0, 2024-09-14T23:58:36.450 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "db9d0c67-2d60-4592-9ad3-f99e5294f93f", 2024-09-14T23:58:36.451 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-14T23:58:36.451 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-14T23:58:36.451 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-14T23:58:36.451 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 0.3231201171875, 2024-09-14T23:58:36.451 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-14T23:58:36.451 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-14T23:58:36.451 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 14, 2024-09-14T23:58:36.451 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 127, 2024-09-14T23:58:36.451 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-14T23:58:36.451 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-14T23:58:36.451 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-14T23:58:36.451 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:36.451 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.452 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:36.452 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6816", 2024-09-14T23:58:36.452 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 314865173 2024-09-14T23:58:36.452 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.452 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.452 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:36.452 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6817", 2024-09-14T23:58:36.452 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 314865173 2024-09-14T23:58:36.452 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.452 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.452 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.452 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-14T23:58:36.453 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:36.453 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.453 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:36.453 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6818", 2024-09-14T23:58:36.453 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 314865173 2024-09-14T23:58:36.453 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.453 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.453 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:36.453 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6819", 2024-09-14T23:58:36.453 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 314865173 2024-09-14T23:58:36.453 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.453 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.453 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.453 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-14T23:58:36.454 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:36.454 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.454 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:36.454 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6822", 2024-09-14T23:58:36.454 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 314865173 2024-09-14T23:58:36.454 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.454 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.454 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:36.454 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6823", 2024-09-14T23:58:36.454 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 314865173 2024-09-14T23:58:36.454 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.454 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.455 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.455 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-14T23:58:36.455 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:36.455 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.455 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:36.455 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6820", 2024-09-14T23:58:36.455 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 314865173 2024-09-14T23:58:36.455 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.455 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.455 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:36.455 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6821", 2024-09-14T23:58:36.455 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 314865173 2024-09-14T23:58:36.455 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.455 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.456 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.456 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6817/314865173", 2024-09-14T23:58:36.456 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6819/314865173", 2024-09-14T23:58:36.456 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6823/314865173", 2024-09-14T23:58:36.456 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6821/314865173", 2024-09-14T23:58:36.456 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-14T23:58:36.456 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-14T23:58:36.456 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-14T23:58:36.456 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.456 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.456 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.456 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 1, 2024-09-14T23:58:36.456 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "6c1a2765-c591-4c57-8ab3-ee6645970aa5", 2024-09-14T23:58:36.456 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-14T23:58:36.457 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-14T23:58:36.457 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-14T23:58:36.457 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-14T23:58:36.457 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-14T23:58:36.457 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-14T23:58:36.457 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 14, 2024-09-14T23:58:36.457 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 128, 2024-09-14T23:58:36.457 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-14T23:58:36.457 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-14T23:58:36.457 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-14T23:58:36.457 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:36.457 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.457 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:36.457 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6800", 2024-09-14T23:58:36.458 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3197912310 2024-09-14T23:58:36.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.458 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.458 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:36.458 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6801", 2024-09-14T23:58:36.458 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3197912310 2024-09-14T23:58:36.458 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.458 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.458 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-14T23:58:36.458 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:36.458 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.458 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:36.459 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6802", 2024-09-14T23:58:36.459 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3197912310 2024-09-14T23:58:36.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.459 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.459 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:36.459 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6803", 2024-09-14T23:58:36.459 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3197912310 2024-09-14T23:58:36.459 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.460 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.460 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.460 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-14T23:58:36.460 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:36.460 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.460 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:36.460 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6806", 2024-09-14T23:58:36.460 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3197912310 2024-09-14T23:58:36.460 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.460 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.460 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:36.460 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6807", 2024-09-14T23:58:36.460 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3197912310 2024-09-14T23:58:36.460 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.461 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.461 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.461 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-14T23:58:36.461 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:36.461 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.461 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:36.461 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6804", 2024-09-14T23:58:36.461 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3197912310 2024-09-14T23:58:36.461 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.461 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.461 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:36.461 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6805", 2024-09-14T23:58:36.461 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3197912310 2024-09-14T23:58:36.461 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.461 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.462 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6801/3197912310", 2024-09-14T23:58:36.462 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6803/3197912310", 2024-09-14T23:58:36.462 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6807/3197912310", 2024-09-14T23:58:36.462 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6805/3197912310", 2024-09-14T23:58:36.462 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-14T23:58:36.462 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-14T23:58:36.462 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-14T23:58:36.462 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.462 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.462 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 2, 2024-09-14T23:58:36.462 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "930db893-7be6-4645-a7f1-e043c5ec95eb", 2024-09-14T23:58:36.462 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-14T23:58:36.463 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-14T23:58:36.463 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-14T23:58:36.463 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-14T23:58:36.463 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-14T23:58:36.463 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-14T23:58:36.463 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 13, 2024-09-14T23:58:36.463 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 128, 2024-09-14T23:58:36.463 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-14T23:58:36.463 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-14T23:58:36.463 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-14T23:58:36.463 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:36.463 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.463 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:36.463 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6808", 2024-09-14T23:58:36.463 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3241153660 2024-09-14T23:58:36.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.464 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:36.464 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6809", 2024-09-14T23:58:36.464 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3241153660 2024-09-14T23:58:36.464 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.464 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.464 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-14T23:58:36.464 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:36.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.464 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:36.464 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6810", 2024-09-14T23:58:36.465 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3241153660 2024-09-14T23:58:36.465 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.465 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.465 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:36.465 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6811", 2024-09-14T23:58:36.465 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3241153660 2024-09-14T23:58:36.465 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.465 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.465 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.465 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-14T23:58:36.465 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:36.465 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.465 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:36.465 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6814", 2024-09-14T23:58:36.466 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3241153660 2024-09-14T23:58:36.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.466 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.466 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:36.466 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6815", 2024-09-14T23:58:36.466 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3241153660 2024-09-14T23:58:36.466 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.466 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.466 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-14T23:58:36.466 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:36.466 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.466 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:36.466 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6812", 2024-09-14T23:58:36.467 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3241153660 2024-09-14T23:58:36.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.467 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.467 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:36.467 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6813", 2024-09-14T23:58:36.467 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3241153660 2024-09-14T23:58:36.467 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.467 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.467 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6809/3241153660", 2024-09-14T23:58:36.467 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6811/3241153660", 2024-09-14T23:58:36.467 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6815/3241153660", 2024-09-14T23:58:36.467 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6813/3241153660", 2024-09-14T23:58:36.467 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-14T23:58:36.468 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-14T23:58:36.468 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-14T23:58:36.468 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.468 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.468 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 3, 2024-09-14T23:58:36.468 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "45bb3b84-b1c4-4f87-a7d1-0eb77b81741b", 2024-09-14T23:58:36.468 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-14T23:58:36.468 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 0, 2024-09-14T23:58:36.468 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 0, 2024-09-14T23:58:36.469 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-14T23:58:36.469 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-14T23:58:36.469 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-14T23:58:36.469 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-14T23:58:36.469 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 127, 2024-09-14T23:58:36.469 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-14T23:58:36.470 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-14T23:58:36.470 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-14T23:58:36.470 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:36.470 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.470 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:36.470 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6824", 2024-09-14T23:58:36.470 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1736416326 2024-09-14T23:58:36.470 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.470 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.470 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:36.470 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6825", 2024-09-14T23:58:36.471 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1736416326 2024-09-14T23:58:36.471 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.471 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.471 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-14T23:58:36.471 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:36.471 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.471 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:36.471 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6826", 2024-09-14T23:58:36.471 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1736416326 2024-09-14T23:58:36.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.472 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.472 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:36.472 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6827", 2024-09-14T23:58:36.472 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1736416326 2024-09-14T23:58:36.472 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.472 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.472 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-14T23:58:36.472 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:36.472 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.473 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:36.473 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6830", 2024-09-14T23:58:36.473 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1736416326 2024-09-14T23:58:36.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.473 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.474 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:36.474 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6831", 2024-09-14T23:58:36.474 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1736416326 2024-09-14T23:58:36.474 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.474 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.474 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-14T23:58:36.474 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:36.474 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.475 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:36.475 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6828", 2024-09-14T23:58:36.475 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1736416326 2024-09-14T23:58:36.475 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.475 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.475 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:36.475 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6829", 2024-09-14T23:58:36.475 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1736416326 2024-09-14T23:58:36.475 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.475 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.475 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.475 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.42:6825/1736416326", 2024-09-14T23:58:36.475 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.42:6827/1736416326", 2024-09-14T23:58:36.475 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.42:6831/1736416326", 2024-09-14T23:58:36.476 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.42:6829/1736416326", 2024-09-14T23:58:36.476 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-14T23:58:36.476 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-14T23:58:36.476 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-14T23:58:36.476 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.476 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.476 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 4, 2024-09-14T23:58:36.476 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "a812e38f-c0f9-4d33-9373-1910fb09103c", 2024-09-14T23:58:36.476 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-14T23:58:36.476 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-14T23:58:36.477 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-14T23:58:36.477 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 0.9703369140625, 2024-09-14T23:58:36.477 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-14T23:58:36.477 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-14T23:58:36.477 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-14T23:58:36.477 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 127, 2024-09-14T23:58:36.477 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-14T23:58:36.477 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-14T23:58:36.477 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-14T23:58:36.477 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:36.477 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.478 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:36.478 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6808", 2024-09-14T23:58:36.478 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1755154427 2024-09-14T23:58:36.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.478 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:36.478 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6809", 2024-09-14T23:58:36.478 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1755154427 2024-09-14T23:58:36.478 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.478 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.478 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-14T23:58:36.479 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:36.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.479 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:36.479 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6810", 2024-09-14T23:58:36.479 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1755154427 2024-09-14T23:58:36.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.479 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:36.479 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6811", 2024-09-14T23:58:36.479 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1755154427 2024-09-14T23:58:36.479 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.479 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.479 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-14T23:58:36.480 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:36.480 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.480 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:36.480 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6814", 2024-09-14T23:58:36.480 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1755154427 2024-09-14T23:58:36.480 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.480 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.480 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:36.480 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6815", 2024-09-14T23:58:36.480 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1755154427 2024-09-14T23:58:36.480 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.480 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.481 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-14T23:58:36.481 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:36.481 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.481 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:36.481 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6812", 2024-09-14T23:58:36.481 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1755154427 2024-09-14T23:58:36.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.481 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.481 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:36.482 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6813", 2024-09-14T23:58:36.482 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1755154427 2024-09-14T23:58:36.482 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.482 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.482 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.43:6809/1755154427", 2024-09-14T23:58:36.482 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.43:6811/1755154427", 2024-09-14T23:58:36.482 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.43:6815/1755154427", 2024-09-14T23:58:36.483 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.43:6813/1755154427", 2024-09-14T23:58:36.483 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-14T23:58:36.483 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-14T23:58:36.483 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-14T23:58:36.483 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.483 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.483 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.483 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 5, 2024-09-14T23:58:36.483 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "0de87e5b-7a21-4fb4-9543-d96c065786f8", 2024-09-14T23:58:36.484 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-14T23:58:36.484 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-14T23:58:36.484 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 1, 2024-09-14T23:58:36.484 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 0, 2024-09-14T23:58:36.484 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-14T23:58:36.484 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-14T23:58:36.484 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-14T23:58:36.484 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 111, 2024-09-14T23:58:36.485 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-14T23:58:36.485 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-14T23:58:36.485 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-14T23:58:36.485 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:36.485 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.485 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:36.485 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6816", 2024-09-14T23:58:36.485 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3033769028 2024-09-14T23:58:36.485 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.486 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.486 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:36.486 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6817", 2024-09-14T23:58:36.486 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3033769028 2024-09-14T23:58:36.486 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.486 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.486 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.486 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-14T23:58:36.487 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:36.487 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.487 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:36.487 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6818", 2024-09-14T23:58:36.487 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3033769028 2024-09-14T23:58:36.487 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.487 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.487 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:36.487 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6819", 2024-09-14T23:58:36.488 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3033769028 2024-09-14T23:58:36.488 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.488 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.488 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.488 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-14T23:58:36.488 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:36.488 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.488 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:36.489 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6822", 2024-09-14T23:58:36.489 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3033769028 2024-09-14T23:58:36.489 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.489 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.489 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:36.489 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6823", 2024-09-14T23:58:36.489 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3033769028 2024-09-14T23:58:36.489 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.489 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.490 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.490 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-14T23:58:36.490 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:36.490 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.490 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:36.490 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6820", 2024-09-14T23:58:36.490 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3033769028 2024-09-14T23:58:36.490 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.491 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.491 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:36.491 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6821", 2024-09-14T23:58:36.491 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3033769028 2024-09-14T23:58:36.491 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.491 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.491 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.43:6817/3033769028", 2024-09-14T23:58:36.492 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.43:6819/3033769028", 2024-09-14T23:58:36.492 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.43:6823/3033769028", 2024-09-14T23:58:36.492 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.43:6821/3033769028", 2024-09-14T23:58:36.492 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-14T23:58:36.492 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-14T23:58:36.492 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-14T23:58:36.492 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.492 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.493 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.493 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 6, 2024-09-14T23:58:36.493 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "c58c1d59-ad02-40f0-98e3-9078ef7e622b", 2024-09-14T23:58:36.493 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-14T23:58:36.493 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 1, 2024-09-14T23:58:36.493 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 0.924530029296875, 2024-09-14T23:58:36.493 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-14T23:58:36.493 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-14T23:58:36.494 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-14T23:58:36.494 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-14T23:58:36.494 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 127, 2024-09-14T23:58:36.494 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-14T23:58:36.494 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-14T23:58:36.494 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-14T23:58:36.494 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:36.494 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.495 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:36.495 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6800", 2024-09-14T23:58:36.495 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3314282416 2024-09-14T23:58:36.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.495 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.495 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:36.495 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6801", 2024-09-14T23:58:36.495 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3314282416 2024-09-14T23:58:36.495 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.495 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.495 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-14T23:58:36.495 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:36.496 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.496 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:36.496 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6802", 2024-09-14T23:58:36.496 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3314282416 2024-09-14T23:58:36.496 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.496 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.496 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:36.496 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6803", 2024-09-14T23:58:36.496 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3314282416 2024-09-14T23:58:36.497 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.497 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.497 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-14T23:58:36.497 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:36.497 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.497 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:36.497 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6806", 2024-09-14T23:58:36.497 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3314282416 2024-09-14T23:58:36.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.497 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.497 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:36.497 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6807", 2024-09-14T23:58:36.497 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3314282416 2024-09-14T23:58:36.498 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.498 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.498 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-14T23:58:36.498 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:36.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.498 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:36.498 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6804", 2024-09-14T23:58:36.498 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3314282416 2024-09-14T23:58:36.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.499 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.499 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:36.499 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6805", 2024-09-14T23:58:36.499 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3314282416 2024-09-14T23:58:36.499 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.499 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.499 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.499 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.43:6801/3314282416", 2024-09-14T23:58:36.499 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.43:6803/3314282416", 2024-09-14T23:58:36.499 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.43:6807/3314282416", 2024-09-14T23:58:36.499 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.43:6805/3314282416", 2024-09-14T23:58:36.499 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-14T23:58:36.499 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-14T23:58:36.499 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-14T23:58:36.499 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.500 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.500 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 7, 2024-09-14T23:58:36.500 INFO:teuthology.orchestra.run.smithi042.stdout: "uuid": "389bdb04-20e3-4a7a-8504-687e034bd7dc", 2024-09-14T23:58:36.500 INFO:teuthology.orchestra.run.smithi042.stdout: "up": 1, 2024-09-14T23:58:36.500 INFO:teuthology.orchestra.run.smithi042.stdout: "in": 0, 2024-09-14T23:58:36.500 INFO:teuthology.orchestra.run.smithi042.stdout: "weight": 0, 2024-09-14T23:58:36.500 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_affinity": 1, 2024-09-14T23:58:36.500 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_begin": 0, 2024-09-14T23:58:36.500 INFO:teuthology.orchestra.run.smithi042.stdout: "last_clean_end": 0, 2024-09-14T23:58:36.500 INFO:teuthology.orchestra.run.smithi042.stdout: "up_from": 15, 2024-09-14T23:58:36.500 INFO:teuthology.orchestra.run.smithi042.stdout: "up_thru": 101, 2024-09-14T23:58:36.500 INFO:teuthology.orchestra.run.smithi042.stdout: "down_at": 0, 2024-09-14T23:58:36.500 INFO:teuthology.orchestra.run.smithi042.stdout: "lost_at": 0, 2024-09-14T23:58:36.501 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addrs": { 2024-09-14T23:58:36.501 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:36.501 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.501 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:36.501 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6824", 2024-09-14T23:58:36.501 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1481350118 2024-09-14T23:58:36.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.501 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.501 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:36.501 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6825", 2024-09-14T23:58:36.501 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1481350118 2024-09-14T23:58:36.501 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.501 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.502 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addrs": { 2024-09-14T23:58:36.502 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:36.502 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.502 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:36.502 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6826", 2024-09-14T23:58:36.502 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1481350118 2024-09-14T23:58:36.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.502 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.502 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:36.502 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6827", 2024-09-14T23:58:36.502 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1481350118 2024-09-14T23:58:36.502 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.502 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.502 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addrs": { 2024-09-14T23:58:36.503 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:36.503 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.503 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:36.503 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6830", 2024-09-14T23:58:36.503 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1481350118 2024-09-14T23:58:36.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.503 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.503 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:36.503 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6831", 2024-09-14T23:58:36.503 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1481350118 2024-09-14T23:58:36.503 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.503 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.504 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addrs": { 2024-09-14T23:58:36.504 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-09-14T23:58:36.504 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.504 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-09-14T23:58:36.504 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6828", 2024-09-14T23:58:36.504 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1481350118 2024-09-14T23:58:36.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.504 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.504 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-14T23:58:36.504 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.43:6829", 2024-09-14T23:58:36.504 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1481350118 2024-09-14T23:58:36.504 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.504 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.505 INFO:teuthology.orchestra.run.smithi042.stdout: "public_addr": "172.21.15.43:6825/1481350118", 2024-09-14T23:58:36.505 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_addr": "172.21.15.43:6827/1481350118", 2024-09-14T23:58:36.505 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_back_addr": "172.21.15.43:6831/1481350118", 2024-09-14T23:58:36.505 INFO:teuthology.orchestra.run.smithi042.stdout: "heartbeat_front_addr": "172.21.15.43:6829/1481350118", 2024-09-14T23:58:36.505 INFO:teuthology.orchestra.run.smithi042.stdout: "state": [ 2024-09-14T23:58:36.505 INFO:teuthology.orchestra.run.smithi042.stdout: "exists", 2024-09-14T23:58:36.505 INFO:teuthology.orchestra.run.smithi042.stdout: "up" 2024-09-14T23:58:36.505 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.505 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.505 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-14T23:58:36.505 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_xinfo": [ 2024-09-14T23:58:36.505 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.505 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 0, 2024-09-14T23:58:36.506 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-14T23:58:36.506 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-14T23:58:36.506 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-14T23:58:36.506 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-14T23:58:36.506 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-14T23:58:36.506 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-14T23:56:11.077810+0000", 2024-09-14T23:58:36.506 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-14T23:58:36.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.506 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 1, 2024-09-14T23:58:36.506 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-14T23:58:36.506 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-14T23:58:36.506 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-14T23:58:36.507 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-14T23:58:36.507 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-14T23:58:36.507 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-14T23:56:11.578838+0000", 2024-09-14T23:58:36.507 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-14T23:58:36.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.507 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.507 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 2, 2024-09-14T23:58:36.507 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-14T23:58:36.507 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-14T23:58:36.507 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-14T23:58:36.507 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-14T23:58:36.507 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-14T23:58:36.507 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-14T23:56:10.895490+0000", 2024-09-14T23:58:36.508 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-14T23:58:36.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.508 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.508 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 3, 2024-09-14T23:58:36.508 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-14T23:58:36.508 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-14T23:58:36.508 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-14T23:58:36.508 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-14T23:58:36.508 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 65536, 2024-09-14T23:58:36.508 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-14T23:56:12.290821+0000", 2024-09-14T23:58:36.508 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-14T23:58:36.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.509 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.509 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 4, 2024-09-14T23:58:36.509 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-14T23:58:36.509 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-14T23:58:36.509 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-14T23:58:36.509 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-14T23:58:36.509 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-14T23:58:36.509 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-14T23:56:11.800581+0000", 2024-09-14T23:58:36.509 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-14T23:58:36.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.509 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.509 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 5, 2024-09-14T23:58:36.510 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-14T23:58:36.510 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-14T23:58:36.510 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-14T23:58:36.510 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-14T23:58:36.510 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-14T23:58:36.510 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-14T23:56:11.883155+0000", 2024-09-14T23:58:36.510 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-14T23:58:36.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.510 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 6, 2024-09-14T23:58:36.510 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-14T23:58:36.510 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-14T23:58:36.510 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-14T23:58:36.511 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-14T23:58:36.511 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 0, 2024-09-14T23:58:36.511 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-14T23:56:11.604790+0000", 2024-09-14T23:58:36.511 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-14T23:58:36.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.511 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": 7, 2024-09-14T23:58:36.511 INFO:teuthology.orchestra.run.smithi042.stdout: "down_stamp": "0.000000", 2024-09-14T23:58:36.511 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_probability": 0, 2024-09-14T23:58:36.512 INFO:teuthology.orchestra.run.smithi042.stdout: "laggy_interval": 0, 2024-09-14T23:58:36.512 INFO:teuthology.orchestra.run.smithi042.stdout: "features": 4540701547738038271, 2024-09-14T23:58:36.512 INFO:teuthology.orchestra.run.smithi042.stdout: "old_weight": 65536, 2024-09-14T23:58:36.512 INFO:teuthology.orchestra.run.smithi042.stdout: "last_purged_snaps_scrub": "2024-09-14T23:56:12.067759+0000", 2024-09-14T23:58:36.512 INFO:teuthology.orchestra.run.smithi042.stdout: "dead_epoch": 0 2024-09-14T23:58:36.512 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.512 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-14T23:58:36.512 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_upmap": [], 2024-09-14T23:58:36.512 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_upmap_items": [ 2024-09-14T23:58:36.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.512 INFO:teuthology.orchestra.run.smithi042.stdout: "pgid": "3.e", 2024-09-14T23:58:36.512 INFO:teuthology.orchestra.run.smithi042.stdout: "mappings": [ 2024-09-14T23:58:36.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.513 INFO:teuthology.orchestra.run.smithi042.stdout: "from": 4, 2024-09-14T23:58:36.513 INFO:teuthology.orchestra.run.smithi042.stdout: "to": 2 2024-09-14T23:58:36.513 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.513 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.513 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.513 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-14T23:58:36.513 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_upmap_primaries": [], 2024-09-14T23:58:36.513 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_temp": [], 2024-09-14T23:58:36.513 INFO:teuthology.orchestra.run.smithi042.stdout: "primary_temp": [], 2024-09-14T23:58:36.513 INFO:teuthology.orchestra.run.smithi042.stdout: "blocklist": {}, 2024-09-14T23:58:36.513 INFO:teuthology.orchestra.run.smithi042.stdout: "range_blocklist": {}, 2024-09-14T23:58:36.513 INFO:teuthology.orchestra.run.smithi042.stdout: "erasure_code_profiles": { 2024-09-14T23:58:36.513 INFO:teuthology.orchestra.run.smithi042.stdout: "default": { 2024-09-14T23:58:36.514 INFO:teuthology.orchestra.run.smithi042.stdout: "crush-failure-domain": "osd", 2024-09-14T23:58:36.514 INFO:teuthology.orchestra.run.smithi042.stdout: "k": "2", 2024-09-14T23:58:36.514 INFO:teuthology.orchestra.run.smithi042.stdout: "m": "1", 2024-09-14T23:58:36.514 INFO:teuthology.orchestra.run.smithi042.stdout: "plugin": "jerasure", 2024-09-14T23:58:36.514 INFO:teuthology.orchestra.run.smithi042.stdout: "technique": "reed_sol_van" 2024-09-14T23:58:36.514 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.514 INFO:teuthology.orchestra.run.smithi042.stdout: "removed_snaps_queue": [ 2024-09-14T23:58:36.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.514 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 3, 2024-09-14T23:58:36.514 INFO:teuthology.orchestra.run.smithi042.stdout: "snaps": [ 2024-09-14T23:58:36.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.514 INFO:teuthology.orchestra.run.smithi042.stdout: "begin": 36, 2024-09-14T23:58:36.515 INFO:teuthology.orchestra.run.smithi042.stdout: "length": 14 2024-09-14T23:58:36.515 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.515 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.515 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.515 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-14T23:58:36.515 INFO:teuthology.orchestra.run.smithi042.stdout: "new_removed_snaps": [ 2024-09-14T23:58:36.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.515 INFO:teuthology.orchestra.run.smithi042.stdout: "pool": 3, 2024-09-14T23:58:36.515 INFO:teuthology.orchestra.run.smithi042.stdout: "snaps": [ 2024-09-14T23:58:36.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.515 INFO:teuthology.orchestra.run.smithi042.stdout: "begin": 36, 2024-09-14T23:58:36.515 INFO:teuthology.orchestra.run.smithi042.stdout: "length": 1 2024-09-14T23:58:36.515 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.515 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.516 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.516 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-14T23:58:36.516 INFO:teuthology.orchestra.run.smithi042.stdout: "new_purged_snaps": [], 2024-09-14T23:58:36.516 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_node_flags": {}, 2024-09-14T23:58:36.516 INFO:teuthology.orchestra.run.smithi042.stdout: "device_class_flags": {}, 2024-09-14T23:58:36.516 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode": { 2024-09-14T23:58:36.516 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode_enabled": false, 2024-09-14T23:58:36.516 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_bucket_count": 0, 2024-09-14T23:58:36.516 INFO:teuthology.orchestra.run.smithi042.stdout: "degraded_stretch_mode": 0, 2024-09-14T23:58:36.516 INFO:teuthology.orchestra.run.smithi042.stdout: "recovering_stretch_mode": 0, 2024-09-14T23:58:36.516 INFO:teuthology.orchestra.run.smithi042.stdout: "stretch_mode_bucket": 0 2024-09-14T23:58:36.517 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.517 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:36.519 INFO:tasks.thrashosds.thrasher:j is {'epoch': 136, 'fsid': '21916fcc-f95c-4bfa-8b8b-11ca45845613', 'created': '2024-09-14T23:56:03.167948+0000', 'modified': '2024-09-14T23:58:35.982436+0000', 'last_up_change': '2024-09-14T23:56:14.322882+0000', 'last_in_change': '2024-09-14T23:58:24.287442+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T23:56:13.500197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.78668212890625, 'average_primary_affinity': 0.8999999761581421, 'average_primary_affinity_weighted': 0.7900000214576721}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T23:56:21.284705+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3.490000009536743, 'score_stable': 3.490000009536743, 'optimal_score': 0.8600000143051147, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.78668212890625, 'average_primary_affinity': 0.8999999761581421, 'average_primary_affinity_weighted': 0.7900000214576721}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T23:56:30.861504+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '136', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 51, 'snap_epoch': 136, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.619999885559082, 'score_stable': 2.619999885559082, 'optimal_score': 0.8600000143051147, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.78668212890625, 'average_primary_affinity': 0.8999999761581421, 'average_primary_affinity_weighted': 0.7900000214576721}}], 'osds': [{'osd': 0, 'uuid': 'db9d0c67-2d60-4592-9ad3-f99e5294f93f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.3231201171875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 127, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6816', 'nonce': 314865173}, {'type': 'v1', 'addr': '172.21.15.42:6817', 'nonce': 314865173}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6818', 'nonce': 314865173}, {'type': 'v1', 'addr': '172.21.15.42:6819', 'nonce': 314865173}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6822', 'nonce': 314865173}, {'type': 'v1', 'addr': '172.21.15.42:6823', 'nonce': 314865173}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6820', 'nonce': 314865173}, {'type': 'v1', 'addr': '172.21.15.42:6821', 'nonce': 314865173}]}, 'public_addr': '172.21.15.42:6817/314865173', 'cluster_addr': '172.21.15.42:6819/314865173', 'heartbeat_back_addr': '172.21.15.42:6823/314865173', 'heartbeat_front_addr': '172.21.15.42:6821/314865173', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '6c1a2765-c591-4c57-8ab3-ee6645970aa5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 128, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6800', 'nonce': 3197912310}, {'type': 'v1', 'addr': '172.21.15.42:6801', 'nonce': 3197912310}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6802', 'nonce': 3197912310}, {'type': 'v1', 'addr': '172.21.15.42:6803', 'nonce': 3197912310}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6806', 'nonce': 3197912310}, {'type': 'v1', 'addr': '172.21.15.42:6807', 'nonce': 3197912310}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6804', 'nonce': 3197912310}, {'type': 'v1', 'addr': '172.21.15.42:6805', 'nonce': 3197912310}]}, 'public_addr': '172.21.15.42:6801/3197912310', 'cluster_addr': '172.21.15.42:6803/3197912310', 'heartbeat_back_addr': '172.21.15.42:6807/3197912310', 'heartbeat_front_addr': '172.21.15.42:6805/3197912310', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '930db893-7be6-4645-a7f1-e043c5ec95eb', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 128, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6808', 'nonce': 3241153660}, {'type': 'v1', 'addr': '172.21.15.42:6809', 'nonce': 3241153660}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6810', 'nonce': 3241153660}, {'type': 'v1', 'addr': '172.21.15.42:6811', 'nonce': 3241153660}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6814', 'nonce': 3241153660}, {'type': 'v1', 'addr': '172.21.15.42:6815', 'nonce': 3241153660}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6812', 'nonce': 3241153660}, {'type': 'v1', 'addr': '172.21.15.42:6813', 'nonce': 3241153660}]}, 'public_addr': '172.21.15.42:6809/3241153660', 'cluster_addr': '172.21.15.42:6811/3241153660', 'heartbeat_back_addr': '172.21.15.42:6815/3241153660', 'heartbeat_front_addr': '172.21.15.42:6813/3241153660', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '45bb3b84-b1c4-4f87-a7d1-0eb77b81741b', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 127, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6824', 'nonce': 1736416326}, {'type': 'v1', 'addr': '172.21.15.42:6825', 'nonce': 1736416326}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6826', 'nonce': 1736416326}, {'type': 'v1', 'addr': '172.21.15.42:6827', 'nonce': 1736416326}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6830', 'nonce': 1736416326}, {'type': 'v1', 'addr': '172.21.15.42:6831', 'nonce': 1736416326}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6828', 'nonce': 1736416326}, {'type': 'v1', 'addr': '172.21.15.42:6829', 'nonce': 1736416326}]}, 'public_addr': '172.21.15.42:6825/1736416326', 'cluster_addr': '172.21.15.42:6827/1736416326', 'heartbeat_back_addr': '172.21.15.42:6831/1736416326', 'heartbeat_front_addr': '172.21.15.42:6829/1736416326', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'a812e38f-c0f9-4d33-9373-1910fb09103c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.9703369140625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 127, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6808', 'nonce': 1755154427}, {'type': 'v1', 'addr': '172.21.15.43:6809', 'nonce': 1755154427}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6810', 'nonce': 1755154427}, {'type': 'v1', 'addr': '172.21.15.43:6811', 'nonce': 1755154427}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6814', 'nonce': 1755154427}, {'type': 'v1', 'addr': '172.21.15.43:6815', 'nonce': 1755154427}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6812', 'nonce': 1755154427}, {'type': 'v1', 'addr': '172.21.15.43:6813', 'nonce': 1755154427}]}, 'public_addr': '172.21.15.43:6809/1755154427', 'cluster_addr': '172.21.15.43:6811/1755154427', 'heartbeat_back_addr': '172.21.15.43:6815/1755154427', 'heartbeat_front_addr': '172.21.15.43:6813/1755154427', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '0de87e5b-7a21-4fb4-9543-d96c065786f8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 111, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6816', 'nonce': 3033769028}, {'type': 'v1', 'addr': '172.21.15.43:6817', 'nonce': 3033769028}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6818', 'nonce': 3033769028}, {'type': 'v1', 'addr': '172.21.15.43:6819', 'nonce': 3033769028}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6822', 'nonce': 3033769028}, {'type': 'v1', 'addr': '172.21.15.43:6823', 'nonce': 3033769028}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6820', 'nonce': 3033769028}, {'type': 'v1', 'addr': '172.21.15.43:6821', 'nonce': 3033769028}]}, 'public_addr': '172.21.15.43:6817/3033769028', 'cluster_addr': '172.21.15.43:6819/3033769028', 'heartbeat_back_addr': '172.21.15.43:6823/3033769028', 'heartbeat_front_addr': '172.21.15.43:6821/3033769028', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'c58c1d59-ad02-40f0-98e3-9078ef7e622b', 'up': 1, 'in': 1, 'weight': 0.924530029296875, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 127, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6800', 'nonce': 3314282416}, {'type': 'v1', 'addr': '172.21.15.43:6801', 'nonce': 3314282416}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6802', 'nonce': 3314282416}, {'type': 'v1', 'addr': '172.21.15.43:6803', 'nonce': 3314282416}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6806', 'nonce': 3314282416}, {'type': 'v1', 'addr': '172.21.15.43:6807', 'nonce': 3314282416}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6804', 'nonce': 3314282416}, {'type': 'v1', 'addr': '172.21.15.43:6805', 'nonce': 3314282416}]}, 'public_addr': '172.21.15.43:6801/3314282416', 'cluster_addr': '172.21.15.43:6803/3314282416', 'heartbeat_back_addr': '172.21.15.43:6807/3314282416', 'heartbeat_front_addr': '172.21.15.43:6805/3314282416', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '389bdb04-20e3-4a7a-8504-687e034bd7dc', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 101, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6824', 'nonce': 1481350118}, {'type': 'v1', 'addr': '172.21.15.43:6825', 'nonce': 1481350118}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6826', 'nonce': 1481350118}, {'type': 'v1', 'addr': '172.21.15.43:6827', 'nonce': 1481350118}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6830', 'nonce': 1481350118}, {'type': 'v1', 'addr': '172.21.15.43:6831', 'nonce': 1481350118}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6828', 'nonce': 1481350118}, {'type': 'v1', 'addr': '172.21.15.43:6829', 'nonce': 1481350118}]}, 'public_addr': '172.21.15.43:6825/1481350118', 'cluster_addr': '172.21.15.43:6827/1481350118', 'heartbeat_back_addr': '172.21.15.43:6831/1481350118', 'heartbeat_front_addr': '172.21.15.43:6829/1481350118', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T23:56:11.077810+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T23:56:11.578838+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T23:56:10.895490+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-14T23:56:12.290821+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T23:56:11.800581+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T23:56:11.883155+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T23:56:11.604790+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-14T23:56:12.067759+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.e', 'mappings': [{'from': 4, 'to': 2}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 36, 'length': 14}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 36, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T23:58:36.519 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-14T23:58:36.519 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:36.516+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:36.520 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:36.520 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:58:36.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:58:36.520 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:58:36.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.520 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5201 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2232320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:36.520 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:58:36.520 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 117.263517264, 2024-09-14T23:58:36.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057643228999999997, 2024-09-14T23:58:36.521 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:36.521 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:36.521 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:36.521 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:36.521 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:36.521 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:36.521 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5201 2024-09-14T23:58:36.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.521 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:36.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.521 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:36.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:58:36.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:36.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:58:36.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:36.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:58:36.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0699999999999998e-07 2024-09-14T23:58:36.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:36.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253422+0000", 2024-09-14T23:58:36.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.514e-06 2024-09-14T23:58:36.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:36.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253431+0000", 2024-09-14T23:58:36.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1919999999999994e-06 2024-09-14T23:58:36.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:36.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253436+0000", 2024-09-14T23:58:36.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5350000000000004e-06 2024-09-14T23:58:36.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:36.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310856+0000", 2024-09-14T23:58:36.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057419381999999998 2024-09-14T23:58:36.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:36.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310953+0000", 2024-09-14T23:58:36.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6774000000000006e-05 2024-09-14T23:58:36.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:36.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311062+0000", 2024-09-14T23:58:36.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000109425 2024-09-14T23:58:36.525 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.525 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.525 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.525 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5202 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2293760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:36.525 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:58:36.525 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 117.263492516, 2024-09-14T23:58:36.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057818255999999998, 2024-09-14T23:58:36.525 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:36.525 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:36.525 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:36.525 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:36.526 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:36.526 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:36.526 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5202 2024-09-14T23:58:36.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.526 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:36.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.526 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:36.526 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:58:36.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.526 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:36.526 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:58:36.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:36.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253573+0000", 2024-09-14T23:58:36.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012934000000000001 2024-09-14T23:58:36.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:36.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253576+0000", 2024-09-14T23:58:36.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5310000000000002e-06 2024-09-14T23:58:36.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:36.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253587+0000", 2024-09-14T23:58:36.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0522e-05 2024-09-14T23:58:36.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:36.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253595+0000", 2024-09-14T23:58:36.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8290000000000008e-06 2024-09-14T23:58:36.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.528 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:36.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311093+0000", 2024-09-14T23:58:36.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057498312000000003 2024-09-14T23:58:36.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:36.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311167+0000", 2024-09-14T23:58:36.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3582000000000001e-05 2024-09-14T23:58:36.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:36.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311262+0000", 2024-09-14T23:58:36.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5140000000000001e-05 2024-09-14T23:58:36.529 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.530 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.530 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.530 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5203 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2355200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:36.530 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:58:36.530 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 117.263329774, 2024-09-14T23:58:36.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057903938000000002, 2024-09-14T23:58:36.530 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:36.530 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:36.530 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:36.530 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:36.531 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:36.531 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:36.531 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5203 2024-09-14T23:58:36.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.531 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:36.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:36.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:58:36.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:36.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:58:36.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:36.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253902+0000", 2024-09-14T23:58:36.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029615400000000002 2024-09-14T23:58:36.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:36.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253906+0000", 2024-09-14T23:58:36.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9249999999999997e-06 2024-09-14T23:58:36.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:36.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253913+0000", 2024-09-14T23:58:36.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9099999999999999e-06 2024-09-14T23:58:36.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:36.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253917+0000", 2024-09-14T23:58:36.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9210000000000002e-06 2024-09-14T23:58:36.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:36.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311295+0000", 2024-09-14T23:58:36.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057378268000000003 2024-09-14T23:58:36.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:36.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311402+0000", 2024-09-14T23:58:36.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010644100000000001 2024-09-14T23:58:36.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:36.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311510+0000", 2024-09-14T23:58:36.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108319 2024-09-14T23:58:36.534 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.535 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.535 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.535 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5204 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2416640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:36.535 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:58:36.535 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 117.263011577, 2024-09-14T23:58:36.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057774860999999997, 2024-09-14T23:58:36.535 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:36.535 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:36.535 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:36.535 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:36.535 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:36.535 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:36.536 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5204 2024-09-14T23:58:36.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.536 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:36.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:36.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:58:36.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:36.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:58:36.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:36.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255903+0000", 2024-09-14T23:58:36.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001978807 2024-09-14T23:58:36.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:36.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255907+0000", 2024-09-14T23:58:36.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3610000000000001e-06 2024-09-14T23:58:36.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:36.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255914+0000", 2024-09-14T23:58:36.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.357e-06 2024-09-14T23:58:36.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:36.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255918+0000", 2024-09-14T23:58:36.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1779999999999996e-06 2024-09-14T23:58:36.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:36.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311541+0000", 2024-09-14T23:58:36.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055622697999999998 2024-09-14T23:58:36.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:36.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311614+0000", 2024-09-14T23:58:36.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3102999999999994e-05 2024-09-14T23:58:36.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:36.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311699+0000", 2024-09-14T23:58:36.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5357000000000003e-05 2024-09-14T23:58:36.539 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.539 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.539 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.539 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5211 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2846720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:36.539 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:58:36.539 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 117.260913363, 2024-09-14T23:58:36.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057528151, 2024-09-14T23:58:36.539 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:36.539 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:36.540 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:36.540 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:36.540 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:36.540 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:36.540 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5211 2024-09-14T23:58:36.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.540 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:36.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:36.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:58:36.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:36.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:58:36.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:36.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:58:36.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1399999999999998e-07 2024-09-14T23:58:36.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:36.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256024+0000", 2024-09-14T23:58:36.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.114e-06 2024-09-14T23:58:36.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:36.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256028+0000", 2024-09-14T23:58:36.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0840000000000002e-06 2024-09-14T23:58:36.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:36.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256031+0000", 2024-09-14T23:58:36.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6819999999999999e-06 2024-09-14T23:58:36.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:36.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313317+0000", 2024-09-14T23:58:36.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057286295000000001 2024-09-14T23:58:36.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:36.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313403+0000", 2024-09-14T23:58:36.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6287999999999998e-05 2024-09-14T23:58:36.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:36.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313551+0000", 2024-09-14T23:58:36.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000147374 2024-09-14T23:58:36.543 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.543 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.543 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.543 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5212 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2908160~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:36.543 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:58:36.543 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 117.260902169, 2024-09-14T23:58:36.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057735679999999998, 2024-09-14T23:58:36.543 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:36.544 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:36.544 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:36.544 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:36.544 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:36.544 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:36.544 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5212 2024-09-14T23:58:36.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.544 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:36.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:36.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:58:36.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:36.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:58:36.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:36.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:58:36.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0900000000000001e-07 2024-09-14T23:58:36.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:36.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256035+0000", 2024-09-14T23:58:36.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1200000000000001e-06 2024-09-14T23:58:36.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:36.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256038+0000", 2024-09-14T23:58:36.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.05e-06 2024-09-14T23:58:36.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:36.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256041+0000", 2024-09-14T23:58:36.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.667e-06 2024-09-14T23:58:36.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:36.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313584+0000", 2024-09-14T23:58:36.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057543484999999998 2024-09-14T23:58:36.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:36.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313668+0000", 2024-09-14T23:58:36.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3807000000000006e-05 2024-09-14T23:58:36.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:36.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313770+0000", 2024-09-14T23:58:36.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010134200000000001 2024-09-14T23:58:36.548 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.548 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.548 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.549 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5213 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2969600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:36.549 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:58:36.549 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 117.260892082, 2024-09-14T23:58:36.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058145467999999999, 2024-09-14T23:58:36.549 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:36.549 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:36.549 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:36.549 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:36.549 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:36.549 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:36.549 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5213 2024-09-14T23:58:36.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.549 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:36.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:36.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:58:36.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:36.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:58:36.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:36.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257901+0000", 2024-09-14T23:58:36.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018567519999999999 2024-09-14T23:58:36.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:36.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257904+0000", 2024-09-14T23:58:36.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4850000000000001e-06 2024-09-14T23:58:36.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:36.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257912+0000", 2024-09-14T23:58:36.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2090000000000005e-06 2024-09-14T23:58:36.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:36.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257918+0000", 2024-09-14T23:58:36.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6899999999999997e-06 2024-09-14T23:58:36.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:36.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313816+0000", 2024-09-14T23:58:36.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055898075999999998 2024-09-14T23:58:36.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:36.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.314082+0000", 2024-09-14T23:58:36.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000265386 2024-09-14T23:58:36.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:36.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.314189+0000", 2024-09-14T23:58:36.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010787 2024-09-14T23:58:36.552 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.552 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.553 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.553 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5218 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3276800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:36.553 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:58:36.553 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 117.258965073, 2024-09-14T23:58:36.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057497557999999997, 2024-09-14T23:58:36.553 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:36.553 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:36.553 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:36.553 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:36.553 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:36.553 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:36.553 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5218 2024-09-14T23:58:36.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.554 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:36.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:36.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:58:36.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:36.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:58:36.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:36.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:58:36.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1899999999999999e-07 2024-09-14T23:58:36.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:36.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257972+0000", 2024-09-14T23:58:36.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.217e-06 2024-09-14T23:58:36.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:36.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257975+0000", 2024-09-14T23:58:36.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9359999999999999e-06 2024-09-14T23:58:36.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:36.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257978+0000", 2024-09-14T23:58:36.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1049999999999999e-06 2024-09-14T23:58:36.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:36.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315331+0000", 2024-09-14T23:58:36.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057352093999999999 2024-09-14T23:58:36.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:36.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315384+0000", 2024-09-14T23:58:36.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3780999999999999e-05 2024-09-14T23:58:36.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:36.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315468+0000", 2024-09-14T23:58:36.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4206e-05 2024-09-14T23:58:36.557 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.557 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.557 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.557 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5219 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3338240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:36.557 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:58:36.557 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 117.258954397, 2024-09-14T23:58:36.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057637764000000001, 2024-09-14T23:58:36.557 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:36.557 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:36.557 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:36.557 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:36.557 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:36.558 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:36.558 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5219 2024-09-14T23:58:36.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.558 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:36.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:36.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:58:36.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:36.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:58:36.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:36.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:58:36.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.03e-07 2024-09-14T23:58:36.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:36.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257983+0000", 2024-09-14T23:58:36.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9199999999999999e-07 2024-09-14T23:58:36.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:36.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257986+0000", 2024-09-14T23:58:36.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0860000000000002e-06 2024-09-14T23:58:36.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:36.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257989+0000", 2024-09-14T23:58:36.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6589999999999999e-06 2024-09-14T23:58:36.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:36.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315509+0000", 2024-09-14T23:58:36.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057520262000000003 2024-09-14T23:58:36.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:36.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315554+0000", 2024-09-14T23:58:36.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5015999999999997e-05 2024-09-14T23:58:36.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:36.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315619+0000", 2024-09-14T23:58:36.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5545999999999999e-05 2024-09-14T23:58:36.561 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.561 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.561 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.561 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5220 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3399680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:36.561 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:58:36.561 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 117.258944441, 2024-09-14T23:58:36.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057792128999999998, 2024-09-14T23:58:36.562 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:36.562 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:36.562 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:36.562 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:36.562 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:36.562 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:36.562 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5220 2024-09-14T23:58:36.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.562 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:36.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:36.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:58:36.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:36.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:58:36.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:36.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:58:36.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1799999999999999e-07 2024-09-14T23:58:36.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:36.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257993+0000", 2024-09-14T23:58:36.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6200000000000006e-07 2024-09-14T23:58:36.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:36.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257996+0000", 2024-09-14T23:58:36.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9050000000000001e-06 2024-09-14T23:58:36.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:36.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257998+0000", 2024-09-14T23:58:36.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5950000000000001e-06 2024-09-14T23:58:36.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:36.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315672+0000", 2024-09-14T23:58:36.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057673637999999999 2024-09-14T23:58:36.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:36.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315734+0000", 2024-09-14T23:58:36.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2216000000000002e-05 2024-09-14T23:58:36.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:36.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315784+0000", 2024-09-14T23:58:36.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9595000000000002e-05 2024-09-14T23:58:36.566 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.566 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.566 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.566 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5221 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3461120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:36.566 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:58:36.567 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 117.258934723, 2024-09-14T23:58:36.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058003557999999997, 2024-09-14T23:58:36.567 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:36.567 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:36.567 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:36.567 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:36.567 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:36.567 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:36.567 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5221 2024-09-14T23:58:36.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.567 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:36.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:36.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:58:36.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:36.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:58:36.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:36.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259905+0000", 2024-09-14T23:58:36.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019035759999999999 2024-09-14T23:58:36.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:36.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259915+0000", 2024-09-14T23:58:36.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0518e-05 2024-09-14T23:58:36.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:36.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259925+0000", 2024-09-14T23:58:36.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7180000000000008e-06 2024-09-14T23:58:36.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:36.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259930+0000", 2024-09-14T23:58:36.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4769999999999997e-06 2024-09-14T23:58:36.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:36.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315820+0000", 2024-09-14T23:58:36.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055889960000000002 2024-09-14T23:58:36.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:36.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315914+0000", 2024-09-14T23:58:36.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4625999999999998e-05 2024-09-14T23:58:36.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:36.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.316005+0000", 2024-09-14T23:58:36.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0682999999999999e-05 2024-09-14T23:58:36.572 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.572 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.572 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.572 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5231 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 4075520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:36.572 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:58:36.572 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 117.256865989, 2024-09-14T23:58:36.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057524733000000002, 2024-09-14T23:58:36.572 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:36.572 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:36.572 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:36.572 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:36.572 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:36.573 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:36.573 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5231 2024-09-14T23:58:36.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.573 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:36.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:36.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:58:36.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:36.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:58:36.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:36.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261904+0000", 2024-09-14T23:58:36.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018338790000000001 2024-09-14T23:58:36.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:36.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261910+0000", 2024-09-14T23:58:36.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9689999999999999e-06 2024-09-14T23:58:36.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:36.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261923+0000", 2024-09-14T23:58:36.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3061000000000001e-05 2024-09-14T23:58:36.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:36.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261931+0000", 2024-09-14T23:58:36.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3159999999999997e-06 2024-09-14T23:58:36.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:36.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317456+0000", 2024-09-14T23:58:36.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055525219000000001 2024-09-14T23:58:36.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:36.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317520+0000", 2024-09-14T23:58:36.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4010000000000005e-05 2024-09-14T23:58:36.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:36.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317595+0000", 2024-09-14T23:58:36.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4278999999999998e-05 2024-09-14T23:58:36.577 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.577 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.577 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.577 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5236 3.8 3:1071668d:::rbd_data.1118de1168b.0000000000000092:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:36.577 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:58:36.577 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 117.254852525, 2024-09-14T23:58:36.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057499462000000001, 2024-09-14T23:58:36.577 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:36.577 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:36.578 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:36.578 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:36.578 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:36.578 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:36.578 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5236 2024-09-14T23:58:36.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.578 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:36.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:36.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:58:36.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:36.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:58:36.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:36.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262084+0000", 2024-09-14T23:58:36.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3399999999999999e-07 2024-09-14T23:58:36.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:36.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262087+0000", 2024-09-14T23:58:36.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2310000000000001e-06 2024-09-14T23:58:36.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:36.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262095+0000", 2024-09-14T23:58:36.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3799999999999996e-06 2024-09-14T23:58:36.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:36.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262101+0000", 2024-09-14T23:58:36.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0009999999999998e-06 2024-09-14T23:58:36.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:36.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317625+0000", 2024-09-14T23:58:36.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055524111000000001 2024-09-14T23:58:36.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:36.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317702+0000", 2024-09-14T23:58:36.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7533999999999993e-05 2024-09-14T23:58:36.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-14T23:58:36.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317892+0000", 2024-09-14T23:58:36.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018991400000000001 2024-09-14T23:58:36.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:58:36.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319407+0000", 2024-09-14T23:58:36.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015145250000000001 2024-09-14T23:58:36.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:36.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319481+0000", 2024-09-14T23:58:36.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3826999999999996e-05 2024-09-14T23:58:36.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:36.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319541+0000", 2024-09-14T23:58:36.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0470999999999998e-05 2024-09-14T23:58:36.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:36.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319583+0000", 2024-09-14T23:58:36.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1934000000000002e-05 2024-09-14T23:58:36.583 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.583 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.583 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.583 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:132622 3.1e 3.7a50b9e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-14T23:58:36.583 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:43.371285+0000", 2024-09-14T23:58:36.583 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 53.145650623000002, 2024-09-14T23:58:36.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.67037987200000004, 2024-09-14T23:58:36.583 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:36.584 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:36.584 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:36.584 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:36.584 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:36.584 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:36.584 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 132622 2024-09-14T23:58:36.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.584 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:36.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:36.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371285+0000", 2024-09-14T23:58:36.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:36.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371285+0000", 2024-09-14T23:58:36.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:36.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371286+0000", 2024-09-14T23:58:36.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6400000000000002e-07 2024-09-14T23:58:36.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:36.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371289+0000", 2024-09-14T23:58:36.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2059999999999999e-06 2024-09-14T23:58:36.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:36.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371299+0000", 2024-09-14T23:58:36.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0288e-05 2024-09-14T23:58:36.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:36.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371306+0000", 2024-09-14T23:58:36.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1650000000000003e-06 2024-09-14T23:58:36.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:36.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371617+0000", 2024-09-14T23:58:36.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000311043 2024-09-14T23:58:36.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:58:36.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371625+0000", 2024-09-14T23:58:36.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.833e-06 2024-09-14T23:58:36.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:36.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.041651+0000", 2024-09-14T23:58:36.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.67002580199999995 2024-09-14T23:58:36.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:36.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.041665+0000", 2024-09-14T23:58:36.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3971000000000001e-05 2024-09-14T23:58:36.588 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.588 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.588 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.588 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:169975 3.5 3:a704a2f0:::rbd_data.1118427720d0.0000000000000030:32 [read 969728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-14T23:58:36.588 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:03.433958+0000", 2024-09-14T23:58:36.588 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 33.082978267000001, 2024-09-14T23:58:36.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057564811, 2024-09-14T23:58:36.588 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:36.588 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:36.588 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:36.588 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:36.588 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:36.588 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:36.589 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 169975 2024-09-14T23:58:36.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.589 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:36.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:36.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.433958+0000", 2024-09-14T23:58:36.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:36.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.433958+0000", 2024-09-14T23:58:36.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:36.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.435096+0000", 2024-09-14T23:58:36.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001138708 2024-09-14T23:58:36.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:36.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.435101+0000", 2024-09-14T23:58:36.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5050000000000001e-06 2024-09-14T23:58:36.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:36.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.435111+0000", 2024-09-14T23:58:36.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5810000000000003e-06 2024-09-14T23:58:36.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:36.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.435120+0000", 2024-09-14T23:58:36.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0089999999999996e-06 2024-09-14T23:58:36.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:36.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.491345+0000", 2024-09-14T23:58:36.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056225479000000002 2024-09-14T23:58:36.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:36.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.491423+0000", 2024-09-14T23:58:36.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7990000000000004e-05 2024-09-14T23:58:36.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:36.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.491523+0000", 2024-09-14T23:58:36.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9538999999999997e-05 2024-09-14T23:58:36.592 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.592 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.592 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.592 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:172639 3.5 3:a704a2f0:::rbd_data.1118427720d0.0000000000000030:32 [read 1645568~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e107)", 2024-09-14T23:58:36.592 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:03.988365+0000", 2024-09-14T23:58:36.592 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.528570547000001, 2024-09-14T23:58:36.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057556213000000002, 2024-09-14T23:58:36.592 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:36.592 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:36.592 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:36.592 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:36.592 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:36.593 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:36.593 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 172639 2024-09-14T23:58:36.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.593 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:36.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:36.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.988365+0000", 2024-09-14T23:58:36.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:36.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.988365+0000", 2024-09-14T23:58:36.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:36.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.989904+0000", 2024-09-14T23:58:36.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015380859999999999 2024-09-14T23:58:36.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:36.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.989909+0000", 2024-09-14T23:58:36.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2800000000000003e-06 2024-09-14T23:58:36.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:36.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.989925+0000", 2024-09-14T23:58:36.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5781999999999999e-05 2024-09-14T23:58:36.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:36.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.989933+0000", 2024-09-14T23:58:36.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0090000000000007e-06 2024-09-14T23:58:36.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:36.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:04.045741+0000", 2024-09-14T23:58:36.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055808112999999999 2024-09-14T23:58:36.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:36.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:04.045812+0000", 2024-09-14T23:58:36.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1531000000000003e-05 2024-09-14T23:58:36.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:36.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:04.045922+0000", 2024-09-14T23:58:36.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000109412 2024-09-14T23:58:36.597 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.597 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.597 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.597 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184571 3.2 3:46f8d78b:::rbd_data.1118427720d0.0000000000000001:32 [read 1405952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:36.597 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.086404+0000", 2024-09-14T23:58:36.597 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 29.430532190000001, 2024-09-14T23:58:36.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059148573000000003, 2024-09-14T23:58:36.597 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:36.597 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:36.597 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:36.597 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:36.598 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:36.598 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:36.598 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184571 2024-09-14T23:58:36.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.598 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:36.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:36.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.086404+0000", 2024-09-14T23:58:36.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:36.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.086404+0000", 2024-09-14T23:58:36.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:36.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087902+0000", 2024-09-14T23:58:36.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001498625 2024-09-14T23:58:36.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:36.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087907+0000", 2024-09-14T23:58:36.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9130000000000002e-06 2024-09-14T23:58:36.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:36.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087919+0000", 2024-09-14T23:58:36.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1719000000000001e-05 2024-09-14T23:58:36.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:36.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087927+0000", 2024-09-14T23:58:36.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3699999999999995e-06 2024-09-14T23:58:36.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:36.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143711+0000", 2024-09-14T23:58:36.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055783396999999998 2024-09-14T23:58:36.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:36.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143792+0000", 2024-09-14T23:58:36.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1292999999999996e-05 2024-09-14T23:58:36.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:36.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145552+0000", 2024-09-14T23:58:36.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001760256 2024-09-14T23:58:36.602 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.602 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.602 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.602 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184579 3.2 3:43badc91:::rbd_data.1118427720d0.0000000000000002:32 [read 1196032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:36.602 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.088145+0000", 2024-09-14T23:58:36.602 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 29.428790786, 2024-09-14T23:58:36.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057643169000000001, 2024-09-14T23:58:36.602 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:36.603 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:36.603 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:36.603 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:36.603 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:36.603 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:36.603 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184579 2024-09-14T23:58:36.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.603 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:36.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:36.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.088145+0000", 2024-09-14T23:58:36.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:36.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.088145+0000", 2024-09-14T23:58:36.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:36.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089947+0000", 2024-09-14T23:58:36.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001801764 2024-09-14T23:58:36.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:36.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089952+0000", 2024-09-14T23:58:36.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8659999999999997e-06 2024-09-14T23:58:36.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:36.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089966+0000", 2024-09-14T23:58:36.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3929e-05 2024-09-14T23:58:36.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:36.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089974+0000", 2024-09-14T23:58:36.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1429999999999998e-06 2024-09-14T23:58:36.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:36.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145636+0000", 2024-09-14T23:58:36.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055662287999999997 2024-09-14T23:58:36.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:36.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145710+0000", 2024-09-14T23:58:36.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4166999999999996e-05 2024-09-14T23:58:36.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:36.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145788+0000", 2024-09-14T23:58:36.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8011999999999998e-05 2024-09-14T23:58:36.606 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.606 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.606 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.607 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:211800 3.12 3:493125e8:::rbd_data.1118c900944f.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-14T23:58:36.607 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:13.660130+0000", 2024-09-14T23:58:36.607 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.856806460000001, 2024-09-14T23:58:36.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.55364332000000005, 2024-09-14T23:58:36.607 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:36.607 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:36.607 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:36.607 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:36.607 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:36.607 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:36.607 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 211800 2024-09-14T23:58:36.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.607 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:36.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:36.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.660130+0000", 2024-09-14T23:58:36.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:36.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.660130+0000", 2024-09-14T23:58:36.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:36.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.660133+0000", 2024-09-14T23:58:36.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.202e-06 2024-09-14T23:58:36.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:36.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.668135+0000", 2024-09-14T23:58:36.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0080027220000000003 2024-09-14T23:58:36.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:36.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.668172+0000", 2024-09-14T23:58:36.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6295999999999999e-05 2024-09-14T23:58:36.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:36.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.668187+0000", 2024-09-14T23:58:36.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5537e-05 2024-09-14T23:58:36.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:36.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.668284+0000", 2024-09-14T23:58:36.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6297000000000003e-05 2024-09-14T23:58:36.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:58:36.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.668299+0000", 2024-09-14T23:58:36.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5091e-05 2024-09-14T23:58:36.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:36.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:14.195910+0000", 2024-09-14T23:58:36.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.52761157999999997 2024-09-14T23:58:36.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:36.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:14.196106+0000", 2024-09-14T23:58:36.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019597399999999999 2024-09-14T23:58:36.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 0", 2024-09-14T23:58:36.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:14.197262+0000", 2024-09-14T23:58:36.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001155744 2024-09-14T23:58:36.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:58:36.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:14.207440+0000", 2024-09-14T23:58:36.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010178347000000001 2024-09-14T23:58:36.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:36.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:14.213691+0000", 2024-09-14T23:58:36.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0062504420000000002 2024-09-14T23:58:36.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:36.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:14.213735+0000", 2024-09-14T23:58:36.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4385999999999998e-05 2024-09-14T23:58:36.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:36.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:14.213773+0000", 2024-09-14T23:58:36.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7701999999999999e-05 2024-09-14T23:58:36.613 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.613 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.613 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.614 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:251476 3.5 3:a1f6ebd6:::rbd_object_map.11189263ca38.000000000000002a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126)", 2024-09-14T23:58:36.614 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:26.388925+0000", 2024-09-14T23:58:36.614 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.128011427000001, 2024-09-14T23:58:36.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.062808638, 2024-09-14T23:58:36.614 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:36.614 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:36.614 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:36.614 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:36.614 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:36.615 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:36.615 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 251476 2024-09-14T23:58:36.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.615 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:36.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:36.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.388925+0000", 2024-09-14T23:58:36.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:36.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.388925+0000", 2024-09-14T23:58:36.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:36.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:36.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.388926+0000", 2024-09-14T23:58:36.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5200000000000001e-06 2024-09-14T23:58:36.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:36.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.388933+0000", 2024-09-14T23:58:36.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.827e-06 2024-09-14T23:58:36.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:36.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.388945+0000", 2024-09-14T23:58:36.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1701e-05 2024-09-14T23:58:36.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:36.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.388950+0000", 2024-09-14T23:58:36.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6080000000000003e-06 2024-09-14T23:58:36.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:36.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.388999+0000", 2024-09-14T23:58:36.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8711999999999997e-05 2024-09-14T23:58:36.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:58:36.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.389006+0000", 2024-09-14T23:58:36.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.063e-06 2024-09-14T23:58:36.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:36.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.451371+0000", 2024-09-14T23:58:36.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.062365255000000001 2024-09-14T23:58:36.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:36.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.451472+0000", 2024-09-14T23:58:36.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100626 2024-09-14T23:58:36.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:36.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:36.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:36.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.451733+0000", 2024-09-14T23:58:36.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026132599999999998 2024-09-14T23:58:36.619 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.619 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.619 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.619 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:36.619 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:36.619 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:36.621 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T23:58:36.621 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:58:36.619+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:58:36.721 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:36.719+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:36.741 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:36.741 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:58:36.741 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:58:36.741 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:36.750 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T23:58:36.822 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:36.821+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:36.860 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:36.860 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:58:36.860 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:58:36.860 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:58:36.860 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:36.868 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T23:58:36.952 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:36.951+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:36.997 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T23:58:37.022 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:37.022 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:58:37.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:58:37.022 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:58:37.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.022 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186203 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1329152~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:37.023 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.194279+0000", 2024-09-14T23:58:37.023 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 29.825306939000001, 2024-09-14T23:58:37.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8011469710000001, 2024-09-14T23:58:37.023 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:37.023 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:37.023 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:37.023 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:37.023 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:37.023 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:37.023 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186203 2024-09-14T23:58:37.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.023 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:37.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.023 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:37.023 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194279+0000", 2024-09-14T23:58:37.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:37.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.024 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:37.024 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194279+0000", 2024-09-14T23:58:37.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:37.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.024 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:37.024 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194279+0000", 2024-09-14T23:58:37.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8100000000000001e-07 2024-09-14T23:58:37.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.024 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:37.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194291+0000", 2024-09-14T23:58:37.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1559999999999999e-05 2024-09-14T23:58:37.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:37.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194303+0000", 2024-09-14T23:58:37.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2130000000000001e-05 2024-09-14T23:58:37.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:37.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194311+0000", 2024-09-14T23:58:37.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8089999999999993e-06 2024-09-14T23:58:37.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:37.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194370+0000", 2024-09-14T23:58:37.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8739999999999997e-05 2024-09-14T23:58:37.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:37.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194376+0000", 2024-09-14T23:58:37.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2319999999999998e-06 2024-09-14T23:58:37.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:37.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986290+0000", 2024-09-14T23:58:37.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.791914334 2024-09-14T23:58:37.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.027 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.027 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:37.027 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986296+0000", 2024-09-14T23:58:37.027 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6300000000000003e-06 2024-09-14T23:58:37.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.027 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.027 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:37.027 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995193+0000", 2024-09-14T23:58:37.027 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0088975040000001 2024-09-14T23:58:37.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.027 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.027 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:37.027 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995307+0000", 2024-09-14T23:58:37.027 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000113656 2024-09-14T23:58:37.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.028 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:37.028 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995426+0000", 2024-09-14T23:58:37.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118495 2024-09-14T23:58:37.028 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:37.028 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:37.028 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:37.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.028 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186204 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1391616~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:37.028 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.194328+0000", 2024-09-14T23:58:37.028 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 29.825257608000001, 2024-09-14T23:58:37.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8013834310000001, 2024-09-14T23:58:37.029 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:37.029 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:37.029 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:37.029 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:37.029 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:37.029 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:37.029 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186204 2024-09-14T23:58:37.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.029 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:37.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:37.029 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194328+0000", 2024-09-14T23:58:37.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:37.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:37.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194328+0000", 2024-09-14T23:58:37.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:37.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:37.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194328+0000", 2024-09-14T23:58:37.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7100000000000002e-07 2024-09-14T23:58:37.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:37.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194331+0000", 2024-09-14T23:58:37.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.807e-06 2024-09-14T23:58:37.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.031 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:37.031 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194339+0000", 2024-09-14T23:58:37.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9340000000000006e-06 2024-09-14T23:58:37.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.031 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:37.031 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194345+0000", 2024-09-14T23:58:37.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1090000000000003e-06 2024-09-14T23:58:37.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.031 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:37.031 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194412+0000", 2024-09-14T23:58:37.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6229000000000006e-05 2024-09-14T23:58:37.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:37.032 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194422+0000", 2024-09-14T23:58:37.032 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.078e-05 2024-09-14T23:58:37.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:37.032 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986322+0000", 2024-09-14T23:58:37.032 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.79189964899999998 2024-09-14T23:58:37.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:37.032 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986332+0000", 2024-09-14T23:58:37.032 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0095000000000001e-05 2024-09-14T23:58:37.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.033 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:37.033 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995554+0000", 2024-09-14T23:58:37.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0092218630000001 2024-09-14T23:58:37.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.033 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:37.033 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995610+0000", 2024-09-14T23:58:37.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6109000000000001e-05 2024-09-14T23:58:37.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.033 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:37.033 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995711+0000", 2024-09-14T23:58:37.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101285 2024-09-14T23:58:37.034 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:37.034 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:37.034 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:37.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.034 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186205 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1333248~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:37.034 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.194354+0000", 2024-09-14T23:58:37.034 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 29.825231678000002, 2024-09-14T23:58:37.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8015471490000001, 2024-09-14T23:58:37.034 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:37.034 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:37.034 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:37.034 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:37.034 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:37.035 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:37.035 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186205 2024-09-14T23:58:37.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.035 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:37.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:37.035 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194354+0000", 2024-09-14T23:58:37.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:37.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:37.035 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194354+0000", 2024-09-14T23:58:37.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:37.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:37.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194354+0000", 2024-09-14T23:58:37.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.39e-07 2024-09-14T23:58:37.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:37.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194357+0000", 2024-09-14T23:58:37.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3479999999999998e-06 2024-09-14T23:58:37.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:37.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194364+0000", 2024-09-14T23:58:37.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0899999999999999e-06 2024-09-14T23:58:37.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.037 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.037 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:37.037 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194371+0000", 2024-09-14T23:58:37.037 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5530000000000004e-06 2024-09-14T23:58:37.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.037 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.037 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:37.037 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194469+0000", 2024-09-14T23:58:37.037 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8119e-05 2024-09-14T23:58:37.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.037 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.037 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:37.037 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194475+0000", 2024-09-14T23:58:37.037 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7910000000000001e-06 2024-09-14T23:58:37.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.038 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.038 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:37.038 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986367+0000", 2024-09-14T23:58:37.038 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.79189214699999999 2024-09-14T23:58:37.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.038 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.038 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:37.038 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986375+0000", 2024-09-14T23:58:37.038 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7800000000000001e-06 2024-09-14T23:58:37.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.038 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.038 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:37.038 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995746+0000", 2024-09-14T23:58:37.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0093706469999999 2024-09-14T23:58:37.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:37.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995810+0000", 2024-09-14T23:58:37.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4068999999999999e-05 2024-09-14T23:58:37.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:37.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995901+0000", 2024-09-14T23:58:37.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1166000000000001e-05 2024-09-14T23:58:37.039 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:37.039 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:37.039 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:37.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.040 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186206 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1395712~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:37.040 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.194380+0000", 2024-09-14T23:58:37.040 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 29.825205657000001, 2024-09-14T23:58:37.040 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.801678347, 2024-09-14T23:58:37.040 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:37.040 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:37.040 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:37.040 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:37.040 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:37.040 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:37.040 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186206 2024-09-14T23:58:37.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.040 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:37.041 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.041 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:37.041 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194380+0000", 2024-09-14T23:58:37.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:37.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.041 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.041 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:37.041 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194380+0000", 2024-09-14T23:58:37.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:37.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.041 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.041 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:37.041 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194380+0000", 2024-09-14T23:58:37.042 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4400000000000001e-07 2024-09-14T23:58:37.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.042 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.042 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:37.042 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194383+0000", 2024-09-14T23:58:37.042 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3460000000000001e-06 2024-09-14T23:58:37.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.042 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.042 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:37.042 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194390+0000", 2024-09-14T23:58:37.042 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2189999999999998e-06 2024-09-14T23:58:37.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.042 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.042 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:37.042 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194403+0000", 2024-09-14T23:58:37.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2941e-05 2024-09-14T23:58:37.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.043 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:37.043 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194545+0000", 2024-09-14T23:58:37.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014232700000000001 2024-09-14T23:58:37.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.043 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:37.043 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194565+0000", 2024-09-14T23:58:37.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9409999999999999e-05 2024-09-14T23:58:37.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.043 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:37.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986409+0000", 2024-09-14T23:58:37.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.79184416000000002 2024-09-14T23:58:37.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.044 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:37.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986417+0000", 2024-09-14T23:58:37.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.551e-06 2024-09-14T23:58:37.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.044 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:37.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995921+0000", 2024-09-14T23:58:37.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.009504212 2024-09-14T23:58:37.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.045 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:37.045 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995974+0000", 2024-09-14T23:58:37.045 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2617999999999997e-05 2024-09-14T23:58:37.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.045 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.045 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:37.045 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996058+0000", 2024-09-14T23:58:37.045 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4118999999999999e-05 2024-09-14T23:58:37.045 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:37.045 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:37.045 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:37.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.045 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.045 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186207 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1337344~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:37.045 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.194412+0000", 2024-09-14T23:58:37.046 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 29.825173059000001, 2024-09-14T23:58:37.046 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.801848141, 2024-09-14T23:58:37.046 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:37.046 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:37.046 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:37.046 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:37.046 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:37.046 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:37.046 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186207 2024-09-14T23:58:37.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.046 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:37.046 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.046 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:37.046 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194412+0000", 2024-09-14T23:58:37.046 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:37.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.047 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:37.047 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194412+0000", 2024-09-14T23:58:37.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:37.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.047 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:37.047 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194413+0000", 2024-09-14T23:58:37.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5600000000000001e-07 2024-09-14T23:58:37.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.047 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:37.047 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194418+0000", 2024-09-14T23:58:37.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3720000000000004e-06 2024-09-14T23:58:37.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.048 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.048 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:37.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194423+0000", 2024-09-14T23:58:37.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.763e-06 2024-09-14T23:58:37.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.048 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.048 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:37.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194426+0000", 2024-09-14T23:58:37.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1480000000000002e-06 2024-09-14T23:58:37.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.048 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.048 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:37.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194596+0000", 2024-09-14T23:58:37.049 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017009600000000001 2024-09-14T23:58:37.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.049 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.049 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:37.049 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194606+0000", 2024-09-14T23:58:37.049 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6639999999999993e-06 2024-09-14T23:58:37.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.049 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.049 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:37.049 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986460+0000", 2024-09-14T23:58:37.049 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.79185424400000004 2024-09-14T23:58:37.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.049 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.049 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:37.049 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986469+0000", 2024-09-14T23:58:37.050 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4419999999999998e-06 2024-09-14T23:58:37.050 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.050 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.050 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:37.050 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996123+0000", 2024-09-14T23:58:37.050 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0096546070000001 2024-09-14T23:58:37.050 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.050 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.050 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:37.050 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996179+0000", 2024-09-14T23:58:37.050 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5850000000000002e-05 2024-09-14T23:58:37.050 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.050 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.050 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:37.051 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996261+0000", 2024-09-14T23:58:37.051 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1599000000000003e-05 2024-09-14T23:58:37.051 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:37.051 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:37.051 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:37.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.051 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.051 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186208 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1399808~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:37.051 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.194445+0000", 2024-09-14T23:58:37.051 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 29.825140889, 2024-09-14T23:58:37.051 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8019822269999999, 2024-09-14T23:58:37.051 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:37.051 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:37.051 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:37.051 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:37.052 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:37.052 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:37.052 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186208 2024-09-14T23:58:37.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.052 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:37.052 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.052 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:37.052 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194445+0000", 2024-09-14T23:58:37.052 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:37.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.052 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.052 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:37.052 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194445+0000", 2024-09-14T23:58:37.052 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:37.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.053 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:37.053 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978674+0000", 2024-09-14T23:58:37.053 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.784229124 2024-09-14T23:58:37.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.053 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:37.053 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978680+0000", 2024-09-14T23:58:37.053 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7969999999999997e-06 2024-09-14T23:58:37.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.053 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:37.053 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978689+0000", 2024-09-14T23:58:37.053 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6949999999999995e-06 2024-09-14T23:58:37.054 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.054 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.054 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:37.054 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978696+0000", 2024-09-14T23:58:37.054 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1300000000000003e-06 2024-09-14T23:58:37.054 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.054 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.054 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:37.054 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986509+0000", 2024-09-14T23:58:37.054 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0078130969999999998 2024-09-14T23:58:37.054 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.054 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.054 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:37.054 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986518+0000", 2024-09-14T23:58:37.054 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2810000000000008e-06 2024-09-14T23:58:37.055 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.055 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.055 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:37.055 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996280+0000", 2024-09-14T23:58:37.055 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0097623120000001 2024-09-14T23:58:37.055 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.055 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.055 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:37.055 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996337+0000", 2024-09-14T23:58:37.055 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7244000000000003e-05 2024-09-14T23:58:37.055 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.055 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.055 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:37.055 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996427+0000", 2024-09-14T23:58:37.056 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9547000000000002e-05 2024-09-14T23:58:37.056 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:37.056 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:37.056 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:37.056 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.056 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.056 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4172.0:7 3.11 3:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:37.056 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.483386+0000", 2024-09-14T23:58:37.056 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 29.536199463999999, 2024-09-14T23:58:37.056 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.558667389, 2024-09-14T23:58:37.056 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:37.056 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:37.056 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:58:37.056 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:37.056 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4172", 2024-09-14T23:58:37.057 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/2542269203", 2024-09-14T23:58:37.057 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 7 2024-09-14T23:58:37.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.057 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:37.057 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.057 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:37.057 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.483386+0000", 2024-09-14T23:58:37.057 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:37.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.057 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.057 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:37.057 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.483386+0000", 2024-09-14T23:58:37.057 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:37.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.057 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.058 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:37.058 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978674+0000", 2024-09-14T23:58:37.058 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.49528781599999999 2024-09-14T23:58:37.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.058 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.058 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:37.058 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978687+0000", 2024-09-14T23:58:37.058 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3244999999999999e-05 2024-09-14T23:58:37.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.058 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.058 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:37.058 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978699+0000", 2024-09-14T23:58:37.058 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1832e-05 2024-09-14T23:58:37.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.059 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:37.059 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978707+0000", 2024-09-14T23:58:37.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4489999999999999e-06 2024-09-14T23:58:37.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.059 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:37.059 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978801+0000", 2024-09-14T23:58:37.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3511999999999995e-05 2024-09-14T23:58:37.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.059 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:37.059 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978814+0000", 2024-09-14T23:58:37.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2789000000000001e-05 2024-09-14T23:58:37.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.060 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.060 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:37.060 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.041948+0000", 2024-09-14T23:58:37.060 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0631338269999999 2024-09-14T23:58:37.060 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.060 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.060 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:37.060 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.042053+0000", 2024-09-14T23:58:37.060 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105919 2024-09-14T23:58:37.060 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:37.060 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:37.060 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:37.060 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.060 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.061 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187209 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1575936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:37.061 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.991888+0000", 2024-09-14T23:58:37.061 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 29.027697493000002, 2024-09-14T23:58:37.061 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0712115470000001, 2024-09-14T23:58:37.061 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:37.061 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:37.061 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:37.061 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:37.061 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:37.061 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:37.061 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187209 2024-09-14T23:58:37.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.061 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:37.061 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.062 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:37.062 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991888+0000", 2024-09-14T23:58:37.062 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:37.062 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.062 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.062 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:37.062 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991888+0000", 2024-09-14T23:58:37.062 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:37.062 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.062 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.062 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:37.062 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991888+0000", 2024-09-14T23:58:37.062 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1199999999999998e-07 2024-09-14T23:58:37.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:37.063 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991890+0000", 2024-09-14T23:58:37.063 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9810000000000002e-06 2024-09-14T23:58:37.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:37.063 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991897+0000", 2024-09-14T23:58:37.063 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2609999999999999e-06 2024-09-14T23:58:37.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:37.063 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991902+0000", 2024-09-14T23:58:37.063 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0000000000000004e-06 2024-09-14T23:58:37.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:37.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995686+0000", 2024-09-14T23:58:37.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0037839290000000001 2024-09-14T23:58:37.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:37.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995693+0000", 2024-09-14T23:58:37.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7589999999999997e-06 2024-09-14T23:58:37.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:37.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062846+0000", 2024-09-14T23:58:37.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.067152417 2024-09-14T23:58:37.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.065 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.065 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:37.065 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062935+0000", 2024-09-14T23:58:37.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9080999999999996e-05 2024-09-14T23:58:37.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.065 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.065 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:37.065 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.063100+0000", 2024-09-14T23:58:37.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016470699999999999 2024-09-14T23:58:37.065 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:37.065 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:37.065 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:37.066 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.066 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.066 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187210 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1591296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:37.066 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.991908+0000", 2024-09-14T23:58:37.066 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 29.027677579999999, 2024-09-14T23:58:37.066 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0713882530000001, 2024-09-14T23:58:37.066 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:37.066 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:37.066 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:37.066 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:37.066 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:37.066 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:37.067 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187210 2024-09-14T23:58:37.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.067 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:37.067 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.067 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:37.067 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991908+0000", 2024-09-14T23:58:37.067 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:37.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.067 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.067 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:37.067 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991908+0000", 2024-09-14T23:58:37.067 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:37.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.068 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.068 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:37.068 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993673+0000", 2024-09-14T23:58:37.068 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017651030000000001 2024-09-14T23:58:37.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.068 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.068 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:37.068 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993681+0000", 2024-09-14T23:58:37.068 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.858e-06 2024-09-14T23:58:37.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.068 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.068 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:37.068 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993691+0000", 2024-09-14T23:58:37.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9920000000000002e-06 2024-09-14T23:58:37.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.069 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.069 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:37.069 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993698+0000", 2024-09-14T23:58:37.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6490000000000002e-06 2024-09-14T23:58:37.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.069 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.069 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:37.069 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995731+0000", 2024-09-14T23:58:37.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0020338959999999999 2024-09-14T23:58:37.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.069 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.069 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:37.069 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995741+0000", 2024-09-14T23:58:37.070 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3079999999999997e-06 2024-09-14T23:58:37.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.070 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:37.070 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.063130+0000", 2024-09-14T23:58:37.070 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.067389159 2024-09-14T23:58:37.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.070 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:37.070 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.063206+0000", 2024-09-14T23:58:37.070 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6577999999999998e-05 2024-09-14T23:58:37.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.070 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:37.070 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.063296+0000", 2024-09-14T23:58:37.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9710000000000007e-05 2024-09-14T23:58:37.071 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:37.071 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:37.071 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:37.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.071 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.071 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187216 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1603584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:37.071 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.993831+0000", 2024-09-14T23:58:37.071 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 29.025754374000002, 2024-09-14T23:58:37.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.071244627, 2024-09-14T23:58:37.071 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:37.071 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:37.071 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:37.071 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:37.072 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:37.072 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:37.072 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187216 2024-09-14T23:58:37.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.072 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:37.072 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.072 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:37.072 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993831+0000", 2024-09-14T23:58:37.072 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:37.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.072 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.072 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:37.072 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993831+0000", 2024-09-14T23:58:37.072 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:37.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.073 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.073 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:37.073 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993832+0000", 2024-09-14T23:58:37.073 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9700000000000002e-07 2024-09-14T23:58:37.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.073 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.073 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:37.073 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993834+0000", 2024-09-14T23:58:37.073 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6170000000000001e-06 2024-09-14T23:58:37.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.073 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.073 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:37.073 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993841+0000", 2024-09-14T23:58:37.073 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6980000000000001e-06 2024-09-14T23:58:37.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.074 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.074 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:37.074 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993846+0000", 2024-09-14T23:58:37.074 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4190000000000001e-06 2024-09-14T23:58:37.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.074 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.074 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:37.074 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995988+0000", 2024-09-14T23:58:37.074 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0021417390000000001 2024-09-14T23:58:37.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.074 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.074 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:37.074 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995998+0000", 2024-09-14T23:58:37.075 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5100000000000004e-06 2024-09-14T23:58:37.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.075 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.075 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:37.075 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.064883+0000", 2024-09-14T23:58:37.075 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.06888597 2024-09-14T23:58:37.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.075 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.075 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:37.075 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.064977+0000", 2024-09-14T23:58:37.075 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3817e-05 2024-09-14T23:58:37.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.075 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.075 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:37.075 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065076+0000", 2024-09-14T23:58:37.075 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8460000000000003e-05 2024-09-14T23:58:37.076 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:37.076 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:37.076 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:37.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.076 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.076 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187217 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1592320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:37.076 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.993855+0000", 2024-09-14T23:58:37.076 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 29.025730940999999, 2024-09-14T23:58:37.076 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0714239349999999, 2024-09-14T23:58:37.076 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:37.076 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:37.076 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:37.076 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:37.077 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:37.077 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:37.077 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187217 2024-09-14T23:58:37.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.077 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:37.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.077 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:37.077 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993855+0000", 2024-09-14T23:58:37.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:37.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.077 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:37.077 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993855+0000", 2024-09-14T23:58:37.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:37.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.078 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.078 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:37.078 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993855+0000", 2024-09-14T23:58:37.078 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4899999999999995e-07 2024-09-14T23:58:37.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.078 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.078 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:37.078 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993857+0000", 2024-09-14T23:58:37.078 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3379999999999999e-06 2024-09-14T23:58:37.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.078 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.078 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:37.078 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993864+0000", 2024-09-14T23:58:37.078 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6000000000000003e-06 2024-09-14T23:58:37.079 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.079 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.079 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:37.079 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993869+0000", 2024-09-14T23:58:37.079 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3210000000000004e-06 2024-09-14T23:58:37.079 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.079 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.079 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:37.079 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996038+0000", 2024-09-14T23:58:37.079 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0021683050000000001 2024-09-14T23:58:37.079 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.079 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.079 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:37.079 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996048+0000", 2024-09-14T23:58:37.080 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0173999999999999e-05 2024-09-14T23:58:37.080 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.080 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.080 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:37.080 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065108+0000", 2024-09-14T23:58:37.080 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0690600619999999 2024-09-14T23:58:37.080 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.080 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.080 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:37.080 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065171+0000", 2024-09-14T23:58:37.080 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3423999999999994e-05 2024-09-14T23:58:37.080 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.080 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.080 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:37.080 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065279+0000", 2024-09-14T23:58:37.081 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000107162 2024-09-14T23:58:37.081 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:37.081 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:37.081 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:37.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.081 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.081 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187218 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1607680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:37.081 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.993877+0000", 2024-09-14T23:58:37.081 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 29.025708660999999, 2024-09-14T23:58:37.081 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0716976810000001, 2024-09-14T23:58:37.081 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:37.081 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:37.081 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:37.081 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:37.082 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:37.082 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:37.082 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187218 2024-09-14T23:58:37.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.082 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:37.082 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.082 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:37.082 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993877+0000", 2024-09-14T23:58:37.082 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:37.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.082 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.082 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:37.082 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993877+0000", 2024-09-14T23:58:37.082 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:37.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.083 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.083 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:37.083 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993877+0000", 2024-09-14T23:58:37.083 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8299999999999998e-07 2024-09-14T23:58:37.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.083 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.083 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:37.083 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993879+0000", 2024-09-14T23:58:37.083 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5659999999999999e-06 2024-09-14T23:58:37.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.083 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.083 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:37.083 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993882+0000", 2024-09-14T23:58:37.083 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7280000000000001e-06 2024-09-14T23:58:37.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.084 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.084 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:37.084 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993885+0000", 2024-09-14T23:58:37.084 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.751e-06 2024-09-14T23:58:37.084 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.084 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.084 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:37.084 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996081+0000", 2024-09-14T23:58:37.084 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0021960590000000002 2024-09-14T23:58:37.084 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.084 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.084 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:37.084 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996091+0000", 2024-09-14T23:58:37.084 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4930000000000004e-06 2024-09-14T23:58:37.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.085 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:37.085 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065308+0000", 2024-09-14T23:58:37.085 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.069217407 2024-09-14T23:58:37.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.085 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:37.085 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065414+0000", 2024-09-14T23:58:37.085 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105438 2024-09-14T23:58:37.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.085 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:37.085 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065575+0000", 2024-09-14T23:58:37.085 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000160956 2024-09-14T23:58:37.086 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:37.086 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:37.086 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:37.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.086 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.086 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187219 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1596416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:37.086 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.993889+0000", 2024-09-14T23:58:37.086 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 29.025696819, 2024-09-14T23:58:37.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0720031190000001, 2024-09-14T23:58:37.086 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:37.086 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:37.086 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:37.086 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:37.086 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:37.086 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:37.087 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187219 2024-09-14T23:58:37.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.087 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:37.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.087 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:37.087 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993889+0000", 2024-09-14T23:58:37.087 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:37.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.087 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:37.087 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993889+0000", 2024-09-14T23:58:37.087 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:37.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.087 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:37.088 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993889+0000", 2024-09-14T23:58:37.088 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2399999999999999e-07 2024-09-14T23:58:37.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.088 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.088 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:37.088 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993890+0000", 2024-09-14T23:58:37.088 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1939999999999999e-06 2024-09-14T23:58:37.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.088 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.088 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:37.088 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993893+0000", 2024-09-14T23:58:37.088 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1300000000000001e-06 2024-09-14T23:58:37.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.088 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.089 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:37.089 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993896+0000", 2024-09-14T23:58:37.089 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6879999999999999e-06 2024-09-14T23:58:37.089 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.089 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.089 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:37.089 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996119+0000", 2024-09-14T23:58:37.089 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0022226149999999998 2024-09-14T23:58:37.089 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.089 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.089 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:37.089 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996123+0000", 2024-09-14T23:58:37.089 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.211e-06 2024-09-14T23:58:37.089 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.089 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.090 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:37.090 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065616+0000", 2024-09-14T23:58:37.090 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0694935240000001 2024-09-14T23:58:37.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.090 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.090 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:37.090 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065733+0000", 2024-09-14T23:58:37.090 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000116492 2024-09-14T23:58:37.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.090 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.090 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:37.090 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065892+0000", 2024-09-14T23:58:37.090 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015894100000000001 2024-09-14T23:58:37.090 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:37.091 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:37.091 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:37.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.091 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187220 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1611776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:37.091 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.993899+0000", 2024-09-14T23:58:37.091 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 29.025686361000002, 2024-09-14T23:58:37.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.072298041, 2024-09-14T23:58:37.091 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:37.091 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:37.091 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:37.091 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:37.091 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:37.091 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:37.091 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187220 2024-09-14T23:58:37.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.092 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:37.092 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.092 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:37.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993899+0000", 2024-09-14T23:58:37.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:37.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.092 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.092 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:37.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993899+0000", 2024-09-14T23:58:37.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:37.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.092 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.092 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:37.093 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993899+0000", 2024-09-14T23:58:37.093 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2100000000000001e-07 2024-09-14T23:58:37.093 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.093 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.093 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:37.093 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993901+0000", 2024-09-14T23:58:37.093 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.141e-06 2024-09-14T23:58:37.093 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.093 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.093 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:37.093 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993904+0000", 2024-09-14T23:58:37.093 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1470000000000001e-06 2024-09-14T23:58:37.093 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.093 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.093 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:37.094 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993906+0000", 2024-09-14T23:58:37.094 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6340000000000001e-06 2024-09-14T23:58:37.094 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.094 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.094 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:37.094 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996160+0000", 2024-09-14T23:58:37.094 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0022537659999999999 2024-09-14T23:58:37.094 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.094 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.094 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:37.094 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996169+0000", 2024-09-14T23:58:37.094 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2739999999999996e-06 2024-09-14T23:58:37.094 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.094 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:37.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065981+0000", 2024-09-14T23:58:37.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.069811742 2024-09-14T23:58:37.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:37.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.066092+0000", 2024-09-14T23:58:37.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000110701 2024-09-14T23:58:37.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:37.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.066197+0000", 2024-09-14T23:58:37.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010541500000000001 2024-09-14T23:58:37.095 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:37.095 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:37.096 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:37.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.096 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187221 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1600512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:37.096 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.993909+0000", 2024-09-14T23:58:37.096 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 29.025676148999999, 2024-09-14T23:58:37.096 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.072643499, 2024-09-14T23:58:37.096 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:37.096 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:37.096 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:37.096 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:37.096 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:37.096 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:37.096 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187221 2024-09-14T23:58:37.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.097 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:37.097 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.097 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:37.097 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993909+0000", 2024-09-14T23:58:37.097 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:37.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.097 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.097 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:37.097 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993909+0000", 2024-09-14T23:58:37.097 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:37.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.097 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.097 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:37.098 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995673+0000", 2024-09-14T23:58:37.098 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001763141 2024-09-14T23:58:37.098 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.098 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.098 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:37.098 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995677+0000", 2024-09-14T23:58:37.098 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8709999999999996e-06 2024-09-14T23:58:37.098 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.098 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.098 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:37.098 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995686+0000", 2024-09-14T23:58:37.098 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6649999999999992e-06 2024-09-14T23:58:37.098 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.098 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.098 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:37.099 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995691+0000", 2024-09-14T23:58:37.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0289999999999996e-06 2024-09-14T23:58:37.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.099 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:37.099 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996199+0000", 2024-09-14T23:58:37.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00050799099999999995 2024-09-14T23:58:37.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.099 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:37.099 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996205+0000", 2024-09-14T23:58:37.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8259999999999998e-06 2024-09-14T23:58:37.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.099 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:37.100 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.066238+0000", 2024-09-14T23:58:37.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.070032928 2024-09-14T23:58:37.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.100 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.100 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:37.100 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.066393+0000", 2024-09-14T23:58:37.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015549800000000001 2024-09-14T23:58:37.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.100 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.100 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:37.100 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.066553+0000", 2024-09-14T23:58:37.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015955 2024-09-14T23:58:37.100 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:37.101 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:37.101 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:37.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.101 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187223 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1604608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:37.101 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.995728+0000", 2024-09-14T23:58:37.101 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 29.023857332999999, 2024-09-14T23:58:37.101 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.071486478, 2024-09-14T23:58:37.101 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:37.101 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:37.101 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:37.101 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:37.101 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:37.101 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:37.101 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187223 2024-09-14T23:58:37.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.102 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:37.102 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.102 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:37.102 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995728+0000", 2024-09-14T23:58:37.102 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:37.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.102 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.102 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:37.102 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995728+0000", 2024-09-14T23:58:37.102 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:37.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.102 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.102 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:37.102 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995728+0000", 2024-09-14T23:58:37.103 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.65e-07 2024-09-14T23:58:37.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.103 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.103 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:37.103 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995730+0000", 2024-09-14T23:58:37.103 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.406e-06 2024-09-14T23:58:37.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.103 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.103 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:37.103 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995734+0000", 2024-09-14T23:58:37.103 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2640000000000002e-06 2024-09-14T23:58:37.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.103 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.103 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:37.104 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995737+0000", 2024-09-14T23:58:37.104 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0589999999999998e-06 2024-09-14T23:58:37.104 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.104 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.104 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:37.104 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996288+0000", 2024-09-14T23:58:37.104 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00055059599999999998 2024-09-14T23:58:37.104 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.104 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.104 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:37.104 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996294+0000", 2024-09-14T23:58:37.104 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0260000000000004e-06 2024-09-14T23:58:37.104 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.104 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.105 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:37.105 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.066938+0000", 2024-09-14T23:58:37.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0706439510000001 2024-09-14T23:58:37.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.105 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:37.105 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067050+0000", 2024-09-14T23:58:37.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011220100000000001 2024-09-14T23:58:37.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.105 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:37.105 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067215+0000", 2024-09-14T23:58:37.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016470999999999999 2024-09-14T23:58:37.105 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:37.105 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:37.106 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:37.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.106 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187224 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1619968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:37.106 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.995742+0000", 2024-09-14T23:58:37.106 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 29.023843518, 2024-09-14T23:58:37.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0717603609999999, 2024-09-14T23:58:37.106 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:37.106 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:37.106 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:37.106 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:37.106 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:37.106 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:37.106 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187224 2024-09-14T23:58:37.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.107 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:37.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.107 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:37.107 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995742+0000", 2024-09-14T23:58:37.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:37.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.107 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:37.107 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995742+0000", 2024-09-14T23:58:37.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:37.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.107 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:37.107 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995742+0000", 2024-09-14T23:58:37.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2399999999999999e-07 2024-09-14T23:58:37.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:37.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995744+0000", 2024-09-14T23:58:37.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.356e-06 2024-09-14T23:58:37.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:37.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995747+0000", 2024-09-14T23:58:37.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.304e-06 2024-09-14T23:58:37.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:37.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995750+0000", 2024-09-14T23:58:37.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6510000000000001e-06 2024-09-14T23:58:37.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:37.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996327+0000", 2024-09-14T23:58:37.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00057794300000000003 2024-09-14T23:58:37.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:37.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996337+0000", 2024-09-14T23:58:37.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3759999999999997e-06 2024-09-14T23:58:37.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:37.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067244+0000", 2024-09-14T23:58:37.110 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.070907367 2024-09-14T23:58:37.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.110 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.110 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:37.110 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067341+0000", 2024-09-14T23:58:37.110 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7277999999999999e-05 2024-09-14T23:58:37.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.110 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.110 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:37.110 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067502+0000", 2024-09-14T23:58:37.110 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016086200000000001 2024-09-14T23:58:37.110 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:37.110 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:37.110 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:37.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.111 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.111 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187225 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1608704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:37.111 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.995766+0000", 2024-09-14T23:58:37.111 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 29.023819344, 2024-09-14T23:58:37.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.072043649, 2024-09-14T23:58:37.111 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:37.111 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:37.111 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:37.111 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:37.111 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:37.111 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:37.111 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187225 2024-09-14T23:58:37.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.111 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:37.111 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:37.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995766+0000", 2024-09-14T23:58:37.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:37.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:37.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995766+0000", 2024-09-14T23:58:37.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:37.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:37.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995766+0000", 2024-09-14T23:58:37.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9200000000000002e-07 2024-09-14T23:58:37.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.113 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:37.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995768+0000", 2024-09-14T23:58:37.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8679999999999999e-06 2024-09-14T23:58:37.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.113 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:37.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995772+0000", 2024-09-14T23:58:37.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6339999999999999e-06 2024-09-14T23:58:37.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.113 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:37.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995775+0000", 2024-09-14T23:58:37.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8159999999999998e-06 2024-09-14T23:58:37.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.114 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:37.114 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996372+0000", 2024-09-14T23:58:37.114 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00059758800000000003 2024-09-14T23:58:37.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.114 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:37.114 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996377+0000", 2024-09-14T23:58:37.114 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7770000000000002e-06 2024-09-14T23:58:37.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.114 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:37.114 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067534+0000", 2024-09-14T23:58:37.114 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.071156821 2024-09-14T23:58:37.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.115 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:37.115 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067638+0000", 2024-09-14T23:58:37.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000103737 2024-09-14T23:58:37.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.115 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:37.115 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067810+0000", 2024-09-14T23:58:37.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017211599999999999 2024-09-14T23:58:37.115 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:37.115 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:37.115 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:37.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.115 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187226 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1624064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:37.115 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.995778+0000", 2024-09-14T23:58:37.116 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 29.023807165000001, 2024-09-14T23:58:37.116 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.072339081, 2024-09-14T23:58:37.116 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:37.116 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:37.116 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:37.116 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:37.116 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:37.116 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:37.116 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187226 2024-09-14T23:58:37.116 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.116 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:37.116 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.116 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:37.116 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995778+0000", 2024-09-14T23:58:37.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:37.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:37.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995778+0000", 2024-09-14T23:58:37.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:37.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:37.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995779+0000", 2024-09-14T23:58:37.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0699999999999999e-07 2024-09-14T23:58:37.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:37.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995780+0000", 2024-09-14T23:58:37.118 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2139999999999999e-06 2024-09-14T23:58:37.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.118 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.118 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:37.118 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995783+0000", 2024-09-14T23:58:37.118 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1250000000000001e-06 2024-09-14T23:58:37.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.118 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.118 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:37.118 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995787+0000", 2024-09-14T23:58:37.118 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6679999999999999e-06 2024-09-14T23:58:37.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.118 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.118 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:37.118 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996404+0000", 2024-09-14T23:58:37.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00061777499999999999 2024-09-14T23:58:37.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.119 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:37.119 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996414+0000", 2024-09-14T23:58:37.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.431e-06 2024-09-14T23:58:37.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.119 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:37.119 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067844+0000", 2024-09-14T23:58:37.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0714302410000001 2024-09-14T23:58:37.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.119 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:37.119 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067952+0000", 2024-09-14T23:58:37.120 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000107778 2024-09-14T23:58:37.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.120 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.120 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:37.120 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.068117+0000", 2024-09-14T23:58:37.120 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000165642 2024-09-14T23:58:37.120 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:37.120 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:37.120 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:37.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.120 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.120 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187227 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1612800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:37.120 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.995806+0000", 2024-09-14T23:58:37.120 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 29.023779928, 2024-09-14T23:58:37.121 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0726092810000001, 2024-09-14T23:58:37.121 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:37.121 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:37.121 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:37.121 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:37.121 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:37.121 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:37.121 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187227 2024-09-14T23:58:37.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.121 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:37.121 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.121 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:37.121 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995806+0000", 2024-09-14T23:58:37.121 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:37.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.122 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.122 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:37.122 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995806+0000", 2024-09-14T23:58:37.122 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:37.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.122 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.122 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:37.122 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995672+0000", 2024-09-14T23:58:37.122 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99986608099999996 2024-09-14T23:58:37.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.122 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.122 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:37.122 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995677+0000", 2024-09-14T23:58:37.122 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9520000000000002e-06 2024-09-14T23:58:37.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.122 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.123 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:37.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995686+0000", 2024-09-14T23:58:37.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8199999999999992e-06 2024-09-14T23:58:37.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.123 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.123 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:37.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995693+0000", 2024-09-14T23:58:37.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1630000000000001e-06 2024-09-14T23:58:37.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.123 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.123 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:37.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.068149+0000", 2024-09-14T23:58:37.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.072456692000000003 2024-09-14T23:58:37.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.123 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.124 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:37.124 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.068251+0000", 2024-09-14T23:58:37.124 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101963 2024-09-14T23:58:37.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:37.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:37.124 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:37.124 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.068415+0000", 2024-09-14T23:58:37.124 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016360999999999999 2024-09-14T23:58:37.124 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:37.124 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:37.124 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:37.124 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:37.124 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:37.124 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:37.126 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T23:58:37.126 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:37.124+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:37.226 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:37.225+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:37.275 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:37.275 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:58:37.275 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:58:37.275 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:37.283 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T23:58:37.314 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T23:58:37.368 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:37.367+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:37.428 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:37.426+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:37.431 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:37.431 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:58:37.431 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:58:37.431 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:58:37.432 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:37.437 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:58:37.438 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:58:37.440 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T23:58:37.454 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:58:37.455 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:58:37.470 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:58:37.470 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:58:37.487 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:58:37.487 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:58:37.503 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:58:37.504 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:58:37.520 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:58:37.520 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:58:37.529 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:37.527+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:37.536 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:58:37.537 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:58:37.553 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:58:37.553 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:58:37.588 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:37.588 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:37.597 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T23:58:37.629 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:37.628+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:37.730 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:37.728+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:37.745 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:37.745 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:37.754 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T23:58:37.830 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:37.829+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:37.861 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:37.861 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:37.870 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T23:58:37.932 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:37.931+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:37.975 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is unset 2024-09-14T23:58:37.979 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:37.979 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:37.995 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T23:58:38.036 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:38.035+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:38.134 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:38.133+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:38.151 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:38.152 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:38.161 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T23:58:38.235 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:38.234+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:38.268 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:38.268 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:38.276 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T23:58:38.361 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:58:38.360+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:58:38.424 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:38.424 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:38.434 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T23:58:38.437 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:38.435+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:38.538 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:38.536+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:38.545 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:38.545 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:38.553 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T23:58:38.638 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:38.637+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:38.660 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:38.660 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:38.669 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T23:58:38.739 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:58:38.737+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:58:38.776 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:38.776 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:38.785 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T23:58:38.839 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:38.838+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:38.892 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:38.892 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:38.900 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T23:58:38.940 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:38.938+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:39.015 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:39.015 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:39.028 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T23:58:39.071 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:39.070+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:39.141 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:39.139+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:39.149 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:39.149 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:39.158 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T23:58:39.243 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:39.242+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:39.306 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:39.306 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:39.315 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T23:58:39.357 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:39.355+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:39.443 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:39.441+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:39.462 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:39.462 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:39.471 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T23:58:39.544 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:39.543+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:39.565 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T23:58:39.579 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:39.579 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:39.588 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T23:58:39.673 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:39.672+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:39.690 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:58:39.691 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:58:39.708 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:58:39.708 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:58:39.724 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:58:39.724 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:58:39.738 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:39.739 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:39.740 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:58:39.740 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:58:39.744 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:39.743+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:39.748 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T23:58:39.757 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:58:39.758 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:58:39.775 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:58:39.775 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:58:39.793 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:58:39.793 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:58:39.810 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:58:39.810 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:58:39.863 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:39.861+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:39.865 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:39.865 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:39.865 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:58:39.865 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:58:39.865 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:39.874 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T23:58:39.946 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:58:39.945+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:58:40.003 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T23:58:40.032 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:40.032 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:58:40.032 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:58:40.032 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:58:40.032 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.032 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5201 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2232320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:40.032 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:58:40.033 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 120.775933659, 2024-09-14T23:58:40.033 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057643228999999997, 2024-09-14T23:58:40.033 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:40.033 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:40.033 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:40.033 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:40.033 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:40.033 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:40.033 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5201 2024-09-14T23:58:40.033 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.033 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:40.033 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.033 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:40.034 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:58:40.034 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:40.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.034 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.034 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:40.034 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:58:40.034 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:40.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.034 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.034 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:40.034 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:58:40.034 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0699999999999998e-07 2024-09-14T23:58:40.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.035 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.035 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:40.035 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253422+0000", 2024-09-14T23:58:40.035 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.514e-06 2024-09-14T23:58:40.035 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.035 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.035 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:40.035 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253431+0000", 2024-09-14T23:58:40.035 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1919999999999994e-06 2024-09-14T23:58:40.035 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.035 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.035 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:40.035 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253436+0000", 2024-09-14T23:58:40.035 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5350000000000004e-06 2024-09-14T23:58:40.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.036 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.036 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:40.036 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310856+0000", 2024-09-14T23:58:40.036 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057419381999999998 2024-09-14T23:58:40.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.036 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.036 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:40.036 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310953+0000", 2024-09-14T23:58:40.036 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6774000000000006e-05 2024-09-14T23:58:40.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.036 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.036 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:40.036 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311062+0000", 2024-09-14T23:58:40.037 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000109425 2024-09-14T23:58:40.037 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:40.037 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:40.037 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:40.037 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.037 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.037 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5202 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2293760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:40.037 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:58:40.037 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 120.775908911, 2024-09-14T23:58:40.037 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057818255999999998, 2024-09-14T23:58:40.037 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:40.038 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:40.038 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:40.038 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:40.038 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:40.038 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:40.038 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5202 2024-09-14T23:58:40.038 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.038 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:40.038 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.038 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:40.038 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:58:40.039 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:40.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.039 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.039 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:40.039 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:58:40.039 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:40.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.039 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.039 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:40.039 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253573+0000", 2024-09-14T23:58:40.039 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012934000000000001 2024-09-14T23:58:40.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.040 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.040 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:40.040 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253576+0000", 2024-09-14T23:58:40.040 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5310000000000002e-06 2024-09-14T23:58:40.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.040 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.040 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:40.040 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253587+0000", 2024-09-14T23:58:40.040 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0522e-05 2024-09-14T23:58:40.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.040 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.041 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:40.041 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253595+0000", 2024-09-14T23:58:40.041 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8290000000000008e-06 2024-09-14T23:58:40.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.041 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.041 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:40.041 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311093+0000", 2024-09-14T23:58:40.041 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057498312000000003 2024-09-14T23:58:40.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.041 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.041 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:40.042 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311167+0000", 2024-09-14T23:58:40.042 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3582000000000001e-05 2024-09-14T23:58:40.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.042 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.042 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:40.042 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311262+0000", 2024-09-14T23:58:40.042 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5140000000000001e-05 2024-09-14T23:58:40.042 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:40.042 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:40.042 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:40.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.042 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.042 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5203 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2355200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:40.043 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:58:40.043 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 120.775746169, 2024-09-14T23:58:40.043 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057903938000000002, 2024-09-14T23:58:40.043 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:40.043 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:40.043 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:40.043 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:40.043 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:40.043 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:40.043 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5203 2024-09-14T23:58:40.043 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.044 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:40.044 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.044 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:40.044 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:58:40.044 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:40.044 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.044 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.044 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:40.044 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:58:40.044 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:40.044 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.045 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.045 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:40.045 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253902+0000", 2024-09-14T23:58:40.045 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029615400000000002 2024-09-14T23:58:40.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.045 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.045 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:40.045 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253906+0000", 2024-09-14T23:58:40.045 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9249999999999997e-06 2024-09-14T23:58:40.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.045 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.046 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:40.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253913+0000", 2024-09-14T23:58:40.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9099999999999999e-06 2024-09-14T23:58:40.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.046 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:40.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253917+0000", 2024-09-14T23:58:40.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9210000000000002e-06 2024-09-14T23:58:40.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.046 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:40.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311295+0000", 2024-09-14T23:58:40.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057378268000000003 2024-09-14T23:58:40.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:40.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311402+0000", 2024-09-14T23:58:40.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010644100000000001 2024-09-14T23:58:40.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:40.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311510+0000", 2024-09-14T23:58:40.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108319 2024-09-14T23:58:40.047 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:40.048 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:40.048 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:40.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.048 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5204 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2416640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:40.048 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:58:40.048 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 120.775427972, 2024-09-14T23:58:40.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057774860999999997, 2024-09-14T23:58:40.048 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:40.048 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:40.048 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:40.049 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:40.049 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:40.049 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:40.049 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5204 2024-09-14T23:58:40.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.049 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:40.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.049 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:40.049 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:58:40.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:40.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.050 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:40.050 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:58:40.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:40.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.050 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:40.050 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255903+0000", 2024-09-14T23:58:40.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001978807 2024-09-14T23:58:40.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.050 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:40.050 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255907+0000", 2024-09-14T23:58:40.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3610000000000001e-06 2024-09-14T23:58:40.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:40.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255914+0000", 2024-09-14T23:58:40.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.357e-06 2024-09-14T23:58:40.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:40.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255918+0000", 2024-09-14T23:58:40.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1779999999999996e-06 2024-09-14T23:58:40.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.052 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.052 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:40.052 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311541+0000", 2024-09-14T23:58:40.052 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055622697999999998 2024-09-14T23:58:40.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.052 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.052 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:40.052 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311614+0000", 2024-09-14T23:58:40.052 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3102999999999994e-05 2024-09-14T23:58:40.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.053 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:40.053 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311699+0000", 2024-09-14T23:58:40.053 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5357000000000003e-05 2024-09-14T23:58:40.053 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:40.053 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:40.053 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:40.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.053 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5211 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2846720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:40.053 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:58:40.054 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 120.773329758, 2024-09-14T23:58:40.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057528151, 2024-09-14T23:58:40.054 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:40.054 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:40.054 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:40.054 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:40.054 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:40.054 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:40.054 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5211 2024-09-14T23:58:40.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.055 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:40.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:40.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:58:40.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:40.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:40.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:58:40.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:40.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:40.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:58:40.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1399999999999998e-07 2024-09-14T23:58:40.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:40.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256024+0000", 2024-09-14T23:58:40.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.114e-06 2024-09-14T23:58:40.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:40.057 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256028+0000", 2024-09-14T23:58:40.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0840000000000002e-06 2024-09-14T23:58:40.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.057 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:40.057 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256031+0000", 2024-09-14T23:58:40.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6819999999999999e-06 2024-09-14T23:58:40.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.057 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:40.057 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313317+0000", 2024-09-14T23:58:40.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057286295000000001 2024-09-14T23:58:40.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.058 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:40.058 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313403+0000", 2024-09-14T23:58:40.058 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6287999999999998e-05 2024-09-14T23:58:40.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.058 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:40.058 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313551+0000", 2024-09-14T23:58:40.058 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000147374 2024-09-14T23:58:40.058 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:40.058 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:40.058 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:40.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.058 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5212 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2908160~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:40.058 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:58:40.058 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 120.77331856399999, 2024-09-14T23:58:40.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057735679999999998, 2024-09-14T23:58:40.059 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:40.059 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:40.059 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:40.059 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:40.059 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:40.059 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:40.059 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5212 2024-09-14T23:58:40.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.059 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:40.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:40.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:58:40.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:40.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:40.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:58:40.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:40.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:40.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:58:40.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0900000000000001e-07 2024-09-14T23:58:40.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:40.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256035+0000", 2024-09-14T23:58:40.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1200000000000001e-06 2024-09-14T23:58:40.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.061 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:40.061 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256038+0000", 2024-09-14T23:58:40.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.05e-06 2024-09-14T23:58:40.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.061 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:40.061 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256041+0000", 2024-09-14T23:58:40.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.667e-06 2024-09-14T23:58:40.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.061 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:40.061 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313584+0000", 2024-09-14T23:58:40.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057543484999999998 2024-09-14T23:58:40.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.062 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:40.062 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313668+0000", 2024-09-14T23:58:40.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3807000000000006e-05 2024-09-14T23:58:40.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.062 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:40.062 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313770+0000", 2024-09-14T23:58:40.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010134200000000001 2024-09-14T23:58:40.062 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:40.062 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:40.062 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:40.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.063 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5213 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2969600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:40.063 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:58:40.063 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 120.773308477, 2024-09-14T23:58:40.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058145467999999999, 2024-09-14T23:58:40.063 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:40.063 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:40.063 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:40.063 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:40.063 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:40.063 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:40.063 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5213 2024-09-14T23:58:40.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.063 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:40.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:40.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:58:40.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:40.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.064 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:40.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:58:40.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:40.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.064 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:40.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257901+0000", 2024-09-14T23:58:40.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018567519999999999 2024-09-14T23:58:40.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:40.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257904+0000", 2024-09-14T23:58:40.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4850000000000001e-06 2024-09-14T23:58:40.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:40.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257912+0000", 2024-09-14T23:58:40.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2090000000000005e-06 2024-09-14T23:58:40.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:40.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257918+0000", 2024-09-14T23:58:40.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6899999999999997e-06 2024-09-14T23:58:40.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:40.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313816+0000", 2024-09-14T23:58:40.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055898075999999998 2024-09-14T23:58:40.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:40.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.314082+0000", 2024-09-14T23:58:40.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000265386 2024-09-14T23:58:40.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:40.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.314189+0000", 2024-09-14T23:58:40.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010787 2024-09-14T23:58:40.067 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:40.067 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:40.067 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:40.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.067 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5218 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3276800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:40.067 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:58:40.067 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 120.771381468, 2024-09-14T23:58:40.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057497557999999997, 2024-09-14T23:58:40.067 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:40.067 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:40.067 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:40.068 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:40.068 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:40.068 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:40.068 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5218 2024-09-14T23:58:40.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.068 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:40.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:40.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:58:40.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:40.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:40.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:58:40.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:40.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:40.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:58:40.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1899999999999999e-07 2024-09-14T23:58:40.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:40.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257972+0000", 2024-09-14T23:58:40.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.217e-06 2024-09-14T23:58:40.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:40.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257975+0000", 2024-09-14T23:58:40.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9359999999999999e-06 2024-09-14T23:58:40.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:40.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257978+0000", 2024-09-14T23:58:40.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1049999999999999e-06 2024-09-14T23:58:40.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:40.071 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315331+0000", 2024-09-14T23:58:40.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057352093999999999 2024-09-14T23:58:40.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:40.071 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315384+0000", 2024-09-14T23:58:40.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3780999999999999e-05 2024-09-14T23:58:40.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:40.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315468+0000", 2024-09-14T23:58:40.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4206e-05 2024-09-14T23:58:40.072 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:40.072 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:40.072 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:40.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.072 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5219 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3338240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:40.072 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:58:40.072 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 120.771370792, 2024-09-14T23:58:40.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057637764000000001, 2024-09-14T23:58:40.073 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:40.073 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:40.073 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:40.073 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:40.073 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:40.073 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:40.073 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5219 2024-09-14T23:58:40.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.073 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:40.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:40.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:58:40.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:40.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:40.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:58:40.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:40.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:40.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:58:40.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.03e-07 2024-09-14T23:58:40.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:40.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257983+0000", 2024-09-14T23:58:40.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9199999999999999e-07 2024-09-14T23:58:40.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:40.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257986+0000", 2024-09-14T23:58:40.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0860000000000002e-06 2024-09-14T23:58:40.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:40.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257989+0000", 2024-09-14T23:58:40.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6589999999999999e-06 2024-09-14T23:58:40.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:40.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315509+0000", 2024-09-14T23:58:40.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057520262000000003 2024-09-14T23:58:40.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:40.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315554+0000", 2024-09-14T23:58:40.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5015999999999997e-05 2024-09-14T23:58:40.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:40.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315619+0000", 2024-09-14T23:58:40.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5545999999999999e-05 2024-09-14T23:58:40.077 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:40.077 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:40.078 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:40.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.078 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5220 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3399680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:40.078 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:58:40.078 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 120.771360836, 2024-09-14T23:58:40.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057792128999999998, 2024-09-14T23:58:40.078 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:40.078 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:40.078 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:40.078 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:40.079 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:40.079 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:40.079 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5220 2024-09-14T23:58:40.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.079 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:40.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:40.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:58:40.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:40.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:40.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:58:40.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:40.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:40.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:58:40.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1799999999999999e-07 2024-09-14T23:58:40.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:40.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257993+0000", 2024-09-14T23:58:40.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6200000000000006e-07 2024-09-14T23:58:40.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:40.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257996+0000", 2024-09-14T23:58:40.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9050000000000001e-06 2024-09-14T23:58:40.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:40.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257998+0000", 2024-09-14T23:58:40.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5950000000000001e-06 2024-09-14T23:58:40.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:40.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315672+0000", 2024-09-14T23:58:40.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057673637999999999 2024-09-14T23:58:40.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:40.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315734+0000", 2024-09-14T23:58:40.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2216000000000002e-05 2024-09-14T23:58:40.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:40.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315784+0000", 2024-09-14T23:58:40.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9595000000000002e-05 2024-09-14T23:58:40.083 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:40.084 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:40.084 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:40.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.084 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5221 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3461120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:40.084 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:58:40.084 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 120.771351118, 2024-09-14T23:58:40.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058003557999999997, 2024-09-14T23:58:40.084 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:40.084 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:40.085 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:40.085 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:40.085 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:40.085 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:40.085 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5221 2024-09-14T23:58:40.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.085 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:40.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:40.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:58:40.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:40.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:40.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:58:40.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:40.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:40.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259905+0000", 2024-09-14T23:58:40.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019035759999999999 2024-09-14T23:58:40.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:40.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259915+0000", 2024-09-14T23:58:40.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0518e-05 2024-09-14T23:58:40.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:40.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259925+0000", 2024-09-14T23:58:40.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7180000000000008e-06 2024-09-14T23:58:40.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:40.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259930+0000", 2024-09-14T23:58:40.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4769999999999997e-06 2024-09-14T23:58:40.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.087 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:40.087 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315820+0000", 2024-09-14T23:58:40.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055889960000000002 2024-09-14T23:58:40.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:40.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315914+0000", 2024-09-14T23:58:40.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4625999999999998e-05 2024-09-14T23:58:40.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:40.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.316005+0000", 2024-09-14T23:58:40.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0682999999999999e-05 2024-09-14T23:58:40.088 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:40.088 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:40.088 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:40.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.089 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5231 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 4075520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:40.089 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:58:40.089 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 120.76928238399999, 2024-09-14T23:58:40.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057524733000000002, 2024-09-14T23:58:40.089 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:40.089 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:40.089 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:40.089 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:40.089 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:40.089 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:40.089 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5231 2024-09-14T23:58:40.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.089 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:40.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:40.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:58:40.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:40.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:40.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:58:40.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:40.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:40.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261904+0000", 2024-09-14T23:58:40.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018338790000000001 2024-09-14T23:58:40.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:40.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261910+0000", 2024-09-14T23:58:40.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9689999999999999e-06 2024-09-14T23:58:40.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:40.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261923+0000", 2024-09-14T23:58:40.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3061000000000001e-05 2024-09-14T23:58:40.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:40.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261931+0000", 2024-09-14T23:58:40.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3159999999999997e-06 2024-09-14T23:58:40.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:40.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317456+0000", 2024-09-14T23:58:40.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055525219000000001 2024-09-14T23:58:40.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:40.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317520+0000", 2024-09-14T23:58:40.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4010000000000005e-05 2024-09-14T23:58:40.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:40.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317595+0000", 2024-09-14T23:58:40.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4278999999999998e-05 2024-09-14T23:58:40.093 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:40.093 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:40.093 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:40.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.093 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5236 3.8 3:1071668d:::rbd_data.1118de1168b.0000000000000092:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:40.093 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:58:40.093 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 120.76726892000001, 2024-09-14T23:58:40.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057499462000000001, 2024-09-14T23:58:40.093 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:40.093 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:40.093 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:40.093 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:40.094 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:40.094 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:40.094 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5236 2024-09-14T23:58:40.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.094 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:40.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:40.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:58:40.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:40.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:40.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:58:40.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:40.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:40.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262084+0000", 2024-09-14T23:58:40.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3399999999999999e-07 2024-09-14T23:58:40.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:40.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262087+0000", 2024-09-14T23:58:40.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2310000000000001e-06 2024-09-14T23:58:40.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:40.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262095+0000", 2024-09-14T23:58:40.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3799999999999996e-06 2024-09-14T23:58:40.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:40.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262101+0000", 2024-09-14T23:58:40.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0009999999999998e-06 2024-09-14T23:58:40.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:40.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317625+0000", 2024-09-14T23:58:40.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055524111000000001 2024-09-14T23:58:40.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:40.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317702+0000", 2024-09-14T23:58:40.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7533999999999993e-05 2024-09-14T23:58:40.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-14T23:58:40.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317892+0000", 2024-09-14T23:58:40.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018991400000000001 2024-09-14T23:58:40.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:58:40.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319407+0000", 2024-09-14T23:58:40.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015145250000000001 2024-09-14T23:58:40.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:40.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319481+0000", 2024-09-14T23:58:40.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3826999999999996e-05 2024-09-14T23:58:40.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:40.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319541+0000", 2024-09-14T23:58:40.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0470999999999998e-05 2024-09-14T23:58:40.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:40.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319583+0000", 2024-09-14T23:58:40.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1934000000000002e-05 2024-09-14T23:58:40.099 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:40.099 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:40.099 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:40.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.100 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:132622 3.1e 3.7a50b9e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-14T23:58:40.100 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:43.371285+0000", 2024-09-14T23:58:40.100 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 56.658067017999997, 2024-09-14T23:58:40.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.67037987200000004, 2024-09-14T23:58:40.100 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:40.100 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:40.100 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:40.100 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:40.100 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:40.100 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:40.101 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 132622 2024-09-14T23:58:40.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.101 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:40.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:40.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371285+0000", 2024-09-14T23:58:40.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:40.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:40.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371285+0000", 2024-09-14T23:58:40.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:40.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:40.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371286+0000", 2024-09-14T23:58:40.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6400000000000002e-07 2024-09-14T23:58:40.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:40.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371289+0000", 2024-09-14T23:58:40.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2059999999999999e-06 2024-09-14T23:58:40.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:40.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371299+0000", 2024-09-14T23:58:40.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0288e-05 2024-09-14T23:58:40.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:40.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371306+0000", 2024-09-14T23:58:40.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1650000000000003e-06 2024-09-14T23:58:40.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:40.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371617+0000", 2024-09-14T23:58:40.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000311043 2024-09-14T23:58:40.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:58:40.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371625+0000", 2024-09-14T23:58:40.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.833e-06 2024-09-14T23:58:40.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:40.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.041651+0000", 2024-09-14T23:58:40.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.67002580199999995 2024-09-14T23:58:40.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:40.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.041665+0000", 2024-09-14T23:58:40.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3971000000000001e-05 2024-09-14T23:58:40.105 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:40.105 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:40.105 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:40.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.106 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:169975 3.5 3:a704a2f0:::rbd_data.1118427720d0.0000000000000030:32 [read 969728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-14T23:58:40.106 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:03.433958+0000", 2024-09-14T23:58:40.106 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 36.595394661999997, 2024-09-14T23:58:40.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057564811, 2024-09-14T23:58:40.106 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:40.106 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:40.106 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:40.106 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:40.106 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:40.106 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:40.106 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 169975 2024-09-14T23:58:40.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.107 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:40.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:40.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.433958+0000", 2024-09-14T23:58:40.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:40.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:40.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.433958+0000", 2024-09-14T23:58:40.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:40.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:40.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.435096+0000", 2024-09-14T23:58:40.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001138708 2024-09-14T23:58:40.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:40.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.435101+0000", 2024-09-14T23:58:40.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5050000000000001e-06 2024-09-14T23:58:40.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:40.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.435111+0000", 2024-09-14T23:58:40.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5810000000000003e-06 2024-09-14T23:58:40.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:40.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.435120+0000", 2024-09-14T23:58:40.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0089999999999996e-06 2024-09-14T23:58:40.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:40.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.491345+0000", 2024-09-14T23:58:40.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056225479000000002 2024-09-14T23:58:40.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:40.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.491423+0000", 2024-09-14T23:58:40.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7990000000000004e-05 2024-09-14T23:58:40.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:40.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.491523+0000", 2024-09-14T23:58:40.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9538999999999997e-05 2024-09-14T23:58:40.110 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:40.112 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:40.112 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:40.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.112 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:172639 3.5 3:a704a2f0:::rbd_data.1118427720d0.0000000000000030:32 [read 1645568~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e107)", 2024-09-14T23:58:40.112 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:03.988365+0000", 2024-09-14T23:58:40.112 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 36.040986942000004, 2024-09-14T23:58:40.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057556213000000002, 2024-09-14T23:58:40.113 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:40.113 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:40.113 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:40.113 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:40.113 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:40.113 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:40.113 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 172639 2024-09-14T23:58:40.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.113 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:40.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:40.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.988365+0000", 2024-09-14T23:58:40.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:40.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:40.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.988365+0000", 2024-09-14T23:58:40.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:40.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:40.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.989904+0000", 2024-09-14T23:58:40.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015380859999999999 2024-09-14T23:58:40.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:40.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.989909+0000", 2024-09-14T23:58:40.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2800000000000003e-06 2024-09-14T23:58:40.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:40.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.989925+0000", 2024-09-14T23:58:40.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5781999999999999e-05 2024-09-14T23:58:40.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:40.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.989933+0000", 2024-09-14T23:58:40.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0090000000000007e-06 2024-09-14T23:58:40.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:40.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:04.045741+0000", 2024-09-14T23:58:40.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055808112999999999 2024-09-14T23:58:40.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:40.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:04.045812+0000", 2024-09-14T23:58:40.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1531000000000003e-05 2024-09-14T23:58:40.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:40.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:04.045922+0000", 2024-09-14T23:58:40.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000109412 2024-09-14T23:58:40.116 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:40.116 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:40.116 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:40.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.117 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184571 3.2 3:46f8d78b:::rbd_data.1118427720d0.0000000000000001:32 [read 1405952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:40.117 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.086404+0000", 2024-09-14T23:58:40.117 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.942948585000003, 2024-09-14T23:58:40.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059148573000000003, 2024-09-14T23:58:40.117 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:40.117 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:40.117 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:40.117 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:40.117 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:40.117 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:40.117 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184571 2024-09-14T23:58:40.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.118 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:40.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:40.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.086404+0000", 2024-09-14T23:58:40.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:40.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:40.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.086404+0000", 2024-09-14T23:58:40.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:40.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:40.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087902+0000", 2024-09-14T23:58:40.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001498625 2024-09-14T23:58:40.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:40.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087907+0000", 2024-09-14T23:58:40.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9130000000000002e-06 2024-09-14T23:58:40.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:40.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087919+0000", 2024-09-14T23:58:40.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1719000000000001e-05 2024-09-14T23:58:40.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:40.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087927+0000", 2024-09-14T23:58:40.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3699999999999995e-06 2024-09-14T23:58:40.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:40.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143711+0000", 2024-09-14T23:58:40.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055783396999999998 2024-09-14T23:58:40.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:40.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143792+0000", 2024-09-14T23:58:40.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1292999999999996e-05 2024-09-14T23:58:40.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:40.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145552+0000", 2024-09-14T23:58:40.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001760256 2024-09-14T23:58:40.121 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:40.121 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:40.121 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:40.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.121 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184579 3.2 3:43badc91:::rbd_data.1118427720d0.0000000000000002:32 [read 1196032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:40.121 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.088145+0000", 2024-09-14T23:58:40.121 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.941207181000003, 2024-09-14T23:58:40.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057643169000000001, 2024-09-14T23:58:40.121 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:40.122 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:40.122 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:40.122 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:40.122 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:40.122 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:40.122 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184579 2024-09-14T23:58:40.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.122 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:40.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:40.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.088145+0000", 2024-09-14T23:58:40.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:40.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:40.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.088145+0000", 2024-09-14T23:58:40.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:40.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:40.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089947+0000", 2024-09-14T23:58:40.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001801764 2024-09-14T23:58:40.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:40.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089952+0000", 2024-09-14T23:58:40.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8659999999999997e-06 2024-09-14T23:58:40.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:40.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089966+0000", 2024-09-14T23:58:40.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3929e-05 2024-09-14T23:58:40.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:40.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089974+0000", 2024-09-14T23:58:40.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1429999999999998e-06 2024-09-14T23:58:40.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:40.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145636+0000", 2024-09-14T23:58:40.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055662287999999997 2024-09-14T23:58:40.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:40.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145710+0000", 2024-09-14T23:58:40.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4166999999999996e-05 2024-09-14T23:58:40.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:40.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145788+0000", 2024-09-14T23:58:40.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8011999999999998e-05 2024-09-14T23:58:40.125 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:40.125 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:40.125 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:40.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.126 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:211800 3.12 3:493125e8:::rbd_data.1118c900944f.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-14T23:58:40.126 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:13.660130+0000", 2024-09-14T23:58:40.126 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.369222855, 2024-09-14T23:58:40.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.55364332000000005, 2024-09-14T23:58:40.126 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:40.126 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:40.126 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:40.126 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:40.126 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:40.127 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:40.127 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 211800 2024-09-14T23:58:40.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.127 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:40.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:40.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.660130+0000", 2024-09-14T23:58:40.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:40.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:40.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.660130+0000", 2024-09-14T23:58:40.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:40.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:40.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.660133+0000", 2024-09-14T23:58:40.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.202e-06 2024-09-14T23:58:40.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:40.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.668135+0000", 2024-09-14T23:58:40.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0080027220000000003 2024-09-14T23:58:40.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:40.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.668172+0000", 2024-09-14T23:58:40.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6295999999999999e-05 2024-09-14T23:58:40.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:40.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.668187+0000", 2024-09-14T23:58:40.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5537e-05 2024-09-14T23:58:40.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:40.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.668284+0000", 2024-09-14T23:58:40.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6297000000000003e-05 2024-09-14T23:58:40.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:58:40.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.668299+0000", 2024-09-14T23:58:40.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5091e-05 2024-09-14T23:58:40.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:40.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:14.195910+0000", 2024-09-14T23:58:40.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.52761157999999997 2024-09-14T23:58:40.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:40.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:14.196106+0000", 2024-09-14T23:58:40.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019597399999999999 2024-09-14T23:58:40.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 0", 2024-09-14T23:58:40.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:14.197262+0000", 2024-09-14T23:58:40.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001155744 2024-09-14T23:58:40.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:58:40.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:14.207440+0000", 2024-09-14T23:58:40.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010178347000000001 2024-09-14T23:58:40.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:40.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:14.213691+0000", 2024-09-14T23:58:40.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0062504420000000002 2024-09-14T23:58:40.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:40.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:14.213735+0000", 2024-09-14T23:58:40.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4385999999999998e-05 2024-09-14T23:58:40.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:40.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:14.213773+0000", 2024-09-14T23:58:40.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7701999999999999e-05 2024-09-14T23:58:40.134 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:40.134 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:40.134 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:40.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.134 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:251476 3.5 3:a1f6ebd6:::rbd_object_map.11189263ca38.000000000000002a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126)", 2024-09-14T23:58:40.134 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:26.388925+0000", 2024-09-14T23:58:40.134 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.640427821999999, 2024-09-14T23:58:40.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.062808638, 2024-09-14T23:58:40.134 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:40.135 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:40.135 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:40.135 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:40.135 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:40.135 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:40.135 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 251476 2024-09-14T23:58:40.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.135 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:40.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:40.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.388925+0000", 2024-09-14T23:58:40.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:40.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:40.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.388925+0000", 2024-09-14T23:58:40.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:40.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:40.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.388926+0000", 2024-09-14T23:58:40.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5200000000000001e-06 2024-09-14T23:58:40.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:40.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.388933+0000", 2024-09-14T23:58:40.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.827e-06 2024-09-14T23:58:40.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:40.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.388945+0000", 2024-09-14T23:58:40.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1701e-05 2024-09-14T23:58:40.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:40.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.388950+0000", 2024-09-14T23:58:40.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6080000000000003e-06 2024-09-14T23:58:40.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:40.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.388999+0000", 2024-09-14T23:58:40.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8711999999999997e-05 2024-09-14T23:58:40.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:58:40.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.389006+0000", 2024-09-14T23:58:40.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.063e-06 2024-09-14T23:58:40.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:40.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.451371+0000", 2024-09-14T23:58:40.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.062365255000000001 2024-09-14T23:58:40.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:40.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.451472+0000", 2024-09-14T23:58:40.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100626 2024-09-14T23:58:40.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:40.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:40.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:40.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.451733+0000", 2024-09-14T23:58:40.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026132599999999998 2024-09-14T23:58:40.140 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:40.140 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:40.140 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:40.140 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:40.140 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:40.140 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:40.142 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T23:58:40.143 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:40.140+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:40.242 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:40.241+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:40.252 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:40.252 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:58:40.253 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:58:40.253 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:40.261 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T23:58:40.343 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:40.342+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:40.368 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:40.368 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:58:40.368 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:58:40.368 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:58:40.368 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:40.377 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T23:58:40.461 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:58:40.460+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:58:40.526 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:40.527 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:58:40.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:58:40.527 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:58:40.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.527 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186203 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1329152~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:40.527 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.194279+0000", 2024-09-14T23:58:40.527 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 33.329892112000003, 2024-09-14T23:58:40.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8011469710000001, 2024-09-14T23:58:40.527 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:40.527 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:40.527 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:40.527 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:40.528 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:40.528 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:40.528 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186203 2024-09-14T23:58:40.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.528 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:40.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:40.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194279+0000", 2024-09-14T23:58:40.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:40.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:40.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194279+0000", 2024-09-14T23:58:40.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:40.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.529 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:40.529 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194279+0000", 2024-09-14T23:58:40.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8100000000000001e-07 2024-09-14T23:58:40.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.529 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:40.529 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194291+0000", 2024-09-14T23:58:40.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1559999999999999e-05 2024-09-14T23:58:40.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.529 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:40.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194303+0000", 2024-09-14T23:58:40.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2130000000000001e-05 2024-09-14T23:58:40.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.530 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:40.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194311+0000", 2024-09-14T23:58:40.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8089999999999993e-06 2024-09-14T23:58:40.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.530 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:40.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194370+0000", 2024-09-14T23:58:40.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8739999999999997e-05 2024-09-14T23:58:40.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:40.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194376+0000", 2024-09-14T23:58:40.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2319999999999998e-06 2024-09-14T23:58:40.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:40.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986290+0000", 2024-09-14T23:58:40.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.791914334 2024-09-14T23:58:40.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:40.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986296+0000", 2024-09-14T23:58:40.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6300000000000003e-06 2024-09-14T23:58:40.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:40.532 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995193+0000", 2024-09-14T23:58:40.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0088975040000001 2024-09-14T23:58:40.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:40.532 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995307+0000", 2024-09-14T23:58:40.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000113656 2024-09-14T23:58:40.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:40.533 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995426+0000", 2024-09-14T23:58:40.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118495 2024-09-14T23:58:40.533 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:40.533 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:40.533 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:40.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.533 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186204 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1391616~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:40.533 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.194328+0000", 2024-09-14T23:58:40.533 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 33.329842780999996, 2024-09-14T23:58:40.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8013834310000001, 2024-09-14T23:58:40.533 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:40.533 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:40.533 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:40.534 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:40.534 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:40.534 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:40.534 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186204 2024-09-14T23:58:40.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.534 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:40.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:40.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194328+0000", 2024-09-14T23:58:40.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:40.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:40.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194328+0000", 2024-09-14T23:58:40.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:40.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:40.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194328+0000", 2024-09-14T23:58:40.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7100000000000002e-07 2024-09-14T23:58:40.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:40.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194331+0000", 2024-09-14T23:58:40.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.807e-06 2024-09-14T23:58:40.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:40.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194339+0000", 2024-09-14T23:58:40.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9340000000000006e-06 2024-09-14T23:58:40.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:40.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194345+0000", 2024-09-14T23:58:40.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1090000000000003e-06 2024-09-14T23:58:40.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:40.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194412+0000", 2024-09-14T23:58:40.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6229000000000006e-05 2024-09-14T23:58:40.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:40.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194422+0000", 2024-09-14T23:58:40.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.078e-05 2024-09-14T23:58:40.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:40.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986322+0000", 2024-09-14T23:58:40.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.79189964899999998 2024-09-14T23:58:40.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:40.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986332+0000", 2024-09-14T23:58:40.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0095000000000001e-05 2024-09-14T23:58:40.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.538 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:40.538 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995554+0000", 2024-09-14T23:58:40.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0092218630000001 2024-09-14T23:58:40.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.538 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:40.538 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995610+0000", 2024-09-14T23:58:40.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6109000000000001e-05 2024-09-14T23:58:40.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.538 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:40.538 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995711+0000", 2024-09-14T23:58:40.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101285 2024-09-14T23:58:40.538 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:40.539 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:40.539 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:40.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.539 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186205 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1333248~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:40.539 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.194354+0000", 2024-09-14T23:58:40.539 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 33.329816850999997, 2024-09-14T23:58:40.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8015471490000001, 2024-09-14T23:58:40.539 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:40.539 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:40.539 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:40.539 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:40.539 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:40.539 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:40.539 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186205 2024-09-14T23:58:40.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.540 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:40.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:40.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194354+0000", 2024-09-14T23:58:40.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:40.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:40.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194354+0000", 2024-09-14T23:58:40.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:40.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:40.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194354+0000", 2024-09-14T23:58:40.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.39e-07 2024-09-14T23:58:40.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:40.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194357+0000", 2024-09-14T23:58:40.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3479999999999998e-06 2024-09-14T23:58:40.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:40.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194364+0000", 2024-09-14T23:58:40.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0899999999999999e-06 2024-09-14T23:58:40.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:40.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194371+0000", 2024-09-14T23:58:40.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5530000000000004e-06 2024-09-14T23:58:40.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:40.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194469+0000", 2024-09-14T23:58:40.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8119e-05 2024-09-14T23:58:40.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:40.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194475+0000", 2024-09-14T23:58:40.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7910000000000001e-06 2024-09-14T23:58:40.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.543 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:40.543 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986367+0000", 2024-09-14T23:58:40.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.79189214699999999 2024-09-14T23:58:40.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.543 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:40.543 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986375+0000", 2024-09-14T23:58:40.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7800000000000001e-06 2024-09-14T23:58:40.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.543 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:40.543 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995746+0000", 2024-09-14T23:58:40.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0093706469999999 2024-09-14T23:58:40.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:40.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995810+0000", 2024-09-14T23:58:40.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4068999999999999e-05 2024-09-14T23:58:40.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:40.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995901+0000", 2024-09-14T23:58:40.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1166000000000001e-05 2024-09-14T23:58:40.544 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:40.544 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:40.544 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:40.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.545 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186206 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1395712~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:40.545 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.194380+0000", 2024-09-14T23:58:40.545 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 33.32979083, 2024-09-14T23:58:40.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.801678347, 2024-09-14T23:58:40.545 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:40.545 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:40.545 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:40.545 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:40.545 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:40.545 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:40.545 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186206 2024-09-14T23:58:40.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.545 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:40.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:40.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194380+0000", 2024-09-14T23:58:40.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:40.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:40.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194380+0000", 2024-09-14T23:58:40.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:40.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:40.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194380+0000", 2024-09-14T23:58:40.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4400000000000001e-07 2024-09-14T23:58:40.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:40.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194383+0000", 2024-09-14T23:58:40.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3460000000000001e-06 2024-09-14T23:58:40.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:40.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194390+0000", 2024-09-14T23:58:40.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2189999999999998e-06 2024-09-14T23:58:40.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:40.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194403+0000", 2024-09-14T23:58:40.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2941e-05 2024-09-14T23:58:40.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.548 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:40.548 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194545+0000", 2024-09-14T23:58:40.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014232700000000001 2024-09-14T23:58:40.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.548 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.548 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:40.548 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194565+0000", 2024-09-14T23:58:40.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9409999999999999e-05 2024-09-14T23:58:40.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.548 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.548 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:40.548 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986409+0000", 2024-09-14T23:58:40.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.79184416000000002 2024-09-14T23:58:40.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:40.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986417+0000", 2024-09-14T23:58:40.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.551e-06 2024-09-14T23:58:40.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:40.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995921+0000", 2024-09-14T23:58:40.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.009504212 2024-09-14T23:58:40.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:40.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995974+0000", 2024-09-14T23:58:40.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2617999999999997e-05 2024-09-14T23:58:40.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:40.550 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996058+0000", 2024-09-14T23:58:40.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4118999999999999e-05 2024-09-14T23:58:40.550 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:40.550 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:40.550 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:40.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.550 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186207 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1337344~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:40.550 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.194412+0000", 2024-09-14T23:58:40.550 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 33.329758232000003, 2024-09-14T23:58:40.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.801848141, 2024-09-14T23:58:40.551 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:40.551 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:40.551 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:40.551 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:40.551 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:40.551 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:40.551 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186207 2024-09-14T23:58:40.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.551 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:40.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:40.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194412+0000", 2024-09-14T23:58:40.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:40.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.552 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:40.552 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194412+0000", 2024-09-14T23:58:40.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:40.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.552 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.552 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:40.552 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194413+0000", 2024-09-14T23:58:40.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5600000000000001e-07 2024-09-14T23:58:40.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.552 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.552 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:40.552 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194418+0000", 2024-09-14T23:58:40.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3720000000000004e-06 2024-09-14T23:58:40.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.553 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:40.553 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194423+0000", 2024-09-14T23:58:40.553 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.763e-06 2024-09-14T23:58:40.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.553 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:40.553 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194426+0000", 2024-09-14T23:58:40.553 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1480000000000002e-06 2024-09-14T23:58:40.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.553 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:40.553 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194596+0000", 2024-09-14T23:58:40.553 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017009600000000001 2024-09-14T23:58:40.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:40.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194606+0000", 2024-09-14T23:58:40.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6639999999999993e-06 2024-09-14T23:58:40.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:40.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986460+0000", 2024-09-14T23:58:40.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.79185424400000004 2024-09-14T23:58:40.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:40.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986469+0000", 2024-09-14T23:58:40.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4419999999999998e-06 2024-09-14T23:58:40.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:40.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996123+0000", 2024-09-14T23:58:40.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0096546070000001 2024-09-14T23:58:40.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:40.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996179+0000", 2024-09-14T23:58:40.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5850000000000002e-05 2024-09-14T23:58:40.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:40.556 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996261+0000", 2024-09-14T23:58:40.556 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1599000000000003e-05 2024-09-14T23:58:40.556 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:40.556 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:40.556 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:40.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.556 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.556 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186208 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1399808~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:40.556 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.194445+0000", 2024-09-14T23:58:40.556 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 33.329726061999999, 2024-09-14T23:58:40.556 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8019822269999999, 2024-09-14T23:58:40.556 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:40.556 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:40.556 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:40.557 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:40.557 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:40.557 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:40.557 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186208 2024-09-14T23:58:40.557 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.557 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:40.557 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.557 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:40.557 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194445+0000", 2024-09-14T23:58:40.557 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:40.557 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.557 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.557 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:40.557 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194445+0000", 2024-09-14T23:58:40.558 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:40.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.558 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.558 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:40.558 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978674+0000", 2024-09-14T23:58:40.558 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.784229124 2024-09-14T23:58:40.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.558 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.558 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:40.558 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978680+0000", 2024-09-14T23:58:40.558 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7969999999999997e-06 2024-09-14T23:58:40.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.558 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.558 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:40.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978689+0000", 2024-09-14T23:58:40.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6949999999999995e-06 2024-09-14T23:58:40.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:40.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978696+0000", 2024-09-14T23:58:40.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1300000000000003e-06 2024-09-14T23:58:40.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:40.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986509+0000", 2024-09-14T23:58:40.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0078130969999999998 2024-09-14T23:58:40.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:40.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986518+0000", 2024-09-14T23:58:40.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2810000000000008e-06 2024-09-14T23:58:40.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:40.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996280+0000", 2024-09-14T23:58:40.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0097623120000001 2024-09-14T23:58:40.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:40.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996337+0000", 2024-09-14T23:58:40.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7244000000000003e-05 2024-09-14T23:58:40.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.561 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:40.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996427+0000", 2024-09-14T23:58:40.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9547000000000002e-05 2024-09-14T23:58:40.561 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:40.561 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:40.561 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:40.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.561 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4172.0:7 3.11 3:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:40.561 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.483386+0000", 2024-09-14T23:58:40.561 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 33.040784637000002, 2024-09-14T23:58:40.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.558667389, 2024-09-14T23:58:40.561 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:40.561 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:40.561 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:58:40.562 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:40.562 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4172", 2024-09-14T23:58:40.562 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/2542269203", 2024-09-14T23:58:40.562 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 7 2024-09-14T23:58:40.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.562 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:40.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:40.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.483386+0000", 2024-09-14T23:58:40.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:40.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:40.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.483386+0000", 2024-09-14T23:58:40.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:40.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.563 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:40.563 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978674+0000", 2024-09-14T23:58:40.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.49528781599999999 2024-09-14T23:58:40.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.563 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:40.563 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978687+0000", 2024-09-14T23:58:40.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3244999999999999e-05 2024-09-14T23:58:40.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.563 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:40.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978699+0000", 2024-09-14T23:58:40.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1832e-05 2024-09-14T23:58:40.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:40.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978707+0000", 2024-09-14T23:58:40.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4489999999999999e-06 2024-09-14T23:58:40.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:40.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978801+0000", 2024-09-14T23:58:40.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3511999999999995e-05 2024-09-14T23:58:40.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.565 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:40.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978814+0000", 2024-09-14T23:58:40.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2789000000000001e-05 2024-09-14T23:58:40.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.565 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:40.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.041948+0000", 2024-09-14T23:58:40.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0631338269999999 2024-09-14T23:58:40.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.565 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:40.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.042053+0000", 2024-09-14T23:58:40.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105919 2024-09-14T23:58:40.565 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:40.565 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:40.566 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:40.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.566 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187209 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1575936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:40.566 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.991888+0000", 2024-09-14T23:58:40.566 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 32.532282666, 2024-09-14T23:58:40.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0712115470000001, 2024-09-14T23:58:40.566 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:40.566 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:40.566 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:40.566 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:40.566 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:40.566 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:40.566 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187209 2024-09-14T23:58:40.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.567 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:40.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.567 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:40.567 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991888+0000", 2024-09-14T23:58:40.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:40.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.567 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:40.567 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991888+0000", 2024-09-14T23:58:40.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:40.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.567 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:40.567 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991888+0000", 2024-09-14T23:58:40.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1199999999999998e-07 2024-09-14T23:58:40.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.568 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:40.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991890+0000", 2024-09-14T23:58:40.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9810000000000002e-06 2024-09-14T23:58:40.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.568 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:40.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991897+0000", 2024-09-14T23:58:40.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2609999999999999e-06 2024-09-14T23:58:40.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.568 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:40.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991902+0000", 2024-09-14T23:58:40.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0000000000000004e-06 2024-09-14T23:58:40.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.569 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:40.569 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995686+0000", 2024-09-14T23:58:40.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0037839290000000001 2024-09-14T23:58:40.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.569 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:40.569 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995693+0000", 2024-09-14T23:58:40.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7589999999999997e-06 2024-09-14T23:58:40.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.569 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:40.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062846+0000", 2024-09-14T23:58:40.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.067152417 2024-09-14T23:58:40.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.570 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:40.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062935+0000", 2024-09-14T23:58:40.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9080999999999996e-05 2024-09-14T23:58:40.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.570 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:40.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.063100+0000", 2024-09-14T23:58:40.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016470699999999999 2024-09-14T23:58:40.570 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:40.570 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:40.570 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:40.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.571 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187210 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1591296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:40.571 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.991908+0000", 2024-09-14T23:58:40.571 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 32.532262752999998, 2024-09-14T23:58:40.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0713882530000001, 2024-09-14T23:58:40.571 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:40.571 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:40.571 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:40.571 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:40.571 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:40.571 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:40.571 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187210 2024-09-14T23:58:40.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.572 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:40.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:40.572 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991908+0000", 2024-09-14T23:58:40.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:40.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:40.572 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991908+0000", 2024-09-14T23:58:40.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:40.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:40.572 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993673+0000", 2024-09-14T23:58:40.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017651030000000001 2024-09-14T23:58:40.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.573 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:40.573 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993681+0000", 2024-09-14T23:58:40.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.858e-06 2024-09-14T23:58:40.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.573 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:40.573 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993691+0000", 2024-09-14T23:58:40.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9920000000000002e-06 2024-09-14T23:58:40.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.573 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:40.574 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993698+0000", 2024-09-14T23:58:40.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6490000000000002e-06 2024-09-14T23:58:40.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.574 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:40.574 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995731+0000", 2024-09-14T23:58:40.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0020338959999999999 2024-09-14T23:58:40.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.574 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:40.574 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995741+0000", 2024-09-14T23:58:40.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3079999999999997e-06 2024-09-14T23:58:40.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:40.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.063130+0000", 2024-09-14T23:58:40.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.067389159 2024-09-14T23:58:40.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:40.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.063206+0000", 2024-09-14T23:58:40.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6577999999999998e-05 2024-09-14T23:58:40.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:40.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.063296+0000", 2024-09-14T23:58:40.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9710000000000007e-05 2024-09-14T23:58:40.575 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:40.575 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:40.576 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:40.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.576 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187216 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1603584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:40.576 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.993831+0000", 2024-09-14T23:58:40.576 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 32.530339546999997, 2024-09-14T23:58:40.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.071244627, 2024-09-14T23:58:40.576 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:40.576 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:40.576 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:40.576 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:40.576 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:40.576 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:40.576 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187216 2024-09-14T23:58:40.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.577 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:40.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.577 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:40.577 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993831+0000", 2024-09-14T23:58:40.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:40.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.577 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:40.577 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993831+0000", 2024-09-14T23:58:40.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:40.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:40.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993832+0000", 2024-09-14T23:58:40.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9700000000000002e-07 2024-09-14T23:58:40.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:40.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993834+0000", 2024-09-14T23:58:40.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6170000000000001e-06 2024-09-14T23:58:40.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:40.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993841+0000", 2024-09-14T23:58:40.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6980000000000001e-06 2024-09-14T23:58:40.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:40.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993846+0000", 2024-09-14T23:58:40.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4190000000000001e-06 2024-09-14T23:58:40.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:40.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995988+0000", 2024-09-14T23:58:40.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0021417390000000001 2024-09-14T23:58:40.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:40.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995998+0000", 2024-09-14T23:58:40.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5100000000000004e-06 2024-09-14T23:58:40.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:40.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.064883+0000", 2024-09-14T23:58:40.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.06888597 2024-09-14T23:58:40.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:40.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.064977+0000", 2024-09-14T23:58:40.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3817e-05 2024-09-14T23:58:40.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:40.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065076+0000", 2024-09-14T23:58:40.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8460000000000003e-05 2024-09-14T23:58:40.581 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:40.581 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:40.581 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:40.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.581 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187217 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1592320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:40.581 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.993855+0000", 2024-09-14T23:58:40.581 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 32.530316114000001, 2024-09-14T23:58:40.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0714239349999999, 2024-09-14T23:58:40.581 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:40.581 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:40.582 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:40.582 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:40.582 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:40.582 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:40.582 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187217 2024-09-14T23:58:40.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.582 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:40.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:40.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993855+0000", 2024-09-14T23:58:40.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:40.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:40.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993855+0000", 2024-09-14T23:58:40.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:40.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:40.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993855+0000", 2024-09-14T23:58:40.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4899999999999995e-07 2024-09-14T23:58:40.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:40.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993857+0000", 2024-09-14T23:58:40.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3379999999999999e-06 2024-09-14T23:58:40.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.584 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:40.584 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993864+0000", 2024-09-14T23:58:40.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6000000000000003e-06 2024-09-14T23:58:40.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.584 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:40.584 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993869+0000", 2024-09-14T23:58:40.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3210000000000004e-06 2024-09-14T23:58:40.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:40.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996038+0000", 2024-09-14T23:58:40.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0021683050000000001 2024-09-14T23:58:40.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:40.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996048+0000", 2024-09-14T23:58:40.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0173999999999999e-05 2024-09-14T23:58:40.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:40.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065108+0000", 2024-09-14T23:58:40.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0690600619999999 2024-09-14T23:58:40.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:40.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065171+0000", 2024-09-14T23:58:40.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3423999999999994e-05 2024-09-14T23:58:40.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:40.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065279+0000", 2024-09-14T23:58:40.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000107162 2024-09-14T23:58:40.586 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:40.586 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:40.586 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:40.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.587 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187218 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1607680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:40.587 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.993877+0000", 2024-09-14T23:58:40.587 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 32.530293833999998, 2024-09-14T23:58:40.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0716976810000001, 2024-09-14T23:58:40.587 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:40.587 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:40.587 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:40.587 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:40.587 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:40.587 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:40.587 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187218 2024-09-14T23:58:40.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.587 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:40.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:40.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993877+0000", 2024-09-14T23:58:40.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:40.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:40.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993877+0000", 2024-09-14T23:58:40.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:40.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:40.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993877+0000", 2024-09-14T23:58:40.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8299999999999998e-07 2024-09-14T23:58:40.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:40.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993879+0000", 2024-09-14T23:58:40.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5659999999999999e-06 2024-09-14T23:58:40.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:40.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993882+0000", 2024-09-14T23:58:40.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7280000000000001e-06 2024-09-14T23:58:40.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:40.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993885+0000", 2024-09-14T23:58:40.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.751e-06 2024-09-14T23:58:40.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:40.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996081+0000", 2024-09-14T23:58:40.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0021960590000000002 2024-09-14T23:58:40.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:40.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996091+0000", 2024-09-14T23:58:40.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4930000000000004e-06 2024-09-14T23:58:40.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:40.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065308+0000", 2024-09-14T23:58:40.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.069217407 2024-09-14T23:58:40.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:40.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065414+0000", 2024-09-14T23:58:40.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105438 2024-09-14T23:58:40.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:40.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065575+0000", 2024-09-14T23:58:40.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000160956 2024-09-14T23:58:40.591 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:40.591 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:40.592 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:40.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.592 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187219 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1596416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:40.592 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.993889+0000", 2024-09-14T23:58:40.592 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 32.530281991999999, 2024-09-14T23:58:40.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0720031190000001, 2024-09-14T23:58:40.592 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:40.592 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:40.592 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:40.592 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:40.592 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:40.592 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:40.592 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187219 2024-09-14T23:58:40.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.593 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:40.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:40.593 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993889+0000", 2024-09-14T23:58:40.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:40.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:40.593 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993889+0000", 2024-09-14T23:58:40.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:40.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:40.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993889+0000", 2024-09-14T23:58:40.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2399999999999999e-07 2024-09-14T23:58:40.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.594 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:40.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993890+0000", 2024-09-14T23:58:40.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1939999999999999e-06 2024-09-14T23:58:40.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.594 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:40.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993893+0000", 2024-09-14T23:58:40.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1300000000000001e-06 2024-09-14T23:58:40.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.594 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:40.595 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993896+0000", 2024-09-14T23:58:40.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6879999999999999e-06 2024-09-14T23:58:40.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:40.595 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996119+0000", 2024-09-14T23:58:40.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0022226149999999998 2024-09-14T23:58:40.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:40.595 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996123+0000", 2024-09-14T23:58:40.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.211e-06 2024-09-14T23:58:40.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:40.596 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065616+0000", 2024-09-14T23:58:40.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0694935240000001 2024-09-14T23:58:40.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.596 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:40.596 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065733+0000", 2024-09-14T23:58:40.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000116492 2024-09-14T23:58:40.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.596 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:40.596 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065892+0000", 2024-09-14T23:58:40.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015894100000000001 2024-09-14T23:58:40.596 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:40.596 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:40.597 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:40.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.597 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187220 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1611776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:40.597 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.993899+0000", 2024-09-14T23:58:40.597 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 32.530271534000001, 2024-09-14T23:58:40.597 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.072298041, 2024-09-14T23:58:40.597 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:40.597 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:40.597 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:40.597 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:40.597 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:40.597 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:40.597 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187220 2024-09-14T23:58:40.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.598 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:40.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.598 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:40.598 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993899+0000", 2024-09-14T23:58:40.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:40.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.598 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:40.598 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993899+0000", 2024-09-14T23:58:40.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:40.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.598 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:40.599 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993899+0000", 2024-09-14T23:58:40.599 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2100000000000001e-07 2024-09-14T23:58:40.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.599 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:40.599 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993901+0000", 2024-09-14T23:58:40.599 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.141e-06 2024-09-14T23:58:40.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.599 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:40.599 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993904+0000", 2024-09-14T23:58:40.599 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1470000000000001e-06 2024-09-14T23:58:40.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.599 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:40.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993906+0000", 2024-09-14T23:58:40.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6340000000000001e-06 2024-09-14T23:58:40.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.600 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:40.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996160+0000", 2024-09-14T23:58:40.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0022537659999999999 2024-09-14T23:58:40.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.600 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:40.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996169+0000", 2024-09-14T23:58:40.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2739999999999996e-06 2024-09-14T23:58:40.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:40.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065981+0000", 2024-09-14T23:58:40.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.069811742 2024-09-14T23:58:40.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:40.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.066092+0000", 2024-09-14T23:58:40.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000110701 2024-09-14T23:58:40.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:40.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.066197+0000", 2024-09-14T23:58:40.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010541500000000001 2024-09-14T23:58:40.601 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:40.602 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:40.602 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:40.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.602 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187221 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1600512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:40.602 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.993909+0000", 2024-09-14T23:58:40.602 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 32.530261322000001, 2024-09-14T23:58:40.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.072643499, 2024-09-14T23:58:40.602 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:40.602 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:40.602 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:40.602 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:40.602 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:40.602 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:40.603 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187221 2024-09-14T23:58:40.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.603 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:40.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:40.603 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993909+0000", 2024-09-14T23:58:40.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:40.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:40.603 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993909+0000", 2024-09-14T23:58:40.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:40.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:40.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995673+0000", 2024-09-14T23:58:40.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001763141 2024-09-14T23:58:40.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:40.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995677+0000", 2024-09-14T23:58:40.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8709999999999996e-06 2024-09-14T23:58:40.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:40.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995686+0000", 2024-09-14T23:58:40.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6649999999999992e-06 2024-09-14T23:58:40.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:40.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995691+0000", 2024-09-14T23:58:40.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0289999999999996e-06 2024-09-14T23:58:40.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:40.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996199+0000", 2024-09-14T23:58:40.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00050799099999999995 2024-09-14T23:58:40.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:40.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996205+0000", 2024-09-14T23:58:40.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8259999999999998e-06 2024-09-14T23:58:40.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.606 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:40.606 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.066238+0000", 2024-09-14T23:58:40.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.070032928 2024-09-14T23:58:40.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.606 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:40.606 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.066393+0000", 2024-09-14T23:58:40.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015549800000000001 2024-09-14T23:58:40.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.606 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:40.606 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.066553+0000", 2024-09-14T23:58:40.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015955 2024-09-14T23:58:40.606 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:40.606 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:40.607 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:40.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.607 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187223 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1604608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:40.607 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.995728+0000", 2024-09-14T23:58:40.607 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 32.528442505999998, 2024-09-14T23:58:40.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.071486478, 2024-09-14T23:58:40.607 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:40.607 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:40.607 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:40.607 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:40.607 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:40.607 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:40.607 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187223 2024-09-14T23:58:40.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.608 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:40.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:40.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995728+0000", 2024-09-14T23:58:40.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:40.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:40.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995728+0000", 2024-09-14T23:58:40.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:40.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:40.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995728+0000", 2024-09-14T23:58:40.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.65e-07 2024-09-14T23:58:40.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:40.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995730+0000", 2024-09-14T23:58:40.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.406e-06 2024-09-14T23:58:40.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:40.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995734+0000", 2024-09-14T23:58:40.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2640000000000002e-06 2024-09-14T23:58:40.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:40.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995737+0000", 2024-09-14T23:58:40.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0589999999999998e-06 2024-09-14T23:58:40.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:40.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996288+0000", 2024-09-14T23:58:40.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00055059599999999998 2024-09-14T23:58:40.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:40.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996294+0000", 2024-09-14T23:58:40.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0260000000000004e-06 2024-09-14T23:58:40.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.611 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:40.611 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.066938+0000", 2024-09-14T23:58:40.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0706439510000001 2024-09-14T23:58:40.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.611 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:40.611 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067050+0000", 2024-09-14T23:58:40.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011220100000000001 2024-09-14T23:58:40.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.611 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:40.611 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067215+0000", 2024-09-14T23:58:40.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016470999999999999 2024-09-14T23:58:40.611 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:40.611 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:40.612 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:40.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.612 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187224 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1619968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:40.612 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.995742+0000", 2024-09-14T23:58:40.612 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 32.528428691000002, 2024-09-14T23:58:40.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0717603609999999, 2024-09-14T23:58:40.612 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:40.612 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:40.612 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:40.612 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:40.612 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:40.612 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:40.612 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187224 2024-09-14T23:58:40.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.613 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:40.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:40.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995742+0000", 2024-09-14T23:58:40.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:40.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:40.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995742+0000", 2024-09-14T23:58:40.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:40.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:40.614 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995742+0000", 2024-09-14T23:58:40.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2399999999999999e-07 2024-09-14T23:58:40.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.614 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:40.614 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995744+0000", 2024-09-14T23:58:40.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.356e-06 2024-09-14T23:58:40.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.614 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:40.614 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995747+0000", 2024-09-14T23:58:40.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.304e-06 2024-09-14T23:58:40.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.615 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:40.615 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995750+0000", 2024-09-14T23:58:40.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6510000000000001e-06 2024-09-14T23:58:40.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.615 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:40.615 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996327+0000", 2024-09-14T23:58:40.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00057794300000000003 2024-09-14T23:58:40.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.615 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:40.615 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996337+0000", 2024-09-14T23:58:40.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3759999999999997e-06 2024-09-14T23:58:40.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:40.616 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067244+0000", 2024-09-14T23:58:40.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.070907367 2024-09-14T23:58:40.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:40.616 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067341+0000", 2024-09-14T23:58:40.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7277999999999999e-05 2024-09-14T23:58:40.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:40.616 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067502+0000", 2024-09-14T23:58:40.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016086200000000001 2024-09-14T23:58:40.617 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:40.617 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:40.617 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:40.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.617 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187225 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1608704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:40.617 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.995766+0000", 2024-09-14T23:58:40.617 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 32.528404516999998, 2024-09-14T23:58:40.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.072043649, 2024-09-14T23:58:40.617 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:40.617 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:40.617 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:40.617 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:40.617 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:40.617 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:40.618 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187225 2024-09-14T23:58:40.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.618 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:40.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:40.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995766+0000", 2024-09-14T23:58:40.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:40.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:40.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995766+0000", 2024-09-14T23:58:40.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:40.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:40.619 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995766+0000", 2024-09-14T23:58:40.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9200000000000002e-07 2024-09-14T23:58:40.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.619 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:40.619 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995768+0000", 2024-09-14T23:58:40.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8679999999999999e-06 2024-09-14T23:58:40.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.619 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:40.619 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995772+0000", 2024-09-14T23:58:40.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6339999999999999e-06 2024-09-14T23:58:40.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:40.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995775+0000", 2024-09-14T23:58:40.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8159999999999998e-06 2024-09-14T23:58:40.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:40.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996372+0000", 2024-09-14T23:58:40.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00059758800000000003 2024-09-14T23:58:40.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:40.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996377+0000", 2024-09-14T23:58:40.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7770000000000002e-06 2024-09-14T23:58:40.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:40.621 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067534+0000", 2024-09-14T23:58:40.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.071156821 2024-09-14T23:58:40.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:40.621 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067638+0000", 2024-09-14T23:58:40.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000103737 2024-09-14T23:58:40.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:40.621 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067810+0000", 2024-09-14T23:58:40.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017211599999999999 2024-09-14T23:58:40.621 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:40.622 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:40.622 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:40.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.622 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187226 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1624064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:40.622 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.995778+0000", 2024-09-14T23:58:40.622 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 32.528392338000003, 2024-09-14T23:58:40.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.072339081, 2024-09-14T23:58:40.622 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:40.622 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:40.622 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:40.622 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:40.622 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:40.622 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:40.622 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187226 2024-09-14T23:58:40.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.623 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:40.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:40.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995778+0000", 2024-09-14T23:58:40.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:40.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:40.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995778+0000", 2024-09-14T23:58:40.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:40.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:40.624 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995779+0000", 2024-09-14T23:58:40.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0699999999999999e-07 2024-09-14T23:58:40.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.624 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:40.624 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995780+0000", 2024-09-14T23:58:40.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2139999999999999e-06 2024-09-14T23:58:40.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.624 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:40.624 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995783+0000", 2024-09-14T23:58:40.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1250000000000001e-06 2024-09-14T23:58:40.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.625 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:40.625 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995787+0000", 2024-09-14T23:58:40.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6679999999999999e-06 2024-09-14T23:58:40.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.625 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:40.625 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996404+0000", 2024-09-14T23:58:40.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00061777499999999999 2024-09-14T23:58:40.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.625 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:40.625 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996414+0000", 2024-09-14T23:58:40.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.431e-06 2024-09-14T23:58:40.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:40.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067844+0000", 2024-09-14T23:58:40.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0714302410000001 2024-09-14T23:58:40.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:40.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067952+0000", 2024-09-14T23:58:40.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000107778 2024-09-14T23:58:40.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:40.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.068117+0000", 2024-09-14T23:58:40.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000165642 2024-09-14T23:58:40.626 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:40.626 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:40.627 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:40.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.627 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187227 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1612800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:40.627 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.995806+0000", 2024-09-14T23:58:40.627 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 32.528365100999999, 2024-09-14T23:58:40.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0726092810000001, 2024-09-14T23:58:40.627 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:40.627 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:40.627 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:40.627 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:40.627 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:40.627 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:40.627 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187227 2024-09-14T23:58:40.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.628 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:40.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:40.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995806+0000", 2024-09-14T23:58:40.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:40.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:40.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995806+0000", 2024-09-14T23:58:40.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:40.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:40.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995672+0000", 2024-09-14T23:58:40.629 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99986608099999996 2024-09-14T23:58:40.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.629 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:40.629 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995677+0000", 2024-09-14T23:58:40.629 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9520000000000002e-06 2024-09-14T23:58:40.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.629 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:40.629 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995686+0000", 2024-09-14T23:58:40.629 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8199999999999992e-06 2024-09-14T23:58:40.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.629 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:40.629 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995693+0000", 2024-09-14T23:58:40.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1630000000000001e-06 2024-09-14T23:58:40.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:40.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.068149+0000", 2024-09-14T23:58:40.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.072456692000000003 2024-09-14T23:58:40.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:40.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.068251+0000", 2024-09-14T23:58:40.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101963 2024-09-14T23:58:40.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:40.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:40.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:40.631 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.068415+0000", 2024-09-14T23:58:40.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016360999999999999 2024-09-14T23:58:40.631 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:40.631 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:40.631 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:40.631 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:40.631 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:40.631 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:40.633 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T23:58:40.633 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:40.631+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:40.733 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:40.732+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:40.781 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:40.781 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:58:40.781 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:58:40.781 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:40.790 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T23:58:40.874 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:40.873+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:40.934 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:40.933+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:40.937 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:40.937 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:58:40.937 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:58:40.937 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:58:40.938 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:40.946 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T23:58:41.015 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is unset 2024-09-14T23:58:41.035 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:41.034+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:41.104 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:41.104 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:58:41.104 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:58:41.104 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:58:41.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.105 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4284.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-14T23:58:41.105 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:58:41.105 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 138.57767084299999, 2024-09-14T23:58:41.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.004942086, 2024-09-14T23:58:41.105 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:41.105 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:41.105 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:41.105 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:41.105 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4284", 2024-09-14T23:58:41.105 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/4278347926", 2024-09-14T23:58:41.105 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1 2024-09-14T23:58:41.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.106 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:41.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.106 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:41.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:58:41.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.106 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:41.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:58:41.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.106 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:41.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524331+0000", 2024-09-14T23:58:41.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.383e-05 2024-09-14T23:58:41.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.107 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:41.107 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524339+0000", 2024-09-14T23:58:41.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8290000000000008e-06 2024-09-14T23:58:41.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.107 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:41.107 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524359+0000", 2024-09-14T23:58:41.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9998000000000001e-05 2024-09-14T23:58:41.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.107 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:41.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524373+0000", 2024-09-14T23:58:41.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4304000000000001e-05 2024-09-14T23:58:41.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:41.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524488+0000", 2024-09-14T23:58:41.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114663 2024-09-14T23:58:41.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:41.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524501+0000", 2024-09-14T23:58:41.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3349999999999999e-05 2024-09-14T23:58:41.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:41.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.526868+0000", 2024-09-14T23:58:41.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023670090000000002 2024-09-14T23:58:41.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:41.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.527041+0000", 2024-09-14T23:58:41.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000172597 2024-09-14T23:58:41.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-14T23:58:41.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.527384+0000", 2024-09-14T23:58:41.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00034309500000000002 2024-09-14T23:58:41.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.110 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.110 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:41.110 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529103+0000", 2024-09-14T23:58:41.110 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017185220000000001 2024-09-14T23:58:41.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.110 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.110 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:58:41.110 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529188+0000", 2024-09-14T23:58:41.110 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5591000000000001e-05 2024-09-14T23:58:41.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.110 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.110 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:41.110 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529227+0000", 2024-09-14T23:58:41.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8563000000000003e-05 2024-09-14T23:58:41.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.111 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.111 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:41.111 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529259+0000", 2024-09-14T23:58:41.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2734999999999997e-05 2024-09-14T23:58:41.111 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.111 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:41.111 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.111 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.111 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4284.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-14T23:58:41.111 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:23.530683+0000", 2024-09-14T23:58:41.111 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 137.57130517799999, 2024-09-14T23:58:41.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0036133110000000001, 2024-09-14T23:58:41.112 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:41.112 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:41.112 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:41.112 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:41.112 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4284", 2024-09-14T23:58:41.112 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/4278347926", 2024-09-14T23:58:41.112 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5 2024-09-14T23:58:41.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.112 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:41.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:41.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530683+0000", 2024-09-14T23:58:41.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.113 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:41.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530683+0000", 2024-09-14T23:58:41.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.113 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:41.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530685+0000", 2024-09-14T23:58:41.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.075e-06 2024-09-14T23:58:41.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.113 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:41.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530695+0000", 2024-09-14T23:58:41.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0261999999999999e-05 2024-09-14T23:58:41.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.114 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:41.114 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530715+0000", 2024-09-14T23:58:41.114 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9930000000000001e-05 2024-09-14T23:58:41.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.114 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:41.114 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530724+0000", 2024-09-14T23:58:41.114 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.901e-06 2024-09-14T23:58:41.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.114 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:41.114 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530800+0000", 2024-09-14T23:58:41.114 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5873999999999999e-05 2024-09-14T23:58:41.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.115 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-14T23:58:41.115 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530807+0000", 2024-09-14T23:58:41.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2389999999999996e-06 2024-09-14T23:58:41.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.115 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:41.115 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.532494+0000", 2024-09-14T23:58:41.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016871830000000001 2024-09-14T23:58:41.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.115 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:41.115 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.532599+0000", 2024-09-14T23:58:41.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010472 2024-09-14T23:58:41.116 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.116 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.116 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1", 2024-09-14T23:58:41.116 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.532715+0000", 2024-09-14T23:58:41.116 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011568199999999999 2024-09-14T23:58:41.116 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.116 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.116 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:58:41.116 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.533460+0000", 2024-09-14T23:58:41.116 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00074494999999999997 2024-09-14T23:58:41.116 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.116 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.116 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:41.116 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534230+0000", 2024-09-14T23:58:41.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00077036599999999998 2024-09-14T23:58:41.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:41.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534266+0000", 2024-09-14T23:58:41.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5725000000000001e-05 2024-09-14T23:58:41.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:41.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534296+0000", 2024-09-14T23:58:41.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0403999999999999e-05 2024-09-14T23:58:41.117 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.117 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:41.117 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.118 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.118 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56383 3.c e56/49 3:36b698b0:::rbd_data.1118c4b3aebe.0000000000000000:head v 56'194, mlcod=56'192)", 2024-09-14T23:58:41.118 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.270483+0000", 2024-09-14T23:58:41.118 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 94.831505378000003, 2024-09-14T23:58:41.118 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0079685880000000004, 2024-09-14T23:58:41.118 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:41.118 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:41.118 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:41.118 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:41.118 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.118 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:41.118 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.270483+0000", 2024-09-14T23:58:41.118 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.119 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:41.119 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.270483+0000", 2024-09-14T23:58:41.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.119 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:41.119 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.270484+0000", 2024-09-14T23:58:41.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7400000000000002e-07 2024-09-14T23:58:41.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.119 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:41.119 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273680+0000", 2024-09-14T23:58:41.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031966680000000002 2024-09-14T23:58:41.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.120 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.120 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:41.120 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273697+0000", 2024-09-14T23:58:41.120 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6595e-05 2024-09-14T23:58:41.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.120 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.120 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:41.120 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273705+0000", 2024-09-14T23:58:41.120 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7530000000000001e-06 2024-09-14T23:58:41.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.120 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.120 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:41.120 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273763+0000", 2024-09-14T23:58:41.121 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8238e-05 2024-09-14T23:58:41.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.121 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.121 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:41.121 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273781+0000", 2024-09-14T23:58:41.121 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8281e-05 2024-09-14T23:58:41.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.121 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.121 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:41.121 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.278408+0000", 2024-09-14T23:58:41.121 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00462716 2024-09-14T23:58:41.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.121 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.121 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:41.122 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.278451+0000", 2024-09-14T23:58:41.122 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3019000000000003e-05 2024-09-14T23:58:41.122 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.122 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:41.122 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.122 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.122 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56384 3.c e56/49 3:36b698b0:::rbd_data.1118c4b3aebe.0000000000000000:head v 56'196, mlcod=56'192)", 2024-09-14T23:58:41.122 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:58:41.122 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 94.827472705999995, 2024-09-14T23:58:41.122 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0046707529999999997, 2024-09-14T23:58:41.122 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:41.122 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:41.122 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:41.122 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:41.123 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.123 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:41.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:58:41.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.123 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.123 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:41.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:58:41.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.123 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.123 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:41.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:58:41.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1699999999999997e-07 2024-09-14T23:58:41.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.124 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:41.124 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274523+0000", 2024-09-14T23:58:41.124 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9129999999999997e-06 2024-09-14T23:58:41.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.124 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:41.124 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274535+0000", 2024-09-14T23:58:41.124 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1788e-05 2024-09-14T23:58:41.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.124 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:41.125 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274542+0000", 2024-09-14T23:58:41.125 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0539999999999997e-06 2024-09-14T23:58:41.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.125 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.125 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:41.125 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275889+0000", 2024-09-14T23:58:41.125 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00134669 2024-09-14T23:58:41.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.125 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.125 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:41.125 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275917+0000", 2024-09-14T23:58:41.125 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7664000000000001e-05 2024-09-14T23:58:41.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.125 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.125 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:41.126 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279134+0000", 2024-09-14T23:58:41.126 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0032178459999999999 2024-09-14T23:58:41.126 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.126 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.126 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:41.126 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279186+0000", 2024-09-14T23:58:41.126 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1881e-05 2024-09-14T23:58:41.126 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.126 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:41.126 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.126 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.126 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.126 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56381 3.1d e56/49 3:bcf9a8cf:::rbd_data.1118c4b3aebe.0000000000000006:head v 56'210, mlcod=56'208)", 2024-09-14T23:58:41.126 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.275462+0000", 2024-09-14T23:58:41.127 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 94.826526376000004, 2024-09-14T23:58:41.127 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0071310710000000001, 2024-09-14T23:58:41.127 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:41.127 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:41.127 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:41.127 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:41.127 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.127 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:41.127 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275462+0000", 2024-09-14T23:58:41.127 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.127 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.127 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.127 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:41.127 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275462+0000", 2024-09-14T23:58:41.127 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.128 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.128 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:41.128 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275463+0000", 2024-09-14T23:58:41.128 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2699999999999999e-07 2024-09-14T23:58:41.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.128 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.128 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:41.128 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276779+0000", 2024-09-14T23:58:41.128 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0013163649999999999 2024-09-14T23:58:41.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.128 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.128 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:41.129 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276795+0000", 2024-09-14T23:58:41.129 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.59e-05 2024-09-14T23:58:41.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.129 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.129 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:41.129 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276806+0000", 2024-09-14T23:58:41.129 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1106e-05 2024-09-14T23:58:41.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.129 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.129 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:41.129 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276941+0000", 2024-09-14T23:58:41.129 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013477500000000001 2024-09-14T23:58:41.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.129 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.130 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:41.130 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276963+0000", 2024-09-14T23:58:41.130 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2419999999999999e-05 2024-09-14T23:58:41.130 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.130 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.130 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:41.130 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282546+0000", 2024-09-14T23:58:41.130 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0055827100000000003 2024-09-14T23:58:41.130 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.130 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.130 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:41.130 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282593+0000", 2024-09-14T23:58:41.130 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7067999999999997e-05 2024-09-14T23:58:41.130 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.130 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:41.131 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.131 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.131 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56387 3.c e56/49 3:30ee4ba0:::rbd_data.1118c4b3aebe.0000000000000004:head v 56'197, mlcod=56'192)", 2024-09-14T23:58:41.131 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.276498+0000", 2024-09-14T23:58:41.131 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 94.825490281, 2024-09-14T23:58:41.131 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0092192639999999996, 2024-09-14T23:58:41.131 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:41.131 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:41.131 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:41.131 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:41.131 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.131 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:41.131 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276498+0000", 2024-09-14T23:58:41.132 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.132 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.132 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:41.132 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276498+0000", 2024-09-14T23:58:41.132 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.132 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.132 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:41.132 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276499+0000", 2024-09-14T23:58:41.132 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1100000000000004e-07 2024-09-14T23:58:41.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.132 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.133 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:41.133 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279646+0000", 2024-09-14T23:58:41.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031473220000000001 2024-09-14T23:58:41.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.133 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:41.133 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279672+0000", 2024-09-14T23:58:41.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6129999999999999e-05 2024-09-14T23:58:41.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:41.134 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279682+0000", 2024-09-14T23:58:41.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7170000000000003e-06 2024-09-14T23:58:41.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:41.134 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279742+0000", 2024-09-14T23:58:41.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9926999999999997e-05 2024-09-14T23:58:41.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:41.134 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279761+0000", 2024-09-14T23:58:41.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9402000000000002e-05 2024-09-14T23:58:41.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.135 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:41.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.285680+0000", 2024-09-14T23:58:41.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.005918881 2024-09-14T23:58:41.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.135 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:41.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.285717+0000", 2024-09-14T23:58:41.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6974000000000002e-05 2024-09-14T23:58:41.135 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.135 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:41.135 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.136 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56388 3.c e56/49 3:30ee4ba0:::rbd_data.1118c4b3aebe.0000000000000004:head v 56'199, mlcod=56'192)", 2024-09-14T23:58:41.136 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:58:41.136 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 94.821998692999998, 2024-09-14T23:58:41.136 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0060596740000000001, 2024-09-14T23:58:41.136 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:41.136 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:41.136 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:41.136 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:41.136 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.136 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:41.136 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:58:41.136 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.136 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:41.137 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:58:41.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:41.137 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:58:41.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2200000000000003e-07 2024-09-14T23:58:41.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:41.137 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279995+0000", 2024-09-14T23:58:41.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5569999999999997e-06 2024-09-14T23:58:41.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:41.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280004+0000", 2024-09-14T23:58:41.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0080000000000008e-06 2024-09-14T23:58:41.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:41.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280010+0000", 2024-09-14T23:58:41.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5749999999999997e-06 2024-09-14T23:58:41.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:41.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282968+0000", 2024-09-14T23:58:41.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0029579390000000001 2024-09-14T23:58:41.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.139 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:41.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282998+0000", 2024-09-14T23:58:41.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9947999999999999e-05 2024-09-14T23:58:41.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.139 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:41.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.286012+0000", 2024-09-14T23:58:41.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030132900000000001 2024-09-14T23:58:41.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.139 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:41.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.286049+0000", 2024-09-14T23:58:41.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7534999999999999e-05 2024-09-14T23:58:41.140 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.140 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:41.140 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.140 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.140 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.140 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56382 3.1d e56/49 3:bcf9a8cf:::rbd_data.1118c4b3aebe.0000000000000006:head v 56'212, mlcod=56'208)", 2024-09-14T23:58:41.140 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.280285+0000", 2024-09-14T23:58:41.140 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 94.821703009999993, 2024-09-14T23:58:41.140 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035393489999999998, 2024-09-14T23:58:41.140 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:41.140 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:41.140 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:41.140 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:41.140 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.141 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:41.141 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280285+0000", 2024-09-14T23:58:41.141 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.141 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:41.141 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280285+0000", 2024-09-14T23:58:41.141 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.141 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:41.141 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280286+0000", 2024-09-14T23:58:41.141 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5000000000000001e-07 2024-09-14T23:58:41.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.142 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:41.142 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280299+0000", 2024-09-14T23:58:41.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3312000000000001e-05 2024-09-14T23:58:41.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.142 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:41.142 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280311+0000", 2024-09-14T23:58:41.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1677e-05 2024-09-14T23:58:41.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.142 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:41.142 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280318+0000", 2024-09-14T23:58:41.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.742e-06 2024-09-14T23:58:41.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:41.143 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280374+0000", 2024-09-14T23:58:41.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.592e-05 2024-09-14T23:58:41.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:41.143 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280392+0000", 2024-09-14T23:58:41.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8490000000000001e-05 2024-09-14T23:58:41.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:41.143 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.283693+0000", 2024-09-14T23:58:41.144 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033012050000000002 2024-09-14T23:58:41.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.144 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.144 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:41.144 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.283825+0000", 2024-09-14T23:58:41.144 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013105300000000001 2024-09-14T23:58:41.144 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.144 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:41.144 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.144 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.144 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "pg_update_log_missing(3.c epoch 56/49 rep_tid 1493 entries 56'200 (0'0) error 3:370203a3:::rbd_data.1118c4b3aebe.000000000000000d:head by client.4376.0:56396 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 56'192)", 2024-09-14T23:58:41.144 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:58:41.145 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 94.821478553000006, 2024-09-14T23:58:41.145 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057370099999999999, 2024-09-14T23:58:41.145 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:41.145 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:41.145 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:58:41.145 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:41.145 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.145 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:41.145 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:58:41.145 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.145 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.145 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:41.145 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:58:41.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.146 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:41.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:58:41.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0699999999999997e-07 2024-09-14T23:58:41.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.146 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:41.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280519+0000", 2024-09-14T23:58:41.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6260000000000001e-06 2024-09-14T23:58:41.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.146 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:41.147 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280541+0000", 2024-09-14T23:58:41.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1976000000000001e-05 2024-09-14T23:58:41.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.147 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:41.147 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280556+0000", 2024-09-14T23:58:41.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5075000000000001e-05 2024-09-14T23:58:41.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.147 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:41.147 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.285494+0000", 2024-09-14T23:58:41.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0049378690000000001 2024-09-14T23:58:41.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.148 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:41.148 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.286247+0000", 2024-09-14T23:58:41.148 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00075285700000000001 2024-09-14T23:58:41.148 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.148 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:41.148 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.148 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.148 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56400 3.1c e56/49 3:3f5b81ba:::rbd_data.1118c4b3aebe.000000000000000a:head v 56'236, mlcod=56'234)", 2024-09-14T23:58:41.148 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.295064+0000", 2024-09-14T23:58:41.148 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 94.806923760999993, 2024-09-14T23:58:41.148 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0054331589999999999, 2024-09-14T23:58:41.148 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:41.148 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:41.149 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:41.149 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:41.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.149 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:41.149 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.295064+0000", 2024-09-14T23:58:41.149 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.149 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:41.149 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.295064+0000", 2024-09-14T23:58:41.149 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:41.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.295066+0000", 2024-09-14T23:58:41.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.502e-06 2024-09-14T23:58:41.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:41.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296403+0000", 2024-09-14T23:58:41.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001336554 2024-09-14T23:58:41.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:41.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296428+0000", 2024-09-14T23:58:41.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5179000000000001e-05 2024-09-14T23:58:41.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:41.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296440+0000", 2024-09-14T23:58:41.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2387e-05 2024-09-14T23:58:41.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:41.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296554+0000", 2024-09-14T23:58:41.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114269 2024-09-14T23:58:41.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:41.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296581+0000", 2024-09-14T23:58:41.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6673999999999999e-05 2024-09-14T23:58:41.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.152 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:41.152 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.300418+0000", 2024-09-14T23:58:41.152 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0038371249999999998 2024-09-14T23:58:41.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.152 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:41.152 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.300498+0000", 2024-09-14T23:58:41.152 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9468999999999994e-05 2024-09-14T23:58:41.152 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.152 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:41.152 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.152 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:61922 3.5 e57/49 3:a11ebf8e:::rbd_data.11188d356229.000000000000000c:head v 57'280, mlcod=57'277)", 2024-09-14T23:58:41.153 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:07.563144+0000", 2024-09-14T23:58:41.153 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 93.538844108999996, 2024-09-14T23:58:41.153 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003977848, 2024-09-14T23:58:41.153 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:41.153 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:41.153 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:41.153 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:41.153 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.153 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:41.153 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563144+0000", 2024-09-14T23:58:41.153 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.153 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.153 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:41.154 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563144+0000", 2024-09-14T23:58:41.154 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.154 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.154 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:41.154 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563145+0000", 2024-09-14T23:58:41.154 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5400000000000001e-07 2024-09-14T23:58:41.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.154 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.154 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:41.154 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565056+0000", 2024-09-14T23:58:41.154 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0019108160000000001 2024-09-14T23:58:41.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.154 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.154 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:41.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565088+0000", 2024-09-14T23:58:41.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2586999999999998e-05 2024-09-14T23:58:41.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:41.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565100+0000", 2024-09-14T23:58:41.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1917e-05 2024-09-14T23:58:41.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:41.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565166+0000", 2024-09-14T23:58:41.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6245e-05 2024-09-14T23:58:41.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:41.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565190+0000", 2024-09-14T23:58:41.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3825999999999999e-05 2024-09-14T23:58:41.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.156 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:41.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.567080+0000", 2024-09-14T23:58:41.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001889541 2024-09-14T23:58:41.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.156 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:41.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.567122+0000", 2024-09-14T23:58:41.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2262000000000003e-05 2024-09-14T23:58:41.156 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.156 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:41.157 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.157 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.157 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:66564 3.c e60/49 3:33faa7ca:::rbd_data.111873792d9b.0000000000000003:head v 60'236, mlcod=57'233)", 2024-09-14T23:58:41.157 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:11.271596+0000", 2024-09-14T23:58:41.157 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 89.830392531000001, 2024-09-14T23:58:41.157 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0060396099999999999, 2024-09-14T23:58:41.157 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:41.157 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:41.157 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:41.157 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:41.157 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.157 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:41.157 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.271596+0000", 2024-09-14T23:58:41.157 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.158 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.158 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:41.158 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.271596+0000", 2024-09-14T23:58:41.158 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.158 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.158 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:41.158 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.271597+0000", 2024-09-14T23:58:41.158 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2559999999999999e-06 2024-09-14T23:58:41.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.158 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.158 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:41.158 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.274854+0000", 2024-09-14T23:58:41.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0032574470000000001 2024-09-14T23:58:41.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.159 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:41.159 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.274874+0000", 2024-09-14T23:58:41.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9449999999999998e-05 2024-09-14T23:58:41.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.159 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:41.159 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.274883+0000", 2024-09-14T23:58:41.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5980000000000003e-06 2024-09-14T23:58:41.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.159 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:41.160 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.274977+0000", 2024-09-14T23:58:41.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3270999999999995e-05 2024-09-14T23:58:41.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.160 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.160 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:41.160 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.275000+0000", 2024-09-14T23:58:41.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3088e-05 2024-09-14T23:58:41.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.160 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.160 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:41.160 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.277590+0000", 2024-09-14T23:58:41.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0025898660000000001 2024-09-14T23:58:41.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.160 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.161 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:41.161 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.277635+0000", 2024-09-14T23:58:41.161 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5633999999999997e-05 2024-09-14T23:58:41.161 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.161 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:41.161 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.161 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.161 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:133384 3.1f e92/49 3:fce5e529:::rbd_data.11186132a8b6.0000000000000002:head v 92'807, mlcod=89'804)", 2024-09-14T23:58:41.161 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:45.293642+0000", 2024-09-14T23:58:41.161 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 55.808346647999997, 2024-09-14T23:58:41.161 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003760186, 2024-09-14T23:58:41.161 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:41.161 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:41.161 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:41.162 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:41.162 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.162 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:41.162 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.293642+0000", 2024-09-14T23:58:41.162 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.162 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.162 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.162 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:41.162 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.293642+0000", 2024-09-14T23:58:41.162 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.162 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.162 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.162 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:41.162 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.293643+0000", 2024-09-14T23:58:41.162 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2500000000000001e-06 2024-09-14T23:58:41.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.163 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:41.163 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.295402+0000", 2024-09-14T23:58:41.163 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017595410000000001 2024-09-14T23:58:41.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.163 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:41.163 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.295422+0000", 2024-09-14T23:58:41.163 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9763999999999999e-05 2024-09-14T23:58:41.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.163 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:41.163 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.295436+0000", 2024-09-14T23:58:41.164 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.364e-05 2024-09-14T23:58:41.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.164 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.164 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:41.164 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.295499+0000", 2024-09-14T23:58:41.164 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2770999999999999e-05 2024-09-14T23:58:41.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.164 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.164 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:41.164 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.295516+0000", 2024-09-14T23:58:41.164 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7240000000000001e-05 2024-09-14T23:58:41.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.164 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.164 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:41.165 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.297324+0000", 2024-09-14T23:58:41.165 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001807857 2024-09-14T23:58:41.165 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.165 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.165 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:41.165 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.297402+0000", 2024-09-14T23:58:41.165 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8122999999999997e-05 2024-09-14T23:58:41.165 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.165 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:41.165 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.165 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.165 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.165 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:156597 3.5 e103/49 3:a23c5924:::rbd_data.1118b2f495e6.0000000000000011:head v 103'518, mlcod=98'516)", 2024-09-14T23:58:41.165 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:58.656197+0000", 2024-09-14T23:58:41.165 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 42.445791591000003, 2024-09-14T23:58:41.166 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0040423029999999997, 2024-09-14T23:58:41.166 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:41.166 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:41.166 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:41.166 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:41.166 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.166 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:41.166 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656197+0000", 2024-09-14T23:58:41.166 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.166 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.166 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:41.166 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656197+0000", 2024-09-14T23:58:41.166 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.167 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.167 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:41.167 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656197+0000", 2024-09-14T23:58:41.167 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1900000000000002e-07 2024-09-14T23:58:41.167 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.167 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.167 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:41.167 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656676+0000", 2024-09-14T23:58:41.167 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00047921699999999999 2024-09-14T23:58:41.167 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.167 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.167 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:41.167 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656690+0000", 2024-09-14T23:58:41.168 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3916e-05 2024-09-14T23:58:41.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.168 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.168 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:41.168 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656698+0000", 2024-09-14T23:58:41.168 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6529999999999994e-06 2024-09-14T23:58:41.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.168 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.168 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:41.168 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656757+0000", 2024-09-14T23:58:41.168 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9237999999999998e-05 2024-09-14T23:58:41.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.168 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.168 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:41.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656780+0000", 2024-09-14T23:58:41.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2968000000000001e-05 2024-09-14T23:58:41.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.169 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:41.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.660188+0000", 2024-09-14T23:58:41.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0034078630000000001 2024-09-14T23:58:41.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.169 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:41.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.660239+0000", 2024-09-14T23:58:41.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0828999999999997e-05 2024-09-14T23:58:41.169 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.169 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:41.169 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.170 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.170 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:157947 3.1c e103/49 3:383374fe:::rbd_data.1118b2f495e6.000000000000000c:head v 103'771, mlcod=103'769)", 2024-09-14T23:58:41.170 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:58.765271+0000", 2024-09-14T23:58:41.170 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 42.336716709999997, 2024-09-14T23:58:41.170 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0040869829999999998, 2024-09-14T23:58:41.170 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:41.170 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:41.170 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:41.170 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:41.170 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.170 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:41.170 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765271+0000", 2024-09-14T23:58:41.170 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.171 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.171 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:41.171 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765271+0000", 2024-09-14T23:58:41.171 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.171 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.171 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:41.171 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765273+0000", 2024-09-14T23:58:41.171 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1149999999999999e-06 2024-09-14T23:58:41.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.171 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.171 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:41.171 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765854+0000", 2024-09-14T23:58:41.171 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00058119599999999997 2024-09-14T23:58:41.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.172 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.172 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:41.172 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765869+0000", 2024-09-14T23:58:41.172 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5641000000000002e-05 2024-09-14T23:58:41.172 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.172 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.172 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:41.172 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765878+0000", 2024-09-14T23:58:41.172 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4519999999999997e-06 2024-09-14T23:58:41.172 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.172 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.172 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:41.172 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765942+0000", 2024-09-14T23:58:41.172 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3727999999999997e-05 2024-09-14T23:58:41.173 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.173 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.173 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:41.173 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765968+0000", 2024-09-14T23:58:41.173 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6171999999999999e-05 2024-09-14T23:58:41.173 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.173 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.173 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:41.173 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.769319+0000", 2024-09-14T23:58:41.173 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033511629999999999 2024-09-14T23:58:41.173 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.173 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.173 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:41.173 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.769358+0000", 2024-09-14T23:58:41.173 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9515999999999999e-05 2024-09-14T23:58:41.174 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.174 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:41.174 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.174 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.174 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:160196 3.c e103/49 3:34623e68:::rbd_data.1118b2f495e6.0000000000000005:head v 103'551, mlcod=102'549)", 2024-09-14T23:58:41.174 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:59.131433+0000", 2024-09-14T23:58:41.174 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 41.970555400999999, 2024-09-14T23:58:41.174 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0036932530000000001, 2024-09-14T23:58:41.174 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:41.174 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:41.174 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:41.174 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:41.174 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.175 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:41.175 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.131433+0000", 2024-09-14T23:58:41.175 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.175 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.175 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:41.175 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.131433+0000", 2024-09-14T23:58:41.175 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.175 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.175 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:41.175 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.131433+0000", 2024-09-14T23:58:41.175 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8500000000000001e-07 2024-09-14T23:58:41.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.176 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.176 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:41.176 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.132144+0000", 2024-09-14T23:58:41.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00071022800000000003 2024-09-14T23:58:41.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.176 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.176 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:41.176 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.132154+0000", 2024-09-14T23:58:41.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0301e-05 2024-09-14T23:58:41.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.176 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.176 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:41.176 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.132159+0000", 2024-09-14T23:58:41.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6280000000000004e-06 2024-09-14T23:58:41.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.177 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:41.177 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.132208+0000", 2024-09-14T23:58:41.177 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9595000000000002e-05 2024-09-14T23:58:41.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.177 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:41.177 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.132231+0000", 2024-09-14T23:58:41.177 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3010000000000002e-05 2024-09-14T23:58:41.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.177 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:41.177 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.135087+0000", 2024-09-14T23:58:41.177 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0028553239999999998 2024-09-14T23:58:41.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.178 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:41.178 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.135126+0000", 2024-09-14T23:58:41.178 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9481999999999999e-05 2024-09-14T23:58:41.178 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.178 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:41.178 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.178 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188871 3.d 3:b35cff4d:::rbd_header.1118c900944f:head [watch reconnect cookie 140670715433840 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-14T23:58:41.178 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.970653+0000", 2024-09-14T23:58:41.178 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 33.131335667999998, 2024-09-14T23:58:41.178 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0727808270000001, 2024-09-14T23:58:41.178 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:41.179 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:41.179 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:41.179 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:41.179 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:41.179 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:41.179 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188871 2024-09-14T23:58:41.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.179 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:41.179 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.179 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:41.179 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970653+0000", 2024-09-14T23:58:41.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.179 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.180 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:41.180 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970653+0000", 2024-09-14T23:58:41.180 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.180 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.180 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:41.180 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970655+0000", 2024-09-14T23:58:41.180 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.277e-06 2024-09-14T23:58:41.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.180 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.180 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:41.180 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970662+0000", 2024-09-14T23:58:41.180 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9619999999999996e-06 2024-09-14T23:58:41.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.181 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.181 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:41.181 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970682+0000", 2024-09-14T23:58:41.181 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0279999999999999e-05 2024-09-14T23:58:41.181 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.181 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.181 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:41.181 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970691+0000", 2024-09-14T23:58:41.181 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2790000000000004e-06 2024-09-14T23:58:41.181 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.181 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.181 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:41.181 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970762+0000", 2024-09-14T23:58:41.181 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1019999999999994e-05 2024-09-14T23:58:41.181 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.182 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.182 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-14T23:58:41.182 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970770+0000", 2024-09-14T23:58:41.182 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7999999999999999e-06 2024-09-14T23:58:41.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.182 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.182 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:41.182 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.973516+0000", 2024-09-14T23:58:41.182 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0027456469999999999 2024-09-14T23:58:41.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.182 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.182 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:41.182 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.973525+0000", 2024-09-14T23:58:41.182 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4890000000000001e-06 2024-09-14T23:58:41.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.183 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.183 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:41.183 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.041451+0000", 2024-09-14T23:58:41.183 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.067926012 2024-09-14T23:58:41.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.183 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.183 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:41.183 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.041673+0000", 2024-09-14T23:58:41.183 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000222048 2024-09-14T23:58:41.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.183 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.183 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:41.183 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.043433+0000", 2024-09-14T23:58:41.183 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017600129999999999 2024-09-14T23:58:41.184 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.184 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:41.184 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.184 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(osd.0.0:3238 3.1f e122/49 3:f8a59e83:::rbd_data.11187b9f1032.0000000000000006:2f v 122'943, mlcod=120'940)", 2024-09-14T23:58:41.184 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:22.285283+0000", 2024-09-14T23:58:41.184 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.816704845, 2024-09-14T23:58:41.184 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003756194, 2024-09-14T23:58:41.184 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:41.184 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:41.184 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:41.184 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:41.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.185 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:41.185 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.285283+0000", 2024-09-14T23:58:41.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.185 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.185 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:41.185 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.285283+0000", 2024-09-14T23:58:41.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.185 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.185 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:41.185 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.285284+0000", 2024-09-14T23:58:41.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5600000000000002e-07 2024-09-14T23:58:41.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.186 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.186 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:41.186 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.285289+0000", 2024-09-14T23:58:41.186 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9579999999999998e-06 2024-09-14T23:58:41.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.186 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.186 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:41.186 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.285300+0000", 2024-09-14T23:58:41.186 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1688999999999999e-05 2024-09-14T23:58:41.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.186 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.186 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:41.186 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.285309+0000", 2024-09-14T23:58:41.186 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.388e-06 2024-09-14T23:58:41.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.187 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.187 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:41.187 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.285367+0000", 2024-09-14T23:58:41.187 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8811000000000003e-05 2024-09-14T23:58:41.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.187 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.187 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:41.187 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.285388+0000", 2024-09-14T23:58:41.187 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0117000000000001e-05 2024-09-14T23:58:41.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.187 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.187 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:41.187 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.288994+0000", 2024-09-14T23:58:41.188 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0036064270000000002 2024-09-14T23:58:41.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.188 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.188 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:41.188 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.289040+0000", 2024-09-14T23:58:41.188 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5547999999999998e-05 2024-09-14T23:58:41.188 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.188 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:41.188 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.188 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.188 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(osd.6.0:1517 3.d e127/112 3:b69c12aa:::rbd_data.1118f9c96a7.0000000000000007:2c v 127'452, mlcod=126'449)", 2024-09-14T23:58:41.188 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:27.316914+0000", 2024-09-14T23:58:41.188 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.785074237, 2024-09-14T23:58:41.189 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0052453980000000001, 2024-09-14T23:58:41.189 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:41.189 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:41.189 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:41.189 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:41.189 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.189 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:41.189 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.316914+0000", 2024-09-14T23:58:41.189 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.189 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.189 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:41.189 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.316914+0000", 2024-09-14T23:58:41.189 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.190 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.190 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:41.190 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.316914+0000", 2024-09-14T23:58:41.190 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4900000000000002e-07 2024-09-14T23:58:41.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.190 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.190 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:41.190 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.316920+0000", 2024-09-14T23:58:41.190 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7169999999999996e-06 2024-09-14T23:58:41.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.190 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.190 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:41.190 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.316928+0000", 2024-09-14T23:58:41.190 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3769999999999996e-06 2024-09-14T23:58:41.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.191 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.191 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:41.191 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.316938+0000", 2024-09-14T23:58:41.191 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7329999999999998e-06 2024-09-14T23:58:41.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.191 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.191 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:41.191 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.317033+0000", 2024-09-14T23:58:41.191 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5295000000000002e-05 2024-09-14T23:58:41.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.191 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.191 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:41.191 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.317057+0000", 2024-09-14T23:58:41.192 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3989999999999999e-05 2024-09-14T23:58:41.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.192 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.192 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:41.192 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.322122+0000", 2024-09-14T23:58:41.192 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0050642500000000002 2024-09-14T23:58:41.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.192 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.192 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:41.192 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.322159+0000", 2024-09-14T23:58:41.192 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7787000000000003e-05 2024-09-14T23:58:41.192 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.192 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:41.192 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.193 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.193 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.193 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(osd.1.0:1466 3.16 e127/112 3:6c7910cf:::rbd_data.1118f9c96a7.0000000000000006:2c v 127'376, mlcod=121'372)", 2024-09-14T23:58:41.193 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:27.317464+0000", 2024-09-14T23:58:41.193 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 13.784524204, 2024-09-14T23:58:41.193 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0050844499999999999, 2024-09-14T23:58:41.193 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:41.193 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:41.193 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:41.193 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:41.193 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.193 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:41.193 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.317464+0000", 2024-09-14T23:58:41.193 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.193 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.194 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.194 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:41.194 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.317464+0000", 2024-09-14T23:58:41.194 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.194 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.194 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:41.194 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.317464+0000", 2024-09-14T23:58:41.194 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.91e-07 2024-09-14T23:58:41.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.194 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.194 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:41.194 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.317469+0000", 2024-09-14T23:58:41.194 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4719999999999997e-06 2024-09-14T23:58:41.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.195 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.195 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:41.195 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.317481+0000", 2024-09-14T23:58:41.195 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1814000000000001e-05 2024-09-14T23:58:41.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.195 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.195 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:41.195 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.317488+0000", 2024-09-14T23:58:41.195 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2740000000000002e-06 2024-09-14T23:58:41.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.195 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.195 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:41.195 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.317584+0000", 2024-09-14T23:58:41.195 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6191000000000001e-05 2024-09-14T23:58:41.196 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.196 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.196 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:41.196 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.317605+0000", 2024-09-14T23:58:41.196 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0554e-05 2024-09-14T23:58:41.196 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.196 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.196 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:41.196 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.322504+0000", 2024-09-14T23:58:41.196 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0048991089999999996 2024-09-14T23:58:41.196 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.196 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.196 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:41.196 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.322548+0000", 2024-09-14T23:58:41.197 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4744999999999999e-05 2024-09-14T23:58:41.197 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.197 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:41.197 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.197 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.197 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:41.197 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:41.198 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T23:58:41.199 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:41.197+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:41.299 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:58:41.298+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:58:41.346 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:41.346 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:58:41.346 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:58:41.346 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:41.355 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T23:58:41.400 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:41.399+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:41.500 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:41.499+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:41.503 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:41.503 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:58:41.503 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:58:41.503 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:58:41.503 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:41.512 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T23:58:41.520 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 6, 5] out_osds: [7, 3] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T23:58:41.520 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-14T23:58:41.602 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:41.600+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:41.621 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:41.621 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:58:41.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:58:41.621 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:58:41.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.621 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:107472 3.1e 3:7bf21588:::rbd_data.11187de8b0a5.000000000000005d:head [set-alloc-hint object_size 2097152 write_size 2097152,write 317788~1491] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-14T23:58:41.621 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:25.198493+0000", 2024-09-14T23:58:41.621 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 76.420329412000001, 2024-09-14T23:58:41.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.78175149300000002, 2024-09-14T23:58:41.621 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:41.622 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:41.622 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:41.622 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:41.622 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:41.622 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:41.622 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 107472 2024-09-14T23:58:41.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.622 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:41.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:41.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198493+0000", 2024-09-14T23:58:41.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:41.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198493+0000", 2024-09-14T23:58:41.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:41.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198495+0000", 2024-09-14T23:58:41.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.826e-06 2024-09-14T23:58:41.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:41.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198503+0000", 2024-09-14T23:58:41.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0150000000000003e-06 2024-09-14T23:58:41.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.624 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:41.624 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198521+0000", 2024-09-14T23:58:41.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8374000000000001e-05 2024-09-14T23:58:41.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.624 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:41.624 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198533+0000", 2024-09-14T23:58:41.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1593000000000001e-05 2024-09-14T23:58:41.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.624 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:41.624 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198603+0000", 2024-09-14T23:58:41.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0279999999999998e-05 2024-09-14T23:58:41.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.625 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:41.625 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198617+0000", 2024-09-14T23:58:41.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3517e-05 2024-09-14T23:58:41.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.625 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:41.625 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.977067+0000", 2024-09-14T23:58:41.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.77845065800000002 2024-09-14T23:58:41.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.625 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for missing object", 2024-09-14T23:58:41.625 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.977173+0000", 2024-09-14T23:58:41.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010543999999999999 2024-09-14T23:58:41.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:41.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.978737+0000", 2024-09-14T23:58:41.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001564078 2024-09-14T23:58:41.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:41.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.978845+0000", 2024-09-14T23:58:41.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000108643 2024-09-14T23:58:41.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1", 2024-09-14T23:58:41.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.978964+0000", 2024-09-14T23:58:41.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000119085 2024-09-14T23:58:41.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:41.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980109+0000", 2024-09-14T23:58:41.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0011442170000000001 2024-09-14T23:58:41.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:58:41.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980130+0000", 2024-09-14T23:58:41.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1766000000000001e-05 2024-09-14T23:58:41.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:41.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980171+0000", 2024-09-14T23:58:41.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0627999999999998e-05 2024-09-14T23:58:41.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:41.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980244+0000", 2024-09-14T23:58:41.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3373000000000004e-05 2024-09-14T23:58:41.628 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.628 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:41.628 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.628 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129159 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 5996544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:41.628 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.927783+0000", 2024-09-14T23:58:41.629 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 62.691039521, 2024-09-14T23:58:41.629 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054661122999999999, 2024-09-14T23:58:41.629 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:41.629 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:41.629 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:41.629 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:41.629 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:41.629 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:41.629 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129159 2024-09-14T23:58:41.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.629 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:41.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.629 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:41.629 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.927783+0000", 2024-09-14T23:58:41.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:41.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.927783+0000", 2024-09-14T23:58:41.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:41.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929684+0000", 2024-09-14T23:58:41.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0019011080000000001 2024-09-14T23:58:41.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:41.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929688+0000", 2024-09-14T23:58:41.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2610000000000004e-06 2024-09-14T23:58:41.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.631 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:41.631 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929697+0000", 2024-09-14T23:58:41.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0219999999999993e-06 2024-09-14T23:58:41.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.631 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:41.631 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929702+0000", 2024-09-14T23:58:41.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8840000000000002e-06 2024-09-14T23:58:41.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.631 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:41.631 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.982090+0000", 2024-09-14T23:58:41.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052388045000000001 2024-09-14T23:58:41.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.632 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:41.632 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.982208+0000", 2024-09-14T23:58:41.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118236 2024-09-14T23:58:41.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.632 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:41.632 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.982444+0000", 2024-09-14T23:58:41.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000235567 2024-09-14T23:58:41.632 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.632 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:41.632 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.633 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129166 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 8192000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:41.633 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.929794+0000", 2024-09-14T23:58:41.633 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 62.68902877, 2024-09-14T23:58:41.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054471973, 2024-09-14T23:58:41.633 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:41.633 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:41.633 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:41.633 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:41.633 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:41.633 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:41.633 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129166 2024-09-14T23:58:41.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.633 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:41.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:41.634 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929794+0000", 2024-09-14T23:58:41.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:41.634 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929794+0000", 2024-09-14T23:58:41.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:41.634 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929794+0000", 2024-09-14T23:58:41.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6899999999999997e-07 2024-09-14T23:58:41.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:41.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929797+0000", 2024-09-14T23:58:41.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.328e-06 2024-09-14T23:58:41.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:41.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929802+0000", 2024-09-14T23:58:41.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7309999999999998e-06 2024-09-14T23:58:41.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:41.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929807+0000", 2024-09-14T23:58:41.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4349999999999999e-06 2024-09-14T23:58:41.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:41.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984063+0000", 2024-09-14T23:58:41.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054256331999999997 2024-09-14T23:58:41.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:41.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984177+0000", 2024-09-14T23:58:41.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114225 2024-09-14T23:58:41.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.637 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:41.637 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984266+0000", 2024-09-14T23:58:41.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8352999999999999e-05 2024-09-14T23:58:41.637 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.637 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:41.637 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.637 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129167 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 6782976~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:41.637 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.929827+0000", 2024-09-14T23:58:41.637 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 62.688995361000003, 2024-09-14T23:58:41.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054637593999999998, 2024-09-14T23:58:41.638 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:41.638 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:41.638 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:41.638 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:41.638 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:41.638 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:41.638 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129167 2024-09-14T23:58:41.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.638 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:41.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.638 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:41.638 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929827+0000", 2024-09-14T23:58:41.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:41.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929827+0000", 2024-09-14T23:58:41.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:41.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929827+0000", 2024-09-14T23:58:41.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4000000000000002e-07 2024-09-14T23:58:41.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:41.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929830+0000", 2024-09-14T23:58:41.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.419e-06 2024-09-14T23:58:41.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:41.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929837+0000", 2024-09-14T23:58:41.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0450000000000003e-06 2024-09-14T23:58:41.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:41.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929842+0000", 2024-09-14T23:58:41.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.69e-06 2024-09-14T23:58:41.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:41.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984278+0000", 2024-09-14T23:58:41.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054436516999999997 2024-09-14T23:58:41.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:41.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984331+0000", 2024-09-14T23:58:41.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2590999999999998e-05 2024-09-14T23:58:41.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:41.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984465+0000", 2024-09-14T23:58:41.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013389200000000001 2024-09-14T23:58:41.641 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.641 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:41.641 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.642 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129168 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 8224768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:41.642 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.929849+0000", 2024-09-14T23:58:41.642 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 62.688973803000003, 2024-09-14T23:58:41.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054870368000000003, 2024-09-14T23:58:41.642 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:41.642 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:41.642 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:41.642 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:41.642 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:41.642 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:41.642 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129168 2024-09-14T23:58:41.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.642 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:41.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:41.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929849+0000", 2024-09-14T23:58:41.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:41.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929849+0000", 2024-09-14T23:58:41.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:41.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931681+0000", 2024-09-14T23:58:41.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018326989999999999 2024-09-14T23:58:41.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:41.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931686+0000", 2024-09-14T23:58:41.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7190000000000001e-06 2024-09-14T23:58:41.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:41.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931697+0000", 2024-09-14T23:58:41.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1287999999999999e-05 2024-09-14T23:58:41.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:41.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931704+0000", 2024-09-14T23:58:41.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3339999999999998e-06 2024-09-14T23:58:41.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:41.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984505+0000", 2024-09-14T23:58:41.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052801684000000002 2024-09-14T23:58:41.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:41.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984621+0000", 2024-09-14T23:58:41.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000116165 2024-09-14T23:58:41.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:41.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984719+0000", 2024-09-14T23:58:41.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7479000000000006e-05 2024-09-14T23:58:41.646 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.646 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:41.646 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.646 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129176 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 10452992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:41.646 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.931830+0000", 2024-09-14T23:58:41.646 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 62.686992042, 2024-09-14T23:58:41.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054513292999999997, 2024-09-14T23:58:41.646 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:41.646 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:41.646 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:41.646 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:41.646 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:41.647 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:41.647 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129176 2024-09-14T23:58:41.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.647 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:41.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:41.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931830+0000", 2024-09-14T23:58:41.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:41.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931830+0000", 2024-09-14T23:58:41.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:41.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.933678+0000", 2024-09-14T23:58:41.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018472440000000001 2024-09-14T23:58:41.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:41.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.933686+0000", 2024-09-14T23:58:41.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0239999999999997e-06 2024-09-14T23:58:41.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:41.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.933694+0000", 2024-09-14T23:58:41.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8400000000000001e-06 2024-09-14T23:58:41.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:41.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.933701+0000", 2024-09-14T23:58:41.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2870000000000001e-06 2024-09-14T23:58:41.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:41.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.986165+0000", 2024-09-14T23:58:41.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052463900000000001 2024-09-14T23:58:41.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:41.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.986259+0000", 2024-09-14T23:58:41.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4435e-05 2024-09-14T23:58:41.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:41.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.986344+0000", 2024-09-14T23:58:41.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4562999999999997e-05 2024-09-14T23:58:41.650 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.650 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:41.650 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.650 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129201 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 10289152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:41.650 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.935960+0000", 2024-09-14T23:58:41.650 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 62.682862579999998, 2024-09-14T23:58:41.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054338487999999997, 2024-09-14T23:58:41.651 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:41.651 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:41.651 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:41.651 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:41.651 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:41.651 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:41.651 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129201 2024-09-14T23:58:41.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.651 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:41.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:41.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935960+0000", 2024-09-14T23:58:41.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.652 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:41.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935960+0000", 2024-09-14T23:58:41.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.652 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:41.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935960+0000", 2024-09-14T23:58:41.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.72e-07 2024-09-14T23:58:41.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.652 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:41.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935963+0000", 2024-09-14T23:58:41.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5390000000000002e-06 2024-09-14T23:58:41.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:41.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935971+0000", 2024-09-14T23:58:41.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4589999999999998e-06 2024-09-14T23:58:41.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:41.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935977+0000", 2024-09-14T23:58:41.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4110000000000004e-06 2024-09-14T23:58:41.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:41.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990213+0000", 2024-09-14T23:58:41.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054236642000000002 2024-09-14T23:58:41.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:41.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990257+0000", 2024-09-14T23:58:41.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3696000000000003e-05 2024-09-14T23:58:41.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:41.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990298+0000", 2024-09-14T23:58:41.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1369000000000003e-05 2024-09-14T23:58:41.654 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.654 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:41.654 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.655 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129202 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 11370496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:41.655 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.935986+0000", 2024-09-14T23:58:41.655 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 62.682835961999999, 2024-09-14T23:58:41.655 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054543700000000001, 2024-09-14T23:58:41.655 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:41.655 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:41.655 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:41.655 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:41.655 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:41.655 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:41.655 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129202 2024-09-14T23:58:41.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.656 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:41.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:41.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935986+0000", 2024-09-14T23:58:41.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:41.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935986+0000", 2024-09-14T23:58:41.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:41.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937678+0000", 2024-09-14T23:58:41.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016917849999999999 2024-09-14T23:58:41.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:41.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937682+0000", 2024-09-14T23:58:41.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2400000000000001e-06 2024-09-14T23:58:41.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:41.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937695+0000", 2024-09-14T23:58:41.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2999e-05 2024-09-14T23:58:41.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:41.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937700+0000", 2024-09-14T23:58:41.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8420000000000004e-06 2024-09-14T23:58:41.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:41.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990336+0000", 2024-09-14T23:58:41.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052636060999999998 2024-09-14T23:58:41.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:41.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990435+0000", 2024-09-14T23:58:41.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9015e-05 2024-09-14T23:58:41.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:41.659 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990530+0000", 2024-09-14T23:58:41.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4758000000000003e-05 2024-09-14T23:58:41.659 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.659 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:41.659 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.659 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129214 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 11468800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:41.659 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.937939+0000", 2024-09-14T23:58:41.659 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 62.680883047999998, 2024-09-14T23:58:41.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054463733, 2024-09-14T23:58:41.659 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:41.659 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:41.659 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:41.659 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:41.660 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:41.660 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:41.660 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129214 2024-09-14T23:58:41.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.660 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:41.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:41.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937939+0000", 2024-09-14T23:58:41.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:41.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937939+0000", 2024-09-14T23:58:41.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:41.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939674+0000", 2024-09-14T23:58:41.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017342589999999999 2024-09-14T23:58:41.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:41.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939678+0000", 2024-09-14T23:58:41.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2590000000000002e-06 2024-09-14T23:58:41.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:41.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939699+0000", 2024-09-14T23:58:41.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1143000000000001e-05 2024-09-14T23:58:41.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:41.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939706+0000", 2024-09-14T23:58:41.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0269999999999998e-06 2024-09-14T23:58:41.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:41.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.992221+0000", 2024-09-14T23:58:41.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052514981000000002 2024-09-14T23:58:41.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:41.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.992321+0000", 2024-09-14T23:58:41.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9625999999999998e-05 2024-09-14T23:58:41.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.663 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:41.663 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.992403+0000", 2024-09-14T23:58:41.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2437999999999999e-05 2024-09-14T23:58:41.663 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.663 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:41.663 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.663 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129226 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 14090240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:41.663 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.939867+0000", 2024-09-14T23:58:41.663 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 62.678955119999998, 2024-09-14T23:58:41.664 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054523573999999998, 2024-09-14T23:58:41.664 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:41.664 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:41.664 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:41.664 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:41.664 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:41.664 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:41.664 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129226 2024-09-14T23:58:41.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.664 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:41.664 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.664 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:41.664 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939867+0000", 2024-09-14T23:58:41.664 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.665 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.665 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:41.665 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939867+0000", 2024-09-14T23:58:41.665 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.665 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.665 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:41.665 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941669+0000", 2024-09-14T23:58:41.665 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018021549999999999 2024-09-14T23:58:41.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.665 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.665 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:41.665 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941673+0000", 2024-09-14T23:58:41.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1269999999999999e-06 2024-09-14T23:58:41.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:41.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941679+0000", 2024-09-14T23:58:41.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1879999999999999e-06 2024-09-14T23:58:41.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:41.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941682+0000", 2024-09-14T23:58:41.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7110000000000001e-06 2024-09-14T23:58:41.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:41.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.994201+0000", 2024-09-14T23:58:41.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052518265000000001 2024-09-14T23:58:41.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:41.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.994302+0000", 2024-09-14T23:58:41.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101653 2024-09-14T23:58:41.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:41.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.994391+0000", 2024-09-14T23:58:41.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8474999999999996e-05 2024-09-14T23:58:41.667 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.667 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:41.668 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.668 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.668 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129237 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 12386304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:41.668 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.941825+0000", 2024-09-14T23:58:41.668 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 62.676997358000001, 2024-09-14T23:58:41.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054368076000000001, 2024-09-14T23:58:41.668 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:41.668 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:41.668 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:41.668 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:41.668 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:41.668 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:41.668 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129237 2024-09-14T23:58:41.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.669 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:41.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.669 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:41.669 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941825+0000", 2024-09-14T23:58:41.669 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.669 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:41.669 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941825+0000", 2024-09-14T23:58:41.669 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.669 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:41.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941825+0000", 2024-09-14T23:58:41.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.29e-07 2024-09-14T23:58:41.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:41.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941826+0000", 2024-09-14T23:58:41.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0999999999999997e-07 2024-09-14T23:58:41.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:41.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941834+0000", 2024-09-14T23:58:41.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5519999999999999e-06 2024-09-14T23:58:41.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:41.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941843+0000", 2024-09-14T23:58:41.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2960000000000004e-06 2024-09-14T23:58:41.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.671 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.671 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:41.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996108+0000", 2024-09-14T23:58:41.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054265200999999999 2024-09-14T23:58:41.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.671 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.671 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:41.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996152+0000", 2024-09-14T23:58:41.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4066999999999997e-05 2024-09-14T23:58:41.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.671 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.671 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:41.672 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996193+0000", 2024-09-14T23:58:41.672 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0821000000000001e-05 2024-09-14T23:58:41.672 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.672 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:41.672 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.672 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.672 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129238 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 13828096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:41.672 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.941851+0000", 2024-09-14T23:58:41.672 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 62.676971033999997, 2024-09-14T23:58:41.672 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.05458031, 2024-09-14T23:58:41.672 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:41.672 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:41.672 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:41.673 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:41.673 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:41.673 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:41.673 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129238 2024-09-14T23:58:41.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.673 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:41.673 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.673 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:41.673 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941851+0000", 2024-09-14T23:58:41.673 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.673 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.673 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:41.673 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941851+0000", 2024-09-14T23:58:41.674 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.674 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:41.674 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943670+0000", 2024-09-14T23:58:41.674 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018181930000000001 2024-09-14T23:58:41.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.674 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:41.674 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943673+0000", 2024-09-14T23:58:41.674 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3790000000000002e-06 2024-09-14T23:58:41.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.674 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:41.674 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943679+0000", 2024-09-14T23:58:41.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8719999999999999e-06 2024-09-14T23:58:41.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.675 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:41.675 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943683+0000", 2024-09-14T23:58:41.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8539999999999999e-06 2024-09-14T23:58:41.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.675 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:41.675 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996233+0000", 2024-09-14T23:58:41.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052550690999999997 2024-09-14T23:58:41.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.676 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:41.676 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996339+0000", 2024-09-14T23:58:41.676 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105989 2024-09-14T23:58:41.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.676 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:41.676 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996432+0000", 2024-09-14T23:58:41.676 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2331999999999996e-05 2024-09-14T23:58:41.676 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.676 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:41.676 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.676 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129249 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 12124160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:41.677 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.943826+0000", 2024-09-14T23:58:41.677 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 62.674996542999999, 2024-09-14T23:58:41.677 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054373266000000003, 2024-09-14T23:58:41.677 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:41.677 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:41.677 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:41.677 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:41.677 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:41.677 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:41.677 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129249 2024-09-14T23:58:41.677 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.677 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:41.677 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.677 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:41.678 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943826+0000", 2024-09-14T23:58:41.678 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.678 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.678 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.678 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:41.678 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943826+0000", 2024-09-14T23:58:41.678 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.678 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.678 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.678 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:41.678 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943826+0000", 2024-09-14T23:58:41.678 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.03e-07 2024-09-14T23:58:41.678 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.678 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.678 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:41.679 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943827+0000", 2024-09-14T23:58:41.679 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.08e-06 2024-09-14T23:58:41.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.679 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:41.679 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943834+0000", 2024-09-14T23:58:41.679 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4119999999999998e-06 2024-09-14T23:58:41.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.679 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:41.679 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943836+0000", 2024-09-14T23:58:41.679 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7369999999999998e-06 2024-09-14T23:58:41.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.680 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:41.680 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998115+0000", 2024-09-14T23:58:41.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054279040000000001 2024-09-14T23:58:41.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.680 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:41.680 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998159+0000", 2024-09-14T23:58:41.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3486999999999999e-05 2024-09-14T23:58:41.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.680 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:41.680 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998199+0000", 2024-09-14T23:58:41.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0306999999999999e-05 2024-09-14T23:58:41.680 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.681 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:41.681 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.681 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.681 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.681 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129250 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 13565952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:41.681 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.943840+0000", 2024-09-14T23:58:41.681 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 62.674982593000003, 2024-09-14T23:58:41.681 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054573875000000001, 2024-09-14T23:58:41.681 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:41.681 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:41.681 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:41.681 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:41.681 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:41.681 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:41.682 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129250 2024-09-14T23:58:41.682 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.682 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:41.682 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.682 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:41.682 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943840+0000", 2024-09-14T23:58:41.682 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.682 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.682 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.682 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:41.682 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943840+0000", 2024-09-14T23:58:41.682 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.682 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.682 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.683 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:41.683 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945669+0000", 2024-09-14T23:58:41.683 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018289720000000001 2024-09-14T23:58:41.683 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.683 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.683 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:41.683 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945671+0000", 2024-09-14T23:58:41.683 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6869999999999999e-06 2024-09-14T23:58:41.683 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.683 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.683 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:41.683 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945680+0000", 2024-09-14T23:58:41.683 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5259999999999993e-06 2024-09-14T23:58:41.683 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.683 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.684 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:41.684 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945685+0000", 2024-09-14T23:58:41.684 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8559999999999998e-06 2024-09-14T23:58:41.684 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.684 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.684 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:41.684 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998242+0000", 2024-09-14T23:58:41.684 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052557505999999997 2024-09-14T23:58:41.684 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.684 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.684 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:41.684 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998329+0000", 2024-09-14T23:58:41.684 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6410999999999996e-05 2024-09-14T23:58:41.684 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.685 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.685 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:41.685 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998414+0000", 2024-09-14T23:58:41.685 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4917e-05 2024-09-14T23:58:41.685 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.685 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:41.685 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.685 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.685 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.685 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129262 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 13303808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:41.685 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.945825+0000", 2024-09-14T23:58:41.685 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 62.672997651999999, 2024-09-14T23:58:41.685 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054515942999999997, 2024-09-14T23:58:41.685 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:41.686 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:41.686 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:41.686 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:41.686 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:41.686 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:41.686 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129262 2024-09-14T23:58:41.686 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.686 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:41.686 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.686 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:41.686 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945825+0000", 2024-09-14T23:58:41.686 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.686 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.686 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.686 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:41.687 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945825+0000", 2024-09-14T23:58:41.687 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.687 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.687 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.687 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:41.687 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.947670+0000", 2024-09-14T23:58:41.687 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018451229999999999 2024-09-14T23:58:41.687 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.687 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.687 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:41.687 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.947672+0000", 2024-09-14T23:58:41.687 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5289999999999998e-06 2024-09-14T23:58:41.687 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.687 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.688 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:41.688 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.947678+0000", 2024-09-14T23:58:41.688 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9880000000000001e-06 2024-09-14T23:58:41.688 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.688 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.688 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:41.688 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.947682+0000", 2024-09-14T23:58:41.688 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.63e-06 2024-09-14T23:58:41.688 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.688 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.688 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:41.689 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000183+0000", 2024-09-14T23:58:41.689 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052500770000000002 2024-09-14T23:58:41.689 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.689 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.689 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:41.689 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000266+0000", 2024-09-14T23:58:41.689 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3133000000000005e-05 2024-09-14T23:58:41.689 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.689 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.689 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:41.689 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000341+0000", 2024-09-14T23:58:41.689 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4770000000000004e-05 2024-09-14T23:58:41.689 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.689 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:41.690 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.690 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.690 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.690 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129299 3.10 3:0f1e06ee:::rbd_object_map.1118721be852:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:41.690 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.952330+0000", 2024-09-14T23:58:41.690 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 62.666492839, 2024-09-14T23:58:41.690 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054412818000000002, 2024-09-14T23:58:41.690 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:41.690 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:41.690 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:41.690 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:41.690 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:41.690 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:41.690 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129299 2024-09-14T23:58:41.691 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.691 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:41.691 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.691 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:41.691 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952330+0000", 2024-09-14T23:58:41.691 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.691 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.691 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.691 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:41.691 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952330+0000", 2024-09-14T23:58:41.691 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.691 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.691 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.691 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:41.692 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952330+0000", 2024-09-14T23:58:41.692 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4900000000000001e-07 2024-09-14T23:58:41.692 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.692 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.692 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:41.692 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952332+0000", 2024-09-14T23:58:41.692 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2879999999999999e-06 2024-09-14T23:58:41.692 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.692 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.692 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:41.692 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952340+0000", 2024-09-14T23:58:41.692 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9910000000000002e-06 2024-09-14T23:58:41.692 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.692 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.693 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:41.693 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952344+0000", 2024-09-14T23:58:41.693 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0280000000000002e-06 2024-09-14T23:58:41.693 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.693 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.693 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:41.693 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.004722+0000", 2024-09-14T23:58:41.693 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052377932000000002 2024-09-14T23:58:41.693 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.693 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.693 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:41.693 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.004778+0000", 2024-09-14T23:58:41.693 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5795999999999999e-05 2024-09-14T23:58:41.693 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.693 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.694 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:58:41.694 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.005016+0000", 2024-09-14T23:58:41.694 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000238006 2024-09-14T23:58:41.694 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.694 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.694 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:58:41.694 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.006366+0000", 2024-09-14T23:58:41.694 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001349786 2024-09-14T23:58:41.694 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.694 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.694 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:41.694 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.006685+0000", 2024-09-14T23:58:41.694 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00031970000000000002 2024-09-14T23:58:41.694 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.695 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.695 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:41.695 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.006718+0000", 2024-09-14T23:58:41.695 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2329000000000002e-05 2024-09-14T23:58:41.695 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.695 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.695 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:41.695 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.006742+0000", 2024-09-14T23:58:41.695 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4613e-05 2024-09-14T23:58:41.695 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.695 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:41.695 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.695 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.695 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.696 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186210 3.d 3.b2ff3acd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:41.696 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.129044+0000", 2024-09-14T23:58:41.696 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 34.489778475999998, 2024-09-14T23:58:41.696 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.84555443900000005, 2024-09-14T23:58:41.696 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:41.696 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:41.696 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:58:41.696 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:41.696 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:41.696 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:41.696 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186210 2024-09-14T23:58:41.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.696 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:41.696 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.696 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:41.697 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129044+0000", 2024-09-14T23:58:41.697 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.697 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.697 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.697 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:41.697 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129044+0000", 2024-09-14T23:58:41.697 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.697 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.697 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.697 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:41.697 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129046+0000", 2024-09-14T23:58:41.697 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.742e-06 2024-09-14T23:58:41.697 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.698 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.698 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:41.698 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129052+0000", 2024-09-14T23:58:41.698 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7390000000000002e-06 2024-09-14T23:58:41.698 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.698 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.698 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:41.698 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129067+0000", 2024-09-14T23:58:41.698 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.417e-05 2024-09-14T23:58:41.698 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.698 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.698 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:41.698 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129076+0000", 2024-09-14T23:58:41.699 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2289999999999992e-06 2024-09-14T23:58:41.699 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.699 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.699 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:41.699 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.130947+0000", 2024-09-14T23:58:41.699 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001871075 2024-09-14T23:58:41.699 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.699 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.699 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:41.699 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.130955+0000", 2024-09-14T23:58:41.699 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4500000000000004e-06 2024-09-14T23:58:41.699 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.699 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.699 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:41.699 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974587+0000", 2024-09-14T23:58:41.700 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.84363205500000005 2024-09-14T23:58:41.700 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.700 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.700 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:41.700 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974598+0000", 2024-09-14T23:58:41.700 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0978999999999999e-05 2024-09-14T23:58:41.700 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.700 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:41.700 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.700 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.700 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.700 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187415 3.d 3.b2ff3acd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:41.700 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.254681+0000", 2024-09-14T23:58:41.700 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 34.364140955000003, 2024-09-14T23:58:41.701 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.71995530699999999, 2024-09-14T23:58:41.701 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:41.701 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:41.701 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:58:41.701 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:41.701 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:41.701 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:41.701 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187415 2024-09-14T23:58:41.701 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.701 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:41.701 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.701 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:41.701 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254681+0000", 2024-09-14T23:58:41.701 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.702 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.702 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.702 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:41.702 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254681+0000", 2024-09-14T23:58:41.702 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.702 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.702 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.702 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:41.702 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254682+0000", 2024-09-14T23:58:41.702 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2e-07 2024-09-14T23:58:41.702 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.702 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.702 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:41.702 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254684+0000", 2024-09-14T23:58:41.703 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2730000000000001e-06 2024-09-14T23:58:41.703 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.703 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.703 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:41.703 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254689+0000", 2024-09-14T23:58:41.703 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4769999999999997e-06 2024-09-14T23:58:41.703 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.703 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.703 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:41.703 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254692+0000", 2024-09-14T23:58:41.703 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9890000000000001e-06 2024-09-14T23:58:41.703 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.703 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.703 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:41.704 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254721+0000", 2024-09-14T23:58:41.704 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9762999999999999e-05 2024-09-14T23:58:41.704 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.704 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.704 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:41.704 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254725+0000", 2024-09-14T23:58:41.704 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9650000000000002e-06 2024-09-14T23:58:41.704 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.704 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.704 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:41.704 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974628+0000", 2024-09-14T23:58:41.704 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.71990235499999999 2024-09-14T23:58:41.704 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.704 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.705 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:41.705 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974637+0000", 2024-09-14T23:58:41.705 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0650000000000005e-06 2024-09-14T23:58:41.705 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.705 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:41.705 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.705 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.705 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.705 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187457 3.d 3.b2ff3acd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:41.705 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.257980+0000", 2024-09-14T23:58:41.705 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 34.360842380999998, 2024-09-14T23:58:41.705 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.71671711199999999, 2024-09-14T23:58:41.705 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:41.705 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:41.705 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:58:41.706 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:41.706 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:41.706 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:41.706 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187457 2024-09-14T23:58:41.706 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.706 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:41.706 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.706 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:41.706 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257980+0000", 2024-09-14T23:58:41.706 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.706 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.706 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.706 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:41.706 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257980+0000", 2024-09-14T23:58:41.707 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.707 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.707 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.707 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:41.707 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257980+0000", 2024-09-14T23:58:41.707 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.17e-07 2024-09-14T23:58:41.707 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.707 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.707 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:41.707 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257981+0000", 2024-09-14T23:58:41.707 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.172e-06 2024-09-14T23:58:41.707 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.707 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.707 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:41.708 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257984+0000", 2024-09-14T23:58:41.708 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9550000000000001e-06 2024-09-14T23:58:41.708 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.708 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.708 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:41.708 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257987+0000", 2024-09-14T23:58:41.708 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5799999999999999e-06 2024-09-14T23:58:41.708 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.708 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.708 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:41.708 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.258030+0000", 2024-09-14T23:58:41.708 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3118999999999998e-05 2024-09-14T23:58:41.708 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.709 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.709 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:41.709 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.258038+0000", 2024-09-14T23:58:41.709 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6850000000000001e-06 2024-09-14T23:58:41.709 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.709 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.709 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:41.709 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974688+0000", 2024-09-14T23:58:41.709 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.71665016000000004 2024-09-14T23:58:41.709 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.709 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.709 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:41.709 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974697+0000", 2024-09-14T23:58:41.709 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2240000000000001e-06 2024-09-14T23:58:41.710 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.710 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:41.710 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.710 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.710 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.710 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:251656 3.c 3:32436721:::rbd_object_map.1118e09062ff:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-14T23:58:41.710 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:27.433419+0000", 2024-09-14T23:58:41.710 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 14.185403707000001, 2024-09-14T23:58:41.710 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.55083879499999999, 2024-09-14T23:58:41.710 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:41.710 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:41.710 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:41.710 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:41.710 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:41.711 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:41.711 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 251656 2024-09-14T23:58:41.711 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.711 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:41.711 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.711 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:41.711 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433419+0000", 2024-09-14T23:58:41.711 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.711 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.711 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.711 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:41.711 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433419+0000", 2024-09-14T23:58:41.711 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:41.711 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.712 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.712 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:41.712 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433419+0000", 2024-09-14T23:58:41.712 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1500000000000003e-07 2024-09-14T23:58:41.712 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.712 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.712 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:41.712 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433424+0000", 2024-09-14T23:58:41.712 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2409999999999997e-06 2024-09-14T23:58:41.712 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.712 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.712 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:41.712 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433435+0000", 2024-09-14T23:58:41.712 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1009000000000001e-05 2024-09-14T23:58:41.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.713 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.713 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:41.713 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433440+0000", 2024-09-14T23:58:41.713 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1950000000000002e-06 2024-09-14T23:58:41.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.713 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.713 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:41.713 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433487+0000", 2024-09-14T23:58:41.713 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7184000000000001e-05 2024-09-14T23:58:41.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.713 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.713 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:41.713 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433494+0000", 2024-09-14T23:58:41.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5490000000000003e-06 2024-09-14T23:58:41.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.714 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.714 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:41.714 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.981923+0000", 2024-09-14T23:58:41.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.54842898600000001 2024-09-14T23:58:41.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.714 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.715 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:41.716 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.982097+0000", 2024-09-14T23:58:41.716 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017428699999999999 2024-09-14T23:58:41.716 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.716 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.716 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5", 2024-09-14T23:58:41.716 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.982334+0000", 2024-09-14T23:58:41.716 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00023700899999999999 2024-09-14T23:58:41.716 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.716 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.716 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:41.716 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.983927+0000", 2024-09-14T23:58:41.717 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0015932710000000001 2024-09-14T23:58:41.717 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.717 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.717 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:58:41.717 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.984128+0000", 2024-09-14T23:58:41.717 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020104199999999999 2024-09-14T23:58:41.717 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.717 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.717 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:41.717 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.984212+0000", 2024-09-14T23:58:41.717 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3709999999999996e-05 2024-09-14T23:58:41.717 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:41.717 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:41.717 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:41.718 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.984257+0000", 2024-09-14T23:58:41.718 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5497000000000002e-05 2024-09-14T23:58:41.718 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.718 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:41.718 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.718 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:41.718 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:41.718 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:41.720 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T23:58:41.720 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:41.718+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:41.820 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:41.819+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:41.822 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T23:58:41.827 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:41.827 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:58:41.827 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:58:41.827 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:41.836 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T23:58:41.922 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:41.920+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:41.942 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:41.943 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:58:41.943 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:58:41.943 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:58:41.943 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:41.945 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:58:41.945 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:58:41.952 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T23:58:41.962 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:58:41.962 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:58:41.978 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:58:41.978 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:58:41.994 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:58:41.994 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:58:42.010 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:58:42.011 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:58:42.023 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:42.022+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:42.025 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.0 to 1 (10000) 2024-09-14T23:58:42.037 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:58:42.037 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:58:42.043 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-14T23:58:42.056 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:58:42.057 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:58:42.076 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:58:42.076 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:58:42.110 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:42.110 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:42.119 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T23:58:42.123 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:42.122+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:42.224 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:58:42.223+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:58:42.239 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:42.240 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:42.249 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T23:58:42.325 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:42.324+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:42.356 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:42.356 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:42.365 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T23:58:42.449 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:42.448+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:42.513 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:42.513 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:42.522 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T23:58:42.562 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:42.561+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:42.627 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:42.625+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:42.669 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:42.669 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:42.677 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T23:58:42.728 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:42.727+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:42.824 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:42.825 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:42.829 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:42.827+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:42.833 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T23:58:42.931 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:42.929+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:42.980 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.1 to 1 (10000) 2024-09-14T23:58:42.987 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:42.987 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:42.998 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-14T23:58:42.999 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T23:58:43.037 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T23:58:43.043 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:43.041+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:43.132 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:43.131+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:43.161 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:43.162 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:43.173 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T23:58:43.257 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:43.256+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:43.322 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:43.322 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:43.331 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T23:58:43.334 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:43.333+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:43.435 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:43.434+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:43.477 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:43.477 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:43.486 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T23:58:43.536 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:58:43.535+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:58:43.593 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:43.594 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:43.602 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T23:58:43.645 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:43.644+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:43.709 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:43.709 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:43.718 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T23:58:43.758 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:43.757+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:43.838 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:43.836+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:43.865 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:43.865 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:43.874 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T23:58:43.939 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:43.937+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:43.995 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:43.995 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:44.005 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T23:58:44.025 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is set 2024-09-14T23:58:44.026 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.2 to 1 (10000) 2024-09-14T23:58:44.040 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:44.038+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:44.045 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-14T23:58:44.090 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T23:58:44.161 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:44.161 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:44.170 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T23:58:44.172 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:44.171+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:44.242 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:44.240+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:44.255 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:58:44.255 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:58:44.272 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:58:44.273 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:58:44.277 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:44.277 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:44.286 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T23:58:44.289 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:58:44.289 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:58:44.305 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:58:44.305 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:58:44.322 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:58:44.322 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:58:44.340 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:58:44.341 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:58:44.343 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:44.341+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:44.357 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:58:44.357 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:58:44.375 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:58:44.375 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:58:44.397 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:44.397 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:58:44.397 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:58:44.397 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:44.406 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T23:58:44.443 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:44.442+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:44.513 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:44.514 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:58:44.514 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:58:44.514 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:58:44.514 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:44.522 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T23:58:44.544 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:44.543+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:44.630 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:44.631 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:58:44.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:58:44.631 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:58:44.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.631 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4284.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-14T23:58:44.631 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:58:44.631 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 142.103825048, 2024-09-14T23:58:44.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.004942086, 2024-09-14T23:58:44.631 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:44.631 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:44.632 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:44.632 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:44.632 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4284", 2024-09-14T23:58:44.632 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/4278347926", 2024-09-14T23:58:44.632 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1 2024-09-14T23:58:44.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.632 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:44.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.632 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:44.632 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:58:44.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:44.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.633 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:44.633 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:58:44.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:44.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.633 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:44.633 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524331+0000", 2024-09-14T23:58:44.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.383e-05 2024-09-14T23:58:44.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:44.634 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524339+0000", 2024-09-14T23:58:44.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8290000000000008e-06 2024-09-14T23:58:44.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:44.634 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524359+0000", 2024-09-14T23:58:44.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9998000000000001e-05 2024-09-14T23:58:44.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:44.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524373+0000", 2024-09-14T23:58:44.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4304000000000001e-05 2024-09-14T23:58:44.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:44.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524488+0000", 2024-09-14T23:58:44.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114663 2024-09-14T23:58:44.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:44.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524501+0000", 2024-09-14T23:58:44.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3349999999999999e-05 2024-09-14T23:58:44.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:44.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.526868+0000", 2024-09-14T23:58:44.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023670090000000002 2024-09-14T23:58:44.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.637 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:44.637 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.527041+0000", 2024-09-14T23:58:44.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000172597 2024-09-14T23:58:44.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.637 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-14T23:58:44.637 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.527384+0000", 2024-09-14T23:58:44.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00034309500000000002 2024-09-14T23:58:44.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.638 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:44.638 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529103+0000", 2024-09-14T23:58:44.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017185220000000001 2024-09-14T23:58:44.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.638 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:58:44.638 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529188+0000", 2024-09-14T23:58:44.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5591000000000001e-05 2024-09-14T23:58:44.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:44.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529227+0000", 2024-09-14T23:58:44.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8563000000000003e-05 2024-09-14T23:58:44.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:44.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529259+0000", 2024-09-14T23:58:44.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2734999999999997e-05 2024-09-14T23:58:44.639 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:44.639 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:44.639 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:44.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.640 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4284.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-14T23:58:44.640 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:23.530683+0000", 2024-09-14T23:58:44.640 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 141.097459383, 2024-09-14T23:58:44.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0036133110000000001, 2024-09-14T23:58:44.640 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:44.640 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:44.640 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:44.640 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:44.641 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4284", 2024-09-14T23:58:44.641 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/4278347926", 2024-09-14T23:58:44.641 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5 2024-09-14T23:58:44.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.641 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:44.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:44.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530683+0000", 2024-09-14T23:58:44.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:44.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:44.642 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530683+0000", 2024-09-14T23:58:44.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:44.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:44.642 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530685+0000", 2024-09-14T23:58:44.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.075e-06 2024-09-14T23:58:44.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:44.642 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530695+0000", 2024-09-14T23:58:44.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0261999999999999e-05 2024-09-14T23:58:44.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:44.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530715+0000", 2024-09-14T23:58:44.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9930000000000001e-05 2024-09-14T23:58:44.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:44.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530724+0000", 2024-09-14T23:58:44.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.901e-06 2024-09-14T23:58:44.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:44.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530800+0000", 2024-09-14T23:58:44.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5873999999999999e-05 2024-09-14T23:58:44.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-14T23:58:44.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530807+0000", 2024-09-14T23:58:44.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2389999999999996e-06 2024-09-14T23:58:44.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:44.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.532494+0000", 2024-09-14T23:58:44.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016871830000000001 2024-09-14T23:58:44.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:44.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.532599+0000", 2024-09-14T23:58:44.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010472 2024-09-14T23:58:44.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1", 2024-09-14T23:58:44.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.532715+0000", 2024-09-14T23:58:44.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011568199999999999 2024-09-14T23:58:44.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:58:44.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.533460+0000", 2024-09-14T23:58:44.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00074494999999999997 2024-09-14T23:58:44.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:44.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534230+0000", 2024-09-14T23:58:44.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00077036599999999998 2024-09-14T23:58:44.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:44.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534266+0000", 2024-09-14T23:58:44.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5725000000000001e-05 2024-09-14T23:58:44.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:44.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534296+0000", 2024-09-14T23:58:44.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0403999999999999e-05 2024-09-14T23:58:44.647 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:44.647 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:44.647 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:44.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.647 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56383 3.c e56/49 3:36b698b0:::rbd_data.1118c4b3aebe.0000000000000000:head v 56'194, mlcod=56'192)", 2024-09-14T23:58:44.647 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.270483+0000", 2024-09-14T23:58:44.647 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 98.357659583, 2024-09-14T23:58:44.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0079685880000000004, 2024-09-14T23:58:44.648 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:44.648 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:44.648 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:44.648 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:44.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:44.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.270483+0000", 2024-09-14T23:58:44.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:44.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:44.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.270483+0000", 2024-09-14T23:58:44.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:44.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:44.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.270484+0000", 2024-09-14T23:58:44.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7400000000000002e-07 2024-09-14T23:58:44.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:44.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273680+0000", 2024-09-14T23:58:44.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031966680000000002 2024-09-14T23:58:44.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:44.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273697+0000", 2024-09-14T23:58:44.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6595e-05 2024-09-14T23:58:44.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:44.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273705+0000", 2024-09-14T23:58:44.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7530000000000001e-06 2024-09-14T23:58:44.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:44.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273763+0000", 2024-09-14T23:58:44.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8238e-05 2024-09-14T23:58:44.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:44.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273781+0000", 2024-09-14T23:58:44.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8281e-05 2024-09-14T23:58:44.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:44.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.278408+0000", 2024-09-14T23:58:44.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00462716 2024-09-14T23:58:44.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:44.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.278451+0000", 2024-09-14T23:58:44.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3019000000000003e-05 2024-09-14T23:58:44.651 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:44.652 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:44.652 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:44.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.652 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56384 3.c e56/49 3:36b698b0:::rbd_data.1118c4b3aebe.0000000000000000:head v 56'196, mlcod=56'192)", 2024-09-14T23:58:44.652 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:58:44.652 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 98.353626911000006, 2024-09-14T23:58:44.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0046707529999999997, 2024-09-14T23:58:44.652 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:44.652 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:44.652 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:44.652 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:44.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:44.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:58:44.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:44.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:44.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:58:44.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:44.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:44.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:58:44.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1699999999999997e-07 2024-09-14T23:58:44.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:44.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274523+0000", 2024-09-14T23:58:44.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9129999999999997e-06 2024-09-14T23:58:44.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:44.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274535+0000", 2024-09-14T23:58:44.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1788e-05 2024-09-14T23:58:44.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:44.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274542+0000", 2024-09-14T23:58:44.655 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0539999999999997e-06 2024-09-14T23:58:44.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.655 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:44.655 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275889+0000", 2024-09-14T23:58:44.655 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00134669 2024-09-14T23:58:44.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.655 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:44.655 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275917+0000", 2024-09-14T23:58:44.655 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7664000000000001e-05 2024-09-14T23:58:44.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:44.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279134+0000", 2024-09-14T23:58:44.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0032178459999999999 2024-09-14T23:58:44.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:44.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279186+0000", 2024-09-14T23:58:44.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1881e-05 2024-09-14T23:58:44.656 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:44.656 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:44.657 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:44.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.657 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56381 3.1d e56/49 3:bcf9a8cf:::rbd_data.1118c4b3aebe.0000000000000006:head v 56'210, mlcod=56'208)", 2024-09-14T23:58:44.657 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.275462+0000", 2024-09-14T23:58:44.657 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 98.352680581000001, 2024-09-14T23:58:44.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0071310710000000001, 2024-09-14T23:58:44.657 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:44.657 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:44.657 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:44.657 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:44.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:44.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275462+0000", 2024-09-14T23:58:44.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:44.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:44.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275462+0000", 2024-09-14T23:58:44.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:44.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:44.659 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275463+0000", 2024-09-14T23:58:44.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2699999999999999e-07 2024-09-14T23:58:44.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.659 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:44.659 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276779+0000", 2024-09-14T23:58:44.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0013163649999999999 2024-09-14T23:58:44.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.659 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:44.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276795+0000", 2024-09-14T23:58:44.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.59e-05 2024-09-14T23:58:44.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:44.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276806+0000", 2024-09-14T23:58:44.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1106e-05 2024-09-14T23:58:44.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:44.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276941+0000", 2024-09-14T23:58:44.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013477500000000001 2024-09-14T23:58:44.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:44.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276963+0000", 2024-09-14T23:58:44.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2419999999999999e-05 2024-09-14T23:58:44.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:44.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282546+0000", 2024-09-14T23:58:44.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0055827100000000003 2024-09-14T23:58:44.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:44.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282593+0000", 2024-09-14T23:58:44.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7067999999999997e-05 2024-09-14T23:58:44.662 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:44.662 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:44.662 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:44.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.662 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56387 3.c e56/49 3:30ee4ba0:::rbd_data.1118c4b3aebe.0000000000000004:head v 56'197, mlcod=56'192)", 2024-09-14T23:58:44.662 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.276498+0000", 2024-09-14T23:58:44.663 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 98.351644485999998, 2024-09-14T23:58:44.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0092192639999999996, 2024-09-14T23:58:44.663 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:44.663 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:44.663 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:44.663 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:44.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.663 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:44.663 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276498+0000", 2024-09-14T23:58:44.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:44.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.664 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.664 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:44.664 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276498+0000", 2024-09-14T23:58:44.664 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:44.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.664 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.664 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:44.664 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276499+0000", 2024-09-14T23:58:44.664 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1100000000000004e-07 2024-09-14T23:58:44.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.665 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.665 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:44.665 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279646+0000", 2024-09-14T23:58:44.665 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031473220000000001 2024-09-14T23:58:44.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.665 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.665 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:44.665 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279672+0000", 2024-09-14T23:58:44.665 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6129999999999999e-05 2024-09-14T23:58:44.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:44.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279682+0000", 2024-09-14T23:58:44.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7170000000000003e-06 2024-09-14T23:58:44.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:44.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279742+0000", 2024-09-14T23:58:44.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9926999999999997e-05 2024-09-14T23:58:44.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:44.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279761+0000", 2024-09-14T23:58:44.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9402000000000002e-05 2024-09-14T23:58:44.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:44.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.285680+0000", 2024-09-14T23:58:44.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.005918881 2024-09-14T23:58:44.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:44.668 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.285717+0000", 2024-09-14T23:58:44.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6974000000000002e-05 2024-09-14T23:58:44.668 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:44.668 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:44.668 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:44.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.668 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.668 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56388 3.c e56/49 3:30ee4ba0:::rbd_data.1118c4b3aebe.0000000000000004:head v 56'199, mlcod=56'192)", 2024-09-14T23:58:44.668 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:58:44.668 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 98.348152897999995, 2024-09-14T23:58:44.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0060596740000000001, 2024-09-14T23:58:44.669 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:44.669 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:44.669 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:44.669 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:44.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.669 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:44.669 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:58:44.669 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:44.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.669 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:44.669 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:58:44.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:44.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:44.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:58:44.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2200000000000003e-07 2024-09-14T23:58:44.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:44.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279995+0000", 2024-09-14T23:58:44.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5569999999999997e-06 2024-09-14T23:58:44.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.671 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.671 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:44.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280004+0000", 2024-09-14T23:58:44.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0080000000000008e-06 2024-09-14T23:58:44.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.671 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.672 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:44.672 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280010+0000", 2024-09-14T23:58:44.672 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5749999999999997e-06 2024-09-14T23:58:44.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.672 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.672 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:44.672 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282968+0000", 2024-09-14T23:58:44.672 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0029579390000000001 2024-09-14T23:58:44.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.672 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.672 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:44.673 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282998+0000", 2024-09-14T23:58:44.673 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9947999999999999e-05 2024-09-14T23:58:44.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.673 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.673 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:44.673 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.286012+0000", 2024-09-14T23:58:44.673 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030132900000000001 2024-09-14T23:58:44.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.673 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.673 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:44.674 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.286049+0000", 2024-09-14T23:58:44.674 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7534999999999999e-05 2024-09-14T23:58:44.674 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:44.674 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:44.674 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:44.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.674 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56382 3.1d e56/49 3:bcf9a8cf:::rbd_data.1118c4b3aebe.0000000000000006:head v 56'212, mlcod=56'208)", 2024-09-14T23:58:44.674 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.280285+0000", 2024-09-14T23:58:44.674 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 98.347857215000005, 2024-09-14T23:58:44.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035393489999999998, 2024-09-14T23:58:44.675 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:44.675 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:44.675 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:44.675 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:44.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.675 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:44.675 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280285+0000", 2024-09-14T23:58:44.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:44.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.675 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:44.675 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280285+0000", 2024-09-14T23:58:44.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:44.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.676 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:44.676 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280286+0000", 2024-09-14T23:58:44.676 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5000000000000001e-07 2024-09-14T23:58:44.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.676 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:44.676 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280299+0000", 2024-09-14T23:58:44.676 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3312000000000001e-05 2024-09-14T23:58:44.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.676 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:44.676 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280311+0000", 2024-09-14T23:58:44.677 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1677e-05 2024-09-14T23:58:44.677 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.677 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.677 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:44.677 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280318+0000", 2024-09-14T23:58:44.677 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.742e-06 2024-09-14T23:58:44.677 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.677 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.677 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:44.677 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280374+0000", 2024-09-14T23:58:44.677 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.592e-05 2024-09-14T23:58:44.677 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.677 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.677 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:44.678 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280392+0000", 2024-09-14T23:58:44.678 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8490000000000001e-05 2024-09-14T23:58:44.678 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.678 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.678 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:44.678 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.283693+0000", 2024-09-14T23:58:44.678 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033012050000000002 2024-09-14T23:58:44.678 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.678 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.678 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:44.678 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.283825+0000", 2024-09-14T23:58:44.678 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013105300000000001 2024-09-14T23:58:44.678 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:44.678 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:44.678 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:44.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.679 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "pg_update_log_missing(3.c epoch 56/49 rep_tid 1493 entries 56'200 (0'0) error 3:370203a3:::rbd_data.1118c4b3aebe.000000000000000d:head by client.4376.0:56396 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 56'192)", 2024-09-14T23:58:44.679 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:58:44.679 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 98.347632758000003, 2024-09-14T23:58:44.679 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057370099999999999, 2024-09-14T23:58:44.679 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:44.679 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:44.679 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:58:44.679 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:44.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.679 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:44.679 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:58:44.679 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:44.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.680 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:44.680 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:58:44.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:44.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.680 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:44.680 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:58:44.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0699999999999997e-07 2024-09-14T23:58:44.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.680 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:44.680 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280519+0000", 2024-09-14T23:58:44.681 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6260000000000001e-06 2024-09-14T23:58:44.681 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.681 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.681 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:44.681 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280541+0000", 2024-09-14T23:58:44.681 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1976000000000001e-05 2024-09-14T23:58:44.681 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.681 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.681 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:44.681 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280556+0000", 2024-09-14T23:58:44.681 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5075000000000001e-05 2024-09-14T23:58:44.681 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.681 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.682 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:44.682 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.285494+0000", 2024-09-14T23:58:44.682 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0049378690000000001 2024-09-14T23:58:44.682 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.682 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.682 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:44.682 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.286247+0000", 2024-09-14T23:58:44.682 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00075285700000000001 2024-09-14T23:58:44.682 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:44.682 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:44.682 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:44.682 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.682 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.682 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56400 3.1c e56/49 3:3f5b81ba:::rbd_data.1118c4b3aebe.000000000000000a:head v 56'236, mlcod=56'234)", 2024-09-14T23:58:44.683 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.295064+0000", 2024-09-14T23:58:44.683 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 98.333077966000005, 2024-09-14T23:58:44.683 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0054331589999999999, 2024-09-14T23:58:44.683 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:44.683 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:44.683 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:44.683 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:44.683 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.683 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:44.683 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.295064+0000", 2024-09-14T23:58:44.683 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:44.683 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.683 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.684 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:44.684 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.295064+0000", 2024-09-14T23:58:44.684 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:44.684 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.684 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.684 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:44.684 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.295066+0000", 2024-09-14T23:58:44.684 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.502e-06 2024-09-14T23:58:44.684 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.684 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.684 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:44.684 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296403+0000", 2024-09-14T23:58:44.685 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001336554 2024-09-14T23:58:44.685 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.685 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.685 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:44.685 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296428+0000", 2024-09-14T23:58:44.685 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5179000000000001e-05 2024-09-14T23:58:44.685 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.685 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.685 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:44.685 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296440+0000", 2024-09-14T23:58:44.685 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2387e-05 2024-09-14T23:58:44.685 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.685 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.686 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:44.686 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296554+0000", 2024-09-14T23:58:44.686 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114269 2024-09-14T23:58:44.686 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.686 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.686 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:44.686 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296581+0000", 2024-09-14T23:58:44.686 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6673999999999999e-05 2024-09-14T23:58:44.686 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.686 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.686 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:44.686 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.300418+0000", 2024-09-14T23:58:44.686 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0038371249999999998 2024-09-14T23:58:44.686 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.686 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.687 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:44.687 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.300498+0000", 2024-09-14T23:58:44.687 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9468999999999994e-05 2024-09-14T23:58:44.687 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:44.687 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:44.687 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:44.687 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.687 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.687 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:61922 3.5 e57/49 3:a11ebf8e:::rbd_data.11188d356229.000000000000000c:head v 57'280, mlcod=57'277)", 2024-09-14T23:58:44.687 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:07.563144+0000", 2024-09-14T23:58:44.687 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 97.064998313999993, 2024-09-14T23:58:44.687 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003977848, 2024-09-14T23:58:44.687 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:44.688 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:44.688 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:44.688 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:44.688 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.688 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:44.688 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563144+0000", 2024-09-14T23:58:44.688 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:44.688 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.688 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.688 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:44.688 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563144+0000", 2024-09-14T23:58:44.688 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:44.689 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.689 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.689 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:44.689 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563145+0000", 2024-09-14T23:58:44.689 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5400000000000001e-07 2024-09-14T23:58:44.689 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.689 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.689 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:44.689 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565056+0000", 2024-09-14T23:58:44.689 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0019108160000000001 2024-09-14T23:58:44.689 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.689 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.690 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:44.690 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565088+0000", 2024-09-14T23:58:44.690 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2586999999999998e-05 2024-09-14T23:58:44.690 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.690 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.690 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:44.690 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565100+0000", 2024-09-14T23:58:44.690 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1917e-05 2024-09-14T23:58:44.690 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.690 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.690 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:44.690 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565166+0000", 2024-09-14T23:58:44.691 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6245e-05 2024-09-14T23:58:44.691 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.691 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.691 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:44.691 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565190+0000", 2024-09-14T23:58:44.691 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3825999999999999e-05 2024-09-14T23:58:44.691 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.691 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.691 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:44.691 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.567080+0000", 2024-09-14T23:58:44.691 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001889541 2024-09-14T23:58:44.692 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.692 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.692 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:44.692 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.567122+0000", 2024-09-14T23:58:44.692 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2262000000000003e-05 2024-09-14T23:58:44.692 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:44.692 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:44.692 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:44.692 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.692 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.692 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:66564 3.c e60/49 3:33faa7ca:::rbd_data.111873792d9b.0000000000000003:head v 60'236, mlcod=57'233)", 2024-09-14T23:58:44.693 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:11.271596+0000", 2024-09-14T23:58:44.693 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 93.356546735999999, 2024-09-14T23:58:44.693 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0060396099999999999, 2024-09-14T23:58:44.693 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:44.693 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:44.693 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:44.693 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:44.693 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.693 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:44.693 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.271596+0000", 2024-09-14T23:58:44.693 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:44.694 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.694 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.694 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:44.694 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.271596+0000", 2024-09-14T23:58:44.694 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:44.694 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.694 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.694 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:44.694 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.271597+0000", 2024-09-14T23:58:44.694 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2559999999999999e-06 2024-09-14T23:58:44.694 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.695 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.695 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:44.695 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.274854+0000", 2024-09-14T23:58:44.695 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0032574470000000001 2024-09-14T23:58:44.695 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.695 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.695 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:44.695 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.274874+0000", 2024-09-14T23:58:44.695 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9449999999999998e-05 2024-09-14T23:58:44.695 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.695 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.695 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:44.695 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.274883+0000", 2024-09-14T23:58:44.696 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5980000000000003e-06 2024-09-14T23:58:44.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.696 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.696 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:44.696 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.274977+0000", 2024-09-14T23:58:44.696 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3270999999999995e-05 2024-09-14T23:58:44.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.696 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.696 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:44.697 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.275000+0000", 2024-09-14T23:58:44.697 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3088e-05 2024-09-14T23:58:44.697 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.697 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.697 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:44.697 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.277590+0000", 2024-09-14T23:58:44.697 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0025898660000000001 2024-09-14T23:58:44.697 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.697 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.697 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:44.697 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.277635+0000", 2024-09-14T23:58:44.698 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5633999999999997e-05 2024-09-14T23:58:44.698 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:44.698 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:44.698 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:44.698 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.698 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.698 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:133384 3.1f e92/49 3:fce5e529:::rbd_data.11186132a8b6.0000000000000002:head v 92'807, mlcod=89'804)", 2024-09-14T23:58:44.698 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:45.293642+0000", 2024-09-14T23:58:44.698 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 59.334500853000002, 2024-09-14T23:58:44.698 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003760186, 2024-09-14T23:58:44.699 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:44.699 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:44.699 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:44.699 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:44.699 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.699 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:44.699 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.293642+0000", 2024-09-14T23:58:44.700 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:44.700 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.700 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.700 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:44.700 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.293642+0000", 2024-09-14T23:58:44.700 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:44.700 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.700 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.700 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:44.700 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.293643+0000", 2024-09-14T23:58:44.700 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2500000000000001e-06 2024-09-14T23:58:44.701 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.701 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.701 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:44.701 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.295402+0000", 2024-09-14T23:58:44.701 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017595410000000001 2024-09-14T23:58:44.701 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.701 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.701 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:44.701 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.295422+0000", 2024-09-14T23:58:44.701 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9763999999999999e-05 2024-09-14T23:58:44.702 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.702 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.702 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:44.702 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.295436+0000", 2024-09-14T23:58:44.702 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.364e-05 2024-09-14T23:58:44.702 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.702 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.702 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:44.702 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.295499+0000", 2024-09-14T23:58:44.702 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2770999999999999e-05 2024-09-14T23:58:44.702 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.702 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.703 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:44.703 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.295516+0000", 2024-09-14T23:58:44.703 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7240000000000001e-05 2024-09-14T23:58:44.703 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.703 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.703 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:44.703 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.297324+0000", 2024-09-14T23:58:44.703 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001807857 2024-09-14T23:58:44.703 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.703 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.704 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:44.704 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.297402+0000", 2024-09-14T23:58:44.704 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8122999999999997e-05 2024-09-14T23:58:44.704 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:44.704 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:44.704 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:44.704 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.704 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.704 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:156597 3.5 e103/49 3:a23c5924:::rbd_data.1118b2f495e6.0000000000000011:head v 103'518, mlcod=98'516)", 2024-09-14T23:58:44.704 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:58.656197+0000", 2024-09-14T23:58:44.705 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 45.971945796, 2024-09-14T23:58:44.705 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0040423029999999997, 2024-09-14T23:58:44.705 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:44.705 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:44.705 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:44.705 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:44.705 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.705 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:44.705 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656197+0000", 2024-09-14T23:58:44.705 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:44.705 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.705 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.706 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:44.706 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656197+0000", 2024-09-14T23:58:44.706 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:44.706 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.706 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.706 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:44.706 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656197+0000", 2024-09-14T23:58:44.706 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1900000000000002e-07 2024-09-14T23:58:44.706 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.706 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.706 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:44.706 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656676+0000", 2024-09-14T23:58:44.706 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00047921699999999999 2024-09-14T23:58:44.707 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.707 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.707 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:44.707 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656690+0000", 2024-09-14T23:58:44.707 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3916e-05 2024-09-14T23:58:44.707 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.707 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.707 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:44.707 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656698+0000", 2024-09-14T23:58:44.707 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6529999999999994e-06 2024-09-14T23:58:44.707 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.707 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.707 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:44.707 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656757+0000", 2024-09-14T23:58:44.707 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9237999999999998e-05 2024-09-14T23:58:44.708 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.708 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.708 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:44.708 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656780+0000", 2024-09-14T23:58:44.708 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2968000000000001e-05 2024-09-14T23:58:44.708 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.708 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.708 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:44.708 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.660188+0000", 2024-09-14T23:58:44.708 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0034078630000000001 2024-09-14T23:58:44.708 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.708 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.708 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:44.708 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.660239+0000", 2024-09-14T23:58:44.709 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0828999999999997e-05 2024-09-14T23:58:44.709 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:44.709 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:44.709 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:44.709 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.709 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.709 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:157947 3.1c e103/49 3:383374fe:::rbd_data.1118b2f495e6.000000000000000c:head v 103'771, mlcod=103'769)", 2024-09-14T23:58:44.709 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:58.765271+0000", 2024-09-14T23:58:44.709 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 45.862870915000002, 2024-09-14T23:58:44.709 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0040869829999999998, 2024-09-14T23:58:44.709 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:44.709 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:44.709 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:44.709 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:44.709 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.710 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:44.710 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765271+0000", 2024-09-14T23:58:44.710 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:44.710 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.710 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.710 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:44.710 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765271+0000", 2024-09-14T23:58:44.710 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:44.710 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.710 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.710 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:44.710 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765273+0000", 2024-09-14T23:58:44.710 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1149999999999999e-06 2024-09-14T23:58:44.710 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.710 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.710 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:44.711 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765854+0000", 2024-09-14T23:58:44.711 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00058119599999999997 2024-09-14T23:58:44.711 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.711 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.711 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:44.711 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765869+0000", 2024-09-14T23:58:44.711 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5641000000000002e-05 2024-09-14T23:58:44.711 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.711 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.711 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:44.711 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765878+0000", 2024-09-14T23:58:44.711 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4519999999999997e-06 2024-09-14T23:58:44.711 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.711 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.711 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:44.711 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765942+0000", 2024-09-14T23:58:44.712 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3727999999999997e-05 2024-09-14T23:58:44.712 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.712 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.712 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:44.712 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765968+0000", 2024-09-14T23:58:44.712 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6171999999999999e-05 2024-09-14T23:58:44.712 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.712 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.712 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:44.712 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.769319+0000", 2024-09-14T23:58:44.712 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033511629999999999 2024-09-14T23:58:44.712 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.712 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.712 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:44.713 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.769358+0000", 2024-09-14T23:58:44.713 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9515999999999999e-05 2024-09-14T23:58:44.713 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:44.713 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:44.713 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:44.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.713 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.713 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:160196 3.c e103/49 3:34623e68:::rbd_data.1118b2f495e6.0000000000000005:head v 103'551, mlcod=102'549)", 2024-09-14T23:58:44.713 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:59.131433+0000", 2024-09-14T23:58:44.713 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 45.496709606000003, 2024-09-14T23:58:44.713 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0036932530000000001, 2024-09-14T23:58:44.713 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:44.713 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:44.713 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:44.713 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:44.714 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.714 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:44.714 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.131433+0000", 2024-09-14T23:58:44.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:44.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.714 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.714 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:44.714 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.131433+0000", 2024-09-14T23:58:44.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:44.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.714 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.714 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:44.714 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.131433+0000", 2024-09-14T23:58:44.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8500000000000001e-07 2024-09-14T23:58:44.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.715 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.715 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:44.715 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.132144+0000", 2024-09-14T23:58:44.715 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00071022800000000003 2024-09-14T23:58:44.715 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.715 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.715 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:44.715 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.132154+0000", 2024-09-14T23:58:44.715 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0301e-05 2024-09-14T23:58:44.715 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.715 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.715 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:44.715 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.132159+0000", 2024-09-14T23:58:44.715 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6280000000000004e-06 2024-09-14T23:58:44.715 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.716 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.716 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:44.716 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.132208+0000", 2024-09-14T23:58:44.716 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9595000000000002e-05 2024-09-14T23:58:44.716 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.716 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.716 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:44.716 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.132231+0000", 2024-09-14T23:58:44.716 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3010000000000002e-05 2024-09-14T23:58:44.716 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.716 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.716 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:44.716 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.135087+0000", 2024-09-14T23:58:44.716 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0028553239999999998 2024-09-14T23:58:44.717 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.717 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.717 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:44.717 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.135126+0000", 2024-09-14T23:58:44.717 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9481999999999999e-05 2024-09-14T23:58:44.717 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:44.717 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:44.717 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:44.717 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.717 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.717 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188871 3.d 3:b35cff4d:::rbd_header.1118c900944f:head [watch reconnect cookie 140670715433840 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-14T23:58:44.717 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.970653+0000", 2024-09-14T23:58:44.718 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.657489873000003, 2024-09-14T23:58:44.718 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0727808270000001, 2024-09-14T23:58:44.718 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:44.718 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:44.718 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:44.718 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:44.718 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:44.718 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:44.718 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188871 2024-09-14T23:58:44.718 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.718 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:44.718 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.718 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:44.719 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970653+0000", 2024-09-14T23:58:44.719 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:44.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.719 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.719 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:44.719 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970653+0000", 2024-09-14T23:58:44.719 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:44.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.719 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.719 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:44.719 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970655+0000", 2024-09-14T23:58:44.719 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.277e-06 2024-09-14T23:58:44.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.719 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.720 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:44.720 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970662+0000", 2024-09-14T23:58:44.720 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9619999999999996e-06 2024-09-14T23:58:44.720 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.720 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.720 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:44.720 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970682+0000", 2024-09-14T23:58:44.720 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0279999999999999e-05 2024-09-14T23:58:44.720 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.720 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.720 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:44.720 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970691+0000", 2024-09-14T23:58:44.720 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2790000000000004e-06 2024-09-14T23:58:44.721 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.721 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.721 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:44.721 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970762+0000", 2024-09-14T23:58:44.721 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1019999999999994e-05 2024-09-14T23:58:44.721 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.721 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.721 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-14T23:58:44.721 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970770+0000", 2024-09-14T23:58:44.721 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7999999999999999e-06 2024-09-14T23:58:44.721 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.722 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.722 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:44.722 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.973516+0000", 2024-09-14T23:58:44.722 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0027456469999999999 2024-09-14T23:58:44.722 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.722 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.722 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:44.722 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.973525+0000", 2024-09-14T23:58:44.722 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4890000000000001e-06 2024-09-14T23:58:44.722 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.722 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.722 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:44.723 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.041451+0000", 2024-09-14T23:58:44.723 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.067926012 2024-09-14T23:58:44.723 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.723 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.723 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:44.723 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.041673+0000", 2024-09-14T23:58:44.723 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000222048 2024-09-14T23:58:44.723 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.723 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.723 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:44.723 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.043433+0000", 2024-09-14T23:58:44.723 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017600129999999999 2024-09-14T23:58:44.723 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:44.724 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:44.724 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:44.724 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.724 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.724 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(osd.0.0:3238 3.1f e122/49 3:f8a59e83:::rbd_data.11187b9f1032.0000000000000006:2f v 122'943, mlcod=120'940)", 2024-09-14T23:58:44.724 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:22.285283+0000", 2024-09-14T23:58:44.724 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 22.342859050000001, 2024-09-14T23:58:44.724 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003756194, 2024-09-14T23:58:44.724 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:44.724 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:44.725 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:44.725 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:44.725 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.725 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:44.725 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.285283+0000", 2024-09-14T23:58:44.725 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:44.725 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.725 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.725 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:44.725 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.285283+0000", 2024-09-14T23:58:44.725 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:44.726 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.726 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.726 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:44.726 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.285284+0000", 2024-09-14T23:58:44.726 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5600000000000002e-07 2024-09-14T23:58:44.726 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.726 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.726 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:44.726 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.285289+0000", 2024-09-14T23:58:44.726 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9579999999999998e-06 2024-09-14T23:58:44.726 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.726 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.727 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:44.727 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.285300+0000", 2024-09-14T23:58:44.727 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1688999999999999e-05 2024-09-14T23:58:44.727 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.727 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.727 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:44.727 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.285309+0000", 2024-09-14T23:58:44.727 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.388e-06 2024-09-14T23:58:44.727 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.727 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.728 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:44.728 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.285367+0000", 2024-09-14T23:58:44.728 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8811000000000003e-05 2024-09-14T23:58:44.728 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.728 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.728 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:44.728 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.285388+0000", 2024-09-14T23:58:44.729 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0117000000000001e-05 2024-09-14T23:58:44.729 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.729 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.729 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:44.729 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.288994+0000", 2024-09-14T23:58:44.729 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0036064270000000002 2024-09-14T23:58:44.729 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.729 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.729 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:44.729 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.289040+0000", 2024-09-14T23:58:44.730 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5547999999999998e-05 2024-09-14T23:58:44.730 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:44.730 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:44.730 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:44.730 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.730 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.730 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(osd.6.0:1517 3.d e127/112 3:b69c12aa:::rbd_data.1118f9c96a7.0000000000000007:2c v 127'452, mlcod=126'449)", 2024-09-14T23:58:44.730 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:27.316914+0000", 2024-09-14T23:58:44.730 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 17.311228442000001, 2024-09-14T23:58:44.730 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0052453980000000001, 2024-09-14T23:58:44.730 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:44.731 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:44.731 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:44.731 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:44.731 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.731 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:44.731 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.316914+0000", 2024-09-14T23:58:44.731 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:44.731 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.731 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.731 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:44.731 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.316914+0000", 2024-09-14T23:58:44.731 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:44.732 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.732 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.732 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:44.732 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.316914+0000", 2024-09-14T23:58:44.732 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4900000000000002e-07 2024-09-14T23:58:44.732 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.732 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.732 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:44.732 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.316920+0000", 2024-09-14T23:58:44.732 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7169999999999996e-06 2024-09-14T23:58:44.733 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.733 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.733 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:44.733 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.316928+0000", 2024-09-14T23:58:44.733 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3769999999999996e-06 2024-09-14T23:58:44.733 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.733 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.733 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:44.733 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.316938+0000", 2024-09-14T23:58:44.733 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7329999999999998e-06 2024-09-14T23:58:44.734 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.734 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.734 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:44.734 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.317033+0000", 2024-09-14T23:58:44.734 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5295000000000002e-05 2024-09-14T23:58:44.734 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.734 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.734 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:44.734 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.317057+0000", 2024-09-14T23:58:44.734 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3989999999999999e-05 2024-09-14T23:58:44.734 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.735 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.735 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:44.735 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.322122+0000", 2024-09-14T23:58:44.735 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0050642500000000002 2024-09-14T23:58:44.735 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.735 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.735 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:44.735 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.322159+0000", 2024-09-14T23:58:44.735 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7787000000000003e-05 2024-09-14T23:58:44.735 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:44.735 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:44.735 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:44.735 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.735 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.736 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(osd.1.0:1466 3.16 e127/112 3:6c7910cf:::rbd_data.1118f9c96a7.0000000000000006:2c v 127'376, mlcod=121'372)", 2024-09-14T23:58:44.736 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:27.317464+0000", 2024-09-14T23:58:44.736 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 17.310678409000001, 2024-09-14T23:58:44.736 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0050844499999999999, 2024-09-14T23:58:44.736 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:44.736 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:44.736 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:44.736 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:44.736 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.736 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:44.736 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.317464+0000", 2024-09-14T23:58:44.736 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:44.736 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.736 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.736 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:44.737 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.317464+0000", 2024-09-14T23:58:44.737 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:44.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.737 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.737 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:44.737 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.317464+0000", 2024-09-14T23:58:44.737 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.91e-07 2024-09-14T23:58:44.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.737 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.737 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:44.737 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.317469+0000", 2024-09-14T23:58:44.737 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4719999999999997e-06 2024-09-14T23:58:44.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.737 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.737 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:44.738 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.317481+0000", 2024-09-14T23:58:44.738 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1814000000000001e-05 2024-09-14T23:58:44.738 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.738 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.738 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:44.738 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.317488+0000", 2024-09-14T23:58:44.738 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2740000000000002e-06 2024-09-14T23:58:44.738 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.738 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.738 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:44.738 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.317584+0000", 2024-09-14T23:58:44.738 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6191000000000001e-05 2024-09-14T23:58:44.738 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.738 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.738 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:44.738 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.317605+0000", 2024-09-14T23:58:44.739 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0554e-05 2024-09-14T23:58:44.739 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.739 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.739 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:44.739 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.322504+0000", 2024-09-14T23:58:44.739 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0048991089999999996 2024-09-14T23:58:44.739 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:44.739 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:44.739 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:44.739 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.322548+0000", 2024-09-14T23:58:44.739 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4744999999999999e-05 2024-09-14T23:58:44.739 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:44.739 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:44.739 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:44.740 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:44.740 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:44.740 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:44.741 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T23:58:44.742 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:58:44.739+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:58:44.842 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:44.841+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:44.889 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:44.889 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:58:44.889 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:58:44.889 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:44.898 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T23:58:44.943 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:44.942+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:45.005 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:45.005 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:58:45.005 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:58:45.005 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:58:45.005 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:45.014 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T23:58:45.027 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.4 to 1 (10000) 2024-09-14T23:58:45.044 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:45.042+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:45.049 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-14T23:58:45.144 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:45.143+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:45.173 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:45.173 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:58:45.173 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:58:45.173 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:58:45.173 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.173 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:107472 3.1e 3:7bf21588:::rbd_data.11187de8b0a5.000000000000005d:head [set-alloc-hint object_size 2097152 write_size 2097152,write 317788~1491] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-14T23:58:45.173 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:25.198493+0000", 2024-09-14T23:58:45.174 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 79.972440939999998, 2024-09-14T23:58:45.174 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.78175149300000002, 2024-09-14T23:58:45.174 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:45.174 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:45.174 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:45.174 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:45.174 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:45.174 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:45.174 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 107472 2024-09-14T23:58:45.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.174 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:45.174 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.174 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:45.174 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198493+0000", 2024-09-14T23:58:45.175 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.175 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.175 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:45.175 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198493+0000", 2024-09-14T23:58:45.175 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.175 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.175 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:45.175 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198495+0000", 2024-09-14T23:58:45.175 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.826e-06 2024-09-14T23:58:45.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.175 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.176 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:45.176 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198503+0000", 2024-09-14T23:58:45.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0150000000000003e-06 2024-09-14T23:58:45.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.176 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.176 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:45.176 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198521+0000", 2024-09-14T23:58:45.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8374000000000001e-05 2024-09-14T23:58:45.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.176 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.176 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:45.176 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198533+0000", 2024-09-14T23:58:45.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1593000000000001e-05 2024-09-14T23:58:45.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.177 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:45.177 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198603+0000", 2024-09-14T23:58:45.177 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0279999999999998e-05 2024-09-14T23:58:45.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.177 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:45.177 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198617+0000", 2024-09-14T23:58:45.177 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3517e-05 2024-09-14T23:58:45.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.177 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:45.178 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.977067+0000", 2024-09-14T23:58:45.178 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.77845065800000002 2024-09-14T23:58:45.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.178 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for missing object", 2024-09-14T23:58:45.178 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.977173+0000", 2024-09-14T23:58:45.178 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010543999999999999 2024-09-14T23:58:45.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.178 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:45.178 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.978737+0000", 2024-09-14T23:58:45.178 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001564078 2024-09-14T23:58:45.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.179 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:45.179 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.978845+0000", 2024-09-14T23:58:45.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000108643 2024-09-14T23:58:45.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.179 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.179 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1", 2024-09-14T23:58:45.179 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.978964+0000", 2024-09-14T23:58:45.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000119085 2024-09-14T23:58:45.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.179 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.179 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:45.179 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980109+0000", 2024-09-14T23:58:45.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0011442170000000001 2024-09-14T23:58:45.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.179 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.180 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:58:45.180 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980130+0000", 2024-09-14T23:58:45.180 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1766000000000001e-05 2024-09-14T23:58:45.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.180 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.180 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:45.180 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980171+0000", 2024-09-14T23:58:45.180 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0627999999999998e-05 2024-09-14T23:58:45.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.180 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.180 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:45.180 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980244+0000", 2024-09-14T23:58:45.180 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3373000000000004e-05 2024-09-14T23:58:45.181 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.181 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:45.181 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.181 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.181 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.181 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129159 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 5996544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:45.181 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.927783+0000", 2024-09-14T23:58:45.181 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.243151049000005, 2024-09-14T23:58:45.181 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054661122999999999, 2024-09-14T23:58:45.181 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:45.181 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:45.181 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:45.181 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:45.182 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:45.182 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:45.182 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129159 2024-09-14T23:58:45.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.182 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:45.182 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.182 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:45.182 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.927783+0000", 2024-09-14T23:58:45.182 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.182 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.182 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:45.182 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.927783+0000", 2024-09-14T23:58:45.182 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.183 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.183 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:45.183 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929684+0000", 2024-09-14T23:58:45.183 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0019011080000000001 2024-09-14T23:58:45.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.183 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.183 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:45.183 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929688+0000", 2024-09-14T23:58:45.183 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2610000000000004e-06 2024-09-14T23:58:45.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.183 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.183 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:45.183 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929697+0000", 2024-09-14T23:58:45.184 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0219999999999993e-06 2024-09-14T23:58:45.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.184 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:45.184 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929702+0000", 2024-09-14T23:58:45.184 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8840000000000002e-06 2024-09-14T23:58:45.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.184 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:45.184 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.982090+0000", 2024-09-14T23:58:45.184 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052388045000000001 2024-09-14T23:58:45.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.185 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:45.185 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.982208+0000", 2024-09-14T23:58:45.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118236 2024-09-14T23:58:45.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.185 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.185 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:45.185 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.982444+0000", 2024-09-14T23:58:45.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000235567 2024-09-14T23:58:45.185 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.185 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:45.185 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.185 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.185 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129166 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 8192000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:45.186 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.929794+0000", 2024-09-14T23:58:45.186 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.241140298000005, 2024-09-14T23:58:45.186 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054471973, 2024-09-14T23:58:45.186 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:45.186 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:45.186 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:45.186 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:45.186 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:45.186 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:45.186 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129166 2024-09-14T23:58:45.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.186 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:45.186 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.187 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:45.187 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929794+0000", 2024-09-14T23:58:45.187 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.187 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.187 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:45.187 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929794+0000", 2024-09-14T23:58:45.187 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.187 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.187 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:45.187 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929794+0000", 2024-09-14T23:58:45.187 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6899999999999997e-07 2024-09-14T23:58:45.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.188 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.188 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:45.188 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929797+0000", 2024-09-14T23:58:45.188 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.328e-06 2024-09-14T23:58:45.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.188 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.188 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:45.188 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929802+0000", 2024-09-14T23:58:45.188 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7309999999999998e-06 2024-09-14T23:58:45.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.188 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.188 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:45.188 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929807+0000", 2024-09-14T23:58:45.189 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4349999999999999e-06 2024-09-14T23:58:45.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.189 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.189 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:45.189 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984063+0000", 2024-09-14T23:58:45.189 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054256331999999997 2024-09-14T23:58:45.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.189 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.189 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:45.189 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984177+0000", 2024-09-14T23:58:45.189 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114225 2024-09-14T23:58:45.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.189 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.190 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:45.190 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984266+0000", 2024-09-14T23:58:45.190 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8352999999999999e-05 2024-09-14T23:58:45.190 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.190 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:45.190 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.190 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.190 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129167 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 6782976~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:45.190 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.929827+0000", 2024-09-14T23:58:45.190 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.241106888999994, 2024-09-14T23:58:45.190 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054637593999999998, 2024-09-14T23:58:45.190 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:45.190 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:45.191 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:45.191 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:45.191 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:45.191 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:45.191 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129167 2024-09-14T23:58:45.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.191 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:45.191 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.191 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:45.191 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929827+0000", 2024-09-14T23:58:45.191 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.191 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.191 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:45.192 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929827+0000", 2024-09-14T23:58:45.192 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.192 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.192 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:45.192 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929827+0000", 2024-09-14T23:58:45.192 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4000000000000002e-07 2024-09-14T23:58:45.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.192 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.192 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:45.192 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929830+0000", 2024-09-14T23:58:45.192 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.419e-06 2024-09-14T23:58:45.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.192 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.193 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:45.193 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929837+0000", 2024-09-14T23:58:45.193 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0450000000000003e-06 2024-09-14T23:58:45.193 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.193 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.193 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:45.193 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929842+0000", 2024-09-14T23:58:45.193 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.69e-06 2024-09-14T23:58:45.193 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.193 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.193 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:45.193 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984278+0000", 2024-09-14T23:58:45.193 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054436516999999997 2024-09-14T23:58:45.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.194 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.194 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:45.194 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984331+0000", 2024-09-14T23:58:45.194 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2590999999999998e-05 2024-09-14T23:58:45.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.194 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.194 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:45.194 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984465+0000", 2024-09-14T23:58:45.194 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013389200000000001 2024-09-14T23:58:45.194 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.194 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:45.194 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.195 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.195 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129168 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 8224768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:45.195 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.929849+0000", 2024-09-14T23:58:45.195 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.241085330999994, 2024-09-14T23:58:45.195 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054870368000000003, 2024-09-14T23:58:45.195 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:45.195 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:45.195 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:45.195 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:45.195 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:45.195 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:45.195 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129168 2024-09-14T23:58:45.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.196 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:45.196 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.196 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:45.196 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929849+0000", 2024-09-14T23:58:45.196 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.196 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.196 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.196 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:45.196 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929849+0000", 2024-09-14T23:58:45.196 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.196 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.196 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.196 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:45.196 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931681+0000", 2024-09-14T23:58:45.197 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018326989999999999 2024-09-14T23:58:45.197 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.197 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.197 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:45.197 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931686+0000", 2024-09-14T23:58:45.197 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7190000000000001e-06 2024-09-14T23:58:45.197 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.197 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.197 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:45.197 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931697+0000", 2024-09-14T23:58:45.197 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1287999999999999e-05 2024-09-14T23:58:45.197 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.197 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.198 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:45.198 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931704+0000", 2024-09-14T23:58:45.198 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3339999999999998e-06 2024-09-14T23:58:45.198 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.198 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.198 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:45.198 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984505+0000", 2024-09-14T23:58:45.198 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052801684000000002 2024-09-14T23:58:45.198 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.198 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.198 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:45.198 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984621+0000", 2024-09-14T23:58:45.198 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000116165 2024-09-14T23:58:45.198 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.199 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.199 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:45.199 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984719+0000", 2024-09-14T23:58:45.199 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7479000000000006e-05 2024-09-14T23:58:45.199 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.199 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:45.199 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.199 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.199 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129176 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 10452992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:45.199 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.931830+0000", 2024-09-14T23:58:45.199 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.239103569999997, 2024-09-14T23:58:45.199 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054513292999999997, 2024-09-14T23:58:45.199 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:45.200 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:45.200 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:45.200 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:45.200 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:45.200 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:45.200 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129176 2024-09-14T23:58:45.200 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.200 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:45.200 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.200 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:45.200 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931830+0000", 2024-09-14T23:58:45.200 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.200 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.201 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.201 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:45.201 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931830+0000", 2024-09-14T23:58:45.201 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.201 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.201 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.201 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:45.201 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.933678+0000", 2024-09-14T23:58:45.201 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018472440000000001 2024-09-14T23:58:45.201 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.201 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.201 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:45.201 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.933686+0000", 2024-09-14T23:58:45.201 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0239999999999997e-06 2024-09-14T23:58:45.202 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.202 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.202 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:45.202 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.933694+0000", 2024-09-14T23:58:45.202 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8400000000000001e-06 2024-09-14T23:58:45.202 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.202 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.202 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:45.202 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.933701+0000", 2024-09-14T23:58:45.202 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2870000000000001e-06 2024-09-14T23:58:45.202 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.202 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.202 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:45.202 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.986165+0000", 2024-09-14T23:58:45.203 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052463900000000001 2024-09-14T23:58:45.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.203 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.203 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:45.203 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.986259+0000", 2024-09-14T23:58:45.203 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4435e-05 2024-09-14T23:58:45.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.203 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.203 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:45.203 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.986344+0000", 2024-09-14T23:58:45.203 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4562999999999997e-05 2024-09-14T23:58:45.203 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.203 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:45.203 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.204 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.204 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129201 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 10289152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:45.204 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.935960+0000", 2024-09-14T23:58:45.204 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.234974108000003, 2024-09-14T23:58:45.204 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054338487999999997, 2024-09-14T23:58:45.204 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:45.204 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:45.204 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:45.204 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:45.204 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:45.204 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:45.204 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129201 2024-09-14T23:58:45.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.205 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:45.205 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.205 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:45.205 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935960+0000", 2024-09-14T23:58:45.205 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.205 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.205 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:45.205 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935960+0000", 2024-09-14T23:58:45.205 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.205 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.205 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:45.206 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935960+0000", 2024-09-14T23:58:45.206 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.72e-07 2024-09-14T23:58:45.206 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.206 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.206 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:45.206 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935963+0000", 2024-09-14T23:58:45.206 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5390000000000002e-06 2024-09-14T23:58:45.206 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.206 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.206 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:45.206 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935971+0000", 2024-09-14T23:58:45.206 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4589999999999998e-06 2024-09-14T23:58:45.206 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.206 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.207 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:45.207 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935977+0000", 2024-09-14T23:58:45.207 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4110000000000004e-06 2024-09-14T23:58:45.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.207 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.207 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:45.207 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990213+0000", 2024-09-14T23:58:45.207 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054236642000000002 2024-09-14T23:58:45.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.207 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.207 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:45.207 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990257+0000", 2024-09-14T23:58:45.207 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3696000000000003e-05 2024-09-14T23:58:45.208 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.208 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.208 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:45.208 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990298+0000", 2024-09-14T23:58:45.208 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1369000000000003e-05 2024-09-14T23:58:45.208 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.208 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:45.208 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.208 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.208 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.208 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129202 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 11370496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:45.208 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.935986+0000", 2024-09-14T23:58:45.208 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.234947489999996, 2024-09-14T23:58:45.209 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054543700000000001, 2024-09-14T23:58:45.209 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:45.209 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:45.209 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:45.209 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:45.209 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:45.209 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:45.209 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129202 2024-09-14T23:58:45.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.209 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:45.209 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.209 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:45.209 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935986+0000", 2024-09-14T23:58:45.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.210 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:45.210 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935986+0000", 2024-09-14T23:58:45.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.210 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:45.210 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937678+0000", 2024-09-14T23:58:45.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016917849999999999 2024-09-14T23:58:45.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.210 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:45.211 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937682+0000", 2024-09-14T23:58:45.211 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2400000000000001e-06 2024-09-14T23:58:45.211 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.211 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.211 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:45.211 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937695+0000", 2024-09-14T23:58:45.211 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2999e-05 2024-09-14T23:58:45.211 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.211 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.211 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:45.211 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937700+0000", 2024-09-14T23:58:45.211 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8420000000000004e-06 2024-09-14T23:58:45.211 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.212 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.212 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:45.212 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990336+0000", 2024-09-14T23:58:45.212 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052636060999999998 2024-09-14T23:58:45.212 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.212 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.212 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:45.212 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990435+0000", 2024-09-14T23:58:45.212 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9015e-05 2024-09-14T23:58:45.212 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.212 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.212 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:45.212 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990530+0000", 2024-09-14T23:58:45.212 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4758000000000003e-05 2024-09-14T23:58:45.213 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.213 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:45.213 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.213 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.213 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.213 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129214 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 11468800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:45.213 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.937939+0000", 2024-09-14T23:58:45.213 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.232994575999996, 2024-09-14T23:58:45.213 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054463733, 2024-09-14T23:58:45.213 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:45.213 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:45.213 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:45.213 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:45.214 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:45.214 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:45.214 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129214 2024-09-14T23:58:45.214 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.214 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:45.214 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.214 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:45.214 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937939+0000", 2024-09-14T23:58:45.214 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.214 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.214 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.214 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:45.214 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937939+0000", 2024-09-14T23:58:45.214 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.215 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.215 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.215 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:45.215 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939674+0000", 2024-09-14T23:58:45.215 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017342589999999999 2024-09-14T23:58:45.215 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.215 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.215 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:45.215 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939678+0000", 2024-09-14T23:58:45.215 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2590000000000002e-06 2024-09-14T23:58:45.215 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.215 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.215 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:45.215 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939699+0000", 2024-09-14T23:58:45.215 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1143000000000001e-05 2024-09-14T23:58:45.216 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.216 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.216 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:45.216 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939706+0000", 2024-09-14T23:58:45.216 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0269999999999998e-06 2024-09-14T23:58:45.216 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.216 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.216 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:45.216 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.992221+0000", 2024-09-14T23:58:45.216 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052514981000000002 2024-09-14T23:58:45.216 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.216 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.216 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:45.216 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.992321+0000", 2024-09-14T23:58:45.217 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9625999999999998e-05 2024-09-14T23:58:45.217 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.217 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.217 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:45.217 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.992403+0000", 2024-09-14T23:58:45.217 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2437999999999999e-05 2024-09-14T23:58:45.217 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.217 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:45.217 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.217 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.217 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.217 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129226 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 14090240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:45.217 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.939867+0000", 2024-09-14T23:58:45.217 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.231066647999995, 2024-09-14T23:58:45.217 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054523573999999998, 2024-09-14T23:58:45.218 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:45.218 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:45.218 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:45.218 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:45.218 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:45.218 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:45.218 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129226 2024-09-14T23:58:45.218 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.218 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:45.218 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.218 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:45.218 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939867+0000", 2024-09-14T23:58:45.218 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.218 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.218 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.219 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:45.219 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939867+0000", 2024-09-14T23:58:45.219 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.219 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.219 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.219 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:45.219 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941669+0000", 2024-09-14T23:58:45.219 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018021549999999999 2024-09-14T23:58:45.219 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.219 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.219 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:45.219 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941673+0000", 2024-09-14T23:58:45.219 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1269999999999999e-06 2024-09-14T23:58:45.219 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.219 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.219 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:45.220 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941679+0000", 2024-09-14T23:58:45.220 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1879999999999999e-06 2024-09-14T23:58:45.220 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.220 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.220 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:45.220 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941682+0000", 2024-09-14T23:58:45.220 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7110000000000001e-06 2024-09-14T23:58:45.220 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.220 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.220 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:45.220 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.994201+0000", 2024-09-14T23:58:45.220 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052518265000000001 2024-09-14T23:58:45.220 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.220 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.220 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:45.221 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.994302+0000", 2024-09-14T23:58:45.221 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101653 2024-09-14T23:58:45.221 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.221 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.221 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:45.221 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.994391+0000", 2024-09-14T23:58:45.221 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8474999999999996e-05 2024-09-14T23:58:45.221 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.221 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:45.221 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.221 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.221 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.221 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129237 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 12386304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:45.221 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.941825+0000", 2024-09-14T23:58:45.222 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.229108886000006, 2024-09-14T23:58:45.222 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054368076000000001, 2024-09-14T23:58:45.222 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:45.222 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:45.222 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:45.222 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:45.222 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:45.222 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:45.222 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129237 2024-09-14T23:58:45.222 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.222 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:45.222 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.222 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:45.222 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941825+0000", 2024-09-14T23:58:45.223 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.223 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.223 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.223 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:45.223 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941825+0000", 2024-09-14T23:58:45.223 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.223 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.223 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.223 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:45.223 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941825+0000", 2024-09-14T23:58:45.223 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.29e-07 2024-09-14T23:58:45.223 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.224 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.224 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:45.224 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941826+0000", 2024-09-14T23:58:45.224 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0999999999999997e-07 2024-09-14T23:58:45.224 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.224 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.224 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:45.224 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941834+0000", 2024-09-14T23:58:45.224 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5519999999999999e-06 2024-09-14T23:58:45.224 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.224 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.224 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:45.225 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941843+0000", 2024-09-14T23:58:45.225 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2960000000000004e-06 2024-09-14T23:58:45.225 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.225 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.225 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:45.225 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996108+0000", 2024-09-14T23:58:45.225 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054265200999999999 2024-09-14T23:58:45.225 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.225 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.225 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:45.225 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996152+0000", 2024-09-14T23:58:45.225 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4066999999999997e-05 2024-09-14T23:58:45.225 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.225 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.226 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:45.226 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996193+0000", 2024-09-14T23:58:45.226 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0821000000000001e-05 2024-09-14T23:58:45.226 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.226 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:45.226 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.226 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.226 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.226 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129238 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 13828096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:45.226 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.941851+0000", 2024-09-14T23:58:45.226 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.229082562000002, 2024-09-14T23:58:45.226 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.05458031, 2024-09-14T23:58:45.226 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:45.226 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:45.226 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:45.227 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:45.227 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:45.227 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:45.227 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129238 2024-09-14T23:58:45.227 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.227 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:45.227 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.227 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:45.227 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941851+0000", 2024-09-14T23:58:45.227 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.227 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.227 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.227 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:45.227 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941851+0000", 2024-09-14T23:58:45.228 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.228 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.228 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.228 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:45.228 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943670+0000", 2024-09-14T23:58:45.228 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018181930000000001 2024-09-14T23:58:45.228 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.228 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.228 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:45.228 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943673+0000", 2024-09-14T23:58:45.228 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3790000000000002e-06 2024-09-14T23:58:45.228 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.228 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.228 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:45.229 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943679+0000", 2024-09-14T23:58:45.229 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8719999999999999e-06 2024-09-14T23:58:45.229 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.229 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.229 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:45.229 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943683+0000", 2024-09-14T23:58:45.229 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8539999999999999e-06 2024-09-14T23:58:45.229 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.229 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.229 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:45.229 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996233+0000", 2024-09-14T23:58:45.229 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052550690999999997 2024-09-14T23:58:45.229 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.230 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.230 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:45.230 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996339+0000", 2024-09-14T23:58:45.230 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105989 2024-09-14T23:58:45.230 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.230 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.230 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:45.230 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996432+0000", 2024-09-14T23:58:45.230 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2331999999999996e-05 2024-09-14T23:58:45.230 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.230 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:45.230 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.230 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.230 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.231 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129249 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 12124160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:45.231 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.943826+0000", 2024-09-14T23:58:45.231 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.227108071000004, 2024-09-14T23:58:45.231 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054373266000000003, 2024-09-14T23:58:45.231 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:45.231 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:45.231 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:45.231 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:45.231 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:45.231 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:45.231 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129249 2024-09-14T23:58:45.231 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.231 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:45.231 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.232 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:45.232 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943826+0000", 2024-09-14T23:58:45.232 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.232 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.232 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.232 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:45.232 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943826+0000", 2024-09-14T23:58:45.232 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.232 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.232 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.232 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:45.232 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943826+0000", 2024-09-14T23:58:45.232 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.03e-07 2024-09-14T23:58:45.232 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.233 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.233 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:45.233 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943827+0000", 2024-09-14T23:58:45.233 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.08e-06 2024-09-14T23:58:45.233 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.233 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.233 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:45.233 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943834+0000", 2024-09-14T23:58:45.233 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4119999999999998e-06 2024-09-14T23:58:45.233 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.233 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.233 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:45.233 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943836+0000", 2024-09-14T23:58:45.233 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7369999999999998e-06 2024-09-14T23:58:45.234 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.234 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.234 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:45.234 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998115+0000", 2024-09-14T23:58:45.234 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054279040000000001 2024-09-14T23:58:45.234 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.234 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.234 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:45.234 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998159+0000", 2024-09-14T23:58:45.234 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3486999999999999e-05 2024-09-14T23:58:45.234 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.234 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.234 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:45.234 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998199+0000", 2024-09-14T23:58:45.234 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0306999999999999e-05 2024-09-14T23:58:45.235 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.235 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:45.235 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.235 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.235 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.235 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129250 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 13565952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:45.235 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.943840+0000", 2024-09-14T23:58:45.235 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.227094120999993, 2024-09-14T23:58:45.235 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054573875000000001, 2024-09-14T23:58:45.235 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:45.235 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:45.235 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:45.235 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:45.235 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:45.236 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:45.236 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129250 2024-09-14T23:58:45.236 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.236 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:45.236 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.236 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:45.236 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943840+0000", 2024-09-14T23:58:45.236 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.236 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.236 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.236 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:45.236 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943840+0000", 2024-09-14T23:58:45.236 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.236 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.236 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.237 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:45.237 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945669+0000", 2024-09-14T23:58:45.237 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018289720000000001 2024-09-14T23:58:45.237 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.237 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.237 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:45.237 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945671+0000", 2024-09-14T23:58:45.237 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6869999999999999e-06 2024-09-14T23:58:45.237 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.237 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.237 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:45.237 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945680+0000", 2024-09-14T23:58:45.237 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5259999999999993e-06 2024-09-14T23:58:45.237 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.238 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.238 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:45.238 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945685+0000", 2024-09-14T23:58:45.238 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8559999999999998e-06 2024-09-14T23:58:45.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.238 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.238 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:45.238 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998242+0000", 2024-09-14T23:58:45.238 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052557505999999997 2024-09-14T23:58:45.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.238 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.238 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:45.238 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998329+0000", 2024-09-14T23:58:45.238 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6410999999999996e-05 2024-09-14T23:58:45.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.238 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.239 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:45.239 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998414+0000", 2024-09-14T23:58:45.239 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4917e-05 2024-09-14T23:58:45.239 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.239 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:45.239 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.239 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.239 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.239 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129262 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 13303808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:45.239 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.945825+0000", 2024-09-14T23:58:45.239 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.225109180000004, 2024-09-14T23:58:45.239 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054515942999999997, 2024-09-14T23:58:45.239 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:45.239 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:45.239 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:45.239 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:45.240 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:45.240 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:45.240 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129262 2024-09-14T23:58:45.240 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.240 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:45.240 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.240 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:45.240 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945825+0000", 2024-09-14T23:58:45.240 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.240 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.240 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.240 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:45.240 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945825+0000", 2024-09-14T23:58:45.240 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.240 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.241 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.241 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:45.241 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.947670+0000", 2024-09-14T23:58:45.241 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018451229999999999 2024-09-14T23:58:45.241 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.241 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.241 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:45.241 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.947672+0000", 2024-09-14T23:58:45.241 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5289999999999998e-06 2024-09-14T23:58:45.241 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.241 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.241 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:45.241 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.947678+0000", 2024-09-14T23:58:45.241 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9880000000000001e-06 2024-09-14T23:58:45.242 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.242 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.242 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:45.242 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.947682+0000", 2024-09-14T23:58:45.242 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.63e-06 2024-09-14T23:58:45.242 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.242 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.242 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:45.242 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000183+0000", 2024-09-14T23:58:45.242 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052500770000000002 2024-09-14T23:58:45.242 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.242 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.242 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:45.242 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000266+0000", 2024-09-14T23:58:45.243 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3133000000000005e-05 2024-09-14T23:58:45.243 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.243 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.243 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:45.243 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000341+0000", 2024-09-14T23:58:45.243 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4770000000000004e-05 2024-09-14T23:58:45.243 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.243 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:45.243 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.243 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.243 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.243 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129299 3.10 3:0f1e06ee:::rbd_object_map.1118721be852:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:45.243 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.952330+0000", 2024-09-14T23:58:45.243 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.218604366999998, 2024-09-14T23:58:45.243 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054412818000000002, 2024-09-14T23:58:45.244 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:45.244 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:45.244 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:45.244 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:45.244 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:45.244 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:45.244 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129299 2024-09-14T23:58:45.244 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.244 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:45.244 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.244 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:45.244 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952330+0000", 2024-09-14T23:58:45.244 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.244 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.244 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.245 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:45.245 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952330+0000", 2024-09-14T23:58:45.245 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.245 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.245 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.245 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:45.245 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952330+0000", 2024-09-14T23:58:45.245 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4900000000000001e-07 2024-09-14T23:58:45.245 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.245 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.245 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:45.245 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952332+0000", 2024-09-14T23:58:45.245 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2879999999999999e-06 2024-09-14T23:58:45.246 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.246 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.246 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:45.246 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952340+0000", 2024-09-14T23:58:45.246 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9910000000000002e-06 2024-09-14T23:58:45.246 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.246 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.246 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:45.246 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952344+0000", 2024-09-14T23:58:45.246 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0280000000000002e-06 2024-09-14T23:58:45.246 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.246 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.246 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:45.246 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.004722+0000", 2024-09-14T23:58:45.246 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052377932000000002 2024-09-14T23:58:45.247 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.247 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.247 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:45.247 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.004778+0000", 2024-09-14T23:58:45.247 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5795999999999999e-05 2024-09-14T23:58:45.247 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.247 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.247 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:58:45.247 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.005016+0000", 2024-09-14T23:58:45.247 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000238006 2024-09-14T23:58:45.247 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.247 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.247 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:58:45.247 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.006366+0000", 2024-09-14T23:58:45.248 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001349786 2024-09-14T23:58:45.248 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.248 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.248 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:45.248 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.006685+0000", 2024-09-14T23:58:45.248 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00031970000000000002 2024-09-14T23:58:45.248 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.248 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.248 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:45.248 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.006718+0000", 2024-09-14T23:58:45.248 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2329000000000002e-05 2024-09-14T23:58:45.249 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.249 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.249 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:45.249 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.006742+0000", 2024-09-14T23:58:45.249 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4613e-05 2024-09-14T23:58:45.249 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.249 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:45.249 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.249 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.249 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.249 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186210 3.d 3.b2ff3acd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:45.249 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.129044+0000", 2024-09-14T23:58:45.250 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 38.041890004000003, 2024-09-14T23:58:45.250 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.84555443900000005, 2024-09-14T23:58:45.250 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:45.250 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:45.250 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:58:45.250 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:45.250 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:45.250 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:45.250 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186210 2024-09-14T23:58:45.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.250 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:45.250 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.250 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:45.250 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129044+0000", 2024-09-14T23:58:45.250 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.251 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.251 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.251 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:45.251 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129044+0000", 2024-09-14T23:58:45.251 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.251 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.251 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.251 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:45.251 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129046+0000", 2024-09-14T23:58:45.251 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.742e-06 2024-09-14T23:58:45.251 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.251 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.251 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:45.252 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129052+0000", 2024-09-14T23:58:45.252 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7390000000000002e-06 2024-09-14T23:58:45.252 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.252 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.252 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:45.252 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129067+0000", 2024-09-14T23:58:45.252 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.417e-05 2024-09-14T23:58:45.252 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.252 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.252 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:45.252 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129076+0000", 2024-09-14T23:58:45.252 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2289999999999992e-06 2024-09-14T23:58:45.252 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.252 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.253 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:45.253 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.130947+0000", 2024-09-14T23:58:45.253 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001871075 2024-09-14T23:58:45.253 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.253 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.253 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:45.253 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.130955+0000", 2024-09-14T23:58:45.253 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4500000000000004e-06 2024-09-14T23:58:45.253 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.253 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.253 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:45.253 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974587+0000", 2024-09-14T23:58:45.253 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.84363205500000005 2024-09-14T23:58:45.254 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.254 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.254 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:45.254 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974598+0000", 2024-09-14T23:58:45.254 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0978999999999999e-05 2024-09-14T23:58:45.254 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.254 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:45.254 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.254 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.254 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.254 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187415 3.d 3.b2ff3acd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:45.254 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.254681+0000", 2024-09-14T23:58:45.254 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 37.916252483000001, 2024-09-14T23:58:45.255 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.71995530699999999, 2024-09-14T23:58:45.255 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:45.255 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:45.255 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:58:45.255 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:45.255 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:45.255 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:45.255 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187415 2024-09-14T23:58:45.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.255 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:45.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.255 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:45.255 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254681+0000", 2024-09-14T23:58:45.255 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.256 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.256 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:45.256 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254681+0000", 2024-09-14T23:58:45.256 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.256 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.256 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.256 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:45.256 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254682+0000", 2024-09-14T23:58:45.256 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2e-07 2024-09-14T23:58:45.256 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.256 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.256 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:45.256 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254684+0000", 2024-09-14T23:58:45.256 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2730000000000001e-06 2024-09-14T23:58:45.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.257 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.257 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:45.257 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254689+0000", 2024-09-14T23:58:45.257 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4769999999999997e-06 2024-09-14T23:58:45.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.257 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.257 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:45.257 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254692+0000", 2024-09-14T23:58:45.257 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9890000000000001e-06 2024-09-14T23:58:45.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.257 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.258 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:45.258 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254721+0000", 2024-09-14T23:58:45.258 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9762999999999999e-05 2024-09-14T23:58:45.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.258 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.258 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:45.258 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254725+0000", 2024-09-14T23:58:45.258 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9650000000000002e-06 2024-09-14T23:58:45.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.258 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.258 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:45.258 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974628+0000", 2024-09-14T23:58:45.258 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.71990235499999999 2024-09-14T23:58:45.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.259 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.259 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:45.259 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974637+0000", 2024-09-14T23:58:45.259 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0650000000000005e-06 2024-09-14T23:58:45.259 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.259 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:45.259 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.259 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.259 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187457 3.d 3.b2ff3acd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:45.259 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.257980+0000", 2024-09-14T23:58:45.259 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 37.912953909000002, 2024-09-14T23:58:45.259 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.71671711199999999, 2024-09-14T23:58:45.259 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:45.260 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:45.260 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:58:45.260 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:45.260 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:45.260 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:45.260 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187457 2024-09-14T23:58:45.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.260 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:45.260 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.260 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:45.260 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257980+0000", 2024-09-14T23:58:45.260 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.260 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.260 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:45.261 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257980+0000", 2024-09-14T23:58:45.261 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.261 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.261 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:45.261 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257980+0000", 2024-09-14T23:58:45.261 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.17e-07 2024-09-14T23:58:45.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.261 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.261 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:45.261 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257981+0000", 2024-09-14T23:58:45.261 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.172e-06 2024-09-14T23:58:45.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.261 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.262 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:45.262 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257984+0000", 2024-09-14T23:58:45.262 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9550000000000001e-06 2024-09-14T23:58:45.262 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.262 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.262 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:45.262 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257987+0000", 2024-09-14T23:58:45.262 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5799999999999999e-06 2024-09-14T23:58:45.262 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.262 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.262 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:45.262 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.258030+0000", 2024-09-14T23:58:45.262 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3118999999999998e-05 2024-09-14T23:58:45.262 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.263 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:45.263 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.258038+0000", 2024-09-14T23:58:45.263 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6850000000000001e-06 2024-09-14T23:58:45.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.263 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:45.263 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974688+0000", 2024-09-14T23:58:45.263 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.71665016000000004 2024-09-14T23:58:45.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.263 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:45.263 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974697+0000", 2024-09-14T23:58:45.263 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2240000000000001e-06 2024-09-14T23:58:45.263 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.263 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:45.264 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.264 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.264 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:251656 3.c 3:32436721:::rbd_object_map.1118e09062ff:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-14T23:58:45.264 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:27.433419+0000", 2024-09-14T23:58:45.264 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 17.737515235, 2024-09-14T23:58:45.264 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.55083879499999999, 2024-09-14T23:58:45.264 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:45.264 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:45.264 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:45.264 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:45.264 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:45.264 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:45.264 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 251656 2024-09-14T23:58:45.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.265 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:45.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.265 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:45.265 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433419+0000", 2024-09-14T23:58:45.265 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.265 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:45.265 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433419+0000", 2024-09-14T23:58:45.265 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.265 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:45.265 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433419+0000", 2024-09-14T23:58:45.265 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1500000000000003e-07 2024-09-14T23:58:45.266 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.266 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.266 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:45.266 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433424+0000", 2024-09-14T23:58:45.266 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2409999999999997e-06 2024-09-14T23:58:45.266 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.266 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.266 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:45.266 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433435+0000", 2024-09-14T23:58:45.266 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1009000000000001e-05 2024-09-14T23:58:45.266 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.266 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.266 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:45.267 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433440+0000", 2024-09-14T23:58:45.267 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1950000000000002e-06 2024-09-14T23:58:45.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.267 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.267 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:45.267 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433487+0000", 2024-09-14T23:58:45.267 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7184000000000001e-05 2024-09-14T23:58:45.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.267 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.267 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:45.267 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433494+0000", 2024-09-14T23:58:45.267 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5490000000000003e-06 2024-09-14T23:58:45.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.267 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.267 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:45.268 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.981923+0000", 2024-09-14T23:58:45.268 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.54842898600000001 2024-09-14T23:58:45.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.268 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.268 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:45.268 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.982097+0000", 2024-09-14T23:58:45.268 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017428699999999999 2024-09-14T23:58:45.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.268 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.269 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5", 2024-09-14T23:58:45.269 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.982334+0000", 2024-09-14T23:58:45.269 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00023700899999999999 2024-09-14T23:58:45.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.269 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.269 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:45.269 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.983927+0000", 2024-09-14T23:58:45.269 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0015932710000000001 2024-09-14T23:58:45.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.269 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.269 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:58:45.269 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.984128+0000", 2024-09-14T23:58:45.269 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020104199999999999 2024-09-14T23:58:45.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.270 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.270 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:45.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.984212+0000", 2024-09-14T23:58:45.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3709999999999996e-05 2024-09-14T23:58:45.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.270 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.270 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:45.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.984257+0000", 2024-09-14T23:58:45.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5497000000000002e-05 2024-09-14T23:58:45.270 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.270 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:45.270 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.270 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.271 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:45.271 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:45.272 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T23:58:45.273 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:45.270+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:45.373 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:45.372+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:45.380 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:45.380 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:58:45.380 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:58:45.380 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:45.389 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T23:58:45.474 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:45.473+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:45.496 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:45.496 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:58:45.496 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:58:45.496 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:58:45.496 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:45.505 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T23:58:45.576 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:45.574+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:45.614 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:45.614 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:58:45.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:58:45.614 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:58:45.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.614 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69262 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 4800512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:45.615 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300811+0000", 2024-09-14T23:58:45.615 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 90.310965780000004, 2024-09-14T23:58:45.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050403796000000001, 2024-09-14T23:58:45.615 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:45.615 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:45.615 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:45.615 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:45.615 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:45.615 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:45.615 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69262 2024-09-14T23:58:45.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.615 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:45.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:45.616 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300811+0000", 2024-09-14T23:58:45.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:45.616 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300811+0000", 2024-09-14T23:58:45.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:45.616 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300811+0000", 2024-09-14T23:58:45.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5700000000000002e-07 2024-09-14T23:58:45.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:45.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300815+0000", 2024-09-14T23:58:45.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.087e-06 2024-09-14T23:58:45.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:45.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300823+0000", 2024-09-14T23:58:45.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8720000000000002e-06 2024-09-14T23:58:45.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:45.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300830+0000", 2024-09-14T23:58:45.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0589999999999997e-06 2024-09-14T23:58:45.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:45.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351020+0000", 2024-09-14T23:58:45.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050189552999999998 2024-09-14T23:58:45.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:45.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351112+0000", 2024-09-14T23:58:45.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2615999999999996e-05 2024-09-14T23:58:45.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:45.619 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351214+0000", 2024-09-14T23:58:45.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101952 2024-09-14T23:58:45.619 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.619 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:45.619 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.619 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69263 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 4915200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:45.619 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300842+0000", 2024-09-14T23:58:45.619 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 90.310934275999998, 2024-09-14T23:58:45.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050590637000000001, 2024-09-14T23:58:45.619 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:45.619 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:45.619 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:45.619 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:45.620 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:45.620 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:45.620 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69263 2024-09-14T23:58:45.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.620 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:45.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:45.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300842+0000", 2024-09-14T23:58:45.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:45.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300842+0000", 2024-09-14T23:58:45.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:45.621 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300843+0000", 2024-09-14T23:58:45.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4000000000000002e-07 2024-09-14T23:58:45.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:45.621 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300846+0000", 2024-09-14T23:58:45.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8720000000000002e-06 2024-09-14T23:58:45.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:45.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300853+0000", 2024-09-14T23:58:45.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0450000000000003e-06 2024-09-14T23:58:45.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:45.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300859+0000", 2024-09-14T23:58:45.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9510000000000003e-06 2024-09-14T23:58:45.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:45.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351260+0000", 2024-09-14T23:58:45.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050401654999999997 2024-09-14T23:58:45.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:45.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351353+0000", 2024-09-14T23:58:45.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2785e-05 2024-09-14T23:58:45.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:45.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351433+0000", 2024-09-14T23:58:45.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9789000000000005e-05 2024-09-14T23:58:45.623 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.623 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:45.623 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.623 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69264 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5029888~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:45.624 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300866+0000", 2024-09-14T23:58:45.624 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 90.310910343000003, 2024-09-14T23:58:45.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050794063, 2024-09-14T23:58:45.624 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:45.624 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:45.624 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:45.624 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:45.624 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:45.624 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:45.624 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69264 2024-09-14T23:58:45.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.624 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:45.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.624 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:45.625 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300866+0000", 2024-09-14T23:58:45.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.625 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:45.625 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300866+0000", 2024-09-14T23:58:45.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.625 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:45.625 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300866+0000", 2024-09-14T23:58:45.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8799999999999998e-07 2024-09-14T23:58:45.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:45.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300869+0000", 2024-09-14T23:58:45.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0880000000000002e-06 2024-09-14T23:58:45.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:45.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300875+0000", 2024-09-14T23:58:45.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9280000000000004e-06 2024-09-14T23:58:45.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:45.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300882+0000", 2024-09-14T23:58:45.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4740000000000002e-06 2024-09-14T23:58:45.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:45.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351472+0000", 2024-09-14T23:58:45.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050590006999999999 2024-09-14T23:58:45.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:45.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351563+0000", 2024-09-14T23:58:45.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0762999999999998e-05 2024-09-14T23:58:45.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:45.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351660+0000", 2024-09-14T23:58:45.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7415000000000002e-05 2024-09-14T23:58:45.627 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.628 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:45.628 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.628 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69265 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5144576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:45.628 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300890+0000", 2024-09-14T23:58:45.628 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 90.310886839000005, 2024-09-14T23:58:45.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050921389999999997, 2024-09-14T23:58:45.628 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:45.628 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:45.628 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:45.628 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:45.628 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:45.629 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:45.629 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69265 2024-09-14T23:58:45.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.629 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:45.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.629 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:45.629 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300890+0000", 2024-09-14T23:58:45.629 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.629 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:45.629 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300890+0000", 2024-09-14T23:58:45.629 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:45.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300890+0000", 2024-09-14T23:58:45.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2500000000000001e-07 2024-09-14T23:58:45.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:45.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300892+0000", 2024-09-14T23:58:45.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2469999999999998e-06 2024-09-14T23:58:45.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:45.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300899+0000", 2024-09-14T23:58:45.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5509999999999996e-06 2024-09-14T23:58:45.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.631 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:45.631 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300904+0000", 2024-09-14T23:58:45.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3299999999999998e-06 2024-09-14T23:58:45.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.631 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:45.631 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351707+0000", 2024-09-14T23:58:45.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050802673999999999 2024-09-14T23:58:45.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.631 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:45.631 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351755+0000", 2024-09-14T23:58:45.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8362000000000002e-05 2024-09-14T23:58:45.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.632 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:45.632 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351811+0000", 2024-09-14T23:58:45.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5801000000000003e-05 2024-09-14T23:58:45.632 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.632 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:45.632 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.632 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69266 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5259264~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:45.632 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300911+0000", 2024-09-14T23:58:45.632 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 90.310865766000006, 2024-09-14T23:58:45.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051125089999999998, 2024-09-14T23:58:45.633 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:45.633 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:45.633 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:45.633 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:45.633 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:45.633 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:45.633 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69266 2024-09-14T23:58:45.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.633 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:45.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.633 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:45.633 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300911+0000", 2024-09-14T23:58:45.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:45.634 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300911+0000", 2024-09-14T23:58:45.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:45.634 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300911+0000", 2024-09-14T23:58:45.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-14T23:58:45.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:45.634 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300913+0000", 2024-09-14T23:58:45.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9300000000000002e-06 2024-09-14T23:58:45.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:45.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300919+0000", 2024-09-14T23:58:45.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.489e-06 2024-09-14T23:58:45.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:45.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300925+0000", 2024-09-14T23:58:45.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1719999999999998e-06 2024-09-14T23:58:45.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:45.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351847+0000", 2024-09-14T23:58:45.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050922621000000001 2024-09-14T23:58:45.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:45.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351938+0000", 2024-09-14T23:58:45.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0830999999999998e-05 2024-09-14T23:58:45.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:45.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352036+0000", 2024-09-14T23:58:45.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7676999999999993e-05 2024-09-14T23:58:45.636 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.636 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:45.636 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.637 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69267 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5373952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:45.637 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300945+0000", 2024-09-14T23:58:45.637 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 90.310830971000001, 2024-09-14T23:58:45.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051217934, 2024-09-14T23:58:45.637 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:45.637 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:45.637 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:45.637 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:45.637 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:45.637 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:45.637 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69267 2024-09-14T23:58:45.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.638 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:45.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.638 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:45.638 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300945+0000", 2024-09-14T23:58:45.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.638 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:45.638 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300945+0000", 2024-09-14T23:58:45.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.638 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:45.638 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300946+0000", 2024-09-14T23:58:45.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2600000000000002e-07 2024-09-14T23:58:45.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:45.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300948+0000", 2024-09-14T23:58:45.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4940000000000002e-06 2024-09-14T23:58:45.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:45.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300956+0000", 2024-09-14T23:58:45.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.622e-06 2024-09-14T23:58:45.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:45.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300962+0000", 2024-09-14T23:58:45.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5790000000000003e-06 2024-09-14T23:58:45.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:45.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352063+0000", 2024-09-14T23:58:45.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051101281999999998 2024-09-14T23:58:45.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:45.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352110+0000", 2024-09-14T23:58:45.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7401000000000002e-05 2024-09-14T23:58:45.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:45.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352163+0000", 2024-09-14T23:58:45.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3029999999999999e-05 2024-09-14T23:58:45.641 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.641 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:45.641 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.641 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69268 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5488640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:45.641 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300969+0000", 2024-09-14T23:58:45.641 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 90.310807299000004, 2024-09-14T23:58:45.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051413420000000001, 2024-09-14T23:58:45.641 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:45.641 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:45.642 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:45.642 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:45.642 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:45.642 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:45.642 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69268 2024-09-14T23:58:45.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.642 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:45.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:45.642 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300969+0000", 2024-09-14T23:58:45.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:45.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300969+0000", 2024-09-14T23:58:45.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:45.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300970+0000", 2024-09-14T23:58:45.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7899999999999999e-07 2024-09-14T23:58:45.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:45.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300982+0000", 2024-09-14T23:58:45.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2741e-05 2024-09-14T23:58:45.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:45.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300990+0000", 2024-09-14T23:58:45.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8169999999999999e-06 2024-09-14T23:58:45.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:45.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300995+0000", 2024-09-14T23:58:45.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2240000000000003e-06 2024-09-14T23:58:45.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:45.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352200+0000", 2024-09-14T23:58:45.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051204223 2024-09-14T23:58:45.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:45.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352285+0000", 2024-09-14T23:58:45.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5519999999999994e-05 2024-09-14T23:58:45.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:45.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352383+0000", 2024-09-14T23:58:45.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7516000000000006e-05 2024-09-14T23:58:45.646 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.646 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:45.646 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.646 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69269 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5603328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:45.646 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.301003+0000", 2024-09-14T23:58:45.646 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 90.310773116999997, 2024-09-14T23:58:45.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051497381000000002, 2024-09-14T23:58:45.646 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:45.646 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:45.646 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:45.646 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:45.646 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:45.647 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:45.647 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69269 2024-09-14T23:58:45.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.647 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:45.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:45.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.301003+0000", 2024-09-14T23:58:45.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:45.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.301003+0000", 2024-09-14T23:58:45.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:45.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302671+0000", 2024-09-14T23:58:45.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001667785 2024-09-14T23:58:45.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:45.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302676+0000", 2024-09-14T23:58:45.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6600000000000003e-06 2024-09-14T23:58:45.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:45.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302697+0000", 2024-09-14T23:58:45.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1288999999999999e-05 2024-09-14T23:58:45.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:45.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302716+0000", 2024-09-14T23:58:45.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9128999999999999e-05 2024-09-14T23:58:45.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:45.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352399+0000", 2024-09-14T23:58:45.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049682995000000001 2024-09-14T23:58:45.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:45.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352446+0000", 2024-09-14T23:58:45.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6341000000000002e-05 2024-09-14T23:58:45.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:45.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352501+0000", 2024-09-14T23:58:45.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5182000000000001e-05 2024-09-14T23:58:45.650 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.650 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:45.650 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.651 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69273 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6062080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:45.651 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302829+0000", 2024-09-14T23:58:45.651 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 90.308947551000003, 2024-09-14T23:58:45.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050384311000000001, 2024-09-14T23:58:45.651 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:45.651 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:45.651 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:45.651 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:45.651 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:45.651 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:45.651 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69273 2024-09-14T23:58:45.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.652 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:45.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.652 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:45.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302829+0000", 2024-09-14T23:58:45.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.652 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:45.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302829+0000", 2024-09-14T23:58:45.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.652 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:45.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302829+0000", 2024-09-14T23:58:45.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7700000000000005e-07 2024-09-14T23:58:45.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:45.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302832+0000", 2024-09-14T23:58:45.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6970000000000002e-06 2024-09-14T23:58:45.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:45.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302840+0000", 2024-09-14T23:58:45.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5719999999999997e-06 2024-09-14T23:58:45.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:45.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302845+0000", 2024-09-14T23:58:45.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7939999999999999e-06 2024-09-14T23:58:45.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:45.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353111+0000", 2024-09-14T23:58:45.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050265588 2024-09-14T23:58:45.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:45.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353160+0000", 2024-09-14T23:58:45.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8859000000000001e-05 2024-09-14T23:58:45.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.655 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:45.655 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353213+0000", 2024-09-14T23:58:45.655 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3324e-05 2024-09-14T23:58:45.655 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.655 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:45.655 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.655 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69274 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6176768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:45.655 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302854+0000", 2024-09-14T23:58:45.655 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 90.308922456999994, 2024-09-14T23:58:45.655 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050579526, 2024-09-14T23:58:45.655 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:45.655 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:45.655 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:45.656 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:45.656 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:45.656 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:45.656 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69274 2024-09-14T23:58:45.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.656 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:45.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:45.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302854+0000", 2024-09-14T23:58:45.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:45.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302854+0000", 2024-09-14T23:58:45.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:45.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302855+0000", 2024-09-14T23:58:45.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0200000000000001e-07 2024-09-14T23:58:45.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:45.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302857+0000", 2024-09-14T23:58:45.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.447e-06 2024-09-14T23:58:45.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:45.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302864+0000", 2024-09-14T23:58:45.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7730000000000002e-06 2024-09-14T23:58:45.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:45.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302869+0000", 2024-09-14T23:58:45.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3199999999999999e-06 2024-09-14T23:58:45.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:45.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353249+0000", 2024-09-14T23:58:45.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050379647999999999 2024-09-14T23:58:45.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.659 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:45.659 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353336+0000", 2024-09-14T23:58:45.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6767999999999993e-05 2024-09-14T23:58:45.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.659 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:45.659 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353433+0000", 2024-09-14T23:58:45.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7868000000000006e-05 2024-09-14T23:58:45.659 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.659 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:45.659 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.659 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69275 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6291456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:45.660 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302889+0000", 2024-09-14T23:58:45.660 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 90.308887838999993, 2024-09-14T23:58:45.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050663118, 2024-09-14T23:58:45.660 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:45.660 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:45.660 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:45.660 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:45.660 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:45.660 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:45.660 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69275 2024-09-14T23:58:45.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.660 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:45.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:45.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302889+0000", 2024-09-14T23:58:45.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:45.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302889+0000", 2024-09-14T23:58:45.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:45.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302889+0000", 2024-09-14T23:58:45.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6399999999999998e-07 2024-09-14T23:58:45.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:45.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302891+0000", 2024-09-14T23:58:45.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.339e-06 2024-09-14T23:58:45.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:45.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302898+0000", 2024-09-14T23:58:45.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0380000000000002e-06 2024-09-14T23:58:45.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:45.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302904+0000", 2024-09-14T23:58:45.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2159999999999997e-06 2024-09-14T23:58:45.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.663 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:45.663 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353449+0000", 2024-09-14T23:58:45.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050545386999999997 2024-09-14T23:58:45.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.663 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:45.663 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353494+0000", 2024-09-14T23:58:45.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4592999999999998e-05 2024-09-14T23:58:45.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.663 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:45.663 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353552+0000", 2024-09-14T23:58:45.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8180999999999997e-05 2024-09-14T23:58:45.663 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.664 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:45.664 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.664 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.664 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69276 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6406144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:45.664 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302911+0000", 2024-09-14T23:58:45.664 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 90.308865425999997, 2024-09-14T23:58:45.664 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050865753, 2024-09-14T23:58:45.664 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:45.664 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:45.664 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:45.664 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:45.665 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:45.665 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:45.665 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69276 2024-09-14T23:58:45.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.665 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:45.665 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.665 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:45.665 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302911+0000", 2024-09-14T23:58:45.665 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.665 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.665 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:45.665 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302911+0000", 2024-09-14T23:58:45.665 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:45.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302912+0000", 2024-09-14T23:58:45.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0800000000000004e-07 2024-09-14T23:58:45.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:45.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302914+0000", 2024-09-14T23:58:45.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9589999999999998e-06 2024-09-14T23:58:45.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:45.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302920+0000", 2024-09-14T23:58:45.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4579999999999998e-06 2024-09-14T23:58:45.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:45.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302925+0000", 2024-09-14T23:58:45.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9429999999999999e-06 2024-09-14T23:58:45.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:45.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353586+0000", 2024-09-14T23:58:45.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050660966000000002 2024-09-14T23:58:45.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:45.668 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353679+0000", 2024-09-14T23:58:45.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3163000000000002e-05 2024-09-14T23:58:45.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.668 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.668 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:45.668 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353777+0000", 2024-09-14T23:58:45.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7555999999999999e-05 2024-09-14T23:58:45.668 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.668 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:45.668 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.668 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.668 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69277 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6520832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:45.669 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302932+0000", 2024-09-14T23:58:45.669 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 90.308844339000004, 2024-09-14T23:58:45.669 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050977674000000001, 2024-09-14T23:58:45.669 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:45.669 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:45.669 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:45.669 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:45.669 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:45.669 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:45.669 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69277 2024-09-14T23:58:45.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.669 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:45.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:45.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302932+0000", 2024-09-14T23:58:45.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:45.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302932+0000", 2024-09-14T23:58:45.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:45.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302932+0000", 2024-09-14T23:58:45.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3299999999999998e-07 2024-09-14T23:58:45.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.671 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:45.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302935+0000", 2024-09-14T23:58:45.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2139999999999999e-06 2024-09-14T23:58:45.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.671 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.671 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:45.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302941+0000", 2024-09-14T23:58:45.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5039999999999999e-06 2024-09-14T23:58:45.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.671 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.671 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:45.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302946+0000", 2024-09-14T23:58:45.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2079999999999999e-06 2024-09-14T23:58:45.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.672 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.672 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:45.672 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353805+0000", 2024-09-14T23:58:45.672 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050858588000000003 2024-09-14T23:58:45.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.672 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.672 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:45.672 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353849+0000", 2024-09-14T23:58:45.672 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4511000000000001e-05 2024-09-14T23:58:45.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.672 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.672 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:45.673 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353910+0000", 2024-09-14T23:58:45.673 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0316000000000003e-05 2024-09-14T23:58:45.673 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.673 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:45.673 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.673 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.673 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69278 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6635520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:45.673 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302953+0000", 2024-09-14T23:58:45.673 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 90.308823211000004, 2024-09-14T23:58:45.673 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051174059000000001, 2024-09-14T23:58:45.673 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:45.673 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:45.674 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:45.674 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:45.674 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:45.674 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:45.674 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69278 2024-09-14T23:58:45.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.674 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:45.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.674 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:45.674 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302953+0000", 2024-09-14T23:58:45.674 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.674 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:45.674 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302953+0000", 2024-09-14T23:58:45.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.675 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:45.675 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302954+0000", 2024-09-14T23:58:45.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5100000000000001e-07 2024-09-14T23:58:45.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.675 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:45.675 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302964+0000", 2024-09-14T23:58:45.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0475000000000001e-05 2024-09-14T23:58:45.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.675 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:45.676 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302970+0000", 2024-09-14T23:58:45.676 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3879999999999997e-06 2024-09-14T23:58:45.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.676 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:45.676 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302975+0000", 2024-09-14T23:58:45.676 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8409999999999999e-06 2024-09-14T23:58:45.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.676 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:45.676 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353944+0000", 2024-09-14T23:58:45.676 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050968952999999997 2024-09-14T23:58:45.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.677 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:45.677 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354031+0000", 2024-09-14T23:58:45.677 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6537000000000002e-05 2024-09-14T23:58:45.677 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.677 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.677 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:45.677 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354127+0000", 2024-09-14T23:58:45.677 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6514000000000005e-05 2024-09-14T23:58:45.677 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.677 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:45.677 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.677 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.677 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.677 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69279 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6750208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:45.678 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302984+0000", 2024-09-14T23:58:45.678 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 90.308792859999997, 2024-09-14T23:58:45.678 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051258845999999997, 2024-09-14T23:58:45.678 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:45.678 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:45.678 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:45.678 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:45.678 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:45.678 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:45.678 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69279 2024-09-14T23:58:45.678 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.678 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:45.678 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.678 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:45.678 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302984+0000", 2024-09-14T23:58:45.679 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.679 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:45.679 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302984+0000", 2024-09-14T23:58:45.679 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.679 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:45.679 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303025+0000", 2024-09-14T23:58:45.679 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1690999999999998e-05 2024-09-14T23:58:45.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.679 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:45.679 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303029+0000", 2024-09-14T23:58:45.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6559999999999998e-06 2024-09-14T23:58:45.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.680 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:45.680 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303037+0000", 2024-09-14T23:58:45.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7950000000000008e-06 2024-09-14T23:58:45.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.680 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:45.680 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303043+0000", 2024-09-14T23:58:45.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9669999999999998e-06 2024-09-14T23:58:45.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.680 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:45.681 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354142+0000", 2024-09-14T23:58:45.681 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051099072000000002 2024-09-14T23:58:45.681 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.681 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.681 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:45.681 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354183+0000", 2024-09-14T23:58:45.681 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0834000000000002e-05 2024-09-14T23:58:45.681 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.681 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.681 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:45.681 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354242+0000", 2024-09-14T23:58:45.681 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9830999999999997e-05 2024-09-14T23:58:45.681 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.681 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:45.682 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.682 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.682 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.682 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69280 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6864896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:45.682 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.303052+0000", 2024-09-14T23:58:45.682 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 90.308724316999999, 2024-09-14T23:58:45.682 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051406463999999999, 2024-09-14T23:58:45.682 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:45.682 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:45.682 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:45.682 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:45.682 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:45.682 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:45.682 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69280 2024-09-14T23:58:45.682 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.683 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:45.683 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.683 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:45.683 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303052+0000", 2024-09-14T23:58:45.683 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.683 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.683 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.683 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:45.683 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303052+0000", 2024-09-14T23:58:45.683 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.683 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.683 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.683 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:45.683 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304673+0000", 2024-09-14T23:58:45.683 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001621389 2024-09-14T23:58:45.684 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.684 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.684 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:45.684 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304681+0000", 2024-09-14T23:58:45.684 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8439999999999998e-06 2024-09-14T23:58:45.684 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.684 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.684 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:45.684 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304691+0000", 2024-09-14T23:58:45.684 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8339999999999994e-06 2024-09-14T23:58:45.684 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.684 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.684 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:45.685 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304698+0000", 2024-09-14T23:58:45.685 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3629999999999999e-06 2024-09-14T23:58:45.685 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.685 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.685 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:45.685 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354278+0000", 2024-09-14T23:58:45.685 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049580618 2024-09-14T23:58:45.685 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.685 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.685 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:45.685 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354364+0000", 2024-09-14T23:58:45.685 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6059999999999999e-05 2024-09-14T23:58:45.685 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.685 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.686 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:45.686 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354459+0000", 2024-09-14T23:58:45.686 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4356000000000003e-05 2024-09-14T23:58:45.686 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.686 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:45.686 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.686 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.686 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.686 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69285 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 7438336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:45.686 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.304792+0000", 2024-09-14T23:58:45.686 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 90.306984908000004, 2024-09-14T23:58:45.686 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050398874000000003, 2024-09-14T23:58:45.686 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:45.686 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:45.687 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:45.687 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:45.687 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:45.687 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:45.687 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69285 2024-09-14T23:58:45.687 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.687 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:45.687 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.687 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:45.687 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304792+0000", 2024-09-14T23:58:45.687 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.687 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.687 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.687 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:45.688 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304792+0000", 2024-09-14T23:58:45.688 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.688 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.688 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.688 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:45.688 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304792+0000", 2024-09-14T23:58:45.688 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2800000000000003e-07 2024-09-14T23:58:45.688 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.688 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.688 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:45.688 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304794+0000", 2024-09-14T23:58:45.688 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8700000000000001e-06 2024-09-14T23:58:45.688 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.688 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.689 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:45.689 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304800+0000", 2024-09-14T23:58:45.689 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0889999999999996e-06 2024-09-14T23:58:45.689 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.689 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.689 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:45.689 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304805+0000", 2024-09-14T23:58:45.689 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7060000000000003e-06 2024-09-14T23:58:45.689 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.689 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.689 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:45.689 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355108+0000", 2024-09-14T23:58:45.689 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050303170000000001 2024-09-14T23:58:45.689 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.690 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.690 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:45.690 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355150+0000", 2024-09-14T23:58:45.690 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2271999999999998e-05 2024-09-14T23:58:45.690 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.690 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.690 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:45.690 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355190+0000", 2024-09-14T23:58:45.690 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0438999999999997e-05 2024-09-14T23:58:45.690 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.690 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:45.690 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.690 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.690 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.691 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69286 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 7553024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:45.691 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.304811+0000", 2024-09-14T23:58:45.691 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 90.306965546000001, 2024-09-14T23:58:45.691 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050585512999999999, 2024-09-14T23:58:45.691 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:45.691 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:45.691 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:45.691 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:45.691 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:45.691 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:45.691 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69286 2024-09-14T23:58:45.691 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.691 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:45.691 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.691 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:45.692 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304811+0000", 2024-09-14T23:58:45.692 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.692 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.692 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.692 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:45.692 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304811+0000", 2024-09-14T23:58:45.692 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.692 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.692 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.692 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:45.692 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304811+0000", 2024-09-14T23:58:45.692 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.89e-07 2024-09-14T23:58:45.692 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.692 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.693 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:45.693 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304814+0000", 2024-09-14T23:58:45.693 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3199999999999998e-06 2024-09-14T23:58:45.693 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.693 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.693 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:45.693 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304821+0000", 2024-09-14T23:58:45.693 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5689999999999999e-06 2024-09-14T23:58:45.693 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.693 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.693 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:45.693 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304826+0000", 2024-09-14T23:58:45.693 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1200000000000001e-06 2024-09-14T23:58:45.693 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.693 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.694 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:45.694 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355226+0000", 2024-09-14T23:58:45.694 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050399653000000003 2024-09-14T23:58:45.694 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.694 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.694 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:45.694 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355314+0000", 2024-09-14T23:58:45.694 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8389999999999999e-05 2024-09-14T23:58:45.694 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.694 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.694 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:45.694 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355396+0000", 2024-09-14T23:58:45.694 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2071999999999996e-05 2024-09-14T23:58:45.694 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.694 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:45.695 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.695 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.695 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.695 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69287 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 7667712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:45.695 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.304833+0000", 2024-09-14T23:58:45.695 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 90.306942948, 2024-09-14T23:58:45.695 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050664740999999999, 2024-09-14T23:58:45.695 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:45.695 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:45.695 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:45.695 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:45.695 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:45.695 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:45.695 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69287 2024-09-14T23:58:45.695 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.696 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:45.696 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.696 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:45.696 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304833+0000", 2024-09-14T23:58:45.696 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.696 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.696 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:45.696 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304833+0000", 2024-09-14T23:58:45.696 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.696 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.697 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:45.697 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304834+0000", 2024-09-14T23:58:45.697 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4999999999999998e-07 2024-09-14T23:58:45.697 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.697 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.697 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:45.697 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304836+0000", 2024-09-14T23:58:45.697 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0279999999999999e-06 2024-09-14T23:58:45.697 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.697 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.697 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:45.697 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304842+0000", 2024-09-14T23:58:45.697 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8919999999999997e-06 2024-09-14T23:58:45.697 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.697 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.698 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:45.698 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304847+0000", 2024-09-14T23:58:45.698 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6160000000000001e-06 2024-09-14T23:58:45.698 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.698 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.698 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:45.698 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355411+0000", 2024-09-14T23:58:45.698 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050563540999999997 2024-09-14T23:58:45.698 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.698 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.698 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:45.698 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355459+0000", 2024-09-14T23:58:45.698 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7979000000000003e-05 2024-09-14T23:58:45.698 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.699 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.699 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:45.699 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355498+0000", 2024-09-14T23:58:45.699 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9335000000000002e-05 2024-09-14T23:58:45.699 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.699 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:45.699 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.699 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.699 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.699 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69288 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 7782400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:45.699 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.304854+0000", 2024-09-14T23:58:45.699 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 90.306922510000007, 2024-09-14T23:58:45.699 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050855145999999997, 2024-09-14T23:58:45.699 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:45.699 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:45.700 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:45.700 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:45.700 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:45.700 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:45.700 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69288 2024-09-14T23:58:45.700 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.700 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:45.700 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.700 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:45.700 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304854+0000", 2024-09-14T23:58:45.700 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.700 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.700 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.700 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:45.701 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304854+0000", 2024-09-14T23:58:45.701 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:45.701 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.701 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.701 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:45.701 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.306670+0000", 2024-09-14T23:58:45.701 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00181631 2024-09-14T23:58:45.701 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.701 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.701 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:45.701 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.306675+0000", 2024-09-14T23:58:45.701 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0220000000000004e-06 2024-09-14T23:58:45.701 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.701 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.702 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:45.702 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.306684+0000", 2024-09-14T23:58:45.702 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7309999999999999e-06 2024-09-14T23:58:45.702 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.702 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.702 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:45.702 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.306689+0000", 2024-09-14T23:58:45.702 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6609999999999999e-06 2024-09-14T23:58:45.702 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.702 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.702 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:45.702 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355533+0000", 2024-09-14T23:58:45.702 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048844578 2024-09-14T23:58:45.702 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.703 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.703 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:45.703 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355622+0000", 2024-09-14T23:58:45.703 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9006999999999997e-05 2024-09-14T23:58:45.703 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:45.703 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:45.703 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:45.703 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355709+0000", 2024-09-14T23:58:45.703 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6836999999999995e-05 2024-09-14T23:58:45.703 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.703 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:45.703 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.703 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:45.703 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:45.703 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:45.705 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T23:58:45.706 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:45.703+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:45.808 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:45.804+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:45.817 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:45.817 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:45.817 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:58:45.817 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:45.825 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T23:58:45.909 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:45.908+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:45.973 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:45.973 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:45.973 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:58:45.973 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:58:45.973 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:45.981 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T23:58:46.022 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:46.021+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:46.035 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.6 to 1 (10000) 2024-09-14T23:58:46.048 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T23:58:46.057 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-14T23:58:46.108 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:46.107+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:46.152 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:46.152 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:58:46.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:58:46.153 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:58:46.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.153 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:43054 3.1f 3:ffbfe596:::rbd_data.1118350d5ae5.00000000000002eb:head [set-alloc-hint object_size 262144 write_size 262144,write 196608~22770] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T23:58:46.153 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:58:46.153 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 113.182991489, 2024-09-14T23:58:46.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.99973334400000002, 2024-09-14T23:58:46.153 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:46.153 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:46.153 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:46.153 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:46.153 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:46.153 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:46.154 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 43054 2024-09-14T23:58:46.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.154 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:46.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:46.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:58:46.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:46.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:46.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:58:46.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:46.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:46.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966552+0000", 2024-09-14T23:58:46.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3790000000000001e-06 2024-09-14T23:58:46.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:46.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966565+0000", 2024-09-14T23:58:46.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2459999999999999e-05 2024-09-14T23:58:46.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:46.156 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966576+0000", 2024-09-14T23:58:46.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1039000000000001e-05 2024-09-14T23:58:46.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.156 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:46.156 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966582+0000", 2024-09-14T23:58:46.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9660000000000001e-06 2024-09-14T23:58:46.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.156 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:46.156 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966625+0000", 2024-09-14T23:58:46.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3529999999999998e-05 2024-09-14T23:58:46.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.157 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:58:46.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966632+0000", 2024-09-14T23:58:46.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9809999999999997e-06 2024-09-14T23:58:46.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.157 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:46.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.964460+0000", 2024-09-14T23:58:46.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.99782768300000002 2024-09-14T23:58:46.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.157 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:46.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.964641+0000", 2024-09-14T23:58:46.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000181089 2024-09-14T23:58:46.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-14T23:58:46.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.964809+0000", 2024-09-14T23:58:46.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000168505 2024-09-14T23:58:46.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:58:46.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966030+0000", 2024-09-14T23:58:46.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012206929999999999 2024-09-14T23:58:46.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:46.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966181+0000", 2024-09-14T23:58:46.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000150569 2024-09-14T23:58:46.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:46.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966243+0000", 2024-09-14T23:58:46.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2331999999999999e-05 2024-09-14T23:58:46.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:46.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966284+0000", 2024-09-14T23:58:46.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1118000000000002e-05 2024-09-14T23:58:46.159 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:46.159 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:46.160 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:46.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.160 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184314 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4162560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:46.160 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.077247+0000", 2024-09-14T23:58:46.160 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 39.072294821, 2024-09-14T23:58:46.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058772256000000002, 2024-09-14T23:58:46.160 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:46.160 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:46.160 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:46.160 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:46.160 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:46.160 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:46.161 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184314 2024-09-14T23:58:46.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.161 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:46.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.161 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:46.161 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.077247+0000", 2024-09-14T23:58:46.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:46.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.161 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:46.161 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.077247+0000", 2024-09-14T23:58:46.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:46.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.161 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:46.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.078906+0000", 2024-09-14T23:58:46.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016583279999999999 2024-09-14T23:58:46.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:46.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.078911+0000", 2024-09-14T23:58:46.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0420000000000002e-06 2024-09-14T23:58:46.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:46.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.078922+0000", 2024-09-14T23:58:46.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1354e-05 2024-09-14T23:58:46.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:46.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.078930+0000", 2024-09-14T23:58:46.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5730000000000002e-06 2024-09-14T23:58:46.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:46.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.135870+0000", 2024-09-14T23:58:46.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056940651000000002 2024-09-14T23:58:46.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:46.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.135951+0000", 2024-09-14T23:58:46.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0890999999999995e-05 2024-09-14T23:58:46.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.164 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:46.164 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.136020+0000", 2024-09-14T23:58:46.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8417000000000006e-05 2024-09-14T23:58:46.164 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:46.164 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:46.164 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:46.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.164 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184323 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4183040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:46.164 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.079228+0000", 2024-09-14T23:58:46.164 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 39.070313747, 2024-09-14T23:58:46.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058454487999999999, 2024-09-14T23:58:46.164 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:46.165 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:46.165 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:46.165 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:46.165 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:46.165 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:46.165 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184323 2024-09-14T23:58:46.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.165 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:46.165 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.165 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:46.165 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079228+0000", 2024-09-14T23:58:46.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:46.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.165 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.165 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:46.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079228+0000", 2024-09-14T23:58:46.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:46.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:46.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079229+0000", 2024-09-14T23:58:46.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5299999999999999e-07 2024-09-14T23:58:46.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:46.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079232+0000", 2024-09-14T23:58:46.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8839999999999998e-06 2024-09-14T23:58:46.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:46.167 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079238+0000", 2024-09-14T23:58:46.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2299999999999996e-06 2024-09-14T23:58:46.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:46.167 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079243+0000", 2024-09-14T23:58:46.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3029999999999999e-06 2024-09-14T23:58:46.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.167 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:46.167 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.137533+0000", 2024-09-14T23:58:46.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058289674999999999 2024-09-14T23:58:46.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.167 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.168 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:46.168 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.137615+0000", 2024-09-14T23:58:46.168 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2288999999999998e-05 2024-09-14T23:58:46.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.168 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:46.168 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.137683+0000", 2024-09-14T23:58:46.168 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7654e-05 2024-09-14T23:58:46.168 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:46.168 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:46.168 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:46.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.168 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184324 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4073472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:46.169 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.079251+0000", 2024-09-14T23:58:46.169 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 39.070291279999999, 2024-09-14T23:58:46.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058609383000000001, 2024-09-14T23:58:46.169 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:46.169 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:46.169 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:46.169 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:46.169 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:46.169 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:46.169 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184324 2024-09-14T23:58:46.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.169 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:46.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.169 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:46.170 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079251+0000", 2024-09-14T23:58:46.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:46.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.170 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.170 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:46.170 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079251+0000", 2024-09-14T23:58:46.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:46.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.170 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.170 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:46.170 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.080920+0000", 2024-09-14T23:58:46.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016687430000000001 2024-09-14T23:58:46.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.170 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:46.171 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.080924+0000", 2024-09-14T23:58:46.171 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3869999999999997e-06 2024-09-14T23:58:46.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:46.171 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.080935+0000", 2024-09-14T23:58:46.171 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0515e-05 2024-09-14T23:58:46.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:46.171 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.080942+0000", 2024-09-14T23:58:46.171 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0990000000000001e-06 2024-09-14T23:58:46.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.172 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:46.172 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.137713+0000", 2024-09-14T23:58:46.172 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056771288000000003 2024-09-14T23:58:46.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.172 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:46.172 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.137794+0000", 2024-09-14T23:58:46.172 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0613999999999998e-05 2024-09-14T23:58:46.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.172 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:46.172 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.137860+0000", 2024-09-14T23:58:46.172 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6736999999999995e-05 2024-09-14T23:58:46.173 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:46.173 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:46.173 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:46.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.173 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.173 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184346 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4099072~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:46.173 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.083616+0000", 2024-09-14T23:58:46.173 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 39.065926095000002, 2024-09-14T23:58:46.173 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058557419999999999, 2024-09-14T23:58:46.173 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:46.173 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:46.173 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:46.173 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:46.173 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:46.174 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:46.174 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184346 2024-09-14T23:58:46.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.174 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:46.174 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.174 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:46.174 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083616+0000", 2024-09-14T23:58:46.174 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:46.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.174 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.174 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:46.174 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083616+0000", 2024-09-14T23:58:46.174 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:46.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.175 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:46.175 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083620+0000", 2024-09-14T23:58:46.175 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0119999999999999e-06 2024-09-14T23:58:46.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.175 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:46.175 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083623+0000", 2024-09-14T23:58:46.175 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9059999999999998e-06 2024-09-14T23:58:46.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.175 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:46.175 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083631+0000", 2024-09-14T23:58:46.175 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2679999999999994e-06 2024-09-14T23:58:46.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.176 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:46.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083637+0000", 2024-09-14T23:58:46.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1210000000000003e-06 2024-09-14T23:58:46.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.176 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:46.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142018+0000", 2024-09-14T23:58:46.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058381111999999999 2024-09-14T23:58:46.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.176 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:46.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142100+0000", 2024-09-14T23:58:46.177 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1308000000000001e-05 2024-09-14T23:58:46.177 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.177 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.177 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:46.177 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142173+0000", 2024-09-14T23:58:46.177 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3693000000000001e-05 2024-09-14T23:58:46.177 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:46.177 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:46.177 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:46.177 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.177 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.177 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184347 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4122624~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:46.177 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.083647+0000", 2024-09-14T23:58:46.177 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 39.065895265000002, 2024-09-14T23:58:46.177 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058706388999999998, 2024-09-14T23:58:46.178 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:46.178 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:46.178 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:46.178 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:46.178 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:46.178 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:46.178 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184347 2024-09-14T23:58:46.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.178 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:46.178 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.178 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:46.178 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083647+0000", 2024-09-14T23:58:46.178 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:46.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.179 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.179 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:46.179 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083647+0000", 2024-09-14T23:58:46.179 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:46.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.179 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.179 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:46.179 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083647+0000", 2024-09-14T23:58:46.179 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3099999999999998e-07 2024-09-14T23:58:46.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.179 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.179 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:46.179 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083650+0000", 2024-09-14T23:58:46.179 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2900000000000001e-06 2024-09-14T23:58:46.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:46.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083657+0000", 2024-09-14T23:58:46.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.486e-06 2024-09-14T23:58:46.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:46.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083663+0000", 2024-09-14T23:58:46.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4330000000000003e-06 2024-09-14T23:58:46.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:46.181 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142206+0000", 2024-09-14T23:58:46.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058543237999999997 2024-09-14T23:58:46.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.181 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.181 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:46.181 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142284+0000", 2024-09-14T23:58:46.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8487999999999998e-05 2024-09-14T23:58:46.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.181 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.181 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:46.181 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142353+0000", 2024-09-14T23:58:46.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9023000000000006e-05 2024-09-14T23:58:46.181 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:46.181 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:46.182 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:46.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.182 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184348 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4103168~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:46.182 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.083669+0000", 2024-09-14T23:58:46.182 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 39.065872847999998, 2024-09-14T23:58:46.182 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058864232000000002, 2024-09-14T23:58:46.182 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:46.182 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:46.182 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:46.182 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:46.182 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:46.182 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:46.182 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184348 2024-09-14T23:58:46.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.183 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:46.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:46.183 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083669+0000", 2024-09-14T23:58:46.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:46.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:46.183 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083669+0000", 2024-09-14T23:58:46.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:46.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:46.183 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083670+0000", 2024-09-14T23:58:46.184 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9999999999999998e-07 2024-09-14T23:58:46.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.184 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.184 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:46.184 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083672+0000", 2024-09-14T23:58:46.184 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1430000000000001e-06 2024-09-14T23:58:46.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.184 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.184 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:46.184 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083678+0000", 2024-09-14T23:58:46.184 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5200000000000003e-06 2024-09-14T23:58:46.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.184 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.184 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:46.185 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083683+0000", 2024-09-14T23:58:46.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9359999999999998e-06 2024-09-14T23:58:46.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.185 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:46.185 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142384+0000", 2024-09-14T23:58:46.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058701159000000003 2024-09-14T23:58:46.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.185 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:46.185 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142466+0000", 2024-09-14T23:58:46.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1024000000000002e-05 2024-09-14T23:58:46.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.186 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:46.186 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142534+0000", 2024-09-14T23:58:46.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8050000000000001e-05 2024-09-14T23:58:46.186 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:46.186 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:46.186 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:46.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.186 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184349 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4126720~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:46.186 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.083690+0000", 2024-09-14T23:58:46.186 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 39.065852313000001, 2024-09-14T23:58:46.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059019911000000001, 2024-09-14T23:58:46.186 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:46.186 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:46.187 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:46.187 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:46.187 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:46.187 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:46.187 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184349 2024-09-14T23:58:46.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.187 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:46.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:46.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083690+0000", 2024-09-14T23:58:46.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:46.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:46.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083690+0000", 2024-09-14T23:58:46.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:46.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.188 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:46.188 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.084900+0000", 2024-09-14T23:58:46.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012099770000000001 2024-09-14T23:58:46.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.188 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:46.188 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.084905+0000", 2024-09-14T23:58:46.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.028e-06 2024-09-14T23:58:46.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:46.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.084914+0000", 2024-09-14T23:58:46.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6619999999999994e-06 2024-09-14T23:58:46.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:46.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.084922+0000", 2024-09-14T23:58:46.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5720000000000003e-06 2024-09-14T23:58:46.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:46.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142565+0000", 2024-09-14T23:58:46.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057642605999999999 2024-09-14T23:58:46.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.190 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:46.190 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142644+0000", 2024-09-14T23:58:46.190 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9370999999999996e-05 2024-09-14T23:58:46.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.190 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.190 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:46.190 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142710+0000", 2024-09-14T23:58:46.190 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5695e-05 2024-09-14T23:58:46.190 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:46.190 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:46.190 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:46.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.190 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.190 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184354 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4115456~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:46.191 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.085029+0000", 2024-09-14T23:58:46.191 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 39.064513663, 2024-09-14T23:58:46.191 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058626327999999998, 2024-09-14T23:58:46.191 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:46.191 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:46.191 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:46.191 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:46.191 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:46.191 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:46.191 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184354 2024-09-14T23:58:46.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.191 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:46.191 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.191 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:46.192 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085029+0000", 2024-09-14T23:58:46.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:46.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.192 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:46.192 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085029+0000", 2024-09-14T23:58:46.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:46.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.192 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:46.192 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085029+0000", 2024-09-14T23:58:46.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0800000000000001e-07 2024-09-14T23:58:46.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.193 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:46.193 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085030+0000", 2024-09-14T23:58:46.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.226e-06 2024-09-14T23:58:46.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.193 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.193 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:46.193 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085033+0000", 2024-09-14T23:58:46.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9569999999999998e-06 2024-09-14T23:58:46.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.193 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.193 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:46.193 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085035+0000", 2024-09-14T23:58:46.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.587e-06 2024-09-14T23:58:46.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.193 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.194 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:46.194 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143491+0000", 2024-09-14T23:58:46.194 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058455337000000003 2024-09-14T23:58:46.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.194 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:46.194 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143575+0000", 2024-09-14T23:58:46.194 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3850000000000005e-05 2024-09-14T23:58:46.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.194 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:46.194 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143655+0000", 2024-09-14T23:58:46.194 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0162999999999998e-05 2024-09-14T23:58:46.194 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:46.195 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:46.195 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:46.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.195 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.195 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184355 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4139008~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:46.195 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.085039+0000", 2024-09-14T23:58:46.195 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 39.064503258999999, 2024-09-14T23:58:46.195 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058801369999999999, 2024-09-14T23:58:46.195 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:46.195 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:46.195 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:46.195 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:46.195 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:46.196 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:46.196 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184355 2024-09-14T23:58:46.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.196 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:46.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.196 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:46.196 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085039+0000", 2024-09-14T23:58:46.196 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:46.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.196 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:46.196 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085039+0000", 2024-09-14T23:58:46.196 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:46.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.197 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:46.197 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085039+0000", 2024-09-14T23:58:46.197 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.03e-07 2024-09-14T23:58:46.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.197 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:46.197 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085040+0000", 2024-09-14T23:58:46.197 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.158e-06 2024-09-14T23:58:46.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.198 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:46.198 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085043+0000", 2024-09-14T23:58:46.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0240000000000002e-06 2024-09-14T23:58:46.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.198 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:46.198 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085046+0000", 2024-09-14T23:58:46.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4600000000000002e-06 2024-09-14T23:58:46.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.198 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:46.198 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143688+0000", 2024-09-14T23:58:46.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058642356 2024-09-14T23:58:46.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.199 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:46.199 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143766+0000", 2024-09-14T23:58:46.199 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8219000000000004e-05 2024-09-14T23:58:46.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.199 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:46.199 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143840+0000", 2024-09-14T23:58:46.199 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3949999999999995e-05 2024-09-14T23:58:46.199 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:46.199 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:46.199 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:46.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.200 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184356 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4119552~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:46.200 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.085049+0000", 2024-09-14T23:58:46.200 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 39.064493087999999, 2024-09-14T23:58:46.200 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058977992, 2024-09-14T23:58:46.200 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:46.200 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:46.200 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:46.200 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:46.200 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:46.200 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:46.200 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184356 2024-09-14T23:58:46.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.201 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:46.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.201 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:46.201 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085049+0000", 2024-09-14T23:58:46.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:46.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.201 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:46.201 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085049+0000", 2024-09-14T23:58:46.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:46.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.201 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:46.202 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.086934+0000", 2024-09-14T23:58:46.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018854189999999999 2024-09-14T23:58:46.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.202 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:46.202 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.086939+0000", 2024-09-14T23:58:46.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.493e-06 2024-09-14T23:58:46.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.202 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:46.202 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.086951+0000", 2024-09-14T23:58:46.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1969e-05 2024-09-14T23:58:46.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.202 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:46.203 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.086959+0000", 2024-09-14T23:58:46.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5780000000000001e-06 2024-09-14T23:58:46.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.203 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:46.203 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143871+0000", 2024-09-14T23:58:46.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056912114 2024-09-14T23:58:46.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.203 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:46.203 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143951+0000", 2024-09-14T23:58:46.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9938000000000006e-05 2024-09-14T23:58:46.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.203 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:46.204 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.144027+0000", 2024-09-14T23:58:46.204 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6481000000000002e-05 2024-09-14T23:58:46.204 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:46.204 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:46.204 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:46.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.204 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184364 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4135936~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:46.204 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.087207+0000", 2024-09-14T23:58:46.204 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 39.062334808000003, 2024-09-14T23:58:46.204 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058466420999999998, 2024-09-14T23:58:46.204 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:46.204 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:46.204 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:46.205 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:46.205 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:46.205 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:46.205 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184364 2024-09-14T23:58:46.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.205 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:46.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:46.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087207+0000", 2024-09-14T23:58:46.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:46.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:46.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087207+0000", 2024-09-14T23:58:46.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:46.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:46.208 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087208+0000", 2024-09-14T23:58:46.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.35e-07 2024-09-14T23:58:46.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:46.208 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087209+0000", 2024-09-14T23:58:46.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.573e-06 2024-09-14T23:58:46.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.209 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:46.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087212+0000", 2024-09-14T23:58:46.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2770000000000001e-06 2024-09-14T23:58:46.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.209 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:46.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087215+0000", 2024-09-14T23:58:46.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5500000000000001e-06 2024-09-14T23:58:46.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.209 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:46.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145520+0000", 2024-09-14T23:58:46.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058304835999999999 2024-09-14T23:58:46.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:46.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145602+0000", 2024-09-14T23:58:46.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1714999999999999e-05 2024-09-14T23:58:46.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:46.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145674+0000", 2024-09-14T23:58:46.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2235000000000002e-05 2024-09-14T23:58:46.210 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:46.211 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:46.211 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:46.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.211 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184365 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4159488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:46.211 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.087219+0000", 2024-09-14T23:58:46.211 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 39.062322776999999, 2024-09-14T23:58:46.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058643533999999997, 2024-09-14T23:58:46.211 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:46.211 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:46.211 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:46.211 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:46.211 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:46.211 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:46.211 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184365 2024-09-14T23:58:46.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.212 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:46.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.212 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:46.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087219+0000", 2024-09-14T23:58:46.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:46.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.212 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:46.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087219+0000", 2024-09-14T23:58:46.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:46.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:46.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087220+0000", 2024-09-14T23:58:46.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.39e-07 2024-09-14T23:58:46.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:46.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087222+0000", 2024-09-14T23:58:46.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0609999999999998e-06 2024-09-14T23:58:46.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:46.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087228+0000", 2024-09-14T23:58:46.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8590000000000001e-06 2024-09-14T23:58:46.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:46.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087232+0000", 2024-09-14T23:58:46.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8149999999999996e-06 2024-09-14T23:58:46.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:46.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145705+0000", 2024-09-14T23:58:46.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058472532000000001 2024-09-14T23:58:46.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:46.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145784+0000", 2024-09-14T23:58:46.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9382000000000007e-05 2024-09-14T23:58:46.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:46.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145863+0000", 2024-09-14T23:58:46.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8546000000000003e-05 2024-09-14T23:58:46.215 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:46.215 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:46.215 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:46.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.215 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184366 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4140032~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:46.215 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.087240+0000", 2024-09-14T23:58:46.215 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 39.062302328999998, 2024-09-14T23:58:46.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058818068000000001, 2024-09-14T23:58:46.216 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:46.216 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:46.216 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:46.216 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:46.216 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:46.216 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:46.216 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184366 2024-09-14T23:58:46.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.216 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:46.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:46.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087240+0000", 2024-09-14T23:58:46.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:46.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:46.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087240+0000", 2024-09-14T23:58:46.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:46.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:46.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.088904+0000", 2024-09-14T23:58:46.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001664315 2024-09-14T23:58:46.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:46.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.088909+0000", 2024-09-14T23:58:46.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7709999999999997e-06 2024-09-14T23:58:46.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.218 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:46.218 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.088921+0000", 2024-09-14T23:58:46.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1661999999999999e-05 2024-09-14T23:58:46.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.218 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:46.218 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.088928+0000", 2024-09-14T23:58:46.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8189999999999992e-06 2024-09-14T23:58:46.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.218 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:46.218 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145901+0000", 2024-09-14T23:58:46.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056972478 2024-09-14T23:58:46.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:46.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145977+0000", 2024-09-14T23:58:46.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5763e-05 2024-09-14T23:58:46.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:46.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.146058+0000", 2024-09-14T23:58:46.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1260000000000005e-05 2024-09-14T23:58:46.219 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:46.219 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:46.219 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:46.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.220 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184375 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4179968~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:46.220 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.089199+0000", 2024-09-14T23:58:46.220 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 39.060342992000002, 2024-09-14T23:58:46.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058597445999999997, 2024-09-14T23:58:46.220 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:46.220 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:46.220 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:46.220 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:46.220 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:46.220 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:46.220 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184375 2024-09-14T23:58:46.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.221 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:46.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:46.221 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089199+0000", 2024-09-14T23:58:46.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:46.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:46.221 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089199+0000", 2024-09-14T23:58:46.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:46.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:46.221 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089200+0000", 2024-09-14T23:58:46.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8100000000000003e-07 2024-09-14T23:58:46.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.222 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:46.222 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089202+0000", 2024-09-14T23:58:46.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7599999999999998e-06 2024-09-14T23:58:46.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.222 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:46.222 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089209+0000", 2024-09-14T23:58:46.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7279999999999998e-06 2024-09-14T23:58:46.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.222 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:46.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089215+0000", 2024-09-14T23:58:46.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5090000000000001e-06 2024-09-14T23:58:46.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:46.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147638+0000", 2024-09-14T23:58:46.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058423105000000003 2024-09-14T23:58:46.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:46.223 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147719+0000", 2024-09-14T23:58:46.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1217000000000005e-05 2024-09-14T23:58:46.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.223 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:46.224 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147797+0000", 2024-09-14T23:58:46.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7645999999999995e-05 2024-09-14T23:58:46.224 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:46.224 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:46.224 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:46.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.224 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184376 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4160512~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:46.224 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.089232+0000", 2024-09-14T23:58:46.224 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 39.060310502, 2024-09-14T23:58:46.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058751338, 2024-09-14T23:58:46.224 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:46.224 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:46.224 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:46.225 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:46.225 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:46.225 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:46.225 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184376 2024-09-14T23:58:46.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.225 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:46.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.225 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:46.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089232+0000", 2024-09-14T23:58:46.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:46.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.225 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:46.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089232+0000", 2024-09-14T23:58:46.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:46.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:46.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089232+0000", 2024-09-14T23:58:46.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5600000000000001e-07 2024-09-14T23:58:46.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:46.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089235+0000", 2024-09-14T23:58:46.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5639999999999999e-06 2024-09-14T23:58:46.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:46.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089242+0000", 2024-09-14T23:58:46.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0369999999999997e-06 2024-09-14T23:58:46.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:46.227 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089247+0000", 2024-09-14T23:58:46.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2079999999999999e-06 2024-09-14T23:58:46.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:46.227 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147831+0000", 2024-09-14T23:58:46.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058583853999999998 2024-09-14T23:58:46.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:46.228 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147915+0000", 2024-09-14T23:58:46.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4208000000000004e-05 2024-09-14T23:58:46.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.228 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:46.228 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147983+0000", 2024-09-14T23:58:46.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8010999999999996e-05 2024-09-14T23:58:46.228 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:46.228 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:46.228 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:46.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.228 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184377 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4184064~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:46.228 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.089255+0000", 2024-09-14T23:58:46.229 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 39.060286945999998, 2024-09-14T23:58:46.229 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058906100000000003, 2024-09-14T23:58:46.229 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:46.229 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:46.229 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:46.229 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:46.229 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:46.229 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:46.229 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184377 2024-09-14T23:58:46.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.229 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:46.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.229 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:46.229 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089255+0000", 2024-09-14T23:58:46.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:46.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:46.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089255+0000", 2024-09-14T23:58:46.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:46.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:46.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.090903+0000", 2024-09-14T23:58:46.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001647871 2024-09-14T23:58:46.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:46.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.090908+0000", 2024-09-14T23:58:46.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7199999999999997e-06 2024-09-14T23:58:46.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.231 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:46.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.090919+0000", 2024-09-14T23:58:46.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1472e-05 2024-09-14T23:58:46.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.231 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:46.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.090927+0000", 2024-09-14T23:58:46.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0390000000000005e-06 2024-09-14T23:58:46.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.232 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:46.232 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.148013+0000", 2024-09-14T23:58:46.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057085662000000002 2024-09-14T23:58:46.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.232 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:46.232 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.148090+0000", 2024-09-14T23:58:46.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6778000000000002e-05 2024-09-14T23:58:46.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.232 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:46.232 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.148161+0000", 2024-09-14T23:58:46.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1557999999999995e-05 2024-09-14T23:58:46.232 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:46.233 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:46.233 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:46.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.233 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184386 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4185088~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:46.233 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.091063+0000", 2024-09-14T23:58:46.233 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 39.058479282, 2024-09-14T23:58:46.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058603404999999997, 2024-09-14T23:58:46.233 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:46.233 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:46.233 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:46.233 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:46.233 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:46.233 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:46.233 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184386 2024-09-14T23:58:46.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.234 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:46.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.234 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:46.234 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091063+0000", 2024-09-14T23:58:46.234 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:46.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.234 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:46.234 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091063+0000", 2024-09-14T23:58:46.234 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:46.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.234 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:46.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091063+0000", 2024-09-14T23:58:46.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1899999999999999e-07 2024-09-14T23:58:46.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.235 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:46.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091064+0000", 2024-09-14T23:58:46.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2699999999999999e-06 2024-09-14T23:58:46.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.235 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:46.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091067+0000", 2024-09-14T23:58:46.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.971e-06 2024-09-14T23:58:46.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.236 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:46.236 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091070+0000", 2024-09-14T23:58:46.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.525e-06 2024-09-14T23:58:46.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.236 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:46.236 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.149510+0000", 2024-09-14T23:58:46.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058439758000000001 2024-09-14T23:58:46.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.236 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:46.236 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.149595+0000", 2024-09-14T23:58:46.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5146000000000001e-05 2024-09-14T23:58:46.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:46.237 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.149666+0000", 2024-09-14T23:58:46.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1515999999999998e-05 2024-09-14T23:58:46.237 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:46.237 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:46.237 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:46.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.237 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184388 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4189184~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:46.237 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.091073+0000", 2024-09-14T23:58:46.237 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 39.058469006999999, 2024-09-14T23:58:46.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058781609999999998, 2024-09-14T23:58:46.237 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:46.237 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:46.238 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:46.238 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:46.238 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:46.238 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:46.238 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184388 2024-09-14T23:58:46.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.238 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:46.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.238 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:46.238 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091073+0000", 2024-09-14T23:58:46.238 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:46.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.238 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:46.239 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091073+0000", 2024-09-14T23:58:46.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:46.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.239 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.239 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:46.239 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.092903+0000", 2024-09-14T23:58:46.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018296289999999999 2024-09-14T23:58:46.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.239 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.239 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:46.239 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.092910+0000", 2024-09-14T23:58:46.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7510000000000002e-06 2024-09-14T23:58:46.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.239 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.240 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:46.240 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.092918+0000", 2024-09-14T23:58:46.240 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7420000000000003e-06 2024-09-14T23:58:46.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.240 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:46.240 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.092925+0000", 2024-09-14T23:58:46.240 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0360000000000001e-06 2024-09-14T23:58:46.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.240 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:46.240 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.149700+0000", 2024-09-14T23:58:46.240 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056774340999999999 2024-09-14T23:58:46.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.241 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.241 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:46.241 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.149784+0000", 2024-09-14T23:58:46.241 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4393000000000003e-05 2024-09-14T23:58:46.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.241 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.241 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:46.241 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.149855+0000", 2024-09-14T23:58:46.241 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0717999999999996e-05 2024-09-14T23:58:46.241 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:46.241 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:46.241 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:46.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.241 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.242 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4172.0:8 3.12 3:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-14T23:58:46.242 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.976999+0000", 2024-09-14T23:58:46.242 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 38.172543597999997, 2024-09-14T23:58:46.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0046036439999999, 2024-09-14T23:58:46.242 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:46.242 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:46.242 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:46.242 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:46.242 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4172", 2024-09-14T23:58:46.242 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/2542269203", 2024-09-14T23:58:46.242 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 8 2024-09-14T23:58:46.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.242 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:46.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.242 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:46.243 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.976999+0000", 2024-09-14T23:58:46.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:46.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.243 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.243 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:46.243 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.976999+0000", 2024-09-14T23:58:46.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:46.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.243 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.243 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:46.243 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.977000+0000", 2024-09-14T23:58:46.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5200000000000001e-06 2024-09-14T23:58:46.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:46.244 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.977005+0000", 2024-09-14T23:58:46.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.04e-06 2024-09-14T23:58:46.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:46.244 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.977013+0000", 2024-09-14T23:58:46.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2889999999999998e-06 2024-09-14T23:58:46.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.244 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:46.244 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.977024+0000", 2024-09-14T23:58:46.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0307e-05 2024-09-14T23:58:46.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.245 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:46.245 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.977086+0000", 2024-09-14T23:58:46.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2248000000000004e-05 2024-09-14T23:58:46.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.245 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:58:46.245 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.977097+0000", 2024-09-14T23:58:46.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1123e-05 2024-09-14T23:58:46.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.245 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:46.245 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:08.981470+0000", 2024-09-14T23:58:46.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.004373116 2024-09-14T23:58:46.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:46.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:46.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:46.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:08.981602+0000", 2024-09-14T23:58:46.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000132001 2024-09-14T23:58:46.246 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:46.246 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:46.246 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:46.246 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:46.246 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:46.246 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:46.248 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T23:58:46.248 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:46.246+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:46.348 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:46.347+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:46.386 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T23:58:46.397 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:46.397 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:46.397 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:58:46.397 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:46.406 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T23:58:46.449 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:46.448+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:46.512 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:58:46.513 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:58:46.530 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:58:46.530 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:58:46.547 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:58:46.547 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:58:46.550 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:46.549+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:46.556 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:46.557 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:46.563 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:58:46.563 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:58:46.566 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T23:58:46.581 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:58:46.581 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:58:46.598 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:58:46.598 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:58:46.615 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:58:46.615 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:58:46.632 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:58:46.632 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:58:46.651 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:46.650+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:46.674 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:46.674 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:46.683 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T23:58:46.767 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:46.766+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:46.831 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:46.831 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:46.839 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T23:58:46.853 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:46.852+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:46.954 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:46.952+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:46.986 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:46.986 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:46.995 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T23:58:47.041 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.5 to 1 (10000) 2024-09-14T23:58:47.042 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is set 2024-09-14T23:58:47.054 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:47.052+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:47.066 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-14T23:58:47.066 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T23:58:47.124 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:47.124 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:47.134 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T23:58:47.154 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:47.153+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:47.255 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:47.254+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:47.286 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:47.286 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:47.295 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T23:58:47.356 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:47.355+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:47.408 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:58:47.409 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-14T23:58:47.421 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":217,"stamp":"2024-09-14T23:58:47.036675+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":631828725,"num_objects":2354,"num_object_clones":352,"num_object_copies":4708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2354,"num_whiteouts":1457,"num_read":303877,"num_read_kb":1025743,"num_write":23357,"num_write_kb":672629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4196,"num_bytes_recovered":273772210,"num_keys_recovered":495,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20133,"ondisk_log_size":20133,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":320932,"kb_used_data":53228,"kb_used_omap":9,"kb_used_meta":267510,"kb_avail":565910108,"statfs":{"total":579820584960,"available":579491950592,"internally_reserved":0,"allocated":54505472,"data_stored":421006108,"data_compressed":31274231,"data_compressed_allocated":34222080,"data_compressed_original":407806356,"omap_allocated":9569,"internal_metadata":273930911},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-127662675,"num_objects":-138,"num_object_clones":-47,"num_object_copies":-276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-138,"num_whiteouts":-33,"num_read":1197,"num_read_kb":1025,"num_write":213,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.060586"},"pg_stats":[{"pgid":"3.1f","version":"143'989","reported_seq":5554,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-14T23:58:43.090535+0000","last_change":"2024-09-14T23:58:42.979652+0000","last_active":"2024-09-14T23:58:43.090535+0000","last_peered":"2024-09-14T23:58:43.090535+0000","last_clean":"2024-09-14T23:58:43.090535+0000","last_became_active":"2024-09-14T23:57:00.062116+0000","last_became_peered":"2024-09-14T23:57:00.062116+0000","last_unstale":"2024-09-14T23:58:43.090535+0000","last_undegraded":"2024-09-14T23:58:43.090535+0000","last_fullsized":"2024-09-14T23:58:43.090535+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":989,"log_dups_size":0,"ondisk_log_size":989,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:58:25.534846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072008499999999997,"stat_sum":{"num_bytes":21756827,"num_objects":82,"num_object_clones":13,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":46,"num_read":7979,"num_read_kb":36865,"num_write":1260,"num_write_kb":38027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":2327658,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1d","length":"6"}]},{"pgid":"3.1e","version":"145'433","reported_seq":6424,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-14T23:58:45.188792+0000","last_change":"2024-09-14T23:58:45.028849+0000","last_active":"2024-09-14T23:58:45.188792+0000","last_peered":"2024-09-14T23:58:45.188792+0000","last_clean":"2024-09-14T23:58:45.188792+0000","last_became_active":"2024-09-14T23:58:27.981159+0000","last_became_peered":"2024-09-14T23:58:27.981159+0000","last_unstale":"2024-09-14T23:58:45.188792+0000","last_undegraded":"2024-09-14T23:58:45.188792+0000","last_fullsized":"2024-09-14T23:58:45.188792+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:51:53.334953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063562000000000004,"stat_sum":{"num_bytes":13278835,"num_objects":72,"num_object_clones":15,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":41,"num_read":5326,"num_read_kb":37530,"num_write":456,"num_write_kb":39218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":368,"num_bytes_recovered":14218815,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"3.1d","version":"145'499","reported_seq":13158,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-14T23:58:45.102793+0000","last_change":"2024-09-14T23:58:45.033923+0000","last_active":"2024-09-14T23:58:45.102793+0000","last_peered":"2024-09-14T23:58:45.102793+0000","last_clean":"2024-09-14T23:58:45.102793+0000","last_became_active":"2024-09-14T23:57:00.062242+0000","last_became_peered":"2024-09-14T23:57:00.062242+0000","last_unstale":"2024-09-14T23:58:45.102793+0000","last_undegraded":"2024-09-14T23:58:45.102793+0000","last_fullsized":"2024-09-14T23:58:45.102793+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":499,"log_dups_size":0,"ondisk_log_size":499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:30:20.134138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001233687,"stat_sum":{"num_bytes":28621232,"num_objects":81,"num_object_clones":15,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":49,"num_read":14022,"num_read_kb":22304,"num_write":471,"num_write_kb":24083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":1249963,"num_keys_recovered":34,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"3.1c","version":"145'911","reported_seq":6189,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-14T23:58:45.163378+0000","last_change":"2024-09-14T23:58:45.034274+0000","last_active":"2024-09-14T23:58:45.163378+0000","last_peered":"2024-09-14T23:58:45.163378+0000","last_clean":"2024-09-14T23:58:45.163378+0000","last_became_active":"2024-09-14T23:57:00.062126+0000","last_became_peered":"2024-09-14T23:57:00.062126+0000","last_unstale":"2024-09-14T23:58:45.163378+0000","last_undegraded":"2024-09-14T23:58:45.163378+0000","last_fullsized":"2024-09-14T23:58:45.163378+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":911,"log_dups_size":0,"ondisk_log_size":911,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:40:36.693694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015300509999999999,"stat_sum":{"num_bytes":9921574,"num_objects":72,"num_object_clones":7,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":50,"num_read":9153,"num_read_kb":25221,"num_write":1219,"num_write_kb":9723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":2675491,"num_keys_recovered":40,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"3.1b","version":"140'500","reported_seq":2148,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-14T23:58:42.978683+0000","last_change":"2024-09-14T23:58:42.978973+0000","last_active":"2024-09-14T23:58:42.978683+0000","last_peered":"2024-09-14T23:58:42.978683+0000","last_clean":"2024-09-14T23:58:42.978683+0000","last_became_active":"2024-09-14T23:56:32.398329+0000","last_became_peered":"2024-09-14T23:56:32.398329+0000","last_unstale":"2024-09-14T23:58:42.978683+0000","last_undegraded":"2024-09-14T23:58:42.978683+0000","last_fullsized":"2024-09-14T23:58:42.978683+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:24:50.601935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086922300000000002,"stat_sum":{"num_bytes":10931520,"num_objects":48,"num_object_clones":6,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":30,"num_read":2076,"num_read_kb":19877,"num_write":611,"num_write_kb":10342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1d","length":"6"}]},{"pgid":"3.1a","version":"142'857","reported_seq":8061,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-14T23:58:45.175681+0000","last_change":"2024-09-14T23:58:45.032928+0000","last_active":"2024-09-14T23:58:45.175681+0000","last_peered":"2024-09-14T23:58:45.175681+0000","last_clean":"2024-09-14T23:58:45.175681+0000","last_became_active":"2024-09-14T23:56:32.399655+0000","last_became_peered":"2024-09-14T23:56:32.399655+0000","last_unstale":"2024-09-14T23:58:45.175681+0000","last_undegraded":"2024-09-14T23:58:45.175681+0000","last_fullsized":"2024-09-14T23:58:45.175681+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":857,"log_dups_size":0,"ondisk_log_size":857,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:26:14.412999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046774799999999998,"stat_sum":{"num_bytes":15076848,"num_objects":86,"num_object_clones":15,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":58,"num_read":9867,"num_read_kb":53644,"num_write":958,"num_write_kb":34825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"3.19","version":"138'647","reported_seq":4079,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-14T23:58:42.983125+0000","last_change":"2024-09-14T23:58:42.983643+0000","last_active":"2024-09-14T23:58:42.983125+0000","last_peered":"2024-09-14T23:58:42.983125+0000","last_clean":"2024-09-14T23:58:42.983125+0000","last_became_active":"2024-09-14T23:58:27.976964+0000","last_became_peered":"2024-09-14T23:58:27.976964+0000","last_unstale":"2024-09-14T23:58:42.983125+0000","last_undegraded":"2024-09-14T23:58:42.983125+0000","last_fullsized":"2024-09-14T23:58:42.983125+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":647,"log_dups_size":0,"ondisk_log_size":647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:52:18.829754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018404599999999999,"stat_sum":{"num_bytes":12145008,"num_objects":79,"num_object_clones":9,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":51,"num_read":5445,"num_read_kb":20509,"num_write":686,"num_write_kb":15830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":159,"num_bytes_recovered":6182078,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1d","length":"6"}]},{"pgid":"3.18","version":"141'653","reported_seq":6087,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-14T23:58:42.983999+0000","last_change":"2024-09-14T23:58:42.984248+0000","last_active":"2024-09-14T23:58:42.983999+0000","last_peered":"2024-09-14T23:58:42.983999+0000","last_clean":"2024-09-14T23:58:42.983999+0000","last_became_active":"2024-09-14T23:58:28.981081+0000","last_became_peered":"2024-09-14T23:58:28.981081+0000","last_unstale":"2024-09-14T23:58:42.983999+0000","last_undegraded":"2024-09-14T23:58:42.983999+0000","last_fullsized":"2024-09-14T23:58:42.983999+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":653,"log_dups_size":0,"ondisk_log_size":653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:52:49.439951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001767188,"stat_sum":{"num_bytes":16912111,"num_objects":91,"num_object_clones":14,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":59,"num_read":6349,"num_read_kb":59036,"num_write":691,"num_write_kb":26083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":12559960,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1d","length":"6"}]},{"pgid":"3.17","version":"142'509","reported_seq":11476,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-14T23:58:42.979819+0000","last_change":"2024-09-14T23:58:42.980067+0000","last_active":"2024-09-14T23:58:42.979819+0000","last_peered":"2024-09-14T23:58:42.979819+0000","last_clean":"2024-09-14T23:58:42.979819+0000","last_became_active":"2024-09-14T23:57:00.062304+0000","last_became_peered":"2024-09-14T23:57:00.062304+0000","last_unstale":"2024-09-14T23:58:42.979819+0000","last_undegraded":"2024-09-14T23:58:42.979819+0000","last_fullsized":"2024-09-14T23:58:42.979819+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":509,"log_dups_size":0,"ondisk_log_size":509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:13.468844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082869300000000003,"stat_sum":{"num_bytes":8610613,"num_objects":65,"num_object_clones":9,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":39,"num_read":12413,"num_read_kb":16848,"num_write":474,"num_write_kb":6294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":3345543,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1d","length":"6"}]},{"pgid":"3.16","version":"142'412","reported_seq":7316,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-14T23:58:42.984044+0000","last_change":"2024-09-14T23:58:42.984273+0000","last_active":"2024-09-14T23:58:42.984044+0000","last_peered":"2024-09-14T23:58:42.984044+0000","last_clean":"2024-09-14T23:58:42.984044+0000","last_became_active":"2024-09-14T23:58:11.139452+0000","last_became_peered":"2024-09-14T23:58:11.139452+0000","last_unstale":"2024-09-14T23:58:42.984044+0000","last_undegraded":"2024-09-14T23:58:42.984044+0000","last_fullsized":"2024-09-14T23:58:42.984044+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":412,"log_dups_size":0,"ondisk_log_size":412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:11:55.056059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001789868,"stat_sum":{"num_bytes":26861480,"num_objects":58,"num_object_clones":6,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":34,"num_read":7597,"num_read_kb":27345,"num_write":348,"num_write_kb":13362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":286,"num_bytes_recovered":24669349,"num_keys_recovered":72,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1d","length":"6"}]},{"pgid":"3.15","version":"145'540","reported_seq":9224,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-14T23:58:45.142145+0000","last_change":"2024-09-14T23:58:45.033602+0000","last_active":"2024-09-14T23:58:45.142145+0000","last_peered":"2024-09-14T23:58:45.142145+0000","last_clean":"2024-09-14T23:58:45.142145+0000","last_became_active":"2024-09-14T23:58:27.976874+0000","last_became_peered":"2024-09-14T23:58:27.976874+0000","last_unstale":"2024-09-14T23:58:45.142145+0000","last_undegraded":"2024-09-14T23:58:45.142145+0000","last_fullsized":"2024-09-14T23:58:45.142145+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:52:43.165351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001117406,"stat_sum":{"num_bytes":35727545,"num_objects":78,"num_object_clones":12,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":50,"num_read":9715,"num_read_kb":35938,"num_write":574,"num_write_kb":23684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":258,"num_bytes_recovered":27784216,"num_keys_recovered":48,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"3.14","version":"145'1033","reported_seq":7588,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-14T23:58:45.034869+0000","last_change":"2024-09-14T23:58:45.034869+0000","last_active":"2024-09-14T23:58:45.034869+0000","last_peered":"2024-09-14T23:58:45.034869+0000","last_clean":"2024-09-14T23:58:45.034869+0000","last_became_active":"2024-09-14T23:58:12.137872+0000","last_became_peered":"2024-09-14T23:58:12.137872+0000","last_unstale":"2024-09-14T23:58:45.034869+0000","last_undegraded":"2024-09-14T23:58:45.034869+0000","last_fullsized":"2024-09-14T23:58:45.034869+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":1033,"log_dups_size":0,"ondisk_log_size":1033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:00.880723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001872199,"stat_sum":{"num_bytes":20525309,"num_objects":75,"num_object_clones":11,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":48,"num_read":9608,"num_read_kb":42272,"num_write":1542,"num_write_kb":28898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":21529916,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"2.7","version":"0'0","reported_seq":273,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-14T23:58:45.028086+0000","last_change":"2024-09-14T23:58:07.977363+0000","last_active":"2024-09-14T23:58:45.028086+0000","last_peered":"2024-09-14T23:58:45.028086+0000","last_clean":"2024-09-14T23:58:45.028086+0000","last_became_active":"2024-09-14T23:58:07.977047+0000","last_became_peered":"2024-09-14T23:58:07.977047+0000","last_unstale":"2024-09-14T23:58:45.028086+0000","last_undegraded":"2024-09-14T23:58:45.028086+0000","last_fullsized":"2024-09-14T23:58:45.028086+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:07:21.332155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045105900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"143'639","reported_seq":4915,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-14T23:58:43.066670+0000","last_change":"2024-09-14T23:58:42.978471+0000","last_active":"2024-09-14T23:58:43.066670+0000","last_peered":"2024-09-14T23:58:43.066670+0000","last_clean":"2024-09-14T23:58:43.066670+0000","last_became_active":"2024-09-14T23:56:32.398664+0000","last_became_peered":"2024-09-14T23:56:32.398664+0000","last_unstale":"2024-09-14T23:58:43.066670+0000","last_undegraded":"2024-09-14T23:58:43.066670+0000","last_fullsized":"2024-09-14T23:58:43.066670+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":639,"log_dups_size":0,"ondisk_log_size":639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:16:09.434724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058035900000000002,"stat_sum":{"num_bytes":15943243,"num_objects":71,"num_object_clones":6,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":45,"num_read":6384,"num_read_kb":24140,"num_write":718,"num_write_kb":5727,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1d","length":"6"}]},{"pgid":"2.6","version":"0'0","reported_seq":264,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-14T23:58:42.982735+0000","last_change":"2024-09-14T23:56:24.534632+0000","last_active":"2024-09-14T23:58:42.982735+0000","last_peered":"2024-09-14T23:58:42.982735+0000","last_clean":"2024-09-14T23:58:42.982735+0000","last_became_active":"2024-09-14T23:56:22.526355+0000","last_became_peered":"2024-09-14T23:56:22.526355+0000","last_unstale":"2024-09-14T23:58:42.982735+0000","last_undegraded":"2024-09-14T23:58:42.982735+0000","last_fullsized":"2024-09-14T23:58:42.982735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:02:16.059627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049910000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"143'443","reported_seq":7206,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-14T23:58:43.057357+0000","last_change":"2024-09-14T23:58:42.979633+0000","last_active":"2024-09-14T23:58:43.057357+0000","last_peered":"2024-09-14T23:58:43.057357+0000","last_clean":"2024-09-14T23:58:43.057357+0000","last_became_active":"2024-09-14T23:58:27.979626+0000","last_became_peered":"2024-09-14T23:58:27.979626+0000","last_unstale":"2024-09-14T23:58:43.057357+0000","last_undegraded":"2024-09-14T23:58:43.057357+0000","last_fullsized":"2024-09-14T23:58:43.057357+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":443,"log_dups_size":0,"ondisk_log_size":443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:18:16.084764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074545000000000004,"stat_sum":{"num_bytes":25401040,"num_objects":83,"num_object_clones":14,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":54,"num_read":7101,"num_read_kb":42993,"num_write":452,"num_write_kb":17287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":320,"num_bytes_recovered":28694058,"num_keys_recovered":42,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1d","length":"6"}]},{"pgid":"2.5","version":"0'0","reported_seq":77,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-14T23:58:42.978807+0000","last_change":"2024-09-14T23:58:07.974964+0000","last_active":"2024-09-14T23:58:42.978807+0000","last_peered":"2024-09-14T23:58:42.978807+0000","last_clean":"2024-09-14T23:58:42.978807+0000","last_became_active":"2024-09-14T23:58:07.974307+0000","last_became_peered":"2024-09-14T23:58:07.974307+0000","last_unstale":"2024-09-14T23:58:42.978807+0000","last_undegraded":"2024-09-14T23:58:42.978807+0000","last_fullsized":"2024-09-14T23:58:42.978807+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:27:42.149000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"143'574","reported_seq":5234,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-14T23:58:43.062234+0000","last_change":"2024-09-14T23:58:42.983349+0000","last_active":"2024-09-14T23:58:43.062234+0000","last_peered":"2024-09-14T23:58:43.062234+0000","last_clean":"2024-09-14T23:58:43.062234+0000","last_became_active":"2024-09-14T23:56:32.401089+0000","last_became_peered":"2024-09-14T23:56:32.401089+0000","last_unstale":"2024-09-14T23:58:43.062234+0000","last_undegraded":"2024-09-14T23:58:43.062234+0000","last_fullsized":"2024-09-14T23:58:43.062234+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":574,"log_dups_size":0,"ondisk_log_size":574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:25:58.264709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015599959999999999,"stat_sum":{"num_bytes":21968624,"num_objects":87,"num_object_clones":13,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":52,"num_read":6571,"num_read_kb":26343,"num_write":574,"num_write_kb":13585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1d","length":"6"}]},{"pgid":"2.4","version":"0'0","reported_seq":264,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-14T23:58:42.982175+0000","last_change":"2024-09-14T23:56:24.534510+0000","last_active":"2024-09-14T23:58:42.982175+0000","last_peered":"2024-09-14T23:58:42.982175+0000","last_clean":"2024-09-14T23:58:42.982175+0000","last_became_active":"2024-09-14T23:56:22.527611+0000","last_became_peered":"2024-09-14T23:56:22.527611+0000","last_unstale":"2024-09-14T23:58:42.982175+0000","last_undegraded":"2024-09-14T23:58:42.982175+0000","last_fullsized":"2024-09-14T23:58:42.982175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:15:45.606026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049916299999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"142'646","reported_seq":7237,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-14T23:58:42.031906+0000","last_change":"2024-09-14T23:58:42.031906+0000","last_active":"2024-09-14T23:58:42.031906+0000","last_peered":"2024-09-14T23:58:42.031906+0000","last_clean":"2024-09-14T23:58:42.031906+0000","last_became_active":"2024-09-14T23:58:28.980804+0000","last_became_peered":"2024-09-14T23:58:28.980804+0000","last_unstale":"2024-09-14T23:58:42.031906+0000","last_undegraded":"2024-09-14T23:58:42.031906+0000","last_fullsized":"2024-09-14T23:58:42.031906+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:51:31.182068+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0047414149999999997,"stat_sum":{"num_bytes":12132899,"num_objects":95,"num_object_clones":15,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":59,"num_read":9046,"num_read_kb":18601,"num_write":726,"num_write_kb":10072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":242,"num_bytes_recovered":8358340,"num_keys_recovered":34,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"1e","length":"a"}]},{"pgid":"3.3","version":"145'662","reported_seq":12242,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-14T23:58:45.161162+0000","last_change":"2024-09-14T23:58:45.034225+0000","last_active":"2024-09-14T23:58:45.161162+0000","last_peered":"2024-09-14T23:58:45.161162+0000","last_clean":"2024-09-14T23:58:45.161162+0000","last_became_active":"2024-09-14T23:56:32.397883+0000","last_became_peered":"2024-09-14T23:56:32.397883+0000","last_unstale":"2024-09-14T23:58:45.161162+0000","last_undegraded":"2024-09-14T23:58:45.161162+0000","last_fullsized":"2024-09-14T23:58:45.161162+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":662,"log_dups_size":0,"ondisk_log_size":662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:42:44.192505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001519502,"stat_sum":{"num_bytes":20549548,"num_objects":78,"num_object_clones":11,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":44,"num_read":12915,"num_read_kb":25375,"num_write":657,"num_write_kb":35365,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"2.2","version":"21'2","reported_seq":80,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-14T23:58:42.982262+0000","last_change":"2024-09-14T23:58:07.978936+0000","last_active":"2024-09-14T23:58:42.982262+0000","last_peered":"2024-09-14T23:58:42.982262+0000","last_clean":"2024-09-14T23:58:42.982262+0000","last_became_active":"2024-09-14T23:58:07.975815+0000","last_became_peered":"2024-09-14T23:58:07.975815+0000","last_unstale":"2024-09-14T23:58:42.982262+0000","last_undegraded":"2024-09-14T23:58:42.982262+0000","last_fullsized":"2024-09-14T23:58:42.982262+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:51:25.065497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"143'485","reported_seq":9094,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-14T23:58:42.983988+0000","last_change":"2024-09-14T23:58:42.983988+0000","last_active":"2024-09-14T23:58:42.983988+0000","last_peered":"2024-09-14T23:58:42.983988+0000","last_clean":"2024-09-14T23:58:42.983988+0000","last_became_active":"2024-09-14T23:56:32.400278+0000","last_became_peered":"2024-09-14T23:56:32.400278+0000","last_unstale":"2024-09-14T23:58:42.983988+0000","last_undegraded":"2024-09-14T23:58:42.983988+0000","last_fullsized":"2024-09-14T23:58:42.983988+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:22:26.177071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001858794,"stat_sum":{"num_bytes":29306721,"num_objects":63,"num_object_clones":7,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":37,"num_read":8534,"num_read_kb":62558,"num_write":413,"num_write_kb":30360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1d","length":"6"}]},{"pgid":"2.1","version":"0'0","reported_seq":266,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-14T23:58:42.027359+0000","last_change":"2024-09-14T23:58:25.299874+0000","last_active":"2024-09-14T23:58:42.027359+0000","last_peered":"2024-09-14T23:58:42.027359+0000","last_clean":"2024-09-14T23:58:42.027359+0000","last_became_active":"2024-09-14T23:58:25.299570+0000","last_became_peered":"2024-09-14T23:58:25.299570+0000","last_unstale":"2024-09-14T23:58:42.027359+0000","last_undegraded":"2024-09-14T23:58:42.027359+0000","last_fullsized":"2024-09-14T23:58:42.027359+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:42:03.033617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026863400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"145'674","reported_seq":6710,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-14T23:58:45.103939+0000","last_change":"2024-09-14T23:58:45.033281+0000","last_active":"2024-09-14T23:58:45.103939+0000","last_peered":"2024-09-14T23:58:45.103939+0000","last_clean":"2024-09-14T23:58:45.103939+0000","last_became_active":"2024-09-14T23:58:14.192509+0000","last_became_peered":"2024-09-14T23:58:14.192509+0000","last_unstale":"2024-09-14T23:58:45.103939+0000","last_undegraded":"2024-09-14T23:58:45.103939+0000","last_fullsized":"2024-09-14T23:58:45.103939+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":674,"log_dups_size":0,"ondisk_log_size":674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:45:08.975859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080818000000000003,"stat_sum":{"num_bytes":11751557,"num_objects":81,"num_object_clones":12,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":52,"num_read":8840,"num_read_kb":21631,"num_write":860,"num_write_kb":5096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"2.0","version":"0'0","reported_seq":77,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-14T23:58:42.982303+0000","last_change":"2024-09-14T23:58:07.974790+0000","last_active":"2024-09-14T23:58:42.982303+0000","last_peered":"2024-09-14T23:58:42.982303+0000","last_clean":"2024-09-14T23:58:42.982303+0000","last_became_active":"2024-09-14T23:58:07.974482+0000","last_became_peered":"2024-09-14T23:58:07.974482+0000","last_unstale":"2024-09-14T23:58:42.982303+0000","last_undegraded":"2024-09-14T23:58:42.982303+0000","last_fullsized":"2024-09-14T23:58:42.982303+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:26:59.848225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"145'1038","reported_seq":29925,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-14T23:58:45.029203+0000","last_change":"2024-09-14T23:58:45.029203+0000","last_active":"2024-09-14T23:58:45.029203+0000","last_peered":"2024-09-14T23:58:45.029203+0000","last_clean":"2024-09-14T23:58:45.029203+0000","last_became_active":"2024-09-14T23:58:27.980535+0000","last_became_peered":"2024-09-14T23:58:27.980535+0000","last_unstale":"2024-09-14T23:58:45.029203+0000","last_undegraded":"2024-09-14T23:58:45.029203+0000","last_fullsized":"2024-09-14T23:58:45.029203+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":1038,"log_dups_size":0,"ondisk_log_size":1038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:20:36.255383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084934099999999998,"stat_sum":{"num_bytes":11095239,"num_objects":93,"num_object_clones":19,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":56,"num_read":31924,"num_read_kb":26355,"num_write":1392,"num_write_kb":17303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":469,"num_bytes_recovered":18776077,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"2.3","version":"19'1","reported_seq":258,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-14T23:58:42.027636+0000","last_change":"2024-09-14T23:56:57.054999+0000","last_active":"2024-09-14T23:58:42.027636+0000","last_peered":"2024-09-14T23:58:42.027636+0000","last_clean":"2024-09-14T23:58:42.027636+0000","last_became_active":"2024-09-14T23:56:57.050729+0000","last_became_peered":"2024-09-14T23:56:57.050729+0000","last_unstale":"2024-09-14T23:58:42.027636+0000","last_undegraded":"2024-09-14T23:58:42.027636+0000","last_fullsized":"2024-09-14T23:58:42.027636+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:23:11.727903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":233,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-14T23:58:45.027993+0000","last_change":"2024-09-14T23:58:14.191995+0000","last_active":"2024-09-14T23:58:45.027993+0000","last_peered":"2024-09-14T23:58:45.027993+0000","last_clean":"2024-09-14T23:58:45.027993+0000","last_became_active":"2024-09-14T23:58:14.191597+0000","last_became_peered":"2024-09-14T23:58:14.191597+0000","last_unstale":"2024-09-14T23:58:45.027993+0000","last_undegraded":"2024-09-14T23:58:45.027993+0000","last_fullsized":"2024-09-14T23:58:45.027993+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:14.322882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:14.322882+0000","last_clean_scrub_stamp":"2024-09-14T23:56:14.322882+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:11:57.219870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.8","version":"143'573","reported_seq":5438,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-14T23:58:42.982624+0000","last_change":"2024-09-14T23:58:42.982624+0000","last_active":"2024-09-14T23:58:42.982624+0000","last_peered":"2024-09-14T23:58:42.982624+0000","last_clean":"2024-09-14T23:58:42.982624+0000","last_became_active":"2024-09-14T23:58:27.976555+0000","last_became_peered":"2024-09-14T23:58:27.976555+0000","last_unstale":"2024-09-14T23:58:42.982624+0000","last_undegraded":"2024-09-14T23:58:42.982624+0000","last_fullsized":"2024-09-14T23:58:42.982624+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":573,"log_dups_size":0,"ondisk_log_size":573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:28:30.558111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088927500000000003,"stat_sum":{"num_bytes":14585339,"num_objects":57,"num_object_clones":6,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":38,"num_read":5898,"num_read_kb":18097,"num_write":502,"num_write_kb":10914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":4505829,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1d","length":"6"}]},{"pgid":"3.9","version":"145'304","reported_seq":10784,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-14T23:58:45.108179+0000","last_change":"2024-09-14T23:58:45.034603+0000","last_active":"2024-09-14T23:58:45.108179+0000","last_peered":"2024-09-14T23:58:45.108179+0000","last_clean":"2024-09-14T23:58:45.108179+0000","last_became_active":"2024-09-14T23:56:32.400017+0000","last_became_peered":"2024-09-14T23:56:32.400017+0000","last_unstale":"2024-09-14T23:58:45.108179+0000","last_undegraded":"2024-09-14T23:58:45.108179+0000","last_fullsized":"2024-09-14T23:58:45.108179+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":304,"log_dups_size":0,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:24:55.466066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001838562,"stat_sum":{"num_bytes":20761281,"num_objects":69,"num_object_clones":11,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":41,"num_read":10186,"num_read_kb":13013,"num_write":280,"num_write_kb":15704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"3.a","version":"145'774","reported_seq":15723,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-14T23:58:45.106474+0000","last_change":"2024-09-14T23:58:45.029054+0000","last_active":"2024-09-14T23:58:45.106474+0000","last_peered":"2024-09-14T23:58:45.106474+0000","last_clean":"2024-09-14T23:58:45.106474+0000","last_became_active":"2024-09-14T23:56:32.402769+0000","last_became_peered":"2024-09-14T23:56:32.402769+0000","last_unstale":"2024-09-14T23:58:45.106474+0000","last_undegraded":"2024-09-14T23:58:45.106474+0000","last_fullsized":"2024-09-14T23:58:45.106474+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":774,"log_dups_size":0,"ondisk_log_size":774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:37:42.085880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000843606,"stat_sum":{"num_bytes":19544689,"num_objects":70,"num_object_clones":8,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":46,"num_read":16864,"num_read_kb":32615,"num_write":1065,"num_write_kb":14958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"3.b","version":"142'615","reported_seq":13437,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-14T23:58:45.034329+0000","last_change":"2024-09-14T23:58:45.034647+0000","last_active":"2024-09-14T23:58:45.034329+0000","last_peered":"2024-09-14T23:58:45.034329+0000","last_clean":"2024-09-14T23:58:45.034329+0000","last_became_active":"2024-09-14T23:58:27.976832+0000","last_became_peered":"2024-09-14T23:58:27.976832+0000","last_unstale":"2024-09-14T23:58:45.034329+0000","last_undegraded":"2024-09-14T23:58:45.034329+0000","last_fullsized":"2024-09-14T23:58:45.034329+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":615,"log_dups_size":0,"ondisk_log_size":615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:04:15.894966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001705074,"stat_sum":{"num_bytes":36361843,"num_objects":64,"num_object_clones":12,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":37,"num_read":13779,"num_read_kb":56640,"num_write":743,"num_write_kb":15947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":129,"num_bytes_recovered":6510884,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"3.c","version":"140'621","reported_seq":15977,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-14T23:58:45.028344+0000","last_change":"2024-09-14T23:58:45.028554+0000","last_active":"2024-09-14T23:58:45.028344+0000","last_peered":"2024-09-14T23:58:45.028344+0000","last_clean":"2024-09-14T23:58:45.028344+0000","last_became_active":"2024-09-14T23:58:27.979946+0000","last_became_peered":"2024-09-14T23:58:27.979946+0000","last_unstale":"2024-09-14T23:58:45.028344+0000","last_undegraded":"2024-09-14T23:58:45.028344+0000","last_fullsized":"2024-09-14T23:58:45.028344+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:45:12.344741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040786199999999999,"stat_sum":{"num_bytes":46489726,"num_objects":75,"num_object_clones":13,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":44,"num_read":15279,"num_read_kb":69418,"num_write":826,"num_write_kb":43367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":189,"num_bytes_recovered":28280964,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"3.d","version":"144'483","reported_seq":3130,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-14T23:58:45.028599+0000","last_change":"2024-09-14T23:58:45.028827+0000","last_active":"2024-09-14T23:58:45.028599+0000","last_peered":"2024-09-14T23:58:45.028599+0000","last_clean":"2024-09-14T23:58:45.028599+0000","last_became_active":"2024-09-14T23:58:11.139554+0000","last_became_peered":"2024-09-14T23:58:11.139554+0000","last_unstale":"2024-09-14T23:58:45.028599+0000","last_undegraded":"2024-09-14T23:58:45.028599+0000","last_fullsized":"2024-09-14T23:58:45.028599+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:41:25.831138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067730899999999996,"stat_sum":{"num_bytes":8653118,"num_objects":59,"num_object_clones":7,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":39,"num_read":3926,"num_read_kb":12286,"num_write":512,"num_write_kb":8311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":318,"num_bytes_recovered":15479703,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"3.e","version":"139'441","reported_seq":5238,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-14T23:58:42.028295+0000","last_change":"2024-09-14T23:58:42.028565+0000","last_active":"2024-09-14T23:58:42.028295+0000","last_peered":"2024-09-14T23:58:42.028295+0000","last_clean":"2024-09-14T23:58:42.028295+0000","last_became_active":"2024-09-14T23:58:12.139534+0000","last_became_peered":"2024-09-14T23:58:12.139534+0000","last_unstale":"2024-09-14T23:58:42.028295+0000","last_undegraded":"2024-09-14T23:58:42.028295+0000","last_fullsized":"2024-09-14T23:58:42.028295+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:14:08.001005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059245999999999995,"stat_sum":{"num_bytes":24001781,"num_objects":75,"num_object_clones":13,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":40,"num_read":7308,"num_read_kb":21003,"num_write":409,"num_write_kb":11344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":7127327,"num_keys_recovered":34,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"1e","length":"a"}]},{"pgid":"3.f","version":"145'560","reported_seq":13512,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-14T23:58:45.104610+0000","last_change":"2024-09-14T23:58:45.033773+0000","last_active":"2024-09-14T23:58:45.104610+0000","last_peered":"2024-09-14T23:58:45.104610+0000","last_clean":"2024-09-14T23:58:45.104610+0000","last_became_active":"2024-09-14T23:58:11.137740+0000","last_became_peered":"2024-09-14T23:58:11.137740+0000","last_unstale":"2024-09-14T23:58:45.104610+0000","last_undegraded":"2024-09-14T23:58:45.104610+0000","last_fullsized":"2024-09-14T23:58:45.104610+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":560,"log_dups_size":0,"ondisk_log_size":560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:42:28.091989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001106203,"stat_sum":{"num_bytes":26736662,"num_objects":73,"num_object_clones":11,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":44,"num_read":14032,"num_read_kb":21106,"num_write":589,"num_write_kb":35760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":8450382,"num_keys_recovered":48,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"3.10","version":"145'755","reported_seq":6242,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-14T23:58:45.111813+0000","last_change":"2024-09-14T23:58:45.028550+0000","last_active":"2024-09-14T23:58:45.111813+0000","last_peered":"2024-09-14T23:58:45.111813+0000","last_clean":"2024-09-14T23:58:45.111813+0000","last_became_active":"2024-09-14T23:56:32.402973+0000","last_became_peered":"2024-09-14T23:56:32.402973+0000","last_unstale":"2024-09-14T23:58:45.111813+0000","last_undegraded":"2024-09-14T23:58:45.111813+0000","last_fullsized":"2024-09-14T23:58:45.111813+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":755,"log_dups_size":0,"ondisk_log_size":755,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:00:27.865670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045600600000000001,"stat_sum":{"num_bytes":17521304,"num_objects":75,"num_object_clones":8,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":52,"num_read":7002,"num_read_kb":57954,"num_write":1154,"num_write_kb":51461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"3.11","version":"145'512","reported_seq":8049,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-14T23:58:45.188224+0000","last_change":"2024-09-14T23:58:45.033108+0000","last_active":"2024-09-14T23:58:45.188224+0000","last_peered":"2024-09-14T23:58:45.188224+0000","last_clean":"2024-09-14T23:58:45.188224+0000","last_became_active":"2024-09-14T23:58:11.138513+0000","last_became_peered":"2024-09-14T23:58:11.138513+0000","last_unstale":"2024-09-14T23:58:45.188224+0000","last_undegraded":"2024-09-14T23:58:45.188224+0000","last_fullsized":"2024-09-14T23:58:45.188224+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:38:34.864577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069932800000000004,"stat_sum":{"num_bytes":25343151,"num_objects":71,"num_object_clones":12,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":43,"num_read":8593,"num_read_kb":36125,"num_write":543,"num_write_kb":28062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":141,"num_bytes_recovered":16337558,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"3.12","version":"141'349","reported_seq":3015,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-14T23:58:42.983615+0000","last_change":"2024-09-14T23:58:42.983938+0000","last_active":"2024-09-14T23:58:42.983615+0000","last_peered":"2024-09-14T23:58:42.983615+0000","last_clean":"2024-09-14T23:58:42.983615+0000","last_became_active":"2024-09-14T23:58:27.975890+0000","last_became_peered":"2024-09-14T23:58:27.975890+0000","last_unstale":"2024-09-14T23:58:42.983615+0000","last_undegraded":"2024-09-14T23:58:42.983615+0000","last_fullsized":"2024-09-14T23:58:42.983615+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:16:07.339427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002109522,"stat_sum":{"num_bytes":6762689,"num_objects":52,"num_object_clones":9,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":34,"num_read":3164,"num_read_kb":16377,"num_write":278,"num_write_kb":14646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":5103606,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1d","length":"6"}]},{"pgid":"3.13","version":"141'882","reported_seq":4106,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-14T23:58:45.148005+0000","last_change":"2024-09-14T23:58:45.033441+0000","last_active":"2024-09-14T23:58:45.148005+0000","last_peered":"2024-09-14T23:58:45.148005+0000","last_clean":"2024-09-14T23:58:45.148005+0000","last_became_active":"2024-09-14T23:58:12.139670+0000","last_became_peered":"2024-09-14T23:58:12.139670+0000","last_unstale":"2024-09-14T23:58:45.148005+0000","last_undegraded":"2024-09-14T23:58:45.148005+0000","last_fullsized":"2024-09-14T23:58:45.148005+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:04:34.762046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096963699999999997,"stat_sum":{"num_bytes":15958982,"num_objects":72,"num_object_clones":13,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":45,"num_read":6865,"num_read_kb":25624,"num_write":1213,"num_write_kb":14573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":9604455,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1b","length":"8"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":631238338,"num_objects":2350,"num_object_clones":352,"num_object_copies":4700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2350,"num_whiteouts":1457,"num_read":303761,"num_read_kb":1025643,"num_write":23196,"num_write_kb":670211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4192,"num_bytes_recovered":273772172,"num_keys_recovered":495,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":68677632,"data_stored":656998560,"data_compressed":49172213,"data_compressed_allocated":52424704,"data_compressed_original":643295180,"omap_allocated":0,"internal_metadata":0},"log_size":20013,"ondisk_log_size":20013,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":1771104,"data_compressed":98526,"data_compressed_allocated":135168,"data_compressed_original":1736704,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":0,"seq":64424509475,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509474,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38952,"kb_used_data":7756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94332888,"statfs":{"total":96636764160,"available":96596877312,"internally_reserved":0,"allocated":7942144,"data_stored":57927275,"data_compressed":4072589,"data_compressed_allocated":4796416,"data_compressed_original":55933805,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509473,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66676,"kb_used_data":9104,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94305164,"statfs":{"total":96636764160,"available":96568487936,"internally_reserved":0,"allocated":9322496,"data_stored":78244876,"data_compressed":5861098,"data_compressed_allocated":6131712,"data_compressed_original":76227095,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509475,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83408,"kb_used_data":11960,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94288432,"statfs":{"total":96636764160,"available":96551354368,"internally_reserved":0,"allocated":12247040,"data_stored":102046541,"data_compressed":7580757,"data_compressed_allocated":8372224,"data_compressed_original":99479620,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":0,"seq":64424509473,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542177,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43372,"kb_used_data":8020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94328468,"statfs":{"total":96636764160,"available":96592351232,"internally_reserved":0,"allocated":8212480,"data_stored":59155955,"data_compressed":4319373,"data_compressed_allocated":4698112,"data_compressed_original":56786611,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542176,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54652,"kb_used_data":9552,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94317188,"statfs":{"total":96636764160,"available":96580800512,"internally_reserved":0,"allocated":9781248,"data_stored":75008249,"data_compressed":5727635,"data_compressed_allocated":6295552,"data_compressed_original":72723650,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33872,"kb_used_data":6836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337968,"statfs":{"total":96636764160,"available":96602079232,"internally_reserved":0,"allocated":7000064,"data_stored":48623212,"data_compressed":3712779,"data_compressed_allocated":3928064,"data_compressed_original":46655575,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32297,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":33932,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32297,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":7839744,"data_stored":72547215,"data_compressed":5560141,"data_compressed_allocated":6086656,"data_compressed_original":71093442,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6348800,"data_stored":57311843,"data_compressed":4180824,"data_compressed_allocated":4538368,"data_compressed_original":55754419,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5148672,"data_stored":47033438,"data_compressed":3565425,"data_compressed_allocated":3764224,"data_compressed_original":45869143,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12812288,"data_stored":122065494,"data_compressed":9432023,"data_compressed_allocated":9170944,"data_compressed_original":118845279,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":9867264,"data_stored":95869156,"data_compressed":7090002,"data_compressed_allocated":7774208,"data_compressed_original":94167108,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":7356416,"data_stored":75299816,"data_compressed":5688840,"data_compressed_allocated":5922816,"data_compressed_original":74129943,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":6000640,"data_stored":55436372,"data_compressed":3934881,"data_compressed_allocated":4624384,"data_compressed_original":54295405,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":13303808,"data_stored":131435226,"data_compressed":9720077,"data_compressed_allocated":10543104,"data_compressed_original":129140441,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T23:58:47.423 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T23:58:47.444 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:47.444 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:47.453 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T23:58:47.495 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:47.493+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:47.559 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:47.557+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:47.604 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:47.604 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:47.613 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T23:58:47.659 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:47.658+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:47.716 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:58:47.717 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-14T23:58:47.728 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":218,"stamp":"2024-09-14T23:58:47.479202+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":625778577,"num_objects":2343,"num_object_clones":350,"num_object_copies":4686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2343,"num_whiteouts":1456,"num_read":304165,"num_read_kb":1025988,"num_write":23381,"num_write_kb":672635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4196,"num_bytes_recovered":273772210,"num_keys_recovered":495,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20167,"ondisk_log_size":20167,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":320136,"kb_used_data":52436,"kb_used_omap":9,"kb_used_meta":267510,"kb_avail":565910904,"statfs":{"total":579820584960,"available":579492765696,"internally_reserved":0,"allocated":53694464,"data_stored":413245090,"data_compressed":30489688,"data_compressed_allocated":33570816,"data_compressed_original":400189581,"omap_allocated":9569,"internal_metadata":273930911},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-62463785,"num_objects":-81,"num_object_clones":-19,"num_object_copies":-162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-81,"num_whiteouts":-16,"num_read":929,"num_read_kb":799,"num_write":158,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000916"},"pg_stats":[{"pgid":"3.1f","version":"143'989","reported_seq":5554,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-14T23:58:43.090535+0000","last_change":"2024-09-14T23:58:42.979652+0000","last_active":"2024-09-14T23:58:43.090535+0000","last_peered":"2024-09-14T23:58:43.090535+0000","last_clean":"2024-09-14T23:58:43.090535+0000","last_became_active":"2024-09-14T23:57:00.062116+0000","last_became_peered":"2024-09-14T23:57:00.062116+0000","last_unstale":"2024-09-14T23:58:43.090535+0000","last_undegraded":"2024-09-14T23:58:43.090535+0000","last_fullsized":"2024-09-14T23:58:43.090535+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":989,"log_dups_size":0,"ondisk_log_size":989,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:58:25.534846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072008499999999997,"stat_sum":{"num_bytes":21756827,"num_objects":82,"num_object_clones":13,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":46,"num_read":7979,"num_read_kb":36865,"num_write":1260,"num_write_kb":38027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":2327658,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1d","length":"6"}]},{"pgid":"3.1e","version":"145'433","reported_seq":6424,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-14T23:58:45.188792+0000","last_change":"2024-09-14T23:58:45.028849+0000","last_active":"2024-09-14T23:58:45.188792+0000","last_peered":"2024-09-14T23:58:45.188792+0000","last_clean":"2024-09-14T23:58:45.188792+0000","last_became_active":"2024-09-14T23:58:27.981159+0000","last_became_peered":"2024-09-14T23:58:27.981159+0000","last_unstale":"2024-09-14T23:58:45.188792+0000","last_undegraded":"2024-09-14T23:58:45.188792+0000","last_fullsized":"2024-09-14T23:58:45.188792+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:51:53.334953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063562000000000004,"stat_sum":{"num_bytes":13278835,"num_objects":72,"num_object_clones":15,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":41,"num_read":5326,"num_read_kb":37530,"num_write":456,"num_write_kb":39218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":368,"num_bytes_recovered":14218815,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"3.1d","version":"145'499","reported_seq":13158,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-14T23:58:45.102793+0000","last_change":"2024-09-14T23:58:45.033923+0000","last_active":"2024-09-14T23:58:45.102793+0000","last_peered":"2024-09-14T23:58:45.102793+0000","last_clean":"2024-09-14T23:58:45.102793+0000","last_became_active":"2024-09-14T23:57:00.062242+0000","last_became_peered":"2024-09-14T23:57:00.062242+0000","last_unstale":"2024-09-14T23:58:45.102793+0000","last_undegraded":"2024-09-14T23:58:45.102793+0000","last_fullsized":"2024-09-14T23:58:45.102793+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":499,"log_dups_size":0,"ondisk_log_size":499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:30:20.134138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001233687,"stat_sum":{"num_bytes":28621232,"num_objects":81,"num_object_clones":15,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":49,"num_read":14022,"num_read_kb":22304,"num_write":471,"num_write_kb":24083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":1249963,"num_keys_recovered":34,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"3.1c","version":"145'911","reported_seq":6189,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-14T23:58:45.163378+0000","last_change":"2024-09-14T23:58:45.034274+0000","last_active":"2024-09-14T23:58:45.163378+0000","last_peered":"2024-09-14T23:58:45.163378+0000","last_clean":"2024-09-14T23:58:45.163378+0000","last_became_active":"2024-09-14T23:57:00.062126+0000","last_became_peered":"2024-09-14T23:57:00.062126+0000","last_unstale":"2024-09-14T23:58:45.163378+0000","last_undegraded":"2024-09-14T23:58:45.163378+0000","last_fullsized":"2024-09-14T23:58:45.163378+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":911,"log_dups_size":0,"ondisk_log_size":911,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:40:36.693694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015300509999999999,"stat_sum":{"num_bytes":9921574,"num_objects":72,"num_object_clones":7,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":50,"num_read":9153,"num_read_kb":25221,"num_write":1219,"num_write_kb":9723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":2675491,"num_keys_recovered":40,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"3.1b","version":"140'500","reported_seq":2148,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-14T23:58:42.978683+0000","last_change":"2024-09-14T23:58:42.978973+0000","last_active":"2024-09-14T23:58:42.978683+0000","last_peered":"2024-09-14T23:58:42.978683+0000","last_clean":"2024-09-14T23:58:42.978683+0000","last_became_active":"2024-09-14T23:56:32.398329+0000","last_became_peered":"2024-09-14T23:56:32.398329+0000","last_unstale":"2024-09-14T23:58:42.978683+0000","last_undegraded":"2024-09-14T23:58:42.978683+0000","last_fullsized":"2024-09-14T23:58:42.978683+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:24:50.601935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086922300000000002,"stat_sum":{"num_bytes":10931520,"num_objects":48,"num_object_clones":6,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":30,"num_read":2076,"num_read_kb":19877,"num_write":611,"num_write_kb":10342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1d","length":"6"}]},{"pgid":"3.1a","version":"142'857","reported_seq":8061,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-14T23:58:45.175681+0000","last_change":"2024-09-14T23:58:45.032928+0000","last_active":"2024-09-14T23:58:45.175681+0000","last_peered":"2024-09-14T23:58:45.175681+0000","last_clean":"2024-09-14T23:58:45.175681+0000","last_became_active":"2024-09-14T23:56:32.399655+0000","last_became_peered":"2024-09-14T23:56:32.399655+0000","last_unstale":"2024-09-14T23:58:45.175681+0000","last_undegraded":"2024-09-14T23:58:45.175681+0000","last_fullsized":"2024-09-14T23:58:45.175681+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":857,"log_dups_size":0,"ondisk_log_size":857,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:26:14.412999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046774799999999998,"stat_sum":{"num_bytes":15076848,"num_objects":86,"num_object_clones":15,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":58,"num_read":9867,"num_read_kb":53644,"num_write":958,"num_write_kb":34825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"3.19","version":"138'647","reported_seq":4079,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-14T23:58:42.983125+0000","last_change":"2024-09-14T23:58:42.983643+0000","last_active":"2024-09-14T23:58:42.983125+0000","last_peered":"2024-09-14T23:58:42.983125+0000","last_clean":"2024-09-14T23:58:42.983125+0000","last_became_active":"2024-09-14T23:58:27.976964+0000","last_became_peered":"2024-09-14T23:58:27.976964+0000","last_unstale":"2024-09-14T23:58:42.983125+0000","last_undegraded":"2024-09-14T23:58:42.983125+0000","last_fullsized":"2024-09-14T23:58:42.983125+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":647,"log_dups_size":0,"ondisk_log_size":647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:52:18.829754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018404599999999999,"stat_sum":{"num_bytes":12145008,"num_objects":79,"num_object_clones":9,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":51,"num_read":5445,"num_read_kb":20509,"num_write":686,"num_write_kb":15830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":159,"num_bytes_recovered":6182078,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1d","length":"6"}]},{"pgid":"3.18","version":"141'653","reported_seq":6087,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-14T23:58:42.983999+0000","last_change":"2024-09-14T23:58:42.984248+0000","last_active":"2024-09-14T23:58:42.983999+0000","last_peered":"2024-09-14T23:58:42.983999+0000","last_clean":"2024-09-14T23:58:42.983999+0000","last_became_active":"2024-09-14T23:58:28.981081+0000","last_became_peered":"2024-09-14T23:58:28.981081+0000","last_unstale":"2024-09-14T23:58:42.983999+0000","last_undegraded":"2024-09-14T23:58:42.983999+0000","last_fullsized":"2024-09-14T23:58:42.983999+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":653,"log_dups_size":0,"ondisk_log_size":653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:52:49.439951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001767188,"stat_sum":{"num_bytes":16912111,"num_objects":91,"num_object_clones":14,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":59,"num_read":6349,"num_read_kb":59036,"num_write":691,"num_write_kb":26083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":12559960,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1d","length":"6"}]},{"pgid":"3.17","version":"142'509","reported_seq":11476,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-14T23:58:42.979819+0000","last_change":"2024-09-14T23:58:42.980067+0000","last_active":"2024-09-14T23:58:42.979819+0000","last_peered":"2024-09-14T23:58:42.979819+0000","last_clean":"2024-09-14T23:58:42.979819+0000","last_became_active":"2024-09-14T23:57:00.062304+0000","last_became_peered":"2024-09-14T23:57:00.062304+0000","last_unstale":"2024-09-14T23:58:42.979819+0000","last_undegraded":"2024-09-14T23:58:42.979819+0000","last_fullsized":"2024-09-14T23:58:42.979819+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":509,"log_dups_size":0,"ondisk_log_size":509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:13.468844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082869300000000003,"stat_sum":{"num_bytes":8610613,"num_objects":65,"num_object_clones":9,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":39,"num_read":12413,"num_read_kb":16848,"num_write":474,"num_write_kb":6294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":3345543,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1d","length":"6"}]},{"pgid":"3.16","version":"142'412","reported_seq":7316,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-14T23:58:42.984044+0000","last_change":"2024-09-14T23:58:42.984273+0000","last_active":"2024-09-14T23:58:42.984044+0000","last_peered":"2024-09-14T23:58:42.984044+0000","last_clean":"2024-09-14T23:58:42.984044+0000","last_became_active":"2024-09-14T23:58:11.139452+0000","last_became_peered":"2024-09-14T23:58:11.139452+0000","last_unstale":"2024-09-14T23:58:42.984044+0000","last_undegraded":"2024-09-14T23:58:42.984044+0000","last_fullsized":"2024-09-14T23:58:42.984044+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":412,"log_dups_size":0,"ondisk_log_size":412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:11:55.056059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001789868,"stat_sum":{"num_bytes":26861480,"num_objects":58,"num_object_clones":6,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":34,"num_read":7597,"num_read_kb":27345,"num_write":348,"num_write_kb":13362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":286,"num_bytes_recovered":24669349,"num_keys_recovered":72,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1d","length":"6"}]},{"pgid":"3.15","version":"145'540","reported_seq":9224,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-14T23:58:45.142145+0000","last_change":"2024-09-14T23:58:45.033602+0000","last_active":"2024-09-14T23:58:45.142145+0000","last_peered":"2024-09-14T23:58:45.142145+0000","last_clean":"2024-09-14T23:58:45.142145+0000","last_became_active":"2024-09-14T23:58:27.976874+0000","last_became_peered":"2024-09-14T23:58:27.976874+0000","last_unstale":"2024-09-14T23:58:45.142145+0000","last_undegraded":"2024-09-14T23:58:45.142145+0000","last_fullsized":"2024-09-14T23:58:45.142145+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:52:43.165351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001117406,"stat_sum":{"num_bytes":35727545,"num_objects":78,"num_object_clones":12,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":50,"num_read":9715,"num_read_kb":35938,"num_write":574,"num_write_kb":23684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":258,"num_bytes_recovered":27784216,"num_keys_recovered":48,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"3.14","version":"145'1033","reported_seq":7588,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-14T23:58:45.034869+0000","last_change":"2024-09-14T23:58:45.034869+0000","last_active":"2024-09-14T23:58:45.034869+0000","last_peered":"2024-09-14T23:58:45.034869+0000","last_clean":"2024-09-14T23:58:45.034869+0000","last_became_active":"2024-09-14T23:58:12.137872+0000","last_became_peered":"2024-09-14T23:58:12.137872+0000","last_unstale":"2024-09-14T23:58:45.034869+0000","last_undegraded":"2024-09-14T23:58:45.034869+0000","last_fullsized":"2024-09-14T23:58:45.034869+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":1033,"log_dups_size":0,"ondisk_log_size":1033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:00.880723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001872199,"stat_sum":{"num_bytes":20525309,"num_objects":75,"num_object_clones":11,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":48,"num_read":9608,"num_read_kb":42272,"num_write":1542,"num_write_kb":28898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":21529916,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"2.7","version":"0'0","reported_seq":273,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-14T23:58:45.028086+0000","last_change":"2024-09-14T23:58:07.977363+0000","last_active":"2024-09-14T23:58:45.028086+0000","last_peered":"2024-09-14T23:58:45.028086+0000","last_clean":"2024-09-14T23:58:45.028086+0000","last_became_active":"2024-09-14T23:58:07.977047+0000","last_became_peered":"2024-09-14T23:58:07.977047+0000","last_unstale":"2024-09-14T23:58:45.028086+0000","last_undegraded":"2024-09-14T23:58:45.028086+0000","last_fullsized":"2024-09-14T23:58:45.028086+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:07:21.332155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045105900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"143'639","reported_seq":4915,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-14T23:58:43.066670+0000","last_change":"2024-09-14T23:58:42.978471+0000","last_active":"2024-09-14T23:58:43.066670+0000","last_peered":"2024-09-14T23:58:43.066670+0000","last_clean":"2024-09-14T23:58:43.066670+0000","last_became_active":"2024-09-14T23:56:32.398664+0000","last_became_peered":"2024-09-14T23:56:32.398664+0000","last_unstale":"2024-09-14T23:58:43.066670+0000","last_undegraded":"2024-09-14T23:58:43.066670+0000","last_fullsized":"2024-09-14T23:58:43.066670+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":639,"log_dups_size":0,"ondisk_log_size":639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:16:09.434724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058035900000000002,"stat_sum":{"num_bytes":15943243,"num_objects":71,"num_object_clones":6,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":45,"num_read":6384,"num_read_kb":24140,"num_write":718,"num_write_kb":5727,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1d","length":"6"}]},{"pgid":"2.6","version":"0'0","reported_seq":264,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-14T23:58:42.982735+0000","last_change":"2024-09-14T23:56:24.534632+0000","last_active":"2024-09-14T23:58:42.982735+0000","last_peered":"2024-09-14T23:58:42.982735+0000","last_clean":"2024-09-14T23:58:42.982735+0000","last_became_active":"2024-09-14T23:56:22.526355+0000","last_became_peered":"2024-09-14T23:56:22.526355+0000","last_unstale":"2024-09-14T23:58:42.982735+0000","last_undegraded":"2024-09-14T23:58:42.982735+0000","last_fullsized":"2024-09-14T23:58:42.982735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:02:16.059627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049910000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"143'443","reported_seq":7206,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-14T23:58:43.057357+0000","last_change":"2024-09-14T23:58:42.979633+0000","last_active":"2024-09-14T23:58:43.057357+0000","last_peered":"2024-09-14T23:58:43.057357+0000","last_clean":"2024-09-14T23:58:43.057357+0000","last_became_active":"2024-09-14T23:58:27.979626+0000","last_became_peered":"2024-09-14T23:58:27.979626+0000","last_unstale":"2024-09-14T23:58:43.057357+0000","last_undegraded":"2024-09-14T23:58:43.057357+0000","last_fullsized":"2024-09-14T23:58:43.057357+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":443,"log_dups_size":0,"ondisk_log_size":443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:18:16.084764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074545000000000004,"stat_sum":{"num_bytes":25401040,"num_objects":83,"num_object_clones":14,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":54,"num_read":7101,"num_read_kb":42993,"num_write":452,"num_write_kb":17287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":320,"num_bytes_recovered":28694058,"num_keys_recovered":42,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1d","length":"6"}]},{"pgid":"2.5","version":"0'0","reported_seq":77,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-14T23:58:42.978807+0000","last_change":"2024-09-14T23:58:07.974964+0000","last_active":"2024-09-14T23:58:42.978807+0000","last_peered":"2024-09-14T23:58:42.978807+0000","last_clean":"2024-09-14T23:58:42.978807+0000","last_became_active":"2024-09-14T23:58:07.974307+0000","last_became_peered":"2024-09-14T23:58:07.974307+0000","last_unstale":"2024-09-14T23:58:42.978807+0000","last_undegraded":"2024-09-14T23:58:42.978807+0000","last_fullsized":"2024-09-14T23:58:42.978807+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:27:42.149000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"143'574","reported_seq":5234,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-14T23:58:43.062234+0000","last_change":"2024-09-14T23:58:42.983349+0000","last_active":"2024-09-14T23:58:43.062234+0000","last_peered":"2024-09-14T23:58:43.062234+0000","last_clean":"2024-09-14T23:58:43.062234+0000","last_became_active":"2024-09-14T23:56:32.401089+0000","last_became_peered":"2024-09-14T23:56:32.401089+0000","last_unstale":"2024-09-14T23:58:43.062234+0000","last_undegraded":"2024-09-14T23:58:43.062234+0000","last_fullsized":"2024-09-14T23:58:43.062234+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":574,"log_dups_size":0,"ondisk_log_size":574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:25:58.264709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015599959999999999,"stat_sum":{"num_bytes":21968624,"num_objects":87,"num_object_clones":13,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":52,"num_read":6571,"num_read_kb":26343,"num_write":574,"num_write_kb":13585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1d","length":"6"}]},{"pgid":"2.4","version":"0'0","reported_seq":264,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-14T23:58:42.982175+0000","last_change":"2024-09-14T23:56:24.534510+0000","last_active":"2024-09-14T23:58:42.982175+0000","last_peered":"2024-09-14T23:58:42.982175+0000","last_clean":"2024-09-14T23:58:42.982175+0000","last_became_active":"2024-09-14T23:56:22.527611+0000","last_became_peered":"2024-09-14T23:56:22.527611+0000","last_unstale":"2024-09-14T23:58:42.982175+0000","last_undegraded":"2024-09-14T23:58:42.982175+0000","last_fullsized":"2024-09-14T23:58:42.982175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:15:45.606026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049916299999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"147'650","reported_seq":7279,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-14T23:58:47.042642+0000","last_change":"2024-09-14T23:58:47.043012+0000","last_active":"2024-09-14T23:58:47.042642+0000","last_peered":"2024-09-14T23:58:47.042642+0000","last_clean":"2024-09-14T23:58:47.042642+0000","last_became_active":"2024-09-14T23:58:28.980804+0000","last_became_peered":"2024-09-14T23:58:28.980804+0000","last_unstale":"2024-09-14T23:58:47.042642+0000","last_undegraded":"2024-09-14T23:58:47.042642+0000","last_fullsized":"2024-09-14T23:58:47.042642+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":650,"log_dups_size":0,"ondisk_log_size":650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:35:14.164479+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0047818169999999998,"stat_sum":{"num_bytes":12050733,"num_objects":91,"num_object_clones":14,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":58,"num_read":9052,"num_read_kb":18604,"num_write":728,"num_write_kb":10072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":242,"num_bytes_recovered":8358340,"num_keys_recovered":34,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"19","length":"a"}]},{"pgid":"3.3","version":"145'662","reported_seq":12242,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-14T23:58:45.161162+0000","last_change":"2024-09-14T23:58:45.034225+0000","last_active":"2024-09-14T23:58:45.161162+0000","last_peered":"2024-09-14T23:58:45.161162+0000","last_clean":"2024-09-14T23:58:45.161162+0000","last_became_active":"2024-09-14T23:56:32.397883+0000","last_became_peered":"2024-09-14T23:56:32.397883+0000","last_unstale":"2024-09-14T23:58:45.161162+0000","last_undegraded":"2024-09-14T23:58:45.161162+0000","last_fullsized":"2024-09-14T23:58:45.161162+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":662,"log_dups_size":0,"ondisk_log_size":662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:42:44.192505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001519502,"stat_sum":{"num_bytes":20549548,"num_objects":78,"num_object_clones":11,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":44,"num_read":12915,"num_read_kb":25375,"num_write":657,"num_write_kb":35365,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"2.2","version":"21'2","reported_seq":80,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-14T23:58:42.982262+0000","last_change":"2024-09-14T23:58:07.978936+0000","last_active":"2024-09-14T23:58:42.982262+0000","last_peered":"2024-09-14T23:58:42.982262+0000","last_clean":"2024-09-14T23:58:42.982262+0000","last_became_active":"2024-09-14T23:58:07.975815+0000","last_became_peered":"2024-09-14T23:58:07.975815+0000","last_unstale":"2024-09-14T23:58:42.982262+0000","last_undegraded":"2024-09-14T23:58:42.982262+0000","last_fullsized":"2024-09-14T23:58:42.982262+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:51:25.065497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"143'485","reported_seq":9094,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-14T23:58:42.983988+0000","last_change":"2024-09-14T23:58:42.983988+0000","last_active":"2024-09-14T23:58:42.983988+0000","last_peered":"2024-09-14T23:58:42.983988+0000","last_clean":"2024-09-14T23:58:42.983988+0000","last_became_active":"2024-09-14T23:56:32.400278+0000","last_became_peered":"2024-09-14T23:56:32.400278+0000","last_unstale":"2024-09-14T23:58:42.983988+0000","last_undegraded":"2024-09-14T23:58:42.983988+0000","last_fullsized":"2024-09-14T23:58:42.983988+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:22:26.177071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001858794,"stat_sum":{"num_bytes":29306721,"num_objects":63,"num_object_clones":7,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":37,"num_read":8534,"num_read_kb":62558,"num_write":413,"num_write_kb":30360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1d","length":"6"}]},{"pgid":"2.1","version":"0'0","reported_seq":276,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-14T23:58:47.037978+0000","last_change":"2024-09-14T23:58:25.299874+0000","last_active":"2024-09-14T23:58:47.037978+0000","last_peered":"2024-09-14T23:58:47.037978+0000","last_clean":"2024-09-14T23:58:47.037978+0000","last_became_active":"2024-09-14T23:58:25.299570+0000","last_became_peered":"2024-09-14T23:58:25.299570+0000","last_unstale":"2024-09-14T23:58:47.037978+0000","last_undegraded":"2024-09-14T23:58:47.037978+0000","last_fullsized":"2024-09-14T23:58:47.037978+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:42:03.033617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026863400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"145'674","reported_seq":6710,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-14T23:58:45.103939+0000","last_change":"2024-09-14T23:58:45.033281+0000","last_active":"2024-09-14T23:58:45.103939+0000","last_peered":"2024-09-14T23:58:45.103939+0000","last_clean":"2024-09-14T23:58:45.103939+0000","last_became_active":"2024-09-14T23:58:14.192509+0000","last_became_peered":"2024-09-14T23:58:14.192509+0000","last_unstale":"2024-09-14T23:58:45.103939+0000","last_undegraded":"2024-09-14T23:58:45.103939+0000","last_fullsized":"2024-09-14T23:58:45.103939+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":674,"log_dups_size":0,"ondisk_log_size":674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:45:08.975859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080818000000000003,"stat_sum":{"num_bytes":11751557,"num_objects":81,"num_object_clones":12,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":52,"num_read":8840,"num_read_kb":21631,"num_write":860,"num_write_kb":5096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"2.0","version":"0'0","reported_seq":77,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-14T23:58:42.982303+0000","last_change":"2024-09-14T23:58:07.974790+0000","last_active":"2024-09-14T23:58:42.982303+0000","last_peered":"2024-09-14T23:58:42.982303+0000","last_clean":"2024-09-14T23:58:42.982303+0000","last_became_active":"2024-09-14T23:58:07.974482+0000","last_became_peered":"2024-09-14T23:58:07.974482+0000","last_unstale":"2024-09-14T23:58:42.982303+0000","last_undegraded":"2024-09-14T23:58:42.982303+0000","last_fullsized":"2024-09-14T23:58:42.982303+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:26:59.848225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"145'1038","reported_seq":29925,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-14T23:58:45.029203+0000","last_change":"2024-09-14T23:58:45.029203+0000","last_active":"2024-09-14T23:58:45.029203+0000","last_peered":"2024-09-14T23:58:45.029203+0000","last_clean":"2024-09-14T23:58:45.029203+0000","last_became_active":"2024-09-14T23:58:27.980535+0000","last_became_peered":"2024-09-14T23:58:27.980535+0000","last_unstale":"2024-09-14T23:58:45.029203+0000","last_undegraded":"2024-09-14T23:58:45.029203+0000","last_fullsized":"2024-09-14T23:58:45.029203+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":1038,"log_dups_size":0,"ondisk_log_size":1038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:20:36.255383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084934099999999998,"stat_sum":{"num_bytes":11095239,"num_objects":93,"num_object_clones":19,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":56,"num_read":31924,"num_read_kb":26355,"num_write":1392,"num_write_kb":17303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":469,"num_bytes_recovered":18776077,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"2.3","version":"19'1","reported_seq":269,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-14T23:58:47.038123+0000","last_change":"2024-09-14T23:56:57.054999+0000","last_active":"2024-09-14T23:58:47.038123+0000","last_peered":"2024-09-14T23:58:47.038123+0000","last_clean":"2024-09-14T23:58:47.038123+0000","last_became_active":"2024-09-14T23:56:57.050729+0000","last_became_peered":"2024-09-14T23:56:57.050729+0000","last_unstale":"2024-09-14T23:58:47.038123+0000","last_undegraded":"2024-09-14T23:58:47.038123+0000","last_fullsized":"2024-09-14T23:58:47.038123+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:23:11.727903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":233,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-14T23:58:45.027993+0000","last_change":"2024-09-14T23:58:14.191995+0000","last_active":"2024-09-14T23:58:45.027993+0000","last_peered":"2024-09-14T23:58:45.027993+0000","last_clean":"2024-09-14T23:58:45.027993+0000","last_became_active":"2024-09-14T23:58:14.191597+0000","last_became_peered":"2024-09-14T23:58:14.191597+0000","last_unstale":"2024-09-14T23:58:45.027993+0000","last_undegraded":"2024-09-14T23:58:45.027993+0000","last_fullsized":"2024-09-14T23:58:45.027993+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:14.322882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:14.322882+0000","last_clean_scrub_stamp":"2024-09-14T23:56:14.322882+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:11:57.219870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.8","version":"143'573","reported_seq":5438,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-14T23:58:42.982624+0000","last_change":"2024-09-14T23:58:42.982624+0000","last_active":"2024-09-14T23:58:42.982624+0000","last_peered":"2024-09-14T23:58:42.982624+0000","last_clean":"2024-09-14T23:58:42.982624+0000","last_became_active":"2024-09-14T23:58:27.976555+0000","last_became_peered":"2024-09-14T23:58:27.976555+0000","last_unstale":"2024-09-14T23:58:42.982624+0000","last_undegraded":"2024-09-14T23:58:42.982624+0000","last_fullsized":"2024-09-14T23:58:42.982624+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":573,"log_dups_size":0,"ondisk_log_size":573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:28:30.558111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088927500000000003,"stat_sum":{"num_bytes":14585339,"num_objects":57,"num_object_clones":6,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":38,"num_read":5898,"num_read_kb":18097,"num_write":502,"num_write_kb":10914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":4505829,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1d","length":"6"}]},{"pgid":"3.9","version":"145'304","reported_seq":10784,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-14T23:58:45.108179+0000","last_change":"2024-09-14T23:58:45.034603+0000","last_active":"2024-09-14T23:58:45.108179+0000","last_peered":"2024-09-14T23:58:45.108179+0000","last_clean":"2024-09-14T23:58:45.108179+0000","last_became_active":"2024-09-14T23:56:32.400017+0000","last_became_peered":"2024-09-14T23:56:32.400017+0000","last_unstale":"2024-09-14T23:58:45.108179+0000","last_undegraded":"2024-09-14T23:58:45.108179+0000","last_fullsized":"2024-09-14T23:58:45.108179+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":304,"log_dups_size":0,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:24:55.466066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001838562,"stat_sum":{"num_bytes":20761281,"num_objects":69,"num_object_clones":11,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":41,"num_read":10186,"num_read_kb":13013,"num_write":280,"num_write_kb":15704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"3.a","version":"145'774","reported_seq":15723,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-14T23:58:45.106474+0000","last_change":"2024-09-14T23:58:45.029054+0000","last_active":"2024-09-14T23:58:45.106474+0000","last_peered":"2024-09-14T23:58:45.106474+0000","last_clean":"2024-09-14T23:58:45.106474+0000","last_became_active":"2024-09-14T23:56:32.402769+0000","last_became_peered":"2024-09-14T23:56:32.402769+0000","last_unstale":"2024-09-14T23:58:45.106474+0000","last_undegraded":"2024-09-14T23:58:45.106474+0000","last_fullsized":"2024-09-14T23:58:45.106474+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":774,"log_dups_size":0,"ondisk_log_size":774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:37:42.085880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000843606,"stat_sum":{"num_bytes":19544689,"num_objects":70,"num_object_clones":8,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":46,"num_read":16864,"num_read_kb":32615,"num_write":1065,"num_write_kb":14958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"3.b","version":"142'615","reported_seq":13437,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-14T23:58:45.034329+0000","last_change":"2024-09-14T23:58:45.034647+0000","last_active":"2024-09-14T23:58:45.034329+0000","last_peered":"2024-09-14T23:58:45.034329+0000","last_clean":"2024-09-14T23:58:45.034329+0000","last_became_active":"2024-09-14T23:58:27.976832+0000","last_became_peered":"2024-09-14T23:58:27.976832+0000","last_unstale":"2024-09-14T23:58:45.034329+0000","last_undegraded":"2024-09-14T23:58:45.034329+0000","last_fullsized":"2024-09-14T23:58:45.034329+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":615,"log_dups_size":0,"ondisk_log_size":615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:04:15.894966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001705074,"stat_sum":{"num_bytes":36361843,"num_objects":64,"num_object_clones":12,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":37,"num_read":13779,"num_read_kb":56640,"num_write":743,"num_write_kb":15947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":129,"num_bytes_recovered":6510884,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"3.c","version":"140'621","reported_seq":15977,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-14T23:58:45.028344+0000","last_change":"2024-09-14T23:58:45.028554+0000","last_active":"2024-09-14T23:58:45.028344+0000","last_peered":"2024-09-14T23:58:45.028344+0000","last_clean":"2024-09-14T23:58:45.028344+0000","last_became_active":"2024-09-14T23:58:27.979946+0000","last_became_peered":"2024-09-14T23:58:27.979946+0000","last_unstale":"2024-09-14T23:58:45.028344+0000","last_undegraded":"2024-09-14T23:58:45.028344+0000","last_fullsized":"2024-09-14T23:58:45.028344+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:45:12.344741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040786199999999999,"stat_sum":{"num_bytes":46489726,"num_objects":75,"num_object_clones":13,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":44,"num_read":15279,"num_read_kb":69418,"num_write":826,"num_write_kb":43367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":189,"num_bytes_recovered":28280964,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"3.d","version":"144'483","reported_seq":3130,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-14T23:58:45.028599+0000","last_change":"2024-09-14T23:58:45.028827+0000","last_active":"2024-09-14T23:58:45.028599+0000","last_peered":"2024-09-14T23:58:45.028599+0000","last_clean":"2024-09-14T23:58:45.028599+0000","last_became_active":"2024-09-14T23:58:11.139554+0000","last_became_peered":"2024-09-14T23:58:11.139554+0000","last_unstale":"2024-09-14T23:58:45.028599+0000","last_undegraded":"2024-09-14T23:58:45.028599+0000","last_fullsized":"2024-09-14T23:58:45.028599+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:41:25.831138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067730899999999996,"stat_sum":{"num_bytes":8653118,"num_objects":59,"num_object_clones":7,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":39,"num_read":3926,"num_read_kb":12286,"num_write":512,"num_write_kb":8311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":318,"num_bytes_recovered":15479703,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"3.e","version":"147'471","reported_seq":5409,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-14T23:58:47.176794+0000","last_change":"2024-09-14T23:58:47.038944+0000","last_active":"2024-09-14T23:58:47.176794+0000","last_peered":"2024-09-14T23:58:47.176794+0000","last_clean":"2024-09-14T23:58:47.176794+0000","last_became_active":"2024-09-14T23:58:12.139534+0000","last_became_peered":"2024-09-14T23:58:12.139534+0000","last_unstale":"2024-09-14T23:58:47.176794+0000","last_undegraded":"2024-09-14T23:58:47.176794+0000","last_fullsized":"2024-09-14T23:58:47.176794+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:27:20.275629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071115999999999996,"stat_sum":{"num_bytes":18033799,"num_objects":68,"num_object_clones":12,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":40,"num_read":7590,"num_read_kb":21245,"num_write":431,"num_write_kb":11350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":7127327,"num_keys_recovered":34,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"19","length":"a"}]},{"pgid":"3.f","version":"145'560","reported_seq":13512,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-14T23:58:45.104610+0000","last_change":"2024-09-14T23:58:45.033773+0000","last_active":"2024-09-14T23:58:45.104610+0000","last_peered":"2024-09-14T23:58:45.104610+0000","last_clean":"2024-09-14T23:58:45.104610+0000","last_became_active":"2024-09-14T23:58:11.137740+0000","last_became_peered":"2024-09-14T23:58:11.137740+0000","last_unstale":"2024-09-14T23:58:45.104610+0000","last_undegraded":"2024-09-14T23:58:45.104610+0000","last_fullsized":"2024-09-14T23:58:45.104610+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":560,"log_dups_size":0,"ondisk_log_size":560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:42:28.091989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001106203,"stat_sum":{"num_bytes":26736662,"num_objects":73,"num_object_clones":11,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":44,"num_read":14032,"num_read_kb":21106,"num_write":589,"num_write_kb":35760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":8450382,"num_keys_recovered":48,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"3.10","version":"145'755","reported_seq":6242,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-14T23:58:45.111813+0000","last_change":"2024-09-14T23:58:45.028550+0000","last_active":"2024-09-14T23:58:45.111813+0000","last_peered":"2024-09-14T23:58:45.111813+0000","last_clean":"2024-09-14T23:58:45.111813+0000","last_became_active":"2024-09-14T23:56:32.402973+0000","last_became_peered":"2024-09-14T23:56:32.402973+0000","last_unstale":"2024-09-14T23:58:45.111813+0000","last_undegraded":"2024-09-14T23:58:45.111813+0000","last_fullsized":"2024-09-14T23:58:45.111813+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":755,"log_dups_size":0,"ondisk_log_size":755,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:00:27.865670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045600600000000001,"stat_sum":{"num_bytes":17521304,"num_objects":75,"num_object_clones":8,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":52,"num_read":7002,"num_read_kb":57954,"num_write":1154,"num_write_kb":51461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"3.11","version":"145'512","reported_seq":8049,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-14T23:58:45.188224+0000","last_change":"2024-09-14T23:58:45.033108+0000","last_active":"2024-09-14T23:58:45.188224+0000","last_peered":"2024-09-14T23:58:45.188224+0000","last_clean":"2024-09-14T23:58:45.188224+0000","last_became_active":"2024-09-14T23:58:11.138513+0000","last_became_peered":"2024-09-14T23:58:11.138513+0000","last_unstale":"2024-09-14T23:58:45.188224+0000","last_undegraded":"2024-09-14T23:58:45.188224+0000","last_fullsized":"2024-09-14T23:58:45.188224+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:38:34.864577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069932800000000004,"stat_sum":{"num_bytes":25343151,"num_objects":71,"num_object_clones":12,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":43,"num_read":8593,"num_read_kb":36125,"num_write":543,"num_write_kb":28062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":141,"num_bytes_recovered":16337558,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"3.12","version":"141'349","reported_seq":3015,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-14T23:58:42.983615+0000","last_change":"2024-09-14T23:58:42.983938+0000","last_active":"2024-09-14T23:58:42.983615+0000","last_peered":"2024-09-14T23:58:42.983615+0000","last_clean":"2024-09-14T23:58:42.983615+0000","last_became_active":"2024-09-14T23:58:27.975890+0000","last_became_peered":"2024-09-14T23:58:27.975890+0000","last_unstale":"2024-09-14T23:58:42.983615+0000","last_undegraded":"2024-09-14T23:58:42.983615+0000","last_fullsized":"2024-09-14T23:58:42.983615+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:16:07.339427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002109522,"stat_sum":{"num_bytes":6762689,"num_objects":52,"num_object_clones":9,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":34,"num_read":3164,"num_read_kb":16377,"num_write":278,"num_write_kb":14646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":5103606,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1d","length":"6"}]},{"pgid":"3.13","version":"141'882","reported_seq":4106,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-14T23:58:45.148005+0000","last_change":"2024-09-14T23:58:45.033441+0000","last_active":"2024-09-14T23:58:45.148005+0000","last_peered":"2024-09-14T23:58:45.148005+0000","last_clean":"2024-09-14T23:58:45.148005+0000","last_became_active":"2024-09-14T23:58:12.139670+0000","last_became_peered":"2024-09-14T23:58:12.139670+0000","last_unstale":"2024-09-14T23:58:45.148005+0000","last_undegraded":"2024-09-14T23:58:45.148005+0000","last_fullsized":"2024-09-14T23:58:45.148005+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:04:34.762046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096963699999999997,"stat_sum":{"num_bytes":15958982,"num_objects":72,"num_object_clones":13,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":45,"num_read":6865,"num_read_kb":25624,"num_write":1213,"num_write_kb":14573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":9604455,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1b","length":"8"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":625188190,"num_objects":2339,"num_object_clones":350,"num_object_copies":4678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2339,"num_whiteouts":1456,"num_read":304049,"num_read_kb":1025888,"num_write":23220,"num_write_kb":670217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4192,"num_bytes_recovered":273772172,"num_keys_recovered":495,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":67883008,"data_stored":649310650,"data_compressed":48479506,"data_compressed_allocated":51859456,"data_compressed_original":635793093,"omap_allocated":0,"internal_metadata":0},"log_size":20047,"ondisk_log_size":20047,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":1771104,"data_compressed":98526,"data_compressed_allocated":135168,"data_compressed_original":1736704,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":0,"seq":64424509475,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509474,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38952,"kb_used_data":7756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94332888,"statfs":{"total":96636764160,"available":96596877312,"internally_reserved":0,"allocated":7942144,"data_stored":57927275,"data_compressed":4072589,"data_compressed_allocated":4796416,"data_compressed_original":55933805,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509473,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66676,"kb_used_data":9104,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94305164,"statfs":{"total":96636764160,"available":96568487936,"internally_reserved":0,"allocated":9322496,"data_stored":78244876,"data_compressed":5861098,"data_compressed_allocated":6131712,"data_compressed_original":76227095,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509475,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83408,"kb_used_data":11960,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94288432,"statfs":{"total":96636764160,"available":96551354368,"internally_reserved":0,"allocated":12247040,"data_stored":102046541,"data_compressed":7580757,"data_compressed_allocated":8372224,"data_compressed_original":99479620,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":0,"seq":64424509473,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542177,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43372,"kb_used_data":8020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94328468,"statfs":{"total":96636764160,"available":96592351232,"internally_reserved":0,"allocated":8212480,"data_stored":59155955,"data_compressed":4319373,"data_compressed_allocated":4698112,"data_compressed_original":56786611,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542176,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54652,"kb_used_data":9552,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94317188,"statfs":{"total":96636764160,"available":96580800512,"internally_reserved":0,"allocated":9781248,"data_stored":75008249,"data_compressed":5727635,"data_compressed_allocated":6295552,"data_compressed_original":72723650,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33076,"kb_used_data":6044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338764,"statfs":{"total":96636764160,"available":96602894336,"internally_reserved":0,"allocated":6189056,"data_stored":40862194,"data_compressed":2928236,"data_compressed_allocated":3276800,"data_compressed_original":39038800,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32297,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":33932,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32297,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":7839744,"data_stored":72547215,"data_compressed":5560141,"data_compressed_allocated":6086656,"data_compressed_original":71093442,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6348800,"data_stored":57311843,"data_compressed":4180824,"data_compressed_allocated":4538368,"data_compressed_original":55754419,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4354048,"data_stored":39345528,"data_compressed":2872718,"data_compressed_allocated":3198976,"data_compressed_original":38367056,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12812288,"data_stored":122065494,"data_compressed":9432023,"data_compressed_allocated":9170944,"data_compressed_original":118845279,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":9867264,"data_stored":95869156,"data_compressed":7090002,"data_compressed_allocated":7774208,"data_compressed_original":94167108,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":7356416,"data_stored":75299816,"data_compressed":5688840,"data_compressed_allocated":5922816,"data_compressed_original":74129943,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":6000640,"data_stored":55436372,"data_compressed":3934881,"data_compressed_allocated":4624384,"data_compressed_original":54295405,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":13303808,"data_stored":131435226,"data_compressed":9720077,"data_compressed_allocated":10543104,"data_compressed_original":129140441,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T23:58:47.730 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-14T23:58:47.731 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T23:58:47.731 INFO:tasks.thrashosds.thrasher:Adding osd 7 2024-09-14T23:58:47.731 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 7 2024-09-14T23:58:47.732 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:47.732 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:47.741 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T23:58:47.782 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:47.781+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:47.861 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:47.860+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:47.889 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:47.890 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:47.898 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T23:58:47.962 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:47.960+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:48.042 INFO:tasks.ceph.mon.a.smithi042.stderr:2024-09-14T23:58:48.040+0000 7f1713676640 -1 mon.a@0(leader).osd e148 definitely_dead 0 2024-09-14T23:58:48.063 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:48.063 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:48.064 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:48.062+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:48.072 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T23:58:48.165 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:48.163+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:48.181 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:48.181 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:48.190 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T23:58:48.274 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:48.273+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:48.338 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:48.338 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:48.347 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T23:58:48.367 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:48.366+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:48.468 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:48.466+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:48.494 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:48.495 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:48.504 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T23:58:48.569 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:48.567+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:48.611 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:48.611 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:48.620 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T23:58:48.645 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T23:58:48.670 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:48.669+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:48.727 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:48.727 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:48.736 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T23:58:48.777 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:48.776+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:48.779 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:58:48.779 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:58:48.796 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:58:48.797 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:58:48.812 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:58:48.813 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:58:48.828 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:58:48.829 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:58:48.845 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:58:48.845 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:58:48.863 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:58:48.863 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:58:48.873 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:48.872+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:48.878 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:58:48.879 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:58:48.886 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:48.886 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:58:48.886 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:58:48.886 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:58:48.886 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.886 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:107472 3.1e 3:7bf21588:::rbd_data.11187de8b0a5.000000000000005d:head [set-alloc-hint object_size 2097152 write_size 2097152,write 317788~1491] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-14T23:58:48.886 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:25.198493+0000", 2024-09-14T23:58:48.886 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 83.685036930999999, 2024-09-14T23:58:48.886 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.78175149300000002, 2024-09-14T23:58:48.887 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:48.887 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:48.887 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:48.887 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:48.887 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:48.887 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:48.887 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 107472 2024-09-14T23:58:48.887 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.887 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:48.887 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.887 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:48.887 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198493+0000", 2024-09-14T23:58:48.887 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:48.887 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.888 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.888 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:48.888 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198493+0000", 2024-09-14T23:58:48.888 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:48.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.888 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.888 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:48.888 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198495+0000", 2024-09-14T23:58:48.888 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.826e-06 2024-09-14T23:58:48.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.888 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.889 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:48.889 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198503+0000", 2024-09-14T23:58:48.889 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0150000000000003e-06 2024-09-14T23:58:48.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.889 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.889 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:48.889 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198521+0000", 2024-09-14T23:58:48.889 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8374000000000001e-05 2024-09-14T23:58:48.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.889 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.889 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:48.889 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198533+0000", 2024-09-14T23:58:48.890 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1593000000000001e-05 2024-09-14T23:58:48.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.890 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.890 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:48.890 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198603+0000", 2024-09-14T23:58:48.890 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0279999999999998e-05 2024-09-14T23:58:48.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.890 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.890 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:48.890 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198617+0000", 2024-09-14T23:58:48.890 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3517e-05 2024-09-14T23:58:48.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.890 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.891 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:48.891 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.977067+0000", 2024-09-14T23:58:48.891 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.77845065800000002 2024-09-14T23:58:48.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.891 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.891 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for missing object", 2024-09-14T23:58:48.891 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.977173+0000", 2024-09-14T23:58:48.891 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010543999999999999 2024-09-14T23:58:48.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.891 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.891 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:48.891 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.978737+0000", 2024-09-14T23:58:48.891 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001564078 2024-09-14T23:58:48.892 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.892 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.892 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:48.892 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.978845+0000", 2024-09-14T23:58:48.892 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000108643 2024-09-14T23:58:48.892 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.892 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.892 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1", 2024-09-14T23:58:48.892 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.978964+0000", 2024-09-14T23:58:48.892 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000119085 2024-09-14T23:58:48.892 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.892 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.892 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:48.893 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980109+0000", 2024-09-14T23:58:48.893 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0011442170000000001 2024-09-14T23:58:48.893 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.893 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.893 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:58:48.893 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980130+0000", 2024-09-14T23:58:48.893 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1766000000000001e-05 2024-09-14T23:58:48.893 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.893 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.893 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:48.893 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980171+0000", 2024-09-14T23:58:48.893 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0627999999999998e-05 2024-09-14T23:58:48.893 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.893 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.894 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:48.894 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980244+0000", 2024-09-14T23:58:48.894 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3373000000000004e-05 2024-09-14T23:58:48.894 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:48.894 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:48.894 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:48.894 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.894 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.894 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129159 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 5996544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:48.894 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.927783+0000", 2024-09-14T23:58:48.894 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 69.955747040000006, 2024-09-14T23:58:48.894 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054661122999999999, 2024-09-14T23:58:48.895 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:48.895 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:48.895 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:48.895 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:48.895 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:48.895 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:48.895 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129159 2024-09-14T23:58:48.895 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.895 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:48.895 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.895 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:48.896 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.927783+0000", 2024-09-14T23:58:48.896 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:48.896 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.896 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.896 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:48.896 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.927783+0000", 2024-09-14T23:58:48.896 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:48.896 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.896 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.896 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:48.896 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929684+0000", 2024-09-14T23:58:48.897 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0019011080000000001 2024-09-14T23:58:48.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.897 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.897 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:48.897 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929688+0000", 2024-09-14T23:58:48.897 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2610000000000004e-06 2024-09-14T23:58:48.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.897 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.897 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:48.897 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929697+0000", 2024-09-14T23:58:48.898 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0219999999999993e-06 2024-09-14T23:58:48.898 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.898 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.898 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:48.898 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929702+0000", 2024-09-14T23:58:48.898 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8840000000000002e-06 2024-09-14T23:58:48.898 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.898 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.898 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:48.898 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.982090+0000", 2024-09-14T23:58:48.898 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052388045000000001 2024-09-14T23:58:48.899 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.899 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.899 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:48.899 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.982208+0000", 2024-09-14T23:58:48.899 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118236 2024-09-14T23:58:48.899 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.899 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.899 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:48.899 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.982444+0000", 2024-09-14T23:58:48.899 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000235567 2024-09-14T23:58:48.900 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:48.900 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:48.900 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:48.900 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.900 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.900 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129166 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 8192000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:48.900 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.929794+0000", 2024-09-14T23:58:48.900 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 69.953736289000005, 2024-09-14T23:58:48.900 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054471973, 2024-09-14T23:58:48.900 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:48.901 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:48.901 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:48.901 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:48.901 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:48.901 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:48.901 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129166 2024-09-14T23:58:48.901 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.901 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:48.901 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.901 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:48.901 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929794+0000", 2024-09-14T23:58:48.901 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:48.901 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.902 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.902 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:48.902 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929794+0000", 2024-09-14T23:58:48.902 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:48.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.902 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.902 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:48.902 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929794+0000", 2024-09-14T23:58:48.902 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6899999999999997e-07 2024-09-14T23:58:48.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.902 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.902 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:48.902 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929797+0000", 2024-09-14T23:58:48.902 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.328e-06 2024-09-14T23:58:48.903 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.903 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.903 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:48.903 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929802+0000", 2024-09-14T23:58:48.903 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7309999999999998e-06 2024-09-14T23:58:48.903 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.903 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.903 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:48.903 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929807+0000", 2024-09-14T23:58:48.903 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4349999999999999e-06 2024-09-14T23:58:48.903 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.903 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.903 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:48.904 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984063+0000", 2024-09-14T23:58:48.904 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054256331999999997 2024-09-14T23:58:48.904 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.904 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.904 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:48.904 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984177+0000", 2024-09-14T23:58:48.904 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114225 2024-09-14T23:58:48.904 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.904 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.904 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:48.904 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984266+0000", 2024-09-14T23:58:48.904 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8352999999999999e-05 2024-09-14T23:58:48.904 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:48.904 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:48.905 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:48.905 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.905 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.905 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129167 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 6782976~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:48.905 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.929827+0000", 2024-09-14T23:58:48.905 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 69.953702879999994, 2024-09-14T23:58:48.905 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054637593999999998, 2024-09-14T23:58:48.905 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:48.905 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:48.905 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:48.905 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:48.905 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:48.905 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:48.906 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129167 2024-09-14T23:58:48.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.906 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:48.906 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.906 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:48.906 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929827+0000", 2024-09-14T23:58:48.906 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:48.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.906 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.906 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:48.906 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929827+0000", 2024-09-14T23:58:48.906 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:48.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.906 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.907 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:48.907 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929827+0000", 2024-09-14T23:58:48.907 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4000000000000002e-07 2024-09-14T23:58:48.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.907 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.907 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:48.907 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929830+0000", 2024-09-14T23:58:48.907 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.419e-06 2024-09-14T23:58:48.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.907 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.907 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:48.907 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929837+0000", 2024-09-14T23:58:48.907 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0450000000000003e-06 2024-09-14T23:58:48.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.907 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.908 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:48.908 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929842+0000", 2024-09-14T23:58:48.908 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.69e-06 2024-09-14T23:58:48.908 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.908 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.908 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:48.908 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984278+0000", 2024-09-14T23:58:48.908 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054436516999999997 2024-09-14T23:58:48.908 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.908 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.908 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:48.908 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984331+0000", 2024-09-14T23:58:48.908 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2590999999999998e-05 2024-09-14T23:58:48.909 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.909 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.909 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:48.909 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984465+0000", 2024-09-14T23:58:48.909 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013389200000000001 2024-09-14T23:58:48.909 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:48.909 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:48.909 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:48.909 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.909 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.909 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129168 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 8224768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:48.909 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.929849+0000", 2024-09-14T23:58:48.909 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 69.953681321999994, 2024-09-14T23:58:48.910 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054870368000000003, 2024-09-14T23:58:48.910 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:48.910 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:48.910 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:48.910 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:48.910 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:48.910 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:48.910 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129168 2024-09-14T23:58:48.910 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.910 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:48.910 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.910 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:48.910 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929849+0000", 2024-09-14T23:58:48.910 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:48.910 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.911 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.911 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:48.911 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929849+0000", 2024-09-14T23:58:48.911 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:48.911 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.911 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.911 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:48.911 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931681+0000", 2024-09-14T23:58:48.911 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018326989999999999 2024-09-14T23:58:48.911 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.911 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.911 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:48.911 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931686+0000", 2024-09-14T23:58:48.911 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7190000000000001e-06 2024-09-14T23:58:48.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.912 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.912 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:48.912 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931697+0000", 2024-09-14T23:58:48.912 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1287999999999999e-05 2024-09-14T23:58:48.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.912 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.912 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:48.912 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931704+0000", 2024-09-14T23:58:48.912 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3339999999999998e-06 2024-09-14T23:58:48.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.912 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.912 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:48.913 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984505+0000", 2024-09-14T23:58:48.913 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052801684000000002 2024-09-14T23:58:48.913 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.913 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.913 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:48.913 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984621+0000", 2024-09-14T23:58:48.913 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000116165 2024-09-14T23:58:48.913 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.913 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.913 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:48.913 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984719+0000", 2024-09-14T23:58:48.913 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7479000000000006e-05 2024-09-14T23:58:48.913 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:48.914 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:48.914 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:48.914 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.914 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.914 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129176 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 10452992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:48.914 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.931830+0000", 2024-09-14T23:58:48.914 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 69.951699560999998, 2024-09-14T23:58:48.914 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054513292999999997, 2024-09-14T23:58:48.914 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:48.914 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:48.914 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:48.914 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:48.914 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:48.915 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:48.915 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129176 2024-09-14T23:58:48.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.915 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:48.915 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.915 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:48.915 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931830+0000", 2024-09-14T23:58:48.915 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:48.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.915 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.915 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:48.915 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931830+0000", 2024-09-14T23:58:48.916 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:48.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.916 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.916 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:48.916 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.933678+0000", 2024-09-14T23:58:48.916 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018472440000000001 2024-09-14T23:58:48.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.916 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.916 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:48.916 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.933686+0000", 2024-09-14T23:58:48.916 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0239999999999997e-06 2024-09-14T23:58:48.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.916 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.917 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:48.917 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.933694+0000", 2024-09-14T23:58:48.917 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8400000000000001e-06 2024-09-14T23:58:48.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.917 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.917 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:48.917 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.933701+0000", 2024-09-14T23:58:48.917 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2870000000000001e-06 2024-09-14T23:58:48.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.917 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.917 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:48.917 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.986165+0000", 2024-09-14T23:58:48.917 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052463900000000001 2024-09-14T23:58:48.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.918 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.918 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:48.918 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.986259+0000", 2024-09-14T23:58:48.918 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4435e-05 2024-09-14T23:58:48.918 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.918 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.918 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:48.918 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.986344+0000", 2024-09-14T23:58:48.918 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4562999999999997e-05 2024-09-14T23:58:48.918 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:48.918 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:48.918 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:48.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.919 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.919 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129201 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 10289152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:48.919 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.935960+0000", 2024-09-14T23:58:48.919 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 69.947570099000004, 2024-09-14T23:58:48.919 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054338487999999997, 2024-09-14T23:58:48.919 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:48.919 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:48.919 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:48.919 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:48.919 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:48.919 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:48.919 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129201 2024-09-14T23:58:48.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.920 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:48.920 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.920 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:48.920 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935960+0000", 2024-09-14T23:58:48.920 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:48.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.920 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.920 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:48.920 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935960+0000", 2024-09-14T23:58:48.920 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:48.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.920 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.920 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:48.920 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935960+0000", 2024-09-14T23:58:48.921 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.72e-07 2024-09-14T23:58:48.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.921 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.921 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:48.921 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935963+0000", 2024-09-14T23:58:48.921 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5390000000000002e-06 2024-09-14T23:58:48.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.921 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.921 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:48.921 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935971+0000", 2024-09-14T23:58:48.921 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4589999999999998e-06 2024-09-14T23:58:48.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.921 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.921 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:48.922 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935977+0000", 2024-09-14T23:58:48.922 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4110000000000004e-06 2024-09-14T23:58:48.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.922 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.922 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:48.922 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990213+0000", 2024-09-14T23:58:48.922 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054236642000000002 2024-09-14T23:58:48.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.922 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.922 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:48.922 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990257+0000", 2024-09-14T23:58:48.922 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3696000000000003e-05 2024-09-14T23:58:48.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.923 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.923 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:48.923 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990298+0000", 2024-09-14T23:58:48.923 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1369000000000003e-05 2024-09-14T23:58:48.923 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:48.923 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:48.923 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:48.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.923 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.923 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129202 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 11370496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:48.923 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.935986+0000", 2024-09-14T23:58:48.923 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 69.947543480999997, 2024-09-14T23:58:48.923 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054543700000000001, 2024-09-14T23:58:48.923 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:48.924 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:48.924 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:48.924 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:48.924 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:48.924 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:48.924 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129202 2024-09-14T23:58:48.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.924 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:48.924 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.924 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:48.924 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935986+0000", 2024-09-14T23:58:48.924 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:48.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.925 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.925 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:48.925 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935986+0000", 2024-09-14T23:58:48.925 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:48.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.925 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.925 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:48.925 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937678+0000", 2024-09-14T23:58:48.925 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016917849999999999 2024-09-14T23:58:48.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.925 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.925 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:48.925 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937682+0000", 2024-09-14T23:58:48.925 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2400000000000001e-06 2024-09-14T23:58:48.926 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.926 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.926 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:48.926 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937695+0000", 2024-09-14T23:58:48.926 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2999e-05 2024-09-14T23:58:48.926 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.926 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.926 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:48.926 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937700+0000", 2024-09-14T23:58:48.926 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8420000000000004e-06 2024-09-14T23:58:48.926 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.926 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.926 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:48.926 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990336+0000", 2024-09-14T23:58:48.927 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052636060999999998 2024-09-14T23:58:48.927 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.927 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.927 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:48.927 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990435+0000", 2024-09-14T23:58:48.927 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9015e-05 2024-09-14T23:58:48.927 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.927 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.927 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:48.927 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990530+0000", 2024-09-14T23:58:48.927 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4758000000000003e-05 2024-09-14T23:58:48.927 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:48.927 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:48.927 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:48.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.928 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.928 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129214 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 11468800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:48.928 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.937939+0000", 2024-09-14T23:58:48.928 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 69.945590566999996, 2024-09-14T23:58:48.928 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054463733, 2024-09-14T23:58:48.928 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:48.928 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:48.928 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:48.928 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:48.928 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:48.928 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:48.928 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129214 2024-09-14T23:58:48.929 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.929 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:48.929 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.929 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:48.929 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937939+0000", 2024-09-14T23:58:48.929 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:48.929 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.929 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.929 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:48.929 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937939+0000", 2024-09-14T23:58:48.929 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:48.929 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.929 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.930 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:48.930 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939674+0000", 2024-09-14T23:58:48.930 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017342589999999999 2024-09-14T23:58:48.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.930 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.930 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:48.930 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939678+0000", 2024-09-14T23:58:48.930 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2590000000000002e-06 2024-09-14T23:58:48.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.930 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.930 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:48.930 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939699+0000", 2024-09-14T23:58:48.930 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1143000000000001e-05 2024-09-14T23:58:48.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.930 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.931 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:48.931 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939706+0000", 2024-09-14T23:58:48.931 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0269999999999998e-06 2024-09-14T23:58:48.931 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.931 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.931 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:48.931 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.992221+0000", 2024-09-14T23:58:48.931 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052514981000000002 2024-09-14T23:58:48.931 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.931 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.931 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:48.931 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.992321+0000", 2024-09-14T23:58:48.931 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9625999999999998e-05 2024-09-14T23:58:48.932 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.932 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.932 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:48.932 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.992403+0000", 2024-09-14T23:58:48.932 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2437999999999999e-05 2024-09-14T23:58:48.932 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:48.932 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:48.932 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:48.932 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.932 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.932 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129226 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 14090240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:48.932 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.939867+0000", 2024-09-14T23:58:48.932 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 69.943662638999996, 2024-09-14T23:58:48.933 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054523573999999998, 2024-09-14T23:58:48.933 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:48.933 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:48.933 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:48.933 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:48.933 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:48.933 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:48.933 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129226 2024-09-14T23:58:48.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.933 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:48.933 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.933 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:48.933 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939867+0000", 2024-09-14T23:58:48.933 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:48.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.934 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.934 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:48.934 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939867+0000", 2024-09-14T23:58:48.934 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:48.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.934 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.934 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:48.934 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941669+0000", 2024-09-14T23:58:48.934 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018021549999999999 2024-09-14T23:58:48.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.934 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.934 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:48.934 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941673+0000", 2024-09-14T23:58:48.934 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1269999999999999e-06 2024-09-14T23:58:48.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.935 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:48.935 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941679+0000", 2024-09-14T23:58:48.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1879999999999999e-06 2024-09-14T23:58:48.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.935 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:48.935 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941682+0000", 2024-09-14T23:58:48.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7110000000000001e-06 2024-09-14T23:58:48.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.935 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:48.935 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.994201+0000", 2024-09-14T23:58:48.936 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052518265000000001 2024-09-14T23:58:48.936 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.936 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.936 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:48.936 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.994302+0000", 2024-09-14T23:58:48.936 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101653 2024-09-14T23:58:48.936 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.936 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.936 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:48.936 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.994391+0000", 2024-09-14T23:58:48.936 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8474999999999996e-05 2024-09-14T23:58:48.936 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:48.936 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:48.936 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:48.936 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.937 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.937 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129237 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 12386304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:48.937 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.941825+0000", 2024-09-14T23:58:48.937 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 69.941704877000006, 2024-09-14T23:58:48.937 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054368076000000001, 2024-09-14T23:58:48.937 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:48.937 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:48.937 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:48.937 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:48.937 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:48.937 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:48.937 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129237 2024-09-14T23:58:48.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.937 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:48.938 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.938 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:48.938 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941825+0000", 2024-09-14T23:58:48.938 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:48.938 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.938 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.938 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:48.938 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941825+0000", 2024-09-14T23:58:48.938 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:48.938 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.938 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.938 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:48.938 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941825+0000", 2024-09-14T23:58:48.938 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.29e-07 2024-09-14T23:58:48.938 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.938 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.939 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:48.939 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941826+0000", 2024-09-14T23:58:48.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0999999999999997e-07 2024-09-14T23:58:48.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.939 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:48.939 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941834+0000", 2024-09-14T23:58:48.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5519999999999999e-06 2024-09-14T23:58:48.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.939 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:48.939 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941843+0000", 2024-09-14T23:58:48.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2960000000000004e-06 2024-09-14T23:58:48.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:48.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996108+0000", 2024-09-14T23:58:48.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054265200999999999 2024-09-14T23:58:48.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:48.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996152+0000", 2024-09-14T23:58:48.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4066999999999997e-05 2024-09-14T23:58:48.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:48.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996193+0000", 2024-09-14T23:58:48.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0821000000000001e-05 2024-09-14T23:58:48.941 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:48.941 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:48.941 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:48.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.941 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129238 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 13828096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:48.941 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.941851+0000", 2024-09-14T23:58:48.941 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 69.941678553000003, 2024-09-14T23:58:48.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.05458031, 2024-09-14T23:58:48.941 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:48.941 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:48.941 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:48.941 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:48.941 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:48.941 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:48.942 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129238 2024-09-14T23:58:48.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.942 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:48.942 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.942 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:48.942 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941851+0000", 2024-09-14T23:58:48.942 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:48.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.942 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.942 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:48.942 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941851+0000", 2024-09-14T23:58:48.942 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:48.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.943 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:48.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943670+0000", 2024-09-14T23:58:48.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018181930000000001 2024-09-14T23:58:48.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.943 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:48.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943673+0000", 2024-09-14T23:58:48.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3790000000000002e-06 2024-09-14T23:58:48.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.943 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:48.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943679+0000", 2024-09-14T23:58:48.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8719999999999999e-06 2024-09-14T23:58:48.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:48.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943683+0000", 2024-09-14T23:58:48.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8539999999999999e-06 2024-09-14T23:58:48.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:48.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996233+0000", 2024-09-14T23:58:48.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052550690999999997 2024-09-14T23:58:48.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:48.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996339+0000", 2024-09-14T23:58:48.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105989 2024-09-14T23:58:48.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:48.945 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996432+0000", 2024-09-14T23:58:48.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2331999999999996e-05 2024-09-14T23:58:48.945 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:48.945 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:48.945 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:48.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.945 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129249 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 12124160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:48.945 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.943826+0000", 2024-09-14T23:58:48.945 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 69.939704062000004, 2024-09-14T23:58:48.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054373266000000003, 2024-09-14T23:58:48.946 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:48.946 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:48.946 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:48.946 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:48.946 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:48.946 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:48.946 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129249 2024-09-14T23:58:48.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.946 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:48.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.946 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:48.946 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943826+0000", 2024-09-14T23:58:48.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:48.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.946 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:48.947 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943826+0000", 2024-09-14T23:58:48.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:48.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.947 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:48.947 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943826+0000", 2024-09-14T23:58:48.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.03e-07 2024-09-14T23:58:48.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.947 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:48.947 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943827+0000", 2024-09-14T23:58:48.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.08e-06 2024-09-14T23:58:48.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.948 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:48.948 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943834+0000", 2024-09-14T23:58:48.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4119999999999998e-06 2024-09-14T23:58:48.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.948 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:48.948 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943836+0000", 2024-09-14T23:58:48.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7369999999999998e-06 2024-09-14T23:58:48.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.948 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:48.948 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998115+0000", 2024-09-14T23:58:48.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054279040000000001 2024-09-14T23:58:48.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:48.949 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998159+0000", 2024-09-14T23:58:48.949 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3486999999999999e-05 2024-09-14T23:58:48.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:48.949 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998199+0000", 2024-09-14T23:58:48.949 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0306999999999999e-05 2024-09-14T23:58:48.949 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:48.949 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:48.949 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:48.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.950 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129250 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 13565952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:48.950 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.943840+0000", 2024-09-14T23:58:48.950 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 69.939690111999994, 2024-09-14T23:58:48.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054573875000000001, 2024-09-14T23:58:48.950 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:48.950 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:48.950 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:48.950 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:48.950 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:48.950 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:48.950 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129250 2024-09-14T23:58:48.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.950 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:48.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.951 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:48.951 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943840+0000", 2024-09-14T23:58:48.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:48.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.951 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:48.951 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943840+0000", 2024-09-14T23:58:48.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:48.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.951 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:48.951 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945669+0000", 2024-09-14T23:58:48.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018289720000000001 2024-09-14T23:58:48.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.952 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:48.952 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945671+0000", 2024-09-14T23:58:48.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6869999999999999e-06 2024-09-14T23:58:48.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.952 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:48.952 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945680+0000", 2024-09-14T23:58:48.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5259999999999993e-06 2024-09-14T23:58:48.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.952 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:48.952 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945685+0000", 2024-09-14T23:58:48.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8559999999999998e-06 2024-09-14T23:58:48.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.953 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:48.953 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998242+0000", 2024-09-14T23:58:48.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052557505999999997 2024-09-14T23:58:48.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.953 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:48.953 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998329+0000", 2024-09-14T23:58:48.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6410999999999996e-05 2024-09-14T23:58:48.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.953 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:48.953 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998414+0000", 2024-09-14T23:58:48.954 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4917e-05 2024-09-14T23:58:48.954 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:48.954 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:48.954 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:48.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.954 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.954 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129262 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 13303808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:48.954 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.945825+0000", 2024-09-14T23:58:48.954 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 69.937705171000005, 2024-09-14T23:58:48.954 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054515942999999997, 2024-09-14T23:58:48.954 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:48.954 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:48.954 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:48.954 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:48.955 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:48.955 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:48.955 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129262 2024-09-14T23:58:48.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.955 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:48.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.955 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:48.955 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945825+0000", 2024-09-14T23:58:48.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:48.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.955 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:48.955 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945825+0000", 2024-09-14T23:58:48.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:48.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:48.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.947670+0000", 2024-09-14T23:58:48.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018451229999999999 2024-09-14T23:58:48.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:48.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.947672+0000", 2024-09-14T23:58:48.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5289999999999998e-06 2024-09-14T23:58:48.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:48.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.947678+0000", 2024-09-14T23:58:48.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9880000000000001e-06 2024-09-14T23:58:48.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:48.957 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.947682+0000", 2024-09-14T23:58:48.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.63e-06 2024-09-14T23:58:48.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:48.957 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000183+0000", 2024-09-14T23:58:48.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052500770000000002 2024-09-14T23:58:48.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.958 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:48.958 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000266+0000", 2024-09-14T23:58:48.958 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3133000000000005e-05 2024-09-14T23:58:48.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.958 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.958 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:48.958 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000341+0000", 2024-09-14T23:58:48.958 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4770000000000004e-05 2024-09-14T23:58:48.958 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:48.958 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:48.959 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:48.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.959 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.959 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129299 3.10 3:0f1e06ee:::rbd_object_map.1118721be852:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:58:48.959 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.952330+0000", 2024-09-14T23:58:48.959 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 69.931200357999998, 2024-09-14T23:58:48.959 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054412818000000002, 2024-09-14T23:58:48.959 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:48.959 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:48.959 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:48.959 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:48.960 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:48.960 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:48.960 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129299 2024-09-14T23:58:48.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.960 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:48.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.960 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:48.960 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952330+0000", 2024-09-14T23:58:48.960 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:48.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:48.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952330+0000", 2024-09-14T23:58:48.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:48.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:48.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952330+0000", 2024-09-14T23:58:48.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4900000000000001e-07 2024-09-14T23:58:48.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:48.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952332+0000", 2024-09-14T23:58:48.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2879999999999999e-06 2024-09-14T23:58:48.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.962 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:48.962 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952340+0000", 2024-09-14T23:58:48.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9910000000000002e-06 2024-09-14T23:58:48.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.962 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:48.962 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952344+0000", 2024-09-14T23:58:48.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0280000000000002e-06 2024-09-14T23:58:48.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.962 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:48.963 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.004722+0000", 2024-09-14T23:58:48.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052377932000000002 2024-09-14T23:58:48.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.963 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.963 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:48.963 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.004778+0000", 2024-09-14T23:58:48.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5795999999999999e-05 2024-09-14T23:58:48.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.963 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.963 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:58:48.963 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.005016+0000", 2024-09-14T23:58:48.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000238006 2024-09-14T23:58:48.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.964 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:58:48.964 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.006366+0000", 2024-09-14T23:58:48.964 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001349786 2024-09-14T23:58:48.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.964 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:48.964 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.006685+0000", 2024-09-14T23:58:48.964 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00031970000000000002 2024-09-14T23:58:48.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.964 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:48.964 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.006718+0000", 2024-09-14T23:58:48.965 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2329000000000002e-05 2024-09-14T23:58:48.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.965 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.965 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:48.965 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.006742+0000", 2024-09-14T23:58:48.965 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4613e-05 2024-09-14T23:58:48.965 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:48.965 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:48.965 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:48.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.965 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.965 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186210 3.d 3.b2ff3acd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:48.965 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.129044+0000", 2024-09-14T23:58:48.966 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 41.754485995000003, 2024-09-14T23:58:48.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.84555443900000005, 2024-09-14T23:58:48.966 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:48.966 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:48.966 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:58:48.966 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:48.966 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:48.966 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:48.966 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186210 2024-09-14T23:58:48.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.966 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:48.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:48.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129044+0000", 2024-09-14T23:58:48.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:48.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.967 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:48.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129044+0000", 2024-09-14T23:58:48.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:48.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.967 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:48.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129046+0000", 2024-09-14T23:58:48.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.742e-06 2024-09-14T23:58:48.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.968 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:48.968 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129052+0000", 2024-09-14T23:58:48.968 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7390000000000002e-06 2024-09-14T23:58:48.968 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.968 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.968 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:48.968 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129067+0000", 2024-09-14T23:58:48.968 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.417e-05 2024-09-14T23:58:48.968 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.968 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.968 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:48.969 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129076+0000", 2024-09-14T23:58:48.969 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2289999999999992e-06 2024-09-14T23:58:48.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.969 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.969 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:48.969 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.130947+0000", 2024-09-14T23:58:48.969 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001871075 2024-09-14T23:58:48.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.969 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.969 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:48.969 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.130955+0000", 2024-09-14T23:58:48.969 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4500000000000004e-06 2024-09-14T23:58:48.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.969 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.970 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:48.970 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974587+0000", 2024-09-14T23:58:48.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.84363205500000005 2024-09-14T23:58:48.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.970 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:48.970 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974598+0000", 2024-09-14T23:58:48.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0978999999999999e-05 2024-09-14T23:58:48.970 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:48.970 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:48.970 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:48.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.970 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187415 3.d 3.b2ff3acd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:48.970 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.254681+0000", 2024-09-14T23:58:48.971 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 41.628848474000002, 2024-09-14T23:58:48.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.71995530699999999, 2024-09-14T23:58:48.971 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:48.971 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:48.971 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:58:48.971 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:48.971 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:48.971 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:48.971 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187415 2024-09-14T23:58:48.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.971 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:48.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.971 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:48.971 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254681+0000", 2024-09-14T23:58:48.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:48.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.972 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.972 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:48.972 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254681+0000", 2024-09-14T23:58:48.972 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:48.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.972 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.972 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:48.972 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254682+0000", 2024-09-14T23:58:48.972 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2e-07 2024-09-14T23:58:48.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.972 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.972 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:48.972 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254684+0000", 2024-09-14T23:58:48.972 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2730000000000001e-06 2024-09-14T23:58:48.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.973 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:48.973 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254689+0000", 2024-09-14T23:58:48.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4769999999999997e-06 2024-09-14T23:58:48.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.973 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:48.973 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254692+0000", 2024-09-14T23:58:48.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9890000000000001e-06 2024-09-14T23:58:48.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.973 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:48.974 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254721+0000", 2024-09-14T23:58:48.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9762999999999999e-05 2024-09-14T23:58:48.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.974 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:48.974 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254725+0000", 2024-09-14T23:58:48.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9650000000000002e-06 2024-09-14T23:58:48.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.974 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:48.974 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974628+0000", 2024-09-14T23:58:48.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.71990235499999999 2024-09-14T23:58:48.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.975 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.975 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:48.975 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974637+0000", 2024-09-14T23:58:48.975 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0650000000000005e-06 2024-09-14T23:58:48.975 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:48.975 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:48.975 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:48.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.975 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.975 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187457 3.d 3.b2ff3acd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:48.975 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.257980+0000", 2024-09-14T23:58:48.975 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 41.625549900000003, 2024-09-14T23:58:48.975 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.71671711199999999, 2024-09-14T23:58:48.976 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:48.976 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:48.976 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:58:48.976 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:48.976 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:48.976 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:48.976 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187457 2024-09-14T23:58:48.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.976 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:48.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.976 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:48.976 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257980+0000", 2024-09-14T23:58:48.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:48.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.977 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:48.977 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257980+0000", 2024-09-14T23:58:48.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:48.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.977 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:48.977 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257980+0000", 2024-09-14T23:58:48.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.17e-07 2024-09-14T23:58:48.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.977 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:48.978 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257981+0000", 2024-09-14T23:58:48.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.172e-06 2024-09-14T23:58:48.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.978 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:48.978 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257984+0000", 2024-09-14T23:58:48.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9550000000000001e-06 2024-09-14T23:58:48.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.978 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:48.978 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257987+0000", 2024-09-14T23:58:48.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5799999999999999e-06 2024-09-14T23:58:48.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.979 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.979 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:48.979 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.258030+0000", 2024-09-14T23:58:48.979 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3118999999999998e-05 2024-09-14T23:58:48.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.979 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.979 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:48.979 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.258038+0000", 2024-09-14T23:58:48.979 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6850000000000001e-06 2024-09-14T23:58:48.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.979 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.979 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:48.979 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974688+0000", 2024-09-14T23:58:48.980 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.71665016000000004 2024-09-14T23:58:48.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.980 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:48.980 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974697+0000", 2024-09-14T23:58:48.980 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2240000000000001e-06 2024-09-14T23:58:48.980 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:48.980 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:48.980 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:48.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.980 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:251656 3.c 3:32436721:::rbd_object_map.1118e09062ff:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-14T23:58:48.980 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:27.433419+0000", 2024-09-14T23:58:48.981 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 21.450111226000001, 2024-09-14T23:58:48.981 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.55083879499999999, 2024-09-14T23:58:48.981 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:48.981 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:48.981 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:48.981 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:48.981 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:48.981 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:48.981 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 251656 2024-09-14T23:58:48.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.981 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:48.981 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.981 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:48.981 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433419+0000", 2024-09-14T23:58:48.981 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:48.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.982 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:48.982 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433419+0000", 2024-09-14T23:58:48.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:48.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.982 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:48.982 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433419+0000", 2024-09-14T23:58:48.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1500000000000003e-07 2024-09-14T23:58:48.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.982 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:48.983 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433424+0000", 2024-09-14T23:58:48.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2409999999999997e-06 2024-09-14T23:58:48.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.983 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.983 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:48.983 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433435+0000", 2024-09-14T23:58:48.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1009000000000001e-05 2024-09-14T23:58:48.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.983 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.983 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:48.983 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433440+0000", 2024-09-14T23:58:48.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1950000000000002e-06 2024-09-14T23:58:48.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.983 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.984 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:48.984 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433487+0000", 2024-09-14T23:58:48.984 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7184000000000001e-05 2024-09-14T23:58:48.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.984 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:48.984 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433494+0000", 2024-09-14T23:58:48.984 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5490000000000003e-06 2024-09-14T23:58:48.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.984 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:48.984 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.981923+0000", 2024-09-14T23:58:48.984 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.54842898600000001 2024-09-14T23:58:48.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.985 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:48.985 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.982097+0000", 2024-09-14T23:58:48.985 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017428699999999999 2024-09-14T23:58:48.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.985 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.985 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5", 2024-09-14T23:58:48.985 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.982334+0000", 2024-09-14T23:58:48.985 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00023700899999999999 2024-09-14T23:58:48.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.985 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.985 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:48.985 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.983927+0000", 2024-09-14T23:58:48.985 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0015932710000000001 2024-09-14T23:58:48.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.986 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:58:48.986 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.984128+0000", 2024-09-14T23:58:48.986 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020104199999999999 2024-09-14T23:58:48.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.986 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:48.986 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.984212+0000", 2024-09-14T23:58:48.986 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3709999999999996e-05 2024-09-14T23:58:48.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:48.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:48.986 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:48.986 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.984257+0000", 2024-09-14T23:58:48.986 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5497000000000002e-05 2024-09-14T23:58:48.987 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:48.987 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:48.987 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:48.987 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:48.987 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:48.987 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:48.988 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:58:48.989 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T23:58:48.989 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:48.987+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:48.990 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:58:48.990 INFO:teuthology.orchestra.run.smithi042.stderr:marked in osd.7. 2024-09-14T23:58:48.997 INFO:tasks.thrashosds.thrasher:Added osd 7 2024-09-14T23:58:49.066 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T23:58:49.110 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:49.109+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:49.137 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:49.137 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:58:49.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.137 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:255885 3.13 3.54329f53 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-14T23:58:49.137 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:48.978834+0000", 2024-09-14T23:58:49.137 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.15744878700000001, 2024-09-14T23:58:49.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.15747040000000001, 2024-09-14T23:58:49.138 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:49.138 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:49.138 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "waiting for peered", 2024-09-14T23:58:49.138 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:49.138 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:49.138 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:49.138 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 255885 2024-09-14T23:58:49.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.138 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:49.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.139 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:49.140 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.978834+0000", 2024-09-14T23:58:49.140 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:49.140 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.140 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.140 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:49.140 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.978834+0000", 2024-09-14T23:58:49.140 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:49.140 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.140 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.140 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:49.140 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.978836+0000", 2024-09-14T23:58:49.141 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.288e-06 2024-09-14T23:58:49.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.141 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:49.141 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.978844+0000", 2024-09-14T23:58:49.141 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9880000000000004e-06 2024-09-14T23:58:49.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.141 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:49.141 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.978861+0000", 2024-09-14T23:58:49.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7020999999999998e-05 2024-09-14T23:58:49.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.142 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:49.142 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.978872+0000", 2024-09-14T23:58:49.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1274000000000001e-05 2024-09-14T23:58:49.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.142 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:49.142 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.982358+0000", 2024-09-14T23:58:49.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003486393 2024-09-14T23:58:49.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:49.143 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.982368+0000", 2024-09-14T23:58:49.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0690000000000008e-06 2024-09-14T23:58:49.143 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:49.143 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:49.143 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:49.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.144 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:255886 3.13 3.54329f53 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-14T23:58:49.144 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:48.981728+0000", 2024-09-14T23:58:49.144 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.15455486900000001, 2024-09-14T23:58:49.144 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.154618741, 2024-09-14T23:58:49.144 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:49.144 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:49.144 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "waiting for peered", 2024-09-14T23:58:49.144 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:49.144 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:49.144 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:49.145 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 255886 2024-09-14T23:58:49.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.145 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:49.145 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.145 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:49.145 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.981728+0000", 2024-09-14T23:58:49.145 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:49.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.145 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.145 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:49.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.981728+0000", 2024-09-14T23:58:49.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:49.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.146 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:49.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.981730+0000", 2024-09-14T23:58:49.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5179999999999999e-06 2024-09-14T23:58:49.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.146 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:49.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.981737+0000", 2024-09-14T23:58:49.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.956e-06 2024-09-14T23:58:49.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.147 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:49.147 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.981748+0000", 2024-09-14T23:58:49.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0754000000000001e-05 2024-09-14T23:58:49.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.147 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:49.147 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.981757+0000", 2024-09-14T23:58:49.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8850000000000005e-06 2024-09-14T23:58:49.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.147 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:49.147 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.983225+0000", 2024-09-14T23:58:49.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001468605 2024-09-14T23:58:49.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.148 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.148 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:49.148 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.983232+0000", 2024-09-14T23:58:49.148 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0090000000000001e-06 2024-09-14T23:58:49.148 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:49.148 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:49.148 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:49.148 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:49.148 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-14T23:58:49.148 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 2 2024-09-14T23:58:49.148 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:49.149 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T23:58:49.190 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:49.188+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:49.257 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:49.257 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:58:49.257 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:58:49.257 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:58:49.257 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:49.265 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T23:58:49.291 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:49.289+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:49.374 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:49.374 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:58:49.374 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:58:49.374 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:58:49.374 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.374 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69262 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 4800512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:49.375 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300811+0000", 2024-09-14T23:58:49.375 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 94.071122681000006, 2024-09-14T23:58:49.375 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050403796000000001, 2024-09-14T23:58:49.375 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:49.375 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:49.375 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:49.375 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:49.375 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:49.375 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:49.375 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69262 2024-09-14T23:58:49.375 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.375 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:49.375 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.375 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:49.375 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300811+0000", 2024-09-14T23:58:49.376 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:49.376 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.376 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.376 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:49.376 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300811+0000", 2024-09-14T23:58:49.376 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:49.376 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.376 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.376 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:49.376 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300811+0000", 2024-09-14T23:58:49.376 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5700000000000002e-07 2024-09-14T23:58:49.376 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.376 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.377 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:49.377 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300815+0000", 2024-09-14T23:58:49.377 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.087e-06 2024-09-14T23:58:49.377 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.377 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.377 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:49.377 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300823+0000", 2024-09-14T23:58:49.377 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8720000000000002e-06 2024-09-14T23:58:49.377 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.377 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.377 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:49.377 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300830+0000", 2024-09-14T23:58:49.377 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0589999999999997e-06 2024-09-14T23:58:49.378 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.378 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.378 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:49.378 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351020+0000", 2024-09-14T23:58:49.378 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050189552999999998 2024-09-14T23:58:49.378 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.378 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.378 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:49.378 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351112+0000", 2024-09-14T23:58:49.378 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2615999999999996e-05 2024-09-14T23:58:49.378 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.378 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.378 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:49.378 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351214+0000", 2024-09-14T23:58:49.379 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101952 2024-09-14T23:58:49.379 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:49.379 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:49.379 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:49.379 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.379 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.379 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69263 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 4915200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:49.379 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300842+0000", 2024-09-14T23:58:49.379 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 94.071091177, 2024-09-14T23:58:49.379 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050590637000000001, 2024-09-14T23:58:49.379 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:49.379 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:49.379 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:49.380 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:49.380 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:49.380 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:49.380 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69263 2024-09-14T23:58:49.380 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.380 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:49.380 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.380 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:49.380 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300842+0000", 2024-09-14T23:58:49.380 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:49.380 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.380 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.380 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:49.381 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300842+0000", 2024-09-14T23:58:49.381 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:49.381 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.381 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.381 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:49.381 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300843+0000", 2024-09-14T23:58:49.381 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4000000000000002e-07 2024-09-14T23:58:49.381 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.381 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.381 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:49.381 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300846+0000", 2024-09-14T23:58:49.381 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8720000000000002e-06 2024-09-14T23:58:49.381 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.382 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.382 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:49.382 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300853+0000", 2024-09-14T23:58:49.382 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0450000000000003e-06 2024-09-14T23:58:49.382 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.382 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.382 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:49.382 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300859+0000", 2024-09-14T23:58:49.382 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9510000000000003e-06 2024-09-14T23:58:49.382 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.382 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.382 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:49.382 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351260+0000", 2024-09-14T23:58:49.383 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050401654999999997 2024-09-14T23:58:49.383 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.383 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.383 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:49.383 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351353+0000", 2024-09-14T23:58:49.383 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2785e-05 2024-09-14T23:58:49.383 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.383 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.383 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:49.383 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351433+0000", 2024-09-14T23:58:49.383 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9789000000000005e-05 2024-09-14T23:58:49.383 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:49.383 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:49.383 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:49.384 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.384 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.384 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69264 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5029888~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:49.384 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300866+0000", 2024-09-14T23:58:49.384 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 94.071067244000005, 2024-09-14T23:58:49.384 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050794063, 2024-09-14T23:58:49.384 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:49.384 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:49.384 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:49.384 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:49.384 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:49.384 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:49.384 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69264 2024-09-14T23:58:49.385 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.385 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:49.385 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.385 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:49.385 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300866+0000", 2024-09-14T23:58:49.385 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:49.385 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.385 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.385 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:49.385 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300866+0000", 2024-09-14T23:58:49.385 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:49.385 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.385 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.386 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:49.386 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300866+0000", 2024-09-14T23:58:49.386 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8799999999999998e-07 2024-09-14T23:58:49.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.386 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.386 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:49.386 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300869+0000", 2024-09-14T23:58:49.386 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0880000000000002e-06 2024-09-14T23:58:49.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.386 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.386 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:49.386 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300875+0000", 2024-09-14T23:58:49.386 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9280000000000004e-06 2024-09-14T23:58:49.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.387 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.387 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:49.387 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300882+0000", 2024-09-14T23:58:49.387 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4740000000000002e-06 2024-09-14T23:58:49.387 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.387 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.387 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:49.387 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351472+0000", 2024-09-14T23:58:49.387 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050590006999999999 2024-09-14T23:58:49.387 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.387 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.387 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:49.387 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351563+0000", 2024-09-14T23:58:49.387 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0762999999999998e-05 2024-09-14T23:58:49.388 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.388 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.388 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:49.388 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351660+0000", 2024-09-14T23:58:49.388 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7415000000000002e-05 2024-09-14T23:58:49.388 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:49.388 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:49.388 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:49.388 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.388 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.388 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69265 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5144576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:49.388 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300890+0000", 2024-09-14T23:58:49.388 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 94.071043739999993, 2024-09-14T23:58:49.389 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050921389999999997, 2024-09-14T23:58:49.389 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:49.389 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:49.389 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:49.389 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:49.389 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:49.389 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:49.389 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69265 2024-09-14T23:58:49.389 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.389 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:49.389 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.389 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:49.389 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300890+0000", 2024-09-14T23:58:49.390 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:49.390 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.390 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.390 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:49.390 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300890+0000", 2024-09-14T23:58:49.390 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:49.390 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.390 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.390 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:49.390 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300890+0000", 2024-09-14T23:58:49.390 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2500000000000001e-07 2024-09-14T23:58:49.390 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.390 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.390 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:49.391 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300892+0000", 2024-09-14T23:58:49.391 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2469999999999998e-06 2024-09-14T23:58:49.391 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.391 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.391 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:49.391 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300899+0000", 2024-09-14T23:58:49.391 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5509999999999996e-06 2024-09-14T23:58:49.391 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.391 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.391 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:49.391 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300904+0000", 2024-09-14T23:58:49.391 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3299999999999998e-06 2024-09-14T23:58:49.391 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.392 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.392 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:49.392 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351707+0000", 2024-09-14T23:58:49.392 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050802673999999999 2024-09-14T23:58:49.392 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.392 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.392 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:49.392 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351755+0000", 2024-09-14T23:58:49.392 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8362000000000002e-05 2024-09-14T23:58:49.392 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.392 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.392 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:49.393 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351811+0000", 2024-09-14T23:58:49.393 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5801000000000003e-05 2024-09-14T23:58:49.393 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:49.393 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:49.393 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:49.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.393 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.393 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69266 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5259264~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:49.393 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300911+0000", 2024-09-14T23:58:49.393 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 94.071022666999994, 2024-09-14T23:58:49.393 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051125089999999998, 2024-09-14T23:58:49.393 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:49.393 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:49.393 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:49.394 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:49.394 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:49.394 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:49.394 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69266 2024-09-14T23:58:49.394 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.394 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:49.394 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.394 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:49.394 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300911+0000", 2024-09-14T23:58:49.394 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:49.394 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.394 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.394 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:49.394 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300911+0000", 2024-09-14T23:58:49.394 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:49.395 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.395 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.395 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:49.395 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300911+0000", 2024-09-14T23:58:49.395 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-14T23:58:49.395 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.395 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.395 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:49.395 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300913+0000", 2024-09-14T23:58:49.395 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9300000000000002e-06 2024-09-14T23:58:49.395 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.395 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.395 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:49.395 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300919+0000", 2024-09-14T23:58:49.396 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.489e-06 2024-09-14T23:58:49.396 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.396 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.396 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:49.396 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300925+0000", 2024-09-14T23:58:49.396 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1719999999999998e-06 2024-09-14T23:58:49.396 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.396 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.396 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:49.396 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351847+0000", 2024-09-14T23:58:49.396 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050922621000000001 2024-09-14T23:58:49.396 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.396 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.396 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:49.396 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351938+0000", 2024-09-14T23:58:49.396 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0830999999999998e-05 2024-09-14T23:58:49.397 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.397 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.397 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:49.397 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352036+0000", 2024-09-14T23:58:49.397 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7676999999999993e-05 2024-09-14T23:58:49.397 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:49.397 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:49.397 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:49.397 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.397 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.397 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69267 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5373952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:49.397 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300945+0000", 2024-09-14T23:58:49.397 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 94.070987872000003, 2024-09-14T23:58:49.397 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051217934, 2024-09-14T23:58:49.398 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:49.398 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:49.398 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:49.398 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:49.398 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:49.398 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:49.398 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69267 2024-09-14T23:58:49.398 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.398 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:49.398 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.398 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:49.398 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300945+0000", 2024-09-14T23:58:49.398 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:49.398 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.399 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.399 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:49.399 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300945+0000", 2024-09-14T23:58:49.399 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:49.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.399 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.399 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:49.399 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300946+0000", 2024-09-14T23:58:49.399 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2600000000000002e-07 2024-09-14T23:58:49.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.399 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.399 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:49.399 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300948+0000", 2024-09-14T23:58:49.399 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4940000000000002e-06 2024-09-14T23:58:49.400 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.400 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.400 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:49.400 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300956+0000", 2024-09-14T23:58:49.400 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.622e-06 2024-09-14T23:58:49.400 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.400 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.400 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:49.400 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300962+0000", 2024-09-14T23:58:49.400 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5790000000000003e-06 2024-09-14T23:58:49.400 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.400 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.400 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:49.400 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352063+0000", 2024-09-14T23:58:49.401 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051101281999999998 2024-09-14T23:58:49.401 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.401 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.401 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:49.401 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352110+0000", 2024-09-14T23:58:49.401 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7401000000000002e-05 2024-09-14T23:58:49.401 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.401 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.401 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:49.401 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352163+0000", 2024-09-14T23:58:49.401 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3029999999999999e-05 2024-09-14T23:58:49.401 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:49.401 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:49.401 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:49.401 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.402 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.402 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69268 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5488640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:49.402 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300969+0000", 2024-09-14T23:58:49.402 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 94.070964200000006, 2024-09-14T23:58:49.402 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051413420000000001, 2024-09-14T23:58:49.402 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:49.402 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:49.402 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:49.402 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:49.402 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:49.402 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:49.402 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69268 2024-09-14T23:58:49.402 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.403 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:49.403 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.403 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:49.403 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300969+0000", 2024-09-14T23:58:49.403 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:49.403 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.403 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.403 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:49.403 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300969+0000", 2024-09-14T23:58:49.403 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:49.403 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.403 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.403 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:49.403 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300970+0000", 2024-09-14T23:58:49.404 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7899999999999999e-07 2024-09-14T23:58:49.404 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.404 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.404 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:49.404 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300982+0000", 2024-09-14T23:58:49.404 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2741e-05 2024-09-14T23:58:49.404 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.404 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.404 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:49.404 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300990+0000", 2024-09-14T23:58:49.404 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8169999999999999e-06 2024-09-14T23:58:49.404 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.404 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.405 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:49.405 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300995+0000", 2024-09-14T23:58:49.405 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2240000000000003e-06 2024-09-14T23:58:49.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.405 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.405 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:49.405 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352200+0000", 2024-09-14T23:58:49.405 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051204223 2024-09-14T23:58:49.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.405 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.405 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:49.405 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352285+0000", 2024-09-14T23:58:49.405 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5519999999999994e-05 2024-09-14T23:58:49.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.406 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:49.406 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352383+0000", 2024-09-14T23:58:49.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7516000000000006e-05 2024-09-14T23:58:49.406 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:49.406 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:49.406 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:49.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.406 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69269 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5603328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:49.406 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.301003+0000", 2024-09-14T23:58:49.406 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 94.070930017999999, 2024-09-14T23:58:49.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051497381000000002, 2024-09-14T23:58:49.406 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:49.407 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:49.407 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:49.407 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:49.407 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:49.407 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:49.407 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69269 2024-09-14T23:58:49.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.407 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:49.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.407 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:49.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.301003+0000", 2024-09-14T23:58:49.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:49.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.408 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.408 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:49.408 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.301003+0000", 2024-09-14T23:58:49.408 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:49.408 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.408 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.408 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:49.408 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302671+0000", 2024-09-14T23:58:49.408 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001667785 2024-09-14T23:58:49.408 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.408 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.408 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:49.408 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302676+0000", 2024-09-14T23:58:49.408 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6600000000000003e-06 2024-09-14T23:58:49.409 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.409 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.409 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:49.409 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302697+0000", 2024-09-14T23:58:49.409 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1288999999999999e-05 2024-09-14T23:58:49.409 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.409 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.409 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:49.409 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302716+0000", 2024-09-14T23:58:49.409 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9128999999999999e-05 2024-09-14T23:58:49.409 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.409 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.409 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:49.410 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352399+0000", 2024-09-14T23:58:49.410 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049682995000000001 2024-09-14T23:58:49.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.410 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.410 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:49.410 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352446+0000", 2024-09-14T23:58:49.410 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6341000000000002e-05 2024-09-14T23:58:49.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.410 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.410 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:49.410 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352501+0000", 2024-09-14T23:58:49.410 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5182000000000001e-05 2024-09-14T23:58:49.410 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:49.410 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:49.411 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:49.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.411 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.411 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69273 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6062080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:49.411 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302829+0000", 2024-09-14T23:58:49.411 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 94.069104452000005, 2024-09-14T23:58:49.411 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050384311000000001, 2024-09-14T23:58:49.411 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:49.411 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:49.411 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:49.411 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:49.411 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:49.411 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:49.411 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69273 2024-09-14T23:58:49.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.412 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:49.412 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.412 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:49.412 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302829+0000", 2024-09-14T23:58:49.412 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:49.412 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.412 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.412 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:49.412 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302829+0000", 2024-09-14T23:58:49.412 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:49.412 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.412 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.412 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:49.413 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302829+0000", 2024-09-14T23:58:49.413 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7700000000000005e-07 2024-09-14T23:58:49.413 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.413 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.413 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:49.413 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302832+0000", 2024-09-14T23:58:49.413 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6970000000000002e-06 2024-09-14T23:58:49.413 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.413 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.413 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:49.413 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302840+0000", 2024-09-14T23:58:49.413 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5719999999999997e-06 2024-09-14T23:58:49.413 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.413 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.414 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:49.414 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302845+0000", 2024-09-14T23:58:49.414 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7939999999999999e-06 2024-09-14T23:58:49.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.414 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.414 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:49.414 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353111+0000", 2024-09-14T23:58:49.414 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050265588 2024-09-14T23:58:49.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.414 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.414 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:49.414 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353160+0000", 2024-09-14T23:58:49.414 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8859000000000001e-05 2024-09-14T23:58:49.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.415 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.415 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:49.415 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353213+0000", 2024-09-14T23:58:49.415 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3324e-05 2024-09-14T23:58:49.415 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:49.415 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:49.415 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:49.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.415 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.415 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69274 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6176768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:49.415 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302854+0000", 2024-09-14T23:58:49.415 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 94.069079357999996, 2024-09-14T23:58:49.415 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050579526, 2024-09-14T23:58:49.416 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:49.416 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:49.416 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:49.416 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:49.416 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:49.416 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:49.416 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69274 2024-09-14T23:58:49.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.416 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:49.416 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.416 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:49.416 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302854+0000", 2024-09-14T23:58:49.416 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:49.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.417 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.417 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:49.417 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302854+0000", 2024-09-14T23:58:49.417 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:49.417 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.417 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.417 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:49.417 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302855+0000", 2024-09-14T23:58:49.417 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0200000000000001e-07 2024-09-14T23:58:49.417 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.417 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.417 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:49.417 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302857+0000", 2024-09-14T23:58:49.418 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.447e-06 2024-09-14T23:58:49.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.418 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:49.418 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302864+0000", 2024-09-14T23:58:49.418 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7730000000000002e-06 2024-09-14T23:58:49.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.418 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:49.418 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302869+0000", 2024-09-14T23:58:49.418 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3199999999999999e-06 2024-09-14T23:58:49.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.418 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:49.419 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353249+0000", 2024-09-14T23:58:49.419 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050379647999999999 2024-09-14T23:58:49.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.419 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.419 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:49.419 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353336+0000", 2024-09-14T23:58:49.419 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6767999999999993e-05 2024-09-14T23:58:49.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.419 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.419 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:49.419 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353433+0000", 2024-09-14T23:58:49.419 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7868000000000006e-05 2024-09-14T23:58:49.419 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:49.419 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:49.420 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:49.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.420 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.420 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69275 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6291456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:49.420 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302889+0000", 2024-09-14T23:58:49.420 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 94.069044739999995, 2024-09-14T23:58:49.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050663118, 2024-09-14T23:58:49.420 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:49.420 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:49.420 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:49.420 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:49.420 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:49.420 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:49.421 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69275 2024-09-14T23:58:49.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.421 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:49.421 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.421 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:49.421 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302889+0000", 2024-09-14T23:58:49.421 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:49.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.421 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.421 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:49.421 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302889+0000", 2024-09-14T23:58:49.421 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:49.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.421 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.422 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:49.422 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302889+0000", 2024-09-14T23:58:49.422 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6399999999999998e-07 2024-09-14T23:58:49.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.422 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.422 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:49.422 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302891+0000", 2024-09-14T23:58:49.422 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.339e-06 2024-09-14T23:58:49.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.422 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.422 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:49.422 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302898+0000", 2024-09-14T23:58:49.422 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0380000000000002e-06 2024-09-14T23:58:49.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.423 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:49.423 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302904+0000", 2024-09-14T23:58:49.423 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2159999999999997e-06 2024-09-14T23:58:49.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.423 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:49.423 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353449+0000", 2024-09-14T23:58:49.423 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050545386999999997 2024-09-14T23:58:49.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.423 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:49.423 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353494+0000", 2024-09-14T23:58:49.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4592999999999998e-05 2024-09-14T23:58:49.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:49.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353552+0000", 2024-09-14T23:58:49.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8180999999999997e-05 2024-09-14T23:58:49.424 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:49.424 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:49.424 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:49.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.424 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69276 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6406144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:49.424 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302911+0000", 2024-09-14T23:58:49.424 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 94.069022326999999, 2024-09-14T23:58:49.425 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050865753, 2024-09-14T23:58:49.425 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:49.425 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:49.425 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:49.425 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:49.425 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:49.425 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:49.425 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69276 2024-09-14T23:58:49.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.425 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:49.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.425 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:49.425 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302911+0000", 2024-09-14T23:58:49.426 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:49.426 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.426 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.426 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:49.426 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302911+0000", 2024-09-14T23:58:49.426 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:49.426 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.426 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.426 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:49.426 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302912+0000", 2024-09-14T23:58:49.426 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0800000000000004e-07 2024-09-14T23:58:49.426 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.426 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.426 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:49.427 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302914+0000", 2024-09-14T23:58:49.427 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9589999999999998e-06 2024-09-14T23:58:49.427 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.427 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.427 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:49.427 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302920+0000", 2024-09-14T23:58:49.427 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4579999999999998e-06 2024-09-14T23:58:49.427 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.427 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.427 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:49.427 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302925+0000", 2024-09-14T23:58:49.427 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9429999999999999e-06 2024-09-14T23:58:49.427 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.427 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.428 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:49.428 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353586+0000", 2024-09-14T23:58:49.428 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050660966000000002 2024-09-14T23:58:49.428 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.428 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.428 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:49.428 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353679+0000", 2024-09-14T23:58:49.428 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3163000000000002e-05 2024-09-14T23:58:49.428 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.428 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.428 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:49.428 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353777+0000", 2024-09-14T23:58:49.428 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7555999999999999e-05 2024-09-14T23:58:49.428 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:49.429 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:49.429 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:49.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.429 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69277 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6520832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:49.429 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302932+0000", 2024-09-14T23:58:49.429 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 94.069001240000006, 2024-09-14T23:58:49.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050977674000000001, 2024-09-14T23:58:49.429 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:49.429 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:49.429 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:49.429 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:49.429 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:49.429 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:49.430 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69277 2024-09-14T23:58:49.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.430 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:49.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.430 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:49.430 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302932+0000", 2024-09-14T23:58:49.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:49.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.430 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:49.430 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302932+0000", 2024-09-14T23:58:49.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:49.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.431 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:49.431 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302932+0000", 2024-09-14T23:58:49.431 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3299999999999998e-07 2024-09-14T23:58:49.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.431 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.431 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:49.431 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302935+0000", 2024-09-14T23:58:49.431 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2139999999999999e-06 2024-09-14T23:58:49.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.431 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.431 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:49.431 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302941+0000", 2024-09-14T23:58:49.431 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5039999999999999e-06 2024-09-14T23:58:49.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.432 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:49.432 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302946+0000", 2024-09-14T23:58:49.432 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2079999999999999e-06 2024-09-14T23:58:49.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.432 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:49.432 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353805+0000", 2024-09-14T23:58:49.432 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050858588000000003 2024-09-14T23:58:49.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.432 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:49.433 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353849+0000", 2024-09-14T23:58:49.433 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4511000000000001e-05 2024-09-14T23:58:49.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.433 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:49.433 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353910+0000", 2024-09-14T23:58:49.433 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0316000000000003e-05 2024-09-14T23:58:49.433 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:49.433 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:49.433 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:49.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.433 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69278 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6635520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:49.433 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302953+0000", 2024-09-14T23:58:49.434 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 94.068980112000006, 2024-09-14T23:58:49.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051174059000000001, 2024-09-14T23:58:49.434 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:49.434 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:49.434 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:49.434 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:49.434 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:49.434 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:49.434 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69278 2024-09-14T23:58:49.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.434 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:49.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.434 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:49.434 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302953+0000", 2024-09-14T23:58:49.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:49.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.435 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:49.435 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302953+0000", 2024-09-14T23:58:49.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:49.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.435 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:49.435 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302954+0000", 2024-09-14T23:58:49.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5100000000000001e-07 2024-09-14T23:58:49.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.435 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:49.435 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302964+0000", 2024-09-14T23:58:49.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0475000000000001e-05 2024-09-14T23:58:49.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.436 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:49.436 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302970+0000", 2024-09-14T23:58:49.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3879999999999997e-06 2024-09-14T23:58:49.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.436 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:49.436 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302975+0000", 2024-09-14T23:58:49.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8409999999999999e-06 2024-09-14T23:58:49.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.436 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:49.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353944+0000", 2024-09-14T23:58:49.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050968952999999997 2024-09-14T23:58:49.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.437 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:49.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354031+0000", 2024-09-14T23:58:49.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6537000000000002e-05 2024-09-14T23:58:49.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.437 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:49.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354127+0000", 2024-09-14T23:58:49.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6514000000000005e-05 2024-09-14T23:58:49.437 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:49.437 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:49.438 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:49.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.438 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69279 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6750208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:49.438 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302984+0000", 2024-09-14T23:58:49.438 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 94.068949760999999, 2024-09-14T23:58:49.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051258845999999997, 2024-09-14T23:58:49.438 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:49.438 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:49.438 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:49.439 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:49.439 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:49.439 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:49.439 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69279 2024-09-14T23:58:49.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.439 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:49.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.439 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:49.439 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302984+0000", 2024-09-14T23:58:49.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:49.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.439 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:49.440 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302984+0000", 2024-09-14T23:58:49.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:49.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.440 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:49.440 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303025+0000", 2024-09-14T23:58:49.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1690999999999998e-05 2024-09-14T23:58:49.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.440 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:49.440 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303029+0000", 2024-09-14T23:58:49.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6559999999999998e-06 2024-09-14T23:58:49.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:49.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303037+0000", 2024-09-14T23:58:49.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7950000000000008e-06 2024-09-14T23:58:49.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:49.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303043+0000", 2024-09-14T23:58:49.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9669999999999998e-06 2024-09-14T23:58:49.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:49.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354142+0000", 2024-09-14T23:58:49.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051099072000000002 2024-09-14T23:58:49.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:49.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354183+0000", 2024-09-14T23:58:49.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0834000000000002e-05 2024-09-14T23:58:49.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:49.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354242+0000", 2024-09-14T23:58:49.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9830999999999997e-05 2024-09-14T23:58:49.442 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:49.442 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:49.443 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:49.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.443 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69280 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6864896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:49.443 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.303052+0000", 2024-09-14T23:58:49.443 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 94.068881218000001, 2024-09-14T23:58:49.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051406463999999999, 2024-09-14T23:58:49.443 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:49.443 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:49.443 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:49.443 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:49.443 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:49.443 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:49.444 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69280 2024-09-14T23:58:49.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.444 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:49.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:49.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303052+0000", 2024-09-14T23:58:49.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:49.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:49.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303052+0000", 2024-09-14T23:58:49.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:49.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:49.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304673+0000", 2024-09-14T23:58:49.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001621389 2024-09-14T23:58:49.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:49.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304681+0000", 2024-09-14T23:58:49.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8439999999999998e-06 2024-09-14T23:58:49.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:49.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304691+0000", 2024-09-14T23:58:49.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8339999999999994e-06 2024-09-14T23:58:49.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.446 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:49.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304698+0000", 2024-09-14T23:58:49.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3629999999999999e-06 2024-09-14T23:58:49.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.446 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:49.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354278+0000", 2024-09-14T23:58:49.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049580618 2024-09-14T23:58:49.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.446 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:49.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354364+0000", 2024-09-14T23:58:49.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6059999999999999e-05 2024-09-14T23:58:49.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:49.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354459+0000", 2024-09-14T23:58:49.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4356000000000003e-05 2024-09-14T23:58:49.447 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:49.447 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:49.447 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:49.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.447 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69285 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 7438336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:49.447 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.304792+0000", 2024-09-14T23:58:49.447 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 94.067141809000006, 2024-09-14T23:58:49.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050398874000000003, 2024-09-14T23:58:49.448 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:49.448 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:49.448 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:49.448 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:49.448 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:49.448 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:49.448 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69285 2024-09-14T23:58:49.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.448 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:49.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:49.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304792+0000", 2024-09-14T23:58:49.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:49.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:49.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304792+0000", 2024-09-14T23:58:49.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:49.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:49.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304792+0000", 2024-09-14T23:58:49.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2800000000000003e-07 2024-09-14T23:58:49.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:49.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304794+0000", 2024-09-14T23:58:49.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8700000000000001e-06 2024-09-14T23:58:49.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.450 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:49.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304800+0000", 2024-09-14T23:58:49.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0889999999999996e-06 2024-09-14T23:58:49.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.450 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:49.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304805+0000", 2024-09-14T23:58:49.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7060000000000003e-06 2024-09-14T23:58:49.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.450 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:49.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355108+0000", 2024-09-14T23:58:49.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050303170000000001 2024-09-14T23:58:49.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:49.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355150+0000", 2024-09-14T23:58:49.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2271999999999998e-05 2024-09-14T23:58:49.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:49.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355190+0000", 2024-09-14T23:58:49.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0438999999999997e-05 2024-09-14T23:58:49.451 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:49.451 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:49.451 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:49.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.451 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69286 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 7553024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:49.452 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.304811+0000", 2024-09-14T23:58:49.452 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 94.067122447000003, 2024-09-14T23:58:49.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050585512999999999, 2024-09-14T23:58:49.452 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:49.452 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:49.452 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:49.452 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:49.452 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:49.452 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:49.452 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69286 2024-09-14T23:58:49.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.452 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:49.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:49.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304811+0000", 2024-09-14T23:58:49.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:49.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:49.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304811+0000", 2024-09-14T23:58:49.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:49.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:49.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304811+0000", 2024-09-14T23:58:49.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.89e-07 2024-09-14T23:58:49.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:49.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304814+0000", 2024-09-14T23:58:49.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3199999999999998e-06 2024-09-14T23:58:49.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:49.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304821+0000", 2024-09-14T23:58:49.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5689999999999999e-06 2024-09-14T23:58:49.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:49.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304826+0000", 2024-09-14T23:58:49.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1200000000000001e-06 2024-09-14T23:58:49.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:49.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355226+0000", 2024-09-14T23:58:49.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050399653000000003 2024-09-14T23:58:49.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:49.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355314+0000", 2024-09-14T23:58:49.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8389999999999999e-05 2024-09-14T23:58:49.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:49.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355396+0000", 2024-09-14T23:58:49.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2071999999999996e-05 2024-09-14T23:58:49.456 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:49.456 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:49.456 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:49.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.456 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69287 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 7667712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:49.456 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.304833+0000", 2024-09-14T23:58:49.456 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 94.067099849000002, 2024-09-14T23:58:49.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050664740999999999, 2024-09-14T23:58:49.456 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:49.456 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:49.456 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:49.456 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:49.457 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:49.457 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:49.457 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69287 2024-09-14T23:58:49.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.457 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:49.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:49.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304833+0000", 2024-09-14T23:58:49.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:49.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:49.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304833+0000", 2024-09-14T23:58:49.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:49.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:49.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304834+0000", 2024-09-14T23:58:49.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4999999999999998e-07 2024-09-14T23:58:49.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:49.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304836+0000", 2024-09-14T23:58:49.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0279999999999999e-06 2024-09-14T23:58:49.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:49.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304842+0000", 2024-09-14T23:58:49.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8919999999999997e-06 2024-09-14T23:58:49.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:49.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304847+0000", 2024-09-14T23:58:49.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6160000000000001e-06 2024-09-14T23:58:49.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:49.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355411+0000", 2024-09-14T23:58:49.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050563540999999997 2024-09-14T23:58:49.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:49.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355459+0000", 2024-09-14T23:58:49.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7979000000000003e-05 2024-09-14T23:58:49.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:49.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355498+0000", 2024-09-14T23:58:49.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9335000000000002e-05 2024-09-14T23:58:49.460 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:49.460 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:49.460 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:49.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.460 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69288 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 7782400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:58:49.460 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.304854+0000", 2024-09-14T23:58:49.461 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 94.067079410999995, 2024-09-14T23:58:49.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050855145999999997, 2024-09-14T23:58:49.461 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:49.461 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:49.461 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:49.461 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:49.461 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:49.461 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:49.461 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69288 2024-09-14T23:58:49.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.461 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:49.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:49.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304854+0000", 2024-09-14T23:58:49.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:49.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:49.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304854+0000", 2024-09-14T23:58:49.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:49.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:49.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.306670+0000", 2024-09-14T23:58:49.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00181631 2024-09-14T23:58:49.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:49.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.306675+0000", 2024-09-14T23:58:49.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0220000000000004e-06 2024-09-14T23:58:49.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:49.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.306684+0000", 2024-09-14T23:58:49.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7309999999999999e-06 2024-09-14T23:58:49.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:49.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.306689+0000", 2024-09-14T23:58:49.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6609999999999999e-06 2024-09-14T23:58:49.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:49.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355533+0000", 2024-09-14T23:58:49.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048844578 2024-09-14T23:58:49.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:49.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355622+0000", 2024-09-14T23:58:49.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9006999999999997e-05 2024-09-14T23:58:49.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:49.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:49.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:49.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355709+0000", 2024-09-14T23:58:49.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6836999999999995e-05 2024-09-14T23:58:49.465 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:49.465 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:49.465 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:49.465 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:49.465 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:49.465 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:49.466 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:49.465+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:49.467 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T23:58:49.567 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:49.565+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:49.575 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:49.575 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:49.575 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:58:49.575 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:49.584 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T23:58:49.667 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:49.666+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:49.690 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:49.690 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:49.690 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:58:49.691 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:58:49.691 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:49.699 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T23:58:49.769 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:49.767+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:49.808 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:49.808 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:58:49.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:58:49.808 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:58:49.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.808 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:43054 3.1f 3:ffbfe596:::rbd_data.1118350d5ae5.00000000000002eb:head [set-alloc-hint object_size 262144 write_size 262144,write 196608~22770] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T23:58:49.808 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:58:49.808 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 116.838946491, 2024-09-14T23:58:49.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.99973334400000002, 2024-09-14T23:58:49.808 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:49.810 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:49.810 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:49.810 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:49.810 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:49.810 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:49.810 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 43054 2024-09-14T23:58:49.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.810 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:49.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:49.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:58:49.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:49.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:49.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:58:49.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:49.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:49.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966552+0000", 2024-09-14T23:58:49.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3790000000000001e-06 2024-09-14T23:58:49.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:49.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966565+0000", 2024-09-14T23:58:49.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2459999999999999e-05 2024-09-14T23:58:49.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:49.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966576+0000", 2024-09-14T23:58:49.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1039000000000001e-05 2024-09-14T23:58:49.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:49.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966582+0000", 2024-09-14T23:58:49.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9660000000000001e-06 2024-09-14T23:58:49.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:49.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966625+0000", 2024-09-14T23:58:49.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3529999999999998e-05 2024-09-14T23:58:49.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:58:49.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966632+0000", 2024-09-14T23:58:49.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9809999999999997e-06 2024-09-14T23:58:49.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:49.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.964460+0000", 2024-09-14T23:58:49.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.99782768300000002 2024-09-14T23:58:49.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:49.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.964641+0000", 2024-09-14T23:58:49.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000181089 2024-09-14T23:58:49.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-14T23:58:49.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.964809+0000", 2024-09-14T23:58:49.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000168505 2024-09-14T23:58:49.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:58:49.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966030+0000", 2024-09-14T23:58:49.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012206929999999999 2024-09-14T23:58:49.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:49.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966181+0000", 2024-09-14T23:58:49.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000150569 2024-09-14T23:58:49.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:49.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966243+0000", 2024-09-14T23:58:49.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2331999999999999e-05 2024-09-14T23:58:49.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:49.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966284+0000", 2024-09-14T23:58:49.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1118000000000002e-05 2024-09-14T23:58:49.816 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:49.816 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:49.816 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:49.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.816 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184314 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4162560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:49.817 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.077247+0000", 2024-09-14T23:58:49.817 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.728249822999999, 2024-09-14T23:58:49.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058772256000000002, 2024-09-14T23:58:49.817 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:49.817 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:49.817 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:49.817 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:49.817 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:49.817 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:49.817 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184314 2024-09-14T23:58:49.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.817 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:49.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:49.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.077247+0000", 2024-09-14T23:58:49.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:49.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:49.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.077247+0000", 2024-09-14T23:58:49.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:49.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:49.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.078906+0000", 2024-09-14T23:58:49.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016583279999999999 2024-09-14T23:58:49.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:49.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.078911+0000", 2024-09-14T23:58:49.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0420000000000002e-06 2024-09-14T23:58:49.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:49.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.078922+0000", 2024-09-14T23:58:49.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1354e-05 2024-09-14T23:58:49.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:49.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.078930+0000", 2024-09-14T23:58:49.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5730000000000002e-06 2024-09-14T23:58:49.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:49.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.135870+0000", 2024-09-14T23:58:49.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056940651000000002 2024-09-14T23:58:49.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:49.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.135951+0000", 2024-09-14T23:58:49.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0890999999999995e-05 2024-09-14T23:58:49.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:49.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.136020+0000", 2024-09-14T23:58:49.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8417000000000006e-05 2024-09-14T23:58:49.821 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:49.821 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:49.821 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:49.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.821 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184323 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4183040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:49.821 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.079228+0000", 2024-09-14T23:58:49.821 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.726268748999999, 2024-09-14T23:58:49.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058454487999999999, 2024-09-14T23:58:49.822 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:49.822 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:49.822 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:49.822 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:49.822 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:49.822 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:49.822 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184323 2024-09-14T23:58:49.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.822 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:49.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:49.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079228+0000", 2024-09-14T23:58:49.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:49.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:49.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079228+0000", 2024-09-14T23:58:49.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:49.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:49.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079229+0000", 2024-09-14T23:58:49.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5299999999999999e-07 2024-09-14T23:58:49.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:49.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079232+0000", 2024-09-14T23:58:49.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8839999999999998e-06 2024-09-14T23:58:49.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:49.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079238+0000", 2024-09-14T23:58:49.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2299999999999996e-06 2024-09-14T23:58:49.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:49.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079243+0000", 2024-09-14T23:58:49.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3029999999999999e-06 2024-09-14T23:58:49.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:49.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.137533+0000", 2024-09-14T23:58:49.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058289674999999999 2024-09-14T23:58:49.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:49.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.137615+0000", 2024-09-14T23:58:49.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2288999999999998e-05 2024-09-14T23:58:49.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:49.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.137683+0000", 2024-09-14T23:58:49.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7654e-05 2024-09-14T23:58:49.826 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:49.826 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:49.826 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:49.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.826 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184324 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4073472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:49.826 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.079251+0000", 2024-09-14T23:58:49.826 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.726246281999998, 2024-09-14T23:58:49.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058609383000000001, 2024-09-14T23:58:49.826 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:49.826 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:49.826 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:49.826 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:49.826 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:49.826 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:49.827 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184324 2024-09-14T23:58:49.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.827 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:49.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:49.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079251+0000", 2024-09-14T23:58:49.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:49.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:49.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079251+0000", 2024-09-14T23:58:49.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:49.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:49.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.080920+0000", 2024-09-14T23:58:49.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016687430000000001 2024-09-14T23:58:49.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:49.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.080924+0000", 2024-09-14T23:58:49.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3869999999999997e-06 2024-09-14T23:58:49.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:49.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.080935+0000", 2024-09-14T23:58:49.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0515e-05 2024-09-14T23:58:49.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:49.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.080942+0000", 2024-09-14T23:58:49.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0990000000000001e-06 2024-09-14T23:58:49.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:49.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.137713+0000", 2024-09-14T23:58:49.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056771288000000003 2024-09-14T23:58:49.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:49.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.137794+0000", 2024-09-14T23:58:49.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0613999999999998e-05 2024-09-14T23:58:49.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:49.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.137860+0000", 2024-09-14T23:58:49.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6736999999999995e-05 2024-09-14T23:58:49.830 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:49.830 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:49.830 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:49.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.830 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184346 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4099072~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:49.831 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.083616+0000", 2024-09-14T23:58:49.831 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.721881097000001, 2024-09-14T23:58:49.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058557419999999999, 2024-09-14T23:58:49.831 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:49.831 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:49.831 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:49.831 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:49.831 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:49.831 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:49.831 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184346 2024-09-14T23:58:49.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.831 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:49.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:49.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083616+0000", 2024-09-14T23:58:49.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:49.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:49.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083616+0000", 2024-09-14T23:58:49.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:49.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:49.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083620+0000", 2024-09-14T23:58:49.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0119999999999999e-06 2024-09-14T23:58:49.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:49.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083623+0000", 2024-09-14T23:58:49.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9059999999999998e-06 2024-09-14T23:58:49.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:49.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083631+0000", 2024-09-14T23:58:49.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2679999999999994e-06 2024-09-14T23:58:49.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:49.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083637+0000", 2024-09-14T23:58:49.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1210000000000003e-06 2024-09-14T23:58:49.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:49.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142018+0000", 2024-09-14T23:58:49.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058381111999999999 2024-09-14T23:58:49.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:49.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142100+0000", 2024-09-14T23:58:49.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1308000000000001e-05 2024-09-14T23:58:49.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:49.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142173+0000", 2024-09-14T23:58:49.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3693000000000001e-05 2024-09-14T23:58:49.835 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:49.835 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:49.835 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:49.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.835 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184347 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4122624~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:49.835 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.083647+0000", 2024-09-14T23:58:49.835 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.721850267000001, 2024-09-14T23:58:49.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058706388999999998, 2024-09-14T23:58:49.835 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:49.835 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:49.835 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:49.835 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:49.836 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:49.836 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:49.836 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184347 2024-09-14T23:58:49.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.836 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:49.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:49.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083647+0000", 2024-09-14T23:58:49.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:49.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:49.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083647+0000", 2024-09-14T23:58:49.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:49.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:49.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083647+0000", 2024-09-14T23:58:49.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3099999999999998e-07 2024-09-14T23:58:49.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:49.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083650+0000", 2024-09-14T23:58:49.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2900000000000001e-06 2024-09-14T23:58:49.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:49.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083657+0000", 2024-09-14T23:58:49.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.486e-06 2024-09-14T23:58:49.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:49.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083663+0000", 2024-09-14T23:58:49.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4330000000000003e-06 2024-09-14T23:58:49.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:49.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142206+0000", 2024-09-14T23:58:49.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058543237999999997 2024-09-14T23:58:49.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:49.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142284+0000", 2024-09-14T23:58:49.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8487999999999998e-05 2024-09-14T23:58:49.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:49.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142353+0000", 2024-09-14T23:58:49.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9023000000000006e-05 2024-09-14T23:58:49.839 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:49.839 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:49.839 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:49.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.839 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184348 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4103168~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:49.839 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.083669+0000", 2024-09-14T23:58:49.840 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.721827849999997, 2024-09-14T23:58:49.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058864232000000002, 2024-09-14T23:58:49.840 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:49.840 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:49.840 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:49.840 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:49.840 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:49.840 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:49.840 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184348 2024-09-14T23:58:49.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.840 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:49.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:49.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083669+0000", 2024-09-14T23:58:49.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:49.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:49.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083669+0000", 2024-09-14T23:58:49.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:49.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:49.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083670+0000", 2024-09-14T23:58:49.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9999999999999998e-07 2024-09-14T23:58:49.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:49.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083672+0000", 2024-09-14T23:58:49.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1430000000000001e-06 2024-09-14T23:58:49.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:49.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083678+0000", 2024-09-14T23:58:49.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5200000000000003e-06 2024-09-14T23:58:49.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:49.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083683+0000", 2024-09-14T23:58:49.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9359999999999998e-06 2024-09-14T23:58:49.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:49.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142384+0000", 2024-09-14T23:58:49.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058701159000000003 2024-09-14T23:58:49.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:49.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142466+0000", 2024-09-14T23:58:49.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1024000000000002e-05 2024-09-14T23:58:49.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:49.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142534+0000", 2024-09-14T23:58:49.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8050000000000001e-05 2024-09-14T23:58:49.844 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:49.844 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:49.844 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:49.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.844 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184349 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4126720~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:49.844 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.083690+0000", 2024-09-14T23:58:49.844 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.721807315, 2024-09-14T23:58:49.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059019911000000001, 2024-09-14T23:58:49.844 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:49.844 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:49.844 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:49.845 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:49.845 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:49.845 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:49.845 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184349 2024-09-14T23:58:49.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.845 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:49.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:49.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083690+0000", 2024-09-14T23:58:49.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:49.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:49.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083690+0000", 2024-09-14T23:58:49.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:49.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:49.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.084900+0000", 2024-09-14T23:58:49.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012099770000000001 2024-09-14T23:58:49.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:49.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.084905+0000", 2024-09-14T23:58:49.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.028e-06 2024-09-14T23:58:49.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:49.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.084914+0000", 2024-09-14T23:58:49.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6619999999999994e-06 2024-09-14T23:58:49.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:49.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.084922+0000", 2024-09-14T23:58:49.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5720000000000003e-06 2024-09-14T23:58:49.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:49.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142565+0000", 2024-09-14T23:58:49.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057642605999999999 2024-09-14T23:58:49.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:49.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142644+0000", 2024-09-14T23:58:49.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9370999999999996e-05 2024-09-14T23:58:49.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:49.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142710+0000", 2024-09-14T23:58:49.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5695e-05 2024-09-14T23:58:49.849 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:49.849 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:49.849 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:49.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.849 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184354 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4115456~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:49.849 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.085029+0000", 2024-09-14T23:58:49.849 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.720468664999999, 2024-09-14T23:58:49.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058626327999999998, 2024-09-14T23:58:49.849 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:49.849 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:49.849 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:49.849 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:49.850 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:49.850 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:49.850 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184354 2024-09-14T23:58:49.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.850 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:49.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:49.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085029+0000", 2024-09-14T23:58:49.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:49.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:49.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085029+0000", 2024-09-14T23:58:49.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:49.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:49.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085029+0000", 2024-09-14T23:58:49.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0800000000000001e-07 2024-09-14T23:58:49.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:49.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085030+0000", 2024-09-14T23:58:49.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.226e-06 2024-09-14T23:58:49.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:49.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085033+0000", 2024-09-14T23:58:49.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9569999999999998e-06 2024-09-14T23:58:49.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:49.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085035+0000", 2024-09-14T23:58:49.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.587e-06 2024-09-14T23:58:49.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:49.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143491+0000", 2024-09-14T23:58:49.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058455337000000003 2024-09-14T23:58:49.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:49.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143575+0000", 2024-09-14T23:58:49.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3850000000000005e-05 2024-09-14T23:58:49.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:49.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143655+0000", 2024-09-14T23:58:49.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0162999999999998e-05 2024-09-14T23:58:49.853 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:49.853 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:49.853 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:49.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.854 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184355 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4139008~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:49.854 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.085039+0000", 2024-09-14T23:58:49.854 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.720458260999997, 2024-09-14T23:58:49.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058801369999999999, 2024-09-14T23:58:49.854 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:49.854 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:49.854 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:49.854 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:49.854 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:49.854 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:49.854 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184355 2024-09-14T23:58:49.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.855 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:49.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:49.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085039+0000", 2024-09-14T23:58:49.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:49.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:49.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085039+0000", 2024-09-14T23:58:49.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:49.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:49.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085039+0000", 2024-09-14T23:58:49.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.03e-07 2024-09-14T23:58:49.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:49.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085040+0000", 2024-09-14T23:58:49.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.158e-06 2024-09-14T23:58:49.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:49.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085043+0000", 2024-09-14T23:58:49.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0240000000000002e-06 2024-09-14T23:58:49.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:49.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085046+0000", 2024-09-14T23:58:49.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4600000000000002e-06 2024-09-14T23:58:49.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:49.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143688+0000", 2024-09-14T23:58:49.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058642356 2024-09-14T23:58:49.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:49.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143766+0000", 2024-09-14T23:58:49.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8219000000000004e-05 2024-09-14T23:58:49.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:49.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143840+0000", 2024-09-14T23:58:49.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3949999999999995e-05 2024-09-14T23:58:49.858 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:49.858 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:49.858 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:49.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.858 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184356 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4119552~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:49.858 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.085049+0000", 2024-09-14T23:58:49.858 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.720448089999998, 2024-09-14T23:58:49.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058977992, 2024-09-14T23:58:49.859 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:49.859 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:49.859 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:49.859 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:49.859 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:49.859 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:49.859 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184356 2024-09-14T23:58:49.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.859 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:49.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:49.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085049+0000", 2024-09-14T23:58:49.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:49.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:49.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085049+0000", 2024-09-14T23:58:49.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:49.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:49.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.086934+0000", 2024-09-14T23:58:49.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018854189999999999 2024-09-14T23:58:49.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:49.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.086939+0000", 2024-09-14T23:58:49.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.493e-06 2024-09-14T23:58:49.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:49.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.086951+0000", 2024-09-14T23:58:49.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1969e-05 2024-09-14T23:58:49.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:49.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.086959+0000", 2024-09-14T23:58:49.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5780000000000001e-06 2024-09-14T23:58:49.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:49.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143871+0000", 2024-09-14T23:58:49.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056912114 2024-09-14T23:58:49.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:49.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143951+0000", 2024-09-14T23:58:49.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9938000000000006e-05 2024-09-14T23:58:49.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:49.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.144027+0000", 2024-09-14T23:58:49.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6481000000000002e-05 2024-09-14T23:58:49.863 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:49.863 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:49.863 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:49.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.863 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184364 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4135936~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:49.863 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.087207+0000", 2024-09-14T23:58:49.863 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.718289810000002, 2024-09-14T23:58:49.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058466420999999998, 2024-09-14T23:58:49.863 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:49.863 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:49.863 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:49.863 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:49.863 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:49.864 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:49.864 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184364 2024-09-14T23:58:49.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.864 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:49.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.864 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:49.864 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087207+0000", 2024-09-14T23:58:49.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:49.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.864 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:49.864 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087207+0000", 2024-09-14T23:58:49.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:49.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:49.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087208+0000", 2024-09-14T23:58:49.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.35e-07 2024-09-14T23:58:49.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:49.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087209+0000", 2024-09-14T23:58:49.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.573e-06 2024-09-14T23:58:49.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:49.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087212+0000", 2024-09-14T23:58:49.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2770000000000001e-06 2024-09-14T23:58:49.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:49.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087215+0000", 2024-09-14T23:58:49.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5500000000000001e-06 2024-09-14T23:58:49.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:49.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145520+0000", 2024-09-14T23:58:49.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058304835999999999 2024-09-14T23:58:49.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:49.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145602+0000", 2024-09-14T23:58:49.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1714999999999999e-05 2024-09-14T23:58:49.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.867 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:49.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145674+0000", 2024-09-14T23:58:49.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2235000000000002e-05 2024-09-14T23:58:49.867 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:49.867 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:49.867 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:49.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.867 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184365 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4159488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:49.867 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.087219+0000", 2024-09-14T23:58:49.868 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.718277778999997, 2024-09-14T23:58:49.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058643533999999997, 2024-09-14T23:58:49.868 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:49.868 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:49.868 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:49.868 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:49.868 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:49.868 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:49.868 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184365 2024-09-14T23:58:49.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.868 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:49.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.868 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:49.868 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087219+0000", 2024-09-14T23:58:49.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:49.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.869 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:49.869 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087219+0000", 2024-09-14T23:58:49.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:49.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.869 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:49.869 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087220+0000", 2024-09-14T23:58:49.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.39e-07 2024-09-14T23:58:49.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:49.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087222+0000", 2024-09-14T23:58:49.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0609999999999998e-06 2024-09-14T23:58:49.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:49.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087228+0000", 2024-09-14T23:58:49.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8590000000000001e-06 2024-09-14T23:58:49.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:49.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087232+0000", 2024-09-14T23:58:49.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8149999999999996e-06 2024-09-14T23:58:49.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:49.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145705+0000", 2024-09-14T23:58:49.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058472532000000001 2024-09-14T23:58:49.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:49.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145784+0000", 2024-09-14T23:58:49.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9382000000000007e-05 2024-09-14T23:58:49.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:49.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145863+0000", 2024-09-14T23:58:49.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8546000000000003e-05 2024-09-14T23:58:49.872 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:49.872 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:49.872 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:49.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.872 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184366 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4140032~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:49.872 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.087240+0000", 2024-09-14T23:58:49.872 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.718257330999997, 2024-09-14T23:58:49.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058818068000000001, 2024-09-14T23:58:49.872 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:49.872 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:49.872 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:49.872 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:49.872 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:49.873 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:49.873 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184366 2024-09-14T23:58:49.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.873 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:49.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.873 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:49.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087240+0000", 2024-09-14T23:58:49.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:49.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.873 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:49.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087240+0000", 2024-09-14T23:58:49.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:49.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:49.874 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.088904+0000", 2024-09-14T23:58:49.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001664315 2024-09-14T23:58:49.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:49.874 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.088909+0000", 2024-09-14T23:58:49.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7709999999999997e-06 2024-09-14T23:58:49.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:49.874 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.088921+0000", 2024-09-14T23:58:49.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1661999999999999e-05 2024-09-14T23:58:49.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:49.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.088928+0000", 2024-09-14T23:58:49.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8189999999999992e-06 2024-09-14T23:58:49.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:49.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145901+0000", 2024-09-14T23:58:49.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056972478 2024-09-14T23:58:49.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:49.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145977+0000", 2024-09-14T23:58:49.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5763e-05 2024-09-14T23:58:49.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:49.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.146058+0000", 2024-09-14T23:58:49.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1260000000000005e-05 2024-09-14T23:58:49.876 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:49.876 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:49.876 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:49.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.876 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184375 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4179968~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:49.876 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.089199+0000", 2024-09-14T23:58:49.877 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.716297994000001, 2024-09-14T23:58:49.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058597445999999997, 2024-09-14T23:58:49.877 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:49.877 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:49.877 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:49.877 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:49.877 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:49.877 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:49.877 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184375 2024-09-14T23:58:49.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.877 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:49.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:49.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089199+0000", 2024-09-14T23:58:49.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:49.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:49.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089199+0000", 2024-09-14T23:58:49.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:49.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:49.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089200+0000", 2024-09-14T23:58:49.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8100000000000003e-07 2024-09-14T23:58:49.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:49.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089202+0000", 2024-09-14T23:58:49.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7599999999999998e-06 2024-09-14T23:58:49.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:49.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089209+0000", 2024-09-14T23:58:49.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7279999999999998e-06 2024-09-14T23:58:49.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:49.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089215+0000", 2024-09-14T23:58:49.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5090000000000001e-06 2024-09-14T23:58:49.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.880 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:49.880 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147638+0000", 2024-09-14T23:58:49.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058423105000000003 2024-09-14T23:58:49.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.880 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:49.880 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147719+0000", 2024-09-14T23:58:49.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1217000000000005e-05 2024-09-14T23:58:49.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.880 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:49.880 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147797+0000", 2024-09-14T23:58:49.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7645999999999995e-05 2024-09-14T23:58:49.880 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:49.881 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:49.881 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:49.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.881 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184376 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4160512~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:49.881 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.089232+0000", 2024-09-14T23:58:49.881 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.716265503999999, 2024-09-14T23:58:49.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058751338, 2024-09-14T23:58:49.881 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:49.881 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:49.881 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:49.881 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:49.881 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:49.881 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:49.882 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184376 2024-09-14T23:58:49.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.882 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:49.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.882 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:49.882 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089232+0000", 2024-09-14T23:58:49.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:49.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.882 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:49.882 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089232+0000", 2024-09-14T23:58:49.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:49.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.883 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:49.883 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089232+0000", 2024-09-14T23:58:49.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5600000000000001e-07 2024-09-14T23:58:49.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.883 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:49.883 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089235+0000", 2024-09-14T23:58:49.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5639999999999999e-06 2024-09-14T23:58:49.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.883 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:49.883 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089242+0000", 2024-09-14T23:58:49.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0369999999999997e-06 2024-09-14T23:58:49.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:49.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089247+0000", 2024-09-14T23:58:49.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2079999999999999e-06 2024-09-14T23:58:49.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:49.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147831+0000", 2024-09-14T23:58:49.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058583853999999998 2024-09-14T23:58:49.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:49.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147915+0000", 2024-09-14T23:58:49.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4208000000000004e-05 2024-09-14T23:58:49.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.885 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:49.885 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147983+0000", 2024-09-14T23:58:49.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8010999999999996e-05 2024-09-14T23:58:49.885 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:49.885 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:49.885 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:49.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.885 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184377 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4184064~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:49.885 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.089255+0000", 2024-09-14T23:58:49.885 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.716241947999997, 2024-09-14T23:58:49.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058906100000000003, 2024-09-14T23:58:49.886 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:49.886 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:49.886 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:49.886 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:49.886 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:49.886 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:49.886 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184377 2024-09-14T23:58:49.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.886 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:49.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:49.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089255+0000", 2024-09-14T23:58:49.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:49.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:49.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089255+0000", 2024-09-14T23:58:49.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:49.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:49.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.090903+0000", 2024-09-14T23:58:49.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001647871 2024-09-14T23:58:49.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:49.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.090908+0000", 2024-09-14T23:58:49.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7199999999999997e-06 2024-09-14T23:58:49.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.888 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:49.888 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.090919+0000", 2024-09-14T23:58:49.888 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1472e-05 2024-09-14T23:58:49.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.888 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:49.888 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.090927+0000", 2024-09-14T23:58:49.888 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0390000000000005e-06 2024-09-14T23:58:49.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.888 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:49.888 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.148013+0000", 2024-09-14T23:58:49.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057085662000000002 2024-09-14T23:58:49.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:49.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.148090+0000", 2024-09-14T23:58:49.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6778000000000002e-05 2024-09-14T23:58:49.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:49.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.148161+0000", 2024-09-14T23:58:49.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1557999999999995e-05 2024-09-14T23:58:49.889 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:49.889 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:49.890 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:49.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.890 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184386 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4185088~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:49.890 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.091063+0000", 2024-09-14T23:58:49.890 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.714434283999999, 2024-09-14T23:58:49.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058603404999999997, 2024-09-14T23:58:49.890 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:49.890 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:49.890 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:49.890 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:49.890 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:49.890 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:49.890 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184386 2024-09-14T23:58:49.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.891 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:49.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:49.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091063+0000", 2024-09-14T23:58:49.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:49.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:49.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091063+0000", 2024-09-14T23:58:49.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:49.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:49.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091063+0000", 2024-09-14T23:58:49.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1899999999999999e-07 2024-09-14T23:58:49.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:49.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091064+0000", 2024-09-14T23:58:49.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2699999999999999e-06 2024-09-14T23:58:49.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:49.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091067+0000", 2024-09-14T23:58:49.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.971e-06 2024-09-14T23:58:49.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:49.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091070+0000", 2024-09-14T23:58:49.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.525e-06 2024-09-14T23:58:49.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:49.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.149510+0000", 2024-09-14T23:58:49.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058439758000000001 2024-09-14T23:58:49.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:49.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.149595+0000", 2024-09-14T23:58:49.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5146000000000001e-05 2024-09-14T23:58:49.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.894 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:49.894 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.149666+0000", 2024-09-14T23:58:49.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1515999999999998e-05 2024-09-14T23:58:49.894 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:49.894 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:49.894 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:49.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.894 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184388 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4189184~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:49.894 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.091073+0000", 2024-09-14T23:58:49.894 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.714424008999998, 2024-09-14T23:58:49.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058781609999999998, 2024-09-14T23:58:49.895 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:49.895 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:49.895 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:49.895 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:49.895 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:49.895 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:49.895 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184388 2024-09-14T23:58:49.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.895 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:49.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:49.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091073+0000", 2024-09-14T23:58:49.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:49.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.896 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:49.896 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091073+0000", 2024-09-14T23:58:49.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:49.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.896 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:49.896 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.092903+0000", 2024-09-14T23:58:49.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018296289999999999 2024-09-14T23:58:49.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.897 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:49.897 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.092910+0000", 2024-09-14T23:58:49.897 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7510000000000002e-06 2024-09-14T23:58:49.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.897 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:49.897 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.092918+0000", 2024-09-14T23:58:49.897 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7420000000000003e-06 2024-09-14T23:58:49.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.897 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:49.897 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.092925+0000", 2024-09-14T23:58:49.897 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0360000000000001e-06 2024-09-14T23:58:49.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.898 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.898 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:49.898 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.149700+0000", 2024-09-14T23:58:49.898 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056774340999999999 2024-09-14T23:58:49.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.898 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.898 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:49.898 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.149784+0000", 2024-09-14T23:58:49.898 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4393000000000003e-05 2024-09-14T23:58:49.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.898 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.899 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:49.899 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.149855+0000", 2024-09-14T23:58:49.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0717999999999996e-05 2024-09-14T23:58:49.899 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:49.899 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:49.899 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:49.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.899 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4172.0:8 3.12 3:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-14T23:58:49.899 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.976999+0000", 2024-09-14T23:58:49.900 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 41.828498600000003, 2024-09-14T23:58:49.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0046036439999999, 2024-09-14T23:58:49.900 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:49.900 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:49.900 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:49.900 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:49.900 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4172", 2024-09-14T23:58:49.900 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/2542269203", 2024-09-14T23:58:49.900 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 8 2024-09-14T23:58:49.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.900 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:49.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:49.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.976999+0000", 2024-09-14T23:58:49.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:49.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:49.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.976999+0000", 2024-09-14T23:58:49.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:49.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:49.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.977000+0000", 2024-09-14T23:58:49.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5200000000000001e-06 2024-09-14T23:58:49.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.902 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:49.902 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.977005+0000", 2024-09-14T23:58:49.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.04e-06 2024-09-14T23:58:49.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.902 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:49.902 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.977013+0000", 2024-09-14T23:58:49.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2889999999999998e-06 2024-09-14T23:58:49.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.902 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:49.902 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.977024+0000", 2024-09-14T23:58:49.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0307e-05 2024-09-14T23:58:49.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.903 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.903 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:49.903 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.977086+0000", 2024-09-14T23:58:49.903 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2248000000000004e-05 2024-09-14T23:58:49.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.903 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.903 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:58:49.903 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.977097+0000", 2024-09-14T23:58:49.903 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1123e-05 2024-09-14T23:58:49.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.903 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.903 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:49.904 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:08.981470+0000", 2024-09-14T23:58:49.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.004373116 2024-09-14T23:58:49.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:49.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:49.904 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:49.904 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:08.981602+0000", 2024-09-14T23:58:49.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000132001 2024-09-14T23:58:49.904 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:49.904 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:49.904 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:49.904 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:49.904 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:49.904 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:49.906 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T23:58:49.906 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:58:49.905+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:58:49.985 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is unset 2024-09-14T23:58:50.041 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:50.040+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:50.060 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:50.061 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:50.061 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:58:50.061 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:50.070 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T23:58:50.110 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:50.109+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:50.209 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:50.208+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:50.217 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:50.217 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:50.217 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:58:50.217 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:58:50.218 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:50.226 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T23:58:50.310 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:58:50.309+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:58:50.375 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:50.375 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:58:50.375 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:58:50.375 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:58:50.375 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.375 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5882 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:50.375 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:58:50.375 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 130.997090171, 2024-09-14T23:58:50.376 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050549666, 2024-09-14T23:58:50.376 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:50.376 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:50.376 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:50.376 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:50.376 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:50.376 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:50.376 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5882 2024-09-14T23:58:50.376 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.376 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:50.376 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.376 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:50.377 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:58:50.377 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.377 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.377 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.377 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:50.377 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:58:50.377 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.377 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.377 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.377 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:50.377 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:58:50.377 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8099999999999998e-07 2024-09-14T23:58:50.377 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.378 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.378 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:50.378 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375842+0000", 2024-09-14T23:58:50.378 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9900000000000009e-07 2024-09-14T23:58:50.378 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.378 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.378 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:50.378 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375845+0000", 2024-09-14T23:58:50.378 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.055e-06 2024-09-14T23:58:50.378 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.378 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.378 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:50.378 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375848+0000", 2024-09-14T23:58:50.379 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7609999999999999e-06 2024-09-14T23:58:50.379 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.379 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.379 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:50.379 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426381+0000", 2024-09-14T23:58:50.379 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050533392000000003 2024-09-14T23:58:50.379 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.379 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.379 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:50.379 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426390+0000", 2024-09-14T23:58:50.379 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0780000000000002e-06 2024-09-14T23:58:50.379 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.379 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:50.380 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.380 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.380 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.380 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5884 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:50.380 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:58:50.380 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 130.997080031, 2024-09-14T23:58:50.380 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050717439000000003, 2024-09-14T23:58:50.380 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:50.380 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:50.380 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:50.380 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:50.380 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:50.380 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:50.381 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5884 2024-09-14T23:58:50.381 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.381 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:50.381 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.381 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:50.381 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:58:50.381 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.381 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.381 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.381 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:50.381 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:58:50.381 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.381 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.382 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.382 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:50.382 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:58:50.382 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0800000000000001e-07 2024-09-14T23:58:50.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.382 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.382 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:50.382 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375852+0000", 2024-09-14T23:58:50.382 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.02e-06 2024-09-14T23:58:50.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.382 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.382 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:50.382 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375855+0000", 2024-09-14T23:58:50.383 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9900000000000002e-06 2024-09-14T23:58:50.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.383 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.383 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:50.383 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375857+0000", 2024-09-14T23:58:50.383 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3499999999999999e-06 2024-09-14T23:58:50.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.383 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.383 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:50.383 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426556+0000", 2024-09-14T23:58:50.383 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050698139000000003 2024-09-14T23:58:50.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.383 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.383 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:50.383 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426568+0000", 2024-09-14T23:58:50.384 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2731999999999999e-05 2024-09-14T23:58:50.384 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.384 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:50.384 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.384 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.384 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.384 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5885 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:50.384 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:58:50.384 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 130.997061006, 2024-09-14T23:58:50.384 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050737333000000003, 2024-09-14T23:58:50.384 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:50.384 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:50.384 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:50.385 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:50.385 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:50.385 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:50.385 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5885 2024-09-14T23:58:50.385 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.385 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:50.385 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.385 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:50.385 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:58:50.385 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.385 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.385 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.385 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:50.386 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:58:50.386 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.386 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.386 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:50.386 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:58:50.386 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5899999999999998e-07 2024-09-14T23:58:50.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.386 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.386 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:50.386 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375872+0000", 2024-09-14T23:58:50.386 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4219999999999999e-06 2024-09-14T23:58:50.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.386 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.387 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:50.387 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375878+0000", 2024-09-14T23:58:50.387 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4980000000000003e-06 2024-09-14T23:58:50.387 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.387 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.387 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:50.387 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375881+0000", 2024-09-14T23:58:50.387 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6110000000000001e-06 2024-09-14T23:58:50.387 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.387 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.387 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:50.387 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426596+0000", 2024-09-14T23:58:50.387 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050715320000000001 2024-09-14T23:58:50.387 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.388 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.388 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:50.388 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426607+0000", 2024-09-14T23:58:50.388 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1223000000000001e-05 2024-09-14T23:58:50.388 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.388 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:50.388 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.388 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.388 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.388 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5886 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:50.388 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:58:50.388 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 130.997046893, 2024-09-14T23:58:50.388 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050803395000000001, 2024-09-14T23:58:50.389 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:50.389 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:50.389 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:50.389 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:50.389 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:50.389 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:50.389 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5886 2024-09-14T23:58:50.389 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.389 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:50.389 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.389 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:50.389 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:58:50.389 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.389 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.390 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.390 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:50.390 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:58:50.390 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.390 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.390 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.390 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:50.390 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:58:50.390 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2700000000000001e-07 2024-09-14T23:58:50.390 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.390 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.390 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:50.391 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375886+0000", 2024-09-14T23:58:50.391 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7060000000000001e-06 2024-09-14T23:58:50.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.391 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.391 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:50.391 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375890+0000", 2024-09-14T23:58:50.391 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2259999999999999e-06 2024-09-14T23:58:50.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.391 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.391 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:50.391 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375893+0000", 2024-09-14T23:58:50.391 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0819999999999999e-06 2024-09-14T23:58:50.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.391 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.392 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:50.392 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426677+0000", 2024-09-14T23:58:50.392 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050784158000000003 2024-09-14T23:58:50.392 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.392 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.392 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:50.392 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426687+0000", 2024-09-14T23:58:50.392 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9960000000000005e-06 2024-09-14T23:58:50.392 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.392 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:50.392 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.392 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.392 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.393 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5887 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:50.393 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:58:50.393 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 130.997024807, 2024-09-14T23:58:50.393 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050833070000000001, 2024-09-14T23:58:50.393 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:50.393 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:50.393 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:50.393 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:50.393 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:50.393 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:50.393 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5887 2024-09-14T23:58:50.393 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.393 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:50.394 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.394 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:50.394 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:58:50.394 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.394 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.394 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.394 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:50.394 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:58:50.394 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.394 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.394 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.394 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:50.394 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:58:50.394 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3299999999999998e-07 2024-09-14T23:58:50.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.395 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:50.395 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375908+0000", 2024-09-14T23:58:50.395 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7689999999999999e-06 2024-09-14T23:58:50.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.395 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:50.395 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375912+0000", 2024-09-14T23:58:50.395 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.535e-06 2024-09-14T23:58:50.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.395 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:50.396 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375916+0000", 2024-09-14T23:58:50.396 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8979999999999998e-06 2024-09-14T23:58:50.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.396 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.396 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:50.396 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426729+0000", 2024-09-14T23:58:50.396 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050813453000000001 2024-09-14T23:58:50.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.396 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.396 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:50.396 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426739+0000", 2024-09-14T23:58:50.396 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0081999999999999e-05 2024-09-14T23:58:50.396 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.397 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:50.397 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.397 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.397 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.397 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5888 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:50.397 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:58:50.397 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 130.99701173, 2024-09-14T23:58:50.397 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050854652, 2024-09-14T23:58:50.397 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:50.397 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:50.397 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:50.397 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:50.397 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:50.397 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:50.398 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5888 2024-09-14T23:58:50.398 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.398 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:50.398 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.398 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:50.398 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:58:50.398 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.398 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.398 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.398 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:50.398 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:58:50.398 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.398 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.398 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.399 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:50.399 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:58:50.399 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1400000000000001e-07 2024-09-14T23:58:50.399 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.399 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.399 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:50.399 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375921+0000", 2024-09-14T23:58:50.399 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.119e-06 2024-09-14T23:58:50.399 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.399 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.399 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:50.399 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375928+0000", 2024-09-14T23:58:50.399 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3170000000000004e-06 2024-09-14T23:58:50.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.400 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:50.400 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375931+0000", 2024-09-14T23:58:50.400 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6970000000000002e-06 2024-09-14T23:58:50.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.400 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:50.400 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426768+0000", 2024-09-14T23:58:50.400 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050837355000000001 2024-09-14T23:58:50.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.400 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:50.401 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426774+0000", 2024-09-14T23:58:50.401 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9499999999999998e-06 2024-09-14T23:58:50.401 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.401 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:50.401 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.401 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.401 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.401 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5889 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:50.401 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:58:50.401 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 130.996994806, 2024-09-14T23:58:50.401 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050885446000000001, 2024-09-14T23:58:50.401 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:50.401 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:50.402 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:50.402 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:50.402 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:50.402 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:50.402 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5889 2024-09-14T23:58:50.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.402 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:50.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.402 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:50.402 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:58:50.402 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.403 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:50.403 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:58:50.403 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.403 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.403 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:50.403 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:58:50.403 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.36e-07 2024-09-14T23:58:50.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.403 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.403 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:50.403 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375940+0000", 2024-09-14T23:58:50.403 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.844e-06 2024-09-14T23:58:50.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.404 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:50.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375944+0000", 2024-09-14T23:58:50.404 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6720000000000002e-06 2024-09-14T23:58:50.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.404 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:50.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375946+0000", 2024-09-14T23:58:50.404 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5629999999999999e-06 2024-09-14T23:58:50.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.404 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:50.405 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426810+0000", 2024-09-14T23:58:50.405 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050863255000000003 2024-09-14T23:58:50.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.405 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:50.405 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426822+0000", 2024-09-14T23:58:50.405 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1875999999999999e-05 2024-09-14T23:58:50.405 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.405 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:50.405 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.406 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.406 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5890 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:50.406 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:58:50.406 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 130.99698099400001, 2024-09-14T23:58:50.406 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050917549999999999, 2024-09-14T23:58:50.406 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:50.406 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:50.406 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:50.406 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:50.406 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:50.406 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:50.406 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5890 2024-09-14T23:58:50.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.407 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:50.407 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.407 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:50.407 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:58:50.407 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.407 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.407 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:50.407 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:58:50.407 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.407 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.408 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:50.408 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:58:50.408 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4299999999999999e-07 2024-09-14T23:58:50.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.408 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:50.408 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375951+0000", 2024-09-14T23:58:50.408 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1000000000000001e-06 2024-09-14T23:58:50.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.408 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:50.408 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375956+0000", 2024-09-14T23:58:50.408 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3370000000000001e-06 2024-09-14T23:58:50.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:50.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375958+0000", 2024-09-14T23:58:50.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4820000000000001e-06 2024-09-14T23:58:50.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:50.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426858+0000", 2024-09-14T23:58:50.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050899666000000003 2024-09-14T23:58:50.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:50.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426867+0000", 2024-09-14T23:58:50.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7219999999999994e-06 2024-09-14T23:58:50.410 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.410 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:50.410 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.410 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.410 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5891 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:50.410 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:58:50.410 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 130.99696967200001, 2024-09-14T23:58:50.410 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050948067, 2024-09-14T23:58:50.410 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:50.410 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:50.410 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:50.410 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:50.411 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:50.411 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:50.411 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5891 2024-09-14T23:58:50.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.411 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:50.411 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.411 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:50.411 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:58:50.411 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.411 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.411 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:50.411 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:58:50.412 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.412 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.412 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:50.412 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:58:50.412 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0800000000000001e-07 2024-09-14T23:58:50.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.412 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.412 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:50.412 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375962+0000", 2024-09-14T23:58:50.412 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9600000000000008e-07 2024-09-14T23:58:50.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.412 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.413 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:50.413 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375970+0000", 2024-09-14T23:58:50.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1600000000000001e-06 2024-09-14T23:58:50.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.413 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:50.413 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375972+0000", 2024-09-14T23:58:50.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4849999999999999e-06 2024-09-14T23:58:50.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.413 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.413 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:50.413 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426902+0000", 2024-09-14T23:58:50.413 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050929813999999997 2024-09-14T23:58:50.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.414 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.414 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:50.414 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426909+0000", 2024-09-14T23:58:50.414 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4039999999999998e-06 2024-09-14T23:58:50.414 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.414 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:50.414 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.414 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.414 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5892 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:50.414 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:58:50.414 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 130.99695481500001, 2024-09-14T23:58:50.415 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050965036999999998, 2024-09-14T23:58:50.415 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:50.415 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:50.415 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:50.415 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:50.415 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:50.415 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:50.415 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5892 2024-09-14T23:58:50.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.415 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:50.415 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.415 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:50.415 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:58:50.416 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.416 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.416 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:50.416 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:58:50.416 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.416 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.416 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:50.416 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:58:50.416 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.34e-07 2024-09-14T23:58:50.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.416 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.417 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:50.417 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375978+0000", 2024-09-14T23:58:50.417 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1880000000000001e-06 2024-09-14T23:58:50.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.417 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.417 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:50.417 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375981+0000", 2024-09-14T23:58:50.417 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2399999999999999e-06 2024-09-14T23:58:50.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.417 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.417 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:50.417 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375984+0000", 2024-09-14T23:58:50.417 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9110000000000002e-06 2024-09-14T23:58:50.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.418 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:50.418 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426934+0000", 2024-09-14T23:58:50.418 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050949912 2024-09-14T23:58:50.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.418 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.418 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:50.418 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426941+0000", 2024-09-14T23:58:50.418 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5519999999999999e-06 2024-09-14T23:58:50.418 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.418 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:50.418 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.419 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.419 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5893 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:50.419 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:58:50.419 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 130.99694421800001, 2024-09-14T23:58:50.419 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050975131999999999, 2024-09-14T23:58:50.419 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:50.419 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:50.419 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:50.419 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:50.419 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:50.419 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:50.419 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5893 2024-09-14T23:58:50.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.419 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:50.420 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.420 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:50.420 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:58:50.420 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.420 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.420 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.420 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:50.420 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:58:50.420 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.420 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.420 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.420 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:50.420 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:58:50.421 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2999999999999999e-07 2024-09-14T23:58:50.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.421 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.421 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:50.421 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375988+0000", 2024-09-14T23:58:50.421 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0410000000000001e-06 2024-09-14T23:58:50.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.421 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.421 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:50.421 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375992+0000", 2024-09-14T23:58:50.421 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.104e-06 2024-09-14T23:58:50.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.421 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.421 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:50.422 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375995+0000", 2024-09-14T23:58:50.422 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4310000000000001e-06 2024-09-14T23:58:50.422 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.422 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.422 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:50.422 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426958+0000", 2024-09-14T23:58:50.422 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050963642000000003 2024-09-14T23:58:50.422 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.422 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.422 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:50.422 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426962+0000", 2024-09-14T23:58:50.422 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6839999999999998e-06 2024-09-14T23:58:50.422 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.423 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:50.423 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.423 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.423 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.423 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5894 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:50.423 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:58:50.423 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 130.996932818, 2024-09-14T23:58:50.423 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050978333000000001, 2024-09-14T23:58:50.423 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:50.423 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:50.423 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:50.423 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:50.423 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:50.423 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:50.424 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5894 2024-09-14T23:58:50.424 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.424 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:50.424 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.424 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:50.424 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:58:50.424 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.424 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.424 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.424 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:50.424 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:58:50.424 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.424 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.425 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.425 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:50.425 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:58:50.425 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.23e-07 2024-09-14T23:58:50.425 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.425 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.425 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:50.425 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375999+0000", 2024-09-14T23:58:50.425 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9800000000000002e-07 2024-09-14T23:58:50.425 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.425 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.425 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:50.425 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376007+0000", 2024-09-14T23:58:50.426 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8539999999999997e-06 2024-09-14T23:58:50.426 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.426 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.426 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:50.426 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376011+0000", 2024-09-14T23:58:50.426 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.399e-06 2024-09-14T23:58:50.426 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.426 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.426 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:50.426 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426973+0000", 2024-09-14T23:58:50.426 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050962454999999997 2024-09-14T23:58:50.426 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.426 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.427 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:50.427 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426976+0000", 2024-09-14T23:58:50.427 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4039999999999999e-06 2024-09-14T23:58:50.427 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.427 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:50.427 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.427 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.427 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.427 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5895 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:50.427 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376014+0000", 2024-09-14T23:58:50.427 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 130.99691647399999, 2024-09-14T23:58:50.427 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050975951999999998, 2024-09-14T23:58:50.427 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:50.428 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:50.428 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:50.428 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:50.428 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:50.428 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:50.428 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5895 2024-09-14T23:58:50.428 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.428 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:50.428 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.428 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:50.428 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376014+0000", 2024-09-14T23:58:50.428 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.429 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:50.429 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376014+0000", 2024-09-14T23:58:50.429 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.429 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:50.429 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376015+0000", 2024-09-14T23:58:50.429 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2999999999999999e-07 2024-09-14T23:58:50.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.429 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.429 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:50.430 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376016+0000", 2024-09-14T23:58:50.430 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1659999999999999e-06 2024-09-14T23:58:50.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.430 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:50.430 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376024+0000", 2024-09-14T23:58:50.430 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8930000000000005e-06 2024-09-14T23:58:50.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.430 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.430 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:50.430 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376026+0000", 2024-09-14T23:58:50.430 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6869999999999999e-06 2024-09-14T23:58:50.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.431 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:50.431 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426987+0000", 2024-09-14T23:58:50.431 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050960601000000001 2024-09-14T23:58:50.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.431 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:50.431 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426990+0000", 2024-09-14T23:58:50.431 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3749999999999999e-06 2024-09-14T23:58:50.431 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.431 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:50.431 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.432 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5896 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:50.432 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:58:50.432 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 130.996900737, 2024-09-14T23:58:50.432 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050975656000000001, 2024-09-14T23:58:50.432 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:50.432 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:50.432 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:50.432 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:50.432 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:50.432 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:50.432 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5896 2024-09-14T23:58:50.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.432 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:50.433 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.433 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:50.433 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:58:50.433 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.433 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.433 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:50.433 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:58:50.433 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.433 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.433 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:50.433 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:58:50.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5100000000000001e-07 2024-09-14T23:58:50.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.434 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:50.434 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376032+0000", 2024-09-14T23:58:50.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.141e-06 2024-09-14T23:58:50.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.434 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:50.434 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376036+0000", 2024-09-14T23:58:50.434 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.25e-06 2024-09-14T23:58:50.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.434 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.435 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:50.435 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376038+0000", 2024-09-14T23:58:50.435 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5469999999999999e-06 2024-09-14T23:58:50.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.435 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:50.435 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427002+0000", 2024-09-14T23:58:50.435 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050964029000000001 2024-09-14T23:58:50.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.435 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:50.435 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427006+0000", 2024-09-14T23:58:50.435 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4379999999999999e-06 2024-09-14T23:58:50.436 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.436 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:50.436 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.436 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.436 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5897 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:50.436 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:58:50.436 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 130.99688893699999, 2024-09-14T23:58:50.436 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050978001000000002, 2024-09-14T23:58:50.436 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:50.436 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:50.436 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:50.437 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:50.437 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:50.437 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:50.437 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5897 2024-09-14T23:58:50.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.437 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:50.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.437 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:50.437 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:58:50.437 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.437 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.438 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:50.438 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:58:50.438 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.438 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.438 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:50.438 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:58:50.438 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2600000000000001e-07 2024-09-14T23:58:50.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.438 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.438 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:50.438 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376043+0000", 2024-09-14T23:58:50.438 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.054e-06 2024-09-14T23:58:50.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.439 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.439 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:50.439 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376047+0000", 2024-09-14T23:58:50.439 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0659999999999997e-06 2024-09-14T23:58:50.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.439 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.439 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:50.439 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376050+0000", 2024-09-14T23:58:50.439 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.88e-06 2024-09-14T23:58:50.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.439 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.439 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:50.440 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427017+0000", 2024-09-14T23:58:50.440 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050966453000000002 2024-09-14T23:58:50.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.440 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.440 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:50.440 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427020+0000", 2024-09-14T23:58:50.440 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3220000000000001e-06 2024-09-14T23:58:50.440 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.440 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:50.440 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.440 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.440 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5898 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:50.440 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376053+0000", 2024-09-14T23:58:50.441 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 130.99687745200001, 2024-09-14T23:58:50.441 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050982024000000001, 2024-09-14T23:58:50.441 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:50.441 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:50.441 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:50.441 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:50.441 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:50.441 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:50.441 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5898 2024-09-14T23:58:50.441 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.441 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:50.441 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.441 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:50.442 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376053+0000", 2024-09-14T23:58:50.442 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.442 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.442 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:50.442 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376053+0000", 2024-09-14T23:58:50.442 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.442 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.442 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:50.442 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376278+0000", 2024-09-14T23:58:50.442 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022426400000000001 2024-09-14T23:58:50.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.443 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.443 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:50.443 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376282+0000", 2024-09-14T23:58:50.443 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4739999999999999e-06 2024-09-14T23:58:50.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.443 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.443 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:50.443 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376296+0000", 2024-09-14T23:58:50.443 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4229999999999999e-05 2024-09-14T23:58:50.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.443 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.443 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:50.443 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376303+0000", 2024-09-14T23:58:50.443 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2199999999999997e-06 2024-09-14T23:58:50.444 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.444 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.444 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:50.444 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427031+0000", 2024-09-14T23:58:50.444 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050728154999999997 2024-09-14T23:58:50.444 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.444 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.444 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:50.444 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427036+0000", 2024-09-14T23:58:50.444 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6809999999999997e-06 2024-09-14T23:58:50.444 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.444 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:50.445 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.445 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.445 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.445 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5899 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:50.445 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376338+0000", 2024-09-14T23:58:50.445 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 130.996592728, 2024-09-14T23:58:50.445 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050712100000000003, 2024-09-14T23:58:50.445 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:50.445 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:50.445 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:50.445 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:50.445 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:50.445 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:50.445 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5899 2024-09-14T23:58:50.446 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.446 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:50.446 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.446 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:50.446 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376338+0000", 2024-09-14T23:58:50.446 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.446 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.446 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.446 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:50.446 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376338+0000", 2024-09-14T23:58:50.446 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.446 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.446 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.447 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:50.447 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376339+0000", 2024-09-14T23:58:50.447 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9100000000000003e-07 2024-09-14T23:58:50.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.447 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.447 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:50.447 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376342+0000", 2024-09-14T23:58:50.447 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7e-06 2024-09-14T23:58:50.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.447 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.447 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:50.447 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376350+0000", 2024-09-14T23:58:50.447 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5280000000000003e-06 2024-09-14T23:58:50.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.448 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.448 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:50.448 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376356+0000", 2024-09-14T23:58:50.448 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6010000000000002e-06 2024-09-14T23:58:50.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.448 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.448 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:50.448 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427047+0000", 2024-09-14T23:58:50.448 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050690962999999999 2024-09-14T23:58:50.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.448 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.448 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:50.448 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427050+0000", 2024-09-14T23:58:50.449 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.517e-06 2024-09-14T23:58:50.449 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.449 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:50.449 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.449 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.449 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5900 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:50.449 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376366+0000", 2024-09-14T23:58:50.449 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 130.99656446899999, 2024-09-14T23:58:50.449 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050697858999999998, 2024-09-14T23:58:50.449 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:50.449 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:50.449 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:50.450 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:50.450 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:50.450 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:50.450 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5900 2024-09-14T23:58:50.450 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.450 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:50.450 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.450 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:50.450 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376366+0000", 2024-09-14T23:58:50.450 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.450 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.450 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.450 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:50.450 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376366+0000", 2024-09-14T23:58:50.450 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.451 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.451 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.451 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:50.451 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377650+0000", 2024-09-14T23:58:50.451 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001283834 2024-09-14T23:58:50.451 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.451 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.451 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:50.451 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377655+0000", 2024-09-14T23:58:50.451 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7230000000000004e-06 2024-09-14T23:58:50.451 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.451 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.452 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:50.452 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377667+0000", 2024-09-14T23:58:50.452 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1746000000000001e-05 2024-09-14T23:58:50.452 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.452 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.452 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:50.452 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377676+0000", 2024-09-14T23:58:50.452 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3619999999999995e-06 2024-09-14T23:58:50.452 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.452 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.452 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:50.452 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427061+0000", 2024-09-14T23:58:50.452 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049384755000000002 2024-09-14T23:58:50.452 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.453 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.453 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:50.453 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427064+0000", 2024-09-14T23:58:50.453 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.439e-06 2024-09-14T23:58:50.453 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.453 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:50.453 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.453 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.453 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.453 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:132622 3.1e 3:79d0a5e0:::rbd_data.111847077d5e.0000000000000007:head [sparse-read 0~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T23:58:50.453 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:44.037901+0000", 2024-09-14T23:58:50.453 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 66.335029860000006, 2024-09-14T23:58:50.453 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1083420859999999, 2024-09-14T23:58:50.454 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:50.454 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:50.454 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:50.454 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:50.454 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:50.454 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:50.454 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 132622 2024-09-14T23:58:50.454 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.454 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:50.454 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.454 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:50.454 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037901+0000", 2024-09-14T23:58:50.454 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.455 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.455 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.455 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:50.455 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037901+0000", 2024-09-14T23:58:50.455 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.455 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.455 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.455 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:50.455 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037903+0000", 2024-09-14T23:58:50.455 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4619999999999999e-06 2024-09-14T23:58:50.455 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.455 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.455 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:50.456 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037910+0000", 2024-09-14T23:58:50.456 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7449999999999995e-06 2024-09-14T23:58:50.456 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.456 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.456 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:50.456 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037928+0000", 2024-09-14T23:58:50.456 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7305000000000001e-05 2024-09-14T23:58:50.456 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.456 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.456 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:50.456 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037939+0000", 2024-09-14T23:58:50.456 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1599e-05 2024-09-14T23:58:50.456 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.457 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.457 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:50.457 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.038049+0000", 2024-09-14T23:58:50.457 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000109577 2024-09-14T23:58:50.457 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.457 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.457 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op must wait for map", 2024-09-14T23:58:50.457 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.038057+0000", 2024-09-14T23:58:50.457 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1450000000000008e-06 2024-09-14T23:58:50.457 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.457 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.457 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:50.457 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.039937+0000", 2024-09-14T23:58:50.457 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00188024 2024-09-14T23:58:50.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.458 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.458 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:58:50.458 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.040039+0000", 2024-09-14T23:58:50.458 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000101934 2024-09-14T23:58:50.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.458 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.458 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:50.458 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:45.145805+0000", 2024-09-14T23:58:50.458 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.105765651 2024-09-14T23:58:50.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.458 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.458 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:50.459 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:45.145997+0000", 2024-09-14T23:58:50.459 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000192084 2024-09-14T23:58:50.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.459 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.459 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:50.459 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:45.146243+0000", 2024-09-14T23:58:50.459 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024634400000000001 2024-09-14T23:58:50.459 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.459 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:50.459 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.459 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.459 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4172.0:5 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:50.460 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.482606+0000", 2024-09-14T23:58:50.460 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 42.890324878999998, 2024-09-14T23:58:50.460 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.49381641300000001, 2024-09-14T23:58:50.460 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:50.460 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:50.460 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:50.460 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:50.460 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4172", 2024-09-14T23:58:50.460 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/2542269203", 2024-09-14T23:58:50.460 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5 2024-09-14T23:58:50.460 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.460 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:50.460 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.460 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:50.461 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482606+0000", 2024-09-14T23:58:50.461 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.461 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.461 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.461 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:50.461 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482606+0000", 2024-09-14T23:58:50.461 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.461 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.461 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.461 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:50.461 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482607+0000", 2024-09-14T23:58:50.461 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3120000000000001e-06 2024-09-14T23:58:50.461 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.462 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.462 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:50.462 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482615+0000", 2024-09-14T23:58:50.462 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9300000000000003e-06 2024-09-14T23:58:50.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.462 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.462 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:50.462 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482632+0000", 2024-09-14T23:58:50.462 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6966999999999999e-05 2024-09-14T23:58:50.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.462 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.462 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:50.462 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482640+0000", 2024-09-14T23:58:50.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1820000000000006e-06 2024-09-14T23:58:50.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.463 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.463 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:50.463 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482703+0000", 2024-09-14T23:58:50.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2098000000000001e-05 2024-09-14T23:58:50.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.463 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.463 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:58:50.463 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482713+0000", 2024-09-14T23:58:50.463 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0064000000000001e-05 2024-09-14T23:58:50.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.463 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.464 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:50.464 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.976307+0000", 2024-09-14T23:58:50.464 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.49359442999999997 2024-09-14T23:58:50.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.464 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.464 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:50.464 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.976422+0000", 2024-09-14T23:58:50.464 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011543 2024-09-14T23:58:50.464 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.464 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:50.464 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.464 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.464 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:50.465 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:50.466 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T23:58:50.466 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:50.465+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:50.567 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:58:50.566+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:58:50.574 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:50.574 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:58:50.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.574 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:255886 3.13 3.54329f53 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e150)", 2024-09-14T23:58:50.574 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:49.986852+0000", 2024-09-14T23:58:50.574 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.58637693300000004, 2024-09-14T23:58:50.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.58639902399999999, 2024-09-14T23:58:50.574 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:50.574 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:50.574 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "waiting for peered", 2024-09-14T23:58:50.574 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:50.574 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:50.575 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:50.575 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 255886 2024-09-14T23:58:50.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.575 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:50.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:50.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:49.986852+0000", 2024-09-14T23:58:50.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:50.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:49.986852+0000", 2024-09-14T23:58:50.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:50.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:49.986854+0000", 2024-09-14T23:58:50.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.494e-06 2024-09-14T23:58:50.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:50.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:49.986860+0000", 2024-09-14T23:58:50.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0839999999999997e-06 2024-09-14T23:58:50.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:50.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:49.986875+0000", 2024-09-14T23:58:50.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5041e-05 2024-09-14T23:58:50.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:50.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:49.986884+0000", 2024-09-14T23:58:50.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8190000000000003e-06 2024-09-14T23:58:50.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:50.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:49.986962+0000", 2024-09-14T23:58:50.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7929000000000006e-05 2024-09-14T23:58:50.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:58:50.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:49.986976+0000", 2024-09-14T23:58:50.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3868e-05 2024-09-14T23:58:50.578 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.578 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:50.578 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.578 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:255887 3.13 3.54329f53 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e150)", 2024-09-14T23:58:50.578 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:49.986922+0000", 2024-09-14T23:58:50.578 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 0.586307248, 2024-09-14T23:58:50.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.58637079199999997, 2024-09-14T23:58:50.578 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:50.578 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:50.578 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "waiting for peered", 2024-09-14T23:58:50.579 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:50.579 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:50.579 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:50.579 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 255887 2024-09-14T23:58:50.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.579 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:50.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:50.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:49.986922+0000", 2024-09-14T23:58:50.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:50.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:49.986922+0000", 2024-09-14T23:58:50.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:50.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:49.986922+0000", 2024-09-14T23:58:50.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1099999999999995e-07 2024-09-14T23:58:50.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:50.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:49.986926+0000", 2024-09-14T23:58:50.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3069999999999998e-06 2024-09-14T23:58:50.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:50.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:49.986940+0000", 2024-09-14T23:58:50.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4270999999999999e-05 2024-09-14T23:58:50.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:50.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:49.986956+0000", 2024-09-14T23:58:50.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5875999999999998e-05 2024-09-14T23:58:50.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:50.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:49.987004+0000", 2024-09-14T23:58:50.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7929000000000001e-05 2024-09-14T23:58:50.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:58:50.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:49.987014+0000", 2024-09-14T23:58:50.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0586e-05 2024-09-14T23:58:50.582 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.582 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:50.582 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.582 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.582 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-14T23:58:50.582 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 2 2024-09-14T23:58:50.582 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:50.583 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T23:58:50.668 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:50.667+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:50.691 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:50.691 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:50.691 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:58:50.691 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:58:50.691 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:50.700 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T23:58:50.769 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:50.768+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:50.809 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:50.809 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:58:50.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:58:50.809 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:58:50.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.809 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:56107 3.0 e56/23 3:00c47890:::rbd_data.1118c4b3aebe.0000000000000009:head v 56'125, mlcod=52'123)", 2024-09-14T23:58:50.809 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.236123+0000", 2024-09-14T23:58:50.809 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 104.570540607, 2024-09-14T23:58:50.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0061273220000000002, 2024-09-14T23:58:50.810 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:50.810 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:50.810 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:50.810 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:50.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:50.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.236123+0000", 2024-09-14T23:58:50.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:50.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.236123+0000", 2024-09-14T23:58:50.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:50.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.236125+0000", 2024-09-14T23:58:50.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.313e-06 2024-09-14T23:58:50.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:50.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237428+0000", 2024-09-14T23:58:50.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013029890000000001 2024-09-14T23:58:50.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:50.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237452+0000", 2024-09-14T23:58:50.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4090000000000001e-05 2024-09-14T23:58:50.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:50.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237462+0000", 2024-09-14T23:58:50.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0421999999999999e-05 2024-09-14T23:58:50.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:50.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237559+0000", 2024-09-14T23:58:50.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6378000000000005e-05 2024-09-14T23:58:50.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:50.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237583+0000", 2024-09-14T23:58:50.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4182e-05 2024-09-14T23:58:50.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:50.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.242205+0000", 2024-09-14T23:58:50.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0046217289999999998 2024-09-14T23:58:50.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:50.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.242251+0000", 2024-09-14T23:58:50.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6218999999999999e-05 2024-09-14T23:58:50.814 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.814 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:50.814 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.814 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:56108 3.0 e56/23 3:00c47890:::rbd_data.1118c4b3aebe.0000000000000009:head v 56'127, mlcod=52'123)", 2024-09-14T23:58:50.814 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.239693+0000", 2024-09-14T23:58:50.814 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 104.566971137, 2024-09-14T23:58:50.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0043496209999999997, 2024-09-14T23:58:50.814 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:50.814 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:50.814 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:50.814 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:50.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:50.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239693+0000", 2024-09-14T23:58:50.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:50.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239693+0000", 2024-09-14T23:58:50.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:50.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239694+0000", 2024-09-14T23:58:50.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0499999999999999e-06 2024-09-14T23:58:50.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:50.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239702+0000", 2024-09-14T23:58:50.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1170000000000004e-06 2024-09-14T23:58:50.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:50.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239716+0000", 2024-09-14T23:58:50.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4306e-05 2024-09-14T23:58:50.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:50.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239724+0000", 2024-09-14T23:58:50.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0399999999999993e-06 2024-09-14T23:58:50.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:50.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239778+0000", 2024-09-14T23:58:50.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4032e-05 2024-09-14T23:58:50.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:50.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239801+0000", 2024-09-14T23:58:50.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2770000000000001e-05 2024-09-14T23:58:50.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:50.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.243990+0000", 2024-09-14T23:58:50.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0041890909999999998 2024-09-14T23:58:50.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:50.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.244043+0000", 2024-09-14T23:58:50.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2215000000000001e-05 2024-09-14T23:58:50.818 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.818 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:50.818 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.819 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:56385 3.9 e56/23 3:963d31c7:::rbd_data.1118c4b3aebe.0000000000000003:head v 56'87, mlcod=53'85)", 2024-09-14T23:58:50.819 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.283936+0000", 2024-09-14T23:58:50.819 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 104.522728227, 2024-09-14T23:58:50.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0063042080000000004, 2024-09-14T23:58:50.819 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:50.820 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:50.820 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:50.820 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:50.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:50.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.283936+0000", 2024-09-14T23:58:50.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:50.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.283936+0000", 2024-09-14T23:58:50.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:50.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.283937+0000", 2024-09-14T23:58:50.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1179999999999999e-06 2024-09-14T23:58:50.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:50.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286338+0000", 2024-09-14T23:58:50.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024010149999999998 2024-09-14T23:58:50.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:50.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286360+0000", 2024-09-14T23:58:50.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2212e-05 2024-09-14T23:58:50.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:50.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286370+0000", 2024-09-14T23:58:50.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5850000000000006e-06 2024-09-14T23:58:50.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:50.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286443+0000", 2024-09-14T23:58:50.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2757999999999997e-05 2024-09-14T23:58:50.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:50.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286466+0000", 2024-09-14T23:58:50.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3844000000000001e-05 2024-09-14T23:58:50.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:50.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.290197+0000", 2024-09-14T23:58:50.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003730176 2024-09-14T23:58:50.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:50.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.290240+0000", 2024-09-14T23:58:50.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.35e-05 2024-09-14T23:58:50.823 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.823 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:50.824 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.824 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:56386 3.9 e56/23 3:963d31c7:::rbd_data.1118c4b3aebe.0000000000000003:head v 56'89, mlcod=53'85)", 2024-09-14T23:58:50.824 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.287578+0000", 2024-09-14T23:58:50.824 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 104.519086501, 2024-09-14T23:58:50.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033580250000000002, 2024-09-14T23:58:50.824 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:50.824 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:50.824 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:50.824 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:50.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:50.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287578+0000", 2024-09-14T23:58:50.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:50.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287578+0000", 2024-09-14T23:58:50.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:50.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287579+0000", 2024-09-14T23:58:50.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.64e-07 2024-09-14T23:58:50.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:50.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287586+0000", 2024-09-14T23:58:50.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5310000000000004e-06 2024-09-14T23:58:50.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:50.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287597+0000", 2024-09-14T23:58:50.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0903999999999999e-05 2024-09-14T23:58:50.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:50.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287605+0000", 2024-09-14T23:58:50.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7889999999999995e-06 2024-09-14T23:58:50.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:50.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.288437+0000", 2024-09-14T23:58:50.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000831914 2024-09-14T23:58:50.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:50.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.288460+0000", 2024-09-14T23:58:50.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3410000000000001e-05 2024-09-14T23:58:50.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:50.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.290883+0000", 2024-09-14T23:58:50.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024230599999999999 2024-09-14T23:58:50.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:50.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.290936+0000", 2024-09-14T23:58:50.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2453000000000001e-05 2024-09-14T23:58:50.828 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.828 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:50.828 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.828 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:64263 3.4 e57/23 3:20c6a768:::rbd_data.11188d356229.0000000000000001:head v 57'165, mlcod=57'163)", 2024-09-14T23:58:50.829 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:07.999031+0000", 2024-09-14T23:58:50.829 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 102.807633353, 2024-09-14T23:58:50.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023815210000000002, 2024-09-14T23:58:50.829 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:50.829 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:50.829 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:50.829 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:50.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:50.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999031+0000", 2024-09-14T23:58:50.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:50.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999031+0000", 2024-09-14T23:58:50.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:50.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999032+0000", 2024-09-14T23:58:50.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.176e-06 2024-09-14T23:58:50.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:50.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999660+0000", 2024-09-14T23:58:50.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00062817099999999996 2024-09-14T23:58:50.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:50.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999683+0000", 2024-09-14T23:58:50.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2816999999999999e-05 2024-09-14T23:58:50.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:50.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999693+0000", 2024-09-14T23:58:50.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0088999999999999e-05 2024-09-14T23:58:50.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:50.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999767+0000", 2024-09-14T23:58:50.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4056999999999999e-05 2024-09-14T23:58:50.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:50.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999793+0000", 2024-09-14T23:58:50.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5573000000000001e-05 2024-09-14T23:58:50.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:50.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:08.001379+0000", 2024-09-14T23:58:50.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015859019999999999 2024-09-14T23:58:50.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:50.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:08.001412+0000", 2024-09-14T23:58:50.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3735999999999997e-05 2024-09-14T23:58:50.833 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.833 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:50.833 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.833 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:66570 3.0 e60/23 3:055350b3:::rbd_data.111873792d9b.0000000000000005:head v 60'162, mlcod=57'159)", 2024-09-14T23:58:50.833 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:11.277517+0000", 2024-09-14T23:58:50.833 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 99.529147197, 2024-09-14T23:58:50.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003472199, 2024-09-14T23:58:50.833 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:50.833 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:50.834 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:50.834 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:50.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:50.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.277517+0000", 2024-09-14T23:58:50.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:50.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.277517+0000", 2024-09-14T23:58:50.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:50.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.277518+0000", 2024-09-14T23:58:50.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1379999999999999e-06 2024-09-14T23:58:50.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:50.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278504+0000", 2024-09-14T23:58:50.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00098617499999999994 2024-09-14T23:58:50.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:50.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278525+0000", 2024-09-14T23:58:50.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.09e-05 2024-09-14T23:58:50.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:50.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278534+0000", 2024-09-14T23:58:50.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6209999999999993e-06 2024-09-14T23:58:50.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:50.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278611+0000", 2024-09-14T23:58:50.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6926000000000002e-05 2024-09-14T23:58:50.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:50.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278641+0000", 2024-09-14T23:58:50.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0861999999999997e-05 2024-09-14T23:58:50.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:50.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.280953+0000", 2024-09-14T23:58:50.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023114239999999999 2024-09-14T23:58:50.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:50.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.280989+0000", 2024-09-14T23:58:50.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6152999999999998e-05 2024-09-14T23:58:50.837 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.837 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:50.837 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.838 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:158983 3.4 e103/23 3:20e43b29:::rbd_data.1118b2f495e6.0000000000000012:head v 103'454, mlcod=103'452)", 2024-09-14T23:58:50.838 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:58.917699+0000", 2024-09-14T23:58:50.838 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 51.888965147, 2024-09-14T23:58:50.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037933799999999998, 2024-09-14T23:58:50.838 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:50.838 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:50.838 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:50.838 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:50.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:50.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917699+0000", 2024-09-14T23:58:50.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:50.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917699+0000", 2024-09-14T23:58:50.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:50.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917700+0000", 2024-09-14T23:58:50.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3850000000000001e-06 2024-09-14T23:58:50.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:50.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917939+0000", 2024-09-14T23:58:50.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000238249 2024-09-14T23:58:50.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:50.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917961+0000", 2024-09-14T23:58:50.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2894000000000001e-05 2024-09-14T23:58:50.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:50.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917972+0000", 2024-09-14T23:58:50.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0261000000000001e-05 2024-09-14T23:58:50.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:50.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.918038+0000", 2024-09-14T23:58:50.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6223000000000006e-05 2024-09-14T23:58:50.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:50.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.918063+0000", 2024-09-14T23:58:50.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5252000000000001e-05 2024-09-14T23:58:50.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:50.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.921433+0000", 2024-09-14T23:58:50.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033699960000000001 2024-09-14T23:58:50.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:50.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.921492+0000", 2024-09-14T23:58:50.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9120000000000003e-05 2024-09-14T23:58:50.842 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.842 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:50.842 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.842 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(osd.1.0:1460 3.0 e122/23 3:00cc1e32:::rbd_data.11187b9f1032.0000000000000001:2f v 122'453, mlcod=118'449)", 2024-09-14T23:58:50.842 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:22.287423+0000", 2024-09-14T23:58:50.843 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.519241489999999, 2024-09-14T23:58:50.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0046041659999999998, 2024-09-14T23:58:50.843 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:50.843 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:50.843 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:50.843 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:50.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:50.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.287423+0000", 2024-09-14T23:58:50.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:50.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.287423+0000", 2024-09-14T23:58:50.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:50.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.287423+0000", 2024-09-14T23:58:50.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7500000000000001e-07 2024-09-14T23:58:50.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:50.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.287435+0000", 2024-09-14T23:58:50.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2006e-05 2024-09-14T23:58:50.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:50.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.287444+0000", 2024-09-14T23:58:50.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.465e-06 2024-09-14T23:58:50.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:50.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.287453+0000", 2024-09-14T23:58:50.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0100000000000001e-06 2024-09-14T23:58:50.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:50.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.287519+0000", 2024-09-14T23:58:50.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6149999999999995e-05 2024-09-14T23:58:50.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:50.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.287543+0000", 2024-09-14T23:58:50.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3682000000000001e-05 2024-09-14T23:58:50.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:50.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.291977+0000", 2024-09-14T23:58:50.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0044333829999999999 2024-09-14T23:58:50.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:50.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.292027+0000", 2024-09-14T23:58:50.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0195000000000003e-05 2024-09-14T23:58:50.847 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.847 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:50.847 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.847 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDPGPush(3.1e 126/125 [PushOp(3:7f7265c3:::rbd_data.11188d356229.000000000000000d:16, version: 58'124, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:7f7265c3:::rbd_data.11188d356229.000000000000000d:16@58'124, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 16=[]:{16: [16]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T23:58:50.847 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:26.552687+0000", 2024-09-14T23:58:50.847 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 24.253977226, 2024-09-14T23:58:50.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023212060000000001, 2024-09-14T23:58:50.847 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:50.847 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:50.848 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:50.848 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:50.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:50.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.552687+0000", 2024-09-14T23:58:50.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:50.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.552687+0000", 2024-09-14T23:58:50.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:50.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.552688+0000", 2024-09-14T23:58:50.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.47e-07 2024-09-14T23:58:50.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:50.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.553659+0000", 2024-09-14T23:58:50.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00097088899999999995 2024-09-14T23:58:50.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:50.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.553682+0000", 2024-09-14T23:58:50.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3862e-05 2024-09-14T23:58:50.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:50.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.553703+0000", 2024-09-14T23:58:50.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0310999999999999e-05 2024-09-14T23:58:50.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:50.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.553813+0000", 2024-09-14T23:58:50.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011051 2024-09-14T23:58:50.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:50.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.553828+0000", 2024-09-14T23:58:50.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4625e-05 2024-09-14T23:58:50.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:50.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.555008+0000", 2024-09-14T23:58:50.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001180162 2024-09-14T23:58:50.851 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.851 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:50.851 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.851 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(osd.1.0:1465 3.4 e127/23 3:265f7763:::rbd_data.1118f9c96a7.0000000000000004:2c v 127'508, mlcod=120'505)", 2024-09-14T23:58:50.851 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:27.318056+0000", 2024-09-14T23:58:50.851 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 23.488607803000001, 2024-09-14T23:58:50.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0049337449999999998, 2024-09-14T23:58:50.852 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:50.852 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:50.852 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:50.852 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:50.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:50.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.318056+0000", 2024-09-14T23:58:50.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:50.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.318056+0000", 2024-09-14T23:58:50.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:50.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.318057+0000", 2024-09-14T23:58:50.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9799999999999999e-07 2024-09-14T23:58:50.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:50.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.318061+0000", 2024-09-14T23:58:50.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7899999999999999e-06 2024-09-14T23:58:50.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:50.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.318074+0000", 2024-09-14T23:58:50.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2537999999999999e-05 2024-09-14T23:58:50.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:50.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.318083+0000", 2024-09-14T23:58:50.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3829999999999998e-06 2024-09-14T23:58:50.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:50.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.318165+0000", 2024-09-14T23:58:50.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1928999999999994e-05 2024-09-14T23:58:50.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:50.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.318199+0000", 2024-09-14T23:58:50.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3587000000000002e-05 2024-09-14T23:58:50.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:50.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.322939+0000", 2024-09-14T23:58:50.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0047397730000000001 2024-09-14T23:58:50.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:50.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.322990+0000", 2024-09-14T23:58:50.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1347000000000002e-05 2024-09-14T23:58:50.855 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.856 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:50.856 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.856 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:252476 3.e 3:725cbf00:::rbd_header.11186acd4a72:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e132)", 2024-09-14T23:58:50.856 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:32.121781+0000", 2024-09-14T23:58:50.856 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.684882966, 2024-09-14T23:58:50.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022671459999999998, 2024-09-14T23:58:50.856 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:50.856 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:50.856 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:50.856 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:50.857 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:50.857 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:50.857 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 252476 2024-09-14T23:58:50.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.857 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:50.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:50.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:32.121781+0000", 2024-09-14T23:58:50.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:50.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:32.121781+0000", 2024-09-14T23:58:50.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:50.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:32.121783+0000", 2024-09-14T23:58:50.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.531e-06 2024-09-14T23:58:50.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:50.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:32.121790+0000", 2024-09-14T23:58:50.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7670000000000003e-06 2024-09-14T23:58:50.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:50.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:32.121806+0000", 2024-09-14T23:58:50.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5291999999999999e-05 2024-09-14T23:58:50.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:50.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:32.121813+0000", 2024-09-14T23:58:50.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.418e-06 2024-09-14T23:58:50.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.859 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:50.859 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:32.121879+0000", 2024-09-14T23:58:50.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5811999999999999e-05 2024-09-14T23:58:50.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op must wait for map", 2024-09-14T23:58:50.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:32.121888+0000", 2024-09-14T23:58:50.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6600000000000001e-06 2024-09-14T23:58:50.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:50.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:32.123710+0000", 2024-09-14T23:58:50.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001822838 2024-09-14T23:58:50.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:50.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:32.123803+0000", 2024-09-14T23:58:50.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3077000000000003e-05 2024-09-14T23:58:50.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:50.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:32.124048+0000", 2024-09-14T23:58:50.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024475099999999997 2024-09-14T23:58:50.861 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.861 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:50.861 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.861 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:253435 3.5 3:a6fcd3e0:::rbd_data.1118e50f549.000000000000004a:head [delete] snapc 24=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-14T23:58:50.861 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:36.070548+0000", 2024-09-14T23:58:50.861 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.736116484, 2024-09-14T23:58:50.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022804000000000001, 2024-09-14T23:58:50.862 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:50.862 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:50.862 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:50.862 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:50.862 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:50.862 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:50.862 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 253435 2024-09-14T23:58:50.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.862 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:50.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:50.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.070548+0000", 2024-09-14T23:58:50.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:50.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.070548+0000", 2024-09-14T23:58:50.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:50.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.070549+0000", 2024-09-14T23:58:50.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6500000000000001e-06 2024-09-14T23:58:50.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:50.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.070555+0000", 2024-09-14T23:58:50.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1750000000000002e-06 2024-09-14T23:58:50.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.864 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:50.864 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.070571+0000", 2024-09-14T23:58:50.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5719999999999999e-05 2024-09-14T23:58:50.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.864 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:50.864 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.070580+0000", 2024-09-14T23:58:50.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0699999999999996e-06 2024-09-14T23:58:50.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.864 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.864 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:50.864 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.070658+0000", 2024-09-14T23:58:50.864 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7525000000000001e-05 2024-09-14T23:58:50.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:50.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.070799+0000", 2024-09-14T23:58:50.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014083499999999999 2024-09-14T23:58:50.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-14T23:58:50.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.070927+0000", 2024-09-14T23:58:50.865 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012853100000000001 2024-09-14T23:58:50.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.865 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.865 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:58:50.865 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.071666+0000", 2024-09-14T23:58:50.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00073853000000000005 2024-09-14T23:58:50.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:50.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.072750+0000", 2024-09-14T23:58:50.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001084738 2024-09-14T23:58:50.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:50.866 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.072791+0000", 2024-09-14T23:58:50.866 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0330999999999997e-05 2024-09-14T23:58:50.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.866 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.866 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:50.867 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.072828+0000", 2024-09-14T23:58:50.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7295000000000001e-05 2024-09-14T23:58:50.867 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.867 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:50.867 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.867 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.867 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:253690 3.e 3:725cbf00:::rbd_header.11186acd4a72:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-14T23:58:50.867 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:37.091642+0000", 2024-09-14T23:58:50.867 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.715022186000001, 2024-09-14T23:58:50.867 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0026525030000000001, 2024-09-14T23:58:50.867 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:50.867 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:50.867 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:50.868 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:50.868 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:50.868 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:50.868 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 253690 2024-09-14T23:58:50.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.868 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:50.868 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.868 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:50.868 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:37.091642+0000", 2024-09-14T23:58:50.868 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.869 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:50.869 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:37.091642+0000", 2024-09-14T23:58:50.869 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.869 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.869 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:50.869 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:37.091644+0000", 2024-09-14T23:58:50.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9829999999999999e-06 2024-09-14T23:58:50.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:50.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:37.091654+0000", 2024-09-14T23:58:50.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8619999999999998e-06 2024-09-14T23:58:50.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.870 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:50.870 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:37.091673+0000", 2024-09-14T23:58:50.870 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9094e-05 2024-09-14T23:58:50.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:50.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:37.091681+0000", 2024-09-14T23:58:50.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9449999999999993e-06 2024-09-14T23:58:50.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:50.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:37.091752+0000", 2024-09-14T23:58:50.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1468000000000001e-05 2024-09-14T23:58:50.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.871 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op must wait for map", 2024-09-14T23:58:50.871 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:37.091764+0000", 2024-09-14T23:58:50.871 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2067e-05 2024-09-14T23:58:50.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:50.872 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:37.093986+0000", 2024-09-14T23:58:50.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022213720000000001 2024-09-14T23:58:50.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:50.872 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:37.094082+0000", 2024-09-14T23:58:50.872 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5872999999999995e-05 2024-09-14T23:58:50.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.872 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:50.873 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:37.094294+0000", 2024-09-14T23:58:50.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021283900000000001 2024-09-14T23:58:50.873 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.873 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:50.873 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.873 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:254188 3.5 3:a13778a5:::rbd_header.1118ce574610:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e140)", 2024-09-14T23:58:50.873 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:40.008804+0000", 2024-09-14T23:58:50.873 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.79785989, 2024-09-14T23:58:50.873 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0044649980000000004, 2024-09-14T23:58:50.873 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:50.873 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:50.873 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:50.874 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:50.874 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:50.874 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:50.874 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254188 2024-09-14T23:58:50.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.874 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:50.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:50.874 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.008804+0000", 2024-09-14T23:58:50.874 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.874 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:50.874 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.008804+0000", 2024-09-14T23:58:50.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:50.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.008806+0000", 2024-09-14T23:58:50.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.824e-06 2024-09-14T23:58:50.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.875 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:50.875 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.008813+0000", 2024-09-14T23:58:50.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8889999999999996e-06 2024-09-14T23:58:50.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.875 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:50.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.008826+0000", 2024-09-14T23:58:50.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.29e-05 2024-09-14T23:58:50.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:50.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.008836+0000", 2024-09-14T23:58:50.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9660000000000008e-06 2024-09-14T23:58:50.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.876 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:50.876 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.008906+0000", 2024-09-14T23:58:50.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0165000000000003e-05 2024-09-14T23:58:50.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:50.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.009004+0000", 2024-09-14T23:58:50.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8412999999999994e-05 2024-09-14T23:58:50.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-14T23:58:50.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.009349+0000", 2024-09-14T23:58:50.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00034510200000000002 2024-09-14T23:58:50.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:58:50.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.010086+0000", 2024-09-14T23:58:50.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00073696000000000005 2024-09-14T23:58:50.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:50.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.013157+0000", 2024-09-14T23:58:50.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0030706470000000001 2024-09-14T23:58:50.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:50.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.013222+0000", 2024-09-14T23:58:50.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4479000000000004e-05 2024-09-14T23:58:50.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:50.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.013269+0000", 2024-09-14T23:58:50.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7652999999999999e-05 2024-09-14T23:58:50.879 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.879 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:50.879 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.879 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:254194 3.5 3:a13778a5:::rbd_header.1118ce574610:head [watch unwatch cookie 140670715072400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e140)", 2024-09-14T23:58:50.879 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:40.018920+0000", 2024-09-14T23:58:50.879 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 10.787744514, 2024-09-14T23:58:50.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023612640000000001, 2024-09-14T23:58:50.880 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:50.880 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:50.880 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:50.880 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:50.880 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:50.880 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:50.880 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254194 2024-09-14T23:58:50.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.880 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:50.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.880 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:50.880 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.018920+0000", 2024-09-14T23:58:50.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:50.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.018920+0000", 2024-09-14T23:58:50.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:50.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.018920+0000", 2024-09-14T23:58:50.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7700000000000004e-07 2024-09-14T23:58:50.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.881 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:50.881 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.018923+0000", 2024-09-14T23:58:50.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9840000000000001e-06 2024-09-14T23:58:50.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.882 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:50.882 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.018931+0000", 2024-09-14T23:58:50.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8089999999999993e-06 2024-09-14T23:58:50.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.882 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:50.882 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.018937+0000", 2024-09-14T23:58:50.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1770000000000003e-06 2024-09-14T23:58:50.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.882 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:50.883 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.019163+0000", 2024-09-14T23:58:50.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022563800000000001 2024-09-14T23:58:50.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.883 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:50.883 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.019233+0000", 2024-09-14T23:58:50.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9698999999999998e-05 2024-09-14T23:58:50.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.883 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-14T23:58:50.883 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.019383+0000", 2024-09-14T23:58:50.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015073800000000001 2024-09-14T23:58:50.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:58:50.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.020341+0000", 2024-09-14T23:58:50.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00095732900000000004 2024-09-14T23:58:50.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:50.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.021141+0000", 2024-09-14T23:58:50.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00080075600000000002 2024-09-14T23:58:50.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:50.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.021189+0000", 2024-09-14T23:58:50.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7318999999999999e-05 2024-09-14T23:58:50.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.885 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:50.885 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.021281+0000", 2024-09-14T23:58:50.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2038000000000002e-05 2024-09-14T23:58:50.885 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.885 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:50.885 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.885 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(osd.4.0:4196 3.13 e141/113 3:cecefa16:::rbd_data.1118c250b6de.0000000000000001:1f v 141'878, mlcod=138'874)", 2024-09-14T23:58:50.885 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:41.015011+0000", 2024-09-14T23:58:50.885 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.7916535670000009, 2024-09-14T23:58:50.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025205499999999999, 2024-09-14T23:58:50.886 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:50.886 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:50.886 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:50.886 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:50.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:50.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:41.015011+0000", 2024-09-14T23:58:50.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.886 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:50.886 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:41.015011+0000", 2024-09-14T23:58:50.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:50.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:41.015011+0000", 2024-09-14T23:58:50.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9900000000000002e-07 2024-09-14T23:58:50.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:50.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:41.015018+0000", 2024-09-14T23:58:50.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2910000000000002e-06 2024-09-14T23:58:50.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:50.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:41.015033+0000", 2024-09-14T23:58:50.888 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4625e-05 2024-09-14T23:58:50.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.888 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:50.888 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:41.015043+0000", 2024-09-14T23:58:50.888 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0584999999999999e-05 2024-09-14T23:58:50.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.888 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:50.888 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:41.015114+0000", 2024-09-14T23:58:50.888 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0541000000000001e-05 2024-09-14T23:58:50.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.888 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:50.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:41.015138+0000", 2024-09-14T23:58:50.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3810000000000001e-05 2024-09-14T23:58:50.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:50.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:41.017481+0000", 2024-09-14T23:58:50.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023433619999999999 2024-09-14T23:58:50.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:50.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:41.017531+0000", 2024-09-14T23:58:50.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0037000000000002e-05 2024-09-14T23:58:50.889 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.890 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:50.890 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.890 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(osd.4.0:4316 3.9 e146/23 3:91eaed90:::rbd_data.11182ee2885e.000000000000000c:1a v 146'308, mlcod=142'303)", 2024-09-14T23:58:50.890 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:46.046915+0000", 2024-09-14T23:58:50.890 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.7597490909999998, 2024-09-14T23:58:50.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022583619999999999, 2024-09-14T23:58:50.890 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:50.890 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:50.890 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:50.890 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:50.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:50.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.046915+0000", 2024-09-14T23:58:50.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:50.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.046915+0000", 2024-09-14T23:58:50.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.891 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:50.891 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.046916+0000", 2024-09-14T23:58:50.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6300000000000004e-07 2024-09-14T23:58:50.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:50.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.046922+0000", 2024-09-14T23:58:50.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2339999999999999e-06 2024-09-14T23:58:50.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:50.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.046933+0000", 2024-09-14T23:58:50.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0795000000000001e-05 2024-09-14T23:58:50.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:50.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.046950+0000", 2024-09-14T23:58:50.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7149000000000001e-05 2024-09-14T23:58:50.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:50.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.047000+0000", 2024-09-14T23:58:50.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9654000000000003e-05 2024-09-14T23:58:50.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:50.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.047020+0000", 2024-09-14T23:58:50.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0199000000000001e-05 2024-09-14T23:58:50.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:50.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.049110+0000", 2024-09-14T23:58:50.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0020902970000000001 2024-09-14T23:58:50.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.894 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:50.894 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.049173+0000", 2024-09-14T23:58:50.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3270999999999998e-05 2024-09-14T23:58:50.894 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.894 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:50.894 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.894 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(osd.6.0:1718 3.1e e146/127 3:7a74d0d0:::rbd_data.11182ee2885e.0000000000000006:1a v 146'435, mlcod=145'432)", 2024-09-14T23:58:50.894 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:46.047607+0000", 2024-09-14T23:58:50.894 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.7590573689999998, 2024-09-14T23:58:50.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0026434779999999999, 2024-09-14T23:58:50.895 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:50.895 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:50.895 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:50.895 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:50.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:50.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.047607+0000", 2024-09-14T23:58:50.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:50.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.047607+0000", 2024-09-14T23:58:50.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.896 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:50.896 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.047607+0000", 2024-09-14T23:58:50.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4700000000000002e-07 2024-09-14T23:58:50.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.896 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:50.896 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.047612+0000", 2024-09-14T23:58:50.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1959999999999999e-06 2024-09-14T23:58:50.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.896 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:50.897 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.047622+0000", 2024-09-14T23:58:50.897 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0169999999999999e-05 2024-09-14T23:58:50.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.897 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:50.897 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.047630+0000", 2024-09-14T23:58:50.897 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7249999999999997e-06 2024-09-14T23:58:50.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.897 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:50.897 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.047739+0000", 2024-09-14T23:58:50.897 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108625 2024-09-14T23:58:50.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.898 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:50.898 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.047756+0000", 2024-09-14T23:58:50.898 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6866000000000001e-05 2024-09-14T23:58:50.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.898 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.898 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:50.898 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.050197+0000", 2024-09-14T23:58:50.898 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024412689999999998 2024-09-14T23:58:50.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.898 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.898 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:50.898 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.050250+0000", 2024-09-14T23:58:50.898 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3279999999999998e-05 2024-09-14T23:58:50.898 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.899 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:50.899 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.899 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:255323 3.e 3:74b99397:::rbd_data.11182ee2885e.0000000000000004:head [delete] snapc 1a=[] ondisk+write+known_if_redirected+supports_pool_eio e146)", 2024-09-14T23:58:50.899 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:46.111665+0000", 2024-09-14T23:58:50.899 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.694999074, 2024-09-14T23:58:50.899 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038303019999999998, 2024-09-14T23:58:50.899 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:50.899 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:50.899 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:50.899 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:50.899 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:50.900 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:50.900 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 255323 2024-09-14T23:58:50.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.900 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:50.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:50.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.111665+0000", 2024-09-14T23:58:50.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:50.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.111665+0000", 2024-09-14T23:58:50.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:50.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.111667+0000", 2024-09-14T23:58:50.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3e-06 2024-09-14T23:58:50.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:50.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.111676+0000", 2024-09-14T23:58:50.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5699999999999993e-06 2024-09-14T23:58:50.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:50.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.111698+0000", 2024-09-14T23:58:50.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2263e-05 2024-09-14T23:58:50.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.902 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:50.902 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.111709+0000", 2024-09-14T23:58:50.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1087000000000001e-05 2024-09-14T23:58:50.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.902 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:50.902 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.111787+0000", 2024-09-14T23:58:50.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8212000000000003e-05 2024-09-14T23:58:50.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.903 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:50.903 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.111886+0000", 2024-09-14T23:58:50.903 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8645000000000002e-05 2024-09-14T23:58:50.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.903 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.903 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-14T23:58:50.903 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.112014+0000", 2024-09-14T23:58:50.903 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012753800000000001 2024-09-14T23:58:50.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.903 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.903 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:58:50.903 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.113790+0000", 2024-09-14T23:58:50.903 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00177669 2024-09-14T23:58:50.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.904 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:50.904 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.115396+0000", 2024-09-14T23:58:50.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001605216 2024-09-14T23:58:50.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.904 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:50.904 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.115451+0000", 2024-09-14T23:58:50.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5534000000000001e-05 2024-09-14T23:58:50.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.904 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:50.904 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.115495+0000", 2024-09-14T23:58:50.905 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4246999999999998e-05 2024-09-14T23:58:50.905 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.905 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:50.905 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.905 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:255325 3.1e e146/127 3:7a74d0d0:::rbd_data.11182ee2885e.0000000000000006:head v 146'436, mlcod=145'433)", 2024-09-14T23:58:50.905 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:46.112042+0000", 2024-09-14T23:58:50.905 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 4.6946221369999996, 2024-09-14T23:58:50.905 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023640530000000001, 2024-09-14T23:58:50.905 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:50.905 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:50.905 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:50.906 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:50.906 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.906 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:50.906 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.112042+0000", 2024-09-14T23:58:50.906 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.906 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.906 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:50.906 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.112042+0000", 2024-09-14T23:58:50.906 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:50.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.906 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.906 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:50.906 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.112043+0000", 2024-09-14T23:58:50.907 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1200000000000001e-06 2024-09-14T23:58:50.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.907 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:50.907 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.112051+0000", 2024-09-14T23:58:50.907 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3310000000000004e-06 2024-09-14T23:58:50.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.907 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:50.907 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.112063+0000", 2024-09-14T23:58:50.907 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2028000000000001e-05 2024-09-14T23:58:50.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.907 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:50.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.112071+0000", 2024-09-14T23:58:50.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4000000000000003e-06 2024-09-14T23:58:50.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.908 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:50.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.112155+0000", 2024-09-14T23:58:50.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4064e-05 2024-09-14T23:58:50.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.908 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:50.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.112181+0000", 2024-09-14T23:58:50.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5817999999999999e-05 2024-09-14T23:58:50.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:50.909 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.114372+0000", 2024-09-14T23:58:50.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0021908129999999998 2024-09-14T23:58:50.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:50.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:50.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:50.909 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.114406+0000", 2024-09-14T23:58:50.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4479e-05 2024-09-14T23:58:50.909 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.909 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:50.909 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.909 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:50.909 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:50.910 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:50.911 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T23:58:50.912 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:50.910+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:50.991 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T23:58:51.011 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:51.010+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:51.067 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:51.067 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:51.067 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:58:51.067 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:51.076 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T23:58:51.119 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:51.118+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:51.120 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:58:51.121 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:58:51.138 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:58:51.138 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:58:51.155 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:58:51.155 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:58:51.171 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:58:51.172 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:58:51.185 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:51.185 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:51.189 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:58:51.189 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:58:51.194 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T23:58:51.207 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:58:51.207 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:58:51.213 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:51.212+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:51.226 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:58:51.226 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:58:51.245 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:58:51.245 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:58:51.314 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:51.312+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:51.324 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:51.324 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:51.336 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T23:58:51.414 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:51.413+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:51.472 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:51.472 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:51.482 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T23:58:51.515 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:51.513+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:51.602 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:51.603 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:51.612 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T23:58:51.615 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:51.614+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:51.716 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:51.715+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:51.730 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:51.730 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:51.739 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T23:58:51.823 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:51.822+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:51.906 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:51.906 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:51.916 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T23:58:51.918 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:51.917+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:52.000 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T23:58:52.045 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:52.044+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:52.083 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:52.084 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:52.093 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T23:58:52.119 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:52.118+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:52.209 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:52.209 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:52.219 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T23:58:52.220 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:52.218+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:52.321 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:58:52.320+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:58:52.339 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:52.339 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:52.349 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T23:58:52.423 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:52.421+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:52.515 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:52.515 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:52.523 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:52.521+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:52.524 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T23:58:52.623 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:52.622+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:52.636 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:52.636 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:52.646 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T23:58:52.724 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:52.723+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:52.802 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:52.802 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:52.812 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T23:58:52.825 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:52.823+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:52.926 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:58:52.924+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:58:52.928 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:52.928 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:52.937 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T23:58:52.978 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is unset 2024-09-14T23:58:53.026 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:53.025+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:53.100 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:53.100 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:53.109 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T23:58:53.150 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:53.149+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:53.222 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:53.222 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:53.229 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:53.227+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:53.231 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T23:58:53.257 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T23:58:53.330 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:53.328+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:53.383 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:53.384 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:53.394 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T23:58:53.405 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:58:53.405 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:58:53.422 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:58:53.423 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:58:53.431 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:53.429+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:53.440 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:58:53.440 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:58:53.459 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:58:53.459 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:58:53.476 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:58:53.477 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:58:53.494 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:58:53.494 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:58:53.506 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:53.507 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:58:53.507 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:58:53.507 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:58:53.507 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.507 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:43054 3.1f 3:ffbfe596:::rbd_data.1118350d5ae5.00000000000002eb:head [set-alloc-hint object_size 262144 write_size 262144,write 196608~22770] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T23:58:53.507 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:58:53.507 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 120.53784665000001, 2024-09-14T23:58:53.508 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.99973334400000002, 2024-09-14T23:58:53.508 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:53.508 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:53.508 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:53.508 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:53.508 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:53.508 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:53.508 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 43054 2024-09-14T23:58:53.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.508 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:53.508 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.508 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:53.508 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:58:53.509 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:53.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.509 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.509 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:53.510 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:58:53.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:53.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.510 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:53.510 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966552+0000", 2024-09-14T23:58:53.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3790000000000001e-06 2024-09-14T23:58:53.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.510 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.510 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:53.510 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966565+0000", 2024-09-14T23:58:53.510 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2459999999999999e-05 2024-09-14T23:58:53.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.511 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:53.511 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966576+0000", 2024-09-14T23:58:53.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1039000000000001e-05 2024-09-14T23:58:53.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.511 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:53.511 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966582+0000", 2024-09-14T23:58:53.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9660000000000001e-06 2024-09-14T23:58:53.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.511 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:53.511 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966625+0000", 2024-09-14T23:58:53.511 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3529999999999998e-05 2024-09-14T23:58:53.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.512 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:58:53.512 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966632+0000", 2024-09-14T23:58:53.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9809999999999997e-06 2024-09-14T23:58:53.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.512 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:53.512 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.964460+0000", 2024-09-14T23:58:53.512 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.99782768300000002 2024-09-14T23:58:53.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.512 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:53.513 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.964641+0000", 2024-09-14T23:58:53.513 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000181089 2024-09-14T23:58:53.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.513 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-14T23:58:53.513 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.964809+0000", 2024-09-14T23:58:53.513 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000168505 2024-09-14T23:58:53.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.513 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:58:53.513 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966030+0000", 2024-09-14T23:58:53.513 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012206929999999999 2024-09-14T23:58:53.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:53.514 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966181+0000", 2024-09-14T23:58:53.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000150569 2024-09-14T23:58:53.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:53.514 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966243+0000", 2024-09-14T23:58:53.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2331999999999999e-05 2024-09-14T23:58:53.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.514 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:53.514 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966284+0000", 2024-09-14T23:58:53.514 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1118000000000002e-05 2024-09-14T23:58:53.514 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:53.515 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:53.515 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:53.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.515 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184314 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4162560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:53.515 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.077247+0000", 2024-09-14T23:58:53.515 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.427149982000003, 2024-09-14T23:58:53.515 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058772256000000002, 2024-09-14T23:58:53.515 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:53.515 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:53.515 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:53.515 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:53.515 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:53.516 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:53.516 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184314 2024-09-14T23:58:53.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.516 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:53.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.516 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:53.516 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.077247+0000", 2024-09-14T23:58:53.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:53.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.516 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:53.516 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.077247+0000", 2024-09-14T23:58:53.516 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:53.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.517 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:53.517 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.078906+0000", 2024-09-14T23:58:53.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016583279999999999 2024-09-14T23:58:53.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.517 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:53.517 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.078911+0000", 2024-09-14T23:58:53.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0420000000000002e-06 2024-09-14T23:58:53.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.517 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:53.517 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.078922+0000", 2024-09-14T23:58:53.517 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1354e-05 2024-09-14T23:58:53.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.518 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:53.518 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.078930+0000", 2024-09-14T23:58:53.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5730000000000002e-06 2024-09-14T23:58:53.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.518 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:53.518 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.135870+0000", 2024-09-14T23:58:53.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056940651000000002 2024-09-14T23:58:53.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.518 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:53.518 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.135951+0000", 2024-09-14T23:58:53.518 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0890999999999995e-05 2024-09-14T23:58:53.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.519 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:53.519 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.136020+0000", 2024-09-14T23:58:53.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8417000000000006e-05 2024-09-14T23:58:53.519 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:53.519 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:53.519 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:53.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.519 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184323 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4183040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:53.519 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.079228+0000", 2024-09-14T23:58:53.519 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.425168908000003, 2024-09-14T23:58:53.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058454487999999999, 2024-09-14T23:58:53.520 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:53.520 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:53.520 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:53.520 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:53.520 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:53.520 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:53.520 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184323 2024-09-14T23:58:53.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.520 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:53.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.520 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:53.520 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079228+0000", 2024-09-14T23:58:53.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:53.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.521 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:53.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079228+0000", 2024-09-14T23:58:53.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:53.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.521 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:53.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079229+0000", 2024-09-14T23:58:53.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5299999999999999e-07 2024-09-14T23:58:53.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.521 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:53.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079232+0000", 2024-09-14T23:58:53.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8839999999999998e-06 2024-09-14T23:58:53.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:53.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079238+0000", 2024-09-14T23:58:53.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2299999999999996e-06 2024-09-14T23:58:53.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:53.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079243+0000", 2024-09-14T23:58:53.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3029999999999999e-06 2024-09-14T23:58:53.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:53.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.137533+0000", 2024-09-14T23:58:53.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058289674999999999 2024-09-14T23:58:53.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:53.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.137615+0000", 2024-09-14T23:58:53.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2288999999999998e-05 2024-09-14T23:58:53.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:53.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.137683+0000", 2024-09-14T23:58:53.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7654e-05 2024-09-14T23:58:53.523 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:53.523 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:53.523 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:53.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.524 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184324 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4073472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:53.524 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.079251+0000", 2024-09-14T23:58:53.524 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.425146441000003, 2024-09-14T23:58:53.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058609383000000001, 2024-09-14T23:58:53.524 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:53.524 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:53.524 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:53.524 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:53.524 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:53.524 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:53.524 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184324 2024-09-14T23:58:53.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.525 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:53.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:53.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079251+0000", 2024-09-14T23:58:53.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:53.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:53.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079251+0000", 2024-09-14T23:58:53.525 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:53.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.525 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.525 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:53.525 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.080920+0000", 2024-09-14T23:58:53.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016687430000000001 2024-09-14T23:58:53.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.526 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:53.526 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.080924+0000", 2024-09-14T23:58:53.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3869999999999997e-06 2024-09-14T23:58:53.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.526 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:53.526 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.080935+0000", 2024-09-14T23:58:53.526 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0515e-05 2024-09-14T23:58:53.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.526 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:53.526 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.080942+0000", 2024-09-14T23:58:53.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0990000000000001e-06 2024-09-14T23:58:53.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:53.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.137713+0000", 2024-09-14T23:58:53.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056771288000000003 2024-09-14T23:58:53.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:53.527 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.137794+0000", 2024-09-14T23:58:53.527 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0613999999999998e-05 2024-09-14T23:58:53.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.527 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:53.528 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.137860+0000", 2024-09-14T23:58:53.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6736999999999995e-05 2024-09-14T23:58:53.528 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:53.528 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:53.528 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:53.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.528 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.528 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184346 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4099072~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:53.528 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.083616+0000", 2024-09-14T23:58:53.528 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.420781255999998, 2024-09-14T23:58:53.528 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058557419999999999, 2024-09-14T23:58:53.528 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:53.528 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:53.528 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:53.529 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:53.529 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:53.529 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:53.529 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184346 2024-09-14T23:58:53.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.529 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:53.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:53.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083616+0000", 2024-09-14T23:58:53.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:53.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:53.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083616+0000", 2024-09-14T23:58:53.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:53.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:53.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083620+0000", 2024-09-14T23:58:53.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0119999999999999e-06 2024-09-14T23:58:53.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:53.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083623+0000", 2024-09-14T23:58:53.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9059999999999998e-06 2024-09-14T23:58:53.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:53.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083631+0000", 2024-09-14T23:58:53.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2679999999999994e-06 2024-09-14T23:58:53.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:53.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083637+0000", 2024-09-14T23:58:53.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1210000000000003e-06 2024-09-14T23:58:53.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:53.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142018+0000", 2024-09-14T23:58:53.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058381111999999999 2024-09-14T23:58:53.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:53.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142100+0000", 2024-09-14T23:58:53.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1308000000000001e-05 2024-09-14T23:58:53.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:53.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142173+0000", 2024-09-14T23:58:53.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3693000000000001e-05 2024-09-14T23:58:53.532 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:53.532 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:53.532 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:53.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.532 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184347 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4122624~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:53.532 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.083647+0000", 2024-09-14T23:58:53.532 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.420750425999998, 2024-09-14T23:58:53.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058706388999999998, 2024-09-14T23:58:53.533 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:53.533 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:53.533 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:53.533 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:53.533 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:53.533 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:53.533 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184347 2024-09-14T23:58:53.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.533 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:53.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:53.533 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083647+0000", 2024-09-14T23:58:53.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:53.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:53.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083647+0000", 2024-09-14T23:58:53.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:53.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:53.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083647+0000", 2024-09-14T23:58:53.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3099999999999998e-07 2024-09-14T23:58:53.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:53.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083650+0000", 2024-09-14T23:58:53.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2900000000000001e-06 2024-09-14T23:58:53.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:53.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083657+0000", 2024-09-14T23:58:53.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.486e-06 2024-09-14T23:58:53.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:53.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083663+0000", 2024-09-14T23:58:53.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4330000000000003e-06 2024-09-14T23:58:53.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:53.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142206+0000", 2024-09-14T23:58:53.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058543237999999997 2024-09-14T23:58:53.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:53.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142284+0000", 2024-09-14T23:58:53.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8487999999999998e-05 2024-09-14T23:58:53.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:53.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142353+0000", 2024-09-14T23:58:53.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9023000000000006e-05 2024-09-14T23:58:53.536 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:53.536 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:53.537 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:53.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.537 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184348 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4103168~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:53.537 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.083669+0000", 2024-09-14T23:58:53.537 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.420728009000001, 2024-09-14T23:58:53.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058864232000000002, 2024-09-14T23:58:53.537 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:53.537 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:53.537 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:53.537 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:53.537 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:53.537 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:53.537 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184348 2024-09-14T23:58:53.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.538 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:53.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:53.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083669+0000", 2024-09-14T23:58:53.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:53.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:53.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083669+0000", 2024-09-14T23:58:53.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:53.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:53.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083670+0000", 2024-09-14T23:58:53.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9999999999999998e-07 2024-09-14T23:58:53.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:53.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083672+0000", 2024-09-14T23:58:53.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1430000000000001e-06 2024-09-14T23:58:53.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:53.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083678+0000", 2024-09-14T23:58:53.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5200000000000003e-06 2024-09-14T23:58:53.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:53.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083683+0000", 2024-09-14T23:58:53.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9359999999999998e-06 2024-09-14T23:58:53.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:53.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142384+0000", 2024-09-14T23:58:53.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058701159000000003 2024-09-14T23:58:53.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:53.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142466+0000", 2024-09-14T23:58:53.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1024000000000002e-05 2024-09-14T23:58:53.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:53.541 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142534+0000", 2024-09-14T23:58:53.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8050000000000001e-05 2024-09-14T23:58:53.541 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:53.541 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:53.541 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:53.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.541 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184349 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4126720~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:53.541 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.083690+0000", 2024-09-14T23:58:53.541 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.420707473999997, 2024-09-14T23:58:53.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059019911000000001, 2024-09-14T23:58:53.541 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:53.541 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:53.542 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:53.542 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:53.542 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:53.542 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:53.542 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184349 2024-09-14T23:58:53.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.542 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:53.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:53.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083690+0000", 2024-09-14T23:58:53.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:53.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:53.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083690+0000", 2024-09-14T23:58:53.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:53.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:53.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.084900+0000", 2024-09-14T23:58:53.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012099770000000001 2024-09-14T23:58:53.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:53.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.084905+0000", 2024-09-14T23:58:53.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.028e-06 2024-09-14T23:58:53.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:53.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.084914+0000", 2024-09-14T23:58:53.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6619999999999994e-06 2024-09-14T23:58:53.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:53.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.084922+0000", 2024-09-14T23:58:53.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5720000000000003e-06 2024-09-14T23:58:53.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:53.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142565+0000", 2024-09-14T23:58:53.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057642605999999999 2024-09-14T23:58:53.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:53.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142644+0000", 2024-09-14T23:58:53.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9370999999999996e-05 2024-09-14T23:58:53.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.545 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:53.545 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142710+0000", 2024-09-14T23:58:53.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5695e-05 2024-09-14T23:58:53.545 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:53.545 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:53.545 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:53.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.545 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184354 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4115456~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:53.545 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.085029+0000", 2024-09-14T23:58:53.545 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.419368824000003, 2024-09-14T23:58:53.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058626327999999998, 2024-09-14T23:58:53.545 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:53.546 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:53.546 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:53.546 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:53.546 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:53.546 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:53.546 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184354 2024-09-14T23:58:53.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.546 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:53.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:53.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085029+0000", 2024-09-14T23:58:53.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:53.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:53.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085029+0000", 2024-09-14T23:58:53.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:53.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:53.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085029+0000", 2024-09-14T23:58:53.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0800000000000001e-07 2024-09-14T23:58:53.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:53.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085030+0000", 2024-09-14T23:58:53.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.226e-06 2024-09-14T23:58:53.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:53.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085033+0000", 2024-09-14T23:58:53.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9569999999999998e-06 2024-09-14T23:58:53.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:53.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085035+0000", 2024-09-14T23:58:53.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.587e-06 2024-09-14T23:58:53.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:53.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143491+0000", 2024-09-14T23:58:53.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058455337000000003 2024-09-14T23:58:53.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:53.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143575+0000", 2024-09-14T23:58:53.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3850000000000005e-05 2024-09-14T23:58:53.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:53.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143655+0000", 2024-09-14T23:58:53.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0162999999999998e-05 2024-09-14T23:58:53.549 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:53.549 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:53.549 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:53.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.550 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184355 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4139008~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:53.550 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.085039+0000", 2024-09-14T23:58:53.550 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.419358420000002, 2024-09-14T23:58:53.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058801369999999999, 2024-09-14T23:58:53.550 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:53.550 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:53.550 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:53.550 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:53.550 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:53.550 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:53.550 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184355 2024-09-14T23:58:53.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.550 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:53.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:53.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085039+0000", 2024-09-14T23:58:53.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:53.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:53.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085039+0000", 2024-09-14T23:58:53.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:53.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:53.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085039+0000", 2024-09-14T23:58:53.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.03e-07 2024-09-14T23:58:53.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:53.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085040+0000", 2024-09-14T23:58:53.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.158e-06 2024-09-14T23:58:53.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:53.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085043+0000", 2024-09-14T23:58:53.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0240000000000002e-06 2024-09-14T23:58:53.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:53.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085046+0000", 2024-09-14T23:58:53.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4600000000000002e-06 2024-09-14T23:58:53.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:53.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143688+0000", 2024-09-14T23:58:53.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058642356 2024-09-14T23:58:53.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:53.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143766+0000", 2024-09-14T23:58:53.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8219000000000004e-05 2024-09-14T23:58:53.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:53.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143840+0000", 2024-09-14T23:58:53.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3949999999999995e-05 2024-09-14T23:58:53.553 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:53.554 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:53.554 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:53.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.554 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184356 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4119552~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:53.554 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.085049+0000", 2024-09-14T23:58:53.554 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.419348249000002, 2024-09-14T23:58:53.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058977992, 2024-09-14T23:58:53.554 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:53.554 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:53.554 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:53.554 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:53.554 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:53.554 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:53.555 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184356 2024-09-14T23:58:53.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.555 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:53.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:53.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085049+0000", 2024-09-14T23:58:53.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:53.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:53.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085049+0000", 2024-09-14T23:58:53.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:53.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:53.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.086934+0000", 2024-09-14T23:58:53.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018854189999999999 2024-09-14T23:58:53.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:53.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.086939+0000", 2024-09-14T23:58:53.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.493e-06 2024-09-14T23:58:53.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:53.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.086951+0000", 2024-09-14T23:58:53.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1969e-05 2024-09-14T23:58:53.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:53.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.086959+0000", 2024-09-14T23:58:53.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5780000000000001e-06 2024-09-14T23:58:53.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:53.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143871+0000", 2024-09-14T23:58:53.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056912114 2024-09-14T23:58:53.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:53.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143951+0000", 2024-09-14T23:58:53.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9938000000000006e-05 2024-09-14T23:58:53.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:53.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.144027+0000", 2024-09-14T23:58:53.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6481000000000002e-05 2024-09-14T23:58:53.558 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:53.558 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:53.558 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:53.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.558 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184364 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4135936~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:53.558 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.087207+0000", 2024-09-14T23:58:53.558 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.417189968999999, 2024-09-14T23:58:53.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058466420999999998, 2024-09-14T23:58:53.558 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:53.558 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:53.559 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:53.559 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:53.559 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:53.559 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:53.559 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184364 2024-09-14T23:58:53.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.559 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:53.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:53.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087207+0000", 2024-09-14T23:58:53.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:53.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:53.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087207+0000", 2024-09-14T23:58:53.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:53.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:53.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087208+0000", 2024-09-14T23:58:53.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.35e-07 2024-09-14T23:58:53.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:53.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087209+0000", 2024-09-14T23:58:53.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.573e-06 2024-09-14T23:58:53.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:53.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087212+0000", 2024-09-14T23:58:53.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2770000000000001e-06 2024-09-14T23:58:53.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:53.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087215+0000", 2024-09-14T23:58:53.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5500000000000001e-06 2024-09-14T23:58:53.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:53.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145520+0000", 2024-09-14T23:58:53.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058304835999999999 2024-09-14T23:58:53.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:53.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145602+0000", 2024-09-14T23:58:53.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1714999999999999e-05 2024-09-14T23:58:53.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:53.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145674+0000", 2024-09-14T23:58:53.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2235000000000002e-05 2024-09-14T23:58:53.562 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:53.562 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:53.562 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:53.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.562 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184365 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4159488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:53.562 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.087219+0000", 2024-09-14T23:58:53.563 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.417177938000002, 2024-09-14T23:58:53.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058643533999999997, 2024-09-14T23:58:53.563 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:53.563 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:53.563 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:53.563 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:53.563 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:53.563 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:53.563 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184365 2024-09-14T23:58:53.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.563 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:53.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:53.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087219+0000", 2024-09-14T23:58:53.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:53.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:53.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087219+0000", 2024-09-14T23:58:53.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:53.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:53.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087220+0000", 2024-09-14T23:58:53.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.39e-07 2024-09-14T23:58:53.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:53.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087222+0000", 2024-09-14T23:58:53.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0609999999999998e-06 2024-09-14T23:58:53.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:53.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087228+0000", 2024-09-14T23:58:53.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8590000000000001e-06 2024-09-14T23:58:53.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:53.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087232+0000", 2024-09-14T23:58:53.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8149999999999996e-06 2024-09-14T23:58:53.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:53.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145705+0000", 2024-09-14T23:58:53.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058472532000000001 2024-09-14T23:58:53.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:53.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145784+0000", 2024-09-14T23:58:53.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9382000000000007e-05 2024-09-14T23:58:53.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:53.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145863+0000", 2024-09-14T23:58:53.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8546000000000003e-05 2024-09-14T23:58:53.566 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:53.566 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:53.566 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:53.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.567 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184366 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4140032~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:53.567 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.087240+0000", 2024-09-14T23:58:53.567 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.417157490000001, 2024-09-14T23:58:53.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058818068000000001, 2024-09-14T23:58:53.567 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:53.567 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:53.567 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:53.567 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:53.567 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:53.567 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:53.567 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184366 2024-09-14T23:58:53.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.568 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:53.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:53.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087240+0000", 2024-09-14T23:58:53.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:53.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:53.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087240+0000", 2024-09-14T23:58:53.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:53.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:53.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.088904+0000", 2024-09-14T23:58:53.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001664315 2024-09-14T23:58:53.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:53.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.088909+0000", 2024-09-14T23:58:53.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7709999999999997e-06 2024-09-14T23:58:53.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:53.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.088921+0000", 2024-09-14T23:58:53.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1661999999999999e-05 2024-09-14T23:58:53.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:53.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.088928+0000", 2024-09-14T23:58:53.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8189999999999992e-06 2024-09-14T23:58:53.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:53.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145901+0000", 2024-09-14T23:58:53.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056972478 2024-09-14T23:58:53.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:53.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145977+0000", 2024-09-14T23:58:53.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5763e-05 2024-09-14T23:58:53.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:53.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.146058+0000", 2024-09-14T23:58:53.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1260000000000005e-05 2024-09-14T23:58:53.571 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:53.571 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:53.571 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:53.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.571 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184375 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4179968~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:53.571 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.089199+0000", 2024-09-14T23:58:53.571 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.415198152999999, 2024-09-14T23:58:53.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058597445999999997, 2024-09-14T23:58:53.571 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:53.571 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:53.571 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:53.571 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:53.572 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:53.572 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:53.572 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184375 2024-09-14T23:58:53.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.572 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:53.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:53.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089199+0000", 2024-09-14T23:58:53.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:53.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:53.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089199+0000", 2024-09-14T23:58:53.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:53.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:53.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089200+0000", 2024-09-14T23:58:53.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8100000000000003e-07 2024-09-14T23:58:53.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:53.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089202+0000", 2024-09-14T23:58:53.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7599999999999998e-06 2024-09-14T23:58:53.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:53.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089209+0000", 2024-09-14T23:58:53.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7279999999999998e-06 2024-09-14T23:58:53.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:53.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089215+0000", 2024-09-14T23:58:53.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5090000000000001e-06 2024-09-14T23:58:53.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:53.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147638+0000", 2024-09-14T23:58:53.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058423105000000003 2024-09-14T23:58:53.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:53.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147719+0000", 2024-09-14T23:58:53.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1217000000000005e-05 2024-09-14T23:58:53.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:53.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147797+0000", 2024-09-14T23:58:53.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7645999999999995e-05 2024-09-14T23:58:53.578 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:53.578 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:53.579 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:53.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.579 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184376 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4160512~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:53.579 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.089232+0000", 2024-09-14T23:58:53.579 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.415165663000003, 2024-09-14T23:58:53.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058751338, 2024-09-14T23:58:53.579 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:53.579 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:53.579 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:53.579 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:53.579 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:53.580 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:53.580 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184376 2024-09-14T23:58:53.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.580 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:53.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:53.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089232+0000", 2024-09-14T23:58:53.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:53.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:53.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089232+0000", 2024-09-14T23:58:53.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:53.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:53.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089232+0000", 2024-09-14T23:58:53.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5600000000000001e-07 2024-09-14T23:58:53.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:53.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089235+0000", 2024-09-14T23:58:53.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5639999999999999e-06 2024-09-14T23:58:53.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:53.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089242+0000", 2024-09-14T23:58:53.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0369999999999997e-06 2024-09-14T23:58:53.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:53.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089247+0000", 2024-09-14T23:58:53.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2079999999999999e-06 2024-09-14T23:58:53.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:53.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147831+0000", 2024-09-14T23:58:53.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058583853999999998 2024-09-14T23:58:53.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:53.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147915+0000", 2024-09-14T23:58:53.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4208000000000004e-05 2024-09-14T23:58:53.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:53.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147983+0000", 2024-09-14T23:58:53.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8010999999999996e-05 2024-09-14T23:58:53.583 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:53.583 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:53.583 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:53.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.584 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184377 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4184064~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:53.584 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.089255+0000", 2024-09-14T23:58:53.584 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.415142107000001, 2024-09-14T23:58:53.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058906100000000003, 2024-09-14T23:58:53.584 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:53.584 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:53.584 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:53.584 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:53.584 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:53.584 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:53.584 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184377 2024-09-14T23:58:53.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.584 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:53.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:53.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089255+0000", 2024-09-14T23:58:53.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:53.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:53.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089255+0000", 2024-09-14T23:58:53.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:53.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:53.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.090903+0000", 2024-09-14T23:58:53.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001647871 2024-09-14T23:58:53.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:53.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.090908+0000", 2024-09-14T23:58:53.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7199999999999997e-06 2024-09-14T23:58:53.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:53.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.090919+0000", 2024-09-14T23:58:53.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1472e-05 2024-09-14T23:58:53.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:53.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.090927+0000", 2024-09-14T23:58:53.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0390000000000005e-06 2024-09-14T23:58:53.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:53.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.148013+0000", 2024-09-14T23:58:53.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057085662000000002 2024-09-14T23:58:53.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:53.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.148090+0000", 2024-09-14T23:58:53.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6778000000000002e-05 2024-09-14T23:58:53.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:53.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.148161+0000", 2024-09-14T23:58:53.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1557999999999995e-05 2024-09-14T23:58:53.588 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:53.588 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:53.588 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:53.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.588 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184386 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4185088~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:53.588 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.091063+0000", 2024-09-14T23:58:53.588 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.413334442999997, 2024-09-14T23:58:53.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058603404999999997, 2024-09-14T23:58:53.588 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:53.589 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:53.589 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:53.589 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:53.589 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:53.589 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:53.589 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184386 2024-09-14T23:58:53.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.589 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:53.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:53.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091063+0000", 2024-09-14T23:58:53.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:53.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:53.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091063+0000", 2024-09-14T23:58:53.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:53.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:53.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091063+0000", 2024-09-14T23:58:53.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1899999999999999e-07 2024-09-14T23:58:53.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:53.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091064+0000", 2024-09-14T23:58:53.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2699999999999999e-06 2024-09-14T23:58:53.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:53.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091067+0000", 2024-09-14T23:58:53.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.971e-06 2024-09-14T23:58:53.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:53.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091070+0000", 2024-09-14T23:58:53.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.525e-06 2024-09-14T23:58:53.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:53.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.149510+0000", 2024-09-14T23:58:53.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058439758000000001 2024-09-14T23:58:53.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:53.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.149595+0000", 2024-09-14T23:58:53.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5146000000000001e-05 2024-09-14T23:58:53.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:53.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.149666+0000", 2024-09-14T23:58:53.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1515999999999998e-05 2024-09-14T23:58:53.592 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:53.592 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:53.592 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:53.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.593 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184388 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4189184~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:53.593 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.091073+0000", 2024-09-14T23:58:53.593 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.413324168000003, 2024-09-14T23:58:53.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058781609999999998, 2024-09-14T23:58:53.593 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:53.593 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:53.593 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:53.593 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:53.593 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:53.593 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:53.593 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184388 2024-09-14T23:58:53.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.594 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:53.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:53.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091073+0000", 2024-09-14T23:58:53.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:53.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:53.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091073+0000", 2024-09-14T23:58:53.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:53.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:53.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.092903+0000", 2024-09-14T23:58:53.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018296289999999999 2024-09-14T23:58:53.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:53.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.092910+0000", 2024-09-14T23:58:53.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7510000000000002e-06 2024-09-14T23:58:53.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:53.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.092918+0000", 2024-09-14T23:58:53.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7420000000000003e-06 2024-09-14T23:58:53.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:53.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.092925+0000", 2024-09-14T23:58:53.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0360000000000001e-06 2024-09-14T23:58:53.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:53.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.149700+0000", 2024-09-14T23:58:53.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056774340999999999 2024-09-14T23:58:53.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:53.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.149784+0000", 2024-09-14T23:58:53.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4393000000000003e-05 2024-09-14T23:58:53.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:53.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.149855+0000", 2024-09-14T23:58:53.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0717999999999996e-05 2024-09-14T23:58:53.597 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:53.597 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:53.597 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:53.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.597 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4172.0:8 3.12 3:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-14T23:58:53.597 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.976999+0000", 2024-09-14T23:58:53.597 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.527398759, 2024-09-14T23:58:53.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0046036439999999, 2024-09-14T23:58:53.598 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:53.598 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:53.598 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:53.598 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:53.598 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4172", 2024-09-14T23:58:53.598 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/2542269203", 2024-09-14T23:58:53.598 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 8 2024-09-14T23:58:53.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.598 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:53.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:53.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.976999+0000", 2024-09-14T23:58:53.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:53.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:53.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.976999+0000", 2024-09-14T23:58:53.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:53.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:53.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.977000+0000", 2024-09-14T23:58:53.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5200000000000001e-06 2024-09-14T23:58:53.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:53.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.977005+0000", 2024-09-14T23:58:53.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.04e-06 2024-09-14T23:58:53.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:53.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.977013+0000", 2024-09-14T23:58:53.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2889999999999998e-06 2024-09-14T23:58:53.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:53.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.977024+0000", 2024-09-14T23:58:53.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0307e-05 2024-09-14T23:58:53.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:53.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.977086+0000", 2024-09-14T23:58:53.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2248000000000004e-05 2024-09-14T23:58:53.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:58:53.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.977097+0000", 2024-09-14T23:58:53.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1123e-05 2024-09-14T23:58:53.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:53.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:08.981470+0000", 2024-09-14T23:58:53.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.004373116 2024-09-14T23:58:53.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:53.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:53.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:53.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:08.981602+0000", 2024-09-14T23:58:53.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000132001 2024-09-14T23:58:53.602 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:53.602 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:53.602 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:53.602 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:53.602 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:53.602 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:53.603 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:58:53.603 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:58:53.604 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T23:58:53.605 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:58:53.605 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:58:53.606 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:53.602+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:53.704 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:53.703+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:53.753 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:53.753 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:53.753 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:58:53.753 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:53.761 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T23:58:53.805 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:53.804+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:53.868 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:53.869 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:53.869 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:58:53.869 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:58:53.869 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:53.877 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T23:58:53.907 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:53.906+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:53.998 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 6, 5, 7] out_osds: [3] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T23:58:53.998 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T23:58:53.998 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-14T23:58:53.998 INFO:tasks.thrashosds.thrasher:Setting osd 2 primary_affinity to 1.000000 2024-09-14T23:58:53.998 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-14T23:58:54.007 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:54.006+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:54.036 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:54.036 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:58:54.036 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:58:54.036 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:58:54.036 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.036 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5882 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:54.036 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:58:54.037 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 134.65814646199999, 2024-09-14T23:58:54.037 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050549666, 2024-09-14T23:58:54.037 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:54.037 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:54.037 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:54.037 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:54.037 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:54.037 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:54.037 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5882 2024-09-14T23:58:54.037 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.037 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:54.037 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.037 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:54.038 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:58:54.038 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.038 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.038 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.038 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:54.038 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:58:54.038 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.038 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.038 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.038 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:54.038 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:58:54.038 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8099999999999998e-07 2024-09-14T23:58:54.038 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.039 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.039 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:54.039 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375842+0000", 2024-09-14T23:58:54.039 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9900000000000009e-07 2024-09-14T23:58:54.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.039 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.039 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:54.039 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375845+0000", 2024-09-14T23:58:54.039 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.055e-06 2024-09-14T23:58:54.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.039 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.039 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:54.039 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375848+0000", 2024-09-14T23:58:54.040 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7609999999999999e-06 2024-09-14T23:58:54.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.040 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.040 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:54.040 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426381+0000", 2024-09-14T23:58:54.040 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050533392000000003 2024-09-14T23:58:54.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.040 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.040 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:54.040 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426390+0000", 2024-09-14T23:58:54.040 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0780000000000002e-06 2024-09-14T23:58:54.040 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.040 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:54.041 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.041 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.041 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5884 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:54.041 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:58:54.041 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 134.65813632199999, 2024-09-14T23:58:54.041 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050717439000000003, 2024-09-14T23:58:54.041 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:54.041 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:54.041 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:54.041 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:54.041 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:54.041 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:54.041 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5884 2024-09-14T23:58:54.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.042 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:54.042 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.042 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:54.042 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:58:54.042 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.042 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.042 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:54.042 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:58:54.042 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.042 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.043 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:54.043 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:58:54.043 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0800000000000001e-07 2024-09-14T23:58:54.043 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.043 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.043 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:54.043 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375852+0000", 2024-09-14T23:58:54.043 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.02e-06 2024-09-14T23:58:54.043 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.043 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.043 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:54.043 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375855+0000", 2024-09-14T23:58:54.043 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9900000000000002e-06 2024-09-14T23:58:54.043 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.044 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.044 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:54.044 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375857+0000", 2024-09-14T23:58:54.044 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3499999999999999e-06 2024-09-14T23:58:54.044 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.044 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.044 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:54.044 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426556+0000", 2024-09-14T23:58:54.044 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050698139000000003 2024-09-14T23:58:54.044 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.044 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.044 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:54.044 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426568+0000", 2024-09-14T23:58:54.045 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2731999999999999e-05 2024-09-14T23:58:54.045 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.045 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:54.045 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.045 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.045 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5885 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:54.045 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:58:54.045 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 134.65811729699999, 2024-09-14T23:58:54.045 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050737333000000003, 2024-09-14T23:58:54.045 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:54.045 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:54.045 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:54.045 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:54.046 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:54.046 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:54.046 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5885 2024-09-14T23:58:54.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.046 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:54.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.046 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:54.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:58:54.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.046 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:54.046 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:58:54.046 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:54.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:58:54.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5899999999999998e-07 2024-09-14T23:58:54.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:54.047 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375872+0000", 2024-09-14T23:58:54.047 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4219999999999999e-06 2024-09-14T23:58:54.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.047 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:54.048 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375878+0000", 2024-09-14T23:58:54.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4980000000000003e-06 2024-09-14T23:58:54.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.048 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:54.048 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375881+0000", 2024-09-14T23:58:54.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6110000000000001e-06 2024-09-14T23:58:54.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.048 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:54.048 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426596+0000", 2024-09-14T23:58:54.048 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050715320000000001 2024-09-14T23:58:54.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.049 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:54.049 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426607+0000", 2024-09-14T23:58:54.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1223000000000001e-05 2024-09-14T23:58:54.049 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.049 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:54.049 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.049 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5886 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:54.049 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:58:54.049 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 134.658103184, 2024-09-14T23:58:54.049 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050803395000000001, 2024-09-14T23:58:54.049 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:54.050 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:54.050 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:54.050 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:54.050 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:54.050 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:54.050 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5886 2024-09-14T23:58:54.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.050 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:54.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.050 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:54.050 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:58:54.050 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:54.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:58:54.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:54.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:58:54.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2700000000000001e-07 2024-09-14T23:58:54.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.051 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.051 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:54.051 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375886+0000", 2024-09-14T23:58:54.051 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7060000000000001e-06 2024-09-14T23:58:54.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.052 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.052 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:54.052 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375890+0000", 2024-09-14T23:58:54.052 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2259999999999999e-06 2024-09-14T23:58:54.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.052 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.052 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:54.052 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375893+0000", 2024-09-14T23:58:54.052 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0819999999999999e-06 2024-09-14T23:58:54.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.052 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.052 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:54.053 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426677+0000", 2024-09-14T23:58:54.053 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050784158000000003 2024-09-14T23:58:54.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.053 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:54.053 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426687+0000", 2024-09-14T23:58:54.053 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9960000000000005e-06 2024-09-14T23:58:54.053 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.053 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:54.053 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.053 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.053 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5887 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:54.053 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:58:54.054 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 134.658081098, 2024-09-14T23:58:54.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050833070000000001, 2024-09-14T23:58:54.054 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:54.054 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:54.054 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:54.054 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:54.054 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:54.054 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:54.054 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5887 2024-09-14T23:58:54.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.054 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:54.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.054 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:54.054 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:58:54.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:54.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:58:54.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:54.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:58:54.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3299999999999998e-07 2024-09-14T23:58:54.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:54.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375908+0000", 2024-09-14T23:58:54.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7689999999999999e-06 2024-09-14T23:58:54.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:54.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375912+0000", 2024-09-14T23:58:54.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.535e-06 2024-09-14T23:58:54.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:54.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375916+0000", 2024-09-14T23:58:54.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8979999999999998e-06 2024-09-14T23:58:54.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.057 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:54.057 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426729+0000", 2024-09-14T23:58:54.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050813453000000001 2024-09-14T23:58:54.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.057 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:54.057 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426739+0000", 2024-09-14T23:58:54.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0081999999999999e-05 2024-09-14T23:58:54.057 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.057 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:54.057 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.058 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5888 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:54.058 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:58:54.058 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 134.65806802099999, 2024-09-14T23:58:54.058 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050854652, 2024-09-14T23:58:54.058 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:54.058 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:54.058 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:54.058 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:54.058 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:54.058 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:54.058 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5888 2024-09-14T23:58:54.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.058 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:54.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.058 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:54.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:58:54.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:54.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:58:54.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:54.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:58:54.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1400000000000001e-07 2024-09-14T23:58:54.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:54.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375921+0000", 2024-09-14T23:58:54.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.119e-06 2024-09-14T23:58:54.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:54.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375928+0000", 2024-09-14T23:58:54.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3170000000000004e-06 2024-09-14T23:58:54.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:54.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375931+0000", 2024-09-14T23:58:54.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6970000000000002e-06 2024-09-14T23:58:54.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.061 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:54.061 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426768+0000", 2024-09-14T23:58:54.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050837355000000001 2024-09-14T23:58:54.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.061 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:54.061 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426774+0000", 2024-09-14T23:58:54.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9499999999999998e-06 2024-09-14T23:58:54.061 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.061 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:54.061 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.062 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5889 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:54.062 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:58:54.062 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 134.658051097, 2024-09-14T23:58:54.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050885446000000001, 2024-09-14T23:58:54.062 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:54.062 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:54.062 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:54.062 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:54.062 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:54.062 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:54.062 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5889 2024-09-14T23:58:54.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.063 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:54.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:54.063 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:58:54.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:54.063 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:58:54.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:54.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:58:54.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.36e-07 2024-09-14T23:58:54.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.064 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:54.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375940+0000", 2024-09-14T23:58:54.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.844e-06 2024-09-14T23:58:54.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.064 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:54.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375944+0000", 2024-09-14T23:58:54.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6720000000000002e-06 2024-09-14T23:58:54.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:54.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375946+0000", 2024-09-14T23:58:54.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5629999999999999e-06 2024-09-14T23:58:54.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:54.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426810+0000", 2024-09-14T23:58:54.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050863255000000003 2024-09-14T23:58:54.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:54.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426822+0000", 2024-09-14T23:58:54.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1875999999999999e-05 2024-09-14T23:58:54.065 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.066 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:54.066 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.066 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5890 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:54.066 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:58:54.066 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 134.65803728500001, 2024-09-14T23:58:54.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050917549999999999, 2024-09-14T23:58:54.066 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:54.066 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:54.066 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:54.066 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:54.066 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:54.067 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:54.067 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5890 2024-09-14T23:58:54.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.067 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:54.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:54.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:58:54.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:54.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:58:54.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:54.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:58:54.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4299999999999999e-07 2024-09-14T23:58:54.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:54.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375951+0000", 2024-09-14T23:58:54.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1000000000000001e-06 2024-09-14T23:58:54.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:54.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375956+0000", 2024-09-14T23:58:54.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3370000000000001e-06 2024-09-14T23:58:54.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:54.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375958+0000", 2024-09-14T23:58:54.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4820000000000001e-06 2024-09-14T23:58:54.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:54.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426858+0000", 2024-09-14T23:58:54.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050899666000000003 2024-09-14T23:58:54.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:54.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426867+0000", 2024-09-14T23:58:54.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7219999999999994e-06 2024-09-14T23:58:54.070 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.070 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:54.070 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.070 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5891 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:54.070 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:58:54.070 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 134.658025963, 2024-09-14T23:58:54.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050948067, 2024-09-14T23:58:54.070 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:54.070 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:54.070 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:54.071 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:54.071 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:54.071 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:54.071 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5891 2024-09-14T23:58:54.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.071 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:54.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:54.071 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:58:54.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:54.071 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:58:54.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:54.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:58:54.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0800000000000001e-07 2024-09-14T23:58:54.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:54.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375962+0000", 2024-09-14T23:58:54.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9600000000000008e-07 2024-09-14T23:58:54.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:54.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375970+0000", 2024-09-14T23:58:54.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1600000000000001e-06 2024-09-14T23:58:54.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:54.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375972+0000", 2024-09-14T23:58:54.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4849999999999999e-06 2024-09-14T23:58:54.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:54.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426902+0000", 2024-09-14T23:58:54.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050929813999999997 2024-09-14T23:58:54.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:54.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426909+0000", 2024-09-14T23:58:54.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4039999999999998e-06 2024-09-14T23:58:54.074 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.074 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:54.074 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.074 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5892 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:54.074 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:58:54.074 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 134.658011106, 2024-09-14T23:58:54.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050965036999999998, 2024-09-14T23:58:54.075 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:54.075 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:54.075 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:54.075 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:54.075 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:54.075 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:54.075 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5892 2024-09-14T23:58:54.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.075 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:54.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.075 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:54.075 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:58:54.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:54.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:58:54.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:54.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:58:54.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.34e-07 2024-09-14T23:58:54.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:54.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375978+0000", 2024-09-14T23:58:54.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1880000000000001e-06 2024-09-14T23:58:54.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:54.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375981+0000", 2024-09-14T23:58:54.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2399999999999999e-06 2024-09-14T23:58:54.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:54.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375984+0000", 2024-09-14T23:58:54.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9110000000000002e-06 2024-09-14T23:58:54.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:54.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426934+0000", 2024-09-14T23:58:54.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050949912 2024-09-14T23:58:54.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:54.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426941+0000", 2024-09-14T23:58:54.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5519999999999999e-06 2024-09-14T23:58:54.078 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.078 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:54.078 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.078 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5893 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:54.078 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:58:54.078 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 134.658000509, 2024-09-14T23:58:54.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050975131999999999, 2024-09-14T23:58:54.079 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:54.079 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:54.079 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:54.079 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:54.079 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:54.079 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:54.079 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5893 2024-09-14T23:58:54.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.079 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:54.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.079 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:54.079 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:58:54.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:54.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:58:54.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:54.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:58:54.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2999999999999999e-07 2024-09-14T23:58:54.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:54.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375988+0000", 2024-09-14T23:58:54.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0410000000000001e-06 2024-09-14T23:58:54.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:54.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375992+0000", 2024-09-14T23:58:54.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.104e-06 2024-09-14T23:58:54.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:54.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375995+0000", 2024-09-14T23:58:54.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4310000000000001e-06 2024-09-14T23:58:54.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:54.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426958+0000", 2024-09-14T23:58:54.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050963642000000003 2024-09-14T23:58:54.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:54.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426962+0000", 2024-09-14T23:58:54.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6839999999999998e-06 2024-09-14T23:58:54.082 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.082 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:54.082 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.082 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5894 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:54.082 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:58:54.083 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 134.657989109, 2024-09-14T23:58:54.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050978333000000001, 2024-09-14T23:58:54.083 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:54.083 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:54.083 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:54.083 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:54.083 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:54.083 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:54.083 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5894 2024-09-14T23:58:54.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.083 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:54.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.083 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:54.083 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:58:54.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:54.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:58:54.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:54.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:58:54.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.23e-07 2024-09-14T23:58:54.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:54.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375999+0000", 2024-09-14T23:58:54.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9800000000000002e-07 2024-09-14T23:58:54.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:54.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376007+0000", 2024-09-14T23:58:54.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8539999999999997e-06 2024-09-14T23:58:54.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:54.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376011+0000", 2024-09-14T23:58:54.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.399e-06 2024-09-14T23:58:54.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:54.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426973+0000", 2024-09-14T23:58:54.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050962454999999997 2024-09-14T23:58:54.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:54.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426976+0000", 2024-09-14T23:58:54.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4039999999999999e-06 2024-09-14T23:58:54.086 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.086 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:54.086 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.087 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5895 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:54.087 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376014+0000", 2024-09-14T23:58:54.087 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 134.65797276500001, 2024-09-14T23:58:54.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050975951999999998, 2024-09-14T23:58:54.087 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:54.087 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:54.087 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:54.087 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:54.087 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:54.087 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:54.087 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5895 2024-09-14T23:58:54.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.087 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:54.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:54.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376014+0000", 2024-09-14T23:58:54.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:54.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376014+0000", 2024-09-14T23:58:54.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:54.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376015+0000", 2024-09-14T23:58:54.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2999999999999999e-07 2024-09-14T23:58:54.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:54.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376016+0000", 2024-09-14T23:58:54.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1659999999999999e-06 2024-09-14T23:58:54.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:54.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376024+0000", 2024-09-14T23:58:54.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8930000000000005e-06 2024-09-14T23:58:54.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:54.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376026+0000", 2024-09-14T23:58:54.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6869999999999999e-06 2024-09-14T23:58:54.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:54.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426987+0000", 2024-09-14T23:58:54.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050960601000000001 2024-09-14T23:58:54.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:54.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426990+0000", 2024-09-14T23:58:54.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3749999999999999e-06 2024-09-14T23:58:54.090 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.090 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:54.090 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.091 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5896 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:54.091 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:58:54.091 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 134.657957028, 2024-09-14T23:58:54.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050975656000000001, 2024-09-14T23:58:54.091 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:54.091 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:54.091 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:54.091 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:54.091 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:54.091 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:54.091 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5896 2024-09-14T23:58:54.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.092 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:54.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:54.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:58:54.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:54.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:58:54.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:54.092 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:58:54.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5100000000000001e-07 2024-09-14T23:58:54.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:54.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376032+0000", 2024-09-14T23:58:54.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.141e-06 2024-09-14T23:58:54.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:54.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376036+0000", 2024-09-14T23:58:54.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.25e-06 2024-09-14T23:58:54.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:54.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376038+0000", 2024-09-14T23:58:54.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5469999999999999e-06 2024-09-14T23:58:54.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:54.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427002+0000", 2024-09-14T23:58:54.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050964029000000001 2024-09-14T23:58:54.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:54.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427006+0000", 2024-09-14T23:58:54.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4379999999999999e-06 2024-09-14T23:58:54.094 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.094 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:54.095 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.095 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5897 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:54.095 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:58:54.095 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 134.65794522799999, 2024-09-14T23:58:54.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050978001000000002, 2024-09-14T23:58:54.095 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:54.095 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:54.095 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:54.095 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:54.095 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:54.096 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:54.096 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5897 2024-09-14T23:58:54.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.096 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:54.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:54.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:58:54.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.096 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:54.096 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:58:54.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:54.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:58:54.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2600000000000001e-07 2024-09-14T23:58:54.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:54.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376043+0000", 2024-09-14T23:58:54.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.054e-06 2024-09-14T23:58:54.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:54.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376047+0000", 2024-09-14T23:58:54.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0659999999999997e-06 2024-09-14T23:58:54.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:54.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376050+0000", 2024-09-14T23:58:54.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.88e-06 2024-09-14T23:58:54.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:54.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427017+0000", 2024-09-14T23:58:54.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050966453000000002 2024-09-14T23:58:54.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:54.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427020+0000", 2024-09-14T23:58:54.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3220000000000001e-06 2024-09-14T23:58:54.099 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.099 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:54.099 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.099 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5898 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:54.099 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376053+0000", 2024-09-14T23:58:54.099 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 134.657933743, 2024-09-14T23:58:54.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050982024000000001, 2024-09-14T23:58:54.099 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:54.099 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:54.099 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:54.100 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:54.100 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:54.100 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:54.100 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5898 2024-09-14T23:58:54.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.100 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:54.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:54.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376053+0000", 2024-09-14T23:58:54.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.100 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:54.100 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376053+0000", 2024-09-14T23:58:54.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:54.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376278+0000", 2024-09-14T23:58:54.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022426400000000001 2024-09-14T23:58:54.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:54.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376282+0000", 2024-09-14T23:58:54.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4739999999999999e-06 2024-09-14T23:58:54.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:54.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376296+0000", 2024-09-14T23:58:54.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4229999999999999e-05 2024-09-14T23:58:54.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:54.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376303+0000", 2024-09-14T23:58:54.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2199999999999997e-06 2024-09-14T23:58:54.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:54.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427031+0000", 2024-09-14T23:58:54.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050728154999999997 2024-09-14T23:58:54.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:54.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427036+0000", 2024-09-14T23:58:54.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6809999999999997e-06 2024-09-14T23:58:54.103 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.103 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:54.103 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.103 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5899 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:54.103 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376338+0000", 2024-09-14T23:58:54.103 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 134.65764901899999, 2024-09-14T23:58:54.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050712100000000003, 2024-09-14T23:58:54.103 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:54.104 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:54.104 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:54.104 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:54.104 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:54.104 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:54.104 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5899 2024-09-14T23:58:54.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.104 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:54.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:54.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376338+0000", 2024-09-14T23:58:54.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:54.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376338+0000", 2024-09-14T23:58:54.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:54.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376339+0000", 2024-09-14T23:58:54.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9100000000000003e-07 2024-09-14T23:58:54.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:54.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376342+0000", 2024-09-14T23:58:54.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7e-06 2024-09-14T23:58:54.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:54.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376350+0000", 2024-09-14T23:58:54.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5280000000000003e-06 2024-09-14T23:58:54.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:54.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376356+0000", 2024-09-14T23:58:54.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6010000000000002e-06 2024-09-14T23:58:54.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:54.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427047+0000", 2024-09-14T23:58:54.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050690962999999999 2024-09-14T23:58:54.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:54.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427050+0000", 2024-09-14T23:58:54.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.517e-06 2024-09-14T23:58:54.107 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.107 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:54.107 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.107 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5900 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:54.107 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376366+0000", 2024-09-14T23:58:54.107 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 134.65762075999999, 2024-09-14T23:58:54.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050697858999999998, 2024-09-14T23:58:54.108 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:54.108 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:54.108 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:54.108 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:54.108 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:54.108 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:54.108 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5900 2024-09-14T23:58:54.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.108 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:54.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.108 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:54.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376366+0000", 2024-09-14T23:58:54.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:54.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376366+0000", 2024-09-14T23:58:54.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:54.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377650+0000", 2024-09-14T23:58:54.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001283834 2024-09-14T23:58:54.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:54.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377655+0000", 2024-09-14T23:58:54.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7230000000000004e-06 2024-09-14T23:58:54.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:54.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377667+0000", 2024-09-14T23:58:54.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1746000000000001e-05 2024-09-14T23:58:54.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:54.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377676+0000", 2024-09-14T23:58:54.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3619999999999995e-06 2024-09-14T23:58:54.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:54.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427061+0000", 2024-09-14T23:58:54.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049384755000000002 2024-09-14T23:58:54.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:54.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427064+0000", 2024-09-14T23:58:54.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.439e-06 2024-09-14T23:58:54.111 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.111 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:54.111 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.111 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:132622 3.1e 3:79d0a5e0:::rbd_data.111847077d5e.0000000000000007:head [sparse-read 0~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T23:58:54.112 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:44.037901+0000", 2024-09-14T23:58:54.112 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 69.996086151, 2024-09-14T23:58:54.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1083420859999999, 2024-09-14T23:58:54.112 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:54.112 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:54.112 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:54.112 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:54.112 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:54.112 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:54.112 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 132622 2024-09-14T23:58:54.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.112 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:54.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:54.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037901+0000", 2024-09-14T23:58:54.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:54.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037901+0000", 2024-09-14T23:58:54.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:54.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037903+0000", 2024-09-14T23:58:54.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4619999999999999e-06 2024-09-14T23:58:54.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:54.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037910+0000", 2024-09-14T23:58:54.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7449999999999995e-06 2024-09-14T23:58:54.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:54.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037928+0000", 2024-09-14T23:58:54.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7305000000000001e-05 2024-09-14T23:58:54.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:54.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037939+0000", 2024-09-14T23:58:54.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1599e-05 2024-09-14T23:58:54.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:54.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.038049+0000", 2024-09-14T23:58:54.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000109577 2024-09-14T23:58:54.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op must wait for map", 2024-09-14T23:58:54.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.038057+0000", 2024-09-14T23:58:54.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1450000000000008e-06 2024-09-14T23:58:54.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:54.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.039937+0000", 2024-09-14T23:58:54.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00188024 2024-09-14T23:58:54.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:58:54.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.040039+0000", 2024-09-14T23:58:54.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000101934 2024-09-14T23:58:54.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:54.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:45.145805+0000", 2024-09-14T23:58:54.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.105765651 2024-09-14T23:58:54.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:54.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:45.145997+0000", 2024-09-14T23:58:54.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000192084 2024-09-14T23:58:54.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:54.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:45.146243+0000", 2024-09-14T23:58:54.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024634400000000001 2024-09-14T23:58:54.117 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.117 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:54.117 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.117 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4172.0:5 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:54.117 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.482606+0000", 2024-09-14T23:58:54.117 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.551381169999999, 2024-09-14T23:58:54.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.49381641300000001, 2024-09-14T23:58:54.118 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:54.118 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:54.118 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:54.118 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:54.118 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4172", 2024-09-14T23:58:54.118 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/2542269203", 2024-09-14T23:58:54.118 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5 2024-09-14T23:58:54.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.118 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:54.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:54.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482606+0000", 2024-09-14T23:58:54.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:54.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482606+0000", 2024-09-14T23:58:54.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:54.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482607+0000", 2024-09-14T23:58:54.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3120000000000001e-06 2024-09-14T23:58:54.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:54.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482615+0000", 2024-09-14T23:58:54.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9300000000000003e-06 2024-09-14T23:58:54.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:54.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482632+0000", 2024-09-14T23:58:54.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6966999999999999e-05 2024-09-14T23:58:54.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:54.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482640+0000", 2024-09-14T23:58:54.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1820000000000006e-06 2024-09-14T23:58:54.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:54.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482703+0000", 2024-09-14T23:58:54.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2098000000000001e-05 2024-09-14T23:58:54.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:58:54.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482713+0000", 2024-09-14T23:58:54.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0064000000000001e-05 2024-09-14T23:58:54.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:54.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.976307+0000", 2024-09-14T23:58:54.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.49359442999999997 2024-09-14T23:58:54.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:54.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.976422+0000", 2024-09-14T23:58:54.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011543 2024-09-14T23:58:54.122 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.122 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:54.122 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.122 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.122 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:54.122 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:54.124 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T23:58:54.124 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:54.122+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:54.224 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:54.223+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:54.271 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:54.272 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:54.272 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:58:54.272 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:54.281 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T23:58:54.325 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:54.323+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:54.388 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:54.388 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:54.388 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:58:54.388 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:58:54.388 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:54.397 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T23:58:54.425 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:54.424+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:54.527 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:54.526+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:54.546 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:54.546 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:58:54.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:58:54.546 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:58:54.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.546 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:56107 3.0 e56/23 3:00c47890:::rbd_data.1118c4b3aebe.0000000000000009:head v 56'125, mlcod=52'123)", 2024-09-14T23:58:54.547 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.236123+0000", 2024-09-14T23:58:54.547 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 108.307922611, 2024-09-14T23:58:54.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0061273220000000002, 2024-09-14T23:58:54.547 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:54.547 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:54.547 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:54.547 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:54.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:54.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.236123+0000", 2024-09-14T23:58:54.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:54.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.236123+0000", 2024-09-14T23:58:54.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:54.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.236125+0000", 2024-09-14T23:58:54.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.313e-06 2024-09-14T23:58:54.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:54.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237428+0000", 2024-09-14T23:58:54.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013029890000000001 2024-09-14T23:58:54.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:54.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237452+0000", 2024-09-14T23:58:54.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4090000000000001e-05 2024-09-14T23:58:54.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:54.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237462+0000", 2024-09-14T23:58:54.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0421999999999999e-05 2024-09-14T23:58:54.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:54.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237559+0000", 2024-09-14T23:58:54.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6378000000000005e-05 2024-09-14T23:58:54.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:54.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237583+0000", 2024-09-14T23:58:54.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4182e-05 2024-09-14T23:58:54.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:54.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.242205+0000", 2024-09-14T23:58:54.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0046217289999999998 2024-09-14T23:58:54.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:54.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.242251+0000", 2024-09-14T23:58:54.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6218999999999999e-05 2024-09-14T23:58:54.551 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.551 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:54.551 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.551 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:56108 3.0 e56/23 3:00c47890:::rbd_data.1118c4b3aebe.0000000000000009:head v 56'127, mlcod=52'123)", 2024-09-14T23:58:54.551 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.239693+0000", 2024-09-14T23:58:54.551 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 108.30435314100001, 2024-09-14T23:58:54.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0043496209999999997, 2024-09-14T23:58:54.551 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:54.551 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:54.552 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:54.552 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:54.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:54.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239693+0000", 2024-09-14T23:58:54.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:54.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239693+0000", 2024-09-14T23:58:54.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:54.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239694+0000", 2024-09-14T23:58:54.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0499999999999999e-06 2024-09-14T23:58:54.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:54.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239702+0000", 2024-09-14T23:58:54.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1170000000000004e-06 2024-09-14T23:58:54.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:54.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239716+0000", 2024-09-14T23:58:54.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4306e-05 2024-09-14T23:58:54.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:54.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239724+0000", 2024-09-14T23:58:54.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0399999999999993e-06 2024-09-14T23:58:54.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:54.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239778+0000", 2024-09-14T23:58:54.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4032e-05 2024-09-14T23:58:54.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:54.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239801+0000", 2024-09-14T23:58:54.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2770000000000001e-05 2024-09-14T23:58:54.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:54.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.243990+0000", 2024-09-14T23:58:54.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0041890909999999998 2024-09-14T23:58:54.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:54.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.244043+0000", 2024-09-14T23:58:54.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2215000000000001e-05 2024-09-14T23:58:54.555 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.555 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:54.555 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.556 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:56385 3.9 e56/23 3:963d31c7:::rbd_data.1118c4b3aebe.0000000000000003:head v 56'87, mlcod=53'85)", 2024-09-14T23:58:54.556 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.283936+0000", 2024-09-14T23:58:54.556 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 108.260110231, 2024-09-14T23:58:54.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0063042080000000004, 2024-09-14T23:58:54.556 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:54.556 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:54.556 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:54.556 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:54.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:54.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.283936+0000", 2024-09-14T23:58:54.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:54.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.283936+0000", 2024-09-14T23:58:54.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:54.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.283937+0000", 2024-09-14T23:58:54.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1179999999999999e-06 2024-09-14T23:58:54.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:54.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286338+0000", 2024-09-14T23:58:54.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024010149999999998 2024-09-14T23:58:54.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:54.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286360+0000", 2024-09-14T23:58:54.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2212e-05 2024-09-14T23:58:54.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:54.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286370+0000", 2024-09-14T23:58:54.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5850000000000006e-06 2024-09-14T23:58:54.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.558 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:54.558 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286443+0000", 2024-09-14T23:58:54.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2757999999999997e-05 2024-09-14T23:58:54.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:54.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286466+0000", 2024-09-14T23:58:54.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3844000000000001e-05 2024-09-14T23:58:54.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:54.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.290197+0000", 2024-09-14T23:58:54.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003730176 2024-09-14T23:58:54.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:54.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.290240+0000", 2024-09-14T23:58:54.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.35e-05 2024-09-14T23:58:54.560 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.560 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:54.560 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.560 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:56386 3.9 e56/23 3:963d31c7:::rbd_data.1118c4b3aebe.0000000000000003:head v 56'89, mlcod=53'85)", 2024-09-14T23:58:54.560 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.287578+0000", 2024-09-14T23:58:54.560 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 108.256468505, 2024-09-14T23:58:54.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033580250000000002, 2024-09-14T23:58:54.560 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:54.561 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:54.561 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:54.561 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:54.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:54.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287578+0000", 2024-09-14T23:58:54.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:54.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287578+0000", 2024-09-14T23:58:54.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:54.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287579+0000", 2024-09-14T23:58:54.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.64e-07 2024-09-14T23:58:54.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:54.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287586+0000", 2024-09-14T23:58:54.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5310000000000004e-06 2024-09-14T23:58:54.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.562 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:54.562 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287597+0000", 2024-09-14T23:58:54.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0903999999999999e-05 2024-09-14T23:58:54.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:54.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287605+0000", 2024-09-14T23:58:54.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7889999999999995e-06 2024-09-14T23:58:54.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:54.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.288437+0000", 2024-09-14T23:58:54.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000831914 2024-09-14T23:58:54.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:54.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.288460+0000", 2024-09-14T23:58:54.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3410000000000001e-05 2024-09-14T23:58:54.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:54.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.290883+0000", 2024-09-14T23:58:54.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024230599999999999 2024-09-14T23:58:54.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:54.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.290936+0000", 2024-09-14T23:58:54.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2453000000000001e-05 2024-09-14T23:58:54.564 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.565 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:54.565 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.565 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:64263 3.4 e57/23 3:20c6a768:::rbd_data.11188d356229.0000000000000001:head v 57'165, mlcod=57'163)", 2024-09-14T23:58:54.565 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:07.999031+0000", 2024-09-14T23:58:54.565 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 106.545015357, 2024-09-14T23:58:54.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023815210000000002, 2024-09-14T23:58:54.565 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:54.565 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:54.565 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:54.565 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:54.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:54.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999031+0000", 2024-09-14T23:58:54.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:54.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999031+0000", 2024-09-14T23:58:54.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:54.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999032+0000", 2024-09-14T23:58:54.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.176e-06 2024-09-14T23:58:54.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:54.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999660+0000", 2024-09-14T23:58:54.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00062817099999999996 2024-09-14T23:58:54.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:54.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999683+0000", 2024-09-14T23:58:54.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2816999999999999e-05 2024-09-14T23:58:54.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:54.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999693+0000", 2024-09-14T23:58:54.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0088999999999999e-05 2024-09-14T23:58:54.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:54.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999767+0000", 2024-09-14T23:58:54.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4056999999999999e-05 2024-09-14T23:58:54.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:54.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999793+0000", 2024-09-14T23:58:54.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5573000000000001e-05 2024-09-14T23:58:54.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:54.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:08.001379+0000", 2024-09-14T23:58:54.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015859019999999999 2024-09-14T23:58:54.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:54.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:08.001412+0000", 2024-09-14T23:58:54.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3735999999999997e-05 2024-09-14T23:58:54.569 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.569 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:54.569 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.569 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:66570 3.0 e60/23 3:055350b3:::rbd_data.111873792d9b.0000000000000005:head v 60'162, mlcod=57'159)", 2024-09-14T23:58:54.569 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:11.277517+0000", 2024-09-14T23:58:54.570 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 103.266529201, 2024-09-14T23:58:54.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003472199, 2024-09-14T23:58:54.570 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:54.570 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:54.570 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:54.570 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:54.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:54.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.277517+0000", 2024-09-14T23:58:54.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:54.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.277517+0000", 2024-09-14T23:58:54.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:54.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.277518+0000", 2024-09-14T23:58:54.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1379999999999999e-06 2024-09-14T23:58:54.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:54.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278504+0000", 2024-09-14T23:58:54.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00098617499999999994 2024-09-14T23:58:54.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:54.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278525+0000", 2024-09-14T23:58:54.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.09e-05 2024-09-14T23:58:54.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:54.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278534+0000", 2024-09-14T23:58:54.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6209999999999993e-06 2024-09-14T23:58:54.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:54.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278611+0000", 2024-09-14T23:58:54.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6926000000000002e-05 2024-09-14T23:58:54.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:54.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278641+0000", 2024-09-14T23:58:54.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0861999999999997e-05 2024-09-14T23:58:54.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:54.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.280953+0000", 2024-09-14T23:58:54.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023114239999999999 2024-09-14T23:58:54.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:54.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.280989+0000", 2024-09-14T23:58:54.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6152999999999998e-05 2024-09-14T23:58:54.574 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.574 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:54.574 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.574 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:158983 3.4 e103/23 3:20e43b29:::rbd_data.1118b2f495e6.0000000000000012:head v 103'454, mlcod=103'452)", 2024-09-14T23:58:54.574 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:58.917699+0000", 2024-09-14T23:58:54.574 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 55.626347150999997, 2024-09-14T23:58:54.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037933799999999998, 2024-09-14T23:58:54.574 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:54.574 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:54.574 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:54.574 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:54.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:54.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917699+0000", 2024-09-14T23:58:54.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:54.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917699+0000", 2024-09-14T23:58:54.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:54.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917700+0000", 2024-09-14T23:58:54.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3850000000000001e-06 2024-09-14T23:58:54.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:54.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917939+0000", 2024-09-14T23:58:54.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000238249 2024-09-14T23:58:54.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:54.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917961+0000", 2024-09-14T23:58:54.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2894000000000001e-05 2024-09-14T23:58:54.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:54.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917972+0000", 2024-09-14T23:58:54.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0261000000000001e-05 2024-09-14T23:58:54.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:54.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.918038+0000", 2024-09-14T23:58:54.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6223000000000006e-05 2024-09-14T23:58:54.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:54.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.918063+0000", 2024-09-14T23:58:54.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5252000000000001e-05 2024-09-14T23:58:54.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:54.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.921433+0000", 2024-09-14T23:58:54.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033699960000000001 2024-09-14T23:58:54.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:54.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.921492+0000", 2024-09-14T23:58:54.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9120000000000003e-05 2024-09-14T23:58:54.578 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.578 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:54.578 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.578 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(osd.1.0:1460 3.0 e122/23 3:00cc1e32:::rbd_data.11187b9f1032.0000000000000001:2f v 122'453, mlcod=118'449)", 2024-09-14T23:58:54.578 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:22.287423+0000", 2024-09-14T23:58:54.579 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.256623494000003, 2024-09-14T23:58:54.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0046041659999999998, 2024-09-14T23:58:54.579 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:54.579 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:54.579 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:54.579 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:54.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:54.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.287423+0000", 2024-09-14T23:58:54.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:54.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.287423+0000", 2024-09-14T23:58:54.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:54.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.287423+0000", 2024-09-14T23:58:54.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7500000000000001e-07 2024-09-14T23:58:54.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:54.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.287435+0000", 2024-09-14T23:58:54.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2006e-05 2024-09-14T23:58:54.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:54.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.287444+0000", 2024-09-14T23:58:54.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.465e-06 2024-09-14T23:58:54.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:54.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.287453+0000", 2024-09-14T23:58:54.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0100000000000001e-06 2024-09-14T23:58:54.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:54.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.287519+0000", 2024-09-14T23:58:54.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6149999999999995e-05 2024-09-14T23:58:54.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:54.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.287543+0000", 2024-09-14T23:58:54.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3682000000000001e-05 2024-09-14T23:58:54.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:54.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.291977+0000", 2024-09-14T23:58:54.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0044333829999999999 2024-09-14T23:58:54.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:54.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.292027+0000", 2024-09-14T23:58:54.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0195000000000003e-05 2024-09-14T23:58:54.583 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.583 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:54.583 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.583 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDPGPush(3.1e 126/125 [PushOp(3:7f7265c3:::rbd_data.11188d356229.000000000000000d:16, version: 58'124, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:7f7265c3:::rbd_data.11188d356229.000000000000000d:16@58'124, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 16=[]:{16: [16]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T23:58:54.583 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:26.552687+0000", 2024-09-14T23:58:54.583 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.99135923, 2024-09-14T23:58:54.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023212060000000001, 2024-09-14T23:58:54.583 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:54.583 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:54.583 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:54.583 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:54.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:54.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.552687+0000", 2024-09-14T23:58:54.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:54.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.552687+0000", 2024-09-14T23:58:54.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:54.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.552688+0000", 2024-09-14T23:58:54.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.47e-07 2024-09-14T23:58:54.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:54.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.553659+0000", 2024-09-14T23:58:54.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00097088899999999995 2024-09-14T23:58:54.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:54.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.553682+0000", 2024-09-14T23:58:54.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3862e-05 2024-09-14T23:58:54.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:54.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.553703+0000", 2024-09-14T23:58:54.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0310999999999999e-05 2024-09-14T23:58:54.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:54.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.553813+0000", 2024-09-14T23:58:54.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011051 2024-09-14T23:58:54.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:54.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.553828+0000", 2024-09-14T23:58:54.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4625e-05 2024-09-14T23:58:54.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:54.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.555008+0000", 2024-09-14T23:58:54.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001180162 2024-09-14T23:58:54.587 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.587 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:54.587 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.587 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(osd.1.0:1465 3.4 e127/23 3:265f7763:::rbd_data.1118f9c96a7.0000000000000004:2c v 127'508, mlcod=120'505)", 2024-09-14T23:58:54.587 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:27.318056+0000", 2024-09-14T23:58:54.587 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.225989807000001, 2024-09-14T23:58:54.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0049337449999999998, 2024-09-14T23:58:54.587 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:54.587 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:54.587 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:54.587 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:54.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:54.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.318056+0000", 2024-09-14T23:58:54.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:54.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.318056+0000", 2024-09-14T23:58:54.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:54.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.318057+0000", 2024-09-14T23:58:54.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9799999999999999e-07 2024-09-14T23:58:54.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:54.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.318061+0000", 2024-09-14T23:58:54.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7899999999999999e-06 2024-09-14T23:58:54.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:54.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.318074+0000", 2024-09-14T23:58:54.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2537999999999999e-05 2024-09-14T23:58:54.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:54.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.318083+0000", 2024-09-14T23:58:54.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3829999999999998e-06 2024-09-14T23:58:54.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:54.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.318165+0000", 2024-09-14T23:58:54.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1928999999999994e-05 2024-09-14T23:58:54.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:54.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.318199+0000", 2024-09-14T23:58:54.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3587000000000002e-05 2024-09-14T23:58:54.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:54.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.322939+0000", 2024-09-14T23:58:54.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0047397730000000001 2024-09-14T23:58:54.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:54.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.322990+0000", 2024-09-14T23:58:54.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1347000000000002e-05 2024-09-14T23:58:54.591 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.591 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:54.591 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.591 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:252476 3.e 3:725cbf00:::rbd_header.11186acd4a72:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e132)", 2024-09-14T23:58:54.592 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:32.121781+0000", 2024-09-14T23:58:54.592 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.422264970000001, 2024-09-14T23:58:54.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022671459999999998, 2024-09-14T23:58:54.592 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:54.592 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:54.592 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:54.592 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:54.592 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:54.592 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:54.592 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 252476 2024-09-14T23:58:54.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.592 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:54.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:54.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:32.121781+0000", 2024-09-14T23:58:54.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:54.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:32.121781+0000", 2024-09-14T23:58:54.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:54.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:32.121783+0000", 2024-09-14T23:58:54.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.531e-06 2024-09-14T23:58:54.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:54.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:32.121790+0000", 2024-09-14T23:58:54.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7670000000000003e-06 2024-09-14T23:58:54.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:54.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:32.121806+0000", 2024-09-14T23:58:54.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5291999999999999e-05 2024-09-14T23:58:54.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:54.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:32.121813+0000", 2024-09-14T23:58:54.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.418e-06 2024-09-14T23:58:54.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:54.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:32.121879+0000", 2024-09-14T23:58:54.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5811999999999999e-05 2024-09-14T23:58:54.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op must wait for map", 2024-09-14T23:58:54.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:32.121888+0000", 2024-09-14T23:58:54.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6600000000000001e-06 2024-09-14T23:58:54.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:54.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:32.123710+0000", 2024-09-14T23:58:54.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001822838 2024-09-14T23:58:54.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:54.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:32.123803+0000", 2024-09-14T23:58:54.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3077000000000003e-05 2024-09-14T23:58:54.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:54.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:32.124048+0000", 2024-09-14T23:58:54.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024475099999999997 2024-09-14T23:58:54.596 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.596 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:54.596 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.597 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:253435 3.5 3:a6fcd3e0:::rbd_data.1118e50f549.000000000000004a:head [delete] snapc 24=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-14T23:58:54.597 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:36.070548+0000", 2024-09-14T23:58:54.597 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.473498488000001, 2024-09-14T23:58:54.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022804000000000001, 2024-09-14T23:58:54.597 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:54.597 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:54.597 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:54.597 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:54.597 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:54.597 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:54.597 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 253435 2024-09-14T23:58:54.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.598 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:54.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:54.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.070548+0000", 2024-09-14T23:58:54.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:54.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.070548+0000", 2024-09-14T23:58:54.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:54.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.070549+0000", 2024-09-14T23:58:54.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6500000000000001e-06 2024-09-14T23:58:54.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:54.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.070555+0000", 2024-09-14T23:58:54.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1750000000000002e-06 2024-09-14T23:58:54.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:54.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.070571+0000", 2024-09-14T23:58:54.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5719999999999999e-05 2024-09-14T23:58:54.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:54.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.070580+0000", 2024-09-14T23:58:54.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0699999999999996e-06 2024-09-14T23:58:54.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:54.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.070658+0000", 2024-09-14T23:58:54.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7525000000000001e-05 2024-09-14T23:58:54.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:54.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.070799+0000", 2024-09-14T23:58:54.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014083499999999999 2024-09-14T23:58:54.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-14T23:58:54.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.070927+0000", 2024-09-14T23:58:54.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012853100000000001 2024-09-14T23:58:54.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:58:54.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.071666+0000", 2024-09-14T23:58:54.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00073853000000000005 2024-09-14T23:58:54.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:54.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.072750+0000", 2024-09-14T23:58:54.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001084738 2024-09-14T23:58:54.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:54.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.072791+0000", 2024-09-14T23:58:54.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0330999999999997e-05 2024-09-14T23:58:54.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:54.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.072828+0000", 2024-09-14T23:58:54.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7295000000000001e-05 2024-09-14T23:58:54.602 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.603 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:54.603 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.603 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:253690 3.e 3:725cbf00:::rbd_header.11186acd4a72:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-14T23:58:54.603 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:37.091642+0000", 2024-09-14T23:58:54.603 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 17.452404189999999, 2024-09-14T23:58:54.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0026525030000000001, 2024-09-14T23:58:54.603 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:54.603 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:54.603 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:54.603 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:54.603 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:54.603 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:54.604 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 253690 2024-09-14T23:58:54.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.604 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:54.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:54.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:37.091642+0000", 2024-09-14T23:58:54.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:54.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:37.091642+0000", 2024-09-14T23:58:54.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:54.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:37.091644+0000", 2024-09-14T23:58:54.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9829999999999999e-06 2024-09-14T23:58:54.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:54.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:37.091654+0000", 2024-09-14T23:58:54.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8619999999999998e-06 2024-09-14T23:58:54.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:54.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:37.091673+0000", 2024-09-14T23:58:54.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9094e-05 2024-09-14T23:58:54.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:54.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:37.091681+0000", 2024-09-14T23:58:54.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9449999999999993e-06 2024-09-14T23:58:54.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:54.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:37.091752+0000", 2024-09-14T23:58:54.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1468000000000001e-05 2024-09-14T23:58:54.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op must wait for map", 2024-09-14T23:58:54.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:37.091764+0000", 2024-09-14T23:58:54.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2067e-05 2024-09-14T23:58:54.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:54.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:37.093986+0000", 2024-09-14T23:58:54.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022213720000000001 2024-09-14T23:58:54.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:54.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:37.094082+0000", 2024-09-14T23:58:54.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5872999999999995e-05 2024-09-14T23:58:54.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:54.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:37.094294+0000", 2024-09-14T23:58:54.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021283900000000001 2024-09-14T23:58:54.608 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.608 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:54.608 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.608 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:254188 3.5 3:a13778a5:::rbd_header.1118ce574610:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e140)", 2024-09-14T23:58:54.608 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:40.008804+0000", 2024-09-14T23:58:54.608 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.535241894, 2024-09-14T23:58:54.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0044649980000000004, 2024-09-14T23:58:54.608 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:54.608 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:54.608 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:54.609 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:54.609 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:54.609 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:54.609 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254188 2024-09-14T23:58:54.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.609 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:54.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:54.609 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.008804+0000", 2024-09-14T23:58:54.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.609 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:54.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.008804+0000", 2024-09-14T23:58:54.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:54.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.008806+0000", 2024-09-14T23:58:54.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.824e-06 2024-09-14T23:58:54.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:54.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.008813+0000", 2024-09-14T23:58:54.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8889999999999996e-06 2024-09-14T23:58:54.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:54.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.008826+0000", 2024-09-14T23:58:54.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.29e-05 2024-09-14T23:58:54.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:54.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.008836+0000", 2024-09-14T23:58:54.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9660000000000008e-06 2024-09-14T23:58:54.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:54.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.008906+0000", 2024-09-14T23:58:54.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0165000000000003e-05 2024-09-14T23:58:54.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:54.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.009004+0000", 2024-09-14T23:58:54.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8412999999999994e-05 2024-09-14T23:58:54.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-14T23:58:54.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.009349+0000", 2024-09-14T23:58:54.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00034510200000000002 2024-09-14T23:58:54.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:58:54.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.010086+0000", 2024-09-14T23:58:54.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00073696000000000005 2024-09-14T23:58:54.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:54.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.013157+0000", 2024-09-14T23:58:54.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0030706470000000001 2024-09-14T23:58:54.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:54.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.013222+0000", 2024-09-14T23:58:54.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4479000000000004e-05 2024-09-14T23:58:54.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:54.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.013269+0000", 2024-09-14T23:58:54.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7652999999999999e-05 2024-09-14T23:58:54.614 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.614 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:54.614 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.614 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:254194 3.5 3:a13778a5:::rbd_header.1118ce574610:head [watch unwatch cookie 140670715072400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e140)", 2024-09-14T23:58:54.614 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:40.018920+0000", 2024-09-14T23:58:54.614 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 14.525126518, 2024-09-14T23:58:54.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023612640000000001, 2024-09-14T23:58:54.614 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:54.614 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:54.614 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:54.615 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:54.615 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:54.615 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:54.615 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254194 2024-09-14T23:58:54.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.615 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:54.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:54.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.018920+0000", 2024-09-14T23:58:54.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:54.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.018920+0000", 2024-09-14T23:58:54.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:54.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.018920+0000", 2024-09-14T23:58:54.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7700000000000004e-07 2024-09-14T23:58:54.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:54.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.018923+0000", 2024-09-14T23:58:54.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9840000000000001e-06 2024-09-14T23:58:54.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:54.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.018931+0000", 2024-09-14T23:58:54.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8089999999999993e-06 2024-09-14T23:58:54.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:54.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.018937+0000", 2024-09-14T23:58:54.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1770000000000003e-06 2024-09-14T23:58:54.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:54.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.019163+0000", 2024-09-14T23:58:54.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022563800000000001 2024-09-14T23:58:54.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:54.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.019233+0000", 2024-09-14T23:58:54.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9698999999999998e-05 2024-09-14T23:58:54.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-14T23:58:54.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.019383+0000", 2024-09-14T23:58:54.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015073800000000001 2024-09-14T23:58:54.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:58:54.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.020341+0000", 2024-09-14T23:58:54.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00095732900000000004 2024-09-14T23:58:54.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:54.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.021141+0000", 2024-09-14T23:58:54.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00080075600000000002 2024-09-14T23:58:54.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:54.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.021189+0000", 2024-09-14T23:58:54.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7318999999999999e-05 2024-09-14T23:58:54.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:54.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.021281+0000", 2024-09-14T23:58:54.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2038000000000002e-05 2024-09-14T23:58:54.620 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.620 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:54.620 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.620 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(osd.4.0:4196 3.13 e141/113 3:cecefa16:::rbd_data.1118c250b6de.0000000000000001:1f v 141'878, mlcod=138'874)", 2024-09-14T23:58:54.620 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:41.015011+0000", 2024-09-14T23:58:54.620 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 13.529035571, 2024-09-14T23:58:54.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025205499999999999, 2024-09-14T23:58:54.620 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:54.620 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:54.620 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:54.621 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:54.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:54.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:41.015011+0000", 2024-09-14T23:58:54.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:54.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:41.015011+0000", 2024-09-14T23:58:54.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:54.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:41.015011+0000", 2024-09-14T23:58:54.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9900000000000002e-07 2024-09-14T23:58:54.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:54.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:41.015018+0000", 2024-09-14T23:58:54.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2910000000000002e-06 2024-09-14T23:58:54.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:54.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:41.015033+0000", 2024-09-14T23:58:54.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4625e-05 2024-09-14T23:58:54.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.622 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:54.622 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:41.015043+0000", 2024-09-14T23:58:54.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0584999999999999e-05 2024-09-14T23:58:54.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:54.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:41.015114+0000", 2024-09-14T23:58:54.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0541000000000001e-05 2024-09-14T23:58:54.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:54.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:41.015138+0000", 2024-09-14T23:58:54.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3810000000000001e-05 2024-09-14T23:58:54.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:54.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:41.017481+0000", 2024-09-14T23:58:54.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023433619999999999 2024-09-14T23:58:54.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:54.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:41.017531+0000", 2024-09-14T23:58:54.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0037000000000002e-05 2024-09-14T23:58:54.624 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.624 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:54.624 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.624 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(osd.4.0:4316 3.9 e146/23 3:91eaed90:::rbd_data.11182ee2885e.000000000000000c:1a v 146'308, mlcod=142'303)", 2024-09-14T23:58:54.625 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:46.046915+0000", 2024-09-14T23:58:54.625 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.4971310950000003, 2024-09-14T23:58:54.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022583619999999999, 2024-09-14T23:58:54.625 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:54.625 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:54.625 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:54.625 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:54.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:54.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.046915+0000", 2024-09-14T23:58:54.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:54.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.046915+0000", 2024-09-14T23:58:54.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:54.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.046916+0000", 2024-09-14T23:58:54.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6300000000000004e-07 2024-09-14T23:58:54.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:54.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.046922+0000", 2024-09-14T23:58:54.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2339999999999999e-06 2024-09-14T23:58:54.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:54.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.046933+0000", 2024-09-14T23:58:54.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0795000000000001e-05 2024-09-14T23:58:54.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:54.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.046950+0000", 2024-09-14T23:58:54.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7149000000000001e-05 2024-09-14T23:58:54.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:54.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.047000+0000", 2024-09-14T23:58:54.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9654000000000003e-05 2024-09-14T23:58:54.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:54.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.047020+0000", 2024-09-14T23:58:54.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0199000000000001e-05 2024-09-14T23:58:54.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:54.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.049110+0000", 2024-09-14T23:58:54.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0020902970000000001 2024-09-14T23:58:54.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:54.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.049173+0000", 2024-09-14T23:58:54.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3270999999999998e-05 2024-09-14T23:58:54.629 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.629 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:54.629 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.629 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(osd.6.0:1718 3.1e e146/127 3:7a74d0d0:::rbd_data.11182ee2885e.0000000000000006:1a v 146'435, mlcod=145'432)", 2024-09-14T23:58:54.629 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:46.047607+0000", 2024-09-14T23:58:54.629 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.4964393729999994, 2024-09-14T23:58:54.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0026434779999999999, 2024-09-14T23:58:54.629 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:54.629 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:54.629 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:54.629 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:54.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:54.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.047607+0000", 2024-09-14T23:58:54.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:54.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.047607+0000", 2024-09-14T23:58:54.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:54.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.047607+0000", 2024-09-14T23:58:54.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4700000000000002e-07 2024-09-14T23:58:54.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:54.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.047612+0000", 2024-09-14T23:58:54.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1959999999999999e-06 2024-09-14T23:58:54.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:54.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.047622+0000", 2024-09-14T23:58:54.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0169999999999999e-05 2024-09-14T23:58:54.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:54.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.047630+0000", 2024-09-14T23:58:54.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7249999999999997e-06 2024-09-14T23:58:54.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:54.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.047739+0000", 2024-09-14T23:58:54.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108625 2024-09-14T23:58:54.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:54.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.047756+0000", 2024-09-14T23:58:54.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6866000000000001e-05 2024-09-14T23:58:54.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:54.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.050197+0000", 2024-09-14T23:58:54.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024412689999999998 2024-09-14T23:58:54.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:54.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.050250+0000", 2024-09-14T23:58:54.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3279999999999998e-05 2024-09-14T23:58:54.633 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.633 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:54.633 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.634 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:255323 3.e 3:74b99397:::rbd_data.11182ee2885e.0000000000000004:head [delete] snapc 1a=[] ondisk+write+known_if_redirected+supports_pool_eio e146)", 2024-09-14T23:58:54.634 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:46.111665+0000", 2024-09-14T23:58:54.634 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.4323810780000006, 2024-09-14T23:58:54.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038303019999999998, 2024-09-14T23:58:54.634 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:54.634 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:54.634 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:54.634 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:54.634 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:54.634 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:54.634 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 255323 2024-09-14T23:58:54.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.634 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:54.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:54.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.111665+0000", 2024-09-14T23:58:54.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:54.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.111665+0000", 2024-09-14T23:58:54.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:54.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.111667+0000", 2024-09-14T23:58:54.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3e-06 2024-09-14T23:58:54.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:54.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.111676+0000", 2024-09-14T23:58:54.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5699999999999993e-06 2024-09-14T23:58:54.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:54.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.111698+0000", 2024-09-14T23:58:54.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2263e-05 2024-09-14T23:58:54.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:54.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.111709+0000", 2024-09-14T23:58:54.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1087000000000001e-05 2024-09-14T23:58:54.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:54.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.111787+0000", 2024-09-14T23:58:54.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8212000000000003e-05 2024-09-14T23:58:54.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:54.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.111886+0000", 2024-09-14T23:58:54.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8645000000000002e-05 2024-09-14T23:58:54.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-14T23:58:54.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.112014+0000", 2024-09-14T23:58:54.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012753800000000001 2024-09-14T23:58:54.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:58:54.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.113790+0000", 2024-09-14T23:58:54.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00177669 2024-09-14T23:58:54.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:54.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.115396+0000", 2024-09-14T23:58:54.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001605216 2024-09-14T23:58:54.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:54.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.115451+0000", 2024-09-14T23:58:54.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5534000000000001e-05 2024-09-14T23:58:54.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:54.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.115495+0000", 2024-09-14T23:58:54.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4246999999999998e-05 2024-09-14T23:58:54.639 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.639 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:54.639 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.639 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:255325 3.1e e146/127 3:7a74d0d0:::rbd_data.11182ee2885e.0000000000000006:head v 146'436, mlcod=145'433)", 2024-09-14T23:58:54.639 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:46.112042+0000", 2024-09-14T23:58:54.640 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.4320041410000002, 2024-09-14T23:58:54.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023640530000000001, 2024-09-14T23:58:54.640 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:54.640 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:54.640 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:54.640 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:54.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:54.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.112042+0000", 2024-09-14T23:58:54.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:54.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.112042+0000", 2024-09-14T23:58:54.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:54.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:54.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.112043+0000", 2024-09-14T23:58:54.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1200000000000001e-06 2024-09-14T23:58:54.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:54.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.112051+0000", 2024-09-14T23:58:54.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3310000000000004e-06 2024-09-14T23:58:54.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:54.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.112063+0000", 2024-09-14T23:58:54.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2028000000000001e-05 2024-09-14T23:58:54.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:54.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.112071+0000", 2024-09-14T23:58:54.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4000000000000003e-06 2024-09-14T23:58:54.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:54.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.112155+0000", 2024-09-14T23:58:54.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4064e-05 2024-09-14T23:58:54.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:54.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.112181+0000", 2024-09-14T23:58:54.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5817999999999999e-05 2024-09-14T23:58:54.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:54.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.114372+0000", 2024-09-14T23:58:54.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0021908129999999998 2024-09-14T23:58:54.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:54.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:54.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:54.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.114406+0000", 2024-09-14T23:58:54.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4479e-05 2024-09-14T23:58:54.644 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.644 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:54.644 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.644 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:54.644 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:54.644 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:54.645 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T23:58:54.646 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:54.644+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:54.746 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:54.745+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:54.796 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:54.796 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:54.796 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:58:54.796 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:54.805 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T23:58:54.848 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:54.847+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:54.912 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:54.912 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:54.913 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:58:54.913 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:58:54.913 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:54.921 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T23:58:54.949 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:54.947+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:54.981 INFO:teuthology.orchestra.run.smithi042.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-14T23:58:54.998 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T23:58:55.049 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:55.048+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:55.102 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:55.102 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:58:55.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:58:55.103 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:58:55.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.103 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5201 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2232320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:55.103 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:58:55.103 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 135.84653533100001, 2024-09-14T23:58:55.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057643228999999997, 2024-09-14T23:58:55.103 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:55.103 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:55.103 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:55.103 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:55.104 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:55.104 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:55.104 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5201 2024-09-14T23:58:55.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.104 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:55.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.104 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:55.104 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:58:55.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:55.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:55.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:58:55.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:55.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:55.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:58:55.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0699999999999998e-07 2024-09-14T23:58:55.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:55.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253422+0000", 2024-09-14T23:58:55.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.514e-06 2024-09-14T23:58:55.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:55.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253431+0000", 2024-09-14T23:58:55.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1919999999999994e-06 2024-09-14T23:58:55.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:55.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253436+0000", 2024-09-14T23:58:55.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5350000000000004e-06 2024-09-14T23:58:55.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:55.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310856+0000", 2024-09-14T23:58:55.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057419381999999998 2024-09-14T23:58:55.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:55.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310953+0000", 2024-09-14T23:58:55.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6774000000000006e-05 2024-09-14T23:58:55.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:55.108 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311062+0000", 2024-09-14T23:58:55.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000109425 2024-09-14T23:58:55.108 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:55.108 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:55.108 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:55.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.108 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5202 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2293760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:55.108 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:58:55.108 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 135.846510583, 2024-09-14T23:58:55.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057818255999999998, 2024-09-14T23:58:55.109 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:55.109 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:55.109 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:55.109 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:55.109 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:55.109 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:55.109 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5202 2024-09-14T23:58:55.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.109 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:55.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:55.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:58:55.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:55.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:55.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:58:55.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:55.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:55.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253573+0000", 2024-09-14T23:58:55.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012934000000000001 2024-09-14T23:58:55.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:55.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253576+0000", 2024-09-14T23:58:55.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5310000000000002e-06 2024-09-14T23:58:55.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:55.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253587+0000", 2024-09-14T23:58:55.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0522e-05 2024-09-14T23:58:55.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:55.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253595+0000", 2024-09-14T23:58:55.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8290000000000008e-06 2024-09-14T23:58:55.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:55.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311093+0000", 2024-09-14T23:58:55.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057498312000000003 2024-09-14T23:58:55.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:55.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311167+0000", 2024-09-14T23:58:55.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3582000000000001e-05 2024-09-14T23:58:55.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.113 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:55.113 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311262+0000", 2024-09-14T23:58:55.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5140000000000001e-05 2024-09-14T23:58:55.113 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:55.113 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:55.113 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:55.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.113 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5203 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2355200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:55.114 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:58:55.114 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 135.84634784100001, 2024-09-14T23:58:55.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057903938000000002, 2024-09-14T23:58:55.114 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:55.114 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:55.114 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:55.114 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:55.114 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:55.114 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:55.114 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5203 2024-09-14T23:58:55.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.114 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:55.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:55.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:58:55.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:55.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:55.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:58:55.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:55.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:55.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253902+0000", 2024-09-14T23:58:55.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029615400000000002 2024-09-14T23:58:55.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:55.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253906+0000", 2024-09-14T23:58:55.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9249999999999997e-06 2024-09-14T23:58:55.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:55.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253913+0000", 2024-09-14T23:58:55.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9099999999999999e-06 2024-09-14T23:58:55.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:55.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253917+0000", 2024-09-14T23:58:55.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9210000000000002e-06 2024-09-14T23:58:55.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:55.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311295+0000", 2024-09-14T23:58:55.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057378268000000003 2024-09-14T23:58:55.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:55.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311402+0000", 2024-09-14T23:58:55.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010644100000000001 2024-09-14T23:58:55.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.118 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:55.118 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311510+0000", 2024-09-14T23:58:55.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108319 2024-09-14T23:58:55.118 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:55.118 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:55.118 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:55.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.118 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.119 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5204 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2416640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:55.119 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:58:55.119 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 135.846029644, 2024-09-14T23:58:55.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057774860999999997, 2024-09-14T23:58:55.119 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:55.119 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:55.119 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:55.119 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:55.119 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:55.119 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:55.119 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5204 2024-09-14T23:58:55.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.120 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:55.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:55.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:58:55.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:55.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:55.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:58:55.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:55.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:55.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255903+0000", 2024-09-14T23:58:55.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001978807 2024-09-14T23:58:55.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:55.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255907+0000", 2024-09-14T23:58:55.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3610000000000001e-06 2024-09-14T23:58:55.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:55.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255914+0000", 2024-09-14T23:58:55.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.357e-06 2024-09-14T23:58:55.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:55.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255918+0000", 2024-09-14T23:58:55.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1779999999999996e-06 2024-09-14T23:58:55.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:55.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311541+0000", 2024-09-14T23:58:55.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055622697999999998 2024-09-14T23:58:55.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:55.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311614+0000", 2024-09-14T23:58:55.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3102999999999994e-05 2024-09-14T23:58:55.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.123 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:55.123 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311699+0000", 2024-09-14T23:58:55.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5357000000000003e-05 2024-09-14T23:58:55.123 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:55.123 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:55.124 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:55.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.124 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5211 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2846720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:55.124 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:58:55.124 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 135.84393143, 2024-09-14T23:58:55.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057528151, 2024-09-14T23:58:55.124 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:55.124 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:55.124 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:55.124 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:55.124 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:55.125 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:55.125 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5211 2024-09-14T23:58:55.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.125 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:55.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:55.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:58:55.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:55.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:55.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:58:55.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:55.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:55.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:58:55.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1399999999999998e-07 2024-09-14T23:58:55.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:55.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256024+0000", 2024-09-14T23:58:55.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.114e-06 2024-09-14T23:58:55.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:55.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256028+0000", 2024-09-14T23:58:55.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0840000000000002e-06 2024-09-14T23:58:55.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:55.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256031+0000", 2024-09-14T23:58:55.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6819999999999999e-06 2024-09-14T23:58:55.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:55.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313317+0000", 2024-09-14T23:58:55.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057286295000000001 2024-09-14T23:58:55.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:55.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313403+0000", 2024-09-14T23:58:55.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6287999999999998e-05 2024-09-14T23:58:55.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.128 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:55.128 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313551+0000", 2024-09-14T23:58:55.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000147374 2024-09-14T23:58:55.129 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:55.129 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:55.129 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:55.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.129 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5212 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2908160~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:55.129 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:58:55.129 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 135.843920236, 2024-09-14T23:58:55.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057735679999999998, 2024-09-14T23:58:55.129 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:55.129 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:55.129 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:55.130 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:55.130 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:55.130 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:55.130 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5212 2024-09-14T23:58:55.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.130 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:55.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:55.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:58:55.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:55.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:55.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:58:55.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:55.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:55.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:58:55.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0900000000000001e-07 2024-09-14T23:58:55.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:55.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256035+0000", 2024-09-14T23:58:55.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1200000000000001e-06 2024-09-14T23:58:55.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:55.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256038+0000", 2024-09-14T23:58:55.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.05e-06 2024-09-14T23:58:55.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.132 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:55.132 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256041+0000", 2024-09-14T23:58:55.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.667e-06 2024-09-14T23:58:55.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:55.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313584+0000", 2024-09-14T23:58:55.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057543484999999998 2024-09-14T23:58:55.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:55.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313668+0000", 2024-09-14T23:58:55.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3807000000000006e-05 2024-09-14T23:58:55.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:55.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313770+0000", 2024-09-14T23:58:55.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010134200000000001 2024-09-14T23:58:55.134 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:55.134 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:55.134 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:55.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.134 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5213 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2969600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:55.134 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:58:55.134 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 135.84391014900001, 2024-09-14T23:58:55.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058145467999999999, 2024-09-14T23:58:55.134 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:55.134 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:55.135 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:55.135 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:55.135 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:55.135 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:55.135 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5213 2024-09-14T23:58:55.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.135 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:55.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:55.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:58:55.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:55.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:55.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:58:55.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:55.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:55.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257901+0000", 2024-09-14T23:58:55.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018567519999999999 2024-09-14T23:58:55.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:55.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257904+0000", 2024-09-14T23:58:55.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4850000000000001e-06 2024-09-14T23:58:55.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:55.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257912+0000", 2024-09-14T23:58:55.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2090000000000005e-06 2024-09-14T23:58:55.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:55.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257918+0000", 2024-09-14T23:58:55.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6899999999999997e-06 2024-09-14T23:58:55.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:55.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313816+0000", 2024-09-14T23:58:55.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055898075999999998 2024-09-14T23:58:55.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.138 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:55.138 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.314082+0000", 2024-09-14T23:58:55.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000265386 2024-09-14T23:58:55.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:55.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.314189+0000", 2024-09-14T23:58:55.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010787 2024-09-14T23:58:55.139 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:55.139 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:55.139 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:55.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.139 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5218 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3276800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:55.139 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:58:55.139 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 135.84198314, 2024-09-14T23:58:55.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057497557999999997, 2024-09-14T23:58:55.140 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:55.140 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:55.140 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:55.140 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:55.140 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:55.140 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:55.140 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5218 2024-09-14T23:58:55.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.140 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:55.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:55.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:58:55.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:55.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:55.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:58:55.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:55.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:55.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:58:55.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1899999999999999e-07 2024-09-14T23:58:55.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:55.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257972+0000", 2024-09-14T23:58:55.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.217e-06 2024-09-14T23:58:55.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:55.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257975+0000", 2024-09-14T23:58:55.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9359999999999999e-06 2024-09-14T23:58:55.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:55.143 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257978+0000", 2024-09-14T23:58:55.143 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1049999999999999e-06 2024-09-14T23:58:55.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.143 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.143 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:55.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315331+0000", 2024-09-14T23:58:55.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057352093999999999 2024-09-14T23:58:55.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:55.144 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315384+0000", 2024-09-14T23:58:55.144 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3780999999999999e-05 2024-09-14T23:58:55.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.144 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.144 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:55.145 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315468+0000", 2024-09-14T23:58:55.145 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4206e-05 2024-09-14T23:58:55.145 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:55.145 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:55.145 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:55.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.145 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5219 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3338240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:55.145 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:58:55.145 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 135.84197246400001, 2024-09-14T23:58:55.146 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057637764000000001, 2024-09-14T23:58:55.146 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:55.146 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:55.146 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:55.146 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:55.146 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:55.146 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:55.146 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5219 2024-09-14T23:58:55.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.146 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:55.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:55.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:58:55.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:55.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:55.147 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:58:55.147 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:55.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.147 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.147 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:55.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:58:55.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.03e-07 2024-09-14T23:58:55.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:55.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257983+0000", 2024-09-14T23:58:55.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9199999999999999e-07 2024-09-14T23:58:55.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.148 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.148 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:55.148 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257986+0000", 2024-09-14T23:58:55.148 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0860000000000002e-06 2024-09-14T23:58:55.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:55.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257989+0000", 2024-09-14T23:58:55.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6589999999999999e-06 2024-09-14T23:58:55.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:55.149 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315509+0000", 2024-09-14T23:58:55.149 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057520262000000003 2024-09-14T23:58:55.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.149 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.149 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:55.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315554+0000", 2024-09-14T23:58:55.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5015999999999997e-05 2024-09-14T23:58:55.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.150 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:55.150 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315619+0000", 2024-09-14T23:58:55.150 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5545999999999999e-05 2024-09-14T23:58:55.150 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:55.150 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:55.150 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:55.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.150 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.151 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5220 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3399680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:55.151 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:58:55.151 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 135.84196250799999, 2024-09-14T23:58:55.151 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057792128999999998, 2024-09-14T23:58:55.151 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:55.151 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:55.151 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:55.151 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:55.151 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:55.151 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:55.151 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5220 2024-09-14T23:58:55.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.152 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:55.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.152 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:55.152 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:58:55.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:55.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.152 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.152 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:55.152 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:58:55.152 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:55.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:55.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:58:55.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1799999999999999e-07 2024-09-14T23:58:55.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:55.153 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257993+0000", 2024-09-14T23:58:55.153 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6200000000000006e-07 2024-09-14T23:58:55.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.153 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.153 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:55.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257996+0000", 2024-09-14T23:58:55.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9050000000000001e-06 2024-09-14T23:58:55.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:55.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257998+0000", 2024-09-14T23:58:55.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5950000000000001e-06 2024-09-14T23:58:55.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.154 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.154 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:55.154 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315672+0000", 2024-09-14T23:58:55.154 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057673637999999999 2024-09-14T23:58:55.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:55.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315734+0000", 2024-09-14T23:58:55.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2216000000000002e-05 2024-09-14T23:58:55.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.155 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:55.155 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315784+0000", 2024-09-14T23:58:55.155 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9595000000000002e-05 2024-09-14T23:58:55.155 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:55.155 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:55.155 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:55.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.156 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.156 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5221 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3461120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:55.156 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:58:55.156 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 135.84195278999999, 2024-09-14T23:58:55.156 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058003557999999997, 2024-09-14T23:58:55.156 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:55.156 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:55.156 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:55.156 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:55.156 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:55.157 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:55.157 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5221 2024-09-14T23:58:55.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.157 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:55.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.157 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:55.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:58:55.157 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:55.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.157 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:55.157 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:58:55.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:55.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:55.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259905+0000", 2024-09-14T23:58:55.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019035759999999999 2024-09-14T23:58:55.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.158 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:55.158 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259915+0000", 2024-09-14T23:58:55.158 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0518e-05 2024-09-14T23:58:55.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:55.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259925+0000", 2024-09-14T23:58:55.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7180000000000008e-06 2024-09-14T23:58:55.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.159 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:55.159 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259930+0000", 2024-09-14T23:58:55.159 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4769999999999997e-06 2024-09-14T23:58:55.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.159 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:55.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315820+0000", 2024-09-14T23:58:55.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055889960000000002 2024-09-14T23:58:55.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:55.160 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315914+0000", 2024-09-14T23:58:55.160 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4625999999999998e-05 2024-09-14T23:58:55.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.160 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:55.161 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.316005+0000", 2024-09-14T23:58:55.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0682999999999999e-05 2024-09-14T23:58:55.161 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:55.161 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:55.161 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:55.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.161 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.161 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5231 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 4075520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:55.161 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:58:55.161 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 135.83988405599999, 2024-09-14T23:58:55.161 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057524733000000002, 2024-09-14T23:58:55.161 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:55.162 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:55.162 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:55.162 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:55.162 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:55.162 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:55.162 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5231 2024-09-14T23:58:55.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.162 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:55.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.162 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:55.162 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:58:55.162 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:55.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:55.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:58:55.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:55.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.163 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:55.163 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261904+0000", 2024-09-14T23:58:55.163 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018338790000000001 2024-09-14T23:58:55.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.163 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.164 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:55.164 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261910+0000", 2024-09-14T23:58:55.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9689999999999999e-06 2024-09-14T23:58:55.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.164 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:55.164 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261923+0000", 2024-09-14T23:58:55.164 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3061000000000001e-05 2024-09-14T23:58:55.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.164 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.164 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:55.165 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261931+0000", 2024-09-14T23:58:55.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3159999999999997e-06 2024-09-14T23:58:55.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.165 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.165 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:55.165 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317456+0000", 2024-09-14T23:58:55.165 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055525219000000001 2024-09-14T23:58:55.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.165 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.165 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:55.165 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317520+0000", 2024-09-14T23:58:55.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4010000000000005e-05 2024-09-14T23:58:55.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.166 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:55.166 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317595+0000", 2024-09-14T23:58:55.166 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4278999999999998e-05 2024-09-14T23:58:55.166 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:55.166 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:55.166 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:55.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.166 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5236 3.8 3:1071668d:::rbd_data.1118de1168b.0000000000000092:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:55.167 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:58:55.167 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 135.837870592, 2024-09-14T23:58:55.167 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057499462000000001, 2024-09-14T23:58:55.167 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:55.167 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:55.167 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:55.167 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:55.167 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:55.167 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:55.167 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5236 2024-09-14T23:58:55.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.167 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:55.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.168 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:55.168 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:58:55.168 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:55.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.168 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:55.168 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:58:55.168 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:55.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.168 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.169 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:55.169 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262084+0000", 2024-09-14T23:58:55.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3399999999999999e-07 2024-09-14T23:58:55.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.169 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:55.169 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262087+0000", 2024-09-14T23:58:55.169 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2310000000000001e-06 2024-09-14T23:58:55.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.169 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.169 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:55.170 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262095+0000", 2024-09-14T23:58:55.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3799999999999996e-06 2024-09-14T23:58:55.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.170 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.170 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:55.170 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262101+0000", 2024-09-14T23:58:55.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0009999999999998e-06 2024-09-14T23:58:55.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.170 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.170 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:55.170 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317625+0000", 2024-09-14T23:58:55.170 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055524111000000001 2024-09-14T23:58:55.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:55.171 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317702+0000", 2024-09-14T23:58:55.171 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7533999999999993e-05 2024-09-14T23:58:55.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.171 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-14T23:58:55.171 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317892+0000", 2024-09-14T23:58:55.171 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018991400000000001 2024-09-14T23:58:55.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.171 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.172 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:58:55.172 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319407+0000", 2024-09-14T23:58:55.172 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015145250000000001 2024-09-14T23:58:55.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.172 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:55.172 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319481+0000", 2024-09-14T23:58:55.172 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3826999999999996e-05 2024-09-14T23:58:55.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.172 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.172 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:55.172 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319541+0000", 2024-09-14T23:58:55.173 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0470999999999998e-05 2024-09-14T23:58:55.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.173 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.173 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:55.173 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319583+0000", 2024-09-14T23:58:55.173 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1934000000000002e-05 2024-09-14T23:58:55.173 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:55.173 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:55.173 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:55.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.173 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.174 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:132622 3.1e 3.7a50b9e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-14T23:58:55.174 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:43.371285+0000", 2024-09-14T23:58:55.174 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 71.728668690000006, 2024-09-14T23:58:55.174 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.67037987200000004, 2024-09-14T23:58:55.174 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:55.174 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:55.174 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:55.174 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:55.174 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:55.174 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:55.174 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 132622 2024-09-14T23:58:55.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.175 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:55.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.175 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:55.175 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371285+0000", 2024-09-14T23:58:55.175 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:55.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.175 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:55.175 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371285+0000", 2024-09-14T23:58:55.175 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:55.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.176 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:55.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371286+0000", 2024-09-14T23:58:55.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6400000000000002e-07 2024-09-14T23:58:55.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.176 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:55.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371289+0000", 2024-09-14T23:58:55.176 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2059999999999999e-06 2024-09-14T23:58:55.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.176 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.176 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:55.176 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371299+0000", 2024-09-14T23:58:55.177 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0288e-05 2024-09-14T23:58:55.177 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.177 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.177 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:55.177 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371306+0000", 2024-09-14T23:58:55.177 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1650000000000003e-06 2024-09-14T23:58:55.177 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.177 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.177 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:55.177 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371617+0000", 2024-09-14T23:58:55.177 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000311043 2024-09-14T23:58:55.177 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.178 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.178 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:58:55.178 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371625+0000", 2024-09-14T23:58:55.178 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.833e-06 2024-09-14T23:58:55.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.178 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.178 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:55.178 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.041651+0000", 2024-09-14T23:58:55.178 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.67002580199999995 2024-09-14T23:58:55.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.178 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.178 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:55.178 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.041665+0000", 2024-09-14T23:58:55.179 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3971000000000001e-05 2024-09-14T23:58:55.179 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:55.179 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:55.179 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:55.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.179 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.179 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:169975 3.5 3:a704a2f0:::rbd_data.1118427720d0.0000000000000030:32 [read 969728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-14T23:58:55.179 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:03.433958+0000", 2024-09-14T23:58:55.179 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 51.665996333999999, 2024-09-14T23:58:55.179 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057564811, 2024-09-14T23:58:55.179 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:55.179 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:55.180 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:55.180 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:55.180 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:55.180 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:55.180 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 169975 2024-09-14T23:58:55.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.180 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:55.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.180 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:55.180 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.433958+0000", 2024-09-14T23:58:55.180 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:55.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.181 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.181 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:55.181 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.433958+0000", 2024-09-14T23:58:55.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:55.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.181 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.181 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:55.181 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.435096+0000", 2024-09-14T23:58:55.181 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001138708 2024-09-14T23:58:55.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.181 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.181 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:55.182 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.435101+0000", 2024-09-14T23:58:55.182 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5050000000000001e-06 2024-09-14T23:58:55.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.182 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:55.182 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.435111+0000", 2024-09-14T23:58:55.182 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5810000000000003e-06 2024-09-14T23:58:55.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.182 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:55.182 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.435120+0000", 2024-09-14T23:58:55.182 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0089999999999996e-06 2024-09-14T23:58:55.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:55.183 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.491345+0000", 2024-09-14T23:58:55.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056225479000000002 2024-09-14T23:58:55.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.183 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:55.183 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.491423+0000", 2024-09-14T23:58:55.183 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7990000000000004e-05 2024-09-14T23:58:55.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.184 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:55.184 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.491523+0000", 2024-09-14T23:58:55.184 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9538999999999997e-05 2024-09-14T23:58:55.184 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:55.184 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:55.184 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:55.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.184 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.184 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:172639 3.5 3:a704a2f0:::rbd_data.1118427720d0.0000000000000030:32 [read 1645568~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e107)", 2024-09-14T23:58:55.184 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:03.988365+0000", 2024-09-14T23:58:55.184 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 51.111588613999999, 2024-09-14T23:58:55.185 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057556213000000002, 2024-09-14T23:58:55.185 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:55.185 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:55.185 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:55.185 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:55.185 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:55.185 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:55.185 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 172639 2024-09-14T23:58:55.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.185 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:55.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.186 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:55.186 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.988365+0000", 2024-09-14T23:58:55.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:55.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.186 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:55.186 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.988365+0000", 2024-09-14T23:58:55.186 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:55.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.186 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:55.186 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.989904+0000", 2024-09-14T23:58:55.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015380859999999999 2024-09-14T23:58:55.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:55.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.989909+0000", 2024-09-14T23:58:55.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2800000000000003e-06 2024-09-14T23:58:55.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.187 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:55.187 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.989925+0000", 2024-09-14T23:58:55.187 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5781999999999999e-05 2024-09-14T23:58:55.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.188 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:55.188 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.989933+0000", 2024-09-14T23:58:55.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0090000000000007e-06 2024-09-14T23:58:55.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.188 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:55.188 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:04.045741+0000", 2024-09-14T23:58:55.188 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055808112999999999 2024-09-14T23:58:55.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:55.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:04.045812+0000", 2024-09-14T23:58:55.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1531000000000003e-05 2024-09-14T23:58:55.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.189 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:55.189 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:04.045922+0000", 2024-09-14T23:58:55.189 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000109412 2024-09-14T23:58:55.189 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:55.189 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:55.189 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:55.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.190 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.190 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184571 3.2 3:46f8d78b:::rbd_data.1118427720d0.0000000000000001:32 [read 1405952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:55.190 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.086404+0000", 2024-09-14T23:58:55.190 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 48.013550256999999, 2024-09-14T23:58:55.190 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059148573000000003, 2024-09-14T23:58:55.190 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:55.190 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:55.190 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:55.190 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:55.190 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:55.190 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:55.190 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184571 2024-09-14T23:58:55.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.191 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:55.191 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.191 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:55.191 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.086404+0000", 2024-09-14T23:58:55.191 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:55.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.191 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.191 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:55.191 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.086404+0000", 2024-09-14T23:58:55.191 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:55.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.192 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:55.192 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087902+0000", 2024-09-14T23:58:55.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001498625 2024-09-14T23:58:55.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.192 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:55.192 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087907+0000", 2024-09-14T23:58:55.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9130000000000002e-06 2024-09-14T23:58:55.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.193 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:55.193 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087919+0000", 2024-09-14T23:58:55.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1719000000000001e-05 2024-09-14T23:58:55.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.193 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.193 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:55.193 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087927+0000", 2024-09-14T23:58:55.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3699999999999995e-06 2024-09-14T23:58:55.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.193 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.193 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:55.193 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143711+0000", 2024-09-14T23:58:55.194 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055783396999999998 2024-09-14T23:58:55.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.194 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:55.194 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143792+0000", 2024-09-14T23:58:55.194 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1292999999999996e-05 2024-09-14T23:58:55.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.194 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:55.194 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145552+0000", 2024-09-14T23:58:55.194 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001760256 2024-09-14T23:58:55.194 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:55.195 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:55.195 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:55.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.195 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.195 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184579 3.2 3:43badc91:::rbd_data.1118427720d0.0000000000000002:32 [read 1196032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:55.195 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.088145+0000", 2024-09-14T23:58:55.195 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 48.011808852999998, 2024-09-14T23:58:55.195 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057643169000000001, 2024-09-14T23:58:55.195 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:55.195 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:55.195 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:55.195 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:55.195 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:55.196 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:55.196 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184579 2024-09-14T23:58:55.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.196 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:55.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.196 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:55.196 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.088145+0000", 2024-09-14T23:58:55.196 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:55.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.196 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:55.197 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.088145+0000", 2024-09-14T23:58:55.197 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:55.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.197 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:55.197 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089947+0000", 2024-09-14T23:58:55.197 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001801764 2024-09-14T23:58:55.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.197 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:55.197 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089952+0000", 2024-09-14T23:58:55.197 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8659999999999997e-06 2024-09-14T23:58:55.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.198 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:55.198 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089966+0000", 2024-09-14T23:58:55.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3929e-05 2024-09-14T23:58:55.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.198 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:55.198 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089974+0000", 2024-09-14T23:58:55.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1429999999999998e-06 2024-09-14T23:58:55.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.198 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:55.199 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145636+0000", 2024-09-14T23:58:55.199 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055662287999999997 2024-09-14T23:58:55.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.199 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:55.199 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145710+0000", 2024-09-14T23:58:55.199 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4166999999999996e-05 2024-09-14T23:58:55.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.199 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:55.199 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145788+0000", 2024-09-14T23:58:55.199 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8011999999999998e-05 2024-09-14T23:58:55.200 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:55.200 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:55.200 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:55.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.200 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:211800 3.12 3:493125e8:::rbd_data.1118c900944f.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-14T23:58:55.200 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:13.660130+0000", 2024-09-14T23:58:55.200 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 41.439824526999999, 2024-09-14T23:58:55.200 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.55364332000000005, 2024-09-14T23:58:55.200 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:55.200 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:55.200 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:55.201 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:55.201 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:55.201 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:55.201 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 211800 2024-09-14T23:58:55.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.201 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:55.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.201 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:55.201 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.660130+0000", 2024-09-14T23:58:55.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:55.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.202 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:55.202 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.660130+0000", 2024-09-14T23:58:55.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:55.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.202 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:55.202 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.660133+0000", 2024-09-14T23:58:55.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.202e-06 2024-09-14T23:58:55.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.202 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:55.202 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.668135+0000", 2024-09-14T23:58:55.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0080027220000000003 2024-09-14T23:58:55.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.203 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:55.203 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.668172+0000", 2024-09-14T23:58:55.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6295999999999999e-05 2024-09-14T23:58:55.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.203 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:55.203 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.668187+0000", 2024-09-14T23:58:55.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5537e-05 2024-09-14T23:58:55.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.204 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:55.204 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.668284+0000", 2024-09-14T23:58:55.204 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6297000000000003e-05 2024-09-14T23:58:55.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.204 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:58:55.204 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.668299+0000", 2024-09-14T23:58:55.204 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5091e-05 2024-09-14T23:58:55.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.204 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:55.204 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:14.195910+0000", 2024-09-14T23:58:55.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.52761157999999997 2024-09-14T23:58:55.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:55.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:14.196106+0000", 2024-09-14T23:58:55.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019597399999999999 2024-09-14T23:58:55.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 0", 2024-09-14T23:58:55.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:14.197262+0000", 2024-09-14T23:58:55.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001155744 2024-09-14T23:58:55.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:58:55.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:14.207440+0000", 2024-09-14T23:58:55.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010178347000000001 2024-09-14T23:58:55.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:55.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:14.213691+0000", 2024-09-14T23:58:55.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0062504420000000002 2024-09-14T23:58:55.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:55.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:14.213735+0000", 2024-09-14T23:58:55.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4385999999999998e-05 2024-09-14T23:58:55.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.207 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:55.207 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:14.213773+0000", 2024-09-14T23:58:55.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7701999999999999e-05 2024-09-14T23:58:55.207 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:55.207 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:55.207 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:55.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.208 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:251476 3.5 3:a1f6ebd6:::rbd_object_map.11189263ca38.000000000000002a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126)", 2024-09-14T23:58:55.208 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:26.388925+0000", 2024-09-14T23:58:55.208 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 28.711029494000002, 2024-09-14T23:58:55.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.062808638, 2024-09-14T23:58:55.208 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:55.208 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:55.209 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:55.209 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:55.209 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:55.209 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:55.209 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 251476 2024-09-14T23:58:55.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.209 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:55.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.209 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:55.209 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.388925+0000", 2024-09-14T23:58:55.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:55.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:55.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.388925+0000", 2024-09-14T23:58:55.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:55.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:55.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.388926+0000", 2024-09-14T23:58:55.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5200000000000001e-06 2024-09-14T23:58:55.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:55.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.388933+0000", 2024-09-14T23:58:55.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.827e-06 2024-09-14T23:58:55.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:55.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.388945+0000", 2024-09-14T23:58:55.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1701e-05 2024-09-14T23:58:55.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:55.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.388950+0000", 2024-09-14T23:58:55.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6080000000000003e-06 2024-09-14T23:58:55.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.212 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:55.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.388999+0000", 2024-09-14T23:58:55.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8711999999999997e-05 2024-09-14T23:58:55.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.212 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:58:55.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.389006+0000", 2024-09-14T23:58:55.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.063e-06 2024-09-14T23:58:55.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:55.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.451371+0000", 2024-09-14T23:58:55.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.062365255000000001 2024-09-14T23:58:55.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:55.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.451472+0000", 2024-09-14T23:58:55.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100626 2024-09-14T23:58:55.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:55.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:55.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:55.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.451733+0000", 2024-09-14T23:58:55.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026132599999999998 2024-09-14T23:58:55.214 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:55.214 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:55.214 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:55.214 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:55.214 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:55.214 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:55.216 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T23:58:55.216 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:55.214+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:55.316 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:55.315+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:55.327 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:55.327 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:58:55.327 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:58:55.327 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:55.336 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T23:58:55.417 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:55.416+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:55.443 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:55.443 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:58:55.443 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:58:55.443 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:58:55.443 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:55.452 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T23:58:55.518 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:55.517+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:55.561 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:55.561 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:58:55.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:58:55.561 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:58:55.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.561 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186203 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1329152~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:55.561 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.194279+0000", 2024-09-14T23:58:55.561 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 48.364273343999997, 2024-09-14T23:58:55.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8011469710000001, 2024-09-14T23:58:55.562 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:55.562 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:55.562 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:55.562 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:55.562 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:55.562 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:55.562 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186203 2024-09-14T23:58:55.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.562 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:55.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:55.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194279+0000", 2024-09-14T23:58:55.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:55.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.563 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:55.563 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194279+0000", 2024-09-14T23:58:55.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:55.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.563 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:55.563 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194279+0000", 2024-09-14T23:58:55.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8100000000000001e-07 2024-09-14T23:58:55.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.563 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:55.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194291+0000", 2024-09-14T23:58:55.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1559999999999999e-05 2024-09-14T23:58:55.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:55.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194303+0000", 2024-09-14T23:58:55.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2130000000000001e-05 2024-09-14T23:58:55.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:55.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194311+0000", 2024-09-14T23:58:55.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8089999999999993e-06 2024-09-14T23:58:55.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.565 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:55.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194370+0000", 2024-09-14T23:58:55.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8739999999999997e-05 2024-09-14T23:58:55.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.565 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:55.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194376+0000", 2024-09-14T23:58:55.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2319999999999998e-06 2024-09-14T23:58:55.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.565 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:55.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986290+0000", 2024-09-14T23:58:55.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.791914334 2024-09-14T23:58:55.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:55.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986296+0000", 2024-09-14T23:58:55.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6300000000000003e-06 2024-09-14T23:58:55.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:55.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995193+0000", 2024-09-14T23:58:55.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0088975040000001 2024-09-14T23:58:55.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:55.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995307+0000", 2024-09-14T23:58:55.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000113656 2024-09-14T23:58:55.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.567 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:55.567 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995426+0000", 2024-09-14T23:58:55.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118495 2024-09-14T23:58:55.567 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:55.567 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:55.567 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:55.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.567 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186204 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1391616~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:55.567 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.194328+0000", 2024-09-14T23:58:55.567 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 48.364224012999998, 2024-09-14T23:58:55.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8013834310000001, 2024-09-14T23:58:55.567 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:55.567 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:55.567 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:55.568 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:55.568 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:55.568 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:55.568 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186204 2024-09-14T23:58:55.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.568 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:55.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.568 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:55.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194328+0000", 2024-09-14T23:58:55.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:55.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.569 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:55.569 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194328+0000", 2024-09-14T23:58:55.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:55.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.569 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:55.569 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194328+0000", 2024-09-14T23:58:55.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7100000000000002e-07 2024-09-14T23:58:55.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.569 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:55.569 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194331+0000", 2024-09-14T23:58:55.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.807e-06 2024-09-14T23:58:55.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.570 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:55.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194339+0000", 2024-09-14T23:58:55.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9340000000000006e-06 2024-09-14T23:58:55.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.570 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:55.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194345+0000", 2024-09-14T23:58:55.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1090000000000003e-06 2024-09-14T23:58:55.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.570 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:55.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194412+0000", 2024-09-14T23:58:55.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6229000000000006e-05 2024-09-14T23:58:55.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:55.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194422+0000", 2024-09-14T23:58:55.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.078e-05 2024-09-14T23:58:55.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:55.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986322+0000", 2024-09-14T23:58:55.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.79189964899999998 2024-09-14T23:58:55.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:55.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986332+0000", 2024-09-14T23:58:55.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0095000000000001e-05 2024-09-14T23:58:55.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:55.572 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995554+0000", 2024-09-14T23:58:55.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0092218630000001 2024-09-14T23:58:55.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:55.572 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995610+0000", 2024-09-14T23:58:55.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6109000000000001e-05 2024-09-14T23:58:55.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.573 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:55.573 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995711+0000", 2024-09-14T23:58:55.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101285 2024-09-14T23:58:55.573 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:55.573 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:55.573 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:55.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.573 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186205 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1333248~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:55.573 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.194354+0000", 2024-09-14T23:58:55.573 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 48.364198082999998, 2024-09-14T23:58:55.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8015471490000001, 2024-09-14T23:58:55.573 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:55.573 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:55.574 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:55.574 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:55.574 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:55.574 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:55.574 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186205 2024-09-14T23:58:55.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.574 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:55.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.574 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:55.574 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194354+0000", 2024-09-14T23:58:55.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:55.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.574 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:55.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194354+0000", 2024-09-14T23:58:55.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:55.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:55.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194354+0000", 2024-09-14T23:58:55.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.39e-07 2024-09-14T23:58:55.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:55.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194357+0000", 2024-09-14T23:58:55.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3479999999999998e-06 2024-09-14T23:58:55.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:55.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194364+0000", 2024-09-14T23:58:55.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0899999999999999e-06 2024-09-14T23:58:55.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:55.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194371+0000", 2024-09-14T23:58:55.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5530000000000004e-06 2024-09-14T23:58:55.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:55.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194469+0000", 2024-09-14T23:58:55.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8119e-05 2024-09-14T23:58:55.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.577 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:55.577 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194475+0000", 2024-09-14T23:58:55.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7910000000000001e-06 2024-09-14T23:58:55.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.577 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:55.577 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986367+0000", 2024-09-14T23:58:55.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.79189214699999999 2024-09-14T23:58:55.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.577 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:55.577 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986375+0000", 2024-09-14T23:58:55.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7800000000000001e-06 2024-09-14T23:58:55.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:55.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995746+0000", 2024-09-14T23:58:55.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0093706469999999 2024-09-14T23:58:55.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:55.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995810+0000", 2024-09-14T23:58:55.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4068999999999999e-05 2024-09-14T23:58:55.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:55.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995901+0000", 2024-09-14T23:58:55.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1166000000000001e-05 2024-09-14T23:58:55.579 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:55.579 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:55.579 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:55.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.579 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186206 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1395712~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:55.579 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.194380+0000", 2024-09-14T23:58:55.579 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 48.364172062000002, 2024-09-14T23:58:55.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.801678347, 2024-09-14T23:58:55.579 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:55.579 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:55.579 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:55.579 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:55.579 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:55.580 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:55.580 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186206 2024-09-14T23:58:55.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.580 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:55.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:55.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194380+0000", 2024-09-14T23:58:55.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:55.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:55.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194380+0000", 2024-09-14T23:58:55.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:55.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:55.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194380+0000", 2024-09-14T23:58:55.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4400000000000001e-07 2024-09-14T23:58:55.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:55.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194383+0000", 2024-09-14T23:58:55.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3460000000000001e-06 2024-09-14T23:58:55.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:55.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194390+0000", 2024-09-14T23:58:55.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2189999999999998e-06 2024-09-14T23:58:55.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:55.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194403+0000", 2024-09-14T23:58:55.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2941e-05 2024-09-14T23:58:55.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:55.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194545+0000", 2024-09-14T23:58:55.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014232700000000001 2024-09-14T23:58:55.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:55.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194565+0000", 2024-09-14T23:58:55.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9409999999999999e-05 2024-09-14T23:58:55.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:55.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986409+0000", 2024-09-14T23:58:55.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.79184416000000002 2024-09-14T23:58:55.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:55.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986417+0000", 2024-09-14T23:58:55.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.551e-06 2024-09-14T23:58:55.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:55.584 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995921+0000", 2024-09-14T23:58:55.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.009504212 2024-09-14T23:58:55.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.584 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:55.584 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995974+0000", 2024-09-14T23:58:55.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2617999999999997e-05 2024-09-14T23:58:55.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.584 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:55.584 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996058+0000", 2024-09-14T23:58:55.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4118999999999999e-05 2024-09-14T23:58:55.584 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:55.584 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:55.584 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:55.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.585 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186207 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1337344~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:55.585 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.194412+0000", 2024-09-14T23:58:55.585 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 48.364139463999997, 2024-09-14T23:58:55.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.801848141, 2024-09-14T23:58:55.585 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:55.585 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:55.585 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:55.585 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:55.585 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:55.585 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:55.585 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186207 2024-09-14T23:58:55.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.586 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:55.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:55.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194412+0000", 2024-09-14T23:58:55.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:55.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:55.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194412+0000", 2024-09-14T23:58:55.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:55.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:55.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194413+0000", 2024-09-14T23:58:55.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5600000000000001e-07 2024-09-14T23:58:55.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:55.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194418+0000", 2024-09-14T23:58:55.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3720000000000004e-06 2024-09-14T23:58:55.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:55.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194423+0000", 2024-09-14T23:58:55.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.763e-06 2024-09-14T23:58:55.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:55.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194426+0000", 2024-09-14T23:58:55.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1480000000000002e-06 2024-09-14T23:58:55.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:55.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194596+0000", 2024-09-14T23:58:55.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017009600000000001 2024-09-14T23:58:55.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:55.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194606+0000", 2024-09-14T23:58:55.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6639999999999993e-06 2024-09-14T23:58:55.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:55.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986460+0000", 2024-09-14T23:58:55.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.79185424400000004 2024-09-14T23:58:55.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:55.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986469+0000", 2024-09-14T23:58:55.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4419999999999998e-06 2024-09-14T23:58:55.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:55.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996123+0000", 2024-09-14T23:58:55.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0096546070000001 2024-09-14T23:58:55.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:55.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996179+0000", 2024-09-14T23:58:55.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5850000000000002e-05 2024-09-14T23:58:55.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:55.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996261+0000", 2024-09-14T23:58:55.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1599000000000003e-05 2024-09-14T23:58:55.590 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:55.590 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:55.590 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:55.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.591 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186208 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1399808~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:55.591 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.194445+0000", 2024-09-14T23:58:55.591 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 48.364107294, 2024-09-14T23:58:55.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8019822269999999, 2024-09-14T23:58:55.591 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:55.591 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:55.591 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:55.591 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:55.591 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:55.591 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:55.591 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186208 2024-09-14T23:58:55.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.591 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:55.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:55.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194445+0000", 2024-09-14T23:58:55.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:55.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:55.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194445+0000", 2024-09-14T23:58:55.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:55.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:55.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978674+0000", 2024-09-14T23:58:55.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.784229124 2024-09-14T23:58:55.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:55.593 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978680+0000", 2024-09-14T23:58:55.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7969999999999997e-06 2024-09-14T23:58:55.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:55.593 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978689+0000", 2024-09-14T23:58:55.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6949999999999995e-06 2024-09-14T23:58:55.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:55.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978696+0000", 2024-09-14T23:58:55.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1300000000000003e-06 2024-09-14T23:58:55.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.594 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:55.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986509+0000", 2024-09-14T23:58:55.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0078130969999999998 2024-09-14T23:58:55.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.594 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:55.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986518+0000", 2024-09-14T23:58:55.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2810000000000008e-06 2024-09-14T23:58:55.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:55.595 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996280+0000", 2024-09-14T23:58:55.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0097623120000001 2024-09-14T23:58:55.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:55.595 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996337+0000", 2024-09-14T23:58:55.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7244000000000003e-05 2024-09-14T23:58:55.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:55.595 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996427+0000", 2024-09-14T23:58:55.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9547000000000002e-05 2024-09-14T23:58:55.595 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:55.596 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:55.596 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:55.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.596 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4172.0:7 3.11 3:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:55.596 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.483386+0000", 2024-09-14T23:58:55.596 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 48.075165869000003, 2024-09-14T23:58:55.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.558667389, 2024-09-14T23:58:55.596 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:55.596 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:55.596 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:58:55.596 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:55.596 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4172", 2024-09-14T23:58:55.596 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/2542269203", 2024-09-14T23:58:55.597 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 7 2024-09-14T23:58:55.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.597 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:55.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.597 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:55.597 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.483386+0000", 2024-09-14T23:58:55.597 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:55.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.597 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:55.597 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.483386+0000", 2024-09-14T23:58:55.597 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:55.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.598 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:55.598 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978674+0000", 2024-09-14T23:58:55.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.49528781599999999 2024-09-14T23:58:55.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.598 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:55.598 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978687+0000", 2024-09-14T23:58:55.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3244999999999999e-05 2024-09-14T23:58:55.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.598 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:55.598 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978699+0000", 2024-09-14T23:58:55.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1832e-05 2024-09-14T23:58:55.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.599 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:55.599 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978707+0000", 2024-09-14T23:58:55.599 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4489999999999999e-06 2024-09-14T23:58:55.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.599 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:55.599 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978801+0000", 2024-09-14T23:58:55.599 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3511999999999995e-05 2024-09-14T23:58:55.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.599 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:55.599 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978814+0000", 2024-09-14T23:58:55.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2789000000000001e-05 2024-09-14T23:58:55.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.600 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:55.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.041948+0000", 2024-09-14T23:58:55.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0631338269999999 2024-09-14T23:58:55.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.600 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:55.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.042053+0000", 2024-09-14T23:58:55.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105919 2024-09-14T23:58:55.600 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:55.600 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:55.600 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:55.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.601 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187209 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1575936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:55.601 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.991888+0000", 2024-09-14T23:58:55.601 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.566663898000002, 2024-09-14T23:58:55.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0712115470000001, 2024-09-14T23:58:55.601 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:55.601 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:55.601 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:55.601 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:55.601 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:55.601 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:55.601 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187209 2024-09-14T23:58:55.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.602 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:55.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.602 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:55.602 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991888+0000", 2024-09-14T23:58:55.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:55.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.602 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:55.602 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991888+0000", 2024-09-14T23:58:55.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:55.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.602 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:55.603 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991888+0000", 2024-09-14T23:58:55.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1199999999999998e-07 2024-09-14T23:58:55.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:55.603 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991890+0000", 2024-09-14T23:58:55.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9810000000000002e-06 2024-09-14T23:58:55.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:55.603 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991897+0000", 2024-09-14T23:58:55.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2609999999999999e-06 2024-09-14T23:58:55.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:55.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991902+0000", 2024-09-14T23:58:55.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0000000000000004e-06 2024-09-14T23:58:55.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:55.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995686+0000", 2024-09-14T23:58:55.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0037839290000000001 2024-09-14T23:58:55.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:55.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995693+0000", 2024-09-14T23:58:55.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7589999999999997e-06 2024-09-14T23:58:55.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:55.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062846+0000", 2024-09-14T23:58:55.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.067152417 2024-09-14T23:58:55.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:55.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062935+0000", 2024-09-14T23:58:55.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9080999999999996e-05 2024-09-14T23:58:55.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:55.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.063100+0000", 2024-09-14T23:58:55.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016470699999999999 2024-09-14T23:58:55.606 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:55.606 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:55.606 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:55.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.606 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187210 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1591296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:55.606 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.991908+0000", 2024-09-14T23:58:55.606 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.566643984999999, 2024-09-14T23:58:55.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0713882530000001, 2024-09-14T23:58:55.606 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:55.606 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:55.606 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:55.606 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:55.607 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:55.607 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:55.607 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187210 2024-09-14T23:58:55.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.607 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:55.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.607 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:55.607 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991908+0000", 2024-09-14T23:58:55.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:55.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.607 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:55.607 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991908+0000", 2024-09-14T23:58:55.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:55.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:55.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993673+0000", 2024-09-14T23:58:55.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017651030000000001 2024-09-14T23:58:55.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:55.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993681+0000", 2024-09-14T23:58:55.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.858e-06 2024-09-14T23:58:55.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:55.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993691+0000", 2024-09-14T23:58:55.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9920000000000002e-06 2024-09-14T23:58:55.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:55.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993698+0000", 2024-09-14T23:58:55.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6490000000000002e-06 2024-09-14T23:58:55.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:55.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995731+0000", 2024-09-14T23:58:55.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0020338959999999999 2024-09-14T23:58:55.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:55.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995741+0000", 2024-09-14T23:58:55.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3079999999999997e-06 2024-09-14T23:58:55.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:55.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.063130+0000", 2024-09-14T23:58:55.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.067389159 2024-09-14T23:58:55.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:55.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.063206+0000", 2024-09-14T23:58:55.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6577999999999998e-05 2024-09-14T23:58:55.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:55.611 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.063296+0000", 2024-09-14T23:58:55.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9710000000000007e-05 2024-09-14T23:58:55.611 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:55.611 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:55.611 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:55.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.611 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187216 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1603584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:55.611 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.993831+0000", 2024-09-14T23:58:55.611 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.564720778999998, 2024-09-14T23:58:55.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.071244627, 2024-09-14T23:58:55.611 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:55.611 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:55.611 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:55.612 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:55.612 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:55.612 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:55.612 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187216 2024-09-14T23:58:55.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.612 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:55.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.612 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:55.612 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993831+0000", 2024-09-14T23:58:55.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:55.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.612 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:55.612 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993831+0000", 2024-09-14T23:58:55.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:55.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:55.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993832+0000", 2024-09-14T23:58:55.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9700000000000002e-07 2024-09-14T23:58:55.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:55.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993834+0000", 2024-09-14T23:58:55.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6170000000000001e-06 2024-09-14T23:58:55.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:55.616 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993841+0000", 2024-09-14T23:58:55.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6980000000000001e-06 2024-09-14T23:58:55.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:55.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993846+0000", 2024-09-14T23:58:55.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4190000000000001e-06 2024-09-14T23:58:55.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:55.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995988+0000", 2024-09-14T23:58:55.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0021417390000000001 2024-09-14T23:58:55.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:55.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995998+0000", 2024-09-14T23:58:55.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5100000000000004e-06 2024-09-14T23:58:55.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:55.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.064883+0000", 2024-09-14T23:58:55.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.06888597 2024-09-14T23:58:55.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:55.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.064977+0000", 2024-09-14T23:58:55.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3817e-05 2024-09-14T23:58:55.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:55.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065076+0000", 2024-09-14T23:58:55.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8460000000000003e-05 2024-09-14T23:58:55.618 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:55.618 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:55.618 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:55.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.619 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187217 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1592320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:55.619 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.993855+0000", 2024-09-14T23:58:55.619 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.564697346000003, 2024-09-14T23:58:55.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0714239349999999, 2024-09-14T23:58:55.619 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:55.619 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:55.619 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:55.619 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:55.619 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:55.619 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:55.619 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187217 2024-09-14T23:58:55.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.620 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:55.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:55.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993855+0000", 2024-09-14T23:58:55.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:55.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:55.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993855+0000", 2024-09-14T23:58:55.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:55.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:55.621 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993855+0000", 2024-09-14T23:58:55.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4899999999999995e-07 2024-09-14T23:58:55.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:55.621 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993857+0000", 2024-09-14T23:58:55.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3379999999999999e-06 2024-09-14T23:58:55.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:55.621 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993864+0000", 2024-09-14T23:58:55.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6000000000000003e-06 2024-09-14T23:58:55.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:55.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993869+0000", 2024-09-14T23:58:55.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3210000000000004e-06 2024-09-14T23:58:55.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:55.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996038+0000", 2024-09-14T23:58:55.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0021683050000000001 2024-09-14T23:58:55.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:55.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996048+0000", 2024-09-14T23:58:55.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0173999999999999e-05 2024-09-14T23:58:55.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:55.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065108+0000", 2024-09-14T23:58:55.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0690600619999999 2024-09-14T23:58:55.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:55.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065171+0000", 2024-09-14T23:58:55.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3423999999999994e-05 2024-09-14T23:58:55.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:55.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065279+0000", 2024-09-14T23:58:55.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000107162 2024-09-14T23:58:55.623 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:55.624 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:55.624 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:55.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.624 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187218 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1607680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:55.624 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.993877+0000", 2024-09-14T23:58:55.624 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.564675066, 2024-09-14T23:58:55.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0716976810000001, 2024-09-14T23:58:55.624 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:55.624 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:55.624 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:55.624 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:55.624 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:55.624 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:55.624 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187218 2024-09-14T23:58:55.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.625 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:55.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.625 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:55.625 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993877+0000", 2024-09-14T23:58:55.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:55.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.625 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:55.625 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993877+0000", 2024-09-14T23:58:55.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:55.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.625 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:55.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993877+0000", 2024-09-14T23:58:55.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8299999999999998e-07 2024-09-14T23:58:55.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:55.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993879+0000", 2024-09-14T23:58:55.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5659999999999999e-06 2024-09-14T23:58:55.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:55.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993882+0000", 2024-09-14T23:58:55.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7280000000000001e-06 2024-09-14T23:58:55.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:55.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993885+0000", 2024-09-14T23:58:55.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.751e-06 2024-09-14T23:58:55.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:55.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996081+0000", 2024-09-14T23:58:55.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0021960590000000002 2024-09-14T23:58:55.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:55.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996091+0000", 2024-09-14T23:58:55.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4930000000000004e-06 2024-09-14T23:58:55.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:55.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065308+0000", 2024-09-14T23:58:55.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.069217407 2024-09-14T23:58:55.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:55.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065414+0000", 2024-09-14T23:58:55.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105438 2024-09-14T23:58:55.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:55.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065575+0000", 2024-09-14T23:58:55.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000160956 2024-09-14T23:58:55.628 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:55.628 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:55.628 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:55.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.628 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187219 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1596416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:55.629 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.993889+0000", 2024-09-14T23:58:55.629 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.564663224, 2024-09-14T23:58:55.629 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0720031190000001, 2024-09-14T23:58:55.629 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:55.629 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:55.629 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:55.629 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:55.629 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:55.629 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:55.629 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187219 2024-09-14T23:58:55.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.629 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:55.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.629 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:55.629 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993889+0000", 2024-09-14T23:58:55.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:55.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:55.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993889+0000", 2024-09-14T23:58:55.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:55.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:55.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993889+0000", 2024-09-14T23:58:55.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2399999999999999e-07 2024-09-14T23:58:55.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:55.631 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993890+0000", 2024-09-14T23:58:55.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1939999999999999e-06 2024-09-14T23:58:55.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.631 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:55.631 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993893+0000", 2024-09-14T23:58:55.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1300000000000001e-06 2024-09-14T23:58:55.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.631 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:55.631 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993896+0000", 2024-09-14T23:58:55.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6879999999999999e-06 2024-09-14T23:58:55.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.632 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:55.632 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996119+0000", 2024-09-14T23:58:55.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0022226149999999998 2024-09-14T23:58:55.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.632 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:55.632 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996123+0000", 2024-09-14T23:58:55.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.211e-06 2024-09-14T23:58:55.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.632 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:55.632 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065616+0000", 2024-09-14T23:58:55.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0694935240000001 2024-09-14T23:58:55.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.633 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:55.633 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065733+0000", 2024-09-14T23:58:55.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000116492 2024-09-14T23:58:55.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.633 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:55.633 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065892+0000", 2024-09-14T23:58:55.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015894100000000001 2024-09-14T23:58:55.633 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:55.633 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:55.633 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:55.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.634 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187220 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1611776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:55.634 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.993899+0000", 2024-09-14T23:58:55.634 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.564652766000002, 2024-09-14T23:58:55.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.072298041, 2024-09-14T23:58:55.634 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:55.634 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:55.634 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:55.634 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:55.634 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:55.634 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:55.634 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187220 2024-09-14T23:58:55.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.635 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:55.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:55.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993899+0000", 2024-09-14T23:58:55.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:55.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:55.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993899+0000", 2024-09-14T23:58:55.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:55.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:55.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993899+0000", 2024-09-14T23:58:55.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2100000000000001e-07 2024-09-14T23:58:55.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:55.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993901+0000", 2024-09-14T23:58:55.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.141e-06 2024-09-14T23:58:55.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:55.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993904+0000", 2024-09-14T23:58:55.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1470000000000001e-06 2024-09-14T23:58:55.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:55.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993906+0000", 2024-09-14T23:58:55.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6340000000000001e-06 2024-09-14T23:58:55.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.637 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:55.637 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996160+0000", 2024-09-14T23:58:55.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0022537659999999999 2024-09-14T23:58:55.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.637 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:55.637 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996169+0000", 2024-09-14T23:58:55.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2739999999999996e-06 2024-09-14T23:58:55.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.638 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:55.638 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065981+0000", 2024-09-14T23:58:55.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.069811742 2024-09-14T23:58:55.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.638 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:55.638 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.066092+0000", 2024-09-14T23:58:55.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000110701 2024-09-14T23:58:55.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.638 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:55.638 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.066197+0000", 2024-09-14T23:58:55.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010541500000000001 2024-09-14T23:58:55.638 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:55.639 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:55.639 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:55.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.639 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187221 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1600512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:55.639 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.993909+0000", 2024-09-14T23:58:55.639 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.564642554000002, 2024-09-14T23:58:55.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.072643499, 2024-09-14T23:58:55.639 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:55.639 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:55.639 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:55.639 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:55.639 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:55.639 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:55.640 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187221 2024-09-14T23:58:55.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.640 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:55.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:55.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993909+0000", 2024-09-14T23:58:55.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:55.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:55.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993909+0000", 2024-09-14T23:58:55.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:55.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:55.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995673+0000", 2024-09-14T23:58:55.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001763141 2024-09-14T23:58:55.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:55.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995677+0000", 2024-09-14T23:58:55.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8709999999999996e-06 2024-09-14T23:58:55.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:55.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995686+0000", 2024-09-14T23:58:55.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6649999999999992e-06 2024-09-14T23:58:55.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:55.642 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995691+0000", 2024-09-14T23:58:55.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0289999999999996e-06 2024-09-14T23:58:55.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:55.642 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996199+0000", 2024-09-14T23:58:55.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00050799099999999995 2024-09-14T23:58:55.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:55.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996205+0000", 2024-09-14T23:58:55.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8259999999999998e-06 2024-09-14T23:58:55.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:55.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.066238+0000", 2024-09-14T23:58:55.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.070032928 2024-09-14T23:58:55.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:55.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.066393+0000", 2024-09-14T23:58:55.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015549800000000001 2024-09-14T23:58:55.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:55.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.066553+0000", 2024-09-14T23:58:55.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015955 2024-09-14T23:58:55.644 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:55.644 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:55.644 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:55.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.644 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187223 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1604608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:55.644 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.995728+0000", 2024-09-14T23:58:55.644 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.562823737999999, 2024-09-14T23:58:55.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.071486478, 2024-09-14T23:58:55.644 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:55.644 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:55.644 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:55.645 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:55.645 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:55.645 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:55.645 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187223 2024-09-14T23:58:55.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.645 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:55.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:55.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995728+0000", 2024-09-14T23:58:55.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:55.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:55.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995728+0000", 2024-09-14T23:58:55.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:55.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:55.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995728+0000", 2024-09-14T23:58:55.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.65e-07 2024-09-14T23:58:55.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:55.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995730+0000", 2024-09-14T23:58:55.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.406e-06 2024-09-14T23:58:55.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:55.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995734+0000", 2024-09-14T23:58:55.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2640000000000002e-06 2024-09-14T23:58:55.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:55.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995737+0000", 2024-09-14T23:58:55.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0589999999999998e-06 2024-09-14T23:58:55.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:55.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996288+0000", 2024-09-14T23:58:55.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00055059599999999998 2024-09-14T23:58:55.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:55.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996294+0000", 2024-09-14T23:58:55.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0260000000000004e-06 2024-09-14T23:58:55.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:55.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.066938+0000", 2024-09-14T23:58:55.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0706439510000001 2024-09-14T23:58:55.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:55.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067050+0000", 2024-09-14T23:58:55.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011220100000000001 2024-09-14T23:58:55.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:55.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067215+0000", 2024-09-14T23:58:55.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016470999999999999 2024-09-14T23:58:55.649 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:55.649 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:55.649 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:55.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.649 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187224 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1619968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:55.649 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.995742+0000", 2024-09-14T23:58:55.649 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.562809923000003, 2024-09-14T23:58:55.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0717603609999999, 2024-09-14T23:58:55.649 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:55.650 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:55.650 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:55.650 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:55.650 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:55.650 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:55.650 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187224 2024-09-14T23:58:55.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.650 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:55.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:55.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995742+0000", 2024-09-14T23:58:55.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:55.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:55.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995742+0000", 2024-09-14T23:58:55.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:55.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:55.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995742+0000", 2024-09-14T23:58:55.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2399999999999999e-07 2024-09-14T23:58:55.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:55.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995744+0000", 2024-09-14T23:58:55.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.356e-06 2024-09-14T23:58:55.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.652 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:55.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995747+0000", 2024-09-14T23:58:55.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.304e-06 2024-09-14T23:58:55.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.652 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:55.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995750+0000", 2024-09-14T23:58:55.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6510000000000001e-06 2024-09-14T23:58:55.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.652 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:55.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996327+0000", 2024-09-14T23:58:55.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00057794300000000003 2024-09-14T23:58:55.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:55.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996337+0000", 2024-09-14T23:58:55.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3759999999999997e-06 2024-09-14T23:58:55.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:55.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067244+0000", 2024-09-14T23:58:55.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.070907367 2024-09-14T23:58:55.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:55.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067341+0000", 2024-09-14T23:58:55.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7277999999999999e-05 2024-09-14T23:58:55.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:55.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067502+0000", 2024-09-14T23:58:55.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016086200000000001 2024-09-14T23:58:55.654 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:55.654 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:55.654 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:55.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.654 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187225 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1608704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:55.654 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.995766+0000", 2024-09-14T23:58:55.654 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.562785749, 2024-09-14T23:58:55.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.072043649, 2024-09-14T23:58:55.655 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:55.655 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:55.655 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:55.655 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:55.655 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:55.655 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:55.655 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187225 2024-09-14T23:58:55.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.655 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:55.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.655 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:55.655 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995766+0000", 2024-09-14T23:58:55.655 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:55.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:55.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995766+0000", 2024-09-14T23:58:55.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:55.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:55.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995766+0000", 2024-09-14T23:58:55.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9200000000000002e-07 2024-09-14T23:58:55.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:55.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995768+0000", 2024-09-14T23:58:55.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8679999999999999e-06 2024-09-14T23:58:55.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:55.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995772+0000", 2024-09-14T23:58:55.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6339999999999999e-06 2024-09-14T23:58:55.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:55.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995775+0000", 2024-09-14T23:58:55.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8159999999999998e-06 2024-09-14T23:58:55.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:55.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996372+0000", 2024-09-14T23:58:55.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00059758800000000003 2024-09-14T23:58:55.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:55.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996377+0000", 2024-09-14T23:58:55.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7770000000000002e-06 2024-09-14T23:58:55.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:55.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067534+0000", 2024-09-14T23:58:55.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.071156821 2024-09-14T23:58:55.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:55.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067638+0000", 2024-09-14T23:58:55.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000103737 2024-09-14T23:58:55.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.659 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:55.659 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067810+0000", 2024-09-14T23:58:55.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017211599999999999 2024-09-14T23:58:55.659 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:55.659 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:55.659 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:55.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.659 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187226 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1624064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:55.659 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.995778+0000", 2024-09-14T23:58:55.659 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.562773569999997, 2024-09-14T23:58:55.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.072339081, 2024-09-14T23:58:55.660 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:55.660 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:55.660 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:55.660 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:55.660 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:55.660 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:55.660 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187226 2024-09-14T23:58:55.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.660 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:55.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:55.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995778+0000", 2024-09-14T23:58:55.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:55.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:55.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995778+0000", 2024-09-14T23:58:55.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:55.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:55.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995779+0000", 2024-09-14T23:58:55.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0699999999999999e-07 2024-09-14T23:58:55.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:55.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995780+0000", 2024-09-14T23:58:55.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2139999999999999e-06 2024-09-14T23:58:55.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:55.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995783+0000", 2024-09-14T23:58:55.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1250000000000001e-06 2024-09-14T23:58:55.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:55.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995787+0000", 2024-09-14T23:58:55.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6679999999999999e-06 2024-09-14T23:58:55.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:55.663 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996404+0000", 2024-09-14T23:58:55.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00061777499999999999 2024-09-14T23:58:55.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.663 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:55.663 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996414+0000", 2024-09-14T23:58:55.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.431e-06 2024-09-14T23:58:55.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.663 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:55.663 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067844+0000", 2024-09-14T23:58:55.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0714302410000001 2024-09-14T23:58:55.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.663 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:55.664 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067952+0000", 2024-09-14T23:58:55.664 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000107778 2024-09-14T23:58:55.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.664 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.664 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:55.664 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.068117+0000", 2024-09-14T23:58:55.664 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000165642 2024-09-14T23:58:55.664 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:55.664 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:55.664 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:55.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.664 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.664 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187227 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1612800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:55.664 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.995806+0000", 2024-09-14T23:58:55.665 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.562746333, 2024-09-14T23:58:55.665 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0726092810000001, 2024-09-14T23:58:55.665 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:55.665 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:55.665 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:55.665 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:55.665 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:55.665 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:55.665 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187227 2024-09-14T23:58:55.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.665 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:55.665 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.665 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:55.665 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995806+0000", 2024-09-14T23:58:55.665 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:55.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:55.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995806+0000", 2024-09-14T23:58:55.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:55.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:55.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995672+0000", 2024-09-14T23:58:55.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99986608099999996 2024-09-14T23:58:55.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:55.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995677+0000", 2024-09-14T23:58:55.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9520000000000002e-06 2024-09-14T23:58:55.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:55.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995686+0000", 2024-09-14T23:58:55.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8199999999999992e-06 2024-09-14T23:58:55.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:55.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995693+0000", 2024-09-14T23:58:55.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1630000000000001e-06 2024-09-14T23:58:55.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:55.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.068149+0000", 2024-09-14T23:58:55.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.072456692000000003 2024-09-14T23:58:55.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.668 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.668 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:55.668 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.068251+0000", 2024-09-14T23:58:55.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101963 2024-09-14T23:58:55.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:55.668 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:55.668 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:55.668 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.068415+0000", 2024-09-14T23:58:55.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016360999999999999 2024-09-14T23:58:55.668 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:55.668 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:55.668 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:55.669 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:55.669 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:55.669 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:55.669 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T23:58:55.671 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T23:58:55.671 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:55.669+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:55.771 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:58:55.770+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:58:55.796 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:58:55.797 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:58:55.813 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:58:55.814 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:58:55.820 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:55.820 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:58:55.820 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:58:55.820 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:55.829 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T23:58:55.829 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:58:55.830 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:58:55.845 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:58:55.845 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:58:55.862 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:58:55.862 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:58:55.872 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:55.870+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:55.878 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:58:55.879 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:58:55.895 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:58:55.895 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:58:55.912 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:58:55.912 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:58:55.973 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:55.972+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:55.976 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:55.977 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:55.985 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T23:58:56.070 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is set 2024-09-14T23:58:56.073 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:56.072+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:56.154 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:56.154 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:56.164 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T23:58:56.207 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:56.206+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:56.272 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:56.272 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:56.276 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:56.274+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:56.281 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T23:58:56.376 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:56.375+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:56.428 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:56.429 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:56.437 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T23:58:56.477 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:56.475+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:56.545 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:56.545 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:56.553 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T23:58:56.597 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:56.596+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:56.661 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:56.661 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:56.669 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T23:58:56.710 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:56.709+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:56.780 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:56.779+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:56.816 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:56.816 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:56.825 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T23:58:56.909 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:56.908+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:56.973 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:56.973 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:56.981 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T23:58:56.982 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:56.980+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:57.082 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:57.080+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:57.107 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:57.107 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:57.118 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T23:58:57.203 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:57.202+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:57.270 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:57.270 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:57.279 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T23:58:57.283 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:57.282+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:57.384 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:57.382+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:57.426 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:57.426 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:57.435 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T23:58:57.519 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:57.518+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:57.584 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:57.584 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:57.585 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:57.584+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:57.592 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T23:58:57.686 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:58:57.685+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:58:57.740 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:57.740 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:57.749 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T23:58:57.792 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:57.791+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:57.856 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:57.856 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:57.865 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T23:58:57.906 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:57.904+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:57.923 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T23:58:57.988 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:58:57.987+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:58:58.019 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:58:58.020 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:58:58.029 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T23:58:58.086 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T23:58:58.089 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:58.087+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:58.104 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:58:58.104 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:58:58.134 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:58:58.135 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:58:58.151 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:58:58.151 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:58:58.163 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:58.164 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:58.164 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:58:58.164 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:58:58.164 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:58.169 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:58:58.169 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:58:58.174 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T23:58:58.187 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:58:58.188 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:58:58.189 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:58:58.188+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:58:58.207 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:58:58.207 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:58:58.225 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:58:58.225 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:58:58.242 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:58:58.243 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:58:58.290 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:58.289+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:58.300 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:58.300 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:58:58.300 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:58:58.300 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:58:58.300 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.300 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:56107 3.0 e56/23 3:00c47890:::rbd_data.1118c4b3aebe.0000000000000009:head v 56'125, mlcod=52'123)", 2024-09-14T23:58:58.300 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.236123+0000", 2024-09-14T23:58:58.300 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.06140566400001, 2024-09-14T23:58:58.301 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0061273220000000002, 2024-09-14T23:58:58.301 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:58.301 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:58.301 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:58.301 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:58.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.301 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:58.301 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.236123+0000", 2024-09-14T23:58:58.301 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.301 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.302 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:58.302 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.236123+0000", 2024-09-14T23:58:58.302 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.302 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.302 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:58.302 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.236125+0000", 2024-09-14T23:58:58.302 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.313e-06 2024-09-14T23:58:58.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.302 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.303 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:58.303 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237428+0000", 2024-09-14T23:58:58.303 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013029890000000001 2024-09-14T23:58:58.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.303 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.303 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:58.303 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237452+0000", 2024-09-14T23:58:58.303 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4090000000000001e-05 2024-09-14T23:58:58.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.303 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.303 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:58.303 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237462+0000", 2024-09-14T23:58:58.304 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0421999999999999e-05 2024-09-14T23:58:58.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.304 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.304 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:58.304 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237559+0000", 2024-09-14T23:58:58.304 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6378000000000005e-05 2024-09-14T23:58:58.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.304 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.304 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:58.304 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237583+0000", 2024-09-14T23:58:58.304 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4182e-05 2024-09-14T23:58:58.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.305 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.305 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:58.305 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.242205+0000", 2024-09-14T23:58:58.305 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0046217289999999998 2024-09-14T23:58:58.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.305 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.305 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:58.305 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.242251+0000", 2024-09-14T23:58:58.305 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6218999999999999e-05 2024-09-14T23:58:58.305 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.305 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:58.305 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.306 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.306 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:56108 3.0 e56/23 3:00c47890:::rbd_data.1118c4b3aebe.0000000000000009:head v 56'127, mlcod=52'123)", 2024-09-14T23:58:58.306 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.239693+0000", 2024-09-14T23:58:58.306 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.057836194, 2024-09-14T23:58:58.306 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0043496209999999997, 2024-09-14T23:58:58.306 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:58.306 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:58.306 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:58.306 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:58.306 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.306 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:58.306 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239693+0000", 2024-09-14T23:58:58.306 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.307 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.307 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:58.307 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239693+0000", 2024-09-14T23:58:58.307 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.307 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.307 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.307 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:58.307 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239694+0000", 2024-09-14T23:58:58.307 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0499999999999999e-06 2024-09-14T23:58:58.307 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.307 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.307 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:58.307 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239702+0000", 2024-09-14T23:58:58.308 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1170000000000004e-06 2024-09-14T23:58:58.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.308 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.308 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:58.308 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239716+0000", 2024-09-14T23:58:58.308 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4306e-05 2024-09-14T23:58:58.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.308 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.308 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:58.308 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239724+0000", 2024-09-14T23:58:58.308 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0399999999999993e-06 2024-09-14T23:58:58.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.308 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.309 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:58.309 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239778+0000", 2024-09-14T23:58:58.309 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4032e-05 2024-09-14T23:58:58.309 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.309 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.309 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:58.309 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239801+0000", 2024-09-14T23:58:58.309 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2770000000000001e-05 2024-09-14T23:58:58.309 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.309 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.309 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:58.309 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.243990+0000", 2024-09-14T23:58:58.309 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0041890909999999998 2024-09-14T23:58:58.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.310 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:58.310 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.244043+0000", 2024-09-14T23:58:58.310 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2215000000000001e-05 2024-09-14T23:58:58.310 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.310 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:58.310 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.310 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:56385 3.9 e56/23 3:963d31c7:::rbd_data.1118c4b3aebe.0000000000000003:head v 56'87, mlcod=53'85)", 2024-09-14T23:58:58.310 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.283936+0000", 2024-09-14T23:58:58.310 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.013593284, 2024-09-14T23:58:58.310 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0063042080000000004, 2024-09-14T23:58:58.310 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:58.311 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:58.311 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:58.311 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:58.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.311 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:58.311 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.283936+0000", 2024-09-14T23:58:58.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.311 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:58.311 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.283936+0000", 2024-09-14T23:58:58.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.312 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.312 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:58.312 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.283937+0000", 2024-09-14T23:58:58.312 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1179999999999999e-06 2024-09-14T23:58:58.312 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.312 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.312 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:58.312 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286338+0000", 2024-09-14T23:58:58.312 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024010149999999998 2024-09-14T23:58:58.312 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.312 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.312 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:58.312 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286360+0000", 2024-09-14T23:58:58.313 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2212e-05 2024-09-14T23:58:58.313 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.313 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.313 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:58.313 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286370+0000", 2024-09-14T23:58:58.313 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5850000000000006e-06 2024-09-14T23:58:58.313 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.313 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.313 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:58.313 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286443+0000", 2024-09-14T23:58:58.313 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2757999999999997e-05 2024-09-14T23:58:58.313 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.313 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.314 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:58.314 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286466+0000", 2024-09-14T23:58:58.314 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3844000000000001e-05 2024-09-14T23:58:58.314 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.314 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.314 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:58.314 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.290197+0000", 2024-09-14T23:58:58.314 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003730176 2024-09-14T23:58:58.314 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.314 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.314 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:58.314 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.290240+0000", 2024-09-14T23:58:58.314 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.35e-05 2024-09-14T23:58:58.314 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.315 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:58.315 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.315 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:56386 3.9 e56/23 3:963d31c7:::rbd_data.1118c4b3aebe.0000000000000003:head v 56'89, mlcod=53'85)", 2024-09-14T23:58:58.315 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.287578+0000", 2024-09-14T23:58:58.315 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 112.009951558, 2024-09-14T23:58:58.315 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033580250000000002, 2024-09-14T23:58:58.315 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:58.315 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:58.315 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:58.315 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:58.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.316 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:58.316 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287578+0000", 2024-09-14T23:58:58.316 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.316 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.316 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:58.316 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287578+0000", 2024-09-14T23:58:58.316 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.317 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.317 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:58.317 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287579+0000", 2024-09-14T23:58:58.317 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.64e-07 2024-09-14T23:58:58.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.317 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.317 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:58.317 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287586+0000", 2024-09-14T23:58:58.317 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5310000000000004e-06 2024-09-14T23:58:58.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.317 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.317 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:58.318 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287597+0000", 2024-09-14T23:58:58.318 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0903999999999999e-05 2024-09-14T23:58:58.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.318 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.318 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:58.318 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287605+0000", 2024-09-14T23:58:58.318 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7889999999999995e-06 2024-09-14T23:58:58.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.318 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.318 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:58.318 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.288437+0000", 2024-09-14T23:58:58.318 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000831914 2024-09-14T23:58:58.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.319 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.319 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:58.319 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.288460+0000", 2024-09-14T23:58:58.319 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3410000000000001e-05 2024-09-14T23:58:58.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.319 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.319 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:58.320 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.290883+0000", 2024-09-14T23:58:58.320 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024230599999999999 2024-09-14T23:58:58.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.320 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.320 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:58.320 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.290936+0000", 2024-09-14T23:58:58.320 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2453000000000001e-05 2024-09-14T23:58:58.320 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.320 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:58.320 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.320 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.321 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:64263 3.4 e57/23 3:20c6a768:::rbd_data.11188d356229.0000000000000001:head v 57'165, mlcod=57'163)", 2024-09-14T23:58:58.321 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:07.999031+0000", 2024-09-14T23:58:58.321 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 110.29849840999999, 2024-09-14T23:58:58.321 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023815210000000002, 2024-09-14T23:58:58.321 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:58.321 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:58.321 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:58.321 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:58.321 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.321 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:58.322 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999031+0000", 2024-09-14T23:58:58.322 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.322 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.322 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:58.322 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999031+0000", 2024-09-14T23:58:58.322 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.322 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.322 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:58.322 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999032+0000", 2024-09-14T23:58:58.323 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.176e-06 2024-09-14T23:58:58.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.323 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.323 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:58.323 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999660+0000", 2024-09-14T23:58:58.323 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00062817099999999996 2024-09-14T23:58:58.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.323 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.323 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:58.323 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999683+0000", 2024-09-14T23:58:58.323 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2816999999999999e-05 2024-09-14T23:58:58.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.324 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:58.324 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999693+0000", 2024-09-14T23:58:58.324 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0088999999999999e-05 2024-09-14T23:58:58.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.324 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:58.324 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999767+0000", 2024-09-14T23:58:58.324 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4056999999999999e-05 2024-09-14T23:58:58.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.325 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:58.325 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999793+0000", 2024-09-14T23:58:58.325 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5573000000000001e-05 2024-09-14T23:58:58.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.325 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:58.325 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:08.001379+0000", 2024-09-14T23:58:58.325 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015859019999999999 2024-09-14T23:58:58.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.326 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:58.326 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:08.001412+0000", 2024-09-14T23:58:58.326 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3735999999999997e-05 2024-09-14T23:58:58.326 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.326 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:58.326 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.326 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.326 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:66570 3.0 e60/23 3:055350b3:::rbd_data.111873792d9b.0000000000000005:head v 60'162, mlcod=57'159)", 2024-09-14T23:58:58.326 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:11.277517+0000", 2024-09-14T23:58:58.326 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 107.02001225399999, 2024-09-14T23:58:58.327 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003472199, 2024-09-14T23:58:58.327 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:58.327 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:58.327 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:58.327 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:58.327 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.327 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:58.327 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.277517+0000", 2024-09-14T23:58:58.327 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.328 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.328 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.328 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:58.328 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.277517+0000", 2024-09-14T23:58:58.328 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.328 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.328 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.328 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:58.328 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.277518+0000", 2024-09-14T23:58:58.328 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1379999999999999e-06 2024-09-14T23:58:58.328 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.328 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.329 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:58.329 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278504+0000", 2024-09-14T23:58:58.329 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00098617499999999994 2024-09-14T23:58:58.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.329 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.329 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:58.329 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278525+0000", 2024-09-14T23:58:58.329 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.09e-05 2024-09-14T23:58:58.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.329 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.329 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:58.330 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278534+0000", 2024-09-14T23:58:58.330 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6209999999999993e-06 2024-09-14T23:58:58.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.330 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.330 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:58.330 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278611+0000", 2024-09-14T23:58:58.330 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6926000000000002e-05 2024-09-14T23:58:58.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.330 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.330 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:58.330 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278641+0000", 2024-09-14T23:58:58.331 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0861999999999997e-05 2024-09-14T23:58:58.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.331 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.331 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:58.331 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.280953+0000", 2024-09-14T23:58:58.331 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023114239999999999 2024-09-14T23:58:58.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.331 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.331 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:58.331 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.280989+0000", 2024-09-14T23:58:58.331 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6152999999999998e-05 2024-09-14T23:58:58.331 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.332 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:58.332 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.332 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.332 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.332 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:158983 3.4 e103/23 3:20e43b29:::rbd_data.1118b2f495e6.0000000000000012:head v 103'454, mlcod=103'452)", 2024-09-14T23:58:58.332 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:58.917699+0000", 2024-09-14T23:58:58.332 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 59.379830204000001, 2024-09-14T23:58:58.332 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037933799999999998, 2024-09-14T23:58:58.332 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:58.332 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:58.332 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:58.332 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:58.333 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.333 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:58.333 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917699+0000", 2024-09-14T23:58:58.333 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.333 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.333 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:58.333 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917699+0000", 2024-09-14T23:58:58.333 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.333 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.334 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:58.334 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917700+0000", 2024-09-14T23:58:58.334 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3850000000000001e-06 2024-09-14T23:58:58.334 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.334 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.334 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:58.334 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917939+0000", 2024-09-14T23:58:58.334 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000238249 2024-09-14T23:58:58.334 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.334 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.334 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:58.335 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917961+0000", 2024-09-14T23:58:58.335 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2894000000000001e-05 2024-09-14T23:58:58.335 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.335 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.335 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:58.335 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917972+0000", 2024-09-14T23:58:58.335 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0261000000000001e-05 2024-09-14T23:58:58.335 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.335 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.335 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:58.335 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.918038+0000", 2024-09-14T23:58:58.335 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6223000000000006e-05 2024-09-14T23:58:58.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.336 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:58.336 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.918063+0000", 2024-09-14T23:58:58.336 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5252000000000001e-05 2024-09-14T23:58:58.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.336 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:58.336 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.921433+0000", 2024-09-14T23:58:58.336 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033699960000000001 2024-09-14T23:58:58.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.337 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:58.337 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.921492+0000", 2024-09-14T23:58:58.337 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9120000000000003e-05 2024-09-14T23:58:58.337 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.337 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:58.337 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.337 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.337 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(osd.1.0:1460 3.0 e122/23 3:00cc1e32:::rbd_data.11187b9f1032.0000000000000001:2f v 122'453, mlcod=118'449)", 2024-09-14T23:58:58.337 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:22.287423+0000", 2024-09-14T23:58:58.337 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 36.010106546999999, 2024-09-14T23:58:58.337 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0046041659999999998, 2024-09-14T23:58:58.337 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:58.337 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:58.338 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:58.338 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:58.338 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.338 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:58.338 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.287423+0000", 2024-09-14T23:58:58.338 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.338 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.338 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.338 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:58.338 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.287423+0000", 2024-09-14T23:58:58.338 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.338 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.338 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.338 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:58.339 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.287423+0000", 2024-09-14T23:58:58.339 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7500000000000001e-07 2024-09-14T23:58:58.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.339 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.339 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:58.339 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.287435+0000", 2024-09-14T23:58:58.339 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2006e-05 2024-09-14T23:58:58.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.339 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.339 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:58.340 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.287444+0000", 2024-09-14T23:58:58.340 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.465e-06 2024-09-14T23:58:58.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.340 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.340 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:58.340 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.287453+0000", 2024-09-14T23:58:58.340 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0100000000000001e-06 2024-09-14T23:58:58.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.340 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.340 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:58.340 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.287519+0000", 2024-09-14T23:58:58.340 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6149999999999995e-05 2024-09-14T23:58:58.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.341 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.341 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:58.341 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.287543+0000", 2024-09-14T23:58:58.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3682000000000001e-05 2024-09-14T23:58:58.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.341 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.341 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:58.341 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.291977+0000", 2024-09-14T23:58:58.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0044333829999999999 2024-09-14T23:58:58.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.341 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.341 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:58.341 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.292027+0000", 2024-09-14T23:58:58.342 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0195000000000003e-05 2024-09-14T23:58:58.342 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.342 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:58.342 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.342 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.342 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDPGPush(3.1e 126/125 [PushOp(3:7f7265c3:::rbd_data.11188d356229.000000000000000d:16, version: 58'124, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:7f7265c3:::rbd_data.11188d356229.000000000000000d:16@58'124, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 16=[]:{16: [16]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T23:58:58.342 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:26.552687+0000", 2024-09-14T23:58:58.342 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 31.744842283000001, 2024-09-14T23:58:58.342 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023212060000000001, 2024-09-14T23:58:58.342 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:58.342 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:58.343 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:58.343 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:58.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.343 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:58.343 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.552687+0000", 2024-09-14T23:58:58.343 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.343 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:58.343 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.552687+0000", 2024-09-14T23:58:58.343 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.344 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:58.344 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.552688+0000", 2024-09-14T23:58:58.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.47e-07 2024-09-14T23:58:58.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.344 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:58.344 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.553659+0000", 2024-09-14T23:58:58.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00097088899999999995 2024-09-14T23:58:58.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.344 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:58.344 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.553682+0000", 2024-09-14T23:58:58.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3862e-05 2024-09-14T23:58:58.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:58.345 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.553703+0000", 2024-09-14T23:58:58.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0310999999999999e-05 2024-09-14T23:58:58.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:58.345 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.553813+0000", 2024-09-14T23:58:58.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011051 2024-09-14T23:58:58.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:58.345 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.553828+0000", 2024-09-14T23:58:58.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4625e-05 2024-09-14T23:58:58.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.346 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:58.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.555008+0000", 2024-09-14T23:58:58.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001180162 2024-09-14T23:58:58.346 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.346 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:58.346 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.346 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(osd.1.0:1465 3.4 e127/23 3:265f7763:::rbd_data.1118f9c96a7.0000000000000004:2c v 127'508, mlcod=120'505)", 2024-09-14T23:58:58.346 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:27.318056+0000", 2024-09-14T23:58:58.346 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 30.979472860000001, 2024-09-14T23:58:58.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0049337449999999998, 2024-09-14T23:58:58.347 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:58.347 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:58.347 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:58.347 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:58.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.347 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:58.347 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.318056+0000", 2024-09-14T23:58:58.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.348 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:58.348 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.318056+0000", 2024-09-14T23:58:58.348 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.348 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:58.348 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.318057+0000", 2024-09-14T23:58:58.349 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9799999999999999e-07 2024-09-14T23:58:58.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.349 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:58.349 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.318061+0000", 2024-09-14T23:58:58.349 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7899999999999999e-06 2024-09-14T23:58:58.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.349 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:58.349 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.318074+0000", 2024-09-14T23:58:58.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2537999999999999e-05 2024-09-14T23:58:58.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.350 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:58.350 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.318083+0000", 2024-09-14T23:58:58.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3829999999999998e-06 2024-09-14T23:58:58.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.350 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:58.350 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.318165+0000", 2024-09-14T23:58:58.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1928999999999994e-05 2024-09-14T23:58:58.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.351 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:58.351 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.318199+0000", 2024-09-14T23:58:58.351 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3587000000000002e-05 2024-09-14T23:58:58.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.351 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:58.351 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.322939+0000", 2024-09-14T23:58:58.351 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0047397730000000001 2024-09-14T23:58:58.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.352 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:58.352 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.322990+0000", 2024-09-14T23:58:58.352 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1347000000000002e-05 2024-09-14T23:58:58.352 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.352 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:58.352 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.352 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.352 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:252476 3.e 3:725cbf00:::rbd_header.11186acd4a72:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e132)", 2024-09-14T23:58:58.352 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:32.121781+0000", 2024-09-14T23:58:58.352 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 26.175748023000001, 2024-09-14T23:58:58.353 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022671459999999998, 2024-09-14T23:58:58.353 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:58.353 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:58.353 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:58.353 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:58.353 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:58.353 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:58.353 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 252476 2024-09-14T23:58:58.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.353 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:58.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.354 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:58.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:32.121781+0000", 2024-09-14T23:58:58.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.354 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:58.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:32.121781+0000", 2024-09-14T23:58:58.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.354 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:58.355 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:32.121783+0000", 2024-09-14T23:58:58.355 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.531e-06 2024-09-14T23:58:58.355 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.355 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.355 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:58.355 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:32.121790+0000", 2024-09-14T23:58:58.355 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7670000000000003e-06 2024-09-14T23:58:58.355 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.355 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.355 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:58.355 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:32.121806+0000", 2024-09-14T23:58:58.355 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5291999999999999e-05 2024-09-14T23:58:58.356 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.356 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.356 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:58.356 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:32.121813+0000", 2024-09-14T23:58:58.356 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.418e-06 2024-09-14T23:58:58.356 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.356 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.356 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:58.356 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:32.121879+0000", 2024-09-14T23:58:58.356 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5811999999999999e-05 2024-09-14T23:58:58.356 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.357 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.357 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op must wait for map", 2024-09-14T23:58:58.357 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:32.121888+0000", 2024-09-14T23:58:58.357 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6600000000000001e-06 2024-09-14T23:58:58.357 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.357 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.357 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:58.357 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:32.123710+0000", 2024-09-14T23:58:58.357 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001822838 2024-09-14T23:58:58.357 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.358 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:58.358 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:32.123803+0000", 2024-09-14T23:58:58.358 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3077000000000003e-05 2024-09-14T23:58:58.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.358 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:58.358 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:32.124048+0000", 2024-09-14T23:58:58.358 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024475099999999997 2024-09-14T23:58:58.358 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.358 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:58.359 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.359 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.359 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:253435 3.5 3:a6fcd3e0:::rbd_data.1118e50f549.000000000000004a:head [delete] snapc 24=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-14T23:58:58.359 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:36.070548+0000", 2024-09-14T23:58:58.359 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 22.226981541000001, 2024-09-14T23:58:58.359 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022804000000000001, 2024-09-14T23:58:58.359 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:58.359 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:58.359 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:58.359 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:58.359 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:58.359 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:58.360 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 253435 2024-09-14T23:58:58.360 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.360 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:58.360 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.360 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:58.360 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.070548+0000", 2024-09-14T23:58:58.360 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.360 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.360 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.360 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:58.360 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.070548+0000", 2024-09-14T23:58:58.361 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.361 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.361 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.361 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:58.361 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.070549+0000", 2024-09-14T23:58:58.361 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6500000000000001e-06 2024-09-14T23:58:58.361 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.361 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.361 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:58.361 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.070555+0000", 2024-09-14T23:58:58.361 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1750000000000002e-06 2024-09-14T23:58:58.361 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.362 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.362 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:58.362 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.070571+0000", 2024-09-14T23:58:58.362 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5719999999999999e-05 2024-09-14T23:58:58.362 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.362 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.362 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:58.362 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.070580+0000", 2024-09-14T23:58:58.362 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0699999999999996e-06 2024-09-14T23:58:58.362 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.362 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.363 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:58.363 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.070658+0000", 2024-09-14T23:58:58.363 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7525000000000001e-05 2024-09-14T23:58:58.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.363 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.363 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:58.363 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.070799+0000", 2024-09-14T23:58:58.363 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014083499999999999 2024-09-14T23:58:58.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.363 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.363 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-14T23:58:58.363 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.070927+0000", 2024-09-14T23:58:58.364 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012853100000000001 2024-09-14T23:58:58.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.364 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.364 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:58:58.364 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.071666+0000", 2024-09-14T23:58:58.364 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00073853000000000005 2024-09-14T23:58:58.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.364 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.364 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:58.364 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.072750+0000", 2024-09-14T23:58:58.364 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001084738 2024-09-14T23:58:58.365 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.365 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.365 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:58.365 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.072791+0000", 2024-09-14T23:58:58.365 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0330999999999997e-05 2024-09-14T23:58:58.365 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.365 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.365 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:58.365 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.072828+0000", 2024-09-14T23:58:58.365 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7295000000000001e-05 2024-09-14T23:58:58.365 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.366 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:58.366 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.366 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.366 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.366 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:253690 3.e 3:725cbf00:::rbd_header.11186acd4a72:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-14T23:58:58.366 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:37.091642+0000", 2024-09-14T23:58:58.366 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 21.205887242999999, 2024-09-14T23:58:58.366 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0026525030000000001, 2024-09-14T23:58:58.366 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:58.366 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:58.366 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:58.366 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:58.366 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:58.366 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:58.367 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 253690 2024-09-14T23:58:58.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.367 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:58.367 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.367 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:58.367 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:37.091642+0000", 2024-09-14T23:58:58.367 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.367 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.367 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:58.367 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:37.091642+0000", 2024-09-14T23:58:58.367 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.367 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.368 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:58.368 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:37.091644+0000", 2024-09-14T23:58:58.368 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9829999999999999e-06 2024-09-14T23:58:58.368 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.368 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.368 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:58.368 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:37.091654+0000", 2024-09-14T23:58:58.368 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8619999999999998e-06 2024-09-14T23:58:58.368 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.368 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.368 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:58.368 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:37.091673+0000", 2024-09-14T23:58:58.368 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9094e-05 2024-09-14T23:58:58.369 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.369 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.369 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:58.369 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:37.091681+0000", 2024-09-14T23:58:58.369 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9449999999999993e-06 2024-09-14T23:58:58.369 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.369 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.369 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:58.369 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:37.091752+0000", 2024-09-14T23:58:58.369 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1468000000000001e-05 2024-09-14T23:58:58.369 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.369 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.370 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op must wait for map", 2024-09-14T23:58:58.370 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:37.091764+0000", 2024-09-14T23:58:58.370 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2067e-05 2024-09-14T23:58:58.370 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.370 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.370 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:58.370 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:37.093986+0000", 2024-09-14T23:58:58.370 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022213720000000001 2024-09-14T23:58:58.370 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.370 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.370 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:58.370 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:37.094082+0000", 2024-09-14T23:58:58.371 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5872999999999995e-05 2024-09-14T23:58:58.371 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.371 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.371 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:58.371 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:37.094294+0000", 2024-09-14T23:58:58.371 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021283900000000001 2024-09-14T23:58:58.371 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.371 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:58.371 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.371 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.371 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.371 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:254188 3.5 3:a13778a5:::rbd_header.1118ce574610:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e140)", 2024-09-14T23:58:58.371 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:40.008804+0000", 2024-09-14T23:58:58.372 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.288724946999999, 2024-09-14T23:58:58.372 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0044649980000000004, 2024-09-14T23:58:58.372 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:58.372 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:58.372 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:58.372 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:58.372 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:58.372 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:58.372 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254188 2024-09-14T23:58:58.372 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.372 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:58.372 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.372 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:58.373 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.008804+0000", 2024-09-14T23:58:58.373 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.373 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.373 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.373 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:58.373 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.008804+0000", 2024-09-14T23:58:58.373 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.373 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.373 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.373 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:58.373 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.008806+0000", 2024-09-14T23:58:58.373 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.824e-06 2024-09-14T23:58:58.373 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.373 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.374 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:58.374 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.008813+0000", 2024-09-14T23:58:58.374 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8889999999999996e-06 2024-09-14T23:58:58.374 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.374 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.374 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:58.374 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.008826+0000", 2024-09-14T23:58:58.374 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.29e-05 2024-09-14T23:58:58.374 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.374 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.374 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:58.374 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.008836+0000", 2024-09-14T23:58:58.375 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9660000000000008e-06 2024-09-14T23:58:58.375 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.375 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.375 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:58.375 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.008906+0000", 2024-09-14T23:58:58.375 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0165000000000003e-05 2024-09-14T23:58:58.375 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.375 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.375 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:58.375 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.009004+0000", 2024-09-14T23:58:58.375 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8412999999999994e-05 2024-09-14T23:58:58.375 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.375 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.375 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-14T23:58:58.376 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.009349+0000", 2024-09-14T23:58:58.376 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00034510200000000002 2024-09-14T23:58:58.376 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.376 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.376 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:58:58.376 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.010086+0000", 2024-09-14T23:58:58.376 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00073696000000000005 2024-09-14T23:58:58.376 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.376 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.376 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:58.376 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.013157+0000", 2024-09-14T23:58:58.376 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0030706470000000001 2024-09-14T23:58:58.376 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.377 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.377 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:58.377 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.013222+0000", 2024-09-14T23:58:58.377 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4479000000000004e-05 2024-09-14T23:58:58.377 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.377 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.377 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:58.377 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.013269+0000", 2024-09-14T23:58:58.377 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7652999999999999e-05 2024-09-14T23:58:58.377 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.377 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:58.377 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.377 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.377 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.378 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:254194 3.5 3:a13778a5:::rbd_header.1118ce574610:head [watch unwatch cookie 140670715072400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e140)", 2024-09-14T23:58:58.378 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:40.018920+0000", 2024-09-14T23:58:58.378 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 18.278609571, 2024-09-14T23:58:58.378 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023612640000000001, 2024-09-14T23:58:58.378 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:58.378 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:58.378 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:58.378 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:58.378 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:58.378 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:58.378 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254194 2024-09-14T23:58:58.378 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.378 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:58.378 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.379 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:58.379 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.018920+0000", 2024-09-14T23:58:58.379 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.379 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.379 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.379 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:58.379 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.018920+0000", 2024-09-14T23:58:58.379 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.379 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.379 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.379 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:58.379 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.018920+0000", 2024-09-14T23:58:58.380 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7700000000000004e-07 2024-09-14T23:58:58.380 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.380 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.380 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:58.380 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.018923+0000", 2024-09-14T23:58:58.380 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9840000000000001e-06 2024-09-14T23:58:58.380 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.380 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.380 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:58.380 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.018931+0000", 2024-09-14T23:58:58.380 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8089999999999993e-06 2024-09-14T23:58:58.381 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.381 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.381 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:58.381 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.018937+0000", 2024-09-14T23:58:58.381 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1770000000000003e-06 2024-09-14T23:58:58.381 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.381 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.381 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:58.381 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.019163+0000", 2024-09-14T23:58:58.381 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022563800000000001 2024-09-14T23:58:58.381 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.381 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.382 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:58.382 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.019233+0000", 2024-09-14T23:58:58.382 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9698999999999998e-05 2024-09-14T23:58:58.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.382 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.382 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-14T23:58:58.382 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.019383+0000", 2024-09-14T23:58:58.382 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015073800000000001 2024-09-14T23:58:58.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.382 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.382 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:58:58.382 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.020341+0000", 2024-09-14T23:58:58.383 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00095732900000000004 2024-09-14T23:58:58.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.383 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.383 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:58.383 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.021141+0000", 2024-09-14T23:58:58.383 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00080075600000000002 2024-09-14T23:58:58.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.383 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.383 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:58.383 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.021189+0000", 2024-09-14T23:58:58.383 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7318999999999999e-05 2024-09-14T23:58:58.384 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.384 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.384 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:58.384 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.021281+0000", 2024-09-14T23:58:58.384 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2038000000000002e-05 2024-09-14T23:58:58.384 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.384 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:58.384 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.384 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.384 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.384 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(osd.4.0:4196 3.13 e141/113 3:cecefa16:::rbd_data.1118c250b6de.0000000000000001:1f v 141'878, mlcod=138'874)", 2024-09-14T23:58:58.385 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:41.015011+0000", 2024-09-14T23:58:58.385 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 17.282518624000001, 2024-09-14T23:58:58.385 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025205499999999999, 2024-09-14T23:58:58.385 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:58.385 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:58.385 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:58.385 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:58.385 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.385 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:58.385 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:41.015011+0000", 2024-09-14T23:58:58.385 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.386 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.386 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:58.386 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:41.015011+0000", 2024-09-14T23:58:58.386 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.386 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.386 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:58.386 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:41.015011+0000", 2024-09-14T23:58:58.386 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9900000000000002e-07 2024-09-14T23:58:58.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.387 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.387 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:58.387 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:41.015018+0000", 2024-09-14T23:58:58.387 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2910000000000002e-06 2024-09-14T23:58:58.387 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.387 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.387 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:58.387 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:41.015033+0000", 2024-09-14T23:58:58.387 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4625e-05 2024-09-14T23:58:58.387 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.387 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.388 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:58.388 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:41.015043+0000", 2024-09-14T23:58:58.388 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0584999999999999e-05 2024-09-14T23:58:58.388 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.388 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.388 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:58.388 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:41.015114+0000", 2024-09-14T23:58:58.388 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0541000000000001e-05 2024-09-14T23:58:58.388 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.389 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.389 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:58.389 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:41.015138+0000", 2024-09-14T23:58:58.389 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3810000000000001e-05 2024-09-14T23:58:58.389 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.389 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.389 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:58.389 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:41.017481+0000", 2024-09-14T23:58:58.389 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023433619999999999 2024-09-14T23:58:58.389 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.389 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.390 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:58.390 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:41.017531+0000", 2024-09-14T23:58:58.390 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0037000000000002e-05 2024-09-14T23:58:58.390 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.390 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:58.390 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.390 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.390 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.390 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(osd.4.0:4316 3.9 e146/23 3:91eaed90:::rbd_data.11182ee2885e.000000000000000c:1a v 146'308, mlcod=142'303)", 2024-09-14T23:58:58.390 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:46.046915+0000", 2024-09-14T23:58:58.390 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 12.250614148, 2024-09-14T23:58:58.391 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022583619999999999, 2024-09-14T23:58:58.391 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:58.391 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:58.391 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:58.391 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:58.391 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.391 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:58.391 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.046915+0000", 2024-09-14T23:58:58.391 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.392 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.392 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:58.392 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.046915+0000", 2024-09-14T23:58:58.392 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.392 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.392 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.392 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:58.392 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.046916+0000", 2024-09-14T23:58:58.392 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6300000000000004e-07 2024-09-14T23:58:58.392 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.393 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.393 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:58.393 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.046922+0000", 2024-09-14T23:58:58.393 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2339999999999999e-06 2024-09-14T23:58:58.393 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.393 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.393 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:58.393 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.046933+0000", 2024-09-14T23:58:58.393 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0795000000000001e-05 2024-09-14T23:58:58.393 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.393 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.394 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:58.394 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.046950+0000", 2024-09-14T23:58:58.394 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7149000000000001e-05 2024-09-14T23:58:58.394 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.394 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.394 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:58.394 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.047000+0000", 2024-09-14T23:58:58.394 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9654000000000003e-05 2024-09-14T23:58:58.394 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.394 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.394 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:58.395 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.047020+0000", 2024-09-14T23:58:58.395 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0199000000000001e-05 2024-09-14T23:58:58.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.395 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:58.395 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.049110+0000", 2024-09-14T23:58:58.395 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0020902970000000001 2024-09-14T23:58:58.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.395 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:58.395 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.049173+0000", 2024-09-14T23:58:58.395 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.3270999999999998e-05 2024-09-14T23:58:58.396 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.396 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:58.396 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.396 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.396 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(osd.6.0:1718 3.1e e146/127 3:7a74d0d0:::rbd_data.11182ee2885e.0000000000000006:1a v 146'435, mlcod=145'432)", 2024-09-14T23:58:58.396 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:46.047607+0000", 2024-09-14T23:58:58.396 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 12.249922425999999, 2024-09-14T23:58:58.396 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0026434779999999999, 2024-09-14T23:58:58.396 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:58.396 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:58.397 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:58.397 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:58.397 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.397 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:58.397 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.047607+0000", 2024-09-14T23:58:58.397 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.397 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.397 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.397 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:58.397 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.047607+0000", 2024-09-14T23:58:58.397 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.398 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.398 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.398 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:58.398 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.047607+0000", 2024-09-14T23:58:58.398 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4700000000000002e-07 2024-09-14T23:58:58.398 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.398 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.398 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:58.398 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.047612+0000", 2024-09-14T23:58:58.398 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1959999999999999e-06 2024-09-14T23:58:58.398 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.398 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.398 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:58.399 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.047622+0000", 2024-09-14T23:58:58.399 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0169999999999999e-05 2024-09-14T23:58:58.399 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.399 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.399 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:58.399 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.047630+0000", 2024-09-14T23:58:58.399 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7249999999999997e-06 2024-09-14T23:58:58.399 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.399 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.399 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:58.400 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.047739+0000", 2024-09-14T23:58:58.400 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108625 2024-09-14T23:58:58.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.400 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:58.400 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.047756+0000", 2024-09-14T23:58:58.400 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6866000000000001e-05 2024-09-14T23:58:58.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.400 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:58.400 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.050197+0000", 2024-09-14T23:58:58.400 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024412689999999998 2024-09-14T23:58:58.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.401 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:58.401 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.050250+0000", 2024-09-14T23:58:58.401 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3279999999999998e-05 2024-09-14T23:58:58.401 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.401 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:58.401 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.401 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.401 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.401 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:255323 3.e 3:74b99397:::rbd_data.11182ee2885e.0000000000000004:head [delete] snapc 1a=[] ondisk+write+known_if_redirected+supports_pool_eio e146)", 2024-09-14T23:58:58.401 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:46.111665+0000", 2024-09-14T23:58:58.401 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 12.185864131000001, 2024-09-14T23:58:58.401 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038303019999999998, 2024-09-14T23:58:58.401 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:58.401 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:58.402 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:58.402 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:58.402 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:58.402 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:58.402 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 255323 2024-09-14T23:58:58.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.402 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:58.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.402 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:58.402 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.111665+0000", 2024-09-14T23:58:58.402 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.403 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:58.403 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.111665+0000", 2024-09-14T23:58:58.403 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.403 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.403 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:58.403 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.111667+0000", 2024-09-14T23:58:58.403 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3e-06 2024-09-14T23:58:58.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.403 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.403 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:58.403 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.111676+0000", 2024-09-14T23:58:58.403 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5699999999999993e-06 2024-09-14T23:58:58.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.404 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:58.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.111698+0000", 2024-09-14T23:58:58.404 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2263e-05 2024-09-14T23:58:58.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.404 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:58.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.111709+0000", 2024-09-14T23:58:58.404 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1087000000000001e-05 2024-09-14T23:58:58.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.404 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:58.404 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.111787+0000", 2024-09-14T23:58:58.404 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8212000000000003e-05 2024-09-14T23:58:58.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.405 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:58.405 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.111886+0000", 2024-09-14T23:58:58.405 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8645000000000002e-05 2024-09-14T23:58:58.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.405 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-14T23:58:58.405 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.112014+0000", 2024-09-14T23:58:58.405 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012753800000000001 2024-09-14T23:58:58.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.405 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.405 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:58:58.405 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.113790+0000", 2024-09-14T23:58:58.405 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00177669 2024-09-14T23:58:58.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.406 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.406 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:58.406 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.115396+0000", 2024-09-14T23:58:58.406 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001605216 2024-09-14T23:58:58.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.406 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.406 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:58.406 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.115451+0000", 2024-09-14T23:58:58.406 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5534000000000001e-05 2024-09-14T23:58:58.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.406 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.406 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:58.406 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.115495+0000", 2024-09-14T23:58:58.406 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4246999999999998e-05 2024-09-14T23:58:58.407 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.407 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:58.407 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.407 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.407 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:255325 3.1e e146/127 3:7a74d0d0:::rbd_data.11182ee2885e.0000000000000006:head v 146'436, mlcod=145'433)", 2024-09-14T23:58:58.407 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:46.112042+0000", 2024-09-14T23:58:58.407 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 12.185487194, 2024-09-14T23:58:58.407 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023640530000000001, 2024-09-14T23:58:58.407 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:58.407 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:58.407 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:58.407 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:58.407 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.408 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:58.408 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.112042+0000", 2024-09-14T23:58:58.408 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.408 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:58.408 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.112042+0000", 2024-09-14T23:58:58.408 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.408 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.408 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:58.408 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.112043+0000", 2024-09-14T23:58:58.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1200000000000001e-06 2024-09-14T23:58:58.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:58.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.112051+0000", 2024-09-14T23:58:58.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3310000000000004e-06 2024-09-14T23:58:58.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.409 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:58.409 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.112063+0000", 2024-09-14T23:58:58.409 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2028000000000001e-05 2024-09-14T23:58:58.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.409 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.410 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:58.410 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.112071+0000", 2024-09-14T23:58:58.410 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4000000000000003e-06 2024-09-14T23:58:58.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.410 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.410 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:58.410 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.112155+0000", 2024-09-14T23:58:58.410 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4064e-05 2024-09-14T23:58:58.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.410 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.410 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:58.410 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.112181+0000", 2024-09-14T23:58:58.411 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5817999999999999e-05 2024-09-14T23:58:58.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.411 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.411 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:58.411 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.114372+0000", 2024-09-14T23:58:58.411 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0021908129999999998 2024-09-14T23:58:58.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.411 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.411 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:58.411 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.114406+0000", 2024-09-14T23:58:58.411 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4479e-05 2024-09-14T23:58:58.411 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.412 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:58.412 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.412 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.412 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:58.412 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:58.414 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T23:58:58.414 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:58.412+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:58.514 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:58.513+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:58.562 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:58.562 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:58.562 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:58:58.562 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:58.571 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T23:58:58.615 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:58.614+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:58.715 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:58.714+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:58.718 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:58.718 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:58:58.718 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:58:58.718 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:58:58.718 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:58.727 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T23:58:58.816 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:58.814+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:58.875 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:58:58.875 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:58:58.875 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:58:58.876 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:58:58.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.876 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5201 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2232320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:58.876 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:58:58.876 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 139.61920335400001, 2024-09-14T23:58:58.876 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057643228999999997, 2024-09-14T23:58:58.876 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:58.876 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:58.876 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:58.876 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:58.876 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:58.876 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:58.876 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5201 2024-09-14T23:58:58.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.877 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:58.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:58.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:58:58.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.877 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:58.877 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:58:58.877 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.877 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:58.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:58:58.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0699999999999998e-07 2024-09-14T23:58:58.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:58.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253422+0000", 2024-09-14T23:58:58.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.514e-06 2024-09-14T23:58:58.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.878 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:58.878 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253431+0000", 2024-09-14T23:58:58.878 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1919999999999994e-06 2024-09-14T23:58:58.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:58.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253436+0000", 2024-09-14T23:58:58.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5350000000000004e-06 2024-09-14T23:58:58.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:58.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310856+0000", 2024-09-14T23:58:58.879 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057419381999999998 2024-09-14T23:58:58.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.879 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:58.879 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310953+0000", 2024-09-14T23:58:58.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6774000000000006e-05 2024-09-14T23:58:58.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.880 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:58.880 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311062+0000", 2024-09-14T23:58:58.880 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000109425 2024-09-14T23:58:58.880 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.880 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:58.880 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.880 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5202 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2293760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:58.880 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:58:58.881 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 139.61917860599999, 2024-09-14T23:58:58.881 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057818255999999998, 2024-09-14T23:58:58.881 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:58.881 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:58.881 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:58.881 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:58.881 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:58.881 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:58.881 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5202 2024-09-14T23:58:58.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.881 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:58.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.882 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:58.882 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:58:58.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.882 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:58.882 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:58:58.882 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.883 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:58.883 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253573+0000", 2024-09-14T23:58:58.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012934000000000001 2024-09-14T23:58:58.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.883 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:58.883 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253576+0000", 2024-09-14T23:58:58.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5310000000000002e-06 2024-09-14T23:58:58.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.883 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:58.883 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253587+0000", 2024-09-14T23:58:58.883 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0522e-05 2024-09-14T23:58:58.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:58.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253595+0000", 2024-09-14T23:58:58.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8290000000000008e-06 2024-09-14T23:58:58.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:58.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311093+0000", 2024-09-14T23:58:58.884 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057498312000000003 2024-09-14T23:58:58.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.884 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:58.884 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311167+0000", 2024-09-14T23:58:58.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3582000000000001e-05 2024-09-14T23:58:58.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.885 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.885 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:58.885 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311262+0000", 2024-09-14T23:58:58.885 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5140000000000001e-05 2024-09-14T23:58:58.885 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.885 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:58.885 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.886 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5203 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2355200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:58.886 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:58:58.886 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 139.619015864, 2024-09-14T23:58:58.886 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057903938000000002, 2024-09-14T23:58:58.886 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:58.886 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:58.886 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:58.886 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:58.886 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:58.887 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:58.887 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5203 2024-09-14T23:58:58.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.887 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:58.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:58.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:58:58.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.887 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.887 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:58.887 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:58:58.887 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.888 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:58.888 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253902+0000", 2024-09-14T23:58:58.888 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029615400000000002 2024-09-14T23:58:58.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.888 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.888 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:58.888 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253906+0000", 2024-09-14T23:58:58.888 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9249999999999997e-06 2024-09-14T23:58:58.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:58.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253913+0000", 2024-09-14T23:58:58.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9099999999999999e-06 2024-09-14T23:58:58.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.889 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:58.889 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253917+0000", 2024-09-14T23:58:58.889 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9210000000000002e-06 2024-09-14T23:58:58.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:58.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311295+0000", 2024-09-14T23:58:58.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057378268000000003 2024-09-14T23:58:58.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:58.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311402+0000", 2024-09-14T23:58:58.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010644100000000001 2024-09-14T23:58:58.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.890 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.890 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:58.890 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311510+0000", 2024-09-14T23:58:58.890 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108319 2024-09-14T23:58:58.891 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.891 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:58.891 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.891 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5204 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2416640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:58.891 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:58:58.891 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 139.61869766699999, 2024-09-14T23:58:58.891 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057774860999999997, 2024-09-14T23:58:58.891 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:58.891 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:58.891 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:58.891 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:58.891 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:58.892 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:58.892 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5204 2024-09-14T23:58:58.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.892 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:58.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:58.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:58:58.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.892 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:58.892 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:58:58.892 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:58.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255903+0000", 2024-09-14T23:58:58.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001978807 2024-09-14T23:58:58.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.893 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.893 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:58.893 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255907+0000", 2024-09-14T23:58:58.893 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3610000000000001e-06 2024-09-14T23:58:58.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.894 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:58.894 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255914+0000", 2024-09-14T23:58:58.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.357e-06 2024-09-14T23:58:58.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.894 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:58.894 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255918+0000", 2024-09-14T23:58:58.894 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1779999999999996e-06 2024-09-14T23:58:58.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.894 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:58.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311541+0000", 2024-09-14T23:58:58.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055622697999999998 2024-09-14T23:58:58.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:58.895 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311614+0000", 2024-09-14T23:58:58.895 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3102999999999994e-05 2024-09-14T23:58:58.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.895 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:58.896 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311699+0000", 2024-09-14T23:58:58.896 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5357000000000003e-05 2024-09-14T23:58:58.896 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.896 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:58.896 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.896 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.896 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5211 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2846720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:58.896 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:58:58.896 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 139.61659945299999, 2024-09-14T23:58:58.897 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057528151, 2024-09-14T23:58:58.897 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:58.897 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:58.897 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:58.897 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:58.897 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:58.897 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:58.897 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5211 2024-09-14T23:58:58.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.897 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:58.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.898 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:58.898 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:58:58.898 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.898 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:58.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:58:58.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.900 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:58.900 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:58:58.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1399999999999998e-07 2024-09-14T23:58:58.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:58.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256024+0000", 2024-09-14T23:58:58.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.114e-06 2024-09-14T23:58:58.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.901 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:58.901 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256028+0000", 2024-09-14T23:58:58.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0840000000000002e-06 2024-09-14T23:58:58.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.902 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:58.902 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256031+0000", 2024-09-14T23:58:58.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6819999999999999e-06 2024-09-14T23:58:58.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.902 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:58.902 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313317+0000", 2024-09-14T23:58:58.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057286295000000001 2024-09-14T23:58:58.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.903 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:58.903 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313403+0000", 2024-09-14T23:58:58.903 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6287999999999998e-05 2024-09-14T23:58:58.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.903 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.903 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:58.903 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313551+0000", 2024-09-14T23:58:58.903 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000147374 2024-09-14T23:58:58.903 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.903 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:58.903 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.904 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5212 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2908160~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:58.904 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:58:58.904 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 139.616588259, 2024-09-14T23:58:58.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057735679999999998, 2024-09-14T23:58:58.904 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:58.904 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:58.904 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:58.904 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:58.904 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:58.904 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:58.905 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5212 2024-09-14T23:58:58.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.905 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:58.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.905 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:58.905 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:58:58.905 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.905 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:58.905 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:58:58.906 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.906 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.906 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:58.906 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:58:58.906 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0900000000000001e-07 2024-09-14T23:58:58.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.906 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.906 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:58.906 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256035+0000", 2024-09-14T23:58:58.906 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1200000000000001e-06 2024-09-14T23:58:58.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.907 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:58.907 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256038+0000", 2024-09-14T23:58:58.907 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.05e-06 2024-09-14T23:58:58.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.907 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:58.907 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256041+0000", 2024-09-14T23:58:58.907 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.667e-06 2024-09-14T23:58:58.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.907 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:58.907 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313584+0000", 2024-09-14T23:58:58.907 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057543484999999998 2024-09-14T23:58:58.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.908 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:58.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313668+0000", 2024-09-14T23:58:58.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3807000000000006e-05 2024-09-14T23:58:58.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.908 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:58.908 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313770+0000", 2024-09-14T23:58:58.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010134200000000001 2024-09-14T23:58:58.908 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.908 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:58.908 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.909 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5213 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2969600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:58.909 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:58:58.909 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 139.616578172, 2024-09-14T23:58:58.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058145467999999999, 2024-09-14T23:58:58.909 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:58.909 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:58.909 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:58.909 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:58.909 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:58.909 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:58.909 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5213 2024-09-14T23:58:58.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.909 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:58.910 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.910 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:58.910 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:58:58.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.910 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.910 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:58.910 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:58:58.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.910 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.910 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:58.910 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257901+0000", 2024-09-14T23:58:58.911 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018567519999999999 2024-09-14T23:58:58.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.911 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.911 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:58.911 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257904+0000", 2024-09-14T23:58:58.911 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4850000000000001e-06 2024-09-14T23:58:58.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.911 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.911 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:58.911 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257912+0000", 2024-09-14T23:58:58.911 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2090000000000005e-06 2024-09-14T23:58:58.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.911 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.911 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:58.912 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257918+0000", 2024-09-14T23:58:58.912 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6899999999999997e-06 2024-09-14T23:58:58.912 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.913 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:58.913 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313816+0000", 2024-09-14T23:58:58.913 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055898075999999998 2024-09-14T23:58:58.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.913 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:58.913 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.314082+0000", 2024-09-14T23:58:58.913 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000265386 2024-09-14T23:58:58.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.913 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:58.914 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.314189+0000", 2024-09-14T23:58:58.914 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010787 2024-09-14T23:58:58.914 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.914 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:58.914 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.914 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.914 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5218 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3276800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:58.914 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:58:58.914 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 139.61465116299999, 2024-09-14T23:58:58.914 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057497557999999997, 2024-09-14T23:58:58.914 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:58.914 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:58.915 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:58.915 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:58.915 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:58.915 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:58.915 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5218 2024-09-14T23:58:58.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.915 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:58.915 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.915 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:58.915 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:58:58.915 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.915 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.915 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:58.916 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:58:58.916 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.916 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.916 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.916 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:58.916 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:58:58.916 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1899999999999999e-07 2024-09-14T23:58:58.916 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.916 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.916 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:58.916 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257972+0000", 2024-09-14T23:58:58.916 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.217e-06 2024-09-14T23:58:58.916 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.916 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.917 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:58.917 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257975+0000", 2024-09-14T23:58:58.917 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9359999999999999e-06 2024-09-14T23:58:58.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.917 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.917 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:58.917 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257978+0000", 2024-09-14T23:58:58.917 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1049999999999999e-06 2024-09-14T23:58:58.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.917 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.917 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:58.917 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315331+0000", 2024-09-14T23:58:58.917 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057352093999999999 2024-09-14T23:58:58.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.918 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.918 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:58.918 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315384+0000", 2024-09-14T23:58:58.918 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3780999999999999e-05 2024-09-14T23:58:58.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.918 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.918 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:58.918 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315468+0000", 2024-09-14T23:58:58.918 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4206e-05 2024-09-14T23:58:58.919 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.919 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:58.919 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.919 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.919 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5219 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3338240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:58.919 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:58:58.919 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 139.614640487, 2024-09-14T23:58:58.919 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057637764000000001, 2024-09-14T23:58:58.919 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:58.919 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:58.919 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:58.920 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:58.920 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:58.920 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:58.920 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5219 2024-09-14T23:58:58.920 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.920 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:58.920 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.920 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:58.921 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:58:58.921 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.921 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.921 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.921 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:58.921 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:58:58.921 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.921 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.921 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.921 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:58.922 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:58:58.922 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.03e-07 2024-09-14T23:58:58.922 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.922 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.922 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:58.922 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257983+0000", 2024-09-14T23:58:58.922 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9199999999999999e-07 2024-09-14T23:58:58.922 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.922 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.922 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:58.922 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257986+0000", 2024-09-14T23:58:58.922 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0860000000000002e-06 2024-09-14T23:58:58.923 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.923 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.923 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:58.923 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257989+0000", 2024-09-14T23:58:58.923 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6589999999999999e-06 2024-09-14T23:58:58.923 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.923 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.923 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:58.923 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315509+0000", 2024-09-14T23:58:58.923 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057520262000000003 2024-09-14T23:58:58.924 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.924 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.924 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:58.924 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315554+0000", 2024-09-14T23:58:58.924 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5015999999999997e-05 2024-09-14T23:58:58.924 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.924 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.924 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:58.924 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315619+0000", 2024-09-14T23:58:58.924 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5545999999999999e-05 2024-09-14T23:58:58.924 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.925 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:58.925 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.925 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.925 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.925 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5220 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3399680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:58.925 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:58:58.925 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 139.61463053099999, 2024-09-14T23:58:58.925 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057792128999999998, 2024-09-14T23:58:58.925 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:58.925 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:58.926 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:58.926 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:58.926 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:58.926 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:58.926 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5220 2024-09-14T23:58:58.926 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.926 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:58.926 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.926 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:58.926 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:58:58.926 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.926 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.927 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.927 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:58.927 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:58:58.927 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.927 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.927 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.927 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:58.927 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:58:58.927 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1799999999999999e-07 2024-09-14T23:58:58.927 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.928 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.928 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:58.928 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257993+0000", 2024-09-14T23:58:58.928 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6200000000000006e-07 2024-09-14T23:58:58.928 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.928 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.928 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:58.928 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257996+0000", 2024-09-14T23:58:58.928 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9050000000000001e-06 2024-09-14T23:58:58.928 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.928 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.929 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:58.929 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257998+0000", 2024-09-14T23:58:58.929 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5950000000000001e-06 2024-09-14T23:58:58.929 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.929 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.929 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:58.929 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315672+0000", 2024-09-14T23:58:58.929 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057673637999999999 2024-09-14T23:58:58.929 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.929 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.930 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:58.930 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315734+0000", 2024-09-14T23:58:58.930 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2216000000000002e-05 2024-09-14T23:58:58.930 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.930 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.930 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:58.930 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315784+0000", 2024-09-14T23:58:58.930 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9595000000000002e-05 2024-09-14T23:58:58.930 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.930 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:58.930 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.930 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.931 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.931 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5221 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3461120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:58.931 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:58:58.931 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 139.61462081299999, 2024-09-14T23:58:58.931 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058003557999999997, 2024-09-14T23:58:58.931 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:58.931 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:58.931 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:58.931 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:58.931 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:58.931 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:58.932 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5221 2024-09-14T23:58:58.932 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.932 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:58.932 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.932 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:58.932 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:58:58.932 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.932 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.932 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.932 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:58.932 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:58:58.933 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.933 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.933 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:58.933 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259905+0000", 2024-09-14T23:58:58.933 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019035759999999999 2024-09-14T23:58:58.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.933 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.933 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:58.933 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259915+0000", 2024-09-14T23:58:58.934 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0518e-05 2024-09-14T23:58:58.934 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.934 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.934 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:58.934 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259925+0000", 2024-09-14T23:58:58.934 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7180000000000008e-06 2024-09-14T23:58:58.934 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.934 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.934 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:58.934 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259930+0000", 2024-09-14T23:58:58.934 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4769999999999997e-06 2024-09-14T23:58:58.935 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.935 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.935 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:58.935 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315820+0000", 2024-09-14T23:58:58.935 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055889960000000002 2024-09-14T23:58:58.935 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.935 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.935 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:58.935 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315914+0000", 2024-09-14T23:58:58.935 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4625999999999998e-05 2024-09-14T23:58:58.935 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.935 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.936 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:58.936 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.316005+0000", 2024-09-14T23:58:58.936 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0682999999999999e-05 2024-09-14T23:58:58.936 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.936 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:58.936 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.936 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.936 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.936 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5231 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 4075520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:58.936 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:58:58.936 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 139.61255207900001, 2024-09-14T23:58:58.936 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057524733000000002, 2024-09-14T23:58:58.936 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:58.937 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:58.937 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:58.937 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:58.937 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:58.937 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:58.937 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5231 2024-09-14T23:58:58.937 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.937 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:58.937 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.937 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:58.937 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:58:58.937 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.937 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.937 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.938 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:58.938 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:58:58.938 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.938 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.938 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.938 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:58.938 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261904+0000", 2024-09-14T23:58:58.938 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018338790000000001 2024-09-14T23:58:58.938 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.938 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.938 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:58.938 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261910+0000", 2024-09-14T23:58:58.939 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9689999999999999e-06 2024-09-14T23:58:58.939 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.939 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.939 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:58.939 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261923+0000", 2024-09-14T23:58:58.939 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3061000000000001e-05 2024-09-14T23:58:58.939 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.939 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.939 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:58.939 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261931+0000", 2024-09-14T23:58:58.939 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3159999999999997e-06 2024-09-14T23:58:58.939 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.939 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.939 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:58.940 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317456+0000", 2024-09-14T23:58:58.940 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055525219000000001 2024-09-14T23:58:58.940 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.940 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.940 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:58.940 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317520+0000", 2024-09-14T23:58:58.940 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4010000000000005e-05 2024-09-14T23:58:58.940 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.940 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.940 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:58.940 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317595+0000", 2024-09-14T23:58:58.940 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4278999999999998e-05 2024-09-14T23:58:58.940 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.940 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:58.941 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.941 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.941 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.941 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5236 3.8 3:1071668d:::rbd_data.1118de1168b.0000000000000092:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:58:58.941 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:58:58.941 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 139.610538615, 2024-09-14T23:58:58.941 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057499462000000001, 2024-09-14T23:58:58.941 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:58.941 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:58.941 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:58.941 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:58.941 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:58.941 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:58.942 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5236 2024-09-14T23:58:58.942 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.942 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:58.942 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.942 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:58.942 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:58:58.942 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.942 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.942 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.942 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:58.942 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:58:58.942 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.942 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.942 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.943 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:58.943 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262084+0000", 2024-09-14T23:58:58.943 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3399999999999999e-07 2024-09-14T23:58:58.943 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.943 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.943 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:58.943 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262087+0000", 2024-09-14T23:58:58.943 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2310000000000001e-06 2024-09-14T23:58:58.943 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.943 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.943 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:58.943 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262095+0000", 2024-09-14T23:58:58.943 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3799999999999996e-06 2024-09-14T23:58:58.944 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.944 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.944 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:58.944 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262101+0000", 2024-09-14T23:58:58.944 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0009999999999998e-06 2024-09-14T23:58:58.944 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.944 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.944 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:58.944 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317625+0000", 2024-09-14T23:58:58.944 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055524111000000001 2024-09-14T23:58:58.944 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.944 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.944 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:58.944 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317702+0000", 2024-09-14T23:58:58.945 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7533999999999993e-05 2024-09-14T23:58:58.945 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.945 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.945 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-14T23:58:58.945 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317892+0000", 2024-09-14T23:58:58.945 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018991400000000001 2024-09-14T23:58:58.945 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.945 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.945 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:58:58.945 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319407+0000", 2024-09-14T23:58:58.945 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015145250000000001 2024-09-14T23:58:58.946 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.946 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.946 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:58.946 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319481+0000", 2024-09-14T23:58:58.946 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3826999999999996e-05 2024-09-14T23:58:58.946 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.946 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.946 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:58.946 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319541+0000", 2024-09-14T23:58:58.946 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0470999999999998e-05 2024-09-14T23:58:58.946 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.946 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.947 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:58.947 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319583+0000", 2024-09-14T23:58:58.947 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1934000000000002e-05 2024-09-14T23:58:58.947 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.947 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:58.947 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.947 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.947 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.947 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:132622 3.1e 3.7a50b9e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-14T23:58:58.947 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:43.371285+0000", 2024-09-14T23:58:58.947 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 75.501336713000001, 2024-09-14T23:58:58.947 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.67037987200000004, 2024-09-14T23:58:58.947 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:58.948 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:58.948 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:58:58.948 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:58.948 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:58.948 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:58.948 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 132622 2024-09-14T23:58:58.948 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.948 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:58.948 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.948 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:58.949 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371285+0000", 2024-09-14T23:58:58.949 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.949 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.949 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.949 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:58.949 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371285+0000", 2024-09-14T23:58:58.949 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.949 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.949 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.949 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:58.949 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371286+0000", 2024-09-14T23:58:58.950 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6400000000000002e-07 2024-09-14T23:58:58.950 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.950 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.950 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:58.950 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371289+0000", 2024-09-14T23:58:58.950 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2059999999999999e-06 2024-09-14T23:58:58.950 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.950 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.951 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:58.951 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371299+0000", 2024-09-14T23:58:58.951 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0288e-05 2024-09-14T23:58:58.951 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.951 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.951 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:58.951 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371306+0000", 2024-09-14T23:58:58.951 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1650000000000003e-06 2024-09-14T23:58:58.951 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.951 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.951 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:58.952 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371617+0000", 2024-09-14T23:58:58.952 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000311043 2024-09-14T23:58:58.952 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.952 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.952 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:58:58.952 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371625+0000", 2024-09-14T23:58:58.952 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.833e-06 2024-09-14T23:58:58.952 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.952 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.952 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:58.953 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.041651+0000", 2024-09-14T23:58:58.953 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.67002580199999995 2024-09-14T23:58:58.953 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.953 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.953 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:58.953 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.041665+0000", 2024-09-14T23:58:58.953 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3971000000000001e-05 2024-09-14T23:58:58.953 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.953 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:58.953 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.953 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.953 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.953 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:169975 3.5 3:a704a2f0:::rbd_data.1118427720d0.0000000000000030:32 [read 969728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-14T23:58:58.954 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:03.433958+0000", 2024-09-14T23:58:58.954 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 55.438664357, 2024-09-14T23:58:58.954 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057564811, 2024-09-14T23:58:58.954 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:58.954 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:58.954 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:58.954 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:58.954 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:58.954 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:58.954 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 169975 2024-09-14T23:58:58.954 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.954 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:58.954 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.954 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:58.955 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.433958+0000", 2024-09-14T23:58:58.955 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.955 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.955 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.955 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:58.955 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.433958+0000", 2024-09-14T23:58:58.955 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.955 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.955 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.955 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:58.955 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.435096+0000", 2024-09-14T23:58:58.955 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001138708 2024-09-14T23:58:58.955 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.955 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.956 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:58.956 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.435101+0000", 2024-09-14T23:58:58.956 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5050000000000001e-06 2024-09-14T23:58:58.956 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.956 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.956 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:58.956 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.435111+0000", 2024-09-14T23:58:58.956 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5810000000000003e-06 2024-09-14T23:58:58.956 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.956 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.956 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:58.957 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.435120+0000", 2024-09-14T23:58:58.957 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0089999999999996e-06 2024-09-14T23:58:58.957 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.957 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.957 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:58.957 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.491345+0000", 2024-09-14T23:58:58.957 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056225479000000002 2024-09-14T23:58:58.957 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.957 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.957 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:58.957 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.491423+0000", 2024-09-14T23:58:58.958 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7990000000000004e-05 2024-09-14T23:58:58.958 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.958 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.958 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:58.958 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.491523+0000", 2024-09-14T23:58:58.958 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9538999999999997e-05 2024-09-14T23:58:58.958 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.958 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:58.958 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.958 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.958 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.958 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:172639 3.5 3:a704a2f0:::rbd_data.1118427720d0.0000000000000030:32 [read 1645568~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e107)", 2024-09-14T23:58:58.959 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:03.988365+0000", 2024-09-14T23:58:58.959 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 54.884256637, 2024-09-14T23:58:58.959 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057556213000000002, 2024-09-14T23:58:58.959 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:58.959 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:58.959 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:58.959 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:58.959 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:58.959 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:58.959 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 172639 2024-09-14T23:58:58.959 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.960 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:58.960 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.960 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:58.960 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.988365+0000", 2024-09-14T23:58:58.960 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.960 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.960 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.960 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:58.960 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.988365+0000", 2024-09-14T23:58:58.960 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.960 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.960 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.961 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:58.961 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.989904+0000", 2024-09-14T23:58:58.961 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015380859999999999 2024-09-14T23:58:58.961 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.961 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.961 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:58.961 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.989909+0000", 2024-09-14T23:58:58.961 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2800000000000003e-06 2024-09-14T23:58:58.961 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.961 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.961 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:58.962 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.989925+0000", 2024-09-14T23:58:58.962 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5781999999999999e-05 2024-09-14T23:58:58.962 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.962 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.962 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:58.962 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.989933+0000", 2024-09-14T23:58:58.962 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0090000000000007e-06 2024-09-14T23:58:58.962 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.962 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.962 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:58.962 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:04.045741+0000", 2024-09-14T23:58:58.962 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055808112999999999 2024-09-14T23:58:58.963 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.963 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.963 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:58.963 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:04.045812+0000", 2024-09-14T23:58:58.963 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1531000000000003e-05 2024-09-14T23:58:58.963 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.963 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.963 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:58.963 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:04.045922+0000", 2024-09-14T23:58:58.963 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000109412 2024-09-14T23:58:58.963 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.964 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:58.964 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.964 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.964 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.964 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184571 3.2 3:46f8d78b:::rbd_data.1118427720d0.0000000000000001:32 [read 1405952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:58.964 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.086404+0000", 2024-09-14T23:58:58.964 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 51.78621828, 2024-09-14T23:58:58.964 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059148573000000003, 2024-09-14T23:58:58.964 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:58.964 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:58.964 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:58.965 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:58.965 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:58.965 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:58.965 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184571 2024-09-14T23:58:58.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.965 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:58.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.965 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:58.965 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.086404+0000", 2024-09-14T23:58:58.965 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.966 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:58.966 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.086404+0000", 2024-09-14T23:58:58.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.966 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:58.966 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087902+0000", 2024-09-14T23:58:58.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001498625 2024-09-14T23:58:58.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.966 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:58.966 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087907+0000", 2024-09-14T23:58:58.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9130000000000002e-06 2024-09-14T23:58:58.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.967 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:58.967 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087919+0000", 2024-09-14T23:58:58.967 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1719000000000001e-05 2024-09-14T23:58:58.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.967 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:58.967 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087927+0000", 2024-09-14T23:58:58.967 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3699999999999995e-06 2024-09-14T23:58:58.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.967 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:58.967 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143711+0000", 2024-09-14T23:58:58.967 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055783396999999998 2024-09-14T23:58:58.968 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.968 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.968 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:58.968 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143792+0000", 2024-09-14T23:58:58.968 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1292999999999996e-05 2024-09-14T23:58:58.968 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.968 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.968 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:58.968 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145552+0000", 2024-09-14T23:58:58.968 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001760256 2024-09-14T23:58:58.968 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.968 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:58.968 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.969 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.969 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184579 3.2 3:43badc91:::rbd_data.1118427720d0.0000000000000002:32 [read 1196032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:58:58.969 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.088145+0000", 2024-09-14T23:58:58.969 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 51.784476875999999, 2024-09-14T23:58:58.969 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057643169000000001, 2024-09-14T23:58:58.969 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:58.969 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:58.969 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:58.969 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:58.969 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:58.969 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:58.969 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184579 2024-09-14T23:58:58.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.970 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:58.970 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.970 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:58.970 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.088145+0000", 2024-09-14T23:58:58.970 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.970 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.970 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:58.970 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.088145+0000", 2024-09-14T23:58:58.970 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.970 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.970 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:58.971 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089947+0000", 2024-09-14T23:58:58.971 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001801764 2024-09-14T23:58:58.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.971 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:58.971 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089952+0000", 2024-09-14T23:58:58.971 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8659999999999997e-06 2024-09-14T23:58:58.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.971 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:58.971 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089966+0000", 2024-09-14T23:58:58.971 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3929e-05 2024-09-14T23:58:58.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.972 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:58.972 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089974+0000", 2024-09-14T23:58:58.972 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1429999999999998e-06 2024-09-14T23:58:58.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.972 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.972 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:58.972 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145636+0000", 2024-09-14T23:58:58.972 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055662287999999997 2024-09-14T23:58:58.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.972 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.972 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:58.972 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145710+0000", 2024-09-14T23:58:58.972 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4166999999999996e-05 2024-09-14T23:58:58.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.973 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.973 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:58.973 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145788+0000", 2024-09-14T23:58:58.973 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8011999999999998e-05 2024-09-14T23:58:58.973 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.973 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:58.973 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.973 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.973 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.973 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:211800 3.12 3:493125e8:::rbd_data.1118c900944f.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-14T23:58:58.973 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:13.660130+0000", 2024-09-14T23:58:58.973 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 45.21249255, 2024-09-14T23:58:58.973 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.55364332000000005, 2024-09-14T23:58:58.973 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:58.973 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:58.974 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:58.974 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:58.974 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:58.974 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:58.974 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 211800 2024-09-14T23:58:58.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.974 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:58.974 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.974 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:58.974 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.660130+0000", 2024-09-14T23:58:58.974 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.974 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.974 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:58.975 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.660130+0000", 2024-09-14T23:58:58.975 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.975 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.975 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.975 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:58.975 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.660133+0000", 2024-09-14T23:58:58.975 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.202e-06 2024-09-14T23:58:58.975 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.975 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.975 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:58.975 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.668135+0000", 2024-09-14T23:58:58.976 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0080027220000000003 2024-09-14T23:58:58.976 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.976 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.976 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:58.976 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.668172+0000", 2024-09-14T23:58:58.976 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6295999999999999e-05 2024-09-14T23:58:58.976 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.976 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.976 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:58.976 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.668187+0000", 2024-09-14T23:58:58.977 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5537e-05 2024-09-14T23:58:58.977 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.977 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.977 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:58.977 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.668284+0000", 2024-09-14T23:58:58.977 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6297000000000003e-05 2024-09-14T23:58:58.977 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.977 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.977 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:58:58.977 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.668299+0000", 2024-09-14T23:58:58.977 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5091e-05 2024-09-14T23:58:58.977 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.977 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.978 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:58.978 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:14.195910+0000", 2024-09-14T23:58:58.978 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.52761157999999997 2024-09-14T23:58:58.978 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.978 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.978 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:58.978 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:14.196106+0000", 2024-09-14T23:58:58.978 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019597399999999999 2024-09-14T23:58:58.978 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.978 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.978 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 0", 2024-09-14T23:58:58.979 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:14.197262+0000", 2024-09-14T23:58:58.979 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001155744 2024-09-14T23:58:58.979 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.979 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.979 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:58:58.979 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:14.207440+0000", 2024-09-14T23:58:58.979 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010178347000000001 2024-09-14T23:58:58.979 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.979 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.979 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:58.979 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:14.213691+0000", 2024-09-14T23:58:58.980 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0062504420000000002 2024-09-14T23:58:58.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.980 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.980 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:58:58.980 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:14.213735+0000", 2024-09-14T23:58:58.980 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4385999999999998e-05 2024-09-14T23:58:58.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.980 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.980 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:58.981 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:14.213773+0000", 2024-09-14T23:58:58.981 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7701999999999999e-05 2024-09-14T23:58:58.981 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.981 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:58.981 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.981 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.981 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.981 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:251476 3.5 3:a1f6ebd6:::rbd_object_map.11189263ca38.000000000000002a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126)", 2024-09-14T23:58:58.981 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:26.388925+0000", 2024-09-14T23:58:58.981 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.483697517000003, 2024-09-14T23:58:58.981 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.062808638, 2024-09-14T23:58:58.982 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:58:58.982 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:58:58.982 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:58:58.982 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:58:58.982 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:58:58.982 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:58.982 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 251476 2024-09-14T23:58:58.982 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.983 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:58:58.983 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.983 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:58:58.983 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.388925+0000", 2024-09-14T23:58:58.983 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.983 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.983 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:58:58.983 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.388925+0000", 2024-09-14T23:58:58.983 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:58:58.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.984 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.984 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:58:58.984 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.388926+0000", 2024-09-14T23:58:58.984 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5200000000000001e-06 2024-09-14T23:58:58.984 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.984 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.984 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:58:58.984 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.388933+0000", 2024-09-14T23:58:58.984 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.827e-06 2024-09-14T23:58:58.984 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.984 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.985 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:58:58.985 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.388945+0000", 2024-09-14T23:58:58.985 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1701e-05 2024-09-14T23:58:58.985 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.985 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.985 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:58:58.985 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.388950+0000", 2024-09-14T23:58:58.985 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6080000000000003e-06 2024-09-14T23:58:58.985 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.985 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.985 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:58.986 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.388999+0000", 2024-09-14T23:58:58.986 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8711999999999997e-05 2024-09-14T23:58:58.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.986 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.986 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:58:58.986 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.389006+0000", 2024-09-14T23:58:58.986 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.063e-06 2024-09-14T23:58:58.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.986 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.986 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:58:58.986 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.451371+0000", 2024-09-14T23:58:58.987 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.062365255000000001 2024-09-14T23:58:58.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.987 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.987 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:58:58.987 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.451472+0000", 2024-09-14T23:58:58.987 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100626 2024-09-14T23:58:58.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:58:58.987 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:58:58.987 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:58:58.987 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.451733+0000", 2024-09-14T23:58:58.987 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026132599999999998 2024-09-14T23:58:58.987 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.988 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:58.988 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.988 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:58:58.988 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:58:58.988 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:58:58.990 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T23:58:58.990 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:58.988+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:59.081 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is set 2024-09-14T23:58:59.089 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:59.088+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:59.122 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:59.122 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:58:59.122 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.122 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:256937 3.f e159/153 3:f1b7e4f5:::rbd_header.1118c507b169:head v 159'579, mlcod=159'577)", 2024-09-14T23:58:59.122 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:59.120488+0000", 2024-09-14T23:58:59.122 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.00045048100000000003, 2024-09-14T23:58:59.122 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00046734800000000003, 2024-09-14T23:58:59.123 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:59.123 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:59.123 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:59.123 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:59.123 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.123 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:59.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:59.120488+0000", 2024-09-14T23:58:59.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.123 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.123 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:59.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:59.120488+0000", 2024-09-14T23:58:59.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.124 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:59.124 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:59.120489+0000", 2024-09-14T23:58:59.124 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.16e-07 2024-09-14T23:58:59.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.124 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:59.124 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:59.120494+0000", 2024-09-14T23:58:59.124 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3499999999999996e-06 2024-09-14T23:58:59.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.124 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:59.124 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:59.120505+0000", 2024-09-14T23:58:59.125 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1011e-05 2024-09-14T23:58:59.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.125 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.125 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:59.125 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:59.120510+0000", 2024-09-14T23:58:59.125 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.78e-06 2024-09-14T23:58:59.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.125 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.125 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.125 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:59.120559+0000", 2024-09-14T23:58:59.125 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8869000000000003e-05 2024-09-14T23:58:59.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.125 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.125 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:59.126 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:59.120572+0000", 2024-09-14T23:58:59.126 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3353000000000001e-05 2024-09-14T23:58:59.126 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.126 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:59.126 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.126 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.126 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-14T23:58:59.126 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 1 2024-09-14T23:58:59.126 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:59.132 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T23:58:59.191 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:58:59.189+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:58:59.283 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:59.283 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:58:59.283 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:58:59.283 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:58:59.283 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:59.293 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T23:58:59.294 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:59.290+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:59.393 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:58:59.392+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:58:59.402 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:59.402 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:58:59.402 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:58:59.402 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:58:59.402 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.402 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186203 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1329152~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:59.402 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.194279+0000", 2024-09-14T23:58:59.403 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 52.205638182999998, 2024-09-14T23:58:59.403 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8011469710000001, 2024-09-14T23:58:59.403 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:59.403 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:59.403 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:59.403 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:59.403 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:59.403 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:59.403 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186203 2024-09-14T23:58:59.403 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.403 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:59.403 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.403 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:59.403 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194279+0000", 2024-09-14T23:58:59.404 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.404 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.404 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.404 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:59.404 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194279+0000", 2024-09-14T23:58:59.404 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.404 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.404 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.404 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:59.404 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194279+0000", 2024-09-14T23:58:59.404 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8100000000000001e-07 2024-09-14T23:58:59.404 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.404 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.404 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:59.405 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194291+0000", 2024-09-14T23:58:59.405 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1559999999999999e-05 2024-09-14T23:58:59.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.405 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.405 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:59.405 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194303+0000", 2024-09-14T23:58:59.405 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2130000000000001e-05 2024-09-14T23:58:59.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.405 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.405 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:59.405 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194311+0000", 2024-09-14T23:58:59.405 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8089999999999993e-06 2024-09-14T23:58:59.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.405 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.406 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.406 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194370+0000", 2024-09-14T23:58:59.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8739999999999997e-05 2024-09-14T23:58:59.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.406 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:59.406 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194376+0000", 2024-09-14T23:58:59.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2319999999999998e-06 2024-09-14T23:58:59.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.406 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.406 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986290+0000", 2024-09-14T23:58:59.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.791914334 2024-09-14T23:58:59.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.407 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:59.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986296+0000", 2024-09-14T23:58:59.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6300000000000003e-06 2024-09-14T23:58:59.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.407 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995193+0000", 2024-09-14T23:58:59.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0088975040000001 2024-09-14T23:58:59.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.407 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:59.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995307+0000", 2024-09-14T23:58:59.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000113656 2024-09-14T23:58:59.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.408 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.408 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:59.408 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995426+0000", 2024-09-14T23:58:59.408 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118495 2024-09-14T23:58:59.408 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.408 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:59.408 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.408 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.408 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.408 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186204 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1391616~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:59.408 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.194328+0000", 2024-09-14T23:58:59.408 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 52.205588851999998, 2024-09-14T23:58:59.408 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8013834310000001, 2024-09-14T23:58:59.408 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:59.409 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:59.409 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:59.409 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:59.409 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:59.409 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:59.409 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186204 2024-09-14T23:58:59.409 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.409 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:59.409 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.409 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:59.409 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194328+0000", 2024-09-14T23:58:59.409 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.409 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.409 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.410 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:59.410 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194328+0000", 2024-09-14T23:58:59.410 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.410 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.410 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:59.410 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194328+0000", 2024-09-14T23:58:59.410 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7100000000000002e-07 2024-09-14T23:58:59.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.410 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.410 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:59.410 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194331+0000", 2024-09-14T23:58:59.410 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.807e-06 2024-09-14T23:58:59.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.411 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.411 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:59.411 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194339+0000", 2024-09-14T23:58:59.411 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9340000000000006e-06 2024-09-14T23:58:59.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.411 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.411 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:59.411 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194345+0000", 2024-09-14T23:58:59.411 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1090000000000003e-06 2024-09-14T23:58:59.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.411 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.411 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.412 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194412+0000", 2024-09-14T23:58:59.412 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6229000000000006e-05 2024-09-14T23:58:59.412 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.412 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.412 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:59.412 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194422+0000", 2024-09-14T23:58:59.412 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.078e-05 2024-09-14T23:58:59.412 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.412 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.412 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.412 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986322+0000", 2024-09-14T23:58:59.412 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.79189964899999998 2024-09-14T23:58:59.412 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.413 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.413 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:59.413 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986332+0000", 2024-09-14T23:58:59.413 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0095000000000001e-05 2024-09-14T23:58:59.413 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.413 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.413 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.413 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995554+0000", 2024-09-14T23:58:59.413 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0092218630000001 2024-09-14T23:58:59.413 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.413 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.413 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:59.413 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995610+0000", 2024-09-14T23:58:59.414 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6109000000000001e-05 2024-09-14T23:58:59.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.414 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.414 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:59.414 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995711+0000", 2024-09-14T23:58:59.414 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101285 2024-09-14T23:58:59.414 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.414 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:59.414 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.414 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.414 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186205 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1333248~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:59.414 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.194354+0000", 2024-09-14T23:58:59.414 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 52.205562921999999, 2024-09-14T23:58:59.415 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8015471490000001, 2024-09-14T23:58:59.415 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:59.415 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:59.415 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:59.415 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:59.415 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:59.415 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:59.415 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186205 2024-09-14T23:58:59.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.415 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:59.415 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.415 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:59.415 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194354+0000", 2024-09-14T23:58:59.415 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.416 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.416 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:59.416 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194354+0000", 2024-09-14T23:58:59.416 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.416 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.416 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:59.416 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194354+0000", 2024-09-14T23:58:59.416 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.39e-07 2024-09-14T23:58:59.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.417 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.417 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:59.417 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194357+0000", 2024-09-14T23:58:59.417 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3479999999999998e-06 2024-09-14T23:58:59.417 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.417 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.417 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:59.417 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194364+0000", 2024-09-14T23:58:59.417 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0899999999999999e-06 2024-09-14T23:58:59.417 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.417 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.417 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:59.417 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194371+0000", 2024-09-14T23:58:59.418 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5530000000000004e-06 2024-09-14T23:58:59.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.418 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.418 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194469+0000", 2024-09-14T23:58:59.418 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8119e-05 2024-09-14T23:58:59.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.418 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:59.418 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194475+0000", 2024-09-14T23:58:59.418 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7910000000000001e-06 2024-09-14T23:58:59.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.419 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.419 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986367+0000", 2024-09-14T23:58:59.419 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.79189214699999999 2024-09-14T23:58:59.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.419 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.419 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:59.419 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986375+0000", 2024-09-14T23:58:59.419 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7800000000000001e-06 2024-09-14T23:58:59.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.419 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.420 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.420 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995746+0000", 2024-09-14T23:58:59.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0093706469999999 2024-09-14T23:58:59.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.420 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.420 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:59.420 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995810+0000", 2024-09-14T23:58:59.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4068999999999999e-05 2024-09-14T23:58:59.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.420 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.420 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:59.420 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995901+0000", 2024-09-14T23:58:59.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1166000000000001e-05 2024-09-14T23:58:59.421 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.421 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:59.421 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.421 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.421 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186206 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1395712~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:59.421 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.194380+0000", 2024-09-14T23:58:59.421 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 52.205536901000002, 2024-09-14T23:58:59.421 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.801678347, 2024-09-14T23:58:59.421 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:59.421 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:59.421 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:59.421 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:59.422 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:59.422 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:59.422 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186206 2024-09-14T23:58:59.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.422 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:59.422 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.422 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:59.422 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194380+0000", 2024-09-14T23:58:59.422 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.422 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.422 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:59.422 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194380+0000", 2024-09-14T23:58:59.422 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.423 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:59.423 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194380+0000", 2024-09-14T23:58:59.423 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4400000000000001e-07 2024-09-14T23:58:59.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.423 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:59.423 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194383+0000", 2024-09-14T23:58:59.423 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3460000000000001e-06 2024-09-14T23:58:59.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:59.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194390+0000", 2024-09-14T23:58:59.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2189999999999998e-06 2024-09-14T23:58:59.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:59.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194403+0000", 2024-09-14T23:58:59.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2941e-05 2024-09-14T23:58:59.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194545+0000", 2024-09-14T23:58:59.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014232700000000001 2024-09-14T23:58:59.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.425 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:59.425 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194565+0000", 2024-09-14T23:58:59.425 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9409999999999999e-05 2024-09-14T23:58:59.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.425 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.425 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986409+0000", 2024-09-14T23:58:59.425 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.79184416000000002 2024-09-14T23:58:59.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.426 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:59.426 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986417+0000", 2024-09-14T23:58:59.426 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.551e-06 2024-09-14T23:58:59.426 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.426 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.426 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.426 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995921+0000", 2024-09-14T23:58:59.426 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.009504212 2024-09-14T23:58:59.426 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.426 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.426 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:59.426 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995974+0000", 2024-09-14T23:58:59.426 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2617999999999997e-05 2024-09-14T23:58:59.427 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.427 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.427 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:59.427 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996058+0000", 2024-09-14T23:58:59.427 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4118999999999999e-05 2024-09-14T23:58:59.427 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.427 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:59.427 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.427 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.427 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.427 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186207 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1337344~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:59.427 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.194412+0000", 2024-09-14T23:58:59.427 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 52.205504302999998, 2024-09-14T23:58:59.427 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.801848141, 2024-09-14T23:58:59.428 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:59.428 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:59.428 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:59.428 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:59.428 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:59.428 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:59.428 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186207 2024-09-14T23:58:59.428 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.428 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:59.428 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.428 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:59.428 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194412+0000", 2024-09-14T23:58:59.428 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.428 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.429 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:59.429 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194412+0000", 2024-09-14T23:58:59.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.429 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:59.429 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194413+0000", 2024-09-14T23:58:59.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5600000000000001e-07 2024-09-14T23:58:59.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.429 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:59.429 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194418+0000", 2024-09-14T23:58:59.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3720000000000004e-06 2024-09-14T23:58:59.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.430 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:59.430 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194423+0000", 2024-09-14T23:58:59.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.763e-06 2024-09-14T23:58:59.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.430 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:59.430 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194426+0000", 2024-09-14T23:58:59.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1480000000000002e-06 2024-09-14T23:58:59.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.430 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.430 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194596+0000", 2024-09-14T23:58:59.431 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017009600000000001 2024-09-14T23:58:59.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.431 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.431 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:59.431 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194606+0000", 2024-09-14T23:58:59.431 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6639999999999993e-06 2024-09-14T23:58:59.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.431 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.431 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.431 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986460+0000", 2024-09-14T23:58:59.431 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.79185424400000004 2024-09-14T23:58:59.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.431 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.432 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:59.432 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986469+0000", 2024-09-14T23:58:59.432 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4419999999999998e-06 2024-09-14T23:58:59.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.432 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.432 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996123+0000", 2024-09-14T23:58:59.432 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0096546070000001 2024-09-14T23:58:59.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.432 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:59.433 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996179+0000", 2024-09-14T23:58:59.433 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5850000000000002e-05 2024-09-14T23:58:59.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.433 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:59.433 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996261+0000", 2024-09-14T23:58:59.433 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1599000000000003e-05 2024-09-14T23:58:59.433 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.433 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:59.433 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.433 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186208 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1399808~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:59.433 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.194445+0000", 2024-09-14T23:58:59.434 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 52.205472133000001, 2024-09-14T23:58:59.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8019822269999999, 2024-09-14T23:58:59.434 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:59.434 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:59.434 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:59.434 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:59.434 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:59.434 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:59.434 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186208 2024-09-14T23:58:59.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.434 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:59.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.434 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:59.434 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194445+0000", 2024-09-14T23:58:59.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.435 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:59.435 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194445+0000", 2024-09-14T23:58:59.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.435 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:59.435 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978674+0000", 2024-09-14T23:58:59.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.784229124 2024-09-14T23:58:59.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.435 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:59.436 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978680+0000", 2024-09-14T23:58:59.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7969999999999997e-06 2024-09-14T23:58:59.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.436 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:59.436 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978689+0000", 2024-09-14T23:58:59.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6949999999999995e-06 2024-09-14T23:58:59.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.436 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:59.436 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978696+0000", 2024-09-14T23:58:59.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1300000000000003e-06 2024-09-14T23:58:59.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.437 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986509+0000", 2024-09-14T23:58:59.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0078130969999999998 2024-09-14T23:58:59.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.437 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:59.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986518+0000", 2024-09-14T23:58:59.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2810000000000008e-06 2024-09-14T23:58:59.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.437 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996280+0000", 2024-09-14T23:58:59.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0097623120000001 2024-09-14T23:58:59.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:59.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996337+0000", 2024-09-14T23:58:59.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7244000000000003e-05 2024-09-14T23:58:59.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:59.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996427+0000", 2024-09-14T23:58:59.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9547000000000002e-05 2024-09-14T23:58:59.438 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.438 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:59.438 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.439 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4172.0:7 3.11 3:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:59.439 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.483386+0000", 2024-09-14T23:58:59.439 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.916530708000003, 2024-09-14T23:58:59.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.558667389, 2024-09-14T23:58:59.439 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:59.439 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:59.439 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:58:59.439 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:59.439 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4172", 2024-09-14T23:58:59.439 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/2542269203", 2024-09-14T23:58:59.439 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 7 2024-09-14T23:58:59.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.439 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:59.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.440 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:59.440 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.483386+0000", 2024-09-14T23:58:59.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.440 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:59.440 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.483386+0000", 2024-09-14T23:58:59.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.440 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:59.440 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978674+0000", 2024-09-14T23:58:59.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.49528781599999999 2024-09-14T23:58:59.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:59.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978687+0000", 2024-09-14T23:58:59.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3244999999999999e-05 2024-09-14T23:58:59.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:59.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978699+0000", 2024-09-14T23:58:59.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1832e-05 2024-09-14T23:58:59.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:59.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978707+0000", 2024-09-14T23:58:59.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4489999999999999e-06 2024-09-14T23:58:59.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978801+0000", 2024-09-14T23:58:59.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3511999999999995e-05 2024-09-14T23:58:59.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:59.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978814+0000", 2024-09-14T23:58:59.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2789000000000001e-05 2024-09-14T23:58:59.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.041948+0000", 2024-09-14T23:58:59.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0631338269999999 2024-09-14T23:58:59.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.443 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:59.443 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.042053+0000", 2024-09-14T23:58:59.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105919 2024-09-14T23:58:59.443 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.443 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:59.443 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.443 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187209 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1575936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:59.443 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.991888+0000", 2024-09-14T23:58:59.444 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.408028737000002, 2024-09-14T23:58:59.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0712115470000001, 2024-09-14T23:58:59.444 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:59.444 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:59.444 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:59.444 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:59.444 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:59.444 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:59.444 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187209 2024-09-14T23:58:59.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.444 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:59.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:59.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991888+0000", 2024-09-14T23:58:59.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:59.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991888+0000", 2024-09-14T23:58:59.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:59.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991888+0000", 2024-09-14T23:58:59.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1199999999999998e-07 2024-09-14T23:58:59.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.446 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:59.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991890+0000", 2024-09-14T23:58:59.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9810000000000002e-06 2024-09-14T23:58:59.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.446 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:59.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991897+0000", 2024-09-14T23:58:59.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2609999999999999e-06 2024-09-14T23:58:59.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.446 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:59.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991902+0000", 2024-09-14T23:58:59.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0000000000000004e-06 2024-09-14T23:58:59.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995686+0000", 2024-09-14T23:58:59.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0037839290000000001 2024-09-14T23:58:59.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:59.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995693+0000", 2024-09-14T23:58:59.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7589999999999997e-06 2024-09-14T23:58:59.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062846+0000", 2024-09-14T23:58:59.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.067152417 2024-09-14T23:58:59.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:59.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062935+0000", 2024-09-14T23:58:59.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9080999999999996e-05 2024-09-14T23:58:59.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:59.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.063100+0000", 2024-09-14T23:58:59.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016470699999999999 2024-09-14T23:58:59.449 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.449 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:59.449 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.449 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187210 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1591296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:59.449 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.991908+0000", 2024-09-14T23:58:59.449 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.408008823999999, 2024-09-14T23:58:59.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0713882530000001, 2024-09-14T23:58:59.449 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:59.450 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:59.450 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:59.450 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:59.450 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:59.450 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:59.450 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187210 2024-09-14T23:58:59.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.450 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:59.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.450 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:59.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991908+0000", 2024-09-14T23:58:59.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:59.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991908+0000", 2024-09-14T23:58:59.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:59.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993673+0000", 2024-09-14T23:58:59.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017651030000000001 2024-09-14T23:58:59.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:59.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993681+0000", 2024-09-14T23:58:59.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.858e-06 2024-09-14T23:58:59.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:59.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993691+0000", 2024-09-14T23:58:59.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9920000000000002e-06 2024-09-14T23:58:59.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:59.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993698+0000", 2024-09-14T23:58:59.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6490000000000002e-06 2024-09-14T23:58:59.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995731+0000", 2024-09-14T23:58:59.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0020338959999999999 2024-09-14T23:58:59.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:59.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995741+0000", 2024-09-14T23:58:59.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3079999999999997e-06 2024-09-14T23:58:59.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.063130+0000", 2024-09-14T23:58:59.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.067389159 2024-09-14T23:58:59.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:59.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.063206+0000", 2024-09-14T23:58:59.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6577999999999998e-05 2024-09-14T23:58:59.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:59.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.063296+0000", 2024-09-14T23:58:59.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9710000000000007e-05 2024-09-14T23:58:59.455 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.455 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:59.455 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.455 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187216 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1603584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:59.455 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.993831+0000", 2024-09-14T23:58:59.455 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.406085617999999, 2024-09-14T23:58:59.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.071244627, 2024-09-14T23:58:59.455 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:59.455 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:59.455 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:59.456 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:59.456 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:59.456 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:59.456 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187216 2024-09-14T23:58:59.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.456 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:59.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.456 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:59.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993831+0000", 2024-09-14T23:58:59.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.456 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:59.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993831+0000", 2024-09-14T23:58:59.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:59.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993832+0000", 2024-09-14T23:58:59.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9700000000000002e-07 2024-09-14T23:58:59.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:59.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993834+0000", 2024-09-14T23:58:59.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6170000000000001e-06 2024-09-14T23:58:59.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:59.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993841+0000", 2024-09-14T23:58:59.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6980000000000001e-06 2024-09-14T23:58:59.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:59.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993846+0000", 2024-09-14T23:58:59.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4190000000000001e-06 2024-09-14T23:58:59.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995988+0000", 2024-09-14T23:58:59.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0021417390000000001 2024-09-14T23:58:59.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:59.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995998+0000", 2024-09-14T23:58:59.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5100000000000004e-06 2024-09-14T23:58:59.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.064883+0000", 2024-09-14T23:58:59.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.06888597 2024-09-14T23:58:59.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:59.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.064977+0000", 2024-09-14T23:58:59.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3817e-05 2024-09-14T23:58:59.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:59.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065076+0000", 2024-09-14T23:58:59.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8460000000000003e-05 2024-09-14T23:58:59.460 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.460 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:59.460 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.460 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187217 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1592320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:59.460 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.993855+0000", 2024-09-14T23:58:59.460 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.406062185000003, 2024-09-14T23:58:59.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0714239349999999, 2024-09-14T23:58:59.461 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:59.461 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:59.461 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:59.461 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:59.461 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:59.461 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:59.461 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187217 2024-09-14T23:58:59.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.461 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:59.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:59.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993855+0000", 2024-09-14T23:58:59.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:59.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993855+0000", 2024-09-14T23:58:59.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:59.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993855+0000", 2024-09-14T23:58:59.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4899999999999995e-07 2024-09-14T23:58:59.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:59.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993857+0000", 2024-09-14T23:58:59.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3379999999999999e-06 2024-09-14T23:58:59.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:59.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993864+0000", 2024-09-14T23:58:59.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6000000000000003e-06 2024-09-14T23:58:59.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:59.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993869+0000", 2024-09-14T23:58:59.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3210000000000004e-06 2024-09-14T23:58:59.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996038+0000", 2024-09-14T23:58:59.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0021683050000000001 2024-09-14T23:58:59.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:59.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996048+0000", 2024-09-14T23:58:59.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0173999999999999e-05 2024-09-14T23:58:59.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065108+0000", 2024-09-14T23:58:59.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0690600619999999 2024-09-14T23:58:59.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:59.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065171+0000", 2024-09-14T23:58:59.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3423999999999994e-05 2024-09-14T23:58:59.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:59.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065279+0000", 2024-09-14T23:58:59.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000107162 2024-09-14T23:58:59.466 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.466 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:59.466 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.466 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187218 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1607680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:59.466 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.993877+0000", 2024-09-14T23:58:59.466 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.406039905, 2024-09-14T23:58:59.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0716976810000001, 2024-09-14T23:58:59.466 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:59.466 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:59.466 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:59.466 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:59.466 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:59.467 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:59.467 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187218 2024-09-14T23:58:59.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.467 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:59.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:59.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993877+0000", 2024-09-14T23:58:59.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:59.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993877+0000", 2024-09-14T23:58:59.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:59.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993877+0000", 2024-09-14T23:58:59.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8299999999999998e-07 2024-09-14T23:58:59.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:59.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993879+0000", 2024-09-14T23:58:59.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5659999999999999e-06 2024-09-14T23:58:59.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:59.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993882+0000", 2024-09-14T23:58:59.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7280000000000001e-06 2024-09-14T23:58:59.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.469 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:59.469 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993885+0000", 2024-09-14T23:58:59.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.751e-06 2024-09-14T23:58:59.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.469 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.469 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996081+0000", 2024-09-14T23:58:59.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0021960590000000002 2024-09-14T23:58:59.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.469 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:59.469 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996091+0000", 2024-09-14T23:58:59.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4930000000000004e-06 2024-09-14T23:58:59.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065308+0000", 2024-09-14T23:58:59.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.069217407 2024-09-14T23:58:59.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:59.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065414+0000", 2024-09-14T23:58:59.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105438 2024-09-14T23:58:59.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:59.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065575+0000", 2024-09-14T23:58:59.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000160956 2024-09-14T23:58:59.471 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.471 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:59.471 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.471 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187219 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1596416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:59.471 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.993889+0000", 2024-09-14T23:58:59.471 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.406028063000001, 2024-09-14T23:58:59.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0720031190000001, 2024-09-14T23:58:59.471 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:59.471 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:59.471 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:59.471 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:59.472 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:59.472 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:59.472 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187219 2024-09-14T23:58:59.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.472 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:59.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:59.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993889+0000", 2024-09-14T23:58:59.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:59.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993889+0000", 2024-09-14T23:58:59.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:59.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993889+0000", 2024-09-14T23:58:59.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2399999999999999e-07 2024-09-14T23:58:59.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:59.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993890+0000", 2024-09-14T23:58:59.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1939999999999999e-06 2024-09-14T23:58:59.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:59.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993893+0000", 2024-09-14T23:58:59.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1300000000000001e-06 2024-09-14T23:58:59.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:59.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993896+0000", 2024-09-14T23:58:59.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6879999999999999e-06 2024-09-14T23:58:59.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996119+0000", 2024-09-14T23:58:59.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0022226149999999998 2024-09-14T23:58:59.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:59.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996123+0000", 2024-09-14T23:58:59.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.211e-06 2024-09-14T23:58:59.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065616+0000", 2024-09-14T23:58:59.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0694935240000001 2024-09-14T23:58:59.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:59.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065733+0000", 2024-09-14T23:58:59.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000116492 2024-09-14T23:58:59.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:59.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065892+0000", 2024-09-14T23:58:59.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015894100000000001 2024-09-14T23:58:59.476 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.476 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:59.476 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.476 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187220 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1611776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:59.476 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.993899+0000", 2024-09-14T23:58:59.476 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.406017605000002, 2024-09-14T23:58:59.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.072298041, 2024-09-14T23:58:59.476 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:59.476 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:59.476 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:59.477 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:59.477 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:59.477 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:59.477 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187220 2024-09-14T23:58:59.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.477 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:59.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:59.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993899+0000", 2024-09-14T23:58:59.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:59.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993899+0000", 2024-09-14T23:58:59.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:59.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993899+0000", 2024-09-14T23:58:59.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2100000000000001e-07 2024-09-14T23:58:59.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:59.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993901+0000", 2024-09-14T23:58:59.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.141e-06 2024-09-14T23:58:59.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:59.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993904+0000", 2024-09-14T23:58:59.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1470000000000001e-06 2024-09-14T23:58:59.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:59.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993906+0000", 2024-09-14T23:58:59.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6340000000000001e-06 2024-09-14T23:58:59.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996160+0000", 2024-09-14T23:58:59.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0022537659999999999 2024-09-14T23:58:59.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:59.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996169+0000", 2024-09-14T23:58:59.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2739999999999996e-06 2024-09-14T23:58:59.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065981+0000", 2024-09-14T23:58:59.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.069811742 2024-09-14T23:58:59.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:59.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.066092+0000", 2024-09-14T23:58:59.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000110701 2024-09-14T23:58:59.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:59.481 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.066197+0000", 2024-09-14T23:58:59.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010541500000000001 2024-09-14T23:58:59.481 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.481 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:59.481 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.481 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187221 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1600512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:59.481 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.993909+0000", 2024-09-14T23:58:59.481 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.406007393000003, 2024-09-14T23:58:59.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.072643499, 2024-09-14T23:58:59.482 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:59.482 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:59.482 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:59.482 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:59.482 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:59.482 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:59.482 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187221 2024-09-14T23:58:59.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.482 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:59.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:59.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993909+0000", 2024-09-14T23:58:59.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:59.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993909+0000", 2024-09-14T23:58:59.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:59.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995673+0000", 2024-09-14T23:58:59.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001763141 2024-09-14T23:58:59.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:59.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995677+0000", 2024-09-14T23:58:59.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8709999999999996e-06 2024-09-14T23:58:59.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:59.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995686+0000", 2024-09-14T23:58:59.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6649999999999992e-06 2024-09-14T23:58:59.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:59.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995691+0000", 2024-09-14T23:58:59.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0289999999999996e-06 2024-09-14T23:58:59.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996199+0000", 2024-09-14T23:58:59.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00050799099999999995 2024-09-14T23:58:59.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:59.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996205+0000", 2024-09-14T23:58:59.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8259999999999998e-06 2024-09-14T23:58:59.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.066238+0000", 2024-09-14T23:58:59.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.070032928 2024-09-14T23:58:59.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:59.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.066393+0000", 2024-09-14T23:58:59.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015549800000000001 2024-09-14T23:58:59.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:59.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.066553+0000", 2024-09-14T23:58:59.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015955 2024-09-14T23:58:59.486 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.486 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:59.486 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.486 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187223 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1604608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:59.486 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.995728+0000", 2024-09-14T23:58:59.486 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.404188576999999, 2024-09-14T23:58:59.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.071486478, 2024-09-14T23:58:59.486 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:59.486 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:59.486 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:59.486 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:59.487 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:59.487 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:59.487 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187223 2024-09-14T23:58:59.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.487 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:59.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:59.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995728+0000", 2024-09-14T23:58:59.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:59.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995728+0000", 2024-09-14T23:58:59.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:59.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995728+0000", 2024-09-14T23:58:59.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.65e-07 2024-09-14T23:58:59.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:59.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995730+0000", 2024-09-14T23:58:59.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.406e-06 2024-09-14T23:58:59.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:59.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995734+0000", 2024-09-14T23:58:59.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2640000000000002e-06 2024-09-14T23:58:59.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:59.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995737+0000", 2024-09-14T23:58:59.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0589999999999998e-06 2024-09-14T23:58:59.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996288+0000", 2024-09-14T23:58:59.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00055059599999999998 2024-09-14T23:58:59.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:59.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996294+0000", 2024-09-14T23:58:59.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0260000000000004e-06 2024-09-14T23:58:59.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.066938+0000", 2024-09-14T23:58:59.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0706439510000001 2024-09-14T23:58:59.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:59.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067050+0000", 2024-09-14T23:58:59.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011220100000000001 2024-09-14T23:58:59.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:59.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067215+0000", 2024-09-14T23:58:59.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016470999999999999 2024-09-14T23:58:59.491 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.491 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:59.491 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.491 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187224 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1619968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:59.491 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.995742+0000", 2024-09-14T23:58:59.491 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.404174761999997, 2024-09-14T23:58:59.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0717603609999999, 2024-09-14T23:58:59.491 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:59.491 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:59.492 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:59.492 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:59.492 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:59.492 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:59.492 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187224 2024-09-14T23:58:59.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.492 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:59.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:59.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995742+0000", 2024-09-14T23:58:59.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:59.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995742+0000", 2024-09-14T23:58:59.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:59.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995742+0000", 2024-09-14T23:58:59.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2399999999999999e-07 2024-09-14T23:58:59.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:59.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995744+0000", 2024-09-14T23:58:59.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.356e-06 2024-09-14T23:58:59.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:59.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995747+0000", 2024-09-14T23:58:59.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.304e-06 2024-09-14T23:58:59.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:59.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995750+0000", 2024-09-14T23:58:59.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6510000000000001e-06 2024-09-14T23:58:59.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996327+0000", 2024-09-14T23:58:59.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00057794300000000003 2024-09-14T23:58:59.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:59.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996337+0000", 2024-09-14T23:58:59.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3759999999999997e-06 2024-09-14T23:58:59.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067244+0000", 2024-09-14T23:58:59.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.070907367 2024-09-14T23:58:59.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:59.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067341+0000", 2024-09-14T23:58:59.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7277999999999999e-05 2024-09-14T23:58:59.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:59.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067502+0000", 2024-09-14T23:58:59.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016086200000000001 2024-09-14T23:58:59.496 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.496 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:59.496 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.496 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187225 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1608704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:59.496 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.995766+0000", 2024-09-14T23:58:59.496 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.404150588, 2024-09-14T23:58:59.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.072043649, 2024-09-14T23:58:59.497 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:59.497 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:59.497 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:59.497 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:59.497 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:59.497 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:59.497 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187225 2024-09-14T23:58:59.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.497 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:59.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:59.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995766+0000", 2024-09-14T23:58:59.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.498 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:59.498 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995766+0000", 2024-09-14T23:58:59.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.498 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:59.498 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995766+0000", 2024-09-14T23:58:59.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9200000000000002e-07 2024-09-14T23:58:59.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.498 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:59.498 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995768+0000", 2024-09-14T23:58:59.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8679999999999999e-06 2024-09-14T23:58:59.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.499 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:59.499 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995772+0000", 2024-09-14T23:58:59.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6339999999999999e-06 2024-09-14T23:58:59.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.499 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:59.499 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995775+0000", 2024-09-14T23:58:59.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8159999999999998e-06 2024-09-14T23:58:59.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.499 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996372+0000", 2024-09-14T23:58:59.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00059758800000000003 2024-09-14T23:58:59.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:59.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996377+0000", 2024-09-14T23:58:59.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7770000000000002e-06 2024-09-14T23:58:59.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067534+0000", 2024-09-14T23:58:59.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.071156821 2024-09-14T23:58:59.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:59.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067638+0000", 2024-09-14T23:58:59.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000103737 2024-09-14T23:58:59.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:59.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067810+0000", 2024-09-14T23:58:59.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017211599999999999 2024-09-14T23:58:59.501 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.501 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:59.501 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.502 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187226 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1624064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:59.502 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.995778+0000", 2024-09-14T23:58:59.502 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.404138408999998, 2024-09-14T23:58:59.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.072339081, 2024-09-14T23:58:59.502 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:59.502 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:59.502 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:59.502 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:59.502 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:59.502 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:59.502 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187226 2024-09-14T23:58:59.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.503 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:59.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:59.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995778+0000", 2024-09-14T23:58:59.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:59.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995778+0000", 2024-09-14T23:58:59.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:59.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995779+0000", 2024-09-14T23:58:59.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0699999999999999e-07 2024-09-14T23:58:59.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.504 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:59.504 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995780+0000", 2024-09-14T23:58:59.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2139999999999999e-06 2024-09-14T23:58:59.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.504 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:59.504 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995783+0000", 2024-09-14T23:58:59.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1250000000000001e-06 2024-09-14T23:58:59.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.504 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:59.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995787+0000", 2024-09-14T23:58:59.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6679999999999999e-06 2024-09-14T23:58:59.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996404+0000", 2024-09-14T23:58:59.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00061777499999999999 2024-09-14T23:58:59.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:59.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996414+0000", 2024-09-14T23:58:59.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.431e-06 2024-09-14T23:58:59.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067844+0000", 2024-09-14T23:58:59.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0714302410000001 2024-09-14T23:58:59.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:59.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067952+0000", 2024-09-14T23:58:59.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000107778 2024-09-14T23:58:59.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:59.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.068117+0000", 2024-09-14T23:58:59.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000165642 2024-09-14T23:58:59.506 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.506 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:59.507 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.507 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187227 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1612800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:58:59.507 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.995806+0000", 2024-09-14T23:58:59.507 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.404111172, 2024-09-14T23:58:59.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0726092810000001, 2024-09-14T23:58:59.507 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:59.507 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:59.507 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:58:59.507 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:59.507 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:58:59.507 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:58:59.507 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187227 2024-09-14T23:58:59.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.508 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:59.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:59.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995806+0000", 2024-09-14T23:58:59.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:59.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995806+0000", 2024-09-14T23:58:59.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:59.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995672+0000", 2024-09-14T23:58:59.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99986608099999996 2024-09-14T23:58:59.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:59.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995677+0000", 2024-09-14T23:58:59.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9520000000000002e-06 2024-09-14T23:58:59.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:59.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995686+0000", 2024-09-14T23:58:59.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8199999999999992e-06 2024-09-14T23:58:59.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:59.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995693+0000", 2024-09-14T23:58:59.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1630000000000001e-06 2024-09-14T23:58:59.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.068149+0000", 2024-09-14T23:58:59.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.072456692000000003 2024-09-14T23:58:59.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:59.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.068251+0000", 2024-09-14T23:58:59.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101963 2024-09-14T23:58:59.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:59.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.068415+0000", 2024-09-14T23:58:59.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016360999999999999 2024-09-14T23:58:59.511 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.511 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:59.511 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.511 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.511 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:59.511 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:59.513 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T23:58:59.513 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:58:59.511+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:58:59.613 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:58:59.612+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:58:59.662 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:59.662 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:58:59.662 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:58:59.662 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:59.670 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T23:58:59.716 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:59.715+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:59.777 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:59.777 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:58:59.777 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:58:59.777 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:58:59.777 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:58:59.786 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T23:58:59.828 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:58:59.826+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:58:59.915 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:58:59.914+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:58:59.937 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:58:59.937 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:58:59.937 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:58:59.937 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:58:59.937 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.937 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4284.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-14T23:58:59.937 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:58:59.937 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 157.410323336, 2024-09-14T23:58:59.937 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.004942086, 2024-09-14T23:58:59.937 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:59.938 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:59.938 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:59.938 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:59.938 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4284", 2024-09-14T23:58:59.938 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/4278347926", 2024-09-14T23:58:59.938 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1 2024-09-14T23:58:59.938 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.938 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:59.938 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.938 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:59.938 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:58:59.938 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:58:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524331+0000", 2024-09-14T23:58:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.383e-05 2024-09-14T23:58:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:59.939 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524339+0000", 2024-09-14T23:58:59.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8290000000000008e-06 2024-09-14T23:58:59.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:59.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524359+0000", 2024-09-14T23:58:59.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9998000000000001e-05 2024-09-14T23:58:59.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:59.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524373+0000", 2024-09-14T23:58:59.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4304000000000001e-05 2024-09-14T23:58:59.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.941 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524488+0000", 2024-09-14T23:58:59.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114663 2024-09-14T23:58:59.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.941 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:58:59.941 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524501+0000", 2024-09-14T23:58:59.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3349999999999999e-05 2024-09-14T23:58:59.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.941 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.941 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.526868+0000", 2024-09-14T23:58:59.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023670090000000002 2024-09-14T23:58:59.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.942 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:59.942 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.527041+0000", 2024-09-14T23:58:59.942 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000172597 2024-09-14T23:58:59.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.942 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.942 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-14T23:58:59.942 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.527384+0000", 2024-09-14T23:58:59.942 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00034309500000000002 2024-09-14T23:58:59.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.942 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.942 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529103+0000", 2024-09-14T23:58:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017185220000000001 2024-09-14T23:58:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:58:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529188+0000", 2024-09-14T23:58:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5591000000000001e-05 2024-09-14T23:58:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529227+0000", 2024-09-14T23:58:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8563000000000003e-05 2024-09-14T23:58:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529259+0000", 2024-09-14T23:58:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2734999999999997e-05 2024-09-14T23:58:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4284.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-14T23:58:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:23.530683+0000", 2024-09-14T23:58:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 156.403957671, 2024-09-14T23:58:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0036133110000000001, 2024-09-14T23:58:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:59.944 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:59.945 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:59.945 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:58:59.945 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4284", 2024-09-14T23:58:59.945 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/4278347926", 2024-09-14T23:58:59.945 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5 2024-09-14T23:58:59.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.945 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:59.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:59.945 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530683+0000", 2024-09-14T23:58:59.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530683+0000", 2024-09-14T23:58:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530685+0000", 2024-09-14T23:58:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.075e-06 2024-09-14T23:58:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530695+0000", 2024-09-14T23:58:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0261999999999999e-05 2024-09-14T23:58:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530715+0000", 2024-09-14T23:58:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9930000000000001e-05 2024-09-14T23:58:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530724+0000", 2024-09-14T23:58:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.901e-06 2024-09-14T23:58:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530800+0000", 2024-09-14T23:58:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5873999999999999e-05 2024-09-14T23:58:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.948 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-14T23:58:59.948 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530807+0000", 2024-09-14T23:58:59.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2389999999999996e-06 2024-09-14T23:58:59.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.948 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.948 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.532494+0000", 2024-09-14T23:58:59.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016871830000000001 2024-09-14T23:58:59.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.948 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:59.949 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.532599+0000", 2024-09-14T23:58:59.949 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010472 2024-09-14T23:58:59.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1", 2024-09-14T23:58:59.949 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.532715+0000", 2024-09-14T23:58:59.949 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011568199999999999 2024-09-14T23:58:59.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:58:59.949 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.533460+0000", 2024-09-14T23:58:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00074494999999999997 2024-09-14T23:58:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:58:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534230+0000", 2024-09-14T23:58:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00077036599999999998 2024-09-14T23:58:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534266+0000", 2024-09-14T23:58:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5725000000000001e-05 2024-09-14T23:58:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.950 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:59.951 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534296+0000", 2024-09-14T23:58:59.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0403999999999999e-05 2024-09-14T23:58:59.951 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.951 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:59.951 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.951 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56383 3.c e56/49 3:36b698b0:::rbd_data.1118c4b3aebe.0000000000000000:head v 56'194, mlcod=56'192)", 2024-09-14T23:58:59.951 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.270483+0000", 2024-09-14T23:58:59.951 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 113.664157871, 2024-09-14T23:58:59.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0079685880000000004, 2024-09-14T23:58:59.951 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:59.951 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:59.951 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:59.952 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:59.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.952 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:59.952 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.270483+0000", 2024-09-14T23:58:59.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.952 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:59.952 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.270483+0000", 2024-09-14T23:58:59.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.270484+0000", 2024-09-14T23:58:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7400000000000002e-07 2024-09-14T23:58:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273680+0000", 2024-09-14T23:58:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031966680000000002 2024-09-14T23:58:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.954 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.954 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:59.954 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273697+0000", 2024-09-14T23:58:59.954 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6595e-05 2024-09-14T23:58:59.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273705+0000", 2024-09-14T23:58:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7530000000000001e-06 2024-09-14T23:58:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273763+0000", 2024-09-14T23:58:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8238e-05 2024-09-14T23:58:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273781+0000", 2024-09-14T23:58:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8281e-05 2024-09-14T23:58:59.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:59.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.278408+0000", 2024-09-14T23:58:59.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00462716 2024-09-14T23:58:59.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:59.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.278451+0000", 2024-09-14T23:58:59.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3019000000000003e-05 2024-09-14T23:58:59.956 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.956 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:59.956 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.957 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56384 3.c e56/49 3:36b698b0:::rbd_data.1118c4b3aebe.0000000000000000:head v 56'196, mlcod=56'192)", 2024-09-14T23:58:59.957 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:58:59.957 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 113.66012519900001, 2024-09-14T23:58:59.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0046707529999999997, 2024-09-14T23:58:59.957 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:59.957 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:59.957 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:59.957 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:59.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.958 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:59.958 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:58:59.958 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.958 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.958 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:59.958 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:58:59.958 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.958 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.958 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:59.958 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:58:59.958 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1699999999999997e-07 2024-09-14T23:58:59.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.959 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.959 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:59.959 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274523+0000", 2024-09-14T23:58:59.959 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9129999999999997e-06 2024-09-14T23:58:59.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.959 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.959 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:59.959 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274535+0000", 2024-09-14T23:58:59.959 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1788e-05 2024-09-14T23:58:59.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.960 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:59.960 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274542+0000", 2024-09-14T23:58:59.960 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0539999999999997e-06 2024-09-14T23:58:59.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.960 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.960 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275889+0000", 2024-09-14T23:58:59.960 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00134669 2024-09-14T23:58:59.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:59.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275917+0000", 2024-09-14T23:58:59.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7664000000000001e-05 2024-09-14T23:58:59.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:59.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279134+0000", 2024-09-14T23:58:59.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0032178459999999999 2024-09-14T23:58:59.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279186+0000", 2024-09-14T23:58:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1881e-05 2024-09-14T23:58:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56381 3.1d e56/49 3:bcf9a8cf:::rbd_data.1118c4b3aebe.0000000000000006:head v 56'210, mlcod=56'208)", 2024-09-14T23:58:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.275462+0000", 2024-09-14T23:58:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 113.659178869, 2024-09-14T23:58:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0071310710000000001, 2024-09-14T23:58:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:59.962 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:59.963 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:59.963 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:59.963 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.963 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:59.963 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275462+0000", 2024-09-14T23:58:59.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.963 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.963 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:59.963 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275462+0000", 2024-09-14T23:58:59.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.963 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.963 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:59.964 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275463+0000", 2024-09-14T23:58:59.964 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2699999999999999e-07 2024-09-14T23:58:59.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.964 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:59.964 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276779+0000", 2024-09-14T23:58:59.964 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0013163649999999999 2024-09-14T23:58:59.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.964 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:59.964 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276795+0000", 2024-09-14T23:58:59.964 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.59e-05 2024-09-14T23:58:59.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.965 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:59.965 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276806+0000", 2024-09-14T23:58:59.965 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1106e-05 2024-09-14T23:58:59.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.965 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.965 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.965 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276941+0000", 2024-09-14T23:58:59.965 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013477500000000001 2024-09-14T23:58:59.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.965 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.965 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:59.965 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276963+0000", 2024-09-14T23:58:59.965 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2419999999999999e-05 2024-09-14T23:58:59.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.965 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:59.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282546+0000", 2024-09-14T23:58:59.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0055827100000000003 2024-09-14T23:58:59.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:59.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282593+0000", 2024-09-14T23:58:59.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7067999999999997e-05 2024-09-14T23:58:59.966 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.966 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:59.966 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.966 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56387 3.c e56/49 3:30ee4ba0:::rbd_data.1118c4b3aebe.0000000000000004:head v 56'197, mlcod=56'192)", 2024-09-14T23:58:59.967 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.276498+0000", 2024-09-14T23:58:59.967 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 113.658142774, 2024-09-14T23:58:59.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0092192639999999996, 2024-09-14T23:58:59.967 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:59.967 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:59.967 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:59.967 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:59.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.967 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:59.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276498+0000", 2024-09-14T23:58:59.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.968 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:59.968 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276498+0000", 2024-09-14T23:58:59.968 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.968 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.968 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.968 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:59.968 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276499+0000", 2024-09-14T23:58:59.968 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1100000000000004e-07 2024-09-14T23:58:59.968 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.968 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.968 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:59.968 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279646+0000", 2024-09-14T23:58:59.968 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031473220000000001 2024-09-14T23:58:59.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.969 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.969 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:59.969 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279672+0000", 2024-09-14T23:58:59.969 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6129999999999999e-05 2024-09-14T23:58:59.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.969 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.969 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:59.969 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279682+0000", 2024-09-14T23:58:59.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7170000000000003e-06 2024-09-14T23:58:59.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.970 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.970 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279742+0000", 2024-09-14T23:58:59.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9926999999999997e-05 2024-09-14T23:58:59.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.970 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:59.970 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279761+0000", 2024-09-14T23:58:59.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9402000000000002e-05 2024-09-14T23:58:59.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.970 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:59.970 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.285680+0000", 2024-09-14T23:58:59.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.005918881 2024-09-14T23:58:59.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.971 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:59.971 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.285717+0000", 2024-09-14T23:58:59.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6974000000000002e-05 2024-09-14T23:58:59.971 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.971 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:59.971 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.971 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56388 3.c e56/49 3:30ee4ba0:::rbd_data.1118c4b3aebe.0000000000000004:head v 56'199, mlcod=56'192)", 2024-09-14T23:58:59.971 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:58:59.971 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 113.654651186, 2024-09-14T23:58:59.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0060596740000000001, 2024-09-14T23:58:59.972 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:59.972 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:59.972 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:59.972 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:59.972 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.972 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:59.972 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:58:59.972 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.972 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.972 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:59.972 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:58:59.972 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.973 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:59.973 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:58:59.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2200000000000003e-07 2024-09-14T23:58:59.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.973 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:59.973 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279995+0000", 2024-09-14T23:58:59.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5569999999999997e-06 2024-09-14T23:58:59.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.973 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280004+0000", 2024-09-14T23:58:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0080000000000008e-06 2024-09-14T23:58:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280010+0000", 2024-09-14T23:58:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5749999999999997e-06 2024-09-14T23:58:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282968+0000", 2024-09-14T23:58:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0029579390000000001 2024-09-14T23:58:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.974 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:59.975 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282998+0000", 2024-09-14T23:58:59.975 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9947999999999999e-05 2024-09-14T23:58:59.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.975 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.975 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:59.975 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.286012+0000", 2024-09-14T23:58:59.975 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030132900000000001 2024-09-14T23:58:59.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.975 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.975 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:59.975 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.286049+0000", 2024-09-14T23:58:59.975 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7534999999999999e-05 2024-09-14T23:58:59.975 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.975 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:59.976 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.976 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56382 3.1d e56/49 3:bcf9a8cf:::rbd_data.1118c4b3aebe.0000000000000006:head v 56'212, mlcod=56'208)", 2024-09-14T23:58:59.976 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.280285+0000", 2024-09-14T23:58:59.976 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 113.65435550300001, 2024-09-14T23:58:59.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035393489999999998, 2024-09-14T23:58:59.976 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:59.976 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:59.976 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:59.976 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:59.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.976 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:59.976 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280285+0000", 2024-09-14T23:58:59.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.977 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:59.977 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280285+0000", 2024-09-14T23:58:59.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.977 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:59.977 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280286+0000", 2024-09-14T23:58:59.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5000000000000001e-07 2024-09-14T23:58:59.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.978 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:59.978 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280299+0000", 2024-09-14T23:58:59.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3312000000000001e-05 2024-09-14T23:58:59.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.979 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:59.979 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280311+0000", 2024-09-14T23:58:59.979 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1677e-05 2024-09-14T23:58:59.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.979 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.979 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:59.980 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280318+0000", 2024-09-14T23:58:59.980 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.742e-06 2024-09-14T23:58:59.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.980 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.980 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280374+0000", 2024-09-14T23:58:59.981 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.592e-05 2024-09-14T23:58:59.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.981 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.981 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:59.981 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280392+0000", 2024-09-14T23:58:59.981 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8490000000000001e-05 2024-09-14T23:58:59.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.982 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:59.982 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.283693+0000", 2024-09-14T23:58:59.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033012050000000002 2024-09-14T23:58:59.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.983 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:59.983 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.283825+0000", 2024-09-14T23:58:59.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013105300000000001 2024-09-14T23:58:59.983 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.983 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:59.983 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.984 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "pg_update_log_missing(3.c epoch 56/49 rep_tid 1493 entries 56'200 (0'0) error 3:370203a3:::rbd_data.1118c4b3aebe.000000000000000d:head by client.4376.0:56396 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 56'192)", 2024-09-14T23:58:59.984 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:58:59.984 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 113.654131046, 2024-09-14T23:58:59.984 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057370099999999999, 2024-09-14T23:58:59.984 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:59.985 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:59.985 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:58:59.985 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:59.985 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.985 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:59.985 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:58:59.986 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.986 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:59.986 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:58:59.986 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.987 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:59.987 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:58:59.987 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0699999999999997e-07 2024-09-14T23:58:59.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.988 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.988 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:59.988 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280519+0000", 2024-09-14T23:58:59.988 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6260000000000001e-06 2024-09-14T23:58:59.988 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.989 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.989 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:59.989 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280541+0000", 2024-09-14T23:58:59.989 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1976000000000001e-05 2024-09-14T23:58:59.989 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.989 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.989 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:59.990 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280556+0000", 2024-09-14T23:58:59.990 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5075000000000001e-05 2024-09-14T23:58:59.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.990 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.990 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.990 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.285494+0000", 2024-09-14T23:58:59.991 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0049378690000000001 2024-09-14T23:58:59.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.991 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.991 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:59.991 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.286247+0000", 2024-09-14T23:58:59.991 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00075285700000000001 2024-09-14T23:58:59.991 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.992 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:59.992 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.992 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.992 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.992 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56400 3.1c e56/49 3:3f5b81ba:::rbd_data.1118c4b3aebe.000000000000000a:head v 56'236, mlcod=56'234)", 2024-09-14T23:58:59.992 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.295064+0000", 2024-09-14T23:58:59.993 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 113.639576254, 2024-09-14T23:58:59.993 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0054331589999999999, 2024-09-14T23:58:59.993 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:58:59.993 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:58:59.993 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:58:59.993 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:58:59.994 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.994 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:58:59.994 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.295064+0000", 2024-09-14T23:58:59.994 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.995 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.995 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:58:59.995 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.295064+0000", 2024-09-14T23:58:59.995 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:58:59.995 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.995 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.995 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:58:59.996 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.295066+0000", 2024-09-14T23:58:59.996 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.502e-06 2024-09-14T23:58:59.996 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.996 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.996 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:58:59.996 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296403+0000", 2024-09-14T23:58:59.997 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001336554 2024-09-14T23:58:59.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.997 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.997 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:58:59.997 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296428+0000", 2024-09-14T23:58:59.997 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5179000000000001e-05 2024-09-14T23:58:59.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.997 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.997 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:58:59.998 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296440+0000", 2024-09-14T23:58:59.998 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2387e-05 2024-09-14T23:58:59.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.998 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:58:59.998 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296554+0000", 2024-09-14T23:58:59.998 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114269 2024-09-14T23:58:59.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.998 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:58:59.998 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296581+0000", 2024-09-14T23:58:59.998 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6673999999999999e-05 2024-09-14T23:58:59.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.999 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:58:59.999 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.300418+0000", 2024-09-14T23:58:59.999 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0038371249999999998 2024-09-14T23:58:59.999 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.999 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.999 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:58:59.999 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.300498+0000", 2024-09-14T23:58:59.999 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9468999999999994e-05 2024-09-14T23:58:59.999 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.999 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:58:59.999 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:58:59.999 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:58:59.999 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:58:59.999 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:61922 3.5 e57/49 3:a11ebf8e:::rbd_data.11188d356229.000000000000000c:head v 57'280, mlcod=57'277)", 2024-09-14T23:59:00.000 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:07.563144+0000", 2024-09-14T23:59:00.000 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 112.37149660199999, 2024-09-14T23:59:00.000 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003977848, 2024-09-14T23:59:00.000 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:00.000 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:00.000 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:00.000 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:00.000 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.000 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:00.000 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563144+0000", 2024-09-14T23:59:00.000 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.000 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.000 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.000 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:00.001 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563144+0000", 2024-09-14T23:59:00.001 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:00.001 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563145+0000", 2024-09-14T23:59:00.001 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5400000000000001e-07 2024-09-14T23:59:00.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:00.001 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565056+0000", 2024-09-14T23:59:00.001 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0019108160000000001 2024-09-14T23:59:00.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.002 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:00.002 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565088+0000", 2024-09-14T23:59:00.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2586999999999998e-05 2024-09-14T23:59:00.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.002 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:00.002 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565100+0000", 2024-09-14T23:59:00.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1917e-05 2024-09-14T23:59:00.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.002 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:00.002 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565166+0000", 2024-09-14T23:59:00.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6245e-05 2024-09-14T23:59:00.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.003 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:00.003 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565190+0000", 2024-09-14T23:59:00.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3825999999999999e-05 2024-09-14T23:59:00.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.003 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:00.003 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.567080+0000", 2024-09-14T23:59:00.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001889541 2024-09-14T23:59:00.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:00.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.567122+0000", 2024-09-14T23:59:00.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2262000000000003e-05 2024-09-14T23:59:00.004 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.004 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:00.004 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.004 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.004 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:66564 3.c e60/49 3:33faa7ca:::rbd_data.111873792d9b.0000000000000003:head v 60'236, mlcod=57'233)", 2024-09-14T23:59:00.004 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:11.271596+0000", 2024-09-14T23:59:00.004 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 108.663045024, 2024-09-14T23:59:00.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0060396099999999999, 2024-09-14T23:59:00.005 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:00.005 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:00.005 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:00.005 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:00.005 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.005 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:00.005 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.271596+0000", 2024-09-14T23:59:00.005 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.005 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.005 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:00.005 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.271596+0000", 2024-09-14T23:59:00.005 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.005 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.006 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:00.006 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.271597+0000", 2024-09-14T23:59:00.006 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2559999999999999e-06 2024-09-14T23:59:00.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.006 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:00.006 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.274854+0000", 2024-09-14T23:59:00.006 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0032574470000000001 2024-09-14T23:59:00.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.006 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:00.006 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.274874+0000", 2024-09-14T23:59:00.006 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9449999999999998e-05 2024-09-14T23:59:00.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.007 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:00.007 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.274883+0000", 2024-09-14T23:59:00.007 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5980000000000003e-06 2024-09-14T23:59:00.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.007 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.007 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:00.007 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.274977+0000", 2024-09-14T23:59:00.007 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3270999999999995e-05 2024-09-14T23:59:00.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.007 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.007 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:00.007 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.275000+0000", 2024-09-14T23:59:00.007 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3088e-05 2024-09-14T23:59:00.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:00.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.277590+0000", 2024-09-14T23:59:00.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0025898660000000001 2024-09-14T23:59:00.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:00.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.277635+0000", 2024-09-14T23:59:00.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5633999999999997e-05 2024-09-14T23:59:00.008 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.008 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:00.008 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.009 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:133384 3.1f e92/49 3:fce5e529:::rbd_data.11186132a8b6.0000000000000002:head v 92'807, mlcod=89'804)", 2024-09-14T23:59:00.009 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:45.293642+0000", 2024-09-14T23:59:00.009 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 74.640999140999995, 2024-09-14T23:59:00.009 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003760186, 2024-09-14T23:59:00.009 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:00.009 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:00.009 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:00.009 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:00.009 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.009 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:00.009 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.293642+0000", 2024-09-14T23:59:00.009 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.009 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.010 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:00.010 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.293642+0000", 2024-09-14T23:59:00.010 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.010 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.010 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:00.010 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.293643+0000", 2024-09-14T23:59:00.010 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2500000000000001e-06 2024-09-14T23:59:00.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.010 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.010 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:00.011 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.295402+0000", 2024-09-14T23:59:00.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017595410000000001 2024-09-14T23:59:00.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.011 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.011 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:00.011 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.295422+0000", 2024-09-14T23:59:00.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9763999999999999e-05 2024-09-14T23:59:00.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.011 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.011 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:00.011 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.295436+0000", 2024-09-14T23:59:00.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.364e-05 2024-09-14T23:59:00.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.011 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.012 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:00.012 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.295499+0000", 2024-09-14T23:59:00.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2770999999999999e-05 2024-09-14T23:59:00.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.012 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:00.012 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.295516+0000", 2024-09-14T23:59:00.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7240000000000001e-05 2024-09-14T23:59:00.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.012 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:00.012 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.297324+0000", 2024-09-14T23:59:00.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001807857 2024-09-14T23:59:00.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.013 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:00.013 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.297402+0000", 2024-09-14T23:59:00.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8122999999999997e-05 2024-09-14T23:59:00.013 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.013 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:00.013 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.013 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:156597 3.5 e103/49 3:a23c5924:::rbd_data.1118b2f495e6.0000000000000011:head v 103'518, mlcod=98'516)", 2024-09-14T23:59:00.013 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:58.656197+0000", 2024-09-14T23:59:00.013 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 61.278444084, 2024-09-14T23:59:00.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0040423029999999997, 2024-09-14T23:59:00.013 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:00.014 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:00.014 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:00.014 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:00.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.014 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:00.014 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656197+0000", 2024-09-14T23:59:00.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.014 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:00.014 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656197+0000", 2024-09-14T23:59:00.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.015 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:00.015 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656197+0000", 2024-09-14T23:59:00.015 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1900000000000002e-07 2024-09-14T23:59:00.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.015 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:00.015 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656676+0000", 2024-09-14T23:59:00.015 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00047921699999999999 2024-09-14T23:59:00.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.015 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:00.015 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656690+0000", 2024-09-14T23:59:00.015 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3916e-05 2024-09-14T23:59:00.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.016 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:00.016 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656698+0000", 2024-09-14T23:59:00.016 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6529999999999994e-06 2024-09-14T23:59:00.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.016 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.016 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:00.016 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656757+0000", 2024-09-14T23:59:00.016 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9237999999999998e-05 2024-09-14T23:59:00.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.016 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.016 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:00.016 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656780+0000", 2024-09-14T23:59:00.016 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2968000000000001e-05 2024-09-14T23:59:00.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.017 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.017 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:00.017 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.660188+0000", 2024-09-14T23:59:00.017 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0034078630000000001 2024-09-14T23:59:00.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.017 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.017 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:00.017 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.660239+0000", 2024-09-14T23:59:00.017 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0828999999999997e-05 2024-09-14T23:59:00.017 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.017 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:00.017 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.018 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:157947 3.1c e103/49 3:383374fe:::rbd_data.1118b2f495e6.000000000000000c:head v 103'771, mlcod=103'769)", 2024-09-14T23:59:00.018 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:58.765271+0000", 2024-09-14T23:59:00.018 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 61.169369203000002, 2024-09-14T23:59:00.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0040869829999999998, 2024-09-14T23:59:00.018 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:00.018 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:00.018 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:00.018 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:00.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.018 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:00.018 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765271+0000", 2024-09-14T23:59:00.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:00.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765271+0000", 2024-09-14T23:59:00.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:00.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765273+0000", 2024-09-14T23:59:00.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1149999999999999e-06 2024-09-14T23:59:00.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:00.020 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765854+0000", 2024-09-14T23:59:00.020 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00058119599999999997 2024-09-14T23:59:00.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.020 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:00.020 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765869+0000", 2024-09-14T23:59:00.020 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5641000000000002e-05 2024-09-14T23:59:00.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.020 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:00.020 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765878+0000", 2024-09-14T23:59:00.020 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4519999999999997e-06 2024-09-14T23:59:00.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.021 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:00.021 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765942+0000", 2024-09-14T23:59:00.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3727999999999997e-05 2024-09-14T23:59:00.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.021 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:00.021 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765968+0000", 2024-09-14T23:59:00.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6171999999999999e-05 2024-09-14T23:59:00.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.021 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:00.021 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.769319+0000", 2024-09-14T23:59:00.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033511629999999999 2024-09-14T23:59:00.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.022 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:00.022 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.769358+0000", 2024-09-14T23:59:00.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9515999999999999e-05 2024-09-14T23:59:00.022 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.022 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:00.022 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.022 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:160196 3.c e103/49 3:34623e68:::rbd_data.1118b2f495e6.0000000000000005:head v 103'551, mlcod=102'549)", 2024-09-14T23:59:00.022 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:59.131433+0000", 2024-09-14T23:59:00.022 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 60.803207894000003, 2024-09-14T23:59:00.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0036932530000000001, 2024-09-14T23:59:00.022 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:00.022 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:00.023 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:00.023 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:00.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.023 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:00.023 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.131433+0000", 2024-09-14T23:59:00.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.023 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:00.023 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.131433+0000", 2024-09-14T23:59:00.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.023 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:00.024 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.131433+0000", 2024-09-14T23:59:00.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8500000000000001e-07 2024-09-14T23:59:00.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.024 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:00.024 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.132144+0000", 2024-09-14T23:59:00.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00071022800000000003 2024-09-14T23:59:00.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.024 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:00.024 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.132154+0000", 2024-09-14T23:59:00.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0301e-05 2024-09-14T23:59:00.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:00.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.132159+0000", 2024-09-14T23:59:00.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6280000000000004e-06 2024-09-14T23:59:00.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:00.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.132208+0000", 2024-09-14T23:59:00.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9595000000000002e-05 2024-09-14T23:59:00.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:00.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.132231+0000", 2024-09-14T23:59:00.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3010000000000002e-05 2024-09-14T23:59:00.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:00.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.135087+0000", 2024-09-14T23:59:00.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0028553239999999998 2024-09-14T23:59:00.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:00.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.135126+0000", 2024-09-14T23:59:00.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9481999999999999e-05 2024-09-14T23:59:00.026 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.026 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:00.026 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.027 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188871 3.d 3:b35cff4d:::rbd_header.1118c900944f:head [watch reconnect cookie 140670715433840 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-14T23:59:00.027 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.970653+0000", 2024-09-14T23:59:00.027 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.963988161000003, 2024-09-14T23:59:00.027 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0727808270000001, 2024-09-14T23:59:00.027 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:00.027 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:00.027 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:00.027 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:00.027 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:00.027 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:00.028 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188871 2024-09-14T23:59:00.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.028 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:00.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.028 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:00.028 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970653+0000", 2024-09-14T23:59:00.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.028 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:00.028 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970653+0000", 2024-09-14T23:59:00.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:00.029 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970655+0000", 2024-09-14T23:59:00.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.277e-06 2024-09-14T23:59:00.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:00.029 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970662+0000", 2024-09-14T23:59:00.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9619999999999996e-06 2024-09-14T23:59:00.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:00.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970682+0000", 2024-09-14T23:59:00.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0279999999999999e-05 2024-09-14T23:59:00.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:00.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970691+0000", 2024-09-14T23:59:00.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2790000000000004e-06 2024-09-14T23:59:00.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:00.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970762+0000", 2024-09-14T23:59:00.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1019999999999994e-05 2024-09-14T23:59:00.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-14T23:59:00.031 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970770+0000", 2024-09-14T23:59:00.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7999999999999999e-06 2024-09-14T23:59:00.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.031 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:00.031 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.973516+0000", 2024-09-14T23:59:00.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0027456469999999999 2024-09-14T23:59:00.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.031 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:00.031 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.973525+0000", 2024-09-14T23:59:00.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4890000000000001e-06 2024-09-14T23:59:00.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.031 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:00.032 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.041451+0000", 2024-09-14T23:59:00.032 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.067926012 2024-09-14T23:59:00.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:00.032 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.041673+0000", 2024-09-14T23:59:00.032 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000222048 2024-09-14T23:59:00.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:00.032 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.043433+0000", 2024-09-14T23:59:00.032 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017600129999999999 2024-09-14T23:59:00.032 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.032 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:00.032 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.033 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(osd.0.0:3238 3.1f e122/49 3:f8a59e83:::rbd_data.11187b9f1032.0000000000000006:2f v 122'943, mlcod=120'940)", 2024-09-14T23:59:00.033 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:22.285283+0000", 2024-09-14T23:59:00.033 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 37.649357338000002, 2024-09-14T23:59:00.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003756194, 2024-09-14T23:59:00.033 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:00.033 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:00.033 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:00.033 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:00.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.033 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:00.033 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.285283+0000", 2024-09-14T23:59:00.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.034 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:00.034 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.285283+0000", 2024-09-14T23:59:00.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.034 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:00.034 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.285284+0000", 2024-09-14T23:59:00.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5600000000000002e-07 2024-09-14T23:59:00.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.034 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:00.034 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.285289+0000", 2024-09-14T23:59:00.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9579999999999998e-06 2024-09-14T23:59:00.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:00.035 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.285300+0000", 2024-09-14T23:59:00.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1688999999999999e-05 2024-09-14T23:59:00.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:00.035 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.285309+0000", 2024-09-14T23:59:00.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.388e-06 2024-09-14T23:59:00.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:00.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.285367+0000", 2024-09-14T23:59:00.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8811000000000003e-05 2024-09-14T23:59:00.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:00.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.285388+0000", 2024-09-14T23:59:00.037 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0117000000000001e-05 2024-09-14T23:59:00.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.037 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.037 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:00.037 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.288994+0000", 2024-09-14T23:59:00.037 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0036064270000000002 2024-09-14T23:59:00.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.038 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.038 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:00.038 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.289040+0000", 2024-09-14T23:59:00.038 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5547999999999998e-05 2024-09-14T23:59:00.038 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.039 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:00.039 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.039 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(osd.6.0:1517 3.d e127/112 3:b69c12aa:::rbd_data.1118f9c96a7.0000000000000007:2c v 127'452, mlcod=126'449)", 2024-09-14T23:59:00.039 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:27.316914+0000", 2024-09-14T23:59:00.039 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 32.617726730000001, 2024-09-14T23:59:00.040 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0052453980000000001, 2024-09-14T23:59:00.040 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:00.040 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:00.040 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:00.040 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:00.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.040 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:00.040 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.316914+0000", 2024-09-14T23:59:00.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.041 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.041 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:00.041 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.316914+0000", 2024-09-14T23:59:00.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.042 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.042 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:00.042 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.316914+0000", 2024-09-14T23:59:00.042 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4900000000000002e-07 2024-09-14T23:59:00.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.042 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.042 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:00.043 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.316920+0000", 2024-09-14T23:59:00.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7169999999999996e-06 2024-09-14T23:59:00.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.043 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:00.043 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.316928+0000", 2024-09-14T23:59:00.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3769999999999996e-06 2024-09-14T23:59:00.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.044 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:00.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.316938+0000", 2024-09-14T23:59:00.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7329999999999998e-06 2024-09-14T23:59:00.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.045 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.045 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:00.045 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.317033+0000", 2024-09-14T23:59:00.045 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5295000000000002e-05 2024-09-14T23:59:00.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.045 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.045 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:00.045 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.317057+0000", 2024-09-14T23:59:00.045 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3989999999999999e-05 2024-09-14T23:59:00.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.046 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.046 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:00.046 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.322122+0000", 2024-09-14T23:59:00.046 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0050642500000000002 2024-09-14T23:59:00.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.046 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.046 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:00.046 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.322159+0000", 2024-09-14T23:59:00.046 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7787000000000003e-05 2024-09-14T23:59:00.046 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.046 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:00.046 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.046 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.046 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(osd.1.0:1466 3.16 e127/112 3:6c7910cf:::rbd_data.1118f9c96a7.0000000000000006:2c v 127'376, mlcod=121'372)", 2024-09-14T23:59:00.047 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:27.317464+0000", 2024-09-14T23:59:00.047 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 32.617176696999998, 2024-09-14T23:59:00.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0050844499999999999, 2024-09-14T23:59:00.047 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:00.047 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:00.047 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:00.047 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:00.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.047 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:00.047 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.317464+0000", 2024-09-14T23:59:00.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.047 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:00.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.317464+0000", 2024-09-14T23:59:00.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.048 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.048 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:00.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.317464+0000", 2024-09-14T23:59:00.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.91e-07 2024-09-14T23:59:00.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.048 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.048 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:00.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.317469+0000", 2024-09-14T23:59:00.049 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4719999999999997e-06 2024-09-14T23:59:00.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.049 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.049 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:00.049 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.317481+0000", 2024-09-14T23:59:00.049 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1814000000000001e-05 2024-09-14T23:59:00.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.049 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.049 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:00.049 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.317488+0000", 2024-09-14T23:59:00.049 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2740000000000002e-06 2024-09-14T23:59:00.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.049 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.049 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:00.050 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.317584+0000", 2024-09-14T23:59:00.050 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6191000000000001e-05 2024-09-14T23:59:00.050 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.050 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.050 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:00.050 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.317605+0000", 2024-09-14T23:59:00.050 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0554e-05 2024-09-14T23:59:00.050 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.050 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.050 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:00.050 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.322504+0000", 2024-09-14T23:59:00.050 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0048991089999999996 2024-09-14T23:59:00.050 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.051 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.051 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:00.051 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.322548+0000", 2024-09-14T23:59:00.051 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4744999999999999e-05 2024-09-14T23:59:00.051 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.051 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:00.051 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.051 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.051 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:00.051 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:59:00.051 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 6, 5, 7] out_osds: [3] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T23:59:00.051 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T23:59:00.051 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-14T23:59:00.052 INFO:tasks.thrashosds.thrasher:Setting osd 2 primary_affinity to 0.000000 2024-09-14T23:59:00.052 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 0 2024-09-14T23:59:00.053 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:59:00.052+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:59:00.054 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T23:59:00.154 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:59:00.153+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:59:00.183 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:59:00.184 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:59:00.184 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:59:00.184 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:59:00.193 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T23:59:00.255 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T23:59:00.256 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:59:00.254+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:59:00.301 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:59:00.301 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:59:00.301 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:59:00.302 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:59:00.302 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:59:00.310 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T23:59:00.357 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:59:00.355+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:59:00.380 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:59:00.381 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:59:00.398 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:59:00.398 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:59:00.414 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:59:00.414 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:59:00.420 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:59:00.420 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:59:00.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:59:00.420 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:59:00.420 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.420 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:107472 3.1e 3:7bf21588:::rbd_data.11187de8b0a5.000000000000005d:head [set-alloc-hint object_size 2097152 write_size 2097152,write 317788~1491] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-14T23:59:00.420 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:25.198493+0000", 2024-09-14T23:59:00.420 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 95.219500597000007, 2024-09-14T23:59:00.421 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.78175149300000002, 2024-09-14T23:59:00.421 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:00.421 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:00.421 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:00.421 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:00.421 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:00.421 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:00.421 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 107472 2024-09-14T23:59:00.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.421 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:00.421 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.421 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:00.421 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198493+0000", 2024-09-14T23:59:00.421 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.422 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.422 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:00.422 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198493+0000", 2024-09-14T23:59:00.422 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.422 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.422 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:00.422 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198495+0000", 2024-09-14T23:59:00.422 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.826e-06 2024-09-14T23:59:00.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.422 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.422 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:00.422 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198503+0000", 2024-09-14T23:59:00.423 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0150000000000003e-06 2024-09-14T23:59:00.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.423 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:00.423 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198521+0000", 2024-09-14T23:59:00.423 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8374000000000001e-05 2024-09-14T23:59:00.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.423 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:00.423 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198533+0000", 2024-09-14T23:59:00.423 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1593000000000001e-05 2024-09-14T23:59:00.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:00.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198603+0000", 2024-09-14T23:59:00.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0279999999999998e-05 2024-09-14T23:59:00.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:00.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198617+0000", 2024-09-14T23:59:00.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3517e-05 2024-09-14T23:59:00.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:00.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.977067+0000", 2024-09-14T23:59:00.425 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.77845065800000002 2024-09-14T23:59:00.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.425 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for missing object", 2024-09-14T23:59:00.425 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.977173+0000", 2024-09-14T23:59:00.425 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010543999999999999 2024-09-14T23:59:00.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.425 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:00.425 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.978737+0000", 2024-09-14T23:59:00.425 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001564078 2024-09-14T23:59:00.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.425 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:00.426 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.978845+0000", 2024-09-14T23:59:00.426 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000108643 2024-09-14T23:59:00.426 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.426 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.426 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1", 2024-09-14T23:59:00.426 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.978964+0000", 2024-09-14T23:59:00.426 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000119085 2024-09-14T23:59:00.426 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.426 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.426 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:59:00.426 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980109+0000", 2024-09-14T23:59:00.426 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0011442170000000001 2024-09-14T23:59:00.426 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.426 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.427 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:59:00.427 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980130+0000", 2024-09-14T23:59:00.427 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1766000000000001e-05 2024-09-14T23:59:00.427 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.427 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.427 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:00.427 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980171+0000", 2024-09-14T23:59:00.427 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0627999999999998e-05 2024-09-14T23:59:00.427 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.427 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.427 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:00.427 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980244+0000", 2024-09-14T23:59:00.427 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3373000000000004e-05 2024-09-14T23:59:00.428 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.428 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:00.428 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.428 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.428 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.428 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129159 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 5996544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:00.428 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.927783+0000", 2024-09-14T23:59:00.428 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.490210705999999, 2024-09-14T23:59:00.428 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054661122999999999, 2024-09-14T23:59:00.428 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:00.428 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:00.428 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:00.428 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:00.429 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:00.429 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:00.429 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129159 2024-09-14T23:59:00.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.429 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:00.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.429 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:00.429 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.927783+0000", 2024-09-14T23:59:00.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.429 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:00.429 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.927783+0000", 2024-09-14T23:59:00.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.430 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:00.430 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929684+0000", 2024-09-14T23:59:00.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0019011080000000001 2024-09-14T23:59:00.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.430 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:00.430 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929688+0000", 2024-09-14T23:59:00.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2610000000000004e-06 2024-09-14T23:59:00.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.430 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:00.430 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929697+0000", 2024-09-14T23:59:00.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0219999999999993e-06 2024-09-14T23:59:00.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.431 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.431 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:00.431 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929702+0000", 2024-09-14T23:59:00.431 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8840000000000002e-06 2024-09-14T23:59:00.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.431 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.431 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:00.431 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.982090+0000", 2024-09-14T23:59:00.431 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052388045000000001 2024-09-14T23:59:00.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.431 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.431 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:00.431 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.982208+0000", 2024-09-14T23:59:00.432 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118236 2024-09-14T23:59:00.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.432 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:00.432 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.982444+0000", 2024-09-14T23:59:00.432 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000235567 2024-09-14T23:59:00.432 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.432 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:00.432 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.432 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129166 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 8192000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:00.432 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.929794+0000", 2024-09-14T23:59:00.432 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.488199954999999, 2024-09-14T23:59:00.433 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054471973, 2024-09-14T23:59:00.433 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:00.433 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:00.433 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:00.433 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:00.433 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:00.433 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:00.433 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129166 2024-09-14T23:59:00.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.433 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:00.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.433 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:00.433 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929794+0000", 2024-09-14T23:59:00.433 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.434 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:00.434 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929794+0000", 2024-09-14T23:59:00.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.434 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:00.434 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929794+0000", 2024-09-14T23:59:00.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6899999999999997e-07 2024-09-14T23:59:00.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.434 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:00.434 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929797+0000", 2024-09-14T23:59:00.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.328e-06 2024-09-14T23:59:00.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.435 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:00.435 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929802+0000", 2024-09-14T23:59:00.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7309999999999998e-06 2024-09-14T23:59:00.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.435 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:00.435 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929807+0000", 2024-09-14T23:59:00.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4349999999999999e-06 2024-09-14T23:59:00.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.435 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:00.435 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984063+0000", 2024-09-14T23:59:00.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054256331999999997 2024-09-14T23:59:00.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.436 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:00.436 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984177+0000", 2024-09-14T23:59:00.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114225 2024-09-14T23:59:00.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.436 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:00.436 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984266+0000", 2024-09-14T23:59:00.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8352999999999999e-05 2024-09-14T23:59:00.437 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.437 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:00.437 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.437 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129167 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 6782976~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:00.437 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.929827+0000", 2024-09-14T23:59:00.437 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.488166546000002, 2024-09-14T23:59:00.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054637593999999998, 2024-09-14T23:59:00.437 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:00.437 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:00.437 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:00.437 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:00.438 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:00.438 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:00.438 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129167 2024-09-14T23:59:00.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.438 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:00.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:00.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929827+0000", 2024-09-14T23:59:00.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:00.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929827+0000", 2024-09-14T23:59:00.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.439 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:00.439 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929827+0000", 2024-09-14T23:59:00.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4000000000000002e-07 2024-09-14T23:59:00.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.439 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:00.439 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929830+0000", 2024-09-14T23:59:00.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.419e-06 2024-09-14T23:59:00.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.439 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:00.439 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929837+0000", 2024-09-14T23:59:00.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0450000000000003e-06 2024-09-14T23:59:00.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.440 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:00.440 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929842+0000", 2024-09-14T23:59:00.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.69e-06 2024-09-14T23:59:00.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.440 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:00.440 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984278+0000", 2024-09-14T23:59:00.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054436516999999997 2024-09-14T23:59:00.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:00.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984331+0000", 2024-09-14T23:59:00.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2590999999999998e-05 2024-09-14T23:59:00.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:00.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984465+0000", 2024-09-14T23:59:00.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013389200000000001 2024-09-14T23:59:00.441 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.441 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:00.441 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.441 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129168 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 8224768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:00.442 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.929849+0000", 2024-09-14T23:59:00.442 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.488144988000002, 2024-09-14T23:59:00.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054870368000000003, 2024-09-14T23:59:00.442 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:00.442 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:00.442 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:00.442 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:00.442 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:00.442 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:00.442 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129168 2024-09-14T23:59:00.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.442 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:00.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:00.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929849+0000", 2024-09-14T23:59:00.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.443 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:00.443 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929849+0000", 2024-09-14T23:59:00.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.443 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:00.443 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931681+0000", 2024-09-14T23:59:00.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018326989999999999 2024-09-14T23:59:00.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.443 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:00.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931686+0000", 2024-09-14T23:59:00.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7190000000000001e-06 2024-09-14T23:59:00.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:00.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931697+0000", 2024-09-14T23:59:00.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1287999999999999e-05 2024-09-14T23:59:00.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:00.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931704+0000", 2024-09-14T23:59:00.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3339999999999998e-06 2024-09-14T23:59:00.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:00.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984505+0000", 2024-09-14T23:59:00.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052801684000000002 2024-09-14T23:59:00.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:00.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984621+0000", 2024-09-14T23:59:00.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000116165 2024-09-14T23:59:00.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:00.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984719+0000", 2024-09-14T23:59:00.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7479000000000006e-05 2024-09-14T23:59:00.446 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.446 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:00.446 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.446 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129176 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 10452992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:00.446 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.931830+0000", 2024-09-14T23:59:00.446 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.486163227000006, 2024-09-14T23:59:00.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054513292999999997, 2024-09-14T23:59:00.446 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:00.446 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:00.446 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:00.446 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:00.446 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:00.447 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:00.447 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129176 2024-09-14T23:59:00.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.447 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:00.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:00.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931830+0000", 2024-09-14T23:59:00.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:00.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931830+0000", 2024-09-14T23:59:00.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:00.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.933678+0000", 2024-09-14T23:59:00.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018472440000000001 2024-09-14T23:59:00.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:00.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.933686+0000", 2024-09-14T23:59:00.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0239999999999997e-06 2024-09-14T23:59:00.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:00.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.933694+0000", 2024-09-14T23:59:00.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8400000000000001e-06 2024-09-14T23:59:00.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:00.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.933701+0000", 2024-09-14T23:59:00.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2870000000000001e-06 2024-09-14T23:59:00.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:00.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.986165+0000", 2024-09-14T23:59:00.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052463900000000001 2024-09-14T23:59:00.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:00.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.986259+0000", 2024-09-14T23:59:00.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4435e-05 2024-09-14T23:59:00.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.450 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:00.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.986344+0000", 2024-09-14T23:59:00.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4562999999999997e-05 2024-09-14T23:59:00.450 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.450 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:00.450 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.450 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129201 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 10289152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:00.450 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.935960+0000", 2024-09-14T23:59:00.451 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.482033764999997, 2024-09-14T23:59:00.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054338487999999997, 2024-09-14T23:59:00.451 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:00.451 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:00.451 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:00.451 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:00.451 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:00.451 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:00.451 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129201 2024-09-14T23:59:00.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.451 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:00.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:00.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935960+0000", 2024-09-14T23:59:00.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:00.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935960+0000", 2024-09-14T23:59:00.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:00.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935960+0000", 2024-09-14T23:59:00.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.72e-07 2024-09-14T23:59:00.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:00.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935963+0000", 2024-09-14T23:59:00.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5390000000000002e-06 2024-09-14T23:59:00.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:00.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935971+0000", 2024-09-14T23:59:00.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4589999999999998e-06 2024-09-14T23:59:00.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:00.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935977+0000", 2024-09-14T23:59:00.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4110000000000004e-06 2024-09-14T23:59:00.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:00.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990213+0000", 2024-09-14T23:59:00.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054236642000000002 2024-09-14T23:59:00.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:00.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990257+0000", 2024-09-14T23:59:00.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3696000000000003e-05 2024-09-14T23:59:00.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:00.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990298+0000", 2024-09-14T23:59:00.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1369000000000003e-05 2024-09-14T23:59:00.454 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.454 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:00.455 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.455 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129202 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 11370496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:00.455 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.935986+0000", 2024-09-14T23:59:00.455 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.482007147000004, 2024-09-14T23:59:00.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054543700000000001, 2024-09-14T23:59:00.455 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:00.455 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:00.455 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:00.455 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:00.455 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:00.455 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:00.455 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129202 2024-09-14T23:59:00.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.456 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:00.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.456 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:00.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935986+0000", 2024-09-14T23:59:00.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.456 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:00.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935986+0000", 2024-09-14T23:59:00.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.456 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:00.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937678+0000", 2024-09-14T23:59:00.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016917849999999999 2024-09-14T23:59:00.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:00.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937682+0000", 2024-09-14T23:59:00.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2400000000000001e-06 2024-09-14T23:59:00.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:00.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937695+0000", 2024-09-14T23:59:00.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2999e-05 2024-09-14T23:59:00.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:00.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937700+0000", 2024-09-14T23:59:00.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8420000000000004e-06 2024-09-14T23:59:00.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:00.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990336+0000", 2024-09-14T23:59:00.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052636060999999998 2024-09-14T23:59:00.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:00.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990435+0000", 2024-09-14T23:59:00.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9015e-05 2024-09-14T23:59:00.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:00.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990530+0000", 2024-09-14T23:59:00.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4758000000000003e-05 2024-09-14T23:59:00.459 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.459 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:00.459 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.459 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129214 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 11468800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:00.459 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.937939+0000", 2024-09-14T23:59:00.459 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.480054233000004, 2024-09-14T23:59:00.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054463733, 2024-09-14T23:59:00.459 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:00.459 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:00.460 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:00.460 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:00.460 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:00.460 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:00.460 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129214 2024-09-14T23:59:00.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.460 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:00.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:00.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937939+0000", 2024-09-14T23:59:00.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:00.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937939+0000", 2024-09-14T23:59:00.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:00.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939674+0000", 2024-09-14T23:59:00.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017342589999999999 2024-09-14T23:59:00.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:00.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939678+0000", 2024-09-14T23:59:00.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2590000000000002e-06 2024-09-14T23:59:00.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:00.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939699+0000", 2024-09-14T23:59:00.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1143000000000001e-05 2024-09-14T23:59:00.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:00.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939706+0000", 2024-09-14T23:59:00.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0269999999999998e-06 2024-09-14T23:59:00.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:00.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.992221+0000", 2024-09-14T23:59:00.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052514981000000002 2024-09-14T23:59:00.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:00.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.992321+0000", 2024-09-14T23:59:00.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9625999999999998e-05 2024-09-14T23:59:00.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:00.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.992403+0000", 2024-09-14T23:59:00.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2437999999999999e-05 2024-09-14T23:59:00.464 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.464 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:00.464 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.464 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129226 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 14090240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:00.464 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.939867+0000", 2024-09-14T23:59:00.464 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.478126305000004, 2024-09-14T23:59:00.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054523573999999998, 2024-09-14T23:59:00.464 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:00.464 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:00.464 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:00.465 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:00.465 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:00.465 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:00.465 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129226 2024-09-14T23:59:00.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.465 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:00.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:00.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939867+0000", 2024-09-14T23:59:00.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:00.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939867+0000", 2024-09-14T23:59:00.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:00.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941669+0000", 2024-09-14T23:59:00.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018021549999999999 2024-09-14T23:59:00.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:00.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941673+0000", 2024-09-14T23:59:00.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1269999999999999e-06 2024-09-14T23:59:00.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:00.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941679+0000", 2024-09-14T23:59:00.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1879999999999999e-06 2024-09-14T23:59:00.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:00.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941682+0000", 2024-09-14T23:59:00.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7110000000000001e-06 2024-09-14T23:59:00.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:00.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.994201+0000", 2024-09-14T23:59:00.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052518265000000001 2024-09-14T23:59:00.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:00.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.994302+0000", 2024-09-14T23:59:00.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101653 2024-09-14T23:59:00.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:00.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.994391+0000", 2024-09-14T23:59:00.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8474999999999996e-05 2024-09-14T23:59:00.468 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.468 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:00.468 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.469 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129237 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 12386304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:00.469 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.941825+0000", 2024-09-14T23:59:00.469 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.476168543, 2024-09-14T23:59:00.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054368076000000001, 2024-09-14T23:59:00.469 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:00.469 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:00.469 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:00.469 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:00.469 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:00.469 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:00.469 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129237 2024-09-14T23:59:00.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.469 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:00.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:00.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941825+0000", 2024-09-14T23:59:00.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:00.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941825+0000", 2024-09-14T23:59:00.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:00.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941825+0000", 2024-09-14T23:59:00.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.29e-07 2024-09-14T23:59:00.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:00.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941826+0000", 2024-09-14T23:59:00.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0999999999999997e-07 2024-09-14T23:59:00.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:00.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941834+0000", 2024-09-14T23:59:00.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5519999999999999e-06 2024-09-14T23:59:00.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:00.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941843+0000", 2024-09-14T23:59:00.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2960000000000004e-06 2024-09-14T23:59:00.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:00.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996108+0000", 2024-09-14T23:59:00.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054265200999999999 2024-09-14T23:59:00.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:00.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996152+0000", 2024-09-14T23:59:00.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4066999999999997e-05 2024-09-14T23:59:00.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:00.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996193+0000", 2024-09-14T23:59:00.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0821000000000001e-05 2024-09-14T23:59:00.473 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.473 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:00.473 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.473 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129238 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 13828096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:00.473 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.941851+0000", 2024-09-14T23:59:00.473 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.476142218999996, 2024-09-14T23:59:00.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.05458031, 2024-09-14T23:59:00.473 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:00.473 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:00.473 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:00.473 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:00.473 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:00.474 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:00.474 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129238 2024-09-14T23:59:00.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.474 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:00.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:00.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941851+0000", 2024-09-14T23:59:00.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:00.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941851+0000", 2024-09-14T23:59:00.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:00.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943670+0000", 2024-09-14T23:59:00.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018181930000000001 2024-09-14T23:59:00.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:00.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943673+0000", 2024-09-14T23:59:00.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3790000000000002e-06 2024-09-14T23:59:00.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:00.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943679+0000", 2024-09-14T23:59:00.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8719999999999999e-06 2024-09-14T23:59:00.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:00.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943683+0000", 2024-09-14T23:59:00.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8539999999999999e-06 2024-09-14T23:59:00.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:00.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996233+0000", 2024-09-14T23:59:00.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052550690999999997 2024-09-14T23:59:00.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:00.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996339+0000", 2024-09-14T23:59:00.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105989 2024-09-14T23:59:00.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:00.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996432+0000", 2024-09-14T23:59:00.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2331999999999996e-05 2024-09-14T23:59:00.477 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.477 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:00.477 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.477 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129249 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 12124160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:00.477 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.943826+0000", 2024-09-14T23:59:00.477 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.474167727999998, 2024-09-14T23:59:00.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054373266000000003, 2024-09-14T23:59:00.478 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:00.478 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:00.478 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:00.478 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:00.478 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:00.478 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:00.478 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129249 2024-09-14T23:59:00.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.478 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:00.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:00.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943826+0000", 2024-09-14T23:59:00.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:00.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943826+0000", 2024-09-14T23:59:00.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:00.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943826+0000", 2024-09-14T23:59:00.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.03e-07 2024-09-14T23:59:00.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:00.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943827+0000", 2024-09-14T23:59:00.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.08e-06 2024-09-14T23:59:00.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:00.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943834+0000", 2024-09-14T23:59:00.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4119999999999998e-06 2024-09-14T23:59:00.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:00.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943836+0000", 2024-09-14T23:59:00.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7369999999999998e-06 2024-09-14T23:59:00.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:00.481 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998115+0000", 2024-09-14T23:59:00.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054279040000000001 2024-09-14T23:59:00.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:00.481 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998159+0000", 2024-09-14T23:59:00.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3486999999999999e-05 2024-09-14T23:59:00.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:00.481 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998199+0000", 2024-09-14T23:59:00.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0306999999999999e-05 2024-09-14T23:59:00.481 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.481 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:00.482 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.482 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129250 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 13565952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:00.482 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.943840+0000", 2024-09-14T23:59:00.482 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.474153778000002, 2024-09-14T23:59:00.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054573875000000001, 2024-09-14T23:59:00.482 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:00.482 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:00.482 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:00.482 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:00.482 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:00.482 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:00.482 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129250 2024-09-14T23:59:00.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.483 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:00.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:00.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943840+0000", 2024-09-14T23:59:00.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:00.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943840+0000", 2024-09-14T23:59:00.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:00.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945669+0000", 2024-09-14T23:59:00.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018289720000000001 2024-09-14T23:59:00.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:00.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945671+0000", 2024-09-14T23:59:00.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6869999999999999e-06 2024-09-14T23:59:00.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:00.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945680+0000", 2024-09-14T23:59:00.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5259999999999993e-06 2024-09-14T23:59:00.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:00.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945685+0000", 2024-09-14T23:59:00.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8559999999999998e-06 2024-09-14T23:59:00.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:00.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998242+0000", 2024-09-14T23:59:00.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052557505999999997 2024-09-14T23:59:00.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:00.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998329+0000", 2024-09-14T23:59:00.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6410999999999996e-05 2024-09-14T23:59:00.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:00.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998414+0000", 2024-09-14T23:59:00.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4917e-05 2024-09-14T23:59:00.486 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.486 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:00.486 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.486 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129262 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 13303808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:00.487 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.945825+0000", 2024-09-14T23:59:00.487 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.472168836999998, 2024-09-14T23:59:00.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054515942999999997, 2024-09-14T23:59:00.487 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:00.487 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:00.487 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:00.487 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:00.487 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:00.487 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:00.487 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129262 2024-09-14T23:59:00.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.487 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:00.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:00.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945825+0000", 2024-09-14T23:59:00.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:00.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945825+0000", 2024-09-14T23:59:00.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:00.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.947670+0000", 2024-09-14T23:59:00.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018451229999999999 2024-09-14T23:59:00.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:00.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.947672+0000", 2024-09-14T23:59:00.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5289999999999998e-06 2024-09-14T23:59:00.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:00.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.947678+0000", 2024-09-14T23:59:00.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9880000000000001e-06 2024-09-14T23:59:00.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:00.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.947682+0000", 2024-09-14T23:59:00.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.63e-06 2024-09-14T23:59:00.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:00.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000183+0000", 2024-09-14T23:59:00.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052500770000000002 2024-09-14T23:59:00.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:00.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000266+0000", 2024-09-14T23:59:00.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3133000000000005e-05 2024-09-14T23:59:00.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:00.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000341+0000", 2024-09-14T23:59:00.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4770000000000004e-05 2024-09-14T23:59:00.490 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.491 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:00.491 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.491 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129299 3.10 3:0f1e06ee:::rbd_object_map.1118721be852:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:00.491 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.952330+0000", 2024-09-14T23:59:00.491 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.465664024000006, 2024-09-14T23:59:00.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054412818000000002, 2024-09-14T23:59:00.491 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:00.491 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:00.491 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:00.491 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:00.491 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:00.491 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:00.492 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129299 2024-09-14T23:59:00.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.492 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:00.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:00.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952330+0000", 2024-09-14T23:59:00.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:00.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952330+0000", 2024-09-14T23:59:00.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:00.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952330+0000", 2024-09-14T23:59:00.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4900000000000001e-07 2024-09-14T23:59:00.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:00.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952332+0000", 2024-09-14T23:59:00.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2879999999999999e-06 2024-09-14T23:59:00.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:00.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952340+0000", 2024-09-14T23:59:00.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9910000000000002e-06 2024-09-14T23:59:00.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:00.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952344+0000", 2024-09-14T23:59:00.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0280000000000002e-06 2024-09-14T23:59:00.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:00.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.004722+0000", 2024-09-14T23:59:00.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052377932000000002 2024-09-14T23:59:00.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:00.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.004778+0000", 2024-09-14T23:59:00.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5795999999999999e-05 2024-09-14T23:59:00.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:59:00.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.005016+0000", 2024-09-14T23:59:00.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000238006 2024-09-14T23:59:00.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:59:00.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.006366+0000", 2024-09-14T23:59:00.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001349786 2024-09-14T23:59:00.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:59:00.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.006685+0000", 2024-09-14T23:59:00.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00031970000000000002 2024-09-14T23:59:00.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:00.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.006718+0000", 2024-09-14T23:59:00.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2329000000000002e-05 2024-09-14T23:59:00.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:00.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.006742+0000", 2024-09-14T23:59:00.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4613e-05 2024-09-14T23:59:00.496 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.496 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:00.496 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.497 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186210 3.d 3.b2ff3acd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:59:00.497 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.129044+0000", 2024-09-14T23:59:00.497 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 53.288949660999997, 2024-09-14T23:59:00.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.84555443900000005, 2024-09-14T23:59:00.497 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:00.497 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:00.497 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:59:00.497 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:00.497 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:00.497 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:00.497 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186210 2024-09-14T23:59:00.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.498 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:00.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.498 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:00.498 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129044+0000", 2024-09-14T23:59:00.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.498 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:00.498 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129044+0000", 2024-09-14T23:59:00.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.498 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:00.499 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129046+0000", 2024-09-14T23:59:00.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.742e-06 2024-09-14T23:59:00.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.499 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:00.499 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129052+0000", 2024-09-14T23:59:00.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7390000000000002e-06 2024-09-14T23:59:00.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.499 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:00.499 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129067+0000", 2024-09-14T23:59:00.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.417e-05 2024-09-14T23:59:00.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.499 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:00.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129076+0000", 2024-09-14T23:59:00.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2289999999999992e-06 2024-09-14T23:59:00.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:00.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.130947+0000", 2024-09-14T23:59:00.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001871075 2024-09-14T23:59:00.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:00.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.130955+0000", 2024-09-14T23:59:00.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4500000000000004e-06 2024-09-14T23:59:00.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:00.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974587+0000", 2024-09-14T23:59:00.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.84363205500000005 2024-09-14T23:59:00.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:00.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974598+0000", 2024-09-14T23:59:00.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0978999999999999e-05 2024-09-14T23:59:00.501 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.501 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:00.501 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.501 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187415 3.d 3.b2ff3acd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:59:00.502 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.254681+0000", 2024-09-14T23:59:00.502 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 53.163312140000002, 2024-09-14T23:59:00.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.71995530699999999, 2024-09-14T23:59:00.502 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:00.502 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:00.502 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:59:00.502 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:00.502 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:00.502 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:00.502 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187415 2024-09-14T23:59:00.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.502 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:00.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:00.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254681+0000", 2024-09-14T23:59:00.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:00.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254681+0000", 2024-09-14T23:59:00.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:00.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254682+0000", 2024-09-14T23:59:00.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2e-07 2024-09-14T23:59:00.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.504 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:00.504 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254684+0000", 2024-09-14T23:59:00.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2730000000000001e-06 2024-09-14T23:59:00.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.504 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:00.504 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254689+0000", 2024-09-14T23:59:00.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4769999999999997e-06 2024-09-14T23:59:00.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.504 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:00.504 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254692+0000", 2024-09-14T23:59:00.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9890000000000001e-06 2024-09-14T23:59:00.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:00.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254721+0000", 2024-09-14T23:59:00.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9762999999999999e-05 2024-09-14T23:59:00.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:00.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254725+0000", 2024-09-14T23:59:00.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9650000000000002e-06 2024-09-14T23:59:00.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:00.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974628+0000", 2024-09-14T23:59:00.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.71990235499999999 2024-09-14T23:59:00.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:00.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974637+0000", 2024-09-14T23:59:00.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0650000000000005e-06 2024-09-14T23:59:00.506 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.506 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:00.506 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.507 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187457 3.d 3.b2ff3acd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:59:00.507 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.257980+0000", 2024-09-14T23:59:00.507 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 53.160013566000003, 2024-09-14T23:59:00.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.71671711199999999, 2024-09-14T23:59:00.507 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:00.507 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:00.507 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:59:00.507 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:00.507 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:00.507 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:00.507 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187457 2024-09-14T23:59:00.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.507 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:00.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:00.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257980+0000", 2024-09-14T23:59:00.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:00.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257980+0000", 2024-09-14T23:59:00.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:00.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257980+0000", 2024-09-14T23:59:00.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.17e-07 2024-09-14T23:59:00.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:00.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257981+0000", 2024-09-14T23:59:00.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.172e-06 2024-09-14T23:59:00.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:00.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257984+0000", 2024-09-14T23:59:00.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9550000000000001e-06 2024-09-14T23:59:00.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:00.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257987+0000", 2024-09-14T23:59:00.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5799999999999999e-06 2024-09-14T23:59:00.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:00.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.258030+0000", 2024-09-14T23:59:00.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3118999999999998e-05 2024-09-14T23:59:00.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:00.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.258038+0000", 2024-09-14T23:59:00.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6850000000000001e-06 2024-09-14T23:59:00.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:00.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974688+0000", 2024-09-14T23:59:00.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.71665016000000004 2024-09-14T23:59:00.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:00.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974697+0000", 2024-09-14T23:59:00.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2240000000000001e-06 2024-09-14T23:59:00.511 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.511 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:00.511 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.511 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:251656 3.c 3:32436721:::rbd_object_map.1118e09062ff:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-14T23:59:00.511 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:27.433419+0000", 2024-09-14T23:59:00.512 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 32.984574891999998, 2024-09-14T23:59:00.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.55083879499999999, 2024-09-14T23:59:00.512 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:00.512 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:00.512 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:00.512 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:00.512 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:00.512 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:00.512 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 251656 2024-09-14T23:59:00.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.512 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:00.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.512 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:00.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433419+0000", 2024-09-14T23:59:00.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:00.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433419+0000", 2024-09-14T23:59:00.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:00.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:00.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433419+0000", 2024-09-14T23:59:00.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1500000000000003e-07 2024-09-14T23:59:00.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:00.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433424+0000", 2024-09-14T23:59:00.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2409999999999997e-06 2024-09-14T23:59:00.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:00.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433435+0000", 2024-09-14T23:59:00.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1009000000000001e-05 2024-09-14T23:59:00.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:00.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433440+0000", 2024-09-14T23:59:00.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1950000000000002e-06 2024-09-14T23:59:00.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:00.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433487+0000", 2024-09-14T23:59:00.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7184000000000001e-05 2024-09-14T23:59:00.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:00.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433494+0000", 2024-09-14T23:59:00.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5490000000000003e-06 2024-09-14T23:59:00.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:00.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.981923+0000", 2024-09-14T23:59:00.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.54842898600000001 2024-09-14T23:59:00.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.516 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:00.516 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.982097+0000", 2024-09-14T23:59:00.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017428699999999999 2024-09-14T23:59:00.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.516 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5", 2024-09-14T23:59:00.516 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.982334+0000", 2024-09-14T23:59:00.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00023700899999999999 2024-09-14T23:59:00.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.516 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:59:00.516 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.983927+0000", 2024-09-14T23:59:00.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0015932710000000001 2024-09-14T23:59:00.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:59:00.517 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.984128+0000", 2024-09-14T23:59:00.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020104199999999999 2024-09-14T23:59:00.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:00.517 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.984212+0000", 2024-09-14T23:59:00.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3709999999999996e-05 2024-09-14T23:59:00.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:00.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:00.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:00.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.984257+0000", 2024-09-14T23:59:00.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5497000000000002e-05 2024-09-14T23:59:00.518 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.518 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:00.518 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.518 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:00.518 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:00.518 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:59:00.519 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:59:00.519 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:59:00.520 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:59:00.520 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:59:00.520 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T23:59:00.521 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:59:00.518+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:59:00.521 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:59:00.521 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:59:00.522 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:59:00.522 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:59:00.522 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:59:00.523 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:59:00.620 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:59:00.619+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:59:00.668 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:00.668 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:00.677 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T23:59:00.721 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:59:00.720+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:59:00.821 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:59:00.820+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:59:00.824 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:00.824 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:00.833 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T23:59:00.922 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:59:00.921+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:59:00.982 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:00.982 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:00.990 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T23:59:01.034 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:59:01.033+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:59:01.094 INFO:teuthology.orchestra.run.smithi042.stderr:set osd.2 primary-affinity to 0 (802) 2024-09-14T23:59:01.099 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T23:59:01.119 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:01.119 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:01.123 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:59:01.122+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:59:01.131 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T23:59:01.224 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:59:01.223+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:59:01.283 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:01.284 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:01.293 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T23:59:01.336 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:59:01.335+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:59:01.400 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:01.400 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:01.409 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T23:59:01.450 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:59:01.448+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:59:01.527 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:59:01.526+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:59:01.556 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:01.556 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:01.565 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T23:59:01.629 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:59:01.627+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:59:01.672 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:01.672 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:01.681 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T23:59:01.765 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:59:01.764+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:59:01.829 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:01.830 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:01.830 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:59:01.829+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:59:01.838 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T23:59:01.930 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:59:01.929+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:59:01.986 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:01.987 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:01.995 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T23:59:02.036 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:59:02.035+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:59:02.104 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is unset 2024-09-14T23:59:02.133 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:59:02.131+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:59:02.156 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:02.157 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:02.168 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T23:59:02.250 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:59:02.249+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:59:02.314 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:02.314 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:02.323 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T23:59:02.364 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:59:02.363+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:59:02.435 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:59:02.433+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:59:02.471 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:02.471 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:02.480 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T23:59:02.524 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T23:59:02.563 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:59:02.562+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:59:02.627 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:02.627 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:02.636 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T23:59:02.637 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:59:02.635+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:59:02.688 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:59:02.688 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:59:02.706 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:59:02.707 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:59:02.723 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:59:02.777 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:59:02.778 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:02.778 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:02.779 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T23:59:02.780 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:59:02.779+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:59:02.881 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:59:02.880+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:59:02.899 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:02.900 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:02.909 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T23:59:02.940 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:59:02.941 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:59:02.957 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:59:02.958 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:59:02.974 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:59:02.974 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:59:02.982 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:59:02.980+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:59:02.997 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:59:02.997 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:59:03.015 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:59:03.015 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:59:03.077 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:59:03.077 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:59:03.077 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:59:03.077 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:59:03.077 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:59:03.082 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:59:03.081+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:59:03.086 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T23:59:03.183 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:59:03.182+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:59:03.197 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:59:03.197 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:59:03.198 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:59:03.198 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:59:03.198 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.198 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186203 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1329152~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:59:03.198 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.194279+0000", 2024-09-14T23:59:03.198 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 56.000840824000001, 2024-09-14T23:59:03.198 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8011469710000001, 2024-09-14T23:59:03.198 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:03.198 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:03.198 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:03.198 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:03.198 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:03.199 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:03.199 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186203 2024-09-14T23:59:03.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.199 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:03.199 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.199 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:03.199 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194279+0000", 2024-09-14T23:59:03.199 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.199 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.199 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:03.199 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194279+0000", 2024-09-14T23:59:03.200 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.200 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.200 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.200 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:03.200 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194279+0000", 2024-09-14T23:59:03.200 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8100000000000001e-07 2024-09-14T23:59:03.200 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.201 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.201 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:03.201 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194291+0000", 2024-09-14T23:59:03.201 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1559999999999999e-05 2024-09-14T23:59:03.201 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.201 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.201 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:03.201 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194303+0000", 2024-09-14T23:59:03.202 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2130000000000001e-05 2024-09-14T23:59:03.202 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.202 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.202 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:03.202 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194311+0000", 2024-09-14T23:59:03.202 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8089999999999993e-06 2024-09-14T23:59:03.202 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.202 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.203 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.203 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194370+0000", 2024-09-14T23:59:03.203 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8739999999999997e-05 2024-09-14T23:59:03.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.203 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.203 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:03.203 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194376+0000", 2024-09-14T23:59:03.203 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2319999999999998e-06 2024-09-14T23:59:03.204 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.204 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.204 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.204 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986290+0000", 2024-09-14T23:59:03.204 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.791914334 2024-09-14T23:59:03.204 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.204 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.205 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:03.205 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986296+0000", 2024-09-14T23:59:03.205 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6300000000000003e-06 2024-09-14T23:59:03.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.205 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.205 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.205 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995193+0000", 2024-09-14T23:59:03.206 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0088975040000001 2024-09-14T23:59:03.206 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.206 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.206 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:03.206 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995307+0000", 2024-09-14T23:59:03.206 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000113656 2024-09-14T23:59:03.206 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.206 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.207 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:03.207 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995426+0000", 2024-09-14T23:59:03.207 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118495 2024-09-14T23:59:03.207 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.207 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:03.207 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.207 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.207 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186204 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1391616~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:59:03.207 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.194328+0000", 2024-09-14T23:59:03.207 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 56.000791493000001, 2024-09-14T23:59:03.207 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8013834310000001, 2024-09-14T23:59:03.207 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:03.207 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:03.208 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:03.208 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:03.208 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:03.208 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:03.208 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186204 2024-09-14T23:59:03.208 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.208 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:03.208 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.208 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:03.208 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194328+0000", 2024-09-14T23:59:03.208 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.208 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.208 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.209 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:03.209 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194328+0000", 2024-09-14T23:59:03.209 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.209 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.209 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:03.209 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194328+0000", 2024-09-14T23:59:03.209 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7100000000000002e-07 2024-09-14T23:59:03.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.209 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.209 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:03.209 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194331+0000", 2024-09-14T23:59:03.209 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.807e-06 2024-09-14T23:59:03.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.210 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:03.210 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194339+0000", 2024-09-14T23:59:03.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9340000000000006e-06 2024-09-14T23:59:03.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.210 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:03.210 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194345+0000", 2024-09-14T23:59:03.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1090000000000003e-06 2024-09-14T23:59:03.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.210 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.210 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194412+0000", 2024-09-14T23:59:03.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6229000000000006e-05 2024-09-14T23:59:03.211 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.211 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.211 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:03.211 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194422+0000", 2024-09-14T23:59:03.211 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.078e-05 2024-09-14T23:59:03.211 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.211 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.211 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.211 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986322+0000", 2024-09-14T23:59:03.211 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.79189964899999998 2024-09-14T23:59:03.211 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.211 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.211 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:03.212 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986332+0000", 2024-09-14T23:59:03.212 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0095000000000001e-05 2024-09-14T23:59:03.212 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.212 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.212 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.212 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995554+0000", 2024-09-14T23:59:03.212 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0092218630000001 2024-09-14T23:59:03.212 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.212 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.212 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:03.212 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995610+0000", 2024-09-14T23:59:03.212 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6109000000000001e-05 2024-09-14T23:59:03.212 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.212 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.213 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:03.213 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995711+0000", 2024-09-14T23:59:03.213 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101285 2024-09-14T23:59:03.213 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.213 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:03.213 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.213 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.213 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.213 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186205 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1333248~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:59:03.213 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.194354+0000", 2024-09-14T23:59:03.213 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 56.000765563000002, 2024-09-14T23:59:03.213 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8015471490000001, 2024-09-14T23:59:03.213 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:03.213 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:03.214 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:03.214 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:03.214 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:03.214 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:03.214 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186205 2024-09-14T23:59:03.214 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.214 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:03.214 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.214 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:03.214 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194354+0000", 2024-09-14T23:59:03.214 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.214 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.215 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.215 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:03.215 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194354+0000", 2024-09-14T23:59:03.215 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.215 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.215 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.215 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:03.215 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194354+0000", 2024-09-14T23:59:03.215 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.39e-07 2024-09-14T23:59:03.215 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.215 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.215 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:03.216 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194357+0000", 2024-09-14T23:59:03.216 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3479999999999998e-06 2024-09-14T23:59:03.216 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.216 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.216 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:03.216 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194364+0000", 2024-09-14T23:59:03.216 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0899999999999999e-06 2024-09-14T23:59:03.216 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.216 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.216 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:03.216 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194371+0000", 2024-09-14T23:59:03.216 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5530000000000004e-06 2024-09-14T23:59:03.216 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.216 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.217 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.217 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194469+0000", 2024-09-14T23:59:03.217 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8119e-05 2024-09-14T23:59:03.217 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.217 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.217 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:03.217 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194475+0000", 2024-09-14T23:59:03.217 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7910000000000001e-06 2024-09-14T23:59:03.217 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.217 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.217 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.217 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986367+0000", 2024-09-14T23:59:03.218 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.79189214699999999 2024-09-14T23:59:03.218 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.218 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.218 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:03.218 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986375+0000", 2024-09-14T23:59:03.218 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7800000000000001e-06 2024-09-14T23:59:03.218 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.218 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.218 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.218 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995746+0000", 2024-09-14T23:59:03.218 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0093706469999999 2024-09-14T23:59:03.218 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.218 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.219 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:03.219 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995810+0000", 2024-09-14T23:59:03.219 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4068999999999999e-05 2024-09-14T23:59:03.219 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.219 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.219 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:03.219 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995901+0000", 2024-09-14T23:59:03.219 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1166000000000001e-05 2024-09-14T23:59:03.219 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.220 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:03.220 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.220 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.220 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.220 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186206 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1395712~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:59:03.220 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.194380+0000", 2024-09-14T23:59:03.220 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 56.000739541999998, 2024-09-14T23:59:03.220 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.801678347, 2024-09-14T23:59:03.220 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:03.220 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:03.220 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:03.220 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:03.220 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:03.221 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:03.221 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186206 2024-09-14T23:59:03.221 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.221 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:03.221 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.221 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:03.221 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194380+0000", 2024-09-14T23:59:03.221 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.221 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.221 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.221 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:03.221 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194380+0000", 2024-09-14T23:59:03.222 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.222 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.222 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.222 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:03.222 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194380+0000", 2024-09-14T23:59:03.222 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4400000000000001e-07 2024-09-14T23:59:03.222 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.222 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.222 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:03.222 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194383+0000", 2024-09-14T23:59:03.222 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3460000000000001e-06 2024-09-14T23:59:03.222 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.222 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.223 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:03.223 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194390+0000", 2024-09-14T23:59:03.223 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2189999999999998e-06 2024-09-14T23:59:03.223 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.223 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.223 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:03.223 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194403+0000", 2024-09-14T23:59:03.223 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2941e-05 2024-09-14T23:59:03.223 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.223 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.223 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.223 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194545+0000", 2024-09-14T23:59:03.223 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014232700000000001 2024-09-14T23:59:03.224 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.224 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.224 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:03.224 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194565+0000", 2024-09-14T23:59:03.224 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9409999999999999e-05 2024-09-14T23:59:03.224 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.224 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.224 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.225 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986409+0000", 2024-09-14T23:59:03.225 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.79184416000000002 2024-09-14T23:59:03.225 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.225 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.225 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:03.225 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986417+0000", 2024-09-14T23:59:03.225 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.551e-06 2024-09-14T23:59:03.225 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.226 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.226 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.226 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995921+0000", 2024-09-14T23:59:03.226 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.009504212 2024-09-14T23:59:03.226 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.226 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.226 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:03.226 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995974+0000", 2024-09-14T23:59:03.227 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2617999999999997e-05 2024-09-14T23:59:03.227 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.227 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.227 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:03.227 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996058+0000", 2024-09-14T23:59:03.227 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4118999999999999e-05 2024-09-14T23:59:03.227 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.228 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:03.228 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.228 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.228 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.228 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186207 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1337344~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:59:03.228 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.194412+0000", 2024-09-14T23:59:03.228 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 56.000706944000001, 2024-09-14T23:59:03.229 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.801848141, 2024-09-14T23:59:03.229 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:03.229 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:03.229 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:03.229 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:03.229 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:03.229 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:03.229 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186207 2024-09-14T23:59:03.229 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.230 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:03.230 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.230 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:03.230 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194412+0000", 2024-09-14T23:59:03.230 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.230 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.230 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.230 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:03.230 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194412+0000", 2024-09-14T23:59:03.230 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.231 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.231 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.231 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:03.231 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194413+0000", 2024-09-14T23:59:03.231 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5600000000000001e-07 2024-09-14T23:59:03.231 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.231 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.231 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:03.231 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194418+0000", 2024-09-14T23:59:03.231 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3720000000000004e-06 2024-09-14T23:59:03.231 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.231 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.231 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:03.232 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194423+0000", 2024-09-14T23:59:03.232 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.763e-06 2024-09-14T23:59:03.232 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.232 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.232 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:03.232 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194426+0000", 2024-09-14T23:59:03.232 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1480000000000002e-06 2024-09-14T23:59:03.232 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.232 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.232 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.232 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194596+0000", 2024-09-14T23:59:03.232 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017009600000000001 2024-09-14T23:59:03.233 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.233 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.233 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:03.233 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194606+0000", 2024-09-14T23:59:03.233 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6639999999999993e-06 2024-09-14T23:59:03.233 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.233 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.233 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.233 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986460+0000", 2024-09-14T23:59:03.233 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.79185424400000004 2024-09-14T23:59:03.233 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.233 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.234 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:03.234 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986469+0000", 2024-09-14T23:59:03.234 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4419999999999998e-06 2024-09-14T23:59:03.234 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.234 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.234 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.234 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996123+0000", 2024-09-14T23:59:03.234 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0096546070000001 2024-09-14T23:59:03.234 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.234 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.234 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:03.234 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996179+0000", 2024-09-14T23:59:03.235 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5850000000000002e-05 2024-09-14T23:59:03.235 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.235 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.235 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:03.235 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996261+0000", 2024-09-14T23:59:03.235 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1599000000000003e-05 2024-09-14T23:59:03.235 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.235 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:03.235 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.235 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.235 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.235 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186208 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1399808~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:59:03.235 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.194445+0000", 2024-09-14T23:59:03.236 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 56.000674773999997, 2024-09-14T23:59:03.236 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8019822269999999, 2024-09-14T23:59:03.236 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:03.236 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:03.236 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:03.236 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:03.236 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:03.236 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:03.236 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186208 2024-09-14T23:59:03.236 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.236 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:03.236 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.236 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:03.237 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194445+0000", 2024-09-14T23:59:03.237 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.237 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.237 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.237 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:03.237 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194445+0000", 2024-09-14T23:59:03.237 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.237 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.237 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.237 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:03.237 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978674+0000", 2024-09-14T23:59:03.237 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.784229124 2024-09-14T23:59:03.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.238 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.238 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:03.238 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978680+0000", 2024-09-14T23:59:03.238 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7969999999999997e-06 2024-09-14T23:59:03.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.238 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.238 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:03.238 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978689+0000", 2024-09-14T23:59:03.238 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6949999999999995e-06 2024-09-14T23:59:03.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.239 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.239 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:03.239 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978696+0000", 2024-09-14T23:59:03.239 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1300000000000003e-06 2024-09-14T23:59:03.239 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.239 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.239 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.239 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986509+0000", 2024-09-14T23:59:03.239 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0078130969999999998 2024-09-14T23:59:03.239 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.239 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.239 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:03.240 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986518+0000", 2024-09-14T23:59:03.240 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2810000000000008e-06 2024-09-14T23:59:03.240 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.240 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.240 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.240 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996280+0000", 2024-09-14T23:59:03.240 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0097623120000001 2024-09-14T23:59:03.240 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.240 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.240 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:03.240 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996337+0000", 2024-09-14T23:59:03.240 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7244000000000003e-05 2024-09-14T23:59:03.240 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.241 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.241 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:03.241 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996427+0000", 2024-09-14T23:59:03.241 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9547000000000002e-05 2024-09-14T23:59:03.241 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.241 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:03.241 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.241 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.241 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.241 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4172.0:7 3.11 3:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:59:03.241 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.483386+0000", 2024-09-14T23:59:03.242 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 55.711733348999999, 2024-09-14T23:59:03.242 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.558667389, 2024-09-14T23:59:03.242 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:03.242 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:03.242 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:59:03.242 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:03.242 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4172", 2024-09-14T23:59:03.242 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/2542269203", 2024-09-14T23:59:03.242 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 7 2024-09-14T23:59:03.242 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.242 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:03.242 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.242 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:03.243 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.483386+0000", 2024-09-14T23:59:03.243 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.243 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.243 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.243 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:03.243 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.483386+0000", 2024-09-14T23:59:03.243 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.243 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.243 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.243 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:03.243 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978674+0000", 2024-09-14T23:59:03.243 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.49528781599999999 2024-09-14T23:59:03.243 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.244 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.244 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:03.244 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978687+0000", 2024-09-14T23:59:03.244 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3244999999999999e-05 2024-09-14T23:59:03.244 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.244 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.244 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:03.244 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978699+0000", 2024-09-14T23:59:03.244 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1832e-05 2024-09-14T23:59:03.244 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.244 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.244 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:03.244 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978707+0000", 2024-09-14T23:59:03.244 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4489999999999999e-06 2024-09-14T23:59:03.244 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.245 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.245 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.245 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978801+0000", 2024-09-14T23:59:03.245 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3511999999999995e-05 2024-09-14T23:59:03.245 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.245 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.245 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:03.245 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978814+0000", 2024-09-14T23:59:03.245 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2789000000000001e-05 2024-09-14T23:59:03.245 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.245 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.245 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.245 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.041948+0000", 2024-09-14T23:59:03.246 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0631338269999999 2024-09-14T23:59:03.246 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.246 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.246 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:03.246 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.042053+0000", 2024-09-14T23:59:03.246 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105919 2024-09-14T23:59:03.246 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.246 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:03.246 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.246 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.246 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.246 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187209 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1575936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:59:03.246 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.991888+0000", 2024-09-14T23:59:03.247 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 55.203231377999998, 2024-09-14T23:59:03.247 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0712115470000001, 2024-09-14T23:59:03.247 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:03.247 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:03.247 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:03.247 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:03.247 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:03.247 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:03.247 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187209 2024-09-14T23:59:03.247 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.247 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:03.247 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.247 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:03.247 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991888+0000", 2024-09-14T23:59:03.248 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.248 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.248 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.248 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:03.248 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991888+0000", 2024-09-14T23:59:03.248 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.248 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.248 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.248 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:03.248 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991888+0000", 2024-09-14T23:59:03.248 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1199999999999998e-07 2024-09-14T23:59:03.248 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.249 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.249 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:03.249 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991890+0000", 2024-09-14T23:59:03.249 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9810000000000002e-06 2024-09-14T23:59:03.249 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.249 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.249 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:03.249 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991897+0000", 2024-09-14T23:59:03.249 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2609999999999999e-06 2024-09-14T23:59:03.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.250 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.250 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:03.250 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991902+0000", 2024-09-14T23:59:03.250 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0000000000000004e-06 2024-09-14T23:59:03.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.250 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.250 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.251 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995686+0000", 2024-09-14T23:59:03.251 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0037839290000000001 2024-09-14T23:59:03.251 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.251 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.251 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:03.251 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995693+0000", 2024-09-14T23:59:03.251 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7589999999999997e-06 2024-09-14T23:59:03.251 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.252 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.252 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.252 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062846+0000", 2024-09-14T23:59:03.252 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.067152417 2024-09-14T23:59:03.252 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.252 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.252 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:03.252 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062935+0000", 2024-09-14T23:59:03.253 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9080999999999996e-05 2024-09-14T23:59:03.253 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.253 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.253 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:03.253 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.063100+0000", 2024-09-14T23:59:03.253 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016470699999999999 2024-09-14T23:59:03.253 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.253 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:03.254 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.254 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.254 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.254 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187210 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1591296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:59:03.254 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.991908+0000", 2024-09-14T23:59:03.254 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 55.203211465000003, 2024-09-14T23:59:03.254 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0713882530000001, 2024-09-14T23:59:03.254 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:03.255 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:03.255 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:03.255 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:03.255 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:03.255 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:03.255 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187210 2024-09-14T23:59:03.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.255 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:03.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.256 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:03.256 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991908+0000", 2024-09-14T23:59:03.256 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.256 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.256 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.256 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:03.256 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991908+0000", 2024-09-14T23:59:03.256 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.256 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.256 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.256 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:03.256 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993673+0000", 2024-09-14T23:59:03.256 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017651030000000001 2024-09-14T23:59:03.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.257 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.257 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:03.257 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993681+0000", 2024-09-14T23:59:03.257 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.858e-06 2024-09-14T23:59:03.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.257 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.257 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:03.257 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993691+0000", 2024-09-14T23:59:03.257 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9920000000000002e-06 2024-09-14T23:59:03.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.257 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.257 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:03.258 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993698+0000", 2024-09-14T23:59:03.258 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6490000000000002e-06 2024-09-14T23:59:03.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.258 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.258 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.258 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995731+0000", 2024-09-14T23:59:03.258 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0020338959999999999 2024-09-14T23:59:03.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.258 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.258 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:03.258 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995741+0000", 2024-09-14T23:59:03.258 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3079999999999997e-06 2024-09-14T23:59:03.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.258 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.259 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.259 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.063130+0000", 2024-09-14T23:59:03.259 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.067389159 2024-09-14T23:59:03.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.259 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.259 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:03.259 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.063206+0000", 2024-09-14T23:59:03.259 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6577999999999998e-05 2024-09-14T23:59:03.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.259 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.259 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:03.259 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.063296+0000", 2024-09-14T23:59:03.259 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9710000000000007e-05 2024-09-14T23:59:03.259 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.260 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:03.260 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.260 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.260 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187216 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1603584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:59:03.260 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.993831+0000", 2024-09-14T23:59:03.260 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 55.201288259000002, 2024-09-14T23:59:03.260 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.071244627, 2024-09-14T23:59:03.260 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:03.260 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:03.260 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:03.260 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:03.260 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187216 2024-09-14T23:59:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993831+0000", 2024-09-14T23:59:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993831+0000", 2024-09-14T23:59:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.262 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.262 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:03.262 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993832+0000", 2024-09-14T23:59:03.262 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9700000000000002e-07 2024-09-14T23:59:03.262 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.262 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.262 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:03.262 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993834+0000", 2024-09-14T23:59:03.262 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6170000000000001e-06 2024-09-14T23:59:03.262 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.262 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.262 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:03.262 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993841+0000", 2024-09-14T23:59:03.263 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6980000000000001e-06 2024-09-14T23:59:03.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.263 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:03.263 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993846+0000", 2024-09-14T23:59:03.263 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4190000000000001e-06 2024-09-14T23:59:03.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.263 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.263 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995988+0000", 2024-09-14T23:59:03.263 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0021417390000000001 2024-09-14T23:59:03.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.263 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:03.264 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995998+0000", 2024-09-14T23:59:03.264 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5100000000000004e-06 2024-09-14T23:59:03.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.264 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.264 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.264 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.064883+0000", 2024-09-14T23:59:03.264 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.06888597 2024-09-14T23:59:03.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.264 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.264 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:03.264 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.064977+0000", 2024-09-14T23:59:03.264 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3817e-05 2024-09-14T23:59:03.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.264 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065076+0000", 2024-09-14T23:59:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8460000000000003e-05 2024-09-14T23:59:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187217 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1592320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:59:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.993855+0000", 2024-09-14T23:59:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 55.201264825999999, 2024-09-14T23:59:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0714239349999999, 2024-09-14T23:59:03.266 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:03.266 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:03.266 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:03.266 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:03.266 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:03.266 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:03.266 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187217 2024-09-14T23:59:03.266 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.266 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:03.266 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.266 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:03.266 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993855+0000", 2024-09-14T23:59:03.266 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.267 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.267 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:03.267 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993855+0000", 2024-09-14T23:59:03.267 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.267 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.267 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:03.267 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993855+0000", 2024-09-14T23:59:03.267 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4899999999999995e-07 2024-09-14T23:59:03.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.267 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.267 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:03.267 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993857+0000", 2024-09-14T23:59:03.268 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3379999999999999e-06 2024-09-14T23:59:03.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.268 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.268 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:03.268 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993864+0000", 2024-09-14T23:59:03.268 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6000000000000003e-06 2024-09-14T23:59:03.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.268 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.268 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:03.268 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993869+0000", 2024-09-14T23:59:03.268 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3210000000000004e-06 2024-09-14T23:59:03.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.268 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.269 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.269 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996038+0000", 2024-09-14T23:59:03.269 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0021683050000000001 2024-09-14T23:59:03.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.269 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.269 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:03.269 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996048+0000", 2024-09-14T23:59:03.269 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0173999999999999e-05 2024-09-14T23:59:03.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.269 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.269 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.269 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065108+0000", 2024-09-14T23:59:03.269 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0690600619999999 2024-09-14T23:59:03.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.270 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.270 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:03.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065171+0000", 2024-09-14T23:59:03.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3423999999999994e-05 2024-09-14T23:59:03.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.270 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.270 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:03.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065279+0000", 2024-09-14T23:59:03.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000107162 2024-09-14T23:59:03.270 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.270 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:03.271 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.271 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.271 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187218 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1607680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:59:03.271 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.993877+0000", 2024-09-14T23:59:03.271 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 55.201242546000003, 2024-09-14T23:59:03.271 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0716976810000001, 2024-09-14T23:59:03.271 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:03.271 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:03.271 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:03.271 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:03.271 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:03.271 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:03.272 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187218 2024-09-14T23:59:03.272 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.272 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:03.272 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.272 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:03.272 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993877+0000", 2024-09-14T23:59:03.272 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.272 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.272 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.272 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:03.272 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993877+0000", 2024-09-14T23:59:03.272 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.272 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.272 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.273 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:03.273 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993877+0000", 2024-09-14T23:59:03.273 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8299999999999998e-07 2024-09-14T23:59:03.273 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.273 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.273 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:03.273 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993879+0000", 2024-09-14T23:59:03.273 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5659999999999999e-06 2024-09-14T23:59:03.273 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.273 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.273 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:03.273 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993882+0000", 2024-09-14T23:59:03.273 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7280000000000001e-06 2024-09-14T23:59:03.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.274 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:03.274 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993885+0000", 2024-09-14T23:59:03.274 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.751e-06 2024-09-14T23:59:03.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.274 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.274 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996081+0000", 2024-09-14T23:59:03.274 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0021960590000000002 2024-09-14T23:59:03.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.274 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:03.275 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996091+0000", 2024-09-14T23:59:03.275 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4930000000000004e-06 2024-09-14T23:59:03.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.275 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.275 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.275 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065308+0000", 2024-09-14T23:59:03.275 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.069217407 2024-09-14T23:59:03.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.276 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.276 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:03.276 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065414+0000", 2024-09-14T23:59:03.276 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105438 2024-09-14T23:59:03.276 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.276 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.276 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:03.277 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065575+0000", 2024-09-14T23:59:03.277 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000160956 2024-09-14T23:59:03.277 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.277 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:03.277 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.277 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.277 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.277 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187219 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1596416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:59:03.278 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.993889+0000", 2024-09-14T23:59:03.278 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 55.201230703999997, 2024-09-14T23:59:03.278 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0720031190000001, 2024-09-14T23:59:03.278 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:03.278 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:03.278 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:03.278 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:03.279 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:03.279 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:03.279 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187219 2024-09-14T23:59:03.279 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.279 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:03.279 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.279 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:03.280 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993889+0000", 2024-09-14T23:59:03.280 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.280 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.280 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.280 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:03.280 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993889+0000", 2024-09-14T23:59:03.280 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.280 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.281 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.281 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:03.281 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993889+0000", 2024-09-14T23:59:03.281 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2399999999999999e-07 2024-09-14T23:59:03.281 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.281 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.281 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:03.281 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993890+0000", 2024-09-14T23:59:03.281 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1939999999999999e-06 2024-09-14T23:59:03.281 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.281 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.281 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:03.281 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993893+0000", 2024-09-14T23:59:03.281 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1300000000000001e-06 2024-09-14T23:59:03.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.282 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.282 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:03.282 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993896+0000", 2024-09-14T23:59:03.282 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6879999999999999e-06 2024-09-14T23:59:03.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.282 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.282 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.282 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996119+0000", 2024-09-14T23:59:03.282 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0022226149999999998 2024-09-14T23:59:03.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.282 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.282 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:03.282 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996123+0000", 2024-09-14T23:59:03.283 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.211e-06 2024-09-14T23:59:03.283 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.283 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.283 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.283 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065616+0000", 2024-09-14T23:59:03.283 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0694935240000001 2024-09-14T23:59:03.283 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.283 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.283 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:03.284 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065733+0000", 2024-09-14T23:59:03.284 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000116492 2024-09-14T23:59:03.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.284 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.284 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:03.284 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065892+0000", 2024-09-14T23:59:03.284 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015894100000000001 2024-09-14T23:59:03.284 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.284 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:03.284 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.284 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.285 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187220 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1611776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:59:03.285 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.993899+0000", 2024-09-14T23:59:03.285 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 55.201220245999998, 2024-09-14T23:59:03.285 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.072298041, 2024-09-14T23:59:03.285 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:03.285 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:03.285 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:03.285 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:03.285 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:03.285 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:03.285 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187220 2024-09-14T23:59:03.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.285 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:03.285 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.286 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:03.286 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993899+0000", 2024-09-14T23:59:03.286 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.286 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.286 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.286 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:03.286 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993899+0000", 2024-09-14T23:59:03.286 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.286 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.286 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.286 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:03.286 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993899+0000", 2024-09-14T23:59:03.286 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2100000000000001e-07 2024-09-14T23:59:03.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.287 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.287 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:03.287 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993901+0000", 2024-09-14T23:59:03.287 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.141e-06 2024-09-14T23:59:03.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.287 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.287 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:03.287 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993904+0000", 2024-09-14T23:59:03.287 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1470000000000001e-06 2024-09-14T23:59:03.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.287 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.287 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:03.288 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993906+0000", 2024-09-14T23:59:03.288 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6340000000000001e-06 2024-09-14T23:59:03.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.288 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.288 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.288 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996160+0000", 2024-09-14T23:59:03.288 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0022537659999999999 2024-09-14T23:59:03.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.288 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.288 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:03.288 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996169+0000", 2024-09-14T23:59:03.288 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2739999999999996e-06 2024-09-14T23:59:03.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.289 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.289 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.289 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065981+0000", 2024-09-14T23:59:03.289 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.069811742 2024-09-14T23:59:03.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.289 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.289 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:03.289 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.066092+0000", 2024-09-14T23:59:03.289 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000110701 2024-09-14T23:59:03.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.289 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.289 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:03.289 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.066197+0000", 2024-09-14T23:59:03.289 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010541500000000001 2024-09-14T23:59:03.290 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.290 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:03.290 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.290 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.290 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187221 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1600512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:59:03.290 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.993909+0000", 2024-09-14T23:59:03.290 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 55.201210033999999, 2024-09-14T23:59:03.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.072643499, 2024-09-14T23:59:03.290 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:03.290 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:03.290 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:03.290 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:03.291 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:03.291 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:03.291 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187221 2024-09-14T23:59:03.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.291 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:03.291 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.291 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:03.291 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993909+0000", 2024-09-14T23:59:03.291 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.291 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.291 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:03.291 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993909+0000", 2024-09-14T23:59:03.292 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.292 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.292 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:03.292 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995673+0000", 2024-09-14T23:59:03.292 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001763141 2024-09-14T23:59:03.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.292 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.292 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:03.292 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995677+0000", 2024-09-14T23:59:03.292 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8709999999999996e-06 2024-09-14T23:59:03.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.292 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.293 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:03.293 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995686+0000", 2024-09-14T23:59:03.293 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6649999999999992e-06 2024-09-14T23:59:03.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.293 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.293 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:03.293 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995691+0000", 2024-09-14T23:59:03.293 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0289999999999996e-06 2024-09-14T23:59:03.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.293 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.293 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.293 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996199+0000", 2024-09-14T23:59:03.293 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00050799099999999995 2024-09-14T23:59:03.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.294 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.294 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:03.294 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996205+0000", 2024-09-14T23:59:03.294 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8259999999999998e-06 2024-09-14T23:59:03.294 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.294 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.294 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.294 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.066238+0000", 2024-09-14T23:59:03.294 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.070032928 2024-09-14T23:59:03.294 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.294 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.294 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:03.294 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.066393+0000", 2024-09-14T23:59:03.295 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015549800000000001 2024-09-14T23:59:03.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.295 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.295 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:03.295 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.066553+0000", 2024-09-14T23:59:03.295 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015955 2024-09-14T23:59:03.295 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.295 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:03.295 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.295 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.295 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187223 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1604608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:59:03.295 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.995728+0000", 2024-09-14T23:59:03.295 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 55.199391218000002, 2024-09-14T23:59:03.296 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.071486478, 2024-09-14T23:59:03.296 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:03.296 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:03.296 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:03.296 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:03.296 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:03.296 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:03.296 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187223 2024-09-14T23:59:03.296 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.296 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:03.296 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.296 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:03.296 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995728+0000", 2024-09-14T23:59:03.297 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.297 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.297 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:03.297 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995728+0000", 2024-09-14T23:59:03.297 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.297 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.297 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:03.297 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995728+0000", 2024-09-14T23:59:03.297 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.65e-07 2024-09-14T23:59:03.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.297 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.297 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:03.298 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995730+0000", 2024-09-14T23:59:03.298 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.406e-06 2024-09-14T23:59:03.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.298 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.298 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:03.298 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995734+0000", 2024-09-14T23:59:03.298 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2640000000000002e-06 2024-09-14T23:59:03.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.298 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.298 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:03.298 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995737+0000", 2024-09-14T23:59:03.298 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0589999999999998e-06 2024-09-14T23:59:03.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.299 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.299 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.299 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996288+0000", 2024-09-14T23:59:03.299 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00055059599999999998 2024-09-14T23:59:03.299 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.299 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.299 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:03.299 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996294+0000", 2024-09-14T23:59:03.299 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0260000000000004e-06 2024-09-14T23:59:03.299 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.299 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.299 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.300 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.066938+0000", 2024-09-14T23:59:03.300 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0706439510000001 2024-09-14T23:59:03.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.300 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.300 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:03.300 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067050+0000", 2024-09-14T23:59:03.300 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011220100000000001 2024-09-14T23:59:03.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.301 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.301 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:03.301 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067215+0000", 2024-09-14T23:59:03.301 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016470999999999999 2024-09-14T23:59:03.301 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.301 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:03.301 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.302 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.302 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187224 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1619968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:59:03.302 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.995742+0000", 2024-09-14T23:59:03.302 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 55.199377403, 2024-09-14T23:59:03.302 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0717603609999999, 2024-09-14T23:59:03.302 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:03.302 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:03.303 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:03.303 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:03.303 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:03.303 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:03.303 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187224 2024-09-14T23:59:03.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.303 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:03.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.304 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:03.304 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995742+0000", 2024-09-14T23:59:03.304 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.304 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.304 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:03.304 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995742+0000", 2024-09-14T23:59:03.304 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.305 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:03.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995742+0000", 2024-09-14T23:59:03.305 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2399999999999999e-07 2024-09-14T23:59:03.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.305 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:03.306 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995744+0000", 2024-09-14T23:59:03.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.356e-06 2024-09-14T23:59:03.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.306 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.306 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:03.306 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995747+0000", 2024-09-14T23:59:03.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.304e-06 2024-09-14T23:59:03.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.307 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.307 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:03.307 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995750+0000", 2024-09-14T23:59:03.307 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6510000000000001e-06 2024-09-14T23:59:03.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.307 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.307 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.307 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996327+0000", 2024-09-14T23:59:03.307 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00057794300000000003 2024-09-14T23:59:03.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.307 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.307 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:03.308 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996337+0000", 2024-09-14T23:59:03.308 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3759999999999997e-06 2024-09-14T23:59:03.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.308 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.308 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.308 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067244+0000", 2024-09-14T23:59:03.308 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.070907367 2024-09-14T23:59:03.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.308 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.308 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:03.308 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067341+0000", 2024-09-14T23:59:03.308 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7277999999999999e-05 2024-09-14T23:59:03.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.308 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.309 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:03.309 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067502+0000", 2024-09-14T23:59:03.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016086200000000001 2024-09-14T23:59:03.309 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.309 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:03.309 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.309 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.309 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187225 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1608704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:59:03.309 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.995766+0000", 2024-09-14T23:59:03.309 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 55.199353229000003, 2024-09-14T23:59:03.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.072043649, 2024-09-14T23:59:03.310 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:03.310 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:03.310 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:03.310 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:03.310 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:03.310 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:03.310 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187225 2024-09-14T23:59:03.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.310 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:03.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.310 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:03.310 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995766+0000", 2024-09-14T23:59:03.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.311 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.311 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:03.311 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995766+0000", 2024-09-14T23:59:03.311 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.311 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.311 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:03.311 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995766+0000", 2024-09-14T23:59:03.311 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9200000000000002e-07 2024-09-14T23:59:03.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.311 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.311 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:03.312 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995768+0000", 2024-09-14T23:59:03.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8679999999999999e-06 2024-09-14T23:59:03.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.312 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:03.312 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995772+0000", 2024-09-14T23:59:03.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6339999999999999e-06 2024-09-14T23:59:03.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.312 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:03.312 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995775+0000", 2024-09-14T23:59:03.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8159999999999998e-06 2024-09-14T23:59:03.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.313 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.313 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.313 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996372+0000", 2024-09-14T23:59:03.313 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00059758800000000003 2024-09-14T23:59:03.313 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.313 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.313 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:03.313 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996377+0000", 2024-09-14T23:59:03.313 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7770000000000002e-06 2024-09-14T23:59:03.313 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.313 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.313 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.313 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067534+0000", 2024-09-14T23:59:03.313 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.071156821 2024-09-14T23:59:03.314 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.314 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.314 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:03.314 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067638+0000", 2024-09-14T23:59:03.314 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000103737 2024-09-14T23:59:03.314 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.314 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.314 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:03.314 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067810+0000", 2024-09-14T23:59:03.314 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017211599999999999 2024-09-14T23:59:03.314 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.314 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:03.314 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.314 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.315 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.315 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187226 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1624064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:59:03.315 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.995778+0000", 2024-09-14T23:59:03.315 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 55.199341050000001, 2024-09-14T23:59:03.315 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.072339081, 2024-09-14T23:59:03.315 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:03.315 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:03.315 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:03.315 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:03.315 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:03.315 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:03.315 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187226 2024-09-14T23:59:03.316 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.316 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:03.316 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.316 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:03.316 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995778+0000", 2024-09-14T23:59:03.316 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.316 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.316 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.316 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:03.316 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995778+0000", 2024-09-14T23:59:03.316 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.316 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.316 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.317 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:03.317 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995779+0000", 2024-09-14T23:59:03.317 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0699999999999999e-07 2024-09-14T23:59:03.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.317 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.317 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:03.317 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995780+0000", 2024-09-14T23:59:03.317 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2139999999999999e-06 2024-09-14T23:59:03.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.317 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.317 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:03.317 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995783+0000", 2024-09-14T23:59:03.317 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1250000000000001e-06 2024-09-14T23:59:03.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.318 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.318 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:03.318 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995787+0000", 2024-09-14T23:59:03.318 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6679999999999999e-06 2024-09-14T23:59:03.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.318 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.318 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.318 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996404+0000", 2024-09-14T23:59:03.318 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00061777499999999999 2024-09-14T23:59:03.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.318 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.318 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:03.318 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996414+0000", 2024-09-14T23:59:03.319 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.431e-06 2024-09-14T23:59:03.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.319 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.319 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.319 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067844+0000", 2024-09-14T23:59:03.319 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0714302410000001 2024-09-14T23:59:03.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.319 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.319 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:03.320 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067952+0000", 2024-09-14T23:59:03.320 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000107778 2024-09-14T23:59:03.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.320 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.320 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:03.320 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.068117+0000", 2024-09-14T23:59:03.320 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000165642 2024-09-14T23:59:03.320 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.320 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:03.320 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.320 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.320 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187227 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1612800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:59:03.321 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.995806+0000", 2024-09-14T23:59:03.321 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 55.199313813000003, 2024-09-14T23:59:03.321 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0726092810000001, 2024-09-14T23:59:03.321 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:03.321 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:03.321 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:03.321 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:03.321 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:03.321 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:03.321 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187227 2024-09-14T23:59:03.321 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.321 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:03.321 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.322 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:03.322 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995806+0000", 2024-09-14T23:59:03.322 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.322 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.322 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:03.322 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995806+0000", 2024-09-14T23:59:03.322 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.322 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.322 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:03.322 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995672+0000", 2024-09-14T23:59:03.322 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99986608099999996 2024-09-14T23:59:03.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.323 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.323 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:03.323 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995677+0000", 2024-09-14T23:59:03.323 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9520000000000002e-06 2024-09-14T23:59:03.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.323 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.323 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:03.323 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995686+0000", 2024-09-14T23:59:03.323 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8199999999999992e-06 2024-09-14T23:59:03.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.323 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.323 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:03.323 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995693+0000", 2024-09-14T23:59:03.324 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1630000000000001e-06 2024-09-14T23:59:03.324 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.324 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.324 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.324 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.068149+0000", 2024-09-14T23:59:03.324 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.072456692000000003 2024-09-14T23:59:03.324 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.324 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.324 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:03.324 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.068251+0000", 2024-09-14T23:59:03.324 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101963 2024-09-14T23:59:03.324 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.324 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.325 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:03.325 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.068415+0000", 2024-09-14T23:59:03.325 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016360999999999999 2024-09-14T23:59:03.325 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.325 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:03.325 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.325 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.325 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:03.325 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:59:03.328 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T23:59:03.328 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:59:03.326+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:59:03.428 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:59:03.427+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:59:03.476 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:59:03.476 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:59:03.476 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:59:03.476 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:59:03.485 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T23:59:03.529 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:59:03.528+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:59:03.592 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:59:03.592 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:59:03.592 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:59:03.592 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:59:03.592 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:59:03.601 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T23:59:03.630 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:59:03.629+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:59:03.709 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:59:03.710 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:59:03.710 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:59:03.710 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:59:03.710 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.710 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4284.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-14T23:59:03.710 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:59:03.710 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 161.18283205099999, 2024-09-14T23:59:03.710 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.004942086, 2024-09-14T23:59:03.711 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:03.711 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:03.711 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:03.711 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:03.711 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4284", 2024-09-14T23:59:03.711 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/4278347926", 2024-09-14T23:59:03.711 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1 2024-09-14T23:59:03.711 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.711 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:03.711 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.711 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:03.712 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:59:03.712 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.712 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.712 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.712 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:03.712 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:59:03.712 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.712 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.712 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.712 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:03.712 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524331+0000", 2024-09-14T23:59:03.712 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.383e-05 2024-09-14T23:59:03.712 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.713 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.713 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:03.713 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524339+0000", 2024-09-14T23:59:03.713 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8290000000000008e-06 2024-09-14T23:59:03.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.713 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.713 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:03.713 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524359+0000", 2024-09-14T23:59:03.713 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9998000000000001e-05 2024-09-14T23:59:03.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.713 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.713 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:03.714 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524373+0000", 2024-09-14T23:59:03.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4304000000000001e-05 2024-09-14T23:59:03.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.714 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.714 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.714 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524488+0000", 2024-09-14T23:59:03.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114663 2024-09-14T23:59:03.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.714 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.714 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:03.714 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524501+0000", 2024-09-14T23:59:03.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3349999999999999e-05 2024-09-14T23:59:03.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.715 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.715 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.715 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.526868+0000", 2024-09-14T23:59:03.715 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023670090000000002 2024-09-14T23:59:03.715 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.715 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.715 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:03.715 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.527041+0000", 2024-09-14T23:59:03.715 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000172597 2024-09-14T23:59:03.715 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.715 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.715 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-14T23:59:03.715 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.527384+0000", 2024-09-14T23:59:03.716 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00034309500000000002 2024-09-14T23:59:03.716 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.716 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.716 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:59:03.716 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529103+0000", 2024-09-14T23:59:03.716 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017185220000000001 2024-09-14T23:59:03.716 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.716 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.716 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:59:03.717 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529188+0000", 2024-09-14T23:59:03.717 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5591000000000001e-05 2024-09-14T23:59:03.717 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.717 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.717 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:03.717 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529227+0000", 2024-09-14T23:59:03.717 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8563000000000003e-05 2024-09-14T23:59:03.717 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.717 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.717 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:03.717 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529259+0000", 2024-09-14T23:59:03.717 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2734999999999997e-05 2024-09-14T23:59:03.717 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.718 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:03.718 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.718 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.718 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.718 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4284.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-14T23:59:03.718 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:23.530683+0000", 2024-09-14T23:59:03.718 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 160.17646638599999, 2024-09-14T23:59:03.718 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0036133110000000001, 2024-09-14T23:59:03.718 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:03.718 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:03.718 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:03.718 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:03.718 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4284", 2024-09-14T23:59:03.718 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/4278347926", 2024-09-14T23:59:03.719 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5 2024-09-14T23:59:03.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.719 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:03.719 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.719 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:03.719 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530683+0000", 2024-09-14T23:59:03.719 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.719 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.719 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:03.719 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530683+0000", 2024-09-14T23:59:03.719 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.720 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.720 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:03.720 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530685+0000", 2024-09-14T23:59:03.720 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.075e-06 2024-09-14T23:59:03.720 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.720 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.720 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:03.720 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530695+0000", 2024-09-14T23:59:03.720 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0261999999999999e-05 2024-09-14T23:59:03.720 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.720 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.720 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:03.720 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530715+0000", 2024-09-14T23:59:03.721 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9930000000000001e-05 2024-09-14T23:59:03.721 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.721 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.721 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:03.721 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530724+0000", 2024-09-14T23:59:03.721 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.901e-06 2024-09-14T23:59:03.721 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.721 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.721 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.721 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530800+0000", 2024-09-14T23:59:03.721 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5873999999999999e-05 2024-09-14T23:59:03.721 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.721 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.722 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-14T23:59:03.722 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.530807+0000", 2024-09-14T23:59:03.722 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2389999999999996e-06 2024-09-14T23:59:03.722 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.722 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.722 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.722 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.532494+0000", 2024-09-14T23:59:03.722 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016871830000000001 2024-09-14T23:59:03.722 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.722 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.722 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:03.722 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.532599+0000", 2024-09-14T23:59:03.722 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010472 2024-09-14T23:59:03.723 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.723 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.723 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1", 2024-09-14T23:59:03.723 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.532715+0000", 2024-09-14T23:59:03.723 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011568199999999999 2024-09-14T23:59:03.723 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.723 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.723 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:59:03.723 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.533460+0000", 2024-09-14T23:59:03.723 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00074494999999999997 2024-09-14T23:59:03.723 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.723 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.723 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:59:03.724 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534230+0000", 2024-09-14T23:59:03.724 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00077036599999999998 2024-09-14T23:59:03.724 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.724 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.724 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:03.724 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534266+0000", 2024-09-14T23:59:03.724 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5725000000000001e-05 2024-09-14T23:59:03.724 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.724 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.724 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:03.724 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:23.534296+0000", 2024-09-14T23:59:03.724 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0403999999999999e-05 2024-09-14T23:59:03.724 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.725 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:03.725 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.725 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.725 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.725 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56383 3.c e56/49 3:36b698b0:::rbd_data.1118c4b3aebe.0000000000000000:head v 56'194, mlcod=56'192)", 2024-09-14T23:59:03.725 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.270483+0000", 2024-09-14T23:59:03.725 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 117.436666586, 2024-09-14T23:59:03.725 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0079685880000000004, 2024-09-14T23:59:03.725 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:03.725 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:03.725 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:03.725 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:03.725 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.726 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:03.726 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.270483+0000", 2024-09-14T23:59:03.726 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.726 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.726 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.726 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:03.726 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.270483+0000", 2024-09-14T23:59:03.726 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.726 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.726 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.726 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:03.726 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.270484+0000", 2024-09-14T23:59:03.726 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7400000000000002e-07 2024-09-14T23:59:03.727 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.727 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.727 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:03.727 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273680+0000", 2024-09-14T23:59:03.727 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031966680000000002 2024-09-14T23:59:03.727 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.727 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.727 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:03.727 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273697+0000", 2024-09-14T23:59:03.727 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6595e-05 2024-09-14T23:59:03.727 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.727 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.727 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:03.728 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273705+0000", 2024-09-14T23:59:03.728 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7530000000000001e-06 2024-09-14T23:59:03.728 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.728 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.728 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.728 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273763+0000", 2024-09-14T23:59:03.728 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8238e-05 2024-09-14T23:59:03.728 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.728 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.728 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:03.728 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273781+0000", 2024-09-14T23:59:03.728 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8281e-05 2024-09-14T23:59:03.728 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.728 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.729 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:03.729 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.278408+0000", 2024-09-14T23:59:03.729 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00462716 2024-09-14T23:59:03.729 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.729 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.729 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:03.729 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.278451+0000", 2024-09-14T23:59:03.729 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3019000000000003e-05 2024-09-14T23:59:03.729 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.729 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:03.729 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.729 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.729 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.730 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56384 3.c e56/49 3:36b698b0:::rbd_data.1118c4b3aebe.0000000000000000:head v 56'196, mlcod=56'192)", 2024-09-14T23:59:03.730 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:59:03.730 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 117.43263391399999, 2024-09-14T23:59:03.730 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0046707529999999997, 2024-09-14T23:59:03.730 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:03.730 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:03.730 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:03.730 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:03.730 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.730 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:03.730 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:59:03.730 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.731 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.731 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.731 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:03.731 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:59:03.731 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.731 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.731 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.731 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:03.731 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:59:03.731 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1699999999999997e-07 2024-09-14T23:59:03.731 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.731 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.731 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:03.732 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274523+0000", 2024-09-14T23:59:03.732 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9129999999999997e-06 2024-09-14T23:59:03.732 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.732 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.732 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:03.732 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274535+0000", 2024-09-14T23:59:03.732 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1788e-05 2024-09-14T23:59:03.732 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.732 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.732 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:03.732 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274542+0000", 2024-09-14T23:59:03.732 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0539999999999997e-06 2024-09-14T23:59:03.732 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.733 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.733 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.733 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275889+0000", 2024-09-14T23:59:03.733 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00134669 2024-09-14T23:59:03.733 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.733 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.733 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:03.733 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275917+0000", 2024-09-14T23:59:03.733 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7664000000000001e-05 2024-09-14T23:59:03.733 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.733 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.733 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:03.733 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279134+0000", 2024-09-14T23:59:03.733 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0032178459999999999 2024-09-14T23:59:03.734 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.734 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.734 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:03.734 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279186+0000", 2024-09-14T23:59:03.734 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1881e-05 2024-09-14T23:59:03.734 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.734 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:03.734 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.734 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.734 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.734 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56381 3.1d e56/49 3:bcf9a8cf:::rbd_data.1118c4b3aebe.0000000000000006:head v 56'210, mlcod=56'208)", 2024-09-14T23:59:03.734 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.275462+0000", 2024-09-14T23:59:03.734 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 117.431687584, 2024-09-14T23:59:03.735 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0071310710000000001, 2024-09-14T23:59:03.735 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:03.735 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:03.735 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:03.735 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:03.735 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.735 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:03.735 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275462+0000", 2024-09-14T23:59:03.735 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.735 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.735 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.735 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:03.735 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275462+0000", 2024-09-14T23:59:03.736 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.736 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.736 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.736 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:03.736 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275463+0000", 2024-09-14T23:59:03.736 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2699999999999999e-07 2024-09-14T23:59:03.736 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.736 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.736 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:03.736 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276779+0000", 2024-09-14T23:59:03.736 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0013163649999999999 2024-09-14T23:59:03.736 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.736 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.736 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:03.737 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276795+0000", 2024-09-14T23:59:03.737 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.59e-05 2024-09-14T23:59:03.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.737 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.737 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:03.737 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276806+0000", 2024-09-14T23:59:03.737 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1106e-05 2024-09-14T23:59:03.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.737 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.737 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.737 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276941+0000", 2024-09-14T23:59:03.737 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013477500000000001 2024-09-14T23:59:03.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.738 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.738 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:03.738 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276963+0000", 2024-09-14T23:59:03.738 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2419999999999999e-05 2024-09-14T23:59:03.738 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.738 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.738 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:03.738 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282546+0000", 2024-09-14T23:59:03.738 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0055827100000000003 2024-09-14T23:59:03.738 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.738 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.738 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:03.738 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282593+0000", 2024-09-14T23:59:03.738 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7067999999999997e-05 2024-09-14T23:59:03.739 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.739 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:03.739 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.739 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.739 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.739 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56387 3.c e56/49 3:30ee4ba0:::rbd_data.1118c4b3aebe.0000000000000004:head v 56'197, mlcod=56'192)", 2024-09-14T23:59:03.739 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.276498+0000", 2024-09-14T23:59:03.739 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 117.430651489, 2024-09-14T23:59:03.739 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0092192639999999996, 2024-09-14T23:59:03.739 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:03.739 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:03.739 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:03.739 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:03.740 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.740 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:03.740 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276498+0000", 2024-09-14T23:59:03.740 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.740 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.740 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.740 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:03.740 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276498+0000", 2024-09-14T23:59:03.740 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.740 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.740 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.740 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:03.740 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276499+0000", 2024-09-14T23:59:03.741 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1100000000000004e-07 2024-09-14T23:59:03.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.741 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.741 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:03.741 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279646+0000", 2024-09-14T23:59:03.741 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031473220000000001 2024-09-14T23:59:03.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.741 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.741 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:03.741 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279672+0000", 2024-09-14T23:59:03.741 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6129999999999999e-05 2024-09-14T23:59:03.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.741 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.742 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:03.742 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279682+0000", 2024-09-14T23:59:03.742 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7170000000000003e-06 2024-09-14T23:59:03.742 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.742 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.742 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.742 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279742+0000", 2024-09-14T23:59:03.742 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9926999999999997e-05 2024-09-14T23:59:03.742 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.742 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.742 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:03.742 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279761+0000", 2024-09-14T23:59:03.742 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9402000000000002e-05 2024-09-14T23:59:03.742 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.743 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.743 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:03.743 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.285680+0000", 2024-09-14T23:59:03.743 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.005918881 2024-09-14T23:59:03.743 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.743 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.743 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:03.743 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.285717+0000", 2024-09-14T23:59:03.743 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6974000000000002e-05 2024-09-14T23:59:03.743 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.743 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:03.743 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.743 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.743 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.744 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56388 3.c e56/49 3:30ee4ba0:::rbd_data.1118c4b3aebe.0000000000000004:head v 56'199, mlcod=56'192)", 2024-09-14T23:59:03.744 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:59:03.744 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 117.427159901, 2024-09-14T23:59:03.744 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0060596740000000001, 2024-09-14T23:59:03.744 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:03.744 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:03.744 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:03.744 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:03.744 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.744 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:03.744 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:59:03.744 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.744 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.745 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.745 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:03.745 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:59:03.745 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.745 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.745 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:03.745 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:59:03.745 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2200000000000003e-07 2024-09-14T23:59:03.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.745 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.745 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:03.745 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279995+0000", 2024-09-14T23:59:03.746 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5569999999999997e-06 2024-09-14T23:59:03.746 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.746 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.746 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:03.746 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280004+0000", 2024-09-14T23:59:03.746 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0080000000000008e-06 2024-09-14T23:59:03.746 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.746 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.746 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:03.746 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280010+0000", 2024-09-14T23:59:03.746 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5749999999999997e-06 2024-09-14T23:59:03.746 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.746 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.746 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.747 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282968+0000", 2024-09-14T23:59:03.747 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0029579390000000001 2024-09-14T23:59:03.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.747 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.747 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:03.747 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282998+0000", 2024-09-14T23:59:03.747 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9947999999999999e-05 2024-09-14T23:59:03.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.747 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.747 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:03.747 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.286012+0000", 2024-09-14T23:59:03.747 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030132900000000001 2024-09-14T23:59:03.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.747 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.748 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:03.748 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.286049+0000", 2024-09-14T23:59:03.748 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7534999999999999e-05 2024-09-14T23:59:03.748 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.748 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:03.748 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.748 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.748 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.748 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56382 3.1d e56/49 3:bcf9a8cf:::rbd_data.1118c4b3aebe.0000000000000006:head v 56'212, mlcod=56'208)", 2024-09-14T23:59:03.748 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.280285+0000", 2024-09-14T23:59:03.748 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 117.42686421800001, 2024-09-14T23:59:03.748 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035393489999999998, 2024-09-14T23:59:03.748 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:03.748 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:03.749 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:03.749 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:03.749 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.749 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:03.749 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280285+0000", 2024-09-14T23:59:03.749 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.749 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.749 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.749 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:03.749 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280285+0000", 2024-09-14T23:59:03.749 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.749 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.749 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.750 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:03.750 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280286+0000", 2024-09-14T23:59:03.750 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5000000000000001e-07 2024-09-14T23:59:03.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.750 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.750 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:03.750 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280299+0000", 2024-09-14T23:59:03.750 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3312000000000001e-05 2024-09-14T23:59:03.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.750 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.750 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:03.750 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280311+0000", 2024-09-14T23:59:03.750 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1677e-05 2024-09-14T23:59:03.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.751 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.751 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:03.751 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280318+0000", 2024-09-14T23:59:03.751 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.742e-06 2024-09-14T23:59:03.751 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.751 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.751 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.751 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280374+0000", 2024-09-14T23:59:03.751 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.592e-05 2024-09-14T23:59:03.751 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.751 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.751 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:03.751 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280392+0000", 2024-09-14T23:59:03.751 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8490000000000001e-05 2024-09-14T23:59:03.752 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.752 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.752 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:03.752 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.283693+0000", 2024-09-14T23:59:03.752 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033012050000000002 2024-09-14T23:59:03.752 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.752 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.752 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:03.752 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.283825+0000", 2024-09-14T23:59:03.752 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013105300000000001 2024-09-14T23:59:03.752 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.752 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:03.752 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.753 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.753 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.753 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "pg_update_log_missing(3.c epoch 56/49 rep_tid 1493 entries 56'200 (0'0) error 3:370203a3:::rbd_data.1118c4b3aebe.000000000000000d:head by client.4376.0:56396 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 56'192)", 2024-09-14T23:59:03.753 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:59:03.753 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 117.426639761, 2024-09-14T23:59:03.753 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057370099999999999, 2024-09-14T23:59:03.753 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:03.753 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:03.753 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:59:03.753 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:03.753 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.753 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:03.753 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:59:03.754 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.754 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.754 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:03.754 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:59:03.754 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.754 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.754 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:03.754 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:59:03.754 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0699999999999997e-07 2024-09-14T23:59:03.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.754 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.755 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:03.755 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280519+0000", 2024-09-14T23:59:03.755 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6260000000000001e-06 2024-09-14T23:59:03.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.755 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:03.755 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280541+0000", 2024-09-14T23:59:03.755 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1976000000000001e-05 2024-09-14T23:59:03.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.755 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:03.755 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280556+0000", 2024-09-14T23:59:03.755 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5075000000000001e-05 2024-09-14T23:59:03.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.756 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.756 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.756 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.285494+0000", 2024-09-14T23:59:03.756 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0049378690000000001 2024-09-14T23:59:03.756 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.756 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.756 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:03.756 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.286247+0000", 2024-09-14T23:59:03.756 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00075285700000000001 2024-09-14T23:59:03.756 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.756 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:03.756 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.756 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.757 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.757 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56400 3.1c e56/49 3:3f5b81ba:::rbd_data.1118c4b3aebe.000000000000000a:head v 56'236, mlcod=56'234)", 2024-09-14T23:59:03.757 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.295064+0000", 2024-09-14T23:59:03.757 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 117.41208496900001, 2024-09-14T23:59:03.757 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0054331589999999999, 2024-09-14T23:59:03.757 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:03.757 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:03.757 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:03.757 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:03.757 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.757 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:03.757 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.295064+0000", 2024-09-14T23:59:03.757 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.758 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.758 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.758 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:03.758 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.295064+0000", 2024-09-14T23:59:03.758 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.758 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.758 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.758 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:03.758 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.295066+0000", 2024-09-14T23:59:03.758 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.502e-06 2024-09-14T23:59:03.758 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.758 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.758 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:03.758 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296403+0000", 2024-09-14T23:59:03.759 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001336554 2024-09-14T23:59:03.759 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.759 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.759 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:03.759 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296428+0000", 2024-09-14T23:59:03.759 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5179000000000001e-05 2024-09-14T23:59:03.759 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.759 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.759 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:03.759 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296440+0000", 2024-09-14T23:59:03.759 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2387e-05 2024-09-14T23:59:03.759 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.759 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.760 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.760 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296554+0000", 2024-09-14T23:59:03.760 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114269 2024-09-14T23:59:03.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.760 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.760 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:03.760 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296581+0000", 2024-09-14T23:59:03.760 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6673999999999999e-05 2024-09-14T23:59:03.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.760 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.760 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:03.760 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.300418+0000", 2024-09-14T23:59:03.760 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0038371249999999998 2024-09-14T23:59:03.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.761 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.761 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:03.761 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.300498+0000", 2024-09-14T23:59:03.761 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9468999999999994e-05 2024-09-14T23:59:03.761 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.761 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:03.761 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.761 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.761 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.761 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:61922 3.5 e57/49 3:a11ebf8e:::rbd_data.11188d356229.000000000000000c:head v 57'280, mlcod=57'277)", 2024-09-14T23:59:03.761 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:07.563144+0000", 2024-09-14T23:59:03.761 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 116.14400531699999, 2024-09-14T23:59:03.761 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003977848, 2024-09-14T23:59:03.761 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:03.762 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:03.762 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:03.762 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:03.762 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.762 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:03.762 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563144+0000", 2024-09-14T23:59:03.762 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.762 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.762 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:03.762 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563144+0000", 2024-09-14T23:59:03.762 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.763 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:03.763 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563145+0000", 2024-09-14T23:59:03.763 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5400000000000001e-07 2024-09-14T23:59:03.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.763 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:03.763 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565056+0000", 2024-09-14T23:59:03.763 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0019108160000000001 2024-09-14T23:59:03.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.763 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:03.763 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565088+0000", 2024-09-14T23:59:03.763 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2586999999999998e-05 2024-09-14T23:59:03.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.764 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.764 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:03.764 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565100+0000", 2024-09-14T23:59:03.764 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1917e-05 2024-09-14T23:59:03.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.764 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.764 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.764 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565166+0000", 2024-09-14T23:59:03.764 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6245e-05 2024-09-14T23:59:03.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.764 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.764 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:03.765 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565190+0000", 2024-09-14T23:59:03.765 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3825999999999999e-05 2024-09-14T23:59:03.765 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.765 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.765 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:03.765 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.567080+0000", 2024-09-14T23:59:03.765 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001889541 2024-09-14T23:59:03.765 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.765 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.765 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:03.765 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.567122+0000", 2024-09-14T23:59:03.765 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2262000000000003e-05 2024-09-14T23:59:03.765 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.765 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:03.766 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.766 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.766 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.766 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:66564 3.c e60/49 3:33faa7ca:::rbd_data.111873792d9b.0000000000000003:head v 60'236, mlcod=57'233)", 2024-09-14T23:59:03.766 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:11.271596+0000", 2024-09-14T23:59:03.766 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 112.435553739, 2024-09-14T23:59:03.766 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0060396099999999999, 2024-09-14T23:59:03.766 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:03.766 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:03.766 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:03.766 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:03.766 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.766 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:03.766 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.271596+0000", 2024-09-14T23:59:03.767 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.767 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.767 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:03.767 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.271596+0000", 2024-09-14T23:59:03.767 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.767 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.767 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:03.767 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.271597+0000", 2024-09-14T23:59:03.767 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2559999999999999e-06 2024-09-14T23:59:03.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.767 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.767 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:03.768 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.274854+0000", 2024-09-14T23:59:03.768 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0032574470000000001 2024-09-14T23:59:03.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.768 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.768 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:03.768 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.274874+0000", 2024-09-14T23:59:03.768 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9449999999999998e-05 2024-09-14T23:59:03.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.768 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.768 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:03.768 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.274883+0000", 2024-09-14T23:59:03.768 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5980000000000003e-06 2024-09-14T23:59:03.769 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.769 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.769 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.769 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.274977+0000", 2024-09-14T23:59:03.769 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3270999999999995e-05 2024-09-14T23:59:03.769 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.769 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.769 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:03.769 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.275000+0000", 2024-09-14T23:59:03.769 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3088e-05 2024-09-14T23:59:03.769 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.769 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.769 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:03.769 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.277590+0000", 2024-09-14T23:59:03.770 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0025898660000000001 2024-09-14T23:59:03.770 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.770 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.770 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:03.770 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.277635+0000", 2024-09-14T23:59:03.770 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5633999999999997e-05 2024-09-14T23:59:03.770 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.770 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:03.770 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.770 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.770 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.770 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:133384 3.1f e92/49 3:fce5e529:::rbd_data.11186132a8b6.0000000000000002:head v 92'807, mlcod=89'804)", 2024-09-14T23:59:03.770 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:45.293642+0000", 2024-09-14T23:59:03.771 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 78.413507855999995, 2024-09-14T23:59:03.771 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003760186, 2024-09-14T23:59:03.771 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:03.771 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:03.771 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:03.771 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:03.771 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.771 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:03.771 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.293642+0000", 2024-09-14T23:59:03.771 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.771 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.771 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.771 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:03.771 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.293642+0000", 2024-09-14T23:59:03.772 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.772 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.772 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.772 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:03.772 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.293643+0000", 2024-09-14T23:59:03.772 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2500000000000001e-06 2024-09-14T23:59:03.772 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.772 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.772 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:03.772 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.295402+0000", 2024-09-14T23:59:03.772 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017595410000000001 2024-09-14T23:59:03.772 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.773 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.773 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:03.773 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.295422+0000", 2024-09-14T23:59:03.773 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9763999999999999e-05 2024-09-14T23:59:03.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.773 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.773 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:03.773 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.295436+0000", 2024-09-14T23:59:03.773 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.364e-05 2024-09-14T23:59:03.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.773 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.773 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.773 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.295499+0000", 2024-09-14T23:59:03.774 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2770999999999999e-05 2024-09-14T23:59:03.774 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.774 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.774 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:03.774 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.295516+0000", 2024-09-14T23:59:03.774 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7240000000000001e-05 2024-09-14T23:59:03.774 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.774 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.774 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:03.774 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.297324+0000", 2024-09-14T23:59:03.774 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001807857 2024-09-14T23:59:03.774 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.774 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.774 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:03.774 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.297402+0000", 2024-09-14T23:59:03.775 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8122999999999997e-05 2024-09-14T23:59:03.775 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.775 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:03.775 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.775 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.775 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.775 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:156597 3.5 e103/49 3:a23c5924:::rbd_data.1118b2f495e6.0000000000000011:head v 103'518, mlcod=98'516)", 2024-09-14T23:59:03.775 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:58.656197+0000", 2024-09-14T23:59:03.775 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 65.050952799000001, 2024-09-14T23:59:03.775 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0040423029999999997, 2024-09-14T23:59:03.775 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:03.775 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:03.775 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:03.776 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:03.776 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.776 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:03.776 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656197+0000", 2024-09-14T23:59:03.776 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.776 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.776 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:03.776 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656197+0000", 2024-09-14T23:59:03.776 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.776 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.777 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:03.777 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656197+0000", 2024-09-14T23:59:03.777 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1900000000000002e-07 2024-09-14T23:59:03.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.777 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.777 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:03.777 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656676+0000", 2024-09-14T23:59:03.777 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00047921699999999999 2024-09-14T23:59:03.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.777 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.777 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:03.777 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656690+0000", 2024-09-14T23:59:03.777 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3916e-05 2024-09-14T23:59:03.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.778 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.778 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:03.778 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656698+0000", 2024-09-14T23:59:03.778 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6529999999999994e-06 2024-09-14T23:59:03.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.778 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.778 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.778 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656757+0000", 2024-09-14T23:59:03.778 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9237999999999998e-05 2024-09-14T23:59:03.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.778 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.778 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:03.778 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656780+0000", 2024-09-14T23:59:03.778 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2968000000000001e-05 2024-09-14T23:59:03.779 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.779 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.779 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:03.779 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.660188+0000", 2024-09-14T23:59:03.779 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0034078630000000001 2024-09-14T23:59:03.779 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.779 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.779 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:03.779 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.660239+0000", 2024-09-14T23:59:03.779 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0828999999999997e-05 2024-09-14T23:59:03.779 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.779 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:03.779 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.780 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.780 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.780 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:157947 3.1c e103/49 3:383374fe:::rbd_data.1118b2f495e6.000000000000000c:head v 103'771, mlcod=103'769)", 2024-09-14T23:59:03.780 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:58.765271+0000", 2024-09-14T23:59:03.780 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 64.941877918000003, 2024-09-14T23:59:03.780 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0040869829999999998, 2024-09-14T23:59:03.780 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:03.780 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:03.780 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:03.780 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:03.780 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.780 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:03.780 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765271+0000", 2024-09-14T23:59:03.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.781 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:03.781 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765271+0000", 2024-09-14T23:59:03.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.781 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:03.781 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765273+0000", 2024-09-14T23:59:03.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1149999999999999e-06 2024-09-14T23:59:03.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.782 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:03.782 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765854+0000", 2024-09-14T23:59:03.782 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00058119599999999997 2024-09-14T23:59:03.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.782 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:03.782 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765869+0000", 2024-09-14T23:59:03.782 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5641000000000002e-05 2024-09-14T23:59:03.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.782 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:03.782 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765878+0000", 2024-09-14T23:59:03.782 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4519999999999997e-06 2024-09-14T23:59:03.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.783 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.783 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.783 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765942+0000", 2024-09-14T23:59:03.783 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3727999999999997e-05 2024-09-14T23:59:03.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.783 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.783 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:03.783 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765968+0000", 2024-09-14T23:59:03.783 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6171999999999999e-05 2024-09-14T23:59:03.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.783 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.783 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:03.783 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.769319+0000", 2024-09-14T23:59:03.784 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033511629999999999 2024-09-14T23:59:03.784 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.784 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.784 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:03.784 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.769358+0000", 2024-09-14T23:59:03.784 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9515999999999999e-05 2024-09-14T23:59:03.784 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.784 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:03.784 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.784 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.784 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.784 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:160196 3.c e103/49 3:34623e68:::rbd_data.1118b2f495e6.0000000000000005:head v 103'551, mlcod=102'549)", 2024-09-14T23:59:03.784 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:59.131433+0000", 2024-09-14T23:59:03.784 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 64.575716608999997, 2024-09-14T23:59:03.785 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0036932530000000001, 2024-09-14T23:59:03.785 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:03.785 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:03.785 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:03.785 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:03.785 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.785 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:03.785 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.131433+0000", 2024-09-14T23:59:03.785 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.785 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.785 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.785 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:03.785 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.131433+0000", 2024-09-14T23:59:03.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:03.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.131433+0000", 2024-09-14T23:59:03.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8500000000000001e-07 2024-09-14T23:59:03.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:03.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.132144+0000", 2024-09-14T23:59:03.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00071022800000000003 2024-09-14T23:59:03.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:03.787 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.132154+0000", 2024-09-14T23:59:03.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0301e-05 2024-09-14T23:59:03.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.787 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:03.787 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.132159+0000", 2024-09-14T23:59:03.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6280000000000004e-06 2024-09-14T23:59:03.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.787 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.787 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.132208+0000", 2024-09-14T23:59:03.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9595000000000002e-05 2024-09-14T23:59:03.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.788 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:03.788 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.132231+0000", 2024-09-14T23:59:03.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3010000000000002e-05 2024-09-14T23:59:03.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.788 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:03.788 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.135087+0000", 2024-09-14T23:59:03.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0028553239999999998 2024-09-14T23:59:03.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.788 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:03.788 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.135126+0000", 2024-09-14T23:59:03.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9481999999999999e-05 2024-09-14T23:59:03.789 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.789 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:03.789 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.789 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188871 3.d 3:b35cff4d:::rbd_header.1118c900944f:head [watch reconnect cookie 140670715433840 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-14T23:59:03.789 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.970653+0000", 2024-09-14T23:59:03.789 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 55.736496875999997, 2024-09-14T23:59:03.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0727808270000001, 2024-09-14T23:59:03.789 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:03.789 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:03.789 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:03.789 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:03.790 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:03.790 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:03.790 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188871 2024-09-14T23:59:03.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.790 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:03.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:03.790 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970653+0000", 2024-09-14T23:59:03.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:03.790 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970653+0000", 2024-09-14T23:59:03.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.791 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:03.791 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970655+0000", 2024-09-14T23:59:03.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.277e-06 2024-09-14T23:59:03.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.791 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:03.791 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970662+0000", 2024-09-14T23:59:03.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9619999999999996e-06 2024-09-14T23:59:03.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.791 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:03.791 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970682+0000", 2024-09-14T23:59:03.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0279999999999999e-05 2024-09-14T23:59:03.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.792 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:03.792 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970691+0000", 2024-09-14T23:59:03.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2790000000000004e-06 2024-09-14T23:59:03.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.792 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.792 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970762+0000", 2024-09-14T23:59:03.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1019999999999994e-05 2024-09-14T23:59:03.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.792 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-14T23:59:03.793 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970770+0000", 2024-09-14T23:59:03.793 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7999999999999999e-06 2024-09-14T23:59:03.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.793 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.793 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.973516+0000", 2024-09-14T23:59:03.793 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0027456469999999999 2024-09-14T23:59:03.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.793 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:03.793 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.973525+0000", 2024-09-14T23:59:03.793 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4890000000000001e-06 2024-09-14T23:59:03.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.794 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.794 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.041451+0000", 2024-09-14T23:59:03.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.067926012 2024-09-14T23:59:03.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.794 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:03.794 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.041673+0000", 2024-09-14T23:59:03.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000222048 2024-09-14T23:59:03.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.794 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:03.794 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.043433+0000", 2024-09-14T23:59:03.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017600129999999999 2024-09-14T23:59:03.795 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.795 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:03.795 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.795 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(osd.0.0:3238 3.1f e122/49 3:f8a59e83:::rbd_data.11187b9f1032.0000000000000006:2f v 122'943, mlcod=120'940)", 2024-09-14T23:59:03.795 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:22.285283+0000", 2024-09-14T23:59:03.795 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 41.421866053000002, 2024-09-14T23:59:03.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003756194, 2024-09-14T23:59:03.795 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:03.795 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:03.795 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:03.795 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:03.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:03.796 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.285283+0000", 2024-09-14T23:59:03.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.796 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:03.796 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.285283+0000", 2024-09-14T23:59:03.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.796 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:03.796 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.285284+0000", 2024-09-14T23:59:03.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5600000000000002e-07 2024-09-14T23:59:03.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.797 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:03.797 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.285289+0000", 2024-09-14T23:59:03.797 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9579999999999998e-06 2024-09-14T23:59:03.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.797 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:03.797 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.285300+0000", 2024-09-14T23:59:03.797 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1688999999999999e-05 2024-09-14T23:59:03.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.797 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:03.797 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.285309+0000", 2024-09-14T23:59:03.797 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.388e-06 2024-09-14T23:59:03.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.285367+0000", 2024-09-14T23:59:03.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8811000000000003e-05 2024-09-14T23:59:03.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:03.801 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.285388+0000", 2024-09-14T23:59:03.801 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0117000000000001e-05 2024-09-14T23:59:03.801 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.801 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.801 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:03.801 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.288994+0000", 2024-09-14T23:59:03.801 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0036064270000000002 2024-09-14T23:59:03.801 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.801 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.801 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:03.801 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.289040+0000", 2024-09-14T23:59:03.801 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5547999999999998e-05 2024-09-14T23:59:03.801 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.802 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:03.802 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.802 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.802 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(osd.6.0:1517 3.d e127/112 3:b69c12aa:::rbd_data.1118f9c96a7.0000000000000007:2c v 127'452, mlcod=126'449)", 2024-09-14T23:59:03.802 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:27.316914+0000", 2024-09-14T23:59:03.802 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.390235445000002, 2024-09-14T23:59:03.802 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0052453980000000001, 2024-09-14T23:59:03.802 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:03.802 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:03.802 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:03.802 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:03.802 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.802 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:03.803 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.316914+0000", 2024-09-14T23:59:03.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.803 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:03.803 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.316914+0000", 2024-09-14T23:59:03.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.803 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:03.803 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.316914+0000", 2024-09-14T23:59:03.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4900000000000002e-07 2024-09-14T23:59:03.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:03.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.316920+0000", 2024-09-14T23:59:03.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7169999999999996e-06 2024-09-14T23:59:03.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:03.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.316928+0000", 2024-09-14T23:59:03.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3769999999999996e-06 2024-09-14T23:59:03.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:03.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.316938+0000", 2024-09-14T23:59:03.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7329999999999998e-06 2024-09-14T23:59:03.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.805 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.805 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.805 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.317033+0000", 2024-09-14T23:59:03.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5295000000000002e-05 2024-09-14T23:59:03.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.805 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.805 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:03.805 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.317057+0000", 2024-09-14T23:59:03.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3989999999999999e-05 2024-09-14T23:59:03.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.805 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.805 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:03.805 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.322122+0000", 2024-09-14T23:59:03.806 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0050642500000000002 2024-09-14T23:59:03.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.806 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.806 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:03.806 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.322159+0000", 2024-09-14T23:59:03.806 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7787000000000003e-05 2024-09-14T23:59:03.806 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.806 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:03.806 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.806 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.806 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(osd.1.0:1466 3.16 e127/112 3:6c7910cf:::rbd_data.1118f9c96a7.0000000000000006:2c v 127'376, mlcod=121'372)", 2024-09-14T23:59:03.807 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:27.317464+0000", 2024-09-14T23:59:03.807 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.389685411999999, 2024-09-14T23:59:03.807 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0050844499999999999, 2024-09-14T23:59:03.807 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:03.807 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:03.807 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:03.807 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:03.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.807 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:03.807 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.317464+0000", 2024-09-14T23:59:03.807 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.808 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.808 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:03.808 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.317464+0000", 2024-09-14T23:59:03.808 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:03.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.808 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.808 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:03.808 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.317464+0000", 2024-09-14T23:59:03.808 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.91e-07 2024-09-14T23:59:03.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.808 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.808 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:03.808 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.317469+0000", 2024-09-14T23:59:03.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4719999999999997e-06 2024-09-14T23:59:03.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.809 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:03.809 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.317481+0000", 2024-09-14T23:59:03.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1814000000000001e-05 2024-09-14T23:59:03.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.809 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:03.809 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.317488+0000", 2024-09-14T23:59:03.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2740000000000002e-06 2024-09-14T23:59:03.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.810 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:03.810 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.317584+0000", 2024-09-14T23:59:03.810 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6191000000000001e-05 2024-09-14T23:59:03.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.810 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.810 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:03.810 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.317605+0000", 2024-09-14T23:59:03.810 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0554e-05 2024-09-14T23:59:03.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.810 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.810 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:03.810 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.322504+0000", 2024-09-14T23:59:03.810 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0048991089999999996 2024-09-14T23:59:03.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:03.811 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:03.811 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:03.811 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.322548+0000", 2024-09-14T23:59:03.811 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4744999999999999e-05 2024-09-14T23:59:03.811 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.811 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:03.811 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.811 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:03.811 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:03.811 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:59:03.813 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T23:59:03.813 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:59:03.811+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:59:03.914 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:59:03.912+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:59:03.921 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:59:03.922 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:59:03.922 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:59:03.922 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:59:03.930 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T23:59:04.015 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:59:04.013+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:59:04.078 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:59:04.078 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:59:04.078 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:59:04.078 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:59:04.078 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:59:04.087 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T23:59:04.121 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T23:59:04.132 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:59:04.131+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:59:04.217 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:59:04.217 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:59:04.217 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:59:04.217 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:59:04.218 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.218 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:107472 3.1e 3:7bf21588:::rbd_data.11187de8b0a5.000000000000005d:head [set-alloc-hint object_size 2097152 write_size 2097152,write 317788~1491] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-14T23:59:04.218 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:25.198493+0000", 2024-09-14T23:59:04.218 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 99.016369874000006, 2024-09-14T23:59:04.218 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.78175149300000002, 2024-09-14T23:59:04.218 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:04.218 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:04.218 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:04.218 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:04.218 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:04.218 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:04.218 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 107472 2024-09-14T23:59:04.218 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.218 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:04.219 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.219 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:04.219 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198493+0000", 2024-09-14T23:59:04.219 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.219 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.219 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.219 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:04.219 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198493+0000", 2024-09-14T23:59:04.219 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.219 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.219 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.219 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:04.219 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198495+0000", 2024-09-14T23:59:04.219 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.826e-06 2024-09-14T23:59:04.220 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.220 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.220 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:04.220 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198503+0000", 2024-09-14T23:59:04.220 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0150000000000003e-06 2024-09-14T23:59:04.220 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.220 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.220 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:04.220 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198521+0000", 2024-09-14T23:59:04.220 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8374000000000001e-05 2024-09-14T23:59:04.220 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.220 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.220 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:04.221 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198533+0000", 2024-09-14T23:59:04.221 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1593000000000001e-05 2024-09-14T23:59:04.221 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.221 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.221 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:04.221 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198603+0000", 2024-09-14T23:59:04.221 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0279999999999998e-05 2024-09-14T23:59:04.221 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.221 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.221 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:04.221 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198617+0000", 2024-09-14T23:59:04.221 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3517e-05 2024-09-14T23:59:04.221 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.221 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.222 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:04.222 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.977067+0000", 2024-09-14T23:59:04.222 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.77845065800000002 2024-09-14T23:59:04.222 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.222 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.222 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for missing object", 2024-09-14T23:59:04.222 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.977173+0000", 2024-09-14T23:59:04.222 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010543999999999999 2024-09-14T23:59:04.222 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.222 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.222 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:04.222 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.978737+0000", 2024-09-14T23:59:04.222 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001564078 2024-09-14T23:59:04.223 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.223 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.223 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:04.223 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.978845+0000", 2024-09-14T23:59:04.223 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000108643 2024-09-14T23:59:04.223 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.223 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.223 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1", 2024-09-14T23:59:04.223 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.978964+0000", 2024-09-14T23:59:04.223 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000119085 2024-09-14T23:59:04.223 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.223 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.223 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:59:04.223 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980109+0000", 2024-09-14T23:59:04.224 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0011442170000000001 2024-09-14T23:59:04.224 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.224 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.224 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:59:04.224 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980130+0000", 2024-09-14T23:59:04.224 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1766000000000001e-05 2024-09-14T23:59:04.224 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.224 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.224 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:04.224 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980171+0000", 2024-09-14T23:59:04.224 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0627999999999998e-05 2024-09-14T23:59:04.224 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.224 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.225 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:04.225 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980244+0000", 2024-09-14T23:59:04.225 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3373000000000004e-05 2024-09-14T23:59:04.225 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.225 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:04.225 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.225 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.225 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.225 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129159 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 5996544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:04.225 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.927783+0000", 2024-09-14T23:59:04.225 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 85.287079982999998, 2024-09-14T23:59:04.225 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054661122999999999, 2024-09-14T23:59:04.225 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:04.225 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:04.226 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:04.226 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:04.226 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:04.226 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:04.226 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129159 2024-09-14T23:59:04.226 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.226 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:04.226 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.226 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:04.226 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.927783+0000", 2024-09-14T23:59:04.226 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.226 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.226 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.226 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:04.226 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.927783+0000", 2024-09-14T23:59:04.227 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.227 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.227 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.227 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:04.227 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929684+0000", 2024-09-14T23:59:04.227 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0019011080000000001 2024-09-14T23:59:04.227 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.227 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.227 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:04.227 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929688+0000", 2024-09-14T23:59:04.227 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2610000000000004e-06 2024-09-14T23:59:04.227 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.227 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.228 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:04.228 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929697+0000", 2024-09-14T23:59:04.228 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0219999999999993e-06 2024-09-14T23:59:04.228 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.228 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.228 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:04.228 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929702+0000", 2024-09-14T23:59:04.228 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8840000000000002e-06 2024-09-14T23:59:04.228 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.228 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.228 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:04.228 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.982090+0000", 2024-09-14T23:59:04.228 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052388045000000001 2024-09-14T23:59:04.228 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.229 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.229 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:04.229 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.982208+0000", 2024-09-14T23:59:04.229 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118236 2024-09-14T23:59:04.229 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.229 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.229 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:04.229 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.982444+0000", 2024-09-14T23:59:04.229 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000235567 2024-09-14T23:59:04.229 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.229 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:04.229 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.229 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.229 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.229 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129166 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 8192000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:04.230 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.929794+0000", 2024-09-14T23:59:04.230 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 85.285069231999998, 2024-09-14T23:59:04.230 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054471973, 2024-09-14T23:59:04.230 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:04.230 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:04.230 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:04.230 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:04.230 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:04.230 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:04.230 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129166 2024-09-14T23:59:04.230 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.230 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:04.230 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.230 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:04.231 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929794+0000", 2024-09-14T23:59:04.231 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.231 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.231 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.231 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:04.231 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929794+0000", 2024-09-14T23:59:04.231 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.231 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.231 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.231 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:04.231 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929794+0000", 2024-09-14T23:59:04.231 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6899999999999997e-07 2024-09-14T23:59:04.231 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.231 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.232 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:04.232 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929797+0000", 2024-09-14T23:59:04.232 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.328e-06 2024-09-14T23:59:04.232 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.232 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.232 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:04.232 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929802+0000", 2024-09-14T23:59:04.232 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7309999999999998e-06 2024-09-14T23:59:04.232 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.232 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.232 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:04.232 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929807+0000", 2024-09-14T23:59:04.233 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4349999999999999e-06 2024-09-14T23:59:04.233 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.233 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.233 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:04.233 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984063+0000", 2024-09-14T23:59:04.233 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054256331999999997 2024-09-14T23:59:04.233 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.233 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.233 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:04.233 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984177+0000", 2024-09-14T23:59:04.233 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114225 2024-09-14T23:59:04.233 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.233 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.233 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:04.233 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984266+0000", 2024-09-14T23:59:04.234 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8352999999999999e-05 2024-09-14T23:59:04.234 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.234 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:04.234 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.234 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.234 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.234 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129167 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 6782976~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:04.234 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.929827+0000", 2024-09-14T23:59:04.234 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 85.285035823000001, 2024-09-14T23:59:04.234 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054637593999999998, 2024-09-14T23:59:04.234 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:04.234 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:04.234 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:04.234 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:04.235 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:04.235 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:04.235 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129167 2024-09-14T23:59:04.235 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.235 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:04.235 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.235 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:04.235 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929827+0000", 2024-09-14T23:59:04.235 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.235 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.235 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.235 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:04.235 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929827+0000", 2024-09-14T23:59:04.235 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.235 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.236 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.236 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:04.236 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929827+0000", 2024-09-14T23:59:04.236 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4000000000000002e-07 2024-09-14T23:59:04.236 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.236 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.236 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:04.236 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929830+0000", 2024-09-14T23:59:04.236 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.419e-06 2024-09-14T23:59:04.236 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.236 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.236 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:04.236 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929837+0000", 2024-09-14T23:59:04.237 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0450000000000003e-06 2024-09-14T23:59:04.237 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.237 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.237 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:04.237 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929842+0000", 2024-09-14T23:59:04.237 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.69e-06 2024-09-14T23:59:04.237 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.237 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.237 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:04.237 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984278+0000", 2024-09-14T23:59:04.237 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054436516999999997 2024-09-14T23:59:04.237 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.238 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.238 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:04.238 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984331+0000", 2024-09-14T23:59:04.238 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2590999999999998e-05 2024-09-14T23:59:04.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.238 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.238 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:04.238 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984465+0000", 2024-09-14T23:59:04.238 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013389200000000001 2024-09-14T23:59:04.238 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.238 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:04.238 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.238 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.239 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129168 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 8224768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:04.239 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.929849+0000", 2024-09-14T23:59:04.239 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 85.285014265000001, 2024-09-14T23:59:04.239 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054870368000000003, 2024-09-14T23:59:04.239 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:04.239 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:04.239 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:04.239 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:04.239 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:04.239 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:04.239 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129168 2024-09-14T23:59:04.239 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.239 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:04.239 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.240 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:04.240 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929849+0000", 2024-09-14T23:59:04.240 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.240 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.240 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.240 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:04.240 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929849+0000", 2024-09-14T23:59:04.240 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.240 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.240 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.240 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:04.240 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931681+0000", 2024-09-14T23:59:04.240 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018326989999999999 2024-09-14T23:59:04.241 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.241 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.241 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:04.241 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931686+0000", 2024-09-14T23:59:04.241 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7190000000000001e-06 2024-09-14T23:59:04.241 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.241 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.241 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:04.241 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931697+0000", 2024-09-14T23:59:04.241 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1287999999999999e-05 2024-09-14T23:59:04.241 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.241 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.241 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:04.241 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931704+0000", 2024-09-14T23:59:04.242 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3339999999999998e-06 2024-09-14T23:59:04.242 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.242 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.242 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:04.242 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984505+0000", 2024-09-14T23:59:04.242 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052801684000000002 2024-09-14T23:59:04.242 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.242 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.242 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:04.242 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984621+0000", 2024-09-14T23:59:04.242 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000116165 2024-09-14T23:59:04.242 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.242 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.242 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:04.242 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984719+0000", 2024-09-14T23:59:04.243 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7479000000000006e-05 2024-09-14T23:59:04.243 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.243 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:04.243 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.243 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.243 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.243 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129176 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 10452992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:04.243 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.931830+0000", 2024-09-14T23:59:04.243 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 85.283032504000005, 2024-09-14T23:59:04.243 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054513292999999997, 2024-09-14T23:59:04.243 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:04.243 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:04.243 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:04.244 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:04.244 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:04.244 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:04.244 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129176 2024-09-14T23:59:04.244 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.244 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:04.244 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.244 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:04.244 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931830+0000", 2024-09-14T23:59:04.244 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.244 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.244 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.244 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:04.244 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931830+0000", 2024-09-14T23:59:04.245 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.245 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.245 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.245 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:04.245 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.933678+0000", 2024-09-14T23:59:04.245 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018472440000000001 2024-09-14T23:59:04.245 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.245 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.245 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:04.245 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.933686+0000", 2024-09-14T23:59:04.245 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0239999999999997e-06 2024-09-14T23:59:04.245 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.245 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.245 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:04.246 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.933694+0000", 2024-09-14T23:59:04.246 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8400000000000001e-06 2024-09-14T23:59:04.246 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.246 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.246 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:04.246 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.933701+0000", 2024-09-14T23:59:04.246 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2870000000000001e-06 2024-09-14T23:59:04.246 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.246 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.246 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:04.246 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.986165+0000", 2024-09-14T23:59:04.246 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052463900000000001 2024-09-14T23:59:04.246 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.247 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.247 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:04.247 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.986259+0000", 2024-09-14T23:59:04.247 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4435e-05 2024-09-14T23:59:04.247 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.247 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.247 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:04.247 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.986344+0000", 2024-09-14T23:59:04.247 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4562999999999997e-05 2024-09-14T23:59:04.247 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.247 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:04.247 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.247 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.247 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.247 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129201 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 10289152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:04.248 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.935960+0000", 2024-09-14T23:59:04.248 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 85.278903041999996, 2024-09-14T23:59:04.248 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054338487999999997, 2024-09-14T23:59:04.248 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:04.248 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:04.248 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:04.248 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:04.248 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:04.248 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:04.248 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129201 2024-09-14T23:59:04.248 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.248 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:04.248 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.248 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:04.249 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935960+0000", 2024-09-14T23:59:04.249 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.249 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.249 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.249 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:04.249 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935960+0000", 2024-09-14T23:59:04.249 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.249 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.249 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.249 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:04.249 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935960+0000", 2024-09-14T23:59:04.249 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.72e-07 2024-09-14T23:59:04.249 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.250 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.250 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:04.250 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935963+0000", 2024-09-14T23:59:04.250 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5390000000000002e-06 2024-09-14T23:59:04.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.250 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.250 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:04.250 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935971+0000", 2024-09-14T23:59:04.250 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4589999999999998e-06 2024-09-14T23:59:04.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.250 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.250 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:04.250 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935977+0000", 2024-09-14T23:59:04.250 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4110000000000004e-06 2024-09-14T23:59:04.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.251 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.251 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:04.251 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990213+0000", 2024-09-14T23:59:04.251 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054236642000000002 2024-09-14T23:59:04.251 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.251 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.251 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:04.251 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990257+0000", 2024-09-14T23:59:04.251 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3696000000000003e-05 2024-09-14T23:59:04.251 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.251 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.251 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:04.251 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990298+0000", 2024-09-14T23:59:04.251 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1369000000000003e-05 2024-09-14T23:59:04.252 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.252 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:04.252 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.252 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.252 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.252 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129202 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 11370496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:04.252 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.935986+0000", 2024-09-14T23:59:04.252 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 85.278876424000003, 2024-09-14T23:59:04.252 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054543700000000001, 2024-09-14T23:59:04.252 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:04.252 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:04.252 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:04.252 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:04.252 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:04.253 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:04.253 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129202 2024-09-14T23:59:04.253 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.253 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:04.253 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.253 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:04.253 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935986+0000", 2024-09-14T23:59:04.253 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.253 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.253 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.253 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:04.253 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935986+0000", 2024-09-14T23:59:04.253 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.254 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.254 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.254 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:04.254 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937678+0000", 2024-09-14T23:59:04.254 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016917849999999999 2024-09-14T23:59:04.254 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.254 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.254 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:04.254 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937682+0000", 2024-09-14T23:59:04.254 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2400000000000001e-06 2024-09-14T23:59:04.254 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.254 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.254 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:04.254 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937695+0000", 2024-09-14T23:59:04.254 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2999e-05 2024-09-14T23:59:04.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.255 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:04.255 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937700+0000", 2024-09-14T23:59:04.255 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8420000000000004e-06 2024-09-14T23:59:04.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.255 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:04.255 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990336+0000", 2024-09-14T23:59:04.255 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052636060999999998 2024-09-14T23:59:04.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.255 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:04.255 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990435+0000", 2024-09-14T23:59:04.256 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9015e-05 2024-09-14T23:59:04.256 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.256 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.256 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:04.256 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990530+0000", 2024-09-14T23:59:04.256 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4758000000000003e-05 2024-09-14T23:59:04.256 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.256 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:04.256 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.256 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.256 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.256 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129214 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 11468800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:04.256 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.937939+0000", 2024-09-14T23:59:04.257 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 85.276923510000003, 2024-09-14T23:59:04.257 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054463733, 2024-09-14T23:59:04.257 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:04.257 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:04.257 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:04.257 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:04.257 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:04.257 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:04.257 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129214 2024-09-14T23:59:04.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.257 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:04.257 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.257 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:04.257 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937939+0000", 2024-09-14T23:59:04.258 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.258 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.258 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:04.258 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937939+0000", 2024-09-14T23:59:04.258 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.258 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.258 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:04.258 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939674+0000", 2024-09-14T23:59:04.258 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017342589999999999 2024-09-14T23:59:04.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.258 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.259 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:04.259 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939678+0000", 2024-09-14T23:59:04.259 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2590000000000002e-06 2024-09-14T23:59:04.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.259 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.259 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:04.259 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939699+0000", 2024-09-14T23:59:04.259 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1143000000000001e-05 2024-09-14T23:59:04.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.259 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.259 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:04.259 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939706+0000", 2024-09-14T23:59:04.259 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0269999999999998e-06 2024-09-14T23:59:04.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.260 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.260 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:04.260 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.992221+0000", 2024-09-14T23:59:04.260 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052514981000000002 2024-09-14T23:59:04.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.260 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.260 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:04.260 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.992321+0000", 2024-09-14T23:59:04.260 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9625999999999998e-05 2024-09-14T23:59:04.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.260 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.260 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:04.260 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.992403+0000", 2024-09-14T23:59:04.261 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2437999999999999e-05 2024-09-14T23:59:04.261 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.261 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:04.261 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.261 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.261 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129226 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 14090240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:04.261 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.939867+0000", 2024-09-14T23:59:04.261 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 85.274995582000003, 2024-09-14T23:59:04.261 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054523573999999998, 2024-09-14T23:59:04.261 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:04.261 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:04.261 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:04.261 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:04.262 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:04.262 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:04.262 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129226 2024-09-14T23:59:04.262 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.262 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:04.262 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.262 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:04.262 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939867+0000", 2024-09-14T23:59:04.262 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.262 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.262 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.262 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:04.262 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939867+0000", 2024-09-14T23:59:04.262 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.262 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.263 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:04.263 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941669+0000", 2024-09-14T23:59:04.263 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018021549999999999 2024-09-14T23:59:04.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.263 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:04.263 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941673+0000", 2024-09-14T23:59:04.263 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1269999999999999e-06 2024-09-14T23:59:04.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.263 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:04.263 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941679+0000", 2024-09-14T23:59:04.263 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1879999999999999e-06 2024-09-14T23:59:04.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.264 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.264 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:04.264 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941682+0000", 2024-09-14T23:59:04.264 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7110000000000001e-06 2024-09-14T23:59:04.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.264 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.264 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:04.264 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.994201+0000", 2024-09-14T23:59:04.264 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052518265000000001 2024-09-14T23:59:04.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.264 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.264 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:04.264 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.994302+0000", 2024-09-14T23:59:04.265 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101653 2024-09-14T23:59:04.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.265 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:04.265 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.994391+0000", 2024-09-14T23:59:04.265 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8474999999999996e-05 2024-09-14T23:59:04.265 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.265 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:04.265 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.265 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129237 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 12386304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:04.265 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.941825+0000", 2024-09-14T23:59:04.265 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 85.273037819999999, 2024-09-14T23:59:04.266 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054368076000000001, 2024-09-14T23:59:04.266 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:04.266 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:04.266 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:04.266 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:04.266 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:04.266 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:04.266 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129237 2024-09-14T23:59:04.266 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.266 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:04.266 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.266 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:04.266 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941825+0000", 2024-09-14T23:59:04.266 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.267 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.267 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:04.267 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941825+0000", 2024-09-14T23:59:04.267 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.267 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.267 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:04.267 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941825+0000", 2024-09-14T23:59:04.267 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.29e-07 2024-09-14T23:59:04.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.267 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.267 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:04.267 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941826+0000", 2024-09-14T23:59:04.268 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0999999999999997e-07 2024-09-14T23:59:04.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.268 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.268 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:04.268 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941834+0000", 2024-09-14T23:59:04.268 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5519999999999999e-06 2024-09-14T23:59:04.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.268 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.268 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:04.268 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941843+0000", 2024-09-14T23:59:04.268 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2960000000000004e-06 2024-09-14T23:59:04.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.268 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.268 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:04.269 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996108+0000", 2024-09-14T23:59:04.269 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054265200999999999 2024-09-14T23:59:04.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.269 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.269 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:04.269 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996152+0000", 2024-09-14T23:59:04.269 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4066999999999997e-05 2024-09-14T23:59:04.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.269 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.269 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:04.269 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996193+0000", 2024-09-14T23:59:04.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0821000000000001e-05 2024-09-14T23:59:04.270 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.270 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:04.270 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.270 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.270 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129238 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 13828096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:04.270 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.941851+0000", 2024-09-14T23:59:04.270 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 85.273011495999995, 2024-09-14T23:59:04.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.05458031, 2024-09-14T23:59:04.270 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:04.270 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:04.270 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:04.270 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:04.271 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:04.271 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:04.271 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129238 2024-09-14T23:59:04.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.271 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:04.271 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.271 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:04.271 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941851+0000", 2024-09-14T23:59:04.271 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.271 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.271 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:04.271 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941851+0000", 2024-09-14T23:59:04.271 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.272 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.272 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.272 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:04.272 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943670+0000", 2024-09-14T23:59:04.272 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018181930000000001 2024-09-14T23:59:04.272 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.272 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.272 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:04.272 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943673+0000", 2024-09-14T23:59:04.272 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3790000000000002e-06 2024-09-14T23:59:04.272 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.272 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.272 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:04.272 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943679+0000", 2024-09-14T23:59:04.273 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8719999999999999e-06 2024-09-14T23:59:04.273 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.273 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.273 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:04.273 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943683+0000", 2024-09-14T23:59:04.273 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8539999999999999e-06 2024-09-14T23:59:04.273 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.273 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.273 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:04.273 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996233+0000", 2024-09-14T23:59:04.273 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052550690999999997 2024-09-14T23:59:04.273 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.273 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.274 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:04.274 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996339+0000", 2024-09-14T23:59:04.274 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105989 2024-09-14T23:59:04.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.274 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:04.274 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996432+0000", 2024-09-14T23:59:04.274 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2331999999999996e-05 2024-09-14T23:59:04.274 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.274 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:04.274 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.274 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129249 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 12124160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:04.275 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.943826+0000", 2024-09-14T23:59:04.275 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 85.271037004999997, 2024-09-14T23:59:04.275 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054373266000000003, 2024-09-14T23:59:04.275 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:04.275 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:04.275 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:04.275 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:04.275 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:04.275 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:04.275 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129249 2024-09-14T23:59:04.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.275 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:04.275 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.275 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:04.276 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943826+0000", 2024-09-14T23:59:04.276 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.276 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.276 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.276 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:04.276 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943826+0000", 2024-09-14T23:59:04.276 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.276 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.276 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.276 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:04.276 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943826+0000", 2024-09-14T23:59:04.276 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.03e-07 2024-09-14T23:59:04.276 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.276 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.277 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:04.277 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943827+0000", 2024-09-14T23:59:04.277 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.08e-06 2024-09-14T23:59:04.277 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.277 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.277 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:04.277 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943834+0000", 2024-09-14T23:59:04.277 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4119999999999998e-06 2024-09-14T23:59:04.277 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.277 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.277 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:04.277 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943836+0000", 2024-09-14T23:59:04.277 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7369999999999998e-06 2024-09-14T23:59:04.277 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.278 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.278 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:04.278 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998115+0000", 2024-09-14T23:59:04.278 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054279040000000001 2024-09-14T23:59:04.278 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.278 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.278 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:04.278 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998159+0000", 2024-09-14T23:59:04.278 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3486999999999999e-05 2024-09-14T23:59:04.278 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.278 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.278 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:04.278 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998199+0000", 2024-09-14T23:59:04.279 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0306999999999999e-05 2024-09-14T23:59:04.279 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.279 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:04.279 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.279 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.279 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.279 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129250 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 13565952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:04.279 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.943840+0000", 2024-09-14T23:59:04.279 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 85.271023055000001, 2024-09-14T23:59:04.279 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054573875000000001, 2024-09-14T23:59:04.279 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:04.279 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:04.279 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:04.279 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:04.279 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:04.280 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:04.280 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129250 2024-09-14T23:59:04.280 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.280 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:04.280 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.280 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:04.280 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943840+0000", 2024-09-14T23:59:04.280 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.280 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.280 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.280 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:04.280 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943840+0000", 2024-09-14T23:59:04.280 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.280 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.281 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.281 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:04.281 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945669+0000", 2024-09-14T23:59:04.281 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018289720000000001 2024-09-14T23:59:04.281 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.281 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.281 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:04.281 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945671+0000", 2024-09-14T23:59:04.281 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6869999999999999e-06 2024-09-14T23:59:04.281 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.281 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.281 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:04.281 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945680+0000", 2024-09-14T23:59:04.282 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5259999999999993e-06 2024-09-14T23:59:04.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.282 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.282 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:04.282 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945685+0000", 2024-09-14T23:59:04.282 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8559999999999998e-06 2024-09-14T23:59:04.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.282 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.282 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:04.282 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998242+0000", 2024-09-14T23:59:04.282 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052557505999999997 2024-09-14T23:59:04.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.282 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.282 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:04.283 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998329+0000", 2024-09-14T23:59:04.283 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6410999999999996e-05 2024-09-14T23:59:04.283 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.283 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.283 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:04.283 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998414+0000", 2024-09-14T23:59:04.283 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4917e-05 2024-09-14T23:59:04.283 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.283 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:04.283 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.283 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.283 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.283 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129262 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 13303808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:04.283 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.945825+0000", 2024-09-14T23:59:04.284 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 85.269038113999997, 2024-09-14T23:59:04.284 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054515942999999997, 2024-09-14T23:59:04.284 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:04.284 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:04.284 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:04.284 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:04.284 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:04.284 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:04.284 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129262 2024-09-14T23:59:04.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.284 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:04.284 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.284 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:04.284 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945825+0000", 2024-09-14T23:59:04.284 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.285 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.285 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:04.285 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945825+0000", 2024-09-14T23:59:04.285 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.285 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.285 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:04.285 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.947670+0000", 2024-09-14T23:59:04.285 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018451229999999999 2024-09-14T23:59:04.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.285 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.285 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:04.285 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.947672+0000", 2024-09-14T23:59:04.286 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5289999999999998e-06 2024-09-14T23:59:04.286 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.286 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.286 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:04.286 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.947678+0000", 2024-09-14T23:59:04.286 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9880000000000001e-06 2024-09-14T23:59:04.286 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.286 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.286 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:04.286 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.947682+0000", 2024-09-14T23:59:04.286 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.63e-06 2024-09-14T23:59:04.286 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.286 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.286 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:04.286 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000183+0000", 2024-09-14T23:59:04.287 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052500770000000002 2024-09-14T23:59:04.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.287 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.287 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:04.287 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000266+0000", 2024-09-14T23:59:04.287 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3133000000000005e-05 2024-09-14T23:59:04.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.287 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.287 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:04.287 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000341+0000", 2024-09-14T23:59:04.287 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4770000000000004e-05 2024-09-14T23:59:04.287 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.287 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:04.287 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.288 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.288 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129299 3.10 3:0f1e06ee:::rbd_object_map.1118721be852:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:04.288 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.952330+0000", 2024-09-14T23:59:04.288 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 85.262533301000005, 2024-09-14T23:59:04.288 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054412818000000002, 2024-09-14T23:59:04.288 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:04.288 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:04.288 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:04.288 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:04.288 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:04.288 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:04.288 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129299 2024-09-14T23:59:04.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.289 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:04.289 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.289 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:04.289 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952330+0000", 2024-09-14T23:59:04.289 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.289 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.289 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:04.289 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952330+0000", 2024-09-14T23:59:04.289 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.289 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.289 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:04.289 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952330+0000", 2024-09-14T23:59:04.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4900000000000001e-07 2024-09-14T23:59:04.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.290 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.290 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:04.290 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952332+0000", 2024-09-14T23:59:04.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2879999999999999e-06 2024-09-14T23:59:04.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.290 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.290 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:04.290 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952340+0000", 2024-09-14T23:59:04.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9910000000000002e-06 2024-09-14T23:59:04.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.290 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.290 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:04.290 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952344+0000", 2024-09-14T23:59:04.291 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0280000000000002e-06 2024-09-14T23:59:04.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.291 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.291 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:04.291 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.004722+0000", 2024-09-14T23:59:04.291 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052377932000000002 2024-09-14T23:59:04.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.291 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.291 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:04.291 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.004778+0000", 2024-09-14T23:59:04.291 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5795999999999999e-05 2024-09-14T23:59:04.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.291 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.291 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:59:04.292 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.005016+0000", 2024-09-14T23:59:04.292 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000238006 2024-09-14T23:59:04.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.292 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.292 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:59:04.292 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.006366+0000", 2024-09-14T23:59:04.292 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001349786 2024-09-14T23:59:04.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.292 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.292 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:59:04.292 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.006685+0000", 2024-09-14T23:59:04.292 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00031970000000000002 2024-09-14T23:59:04.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.292 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.293 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:04.293 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.006718+0000", 2024-09-14T23:59:04.293 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2329000000000002e-05 2024-09-14T23:59:04.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.293 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.293 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:04.293 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.006742+0000", 2024-09-14T23:59:04.293 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4613e-05 2024-09-14T23:59:04.293 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.293 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:04.293 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.293 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.293 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186210 3.d 3.b2ff3acd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:59:04.294 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.129044+0000", 2024-09-14T23:59:04.294 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 57.085818938000003, 2024-09-14T23:59:04.294 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.84555443900000005, 2024-09-14T23:59:04.294 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:04.294 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:04.294 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:59:04.294 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:04.294 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:04.294 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:04.294 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186210 2024-09-14T23:59:04.294 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.294 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:04.294 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.294 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:04.295 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129044+0000", 2024-09-14T23:59:04.295 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.295 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.295 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:04.295 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129044+0000", 2024-09-14T23:59:04.295 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.295 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.295 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:04.295 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129046+0000", 2024-09-14T23:59:04.295 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.742e-06 2024-09-14T23:59:04.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.296 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.296 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:04.296 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129052+0000", 2024-09-14T23:59:04.296 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7390000000000002e-06 2024-09-14T23:59:04.296 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.296 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.296 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:04.296 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129067+0000", 2024-09-14T23:59:04.296 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.417e-05 2024-09-14T23:59:04.296 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.296 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.296 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:04.296 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129076+0000", 2024-09-14T23:59:04.297 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2289999999999992e-06 2024-09-14T23:59:04.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.297 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.297 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:04.297 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.130947+0000", 2024-09-14T23:59:04.297 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001871075 2024-09-14T23:59:04.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.297 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.297 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:04.297 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.130955+0000", 2024-09-14T23:59:04.297 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4500000000000004e-06 2024-09-14T23:59:04.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.297 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.297 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:04.298 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974587+0000", 2024-09-14T23:59:04.298 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.84363205500000005 2024-09-14T23:59:04.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.298 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.298 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:04.298 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974598+0000", 2024-09-14T23:59:04.298 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0978999999999999e-05 2024-09-14T23:59:04.298 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.298 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:04.298 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.298 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.298 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187415 3.d 3.b2ff3acd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:59:04.299 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.254681+0000", 2024-09-14T23:59:04.299 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 56.960181417000001, 2024-09-14T23:59:04.299 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.71995530699999999, 2024-09-14T23:59:04.299 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:04.299 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:04.299 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:59:04.299 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:04.299 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:04.299 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:04.299 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187415 2024-09-14T23:59:04.299 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.299 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:04.299 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.299 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:04.300 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254681+0000", 2024-09-14T23:59:04.300 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.300 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.300 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:04.300 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254681+0000", 2024-09-14T23:59:04.300 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.300 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.300 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:04.300 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254682+0000", 2024-09-14T23:59:04.300 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2e-07 2024-09-14T23:59:04.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.301 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.301 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:04.301 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254684+0000", 2024-09-14T23:59:04.301 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2730000000000001e-06 2024-09-14T23:59:04.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.301 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.301 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:04.301 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254689+0000", 2024-09-14T23:59:04.301 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4769999999999997e-06 2024-09-14T23:59:04.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.301 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.301 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:04.302 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254692+0000", 2024-09-14T23:59:04.302 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9890000000000001e-06 2024-09-14T23:59:04.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.302 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.302 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:04.302 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254721+0000", 2024-09-14T23:59:04.302 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9762999999999999e-05 2024-09-14T23:59:04.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.302 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.302 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:04.302 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254725+0000", 2024-09-14T23:59:04.302 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9650000000000002e-06 2024-09-14T23:59:04.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.303 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:04.303 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974628+0000", 2024-09-14T23:59:04.303 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.71990235499999999 2024-09-14T23:59:04.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.303 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:04.303 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974637+0000", 2024-09-14T23:59:04.303 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0650000000000005e-06 2024-09-14T23:59:04.303 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.303 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:04.303 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.304 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.304 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187457 3.d 3.b2ff3acd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:59:04.304 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.257980+0000", 2024-09-14T23:59:04.304 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 56.956882843000002, 2024-09-14T23:59:04.304 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.71671711199999999, 2024-09-14T23:59:04.304 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:04.304 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:04.304 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:59:04.304 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:04.304 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:04.304 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:04.304 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187457 2024-09-14T23:59:04.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.304 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:04.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.305 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:04.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257980+0000", 2024-09-14T23:59:04.305 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.305 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:04.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257980+0000", 2024-09-14T23:59:04.305 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.305 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:04.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257980+0000", 2024-09-14T23:59:04.305 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.17e-07 2024-09-14T23:59:04.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.306 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.306 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:04.306 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257981+0000", 2024-09-14T23:59:04.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.172e-06 2024-09-14T23:59:04.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.306 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.306 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:04.306 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257984+0000", 2024-09-14T23:59:04.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9550000000000001e-06 2024-09-14T23:59:04.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.306 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.306 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:04.306 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257987+0000", 2024-09-14T23:59:04.307 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5799999999999999e-06 2024-09-14T23:59:04.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.307 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.307 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:04.307 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.258030+0000", 2024-09-14T23:59:04.307 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3118999999999998e-05 2024-09-14T23:59:04.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.307 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.307 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:04.307 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.258038+0000", 2024-09-14T23:59:04.307 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6850000000000001e-06 2024-09-14T23:59:04.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.307 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.307 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:04.307 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974688+0000", 2024-09-14T23:59:04.308 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.71665016000000004 2024-09-14T23:59:04.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.308 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.308 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:04.308 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974697+0000", 2024-09-14T23:59:04.308 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2240000000000001e-06 2024-09-14T23:59:04.308 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.308 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:04.308 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.308 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.308 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:251656 3.c 3:32436721:::rbd_object_map.1118e09062ff:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-14T23:59:04.308 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:27.433419+0000", 2024-09-14T23:59:04.308 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 36.781444168999997, 2024-09-14T23:59:04.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.55083879499999999, 2024-09-14T23:59:04.309 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:04.309 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:04.309 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:04.309 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:04.309 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:04.309 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:04.309 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 251656 2024-09-14T23:59:04.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.309 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:04.309 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.309 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:04.309 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433419+0000", 2024-09-14T23:59:04.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.310 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:04.310 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433419+0000", 2024-09-14T23:59:04.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.310 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:04.310 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433419+0000", 2024-09-14T23:59:04.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1500000000000003e-07 2024-09-14T23:59:04.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.310 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:04.310 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433424+0000", 2024-09-14T23:59:04.311 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2409999999999997e-06 2024-09-14T23:59:04.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.311 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.311 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:04.311 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433435+0000", 2024-09-14T23:59:04.311 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1009000000000001e-05 2024-09-14T23:59:04.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.311 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.311 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:04.311 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433440+0000", 2024-09-14T23:59:04.311 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1950000000000002e-06 2024-09-14T23:59:04.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.311 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.311 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:04.311 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433487+0000", 2024-09-14T23:59:04.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7184000000000001e-05 2024-09-14T23:59:04.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.312 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:04.312 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433494+0000", 2024-09-14T23:59:04.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5490000000000003e-06 2024-09-14T23:59:04.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.312 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:04.312 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.981923+0000", 2024-09-14T23:59:04.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.54842898600000001 2024-09-14T23:59:04.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.312 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:04.313 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.982097+0000", 2024-09-14T23:59:04.313 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017428699999999999 2024-09-14T23:59:04.313 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.313 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.313 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5", 2024-09-14T23:59:04.313 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.982334+0000", 2024-09-14T23:59:04.313 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00023700899999999999 2024-09-14T23:59:04.313 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.313 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.313 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:59:04.313 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.983927+0000", 2024-09-14T23:59:04.313 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0015932710000000001 2024-09-14T23:59:04.313 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.313 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.314 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:59:04.314 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.984128+0000", 2024-09-14T23:59:04.314 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020104199999999999 2024-09-14T23:59:04.314 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.314 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.314 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:04.314 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.984212+0000", 2024-09-14T23:59:04.314 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3709999999999996e-05 2024-09-14T23:59:04.314 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.314 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.314 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:04.314 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.984257+0000", 2024-09-14T23:59:04.314 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5497000000000002e-05 2024-09-14T23:59:04.314 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.314 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:04.315 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.315 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.315 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:04.315 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:59:04.315 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:59:04.215+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:59:04.316 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T23:59:04.317 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:59:04.316+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:59:04.418 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:59:04.417+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:59:04.464 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:59:04.464 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:59:04.465 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:59:04.465 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:59:04.473 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T23:59:04.518 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:59:04.518+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:59:04.619 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:59:04.618+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:59:04.620 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:59:04.620 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:59:04.620 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:59:04.620 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:59:04.620 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:59:04.629 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T23:59:04.720 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:59:04.719+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:59:04.779 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:59:04.779 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:59:04.779 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:59:04.780 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:59:04.780 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.780 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69262 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 4800512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:59:04.780 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300811+0000", 2024-09-14T23:59:04.780 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 109.476222073, 2024-09-14T23:59:04.780 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050403796000000001, 2024-09-14T23:59:04.780 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:04.780 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:04.780 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:04.780 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:04.780 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:04.781 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:04.781 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69262 2024-09-14T23:59:04.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.781 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:04.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.781 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:04.781 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300811+0000", 2024-09-14T23:59:04.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.781 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:04.781 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300811+0000", 2024-09-14T23:59:04.782 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.782 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:04.782 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300811+0000", 2024-09-14T23:59:04.782 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5700000000000002e-07 2024-09-14T23:59:04.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.782 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:04.782 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300815+0000", 2024-09-14T23:59:04.782 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.087e-06 2024-09-14T23:59:04.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.783 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.783 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:04.783 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300823+0000", 2024-09-14T23:59:04.783 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8720000000000002e-06 2024-09-14T23:59:04.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.783 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.783 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:04.783 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300830+0000", 2024-09-14T23:59:04.783 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0589999999999997e-06 2024-09-14T23:59:04.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.783 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.784 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:04.784 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351020+0000", 2024-09-14T23:59:04.784 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050189552999999998 2024-09-14T23:59:04.784 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.784 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.784 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:04.784 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351112+0000", 2024-09-14T23:59:04.784 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2615999999999996e-05 2024-09-14T23:59:04.784 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.784 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.784 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:04.785 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351214+0000", 2024-09-14T23:59:04.785 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101952 2024-09-14T23:59:04.785 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.785 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:04.785 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.785 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.785 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.785 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69263 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 4915200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:59:04.785 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300842+0000", 2024-09-14T23:59:04.785 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 109.476190569, 2024-09-14T23:59:04.785 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050590637000000001, 2024-09-14T23:59:04.785 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:04.786 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:04.786 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:04.786 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:04.786 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:04.786 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:04.786 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69263 2024-09-14T23:59:04.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.786 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:04.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:04.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300842+0000", 2024-09-14T23:59:04.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.787 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:04.787 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300842+0000", 2024-09-14T23:59:04.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.787 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:04.787 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300843+0000", 2024-09-14T23:59:04.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4000000000000002e-07 2024-09-14T23:59:04.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.788 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:04.788 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300846+0000", 2024-09-14T23:59:04.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8720000000000002e-06 2024-09-14T23:59:04.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.788 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:04.788 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300853+0000", 2024-09-14T23:59:04.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0450000000000003e-06 2024-09-14T23:59:04.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.788 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:04.789 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300859+0000", 2024-09-14T23:59:04.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9510000000000003e-06 2024-09-14T23:59:04.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.789 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:04.789 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351260+0000", 2024-09-14T23:59:04.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050401654999999997 2024-09-14T23:59:04.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.789 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:04.789 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351353+0000", 2024-09-14T23:59:04.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2785e-05 2024-09-14T23:59:04.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:04.790 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351433+0000", 2024-09-14T23:59:04.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9789000000000005e-05 2024-09-14T23:59:04.790 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.790 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:04.790 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.790 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69264 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5029888~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:59:04.790 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300866+0000", 2024-09-14T23:59:04.791 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 109.476166636, 2024-09-14T23:59:04.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050794063, 2024-09-14T23:59:04.791 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:04.791 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:04.791 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:04.791 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:04.791 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:04.791 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:04.791 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69264 2024-09-14T23:59:04.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.791 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:04.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.792 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:04.792 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300866+0000", 2024-09-14T23:59:04.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.792 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:04.792 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300866+0000", 2024-09-14T23:59:04.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.792 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:04.792 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300866+0000", 2024-09-14T23:59:04.793 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8799999999999998e-07 2024-09-14T23:59:04.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.793 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:04.793 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300869+0000", 2024-09-14T23:59:04.793 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0880000000000002e-06 2024-09-14T23:59:04.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.793 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:04.793 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300875+0000", 2024-09-14T23:59:04.793 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9280000000000004e-06 2024-09-14T23:59:04.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.794 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:04.794 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300882+0000", 2024-09-14T23:59:04.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4740000000000002e-06 2024-09-14T23:59:04.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.794 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:04.794 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351472+0000", 2024-09-14T23:59:04.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050590006999999999 2024-09-14T23:59:04.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.794 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:04.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351563+0000", 2024-09-14T23:59:04.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0762999999999998e-05 2024-09-14T23:59:04.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:04.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351660+0000", 2024-09-14T23:59:04.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7415000000000002e-05 2024-09-14T23:59:04.795 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.795 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:04.795 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.796 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69265 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5144576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:59:04.796 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300890+0000", 2024-09-14T23:59:04.796 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 109.476143132, 2024-09-14T23:59:04.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050921389999999997, 2024-09-14T23:59:04.796 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:04.796 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:04.796 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:04.797 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:04.797 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:04.797 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:04.797 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69265 2024-09-14T23:59:04.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.797 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:04.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.797 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:04.797 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300890+0000", 2024-09-14T23:59:04.797 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.798 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:04.798 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300890+0000", 2024-09-14T23:59:04.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.798 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:04.798 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300890+0000", 2024-09-14T23:59:04.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2500000000000001e-07 2024-09-14T23:59:04.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.799 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.799 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:04.799 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300892+0000", 2024-09-14T23:59:04.799 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2469999999999998e-06 2024-09-14T23:59:04.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.799 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.799 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:04.799 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300899+0000", 2024-09-14T23:59:04.799 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5509999999999996e-06 2024-09-14T23:59:04.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.799 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:04.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300904+0000", 2024-09-14T23:59:04.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3299999999999998e-06 2024-09-14T23:59:04.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:04.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351707+0000", 2024-09-14T23:59:04.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050802673999999999 2024-09-14T23:59:04.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:04.801 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351755+0000", 2024-09-14T23:59:04.801 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8362000000000002e-05 2024-09-14T23:59:04.801 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.801 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.801 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:04.801 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351811+0000", 2024-09-14T23:59:04.801 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5801000000000003e-05 2024-09-14T23:59:04.801 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.801 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:04.801 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.801 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.801 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.802 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69266 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5259264~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:59:04.802 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300911+0000", 2024-09-14T23:59:04.802 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 109.47612205900001, 2024-09-14T23:59:04.802 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051125089999999998, 2024-09-14T23:59:04.802 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:04.802 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:04.802 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:04.802 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:04.802 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:04.802 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:04.802 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69266 2024-09-14T23:59:04.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.803 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:04.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.803 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:04.803 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300911+0000", 2024-09-14T23:59:04.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.803 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:04.803 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300911+0000", 2024-09-14T23:59:04.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:04.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300911+0000", 2024-09-14T23:59:04.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-14T23:59:04.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:04.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300913+0000", 2024-09-14T23:59:04.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9300000000000002e-06 2024-09-14T23:59:04.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:04.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300919+0000", 2024-09-14T23:59:04.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.489e-06 2024-09-14T23:59:04.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.805 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.805 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:04.805 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300925+0000", 2024-09-14T23:59:04.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1719999999999998e-06 2024-09-14T23:59:04.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.805 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.805 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:04.805 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351847+0000", 2024-09-14T23:59:04.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050922621000000001 2024-09-14T23:59:04.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.806 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.806 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:04.806 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351938+0000", 2024-09-14T23:59:04.806 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0830999999999998e-05 2024-09-14T23:59:04.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.806 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.806 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:04.806 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352036+0000", 2024-09-14T23:59:04.806 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7676999999999993e-05 2024-09-14T23:59:04.806 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.806 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:04.807 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.807 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69267 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5373952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:59:04.807 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300945+0000", 2024-09-14T23:59:04.807 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 109.476087264, 2024-09-14T23:59:04.807 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051217934, 2024-09-14T23:59:04.807 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:04.807 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:04.807 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:04.807 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:04.807 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:04.808 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:04.808 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69267 2024-09-14T23:59:04.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.808 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:04.808 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.808 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:04.808 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300945+0000", 2024-09-14T23:59:04.808 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.808 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.808 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:04.808 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300945+0000", 2024-09-14T23:59:04.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.809 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:04.809 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300946+0000", 2024-09-14T23:59:04.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2600000000000002e-07 2024-09-14T23:59:04.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.809 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:04.809 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300948+0000", 2024-09-14T23:59:04.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4940000000000002e-06 2024-09-14T23:59:04.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.810 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.810 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:04.810 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300956+0000", 2024-09-14T23:59:04.810 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.622e-06 2024-09-14T23:59:04.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.810 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.810 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:04.810 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300962+0000", 2024-09-14T23:59:04.810 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5790000000000003e-06 2024-09-14T23:59:04.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.810 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.811 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:04.811 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352063+0000", 2024-09-14T23:59:04.811 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051101281999999998 2024-09-14T23:59:04.811 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.811 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.811 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:04.811 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352110+0000", 2024-09-14T23:59:04.811 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7401000000000002e-05 2024-09-14T23:59:04.811 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.811 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.811 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:04.811 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352163+0000", 2024-09-14T23:59:04.812 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3029999999999999e-05 2024-09-14T23:59:04.812 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.812 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:04.812 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.812 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.812 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69268 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5488640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:59:04.812 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300969+0000", 2024-09-14T23:59:04.812 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 109.476063592, 2024-09-14T23:59:04.812 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051413420000000001, 2024-09-14T23:59:04.812 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:04.813 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:04.813 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:04.813 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:04.813 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:04.813 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:04.813 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69268 2024-09-14T23:59:04.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.813 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:04.813 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.813 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:04.813 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300969+0000", 2024-09-14T23:59:04.814 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.814 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.814 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:04.814 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300969+0000", 2024-09-14T23:59:04.814 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.814 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.814 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:04.814 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300970+0000", 2024-09-14T23:59:04.814 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7899999999999999e-07 2024-09-14T23:59:04.815 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.815 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.815 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:04.815 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300982+0000", 2024-09-14T23:59:04.815 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2741e-05 2024-09-14T23:59:04.815 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.815 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.815 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:04.815 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300990+0000", 2024-09-14T23:59:04.815 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8169999999999999e-06 2024-09-14T23:59:04.815 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.816 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.816 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:04.816 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300995+0000", 2024-09-14T23:59:04.816 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2240000000000003e-06 2024-09-14T23:59:04.816 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.816 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.816 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:04.816 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352200+0000", 2024-09-14T23:59:04.816 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051204223 2024-09-14T23:59:04.816 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.816 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.816 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:04.817 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352285+0000", 2024-09-14T23:59:04.817 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5519999999999994e-05 2024-09-14T23:59:04.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.817 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.821 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:04.821 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352383+0000", 2024-09-14T23:59:04.821 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7516000000000006e-05 2024-09-14T23:59:04.821 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.821 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:04.821 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.821 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.821 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.821 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69269 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5603328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:59:04.821 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.301003+0000", 2024-09-14T23:59:04.822 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 109.47602941, 2024-09-14T23:59:04.822 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051497381000000002, 2024-09-14T23:59:04.822 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:04.822 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:04.822 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:04.822 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:04.822 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:04.822 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:04.822 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69269 2024-09-14T23:59:04.822 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.822 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:04.823 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.823 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:04.823 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.301003+0000", 2024-09-14T23:59:04.823 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.823 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.823 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.823 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:04.823 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.301003+0000", 2024-09-14T23:59:04.823 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.823 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.824 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.824 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:04.824 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302671+0000", 2024-09-14T23:59:04.824 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001667785 2024-09-14T23:59:04.824 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.824 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.824 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:04.824 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302676+0000", 2024-09-14T23:59:04.824 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6600000000000003e-06 2024-09-14T23:59:04.824 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.824 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.825 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:04.825 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302697+0000", 2024-09-14T23:59:04.825 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1288999999999999e-05 2024-09-14T23:59:04.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.825 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.825 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:04.825 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302716+0000", 2024-09-14T23:59:04.825 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9128999999999999e-05 2024-09-14T23:59:04.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.825 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.825 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:04.826 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352399+0000", 2024-09-14T23:59:04.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049682995000000001 2024-09-14T23:59:04.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.826 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.826 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:04.826 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352446+0000", 2024-09-14T23:59:04.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6341000000000002e-05 2024-09-14T23:59:04.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.826 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.826 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:04.827 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352501+0000", 2024-09-14T23:59:04.827 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5182000000000001e-05 2024-09-14T23:59:04.827 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.827 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:04.827 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.827 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.827 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69274 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6176768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:59:04.827 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302854+0000", 2024-09-14T23:59:04.827 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 109.47417874999999, 2024-09-14T23:59:04.828 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050579526, 2024-09-14T23:59:04.828 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:04.828 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:04.828 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:04.828 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:04.828 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:04.828 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:04.828 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69274 2024-09-14T23:59:04.828 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.829 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:04.829 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.829 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:04.829 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302854+0000", 2024-09-14T23:59:04.829 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.829 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.829 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.829 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:04.829 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302854+0000", 2024-09-14T23:59:04.829 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.829 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.830 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.830 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:04.830 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302855+0000", 2024-09-14T23:59:04.830 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0200000000000001e-07 2024-09-14T23:59:04.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.830 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.830 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:04.830 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302857+0000", 2024-09-14T23:59:04.830 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.447e-06 2024-09-14T23:59:04.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.831 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.831 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:04.831 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302864+0000", 2024-09-14T23:59:04.831 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7730000000000002e-06 2024-09-14T23:59:04.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.831 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.831 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:04.831 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302869+0000", 2024-09-14T23:59:04.831 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3199999999999999e-06 2024-09-14T23:59:04.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.832 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.832 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:04.832 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353249+0000", 2024-09-14T23:59:04.832 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050379647999999999 2024-09-14T23:59:04.832 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.832 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.832 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:04.832 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353336+0000", 2024-09-14T23:59:04.832 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6767999999999993e-05 2024-09-14T23:59:04.832 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.833 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.833 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:04.833 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353433+0000", 2024-09-14T23:59:04.833 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7868000000000006e-05 2024-09-14T23:59:04.833 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.833 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:04.833 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.833 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.833 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.833 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69275 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6291456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:59:04.833 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302889+0000", 2024-09-14T23:59:04.834 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 109.47414413200001, 2024-09-14T23:59:04.834 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050663118, 2024-09-14T23:59:04.834 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:04.834 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:04.834 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:04.834 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:04.834 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:04.834 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:04.834 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69275 2024-09-14T23:59:04.834 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.834 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:04.834 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.835 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:04.835 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302889+0000", 2024-09-14T23:59:04.835 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.835 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.835 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.835 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:04.835 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302889+0000", 2024-09-14T23:59:04.835 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.835 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.835 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.835 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:04.835 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302889+0000", 2024-09-14T23:59:04.836 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6399999999999998e-07 2024-09-14T23:59:04.836 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.836 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.836 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:04.836 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302891+0000", 2024-09-14T23:59:04.836 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.339e-06 2024-09-14T23:59:04.836 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.836 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.836 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:04.836 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302898+0000", 2024-09-14T23:59:04.836 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0380000000000002e-06 2024-09-14T23:59:04.837 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.837 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.837 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:04.837 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302904+0000", 2024-09-14T23:59:04.837 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2159999999999997e-06 2024-09-14T23:59:04.837 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.837 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.837 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:04.837 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353449+0000", 2024-09-14T23:59:04.837 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050545386999999997 2024-09-14T23:59:04.837 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.838 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.838 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:04.838 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353494+0000", 2024-09-14T23:59:04.838 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4592999999999998e-05 2024-09-14T23:59:04.838 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.838 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.838 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:04.838 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353552+0000", 2024-09-14T23:59:04.838 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8180999999999997e-05 2024-09-14T23:59:04.838 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.838 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:04.839 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.839 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.839 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.839 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69276 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6406144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:59:04.839 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302911+0000", 2024-09-14T23:59:04.839 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 109.474121719, 2024-09-14T23:59:04.839 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050865753, 2024-09-14T23:59:04.839 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:04.839 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:04.839 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:04.839 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:04.840 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:04.840 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:04.840 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69276 2024-09-14T23:59:04.840 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.840 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:04.840 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.840 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:04.840 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302911+0000", 2024-09-14T23:59:04.840 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.840 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.841 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.841 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:04.841 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302911+0000", 2024-09-14T23:59:04.841 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.841 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.841 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.841 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:04.841 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302912+0000", 2024-09-14T23:59:04.841 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0800000000000004e-07 2024-09-14T23:59:04.842 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.842 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.842 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:04.842 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302914+0000", 2024-09-14T23:59:04.842 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9589999999999998e-06 2024-09-14T23:59:04.842 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.842 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.842 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:04.842 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302920+0000", 2024-09-14T23:59:04.842 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4579999999999998e-06 2024-09-14T23:59:04.842 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.843 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.843 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:04.843 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302925+0000", 2024-09-14T23:59:04.843 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9429999999999999e-06 2024-09-14T23:59:04.843 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.843 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.843 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:04.843 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353586+0000", 2024-09-14T23:59:04.843 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050660966000000002 2024-09-14T23:59:04.843 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.843 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.844 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:04.844 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353679+0000", 2024-09-14T23:59:04.844 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3163000000000002e-05 2024-09-14T23:59:04.844 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.844 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.844 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:04.844 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353777+0000", 2024-09-14T23:59:04.844 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7555999999999999e-05 2024-09-14T23:59:04.844 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.845 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:04.845 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.845 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.845 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.845 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69277 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6520832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:59:04.845 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302932+0000", 2024-09-14T23:59:04.845 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 109.474100632, 2024-09-14T23:59:04.845 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050977674000000001, 2024-09-14T23:59:04.845 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:04.845 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:04.845 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:04.845 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:04.846 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:04.846 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:04.846 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69277 2024-09-14T23:59:04.846 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.846 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:04.846 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.846 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:04.846 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302932+0000", 2024-09-14T23:59:04.846 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.846 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.846 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.847 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:04.847 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302932+0000", 2024-09-14T23:59:04.847 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.847 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.847 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.847 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:04.847 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302932+0000", 2024-09-14T23:59:04.847 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3299999999999998e-07 2024-09-14T23:59:04.847 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.847 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.848 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:04.848 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302935+0000", 2024-09-14T23:59:04.848 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2139999999999999e-06 2024-09-14T23:59:04.848 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.848 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.848 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:04.848 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302941+0000", 2024-09-14T23:59:04.848 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5039999999999999e-06 2024-09-14T23:59:04.848 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.848 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.848 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:04.848 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302946+0000", 2024-09-14T23:59:04.849 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2079999999999999e-06 2024-09-14T23:59:04.849 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.849 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.849 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:04.849 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353805+0000", 2024-09-14T23:59:04.849 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050858588000000003 2024-09-14T23:59:04.849 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.849 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.849 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:04.849 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353849+0000", 2024-09-14T23:59:04.849 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4511000000000001e-05 2024-09-14T23:59:04.850 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.850 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.850 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:04.850 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353910+0000", 2024-09-14T23:59:04.850 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0316000000000003e-05 2024-09-14T23:59:04.850 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.850 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:04.850 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.850 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.851 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.851 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69278 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6635520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:59:04.851 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302953+0000", 2024-09-14T23:59:04.851 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 109.474079504, 2024-09-14T23:59:04.851 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051174059000000001, 2024-09-14T23:59:04.851 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:04.851 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:04.852 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:04.852 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:04.852 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:04.852 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:04.852 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69278 2024-09-14T23:59:04.852 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.852 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:04.852 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.852 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:04.852 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302953+0000", 2024-09-14T23:59:04.853 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.853 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.853 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.853 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:04.853 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302953+0000", 2024-09-14T23:59:04.853 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.853 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.853 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.853 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:04.853 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302954+0000", 2024-09-14T23:59:04.853 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5100000000000001e-07 2024-09-14T23:59:04.854 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.854 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.854 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:04.854 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302964+0000", 2024-09-14T23:59:04.854 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0475000000000001e-05 2024-09-14T23:59:04.854 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.854 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.854 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:04.854 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302970+0000", 2024-09-14T23:59:04.854 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3879999999999997e-06 2024-09-14T23:59:04.855 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.855 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.855 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:04.855 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302975+0000", 2024-09-14T23:59:04.855 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8409999999999999e-06 2024-09-14T23:59:04.855 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.855 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.855 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:04.855 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353944+0000", 2024-09-14T23:59:04.855 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050968952999999997 2024-09-14T23:59:04.856 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.856 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.856 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:04.856 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354031+0000", 2024-09-14T23:59:04.856 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6537000000000002e-05 2024-09-14T23:59:04.856 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.856 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.856 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:04.856 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354127+0000", 2024-09-14T23:59:04.856 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6514000000000005e-05 2024-09-14T23:59:04.856 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.856 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:04.857 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.857 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.857 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.857 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69279 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6750208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:59:04.857 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302984+0000", 2024-09-14T23:59:04.857 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 109.474049153, 2024-09-14T23:59:04.857 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051258845999999997, 2024-09-14T23:59:04.857 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:04.857 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:04.857 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:04.858 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:04.858 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:04.858 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:04.858 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69279 2024-09-14T23:59:04.858 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.858 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:04.858 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.858 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:04.858 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302984+0000", 2024-09-14T23:59:04.858 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.858 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.858 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.859 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:04.859 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302984+0000", 2024-09-14T23:59:04.859 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.859 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.859 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.859 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:04.859 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303025+0000", 2024-09-14T23:59:04.859 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1690999999999998e-05 2024-09-14T23:59:04.859 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.859 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.859 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:04.860 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303029+0000", 2024-09-14T23:59:04.860 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6559999999999998e-06 2024-09-14T23:59:04.860 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.860 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.860 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:04.861 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303037+0000", 2024-09-14T23:59:04.861 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7950000000000008e-06 2024-09-14T23:59:04.861 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.861 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.861 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:04.861 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303043+0000", 2024-09-14T23:59:04.861 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9669999999999998e-06 2024-09-14T23:59:04.861 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.861 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.861 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:04.862 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354142+0000", 2024-09-14T23:59:04.862 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051099072000000002 2024-09-14T23:59:04.862 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.862 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.862 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:04.862 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354183+0000", 2024-09-14T23:59:04.862 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0834000000000002e-05 2024-09-14T23:59:04.862 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.862 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.862 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:04.863 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354242+0000", 2024-09-14T23:59:04.863 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9830999999999997e-05 2024-09-14T23:59:04.863 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.863 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:04.863 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.863 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.863 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.863 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69280 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6864896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:59:04.863 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.303052+0000", 2024-09-14T23:59:04.863 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 109.47398061, 2024-09-14T23:59:04.864 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051406463999999999, 2024-09-14T23:59:04.864 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:04.864 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:04.864 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:04.864 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:04.864 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:04.864 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:04.865 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69280 2024-09-14T23:59:04.865 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.865 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:04.865 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.865 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:04.865 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303052+0000", 2024-09-14T23:59:04.865 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.865 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.865 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.865 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:04.865 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303052+0000", 2024-09-14T23:59:04.865 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.866 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.866 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.866 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:04.866 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304673+0000", 2024-09-14T23:59:04.866 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001621389 2024-09-14T23:59:04.866 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.866 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.866 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:04.866 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304681+0000", 2024-09-14T23:59:04.867 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8439999999999998e-06 2024-09-14T23:59:04.867 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.867 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.867 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:04.867 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304691+0000", 2024-09-14T23:59:04.867 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8339999999999994e-06 2024-09-14T23:59:04.867 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.867 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.867 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:04.867 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304698+0000", 2024-09-14T23:59:04.867 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3629999999999999e-06 2024-09-14T23:59:04.867 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.868 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.868 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:04.868 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354278+0000", 2024-09-14T23:59:04.868 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049580618 2024-09-14T23:59:04.868 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.868 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.868 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:04.868 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354364+0000", 2024-09-14T23:59:04.868 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6059999999999999e-05 2024-09-14T23:59:04.868 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.869 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.869 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:04.869 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354459+0000", 2024-09-14T23:59:04.869 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4356000000000003e-05 2024-09-14T23:59:04.869 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.869 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:04.869 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.869 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.869 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.869 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69286 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 7553024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:59:04.869 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.304811+0000", 2024-09-14T23:59:04.870 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 109.472221839, 2024-09-14T23:59:04.870 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050585512999999999, 2024-09-14T23:59:04.870 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:04.870 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:04.870 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:04.870 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:04.870 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:04.870 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:04.870 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69286 2024-09-14T23:59:04.870 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.870 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:04.871 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.871 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:04.871 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304811+0000", 2024-09-14T23:59:04.871 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.871 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.871 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.871 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:04.871 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304811+0000", 2024-09-14T23:59:04.872 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.872 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.872 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.872 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:04.872 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304811+0000", 2024-09-14T23:59:04.872 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.89e-07 2024-09-14T23:59:04.872 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.872 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.872 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:04.872 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304814+0000", 2024-09-14T23:59:04.873 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3199999999999998e-06 2024-09-14T23:59:04.873 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.873 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.873 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:04.873 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304821+0000", 2024-09-14T23:59:04.873 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5689999999999999e-06 2024-09-14T23:59:04.873 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.873 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.873 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:04.873 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304826+0000", 2024-09-14T23:59:04.873 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1200000000000001e-06 2024-09-14T23:59:04.874 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.874 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.874 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:04.874 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355226+0000", 2024-09-14T23:59:04.874 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050399653000000003 2024-09-14T23:59:04.874 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.874 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.874 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:04.874 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355314+0000", 2024-09-14T23:59:04.874 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8389999999999999e-05 2024-09-14T23:59:04.874 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.875 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.875 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:04.875 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355396+0000", 2024-09-14T23:59:04.875 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2071999999999996e-05 2024-09-14T23:59:04.875 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.875 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:04.875 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.875 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.875 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.876 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69287 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 7667712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:59:04.876 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.304833+0000", 2024-09-14T23:59:04.876 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 109.472199241, 2024-09-14T23:59:04.876 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050664740999999999, 2024-09-14T23:59:04.876 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:04.876 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:04.876 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:04.876 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:04.876 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:04.876 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:04.876 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69287 2024-09-14T23:59:04.877 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.877 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:04.877 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.877 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:04.877 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304833+0000", 2024-09-14T23:59:04.877 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.877 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.877 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.877 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:04.877 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304833+0000", 2024-09-14T23:59:04.878 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.878 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.878 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.878 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:04.878 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304834+0000", 2024-09-14T23:59:04.878 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4999999999999998e-07 2024-09-14T23:59:04.878 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.878 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.878 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:04.878 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304836+0000", 2024-09-14T23:59:04.878 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0279999999999999e-06 2024-09-14T23:59:04.879 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.879 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.879 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:04.879 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304842+0000", 2024-09-14T23:59:04.879 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8919999999999997e-06 2024-09-14T23:59:04.879 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.879 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.879 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:04.879 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304847+0000", 2024-09-14T23:59:04.879 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6160000000000001e-06 2024-09-14T23:59:04.879 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.879 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.880 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:04.880 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355411+0000", 2024-09-14T23:59:04.880 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050563540999999997 2024-09-14T23:59:04.880 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.880 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.880 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:04.880 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355459+0000", 2024-09-14T23:59:04.880 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7979000000000003e-05 2024-09-14T23:59:04.880 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.880 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.880 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:04.881 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355498+0000", 2024-09-14T23:59:04.881 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9335000000000002e-05 2024-09-14T23:59:04.881 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.881 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:04.881 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.881 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.881 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.881 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69288 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 7782400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:59:04.881 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.304854+0000", 2024-09-14T23:59:04.881 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 109.47217880300001, 2024-09-14T23:59:04.881 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050855145999999997, 2024-09-14T23:59:04.881 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:04.882 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:04.882 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:04.882 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:04.882 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:04.882 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:04.882 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69288 2024-09-14T23:59:04.882 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.882 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:04.882 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.882 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:04.882 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304854+0000", 2024-09-14T23:59:04.883 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.883 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.883 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.883 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:04.883 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304854+0000", 2024-09-14T23:59:04.883 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.883 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.883 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.883 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:04.883 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.306670+0000", 2024-09-14T23:59:04.883 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00181631 2024-09-14T23:59:04.883 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.883 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.884 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:04.884 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.306675+0000", 2024-09-14T23:59:04.884 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0220000000000004e-06 2024-09-14T23:59:04.884 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.884 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.884 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:04.884 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.306684+0000", 2024-09-14T23:59:04.884 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7309999999999999e-06 2024-09-14T23:59:04.884 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.884 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.884 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:04.884 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.306689+0000", 2024-09-14T23:59:04.885 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6609999999999999e-06 2024-09-14T23:59:04.885 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.885 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.885 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:04.885 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355533+0000", 2024-09-14T23:59:04.885 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048844578 2024-09-14T23:59:04.885 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.885 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.885 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:04.885 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355622+0000", 2024-09-14T23:59:04.885 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9006999999999997e-05 2024-09-14T23:59:04.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.886 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.886 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:04.886 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355709+0000", 2024-09-14T23:59:04.886 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6836999999999995e-05 2024-09-14T23:59:04.886 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.886 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:04.886 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.886 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.887 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:255885 3.13 3.54329f53 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-14T23:59:04.887 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:48.978834+0000", 2024-09-14T23:59:04.887 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 15.798198218, 2024-09-14T23:59:04.887 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.007469078, 2024-09-14T23:59:04.887 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:04.887 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:04.887 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:59:04.887 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:04.887 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:04.887 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:04.887 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 255885 2024-09-14T23:59:04.887 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.888 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:04.888 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.888 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:04.888 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.978834+0000", 2024-09-14T23:59:04.888 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.888 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.888 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:04.888 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.978834+0000", 2024-09-14T23:59:04.888 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.888 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.889 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:04.889 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.978836+0000", 2024-09-14T23:59:04.889 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.288e-06 2024-09-14T23:59:04.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.889 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.889 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:04.889 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.978844+0000", 2024-09-14T23:59:04.889 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9880000000000004e-06 2024-09-14T23:59:04.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.889 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.889 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:04.890 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.978861+0000", 2024-09-14T23:59:04.890 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7020999999999998e-05 2024-09-14T23:59:04.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.890 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.890 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:04.890 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.978872+0000", 2024-09-14T23:59:04.890 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1274000000000001e-05 2024-09-14T23:59:04.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.890 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.890 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:04.890 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.982358+0000", 2024-09-14T23:59:04.890 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003486393 2024-09-14T23:59:04.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.891 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.891 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:04.891 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.982368+0000", 2024-09-14T23:59:04.891 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0690000000000008e-06 2024-09-14T23:59:04.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.891 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.891 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:04.891 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:49.986277+0000", 2024-09-14T23:59:04.891 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0039092489999999 2024-09-14T23:59:04.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.891 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.892 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:04.892 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:49.986304+0000", 2024-09-14T23:59:04.892 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6795999999999999e-05 2024-09-14T23:59:04.892 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.892 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:04.892 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.892 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.892 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.892 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:255886 3.13 3.54329f53 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-14T23:59:04.892 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:48.981728+0000", 2024-09-14T23:59:04.892 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 15.7953043, 2024-09-14T23:59:04.893 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0046192860000001, 2024-09-14T23:59:04.893 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:04.893 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:04.893 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:59:04.893 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:04.893 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:04.893 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:04.893 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 255886 2024-09-14T23:59:04.893 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.893 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:04.893 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.894 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:04.894 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.981728+0000", 2024-09-14T23:59:04.894 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.894 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.894 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.894 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:04.894 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.981728+0000", 2024-09-14T23:59:04.894 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:04.894 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.894 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.895 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:04.895 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.981730+0000", 2024-09-14T23:59:04.895 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5179999999999999e-06 2024-09-14T23:59:04.895 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.895 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.895 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:04.895 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.981737+0000", 2024-09-14T23:59:04.895 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.956e-06 2024-09-14T23:59:04.895 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.895 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.895 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:04.895 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.981748+0000", 2024-09-14T23:59:04.896 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0754000000000001e-05 2024-09-14T23:59:04.896 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.896 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.896 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:04.896 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.981757+0000", 2024-09-14T23:59:04.896 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8850000000000005e-06 2024-09-14T23:59:04.896 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.896 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.896 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:04.897 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.983225+0000", 2024-09-14T23:59:04.897 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001468605 2024-09-14T23:59:04.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.898 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.898 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:04.898 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.983232+0000", 2024-09-14T23:59:04.898 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0090000000000001e-06 2024-09-14T23:59:04.898 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.899 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.899 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:04.899 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:49.986337+0000", 2024-09-14T23:59:04.899 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0031047470000001 2024-09-14T23:59:04.899 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:04.900 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:04.900 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:04.900 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:49.986348+0000", 2024-09-14T23:59:04.900 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0812000000000001e-05 2024-09-14T23:59:04.900 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.901 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:04.901 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.901 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:04.901 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:04.902 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:59:04.904 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:59:04.902+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:59:04.905 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T23:59:05.004 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:59:05.003+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:59:05.023 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:59:05.023 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:59:05.023 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:59:05.024 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:59:05.027 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T23:59:05.032 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T23:59:05.105 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:59:05.104+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:59:05.110 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is unset 2024-09-14T23:59:05.160 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:59:05.160 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:59:05.181 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:59:05.182 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:59:05.202 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:05.202 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:05.206 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:59:05.204+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:59:05.216 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T23:59:05.217 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:59:05.218 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:59:05.234 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:59:05.234 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:59:05.252 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:59:05.252 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:59:05.273 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:59:05.273 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:59:05.291 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:59:05.291 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:59:05.306 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:59:05.305+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:59:05.308 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:59:05.308 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:59:05.347 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:05.347 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:05.356 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T23:59:05.407 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:59:05.406+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:59:05.505 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:05.505 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:05.508 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:59:05.507+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:59:05.514 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T23:59:05.609 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:59:05.608+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:59:05.661 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:05.661 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:05.670 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T23:59:05.713 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:59:05.712+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:59:05.777 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:05.777 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:05.786 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T23:59:05.811 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:59:05.810+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:59:05.912 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:59:05.911+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:59:05.932 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:05.933 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:05.941 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T23:59:06.013 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:59:06.011+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:59:06.048 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:06.048 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:06.057 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T23:59:06.111 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 6, 5, 7] out_osds: [3] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T23:59:06.112 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T23:59:06.112 INFO:tasks.thrashosds.thrasher:Adding osd 3 2024-09-14T23:59:06.112 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 3 2024-09-14T23:59:06.141 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:59:06.139+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:59:06.214 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:59:06.213+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:59:06.224 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:06.224 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:06.234 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T23:59:06.315 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:59:06.314+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:59:06.345 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:06.345 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:06.354 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T23:59:06.416 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:59:06.414+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:59:06.456 INFO:tasks.ceph.mon.a.smithi042.stderr:2024-09-14T23:59:06.454+0000 7f1713676640 -1 mon.a@0(leader).osd e166 definitely_dead 0 2024-09-14T23:59:06.461 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:06.462 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:06.471 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T23:59:06.516 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:59:06.515+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:59:06.617 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:59:06.616+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:59:06.618 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:06.618 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:06.627 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T23:59:06.718 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:59:06.717+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:59:06.747 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:06.747 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:06.756 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T23:59:06.819 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:59:06.818+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:59:06.864 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:06.865 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:06.873 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T23:59:06.957 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:59:06.956+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:59:07.020 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:07.021 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:07.021 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:59:07.019+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:59:07.029 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T23:59:07.122 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:59:07.120+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:59:07.123 INFO:teuthology.orchestra.run.smithi042.stderr:marked in osd.3. 2024-09-14T23:59:07.128 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T23:59:07.148 INFO:tasks.thrashosds.thrasher:Added osd 3 2024-09-14T23:59:07.189 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:07.189 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:07.198 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T23:59:07.239 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:59:07.238+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:59:07.321 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T23:59:07.324 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:59:07.322+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:59:07.345 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:07.345 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:07.354 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T23:59:07.424 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:59:07.423+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:59:07.447 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:59:07.447 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:59:07.461 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:07.461 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:07.463 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:59:07.464 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:59:07.470 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T23:59:07.480 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:59:07.480 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:59:07.497 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:59:07.497 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:59:07.514 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:59:07.514 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:59:07.525 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:59:07.524+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:59:07.530 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:59:07.530 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:59:07.547 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:59:07.547 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:59:07.563 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:59:07.564 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:59:07.578 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:59:07.578 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:59:07.578 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:59:07.579 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:59:07.587 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T23:59:07.628 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:59:07.627+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:59:07.728 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:59:07.727+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:59:07.734 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:59:07.734 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:59:07.734 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:59:07.734 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:59:07.734 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:59:07.743 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T23:59:07.829 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:59:07.828+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:59:07.852 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:59:07.852 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:59:07.852 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:59:07.852 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:59:07.852 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.852 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:107472 3.1e 3:7bf21588:::rbd_data.11187de8b0a5.000000000000005d:head [set-alloc-hint object_size 2097152 write_size 2097152,write 317788~1491] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-14T23:59:07.852 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:25.198493+0000", 2024-09-14T23:59:07.852 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 102.65128551799999, 2024-09-14T23:59:07.853 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.78175149300000002, 2024-09-14T23:59:07.853 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:07.853 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:07.853 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:07.853 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:07.853 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:07.853 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:07.853 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 107472 2024-09-14T23:59:07.853 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.853 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:07.853 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.853 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:07.853 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198493+0000", 2024-09-14T23:59:07.853 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:07.854 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.854 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.854 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:07.854 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198493+0000", 2024-09-14T23:59:07.854 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:07.854 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.854 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.854 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:07.854 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198495+0000", 2024-09-14T23:59:07.854 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.826e-06 2024-09-14T23:59:07.854 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.854 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.854 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:07.854 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198503+0000", 2024-09-14T23:59:07.855 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0150000000000003e-06 2024-09-14T23:59:07.855 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.855 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.855 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:07.855 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198521+0000", 2024-09-14T23:59:07.855 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8374000000000001e-05 2024-09-14T23:59:07.855 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.855 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.855 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:07.855 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198533+0000", 2024-09-14T23:59:07.856 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1593000000000001e-05 2024-09-14T23:59:07.856 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.856 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.856 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:07.856 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198603+0000", 2024-09-14T23:59:07.856 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0279999999999998e-05 2024-09-14T23:59:07.857 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.857 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.857 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:07.857 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198617+0000", 2024-09-14T23:59:07.857 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3517e-05 2024-09-14T23:59:07.857 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.857 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.858 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:07.858 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.977067+0000", 2024-09-14T23:59:07.858 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.77845065800000002 2024-09-14T23:59:07.858 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.858 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.858 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for missing object", 2024-09-14T23:59:07.858 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.977173+0000", 2024-09-14T23:59:07.859 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010543999999999999 2024-09-14T23:59:07.859 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.859 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.859 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:07.859 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.978737+0000", 2024-09-14T23:59:07.859 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001564078 2024-09-14T23:59:07.859 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.859 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.860 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:07.860 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.978845+0000", 2024-09-14T23:59:07.860 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000108643 2024-09-14T23:59:07.860 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.860 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.860 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1", 2024-09-14T23:59:07.861 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.978964+0000", 2024-09-14T23:59:07.861 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000119085 2024-09-14T23:59:07.861 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.861 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.861 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:59:07.861 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980109+0000", 2024-09-14T23:59:07.861 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0011442170000000001 2024-09-14T23:59:07.861 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.862 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.862 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:59:07.862 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980130+0000", 2024-09-14T23:59:07.862 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1766000000000001e-05 2024-09-14T23:59:07.862 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.862 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.862 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:07.863 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980171+0000", 2024-09-14T23:59:07.863 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0627999999999998e-05 2024-09-14T23:59:07.863 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.863 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.863 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:07.863 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980244+0000", 2024-09-14T23:59:07.863 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3373000000000004e-05 2024-09-14T23:59:07.864 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:07.864 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:07.864 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:07.864 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.864 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.864 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129159 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 5996544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:07.864 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.927783+0000", 2024-09-14T23:59:07.865 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 88.921995627000001, 2024-09-14T23:59:07.865 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054661122999999999, 2024-09-14T23:59:07.865 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:07.865 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:07.865 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:07.865 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:07.866 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:07.866 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:07.866 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129159 2024-09-14T23:59:07.866 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.866 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:07.866 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.867 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:07.867 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.927783+0000", 2024-09-14T23:59:07.867 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:07.867 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.867 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.867 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:07.867 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.927783+0000", 2024-09-14T23:59:07.867 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:07.868 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.868 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.868 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:07.868 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929684+0000", 2024-09-14T23:59:07.868 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0019011080000000001 2024-09-14T23:59:07.868 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.868 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.869 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:07.869 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929688+0000", 2024-09-14T23:59:07.869 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2610000000000004e-06 2024-09-14T23:59:07.869 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.869 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.869 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:07.869 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929697+0000", 2024-09-14T23:59:07.870 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0219999999999993e-06 2024-09-14T23:59:07.870 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.870 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.870 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:07.870 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929702+0000", 2024-09-14T23:59:07.870 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8840000000000002e-06 2024-09-14T23:59:07.870 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.871 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.871 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:07.871 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.982090+0000", 2024-09-14T23:59:07.871 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052388045000000001 2024-09-14T23:59:07.871 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.871 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.871 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:07.871 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.982208+0000", 2024-09-14T23:59:07.872 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118236 2024-09-14T23:59:07.872 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.872 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.872 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:07.872 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.982444+0000", 2024-09-14T23:59:07.872 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000235567 2024-09-14T23:59:07.872 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:07.873 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:07.873 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:07.873 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.873 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.873 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129166 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 8192000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:07.873 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.929794+0000", 2024-09-14T23:59:07.873 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 88.919984876000001, 2024-09-14T23:59:07.874 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054471973, 2024-09-14T23:59:07.874 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:07.874 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:07.874 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:07.874 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:07.874 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:07.874 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:07.875 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129166 2024-09-14T23:59:07.875 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.875 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:07.875 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.875 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:07.876 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929794+0000", 2024-09-14T23:59:07.876 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:07.876 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.876 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.876 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:07.876 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929794+0000", 2024-09-14T23:59:07.876 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:07.877 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.877 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.877 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:07.877 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929794+0000", 2024-09-14T23:59:07.877 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6899999999999997e-07 2024-09-14T23:59:07.877 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.878 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.878 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:07.878 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929797+0000", 2024-09-14T23:59:07.878 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.328e-06 2024-09-14T23:59:07.878 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.878 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.878 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:07.879 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929802+0000", 2024-09-14T23:59:07.879 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7309999999999998e-06 2024-09-14T23:59:07.879 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.879 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.879 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:07.879 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929807+0000", 2024-09-14T23:59:07.879 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4349999999999999e-06 2024-09-14T23:59:07.879 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.880 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.880 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:07.880 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984063+0000", 2024-09-14T23:59:07.880 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054256331999999997 2024-09-14T23:59:07.880 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.880 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.880 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:07.881 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984177+0000", 2024-09-14T23:59:07.881 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114225 2024-09-14T23:59:07.881 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.881 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.881 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:07.881 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984266+0000", 2024-09-14T23:59:07.882 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8352999999999999e-05 2024-09-14T23:59:07.882 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:07.882 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:07.882 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:07.882 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.882 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.882 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129167 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 6782976~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:07.883 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.929827+0000", 2024-09-14T23:59:07.883 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 88.919951467000004, 2024-09-14T23:59:07.883 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054637593999999998, 2024-09-14T23:59:07.883 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:07.883 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:07.883 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:07.883 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:07.883 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:07.884 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:07.884 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129167 2024-09-14T23:59:07.884 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.884 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:07.884 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.884 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:07.884 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929827+0000", 2024-09-14T23:59:07.885 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:07.885 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.885 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.885 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:07.885 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929827+0000", 2024-09-14T23:59:07.885 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:07.885 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.888 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.888 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:07.888 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929827+0000", 2024-09-14T23:59:07.888 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4000000000000002e-07 2024-09-14T23:59:07.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.889 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.889 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:07.889 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929830+0000", 2024-09-14T23:59:07.889 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.419e-06 2024-09-14T23:59:07.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.889 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.889 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:07.890 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929837+0000", 2024-09-14T23:59:07.890 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0450000000000003e-06 2024-09-14T23:59:07.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.890 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.890 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:07.890 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929842+0000", 2024-09-14T23:59:07.890 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.69e-06 2024-09-14T23:59:07.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.891 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.891 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:07.891 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984278+0000", 2024-09-14T23:59:07.891 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054436516999999997 2024-09-14T23:59:07.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.891 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.892 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:07.892 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984331+0000", 2024-09-14T23:59:07.892 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2590999999999998e-05 2024-09-14T23:59:07.892 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.892 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.892 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:07.892 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984465+0000", 2024-09-14T23:59:07.893 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013389200000000001 2024-09-14T23:59:07.893 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:07.893 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:07.893 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:07.893 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.893 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.893 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129168 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 8224768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:07.894 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.929849+0000", 2024-09-14T23:59:07.894 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 88.919929909000004, 2024-09-14T23:59:07.894 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054870368000000003, 2024-09-14T23:59:07.894 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:07.894 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:07.894 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:07.894 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:07.895 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:07.895 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:07.895 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129168 2024-09-14T23:59:07.895 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.895 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:07.895 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.895 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:07.895 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929849+0000", 2024-09-14T23:59:07.896 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:07.896 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.896 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.896 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:07.896 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929849+0000", 2024-09-14T23:59:07.896 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:07.896 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.897 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.897 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:07.897 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931681+0000", 2024-09-14T23:59:07.897 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018326989999999999 2024-09-14T23:59:07.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.897 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.897 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:07.898 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931686+0000", 2024-09-14T23:59:07.898 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7190000000000001e-06 2024-09-14T23:59:07.898 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.898 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.898 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:07.898 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931697+0000", 2024-09-14T23:59:07.898 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1287999999999999e-05 2024-09-14T23:59:07.899 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.899 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.899 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:07.899 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931704+0000", 2024-09-14T23:59:07.899 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3339999999999998e-06 2024-09-14T23:59:07.899 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.899 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.900 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:07.900 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984505+0000", 2024-09-14T23:59:07.900 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052801684000000002 2024-09-14T23:59:07.900 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.900 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.900 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:07.900 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984621+0000", 2024-09-14T23:59:07.901 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000116165 2024-09-14T23:59:07.901 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.901 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.901 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:07.901 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984719+0000", 2024-09-14T23:59:07.901 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7479000000000006e-05 2024-09-14T23:59:07.901 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:07.902 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:07.902 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:07.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.902 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.902 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129176 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 10452992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:07.902 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.931830+0000", 2024-09-14T23:59:07.902 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 88.917948147999994, 2024-09-14T23:59:07.903 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054513292999999997, 2024-09-14T23:59:07.903 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:07.903 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:07.903 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:07.903 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:07.903 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:07.903 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:07.904 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129176 2024-09-14T23:59:07.904 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.904 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:07.904 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.904 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:07.904 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931830+0000", 2024-09-14T23:59:07.904 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:07.905 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.905 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.905 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:07.905 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931830+0000", 2024-09-14T23:59:07.905 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:07.905 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.905 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.906 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:07.906 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.933678+0000", 2024-09-14T23:59:07.906 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018472440000000001 2024-09-14T23:59:07.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.906 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.906 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:07.906 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.933686+0000", 2024-09-14T23:59:07.906 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0239999999999997e-06 2024-09-14T23:59:07.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.907 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.907 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:07.907 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.933694+0000", 2024-09-14T23:59:07.907 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8400000000000001e-06 2024-09-14T23:59:07.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.907 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.908 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:07.908 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.933701+0000", 2024-09-14T23:59:07.908 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2870000000000001e-06 2024-09-14T23:59:07.908 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.908 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.908 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:07.908 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.986165+0000", 2024-09-14T23:59:07.909 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052463900000000001 2024-09-14T23:59:07.909 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.909 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.909 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:07.909 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.986259+0000", 2024-09-14T23:59:07.909 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4435e-05 2024-09-14T23:59:07.909 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.910 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.910 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:07.910 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.986344+0000", 2024-09-14T23:59:07.910 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4562999999999997e-05 2024-09-14T23:59:07.910 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:07.910 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:07.910 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:07.911 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.911 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.911 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129201 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 10289152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:07.911 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.935960+0000", 2024-09-14T23:59:07.911 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 88.913818685999999, 2024-09-14T23:59:07.911 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054338487999999997, 2024-09-14T23:59:07.911 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:07.912 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:07.912 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:07.912 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:07.912 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:07.912 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:07.912 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129201 2024-09-14T23:59:07.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.912 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:07.912 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.912 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:07.912 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935960+0000", 2024-09-14T23:59:07.912 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:07.913 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.913 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.913 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:07.913 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935960+0000", 2024-09-14T23:59:07.913 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:07.913 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.913 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.913 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:07.913 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935960+0000", 2024-09-14T23:59:07.913 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.72e-07 2024-09-14T23:59:07.913 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.913 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.913 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:07.914 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935963+0000", 2024-09-14T23:59:07.914 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5390000000000002e-06 2024-09-14T23:59:07.914 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.914 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.914 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:07.914 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935971+0000", 2024-09-14T23:59:07.914 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4589999999999998e-06 2024-09-14T23:59:07.914 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.914 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.914 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:07.914 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935977+0000", 2024-09-14T23:59:07.914 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4110000000000004e-06 2024-09-14T23:59:07.914 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.914 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.914 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:07.915 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990213+0000", 2024-09-14T23:59:07.915 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054236642000000002 2024-09-14T23:59:07.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.915 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.915 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:07.915 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990257+0000", 2024-09-14T23:59:07.915 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3696000000000003e-05 2024-09-14T23:59:07.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.915 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.915 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:07.915 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990298+0000", 2024-09-14T23:59:07.915 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1369000000000003e-05 2024-09-14T23:59:07.915 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:07.915 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:07.916 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:07.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.916 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.916 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129202 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 11370496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:07.916 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.935986+0000", 2024-09-14T23:59:07.916 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 88.913792068000006, 2024-09-14T23:59:07.916 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054543700000000001, 2024-09-14T23:59:07.916 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:07.916 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:07.916 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:07.916 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:07.916 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:07.916 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:07.917 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129202 2024-09-14T23:59:07.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.917 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:07.917 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.917 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:07.917 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935986+0000", 2024-09-14T23:59:07.917 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:07.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.917 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.917 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:07.917 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935986+0000", 2024-09-14T23:59:07.917 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:07.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.917 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.917 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:07.918 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937678+0000", 2024-09-14T23:59:07.918 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016917849999999999 2024-09-14T23:59:07.918 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.918 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.918 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:07.918 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937682+0000", 2024-09-14T23:59:07.918 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2400000000000001e-06 2024-09-14T23:59:07.918 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.918 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.918 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:07.918 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937695+0000", 2024-09-14T23:59:07.918 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2999e-05 2024-09-14T23:59:07.918 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.918 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.919 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:07.919 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937700+0000", 2024-09-14T23:59:07.919 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8420000000000004e-06 2024-09-14T23:59:07.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.919 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.919 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:07.919 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990336+0000", 2024-09-14T23:59:07.919 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052636060999999998 2024-09-14T23:59:07.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.919 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.919 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:07.919 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990435+0000", 2024-09-14T23:59:07.919 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9015e-05 2024-09-14T23:59:07.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.919 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.920 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:07.920 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990530+0000", 2024-09-14T23:59:07.920 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4758000000000003e-05 2024-09-14T23:59:07.920 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:07.920 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:07.920 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:07.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.920 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.920 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129214 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 11468800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:07.920 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.937939+0000", 2024-09-14T23:59:07.920 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 88.911839154000006, 2024-09-14T23:59:07.920 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054463733, 2024-09-14T23:59:07.920 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:07.921 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:07.921 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:07.921 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:07.921 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:07.921 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:07.921 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129214 2024-09-14T23:59:07.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.921 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:07.921 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.921 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:07.921 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937939+0000", 2024-09-14T23:59:07.921 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:07.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.922 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.922 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:07.922 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937939+0000", 2024-09-14T23:59:07.922 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:07.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.922 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.922 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:07.922 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939674+0000", 2024-09-14T23:59:07.922 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017342589999999999 2024-09-14T23:59:07.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.922 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.922 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:07.922 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939678+0000", 2024-09-14T23:59:07.923 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2590000000000002e-06 2024-09-14T23:59:07.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.923 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.923 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:07.923 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939699+0000", 2024-09-14T23:59:07.923 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1143000000000001e-05 2024-09-14T23:59:07.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.923 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.923 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:07.923 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939706+0000", 2024-09-14T23:59:07.923 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0269999999999998e-06 2024-09-14T23:59:07.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.923 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.923 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:07.924 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.992221+0000", 2024-09-14T23:59:07.924 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052514981000000002 2024-09-14T23:59:07.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.924 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.924 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:07.924 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.992321+0000", 2024-09-14T23:59:07.924 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9625999999999998e-05 2024-09-14T23:59:07.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.924 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.924 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:07.924 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.992403+0000", 2024-09-14T23:59:07.925 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2437999999999999e-05 2024-09-14T23:59:07.925 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:07.925 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:07.925 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:07.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.925 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.925 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129226 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 14090240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:07.925 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.939867+0000", 2024-09-14T23:59:07.925 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 88.909911226000006, 2024-09-14T23:59:07.925 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054523573999999998, 2024-09-14T23:59:07.926 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:07.926 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:07.926 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:07.926 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:07.926 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:07.926 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:07.926 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129226 2024-09-14T23:59:07.926 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.926 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:07.926 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.926 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:07.926 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939867+0000", 2024-09-14T23:59:07.926 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:07.926 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.927 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.927 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:07.927 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939867+0000", 2024-09-14T23:59:07.927 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:07.927 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.927 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.927 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:07.927 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941669+0000", 2024-09-14T23:59:07.927 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018021549999999999 2024-09-14T23:59:07.927 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.927 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.927 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:07.927 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941673+0000", 2024-09-14T23:59:07.927 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1269999999999999e-06 2024-09-14T23:59:07.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.928 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.928 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:07.928 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941679+0000", 2024-09-14T23:59:07.928 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1879999999999999e-06 2024-09-14T23:59:07.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.928 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.928 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:07.928 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941682+0000", 2024-09-14T23:59:07.928 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7110000000000001e-06 2024-09-14T23:59:07.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.928 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.928 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:07.928 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.994201+0000", 2024-09-14T23:59:07.929 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052518265000000001 2024-09-14T23:59:07.929 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.929 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.929 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:07.929 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.994302+0000", 2024-09-14T23:59:07.929 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101653 2024-09-14T23:59:07.929 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.929 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.929 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:07.929 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.994391+0000", 2024-09-14T23:59:07.929 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8474999999999996e-05 2024-09-14T23:59:07.929 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:07.929 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:07.929 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:07.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.930 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.930 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129237 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 12386304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:07.930 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.941825+0000", 2024-09-14T23:59:07.930 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 88.907953464000002, 2024-09-14T23:59:07.930 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054368076000000001, 2024-09-14T23:59:07.930 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:07.930 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:07.930 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:07.930 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:07.930 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:07.930 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:07.930 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129237 2024-09-14T23:59:07.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.930 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:07.931 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.931 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:07.931 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941825+0000", 2024-09-14T23:59:07.931 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:07.931 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.931 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.931 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:07.931 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941825+0000", 2024-09-14T23:59:07.931 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:07.931 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.931 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.931 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:07.931 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941825+0000", 2024-09-14T23:59:07.931 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.29e-07 2024-09-14T23:59:07.932 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.932 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.932 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:07.932 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941826+0000", 2024-09-14T23:59:07.932 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0999999999999997e-07 2024-09-14T23:59:07.932 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.932 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.932 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:07.932 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941834+0000", 2024-09-14T23:59:07.932 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5519999999999999e-06 2024-09-14T23:59:07.932 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.932 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.932 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:07.933 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941843+0000", 2024-09-14T23:59:07.933 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2960000000000004e-06 2024-09-14T23:59:07.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.933 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.933 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:07.933 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996108+0000", 2024-09-14T23:59:07.933 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054265200999999999 2024-09-14T23:59:07.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.933 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.933 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:07.934 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996152+0000", 2024-09-14T23:59:07.934 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4066999999999997e-05 2024-09-14T23:59:07.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.934 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.934 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:07.934 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996193+0000", 2024-09-14T23:59:07.934 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0821000000000001e-05 2024-09-14T23:59:07.934 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:07.934 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:07.934 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:07.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.934 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.934 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129238 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 13828096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:07.934 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.941851+0000", 2024-09-14T23:59:07.935 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 88.907927139999998, 2024-09-14T23:59:07.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.05458031, 2024-09-14T23:59:07.935 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:07.935 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:07.935 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:07.935 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:07.935 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:07.935 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:07.935 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129238 2024-09-14T23:59:07.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.935 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:07.935 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:07.935 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941851+0000", 2024-09-14T23:59:07.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:07.936 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.936 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.936 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:07.936 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941851+0000", 2024-09-14T23:59:07.936 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:07.936 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.936 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.936 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:07.936 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943670+0000", 2024-09-14T23:59:07.936 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018181930000000001 2024-09-14T23:59:07.936 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.936 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.936 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:07.936 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943673+0000", 2024-09-14T23:59:07.937 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3790000000000002e-06 2024-09-14T23:59:07.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.937 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.937 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:07.937 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943679+0000", 2024-09-14T23:59:07.937 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8719999999999999e-06 2024-09-14T23:59:07.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.937 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.937 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:07.937 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943683+0000", 2024-09-14T23:59:07.937 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8539999999999999e-06 2024-09-14T23:59:07.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.937 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.937 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:07.938 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996233+0000", 2024-09-14T23:59:07.938 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052550690999999997 2024-09-14T23:59:07.938 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.938 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.938 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:07.938 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996339+0000", 2024-09-14T23:59:07.938 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105989 2024-09-14T23:59:07.938 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.938 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.938 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:07.938 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996432+0000", 2024-09-14T23:59:07.938 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2331999999999996e-05 2024-09-14T23:59:07.938 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:07.939 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:07.939 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:07.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.939 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129249 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 12124160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:07.939 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.943826+0000", 2024-09-14T23:59:07.939 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 88.905952649, 2024-09-14T23:59:07.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054373266000000003, 2024-09-14T23:59:07.939 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:07.939 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:07.939 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:07.939 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:07.939 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:07.939 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:07.940 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129249 2024-09-14T23:59:07.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.940 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:07.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:07.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943826+0000", 2024-09-14T23:59:07.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:07.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:07.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943826+0000", 2024-09-14T23:59:07.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:07.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.941 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:07.941 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943826+0000", 2024-09-14T23:59:07.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.03e-07 2024-09-14T23:59:07.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.941 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:07.941 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943827+0000", 2024-09-14T23:59:07.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.08e-06 2024-09-14T23:59:07.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.941 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:07.941 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943834+0000", 2024-09-14T23:59:07.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4119999999999998e-06 2024-09-14T23:59:07.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.942 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.942 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:07.942 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943836+0000", 2024-09-14T23:59:07.942 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7369999999999998e-06 2024-09-14T23:59:07.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.942 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.942 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:07.942 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998115+0000", 2024-09-14T23:59:07.942 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054279040000000001 2024-09-14T23:59:07.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.942 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.942 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:07.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998159+0000", 2024-09-14T23:59:07.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3486999999999999e-05 2024-09-14T23:59:07.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.943 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:07.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998199+0000", 2024-09-14T23:59:07.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0306999999999999e-05 2024-09-14T23:59:07.943 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:07.943 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:07.943 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:07.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.943 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129250 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 13565952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:07.944 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.943840+0000", 2024-09-14T23:59:07.944 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 88.905938699000004, 2024-09-14T23:59:07.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054573875000000001, 2024-09-14T23:59:07.944 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:07.944 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:07.944 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:07.944 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:07.944 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:07.944 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:07.944 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129250 2024-09-14T23:59:07.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.944 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:07.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:07.945 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943840+0000", 2024-09-14T23:59:07.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:07.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:07.945 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943840+0000", 2024-09-14T23:59:07.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:07.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:07.945 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945669+0000", 2024-09-14T23:59:07.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018289720000000001 2024-09-14T23:59:07.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.946 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:07.946 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945671+0000", 2024-09-14T23:59:07.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6869999999999999e-06 2024-09-14T23:59:07.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.946 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:07.946 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945680+0000", 2024-09-14T23:59:07.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5259999999999993e-06 2024-09-14T23:59:07.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.946 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:07.946 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945685+0000", 2024-09-14T23:59:07.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8559999999999998e-06 2024-09-14T23:59:07.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.947 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:07.947 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998242+0000", 2024-09-14T23:59:07.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052557505999999997 2024-09-14T23:59:07.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.947 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:07.947 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998329+0000", 2024-09-14T23:59:07.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6410999999999996e-05 2024-09-14T23:59:07.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.947 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:07.947 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998414+0000", 2024-09-14T23:59:07.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4917e-05 2024-09-14T23:59:07.947 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:07.948 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:07.948 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:07.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.948 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129262 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 13303808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:07.948 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.945825+0000", 2024-09-14T23:59:07.948 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 88.903953758, 2024-09-14T23:59:07.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054515942999999997, 2024-09-14T23:59:07.948 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:07.948 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:07.948 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:07.948 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:07.948 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:07.948 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:07.949 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129262 2024-09-14T23:59:07.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.949 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:07.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:07.949 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945825+0000", 2024-09-14T23:59:07.949 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:07.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:07.949 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945825+0000", 2024-09-14T23:59:07.949 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:07.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.950 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:07.950 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.947670+0000", 2024-09-14T23:59:07.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018451229999999999 2024-09-14T23:59:07.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.950 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:07.950 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.947672+0000", 2024-09-14T23:59:07.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5289999999999998e-06 2024-09-14T23:59:07.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.950 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:07.950 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.947678+0000", 2024-09-14T23:59:07.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9880000000000001e-06 2024-09-14T23:59:07.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.951 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:07.951 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.947682+0000", 2024-09-14T23:59:07.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.63e-06 2024-09-14T23:59:07.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.951 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:07.951 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000183+0000", 2024-09-14T23:59:07.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052500770000000002 2024-09-14T23:59:07.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.951 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:07.951 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000266+0000", 2024-09-14T23:59:07.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3133000000000005e-05 2024-09-14T23:59:07.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.952 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:07.952 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000341+0000", 2024-09-14T23:59:07.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4770000000000004e-05 2024-09-14T23:59:07.952 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:07.952 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:07.952 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:07.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.952 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129299 3.10 3:0f1e06ee:::rbd_object_map.1118721be852:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:07.952 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.952330+0000", 2024-09-14T23:59:07.952 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 88.897448944999994, 2024-09-14T23:59:07.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054412818000000002, 2024-09-14T23:59:07.953 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:07.953 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:07.953 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:07.953 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:07.953 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:07.953 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:07.953 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129299 2024-09-14T23:59:07.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.953 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:07.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.953 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:07.953 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952330+0000", 2024-09-14T23:59:07.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:07.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.954 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.954 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:07.954 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952330+0000", 2024-09-14T23:59:07.954 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:07.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.954 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.954 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:07.954 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952330+0000", 2024-09-14T23:59:07.954 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4900000000000001e-07 2024-09-14T23:59:07.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.954 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.954 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:07.954 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952332+0000", 2024-09-14T23:59:07.954 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2879999999999999e-06 2024-09-14T23:59:07.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.955 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:07.955 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952340+0000", 2024-09-14T23:59:07.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9910000000000002e-06 2024-09-14T23:59:07.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.955 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:07.955 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952344+0000", 2024-09-14T23:59:07.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0280000000000002e-06 2024-09-14T23:59:07.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.955 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:07.955 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.004722+0000", 2024-09-14T23:59:07.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052377932000000002 2024-09-14T23:59:07.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:07.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.004778+0000", 2024-09-14T23:59:07.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5795999999999999e-05 2024-09-14T23:59:07.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:59:07.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.005016+0000", 2024-09-14T23:59:07.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000238006 2024-09-14T23:59:07.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:59:07.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.006366+0000", 2024-09-14T23:59:07.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001349786 2024-09-14T23:59:07.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:59:07.957 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.006685+0000", 2024-09-14T23:59:07.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00031970000000000002 2024-09-14T23:59:07.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:07.957 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.006718+0000", 2024-09-14T23:59:07.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2329000000000002e-05 2024-09-14T23:59:07.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:07.958 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.006742+0000", 2024-09-14T23:59:07.958 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4613e-05 2024-09-14T23:59:07.958 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:07.958 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:07.958 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:07.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.958 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.958 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186210 3.d 3.b2ff3acd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:59:07.958 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.129044+0000", 2024-09-14T23:59:07.958 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 60.720734581999999, 2024-09-14T23:59:07.958 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.84555443900000005, 2024-09-14T23:59:07.958 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:07.958 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:07.958 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:59:07.959 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:07.959 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:07.959 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:07.959 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186210 2024-09-14T23:59:07.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.959 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:07.959 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.959 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:07.959 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129044+0000", 2024-09-14T23:59:07.959 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:07.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.959 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.959 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:07.959 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129044+0000", 2024-09-14T23:59:07.960 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:07.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.960 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:07.960 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129046+0000", 2024-09-14T23:59:07.960 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.742e-06 2024-09-14T23:59:07.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.960 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:07.960 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129052+0000", 2024-09-14T23:59:07.960 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7390000000000002e-06 2024-09-14T23:59:07.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:07.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129067+0000", 2024-09-14T23:59:07.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.417e-05 2024-09-14T23:59:07.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:07.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129076+0000", 2024-09-14T23:59:07.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2289999999999992e-06 2024-09-14T23:59:07.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:07.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.130947+0000", 2024-09-14T23:59:07.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001871075 2024-09-14T23:59:07.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.962 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:07.962 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.130955+0000", 2024-09-14T23:59:07.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4500000000000004e-06 2024-09-14T23:59:07.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.962 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:07.962 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974587+0000", 2024-09-14T23:59:07.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.84363205500000005 2024-09-14T23:59:07.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.962 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:07.962 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974598+0000", 2024-09-14T23:59:07.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0978999999999999e-05 2024-09-14T23:59:07.963 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:07.963 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:07.963 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:07.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.963 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.963 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187415 3.d 3.b2ff3acd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:59:07.963 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.254681+0000", 2024-09-14T23:59:07.963 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 60.595097060999997, 2024-09-14T23:59:07.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.71995530699999999, 2024-09-14T23:59:07.963 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:07.963 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:07.963 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:59:07.963 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:07.963 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:07.963 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:07.964 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187415 2024-09-14T23:59:07.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.964 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:07.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.964 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:07.964 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254681+0000", 2024-09-14T23:59:07.964 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:07.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.964 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:07.964 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254681+0000", 2024-09-14T23:59:07.964 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:07.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.965 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:07.965 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254682+0000", 2024-09-14T23:59:07.965 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2e-07 2024-09-14T23:59:07.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.965 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.965 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:07.965 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254684+0000", 2024-09-14T23:59:07.965 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2730000000000001e-06 2024-09-14T23:59:07.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.965 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.965 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:07.965 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254689+0000", 2024-09-14T23:59:07.965 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4769999999999997e-06 2024-09-14T23:59:07.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:07.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254692+0000", 2024-09-14T23:59:07.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9890000000000001e-06 2024-09-14T23:59:07.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:07.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254721+0000", 2024-09-14T23:59:07.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9762999999999999e-05 2024-09-14T23:59:07.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:07.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254725+0000", 2024-09-14T23:59:07.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9650000000000002e-06 2024-09-14T23:59:07.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.967 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:07.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974628+0000", 2024-09-14T23:59:07.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.71990235499999999 2024-09-14T23:59:07.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.967 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:07.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974637+0000", 2024-09-14T23:59:07.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0650000000000005e-06 2024-09-14T23:59:07.967 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:07.967 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:07.967 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:07.968 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.968 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.968 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187457 3.d 3.b2ff3acd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:59:07.968 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.257980+0000", 2024-09-14T23:59:07.968 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 60.591798486999998, 2024-09-14T23:59:07.968 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.71671711199999999, 2024-09-14T23:59:07.968 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:07.968 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:07.968 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:59:07.968 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:07.968 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:07.968 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:07.968 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187457 2024-09-14T23:59:07.968 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.969 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:07.969 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.969 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:07.969 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257980+0000", 2024-09-14T23:59:07.969 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:07.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.969 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.969 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:07.969 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257980+0000", 2024-09-14T23:59:07.969 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:07.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.969 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.969 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:07.969 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257980+0000", 2024-09-14T23:59:07.969 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.17e-07 2024-09-14T23:59:07.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.970 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:07.970 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257981+0000", 2024-09-14T23:59:07.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.172e-06 2024-09-14T23:59:07.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.970 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:07.970 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257984+0000", 2024-09-14T23:59:07.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9550000000000001e-06 2024-09-14T23:59:07.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.970 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:07.970 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257987+0000", 2024-09-14T23:59:07.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5799999999999999e-06 2024-09-14T23:59:07.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.971 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:07.971 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.258030+0000", 2024-09-14T23:59:07.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3118999999999998e-05 2024-09-14T23:59:07.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.971 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:07.971 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.258038+0000", 2024-09-14T23:59:07.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6850000000000001e-06 2024-09-14T23:59:07.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.971 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:07.972 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974688+0000", 2024-09-14T23:59:07.972 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.71665016000000004 2024-09-14T23:59:07.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.972 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.972 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:07.972 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974697+0000", 2024-09-14T23:59:07.972 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2240000000000001e-06 2024-09-14T23:59:07.972 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:07.972 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:07.972 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:07.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.972 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.972 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:251656 3.c 3:32436721:::rbd_object_map.1118e09062ff:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-14T23:59:07.972 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:27.433419+0000", 2024-09-14T23:59:07.973 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 40.416359813, 2024-09-14T23:59:07.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.55083879499999999, 2024-09-14T23:59:07.973 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:07.973 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:07.973 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:07.973 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:07.973 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:07.973 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:07.973 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 251656 2024-09-14T23:59:07.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.973 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:07.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.973 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:07.973 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433419+0000", 2024-09-14T23:59:07.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:07.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.974 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:07.974 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433419+0000", 2024-09-14T23:59:07.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:07.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.974 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:07.974 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433419+0000", 2024-09-14T23:59:07.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1500000000000003e-07 2024-09-14T23:59:07.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.974 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:07.975 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433424+0000", 2024-09-14T23:59:07.975 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2409999999999997e-06 2024-09-14T23:59:07.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.975 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.975 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:07.975 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433435+0000", 2024-09-14T23:59:07.975 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1009000000000001e-05 2024-09-14T23:59:07.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.975 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.975 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:07.975 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433440+0000", 2024-09-14T23:59:07.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1950000000000002e-06 2024-09-14T23:59:07.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.976 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:07.976 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433487+0000", 2024-09-14T23:59:07.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7184000000000001e-05 2024-09-14T23:59:07.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.976 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:07.976 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433494+0000", 2024-09-14T23:59:07.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5490000000000003e-06 2024-09-14T23:59:07.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.977 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:07.977 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.981923+0000", 2024-09-14T23:59:07.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.54842898600000001 2024-09-14T23:59:07.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.977 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:07.977 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.982097+0000", 2024-09-14T23:59:07.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017428699999999999 2024-09-14T23:59:07.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.977 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5", 2024-09-14T23:59:07.977 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.982334+0000", 2024-09-14T23:59:07.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00023700899999999999 2024-09-14T23:59:07.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.978 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:59:07.978 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.983927+0000", 2024-09-14T23:59:07.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0015932710000000001 2024-09-14T23:59:07.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.978 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:59:07.978 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.984128+0000", 2024-09-14T23:59:07.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020104199999999999 2024-09-14T23:59:07.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.978 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:07.978 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.984212+0000", 2024-09-14T23:59:07.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3709999999999996e-05 2024-09-14T23:59:07.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:07.979 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:07.979 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:07.979 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.984257+0000", 2024-09-14T23:59:07.979 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5497000000000002e-05 2024-09-14T23:59:07.979 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:07.979 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:07.979 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:07.979 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:07.979 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:07.979 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:59:07.981 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T23:59:07.981 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:59:07.979+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:59:07.982 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is set 2024-09-14T23:59:08.082 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:59:08.081+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:59:08.092 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:59:08.092 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:59:08.092 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:59:08.093 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:59:08.100 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T23:59:08.186 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:59:08.185+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:59:08.249 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:59:08.250 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:59:08.250 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:59:08.250 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:59:08.250 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:59:08.258 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T23:59:08.284 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:59:08.283+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:59:08.384 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:59:08.383+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:59:08.407 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:59:08.407 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:59:08.408 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:59:08.408 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:59:08.408 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.408 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69262 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 4800512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:59:08.408 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300811+0000", 2024-09-14T23:59:08.408 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 113.10456638700001, 2024-09-14T23:59:08.408 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050403796000000001, 2024-09-14T23:59:08.408 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:08.408 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:08.408 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:08.408 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:08.408 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:08.408 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:08.409 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69262 2024-09-14T23:59:08.409 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.409 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:08.409 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.409 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:08.409 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300811+0000", 2024-09-14T23:59:08.409 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:08.409 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.409 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.409 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:08.409 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300811+0000", 2024-09-14T23:59:08.409 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:08.409 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.410 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.410 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:08.410 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300811+0000", 2024-09-14T23:59:08.410 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5700000000000002e-07 2024-09-14T23:59:08.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.410 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.410 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:08.410 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300815+0000", 2024-09-14T23:59:08.410 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.087e-06 2024-09-14T23:59:08.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.410 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.410 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:08.410 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300823+0000", 2024-09-14T23:59:08.410 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8720000000000002e-06 2024-09-14T23:59:08.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.411 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.411 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:08.411 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300830+0000", 2024-09-14T23:59:08.411 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0589999999999997e-06 2024-09-14T23:59:08.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.411 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.411 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:08.411 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351020+0000", 2024-09-14T23:59:08.411 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050189552999999998 2024-09-14T23:59:08.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.411 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.411 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:08.412 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351112+0000", 2024-09-14T23:59:08.412 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2615999999999996e-05 2024-09-14T23:59:08.412 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.412 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.412 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:08.412 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351214+0000", 2024-09-14T23:59:08.412 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101952 2024-09-14T23:59:08.412 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:08.412 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:08.412 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:08.412 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.412 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.412 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69263 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 4915200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:59:08.412 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300842+0000", 2024-09-14T23:59:08.413 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 113.104534883, 2024-09-14T23:59:08.413 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050590637000000001, 2024-09-14T23:59:08.413 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:08.413 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:08.413 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:08.413 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:08.413 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:08.413 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:08.413 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69263 2024-09-14T23:59:08.413 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.413 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:08.413 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.413 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:08.413 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300842+0000", 2024-09-14T23:59:08.414 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:08.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.414 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.414 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:08.414 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300842+0000", 2024-09-14T23:59:08.414 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:08.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.414 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.414 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:08.414 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300843+0000", 2024-09-14T23:59:08.414 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4000000000000002e-07 2024-09-14T23:59:08.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.414 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.414 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:08.414 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300846+0000", 2024-09-14T23:59:08.415 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8720000000000002e-06 2024-09-14T23:59:08.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.415 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.415 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:08.415 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300853+0000", 2024-09-14T23:59:08.415 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0450000000000003e-06 2024-09-14T23:59:08.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.415 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.415 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:08.415 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300859+0000", 2024-09-14T23:59:08.415 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9510000000000003e-06 2024-09-14T23:59:08.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.415 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.415 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:08.416 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351260+0000", 2024-09-14T23:59:08.416 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050401654999999997 2024-09-14T23:59:08.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.416 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.416 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:08.416 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351353+0000", 2024-09-14T23:59:08.416 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2785e-05 2024-09-14T23:59:08.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.416 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.416 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:08.416 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351433+0000", 2024-09-14T23:59:08.416 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9789000000000005e-05 2024-09-14T23:59:08.416 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:08.417 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:08.417 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:08.417 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.417 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.417 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69264 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5029888~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:59:08.417 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300866+0000", 2024-09-14T23:59:08.417 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 113.10451095000001, 2024-09-14T23:59:08.417 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050794063, 2024-09-14T23:59:08.417 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:08.417 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:08.417 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:08.417 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:08.417 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:08.418 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:08.418 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69264 2024-09-14T23:59:08.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.418 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:08.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.418 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:08.418 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300866+0000", 2024-09-14T23:59:08.418 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:08.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.418 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:08.418 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300866+0000", 2024-09-14T23:59:08.418 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:08.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.419 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.419 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:08.419 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300866+0000", 2024-09-14T23:59:08.419 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8799999999999998e-07 2024-09-14T23:59:08.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.419 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.419 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:08.419 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300869+0000", 2024-09-14T23:59:08.419 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0880000000000002e-06 2024-09-14T23:59:08.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.419 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.419 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:08.419 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300875+0000", 2024-09-14T23:59:08.419 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9280000000000004e-06 2024-09-14T23:59:08.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.420 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.420 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:08.420 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300882+0000", 2024-09-14T23:59:08.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4740000000000002e-06 2024-09-14T23:59:08.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.420 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.420 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:08.420 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351472+0000", 2024-09-14T23:59:08.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050590006999999999 2024-09-14T23:59:08.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.420 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.420 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:08.420 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351563+0000", 2024-09-14T23:59:08.421 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0762999999999998e-05 2024-09-14T23:59:08.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.421 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.421 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:08.421 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351660+0000", 2024-09-14T23:59:08.421 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7415000000000002e-05 2024-09-14T23:59:08.421 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:08.421 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:08.421 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:08.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.421 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.421 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69265 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5144576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:59:08.421 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300890+0000", 2024-09-14T23:59:08.422 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 113.10448744599999, 2024-09-14T23:59:08.422 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050921389999999997, 2024-09-14T23:59:08.422 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:08.422 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:08.422 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:08.422 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:08.422 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:08.422 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:08.422 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69265 2024-09-14T23:59:08.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.422 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:08.422 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.422 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:08.422 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300890+0000", 2024-09-14T23:59:08.423 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:08.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.423 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:08.423 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300890+0000", 2024-09-14T23:59:08.423 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:08.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.423 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:08.423 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300890+0000", 2024-09-14T23:59:08.423 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2500000000000001e-07 2024-09-14T23:59:08.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:08.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300892+0000", 2024-09-14T23:59:08.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2469999999999998e-06 2024-09-14T23:59:08.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:08.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300899+0000", 2024-09-14T23:59:08.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5509999999999996e-06 2024-09-14T23:59:08.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:08.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300904+0000", 2024-09-14T23:59:08.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3299999999999998e-06 2024-09-14T23:59:08.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.425 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:08.425 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351707+0000", 2024-09-14T23:59:08.425 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050802673999999999 2024-09-14T23:59:08.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.425 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:08.425 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351755+0000", 2024-09-14T23:59:08.425 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8362000000000002e-05 2024-09-14T23:59:08.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.425 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:08.425 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351811+0000", 2024-09-14T23:59:08.425 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5801000000000003e-05 2024-09-14T23:59:08.426 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:08.426 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:08.426 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:08.426 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.426 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.426 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69266 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5259264~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:59:08.426 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300911+0000", 2024-09-14T23:59:08.426 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 113.10446637299999, 2024-09-14T23:59:08.426 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051125089999999998, 2024-09-14T23:59:08.426 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:08.426 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:08.426 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:08.426 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:08.426 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:08.427 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:08.427 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69266 2024-09-14T23:59:08.427 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.427 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:08.427 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.427 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:08.427 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300911+0000", 2024-09-14T23:59:08.427 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:08.427 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.427 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.427 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:08.427 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300911+0000", 2024-09-14T23:59:08.427 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:08.427 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.428 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.428 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:08.428 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300911+0000", 2024-09-14T23:59:08.428 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-14T23:59:08.428 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.428 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.428 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:08.428 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300913+0000", 2024-09-14T23:59:08.428 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9300000000000002e-06 2024-09-14T23:59:08.428 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.428 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.428 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:08.428 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300919+0000", 2024-09-14T23:59:08.428 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.489e-06 2024-09-14T23:59:08.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.429 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:08.429 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300925+0000", 2024-09-14T23:59:08.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1719999999999998e-06 2024-09-14T23:59:08.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.429 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:08.429 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351847+0000", 2024-09-14T23:59:08.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050922621000000001 2024-09-14T23:59:08.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.429 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:08.430 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351938+0000", 2024-09-14T23:59:08.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0830999999999998e-05 2024-09-14T23:59:08.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.430 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:08.430 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352036+0000", 2024-09-14T23:59:08.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7676999999999993e-05 2024-09-14T23:59:08.430 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:08.430 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:08.430 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:08.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.430 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69267 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5373952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:59:08.430 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300945+0000", 2024-09-14T23:59:08.431 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 113.104431578, 2024-09-14T23:59:08.431 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051217934, 2024-09-14T23:59:08.431 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:08.431 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:08.431 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:08.431 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:08.431 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:08.431 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:08.431 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69267 2024-09-14T23:59:08.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.431 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:08.431 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.431 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:08.431 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300945+0000", 2024-09-14T23:59:08.431 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:08.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.432 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:08.432 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300945+0000", 2024-09-14T23:59:08.432 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:08.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.432 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:08.432 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300946+0000", 2024-09-14T23:59:08.432 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2600000000000002e-07 2024-09-14T23:59:08.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.432 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:08.432 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300948+0000", 2024-09-14T23:59:08.433 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4940000000000002e-06 2024-09-14T23:59:08.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.433 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:08.433 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300956+0000", 2024-09-14T23:59:08.433 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.622e-06 2024-09-14T23:59:08.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.433 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:08.433 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300962+0000", 2024-09-14T23:59:08.433 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5790000000000003e-06 2024-09-14T23:59:08.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.433 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:08.434 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352063+0000", 2024-09-14T23:59:08.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051101281999999998 2024-09-14T23:59:08.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.434 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:08.434 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352110+0000", 2024-09-14T23:59:08.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7401000000000002e-05 2024-09-14T23:59:08.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.434 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:08.435 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352163+0000", 2024-09-14T23:59:08.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3029999999999999e-05 2024-09-14T23:59:08.435 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:08.435 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:08.435 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:08.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.435 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69268 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5488640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:59:08.435 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300969+0000", 2024-09-14T23:59:08.435 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 113.10440790600001, 2024-09-14T23:59:08.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051413420000000001, 2024-09-14T23:59:08.436 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:08.436 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:08.436 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:08.436 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:08.436 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:08.436 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:08.436 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69268 2024-09-14T23:59:08.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.436 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:08.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.436 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:08.436 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300969+0000", 2024-09-14T23:59:08.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:08.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.437 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:08.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300969+0000", 2024-09-14T23:59:08.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:08.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.437 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:08.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300970+0000", 2024-09-14T23:59:08.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7899999999999999e-07 2024-09-14T23:59:08.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:08.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300982+0000", 2024-09-14T23:59:08.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2741e-05 2024-09-14T23:59:08.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:08.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300990+0000", 2024-09-14T23:59:08.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8169999999999999e-06 2024-09-14T23:59:08.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:08.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300995+0000", 2024-09-14T23:59:08.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2240000000000003e-06 2024-09-14T23:59:08.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.439 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:08.439 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352200+0000", 2024-09-14T23:59:08.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051204223 2024-09-14T23:59:08.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.439 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:08.439 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352285+0000", 2024-09-14T23:59:08.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5519999999999994e-05 2024-09-14T23:59:08.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.439 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:08.440 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352383+0000", 2024-09-14T23:59:08.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7516000000000006e-05 2024-09-14T23:59:08.440 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:08.440 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:08.440 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:08.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.440 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69269 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5603328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:59:08.440 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.301003+0000", 2024-09-14T23:59:08.440 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 113.104373724, 2024-09-14T23:59:08.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051497381000000002, 2024-09-14T23:59:08.440 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:08.441 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:08.441 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:08.441 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:08.441 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:08.441 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:08.441 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69269 2024-09-14T23:59:08.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.441 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:08.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:08.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.301003+0000", 2024-09-14T23:59:08.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:08.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:08.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.301003+0000", 2024-09-14T23:59:08.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:08.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:08.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302671+0000", 2024-09-14T23:59:08.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001667785 2024-09-14T23:59:08.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:08.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302676+0000", 2024-09-14T23:59:08.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6600000000000003e-06 2024-09-14T23:59:08.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.443 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:08.443 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302697+0000", 2024-09-14T23:59:08.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1288999999999999e-05 2024-09-14T23:59:08.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.443 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:08.443 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302716+0000", 2024-09-14T23:59:08.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9128999999999999e-05 2024-09-14T23:59:08.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.443 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:08.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352399+0000", 2024-09-14T23:59:08.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049682995000000001 2024-09-14T23:59:08.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:08.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352446+0000", 2024-09-14T23:59:08.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6341000000000002e-05 2024-09-14T23:59:08.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:08.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352501+0000", 2024-09-14T23:59:08.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5182000000000001e-05 2024-09-14T23:59:08.444 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:08.444 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:08.445 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:08.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.445 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69274 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6176768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:59:08.445 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302854+0000", 2024-09-14T23:59:08.445 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 113.102523064, 2024-09-14T23:59:08.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050579526, 2024-09-14T23:59:08.445 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:08.445 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:08.445 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:08.445 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:08.445 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:08.445 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:08.446 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69274 2024-09-14T23:59:08.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.446 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:08.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.446 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:08.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302854+0000", 2024-09-14T23:59:08.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:08.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.446 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:08.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302854+0000", 2024-09-14T23:59:08.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:08.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:08.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302855+0000", 2024-09-14T23:59:08.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0200000000000001e-07 2024-09-14T23:59:08.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:08.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302857+0000", 2024-09-14T23:59:08.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.447e-06 2024-09-14T23:59:08.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:08.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302864+0000", 2024-09-14T23:59:08.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7730000000000002e-06 2024-09-14T23:59:08.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:08.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302869+0000", 2024-09-14T23:59:08.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3199999999999999e-06 2024-09-14T23:59:08.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:08.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353249+0000", 2024-09-14T23:59:08.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050379647999999999 2024-09-14T23:59:08.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:08.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353336+0000", 2024-09-14T23:59:08.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6767999999999993e-05 2024-09-14T23:59:08.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:08.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353433+0000", 2024-09-14T23:59:08.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7868000000000006e-05 2024-09-14T23:59:08.449 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:08.449 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:08.449 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:08.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.449 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69275 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6291456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:59:08.449 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302889+0000", 2024-09-14T23:59:08.449 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 113.102488446, 2024-09-14T23:59:08.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050663118, 2024-09-14T23:59:08.450 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:08.450 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:08.450 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:08.450 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:08.450 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:08.450 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:08.450 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69275 2024-09-14T23:59:08.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.450 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:08.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.450 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:08.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302889+0000", 2024-09-14T23:59:08.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:08.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:08.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302889+0000", 2024-09-14T23:59:08.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:08.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:08.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302889+0000", 2024-09-14T23:59:08.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6399999999999998e-07 2024-09-14T23:59:08.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:08.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302891+0000", 2024-09-14T23:59:08.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.339e-06 2024-09-14T23:59:08.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:08.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302898+0000", 2024-09-14T23:59:08.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0380000000000002e-06 2024-09-14T23:59:08.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:08.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302904+0000", 2024-09-14T23:59:08.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2159999999999997e-06 2024-09-14T23:59:08.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:08.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353449+0000", 2024-09-14T23:59:08.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050545386999999997 2024-09-14T23:59:08.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:08.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353494+0000", 2024-09-14T23:59:08.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4592999999999998e-05 2024-09-14T23:59:08.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:08.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353552+0000", 2024-09-14T23:59:08.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8180999999999997e-05 2024-09-14T23:59:08.453 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:08.453 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:08.454 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:08.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.454 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69276 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6406144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:59:08.454 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302911+0000", 2024-09-14T23:59:08.454 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 113.102466033, 2024-09-14T23:59:08.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050865753, 2024-09-14T23:59:08.454 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:08.454 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:08.454 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:08.454 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:08.454 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:08.454 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:08.455 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69276 2024-09-14T23:59:08.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.455 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:08.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:08.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302911+0000", 2024-09-14T23:59:08.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:08.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:08.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302911+0000", 2024-09-14T23:59:08.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:08.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.456 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:08.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302912+0000", 2024-09-14T23:59:08.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0800000000000004e-07 2024-09-14T23:59:08.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.456 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:08.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302914+0000", 2024-09-14T23:59:08.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9589999999999998e-06 2024-09-14T23:59:08.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.456 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:08.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302920+0000", 2024-09-14T23:59:08.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4579999999999998e-06 2024-09-14T23:59:08.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:08.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302925+0000", 2024-09-14T23:59:08.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9429999999999999e-06 2024-09-14T23:59:08.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:08.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353586+0000", 2024-09-14T23:59:08.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050660966000000002 2024-09-14T23:59:08.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:08.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353679+0000", 2024-09-14T23:59:08.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3163000000000002e-05 2024-09-14T23:59:08.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:08.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353777+0000", 2024-09-14T23:59:08.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7555999999999999e-05 2024-09-14T23:59:08.458 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:08.458 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:08.458 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:08.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.458 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69277 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6520832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:59:08.458 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302932+0000", 2024-09-14T23:59:08.458 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 113.10244494600001, 2024-09-14T23:59:08.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050977674000000001, 2024-09-14T23:59:08.459 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:08.459 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:08.459 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:08.459 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:08.459 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:08.459 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:08.459 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69277 2024-09-14T23:59:08.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.459 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:08.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:08.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302932+0000", 2024-09-14T23:59:08.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:08.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:08.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302932+0000", 2024-09-14T23:59:08.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:08.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:08.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302932+0000", 2024-09-14T23:59:08.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3299999999999998e-07 2024-09-14T23:59:08.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:08.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302935+0000", 2024-09-14T23:59:08.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2139999999999999e-06 2024-09-14T23:59:08.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:08.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302941+0000", 2024-09-14T23:59:08.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5039999999999999e-06 2024-09-14T23:59:08.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:08.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302946+0000", 2024-09-14T23:59:08.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2079999999999999e-06 2024-09-14T23:59:08.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:08.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353805+0000", 2024-09-14T23:59:08.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050858588000000003 2024-09-14T23:59:08.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:08.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353849+0000", 2024-09-14T23:59:08.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4511000000000001e-05 2024-09-14T23:59:08.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:08.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353910+0000", 2024-09-14T23:59:08.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0316000000000003e-05 2024-09-14T23:59:08.463 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:08.463 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:08.463 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:08.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.463 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69278 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6635520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:59:08.463 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302953+0000", 2024-09-14T23:59:08.463 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 113.10242381800001, 2024-09-14T23:59:08.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051174059000000001, 2024-09-14T23:59:08.463 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:08.463 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:08.463 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:08.463 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:08.463 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:08.464 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:08.464 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69278 2024-09-14T23:59:08.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.464 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:08.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:08.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302953+0000", 2024-09-14T23:59:08.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:08.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:08.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302953+0000", 2024-09-14T23:59:08.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:08.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:08.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302954+0000", 2024-09-14T23:59:08.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5100000000000001e-07 2024-09-14T23:59:08.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:08.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302964+0000", 2024-09-14T23:59:08.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0475000000000001e-05 2024-09-14T23:59:08.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:08.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302970+0000", 2024-09-14T23:59:08.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3879999999999997e-06 2024-09-14T23:59:08.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:08.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302975+0000", 2024-09-14T23:59:08.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8409999999999999e-06 2024-09-14T23:59:08.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:08.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353944+0000", 2024-09-14T23:59:08.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050968952999999997 2024-09-14T23:59:08.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:08.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354031+0000", 2024-09-14T23:59:08.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6537000000000002e-05 2024-09-14T23:59:08.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:08.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354127+0000", 2024-09-14T23:59:08.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6514000000000005e-05 2024-09-14T23:59:08.467 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:08.467 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:08.467 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:08.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.467 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69279 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6750208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:59:08.467 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302984+0000", 2024-09-14T23:59:08.467 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 113.102393467, 2024-09-14T23:59:08.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051258845999999997, 2024-09-14T23:59:08.468 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:08.468 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:08.468 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:08.468 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:08.468 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:08.468 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:08.468 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69279 2024-09-14T23:59:08.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.468 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:08.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:08.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302984+0000", 2024-09-14T23:59:08.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:08.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.469 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:08.469 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302984+0000", 2024-09-14T23:59:08.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:08.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.469 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:08.469 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303025+0000", 2024-09-14T23:59:08.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1690999999999998e-05 2024-09-14T23:59:08.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.469 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:08.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303029+0000", 2024-09-14T23:59:08.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6559999999999998e-06 2024-09-14T23:59:08.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:08.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303037+0000", 2024-09-14T23:59:08.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7950000000000008e-06 2024-09-14T23:59:08.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:08.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303043+0000", 2024-09-14T23:59:08.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9669999999999998e-06 2024-09-14T23:59:08.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:08.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354142+0000", 2024-09-14T23:59:08.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051099072000000002 2024-09-14T23:59:08.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:08.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354183+0000", 2024-09-14T23:59:08.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0834000000000002e-05 2024-09-14T23:59:08.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:08.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354242+0000", 2024-09-14T23:59:08.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9830999999999997e-05 2024-09-14T23:59:08.471 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:08.472 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:08.472 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:08.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.472 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69280 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6864896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:59:08.472 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.303052+0000", 2024-09-14T23:59:08.472 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 113.102324924, 2024-09-14T23:59:08.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051406463999999999, 2024-09-14T23:59:08.472 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:08.472 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:08.472 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:08.472 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:08.472 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:08.473 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:08.473 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69280 2024-09-14T23:59:08.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.473 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:08.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:08.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303052+0000", 2024-09-14T23:59:08.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:08.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:08.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303052+0000", 2024-09-14T23:59:08.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:08.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:08.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304673+0000", 2024-09-14T23:59:08.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001621389 2024-09-14T23:59:08.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:08.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304681+0000", 2024-09-14T23:59:08.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8439999999999998e-06 2024-09-14T23:59:08.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:08.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304691+0000", 2024-09-14T23:59:08.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8339999999999994e-06 2024-09-14T23:59:08.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:08.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304698+0000", 2024-09-14T23:59:08.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3629999999999999e-06 2024-09-14T23:59:08.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:08.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354278+0000", 2024-09-14T23:59:08.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049580618 2024-09-14T23:59:08.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:08.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354364+0000", 2024-09-14T23:59:08.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6059999999999999e-05 2024-09-14T23:59:08.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:08.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354459+0000", 2024-09-14T23:59:08.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4356000000000003e-05 2024-09-14T23:59:08.476 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:08.476 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:08.476 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:08.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.476 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69286 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 7553024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:59:08.476 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.304811+0000", 2024-09-14T23:59:08.476 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 113.100566153, 2024-09-14T23:59:08.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050585512999999999, 2024-09-14T23:59:08.477 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:08.477 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:08.477 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:08.477 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:08.477 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:08.477 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:08.477 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69286 2024-09-14T23:59:08.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.478 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:08.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:08.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304811+0000", 2024-09-14T23:59:08.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:08.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:08.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304811+0000", 2024-09-14T23:59:08.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:08.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:08.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304811+0000", 2024-09-14T23:59:08.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.89e-07 2024-09-14T23:59:08.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:08.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304814+0000", 2024-09-14T23:59:08.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3199999999999998e-06 2024-09-14T23:59:08.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:08.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304821+0000", 2024-09-14T23:59:08.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5689999999999999e-06 2024-09-14T23:59:08.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:08.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304826+0000", 2024-09-14T23:59:08.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1200000000000001e-06 2024-09-14T23:59:08.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:08.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355226+0000", 2024-09-14T23:59:08.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050399653000000003 2024-09-14T23:59:08.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:08.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355314+0000", 2024-09-14T23:59:08.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8389999999999999e-05 2024-09-14T23:59:08.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:08.481 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355396+0000", 2024-09-14T23:59:08.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2071999999999996e-05 2024-09-14T23:59:08.481 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:08.481 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:08.481 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:08.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.481 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69287 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 7667712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:59:08.481 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.304833+0000", 2024-09-14T23:59:08.481 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 113.100543555, 2024-09-14T23:59:08.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050664740999999999, 2024-09-14T23:59:08.482 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:08.482 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:08.482 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:08.482 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:08.482 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:08.482 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:08.482 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69287 2024-09-14T23:59:08.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.482 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:08.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:08.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304833+0000", 2024-09-14T23:59:08.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:08.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:08.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304833+0000", 2024-09-14T23:59:08.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:08.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:08.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304834+0000", 2024-09-14T23:59:08.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4999999999999998e-07 2024-09-14T23:59:08.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:08.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304836+0000", 2024-09-14T23:59:08.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0279999999999999e-06 2024-09-14T23:59:08.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:08.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304842+0000", 2024-09-14T23:59:08.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8919999999999997e-06 2024-09-14T23:59:08.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:08.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304847+0000", 2024-09-14T23:59:08.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6160000000000001e-06 2024-09-14T23:59:08.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:08.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355411+0000", 2024-09-14T23:59:08.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050563540999999997 2024-09-14T23:59:08.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:08.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355459+0000", 2024-09-14T23:59:08.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7979000000000003e-05 2024-09-14T23:59:08.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:08.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355498+0000", 2024-09-14T23:59:08.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9335000000000002e-05 2024-09-14T23:59:08.486 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:08.486 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:08.486 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:08.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.486 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69288 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 7782400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:59:08.486 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.304854+0000", 2024-09-14T23:59:08.486 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 113.10052311699999, 2024-09-14T23:59:08.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050855145999999997, 2024-09-14T23:59:08.486 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:08.486 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:08.486 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:08.487 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:08.487 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:08.487 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:08.487 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69288 2024-09-14T23:59:08.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.487 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:08.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:08.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304854+0000", 2024-09-14T23:59:08.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:08.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:08.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304854+0000", 2024-09-14T23:59:08.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:08.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:08.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.306670+0000", 2024-09-14T23:59:08.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00181631 2024-09-14T23:59:08.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:08.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.306675+0000", 2024-09-14T23:59:08.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0220000000000004e-06 2024-09-14T23:59:08.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:08.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.306684+0000", 2024-09-14T23:59:08.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7309999999999999e-06 2024-09-14T23:59:08.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:08.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.306689+0000", 2024-09-14T23:59:08.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6609999999999999e-06 2024-09-14T23:59:08.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:08.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355533+0000", 2024-09-14T23:59:08.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048844578 2024-09-14T23:59:08.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:08.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355622+0000", 2024-09-14T23:59:08.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9006999999999997e-05 2024-09-14T23:59:08.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:08.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355709+0000", 2024-09-14T23:59:08.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6836999999999995e-05 2024-09-14T23:59:08.490 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:08.490 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:08.490 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:08.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.490 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:255885 3.13 3.54329f53 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-14T23:59:08.491 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:48.978834+0000", 2024-09-14T23:59:08.491 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.426542531999999, 2024-09-14T23:59:08.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.007469078, 2024-09-14T23:59:08.491 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:08.491 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:08.491 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:59:08.491 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:08.491 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:08.491 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:08.491 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 255885 2024-09-14T23:59:08.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.491 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:08.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:08.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.978834+0000", 2024-09-14T23:59:08.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:08.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:08.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.978834+0000", 2024-09-14T23:59:08.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:08.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:08.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.978836+0000", 2024-09-14T23:59:08.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.288e-06 2024-09-14T23:59:08.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:08.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.978844+0000", 2024-09-14T23:59:08.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9880000000000004e-06 2024-09-14T23:59:08.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:08.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.978861+0000", 2024-09-14T23:59:08.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7020999999999998e-05 2024-09-14T23:59:08.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:08.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.978872+0000", 2024-09-14T23:59:08.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1274000000000001e-05 2024-09-14T23:59:08.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:08.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.982358+0000", 2024-09-14T23:59:08.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003486393 2024-09-14T23:59:08.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:08.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.982368+0000", 2024-09-14T23:59:08.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0690000000000008e-06 2024-09-14T23:59:08.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:08.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:49.986277+0000", 2024-09-14T23:59:08.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0039092489999999 2024-09-14T23:59:08.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:08.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:49.986304+0000", 2024-09-14T23:59:08.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6795999999999999e-05 2024-09-14T23:59:08.495 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:08.495 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:08.495 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:08.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.495 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:255886 3.13 3.54329f53 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-14T23:59:08.495 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:48.981728+0000", 2024-09-14T23:59:08.495 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.423648614000001, 2024-09-14T23:59:08.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0046192860000001, 2024-09-14T23:59:08.496 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:08.496 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:08.496 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:59:08.496 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:08.496 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:08.496 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:08.496 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 255886 2024-09-14T23:59:08.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.496 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:08.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:08.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.981728+0000", 2024-09-14T23:59:08.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:08.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:08.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.981728+0000", 2024-09-14T23:59:08.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:08.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:08.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.981730+0000", 2024-09-14T23:59:08.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5179999999999999e-06 2024-09-14T23:59:08.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:08.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.981737+0000", 2024-09-14T23:59:08.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.956e-06 2024-09-14T23:59:08.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.498 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:08.498 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.981748+0000", 2024-09-14T23:59:08.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0754000000000001e-05 2024-09-14T23:59:08.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.498 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:08.498 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.981757+0000", 2024-09-14T23:59:08.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8850000000000005e-06 2024-09-14T23:59:08.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.498 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:08.498 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.983225+0000", 2024-09-14T23:59:08.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001468605 2024-09-14T23:59:08.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:08.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.983232+0000", 2024-09-14T23:59:08.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0090000000000001e-06 2024-09-14T23:59:08.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:08.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:49.986337+0000", 2024-09-14T23:59:08.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0031047470000001 2024-09-14T23:59:08.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:08.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:08.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:08.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:49.986348+0000", 2024-09-14T23:59:08.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0812000000000001e-05 2024-09-14T23:59:08.501 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:08.501 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:08.501 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:08.501 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:08.501 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:08.501 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:59:08.503 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:59:08.502+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:59:08.503 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T23:59:08.604 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:59:08.602+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:59:08.624 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:59:08.624 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:59:08.624 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:59:08.624 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:59:08.633 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T23:59:08.705 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:59:08.704+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:59:08.782 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:59:08.783 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:59:08.783 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:59:08.783 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:59:08.783 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:59:08.791 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T23:59:08.835 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:59:08.834+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:59:08.900 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:59:08.900 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:59:08.900 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:59:08.900 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:59:08.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.900 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:43054 3.1f 3:ffbfe596:::rbd_data.1118350d5ae5.00000000000002eb:head [set-alloc-hint object_size 262144 write_size 262144,write 196608~22770] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T23:59:08.901 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:59:08.901 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 135.93129973200001, 2024-09-14T23:59:08.901 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.99973334400000002, 2024-09-14T23:59:08.901 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:08.901 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:08.901 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:08.901 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:08.901 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:08.901 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:08.901 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 43054 2024-09-14T23:59:08.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.901 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:08.901 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.902 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:08.902 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:59:08.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:08.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.902 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:08.902 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:59:08.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:08.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.902 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:08.902 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966552+0000", 2024-09-14T23:59:08.902 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3790000000000001e-06 2024-09-14T23:59:08.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.903 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.903 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:08.903 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966565+0000", 2024-09-14T23:59:08.903 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2459999999999999e-05 2024-09-14T23:59:08.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.903 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.903 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:08.903 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966576+0000", 2024-09-14T23:59:08.903 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1039000000000001e-05 2024-09-14T23:59:08.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.904 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:08.904 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966582+0000", 2024-09-14T23:59:08.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9660000000000001e-06 2024-09-14T23:59:08.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.904 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:08.904 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966625+0000", 2024-09-14T23:59:08.904 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3529999999999998e-05 2024-09-14T23:59:08.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.904 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:59:08.905 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966632+0000", 2024-09-14T23:59:08.905 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9809999999999997e-06 2024-09-14T23:59:08.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.905 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:08.905 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.964460+0000", 2024-09-14T23:59:08.905 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.99782768300000002 2024-09-14T23:59:08.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.905 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:08.905 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.964641+0000", 2024-09-14T23:59:08.905 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000181089 2024-09-14T23:59:08.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.906 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-14T23:59:08.906 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.964809+0000", 2024-09-14T23:59:08.906 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000168505 2024-09-14T23:59:08.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.906 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.906 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:59:08.906 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966030+0000", 2024-09-14T23:59:08.906 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012206929999999999 2024-09-14T23:59:08.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.906 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.906 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:59:08.906 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966181+0000", 2024-09-14T23:59:08.906 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000150569 2024-09-14T23:59:08.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.907 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:59:08.907 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966243+0000", 2024-09-14T23:59:08.907 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2331999999999999e-05 2024-09-14T23:59:08.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.907 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:08.907 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966284+0000", 2024-09-14T23:59:08.907 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1118000000000002e-05 2024-09-14T23:59:08.907 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:08.907 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:08.907 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:08.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.908 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184314 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4162560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:59:08.908 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.077247+0000", 2024-09-14T23:59:08.908 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 61.820603063999997, 2024-09-14T23:59:08.908 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058772256000000002, 2024-09-14T23:59:08.908 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:08.908 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:08.908 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:08.908 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:08.908 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:08.908 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:08.908 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184314 2024-09-14T23:59:08.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.908 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:08.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:08.909 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.077247+0000", 2024-09-14T23:59:08.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:08.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:08.909 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.077247+0000", 2024-09-14T23:59:08.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:08.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.909 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:08.909 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.078906+0000", 2024-09-14T23:59:08.909 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016583279999999999 2024-09-14T23:59:08.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.910 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.910 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:08.910 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.078911+0000", 2024-09-14T23:59:08.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0420000000000002e-06 2024-09-14T23:59:08.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.910 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.910 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:08.910 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.078922+0000", 2024-09-14T23:59:08.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1354e-05 2024-09-14T23:59:08.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.910 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.910 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:08.910 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.078930+0000", 2024-09-14T23:59:08.910 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5730000000000002e-06 2024-09-14T23:59:08.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.911 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.911 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:08.912 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.135870+0000", 2024-09-14T23:59:08.912 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056940651000000002 2024-09-14T23:59:08.912 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.912 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.912 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:08.912 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.135951+0000", 2024-09-14T23:59:08.913 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0890999999999995e-05 2024-09-14T23:59:08.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.913 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:08.913 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.136020+0000", 2024-09-14T23:59:08.913 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8417000000000006e-05 2024-09-14T23:59:08.913 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:08.913 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:08.913 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:08.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.913 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184323 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4183040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:59:08.913 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.079228+0000", 2024-09-14T23:59:08.914 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 61.818621989999997, 2024-09-14T23:59:08.914 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058454487999999999, 2024-09-14T23:59:08.914 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:08.914 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:08.914 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:08.914 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:08.914 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:08.914 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:08.914 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184323 2024-09-14T23:59:08.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.914 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:08.914 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.914 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:08.914 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079228+0000", 2024-09-14T23:59:08.915 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:08.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.915 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.915 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:08.915 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079228+0000", 2024-09-14T23:59:08.915 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:08.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.915 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.915 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:08.915 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079229+0000", 2024-09-14T23:59:08.915 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5299999999999999e-07 2024-09-14T23:59:08.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.915 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.915 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:08.916 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079232+0000", 2024-09-14T23:59:08.916 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8839999999999998e-06 2024-09-14T23:59:08.916 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.916 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.916 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:08.916 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079238+0000", 2024-09-14T23:59:08.916 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2299999999999996e-06 2024-09-14T23:59:08.916 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.916 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.916 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:08.916 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079243+0000", 2024-09-14T23:59:08.916 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3029999999999999e-06 2024-09-14T23:59:08.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.917 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.917 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:08.917 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.137533+0000", 2024-09-14T23:59:08.917 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058289674999999999 2024-09-14T23:59:08.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.917 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.917 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:08.917 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.137615+0000", 2024-09-14T23:59:08.917 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2288999999999998e-05 2024-09-14T23:59:08.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.917 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.918 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:08.918 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.137683+0000", 2024-09-14T23:59:08.918 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7654e-05 2024-09-14T23:59:08.918 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:08.918 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:08.918 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:08.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.918 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.918 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184324 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4073472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:59:08.918 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.079251+0000", 2024-09-14T23:59:08.918 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 61.818599523000003, 2024-09-14T23:59:08.918 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058609383000000001, 2024-09-14T23:59:08.918 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:08.918 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:08.919 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:08.919 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:08.919 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:08.919 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:08.919 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184324 2024-09-14T23:59:08.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.919 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:08.919 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.919 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:08.919 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079251+0000", 2024-09-14T23:59:08.919 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:08.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.919 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.919 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:08.920 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079251+0000", 2024-09-14T23:59:08.920 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:08.920 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.920 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.920 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:08.920 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.080920+0000", 2024-09-14T23:59:08.920 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016687430000000001 2024-09-14T23:59:08.920 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.920 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.920 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:08.920 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.080924+0000", 2024-09-14T23:59:08.920 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3869999999999997e-06 2024-09-14T23:59:08.920 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.920 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.921 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:08.921 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.080935+0000", 2024-09-14T23:59:08.921 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0515e-05 2024-09-14T23:59:08.921 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.921 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.921 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:08.921 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.080942+0000", 2024-09-14T23:59:08.921 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0990000000000001e-06 2024-09-14T23:59:08.921 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.921 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.921 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:08.921 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.137713+0000", 2024-09-14T23:59:08.921 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056771288000000003 2024-09-14T23:59:08.921 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.922 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.922 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:08.922 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.137794+0000", 2024-09-14T23:59:08.922 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0613999999999998e-05 2024-09-14T23:59:08.922 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.922 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.922 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:08.922 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.137860+0000", 2024-09-14T23:59:08.922 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6736999999999995e-05 2024-09-14T23:59:08.922 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:08.922 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:08.922 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:08.922 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.922 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.923 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184346 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4099072~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:59:08.923 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.083616+0000", 2024-09-14T23:59:08.923 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 61.814234337999999, 2024-09-14T23:59:08.923 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058557419999999999, 2024-09-14T23:59:08.923 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:08.923 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:08.923 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:08.923 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:08.923 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:08.923 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:08.923 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184346 2024-09-14T23:59:08.923 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.923 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:08.923 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.924 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:08.924 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083616+0000", 2024-09-14T23:59:08.924 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:08.924 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.924 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.924 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:08.924 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083616+0000", 2024-09-14T23:59:08.924 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:08.924 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.924 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.924 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:08.924 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083620+0000", 2024-09-14T23:59:08.924 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0119999999999999e-06 2024-09-14T23:59:08.925 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.925 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.925 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:08.925 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083623+0000", 2024-09-14T23:59:08.925 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9059999999999998e-06 2024-09-14T23:59:08.925 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.925 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.925 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:08.925 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083631+0000", 2024-09-14T23:59:08.925 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2679999999999994e-06 2024-09-14T23:59:08.925 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.925 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.925 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:08.925 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083637+0000", 2024-09-14T23:59:08.926 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1210000000000003e-06 2024-09-14T23:59:08.926 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.926 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.926 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:08.926 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142018+0000", 2024-09-14T23:59:08.926 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058381111999999999 2024-09-14T23:59:08.926 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.926 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.926 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:08.926 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142100+0000", 2024-09-14T23:59:08.926 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1308000000000001e-05 2024-09-14T23:59:08.926 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.926 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.927 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:08.927 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142173+0000", 2024-09-14T23:59:08.927 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3693000000000001e-05 2024-09-14T23:59:08.927 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:08.927 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:08.927 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:08.927 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.927 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.927 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184347 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4122624~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:59:08.927 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.083647+0000", 2024-09-14T23:59:08.927 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 61.814203507999999, 2024-09-14T23:59:08.927 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058706388999999998, 2024-09-14T23:59:08.927 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:08.927 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:08.928 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:08.928 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:08.928 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:08.928 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:08.928 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184347 2024-09-14T23:59:08.928 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.928 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:08.928 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.928 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:08.928 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083647+0000", 2024-09-14T23:59:08.928 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:08.928 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.929 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.929 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:08.929 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083647+0000", 2024-09-14T23:59:08.929 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:08.929 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.929 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.929 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:08.929 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083647+0000", 2024-09-14T23:59:08.929 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3099999999999998e-07 2024-09-14T23:59:08.929 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.929 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.929 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:08.929 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083650+0000", 2024-09-14T23:59:08.929 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2900000000000001e-06 2024-09-14T23:59:08.930 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.930 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.930 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:08.930 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083657+0000", 2024-09-14T23:59:08.930 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.486e-06 2024-09-14T23:59:08.930 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.930 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.930 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:08.930 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083663+0000", 2024-09-14T23:59:08.930 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4330000000000003e-06 2024-09-14T23:59:08.930 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.930 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.930 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:08.931 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142206+0000", 2024-09-14T23:59:08.931 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058543237999999997 2024-09-14T23:59:08.931 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.931 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.931 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:08.931 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142284+0000", 2024-09-14T23:59:08.931 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8487999999999998e-05 2024-09-14T23:59:08.931 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.931 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.931 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:08.931 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142353+0000", 2024-09-14T23:59:08.931 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9023000000000006e-05 2024-09-14T23:59:08.931 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:08.931 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:08.932 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:08.932 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.932 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.932 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184348 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4103168~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:59:08.932 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.083669+0000", 2024-09-14T23:59:08.932 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 61.814181091000002, 2024-09-14T23:59:08.932 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058864232000000002, 2024-09-14T23:59:08.932 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:08.932 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:08.932 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:08.932 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:08.932 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:08.932 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:08.933 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184348 2024-09-14T23:59:08.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.933 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:08.933 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.933 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:08.933 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083669+0000", 2024-09-14T23:59:08.933 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:08.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.933 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.933 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:08.933 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083669+0000", 2024-09-14T23:59:08.933 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:08.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.933 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.934 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:08.934 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083670+0000", 2024-09-14T23:59:08.934 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9999999999999998e-07 2024-09-14T23:59:08.934 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.934 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.934 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:08.934 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083672+0000", 2024-09-14T23:59:08.934 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1430000000000001e-06 2024-09-14T23:59:08.934 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.934 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.934 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:08.934 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083678+0000", 2024-09-14T23:59:08.934 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5200000000000003e-06 2024-09-14T23:59:08.934 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.935 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.935 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:08.935 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083683+0000", 2024-09-14T23:59:08.935 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9359999999999998e-06 2024-09-14T23:59:08.935 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.935 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.935 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:08.935 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142384+0000", 2024-09-14T23:59:08.935 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058701159000000003 2024-09-14T23:59:08.935 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.935 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.935 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:08.935 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142466+0000", 2024-09-14T23:59:08.935 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1024000000000002e-05 2024-09-14T23:59:08.936 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.936 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.936 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:08.936 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142534+0000", 2024-09-14T23:59:08.936 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8050000000000001e-05 2024-09-14T23:59:08.936 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:08.936 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:08.936 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:08.936 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.936 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.936 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184349 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4126720~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:59:08.936 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.083690+0000", 2024-09-14T23:59:08.936 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 61.814160555999997, 2024-09-14T23:59:08.936 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059019911000000001, 2024-09-14T23:59:08.937 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:08.937 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:08.937 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:08.937 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:08.937 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:08.937 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:08.937 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184349 2024-09-14T23:59:08.937 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.937 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:08.937 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.937 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:08.937 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083690+0000", 2024-09-14T23:59:08.937 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:08.938 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.938 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.938 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:08.938 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083690+0000", 2024-09-14T23:59:08.938 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:08.938 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.938 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.938 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:08.938 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.084900+0000", 2024-09-14T23:59:08.938 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012099770000000001 2024-09-14T23:59:08.938 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.938 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.938 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:08.938 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.084905+0000", 2024-09-14T23:59:08.939 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.028e-06 2024-09-14T23:59:08.939 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.939 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.939 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:08.939 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.084914+0000", 2024-09-14T23:59:08.939 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6619999999999994e-06 2024-09-14T23:59:08.939 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.939 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.939 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:08.939 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.084922+0000", 2024-09-14T23:59:08.939 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5720000000000003e-06 2024-09-14T23:59:08.939 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.939 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.939 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:08.940 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142565+0000", 2024-09-14T23:59:08.940 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057642605999999999 2024-09-14T23:59:08.940 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.940 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.940 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:08.940 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142644+0000", 2024-09-14T23:59:08.940 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9370999999999996e-05 2024-09-14T23:59:08.940 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.940 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.940 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:08.940 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142710+0000", 2024-09-14T23:59:08.940 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5695e-05 2024-09-14T23:59:08.940 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:08.940 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:08.941 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:08.941 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.941 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.941 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184354 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4115456~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:59:08.941 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.085029+0000", 2024-09-14T23:59:08.941 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 61.812821906000003, 2024-09-14T23:59:08.941 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058626327999999998, 2024-09-14T23:59:08.941 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:08.941 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:08.941 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:08.941 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:08.941 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:08.941 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:08.941 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184354 2024-09-14T23:59:08.942 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.942 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:08.942 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.942 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:08.942 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085029+0000", 2024-09-14T23:59:08.942 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:08.942 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.942 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.942 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:08.942 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085029+0000", 2024-09-14T23:59:08.942 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:08.942 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.942 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.942 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:08.943 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085029+0000", 2024-09-14T23:59:08.943 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0800000000000001e-07 2024-09-14T23:59:08.943 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.943 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.943 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:08.943 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085030+0000", 2024-09-14T23:59:08.943 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.226e-06 2024-09-14T23:59:08.943 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.943 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.943 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:08.943 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085033+0000", 2024-09-14T23:59:08.943 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9569999999999998e-06 2024-09-14T23:59:08.943 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.943 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.944 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:08.944 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085035+0000", 2024-09-14T23:59:08.944 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.587e-06 2024-09-14T23:59:08.944 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.944 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.944 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:08.944 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143491+0000", 2024-09-14T23:59:08.944 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058455337000000003 2024-09-14T23:59:08.944 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.944 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.944 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:08.944 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143575+0000", 2024-09-14T23:59:08.944 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3850000000000005e-05 2024-09-14T23:59:08.945 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.945 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.945 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:08.945 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143655+0000", 2024-09-14T23:59:08.945 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0162999999999998e-05 2024-09-14T23:59:08.945 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:08.945 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:08.945 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:08.945 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.945 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.945 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184355 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4139008~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:59:08.945 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.085039+0000", 2024-09-14T23:59:08.945 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 61.812811502000002, 2024-09-14T23:59:08.946 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058801369999999999, 2024-09-14T23:59:08.946 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:08.946 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:08.946 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:08.946 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:08.946 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:08.946 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:08.946 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184355 2024-09-14T23:59:08.946 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.946 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:08.946 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.946 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:08.946 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085039+0000", 2024-09-14T23:59:08.946 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:08.946 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.947 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.947 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:08.947 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085039+0000", 2024-09-14T23:59:08.947 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:08.947 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.947 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.947 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:08.947 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085039+0000", 2024-09-14T23:59:08.947 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.03e-07 2024-09-14T23:59:08.947 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.947 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.947 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:08.947 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085040+0000", 2024-09-14T23:59:08.947 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.158e-06 2024-09-14T23:59:08.948 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.948 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.948 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:08.948 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085043+0000", 2024-09-14T23:59:08.948 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0240000000000002e-06 2024-09-14T23:59:08.948 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.948 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.948 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:08.948 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085046+0000", 2024-09-14T23:59:08.948 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4600000000000002e-06 2024-09-14T23:59:08.948 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.948 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.948 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:08.949 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143688+0000", 2024-09-14T23:59:08.949 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058642356 2024-09-14T23:59:08.949 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.949 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.949 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:08.949 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143766+0000", 2024-09-14T23:59:08.949 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8219000000000004e-05 2024-09-14T23:59:08.949 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.949 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.949 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:08.949 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143840+0000", 2024-09-14T23:59:08.949 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3949999999999995e-05 2024-09-14T23:59:08.949 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:08.949 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:08.950 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:08.950 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.950 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.950 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184356 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4119552~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:59:08.950 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.085049+0000", 2024-09-14T23:59:08.950 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 61.812801331000003, 2024-09-14T23:59:08.950 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058977992, 2024-09-14T23:59:08.950 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:08.950 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:08.950 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:08.950 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:08.950 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:08.950 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:08.950 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184356 2024-09-14T23:59:08.950 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.951 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:08.951 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.951 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:08.951 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085049+0000", 2024-09-14T23:59:08.951 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:08.951 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.951 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.951 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:08.951 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085049+0000", 2024-09-14T23:59:08.951 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:08.951 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.951 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.951 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:08.951 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.086934+0000", 2024-09-14T23:59:08.952 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018854189999999999 2024-09-14T23:59:08.952 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.952 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.952 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:08.952 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.086939+0000", 2024-09-14T23:59:08.952 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.493e-06 2024-09-14T23:59:08.952 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.952 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.952 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:08.952 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.086951+0000", 2024-09-14T23:59:08.952 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1969e-05 2024-09-14T23:59:08.952 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.952 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.953 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:08.953 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.086959+0000", 2024-09-14T23:59:08.953 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5780000000000001e-06 2024-09-14T23:59:08.953 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.953 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.953 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:08.953 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143871+0000", 2024-09-14T23:59:08.953 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056912114 2024-09-14T23:59:08.953 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.953 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.953 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:08.953 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143951+0000", 2024-09-14T23:59:08.953 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9938000000000006e-05 2024-09-14T23:59:08.954 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.954 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.954 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:08.954 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.144027+0000", 2024-09-14T23:59:08.954 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6481000000000002e-05 2024-09-14T23:59:08.954 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:08.954 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:08.954 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:08.954 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.954 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.954 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184364 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4135936~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:59:08.954 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.087207+0000", 2024-09-14T23:59:08.954 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 61.810643051, 2024-09-14T23:59:08.954 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058466420999999998, 2024-09-14T23:59:08.955 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:08.955 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:08.955 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:08.955 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:08.955 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:08.955 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:08.955 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184364 2024-09-14T23:59:08.955 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.955 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:08.955 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.955 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:08.955 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087207+0000", 2024-09-14T23:59:08.955 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:08.955 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.956 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.956 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:08.956 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087207+0000", 2024-09-14T23:59:08.956 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:08.956 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.956 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.956 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:08.956 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087208+0000", 2024-09-14T23:59:08.956 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.35e-07 2024-09-14T23:59:08.956 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.956 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.956 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:08.956 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087209+0000", 2024-09-14T23:59:08.956 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.573e-06 2024-09-14T23:59:08.957 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.957 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.957 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:08.957 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087212+0000", 2024-09-14T23:59:08.957 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2770000000000001e-06 2024-09-14T23:59:08.957 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.957 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.957 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:08.957 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087215+0000", 2024-09-14T23:59:08.957 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5500000000000001e-06 2024-09-14T23:59:08.957 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.957 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.957 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:08.958 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145520+0000", 2024-09-14T23:59:08.958 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058304835999999999 2024-09-14T23:59:08.958 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.958 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.958 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:08.958 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145602+0000", 2024-09-14T23:59:08.958 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1714999999999999e-05 2024-09-14T23:59:08.958 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.958 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.958 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:08.958 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145674+0000", 2024-09-14T23:59:08.958 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2235000000000002e-05 2024-09-14T23:59:08.958 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:08.958 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:08.959 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:08.959 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.959 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.959 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184365 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4159488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:59:08.959 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.087219+0000", 2024-09-14T23:59:08.959 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 61.810631020000002, 2024-09-14T23:59:08.959 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058643533999999997, 2024-09-14T23:59:08.959 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:08.959 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:08.959 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:08.959 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:08.959 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:08.959 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:08.959 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184365 2024-09-14T23:59:08.960 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.960 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:08.960 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.960 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:08.960 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087219+0000", 2024-09-14T23:59:08.960 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:08.960 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.960 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.960 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:08.960 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087219+0000", 2024-09-14T23:59:08.960 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:08.960 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.960 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.961 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:08.961 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087220+0000", 2024-09-14T23:59:08.961 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.39e-07 2024-09-14T23:59:08.961 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.961 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.961 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:08.961 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087222+0000", 2024-09-14T23:59:08.961 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0609999999999998e-06 2024-09-14T23:59:08.961 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.961 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.961 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:08.961 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087228+0000", 2024-09-14T23:59:08.961 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8590000000000001e-06 2024-09-14T23:59:08.962 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.962 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.962 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:08.962 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087232+0000", 2024-09-14T23:59:08.962 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8149999999999996e-06 2024-09-14T23:59:08.962 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.962 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.962 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:08.962 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145705+0000", 2024-09-14T23:59:08.962 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058472532000000001 2024-09-14T23:59:08.962 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.962 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.962 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:08.963 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145784+0000", 2024-09-14T23:59:08.963 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9382000000000007e-05 2024-09-14T23:59:08.963 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.963 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.963 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:08.963 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145863+0000", 2024-09-14T23:59:08.963 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8546000000000003e-05 2024-09-14T23:59:08.963 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:08.963 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:08.963 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:08.963 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.963 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.963 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184366 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4140032~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:59:08.963 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.087240+0000", 2024-09-14T23:59:08.964 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 61.810610572000002, 2024-09-14T23:59:08.964 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058818068000000001, 2024-09-14T23:59:08.964 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:08.964 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:08.964 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:08.964 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:08.964 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:08.964 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:08.964 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184366 2024-09-14T23:59:08.964 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.964 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:08.964 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.964 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:08.965 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087240+0000", 2024-09-14T23:59:08.965 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:08.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.965 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:08.965 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087240+0000", 2024-09-14T23:59:08.965 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:08.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.965 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:08.965 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.088904+0000", 2024-09-14T23:59:08.965 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001664315 2024-09-14T23:59:08.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.966 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:08.966 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.088909+0000", 2024-09-14T23:59:08.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7709999999999997e-06 2024-09-14T23:59:08.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.966 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:08.966 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.088921+0000", 2024-09-14T23:59:08.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1661999999999999e-05 2024-09-14T23:59:08.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.966 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:08.966 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.088928+0000", 2024-09-14T23:59:08.966 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8189999999999992e-06 2024-09-14T23:59:08.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.967 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:08.967 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145901+0000", 2024-09-14T23:59:08.967 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056972478 2024-09-14T23:59:08.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.967 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:08.967 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145977+0000", 2024-09-14T23:59:08.967 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5763e-05 2024-09-14T23:59:08.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.967 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:08.967 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.146058+0000", 2024-09-14T23:59:08.967 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1260000000000005e-05 2024-09-14T23:59:08.968 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:08.968 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:08.968 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:08.968 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.968 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.968 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184375 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4179968~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:59:08.968 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.089199+0000", 2024-09-14T23:59:08.968 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 61.808651234999999, 2024-09-14T23:59:08.968 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058597445999999997, 2024-09-14T23:59:08.968 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:08.968 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:08.968 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:08.968 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:08.969 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:08.969 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:08.969 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184375 2024-09-14T23:59:08.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.969 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:08.969 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.969 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:08.969 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089199+0000", 2024-09-14T23:59:08.969 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:08.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.969 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.969 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:08.969 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089199+0000", 2024-09-14T23:59:08.969 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:08.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.970 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.970 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:08.970 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089200+0000", 2024-09-14T23:59:08.970 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8100000000000003e-07 2024-09-14T23:59:08.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.970 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.970 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:08.970 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089202+0000", 2024-09-14T23:59:08.970 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7599999999999998e-06 2024-09-14T23:59:08.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.970 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.970 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:08.970 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089209+0000", 2024-09-14T23:59:08.971 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7279999999999998e-06 2024-09-14T23:59:08.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.971 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:08.971 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089215+0000", 2024-09-14T23:59:08.971 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5090000000000001e-06 2024-09-14T23:59:08.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.971 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:08.971 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147638+0000", 2024-09-14T23:59:08.971 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058423105000000003 2024-09-14T23:59:08.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.971 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:08.972 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147719+0000", 2024-09-14T23:59:08.972 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1217000000000005e-05 2024-09-14T23:59:08.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.972 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.972 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:08.972 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147797+0000", 2024-09-14T23:59:08.972 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7645999999999995e-05 2024-09-14T23:59:08.972 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:08.972 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:08.972 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:08.973 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.973 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.973 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184376 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4160512~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:59:08.973 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.089232+0000", 2024-09-14T23:59:08.973 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 61.808618744999997, 2024-09-14T23:59:08.973 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058751338, 2024-09-14T23:59:08.973 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:08.973 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:08.973 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:08.973 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:08.973 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:08.973 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:08.974 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184376 2024-09-14T23:59:08.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.974 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:08.974 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.974 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:08.974 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089232+0000", 2024-09-14T23:59:08.974 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:08.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.974 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.974 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:08.974 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089232+0000", 2024-09-14T23:59:08.974 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:08.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.975 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.975 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:08.975 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089232+0000", 2024-09-14T23:59:08.975 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5600000000000001e-07 2024-09-14T23:59:08.975 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.975 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.975 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:08.975 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089235+0000", 2024-09-14T23:59:08.975 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5639999999999999e-06 2024-09-14T23:59:08.975 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.975 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.975 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:08.975 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089242+0000", 2024-09-14T23:59:08.975 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0369999999999997e-06 2024-09-14T23:59:08.976 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.976 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.976 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:08.976 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089247+0000", 2024-09-14T23:59:08.976 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2079999999999999e-06 2024-09-14T23:59:08.976 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.976 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.976 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:08.976 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147831+0000", 2024-09-14T23:59:08.976 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058583853999999998 2024-09-14T23:59:08.976 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.976 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.976 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:08.977 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147915+0000", 2024-09-14T23:59:08.977 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4208000000000004e-05 2024-09-14T23:59:08.977 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.977 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.977 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:08.977 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147983+0000", 2024-09-14T23:59:08.977 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8010999999999996e-05 2024-09-14T23:59:08.977 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:08.977 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:08.977 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:08.977 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.977 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.977 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184377 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4184064~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:59:08.977 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.089255+0000", 2024-09-14T23:59:08.978 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 61.808595189000002, 2024-09-14T23:59:08.978 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058906100000000003, 2024-09-14T23:59:08.978 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:08.978 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:08.978 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:08.978 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:08.978 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:08.978 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:08.978 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184377 2024-09-14T23:59:08.978 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.978 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:08.978 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.978 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:08.978 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089255+0000", 2024-09-14T23:59:08.979 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:08.979 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.979 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.979 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:08.979 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089255+0000", 2024-09-14T23:59:08.979 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:08.979 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.979 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.979 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:08.979 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.090903+0000", 2024-09-14T23:59:08.979 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001647871 2024-09-14T23:59:08.979 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.979 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.979 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:08.980 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.090908+0000", 2024-09-14T23:59:08.980 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7199999999999997e-06 2024-09-14T23:59:08.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.980 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.980 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:08.980 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.090919+0000", 2024-09-14T23:59:08.980 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1472e-05 2024-09-14T23:59:08.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.980 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.980 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:08.980 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.090927+0000", 2024-09-14T23:59:08.980 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0390000000000005e-06 2024-09-14T23:59:08.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.980 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.981 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:08.981 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.148013+0000", 2024-09-14T23:59:08.981 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057085662000000002 2024-09-14T23:59:08.981 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.981 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.981 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:08.981 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.148090+0000", 2024-09-14T23:59:08.981 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6778000000000002e-05 2024-09-14T23:59:08.981 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.981 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.981 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:08.981 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.148161+0000", 2024-09-14T23:59:08.981 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1557999999999995e-05 2024-09-14T23:59:08.982 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:08.982 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:08.982 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:08.982 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.982 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.982 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184386 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4185088~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:59:08.982 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.091063+0000", 2024-09-14T23:59:08.982 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 61.806787524999997, 2024-09-14T23:59:08.982 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058603404999999997, 2024-09-14T23:59:08.982 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:08.982 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:08.982 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:08.982 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:08.982 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:08.983 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:08.983 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184386 2024-09-14T23:59:08.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.983 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:08.983 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.983 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:08.983 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091063+0000", 2024-09-14T23:59:08.983 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:08.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.983 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.983 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:08.983 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091063+0000", 2024-09-14T23:59:08.983 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:08.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.984 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.984 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:08.984 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091063+0000", 2024-09-14T23:59:08.984 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1899999999999999e-07 2024-09-14T23:59:08.984 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.984 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.984 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:08.984 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091064+0000", 2024-09-14T23:59:08.984 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2699999999999999e-06 2024-09-14T23:59:08.984 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.984 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.984 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:08.984 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091067+0000", 2024-09-14T23:59:08.985 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.971e-06 2024-09-14T23:59:08.985 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.985 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.985 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:08.985 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091070+0000", 2024-09-14T23:59:08.985 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.525e-06 2024-09-14T23:59:08.985 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.985 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.985 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:08.985 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.149510+0000", 2024-09-14T23:59:08.985 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058439758000000001 2024-09-14T23:59:08.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.986 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.986 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:08.986 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.149595+0000", 2024-09-14T23:59:08.986 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5146000000000001e-05 2024-09-14T23:59:08.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.986 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.986 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:08.986 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.149666+0000", 2024-09-14T23:59:08.986 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1515999999999998e-05 2024-09-14T23:59:08.986 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:08.986 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:08.986 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:08.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.987 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.987 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184388 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4189184~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:59:08.987 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.091073+0000", 2024-09-14T23:59:08.987 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 61.806777250000003, 2024-09-14T23:59:08.987 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058781609999999998, 2024-09-14T23:59:08.987 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:08.987 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:08.987 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:08.987 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:08.987 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:08.987 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:08.987 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184388 2024-09-14T23:59:08.988 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.988 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:08.988 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.988 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:08.988 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091073+0000", 2024-09-14T23:59:08.988 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:08.988 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.988 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.988 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:08.988 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091073+0000", 2024-09-14T23:59:08.988 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:08.988 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.988 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.988 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:08.988 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.092903+0000", 2024-09-14T23:59:08.989 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018296289999999999 2024-09-14T23:59:08.989 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.989 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.989 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:08.989 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.092910+0000", 2024-09-14T23:59:08.989 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7510000000000002e-06 2024-09-14T23:59:08.989 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.989 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.989 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:08.989 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.092918+0000", 2024-09-14T23:59:08.989 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7420000000000003e-06 2024-09-14T23:59:08.989 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.989 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.989 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:08.990 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.092925+0000", 2024-09-14T23:59:08.990 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0360000000000001e-06 2024-09-14T23:59:08.990 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.990 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.990 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:08.990 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.149700+0000", 2024-09-14T23:59:08.990 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056774340999999999 2024-09-14T23:59:08.990 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.990 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.990 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:08.990 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.149784+0000", 2024-09-14T23:59:08.990 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4393000000000003e-05 2024-09-14T23:59:08.990 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.990 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.991 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:08.991 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.149855+0000", 2024-09-14T23:59:08.991 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0717999999999996e-05 2024-09-14T23:59:08.991 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:08.991 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:08.991 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:08.991 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.991 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.991 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4172.0:8 3.12 3:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-14T23:59:08.991 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.976999+0000", 2024-09-14T23:59:08.991 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 60.920851841000001, 2024-09-14T23:59:08.991 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0046036439999999, 2024-09-14T23:59:08.992 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:08.992 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:08.992 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:08.992 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:08.992 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4172", 2024-09-14T23:59:08.992 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/2542269203", 2024-09-14T23:59:08.992 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 8 2024-09-14T23:59:08.992 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.992 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:08.992 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.992 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:08.992 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.976999+0000", 2024-09-14T23:59:08.992 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:08.992 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.993 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.993 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:08.993 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.976999+0000", 2024-09-14T23:59:08.993 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:08.993 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.993 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.993 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:08.993 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.977000+0000", 2024-09-14T23:59:08.993 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5200000000000001e-06 2024-09-14T23:59:08.993 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.993 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.993 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:08.993 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.977005+0000", 2024-09-14T23:59:08.994 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.04e-06 2024-09-14T23:59:08.994 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.995 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.995 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:08.995 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.977013+0000", 2024-09-14T23:59:08.995 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2889999999999998e-06 2024-09-14T23:59:08.995 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.995 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.995 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:08.995 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.977024+0000", 2024-09-14T23:59:08.996 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0307e-05 2024-09-14T23:59:08.996 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.996 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.996 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:08.996 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.977086+0000", 2024-09-14T23:59:08.996 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2248000000000004e-05 2024-09-14T23:59:08.996 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.996 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.996 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:59:08.996 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.977097+0000", 2024-09-14T23:59:08.996 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1123e-05 2024-09-14T23:59:08.996 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.996 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.996 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:08.997 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:08.981470+0000", 2024-09-14T23:59:08.997 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.004373116 2024-09-14T23:59:08.997 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:08.997 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:08.997 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:08.997 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:08.981602+0000", 2024-09-14T23:59:08.997 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000132001 2024-09-14T23:59:08.997 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:08.997 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:08.997 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:08.997 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:08.997 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:08.997 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:59:08.999 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T23:59:08.999 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:59:08.997+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:59:09.099 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:59:09.098+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:59:09.181 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:59:09.181 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:59:09.181 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:59:09.181 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:59:09.191 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T23:59:09.233 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:59:09.231+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:59:09.301 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:59:09.300+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:59:09.349 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:59:09.349 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:59:09.349 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:59:09.349 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:59:09.349 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:59:09.359 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T23:59:09.402 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:59:09.401+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:59:09.503 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:59:09.501+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:59:09.519 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:59:09.519 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:59:09.519 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:59:09.519 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:59:09.519 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.519 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5882 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:09.520 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:59:09.520 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 150.14104845099999, 2024-09-14T23:59:09.520 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050549666, 2024-09-14T23:59:09.520 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:09.520 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:09.520 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:09.520 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:09.520 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:09.520 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:09.521 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5882 2024-09-14T23:59:09.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.521 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:09.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.521 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:09.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:59:09.521 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:09.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.521 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:09.521 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:59:09.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:09.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:09.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:59:09.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8099999999999998e-07 2024-09-14T23:59:09.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:09.522 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375842+0000", 2024-09-14T23:59:09.522 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9900000000000009e-07 2024-09-14T23:59:09.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.522 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:09.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375845+0000", 2024-09-14T23:59:09.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.055e-06 2024-09-14T23:59:09.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:09.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375848+0000", 2024-09-14T23:59:09.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7609999999999999e-06 2024-09-14T23:59:09.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.523 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:09.523 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426381+0000", 2024-09-14T23:59:09.523 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050533392000000003 2024-09-14T23:59:09.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.523 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.524 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:09.524 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426390+0000", 2024-09-14T23:59:09.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0780000000000002e-06 2024-09-14T23:59:09.524 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:09.524 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:09.524 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:09.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.524 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.524 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5884 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:09.524 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:59:09.524 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 150.14103831099999, 2024-09-14T23:59:09.524 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050717439000000003, 2024-09-14T23:59:09.524 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:09.524 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:09.525 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:09.525 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:09.525 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:09.525 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:09.525 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5884 2024-09-14T23:59:09.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.529 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:09.529 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.529 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:09.529 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:59:09.529 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:09.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:09.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:59:09.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:09.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:09.530 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:59:09.530 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0800000000000001e-07 2024-09-14T23:59:09.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.530 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.530 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:09.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375852+0000", 2024-09-14T23:59:09.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.02e-06 2024-09-14T23:59:09.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:09.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375855+0000", 2024-09-14T23:59:09.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9900000000000002e-06 2024-09-14T23:59:09.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:09.531 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375857+0000", 2024-09-14T23:59:09.531 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3499999999999999e-06 2024-09-14T23:59:09.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.531 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.531 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:09.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426556+0000", 2024-09-14T23:59:09.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050698139000000003 2024-09-14T23:59:09.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.532 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:09.532 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426568+0000", 2024-09-14T23:59:09.532 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2731999999999999e-05 2024-09-14T23:59:09.532 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:09.532 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:09.532 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:09.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.532 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5885 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:09.533 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:59:09.533 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 150.14101928599999, 2024-09-14T23:59:09.533 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050737333000000003, 2024-09-14T23:59:09.533 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:09.533 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:09.533 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:09.533 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:09.533 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:09.533 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:09.533 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5885 2024-09-14T23:59:09.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.533 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:09.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.533 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:09.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:59:09.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:09.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:09.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:59:09.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:09.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:09.534 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:59:09.534 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5899999999999998e-07 2024-09-14T23:59:09.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.534 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:09.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375872+0000", 2024-09-14T23:59:09.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4219999999999999e-06 2024-09-14T23:59:09.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:09.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375878+0000", 2024-09-14T23:59:09.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4980000000000003e-06 2024-09-14T23:59:09.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:09.535 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375881+0000", 2024-09-14T23:59:09.535 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6110000000000001e-06 2024-09-14T23:59:09.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.535 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.535 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:09.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426596+0000", 2024-09-14T23:59:09.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050715320000000001 2024-09-14T23:59:09.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.536 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:09.536 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426607+0000", 2024-09-14T23:59:09.536 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1223000000000001e-05 2024-09-14T23:59:09.536 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:09.536 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:09.536 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:09.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.536 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5886 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:09.536 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:59:09.537 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 150.141005173, 2024-09-14T23:59:09.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050803395000000001, 2024-09-14T23:59:09.537 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:09.537 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:09.537 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:09.537 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:09.537 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:09.537 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:09.537 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5886 2024-09-14T23:59:09.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.537 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:09.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.537 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:09.537 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:59:09.537 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:09.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:09.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:59:09.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:09.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:09.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:59:09.538 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2700000000000001e-07 2024-09-14T23:59:09.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.538 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:09.538 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375886+0000", 2024-09-14T23:59:09.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7060000000000001e-06 2024-09-14T23:59:09.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:09.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375890+0000", 2024-09-14T23:59:09.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2259999999999999e-06 2024-09-14T23:59:09.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:09.539 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375893+0000", 2024-09-14T23:59:09.539 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0819999999999999e-06 2024-09-14T23:59:09.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.539 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.539 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:09.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426677+0000", 2024-09-14T23:59:09.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050784158000000003 2024-09-14T23:59:09.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.540 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:09.540 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426687+0000", 2024-09-14T23:59:09.540 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9960000000000005e-06 2024-09-14T23:59:09.540 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:09.540 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:09.540 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:09.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.540 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.540 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5887 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:09.541 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:59:09.541 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 150.140983087, 2024-09-14T23:59:09.541 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050833070000000001, 2024-09-14T23:59:09.541 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:09.541 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:09.541 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:09.541 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:09.541 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:09.541 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:09.541 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5887 2024-09-14T23:59:09.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.541 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:09.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.541 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:09.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:59:09.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:09.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:09.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:59:09.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:09.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.542 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:09.542 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:59:09.542 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3299999999999998e-07 2024-09-14T23:59:09.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:09.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375908+0000", 2024-09-14T23:59:09.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7689999999999999e-06 2024-09-14T23:59:09.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:09.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375912+0000", 2024-09-14T23:59:09.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.535e-06 2024-09-14T23:59:09.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.543 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.543 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:09.543 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375916+0000", 2024-09-14T23:59:09.543 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8979999999999998e-06 2024-09-14T23:59:09.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:09.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426729+0000", 2024-09-14T23:59:09.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050813453000000001 2024-09-14T23:59:09.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.544 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.544 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:09.544 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426739+0000", 2024-09-14T23:59:09.544 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0081999999999999e-05 2024-09-14T23:59:09.544 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:09.544 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:09.544 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:09.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.545 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5888 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:09.545 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:59:09.545 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 150.14097000999999, 2024-09-14T23:59:09.545 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050854652, 2024-09-14T23:59:09.545 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:09.545 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:09.545 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:09.545 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:09.545 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:09.545 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:09.545 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5888 2024-09-14T23:59:09.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.546 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:09.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:09.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:59:09.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:09.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.546 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.546 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:09.546 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:59:09.546 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:09.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:09.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:59:09.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1400000000000001e-07 2024-09-14T23:59:09.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:09.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375921+0000", 2024-09-14T23:59:09.547 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.119e-06 2024-09-14T23:59:09.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.547 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.547 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:09.547 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375928+0000", 2024-09-14T23:59:09.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3170000000000004e-06 2024-09-14T23:59:09.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:09.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375931+0000", 2024-09-14T23:59:09.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6970000000000002e-06 2024-09-14T23:59:09.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.548 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:09.548 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426768+0000", 2024-09-14T23:59:09.548 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050837355000000001 2024-09-14T23:59:09.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.548 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.549 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:09.549 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426774+0000", 2024-09-14T23:59:09.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9499999999999998e-06 2024-09-14T23:59:09.549 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:09.549 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:09.549 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:09.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.549 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.549 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5889 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:09.549 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:59:09.549 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 150.140953086, 2024-09-14T23:59:09.549 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050885446000000001, 2024-09-14T23:59:09.549 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:09.549 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:09.550 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:09.550 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:09.550 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:09.550 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:09.550 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5889 2024-09-14T23:59:09.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.550 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:09.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:09.550 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:59:09.550 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:09.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.550 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:09.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:59:09.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:09.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:09.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:59:09.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.36e-07 2024-09-14T23:59:09.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.551 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:09.551 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375940+0000", 2024-09-14T23:59:09.551 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.844e-06 2024-09-14T23:59:09.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.551 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:09.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375944+0000", 2024-09-14T23:59:09.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6720000000000002e-06 2024-09-14T23:59:09.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:09.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375946+0000", 2024-09-14T23:59:09.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5629999999999999e-06 2024-09-14T23:59:09.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.552 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.552 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:09.552 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426810+0000", 2024-09-14T23:59:09.552 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050863255000000003 2024-09-14T23:59:09.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.553 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:09.553 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426822+0000", 2024-09-14T23:59:09.553 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1875999999999999e-05 2024-09-14T23:59:09.553 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:09.553 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:09.553 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:09.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.553 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5890 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:09.553 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:59:09.554 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 150.140939274, 2024-09-14T23:59:09.554 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050917549999999999, 2024-09-14T23:59:09.554 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:09.554 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:09.554 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:09.554 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:09.554 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:09.554 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:09.554 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5890 2024-09-14T23:59:09.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.554 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:09.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.554 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:09.554 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:59:09.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:09.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:09.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:59:09.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:09.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.555 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:09.555 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:59:09.555 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4299999999999999e-07 2024-09-14T23:59:09.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:09.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375951+0000", 2024-09-14T23:59:09.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1000000000000001e-06 2024-09-14T23:59:09.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:09.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375956+0000", 2024-09-14T23:59:09.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3370000000000001e-06 2024-09-14T23:59:09.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.556 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.556 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:09.556 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375958+0000", 2024-09-14T23:59:09.556 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4820000000000001e-06 2024-09-14T23:59:09.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:09.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426858+0000", 2024-09-14T23:59:09.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050899666000000003 2024-09-14T23:59:09.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.557 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:09.557 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426867+0000", 2024-09-14T23:59:09.557 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7219999999999994e-06 2024-09-14T23:59:09.557 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:09.557 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:09.557 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:09.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.558 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.558 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5891 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:09.558 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:59:09.558 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 150.140927952, 2024-09-14T23:59:09.558 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050948067, 2024-09-14T23:59:09.558 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:09.558 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:09.558 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:09.558 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:09.558 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:09.558 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:09.558 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5891 2024-09-14T23:59:09.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.558 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:09.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:09.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:59:09.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:09.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:09.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:59:09.559 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:09.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.559 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.559 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:09.559 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:59:09.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0800000000000001e-07 2024-09-14T23:59:09.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:09.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375962+0000", 2024-09-14T23:59:09.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9600000000000008e-07 2024-09-14T23:59:09.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.560 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:09.560 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375970+0000", 2024-09-14T23:59:09.560 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1600000000000001e-06 2024-09-14T23:59:09.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:09.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375972+0000", 2024-09-14T23:59:09.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4849999999999999e-06 2024-09-14T23:59:09.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:09.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426902+0000", 2024-09-14T23:59:09.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050929813999999997 2024-09-14T23:59:09.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.561 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.561 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:09.561 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426909+0000", 2024-09-14T23:59:09.561 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4039999999999998e-06 2024-09-14T23:59:09.562 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:09.562 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:09.562 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:09.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.562 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.562 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5892 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:09.562 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:59:09.562 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 150.140913095, 2024-09-14T23:59:09.562 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050965036999999998, 2024-09-14T23:59:09.562 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:09.562 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:09.562 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:09.562 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:09.563 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:09.563 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:09.563 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5892 2024-09-14T23:59:09.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.563 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:09.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:09.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:59:09.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:09.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.563 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:09.563 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:59:09.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:09.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:09.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:59:09.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.34e-07 2024-09-14T23:59:09.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:09.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375978+0000", 2024-09-14T23:59:09.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1880000000000001e-06 2024-09-14T23:59:09.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:09.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375981+0000", 2024-09-14T23:59:09.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2399999999999999e-06 2024-09-14T23:59:09.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:09.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375984+0000", 2024-09-14T23:59:09.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9110000000000002e-06 2024-09-14T23:59:09.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:09.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426934+0000", 2024-09-14T23:59:09.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050949912 2024-09-14T23:59:09.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:09.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426941+0000", 2024-09-14T23:59:09.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5519999999999999e-06 2024-09-14T23:59:09.566 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:09.566 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:09.566 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:09.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.566 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5893 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:09.566 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:59:09.566 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 150.140902498, 2024-09-14T23:59:09.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050975131999999999, 2024-09-14T23:59:09.567 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:09.567 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:09.567 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:09.567 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:09.567 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:09.567 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:09.567 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5893 2024-09-14T23:59:09.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.567 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:09.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:09.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:59:09.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:09.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:09.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:59:09.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:09.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:09.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:59:09.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2999999999999999e-07 2024-09-14T23:59:09.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:09.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375988+0000", 2024-09-14T23:59:09.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0410000000000001e-06 2024-09-14T23:59:09.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:09.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375992+0000", 2024-09-14T23:59:09.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.104e-06 2024-09-14T23:59:09.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:09.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375995+0000", 2024-09-14T23:59:09.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4310000000000001e-06 2024-09-14T23:59:09.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.570 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:09.570 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426958+0000", 2024-09-14T23:59:09.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050963642000000003 2024-09-14T23:59:09.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:09.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426962+0000", 2024-09-14T23:59:09.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6839999999999998e-06 2024-09-14T23:59:09.571 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:09.571 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:09.571 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:09.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.571 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5894 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:09.572 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:59:09.572 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 150.140891098, 2024-09-14T23:59:09.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050978333000000001, 2024-09-14T23:59:09.572 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:09.572 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:09.572 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:09.572 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:09.572 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:09.572 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:09.572 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5894 2024-09-14T23:59:09.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.572 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:09.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:09.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:59:09.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:09.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:09.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:59:09.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:09.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:09.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:59:09.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.23e-07 2024-09-14T23:59:09.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:09.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375999+0000", 2024-09-14T23:59:09.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9800000000000002e-07 2024-09-14T23:59:09.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:09.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376007+0000", 2024-09-14T23:59:09.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8539999999999997e-06 2024-09-14T23:59:09.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:09.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376011+0000", 2024-09-14T23:59:09.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.399e-06 2024-09-14T23:59:09.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:09.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426973+0000", 2024-09-14T23:59:09.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050962454999999997 2024-09-14T23:59:09.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:09.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426976+0000", 2024-09-14T23:59:09.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4039999999999999e-06 2024-09-14T23:59:09.576 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:09.576 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:09.576 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:09.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.576 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5895 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:09.576 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376014+0000", 2024-09-14T23:59:09.576 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 150.14087475400001, 2024-09-14T23:59:09.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050975951999999998, 2024-09-14T23:59:09.577 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:09.577 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:09.577 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:09.577 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:09.577 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:09.577 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:09.577 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5895 2024-09-14T23:59:09.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.577 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:09.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:09.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376014+0000", 2024-09-14T23:59:09.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:09.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:09.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376014+0000", 2024-09-14T23:59:09.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:09.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:09.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376015+0000", 2024-09-14T23:59:09.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2999999999999999e-07 2024-09-14T23:59:09.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:09.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376016+0000", 2024-09-14T23:59:09.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1659999999999999e-06 2024-09-14T23:59:09.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:09.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376024+0000", 2024-09-14T23:59:09.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8930000000000005e-06 2024-09-14T23:59:09.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.579 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:09.579 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376026+0000", 2024-09-14T23:59:09.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6869999999999999e-06 2024-09-14T23:59:09.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:09.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426987+0000", 2024-09-14T23:59:09.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050960601000000001 2024-09-14T23:59:09.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:09.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426990+0000", 2024-09-14T23:59:09.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3749999999999999e-06 2024-09-14T23:59:09.580 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:09.580 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:09.580 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:09.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.581 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5896 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:09.581 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:59:09.581 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 150.140859017, 2024-09-14T23:59:09.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050975656000000001, 2024-09-14T23:59:09.581 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:09.581 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:09.581 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:09.581 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:09.581 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:09.581 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:09.581 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5896 2024-09-14T23:59:09.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.581 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:09.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:09.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:59:09.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:09.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:09.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:59:09.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:09.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:09.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:59:09.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5100000000000001e-07 2024-09-14T23:59:09.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:09.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376032+0000", 2024-09-14T23:59:09.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.141e-06 2024-09-14T23:59:09.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:09.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376036+0000", 2024-09-14T23:59:09.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.25e-06 2024-09-14T23:59:09.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:09.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376038+0000", 2024-09-14T23:59:09.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5469999999999999e-06 2024-09-14T23:59:09.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:09.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427002+0000", 2024-09-14T23:59:09.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050964029000000001 2024-09-14T23:59:09.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:09.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427006+0000", 2024-09-14T23:59:09.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4379999999999999e-06 2024-09-14T23:59:09.584 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:09.584 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:09.585 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:09.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.585 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5897 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:09.585 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:59:09.585 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 150.14084721699999, 2024-09-14T23:59:09.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050978001000000002, 2024-09-14T23:59:09.585 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:09.585 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:09.585 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:09.585 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:09.585 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:09.585 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:09.585 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5897 2024-09-14T23:59:09.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.586 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:09.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:09.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:59:09.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:09.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:09.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:59:09.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:09.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:09.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:59:09.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2600000000000001e-07 2024-09-14T23:59:09.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:09.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376043+0000", 2024-09-14T23:59:09.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.054e-06 2024-09-14T23:59:09.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:09.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376047+0000", 2024-09-14T23:59:09.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0659999999999997e-06 2024-09-14T23:59:09.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:09.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376050+0000", 2024-09-14T23:59:09.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.88e-06 2024-09-14T23:59:09.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:09.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427017+0000", 2024-09-14T23:59:09.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050966453000000002 2024-09-14T23:59:09.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.588 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:09.588 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427020+0000", 2024-09-14T23:59:09.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3220000000000001e-06 2024-09-14T23:59:09.589 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:09.589 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:09.589 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:09.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.589 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5898 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:09.589 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376053+0000", 2024-09-14T23:59:09.589 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 150.140835732, 2024-09-14T23:59:09.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050982024000000001, 2024-09-14T23:59:09.589 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:09.590 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:09.590 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:09.590 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:09.590 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:09.590 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:09.590 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5898 2024-09-14T23:59:09.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.590 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:09.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:09.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376053+0000", 2024-09-14T23:59:09.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:09.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:09.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376053+0000", 2024-09-14T23:59:09.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:09.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:09.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376278+0000", 2024-09-14T23:59:09.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022426400000000001 2024-09-14T23:59:09.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:09.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376282+0000", 2024-09-14T23:59:09.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4739999999999999e-06 2024-09-14T23:59:09.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:09.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376296+0000", 2024-09-14T23:59:09.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4229999999999999e-05 2024-09-14T23:59:09.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:09.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376303+0000", 2024-09-14T23:59:09.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2199999999999997e-06 2024-09-14T23:59:09.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:09.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427031+0000", 2024-09-14T23:59:09.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050728154999999997 2024-09-14T23:59:09.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.593 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:09.593 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427036+0000", 2024-09-14T23:59:09.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6809999999999997e-06 2024-09-14T23:59:09.593 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:09.594 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:09.594 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:09.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.594 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5899 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:09.594 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376338+0000", 2024-09-14T23:59:09.594 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 150.14055100799999, 2024-09-14T23:59:09.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050712100000000003, 2024-09-14T23:59:09.594 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:09.594 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:09.594 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:09.594 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:09.594 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:09.594 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:09.595 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5899 2024-09-14T23:59:09.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.595 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:09.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:09.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376338+0000", 2024-09-14T23:59:09.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:09.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:09.595 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376338+0000", 2024-09-14T23:59:09.595 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:09.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.595 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:09.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376339+0000", 2024-09-14T23:59:09.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9100000000000003e-07 2024-09-14T23:59:09.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:09.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376342+0000", 2024-09-14T23:59:09.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7e-06 2024-09-14T23:59:09.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:09.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376350+0000", 2024-09-14T23:59:09.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5280000000000003e-06 2024-09-14T23:59:09.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:09.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376356+0000", 2024-09-14T23:59:09.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6010000000000002e-06 2024-09-14T23:59:09.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:09.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427047+0000", 2024-09-14T23:59:09.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050690962999999999 2024-09-14T23:59:09.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:09.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427050+0000", 2024-09-14T23:59:09.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.517e-06 2024-09-14T23:59:09.597 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:09.598 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:09.598 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:09.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.598 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5900 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:09.598 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376366+0000", 2024-09-14T23:59:09.598 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 150.14052274900001, 2024-09-14T23:59:09.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050697858999999998, 2024-09-14T23:59:09.598 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:09.598 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:09.598 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:09.598 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:09.598 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:09.598 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:09.599 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5900 2024-09-14T23:59:09.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.599 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:09.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:09.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376366+0000", 2024-09-14T23:59:09.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:09.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:09.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376366+0000", 2024-09-14T23:59:09.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:09.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:09.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377650+0000", 2024-09-14T23:59:09.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001283834 2024-09-14T23:59:09.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:09.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377655+0000", 2024-09-14T23:59:09.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7230000000000004e-06 2024-09-14T23:59:09.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.600 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:09.600 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377667+0000", 2024-09-14T23:59:09.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1746000000000001e-05 2024-09-14T23:59:09.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:09.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377676+0000", 2024-09-14T23:59:09.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3619999999999995e-06 2024-09-14T23:59:09.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:09.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427061+0000", 2024-09-14T23:59:09.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049384755000000002 2024-09-14T23:59:09.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:09.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427064+0000", 2024-09-14T23:59:09.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.439e-06 2024-09-14T23:59:09.602 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:09.602 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:09.602 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:09.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.602 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:132622 3.1e 3:79d0a5e0:::rbd_data.111847077d5e.0000000000000007:head [sparse-read 0~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T23:59:09.602 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:44.037901+0000", 2024-09-14T23:59:09.602 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 85.478988139999998, 2024-09-14T23:59:09.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1083420859999999, 2024-09-14T23:59:09.602 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:09.602 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:09.602 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:09.602 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:09.602 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:09.603 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:09.603 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 132622 2024-09-14T23:59:09.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.603 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:09.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:09.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037901+0000", 2024-09-14T23:59:09.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:09.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:09.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037901+0000", 2024-09-14T23:59:09.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:09.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:09.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037903+0000", 2024-09-14T23:59:09.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4619999999999999e-06 2024-09-14T23:59:09.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.604 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:09.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037910+0000", 2024-09-14T23:59:09.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7449999999999995e-06 2024-09-14T23:59:09.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:09.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037928+0000", 2024-09-14T23:59:09.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7305000000000001e-05 2024-09-14T23:59:09.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:09.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037939+0000", 2024-09-14T23:59:09.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1599e-05 2024-09-14T23:59:09.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:09.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.038049+0000", 2024-09-14T23:59:09.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000109577 2024-09-14T23:59:09.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op must wait for map", 2024-09-14T23:59:09.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.038057+0000", 2024-09-14T23:59:09.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1450000000000008e-06 2024-09-14T23:59:09.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:09.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.039937+0000", 2024-09-14T23:59:09.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00188024 2024-09-14T23:59:09.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:59:09.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.040039+0000", 2024-09-14T23:59:09.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000101934 2024-09-14T23:59:09.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:09.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:45.145805+0000", 2024-09-14T23:59:09.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.105765651 2024-09-14T23:59:09.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:09.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:45.145997+0000", 2024-09-14T23:59:09.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000192084 2024-09-14T23:59:09.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:09.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:45.146243+0000", 2024-09-14T23:59:09.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024634400000000001 2024-09-14T23:59:09.608 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:09.608 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:09.609 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:09.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.609 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4172.0:5 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:59:09.609 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.482606+0000", 2024-09-14T23:59:09.609 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 62.034283158999997, 2024-09-14T23:59:09.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.49381641300000001, 2024-09-14T23:59:09.609 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:09.609 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:09.609 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:09.609 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:09.610 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4172", 2024-09-14T23:59:09.610 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/2542269203", 2024-09-14T23:59:09.610 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5 2024-09-14T23:59:09.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.610 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:09.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:09.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482606+0000", 2024-09-14T23:59:09.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:09.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:09.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482606+0000", 2024-09-14T23:59:09.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:09.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:09.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482607+0000", 2024-09-14T23:59:09.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3120000000000001e-06 2024-09-14T23:59:09.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:09.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482615+0000", 2024-09-14T23:59:09.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9300000000000003e-06 2024-09-14T23:59:09.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:09.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482632+0000", 2024-09-14T23:59:09.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6966999999999999e-05 2024-09-14T23:59:09.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:09.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482640+0000", 2024-09-14T23:59:09.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1820000000000006e-06 2024-09-14T23:59:09.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:09.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482703+0000", 2024-09-14T23:59:09.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2098000000000001e-05 2024-09-14T23:59:09.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:59:09.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482713+0000", 2024-09-14T23:59:09.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0064000000000001e-05 2024-09-14T23:59:09.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:09.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.976307+0000", 2024-09-14T23:59:09.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.49359442999999997 2024-09-14T23:59:09.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:09.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:09.613 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:09.613 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.976422+0000", 2024-09-14T23:59:09.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011543 2024-09-14T23:59:09.614 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:09.614 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:09.614 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:09.614 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:09.614 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:09.614 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:59:09.614 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T23:59:09.616 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T23:59:09.616 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:59:09.614+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:59:09.717 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:59:09.715+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:59:09.788 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:59:09.788 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:59:09.788 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:59:09.788 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:59:09.798 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T23:59:09.800 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:59:09.800 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:59:09.818 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:59:09.817+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:59:09.824 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:59:09.824 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:59:09.843 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:59:09.843 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:59:09.861 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:59:09.861 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:59:09.879 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:59:09.879 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:59:09.898 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:59:09.898 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:59:09.917 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:59:09.917 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:59:09.918 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:59:09.917+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:59:09.921 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:09.921 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:09.931 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T23:59:09.935 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:59:09.936 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:59:10.005 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T23:59:10.019 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:59:10.018+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:59:10.070 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:10.070 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:10.081 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T23:59:10.120 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:59:10.119+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:59:10.221 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:59:10.220+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:59:10.222 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:10.222 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:10.232 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T23:59:10.322 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:59:10.320+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:59:10.354 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:10.354 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:10.364 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T23:59:10.449 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:59:10.448+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:59:10.524 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:59:10.522+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:59:10.535 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:10.536 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:10.546 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T23:59:10.624 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:59:10.623+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:59:10.664 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:10.664 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:10.673 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T23:59:10.725 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:59:10.724+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:59:10.784 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:10.785 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:10.794 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T23:59:10.835 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:59:10.834+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:59:10.927 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:59:10.926+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:59:10.942 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:10.942 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:10.951 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T23:59:11.029 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:59:11.027+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:59:11.058 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:11.058 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:11.067 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T23:59:11.130 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:59:11.129+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:59:11.131 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is set 2024-09-14T23:59:11.186 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:11.186 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:11.196 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T23:59:11.241 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:59:11.240+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:59:11.306 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:11.306 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:11.315 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T23:59:11.332 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:59:11.330+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:59:11.432 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:59:11.431+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:59:11.463 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:11.463 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:11.472 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T23:59:11.533 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:59:11.531+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:59:11.619 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:11.619 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:11.628 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T23:59:11.673 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:59:11.672+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:59:11.735 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:11.735 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:11.735 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:59:11.733+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:59:11.744 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T23:59:11.835 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:59:11.834+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:59:11.892 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:11.892 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:11.901 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T23:59:11.945 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:59:11.944+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:59:11.948 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T23:59:12.008 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:12.008 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:12.016 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T23:59:12.057 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:59:12.056+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:59:12.086 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:59:12.086 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:59:12.103 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:59:12.104 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:59:12.120 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:59:12.120 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:59:12.138 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:59:12.137+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:59:12.145 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:59:12.145 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:59:12.148 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 6, 5, 7, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T23:59:12.148 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T23:59:12.149 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-14T23:59:12.149 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-14T23:59:12.149 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-14T23:59:12.149 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-09-14T23:59:12.168 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:59:12.168 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:59:12.170 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:12.170 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:12.179 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T23:59:12.185 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:59:12.185 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:59:12.203 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:59:12.203 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:59:12.220 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:59:12.220 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:59:12.239 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:59:12.238+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:59:12.262 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:59:12.262 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-14T23:59:12.262 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:59:12.289 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:59:12.289 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:59:12.289 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:59:12.289 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:59:12.298 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T23:59:12.341 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:59:12.339+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:59:12.441 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:59:12.440+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:59:12.445 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:59:12.445 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:59:12.445 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:59:12.446 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:59:12.446 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:59:12.454 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T23:59:12.542 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:59:12.541+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:59:12.563 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:59:12.563 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:59:12.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:59:12.563 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:59:12.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.563 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:43054 3.1f 3:ffbfe596:::rbd_data.1118350d5ae5.00000000000002eb:head [set-alloc-hint object_size 262144 write_size 262144,write 196608~22770] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T23:59:12.563 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:59:12.563 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 139.59404419200001, 2024-09-14T23:59:12.563 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.99973334400000002, 2024-09-14T23:59:12.563 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:12.563 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:12.564 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:12.564 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:12.564 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:12.564 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:12.564 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 43054 2024-09-14T23:59:12.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.564 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:12.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.564 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:12.564 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:59:12.564 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:12.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:12.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966551+0000", 2024-09-14T23:59:12.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:12.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:12.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966552+0000", 2024-09-14T23:59:12.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3790000000000001e-06 2024-09-14T23:59:12.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.565 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:12.565 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966565+0000", 2024-09-14T23:59:12.565 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2459999999999999e-05 2024-09-14T23:59:12.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:12.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966576+0000", 2024-09-14T23:59:12.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1039000000000001e-05 2024-09-14T23:59:12.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:12.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966582+0000", 2024-09-14T23:59:12.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9660000000000001e-06 2024-09-14T23:59:12.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.566 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.566 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:12.566 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966625+0000", 2024-09-14T23:59:12.566 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3529999999999998e-05 2024-09-14T23:59:12.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:59:12.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:52.966632+0000", 2024-09-14T23:59:12.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9809999999999997e-06 2024-09-14T23:59:12.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:12.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.964460+0000", 2024-09-14T23:59:12.567 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.99782768300000002 2024-09-14T23:59:12.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.567 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:12.567 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.964641+0000", 2024-09-14T23:59:12.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000181089 2024-09-14T23:59:12.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 2", 2024-09-14T23:59:12.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.964809+0000", 2024-09-14T23:59:12.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000168505 2024-09-14T23:59:12.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.568 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:59:12.568 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966030+0000", 2024-09-14T23:59:12.568 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012206929999999999 2024-09-14T23:59:12.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:59:12.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966181+0000", 2024-09-14T23:59:12.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000150569 2024-09-14T23:59:12.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:59:12.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966243+0000", 2024-09-14T23:59:12.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2331999999999999e-05 2024-09-14T23:59:12.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.569 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:12.569 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:53.966284+0000", 2024-09-14T23:59:12.569 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1118000000000002e-05 2024-09-14T23:59:12.569 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:12.570 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:12.570 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:12.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.570 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184314 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4162560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:59:12.570 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.077247+0000", 2024-09-14T23:59:12.570 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.483347523999996, 2024-09-14T23:59:12.570 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058772256000000002, 2024-09-14T23:59:12.570 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:12.570 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:12.570 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:12.570 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:12.570 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:12.570 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:12.571 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184314 2024-09-14T23:59:12.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.571 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:12.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:12.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.077247+0000", 2024-09-14T23:59:12.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:12.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.571 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:12.571 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.077247+0000", 2024-09-14T23:59:12.571 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:12.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.571 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:12.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.078906+0000", 2024-09-14T23:59:12.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016583279999999999 2024-09-14T23:59:12.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:12.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.078911+0000", 2024-09-14T23:59:12.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0420000000000002e-06 2024-09-14T23:59:12.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.572 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.572 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:12.572 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.078922+0000", 2024-09-14T23:59:12.572 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1354e-05 2024-09-14T23:59:12.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:12.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.078930+0000", 2024-09-14T23:59:12.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5730000000000002e-06 2024-09-14T23:59:12.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:12.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.135870+0000", 2024-09-14T23:59:12.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056940651000000002 2024-09-14T23:59:12.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.573 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.573 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:12.573 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.135951+0000", 2024-09-14T23:59:12.573 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0890999999999995e-05 2024-09-14T23:59:12.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.574 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:12.574 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.136020+0000", 2024-09-14T23:59:12.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8417000000000006e-05 2024-09-14T23:59:12.574 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:12.574 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:12.574 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:12.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.574 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.574 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184323 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4183040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:59:12.574 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.079228+0000", 2024-09-14T23:59:12.574 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.481366449999996, 2024-09-14T23:59:12.574 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058454487999999999, 2024-09-14T23:59:12.574 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:12.575 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:12.575 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:12.575 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:12.575 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:12.575 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:12.575 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184323 2024-09-14T23:59:12.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.575 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:12.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.575 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:12.575 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079228+0000", 2024-09-14T23:59:12.575 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:12.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:12.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079228+0000", 2024-09-14T23:59:12.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:12.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:12.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079229+0000", 2024-09-14T23:59:12.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5299999999999999e-07 2024-09-14T23:59:12.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.576 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:12.576 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079232+0000", 2024-09-14T23:59:12.576 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.8839999999999998e-06 2024-09-14T23:59:12.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:12.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079238+0000", 2024-09-14T23:59:12.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2299999999999996e-06 2024-09-14T23:59:12.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:12.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079243+0000", 2024-09-14T23:59:12.577 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3029999999999999e-06 2024-09-14T23:59:12.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.577 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:12.577 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.137533+0000", 2024-09-14T23:59:12.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058289674999999999 2024-09-14T23:59:12.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:12.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.137615+0000", 2024-09-14T23:59:12.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2288999999999998e-05 2024-09-14T23:59:12.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.578 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:12.578 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.137683+0000", 2024-09-14T23:59:12.578 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7654e-05 2024-09-14T23:59:12.578 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:12.578 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:12.578 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:12.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.579 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.579 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184324 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4073472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:59:12.579 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.079251+0000", 2024-09-14T23:59:12.579 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.481343983000002, 2024-09-14T23:59:12.579 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058609383000000001, 2024-09-14T23:59:12.579 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:12.579 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:12.579 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:12.579 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:12.579 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:12.579 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:12.579 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184324 2024-09-14T23:59:12.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.580 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:12.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:12.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079251+0000", 2024-09-14T23:59:12.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:12.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:12.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.079251+0000", 2024-09-14T23:59:12.580 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:12.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.580 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:12.580 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.080920+0000", 2024-09-14T23:59:12.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0016687430000000001 2024-09-14T23:59:12.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:12.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.080924+0000", 2024-09-14T23:59:12.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3869999999999997e-06 2024-09-14T23:59:12.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:12.581 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.080935+0000", 2024-09-14T23:59:12.581 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0515e-05 2024-09-14T23:59:12.581 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.581 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.581 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:12.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.080942+0000", 2024-09-14T23:59:12.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0990000000000001e-06 2024-09-14T23:59:12.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:12.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.137713+0000", 2024-09-14T23:59:12.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056771288000000003 2024-09-14T23:59:12.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.582 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:12.582 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.137794+0000", 2024-09-14T23:59:12.582 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0613999999999998e-05 2024-09-14T23:59:12.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.583 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:12.583 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.137860+0000", 2024-09-14T23:59:12.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6736999999999995e-05 2024-09-14T23:59:12.583 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:12.583 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:12.583 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:12.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.583 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.583 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184346 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4099072~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:59:12.583 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.083616+0000", 2024-09-14T23:59:12.583 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.476978798000005, 2024-09-14T23:59:12.583 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058557419999999999, 2024-09-14T23:59:12.583 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:12.583 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:12.584 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:12.584 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:12.584 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:12.584 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:12.584 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184346 2024-09-14T23:59:12.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.584 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:12.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:12.584 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083616+0000", 2024-09-14T23:59:12.584 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:12.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.584 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.584 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:12.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083616+0000", 2024-09-14T23:59:12.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:12.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.585 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:12.585 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083620+0000", 2024-09-14T23:59:12.585 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0119999999999999e-06 2024-09-14T23:59:12.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.585 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:12.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083623+0000", 2024-09-14T23:59:12.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9059999999999998e-06 2024-09-14T23:59:12.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:12.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083631+0000", 2024-09-14T23:59:12.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2679999999999994e-06 2024-09-14T23:59:12.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.586 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.586 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:12.586 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083637+0000", 2024-09-14T23:59:12.586 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1210000000000003e-06 2024-09-14T23:59:12.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:12.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142018+0000", 2024-09-14T23:59:12.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058381111999999999 2024-09-14T23:59:12.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:12.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142100+0000", 2024-09-14T23:59:12.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1308000000000001e-05 2024-09-14T23:59:12.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.587 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.587 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:12.587 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142173+0000", 2024-09-14T23:59:12.587 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3693000000000001e-05 2024-09-14T23:59:12.587 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:12.588 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:12.588 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:12.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.588 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184347 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4122624~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:59:12.588 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.083647+0000", 2024-09-14T23:59:12.588 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.476947968000005, 2024-09-14T23:59:12.588 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058706388999999998, 2024-09-14T23:59:12.588 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:12.589 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:12.589 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:12.589 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:12.589 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:12.589 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:12.589 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184347 2024-09-14T23:59:12.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.589 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:12.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.589 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:12.589 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083647+0000", 2024-09-14T23:59:12.589 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:12.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:12.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083647+0000", 2024-09-14T23:59:12.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:12.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:12.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083647+0000", 2024-09-14T23:59:12.590 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3099999999999998e-07 2024-09-14T23:59:12.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.590 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.590 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:12.590 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083650+0000", 2024-09-14T23:59:12.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2900000000000001e-06 2024-09-14T23:59:12.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:12.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083657+0000", 2024-09-14T23:59:12.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.486e-06 2024-09-14T23:59:12.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:12.591 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083663+0000", 2024-09-14T23:59:12.591 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4330000000000003e-06 2024-09-14T23:59:12.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.591 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.591 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:12.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142206+0000", 2024-09-14T23:59:12.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058543237999999997 2024-09-14T23:59:12.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:12.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142284+0000", 2024-09-14T23:59:12.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8487999999999998e-05 2024-09-14T23:59:12.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.592 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.592 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:12.592 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142353+0000", 2024-09-14T23:59:12.592 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9023000000000006e-05 2024-09-14T23:59:12.592 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:12.593 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:12.593 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:12.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.593 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184348 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4103168~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:59:12.593 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.083669+0000", 2024-09-14T23:59:12.593 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.476925550999994, 2024-09-14T23:59:12.593 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058864232000000002, 2024-09-14T23:59:12.593 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:12.593 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:12.593 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:12.593 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:12.593 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:12.593 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:12.594 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184348 2024-09-14T23:59:12.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.594 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:12.594 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.594 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:12.594 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083669+0000", 2024-09-14T23:59:12.594 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:12.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.596 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.596 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:12.596 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083669+0000", 2024-09-14T23:59:12.596 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:12.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:12.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083670+0000", 2024-09-14T23:59:12.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9999999999999998e-07 2024-09-14T23:59:12.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.597 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:12.597 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083672+0000", 2024-09-14T23:59:12.597 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1430000000000001e-06 2024-09-14T23:59:12.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.597 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:12.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083678+0000", 2024-09-14T23:59:12.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5200000000000003e-06 2024-09-14T23:59:12.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:12.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083683+0000", 2024-09-14T23:59:12.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9359999999999998e-06 2024-09-14T23:59:12.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.598 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.598 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:12.598 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142384+0000", 2024-09-14T23:59:12.598 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058701159000000003 2024-09-14T23:59:12.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:12.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142466+0000", 2024-09-14T23:59:12.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1024000000000002e-05 2024-09-14T23:59:12.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.599 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.599 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:12.599 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142534+0000", 2024-09-14T23:59:12.599 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8050000000000001e-05 2024-09-14T23:59:12.599 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:12.599 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:12.599 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:12.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.600 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184349 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4126720~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:59:12.600 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.083690+0000", 2024-09-14T23:59:12.600 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.476905016000003, 2024-09-14T23:59:12.600 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059019911000000001, 2024-09-14T23:59:12.600 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:12.600 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:12.600 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:12.600 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:12.600 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:12.600 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:12.600 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184349 2024-09-14T23:59:12.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.601 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:12.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:12.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083690+0000", 2024-09-14T23:59:12.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:12.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:12.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.083690+0000", 2024-09-14T23:59:12.601 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:12.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.601 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.601 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:12.601 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.084900+0000", 2024-09-14T23:59:12.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0012099770000000001 2024-09-14T23:59:12.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:12.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.084905+0000", 2024-09-14T23:59:12.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.028e-06 2024-09-14T23:59:12.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:12.602 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.084914+0000", 2024-09-14T23:59:12.602 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6619999999999994e-06 2024-09-14T23:59:12.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.602 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.602 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:12.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.084922+0000", 2024-09-14T23:59:12.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5720000000000003e-06 2024-09-14T23:59:12.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:12.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142565+0000", 2024-09-14T23:59:12.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057642605999999999 2024-09-14T23:59:12.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:12.603 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142644+0000", 2024-09-14T23:59:12.603 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9370999999999996e-05 2024-09-14T23:59:12.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.603 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.603 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:12.604 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.142710+0000", 2024-09-14T23:59:12.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5695e-05 2024-09-14T23:59:12.604 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:12.604 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:12.604 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:12.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.604 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184354 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4115456~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:59:12.604 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.085029+0000", 2024-09-14T23:59:12.604 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.475566365999995, 2024-09-14T23:59:12.604 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058626327999999998, 2024-09-14T23:59:12.604 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:12.604 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:12.605 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:12.605 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:12.605 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:12.605 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:12.605 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184354 2024-09-14T23:59:12.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.605 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:12.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:12.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085029+0000", 2024-09-14T23:59:12.605 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:12.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.605 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.605 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:12.605 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085029+0000", 2024-09-14T23:59:12.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:12.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:12.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085029+0000", 2024-09-14T23:59:12.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0800000000000001e-07 2024-09-14T23:59:12.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:12.606 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085030+0000", 2024-09-14T23:59:12.606 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.226e-06 2024-09-14T23:59:12.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.606 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.606 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:12.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085033+0000", 2024-09-14T23:59:12.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9569999999999998e-06 2024-09-14T23:59:12.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:12.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085035+0000", 2024-09-14T23:59:12.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.587e-06 2024-09-14T23:59:12.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.607 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.607 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:12.607 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143491+0000", 2024-09-14T23:59:12.607 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058455337000000003 2024-09-14T23:59:12.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:12.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143575+0000", 2024-09-14T23:59:12.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3850000000000005e-05 2024-09-14T23:59:12.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.608 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:12.608 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143655+0000", 2024-09-14T23:59:12.608 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0162999999999998e-05 2024-09-14T23:59:12.608 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:12.608 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:12.608 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:12.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.608 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.609 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184355 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4139008~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:59:12.609 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.085039+0000", 2024-09-14T23:59:12.609 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.475555962000001, 2024-09-14T23:59:12.609 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058801369999999999, 2024-09-14T23:59:12.609 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:12.609 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:12.609 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:12.609 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:12.609 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:12.609 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:12.609 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184355 2024-09-14T23:59:12.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.609 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:12.609 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:12.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085039+0000", 2024-09-14T23:59:12.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:12.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:12.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085039+0000", 2024-09-14T23:59:12.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:12.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.610 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:12.610 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085039+0000", 2024-09-14T23:59:12.610 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.03e-07 2024-09-14T23:59:12.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:12.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085040+0000", 2024-09-14T23:59:12.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.158e-06 2024-09-14T23:59:12.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:12.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085043+0000", 2024-09-14T23:59:12.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0240000000000002e-06 2024-09-14T23:59:12.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.611 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.611 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:12.611 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085046+0000", 2024-09-14T23:59:12.611 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4600000000000002e-06 2024-09-14T23:59:12.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:12.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143688+0000", 2024-09-14T23:59:12.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058642356 2024-09-14T23:59:12.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:12.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143766+0000", 2024-09-14T23:59:12.612 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8219000000000004e-05 2024-09-14T23:59:12.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.612 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.612 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:12.612 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143840+0000", 2024-09-14T23:59:12.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3949999999999995e-05 2024-09-14T23:59:12.613 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:12.613 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:12.613 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:12.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.613 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184356 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4119552~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:59:12.613 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.085049+0000", 2024-09-14T23:59:12.613 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.475545791000002, 2024-09-14T23:59:12.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058977992, 2024-09-14T23:59:12.613 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:12.613 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:12.613 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:12.613 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:12.614 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:12.614 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:12.614 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184356 2024-09-14T23:59:12.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.614 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:12.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:12.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085049+0000", 2024-09-14T23:59:12.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:12.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:12.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.085049+0000", 2024-09-14T23:59:12.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:12.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:12.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.086934+0000", 2024-09-14T23:59:12.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018854189999999999 2024-09-14T23:59:12.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:12.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.086939+0000", 2024-09-14T23:59:12.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.493e-06 2024-09-14T23:59:12.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:12.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.086951+0000", 2024-09-14T23:59:12.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1969e-05 2024-09-14T23:59:12.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:12.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.086959+0000", 2024-09-14T23:59:12.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5780000000000001e-06 2024-09-14T23:59:12.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:12.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143871+0000", 2024-09-14T23:59:12.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056912114 2024-09-14T23:59:12.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:12.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143951+0000", 2024-09-14T23:59:12.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9938000000000006e-05 2024-09-14T23:59:12.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:12.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.144027+0000", 2024-09-14T23:59:12.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6481000000000002e-05 2024-09-14T23:59:12.617 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:12.617 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:12.617 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:12.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.617 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184364 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4135936~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:59:12.618 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.087207+0000", 2024-09-14T23:59:12.618 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.473387510999999, 2024-09-14T23:59:12.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058466420999999998, 2024-09-14T23:59:12.618 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:12.618 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:12.618 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:12.618 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:12.618 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:12.618 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:12.618 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184364 2024-09-14T23:59:12.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.618 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:12.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:12.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087207+0000", 2024-09-14T23:59:12.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:12.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:12.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087207+0000", 2024-09-14T23:59:12.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:12.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:12.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087208+0000", 2024-09-14T23:59:12.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.35e-07 2024-09-14T23:59:12.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:12.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087209+0000", 2024-09-14T23:59:12.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.573e-06 2024-09-14T23:59:12.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:12.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087212+0000", 2024-09-14T23:59:12.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2770000000000001e-06 2024-09-14T23:59:12.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:12.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087215+0000", 2024-09-14T23:59:12.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5500000000000001e-06 2024-09-14T23:59:12.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:12.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145520+0000", 2024-09-14T23:59:12.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058304835999999999 2024-09-14T23:59:12.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:12.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145602+0000", 2024-09-14T23:59:12.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1714999999999999e-05 2024-09-14T23:59:12.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:12.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145674+0000", 2024-09-14T23:59:12.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2235000000000002e-05 2024-09-14T23:59:12.622 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:12.622 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:12.622 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:12.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.622 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184365 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4159488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:59:12.622 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.087219+0000", 2024-09-14T23:59:12.622 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.473375480000001, 2024-09-14T23:59:12.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058643533999999997, 2024-09-14T23:59:12.622 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:12.622 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:12.622 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:12.623 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:12.623 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:12.623 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:12.623 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184365 2024-09-14T23:59:12.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.623 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:12.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:12.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087219+0000", 2024-09-14T23:59:12.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:12.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:12.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087219+0000", 2024-09-14T23:59:12.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:12.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:12.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087220+0000", 2024-09-14T23:59:12.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.39e-07 2024-09-14T23:59:12.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:12.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087222+0000", 2024-09-14T23:59:12.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0609999999999998e-06 2024-09-14T23:59:12.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:12.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087228+0000", 2024-09-14T23:59:12.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.8590000000000001e-06 2024-09-14T23:59:12.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:12.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087232+0000", 2024-09-14T23:59:12.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8149999999999996e-06 2024-09-14T23:59:12.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:12.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145705+0000", 2024-09-14T23:59:12.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058472532000000001 2024-09-14T23:59:12.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:12.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145784+0000", 2024-09-14T23:59:12.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9382000000000007e-05 2024-09-14T23:59:12.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:12.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145863+0000", 2024-09-14T23:59:12.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8546000000000003e-05 2024-09-14T23:59:12.626 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:12.626 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:12.626 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:12.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.627 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184366 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4140032~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:59:12.627 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.087240+0000", 2024-09-14T23:59:12.627 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.473355032000001, 2024-09-14T23:59:12.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058818068000000001, 2024-09-14T23:59:12.627 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:12.627 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:12.627 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:12.627 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:12.627 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:12.627 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:12.627 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184366 2024-09-14T23:59:12.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.627 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:12.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:12.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087240+0000", 2024-09-14T23:59:12.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:12.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:12.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087240+0000", 2024-09-14T23:59:12.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:12.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:12.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.088904+0000", 2024-09-14T23:59:12.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001664315 2024-09-14T23:59:12.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:12.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.088909+0000", 2024-09-14T23:59:12.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7709999999999997e-06 2024-09-14T23:59:12.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:12.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.088921+0000", 2024-09-14T23:59:12.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1661999999999999e-05 2024-09-14T23:59:12.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:12.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.088928+0000", 2024-09-14T23:59:12.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8189999999999992e-06 2024-09-14T23:59:12.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:12.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145901+0000", 2024-09-14T23:59:12.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056972478 2024-09-14T23:59:12.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:12.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145977+0000", 2024-09-14T23:59:12.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5763e-05 2024-09-14T23:59:12.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.631 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:12.631 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.146058+0000", 2024-09-14T23:59:12.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1260000000000005e-05 2024-09-14T23:59:12.631 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:12.631 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:12.631 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:12.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.631 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184375 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4179968~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:59:12.631 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.089199+0000", 2024-09-14T23:59:12.631 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.471395694999998, 2024-09-14T23:59:12.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058597445999999997, 2024-09-14T23:59:12.631 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:12.631 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:12.632 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:12.632 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:12.632 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:12.632 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:12.632 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184375 2024-09-14T23:59:12.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.632 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:12.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:12.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089199+0000", 2024-09-14T23:59:12.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:12.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:12.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089199+0000", 2024-09-14T23:59:12.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:12.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:12.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089200+0000", 2024-09-14T23:59:12.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8100000000000003e-07 2024-09-14T23:59:12.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:12.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089202+0000", 2024-09-14T23:59:12.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7599999999999998e-06 2024-09-14T23:59:12.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:12.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089209+0000", 2024-09-14T23:59:12.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7279999999999998e-06 2024-09-14T23:59:12.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:12.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089215+0000", 2024-09-14T23:59:12.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5090000000000001e-06 2024-09-14T23:59:12.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:12.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147638+0000", 2024-09-14T23:59:12.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058423105000000003 2024-09-14T23:59:12.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:12.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147719+0000", 2024-09-14T23:59:12.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1217000000000005e-05 2024-09-14T23:59:12.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:12.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147797+0000", 2024-09-14T23:59:12.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7645999999999995e-05 2024-09-14T23:59:12.635 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:12.635 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:12.635 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:12.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.636 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184376 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4160512~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:59:12.636 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.089232+0000", 2024-09-14T23:59:12.636 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.471363205000003, 2024-09-14T23:59:12.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058751338, 2024-09-14T23:59:12.636 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:12.636 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:12.636 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:12.636 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:12.636 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:12.636 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:12.636 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184376 2024-09-14T23:59:12.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.637 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:12.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:12.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089232+0000", 2024-09-14T23:59:12.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:12.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:12.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089232+0000", 2024-09-14T23:59:12.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:12.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:12.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089232+0000", 2024-09-14T23:59:12.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5600000000000001e-07 2024-09-14T23:59:12.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:12.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089235+0000", 2024-09-14T23:59:12.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5639999999999999e-06 2024-09-14T23:59:12.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:12.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089242+0000", 2024-09-14T23:59:12.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0369999999999997e-06 2024-09-14T23:59:12.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:12.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089247+0000", 2024-09-14T23:59:12.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2079999999999999e-06 2024-09-14T23:59:12.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:12.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147831+0000", 2024-09-14T23:59:12.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058583853999999998 2024-09-14T23:59:12.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:12.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147915+0000", 2024-09-14T23:59:12.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4208000000000004e-05 2024-09-14T23:59:12.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:12.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.147983+0000", 2024-09-14T23:59:12.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8010999999999996e-05 2024-09-14T23:59:12.640 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:12.640 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:12.640 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:12.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.640 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184377 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4184064~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:59:12.640 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.089255+0000", 2024-09-14T23:59:12.640 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.471339649000001, 2024-09-14T23:59:12.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058906100000000003, 2024-09-14T23:59:12.640 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:12.641 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:12.641 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:12.641 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:12.641 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:12.641 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:12.641 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184377 2024-09-14T23:59:12.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.641 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:12.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:12.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089255+0000", 2024-09-14T23:59:12.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:12.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:12.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089255+0000", 2024-09-14T23:59:12.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:12.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:12.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.090903+0000", 2024-09-14T23:59:12.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001647871 2024-09-14T23:59:12.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.642 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:12.642 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.090908+0000", 2024-09-14T23:59:12.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7199999999999997e-06 2024-09-14T23:59:12.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:12.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.090919+0000", 2024-09-14T23:59:12.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1472e-05 2024-09-14T23:59:12.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:12.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.090927+0000", 2024-09-14T23:59:12.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0390000000000005e-06 2024-09-14T23:59:12.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:12.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.148013+0000", 2024-09-14T23:59:12.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057085662000000002 2024-09-14T23:59:12.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:12.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.148090+0000", 2024-09-14T23:59:12.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6778000000000002e-05 2024-09-14T23:59:12.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:12.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.148161+0000", 2024-09-14T23:59:12.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1557999999999995e-05 2024-09-14T23:59:12.644 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:12.644 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:12.644 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:12.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.645 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184386 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4185088~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:59:12.645 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.091063+0000", 2024-09-14T23:59:12.645 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.469531985000003, 2024-09-14T23:59:12.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058603404999999997, 2024-09-14T23:59:12.645 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:12.645 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:12.645 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:12.645 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:12.645 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:12.645 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:12.645 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184386 2024-09-14T23:59:12.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.646 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:12.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:12.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091063+0000", 2024-09-14T23:59:12.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:12.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:12.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091063+0000", 2024-09-14T23:59:12.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:12.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:12.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091063+0000", 2024-09-14T23:59:12.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1899999999999999e-07 2024-09-14T23:59:12.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:12.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091064+0000", 2024-09-14T23:59:12.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2699999999999999e-06 2024-09-14T23:59:12.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:12.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091067+0000", 2024-09-14T23:59:12.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.971e-06 2024-09-14T23:59:12.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:12.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091070+0000", 2024-09-14T23:59:12.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.525e-06 2024-09-14T23:59:12.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.648 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:12.648 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.149510+0000", 2024-09-14T23:59:12.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058439758000000001 2024-09-14T23:59:12.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.648 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:12.648 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.149595+0000", 2024-09-14T23:59:12.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5146000000000001e-05 2024-09-14T23:59:12.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.648 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:12.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.149666+0000", 2024-09-14T23:59:12.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1515999999999998e-05 2024-09-14T23:59:12.649 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:12.649 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:12.649 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:12.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.649 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184388 3.17 3:e9240089:::rbd_data.1118427720d0.0000000000000019:32 [read 4189184~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:59:12.649 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.091073+0000", 2024-09-14T23:59:12.649 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.469521709999995, 2024-09-14T23:59:12.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058781609999999998, 2024-09-14T23:59:12.649 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:12.649 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:12.650 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:12.650 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:12.650 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:12.650 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:12.650 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184388 2024-09-14T23:59:12.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.650 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:12.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.650 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:12.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091073+0000", 2024-09-14T23:59:12.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:12.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.651 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:12.651 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.091073+0000", 2024-09-14T23:59:12.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:12.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.651 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:12.651 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.092903+0000", 2024-09-14T23:59:12.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018296289999999999 2024-09-14T23:59:12.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.651 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:12.651 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.092910+0000", 2024-09-14T23:59:12.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.7510000000000002e-06 2024-09-14T23:59:12.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.652 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:12.652 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.092918+0000", 2024-09-14T23:59:12.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.7420000000000003e-06 2024-09-14T23:59:12.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.652 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:12.652 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.092925+0000", 2024-09-14T23:59:12.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0360000000000001e-06 2024-09-14T23:59:12.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.652 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:12.653 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.149700+0000", 2024-09-14T23:59:12.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056774340999999999 2024-09-14T23:59:12.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.653 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:12.653 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.149784+0000", 2024-09-14T23:59:12.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4393000000000003e-05 2024-09-14T23:59:12.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.653 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:12.653 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.149855+0000", 2024-09-14T23:59:12.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0717999999999996e-05 2024-09-14T23:59:12.653 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:12.654 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:12.654 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:12.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.654 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4172.0:8 3.12 3:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-14T23:59:12.654 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.976999+0000", 2024-09-14T23:59:12.654 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 64.583596301, 2024-09-14T23:59:12.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0046036439999999, 2024-09-14T23:59:12.654 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:12.654 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:12.654 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:12.654 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:12.654 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4172", 2024-09-14T23:59:12.654 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/2542269203", 2024-09-14T23:59:12.654 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 8 2024-09-14T23:59:12.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.655 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:12.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.655 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:12.655 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.976999+0000", 2024-09-14T23:59:12.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:12.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.655 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:12.655 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.976999+0000", 2024-09-14T23:59:12.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:12.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.656 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:12.656 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.977000+0000", 2024-09-14T23:59:12.656 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5200000000000001e-06 2024-09-14T23:59:12.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.656 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.656 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:12.656 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.977005+0000", 2024-09-14T23:59:12.656 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.04e-06 2024-09-14T23:59:12.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.656 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.656 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:12.656 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.977013+0000", 2024-09-14T23:59:12.656 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2889999999999998e-06 2024-09-14T23:59:12.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.657 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.657 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:12.657 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.977024+0000", 2024-09-14T23:59:12.657 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0307e-05 2024-09-14T23:59:12.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.657 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.657 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:12.657 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.977086+0000", 2024-09-14T23:59:12.657 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2248000000000004e-05 2024-09-14T23:59:12.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.657 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.657 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:59:12.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.977097+0000", 2024-09-14T23:59:12.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1123e-05 2024-09-14T23:59:12.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:12.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:08.981470+0000", 2024-09-14T23:59:12.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.004373116 2024-09-14T23:59:12.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:12.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:12.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:12.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:08.981602+0000", 2024-09-14T23:59:12.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000132001 2024-09-14T23:59:12.658 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:12.658 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:12.659 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:12.659 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:12.659 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:12.659 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:59:12.660 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T23:59:12.661 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:59:12.659+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:59:12.761 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:59:12.760+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:59:12.767 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:59:12.767 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:59:12.767 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:59:12.768 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:59:12.776 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T23:59:12.862 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:59:12.860+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:59:12.883 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:59:12.884 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:59:12.884 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:59:12.884 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:59:12.884 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:59:12.892 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T23:59:12.976 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:59:12.975+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:59:13.053 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:59:13.054 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:59:13.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:59:13.054 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:59:13.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.054 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5882 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:13.054 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:59:13.054 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 153.67526785300001, 2024-09-14T23:59:13.054 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050549666, 2024-09-14T23:59:13.054 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:13.054 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:13.054 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:13.054 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:13.055 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:13.055 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:13.055 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5882 2024-09-14T23:59:13.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.055 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:13.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:13.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:59:13.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.055 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:13.055 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:59:13.055 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:13.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:59:13.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8099999999999998e-07 2024-09-14T23:59:13.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:13.056 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375842+0000", 2024-09-14T23:59:13.056 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9900000000000009e-07 2024-09-14T23:59:13.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.056 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:13.057 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375845+0000", 2024-09-14T23:59:13.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.055e-06 2024-09-14T23:59:13.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.057 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:13.057 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375848+0000", 2024-09-14T23:59:13.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7609999999999999e-06 2024-09-14T23:59:13.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.057 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:13.057 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426381+0000", 2024-09-14T23:59:13.057 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050533392000000003 2024-09-14T23:59:13.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.058 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:13.058 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426390+0000", 2024-09-14T23:59:13.058 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0780000000000002e-06 2024-09-14T23:59:13.058 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.058 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:13.058 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.058 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5884 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:13.058 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:59:13.058 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 153.67525771300001, 2024-09-14T23:59:13.058 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050717439000000003, 2024-09-14T23:59:13.058 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:13.058 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:13.059 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:13.059 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:13.059 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:13.059 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:13.059 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5884 2024-09-14T23:59:13.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.059 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:13.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:13.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:59:13.059 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.059 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.059 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:13.059 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:59:13.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:13.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:59:13.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0800000000000001e-07 2024-09-14T23:59:13.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.060 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:13.060 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375852+0000", 2024-09-14T23:59:13.060 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.02e-06 2024-09-14T23:59:13.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.060 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.061 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:13.061 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375855+0000", 2024-09-14T23:59:13.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9900000000000002e-06 2024-09-14T23:59:13.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.061 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:13.061 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375857+0000", 2024-09-14T23:59:13.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3499999999999999e-06 2024-09-14T23:59:13.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.061 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.061 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:13.061 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426556+0000", 2024-09-14T23:59:13.061 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050698139000000003 2024-09-14T23:59:13.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.062 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:13.062 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426568+0000", 2024-09-14T23:59:13.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2731999999999999e-05 2024-09-14T23:59:13.062 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.062 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:13.062 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.062 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5885 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:13.062 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:59:13.062 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 153.67523868800001, 2024-09-14T23:59:13.062 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050737333000000003, 2024-09-14T23:59:13.062 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:13.062 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:13.063 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:13.063 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:13.063 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:13.063 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:13.063 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5885 2024-09-14T23:59:13.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.063 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:13.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:13.063 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:59:13.063 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.063 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:13.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:59:13.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.064 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:13.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:59:13.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5899999999999998e-07 2024-09-14T23:59:13.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.064 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:13.064 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375872+0000", 2024-09-14T23:59:13.064 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4219999999999999e-06 2024-09-14T23:59:13.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.064 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:13.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375878+0000", 2024-09-14T23:59:13.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4980000000000003e-06 2024-09-14T23:59:13.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:13.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375881+0000", 2024-09-14T23:59:13.065 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6110000000000001e-06 2024-09-14T23:59:13.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.065 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:13.065 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426596+0000", 2024-09-14T23:59:13.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050715320000000001 2024-09-14T23:59:13.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.066 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:13.066 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426607+0000", 2024-09-14T23:59:13.066 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1223000000000001e-05 2024-09-14T23:59:13.066 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.066 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:13.066 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.066 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5886 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:13.066 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:59:13.066 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 153.67522457499999, 2024-09-14T23:59:13.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050803395000000001, 2024-09-14T23:59:13.067 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:13.067 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:13.067 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:13.067 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:13.067 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:13.067 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:13.067 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5886 2024-09-14T23:59:13.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.067 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:13.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.067 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:13.067 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:59:13.067 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:13.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:59:13.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:13.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:59:13.068 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2700000000000001e-07 2024-09-14T23:59:13.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.068 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:13.068 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375886+0000", 2024-09-14T23:59:13.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7060000000000001e-06 2024-09-14T23:59:13.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:13.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375890+0000", 2024-09-14T23:59:13.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2259999999999999e-06 2024-09-14T23:59:13.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.069 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:13.069 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375893+0000", 2024-09-14T23:59:13.069 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0819999999999999e-06 2024-09-14T23:59:13.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.069 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:13.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426677+0000", 2024-09-14T23:59:13.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050784158000000003 2024-09-14T23:59:13.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.070 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:13.070 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426687+0000", 2024-09-14T23:59:13.070 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9960000000000005e-06 2024-09-14T23:59:13.070 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.070 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:13.070 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.070 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5887 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:13.070 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:59:13.071 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 153.67520248899999, 2024-09-14T23:59:13.071 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050833070000000001, 2024-09-14T23:59:13.071 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:13.071 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:13.071 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:13.071 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:13.071 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:13.071 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:13.071 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5887 2024-09-14T23:59:13.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.071 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:13.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.071 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:13.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:59:13.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:13.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:59:13.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.072 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:13.072 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:59:13.072 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3299999999999998e-07 2024-09-14T23:59:13.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:13.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375908+0000", 2024-09-14T23:59:13.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7689999999999999e-06 2024-09-14T23:59:13.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:13.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375912+0000", 2024-09-14T23:59:13.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.535e-06 2024-09-14T23:59:13.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.073 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:13.073 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375916+0000", 2024-09-14T23:59:13.073 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8979999999999998e-06 2024-09-14T23:59:13.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:13.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426729+0000", 2024-09-14T23:59:13.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050813453000000001 2024-09-14T23:59:13.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.074 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:13.074 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426739+0000", 2024-09-14T23:59:13.074 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0081999999999999e-05 2024-09-14T23:59:13.074 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.074 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:13.074 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.075 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5888 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:13.075 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:59:13.075 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 153.67518941200001, 2024-09-14T23:59:13.075 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050854652, 2024-09-14T23:59:13.075 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:13.075 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:13.075 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:13.075 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:13.075 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:13.075 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:13.075 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5888 2024-09-14T23:59:13.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.075 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:13.075 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:13.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:59:13.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:13.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:59:13.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.076 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.076 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:13.076 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:59:13.076 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1400000000000001e-07 2024-09-14T23:59:13.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:13.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375921+0000", 2024-09-14T23:59:13.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.119e-06 2024-09-14T23:59:13.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:13.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375928+0000", 2024-09-14T23:59:13.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3170000000000004e-06 2024-09-14T23:59:13.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.077 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.077 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:13.077 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375931+0000", 2024-09-14T23:59:13.077 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6970000000000002e-06 2024-09-14T23:59:13.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:13.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426768+0000", 2024-09-14T23:59:13.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050837355000000001 2024-09-14T23:59:13.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.078 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:13.078 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426774+0000", 2024-09-14T23:59:13.078 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9499999999999998e-06 2024-09-14T23:59:13.078 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.078 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:13.078 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.079 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5889 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:13.079 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:59:13.079 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 153.67517248799999, 2024-09-14T23:59:13.079 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050885446000000001, 2024-09-14T23:59:13.079 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:13.079 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:13.079 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:13.079 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:13.079 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:13.079 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:13.079 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5889 2024-09-14T23:59:13.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.079 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:13.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:13.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:59:13.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:13.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:59:13.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.080 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.080 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:13.080 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:59:13.080 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.36e-07 2024-09-14T23:59:13.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:13.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375940+0000", 2024-09-14T23:59:13.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.844e-06 2024-09-14T23:59:13.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:13.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375944+0000", 2024-09-14T23:59:13.081 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6720000000000002e-06 2024-09-14T23:59:13.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.081 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:13.081 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375946+0000", 2024-09-14T23:59:13.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5629999999999999e-06 2024-09-14T23:59:13.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:13.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426810+0000", 2024-09-14T23:59:13.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050863255000000003 2024-09-14T23:59:13.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.082 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.082 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:13.082 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426822+0000", 2024-09-14T23:59:13.082 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1875999999999999e-05 2024-09-14T23:59:13.082 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.082 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:13.083 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.083 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5890 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:13.083 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:59:13.083 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 153.675158676, 2024-09-14T23:59:13.083 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050917549999999999, 2024-09-14T23:59:13.083 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:13.083 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:13.083 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:13.083 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:13.083 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:13.083 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:13.084 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5890 2024-09-14T23:59:13.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.084 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:13.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:13.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:59:13.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.084 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.084 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:13.084 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:59:13.084 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:13.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:59:13.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4299999999999999e-07 2024-09-14T23:59:13.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:13.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375951+0000", 2024-09-14T23:59:13.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1000000000000001e-06 2024-09-14T23:59:13.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.085 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.085 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:13.085 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375956+0000", 2024-09-14T23:59:13.085 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3370000000000001e-06 2024-09-14T23:59:13.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:13.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375958+0000", 2024-09-14T23:59:13.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4820000000000001e-06 2024-09-14T23:59:13.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:13.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426858+0000", 2024-09-14T23:59:13.086 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050899666000000003 2024-09-14T23:59:13.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.086 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.086 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:13.086 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426867+0000", 2024-09-14T23:59:13.087 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7219999999999994e-06 2024-09-14T23:59:13.087 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.087 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:13.087 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.087 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.087 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5891 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:13.087 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:59:13.087 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 153.67514735399999, 2024-09-14T23:59:13.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050948067, 2024-09-14T23:59:13.088 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:13.088 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:13.088 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:13.088 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:13.088 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:13.088 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:13.088 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5891 2024-09-14T23:59:13.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.088 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:13.088 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.088 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:13.088 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:59:13.088 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:13.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:59:13.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:13.089 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:59:13.089 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0800000000000001e-07 2024-09-14T23:59:13.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.089 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.089 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:13.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375962+0000", 2024-09-14T23:59:13.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9600000000000008e-07 2024-09-14T23:59:13.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:13.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375970+0000", 2024-09-14T23:59:13.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1600000000000001e-06 2024-09-14T23:59:13.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.090 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:13.090 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375972+0000", 2024-09-14T23:59:13.090 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4849999999999999e-06 2024-09-14T23:59:13.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:13.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426902+0000", 2024-09-14T23:59:13.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050929813999999997 2024-09-14T23:59:13.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.091 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:13.091 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426909+0000", 2024-09-14T23:59:13.091 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4039999999999998e-06 2024-09-14T23:59:13.091 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.091 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:13.091 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.091 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5892 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:13.092 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:59:13.092 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 153.67513249699999, 2024-09-14T23:59:13.092 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050965036999999998, 2024-09-14T23:59:13.092 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:13.092 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:13.092 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:13.092 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:13.092 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:13.092 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:13.092 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5892 2024-09-14T23:59:13.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.092 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:13.092 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.092 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:13.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:59:13.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:13.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:59:13.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.093 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:13.093 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:59:13.093 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.34e-07 2024-09-14T23:59:13.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:13.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375978+0000", 2024-09-14T23:59:13.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1880000000000001e-06 2024-09-14T23:59:13.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:13.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375981+0000", 2024-09-14T23:59:13.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2399999999999999e-06 2024-09-14T23:59:13.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.094 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:13.094 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375984+0000", 2024-09-14T23:59:13.094 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9110000000000002e-06 2024-09-14T23:59:13.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:13.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426934+0000", 2024-09-14T23:59:13.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050949912 2024-09-14T23:59:13.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.095 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:13.095 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426941+0000", 2024-09-14T23:59:13.095 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5519999999999999e-06 2024-09-14T23:59:13.095 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.095 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:13.095 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.095 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5893 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:13.096 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:59:13.096 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 153.67512189999999, 2024-09-14T23:59:13.096 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050975131999999999, 2024-09-14T23:59:13.096 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:13.096 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:13.096 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:13.096 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:13.096 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:13.096 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:13.096 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5893 2024-09-14T23:59:13.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.096 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:13.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:13.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:59:13.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:13.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:59:13.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.097 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:13.097 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:59:13.097 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2999999999999999e-07 2024-09-14T23:59:13.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:13.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375988+0000", 2024-09-14T23:59:13.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0410000000000001e-06 2024-09-14T23:59:13.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:13.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375992+0000", 2024-09-14T23:59:13.098 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.104e-06 2024-09-14T23:59:13.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.098 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:13.098 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375995+0000", 2024-09-14T23:59:13.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4310000000000001e-06 2024-09-14T23:59:13.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:13.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426958+0000", 2024-09-14T23:59:13.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050963642000000003 2024-09-14T23:59:13.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.099 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:13.099 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426962+0000", 2024-09-14T23:59:13.099 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6839999999999998e-06 2024-09-14T23:59:13.099 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.099 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:13.099 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.100 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5894 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:13.100 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:59:13.100 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 153.67511049999999, 2024-09-14T23:59:13.100 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050978333000000001, 2024-09-14T23:59:13.100 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:13.100 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:13.100 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:13.100 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:13.100 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:13.100 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:13.100 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5894 2024-09-14T23:59:13.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.101 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:13.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:13.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:59:13.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:13.101 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:59:13.101 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.101 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:13.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:59:13.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.23e-07 2024-09-14T23:59:13.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:13.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375999+0000", 2024-09-14T23:59:13.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9800000000000002e-07 2024-09-14T23:59:13.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.102 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:13.102 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376007+0000", 2024-09-14T23:59:13.102 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8539999999999997e-06 2024-09-14T23:59:13.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:13.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376011+0000", 2024-09-14T23:59:13.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.399e-06 2024-09-14T23:59:13.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:13.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426973+0000", 2024-09-14T23:59:13.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050962454999999997 2024-09-14T23:59:13.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.103 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.103 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:13.103 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426976+0000", 2024-09-14T23:59:13.103 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4039999999999999e-06 2024-09-14T23:59:13.103 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.104 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:13.104 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.104 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.104 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5895 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:13.104 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376014+0000", 2024-09-14T23:59:13.104 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 153.675094156, 2024-09-14T23:59:13.104 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050975951999999998, 2024-09-14T23:59:13.104 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:13.104 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:13.104 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:13.104 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:13.104 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:13.104 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:13.105 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5895 2024-09-14T23:59:13.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.105 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:13.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:13.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376014+0000", 2024-09-14T23:59:13.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.105 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:13.105 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376014+0000", 2024-09-14T23:59:13.105 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:13.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376015+0000", 2024-09-14T23:59:13.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2999999999999999e-07 2024-09-14T23:59:13.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:13.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376016+0000", 2024-09-14T23:59:13.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1659999999999999e-06 2024-09-14T23:59:13.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.106 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:13.106 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376024+0000", 2024-09-14T23:59:13.106 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8930000000000005e-06 2024-09-14T23:59:13.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:13.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376026+0000", 2024-09-14T23:59:13.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6869999999999999e-06 2024-09-14T23:59:13.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:13.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426987+0000", 2024-09-14T23:59:13.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050960601000000001 2024-09-14T23:59:13.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.107 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:13.107 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426990+0000", 2024-09-14T23:59:13.107 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3749999999999999e-06 2024-09-14T23:59:13.108 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.108 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:13.108 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.108 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.108 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5896 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:13.108 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:59:13.108 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 153.67507841899999, 2024-09-14T23:59:13.108 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050975656000000001, 2024-09-14T23:59:13.108 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:13.108 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:13.108 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:13.108 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:13.108 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:13.109 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:13.109 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5896 2024-09-14T23:59:13.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.109 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:13.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:13.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:59:13.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.109 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:13.109 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:59:13.109 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:13.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:59:13.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5100000000000001e-07 2024-09-14T23:59:13.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.110 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:13.110 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376032+0000", 2024-09-14T23:59:13.110 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.141e-06 2024-09-14T23:59:13.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:13.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376036+0000", 2024-09-14T23:59:13.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.25e-06 2024-09-14T23:59:13.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:13.111 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376038+0000", 2024-09-14T23:59:13.111 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5469999999999999e-06 2024-09-14T23:59:13.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.111 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:13.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427002+0000", 2024-09-14T23:59:13.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050964029000000001 2024-09-14T23:59:13.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.112 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:13.112 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427006+0000", 2024-09-14T23:59:13.112 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4379999999999999e-06 2024-09-14T23:59:13.112 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.112 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:13.112 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.112 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.113 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5897 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:13.113 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:59:13.113 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 153.67506661900001, 2024-09-14T23:59:13.113 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050978001000000002, 2024-09-14T23:59:13.113 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:13.113 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:13.113 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:13.113 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:13.113 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:13.113 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:13.113 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5897 2024-09-14T23:59:13.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.114 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:13.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:13.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:59:13.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.114 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:13.114 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:59:13.114 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:13.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:59:13.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2600000000000001e-07 2024-09-14T23:59:13.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:13.115 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376043+0000", 2024-09-14T23:59:13.115 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.054e-06 2024-09-14T23:59:13.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.115 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:13.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376047+0000", 2024-09-14T23:59:13.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0659999999999997e-06 2024-09-14T23:59:13.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:13.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376050+0000", 2024-09-14T23:59:13.116 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.88e-06 2024-09-14T23:59:13.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.116 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:13.116 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427017+0000", 2024-09-14T23:59:13.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050966453000000002 2024-09-14T23:59:13.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.117 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:13.117 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427020+0000", 2024-09-14T23:59:13.117 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3220000000000001e-06 2024-09-14T23:59:13.117 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.117 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:13.117 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.117 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.117 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5898 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:13.118 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376053+0000", 2024-09-14T23:59:13.118 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 153.67505513399999, 2024-09-14T23:59:13.118 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050982024000000001, 2024-09-14T23:59:13.118 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:13.118 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:13.118 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:13.118 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:13.118 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:13.118 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:13.118 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5898 2024-09-14T23:59:13.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.118 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:13.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:13.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376053+0000", 2024-09-14T23:59:13.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.119 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:13.119 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376053+0000", 2024-09-14T23:59:13.119 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.119 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:13.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376278+0000", 2024-09-14T23:59:13.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022426400000000001 2024-09-14T23:59:13.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:13.120 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376282+0000", 2024-09-14T23:59:13.120 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4739999999999999e-06 2024-09-14T23:59:13.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.120 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.120 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:13.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376296+0000", 2024-09-14T23:59:13.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4229999999999999e-05 2024-09-14T23:59:13.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:13.121 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376303+0000", 2024-09-14T23:59:13.121 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2199999999999997e-06 2024-09-14T23:59:13.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.121 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.121 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:13.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427031+0000", 2024-09-14T23:59:13.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050728154999999997 2024-09-14T23:59:13.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.122 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:13.122 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427036+0000", 2024-09-14T23:59:13.122 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6809999999999997e-06 2024-09-14T23:59:13.122 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.122 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:13.122 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.122 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.123 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5899 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:13.123 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376338+0000", 2024-09-14T23:59:13.123 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 153.67477041000001, 2024-09-14T23:59:13.123 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050712100000000003, 2024-09-14T23:59:13.123 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:13.123 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:13.123 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:13.123 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:13.123 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:13.123 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:13.124 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5899 2024-09-14T23:59:13.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.124 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:13.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:13.124 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376338+0000", 2024-09-14T23:59:13.124 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.124 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.124 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:13.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376338+0000", 2024-09-14T23:59:13.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:13.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376339+0000", 2024-09-14T23:59:13.125 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9100000000000003e-07 2024-09-14T23:59:13.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.125 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.125 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:13.125 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376342+0000", 2024-09-14T23:59:13.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7e-06 2024-09-14T23:59:13.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:13.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376350+0000", 2024-09-14T23:59:13.126 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5280000000000003e-06 2024-09-14T23:59:13.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.126 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:13.126 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376356+0000", 2024-09-14T23:59:13.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6010000000000002e-06 2024-09-14T23:59:13.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:13.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427047+0000", 2024-09-14T23:59:13.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050690962999999999 2024-09-14T23:59:13.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.127 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.127 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:13.127 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427050+0000", 2024-09-14T23:59:13.127 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.517e-06 2024-09-14T23:59:13.127 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.127 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:13.127 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.128 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5900 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:13.128 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376366+0000", 2024-09-14T23:59:13.128 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 153.674742151, 2024-09-14T23:59:13.128 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050697858999999998, 2024-09-14T23:59:13.128 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:13.128 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:13.128 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:13.128 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:13.128 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:13.128 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:13.128 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5900 2024-09-14T23:59:13.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.129 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:13.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:13.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376366+0000", 2024-09-14T23:59:13.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:13.129 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376366+0000", 2024-09-14T23:59:13.129 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.129 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.129 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:13.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377650+0000", 2024-09-14T23:59:13.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001283834 2024-09-14T23:59:13.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:13.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377655+0000", 2024-09-14T23:59:13.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7230000000000004e-06 2024-09-14T23:59:13.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.130 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:13.130 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377667+0000", 2024-09-14T23:59:13.130 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1746000000000001e-05 2024-09-14T23:59:13.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:13.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377676+0000", 2024-09-14T23:59:13.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3619999999999995e-06 2024-09-14T23:59:13.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:13.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427061+0000", 2024-09-14T23:59:13.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049384755000000002 2024-09-14T23:59:13.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.131 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:13.131 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427064+0000", 2024-09-14T23:59:13.131 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.439e-06 2024-09-14T23:59:13.131 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.132 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:13.132 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.132 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.132 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:132622 3.1e 3:79d0a5e0:::rbd_data.111847077d5e.0000000000000007:head [sparse-read 0~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T23:59:13.132 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:44.037901+0000", 2024-09-14T23:59:13.132 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 89.013207542000004, 2024-09-14T23:59:13.132 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1083420859999999, 2024-09-14T23:59:13.132 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:13.132 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:13.132 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:13.132 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:13.132 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:13.133 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:13.133 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 132622 2024-09-14T23:59:13.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.133 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:13.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:13.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037901+0000", 2024-09-14T23:59:13.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.133 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:13.133 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037901+0000", 2024-09-14T23:59:13.133 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:13.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037903+0000", 2024-09-14T23:59:13.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4619999999999999e-06 2024-09-14T23:59:13.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:13.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037910+0000", 2024-09-14T23:59:13.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7449999999999995e-06 2024-09-14T23:59:13.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.134 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.134 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:13.134 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037928+0000", 2024-09-14T23:59:13.134 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7305000000000001e-05 2024-09-14T23:59:13.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:13.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037939+0000", 2024-09-14T23:59:13.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1599e-05 2024-09-14T23:59:13.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:13.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.038049+0000", 2024-09-14T23:59:13.135 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000109577 2024-09-14T23:59:13.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.135 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op must wait for map", 2024-09-14T23:59:13.135 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.038057+0000", 2024-09-14T23:59:13.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1450000000000008e-06 2024-09-14T23:59:13.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:13.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.039937+0000", 2024-09-14T23:59:13.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00188024 2024-09-14T23:59:13.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:59:13.136 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.040039+0000", 2024-09-14T23:59:13.136 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000101934 2024-09-14T23:59:13.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.136 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.136 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:13.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:45.145805+0000", 2024-09-14T23:59:13.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.105765651 2024-09-14T23:59:13.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:13.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:45.145997+0000", 2024-09-14T23:59:13.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000192084 2024-09-14T23:59:13.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.137 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:13.137 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:45.146243+0000", 2024-09-14T23:59:13.137 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024634400000000001 2024-09-14T23:59:13.137 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.137 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:13.138 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.138 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.138 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4172.0:5 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:59:13.138 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.482606+0000", 2024-09-14T23:59:13.138 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 65.568502561000003, 2024-09-14T23:59:13.138 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.49381641300000001, 2024-09-14T23:59:13.138 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:13.138 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:13.138 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:13.138 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:13.138 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4172", 2024-09-14T23:59:13.138 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/2542269203", 2024-09-14T23:59:13.138 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5 2024-09-14T23:59:13.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.139 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:13.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:13.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482606+0000", 2024-09-14T23:59:13.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.139 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:13.139 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482606+0000", 2024-09-14T23:59:13.139 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.139 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:13.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482607+0000", 2024-09-14T23:59:13.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3120000000000001e-06 2024-09-14T23:59:13.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:13.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482615+0000", 2024-09-14T23:59:13.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9300000000000003e-06 2024-09-14T23:59:13.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.140 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:13.140 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482632+0000", 2024-09-14T23:59:13.140 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6966999999999999e-05 2024-09-14T23:59:13.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:13.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482640+0000", 2024-09-14T23:59:13.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1820000000000006e-06 2024-09-14T23:59:13.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:13.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482703+0000", 2024-09-14T23:59:13.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2098000000000001e-05 2024-09-14T23:59:13.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.141 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:59:13.141 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482713+0000", 2024-09-14T23:59:13.141 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0064000000000001e-05 2024-09-14T23:59:13.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:13.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.976307+0000", 2024-09-14T23:59:13.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.49359442999999997 2024-09-14T23:59:13.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.142 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.142 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:13.142 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.976422+0000", 2024-09-14T23:59:13.142 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011543 2024-09-14T23:59:13.142 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.142 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:13.142 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.143 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.143 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:13.143 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:59:13.145 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T23:59:13.145 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:59:13.143+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:59:13.153 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T23:59:13.245 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:59:13.244+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:59:13.295 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:59:13.296 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:59:13.296 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:59:13.296 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:59:13.306 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T23:59:13.346 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:59:13.345+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:59:13.448 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:59:13.446+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:59:13.454 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:59:13.454 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:59:13.454 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:59:13.454 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:59:13.454 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:59:13.463 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T23:59:13.548 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:59:13.547+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:59:13.612 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:59:13.613 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:59:13.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:59:13.613 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:59:13.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.613 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:56107 3.0 e56/23 3:00c47890:::rbd_data.1118c4b3aebe.0000000000000009:head v 56'125, mlcod=52'123)", 2024-09-14T23:59:13.613 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.236123+0000", 2024-09-14T23:59:13.613 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 127.37402742800001, 2024-09-14T23:59:13.613 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0061273220000000002, 2024-09-14T23:59:13.613 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:13.613 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:13.613 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:13.613 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:13.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:13.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.236123+0000", 2024-09-14T23:59:13.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:13.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.236123+0000", 2024-09-14T23:59:13.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.614 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.614 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:13.614 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.236125+0000", 2024-09-14T23:59:13.614 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.313e-06 2024-09-14T23:59:13.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:13.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237428+0000", 2024-09-14T23:59:13.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013029890000000001 2024-09-14T23:59:13.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:13.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237452+0000", 2024-09-14T23:59:13.615 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4090000000000001e-05 2024-09-14T23:59:13.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.615 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.615 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:13.615 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237462+0000", 2024-09-14T23:59:13.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0421999999999999e-05 2024-09-14T23:59:13.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:13.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237559+0000", 2024-09-14T23:59:13.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6378000000000005e-05 2024-09-14T23:59:13.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:13.616 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237583+0000", 2024-09-14T23:59:13.616 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4182e-05 2024-09-14T23:59:13.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.616 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.616 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:59:13.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.242205+0000", 2024-09-14T23:59:13.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0046217289999999998 2024-09-14T23:59:13.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.617 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:13.617 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.242251+0000", 2024-09-14T23:59:13.617 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6218999999999999e-05 2024-09-14T23:59:13.617 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.617 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:13.617 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.617 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:56108 3.0 e56/23 3:00c47890:::rbd_data.1118c4b3aebe.0000000000000009:head v 56'127, mlcod=52'123)", 2024-09-14T23:59:13.618 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.239693+0000", 2024-09-14T23:59:13.618 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 127.370457958, 2024-09-14T23:59:13.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0043496209999999997, 2024-09-14T23:59:13.618 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:13.618 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:13.618 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:13.618 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:13.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:13.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239693+0000", 2024-09-14T23:59:13.618 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.618 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.618 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.618 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:13.618 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239693+0000", 2024-09-14T23:59:13.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:13.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239694+0000", 2024-09-14T23:59:13.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0499999999999999e-06 2024-09-14T23:59:13.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.619 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:13.619 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239702+0000", 2024-09-14T23:59:13.619 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1170000000000004e-06 2024-09-14T23:59:13.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:13.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239716+0000", 2024-09-14T23:59:13.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4306e-05 2024-09-14T23:59:13.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:13.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239724+0000", 2024-09-14T23:59:13.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0399999999999993e-06 2024-09-14T23:59:13.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.620 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.620 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:13.620 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239778+0000", 2024-09-14T23:59:13.620 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4032e-05 2024-09-14T23:59:13.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:13.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239801+0000", 2024-09-14T23:59:13.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2770000000000001e-05 2024-09-14T23:59:13.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:59:13.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.243990+0000", 2024-09-14T23:59:13.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0041890909999999998 2024-09-14T23:59:13.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.621 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.621 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:13.621 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.244043+0000", 2024-09-14T23:59:13.621 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2215000000000001e-05 2024-09-14T23:59:13.622 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.622 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:13.622 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.622 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:56385 3.9 e56/23 3:963d31c7:::rbd_data.1118c4b3aebe.0000000000000003:head v 56'87, mlcod=53'85)", 2024-09-14T23:59:13.622 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.283936+0000", 2024-09-14T23:59:13.622 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 127.32621504799999, 2024-09-14T23:59:13.622 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0063042080000000004, 2024-09-14T23:59:13.622 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:13.622 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:13.622 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:13.622 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:13.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:13.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.283936+0000", 2024-09-14T23:59:13.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:13.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.283936+0000", 2024-09-14T23:59:13.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.623 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.623 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:13.623 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.283937+0000", 2024-09-14T23:59:13.623 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1179999999999999e-06 2024-09-14T23:59:13.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:13.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286338+0000", 2024-09-14T23:59:13.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024010149999999998 2024-09-14T23:59:13.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:13.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286360+0000", 2024-09-14T23:59:13.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2212e-05 2024-09-14T23:59:13.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.624 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.624 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:13.624 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286370+0000", 2024-09-14T23:59:13.624 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5850000000000006e-06 2024-09-14T23:59:13.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:13.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286443+0000", 2024-09-14T23:59:13.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2757999999999997e-05 2024-09-14T23:59:13.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:13.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286466+0000", 2024-09-14T23:59:13.625 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3844000000000001e-05 2024-09-14T23:59:13.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.625 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:59:13.625 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.290197+0000", 2024-09-14T23:59:13.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003730176 2024-09-14T23:59:13.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.626 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:13.626 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.290240+0000", 2024-09-14T23:59:13.626 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.35e-05 2024-09-14T23:59:13.626 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.626 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:13.626 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.626 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.626 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:56386 3.9 e56/23 3:963d31c7:::rbd_data.1118c4b3aebe.0000000000000003:head v 56'89, mlcod=53'85)", 2024-09-14T23:59:13.626 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.287578+0000", 2024-09-14T23:59:13.626 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 127.322573322, 2024-09-14T23:59:13.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033580250000000002, 2024-09-14T23:59:13.627 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:13.627 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:13.627 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:13.627 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:13.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:13.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287578+0000", 2024-09-14T23:59:13.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.627 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.627 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:13.627 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287578+0000", 2024-09-14T23:59:13.627 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:13.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287579+0000", 2024-09-14T23:59:13.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.64e-07 2024-09-14T23:59:13.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:13.628 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287586+0000", 2024-09-14T23:59:13.628 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5310000000000004e-06 2024-09-14T23:59:13.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.628 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.628 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:13.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287597+0000", 2024-09-14T23:59:13.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0903999999999999e-05 2024-09-14T23:59:13.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:13.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287605+0000", 2024-09-14T23:59:13.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7889999999999995e-06 2024-09-14T23:59:13.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.629 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:13.629 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.288437+0000", 2024-09-14T23:59:13.629 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000831914 2024-09-14T23:59:13.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:13.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.288460+0000", 2024-09-14T23:59:13.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3410000000000001e-05 2024-09-14T23:59:13.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:59:13.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.290883+0000", 2024-09-14T23:59:13.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024230599999999999 2024-09-14T23:59:13.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.630 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:13.630 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.290936+0000", 2024-09-14T23:59:13.630 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2453000000000001e-05 2024-09-14T23:59:13.631 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.631 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:13.631 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.631 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.631 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:64263 3.4 e57/23 3:20c6a768:::rbd_data.11188d356229.0000000000000001:head v 57'165, mlcod=57'163)", 2024-09-14T23:59:13.631 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:07.999031+0000", 2024-09-14T23:59:13.631 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 125.61112017400001, 2024-09-14T23:59:13.631 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023815210000000002, 2024-09-14T23:59:13.631 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:13.631 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:13.631 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:13.632 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:13.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:13.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999031+0000", 2024-09-14T23:59:13.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:13.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999031+0000", 2024-09-14T23:59:13.632 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.632 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.632 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:13.632 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999032+0000", 2024-09-14T23:59:13.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.176e-06 2024-09-14T23:59:13.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:13.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999660+0000", 2024-09-14T23:59:13.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00062817099999999996 2024-09-14T23:59:13.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.633 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.633 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:13.633 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999683+0000", 2024-09-14T23:59:13.633 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2816999999999999e-05 2024-09-14T23:59:13.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:13.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999693+0000", 2024-09-14T23:59:13.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0088999999999999e-05 2024-09-14T23:59:13.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.634 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.634 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:13.634 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999767+0000", 2024-09-14T23:59:13.634 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4056999999999999e-05 2024-09-14T23:59:13.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:13.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999793+0000", 2024-09-14T23:59:13.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5573000000000001e-05 2024-09-14T23:59:13.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.635 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:59:13.635 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:08.001379+0000", 2024-09-14T23:59:13.635 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015859019999999999 2024-09-14T23:59:13.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.636 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:13.636 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:08.001412+0000", 2024-09-14T23:59:13.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3735999999999997e-05 2024-09-14T23:59:13.636 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.636 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:13.636 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.636 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.636 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:66570 3.0 e60/23 3:055350b3:::rbd_data.111873792d9b.0000000000000005:head v 60'162, mlcod=57'159)", 2024-09-14T23:59:13.636 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:11.277517+0000", 2024-09-14T23:59:13.636 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 122.33263401799999, 2024-09-14T23:59:13.636 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003472199, 2024-09-14T23:59:13.637 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:13.637 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:13.637 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:13.637 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:13.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:13.637 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.277517+0000", 2024-09-14T23:59:13.637 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.637 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:13.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.277517+0000", 2024-09-14T23:59:13.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:13.638 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.277518+0000", 2024-09-14T23:59:13.638 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1379999999999999e-06 2024-09-14T23:59:13.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.638 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:13.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278504+0000", 2024-09-14T23:59:13.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00098617499999999994 2024-09-14T23:59:13.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:13.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278525+0000", 2024-09-14T23:59:13.639 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.09e-05 2024-09-14T23:59:13.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.639 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:13.639 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278534+0000", 2024-09-14T23:59:13.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6209999999999993e-06 2024-09-14T23:59:13.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:13.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278611+0000", 2024-09-14T23:59:13.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6926000000000002e-05 2024-09-14T23:59:13.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.640 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:13.640 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278641+0000", 2024-09-14T23:59:13.640 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0861999999999997e-05 2024-09-14T23:59:13.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:59:13.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.280953+0000", 2024-09-14T23:59:13.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023114239999999999 2024-09-14T23:59:13.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.641 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:13.641 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.280989+0000", 2024-09-14T23:59:13.641 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6152999999999998e-05 2024-09-14T23:59:13.641 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.642 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:13.642 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.642 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:158983 3.4 e103/23 3:20e43b29:::rbd_data.1118b2f495e6.0000000000000012:head v 103'454, mlcod=103'452)", 2024-09-14T23:59:13.642 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:58.917699+0000", 2024-09-14T23:59:13.642 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 74.692451968, 2024-09-14T23:59:13.642 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037933799999999998, 2024-09-14T23:59:13.642 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:13.642 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:13.642 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:13.642 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:13.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:13.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917699+0000", 2024-09-14T23:59:13.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:13.643 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917699+0000", 2024-09-14T23:59:13.643 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.643 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:13.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917700+0000", 2024-09-14T23:59:13.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3850000000000001e-06 2024-09-14T23:59:13.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:13.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917939+0000", 2024-09-14T23:59:13.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000238249 2024-09-14T23:59:13.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.644 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:13.644 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917961+0000", 2024-09-14T23:59:13.644 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2894000000000001e-05 2024-09-14T23:59:13.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:13.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917972+0000", 2024-09-14T23:59:13.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0261000000000001e-05 2024-09-14T23:59:13.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:13.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.918038+0000", 2024-09-14T23:59:13.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6223000000000006e-05 2024-09-14T23:59:13.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.645 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:13.645 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.918063+0000", 2024-09-14T23:59:13.645 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5252000000000001e-05 2024-09-14T23:59:13.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:59:13.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.921433+0000", 2024-09-14T23:59:13.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033699960000000001 2024-09-14T23:59:13.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.646 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:13.646 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.921492+0000", 2024-09-14T23:59:13.646 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9120000000000003e-05 2024-09-14T23:59:13.646 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.646 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:13.646 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.647 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(osd.1.0:1460 3.0 e122/23 3:00cc1e32:::rbd_data.11187b9f1032.0000000000000001:2f v 122'453, mlcod=118'449)", 2024-09-14T23:59:13.647 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:22.287423+0000", 2024-09-14T23:59:13.647 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 51.322728310999999, 2024-09-14T23:59:13.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0046041659999999998, 2024-09-14T23:59:13.647 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:13.647 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:13.647 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:13.647 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:13.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.647 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:13.647 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.287423+0000", 2024-09-14T23:59:13.647 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.648 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:13.648 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.287423+0000", 2024-09-14T23:59:13.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.648 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:13.648 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.287423+0000", 2024-09-14T23:59:13.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7500000000000001e-07 2024-09-14T23:59:13.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.648 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:13.648 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.287435+0000", 2024-09-14T23:59:13.648 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2006e-05 2024-09-14T23:59:13.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:13.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.287444+0000", 2024-09-14T23:59:13.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.465e-06 2024-09-14T23:59:13.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:13.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.287453+0000", 2024-09-14T23:59:13.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0100000000000001e-06 2024-09-14T23:59:13.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.649 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:13.649 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.287519+0000", 2024-09-14T23:59:13.649 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6149999999999995e-05 2024-09-14T23:59:13.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.650 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:13.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.287543+0000", 2024-09-14T23:59:13.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3682000000000001e-05 2024-09-14T23:59:13.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.650 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:59:13.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.291977+0000", 2024-09-14T23:59:13.650 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0044333829999999999 2024-09-14T23:59:13.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.650 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:13.650 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.292027+0000", 2024-09-14T23:59:13.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0195000000000003e-05 2024-09-14T23:59:13.651 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.651 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:13.651 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.651 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDPGPush(3.1e 126/125 [PushOp(3:7f7265c3:::rbd_data.11188d356229.000000000000000d:16, version: 58'124, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:7f7265c3:::rbd_data.11188d356229.000000000000000d:16@58'124, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 16=[]:{16: [16]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T23:59:13.651 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:26.552687+0000", 2024-09-14T23:59:13.651 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 47.057464047000003, 2024-09-14T23:59:13.651 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023212060000000001, 2024-09-14T23:59:13.651 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:13.651 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:13.651 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:13.651 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:13.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.652 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:13.652 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.552687+0000", 2024-09-14T23:59:13.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.652 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:13.652 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.552687+0000", 2024-09-14T23:59:13.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.652 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:13.652 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.552688+0000", 2024-09-14T23:59:13.652 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.47e-07 2024-09-14T23:59:13.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.653 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:13.653 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.553659+0000", 2024-09-14T23:59:13.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00097088899999999995 2024-09-14T23:59:13.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.653 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:13.653 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.553682+0000", 2024-09-14T23:59:13.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3862e-05 2024-09-14T23:59:13.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.653 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:13.653 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.553703+0000", 2024-09-14T23:59:13.653 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0310999999999999e-05 2024-09-14T23:59:13.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.654 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:13.654 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.553813+0000", 2024-09-14T23:59:13.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011051 2024-09-14T23:59:13.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.654 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:13.654 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.553828+0000", 2024-09-14T23:59:13.654 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4625e-05 2024-09-14T23:59:13.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.654 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:13.654 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.555008+0000", 2024-09-14T23:59:13.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001180162 2024-09-14T23:59:13.655 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.655 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:13.655 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.655 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(osd.1.0:1465 3.4 e127/23 3:265f7763:::rbd_data.1118f9c96a7.0000000000000004:2c v 127'508, mlcod=120'505)", 2024-09-14T23:59:13.655 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:27.318056+0000", 2024-09-14T23:59:13.655 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 46.292094624000001, 2024-09-14T23:59:13.655 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0049337449999999998, 2024-09-14T23:59:13.655 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:13.656 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:13.656 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:13.656 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:13.656 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.656 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:13.656 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.318056+0000", 2024-09-14T23:59:13.656 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.656 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.656 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:13.656 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.318056+0000", 2024-09-14T23:59:13.656 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.657 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.657 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:13.657 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.318057+0000", 2024-09-14T23:59:13.657 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9799999999999999e-07 2024-09-14T23:59:13.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.657 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.657 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:13.657 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.318061+0000", 2024-09-14T23:59:13.657 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7899999999999999e-06 2024-09-14T23:59:13.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.657 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.657 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:13.657 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.318074+0000", 2024-09-14T23:59:13.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2537999999999999e-05 2024-09-14T23:59:13.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:13.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.318083+0000", 2024-09-14T23:59:13.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3829999999999998e-06 2024-09-14T23:59:13.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.658 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:13.658 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.318165+0000", 2024-09-14T23:59:13.658 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1928999999999994e-05 2024-09-14T23:59:13.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.658 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:13.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.318199+0000", 2024-09-14T23:59:13.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3587000000000002e-05 2024-09-14T23:59:13.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:59:13.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.322939+0000", 2024-09-14T23:59:13.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0047397730000000001 2024-09-14T23:59:13.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.659 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.659 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:13.659 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.322990+0000", 2024-09-14T23:59:13.659 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1347000000000002e-05 2024-09-14T23:59:13.659 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.660 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:13.660 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.660 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.660 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:253435 3.5 3:a6fcd3e0:::rbd_data.1118e50f549.000000000000004a:head [delete] snapc 24=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-14T23:59:13.660 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:36.070548+0000", 2024-09-14T23:59:13.660 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.539603305, 2024-09-14T23:59:13.660 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022804000000000001, 2024-09-14T23:59:13.660 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:13.660 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:13.660 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:13.661 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:13.661 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:13.661 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:13.661 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 253435 2024-09-14T23:59:13.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.661 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:13.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.661 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:13.661 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.070548+0000", 2024-09-14T23:59:13.661 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.661 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.661 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:13.662 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.070548+0000", 2024-09-14T23:59:13.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.662 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:13.662 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.070549+0000", 2024-09-14T23:59:13.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6500000000000001e-06 2024-09-14T23:59:13.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.662 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:13.662 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.070555+0000", 2024-09-14T23:59:13.662 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1750000000000002e-06 2024-09-14T23:59:13.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.662 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:13.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.070571+0000", 2024-09-14T23:59:13.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5719999999999999e-05 2024-09-14T23:59:13.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:13.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.070580+0000", 2024-09-14T23:59:13.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0699999999999996e-06 2024-09-14T23:59:13.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.663 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:13.663 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.070658+0000", 2024-09-14T23:59:13.663 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7525000000000001e-05 2024-09-14T23:59:13.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.664 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:13.664 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.070799+0000", 2024-09-14T23:59:13.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014083499999999999 2024-09-14T23:59:13.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.664 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-14T23:59:13.664 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.070927+0000", 2024-09-14T23:59:13.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012853100000000001 2024-09-14T23:59:13.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.664 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:59:13.664 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.071666+0000", 2024-09-14T23:59:13.664 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00073853000000000005 2024-09-14T23:59:13.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.665 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:59:13.665 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.072750+0000", 2024-09-14T23:59:13.665 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001084738 2024-09-14T23:59:13.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.665 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:59:13.665 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.072791+0000", 2024-09-14T23:59:13.665 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0330999999999997e-05 2024-09-14T23:59:13.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.665 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.665 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:13.665 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.072828+0000", 2024-09-14T23:59:13.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7295000000000001e-05 2024-09-14T23:59:13.666 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.666 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:13.666 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.666 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.666 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:253690 3.e 3:725cbf00:::rbd_header.11186acd4a72:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-14T23:59:13.666 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:37.091642+0000", 2024-09-14T23:59:13.666 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 36.518509006999999, 2024-09-14T23:59:13.666 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0026525030000000001, 2024-09-14T23:59:13.666 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:13.666 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:13.666 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:13.666 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:13.667 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:13.667 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:13.667 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 253690 2024-09-14T23:59:13.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.667 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:13.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.667 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:13.667 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:37.091642+0000", 2024-09-14T23:59:13.667 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.667 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:13.667 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:37.091642+0000", 2024-09-14T23:59:13.667 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.668 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:13.668 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:37.091644+0000", 2024-09-14T23:59:13.668 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9829999999999999e-06 2024-09-14T23:59:13.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.668 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:13.668 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:37.091654+0000", 2024-09-14T23:59:13.668 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8619999999999998e-06 2024-09-14T23:59:13.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.668 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.668 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:13.668 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:37.091673+0000", 2024-09-14T23:59:13.668 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9094e-05 2024-09-14T23:59:13.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.669 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:13.669 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:37.091681+0000", 2024-09-14T23:59:13.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9449999999999993e-06 2024-09-14T23:59:13.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.669 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:13.669 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:37.091752+0000", 2024-09-14T23:59:13.669 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1468000000000001e-05 2024-09-14T23:59:13.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.669 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.669 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op must wait for map", 2024-09-14T23:59:13.669 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:37.091764+0000", 2024-09-14T23:59:13.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2067e-05 2024-09-14T23:59:13.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.670 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:13.670 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:37.093986+0000", 2024-09-14T23:59:13.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022213720000000001 2024-09-14T23:59:13.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.670 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:13.670 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:37.094082+0000", 2024-09-14T23:59:13.670 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5872999999999995e-05 2024-09-14T23:59:13.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.670 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.670 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:13.671 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:37.094294+0000", 2024-09-14T23:59:13.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021283900000000001 2024-09-14T23:59:13.671 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.671 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:13.671 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.671 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:254188 3.5 3:a13778a5:::rbd_header.1118ce574610:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e140)", 2024-09-14T23:59:13.671 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:40.008804+0000", 2024-09-14T23:59:13.671 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 33.601346710999998, 2024-09-14T23:59:13.671 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0044649980000000004, 2024-09-14T23:59:13.671 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:13.671 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:13.671 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:13.672 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:13.672 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:13.672 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:13.672 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254188 2024-09-14T23:59:13.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.672 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:13.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:13.672 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.008804+0000", 2024-09-14T23:59:13.672 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.672 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:13.673 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.008804+0000", 2024-09-14T23:59:13.673 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.673 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:13.673 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.008806+0000", 2024-09-14T23:59:13.673 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.824e-06 2024-09-14T23:59:13.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.673 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:13.673 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.008813+0000", 2024-09-14T23:59:13.673 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8889999999999996e-06 2024-09-14T23:59:13.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.674 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:13.674 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.008826+0000", 2024-09-14T23:59:13.674 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.29e-05 2024-09-14T23:59:13.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.674 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:13.674 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.008836+0000", 2024-09-14T23:59:13.674 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9660000000000008e-06 2024-09-14T23:59:13.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.674 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:13.674 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.008906+0000", 2024-09-14T23:59:13.674 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0165000000000003e-05 2024-09-14T23:59:13.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:13.675 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.009004+0000", 2024-09-14T23:59:13.675 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8412999999999994e-05 2024-09-14T23:59:13.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-14T23:59:13.675 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.009349+0000", 2024-09-14T23:59:13.675 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00034510200000000002 2024-09-14T23:59:13.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.675 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.675 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:59:13.675 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.010086+0000", 2024-09-14T23:59:13.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00073696000000000005 2024-09-14T23:59:13.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.676 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:59:13.676 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.013157+0000", 2024-09-14T23:59:13.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0030706470000000001 2024-09-14T23:59:13.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.676 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:59:13.676 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.013222+0000", 2024-09-14T23:59:13.676 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4479000000000004e-05 2024-09-14T23:59:13.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.677 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.677 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:13.677 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.013269+0000", 2024-09-14T23:59:13.677 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7652999999999999e-05 2024-09-14T23:59:13.677 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.677 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:13.677 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.678 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.678 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:254194 3.5 3:a13778a5:::rbd_header.1118ce574610:head [watch unwatch cookie 140670715072400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e140)", 2024-09-14T23:59:13.678 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:40.018920+0000", 2024-09-14T23:59:13.678 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 33.591231335000003, 2024-09-14T23:59:13.678 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023612640000000001, 2024-09-14T23:59:13.678 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:13.678 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:13.678 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:13.678 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:13.678 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:13.678 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:13.678 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254194 2024-09-14T23:59:13.678 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.679 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:13.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.679 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:13.679 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.018920+0000", 2024-09-14T23:59:13.679 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.679 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:13.679 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.018920+0000", 2024-09-14T23:59:13.679 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.679 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.679 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:13.679 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.018920+0000", 2024-09-14T23:59:13.680 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7700000000000004e-07 2024-09-14T23:59:13.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.680 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:13.680 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.018923+0000", 2024-09-14T23:59:13.680 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9840000000000001e-06 2024-09-14T23:59:13.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.680 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:13.680 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.018931+0000", 2024-09-14T23:59:13.680 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8089999999999993e-06 2024-09-14T23:59:13.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.681 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:13.681 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.018937+0000", 2024-09-14T23:59:13.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1770000000000003e-06 2024-09-14T23:59:13.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.681 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:13.681 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.019163+0000", 2024-09-14T23:59:13.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022563800000000001 2024-09-14T23:59:13.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.681 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:13.681 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.019233+0000", 2024-09-14T23:59:13.681 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9698999999999998e-05 2024-09-14T23:59:13.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.682 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-14T23:59:13.682 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.019383+0000", 2024-09-14T23:59:13.682 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015073800000000001 2024-09-14T23:59:13.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.682 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:59:13.682 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.020341+0000", 2024-09-14T23:59:13.682 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00095732900000000004 2024-09-14T23:59:13.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.682 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:59:13.682 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.021141+0000", 2024-09-14T23:59:13.682 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00080075600000000002 2024-09-14T23:59:13.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.683 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.683 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:59:13.683 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.021189+0000", 2024-09-14T23:59:13.683 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7318999999999999e-05 2024-09-14T23:59:13.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.683 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.683 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:13.683 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.021281+0000", 2024-09-14T23:59:13.683 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2038000000000002e-05 2024-09-14T23:59:13.683 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.683 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:13.683 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.683 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.684 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(osd.4.0:4196 3.13 e141/113 3:cecefa16:::rbd_data.1118c250b6de.0000000000000001:1f v 141'878, mlcod=138'874)", 2024-09-14T23:59:13.684 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:41.015011+0000", 2024-09-14T23:59:13.684 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 32.595140387999997, 2024-09-14T23:59:13.684 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025205499999999999, 2024-09-14T23:59:13.684 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:13.684 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:13.684 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:13.684 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:13.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.684 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:13.684 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:41.015011+0000", 2024-09-14T23:59:13.684 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.684 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.685 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:13.685 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:41.015011+0000", 2024-09-14T23:59:13.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.685 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:13.685 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:41.015011+0000", 2024-09-14T23:59:13.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9900000000000002e-07 2024-09-14T23:59:13.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.685 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.685 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:13.685 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:41.015018+0000", 2024-09-14T23:59:13.685 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2910000000000002e-06 2024-09-14T23:59:13.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.686 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:13.686 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:41.015033+0000", 2024-09-14T23:59:13.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4625e-05 2024-09-14T23:59:13.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.686 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:13.686 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:41.015043+0000", 2024-09-14T23:59:13.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0584999999999999e-05 2024-09-14T23:59:13.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.686 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:13.686 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:41.015114+0000", 2024-09-14T23:59:13.686 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0541000000000001e-05 2024-09-14T23:59:13.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.687 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:13.687 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:41.015138+0000", 2024-09-14T23:59:13.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3810000000000001e-05 2024-09-14T23:59:13.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.687 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:59:13.687 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:41.017481+0000", 2024-09-14T23:59:13.687 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023433619999999999 2024-09-14T23:59:13.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.687 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:13.687 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:41.017531+0000", 2024-09-14T23:59:13.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0037000000000002e-05 2024-09-14T23:59:13.688 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.688 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:13.688 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.688 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.688 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(osd.6.0:1718 3.1e e146/127 3:7a74d0d0:::rbd_data.11182ee2885e.0000000000000006:1a v 146'435, mlcod=145'432)", 2024-09-14T23:59:13.688 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:46.047607+0000", 2024-09-14T23:59:13.688 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.562544190000001, 2024-09-14T23:59:13.688 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0026434779999999999, 2024-09-14T23:59:13.688 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:13.688 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:13.688 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:13.688 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:13.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.689 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:13.689 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.047607+0000", 2024-09-14T23:59:13.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.689 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:13.689 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.047607+0000", 2024-09-14T23:59:13.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.689 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:13.689 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.047607+0000", 2024-09-14T23:59:13.689 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4700000000000002e-07 2024-09-14T23:59:13.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.690 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:13.690 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.047612+0000", 2024-09-14T23:59:13.690 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1959999999999999e-06 2024-09-14T23:59:13.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.690 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:13.690 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.047622+0000", 2024-09-14T23:59:13.690 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0169999999999999e-05 2024-09-14T23:59:13.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.690 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:13.690 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.047630+0000", 2024-09-14T23:59:13.691 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7249999999999997e-06 2024-09-14T23:59:13.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.691 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:13.691 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.047739+0000", 2024-09-14T23:59:13.691 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108625 2024-09-14T23:59:13.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.691 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:13.691 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.047756+0000", 2024-09-14T23:59:13.691 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6866000000000001e-05 2024-09-14T23:59:13.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.691 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.691 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:59:13.692 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.050197+0000", 2024-09-14T23:59:13.692 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024412689999999998 2024-09-14T23:59:13.692 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.692 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.692 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:13.692 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.050250+0000", 2024-09-14T23:59:13.692 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3279999999999998e-05 2024-09-14T23:59:13.692 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.692 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:13.692 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.692 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.692 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.692 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:255323 3.e 3:74b99397:::rbd_data.11182ee2885e.0000000000000004:head [delete] snapc 1a=[] ondisk+write+known_if_redirected+supports_pool_eio e146)", 2024-09-14T23:59:13.692 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:46.111665+0000", 2024-09-14T23:59:13.693 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.498485895000002, 2024-09-14T23:59:13.693 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038303019999999998, 2024-09-14T23:59:13.693 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:13.693 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:13.693 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:13.693 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:13.693 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:13.693 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:13.693 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 255323 2024-09-14T23:59:13.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.693 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:13.693 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.693 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:13.693 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.111665+0000", 2024-09-14T23:59:13.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:13.694 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.111665+0000", 2024-09-14T23:59:13.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:13.694 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.111667+0000", 2024-09-14T23:59:13.694 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3e-06 2024-09-14T23:59:13.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.694 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:13.695 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.111676+0000", 2024-09-14T23:59:13.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5699999999999993e-06 2024-09-14T23:59:13.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.695 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:13.695 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.111698+0000", 2024-09-14T23:59:13.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2263e-05 2024-09-14T23:59:13.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.695 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:13.695 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.111709+0000", 2024-09-14T23:59:13.695 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1087000000000001e-05 2024-09-14T23:59:13.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.695 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.696 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:13.696 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.111787+0000", 2024-09-14T23:59:13.696 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8212000000000003e-05 2024-09-14T23:59:13.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.696 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:13.696 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.111886+0000", 2024-09-14T23:59:13.696 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8645000000000002e-05 2024-09-14T23:59:13.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.696 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-14T23:59:13.696 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.112014+0000", 2024-09-14T23:59:13.696 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012753800000000001 2024-09-14T23:59:13.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.697 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:59:13.697 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.113790+0000", 2024-09-14T23:59:13.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00177669 2024-09-14T23:59:13.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.697 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:59:13.697 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.115396+0000", 2024-09-14T23:59:13.697 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001605216 2024-09-14T23:59:13.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.697 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:59:13.697 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.115451+0000", 2024-09-14T23:59:13.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5534000000000001e-05 2024-09-14T23:59:13.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.698 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:13.698 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.115495+0000", 2024-09-14T23:59:13.698 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4246999999999998e-05 2024-09-14T23:59:13.698 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.698 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:13.698 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.698 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:255325 3.1e e146/127 3:7a74d0d0:::rbd_data.11182ee2885e.0000000000000006:head v 146'436, mlcod=145'433)", 2024-09-14T23:59:13.698 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:46.112042+0000", 2024-09-14T23:59:13.698 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 27.498108958, 2024-09-14T23:59:13.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023640530000000001, 2024-09-14T23:59:13.699 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:13.699 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:13.699 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:13.699 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:13.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.699 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:13.699 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.112042+0000", 2024-09-14T23:59:13.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.699 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.699 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:13.699 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.112042+0000", 2024-09-14T23:59:13.699 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.700 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:13.700 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.112043+0000", 2024-09-14T23:59:13.700 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1200000000000001e-06 2024-09-14T23:59:13.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.700 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:13.700 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.112051+0000", 2024-09-14T23:59:13.700 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3310000000000004e-06 2024-09-14T23:59:13.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.700 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:13.700 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.112063+0000", 2024-09-14T23:59:13.700 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2028000000000001e-05 2024-09-14T23:59:13.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.701 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:13.701 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.112071+0000", 2024-09-14T23:59:13.701 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4000000000000003e-06 2024-09-14T23:59:13.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.701 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:13.701 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.112155+0000", 2024-09-14T23:59:13.701 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4064e-05 2024-09-14T23:59:13.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.701 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.701 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:13.701 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.112181+0000", 2024-09-14T23:59:13.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5817999999999999e-05 2024-09-14T23:59:13.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:59:13.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.114372+0000", 2024-09-14T23:59:13.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0021908129999999998 2024-09-14T23:59:13.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.702 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.702 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:13.702 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.114406+0000", 2024-09-14T23:59:13.702 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4479e-05 2024-09-14T23:59:13.702 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.702 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:13.703 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.703 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:258077 3.e e165/153 3:736dc012:::rbd_data.111814b482f8.0000000000000083:head v 165'515, mlcod=165'513)", 2024-09-14T23:59:13.703 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:59:05.197045+0000", 2024-09-14T23:59:13.703 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 8.4131059130000008, 2024-09-14T23:59:13.703 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038631440000000002, 2024-09-14T23:59:13.703 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:13.703 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:13.703 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:13.703 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:13.703 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.703 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:13.703 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:59:05.197045+0000", 2024-09-14T23:59:13.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.704 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:13.704 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:59:05.197045+0000", 2024-09-14T23:59:13.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.704 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:13.704 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:59:05.197046+0000", 2024-09-14T23:59:13.704 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4000000000000001e-07 2024-09-14T23:59:13.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.704 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:13.705 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:59:05.197052+0000", 2024-09-14T23:59:13.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6030000000000001e-06 2024-09-14T23:59:13.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.705 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:13.705 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:59:05.197063+0000", 2024-09-14T23:59:13.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0963e-05 2024-09-14T23:59:13.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.705 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:13.705 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:59:05.197070+0000", 2024-09-14T23:59:13.705 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9739999999999996e-06 2024-09-14T23:59:13.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:13.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:59:05.197122+0000", 2024-09-14T23:59:13.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1322000000000001e-05 2024-09-14T23:59:13.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:13.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:59:05.197145+0000", 2024-09-14T23:59:13.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3731999999999999e-05 2024-09-14T23:59:13.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.706 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.706 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:59:13.706 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:59:05.200853+0000", 2024-09-14T23:59:13.706 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037077999999999998 2024-09-14T23:59:13.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.707 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:13.707 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:59:05.200908+0000", 2024-09-14T23:59:13.707 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5009999999999997e-05 2024-09-14T23:59:13.707 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.707 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:13.707 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.707 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.707 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:258483 3.5 3:a51c63e5:::rbd_id.image_client.0-clone6:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e168)", 2024-09-14T23:59:13.707 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:59:07.990358+0000", 2024-09-14T23:59:13.707 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 5.6197926010000003, 2024-09-14T23:59:13.707 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0954007159999999, 2024-09-14T23:59:13.707 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:13.708 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:13.708 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:13.708 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:13.708 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:13.708 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:13.708 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 258483 2024-09-14T23:59:13.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.708 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:13.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.708 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:13.708 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:59:07.990358+0000", 2024-09-14T23:59:13.708 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.708 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:13.709 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:59:07.990358+0000", 2024-09-14T23:59:13.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:13.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:13.709 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:59:07.990361+0000", 2024-09-14T23:59:13.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.599e-06 2024-09-14T23:59:13.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.709 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:13.709 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:59:07.990368+0000", 2024-09-14T23:59:13.709 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9029999999999998e-06 2024-09-14T23:59:13.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.710 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:13.710 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:59:07.990382+0000", 2024-09-14T23:59:13.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3740999999999999e-05 2024-09-14T23:59:13.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.710 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:13.710 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:59:07.990393+0000", 2024-09-14T23:59:13.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1421e-05 2024-09-14T23:59:13.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.710 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:13.710 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:59:07.990470+0000", 2024-09-14T23:59:13.710 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6569999999999994e-05 2024-09-14T23:59:13.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:59:13.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:59:07.990480+0000", 2024-09-14T23:59:13.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0627e-05 2024-09-14T23:59:13.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:13.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:59:09.083846+0000", 2024-09-14T23:59:13.711 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.093365519 2024-09-14T23:59:13.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.711 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.711 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:13.711 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:59:09.084020+0000", 2024-09-14T23:59:13.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017424000000000001 2024-09-14T23:59:13.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.712 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.712 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3,5", 2024-09-14T23:59:13.712 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:59:09.084156+0000", 2024-09-14T23:59:13.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013616 2024-09-14T23:59:13.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.712 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.712 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:59:13.712 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:59:09.084998+0000", 2024-09-14T23:59:13.712 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00084217600000000004 2024-09-14T23:59:13.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.712 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.713 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:59:13.713 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:59:09.085383+0000", 2024-09-14T23:59:13.713 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038516899999999998 2024-09-14T23:59:13.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.713 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:59:13.713 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:59:09.085540+0000", 2024-09-14T23:59:13.713 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015627699999999999 2024-09-14T23:59:13.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.713 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:59:13.713 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:59:09.085612+0000", 2024-09-14T23:59:13.713 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2738999999999996e-05 2024-09-14T23:59:13.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:13.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:13.714 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:13.714 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:59:09.085759+0000", 2024-09-14T23:59:13.714 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000146575 2024-09-14T23:59:13.714 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.714 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:13.714 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.714 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:13.714 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:13.714 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:59:13.716 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T23:59:13.716 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:59:13.714+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:59:13.816 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:59:13.815+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:59:13.864 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:59:13.864 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:59:13.864 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:59:13.864 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:59:13.873 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T23:59:13.917 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:59:13.916+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:59:13.987 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is unset 2024-09-14T23:59:14.018 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:59:14.017+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:59:14.028 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:59:14.028 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:59:14.028 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:59:14.028 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:59:14.029 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:59:14.037 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T23:59:14.121 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:59:14.120+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:59:14.192 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:59:14.192 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:59:14.192 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:59:14.192 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:59:14.192 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.192 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5201 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2232320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:14.192 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:59:14.192 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 154.935854726, 2024-09-14T23:59:14.193 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057643228999999997, 2024-09-14T23:59:14.193 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:14.193 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:14.193 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:14.193 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:14.193 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:14.193 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:14.193 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5201 2024-09-14T23:59:14.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.193 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:14.193 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.193 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:14.194 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:59:14.194 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:14.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.194 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:14.194 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:59:14.194 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:14.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.194 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:14.194 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:59:14.195 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0699999999999998e-07 2024-09-14T23:59:14.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.195 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.195 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:14.195 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253422+0000", 2024-09-14T23:59:14.195 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.514e-06 2024-09-14T23:59:14.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.195 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.195 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:14.195 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253431+0000", 2024-09-14T23:59:14.195 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1919999999999994e-06 2024-09-14T23:59:14.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.196 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:14.196 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253436+0000", 2024-09-14T23:59:14.196 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5350000000000004e-06 2024-09-14T23:59:14.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.196 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:14.196 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310856+0000", 2024-09-14T23:59:14.196 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057419381999999998 2024-09-14T23:59:14.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.196 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.197 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:14.197 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310953+0000", 2024-09-14T23:59:14.197 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6774000000000006e-05 2024-09-14T23:59:14.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.197 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.197 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:14.197 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311062+0000", 2024-09-14T23:59:14.197 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000109425 2024-09-14T23:59:14.197 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:14.197 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:14.197 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:14.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.198 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.198 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5202 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2293760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:14.198 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:59:14.198 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 154.93582997799999, 2024-09-14T23:59:14.198 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057818255999999998, 2024-09-14T23:59:14.198 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:14.198 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:14.198 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:14.198 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:14.199 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:14.199 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:14.199 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5202 2024-09-14T23:59:14.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.199 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:14.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.199 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:14.199 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:59:14.199 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:14.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.199 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:14.200 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:59:14.200 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:14.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.200 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:14.200 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253573+0000", 2024-09-14T23:59:14.200 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012934000000000001 2024-09-14T23:59:14.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.200 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.200 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:14.201 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253576+0000", 2024-09-14T23:59:14.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5310000000000002e-06 2024-09-14T23:59:14.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.201 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:14.201 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253587+0000", 2024-09-14T23:59:14.201 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0522e-05 2024-09-14T23:59:14.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.201 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.201 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:14.201 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253595+0000", 2024-09-14T23:59:14.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8290000000000008e-06 2024-09-14T23:59:14.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.202 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:14.202 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311093+0000", 2024-09-14T23:59:14.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057498312000000003 2024-09-14T23:59:14.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.202 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.202 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:14.202 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311167+0000", 2024-09-14T23:59:14.202 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3582000000000001e-05 2024-09-14T23:59:14.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.203 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:14.203 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311262+0000", 2024-09-14T23:59:14.203 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5140000000000001e-05 2024-09-14T23:59:14.203 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:14.203 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:14.203 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:14.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.203 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.203 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5203 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2355200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:14.203 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:59:14.203 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 154.935667236, 2024-09-14T23:59:14.204 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057903938000000002, 2024-09-14T23:59:14.204 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:14.204 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:14.204 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:14.204 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:14.204 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:14.204 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:14.204 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5203 2024-09-14T23:59:14.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.204 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:14.204 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:14.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:59:14.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:14.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:14.205 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:59:14.205 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:14.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.205 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.205 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:14.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253902+0000", 2024-09-14T23:59:14.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029615400000000002 2024-09-14T23:59:14.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:14.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253906+0000", 2024-09-14T23:59:14.206 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9249999999999997e-06 2024-09-14T23:59:14.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.206 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:14.206 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253913+0000", 2024-09-14T23:59:14.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9099999999999999e-06 2024-09-14T23:59:14.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.207 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:14.207 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253917+0000", 2024-09-14T23:59:14.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9210000000000002e-06 2024-09-14T23:59:14.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.207 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.207 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:14.207 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311295+0000", 2024-09-14T23:59:14.207 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057378268000000003 2024-09-14T23:59:14.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:14.208 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311402+0000", 2024-09-14T23:59:14.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010644100000000001 2024-09-14T23:59:14.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.208 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:14.208 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311510+0000", 2024-09-14T23:59:14.208 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108319 2024-09-14T23:59:14.208 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:14.208 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:14.208 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:14.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.208 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.209 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5204 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2416640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:14.209 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:59:14.209 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 154.93534903899999, 2024-09-14T23:59:14.209 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057774860999999997, 2024-09-14T23:59:14.209 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:14.209 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:14.209 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:14.209 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:14.209 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:14.209 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:14.209 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5204 2024-09-14T23:59:14.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.209 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:14.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:14.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:59:14.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:14.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:14.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:59:14.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:14.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.210 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:14.210 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255903+0000", 2024-09-14T23:59:14.210 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001978807 2024-09-14T23:59:14.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:14.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255907+0000", 2024-09-14T23:59:14.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3610000000000001e-06 2024-09-14T23:59:14.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:14.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255914+0000", 2024-09-14T23:59:14.211 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.357e-06 2024-09-14T23:59:14.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.211 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:14.211 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255918+0000", 2024-09-14T23:59:14.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1779999999999996e-06 2024-09-14T23:59:14.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.212 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:14.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311541+0000", 2024-09-14T23:59:14.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055622697999999998 2024-09-14T23:59:14.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.212 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:14.212 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311614+0000", 2024-09-14T23:59:14.212 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3102999999999994e-05 2024-09-14T23:59:14.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.212 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.213 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:14.213 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311699+0000", 2024-09-14T23:59:14.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5357000000000003e-05 2024-09-14T23:59:14.213 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:14.213 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:14.213 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:14.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.213 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5211 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2846720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:14.213 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:59:14.213 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 154.93325082499999, 2024-09-14T23:59:14.213 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057528151, 2024-09-14T23:59:14.213 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:14.213 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:14.214 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:14.214 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:14.214 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:14.214 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:14.214 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5211 2024-09-14T23:59:14.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.214 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:14.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:14.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:59:14.214 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:14.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.214 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.214 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:14.214 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:59:14.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:14.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:14.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:59:14.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1399999999999998e-07 2024-09-14T23:59:14.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.215 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:14.215 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256024+0000", 2024-09-14T23:59:14.215 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.114e-06 2024-09-14T23:59:14.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:14.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256028+0000", 2024-09-14T23:59:14.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0840000000000002e-06 2024-09-14T23:59:14.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:14.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256031+0000", 2024-09-14T23:59:14.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6819999999999999e-06 2024-09-14T23:59:14.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.216 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.216 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:14.216 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313317+0000", 2024-09-14T23:59:14.216 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057286295000000001 2024-09-14T23:59:14.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:14.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313403+0000", 2024-09-14T23:59:14.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6287999999999998e-05 2024-09-14T23:59:14.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.217 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:14.217 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313551+0000", 2024-09-14T23:59:14.217 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000147374 2024-09-14T23:59:14.217 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:14.217 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:14.217 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:14.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.218 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5212 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2908160~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:14.218 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:59:14.218 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 154.93323963099999, 2024-09-14T23:59:14.218 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057735679999999998, 2024-09-14T23:59:14.218 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:14.218 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:14.218 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:14.218 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:14.218 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:14.218 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:14.218 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5212 2024-09-14T23:59:14.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.218 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:14.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:14.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:59:14.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:14.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:14.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:59:14.219 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:14.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.219 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:14.219 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:59:14.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0900000000000001e-07 2024-09-14T23:59:14.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:14.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256035+0000", 2024-09-14T23:59:14.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1200000000000001e-06 2024-09-14T23:59:14.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.220 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.220 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:14.220 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256038+0000", 2024-09-14T23:59:14.220 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.05e-06 2024-09-14T23:59:14.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:14.221 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256041+0000", 2024-09-14T23:59:14.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.667e-06 2024-09-14T23:59:14.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:14.221 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313584+0000", 2024-09-14T23:59:14.221 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057543484999999998 2024-09-14T23:59:14.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.221 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.221 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:14.222 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313668+0000", 2024-09-14T23:59:14.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3807000000000006e-05 2024-09-14T23:59:14.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.222 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:14.222 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313770+0000", 2024-09-14T23:59:14.222 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010134200000000001 2024-09-14T23:59:14.222 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:14.222 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:14.222 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:14.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.222 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.222 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5213 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2969600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:14.223 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:59:14.223 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 154.933229544, 2024-09-14T23:59:14.223 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058145467999999999, 2024-09-14T23:59:14.223 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:14.223 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:14.223 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:14.223 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:14.223 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:14.223 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:14.223 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5213 2024-09-14T23:59:14.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.224 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:14.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.224 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:14.224 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:59:14.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:14.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.224 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.224 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:14.224 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:59:14.224 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:14.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.225 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:14.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257901+0000", 2024-09-14T23:59:14.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018567519999999999 2024-09-14T23:59:14.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.225 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:14.225 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257904+0000", 2024-09-14T23:59:14.225 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4850000000000001e-06 2024-09-14T23:59:14.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:14.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257912+0000", 2024-09-14T23:59:14.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2090000000000005e-06 2024-09-14T23:59:14.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:14.226 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257918+0000", 2024-09-14T23:59:14.226 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6899999999999997e-06 2024-09-14T23:59:14.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.226 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:14.227 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313816+0000", 2024-09-14T23:59:14.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055898075999999998 2024-09-14T23:59:14.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:14.227 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.314082+0000", 2024-09-14T23:59:14.227 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000265386 2024-09-14T23:59:14.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.227 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:14.227 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.314189+0000", 2024-09-14T23:59:14.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010787 2024-09-14T23:59:14.228 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:14.228 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:14.228 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:14.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.228 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5218 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3276800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:14.228 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:59:14.228 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 154.93130253499999, 2024-09-14T23:59:14.228 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057497557999999997, 2024-09-14T23:59:14.229 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:14.229 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:14.229 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:14.229 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:14.229 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:14.229 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:14.229 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5218 2024-09-14T23:59:14.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.229 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:14.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:14.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:59:14.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:14.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:14.230 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:59:14.230 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:14.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.230 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.230 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:14.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:59:14.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1899999999999999e-07 2024-09-14T23:59:14.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.231 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:14.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257972+0000", 2024-09-14T23:59:14.231 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.217e-06 2024-09-14T23:59:14.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.231 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:14.231 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257975+0000", 2024-09-14T23:59:14.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9359999999999999e-06 2024-09-14T23:59:14.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.232 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:14.232 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257978+0000", 2024-09-14T23:59:14.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1049999999999999e-06 2024-09-14T23:59:14.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.232 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.232 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:14.232 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315331+0000", 2024-09-14T23:59:14.232 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057352093999999999 2024-09-14T23:59:14.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.233 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:14.233 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315384+0000", 2024-09-14T23:59:14.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3780999999999999e-05 2024-09-14T23:59:14.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.233 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:14.233 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315468+0000", 2024-09-14T23:59:14.233 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4206e-05 2024-09-14T23:59:14.233 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:14.234 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:14.234 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:14.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.234 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5219 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3338240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:14.234 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:59:14.234 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 154.931291859, 2024-09-14T23:59:14.234 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057637764000000001, 2024-09-14T23:59:14.234 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:14.234 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:14.234 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:14.235 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:14.235 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:14.235 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:14.235 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5219 2024-09-14T23:59:14.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.235 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:14.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.235 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:14.235 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:59:14.235 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:14.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.236 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:14.236 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:59:14.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:14.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.236 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:14.236 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:59:14.236 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.03e-07 2024-09-14T23:59:14.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:14.237 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257983+0000", 2024-09-14T23:59:14.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9199999999999999e-07 2024-09-14T23:59:14.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:14.237 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257986+0000", 2024-09-14T23:59:14.237 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0860000000000002e-06 2024-09-14T23:59:14.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.237 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:14.237 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257989+0000", 2024-09-14T23:59:14.238 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6589999999999999e-06 2024-09-14T23:59:14.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.238 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:14.238 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315509+0000", 2024-09-14T23:59:14.238 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057520262000000003 2024-09-14T23:59:14.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.238 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:14.238 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315554+0000", 2024-09-14T23:59:14.238 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5015999999999997e-05 2024-09-14T23:59:14.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.238 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:14.239 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315619+0000", 2024-09-14T23:59:14.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5545999999999999e-05 2024-09-14T23:59:14.239 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:14.239 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:14.239 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:14.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.239 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.239 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5220 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3399680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:14.239 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:59:14.239 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 154.93128190300001, 2024-09-14T23:59:14.239 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057792128999999998, 2024-09-14T23:59:14.239 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:14.239 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:14.239 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:14.240 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:14.240 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:14.240 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:14.240 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5220 2024-09-14T23:59:14.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.240 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:14.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.240 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:14.240 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:59:14.240 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:14.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.240 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:14.240 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:59:14.241 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:14.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.241 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.241 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:14.241 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:59:14.241 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1799999999999999e-07 2024-09-14T23:59:14.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.241 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.241 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:14.241 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257993+0000", 2024-09-14T23:59:14.241 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6200000000000006e-07 2024-09-14T23:59:14.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.242 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:14.242 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257996+0000", 2024-09-14T23:59:14.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9050000000000001e-06 2024-09-14T23:59:14.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.242 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:14.242 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257998+0000", 2024-09-14T23:59:14.242 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5950000000000001e-06 2024-09-14T23:59:14.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.242 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:14.243 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315672+0000", 2024-09-14T23:59:14.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057673637999999999 2024-09-14T23:59:14.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.243 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.243 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:14.243 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315734+0000", 2024-09-14T23:59:14.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2216000000000002e-05 2024-09-14T23:59:14.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.243 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.243 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:14.243 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315784+0000", 2024-09-14T23:59:14.243 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9595000000000002e-05 2024-09-14T23:59:14.243 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:14.244 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:14.244 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:14.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.244 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5221 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3461120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:14.244 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:59:14.244 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 154.93127218500001, 2024-09-14T23:59:14.244 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058003557999999997, 2024-09-14T23:59:14.244 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:14.244 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:14.244 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:14.244 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:14.244 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:14.245 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:14.245 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5221 2024-09-14T23:59:14.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.245 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:14.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.245 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:14.245 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:59:14.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:14.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.245 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:14.245 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:59:14.245 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:14.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:14.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259905+0000", 2024-09-14T23:59:14.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019035759999999999 2024-09-14T23:59:14.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:14.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259915+0000", 2024-09-14T23:59:14.246 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0518e-05 2024-09-14T23:59:14.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.246 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:14.246 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259925+0000", 2024-09-14T23:59:14.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7180000000000008e-06 2024-09-14T23:59:14.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:14.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259930+0000", 2024-09-14T23:59:14.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4769999999999997e-06 2024-09-14T23:59:14.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:14.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315820+0000", 2024-09-14T23:59:14.247 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055889960000000002 2024-09-14T23:59:14.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.247 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.247 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:14.247 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315914+0000", 2024-09-14T23:59:14.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4625999999999998e-05 2024-09-14T23:59:14.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.248 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:14.248 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.316005+0000", 2024-09-14T23:59:14.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0682999999999999e-05 2024-09-14T23:59:14.248 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:14.248 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:14.248 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:14.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.248 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5231 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 4075520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:14.248 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:59:14.248 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 154.92920345100001, 2024-09-14T23:59:14.248 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057524733000000002, 2024-09-14T23:59:14.249 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:14.249 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:14.249 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:14.249 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:14.249 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:14.249 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:14.249 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5231 2024-09-14T23:59:14.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.249 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:14.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.249 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:14.249 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:59:14.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:14.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.250 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:14.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:59:14.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:14.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.250 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:14.250 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261904+0000", 2024-09-14T23:59:14.250 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018338790000000001 2024-09-14T23:59:14.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:14.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261910+0000", 2024-09-14T23:59:14.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9689999999999999e-06 2024-09-14T23:59:14.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:14.251 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261923+0000", 2024-09-14T23:59:14.251 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3061000000000001e-05 2024-09-14T23:59:14.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.251 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:14.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261931+0000", 2024-09-14T23:59:14.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3159999999999997e-06 2024-09-14T23:59:14.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:14.252 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317456+0000", 2024-09-14T23:59:14.252 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055525219000000001 2024-09-14T23:59:14.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.252 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:14.253 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317520+0000", 2024-09-14T23:59:14.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4010000000000005e-05 2024-09-14T23:59:14.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.253 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.253 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:14.253 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317595+0000", 2024-09-14T23:59:14.253 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4278999999999998e-05 2024-09-14T23:59:14.253 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:14.253 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:14.253 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:14.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.254 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5236 3.8 3:1071668d:::rbd_data.1118de1168b.0000000000000092:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:14.254 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:59:14.254 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 154.92718998699999, 2024-09-14T23:59:14.254 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057499462000000001, 2024-09-14T23:59:14.254 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:14.254 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:14.254 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:14.254 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:14.254 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:14.254 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:14.254 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5236 2024-09-14T23:59:14.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.255 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:14.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:14.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:59:14.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:14.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.255 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:14.255 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:59:14.255 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:14.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:14.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262084+0000", 2024-09-14T23:59:14.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3399999999999999e-07 2024-09-14T23:59:14.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.256 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:14.256 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262087+0000", 2024-09-14T23:59:14.256 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2310000000000001e-06 2024-09-14T23:59:14.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:14.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262095+0000", 2024-09-14T23:59:14.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3799999999999996e-06 2024-09-14T23:59:14.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.257 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:14.257 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262101+0000", 2024-09-14T23:59:14.257 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0009999999999998e-06 2024-09-14T23:59:14.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:14.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317625+0000", 2024-09-14T23:59:14.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055524111000000001 2024-09-14T23:59:14.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:14.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317702+0000", 2024-09-14T23:59:14.258 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7533999999999993e-05 2024-09-14T23:59:14.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.258 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.258 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-14T23:59:14.258 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317892+0000", 2024-09-14T23:59:14.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018991400000000001 2024-09-14T23:59:14.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.259 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:59:14.259 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319407+0000", 2024-09-14T23:59:14.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015145250000000001 2024-09-14T23:59:14.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.259 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:59:14.259 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319481+0000", 2024-09-14T23:59:14.259 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3826999999999996e-05 2024-09-14T23:59:14.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:59:14.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319541+0000", 2024-09-14T23:59:14.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0470999999999998e-05 2024-09-14T23:59:14.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.260 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.260 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:14.260 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319583+0000", 2024-09-14T23:59:14.260 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1934000000000002e-05 2024-09-14T23:59:14.260 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:14.260 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:14.261 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:14.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.261 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:132622 3.1e 3.7a50b9e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-14T23:59:14.261 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:43.371285+0000", 2024-09-14T23:59:14.261 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 90.817988084999996, 2024-09-14T23:59:14.261 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.67037987200000004, 2024-09-14T23:59:14.261 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:14.261 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:14.261 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:14.262 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:14.262 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:14.262 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:14.262 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 132622 2024-09-14T23:59:14.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.262 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:14.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.262 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:14.262 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371285+0000", 2024-09-14T23:59:14.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:14.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.263 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:14.263 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371285+0000", 2024-09-14T23:59:14.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:14.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.263 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:14.263 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371286+0000", 2024-09-14T23:59:14.263 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6400000000000002e-07 2024-09-14T23:59:14.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:14.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371289+0000", 2024-09-14T23:59:14.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2059999999999999e-06 2024-09-14T23:59:14.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:14.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371299+0000", 2024-09-14T23:59:14.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0288e-05 2024-09-14T23:59:14.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.264 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:14.264 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371306+0000", 2024-09-14T23:59:14.264 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1650000000000003e-06 2024-09-14T23:59:14.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.265 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:14.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371617+0000", 2024-09-14T23:59:14.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000311043 2024-09-14T23:59:14.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.265 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:59:14.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371625+0000", 2024-09-14T23:59:14.265 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.833e-06 2024-09-14T23:59:14.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.265 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:14.265 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.041651+0000", 2024-09-14T23:59:14.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.67002580199999995 2024-09-14T23:59:14.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.266 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:14.266 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.041665+0000", 2024-09-14T23:59:14.266 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3971000000000001e-05 2024-09-14T23:59:14.266 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:14.266 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:14.266 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:14.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.266 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:169975 3.5 3:a704a2f0:::rbd_data.1118427720d0.0000000000000030:32 [read 969728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-14T23:59:14.266 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:03.433958+0000", 2024-09-14T23:59:14.266 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 70.755315729000003, 2024-09-14T23:59:14.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057564811, 2024-09-14T23:59:14.267 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:14.267 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:14.267 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:14.267 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:14.267 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:14.267 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:14.267 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 169975 2024-09-14T23:59:14.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.267 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:14.267 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.267 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:14.267 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.433958+0000", 2024-09-14T23:59:14.267 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:14.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:14.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.433958+0000", 2024-09-14T23:59:14.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:14.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:14.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.435096+0000", 2024-09-14T23:59:14.268 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001138708 2024-09-14T23:59:14.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.268 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:14.268 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.435101+0000", 2024-09-14T23:59:14.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5050000000000001e-06 2024-09-14T23:59:14.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:14.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.435111+0000", 2024-09-14T23:59:14.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5810000000000003e-06 2024-09-14T23:59:14.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.269 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:14.269 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.435120+0000", 2024-09-14T23:59:14.269 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0089999999999996e-06 2024-09-14T23:59:14.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:14.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.491345+0000", 2024-09-14T23:59:14.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056225479000000002 2024-09-14T23:59:14.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:14.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.491423+0000", 2024-09-14T23:59:14.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7990000000000004e-05 2024-09-14T23:59:14.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.270 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:14.270 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.491523+0000", 2024-09-14T23:59:14.270 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9538999999999997e-05 2024-09-14T23:59:14.271 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:14.271 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:14.271 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:14.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.271 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.271 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:172639 3.5 3:a704a2f0:::rbd_data.1118427720d0.0000000000000030:32 [read 1645568~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e107)", 2024-09-14T23:59:14.271 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:03.988365+0000", 2024-09-14T23:59:14.271 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 70.200908009000003, 2024-09-14T23:59:14.271 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057556213000000002, 2024-09-14T23:59:14.271 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:14.271 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:14.271 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:14.271 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:14.272 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:14.272 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:14.272 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 172639 2024-09-14T23:59:14.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.272 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:14.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:14.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.988365+0000", 2024-09-14T23:59:14.272 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:14.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.272 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.272 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:14.272 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.988365+0000", 2024-09-14T23:59:14.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:14.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:14.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.989904+0000", 2024-09-14T23:59:14.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015380859999999999 2024-09-14T23:59:14.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:14.273 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.989909+0000", 2024-09-14T23:59:14.273 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2800000000000003e-06 2024-09-14T23:59:14.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.273 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.273 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:14.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.989925+0000", 2024-09-14T23:59:14.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5781999999999999e-05 2024-09-14T23:59:14.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:14.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.989933+0000", 2024-09-14T23:59:14.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0090000000000007e-06 2024-09-14T23:59:14.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.274 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:14.274 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:04.045741+0000", 2024-09-14T23:59:14.274 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055808112999999999 2024-09-14T23:59:14.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.274 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:14.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:04.045812+0000", 2024-09-14T23:59:14.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1531000000000003e-05 2024-09-14T23:59:14.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.275 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:14.275 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:04.045922+0000", 2024-09-14T23:59:14.275 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000109412 2024-09-14T23:59:14.275 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:14.275 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:14.275 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:14.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.276 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184571 3.2 3:46f8d78b:::rbd_data.1118427720d0.0000000000000001:32 [read 1405952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:59:14.276 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.086404+0000", 2024-09-14T23:59:14.276 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 67.102869651999995, 2024-09-14T23:59:14.276 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059148573000000003, 2024-09-14T23:59:14.276 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:14.276 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:14.276 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:14.276 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:14.276 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:14.276 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:14.276 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184571 2024-09-14T23:59:14.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.276 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:14.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:14.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.086404+0000", 2024-09-14T23:59:14.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:14.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:14.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.086404+0000", 2024-09-14T23:59:14.277 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:14.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.277 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:14.277 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087902+0000", 2024-09-14T23:59:14.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001498625 2024-09-14T23:59:14.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:14.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087907+0000", 2024-09-14T23:59:14.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9130000000000002e-06 2024-09-14T23:59:14.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.278 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:14.278 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087919+0000", 2024-09-14T23:59:14.278 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1719000000000001e-05 2024-09-14T23:59:14.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.278 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:14.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087927+0000", 2024-09-14T23:59:14.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3699999999999995e-06 2024-09-14T23:59:14.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:14.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143711+0000", 2024-09-14T23:59:14.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055783396999999998 2024-09-14T23:59:14.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.279 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:14.279 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143792+0000", 2024-09-14T23:59:14.279 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1292999999999996e-05 2024-09-14T23:59:14.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.280 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:14.280 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145552+0000", 2024-09-14T23:59:14.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001760256 2024-09-14T23:59:14.280 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:14.280 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:14.280 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:14.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.280 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184579 3.2 3:43badc91:::rbd_data.1118427720d0.0000000000000002:32 [read 1196032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:59:14.280 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.088145+0000", 2024-09-14T23:59:14.280 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 67.101128247999995, 2024-09-14T23:59:14.280 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057643169000000001, 2024-09-14T23:59:14.281 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:14.281 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:14.281 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:14.281 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:14.281 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:14.281 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:14.281 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184579 2024-09-14T23:59:14.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.281 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:14.281 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.281 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:14.281 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.088145+0000", 2024-09-14T23:59:14.281 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:14.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:14.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.088145+0000", 2024-09-14T23:59:14.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:14.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:14.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089947+0000", 2024-09-14T23:59:14.282 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001801764 2024-09-14T23:59:14.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.282 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:14.282 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089952+0000", 2024-09-14T23:59:14.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8659999999999997e-06 2024-09-14T23:59:14.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:14.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089966+0000", 2024-09-14T23:59:14.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3929e-05 2024-09-14T23:59:14.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:14.283 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089974+0000", 2024-09-14T23:59:14.283 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1429999999999998e-06 2024-09-14T23:59:14.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.283 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.283 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:14.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145636+0000", 2024-09-14T23:59:14.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055662287999999997 2024-09-14T23:59:14.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:14.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145710+0000", 2024-09-14T23:59:14.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4166999999999996e-05 2024-09-14T23:59:14.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.284 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.284 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:14.284 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145788+0000", 2024-09-14T23:59:14.284 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8011999999999998e-05 2024-09-14T23:59:14.284 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:14.284 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:14.285 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:14.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.285 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.285 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:211800 3.12 3:493125e8:::rbd_data.1118c900944f.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-14T23:59:14.285 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:13.660130+0000", 2024-09-14T23:59:14.285 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 60.529143922000003, 2024-09-14T23:59:14.285 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.55364332000000005, 2024-09-14T23:59:14.285 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:14.285 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:14.285 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:14.285 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:14.285 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:14.285 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:14.285 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 211800 2024-09-14T23:59:14.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.286 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:14.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:14.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.660130+0000", 2024-09-14T23:59:14.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:14.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:14.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.660130+0000", 2024-09-14T23:59:14.286 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:14.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.286 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.286 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:14.286 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.660133+0000", 2024-09-14T23:59:14.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.202e-06 2024-09-14T23:59:14.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.287 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:14.287 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.668135+0000", 2024-09-14T23:59:14.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0080027220000000003 2024-09-14T23:59:14.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.287 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:14.287 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.668172+0000", 2024-09-14T23:59:14.287 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6295999999999999e-05 2024-09-14T23:59:14.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.287 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:14.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.668187+0000", 2024-09-14T23:59:14.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5537e-05 2024-09-14T23:59:14.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:14.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.668284+0000", 2024-09-14T23:59:14.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6297000000000003e-05 2024-09-14T23:59:14.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.288 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.288 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:59:14.288 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.668299+0000", 2024-09-14T23:59:14.288 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5091e-05 2024-09-14T23:59:14.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:14.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:14.195910+0000", 2024-09-14T23:59:14.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.52761157999999997 2024-09-14T23:59:14.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:14.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:14.196106+0000", 2024-09-14T23:59:14.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019597399999999999 2024-09-14T23:59:14.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.289 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.289 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 0", 2024-09-14T23:59:14.289 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:14.197262+0000", 2024-09-14T23:59:14.289 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001155744 2024-09-14T23:59:14.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:59:14.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:14.207440+0000", 2024-09-14T23:59:14.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010178347000000001 2024-09-14T23:59:14.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:59:14.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:14.213691+0000", 2024-09-14T23:59:14.290 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0062504420000000002 2024-09-14T23:59:14.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.290 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:59:14.290 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:14.213735+0000", 2024-09-14T23:59:14.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4385999999999998e-05 2024-09-14T23:59:14.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.291 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:14.291 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:14.213773+0000", 2024-09-14T23:59:14.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7701999999999999e-05 2024-09-14T23:59:14.291 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:14.291 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:14.291 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:14.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.291 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.291 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:251476 3.5 3:a1f6ebd6:::rbd_object_map.11189263ca38.000000000000002a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126)", 2024-09-14T23:59:14.291 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:26.388925+0000", 2024-09-14T23:59:14.291 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 47.800348888999999, 2024-09-14T23:59:14.291 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.062808638, 2024-09-14T23:59:14.292 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:14.292 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:14.292 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:14.292 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:14.292 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:14.292 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:14.292 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 251476 2024-09-14T23:59:14.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.292 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:14.292 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.292 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:14.292 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.388925+0000", 2024-09-14T23:59:14.292 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:14.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.293 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:14.293 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.388925+0000", 2024-09-14T23:59:14.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:14.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.293 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:14.293 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.388926+0000", 2024-09-14T23:59:14.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5200000000000001e-06 2024-09-14T23:59:14.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.293 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:14.293 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.388933+0000", 2024-09-14T23:59:14.293 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.827e-06 2024-09-14T23:59:14.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:14.294 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.388945+0000", 2024-09-14T23:59:14.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1701e-05 2024-09-14T23:59:14.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:14.294 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.388950+0000", 2024-09-14T23:59:14.294 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6080000000000003e-06 2024-09-14T23:59:14.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.294 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.294 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:14.294 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.388999+0000", 2024-09-14T23:59:14.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8711999999999997e-05 2024-09-14T23:59:14.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.295 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:59:14.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.389006+0000", 2024-09-14T23:59:14.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.063e-06 2024-09-14T23:59:14.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.295 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:14.295 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.451371+0000", 2024-09-14T23:59:14.295 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.062365255000000001 2024-09-14T23:59:14.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.295 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.295 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:14.296 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.451472+0000", 2024-09-14T23:59:14.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100626 2024-09-14T23:59:14.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:14.296 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:14.296 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:14.296 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.451733+0000", 2024-09-14T23:59:14.296 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026132599999999998 2024-09-14T23:59:14.296 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:14.296 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:14.296 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:14.296 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:14.296 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:14.296 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:59:14.297 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T23:59:14.298 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:59:14.296+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:59:14.299 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T23:59:14.398 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:59:14.397+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:59:14.415 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:59:14.415 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:59:14.415 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:59:14.415 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:59:14.424 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T23:59:14.463 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:59:14.463 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:59:14.480 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:59:14.480 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:59:14.497 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:59:14.497 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:59:14.499 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:59:14.498+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:59:14.513 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:59:14.513 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:59:14.530 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:59:14.530 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:59:14.547 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:59:14.548 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:59:14.564 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:59:14.564 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:59:14.573 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:14.573 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:14.582 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:59:14.582 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:59:14.583 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T23:59:14.627 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:59:14.626+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:59:14.691 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:14.692 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:14.700 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T23:59:14.701 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:59:14.699+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:59:14.801 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:59:14.800+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:59:14.849 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:14.849 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:14.858 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T23:59:14.902 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:59:14.901+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:59:14.964 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:14.965 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:14.973 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T23:59:15.003 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:59:15.001+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:59:15.089 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:15.089 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:15.098 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T23:59:15.142 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:59:15.141+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:59:15.204 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:59:15.203+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:59:15.206 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:15.206 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:15.215 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T23:59:15.306 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:59:15.304+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:59:15.362 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:15.363 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:15.371 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T23:59:15.406 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:59:15.405+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:59:15.479 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:15.479 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:15.488 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T23:59:15.531 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:59:15.530+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:59:15.595 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:15.595 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:15.604 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T23:59:15.645 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:59:15.644+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:59:15.709 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:59:15.707+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:59:15.751 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:15.751 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:15.760 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T23:59:15.809 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:59:15.807+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:59:15.867 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:15.867 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:15.876 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T23:59:15.917 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:59:15.916+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:59:16.008 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T23:59:16.009 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:16.009 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:16.011 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:59:16.010+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:59:16.024 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T23:59:16.111 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-14T23:59:16.110+0000 7face191b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33251) UID: 0 2024-09-14T23:59:16.184 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:16.184 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:16.193 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T23:59:16.237 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-09-14T23:59:16.236+0000 7facb5ae7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33223) UID: 0 2024-09-14T23:59:16.301 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:16.301 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:16.310 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T23:59:16.351 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-09-14T23:59:16.349+0000 7f34ac556640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33226) UID: 0 2024-09-14T23:59:16.414 INFO:tasks.ceph.osd.3.smithi042.stderr:2024-09-14T23:59:16.412+0000 7f3316c93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33229) UID: 0 2024-09-14T23:59:16.457 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:16.458 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:16.466 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T23:59:16.514 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:59:16.513+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:59:16.594 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T23:59:16.614 INFO:teuthology.orchestra.run.smithi042.stdout:ERROR: (22) Invalid argument 2024-09-14T23:59:16.614 INFO:teuthology.orchestra.run.smithi042.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T23:59:16.615 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:59:16.614+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:59:16.623 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T23:59:16.715 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-14T23:59:16.714+0000 7ff0bcf0a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33246) UID: 0 2024-09-14T23:59:16.718 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0: {} 2024-09-14T23:59:16.719 INFO:teuthology.orchestra.run.smithi042.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T23:59:16.737 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1: {} 2024-09-14T23:59:16.737 INFO:teuthology.orchestra.run.smithi042.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T23:59:16.753 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2: {} 2024-09-14T23:59:16.753 INFO:teuthology.orchestra.run.smithi042.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T23:59:16.770 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3: {} 2024-09-14T23:59:16.770 INFO:teuthology.orchestra.run.smithi042.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T23:59:16.775 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:59:16.776 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:59:16.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:59:16.776 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:59:16.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.776 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5882 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:16.776 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:59:16.776 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 157.39799601000001, 2024-09-14T23:59:16.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050549666, 2024-09-14T23:59:16.776 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:16.776 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:16.776 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:16.777 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:16.777 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:16.777 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:16.777 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5882 2024-09-14T23:59:16.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.777 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:16.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.777 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:16.777 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:59:16.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:16.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.777 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:16.777 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:59:16.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:16.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:16.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375841+0000", 2024-09-14T23:59:16.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8099999999999998e-07 2024-09-14T23:59:16.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:16.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375842+0000", 2024-09-14T23:59:16.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9900000000000009e-07 2024-09-14T23:59:16.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:16.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375845+0000", 2024-09-14T23:59:16.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.055e-06 2024-09-14T23:59:16.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:16.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375848+0000", 2024-09-14T23:59:16.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7609999999999999e-06 2024-09-14T23:59:16.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:16.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426381+0000", 2024-09-14T23:59:16.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050533392000000003 2024-09-14T23:59:16.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:16.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426390+0000", 2024-09-14T23:59:16.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0780000000000002e-06 2024-09-14T23:59:16.780 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:16.780 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:16.780 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:16.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.780 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5884 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:16.780 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:59:16.780 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 157.39798587000001, 2024-09-14T23:59:16.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050717439000000003, 2024-09-14T23:59:16.780 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:16.780 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:16.780 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:16.781 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:16.781 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:16.781 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:16.781 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5884 2024-09-14T23:59:16.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.781 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:16.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.781 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:16.781 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:59:16.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:16.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.781 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:16.781 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:59:16.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:16.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:16.782 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375851+0000", 2024-09-14T23:59:16.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0800000000000001e-07 2024-09-14T23:59:16.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:16.782 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375852+0000", 2024-09-14T23:59:16.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.02e-06 2024-09-14T23:59:16.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:16.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375855+0000", 2024-09-14T23:59:16.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9900000000000002e-06 2024-09-14T23:59:16.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:16.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375857+0000", 2024-09-14T23:59:16.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3499999999999999e-06 2024-09-14T23:59:16.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:16.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426556+0000", 2024-09-14T23:59:16.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050698139000000003 2024-09-14T23:59:16.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.784 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:16.784 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426568+0000", 2024-09-14T23:59:16.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2731999999999999e-05 2024-09-14T23:59:16.784 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:16.784 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:16.784 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:16.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.784 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5885 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:16.784 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:59:16.784 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 157.39796684500001, 2024-09-14T23:59:16.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050737333000000003, 2024-09-14T23:59:16.784 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:16.785 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:16.785 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:16.785 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:16.785 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:16.785 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:16.785 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5885 2024-09-14T23:59:16.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.785 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:16.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:16.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:59:16.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:16.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:16.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:59:16.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:16.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:16.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375870+0000", 2024-09-14T23:59:16.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5899999999999998e-07 2024-09-14T23:59:16.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:16.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375872+0000", 2024-09-14T23:59:16.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4219999999999999e-06 2024-09-14T23:59:16.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:16.787 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375878+0000", 2024-09-14T23:59:16.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4980000000000003e-06 2024-09-14T23:59:16.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:16.787 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375881+0000", 2024-09-14T23:59:16.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6110000000000001e-06 2024-09-14T23:59:16.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:16.787 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426596+0000", 2024-09-14T23:59:16.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050715320000000001 2024-09-14T23:59:16.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:16.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426607+0000", 2024-09-14T23:59:16.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1223000000000001e-05 2024-09-14T23:59:16.788 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:16.789 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:16.789 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:16.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.789 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5886 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:16.789 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:59:16.789 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 157.39795273199999, 2024-09-14T23:59:16.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050803395000000001, 2024-09-14T23:59:16.789 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:16.789 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:16.790 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:16.790 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:16.790 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:16.790 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:16.790 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5886 2024-09-14T23:59:16.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.790 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:16.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:16.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:59:16.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:16.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:16.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:59:16.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:16.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:16.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375884+0000", 2024-09-14T23:59:16.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2700000000000001e-07 2024-09-14T23:59:16.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:16.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375886+0000", 2024-09-14T23:59:16.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7060000000000001e-06 2024-09-14T23:59:16.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:16.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375890+0000", 2024-09-14T23:59:16.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.2259999999999999e-06 2024-09-14T23:59:16.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:16.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375893+0000", 2024-09-14T23:59:16.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0819999999999999e-06 2024-09-14T23:59:16.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:16.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426677+0000", 2024-09-14T23:59:16.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050784158000000003 2024-09-14T23:59:16.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.793 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:16.793 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426687+0000", 2024-09-14T23:59:16.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9960000000000005e-06 2024-09-14T23:59:16.793 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:16.793 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:16.793 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:16.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.794 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5887 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:16.794 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:59:16.794 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 157.39793064599999, 2024-09-14T23:59:16.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050833070000000001, 2024-09-14T23:59:16.794 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:16.794 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:16.794 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:16.794 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:16.794 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:16.794 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:16.794 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5887 2024-09-14T23:59:16.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.794 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:16.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:16.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:59:16.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:16.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:16.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:59:16.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:16.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:16.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375906+0000", 2024-09-14T23:59:16.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3299999999999998e-07 2024-09-14T23:59:16.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.796 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:16.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375908+0000", 2024-09-14T23:59:16.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7689999999999999e-06 2024-09-14T23:59:16.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.796 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:16.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375912+0000", 2024-09-14T23:59:16.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.535e-06 2024-09-14T23:59:16.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:16.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375916+0000", 2024-09-14T23:59:16.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.8979999999999998e-06 2024-09-14T23:59:16.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:16.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426729+0000", 2024-09-14T23:59:16.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050813453000000001 2024-09-14T23:59:16.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:16.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426739+0000", 2024-09-14T23:59:16.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0081999999999999e-05 2024-09-14T23:59:16.797 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:16.798 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:16.798 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:16.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.798 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5888 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:16.798 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:59:16.798 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 157.39791756899999, 2024-09-14T23:59:16.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050854652, 2024-09-14T23:59:16.798 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:16.798 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:16.798 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:16.798 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:16.798 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:16.798 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:16.798 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5888 2024-09-14T23:59:16.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.799 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:16.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:16.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:59:16.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:16.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:16.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:59:16.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:16.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:16.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375919+0000", 2024-09-14T23:59:16.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1400000000000001e-07 2024-09-14T23:59:16.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:16.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375921+0000", 2024-09-14T23:59:16.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.119e-06 2024-09-14T23:59:16.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:16.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375928+0000", 2024-09-14T23:59:16.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3170000000000004e-06 2024-09-14T23:59:16.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:16.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375931+0000", 2024-09-14T23:59:16.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6970000000000002e-06 2024-09-14T23:59:16.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:16.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426768+0000", 2024-09-14T23:59:16.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050837355000000001 2024-09-14T23:59:16.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.802 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:16.802 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426774+0000", 2024-09-14T23:59:16.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9499999999999998e-06 2024-09-14T23:59:16.802 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:16.802 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:16.802 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:16.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.802 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5889 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:16.802 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:59:16.802 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 157.39790064499999, 2024-09-14T23:59:16.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050885446000000001, 2024-09-14T23:59:16.802 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:16.802 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:16.803 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:16.803 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:16.803 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:16.803 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:16.803 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5889 2024-09-14T23:59:16.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.803 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:16.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:16.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:59:16.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:16.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:16.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:59:16.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:16.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:16.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375936+0000", 2024-09-14T23:59:16.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.36e-07 2024-09-14T23:59:16.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:16.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375940+0000", 2024-09-14T23:59:16.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.844e-06 2024-09-14T23:59:16.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:16.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375944+0000", 2024-09-14T23:59:16.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6720000000000002e-06 2024-09-14T23:59:16.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:16.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375946+0000", 2024-09-14T23:59:16.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5629999999999999e-06 2024-09-14T23:59:16.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:16.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426810+0000", 2024-09-14T23:59:16.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050863255000000003 2024-09-14T23:59:16.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:16.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426822+0000", 2024-09-14T23:59:16.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1875999999999999e-05 2024-09-14T23:59:16.806 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:16.806 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:16.806 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:16.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.806 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5890 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:16.806 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:59:16.807 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 157.397886833, 2024-09-14T23:59:16.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050917549999999999, 2024-09-14T23:59:16.807 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:16.807 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:16.807 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:16.807 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:16.807 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:16.807 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:16.807 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5890 2024-09-14T23:59:16.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.808 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:16.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:16.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:59:16.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:16.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:16.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:59:16.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:16.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:16.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375950+0000", 2024-09-14T23:59:16.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4299999999999999e-07 2024-09-14T23:59:16.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:16.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375951+0000", 2024-09-14T23:59:16.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1000000000000001e-06 2024-09-14T23:59:16.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:16.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375956+0000", 2024-09-14T23:59:16.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.3370000000000001e-06 2024-09-14T23:59:16.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:16.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375958+0000", 2024-09-14T23:59:16.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4820000000000001e-06 2024-09-14T23:59:16.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:16.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426858+0000", 2024-09-14T23:59:16.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050899666000000003 2024-09-14T23:59:16.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:16.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426867+0000", 2024-09-14T23:59:16.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7219999999999994e-06 2024-09-14T23:59:16.811 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:16.811 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:16.812 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:16.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.812 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5891 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:16.812 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:59:16.812 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 157.397875511, 2024-09-14T23:59:16.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050948067, 2024-09-14T23:59:16.812 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:16.812 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:16.812 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:16.812 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:16.812 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:16.812 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:16.813 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5891 2024-09-14T23:59:16.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.813 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:16.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:16.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:59:16.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:16.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:16.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:59:16.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:16.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:16.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375961+0000", 2024-09-14T23:59:16.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0800000000000001e-07 2024-09-14T23:59:16.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:16.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375962+0000", 2024-09-14T23:59:16.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9600000000000008e-07 2024-09-14T23:59:16.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:16.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375970+0000", 2024-09-14T23:59:16.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1600000000000001e-06 2024-09-14T23:59:16.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:16.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375972+0000", 2024-09-14T23:59:16.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4849999999999999e-06 2024-09-14T23:59:16.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:16.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426902+0000", 2024-09-14T23:59:16.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050929813999999997 2024-09-14T23:59:16.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:16.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426909+0000", 2024-09-14T23:59:16.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4039999999999998e-06 2024-09-14T23:59:16.816 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:16.816 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:16.816 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:16.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.816 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5892 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:16.816 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:59:16.816 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 157.397860654, 2024-09-14T23:59:16.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050965036999999998, 2024-09-14T23:59:16.816 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:16.816 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:16.816 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:16.816 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:16.816 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:16.817 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:16.817 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5892 2024-09-14T23:59:16.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.817 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:16.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:16.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:59:16.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:16.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:16.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:59:16.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:16.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:16.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375976+0000", 2024-09-14T23:59:16.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.34e-07 2024-09-14T23:59:16.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:16.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375978+0000", 2024-09-14T23:59:16.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1880000000000001e-06 2024-09-14T23:59:16.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:16.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375981+0000", 2024-09-14T23:59:16.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2399999999999999e-06 2024-09-14T23:59:16.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:16.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375984+0000", 2024-09-14T23:59:16.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9110000000000002e-06 2024-09-14T23:59:16.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:16.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426934+0000", 2024-09-14T23:59:16.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050949912 2024-09-14T23:59:16.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:16.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426941+0000", 2024-09-14T23:59:16.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5519999999999999e-06 2024-09-14T23:59:16.820 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:16.820 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:16.820 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:16.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.820 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5893 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:16.820 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:59:16.820 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 157.397850057, 2024-09-14T23:59:16.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050975131999999999, 2024-09-14T23:59:16.820 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:16.820 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:16.820 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:16.820 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:16.820 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:16.820 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:16.821 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5893 2024-09-14T23:59:16.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.821 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:16.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:16.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:59:16.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:16.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:16.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:59:16.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:16.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:16.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375987+0000", 2024-09-14T23:59:16.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2999999999999999e-07 2024-09-14T23:59:16.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:16.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375988+0000", 2024-09-14T23:59:16.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0410000000000001e-06 2024-09-14T23:59:16.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:16.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375992+0000", 2024-09-14T23:59:16.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.104e-06 2024-09-14T23:59:16.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:16.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375995+0000", 2024-09-14T23:59:16.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4310000000000001e-06 2024-09-14T23:59:16.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:16.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426958+0000", 2024-09-14T23:59:16.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050963642000000003 2024-09-14T23:59:16.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:16.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426962+0000", 2024-09-14T23:59:16.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6839999999999998e-06 2024-09-14T23:59:16.824 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:16.824 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:16.824 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:16.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.825 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5894 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:16.825 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:59:16.825 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 157.39783865699999, 2024-09-14T23:59:16.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050978333000000001, 2024-09-14T23:59:16.825 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:16.825 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:16.825 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:16.825 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:16.825 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:16.825 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:16.825 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5894 2024-09-14T23:59:16.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.825 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:16.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:16.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:59:16.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:16.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:16.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:59:16.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:16.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.826 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:16.826 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375998+0000", 2024-09-14T23:59:16.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.23e-07 2024-09-14T23:59:16.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:16.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.375999+0000", 2024-09-14T23:59:16.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9800000000000002e-07 2024-09-14T23:59:16.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:16.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376007+0000", 2024-09-14T23:59:16.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8539999999999997e-06 2024-09-14T23:59:16.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:16.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376011+0000", 2024-09-14T23:59:16.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.399e-06 2024-09-14T23:59:16.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:16.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426973+0000", 2024-09-14T23:59:16.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050962454999999997 2024-09-14T23:59:16.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:16.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426976+0000", 2024-09-14T23:59:16.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4039999999999999e-06 2024-09-14T23:59:16.828 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:16.828 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:16.828 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:16.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.829 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5895 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:16.829 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376014+0000", 2024-09-14T23:59:16.829 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 157.39782231300001, 2024-09-14T23:59:16.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050975951999999998, 2024-09-14T23:59:16.829 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:16.829 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:16.829 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:16.829 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:16.829 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:16.829 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:16.829 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5895 2024-09-14T23:59:16.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.829 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:16.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:16.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376014+0000", 2024-09-14T23:59:16.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:16.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:16.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376014+0000", 2024-09-14T23:59:16.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:16.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:16.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376015+0000", 2024-09-14T23:59:16.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2999999999999999e-07 2024-09-14T23:59:16.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:16.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376016+0000", 2024-09-14T23:59:16.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1659999999999999e-06 2024-09-14T23:59:16.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:16.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376024+0000", 2024-09-14T23:59:16.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8930000000000005e-06 2024-09-14T23:59:16.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:16.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376026+0000", 2024-09-14T23:59:16.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6869999999999999e-06 2024-09-14T23:59:16.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:16.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426987+0000", 2024-09-14T23:59:16.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050960601000000001 2024-09-14T23:59:16.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:16.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.426990+0000", 2024-09-14T23:59:16.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3749999999999999e-06 2024-09-14T23:59:16.832 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:16.832 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:16.832 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:16.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.833 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5896 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:16.833 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:59:16.833 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 157.39780657599999, 2024-09-14T23:59:16.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050975656000000001, 2024-09-14T23:59:16.833 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:16.833 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:16.833 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:16.833 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:16.833 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:16.833 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:16.833 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5896 2024-09-14T23:59:16.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.833 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:16.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:16.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:59:16.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:16.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:16.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:59:16.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:16.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:16.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376030+0000", 2024-09-14T23:59:16.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5100000000000001e-07 2024-09-14T23:59:16.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:16.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376032+0000", 2024-09-14T23:59:16.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.141e-06 2024-09-14T23:59:16.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:16.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376036+0000", 2024-09-14T23:59:16.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.25e-06 2024-09-14T23:59:16.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.835 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:16.835 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376038+0000", 2024-09-14T23:59:16.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5469999999999999e-06 2024-09-14T23:59:16.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:16.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427002+0000", 2024-09-14T23:59:16.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050964029000000001 2024-09-14T23:59:16.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:16.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427006+0000", 2024-09-14T23:59:16.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4379999999999999e-06 2024-09-14T23:59:16.836 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:16.836 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:16.836 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:16.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.837 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5897 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:16.837 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:59:16.837 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 157.39779477600001, 2024-09-14T23:59:16.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050978001000000002, 2024-09-14T23:59:16.837 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:16.837 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:16.837 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:16.837 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:16.837 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:16.837 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:16.837 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5897 2024-09-14T23:59:16.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.837 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:16.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:16.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:59:16.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:16.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:16.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:59:16.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:16.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:16.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376042+0000", 2024-09-14T23:59:16.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2600000000000001e-07 2024-09-14T23:59:16.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:16.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376043+0000", 2024-09-14T23:59:16.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.054e-06 2024-09-14T23:59:16.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:16.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376047+0000", 2024-09-14T23:59:16.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0659999999999997e-06 2024-09-14T23:59:16.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:16.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376050+0000", 2024-09-14T23:59:16.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.88e-06 2024-09-14T23:59:16.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:16.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427017+0000", 2024-09-14T23:59:16.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050966453000000002 2024-09-14T23:59:16.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:16.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427020+0000", 2024-09-14T23:59:16.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3220000000000001e-06 2024-09-14T23:59:16.840 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:16.840 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:16.840 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:16.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.841 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5898 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:16.841 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376053+0000", 2024-09-14T23:59:16.841 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 157.397783291, 2024-09-14T23:59:16.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050982024000000001, 2024-09-14T23:59:16.841 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:16.841 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:16.841 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:16.841 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:16.841 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:16.842 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:16.842 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5898 2024-09-14T23:59:16.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.842 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:16.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:16.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376053+0000", 2024-09-14T23:59:16.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:16.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:16.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376053+0000", 2024-09-14T23:59:16.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:16.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:16.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376278+0000", 2024-09-14T23:59:16.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022426400000000001 2024-09-14T23:59:16.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:16.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376282+0000", 2024-09-14T23:59:16.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4739999999999999e-06 2024-09-14T23:59:16.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:16.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376296+0000", 2024-09-14T23:59:16.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4229999999999999e-05 2024-09-14T23:59:16.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:16.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376303+0000", 2024-09-14T23:59:16.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2199999999999997e-06 2024-09-14T23:59:16.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:16.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427031+0000", 2024-09-14T23:59:16.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050728154999999997 2024-09-14T23:59:16.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:16.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427036+0000", 2024-09-14T23:59:16.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6809999999999997e-06 2024-09-14T23:59:16.845 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:16.845 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:16.845 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:16.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.845 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5899 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:16.845 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376338+0000", 2024-09-14T23:59:16.845 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 157.39749856700001, 2024-09-14T23:59:16.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050712100000000003, 2024-09-14T23:59:16.845 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:16.846 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:16.846 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:16.846 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:16.846 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:16.846 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:16.846 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5899 2024-09-14T23:59:16.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.846 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:16.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:16.846 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376338+0000", 2024-09-14T23:59:16.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:16.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.846 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:16.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376338+0000", 2024-09-14T23:59:16.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:16.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:16.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376339+0000", 2024-09-14T23:59:16.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9100000000000003e-07 2024-09-14T23:59:16.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:16.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376342+0000", 2024-09-14T23:59:16.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7e-06 2024-09-14T23:59:16.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:16.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376350+0000", 2024-09-14T23:59:16.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5280000000000003e-06 2024-09-14T23:59:16.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:16.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376356+0000", 2024-09-14T23:59:16.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6010000000000002e-06 2024-09-14T23:59:16.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:16.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427047+0000", 2024-09-14T23:59:16.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050690962999999999 2024-09-14T23:59:16.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:16.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427050+0000", 2024-09-14T23:59:16.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.517e-06 2024-09-14T23:59:16.849 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:16.849 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:16.849 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:16.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.849 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5900 3.4 3.b7496664 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:16.849 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.376366+0000", 2024-09-14T23:59:16.849 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 157.39747030800001, 2024-09-14T23:59:16.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.050697858999999998, 2024-09-14T23:59:16.849 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:16.850 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:16.850 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:16.850 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:16.850 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:16.850 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:16.850 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5900 2024-09-14T23:59:16.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.850 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:16.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:16.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376366+0000", 2024-09-14T23:59:16.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:16.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:16.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.376366+0000", 2024-09-14T23:59:16.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:16.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:16.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377650+0000", 2024-09-14T23:59:16.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001283834 2024-09-14T23:59:16.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:16.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377655+0000", 2024-09-14T23:59:16.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7230000000000004e-06 2024-09-14T23:59:16.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.851 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:16.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377667+0000", 2024-09-14T23:59:16.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1746000000000001e-05 2024-09-14T23:59:16.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:16.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.377676+0000", 2024-09-14T23:59:16.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3619999999999995e-06 2024-09-14T23:59:16.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.852 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:16.852 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427061+0000", 2024-09-14T23:59:16.852 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.049384755000000002 2024-09-14T23:59:16.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.853 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:16.853 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.427064+0000", 2024-09-14T23:59:16.853 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.439e-06 2024-09-14T23:59:16.853 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:16.853 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:16.853 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:16.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.853 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.853 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:132622 3.1e 3:79d0a5e0:::rbd_data.111847077d5e.0000000000000007:head [sparse-read 0~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T23:59:16.853 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:44.037901+0000", 2024-09-14T23:59:16.853 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 92.735935698999995, 2024-09-14T23:59:16.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1083420859999999, 2024-09-14T23:59:16.854 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:16.854 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:16.854 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:16.854 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:16.854 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:16.854 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:16.854 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 132622 2024-09-14T23:59:16.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.854 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:16.854 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.854 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:16.854 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037901+0000", 2024-09-14T23:59:16.854 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:16.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:16.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037901+0000", 2024-09-14T23:59:16.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:16.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:16.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037903+0000", 2024-09-14T23:59:16.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4619999999999999e-06 2024-09-14T23:59:16.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.855 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.855 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:16.855 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037910+0000", 2024-09-14T23:59:16.855 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7449999999999995e-06 2024-09-14T23:59:16.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:16.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037928+0000", 2024-09-14T23:59:16.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.7305000000000001e-05 2024-09-14T23:59:16.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:16.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.037939+0000", 2024-09-14T23:59:16.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1599e-05 2024-09-14T23:59:16.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.856 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.856 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:16.856 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.038049+0000", 2024-09-14T23:59:16.856 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000109577 2024-09-14T23:59:16.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op must wait for map", 2024-09-14T23:59:16.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.038057+0000", 2024-09-14T23:59:16.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1450000000000008e-06 2024-09-14T23:59:16.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:16.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.039937+0000", 2024-09-14T23:59:16.857 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00188024 2024-09-14T23:59:16.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.857 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:59:16.857 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.040039+0000", 2024-09-14T23:59:16.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000101934 2024-09-14T23:59:16.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:16.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:45.145805+0000", 2024-09-14T23:59:16.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.105765651 2024-09-14T23:59:16.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:16.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:45.145997+0000", 2024-09-14T23:59:16.858 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000192084 2024-09-14T23:59:16.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.858 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.858 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:16.858 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:45.146243+0000", 2024-09-14T23:59:16.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00024634400000000001 2024-09-14T23:59:16.859 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:16.859 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:16.859 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:16.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.859 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.859 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4172.0:5 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:59:16.859 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.482606+0000", 2024-09-14T23:59:16.859 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 69.291230717999994, 2024-09-14T23:59:16.859 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.49381641300000001, 2024-09-14T23:59:16.859 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:16.859 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:16.859 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:16.859 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:16.859 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4172", 2024-09-14T23:59:16.860 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/2542269203", 2024-09-14T23:59:16.860 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5 2024-09-14T23:59:16.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.860 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:16.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:16.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482606+0000", 2024-09-14T23:59:16.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:16.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.860 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.860 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:16.860 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482606+0000", 2024-09-14T23:59:16.860 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:16.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:16.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482607+0000", 2024-09-14T23:59:16.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3120000000000001e-06 2024-09-14T23:59:16.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:16.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482615+0000", 2024-09-14T23:59:16.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9300000000000003e-06 2024-09-14T23:59:16.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.861 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:16.861 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482632+0000", 2024-09-14T23:59:16.861 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6966999999999999e-05 2024-09-14T23:59:16.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:16.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482640+0000", 2024-09-14T23:59:16.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1820000000000006e-06 2024-09-14T23:59:16.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:16.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482703+0000", 2024-09-14T23:59:16.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2098000000000001e-05 2024-09-14T23:59:16.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.862 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.862 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:59:16.862 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.482713+0000", 2024-09-14T23:59:16.862 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0064000000000001e-05 2024-09-14T23:59:16.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:16.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.976307+0000", 2024-09-14T23:59:16.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.49359442999999997 2024-09-14T23:59:16.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:16.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:16.863 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:16.863 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.976422+0000", 2024-09-14T23:59:16.863 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011543 2024-09-14T23:59:16.863 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:16.863 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:16.863 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:16.863 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:16.863 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:16.863 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:59:16.865 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T23:59:16.865 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4: {} 2024-09-14T23:59:16.866 INFO:teuthology.orchestra.run.smithi042.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T23:59:16.866 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5: {} 2024-09-14T23:59:16.866 INFO:teuthology.orchestra.run.smithi042.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T23:59:16.867 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6: {} 2024-09-14T23:59:16.867 INFO:teuthology.orchestra.run.smithi042.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T23:59:16.867 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7: {} 2024-09-14T23:59:16.868 INFO:teuthology.orchestra.run.smithi042.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T23:59:16.868 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-14T23:59:16.863+0000 7ff2d7abb640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33243) UID: 0 2024-09-14T23:59:16.965 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-09-14T23:59:16.964+0000 7f987341d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33214) UID: 0 2024-09-14T23:59:16.991 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is unset 2024-09-14T23:59:17.023 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:59:17.024 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:59:17.024 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:59:17.024 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:59:17.033 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T23:59:17.067 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-14T23:59:17.065+0000 7f16147a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33249) UID: 0 2024-09-14T23:59:17.083 INFO:teuthology.orchestra.run.smithi042.stdout:All operations completed A-OK! 2024-09-14T23:59:17.097 DEBUG:teuthology.parallel:result is None 2024-09-14T23:59:17.097 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-09-14T23:59:17.106 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-14T23:59:17.113 INFO:tasks.thrashosds:joining thrashosds 2024-09-14T23:59:17.182 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:59:17.183 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:59:17.183 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:59:17.183 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:59:17.183 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:59:17.191 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T23:59:17.271 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-14T23:59:17.301 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:59:17.301 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:59:17.301 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:59:17.302 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:59:17.302 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.302 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:56107 3.0 e56/23 3:00c47890:::rbd_data.1118c4b3aebe.0000000000000009:head v 56'125, mlcod=52'123)", 2024-09-14T23:59:17.302 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.236123+0000", 2024-09-14T23:59:17.302 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 131.062644554, 2024-09-14T23:59:17.302 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0061273220000000002, 2024-09-14T23:59:17.302 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:17.302 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:17.302 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:17.302 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:17.302 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.303 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:17.303 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.236123+0000", 2024-09-14T23:59:17.303 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.303 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.303 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:17.303 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.236123+0000", 2024-09-14T23:59:17.303 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.303 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.303 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:17.304 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.236125+0000", 2024-09-14T23:59:17.304 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.313e-06 2024-09-14T23:59:17.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.304 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.304 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:17.304 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237428+0000", 2024-09-14T23:59:17.304 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0013029890000000001 2024-09-14T23:59:17.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.304 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.304 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:17.304 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237452+0000", 2024-09-14T23:59:17.305 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4090000000000001e-05 2024-09-14T23:59:17.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.305 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.305 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:17.305 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237462+0000", 2024-09-14T23:59:17.305 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0421999999999999e-05 2024-09-14T23:59:17.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.305 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.305 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:17.305 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237559+0000", 2024-09-14T23:59:17.305 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6378000000000005e-05 2024-09-14T23:59:17.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.306 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.306 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:17.306 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.237583+0000", 2024-09-14T23:59:17.306 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.4182e-05 2024-09-14T23:59:17.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.306 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.306 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:59:17.306 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.242205+0000", 2024-09-14T23:59:17.306 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0046217289999999998 2024-09-14T23:59:17.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.306 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.307 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:17.307 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.242251+0000", 2024-09-14T23:59:17.307 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.6218999999999999e-05 2024-09-14T23:59:17.307 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.307 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:17.307 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.307 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.307 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.307 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:56108 3.0 e56/23 3:00c47890:::rbd_data.1118c4b3aebe.0000000000000009:head v 56'127, mlcod=52'123)", 2024-09-14T23:59:17.307 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.239693+0000", 2024-09-14T23:59:17.307 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 131.059075084, 2024-09-14T23:59:17.307 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0043496209999999997, 2024-09-14T23:59:17.307 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:17.308 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:17.308 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:17.308 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:17.308 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.308 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:17.308 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239693+0000", 2024-09-14T23:59:17.308 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.308 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.308 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:17.308 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239693+0000", 2024-09-14T23:59:17.309 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.309 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.309 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.309 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:17.309 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239694+0000", 2024-09-14T23:59:17.309 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0499999999999999e-06 2024-09-14T23:59:17.309 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.309 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.309 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:17.309 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239702+0000", 2024-09-14T23:59:17.309 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1170000000000004e-06 2024-09-14T23:59:17.309 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.310 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:17.310 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239716+0000", 2024-09-14T23:59:17.310 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4306e-05 2024-09-14T23:59:17.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.310 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:17.310 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239724+0000", 2024-09-14T23:59:17.310 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0399999999999993e-06 2024-09-14T23:59:17.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.310 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.310 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:17.310 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239778+0000", 2024-09-14T23:59:17.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.4032e-05 2024-09-14T23:59:17.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.311 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:17.311 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.239801+0000", 2024-09-14T23:59:17.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2770000000000001e-05 2024-09-14T23:59:17.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.311 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:59:17.311 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.243990+0000", 2024-09-14T23:59:17.311 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0041890909999999998 2024-09-14T23:59:17.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.312 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:17.312 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.244043+0000", 2024-09-14T23:59:17.312 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2215000000000001e-05 2024-09-14T23:59:17.312 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.312 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:17.312 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.312 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.312 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.312 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:56385 3.9 e56/23 3:963d31c7:::rbd_data.1118c4b3aebe.0000000000000003:head v 56'87, mlcod=53'85)", 2024-09-14T23:59:17.312 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.283936+0000", 2024-09-14T23:59:17.312 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 131.01483217399999, 2024-09-14T23:59:17.312 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0063042080000000004, 2024-09-14T23:59:17.312 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:17.313 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:17.313 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:17.313 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:17.313 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.313 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:17.313 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.283936+0000", 2024-09-14T23:59:17.313 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.313 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.313 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.313 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:17.313 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.283936+0000", 2024-09-14T23:59:17.314 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.314 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.314 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.314 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:17.314 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.283937+0000", 2024-09-14T23:59:17.314 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1179999999999999e-06 2024-09-14T23:59:17.314 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.314 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.314 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:17.314 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286338+0000", 2024-09-14T23:59:17.314 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024010149999999998 2024-09-14T23:59:17.314 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.315 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:17.315 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286360+0000", 2024-09-14T23:59:17.315 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2212e-05 2024-09-14T23:59:17.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.315 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.315 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:17.315 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286370+0000", 2024-09-14T23:59:17.315 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5850000000000006e-06 2024-09-14T23:59:17.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.316 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.316 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:17.316 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286443+0000", 2024-09-14T23:59:17.316 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2757999999999997e-05 2024-09-14T23:59:17.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.316 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.317 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:17.317 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.286466+0000", 2024-09-14T23:59:17.317 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3844000000000001e-05 2024-09-14T23:59:17.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.317 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.317 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:59:17.317 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.290197+0000", 2024-09-14T23:59:17.317 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003730176 2024-09-14T23:59:17.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.317 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.317 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:17.317 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.290240+0000", 2024-09-14T23:59:17.317 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.35e-05 2024-09-14T23:59:17.318 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.318 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:17.318 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.318 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.318 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:56386 3.9 e56/23 3:963d31c7:::rbd_data.1118c4b3aebe.0000000000000003:head v 56'89, mlcod=53'85)", 2024-09-14T23:59:17.318 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:06.287578+0000", 2024-09-14T23:59:17.318 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 131.01119044800001, 2024-09-14T23:59:17.318 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033580250000000002, 2024-09-14T23:59:17.318 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:17.318 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:17.318 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:17.318 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:17.319 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.319 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:17.319 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287578+0000", 2024-09-14T23:59:17.319 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.319 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.319 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:17.319 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287578+0000", 2024-09-14T23:59:17.319 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.319 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.319 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:17.319 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287579+0000", 2024-09-14T23:59:17.319 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.64e-07 2024-09-14T23:59:17.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.320 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.320 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:17.320 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287586+0000", 2024-09-14T23:59:17.320 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.5310000000000004e-06 2024-09-14T23:59:17.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.320 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.320 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:17.320 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287597+0000", 2024-09-14T23:59:17.320 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0903999999999999e-05 2024-09-14T23:59:17.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.320 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.320 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:17.320 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.287605+0000", 2024-09-14T23:59:17.320 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7889999999999995e-06 2024-09-14T23:59:17.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.321 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.321 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:17.321 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.288437+0000", 2024-09-14T23:59:17.321 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000831914 2024-09-14T23:59:17.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.321 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.321 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:17.321 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.288460+0000", 2024-09-14T23:59:17.321 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3410000000000001e-05 2024-09-14T23:59:17.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.321 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.321 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:59:17.321 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.290883+0000", 2024-09-14T23:59:17.321 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024230599999999999 2024-09-14T23:59:17.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.322 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.322 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:17.322 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:06.290936+0000", 2024-09-14T23:59:17.322 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2453000000000001e-05 2024-09-14T23:59:17.322 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.322 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:17.322 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.322 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.322 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:64263 3.4 e57/23 3:20c6a768:::rbd_data.11188d356229.0000000000000001:head v 57'165, mlcod=57'163)", 2024-09-14T23:59:17.322 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:07.999031+0000", 2024-09-14T23:59:17.322 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 129.2997373, 2024-09-14T23:59:17.322 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023815210000000002, 2024-09-14T23:59:17.323 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:17.323 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:17.323 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:17.323 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:17.323 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.323 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:17.323 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999031+0000", 2024-09-14T23:59:17.323 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.323 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.323 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:17.323 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999031+0000", 2024-09-14T23:59:17.323 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.324 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:17.324 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999032+0000", 2024-09-14T23:59:17.324 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.176e-06 2024-09-14T23:59:17.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.324 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:17.324 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999660+0000", 2024-09-14T23:59:17.324 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00062817099999999996 2024-09-14T23:59:17.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.324 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:17.324 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999683+0000", 2024-09-14T23:59:17.324 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2816999999999999e-05 2024-09-14T23:59:17.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.325 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:17.325 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999693+0000", 2024-09-14T23:59:17.325 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0088999999999999e-05 2024-09-14T23:59:17.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.325 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:17.325 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999767+0000", 2024-09-14T23:59:17.325 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4056999999999999e-05 2024-09-14T23:59:17.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.325 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:17.325 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:07.999793+0000", 2024-09-14T23:59:17.326 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5573000000000001e-05 2024-09-14T23:59:17.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.326 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.326 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:59:17.326 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:08.001379+0000", 2024-09-14T23:59:17.326 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015859019999999999 2024-09-14T23:59:17.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.326 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.326 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:17.326 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:08.001412+0000", 2024-09-14T23:59:17.326 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3735999999999997e-05 2024-09-14T23:59:17.326 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.326 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:17.326 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.327 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.327 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:66570 3.0 e60/23 3:055350b3:::rbd_data.111873792d9b.0000000000000005:head v 60'162, mlcod=57'159)", 2024-09-14T23:59:17.327 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:11.277517+0000", 2024-09-14T23:59:17.327 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 126.021251144, 2024-09-14T23:59:17.327 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.003472199, 2024-09-14T23:59:17.327 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:17.327 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:17.327 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:17.327 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:17.327 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.327 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:17.327 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.277517+0000", 2024-09-14T23:59:17.327 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.328 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.328 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:17.328 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.277517+0000", 2024-09-14T23:59:17.328 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.328 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.328 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.328 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:17.328 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.277518+0000", 2024-09-14T23:59:17.328 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1379999999999999e-06 2024-09-14T23:59:17.328 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.328 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.328 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:17.328 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278504+0000", 2024-09-14T23:59:17.328 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00098617499999999994 2024-09-14T23:59:17.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.329 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.329 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:17.329 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278525+0000", 2024-09-14T23:59:17.329 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.09e-05 2024-09-14T23:59:17.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.329 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.329 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:17.329 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278534+0000", 2024-09-14T23:59:17.329 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6209999999999993e-06 2024-09-14T23:59:17.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.329 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.329 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:17.329 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278611+0000", 2024-09-14T23:59:17.330 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6926000000000002e-05 2024-09-14T23:59:17.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.330 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.330 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:17.330 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.278641+0000", 2024-09-14T23:59:17.330 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0861999999999997e-05 2024-09-14T23:59:17.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.330 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.330 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:59:17.330 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.280953+0000", 2024-09-14T23:59:17.330 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023114239999999999 2024-09-14T23:59:17.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.330 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.330 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:17.330 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:11.280989+0000", 2024-09-14T23:59:17.331 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6152999999999998e-05 2024-09-14T23:59:17.331 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.331 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:17.331 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.331 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.331 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:158983 3.4 e103/23 3:20e43b29:::rbd_data.1118b2f495e6.0000000000000012:head v 103'454, mlcod=103'452)", 2024-09-14T23:59:17.331 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:58.917699+0000", 2024-09-14T23:59:17.331 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 78.381069093999997, 2024-09-14T23:59:17.331 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037933799999999998, 2024-09-14T23:59:17.331 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:17.331 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:17.331 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:17.331 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:17.332 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.332 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:17.332 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917699+0000", 2024-09-14T23:59:17.332 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.332 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.332 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.332 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:17.332 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917699+0000", 2024-09-14T23:59:17.332 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.332 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.332 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.332 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:17.332 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917700+0000", 2024-09-14T23:59:17.333 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3850000000000001e-06 2024-09-14T23:59:17.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.333 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.333 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:17.333 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917939+0000", 2024-09-14T23:59:17.333 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000238249 2024-09-14T23:59:17.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.333 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.333 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:17.333 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917961+0000", 2024-09-14T23:59:17.333 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2894000000000001e-05 2024-09-14T23:59:17.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.333 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.333 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:17.334 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.917972+0000", 2024-09-14T23:59:17.334 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0261000000000001e-05 2024-09-14T23:59:17.334 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.334 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.334 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:17.334 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.918038+0000", 2024-09-14T23:59:17.334 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6223000000000006e-05 2024-09-14T23:59:17.334 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.334 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.334 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:17.334 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.918063+0000", 2024-09-14T23:59:17.334 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5252000000000001e-05 2024-09-14T23:59:17.334 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.334 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.334 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:59:17.335 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.921433+0000", 2024-09-14T23:59:17.335 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0033699960000000001 2024-09-14T23:59:17.335 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.335 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.335 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:17.335 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:58.921492+0000", 2024-09-14T23:59:17.335 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9120000000000003e-05 2024-09-14T23:59:17.335 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.335 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:17.335 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.335 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.335 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.335 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(osd.1.0:1460 3.0 e122/23 3:00cc1e32:::rbd_data.11187b9f1032.0000000000000001:2f v 122'453, mlcod=118'449)", 2024-09-14T23:59:17.336 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:22.287423+0000", 2024-09-14T23:59:17.336 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 55.011345437000003, 2024-09-14T23:59:17.336 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0046041659999999998, 2024-09-14T23:59:17.336 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:17.336 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:17.336 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:17.336 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:17.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.336 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:17.336 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.287423+0000", 2024-09-14T23:59:17.337 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.337 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.337 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:17.337 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.287423+0000", 2024-09-14T23:59:17.337 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.337 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.337 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:17.337 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.287423+0000", 2024-09-14T23:59:17.337 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.7500000000000001e-07 2024-09-14T23:59:17.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.337 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.338 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:17.338 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.287435+0000", 2024-09-14T23:59:17.338 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2006e-05 2024-09-14T23:59:17.338 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.338 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.338 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:17.338 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.287444+0000", 2024-09-14T23:59:17.338 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.465e-06 2024-09-14T23:59:17.338 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.338 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.338 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:17.338 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.287453+0000", 2024-09-14T23:59:17.338 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0100000000000001e-06 2024-09-14T23:59:17.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.339 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.339 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:17.339 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.287519+0000", 2024-09-14T23:59:17.339 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6149999999999995e-05 2024-09-14T23:59:17.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.339 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.339 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:17.339 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.287543+0000", 2024-09-14T23:59:17.339 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3682000000000001e-05 2024-09-14T23:59:17.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.339 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.339 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:59:17.340 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.291977+0000", 2024-09-14T23:59:17.340 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0044333829999999999 2024-09-14T23:59:17.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.340 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.340 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:17.340 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:22.292027+0000", 2024-09-14T23:59:17.340 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0195000000000003e-05 2024-09-14T23:59:17.340 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.340 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:17.340 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.340 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.340 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "MOSDPGPush(3.1e 126/125 [PushOp(3:7f7265c3:::rbd_data.11188d356229.000000000000000d:16, version: 58'124, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:7f7265c3:::rbd_data.11188d356229.000000000000000d:16@58'124, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 16=[]:{16: [16]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T23:59:17.340 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:26.552687+0000", 2024-09-14T23:59:17.341 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 50.746081173, 2024-09-14T23:59:17.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023212060000000001, 2024-09-14T23:59:17.341 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:17.341 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:17.341 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:17.341 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:17.341 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.341 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:17.341 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.552687+0000", 2024-09-14T23:59:17.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.341 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.341 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:17.341 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.552687+0000", 2024-09-14T23:59:17.341 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.342 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.342 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:17.342 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.552688+0000", 2024-09-14T23:59:17.342 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.47e-07 2024-09-14T23:59:17.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.342 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.342 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:17.342 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.553659+0000", 2024-09-14T23:59:17.342 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00097088899999999995 2024-09-14T23:59:17.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.342 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.342 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:17.343 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.553682+0000", 2024-09-14T23:59:17.343 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3862e-05 2024-09-14T23:59:17.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.343 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:17.343 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.553703+0000", 2024-09-14T23:59:17.343 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0310999999999999e-05 2024-09-14T23:59:17.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.343 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:17.343 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.553813+0000", 2024-09-14T23:59:17.343 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00011051 2024-09-14T23:59:17.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.344 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:17.344 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.553828+0000", 2024-09-14T23:59:17.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4625e-05 2024-09-14T23:59:17.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.344 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:17.344 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.555008+0000", 2024-09-14T23:59:17.344 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001180162 2024-09-14T23:59:17.344 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.344 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:17.344 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.344 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(osd.1.0:1465 3.4 e127/23 3:265f7763:::rbd_data.1118f9c96a7.0000000000000004:2c v 127'508, mlcod=120'505)", 2024-09-14T23:59:17.345 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:27.318056+0000", 2024-09-14T23:59:17.345 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 49.980711749999998, 2024-09-14T23:59:17.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0049337449999999998, 2024-09-14T23:59:17.345 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:17.345 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:17.345 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:17.345 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:17.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:17.345 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.318056+0000", 2024-09-14T23:59:17.345 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.345 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.345 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:17.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.318056+0000", 2024-09-14T23:59:17.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.346 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:17.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.318057+0000", 2024-09-14T23:59:17.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9799999999999999e-07 2024-09-14T23:59:17.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.346 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:17.346 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.318061+0000", 2024-09-14T23:59:17.346 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7899999999999999e-06 2024-09-14T23:59:17.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.347 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:17.347 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.318074+0000", 2024-09-14T23:59:17.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2537999999999999e-05 2024-09-14T23:59:17.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.347 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:17.347 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.318083+0000", 2024-09-14T23:59:17.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.3829999999999998e-06 2024-09-14T23:59:17.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.347 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:17.347 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.318165+0000", 2024-09-14T23:59:17.347 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1928999999999994e-05 2024-09-14T23:59:17.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.348 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:17.348 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.318199+0000", 2024-09-14T23:59:17.348 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3587000000000002e-05 2024-09-14T23:59:17.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.348 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:59:17.348 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.322939+0000", 2024-09-14T23:59:17.348 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0047397730000000001 2024-09-14T23:59:17.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.348 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:17.349 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:27.322990+0000", 2024-09-14T23:59:17.349 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1347000000000002e-05 2024-09-14T23:59:17.349 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.349 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:17.349 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.349 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:253435 3.5 3:a6fcd3e0:::rbd_data.1118e50f549.000000000000004a:head [delete] snapc 24=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-14T23:59:17.349 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:36.070548+0000", 2024-09-14T23:59:17.349 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 41.228220430999997, 2024-09-14T23:59:17.349 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022804000000000001, 2024-09-14T23:59:17.349 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:17.349 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:17.349 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:17.350 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:17.350 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:17.350 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:17.350 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 253435 2024-09-14T23:59:17.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.350 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:17.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.350 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:17.350 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.070548+0000", 2024-09-14T23:59:17.350 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.350 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.350 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:17.350 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.070548+0000", 2024-09-14T23:59:17.351 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.351 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:17.351 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.070549+0000", 2024-09-14T23:59:17.351 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6500000000000001e-06 2024-09-14T23:59:17.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.351 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:17.351 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.070555+0000", 2024-09-14T23:59:17.351 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1750000000000002e-06 2024-09-14T23:59:17.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.351 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:17.352 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.070571+0000", 2024-09-14T23:59:17.352 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5719999999999999e-05 2024-09-14T23:59:17.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.352 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.352 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:17.352 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.070580+0000", 2024-09-14T23:59:17.352 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0699999999999996e-06 2024-09-14T23:59:17.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.352 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.352 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:17.352 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.070658+0000", 2024-09-14T23:59:17.352 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7525000000000001e-05 2024-09-14T23:59:17.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.352 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:17.353 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.070799+0000", 2024-09-14T23:59:17.353 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00014083499999999999 2024-09-14T23:59:17.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-14T23:59:17.353 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.070927+0000", 2024-09-14T23:59:17.353 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012853100000000001 2024-09-14T23:59:17.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.353 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:59:17.353 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.071666+0000", 2024-09-14T23:59:17.353 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00073853000000000005 2024-09-14T23:59:17.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.354 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:59:17.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.072750+0000", 2024-09-14T23:59:17.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001084738 2024-09-14T23:59:17.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.354 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:59:17.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.072791+0000", 2024-09-14T23:59:17.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0330999999999997e-05 2024-09-14T23:59:17.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.354 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.354 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:17.354 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:36.072828+0000", 2024-09-14T23:59:17.354 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7295000000000001e-05 2024-09-14T23:59:17.355 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.355 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:17.355 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.355 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.355 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.355 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:253690 3.e 3:725cbf00:::rbd_header.11186acd4a72:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e137)", 2024-09-14T23:59:17.355 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:37.091642+0000", 2024-09-14T23:59:17.355 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 40.207126133000003, 2024-09-14T23:59:17.355 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0026525030000000001, 2024-09-14T23:59:17.355 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:17.355 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:17.355 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:17.355 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:17.355 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:17.356 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:17.356 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 253690 2024-09-14T23:59:17.356 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.356 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:17.356 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.356 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:17.356 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:37.091642+0000", 2024-09-14T23:59:17.356 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.356 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.356 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.356 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:17.356 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:37.091642+0000", 2024-09-14T23:59:17.356 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.356 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.357 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.357 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:17.357 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:37.091644+0000", 2024-09-14T23:59:17.357 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9829999999999999e-06 2024-09-14T23:59:17.357 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.357 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.357 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:17.357 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:37.091654+0000", 2024-09-14T23:59:17.357 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8619999999999998e-06 2024-09-14T23:59:17.357 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.357 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.357 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:17.357 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:37.091673+0000", 2024-09-14T23:59:17.357 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.9094e-05 2024-09-14T23:59:17.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.358 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:17.358 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:37.091681+0000", 2024-09-14T23:59:17.358 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.9449999999999993e-06 2024-09-14T23:59:17.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.358 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:17.358 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:37.091752+0000", 2024-09-14T23:59:17.358 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1468000000000001e-05 2024-09-14T23:59:17.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.358 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.358 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op must wait for map", 2024-09-14T23:59:17.358 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:37.091764+0000", 2024-09-14T23:59:17.359 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2067e-05 2024-09-14T23:59:17.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.359 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.359 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:17.359 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:37.093986+0000", 2024-09-14T23:59:17.359 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0022213720000000001 2024-09-14T23:59:17.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.359 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.359 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:17.359 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:37.094082+0000", 2024-09-14T23:59:17.359 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5872999999999995e-05 2024-09-14T23:59:17.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.359 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.359 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:17.359 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:37.094294+0000", 2024-09-14T23:59:17.360 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00021283900000000001 2024-09-14T23:59:17.360 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.360 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:17.360 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.360 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.360 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.360 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:254188 3.5 3:a13778a5:::rbd_header.1118ce574610:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e140)", 2024-09-14T23:59:17.360 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:40.008804+0000", 2024-09-14T23:59:17.360 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.289963837000002, 2024-09-14T23:59:17.360 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0044649980000000004, 2024-09-14T23:59:17.360 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:17.360 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:17.360 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:17.360 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:17.360 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:17.361 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:17.361 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254188 2024-09-14T23:59:17.361 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.361 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:17.361 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.361 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:17.361 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.008804+0000", 2024-09-14T23:59:17.361 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.361 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.361 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.361 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:17.361 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.008804+0000", 2024-09-14T23:59:17.362 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.362 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.362 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.362 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:17.362 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.008806+0000", 2024-09-14T23:59:17.362 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.824e-06 2024-09-14T23:59:17.362 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.362 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.362 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:17.362 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.008813+0000", 2024-09-14T23:59:17.362 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.8889999999999996e-06 2024-09-14T23:59:17.362 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.362 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.362 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:17.363 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.008826+0000", 2024-09-14T23:59:17.363 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.29e-05 2024-09-14T23:59:17.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.363 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.363 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:17.363 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.008836+0000", 2024-09-14T23:59:17.363 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9660000000000008e-06 2024-09-14T23:59:17.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.363 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.363 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:17.363 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.008906+0000", 2024-09-14T23:59:17.363 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0165000000000003e-05 2024-09-14T23:59:17.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.363 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.364 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:17.364 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.009004+0000", 2024-09-14T23:59:17.364 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8412999999999994e-05 2024-09-14T23:59:17.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.364 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.364 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-14T23:59:17.364 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.009349+0000", 2024-09-14T23:59:17.364 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00034510200000000002 2024-09-14T23:59:17.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.364 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.364 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:59:17.364 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.010086+0000", 2024-09-14T23:59:17.364 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00073696000000000005 2024-09-14T23:59:17.365 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.365 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.365 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:59:17.365 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.013157+0000", 2024-09-14T23:59:17.365 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0030706470000000001 2024-09-14T23:59:17.365 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.365 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.365 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:59:17.365 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.013222+0000", 2024-09-14T23:59:17.365 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4479000000000004e-05 2024-09-14T23:59:17.365 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.365 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.365 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:17.365 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.013269+0000", 2024-09-14T23:59:17.365 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7652999999999999e-05 2024-09-14T23:59:17.366 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.366 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:17.366 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.366 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.366 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.366 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:254194 3.5 3:a13778a5:::rbd_header.1118ce574610:head [watch unwatch cookie 140670715072400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e140)", 2024-09-14T23:59:17.366 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:40.018920+0000", 2024-09-14T23:59:17.366 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 37.279848461, 2024-09-14T23:59:17.366 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023612640000000001, 2024-09-14T23:59:17.366 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:17.366 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:17.366 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:17.366 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:17.366 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:17.367 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:17.367 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 254194 2024-09-14T23:59:17.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.367 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:17.367 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.367 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:17.367 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.018920+0000", 2024-09-14T23:59:17.367 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.367 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.367 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:17.367 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.018920+0000", 2024-09-14T23:59:17.367 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.368 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.368 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:17.368 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.018920+0000", 2024-09-14T23:59:17.368 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7700000000000004e-07 2024-09-14T23:59:17.368 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.368 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.368 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:17.368 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.018923+0000", 2024-09-14T23:59:17.368 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9840000000000001e-06 2024-09-14T23:59:17.368 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.368 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.368 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:17.368 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.018931+0000", 2024-09-14T23:59:17.368 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8089999999999993e-06 2024-09-14T23:59:17.368 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.369 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.369 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:17.369 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.018937+0000", 2024-09-14T23:59:17.369 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1770000000000003e-06 2024-09-14T23:59:17.369 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.369 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.369 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:17.369 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.019163+0000", 2024-09-14T23:59:17.369 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00022563800000000001 2024-09-14T23:59:17.369 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.369 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.369 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:17.369 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.019233+0000", 2024-09-14T23:59:17.369 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9698999999999998e-05 2024-09-14T23:59:17.370 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.370 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.370 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 5", 2024-09-14T23:59:17.370 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.019383+0000", 2024-09-14T23:59:17.370 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015073800000000001 2024-09-14T23:59:17.370 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.370 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.370 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:59:17.370 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.020341+0000", 2024-09-14T23:59:17.370 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00095732900000000004 2024-09-14T23:59:17.370 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.370 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.370 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:59:17.370 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.021141+0000", 2024-09-14T23:59:17.371 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00080075600000000002 2024-09-14T23:59:17.371 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.371 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.371 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:59:17.371 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.021189+0000", 2024-09-14T23:59:17.371 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.7318999999999999e-05 2024-09-14T23:59:17.371 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.371 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.371 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:17.371 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:40.021281+0000", 2024-09-14T23:59:17.371 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.2038000000000002e-05 2024-09-14T23:59:17.371 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.371 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:17.371 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.372 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.372 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.372 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(osd.4.0:4196 3.13 e141/113 3:cecefa16:::rbd_data.1118c250b6de.0000000000000001:1f v 141'878, mlcod=138'874)", 2024-09-14T23:59:17.372 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:41.015011+0000", 2024-09-14T23:59:17.372 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 36.283757514000001, 2024-09-14T23:59:17.372 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0025205499999999999, 2024-09-14T23:59:17.372 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:17.372 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:17.372 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:17.372 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:17.372 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.372 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:17.372 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:41.015011+0000", 2024-09-14T23:59:17.373 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.373 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.373 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.373 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:17.373 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:41.015011+0000", 2024-09-14T23:59:17.373 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.373 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.373 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.373 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:17.373 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:41.015011+0000", 2024-09-14T23:59:17.373 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9900000000000002e-07 2024-09-14T23:59:17.373 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.373 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.374 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:17.374 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:41.015018+0000", 2024-09-14T23:59:17.374 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2910000000000002e-06 2024-09-14T23:59:17.374 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.374 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.374 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:17.374 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:41.015033+0000", 2024-09-14T23:59:17.374 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.4625e-05 2024-09-14T23:59:17.374 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.374 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.374 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:17.374 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:41.015043+0000", 2024-09-14T23:59:17.374 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0584999999999999e-05 2024-09-14T23:59:17.374 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.375 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.375 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:17.375 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:41.015114+0000", 2024-09-14T23:59:17.375 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.0541000000000001e-05 2024-09-14T23:59:17.375 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.375 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.375 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:17.375 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:41.015138+0000", 2024-09-14T23:59:17.375 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3810000000000001e-05 2024-09-14T23:59:17.375 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.375 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.375 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:59:17.375 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:41.017481+0000", 2024-09-14T23:59:17.375 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023433619999999999 2024-09-14T23:59:17.376 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.376 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.376 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:17.376 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:41.017531+0000", 2024-09-14T23:59:17.376 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.0037000000000002e-05 2024-09-14T23:59:17.376 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.376 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:17.376 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.376 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.376 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.376 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(osd.6.0:1718 3.1e e146/127 3:7a74d0d0:::rbd_data.11182ee2885e.0000000000000006:1a v 146'435, mlcod=145'432)", 2024-09-14T23:59:17.376 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:46.047607+0000", 2024-09-14T23:59:17.376 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 31.251161316000001, 2024-09-14T23:59:17.377 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0026434779999999999, 2024-09-14T23:59:17.377 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:17.377 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:17.377 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:17.377 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:17.377 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.377 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:17.377 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.047607+0000", 2024-09-14T23:59:17.377 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.377 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.377 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.377 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:17.377 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.047607+0000", 2024-09-14T23:59:17.378 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.378 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.378 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.378 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:17.378 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.047607+0000", 2024-09-14T23:59:17.378 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4700000000000002e-07 2024-09-14T23:59:17.378 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.378 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.378 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:17.378 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.047612+0000", 2024-09-14T23:59:17.378 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1959999999999999e-06 2024-09-14T23:59:17.378 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.378 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.379 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:17.379 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.047622+0000", 2024-09-14T23:59:17.379 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0169999999999999e-05 2024-09-14T23:59:17.379 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.379 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.379 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:17.379 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.047630+0000", 2024-09-14T23:59:17.379 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7249999999999997e-06 2024-09-14T23:59:17.379 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.379 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.379 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:17.379 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.047739+0000", 2024-09-14T23:59:17.379 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108625 2024-09-14T23:59:17.379 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.380 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.380 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:17.380 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.047756+0000", 2024-09-14T23:59:17.380 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.6866000000000001e-05 2024-09-14T23:59:17.380 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.380 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.380 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:59:17.380 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.050197+0000", 2024-09-14T23:59:17.380 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0024412689999999998 2024-09-14T23:59:17.380 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.380 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.380 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:17.380 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.050250+0000", 2024-09-14T23:59:17.381 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3279999999999998e-05 2024-09-14T23:59:17.381 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.381 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:17.381 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.381 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.381 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.381 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:255323 3.e 3:74b99397:::rbd_data.11182ee2885e.0000000000000004:head [delete] snapc 1a=[] ondisk+write+known_if_redirected+supports_pool_eio e146)", 2024-09-14T23:59:17.381 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:46.111665+0000", 2024-09-14T23:59:17.381 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 31.187103020999999, 2024-09-14T23:59:17.381 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038303019999999998, 2024-09-14T23:59:17.381 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:17.381 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:17.381 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:17.381 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:17.382 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:17.382 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:17.382 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 255323 2024-09-14T23:59:17.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.382 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:17.382 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.382 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:17.382 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.111665+0000", 2024-09-14T23:59:17.382 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.382 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.382 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:17.382 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.111665+0000", 2024-09-14T23:59:17.382 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.383 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.383 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:17.383 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.111667+0000", 2024-09-14T23:59:17.383 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3e-06 2024-09-14T23:59:17.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.383 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.383 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:17.383 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.111676+0000", 2024-09-14T23:59:17.383 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5699999999999993e-06 2024-09-14T23:59:17.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.383 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.384 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:17.384 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.111698+0000", 2024-09-14T23:59:17.384 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.2263e-05 2024-09-14T23:59:17.384 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.384 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.384 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:17.384 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.111709+0000", 2024-09-14T23:59:17.384 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1087000000000001e-05 2024-09-14T23:59:17.384 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.384 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.384 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:17.384 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.111787+0000", 2024-09-14T23:59:17.384 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8212000000000003e-05 2024-09-14T23:59:17.384 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.385 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.385 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:17.385 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.111886+0000", 2024-09-14T23:59:17.385 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.8645000000000002e-05 2024-09-14T23:59:17.385 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.385 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.385 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-14T23:59:17.385 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.112014+0000", 2024-09-14T23:59:17.385 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012753800000000001 2024-09-14T23:59:17.385 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.385 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.385 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:59:17.385 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.113790+0000", 2024-09-14T23:59:17.386 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00177669 2024-09-14T23:59:17.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.386 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.386 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:59:17.386 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.115396+0000", 2024-09-14T23:59:17.386 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001605216 2024-09-14T23:59:17.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.386 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.386 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:59:17.386 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.115451+0000", 2024-09-14T23:59:17.386 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5534000000000001e-05 2024-09-14T23:59:17.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.386 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.387 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:17.387 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.115495+0000", 2024-09-14T23:59:17.387 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4246999999999998e-05 2024-09-14T23:59:17.387 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.387 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:17.387 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.387 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.387 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.387 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:255325 3.1e e146/127 3:7a74d0d0:::rbd_data.11182ee2885e.0000000000000006:head v 146'436, mlcod=145'433)", 2024-09-14T23:59:17.387 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:46.112042+0000", 2024-09-14T23:59:17.387 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 31.186726084, 2024-09-14T23:59:17.387 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0023640530000000001, 2024-09-14T23:59:17.387 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:17.388 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:17.388 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:17.388 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:17.388 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.388 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:17.388 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.112042+0000", 2024-09-14T23:59:17.388 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.388 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.388 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.388 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:17.388 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.112042+0000", 2024-09-14T23:59:17.388 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.388 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.389 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.389 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:17.389 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.112043+0000", 2024-09-14T23:59:17.389 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1200000000000001e-06 2024-09-14T23:59:17.389 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.389 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.389 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:17.389 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.112051+0000", 2024-09-14T23:59:17.389 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3310000000000004e-06 2024-09-14T23:59:17.389 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.389 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.389 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:17.389 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.112063+0000", 2024-09-14T23:59:17.390 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.2028000000000001e-05 2024-09-14T23:59:17.390 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.390 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.390 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:17.390 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.112071+0000", 2024-09-14T23:59:17.390 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4000000000000003e-06 2024-09-14T23:59:17.390 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.390 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.390 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:17.390 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.112155+0000", 2024-09-14T23:59:17.390 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4064e-05 2024-09-14T23:59:17.390 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.390 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.391 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:17.391 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.112181+0000", 2024-09-14T23:59:17.391 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5817999999999999e-05 2024-09-14T23:59:17.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.391 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.391 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:59:17.391 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.114372+0000", 2024-09-14T23:59:17.391 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0021908129999999998 2024-09-14T23:59:17.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.391 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.391 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:17.391 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:46.114406+0000", 2024-09-14T23:59:17.391 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4479e-05 2024-09-14T23:59:17.392 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.392 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:17.392 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.392 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.392 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.392 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_repop(client.4376.0:258077 3.e e165/153 3:736dc012:::rbd_data.111814b482f8.0000000000000083:head v 165'515, mlcod=165'513)", 2024-09-14T23:59:17.392 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:59:05.197045+0000", 2024-09-14T23:59:17.392 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 12.101723038999999, 2024-09-14T23:59:17.392 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0038631440000000002, 2024-09-14T23:59:17.392 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:17.392 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:17.392 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:17.392 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:17.393 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.393 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:17.393 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:59:05.197045+0000", 2024-09-14T23:59:17.393 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.393 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.393 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.393 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:17.393 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:59:05.197045+0000", 2024-09-14T23:59:17.393 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.393 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.393 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.393 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:17.393 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:59:05.197046+0000", 2024-09-14T23:59:17.393 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4000000000000001e-07 2024-09-14T23:59:17.394 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.394 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.394 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:17.394 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:59:05.197052+0000", 2024-09-14T23:59:17.394 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.6030000000000001e-06 2024-09-14T23:59:17.394 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.394 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.394 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:17.394 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:59:05.197063+0000", 2024-09-14T23:59:17.394 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0963e-05 2024-09-14T23:59:17.394 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.394 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.395 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:17.395 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:59:05.197070+0000", 2024-09-14T23:59:17.395 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9739999999999996e-06 2024-09-14T23:59:17.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.395 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:17.395 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:59:05.197122+0000", 2024-09-14T23:59:17.395 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.1322000000000001e-05 2024-09-14T23:59:17.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.395 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:17.395 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:59:05.197145+0000", 2024-09-14T23:59:17.395 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.3731999999999999e-05 2024-09-14T23:59:17.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.396 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.396 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:59:17.396 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:59:05.200853+0000", 2024-09-14T23:59:17.396 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0037077999999999998 2024-09-14T23:59:17.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.396 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.396 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:17.396 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:59:05.200908+0000", 2024-09-14T23:59:17.396 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5009999999999997e-05 2024-09-14T23:59:17.396 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.396 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:17.396 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.397 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.397 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:258483 3.5 3:a51c63e5:::rbd_id.image_client.0-clone6:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e168)", 2024-09-14T23:59:17.397 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:59:07.990358+0000", 2024-09-14T23:59:17.397 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 9.3084097270000008, 2024-09-14T23:59:17.397 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0954007159999999, 2024-09-14T23:59:17.397 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:17.397 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:17.397 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:17.397 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:17.397 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:17.397 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:17.397 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 258483 2024-09-14T23:59:17.397 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.398 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:17.398 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.398 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:17.398 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:59:07.990358+0000", 2024-09-14T23:59:17.398 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.398 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.398 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.398 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:17.398 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:59:07.990358+0000", 2024-09-14T23:59:17.398 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.398 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.398 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.398 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:17.399 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:59:07.990361+0000", 2024-09-14T23:59:17.399 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.599e-06 2024-09-14T23:59:17.399 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.399 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.399 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:17.399 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:59:07.990368+0000", 2024-09-14T23:59:17.399 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9029999999999998e-06 2024-09-14T23:59:17.399 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.399 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.399 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:17.399 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:59:07.990382+0000", 2024-09-14T23:59:17.399 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3740999999999999e-05 2024-09-14T23:59:17.399 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.399 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.400 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:17.400 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:59:07.990393+0000", 2024-09-14T23:59:17.400 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1421e-05 2024-09-14T23:59:17.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.400 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:17.400 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:59:07.990470+0000", 2024-09-14T23:59:17.400 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.6569999999999994e-05 2024-09-14T23:59:17.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.400 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.400 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:59:17.400 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:59:07.990480+0000", 2024-09-14T23:59:17.400 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0627e-05 2024-09-14T23:59:17.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.401 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.401 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:17.401 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:59:09.083846+0000", 2024-09-14T23:59:17.401 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.093365519 2024-09-14T23:59:17.401 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.401 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.401 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:17.401 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:59:09.084020+0000", 2024-09-14T23:59:17.401 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00017424000000000001 2024-09-14T23:59:17.401 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.401 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.401 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 3,5", 2024-09-14T23:59:17.401 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:59:09.084156+0000", 2024-09-14T23:59:17.401 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00013616 2024-09-14T23:59:17.401 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.402 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:59:17.402 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:59:09.084998+0000", 2024-09-14T23:59:17.402 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00084217600000000004 2024-09-14T23:59:17.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.402 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:59:17.402 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:59:09.085383+0000", 2024-09-14T23:59:17.402 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00038516899999999998 2024-09-14T23:59:17.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.402 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:59:17.402 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:59:09.085540+0000", 2024-09-14T23:59:17.402 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00015627699999999999 2024-09-14T23:59:17.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.403 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.403 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:59:17.403 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:59:09.085612+0000", 2024-09-14T23:59:17.403 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.2738999999999996e-05 2024-09-14T23:59:17.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.403 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.403 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:17.403 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:59:09.085759+0000", 2024-09-14T23:59:17.403 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000146575 2024-09-14T23:59:17.403 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.403 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:17.403 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.403 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.404 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:17.404 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:59:17.405 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T23:59:17.405 INFO:teuthology.orchestra.run.smithi042.stdout:mon.a: {} 2024-09-14T23:59:17.406 INFO:teuthology.orchestra.run.smithi042.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-14T23:59:17.406 INFO:teuthology.orchestra.run.smithi042.stdout:mon.b: {} 2024-09-14T23:59:17.406 INFO:teuthology.orchestra.run.smithi042.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-14T23:59:17.407 INFO:teuthology.orchestra.run.smithi042.stdout:mon.c: {} 2024-09-14T23:59:17.407 INFO:teuthology.orchestra.run.smithi042.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-14T23:59:17.417 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-14T23:59:17.515 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:59:17.515 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:59:17.515 INFO:teuthology.orchestra.run.smithi042.stdout: "num_ops": 0 2024-09-14T23:59:17.515 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:59:17.524 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T23:59:17.638 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:59:17.638 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [], 2024-09-14T23:59:17.638 INFO:teuthology.orchestra.run.smithi042.stdout: "complaint_time": 30, 2024-09-14T23:59:17.638 INFO:teuthology.orchestra.run.smithi042.stdout: "num_blocked_ops": 0 2024-09-14T23:59:17.638 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:59:17.647 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T23:59:17.757 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:59:17.757 INFO:teuthology.orchestra.run.smithi042.stdout: "size": 20, 2024-09-14T23:59:17.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 600, 2024-09-14T23:59:17.757 INFO:teuthology.orchestra.run.smithi042.stdout: "ops": [ 2024-09-14T23:59:17.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.757 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5201 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2232320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:17.757 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:59:17.757 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 158.500224132, 2024-09-14T23:59:17.757 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057643228999999997, 2024-09-14T23:59:17.757 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:17.757 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:17.758 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:17.758 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:17.758 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:17.758 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:17.758 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5201 2024-09-14T23:59:17.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.758 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:17.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.758 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:17.758 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:59:17.758 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.758 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.758 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:17.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:59:17.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:17.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253419+0000", 2024-09-14T23:59:17.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0699999999999998e-07 2024-09-14T23:59:17.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:17.759 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253422+0000", 2024-09-14T23:59:17.759 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.514e-06 2024-09-14T23:59:17.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.759 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:17.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253431+0000", 2024-09-14T23:59:17.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.1919999999999994e-06 2024-09-14T23:59:17.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.760 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:17.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253436+0000", 2024-09-14T23:59:17.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.5350000000000004e-06 2024-09-14T23:59:17.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.760 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:17.760 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310856+0000", 2024-09-14T23:59:17.760 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057419381999999998 2024-09-14T23:59:17.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.760 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.761 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:17.761 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.310953+0000", 2024-09-14T23:59:17.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6774000000000006e-05 2024-09-14T23:59:17.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.761 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:17.761 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311062+0000", 2024-09-14T23:59:17.761 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000109425 2024-09-14T23:59:17.761 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.761 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:17.761 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.761 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.761 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5202 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2293760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:17.762 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:59:17.762 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 158.50019938400001, 2024-09-14T23:59:17.762 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057818255999999998, 2024-09-14T23:59:17.762 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:17.762 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:17.762 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:17.762 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:17.762 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:17.762 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:17.762 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5202 2024-09-14T23:59:17.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.762 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:17.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.762 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:17.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:59:17.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:17.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253443+0000", 2024-09-14T23:59:17.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.763 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:17.763 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253573+0000", 2024-09-14T23:59:17.763 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00012934000000000001 2024-09-14T23:59:17.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.764 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:17.764 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253576+0000", 2024-09-14T23:59:17.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.5310000000000002e-06 2024-09-14T23:59:17.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.764 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:17.764 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253587+0000", 2024-09-14T23:59:17.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0522e-05 2024-09-14T23:59:17.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.764 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:17.764 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253595+0000", 2024-09-14T23:59:17.764 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8290000000000008e-06 2024-09-14T23:59:17.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.765 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:17.765 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311093+0000", 2024-09-14T23:59:17.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057498312000000003 2024-09-14T23:59:17.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.765 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:17.765 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311167+0000", 2024-09-14T23:59:17.765 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3582000000000001e-05 2024-09-14T23:59:17.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.765 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:17.765 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311262+0000", 2024-09-14T23:59:17.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5140000000000001e-05 2024-09-14T23:59:17.766 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.766 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:17.766 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.766 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5203 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2355200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:17.766 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:59:17.766 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 158.500036642, 2024-09-14T23:59:17.766 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057903938000000002, 2024-09-14T23:59:17.766 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:17.766 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:17.766 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:17.766 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:17.767 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:17.767 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:17.767 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5203 2024-09-14T23:59:17.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.767 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:17.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:17.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:59:17.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.767 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:17.767 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253606+0000", 2024-09-14T23:59:17.767 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:17.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253902+0000", 2024-09-14T23:59:17.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00029615400000000002 2024-09-14T23:59:17.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:17.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253906+0000", 2024-09-14T23:59:17.768 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9249999999999997e-06 2024-09-14T23:59:17.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.768 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.768 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:17.768 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253913+0000", 2024-09-14T23:59:17.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.9099999999999999e-06 2024-09-14T23:59:17.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.769 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:17.769 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253917+0000", 2024-09-14T23:59:17.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.9210000000000002e-06 2024-09-14T23:59:17.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.769 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:17.769 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311295+0000", 2024-09-14T23:59:17.769 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057378268000000003 2024-09-14T23:59:17.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.769 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.769 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:17.770 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311402+0000", 2024-09-14T23:59:17.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010644100000000001 2024-09-14T23:59:17.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.770 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:17.770 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311510+0000", 2024-09-14T23:59:17.770 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000108319 2024-09-14T23:59:17.770 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.770 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:17.770 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.770 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5204 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2416640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:17.770 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:59:17.771 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 158.49971844500001, 2024-09-14T23:59:17.771 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057774860999999997, 2024-09-14T23:59:17.771 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:17.771 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:17.771 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:17.771 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:17.771 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:17.771 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:17.771 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5204 2024-09-14T23:59:17.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.771 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:17.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.771 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:17.771 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:59:17.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:17.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.253924+0000", 2024-09-14T23:59:17.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.772 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:17.772 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255903+0000", 2024-09-14T23:59:17.772 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001978807 2024-09-14T23:59:17.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:17.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255907+0000", 2024-09-14T23:59:17.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.3610000000000001e-06 2024-09-14T23:59:17.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:17.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255914+0000", 2024-09-14T23:59:17.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.357e-06 2024-09-14T23:59:17.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.773 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.773 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:17.773 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.255918+0000", 2024-09-14T23:59:17.773 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1779999999999996e-06 2024-09-14T23:59:17.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:17.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311541+0000", 2024-09-14T23:59:17.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055622697999999998 2024-09-14T23:59:17.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:17.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311614+0000", 2024-09-14T23:59:17.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3102999999999994e-05 2024-09-14T23:59:17.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.774 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.774 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:17.774 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.311699+0000", 2024-09-14T23:59:17.774 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.5357000000000003e-05 2024-09-14T23:59:17.775 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.775 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:17.775 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.775 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.775 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5211 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2846720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:17.775 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:59:17.775 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 158.49762023100001, 2024-09-14T23:59:17.775 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057528151, 2024-09-14T23:59:17.775 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:17.775 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:17.775 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:17.775 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:17.775 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:17.775 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:17.776 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5211 2024-09-14T23:59:17.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.776 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:17.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:17.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:59:17.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.776 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:17.776 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:59:17.776 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.777 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:17.777 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256023+0000", 2024-09-14T23:59:17.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1399999999999998e-07 2024-09-14T23:59:17.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.777 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:17.777 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256024+0000", 2024-09-14T23:59:17.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.114e-06 2024-09-14T23:59:17.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.777 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:17.777 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256028+0000", 2024-09-14T23:59:17.777 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.0840000000000002e-06 2024-09-14T23:59:17.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:17.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256031+0000", 2024-09-14T23:59:17.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6819999999999999e-06 2024-09-14T23:59:17.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:17.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313317+0000", 2024-09-14T23:59:17.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057286295000000001 2024-09-14T23:59:17.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.778 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:17.778 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313403+0000", 2024-09-14T23:59:17.778 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.6287999999999998e-05 2024-09-14T23:59:17.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.779 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:17.779 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313551+0000", 2024-09-14T23:59:17.779 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000147374 2024-09-14T23:59:17.779 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.779 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:17.779 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.779 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5212 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2908160~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:17.779 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:59:17.779 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 158.49760903699999, 2024-09-14T23:59:17.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057735679999999998, 2024-09-14T23:59:17.780 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:17.780 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:17.780 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:17.780 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:17.780 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:17.780 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:17.780 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5212 2024-09-14T23:59:17.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.780 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:17.780 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.780 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:17.780 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:59:17.780 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.781 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:17.781 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:59:17.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.781 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:17.781 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256034+0000", 2024-09-14T23:59:17.781 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.0900000000000001e-07 2024-09-14T23:59:17.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.781 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:17.781 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256035+0000", 2024-09-14T23:59:17.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1200000000000001e-06 2024-09-14T23:59:17.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:17.782 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256038+0000", 2024-09-14T23:59:17.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.05e-06 2024-09-14T23:59:17.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:17.782 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256041+0000", 2024-09-14T23:59:17.782 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.667e-06 2024-09-14T23:59:17.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.782 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.782 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:17.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313584+0000", 2024-09-14T23:59:17.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057543484999999998 2024-09-14T23:59:17.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:17.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313668+0000", 2024-09-14T23:59:17.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3807000000000006e-05 2024-09-14T23:59:17.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.783 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.783 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:17.783 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313770+0000", 2024-09-14T23:59:17.783 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010134200000000001 2024-09-14T23:59:17.783 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.783 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:17.784 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.784 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.784 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5213 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 2969600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:17.784 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:59:17.784 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 158.49759895, 2024-09-14T23:59:17.784 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058145467999999999, 2024-09-14T23:59:17.784 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:17.784 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:17.784 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:17.784 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:17.784 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:17.784 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:17.785 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5213 2024-09-14T23:59:17.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.785 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:17.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:17.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:59:17.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.785 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:17.785 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.256044+0000", 2024-09-14T23:59:17.785 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.785 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:17.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257901+0000", 2024-09-14T23:59:17.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018567519999999999 2024-09-14T23:59:17.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:17.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257904+0000", 2024-09-14T23:59:17.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.4850000000000001e-06 2024-09-14T23:59:17.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.786 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.786 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:17.786 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257912+0000", 2024-09-14T23:59:17.786 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.2090000000000005e-06 2024-09-14T23:59:17.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:17.787 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257918+0000", 2024-09-14T23:59:17.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6899999999999997e-06 2024-09-14T23:59:17.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:17.787 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.313816+0000", 2024-09-14T23:59:17.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055898075999999998 2024-09-14T23:59:17.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.787 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.787 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:17.787 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.314082+0000", 2024-09-14T23:59:17.787 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000265386 2024-09-14T23:59:17.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.788 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:17.788 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.314189+0000", 2024-09-14T23:59:17.788 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00010787 2024-09-14T23:59:17.788 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.788 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:17.788 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.788 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.788 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5218 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3276800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:17.788 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:59:17.788 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 158.49567194100001, 2024-09-14T23:59:17.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057497557999999997, 2024-09-14T23:59:17.789 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:17.789 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:17.789 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:17.789 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:17.789 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:17.789 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:17.789 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5218 2024-09-14T23:59:17.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.789 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:17.789 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.789 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:17.789 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:59:17.789 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:17.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:59:17.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:17.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257971+0000", 2024-09-14T23:59:17.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1899999999999999e-07 2024-09-14T23:59:17.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.790 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:17.790 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257972+0000", 2024-09-14T23:59:17.790 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.217e-06 2024-09-14T23:59:17.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:17.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257975+0000", 2024-09-14T23:59:17.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9359999999999999e-06 2024-09-14T23:59:17.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:17.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257978+0000", 2024-09-14T23:59:17.791 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.1049999999999999e-06 2024-09-14T23:59:17.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.791 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:17.791 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315331+0000", 2024-09-14T23:59:17.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057352093999999999 2024-09-14T23:59:17.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:17.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315384+0000", 2024-09-14T23:59:17.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3780999999999999e-05 2024-09-14T23:59:17.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.792 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.792 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:17.792 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315468+0000", 2024-09-14T23:59:17.792 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.4206e-05 2024-09-14T23:59:17.792 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.792 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:17.792 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.793 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.793 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5219 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3338240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:17.793 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:59:17.793 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 158.495661265, 2024-09-14T23:59:17.793 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057637764000000001, 2024-09-14T23:59:17.793 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:17.793 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:17.793 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:17.793 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:17.793 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:17.793 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:17.793 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5219 2024-09-14T23:59:17.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.794 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:17.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:17.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:59:17.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:17.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:59:17.794 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.794 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.794 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:17.794 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257982+0000", 2024-09-14T23:59:17.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.03e-07 2024-09-14T23:59:17.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:17.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257983+0000", 2024-09-14T23:59:17.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9199999999999999e-07 2024-09-14T23:59:17.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:17.795 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257986+0000", 2024-09-14T23:59:17.795 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.0860000000000002e-06 2024-09-14T23:59:17.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.795 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.795 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:17.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257989+0000", 2024-09-14T23:59:17.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.6589999999999999e-06 2024-09-14T23:59:17.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.796 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:17.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315509+0000", 2024-09-14T23:59:17.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057520262000000003 2024-09-14T23:59:17.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.796 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:17.796 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315554+0000", 2024-09-14T23:59:17.796 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5015999999999997e-05 2024-09-14T23:59:17.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.796 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.797 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:17.797 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315619+0000", 2024-09-14T23:59:17.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.5545999999999999e-05 2024-09-14T23:59:17.797 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.797 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:17.797 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.797 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.797 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5220 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3399680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:17.797 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:59:17.797 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 158.49565130900001, 2024-09-14T23:59:17.797 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057792128999999998, 2024-09-14T23:59:17.797 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:17.797 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:17.798 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:17.798 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:17.798 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:17.798 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:17.798 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5220 2024-09-14T23:59:17.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.798 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:17.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.798 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:17.798 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:59:17.798 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.798 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:17.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:59:17.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:17.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257992+0000", 2024-09-14T23:59:17.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.1799999999999999e-07 2024-09-14T23:59:17.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.799 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:17.799 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257993+0000", 2024-09-14T23:59:17.799 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6200000000000006e-07 2024-09-14T23:59:17.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:17.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257996+0000", 2024-09-14T23:59:17.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.9050000000000001e-06 2024-09-14T23:59:17.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:17.800 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.257998+0000", 2024-09-14T23:59:17.800 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 2.5950000000000001e-06 2024-09-14T23:59:17.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.800 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.800 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:17.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315672+0000", 2024-09-14T23:59:17.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057673637999999999 2024-09-14T23:59:17.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:17.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315734+0000", 2024-09-14T23:59:17.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.2216000000000002e-05 2024-09-14T23:59:17.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.801 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.801 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:17.801 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315784+0000", 2024-09-14T23:59:17.801 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9595000000000002e-05 2024-09-14T23:59:17.801 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.801 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:17.802 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.802 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5221 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 3461120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:17.802 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:59:17.802 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 158.49564159100001, 2024-09-14T23:59:17.802 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.058003557999999997, 2024-09-14T23:59:17.802 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:17.802 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:17.802 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:17.802 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:17.802 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:17.802 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:17.802 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5221 2024-09-14T23:59:17.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.803 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:17.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:17.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:59:17.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:17.803 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.258001+0000", 2024-09-14T23:59:17.803 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.803 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.803 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:17.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259905+0000", 2024-09-14T23:59:17.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0019035759999999999 2024-09-14T23:59:17.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:17.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259915+0000", 2024-09-14T23:59:17.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0518e-05 2024-09-14T23:59:17.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.804 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:17.804 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259925+0000", 2024-09-14T23:59:17.804 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.7180000000000008e-06 2024-09-14T23:59:17.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.804 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:17.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.259930+0000", 2024-09-14T23:59:17.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4769999999999997e-06 2024-09-14T23:59:17.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:17.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315820+0000", 2024-09-14T23:59:17.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055889960000000002 2024-09-14T23:59:17.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.805 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:17.805 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.315914+0000", 2024-09-14T23:59:17.805 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.4625999999999998e-05 2024-09-14T23:59:17.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.806 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:17.806 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.316005+0000", 2024-09-14T23:59:17.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0682999999999999e-05 2024-09-14T23:59:17.806 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.806 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:17.806 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.806 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.806 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5231 3.18 3:1db1ce4c:::rbd_data.1118d8884d62.0000000000000009:3 [read 4075520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:17.806 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:59:17.806 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 158.493572857, 2024-09-14T23:59:17.806 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057524733000000002, 2024-09-14T23:59:17.806 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:17.806 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:17.807 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:17.807 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:17.807 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:17.807 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:17.807 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5231 2024-09-14T23:59:17.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.807 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:17.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:17.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:59:17.807 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.807 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:17.807 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.260070+0000", 2024-09-14T23:59:17.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:17.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261904+0000", 2024-09-14T23:59:17.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0018338790000000001 2024-09-14T23:59:17.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:17.808 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261910+0000", 2024-09-14T23:59:17.808 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.9689999999999999e-06 2024-09-14T23:59:17.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.808 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.808 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:17.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261923+0000", 2024-09-14T23:59:17.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3061000000000001e-05 2024-09-14T23:59:17.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:17.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.261931+0000", 2024-09-14T23:59:17.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3159999999999997e-06 2024-09-14T23:59:17.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.809 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:17.809 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317456+0000", 2024-09-14T23:59:17.809 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055525219000000001 2024-09-14T23:59:17.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.809 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:17.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317520+0000", 2024-09-14T23:59:17.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.4010000000000005e-05 2024-09-14T23:59:17.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.810 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:17.810 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317595+0000", 2024-09-14T23:59:17.810 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4278999999999998e-05 2024-09-14T23:59:17.810 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.810 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:17.810 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.810 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.810 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:5236 3.8 3:1071668d:::rbd_data.1118de1168b.0000000000000092:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T23:59:17.810 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:59:17.811 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 158.49155939299999, 2024-09-14T23:59:17.811 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057499462000000001, 2024-09-14T23:59:17.811 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:17.811 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:17.811 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:17.811 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:17.811 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:17.811 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:17.811 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 5236 2024-09-14T23:59:17.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.811 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:17.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.811 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:17.811 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:59:17.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:17.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262083+0000", 2024-09-14T23:59:17.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:17.812 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262084+0000", 2024-09-14T23:59:17.812 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.3399999999999999e-07 2024-09-14T23:59:17.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.812 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.812 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:17.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262087+0000", 2024-09-14T23:59:17.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2310000000000001e-06 2024-09-14T23:59:17.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:17.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262095+0000", 2024-09-14T23:59:17.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3799999999999996e-06 2024-09-14T23:59:17.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.813 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.813 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:17.813 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.262101+0000", 2024-09-14T23:59:17.813 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0009999999999998e-06 2024-09-14T23:59:17.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:17.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317625+0000", 2024-09-14T23:59:17.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055524111000000001 2024-09-14T23:59:17.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:17.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317702+0000", 2024-09-14T23:59:17.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7533999999999993e-05 2024-09-14T23:59:17.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.814 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.814 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 1", 2024-09-14T23:59:17.814 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.317892+0000", 2024-09-14T23:59:17.814 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00018991400000000001 2024-09-14T23:59:17.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:59:17.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319407+0000", 2024-09-14T23:59:17.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015145250000000001 2024-09-14T23:59:17.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:59:17.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319481+0000", 2024-09-14T23:59:17.815 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.3826999999999996e-05 2024-09-14T23:59:17.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.815 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.815 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:59:17.815 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319541+0000", 2024-09-14T23:59:17.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.0470999999999998e-05 2024-09-14T23:59:17.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.816 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:17.816 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:56:39.319583+0000", 2024-09-14T23:59:17.816 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.1934000000000002e-05 2024-09-14T23:59:17.816 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.816 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:17.816 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.816 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.816 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:132622 3.1e 3.7a50b9e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-14T23:59:17.816 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:57:43.371285+0000", 2024-09-14T23:59:17.816 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 94.382357490999993, 2024-09-14T23:59:17.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.67037987200000004, 2024-09-14T23:59:17.817 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:17.817 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:17.817 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "reached pg", 2024-09-14T23:59:17.817 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:17.817 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:17.817 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:17.817 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 132622 2024-09-14T23:59:17.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.817 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:17.817 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.817 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:17.817 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371285+0000", 2024-09-14T23:59:17.817 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:17.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371285+0000", 2024-09-14T23:59:17.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:17.818 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371286+0000", 2024-09-14T23:59:17.818 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6400000000000002e-07 2024-09-14T23:59:17.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.818 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.818 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:17.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371289+0000", 2024-09-14T23:59:17.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.2059999999999999e-06 2024-09-14T23:59:17.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:17.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371299+0000", 2024-09-14T23:59:17.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.0288e-05 2024-09-14T23:59:17.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.819 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:17.819 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371306+0000", 2024-09-14T23:59:17.819 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.1650000000000003e-06 2024-09-14T23:59:17.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:17.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371617+0000", 2024-09-14T23:59:17.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000311043 2024-09-14T23:59:17.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:59:17.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:43.371625+0000", 2024-09-14T23:59:17.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.833e-06 2024-09-14T23:59:17.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.820 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:17.820 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.041651+0000", 2024-09-14T23:59:17.820 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.67002580199999995 2024-09-14T23:59:17.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.821 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:17.821 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:57:44.041665+0000", 2024-09-14T23:59:17.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3971000000000001e-05 2024-09-14T23:59:17.821 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.821 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:17.821 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.821 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.821 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:169975 3.5 3:a704a2f0:::rbd_data.1118427720d0.0000000000000030:32 [read 969728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-14T23:59:17.821 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:03.433958+0000", 2024-09-14T23:59:17.821 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 74.319685135, 2024-09-14T23:59:17.821 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057564811, 2024-09-14T23:59:17.822 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:17.822 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:17.822 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:17.822 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:17.822 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:17.822 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:17.822 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 169975 2024-09-14T23:59:17.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.822 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:17.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.822 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:17.822 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.433958+0000", 2024-09-14T23:59:17.822 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:17.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.433958+0000", 2024-09-14T23:59:17.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:17.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.435096+0000", 2024-09-14T23:59:17.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001138708 2024-09-14T23:59:17.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.823 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.823 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:17.823 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.435101+0000", 2024-09-14T23:59:17.823 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.5050000000000001e-06 2024-09-14T23:59:17.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:17.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.435111+0000", 2024-09-14T23:59:17.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.5810000000000003e-06 2024-09-14T23:59:17.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:17.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.435120+0000", 2024-09-14T23:59:17.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.0089999999999996e-06 2024-09-14T23:59:17.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.824 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.824 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:17.824 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.491345+0000", 2024-09-14T23:59:17.824 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.056225479000000002 2024-09-14T23:59:17.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:17.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.491423+0000", 2024-09-14T23:59:17.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.7990000000000004e-05 2024-09-14T23:59:17.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.825 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.825 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:17.825 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.491523+0000", 2024-09-14T23:59:17.825 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.9538999999999997e-05 2024-09-14T23:59:17.825 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.825 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:17.825 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.826 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:172639 3.5 3:a704a2f0:::rbd_data.1118427720d0.0000000000000030:32 [read 1645568~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e107)", 2024-09-14T23:59:17.826 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:03.988365+0000", 2024-09-14T23:59:17.826 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 73.765277415, 2024-09-14T23:59:17.826 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057556213000000002, 2024-09-14T23:59:17.826 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:17.826 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:17.826 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:17.826 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:17.826 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:17.826 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:17.826 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 172639 2024-09-14T23:59:17.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.827 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:17.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:17.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.988365+0000", 2024-09-14T23:59:17.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:17.827 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.988365+0000", 2024-09-14T23:59:17.827 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.827 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.827 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:17.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.989904+0000", 2024-09-14T23:59:17.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0015380859999999999 2024-09-14T23:59:17.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:17.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.989909+0000", 2024-09-14T23:59:17.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.2800000000000003e-06 2024-09-14T23:59:17.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.828 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:17.828 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.989925+0000", 2024-09-14T23:59:17.828 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5781999999999999e-05 2024-09-14T23:59:17.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.828 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:17.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:03.989933+0000", 2024-09-14T23:59:17.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.0090000000000007e-06 2024-09-14T23:59:17.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:17.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:04.045741+0000", 2024-09-14T23:59:17.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055808112999999999 2024-09-14T23:59:17.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.829 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.829 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:17.829 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:04.045812+0000", 2024-09-14T23:59:17.829 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.1531000000000003e-05 2024-09-14T23:59:17.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.830 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:17.830 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:04.045922+0000", 2024-09-14T23:59:17.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000109412 2024-09-14T23:59:17.830 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.830 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:17.830 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.830 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.830 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184571 3.2 3:46f8d78b:::rbd_data.1118427720d0.0000000000000001:32 [read 1405952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:59:17.830 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.086404+0000", 2024-09-14T23:59:17.830 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 70.667239058000007, 2024-09-14T23:59:17.830 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.059148573000000003, 2024-09-14T23:59:17.831 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:17.831 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:17.831 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:17.831 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:17.831 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:17.831 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:17.831 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184571 2024-09-14T23:59:17.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.831 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:17.831 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.831 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:17.831 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.086404+0000", 2024-09-14T23:59:17.831 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:17.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.086404+0000", 2024-09-14T23:59:17.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:17.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087902+0000", 2024-09-14T23:59:17.832 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001498625 2024-09-14T23:59:17.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.832 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.832 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:17.832 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087907+0000", 2024-09-14T23:59:17.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.9130000000000002e-06 2024-09-14T23:59:17.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:17.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087919+0000", 2024-09-14T23:59:17.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1719000000000001e-05 2024-09-14T23:59:17.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:17.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.087927+0000", 2024-09-14T23:59:17.833 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.3699999999999995e-06 2024-09-14T23:59:17.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.833 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:17.833 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143711+0000", 2024-09-14T23:59:17.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055783396999999998 2024-09-14T23:59:17.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:17.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.143792+0000", 2024-09-14T23:59:17.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1292999999999996e-05 2024-09-14T23:59:17.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.834 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.834 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:17.834 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145552+0000", 2024-09-14T23:59:17.834 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001760256 2024-09-14T23:59:17.834 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.834 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:17.834 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.835 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.835 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:184579 3.2 3:43badc91:::rbd_data.1118427720d0.0000000000000002:32 [read 1196032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T23:59:17.835 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:07.088145+0000", 2024-09-14T23:59:17.835 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 70.665497654000006, 2024-09-14T23:59:17.835 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.057643169000000001, 2024-09-14T23:59:17.835 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:17.835 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:17.835 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:17.835 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:17.835 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:17.835 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:17.835 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 184579 2024-09-14T23:59:17.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.835 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:17.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:17.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.088145+0000", 2024-09-14T23:59:17.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:17.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.088145+0000", 2024-09-14T23:59:17.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.836 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:17.836 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089947+0000", 2024-09-14T23:59:17.836 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001801764 2024-09-14T23:59:17.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:17.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089952+0000", 2024-09-14T23:59:17.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8659999999999997e-06 2024-09-14T23:59:17.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:17.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089966+0000", 2024-09-14T23:59:17.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.3929e-05 2024-09-14T23:59:17.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.837 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.837 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:17.837 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.089974+0000", 2024-09-14T23:59:17.837 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 8.1429999999999998e-06 2024-09-14T23:59:17.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:17.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145636+0000", 2024-09-14T23:59:17.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.055662287999999997 2024-09-14T23:59:17.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.838 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.838 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:17.838 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145710+0000", 2024-09-14T23:59:17.838 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.4166999999999996e-05 2024-09-14T23:59:17.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.839 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:17.839 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:07.145788+0000", 2024-09-14T23:59:17.839 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.8011999999999998e-05 2024-09-14T23:59:17.839 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.839 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:17.839 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.839 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.839 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:211800 3.12 3:493125e8:::rbd_data.1118c900944f.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-14T23:59:17.839 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:13.660130+0000", 2024-09-14T23:59:17.839 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 64.093513328, 2024-09-14T23:59:17.840 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.55364332000000005, 2024-09-14T23:59:17.840 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:17.840 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:17.840 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:17.840 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:17.840 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:17.840 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:17.840 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 211800 2024-09-14T23:59:17.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.840 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:17.840 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.840 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:17.840 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.660130+0000", 2024-09-14T23:59:17.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:17.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.660130+0000", 2024-09-14T23:59:17.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:17.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.660133+0000", 2024-09-14T23:59:17.841 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.202e-06 2024-09-14T23:59:17.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.841 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.841 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:17.841 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.668135+0000", 2024-09-14T23:59:17.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0080027220000000003 2024-09-14T23:59:17.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:17.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.668172+0000", 2024-09-14T23:59:17.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.6295999999999999e-05 2024-09-14T23:59:17.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:17.842 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.668187+0000", 2024-09-14T23:59:17.842 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5537e-05 2024-09-14T23:59:17.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.842 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.842 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:17.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.668284+0000", 2024-09-14T23:59:17.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 9.6297000000000003e-05 2024-09-14T23:59:17.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:59:17.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:13.668299+0000", 2024-09-14T23:59:17.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5091e-05 2024-09-14T23:59:17.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.843 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:17.843 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:14.195910+0000", 2024-09-14T23:59:17.843 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.52761157999999997 2024-09-14T23:59:17.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.843 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:17.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:14.196106+0000", 2024-09-14T23:59:17.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00019597399999999999 2024-09-14T23:59:17.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for subops from 0", 2024-09-14T23:59:17.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:14.197262+0000", 2024-09-14T23:59:17.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.001155744 2024-09-14T23:59:17.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.844 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.844 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "op_commit", 2024-09-14T23:59:17.844 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:14.207440+0000", 2024-09-14T23:59:17.844 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.010178347000000001 2024-09-14T23:59:17.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:59:17.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:14.213691+0000", 2024-09-14T23:59:17.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.0062504420000000002 2024-09-14T23:59:17.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "commit_sent", 2024-09-14T23:59:17.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:14.213735+0000", 2024-09-14T23:59:17.845 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.4385999999999998e-05 2024-09-14T23:59:17.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.845 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.845 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:17.845 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:14.213773+0000", 2024-09-14T23:59:17.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 3.7701999999999999e-05 2024-09-14T23:59:17.846 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.846 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:17.846 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.846 INFO:teuthology.orchestra.run.smithi042.stdout: "description": "osd_op(client.4376.0:251476 3.5 3:a1f6ebd6:::rbd_object_map.11189263ca38.000000000000002a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126)", 2024-09-14T23:59:17.846 INFO:teuthology.orchestra.run.smithi042.stdout: "initiated_at": "2024-09-14T23:58:26.388925+0000", 2024-09-14T23:59:17.846 INFO:teuthology.orchestra.run.smithi042.stdout: "age": 51.364718295000003, 2024-09-14T23:59:17.846 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.062808638, 2024-09-14T23:59:17.846 INFO:teuthology.orchestra.run.smithi042.stdout: "continuous": false, 2024-09-14T23:59:17.846 INFO:teuthology.orchestra.run.smithi042.stdout: "type_data": { 2024-09-14T23:59:17.846 INFO:teuthology.orchestra.run.smithi042.stdout: "flag_point": "started", 2024-09-14T23:59:17.846 INFO:teuthology.orchestra.run.smithi042.stdout: "client_info": { 2024-09-14T23:59:17.847 INFO:teuthology.orchestra.run.smithi042.stdout: "client": "client.4376", 2024-09-14T23:59:17.847 INFO:teuthology.orchestra.run.smithi042.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:17.847 INFO:teuthology.orchestra.run.smithi042.stdout: "tid": 251476 2024-09-14T23:59:17.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.847 INFO:teuthology.orchestra.run.smithi042.stdout: "events": [ 2024-09-14T23:59:17.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "initiated", 2024-09-14T23:59:17.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.388925+0000", 2024-09-14T23:59:17.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.847 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.847 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "header_read", 2024-09-14T23:59:17.847 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.388925+0000", 2024-09-14T23:59:17.847 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0 2024-09-14T23:59:17.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "throttled", 2024-09-14T23:59:17.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.388926+0000", 2024-09-14T23:59:17.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.5200000000000001e-06 2024-09-14T23:59:17.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "all_read", 2024-09-14T23:59:17.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.388933+0000", 2024-09-14T23:59:17.848 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 6.827e-06 2024-09-14T23:59:17.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.848 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.848 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "dispatched", 2024-09-14T23:59:17.848 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.388945+0000", 2024-09-14T23:59:17.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 1.1701e-05 2024-09-14T23:59:17.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "queued_for_pg", 2024-09-14T23:59:17.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.388950+0000", 2024-09-14T23:59:17.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 5.6080000000000003e-06 2024-09-14T23:59:17.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:17.849 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.388999+0000", 2024-09-14T23:59:17.849 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 4.8711999999999997e-05 2024-09-14T23:59:17.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.849 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.849 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "waiting for peered", 2024-09-14T23:59:17.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.389006+0000", 2024-09-14T23:59:17.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 7.063e-06 2024-09-14T23:59:17.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "reached_pg", 2024-09-14T23:59:17.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.451371+0000", 2024-09-14T23:59:17.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.062365255000000001 2024-09-14T23:59:17.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.850 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "started", 2024-09-14T23:59:17.850 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.451472+0000", 2024-09-14T23:59:17.850 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.000100626 2024-09-14T23:59:17.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-14T23:59:17.850 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-09-14T23:59:17.851 INFO:teuthology.orchestra.run.smithi042.stdout: "event": "done", 2024-09-14T23:59:17.851 INFO:teuthology.orchestra.run.smithi042.stdout: "time": "2024-09-14T23:58:26.451733+0000", 2024-09-14T23:59:17.851 INFO:teuthology.orchestra.run.smithi042.stdout: "duration": 0.00026132599999999998 2024-09-14T23:59:17.851 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.851 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:17.851 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.851 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-14T23:59:17.851 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-14T23:59:17.851 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:59:17.852 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T23:59:17.973 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:59:17.973 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:59:17.973 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:59:17.973 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:59:17.982 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T23:59:17.984 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.0 to 1 (10000) 2024-09-14T23:59:18.000 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-14T23:59:18.091 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:59:18.091 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:59:18.091 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:59:18.092 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:59:18.092 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:59:18.100 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T23:59:18.210 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:59:18.469 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:59:18.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:59:18.470 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:59:18.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.470 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186203 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1329152~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:59:18.470 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.194279+0000", 2024-09-14T23:59:18.470 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 71.013476045999994, 2024-09-14T23:59:18.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8011469710000001, 2024-09-14T23:59:18.470 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:18.470 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:18.471 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:18.471 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:18.471 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:18.471 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:18.471 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186203 2024-09-14T23:59:18.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.471 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:18.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:18.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194279+0000", 2024-09-14T23:59:18.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:18.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194279+0000", 2024-09-14T23:59:18.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:18.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194279+0000", 2024-09-14T23:59:18.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8100000000000001e-07 2024-09-14T23:59:18.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:18.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194291+0000", 2024-09-14T23:59:18.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1559999999999999e-05 2024-09-14T23:59:18.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:18.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194303+0000", 2024-09-14T23:59:18.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2130000000000001e-05 2024-09-14T23:59:18.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:18.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194311+0000", 2024-09-14T23:59:18.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8089999999999993e-06 2024-09-14T23:59:18.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194370+0000", 2024-09-14T23:59:18.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8739999999999997e-05 2024-09-14T23:59:18.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:18.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194376+0000", 2024-09-14T23:59:18.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2319999999999998e-06 2024-09-14T23:59:18.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986290+0000", 2024-09-14T23:59:18.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.791914334 2024-09-14T23:59:18.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:18.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986296+0000", 2024-09-14T23:59:18.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6300000000000003e-06 2024-09-14T23:59:18.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995193+0000", 2024-09-14T23:59:18.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0088975040000001 2024-09-14T23:59:18.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:18.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995307+0000", 2024-09-14T23:59:18.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000113656 2024-09-14T23:59:18.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:18.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995426+0000", 2024-09-14T23:59:18.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118495 2024-09-14T23:59:18.476 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.476 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:18.476 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.476 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186204 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1391616~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:59:18.476 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.194328+0000", 2024-09-14T23:59:18.476 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 71.013426714999994, 2024-09-14T23:59:18.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8013834310000001, 2024-09-14T23:59:18.476 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:18.476 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:18.476 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:18.477 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:18.477 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:18.477 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:18.477 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186204 2024-09-14T23:59:18.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.477 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:18.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:18.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194328+0000", 2024-09-14T23:59:18.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:18.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194328+0000", 2024-09-14T23:59:18.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:18.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194328+0000", 2024-09-14T23:59:18.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7100000000000002e-07 2024-09-14T23:59:18.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:18.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194331+0000", 2024-09-14T23:59:18.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.807e-06 2024-09-14T23:59:18.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:18.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194339+0000", 2024-09-14T23:59:18.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9340000000000006e-06 2024-09-14T23:59:18.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:18.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194345+0000", 2024-09-14T23:59:18.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1090000000000003e-06 2024-09-14T23:59:18.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194412+0000", 2024-09-14T23:59:18.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6229000000000006e-05 2024-09-14T23:59:18.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:18.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194422+0000", 2024-09-14T23:59:18.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.078e-05 2024-09-14T23:59:18.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986322+0000", 2024-09-14T23:59:18.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.79189964899999998 2024-09-14T23:59:18.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:18.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986332+0000", 2024-09-14T23:59:18.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0095000000000001e-05 2024-09-14T23:59:18.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.481 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995554+0000", 2024-09-14T23:59:18.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0092218630000001 2024-09-14T23:59:18.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:18.481 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995610+0000", 2024-09-14T23:59:18.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6109000000000001e-05 2024-09-14T23:59:18.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:18.481 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995711+0000", 2024-09-14T23:59:18.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101285 2024-09-14T23:59:18.482 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.482 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:18.482 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.482 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186205 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1333248~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:59:18.482 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.194354+0000", 2024-09-14T23:59:18.482 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 71.013400785000002, 2024-09-14T23:59:18.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8015471490000001, 2024-09-14T23:59:18.482 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:18.482 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:18.482 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:18.482 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:18.483 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:18.483 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:18.483 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186205 2024-09-14T23:59:18.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.483 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:18.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:18.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194354+0000", 2024-09-14T23:59:18.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:18.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194354+0000", 2024-09-14T23:59:18.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:18.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194354+0000", 2024-09-14T23:59:18.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.39e-07 2024-09-14T23:59:18.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:18.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194357+0000", 2024-09-14T23:59:18.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3479999999999998e-06 2024-09-14T23:59:18.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:18.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194364+0000", 2024-09-14T23:59:18.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0899999999999999e-06 2024-09-14T23:59:18.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:18.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194371+0000", 2024-09-14T23:59:18.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5530000000000004e-06 2024-09-14T23:59:18.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194469+0000", 2024-09-14T23:59:18.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8119e-05 2024-09-14T23:59:18.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:18.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194475+0000", 2024-09-14T23:59:18.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7910000000000001e-06 2024-09-14T23:59:18.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986367+0000", 2024-09-14T23:59:18.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.79189214699999999 2024-09-14T23:59:18.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:18.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986375+0000", 2024-09-14T23:59:18.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7800000000000001e-06 2024-09-14T23:59:18.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995746+0000", 2024-09-14T23:59:18.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0093706469999999 2024-09-14T23:59:18.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:18.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995810+0000", 2024-09-14T23:59:18.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4068999999999999e-05 2024-09-14T23:59:18.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:18.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995901+0000", 2024-09-14T23:59:18.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1166000000000001e-05 2024-09-14T23:59:18.488 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.488 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:18.488 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.488 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186206 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1395712~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:59:18.488 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.194380+0000", 2024-09-14T23:59:18.488 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 71.013374764000005, 2024-09-14T23:59:18.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.801678347, 2024-09-14T23:59:18.488 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:18.488 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:18.488 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:18.488 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:18.488 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:18.489 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:18.489 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186206 2024-09-14T23:59:18.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.489 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:18.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:18.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194380+0000", 2024-09-14T23:59:18.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:18.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194380+0000", 2024-09-14T23:59:18.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:18.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194380+0000", 2024-09-14T23:59:18.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4400000000000001e-07 2024-09-14T23:59:18.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:18.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194383+0000", 2024-09-14T23:59:18.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3460000000000001e-06 2024-09-14T23:59:18.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:18.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194390+0000", 2024-09-14T23:59:18.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2189999999999998e-06 2024-09-14T23:59:18.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:18.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194403+0000", 2024-09-14T23:59:18.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2941e-05 2024-09-14T23:59:18.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194545+0000", 2024-09-14T23:59:18.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014232700000000001 2024-09-14T23:59:18.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:18.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194565+0000", 2024-09-14T23:59:18.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9409999999999999e-05 2024-09-14T23:59:18.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986409+0000", 2024-09-14T23:59:18.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.79184416000000002 2024-09-14T23:59:18.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:18.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986417+0000", 2024-09-14T23:59:18.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.551e-06 2024-09-14T23:59:18.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995921+0000", 2024-09-14T23:59:18.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.009504212 2024-09-14T23:59:18.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:18.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995974+0000", 2024-09-14T23:59:18.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2617999999999997e-05 2024-09-14T23:59:18.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:18.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996058+0000", 2024-09-14T23:59:18.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4118999999999999e-05 2024-09-14T23:59:18.494 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.494 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:18.494 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.494 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186207 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1337344~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:59:18.494 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.194412+0000", 2024-09-14T23:59:18.494 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 71.013342166000001, 2024-09-14T23:59:18.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.801848141, 2024-09-14T23:59:18.494 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:18.494 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:18.494 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:18.495 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:18.495 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:18.495 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:18.495 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186207 2024-09-14T23:59:18.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.495 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:18.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:18.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194412+0000", 2024-09-14T23:59:18.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:18.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194412+0000", 2024-09-14T23:59:18.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:18.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194413+0000", 2024-09-14T23:59:18.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5600000000000001e-07 2024-09-14T23:59:18.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:18.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194418+0000", 2024-09-14T23:59:18.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3720000000000004e-06 2024-09-14T23:59:18.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:18.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194423+0000", 2024-09-14T23:59:18.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.763e-06 2024-09-14T23:59:18.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:18.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194426+0000", 2024-09-14T23:59:18.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1480000000000002e-06 2024-09-14T23:59:18.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.498 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.498 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194596+0000", 2024-09-14T23:59:18.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017009600000000001 2024-09-14T23:59:18.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.498 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:18.498 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194606+0000", 2024-09-14T23:59:18.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6639999999999993e-06 2024-09-14T23:59:18.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.499 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.499 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986460+0000", 2024-09-14T23:59:18.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.79185424400000004 2024-09-14T23:59:18.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.499 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:18.499 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986469+0000", 2024-09-14T23:59:18.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4419999999999998e-06 2024-09-14T23:59:18.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996123+0000", 2024-09-14T23:59:18.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0096546070000001 2024-09-14T23:59:18.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:18.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996179+0000", 2024-09-14T23:59:18.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5850000000000002e-05 2024-09-14T23:59:18.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:18.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996261+0000", 2024-09-14T23:59:18.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1599000000000003e-05 2024-09-14T23:59:18.502 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.502 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:18.503 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.503 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186208 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1399808~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:59:18.503 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.194445+0000", 2024-09-14T23:59:18.503 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 71.013309996000004, 2024-09-14T23:59:18.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8019822269999999, 2024-09-14T23:59:18.504 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:18.504 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:18.504 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:18.504 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:18.504 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:18.504 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:18.505 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186208 2024-09-14T23:59:18.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.505 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:18.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:18.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194445+0000", 2024-09-14T23:59:18.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.507 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:18.507 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.194445+0000", 2024-09-14T23:59:18.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:18.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978674+0000", 2024-09-14T23:59:18.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.784229124 2024-09-14T23:59:18.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:18.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978680+0000", 2024-09-14T23:59:18.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7969999999999997e-06 2024-09-14T23:59:18.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:18.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978689+0000", 2024-09-14T23:59:18.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6949999999999995e-06 2024-09-14T23:59:18.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:18.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978696+0000", 2024-09-14T23:59:18.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1300000000000003e-06 2024-09-14T23:59:18.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986509+0000", 2024-09-14T23:59:18.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0078130969999999998 2024-09-14T23:59:18.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:18.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.986518+0000", 2024-09-14T23:59:18.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2810000000000008e-06 2024-09-14T23:59:18.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996280+0000", 2024-09-14T23:59:18.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0097623120000001 2024-09-14T23:59:18.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:18.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996337+0000", 2024-09-14T23:59:18.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7244000000000003e-05 2024-09-14T23:59:18.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:18.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.996427+0000", 2024-09-14T23:59:18.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9547000000000002e-05 2024-09-14T23:59:18.511 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.511 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:18.511 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.511 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4172.0:7 3.11 3:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:59:18.511 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.483386+0000", 2024-09-14T23:59:18.512 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 70.724368570999999, 2024-09-14T23:59:18.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.558667389, 2024-09-14T23:59:18.512 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:18.512 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:18.512 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:59:18.512 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:18.512 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4172", 2024-09-14T23:59:18.512 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/2542269203", 2024-09-14T23:59:18.512 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 7 2024-09-14T23:59:18.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.512 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:18.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.512 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:18.512 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.483386+0000", 2024-09-14T23:59:18.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:18.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.483386+0000", 2024-09-14T23:59:18.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:18.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978674+0000", 2024-09-14T23:59:18.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.49528781599999999 2024-09-14T23:59:18.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:18.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978687+0000", 2024-09-14T23:59:18.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3244999999999999e-05 2024-09-14T23:59:18.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:18.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978699+0000", 2024-09-14T23:59:18.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1832e-05 2024-09-14T23:59:18.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:18.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978707+0000", 2024-09-14T23:59:18.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4489999999999999e-06 2024-09-14T23:59:18.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978801+0000", 2024-09-14T23:59:18.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3511999999999995e-05 2024-09-14T23:59:18.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:18.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.978814+0000", 2024-09-14T23:59:18.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2789000000000001e-05 2024-09-14T23:59:18.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.041948+0000", 2024-09-14T23:59:18.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0631338269999999 2024-09-14T23:59:18.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.516 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:18.516 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.042053+0000", 2024-09-14T23:59:18.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105919 2024-09-14T23:59:18.516 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.516 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:18.516 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.516 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187209 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1575936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:59:18.516 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.991888+0000", 2024-09-14T23:59:18.516 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 70.215866599999998, 2024-09-14T23:59:18.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0712115470000001, 2024-09-14T23:59:18.516 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:18.517 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:18.517 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:18.517 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:18.517 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:18.517 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:18.517 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187209 2024-09-14T23:59:18.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.517 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:18.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:18.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991888+0000", 2024-09-14T23:59:18.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:18.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991888+0000", 2024-09-14T23:59:18.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:18.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991888+0000", 2024-09-14T23:59:18.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1199999999999998e-07 2024-09-14T23:59:18.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:18.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991890+0000", 2024-09-14T23:59:18.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9810000000000002e-06 2024-09-14T23:59:18.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:18.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991897+0000", 2024-09-14T23:59:18.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2609999999999999e-06 2024-09-14T23:59:18.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:18.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991902+0000", 2024-09-14T23:59:18.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0000000000000004e-06 2024-09-14T23:59:18.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995686+0000", 2024-09-14T23:59:18.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0037839290000000001 2024-09-14T23:59:18.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:18.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995693+0000", 2024-09-14T23:59:18.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7589999999999997e-06 2024-09-14T23:59:18.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.521 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.521 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.521 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062846+0000", 2024-09-14T23:59:18.521 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.067152417 2024-09-14T23:59:18.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.521 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.521 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:18.521 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.062935+0000", 2024-09-14T23:59:18.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9080999999999996e-05 2024-09-14T23:59:18.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:18.522 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.063100+0000", 2024-09-14T23:59:18.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016470699999999999 2024-09-14T23:59:18.522 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.522 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:18.522 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.522 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187210 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1591296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:59:18.522 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.991908+0000", 2024-09-14T23:59:18.522 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 70.215846686999996, 2024-09-14T23:59:18.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0713882530000001, 2024-09-14T23:59:18.523 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:18.523 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:18.523 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:18.523 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:18.523 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:18.523 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:18.523 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187210 2024-09-14T23:59:18.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.523 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:18.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:18.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991908+0000", 2024-09-14T23:59:18.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:18.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.991908+0000", 2024-09-14T23:59:18.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:18.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993673+0000", 2024-09-14T23:59:18.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017651030000000001 2024-09-14T23:59:18.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:18.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993681+0000", 2024-09-14T23:59:18.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.858e-06 2024-09-14T23:59:18.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.525 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:18.525 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993691+0000", 2024-09-14T23:59:18.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9920000000000002e-06 2024-09-14T23:59:18.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.525 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:18.525 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993698+0000", 2024-09-14T23:59:18.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6490000000000002e-06 2024-09-14T23:59:18.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.525 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.525 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995731+0000", 2024-09-14T23:59:18.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0020338959999999999 2024-09-14T23:59:18.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.526 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:18.526 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995741+0000", 2024-09-14T23:59:18.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3079999999999997e-06 2024-09-14T23:59:18.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.526 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.526 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.063130+0000", 2024-09-14T23:59:18.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.067389159 2024-09-14T23:59:18.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.526 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:18.526 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.063206+0000", 2024-09-14T23:59:18.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6577999999999998e-05 2024-09-14T23:59:18.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:18.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.063296+0000", 2024-09-14T23:59:18.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9710000000000007e-05 2024-09-14T23:59:18.527 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.527 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:18.527 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.527 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187216 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1603584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:59:18.527 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.993831+0000", 2024-09-14T23:59:18.528 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 70.213923480999995, 2024-09-14T23:59:18.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.071244627, 2024-09-14T23:59:18.528 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:18.528 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:18.528 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:18.528 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:18.528 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:18.528 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:18.528 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187216 2024-09-14T23:59:18.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.528 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:18.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:18.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993831+0000", 2024-09-14T23:59:18.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.529 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:18.529 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993831+0000", 2024-09-14T23:59:18.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.529 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:18.529 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993832+0000", 2024-09-14T23:59:18.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9700000000000002e-07 2024-09-14T23:59:18.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.529 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:18.529 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993834+0000", 2024-09-14T23:59:18.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6170000000000001e-06 2024-09-14T23:59:18.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.530 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:18.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993841+0000", 2024-09-14T23:59:18.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6980000000000001e-06 2024-09-14T23:59:18.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.530 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:18.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993846+0000", 2024-09-14T23:59:18.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4190000000000001e-06 2024-09-14T23:59:18.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995988+0000", 2024-09-14T23:59:18.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0021417390000000001 2024-09-14T23:59:18.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:18.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995998+0000", 2024-09-14T23:59:18.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5100000000000004e-06 2024-09-14T23:59:18.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.064883+0000", 2024-09-14T23:59:18.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.06888597 2024-09-14T23:59:18.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:18.532 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.064977+0000", 2024-09-14T23:59:18.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3817e-05 2024-09-14T23:59:18.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:18.532 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065076+0000", 2024-09-14T23:59:18.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8460000000000003e-05 2024-09-14T23:59:18.532 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.532 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:18.532 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.533 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187217 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1592320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:59:18.533 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.993855+0000", 2024-09-14T23:59:18.533 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 70.213900047999999, 2024-09-14T23:59:18.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0714239349999999, 2024-09-14T23:59:18.533 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:18.533 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:18.533 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:18.533 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:18.533 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:18.533 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:18.534 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187217 2024-09-14T23:59:18.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.534 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:18.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:18.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993855+0000", 2024-09-14T23:59:18.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:18.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993855+0000", 2024-09-14T23:59:18.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:18.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993855+0000", 2024-09-14T23:59:18.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4899999999999995e-07 2024-09-14T23:59:18.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:18.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993857+0000", 2024-09-14T23:59:18.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3379999999999999e-06 2024-09-14T23:59:18.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:18.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993864+0000", 2024-09-14T23:59:18.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6000000000000003e-06 2024-09-14T23:59:18.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:18.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993869+0000", 2024-09-14T23:59:18.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3210000000000004e-06 2024-09-14T23:59:18.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996038+0000", 2024-09-14T23:59:18.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0021683050000000001 2024-09-14T23:59:18.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:18.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996048+0000", 2024-09-14T23:59:18.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0173999999999999e-05 2024-09-14T23:59:18.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065108+0000", 2024-09-14T23:59:18.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0690600619999999 2024-09-14T23:59:18.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:18.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065171+0000", 2024-09-14T23:59:18.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3423999999999994e-05 2024-09-14T23:59:18.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:18.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065279+0000", 2024-09-14T23:59:18.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000107162 2024-09-14T23:59:18.537 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.538 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:18.538 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.538 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187218 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1607680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:59:18.538 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.993877+0000", 2024-09-14T23:59:18.538 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 70.213877768000003, 2024-09-14T23:59:18.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0716976810000001, 2024-09-14T23:59:18.538 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:18.538 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:18.538 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:18.538 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:18.538 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:18.538 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:18.539 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187218 2024-09-14T23:59:18.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.539 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:18.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.539 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:18.539 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993877+0000", 2024-09-14T23:59:18.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.539 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:18.539 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993877+0000", 2024-09-14T23:59:18.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.539 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:18.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993877+0000", 2024-09-14T23:59:18.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8299999999999998e-07 2024-09-14T23:59:18.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:18.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993879+0000", 2024-09-14T23:59:18.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5659999999999999e-06 2024-09-14T23:59:18.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:18.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993882+0000", 2024-09-14T23:59:18.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7280000000000001e-06 2024-09-14T23:59:18.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:18.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993885+0000", 2024-09-14T23:59:18.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.751e-06 2024-09-14T23:59:18.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996081+0000", 2024-09-14T23:59:18.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0021960590000000002 2024-09-14T23:59:18.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:18.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996091+0000", 2024-09-14T23:59:18.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4930000000000004e-06 2024-09-14T23:59:18.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065308+0000", 2024-09-14T23:59:18.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.069217407 2024-09-14T23:59:18.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:18.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065414+0000", 2024-09-14T23:59:18.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105438 2024-09-14T23:59:18.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:18.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065575+0000", 2024-09-14T23:59:18.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000160956 2024-09-14T23:59:18.542 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.543 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:18.543 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.543 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187219 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1596416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:59:18.543 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.993889+0000", 2024-09-14T23:59:18.543 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 70.213865925999997, 2024-09-14T23:59:18.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0720031190000001, 2024-09-14T23:59:18.543 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:18.543 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:18.543 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:18.543 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:18.543 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:18.543 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:18.544 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187219 2024-09-14T23:59:18.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.544 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:18.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:18.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993889+0000", 2024-09-14T23:59:18.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:18.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993889+0000", 2024-09-14T23:59:18.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:18.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993889+0000", 2024-09-14T23:59:18.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2399999999999999e-07 2024-09-14T23:59:18.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:18.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993890+0000", 2024-09-14T23:59:18.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1939999999999999e-06 2024-09-14T23:59:18.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:18.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993893+0000", 2024-09-14T23:59:18.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1300000000000001e-06 2024-09-14T23:59:18.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:18.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993896+0000", 2024-09-14T23:59:18.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6879999999999999e-06 2024-09-14T23:59:18.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996119+0000", 2024-09-14T23:59:18.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0022226149999999998 2024-09-14T23:59:18.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:18.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996123+0000", 2024-09-14T23:59:18.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.211e-06 2024-09-14T23:59:18.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065616+0000", 2024-09-14T23:59:18.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0694935240000001 2024-09-14T23:59:18.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:18.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065733+0000", 2024-09-14T23:59:18.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000116492 2024-09-14T23:59:18.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:18.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065892+0000", 2024-09-14T23:59:18.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015894100000000001 2024-09-14T23:59:18.548 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.548 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:18.548 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.548 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.548 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187220 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1611776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:59:18.548 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.993899+0000", 2024-09-14T23:59:18.548 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 70.213855468000006, 2024-09-14T23:59:18.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.072298041, 2024-09-14T23:59:18.548 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:18.548 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:18.548 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:18.548 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:18.548 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:18.548 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:18.549 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187220 2024-09-14T23:59:18.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.549 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:18.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:18.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993899+0000", 2024-09-14T23:59:18.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:18.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993899+0000", 2024-09-14T23:59:18.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:18.550 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993899+0000", 2024-09-14T23:59:18.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2100000000000001e-07 2024-09-14T23:59:18.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:18.550 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993901+0000", 2024-09-14T23:59:18.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.141e-06 2024-09-14T23:59:18.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:18.550 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993904+0000", 2024-09-14T23:59:18.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1470000000000001e-06 2024-09-14T23:59:18.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:18.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993906+0000", 2024-09-14T23:59:18.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6340000000000001e-06 2024-09-14T23:59:18.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996160+0000", 2024-09-14T23:59:18.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0022537659999999999 2024-09-14T23:59:18.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:18.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996169+0000", 2024-09-14T23:59:18.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2739999999999996e-06 2024-09-14T23:59:18.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.552 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.552 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.065981+0000", 2024-09-14T23:59:18.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.069811742 2024-09-14T23:59:18.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.552 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.552 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:18.552 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.066092+0000", 2024-09-14T23:59:18.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000110701 2024-09-14T23:59:18.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.552 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.552 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:18.552 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.066197+0000", 2024-09-14T23:59:18.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010541500000000001 2024-09-14T23:59:18.552 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.553 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:18.553 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.553 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187221 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1600512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:59:18.553 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.993909+0000", 2024-09-14T23:59:18.553 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 70.213845255999999, 2024-09-14T23:59:18.553 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.072643499, 2024-09-14T23:59:18.553 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:18.553 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:18.553 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:18.553 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:18.553 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:18.553 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:18.553 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187221 2024-09-14T23:59:18.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.554 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:18.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:18.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993909+0000", 2024-09-14T23:59:18.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:18.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.993909+0000", 2024-09-14T23:59:18.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:18.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995673+0000", 2024-09-14T23:59:18.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001763141 2024-09-14T23:59:18.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:18.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995677+0000", 2024-09-14T23:59:18.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8709999999999996e-06 2024-09-14T23:59:18.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:18.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995686+0000", 2024-09-14T23:59:18.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6649999999999992e-06 2024-09-14T23:59:18.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:18.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995691+0000", 2024-09-14T23:59:18.556 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0289999999999996e-06 2024-09-14T23:59:18.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.556 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.556 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.556 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996199+0000", 2024-09-14T23:59:18.556 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00050799099999999995 2024-09-14T23:59:18.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.556 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.556 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:18.556 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996205+0000", 2024-09-14T23:59:18.556 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8259999999999998e-06 2024-09-14T23:59:18.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.556 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.556 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.556 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.066238+0000", 2024-09-14T23:59:18.557 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.070032928 2024-09-14T23:59:18.557 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.557 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.557 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:18.557 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.066393+0000", 2024-09-14T23:59:18.557 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015549800000000001 2024-09-14T23:59:18.557 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.557 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.557 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:18.557 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.066553+0000", 2024-09-14T23:59:18.557 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015955 2024-09-14T23:59:18.557 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.557 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:18.557 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.558 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.558 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187223 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1604608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:59:18.558 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.995728+0000", 2024-09-14T23:59:18.558 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 70.212026440000002, 2024-09-14T23:59:18.558 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.071486478, 2024-09-14T23:59:18.558 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:18.558 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:18.558 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:18.558 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:18.558 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:18.558 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:18.558 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187223 2024-09-14T23:59:18.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.558 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:18.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:18.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995728+0000", 2024-09-14T23:59:18.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:18.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995728+0000", 2024-09-14T23:59:18.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:18.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995728+0000", 2024-09-14T23:59:18.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.65e-07 2024-09-14T23:59:18.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:18.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995730+0000", 2024-09-14T23:59:18.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.406e-06 2024-09-14T23:59:18.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:18.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995734+0000", 2024-09-14T23:59:18.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2640000000000002e-06 2024-09-14T23:59:18.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.561 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:18.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995737+0000", 2024-09-14T23:59:18.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0589999999999998e-06 2024-09-14T23:59:18.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.561 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996288+0000", 2024-09-14T23:59:18.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00055059599999999998 2024-09-14T23:59:18.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:18.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996294+0000", 2024-09-14T23:59:18.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0260000000000004e-06 2024-09-14T23:59:18.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.066938+0000", 2024-09-14T23:59:18.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0706439510000001 2024-09-14T23:59:18.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:18.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067050+0000", 2024-09-14T23:59:18.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011220100000000001 2024-09-14T23:59:18.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.563 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:18.563 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067215+0000", 2024-09-14T23:59:18.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016470999999999999 2024-09-14T23:59:18.563 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.563 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:18.563 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.563 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187224 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1619968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:59:18.563 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.995742+0000", 2024-09-14T23:59:18.563 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 70.212012625, 2024-09-14T23:59:18.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0717603609999999, 2024-09-14T23:59:18.563 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:18.563 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:18.564 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:18.564 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:18.564 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:18.564 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:18.564 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187224 2024-09-14T23:59:18.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.564 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:18.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:18.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995742+0000", 2024-09-14T23:59:18.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:18.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995742+0000", 2024-09-14T23:59:18.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.565 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:18.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995742+0000", 2024-09-14T23:59:18.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2399999999999999e-07 2024-09-14T23:59:18.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.565 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:18.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995744+0000", 2024-09-14T23:59:18.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.356e-06 2024-09-14T23:59:18.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.565 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:18.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995747+0000", 2024-09-14T23:59:18.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.304e-06 2024-09-14T23:59:18.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:18.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995750+0000", 2024-09-14T23:59:18.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6510000000000001e-06 2024-09-14T23:59:18.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996327+0000", 2024-09-14T23:59:18.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00057794300000000003 2024-09-14T23:59:18.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.567 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:18.567 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996337+0000", 2024-09-14T23:59:18.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3759999999999997e-06 2024-09-14T23:59:18.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.567 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.567 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067244+0000", 2024-09-14T23:59:18.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.070907367 2024-09-14T23:59:18.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.567 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:18.567 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067341+0000", 2024-09-14T23:59:18.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7277999999999999e-05 2024-09-14T23:59:18.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.568 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:18.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067502+0000", 2024-09-14T23:59:18.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016086200000000001 2024-09-14T23:59:18.568 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.568 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:18.568 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.568 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187225 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1608704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:59:18.568 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.995766+0000", 2024-09-14T23:59:18.568 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 70.211988450999996, 2024-09-14T23:59:18.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.072043649, 2024-09-14T23:59:18.568 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:18.569 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:18.569 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:18.569 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:18.569 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:18.569 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:18.569 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187225 2024-09-14T23:59:18.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.569 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:18.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.569 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:18.569 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995766+0000", 2024-09-14T23:59:18.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.569 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:18.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995766+0000", 2024-09-14T23:59:18.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.570 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:18.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995766+0000", 2024-09-14T23:59:18.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9200000000000002e-07 2024-09-14T23:59:18.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.570 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:18.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995768+0000", 2024-09-14T23:59:18.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8679999999999999e-06 2024-09-14T23:59:18.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:18.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995772+0000", 2024-09-14T23:59:18.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6339999999999999e-06 2024-09-14T23:59:18.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:18.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995775+0000", 2024-09-14T23:59:18.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8159999999999998e-06 2024-09-14T23:59:18.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996372+0000", 2024-09-14T23:59:18.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00059758800000000003 2024-09-14T23:59:18.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:18.572 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996377+0000", 2024-09-14T23:59:18.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7770000000000002e-06 2024-09-14T23:59:18.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.572 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067534+0000", 2024-09-14T23:59:18.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.071156821 2024-09-14T23:59:18.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:18.572 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067638+0000", 2024-09-14T23:59:18.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000103737 2024-09-14T23:59:18.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.573 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:18.573 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067810+0000", 2024-09-14T23:59:18.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017211599999999999 2024-09-14T23:59:18.573 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.573 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:18.573 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.573 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187226 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1624064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:59:18.573 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.995778+0000", 2024-09-14T23:59:18.573 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 70.211976272000001, 2024-09-14T23:59:18.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.072339081, 2024-09-14T23:59:18.573 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:18.574 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:18.574 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:18.574 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:18.574 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:18.574 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:18.574 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187226 2024-09-14T23:59:18.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.574 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:18.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.574 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:18.574 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995778+0000", 2024-09-14T23:59:18.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:18.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995778+0000", 2024-09-14T23:59:18.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:18.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995779+0000", 2024-09-14T23:59:18.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0699999999999999e-07 2024-09-14T23:59:18.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:18.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995780+0000", 2024-09-14T23:59:18.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2139999999999999e-06 2024-09-14T23:59:18.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:18.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995783+0000", 2024-09-14T23:59:18.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1250000000000001e-06 2024-09-14T23:59:18.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:18.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995787+0000", 2024-09-14T23:59:18.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6679999999999999e-06 2024-09-14T23:59:18.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996404+0000", 2024-09-14T23:59:18.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00061777499999999999 2024-09-14T23:59:18.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.577 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:18.577 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.996414+0000", 2024-09-14T23:59:18.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.431e-06 2024-09-14T23:59:18.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.577 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.577 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067844+0000", 2024-09-14T23:59:18.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0714302410000001 2024-09-14T23:59:18.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.577 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:18.577 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.067952+0000", 2024-09-14T23:59:18.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000107778 2024-09-14T23:59:18.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:18.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.068117+0000", 2024-09-14T23:59:18.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000165642 2024-09-14T23:59:18.578 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.578 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:18.578 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.578 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187227 3.f 3:f768e6da:::rbd_data.1118427720d0.0000000000000003:32 [read 1612800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:59:18.578 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.995806+0000", 2024-09-14T23:59:18.578 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 70.211949035000004, 2024-09-14T23:59:18.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0726092810000001, 2024-09-14T23:59:18.578 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:18.578 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:18.579 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:18.579 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:18.579 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:18.579 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:18.579 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187227 2024-09-14T23:59:18.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.579 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:18.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:18.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995806+0000", 2024-09-14T23:59:18.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:18.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.995806+0000", 2024-09-14T23:59:18.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:18.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995672+0000", 2024-09-14T23:59:18.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99986608099999996 2024-09-14T23:59:18.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:18.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995677+0000", 2024-09-14T23:59:18.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9520000000000002e-06 2024-09-14T23:59:18.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:18.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995686+0000", 2024-09-14T23:59:18.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8199999999999992e-06 2024-09-14T23:59:18.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:18.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:08.995693+0000", 2024-09-14T23:59:18.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1630000000000001e-06 2024-09-14T23:59:18.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.068149+0000", 2024-09-14T23:59:18.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.072456692000000003 2024-09-14T23:59:18.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:18.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.068251+0000", 2024-09-14T23:59:18.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101963 2024-09-14T23:59:18.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:18.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.068415+0000", 2024-09-14T23:59:18.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016360999999999999 2024-09-14T23:59:18.582 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.582 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:18.583 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.583 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.583 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:18.583 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:59:18.584 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T23:59:18.693 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:59:18.694 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:59:18.694 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:59:18.694 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:59:18.702 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T23:59:18.809 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:59:18.810 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:59:18.810 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:59:18.810 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:59:18.810 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:59:18.819 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T23:59:18.933 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:59:18.933 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:59:18.934 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:59:18.934 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:59:18.934 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.934 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4284.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-14T23:59:18.934 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:59:18.934 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 176.40325920500001, 2024-09-14T23:59:18.934 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.004942086, 2024-09-14T23:59:18.934 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:18.934 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:18.934 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:18.934 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:18.934 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4284", 2024-09-14T23:59:18.934 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/4278347926", 2024-09-14T23:59:18.934 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 1 2024-09-14T23:59:18.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.935 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:18.935 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:18.935 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:59:18.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.935 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:18.935 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524317+0000", 2024-09-14T23:59:18.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.935 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:18.936 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524331+0000", 2024-09-14T23:59:18.936 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.383e-05 2024-09-14T23:59:18.936 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.936 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.936 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:18.936 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524339+0000", 2024-09-14T23:59:18.936 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8290000000000008e-06 2024-09-14T23:59:18.936 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.936 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.936 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:18.936 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524359+0000", 2024-09-14T23:59:18.936 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9998000000000001e-05 2024-09-14T23:59:18.936 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.936 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.937 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:18.937 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524373+0000", 2024-09-14T23:59:18.937 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4304000000000001e-05 2024-09-14T23:59:18.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.937 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.937 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.937 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524488+0000", 2024-09-14T23:59:18.937 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114663 2024-09-14T23:59:18.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.937 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.937 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:18.937 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.524501+0000", 2024-09-14T23:59:18.938 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3349999999999999e-05 2024-09-14T23:59:18.938 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.938 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.938 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.938 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.526868+0000", 2024-09-14T23:59:18.938 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023670090000000002 2024-09-14T23:59:18.938 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.938 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.938 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:18.938 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.527041+0000", 2024-09-14T23:59:18.938 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000172597 2024-09-14T23:59:18.938 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.938 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.939 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-14T23:59:18.939 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.527384+0000", 2024-09-14T23:59:18.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00034309500000000002 2024-09-14T23:59:18.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.939 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:59:18.939 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529103+0000", 2024-09-14T23:59:18.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017185220000000001 2024-09-14T23:59:18.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.939 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:59:18.939 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529188+0000", 2024-09-14T23:59:18.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5591000000000001e-05 2024-09-14T23:59:18.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:18.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529227+0000", 2024-09-14T23:59:18.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8563000000000003e-05 2024-09-14T23:59:18.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:18.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:56:22.529259+0000", 2024-09-14T23:59:18.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2734999999999997e-05 2024-09-14T23:59:18.940 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.940 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:18.940 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.941 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56383 3.c e56/49 3:36b698b0:::rbd_data.1118c4b3aebe.0000000000000000:head v 56'194, mlcod=56'192)", 2024-09-14T23:59:18.941 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.270483+0000", 2024-09-14T23:59:18.941 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 132.65709373999999, 2024-09-14T23:59:18.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0079685880000000004, 2024-09-14T23:59:18.941 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:18.941 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:18.941 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:18.941 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:18.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.941 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:18.941 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.270483+0000", 2024-09-14T23:59:18.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.942 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.942 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:18.942 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.270483+0000", 2024-09-14T23:59:18.942 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.942 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.942 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:18.942 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.270484+0000", 2024-09-14T23:59:18.942 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7400000000000002e-07 2024-09-14T23:59:18.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.942 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.942 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:18.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273680+0000", 2024-09-14T23:59:18.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031966680000000002 2024-09-14T23:59:18.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.943 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:18.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273697+0000", 2024-09-14T23:59:18.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6595e-05 2024-09-14T23:59:18.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.943 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:18.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273705+0000", 2024-09-14T23:59:18.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7530000000000001e-06 2024-09-14T23:59:18.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273763+0000", 2024-09-14T23:59:18.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8238e-05 2024-09-14T23:59:18.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:18.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.273781+0000", 2024-09-14T23:59:18.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8281e-05 2024-09-14T23:59:18.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:18.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.278408+0000", 2024-09-14T23:59:18.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00462716 2024-09-14T23:59:18.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:18.945 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.278451+0000", 2024-09-14T23:59:18.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3019000000000003e-05 2024-09-14T23:59:18.945 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.945 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:18.945 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.945 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56384 3.c e56/49 3:36b698b0:::rbd_data.1118c4b3aebe.0000000000000000:head v 56'196, mlcod=56'192)", 2024-09-14T23:59:18.945 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:59:18.945 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 132.653061068, 2024-09-14T23:59:18.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0046707529999999997, 2024-09-14T23:59:18.946 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:18.946 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:18.946 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:18.946 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:18.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.946 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:18.946 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:59:18.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.946 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:18.946 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:59:18.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.947 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:18.947 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274516+0000", 2024-09-14T23:59:18.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1699999999999997e-07 2024-09-14T23:59:18.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.947 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:18.947 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274523+0000", 2024-09-14T23:59:18.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9129999999999997e-06 2024-09-14T23:59:18.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.947 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:18.947 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274535+0000", 2024-09-14T23:59:18.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1788e-05 2024-09-14T23:59:18.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.948 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:18.948 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.274542+0000", 2024-09-14T23:59:18.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0539999999999997e-06 2024-09-14T23:59:18.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.948 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.948 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275889+0000", 2024-09-14T23:59:18.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00134669 2024-09-14T23:59:18.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.948 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:18.949 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275917+0000", 2024-09-14T23:59:18.949 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7664000000000001e-05 2024-09-14T23:59:18.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:18.949 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279134+0000", 2024-09-14T23:59:18.949 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0032178459999999999 2024-09-14T23:59:18.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:18.949 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279186+0000", 2024-09-14T23:59:18.949 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1881e-05 2024-09-14T23:59:18.949 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.949 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:18.950 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.950 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56381 3.1d e56/49 3:bcf9a8cf:::rbd_data.1118c4b3aebe.0000000000000006:head v 56'210, mlcod=56'208)", 2024-09-14T23:59:18.950 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.275462+0000", 2024-09-14T23:59:18.950 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 132.65211473799999, 2024-09-14T23:59:18.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0071310710000000001, 2024-09-14T23:59:18.950 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:18.950 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:18.950 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:18.950 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:18.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.950 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:18.950 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275462+0000", 2024-09-14T23:59:18.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.951 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:18.951 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275462+0000", 2024-09-14T23:59:18.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.951 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:18.951 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.275463+0000", 2024-09-14T23:59:18.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2699999999999999e-07 2024-09-14T23:59:18.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.951 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:18.952 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276779+0000", 2024-09-14T23:59:18.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0013163649999999999 2024-09-14T23:59:18.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.952 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:18.952 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276795+0000", 2024-09-14T23:59:18.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.59e-05 2024-09-14T23:59:18.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.953 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:18.953 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276806+0000", 2024-09-14T23:59:18.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1106e-05 2024-09-14T23:59:18.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.953 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.953 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276941+0000", 2024-09-14T23:59:18.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013477500000000001 2024-09-14T23:59:18.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.953 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:18.953 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276963+0000", 2024-09-14T23:59:18.954 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2419999999999999e-05 2024-09-14T23:59:18.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.954 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.954 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:18.954 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282546+0000", 2024-09-14T23:59:18.954 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0055827100000000003 2024-09-14T23:59:18.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.954 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.954 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:18.954 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282593+0000", 2024-09-14T23:59:18.954 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7067999999999997e-05 2024-09-14T23:59:18.954 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.954 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:18.954 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.954 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.955 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56387 3.c e56/49 3:30ee4ba0:::rbd_data.1118c4b3aebe.0000000000000004:head v 56'197, mlcod=56'192)", 2024-09-14T23:59:18.955 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.276498+0000", 2024-09-14T23:59:18.955 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 132.65107864300001, 2024-09-14T23:59:18.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0092192639999999996, 2024-09-14T23:59:18.955 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:18.955 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:18.955 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:18.955 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:18.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.955 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:18.955 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276498+0000", 2024-09-14T23:59:18.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:18.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276498+0000", 2024-09-14T23:59:18.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:18.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.276499+0000", 2024-09-14T23:59:18.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1100000000000004e-07 2024-09-14T23:59:18.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:18.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279646+0000", 2024-09-14T23:59:18.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031473220000000001 2024-09-14T23:59:18.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:18.957 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279672+0000", 2024-09-14T23:59:18.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6129999999999999e-05 2024-09-14T23:59:18.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:18.957 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279682+0000", 2024-09-14T23:59:18.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7170000000000003e-06 2024-09-14T23:59:18.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.957 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279742+0000", 2024-09-14T23:59:18.958 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9926999999999997e-05 2024-09-14T23:59:18.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.958 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.958 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:18.958 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279761+0000", 2024-09-14T23:59:18.958 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9402000000000002e-05 2024-09-14T23:59:18.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.958 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.958 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:18.958 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.285680+0000", 2024-09-14T23:59:18.958 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.005918881 2024-09-14T23:59:18.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.959 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.959 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:18.959 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.285717+0000", 2024-09-14T23:59:18.959 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6974000000000002e-05 2024-09-14T23:59:18.959 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.959 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:18.959 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.959 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.959 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56388 3.c e56/49 3:30ee4ba0:::rbd_data.1118c4b3aebe.0000000000000004:head v 56'199, mlcod=56'192)", 2024-09-14T23:59:18.959 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:59:18.959 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 132.647587055, 2024-09-14T23:59:18.959 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0060596740000000001, 2024-09-14T23:59:18.959 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:18.960 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:18.960 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:18.960 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:18.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.960 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:18.960 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:59:18.960 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.960 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:18.960 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:59:18.960 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:18.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279990+0000", 2024-09-14T23:59:18.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2200000000000003e-07 2024-09-14T23:59:18.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:18.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.279995+0000", 2024-09-14T23:59:18.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5569999999999997e-06 2024-09-14T23:59:18.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:18.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280004+0000", 2024-09-14T23:59:18.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0080000000000008e-06 2024-09-14T23:59:18.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.962 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:18.962 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280010+0000", 2024-09-14T23:59:18.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5749999999999997e-06 2024-09-14T23:59:18.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.962 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.962 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282968+0000", 2024-09-14T23:59:18.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0029579390000000001 2024-09-14T23:59:18.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.962 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:18.963 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.282998+0000", 2024-09-14T23:59:18.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9947999999999999e-05 2024-09-14T23:59:18.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.963 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.963 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:18.963 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.286012+0000", 2024-09-14T23:59:18.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030132900000000001 2024-09-14T23:59:18.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.963 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.963 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:18.963 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.286049+0000", 2024-09-14T23:59:18.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7534999999999999e-05 2024-09-14T23:59:18.963 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.964 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:18.964 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.964 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "pg_update_log_missing(3.c epoch 56/49 rep_tid 1493 entries 56'200 (0'0) error 3:370203a3:::rbd_data.1118c4b3aebe.000000000000000d:head by client.4376.0:56396 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 56'192)", 2024-09-14T23:59:18.964 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:59:18.964 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 132.64706691500001, 2024-09-14T23:59:18.964 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057370099999999999, 2024-09-14T23:59:18.964 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:18.964 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:18.964 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:59:18.964 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:18.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.965 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:18.965 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:59:18.965 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.965 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.965 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:18.965 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:59:18.965 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.965 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.965 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:18.965 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280510+0000", 2024-09-14T23:59:18.965 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0699999999999997e-07 2024-09-14T23:59:18.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:18.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280519+0000", 2024-09-14T23:59:18.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6260000000000001e-06 2024-09-14T23:59:18.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:18.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280541+0000", 2024-09-14T23:59:18.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1976000000000001e-05 2024-09-14T23:59:18.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:18.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.280556+0000", 2024-09-14T23:59:18.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5075000000000001e-05 2024-09-14T23:59:18.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.967 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.285494+0000", 2024-09-14T23:59:18.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0049378690000000001 2024-09-14T23:59:18.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.967 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:18.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.286247+0000", 2024-09-14T23:59:18.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00075285700000000001 2024-09-14T23:59:18.967 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.967 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:18.968 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.968 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.968 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.968 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:56400 3.1c e56/49 3:3f5b81ba:::rbd_data.1118c4b3aebe.000000000000000a:head v 56'236, mlcod=56'234)", 2024-09-14T23:59:18.968 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:06.295064+0000", 2024-09-14T23:59:18.968 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 132.632512123, 2024-09-14T23:59:18.968 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0054331589999999999, 2024-09-14T23:59:18.968 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:18.968 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:18.968 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:18.968 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:18.968 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.968 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:18.969 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.295064+0000", 2024-09-14T23:59:18.969 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.969 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.969 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:18.969 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.295064+0000", 2024-09-14T23:59:18.969 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.969 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.969 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:18.969 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.295066+0000", 2024-09-14T23:59:18.969 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.502e-06 2024-09-14T23:59:18.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.970 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:18.970 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296403+0000", 2024-09-14T23:59:18.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001336554 2024-09-14T23:59:18.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.970 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:18.970 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296428+0000", 2024-09-14T23:59:18.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5179000000000001e-05 2024-09-14T23:59:18.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.970 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:18.970 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296440+0000", 2024-09-14T23:59:18.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2387e-05 2024-09-14T23:59:18.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.971 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.971 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296554+0000", 2024-09-14T23:59:18.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114269 2024-09-14T23:59:18.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.971 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:18.971 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.296581+0000", 2024-09-14T23:59:18.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6673999999999999e-05 2024-09-14T23:59:18.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.971 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:18.971 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.300418+0000", 2024-09-14T23:59:18.972 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0038371249999999998 2024-09-14T23:59:18.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.972 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.972 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:18.972 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:06.300498+0000", 2024-09-14T23:59:18.972 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9468999999999994e-05 2024-09-14T23:59:18.972 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.972 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:18.972 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.972 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.972 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:61922 3.5 e57/49 3:a11ebf8e:::rbd_data.11188d356229.000000000000000c:head v 57'280, mlcod=57'277)", 2024-09-14T23:59:18.972 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:07.563144+0000", 2024-09-14T23:59:18.973 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 131.36443247099999, 2024-09-14T23:59:18.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003977848, 2024-09-14T23:59:18.973 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:18.973 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:18.973 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:18.973 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:18.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.973 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:18.973 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563144+0000", 2024-09-14T23:59:18.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.973 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:18.973 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563144+0000", 2024-09-14T23:59:18.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.974 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:18.974 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.563145+0000", 2024-09-14T23:59:18.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5400000000000001e-07 2024-09-14T23:59:18.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.974 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:18.974 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565056+0000", 2024-09-14T23:59:18.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0019108160000000001 2024-09-14T23:59:18.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.974 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:18.974 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565088+0000", 2024-09-14T23:59:18.975 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2586999999999998e-05 2024-09-14T23:59:18.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.975 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.975 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:18.975 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565100+0000", 2024-09-14T23:59:18.975 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1917e-05 2024-09-14T23:59:18.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.975 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.975 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.975 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565166+0000", 2024-09-14T23:59:18.975 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6245e-05 2024-09-14T23:59:18.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.975 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.975 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:18.976 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.565190+0000", 2024-09-14T23:59:18.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3825999999999999e-05 2024-09-14T23:59:18.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.976 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:18.976 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.567080+0000", 2024-09-14T23:59:18.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001889541 2024-09-14T23:59:18.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.976 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:18.976 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:07.567122+0000", 2024-09-14T23:59:18.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2262000000000003e-05 2024-09-14T23:59:18.976 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.976 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:18.977 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.977 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:66564 3.c e60/49 3:33faa7ca:::rbd_data.111873792d9b.0000000000000003:head v 60'236, mlcod=57'233)", 2024-09-14T23:59:18.977 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:11.271596+0000", 2024-09-14T23:59:18.977 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 127.65598089300001, 2024-09-14T23:59:18.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0060396099999999999, 2024-09-14T23:59:18.977 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:18.977 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:18.977 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:18.977 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:18.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.977 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:18.978 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.271596+0000", 2024-09-14T23:59:18.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.978 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:18.978 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.271596+0000", 2024-09-14T23:59:18.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.978 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:18.978 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.271597+0000", 2024-09-14T23:59:18.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2559999999999999e-06 2024-09-14T23:59:18.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.979 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.979 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:18.979 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.274854+0000", 2024-09-14T23:59:18.979 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0032574470000000001 2024-09-14T23:59:18.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.979 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.979 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:18.979 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.274874+0000", 2024-09-14T23:59:18.979 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9449999999999998e-05 2024-09-14T23:59:18.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.979 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.979 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:18.979 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.274883+0000", 2024-09-14T23:59:18.979 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5980000000000003e-06 2024-09-14T23:59:18.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.980 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.980 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.274977+0000", 2024-09-14T23:59:18.980 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3270999999999995e-05 2024-09-14T23:59:18.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.980 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:18.980 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.275000+0000", 2024-09-14T23:59:18.980 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3088e-05 2024-09-14T23:59:18.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.980 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:18.980 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.277590+0000", 2024-09-14T23:59:18.981 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0025898660000000001 2024-09-14T23:59:18.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.981 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.981 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:18.981 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:11.277635+0000", 2024-09-14T23:59:18.981 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5633999999999997e-05 2024-09-14T23:59:18.981 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.981 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:18.981 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.981 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.981 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:133384 3.1f e92/49 3:fce5e529:::rbd_data.11186132a8b6.0000000000000002:head v 92'807, mlcod=89'804)", 2024-09-14T23:59:18.981 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:45.293642+0000", 2024-09-14T23:59:18.981 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 93.633935010000002, 2024-09-14T23:59:18.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003760186, 2024-09-14T23:59:18.982 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:18.982 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:18.982 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:18.982 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:18.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.982 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:18.982 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.293642+0000", 2024-09-14T23:59:18.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.982 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:18.982 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.293642+0000", 2024-09-14T23:59:18.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.983 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.983 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:18.983 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.293643+0000", 2024-09-14T23:59:18.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2500000000000001e-06 2024-09-14T23:59:18.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.983 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.983 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:18.983 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.295402+0000", 2024-09-14T23:59:18.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017595410000000001 2024-09-14T23:59:18.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.983 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.983 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:18.983 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.295422+0000", 2024-09-14T23:59:18.984 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9763999999999999e-05 2024-09-14T23:59:18.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.984 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:18.984 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.295436+0000", 2024-09-14T23:59:18.984 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.364e-05 2024-09-14T23:59:18.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.984 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.984 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.295499+0000", 2024-09-14T23:59:18.984 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2770999999999999e-05 2024-09-14T23:59:18.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.984 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:18.985 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.295516+0000", 2024-09-14T23:59:18.985 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7240000000000001e-05 2024-09-14T23:59:18.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.985 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.985 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:18.985 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.297324+0000", 2024-09-14T23:59:18.985 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001807857 2024-09-14T23:59:18.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.985 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.985 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:18.985 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:45.297402+0000", 2024-09-14T23:59:18.985 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8122999999999997e-05 2024-09-14T23:59:18.985 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.985 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:18.986 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.986 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:156597 3.5 e103/49 3:a23c5924:::rbd_data.1118b2f495e6.0000000000000011:head v 103'518, mlcod=98'516)", 2024-09-14T23:59:18.986 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:58.656197+0000", 2024-09-14T23:59:18.986 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 80.271379952999993, 2024-09-14T23:59:18.986 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0040423029999999997, 2024-09-14T23:59:18.986 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:18.986 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:18.986 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:18.987 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:18.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.987 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:18.987 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656197+0000", 2024-09-14T23:59:18.987 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.987 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:18.987 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656197+0000", 2024-09-14T23:59:18.987 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.988 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.988 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.988 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:18.988 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656197+0000", 2024-09-14T23:59:18.988 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1900000000000002e-07 2024-09-14T23:59:18.988 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.988 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.988 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:18.988 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656676+0000", 2024-09-14T23:59:18.988 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00047921699999999999 2024-09-14T23:59:18.988 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.989 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.989 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:18.989 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656690+0000", 2024-09-14T23:59:18.989 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3916e-05 2024-09-14T23:59:18.989 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.989 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.989 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:18.989 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656698+0000", 2024-09-14T23:59:18.989 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6529999999999994e-06 2024-09-14T23:59:18.989 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.989 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.989 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.989 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656757+0000", 2024-09-14T23:59:18.990 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9237999999999998e-05 2024-09-14T23:59:18.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.990 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.990 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:18.990 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.656780+0000", 2024-09-14T23:59:18.990 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2968000000000001e-05 2024-09-14T23:59:18.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.990 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.990 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:18.990 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.660188+0000", 2024-09-14T23:59:18.990 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0034078630000000001 2024-09-14T23:59:18.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.991 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.991 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:18.991 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.660239+0000", 2024-09-14T23:59:18.991 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0828999999999997e-05 2024-09-14T23:59:18.991 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.991 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:18.991 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.991 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.991 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:157947 3.1c e103/49 3:383374fe:::rbd_data.1118b2f495e6.000000000000000c:head v 103'771, mlcod=103'769)", 2024-09-14T23:59:18.991 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:58.765271+0000", 2024-09-14T23:59:18.991 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 80.162305071999995, 2024-09-14T23:59:18.991 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0040869829999999998, 2024-09-14T23:59:18.992 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:18.992 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:18.992 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:18.992 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:18.992 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.992 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:18.992 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765271+0000", 2024-09-14T23:59:18.992 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.992 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.992 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.992 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:18.992 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765271+0000", 2024-09-14T23:59:18.992 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.992 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.993 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.993 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:18.993 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765273+0000", 2024-09-14T23:59:18.993 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1149999999999999e-06 2024-09-14T23:59:18.993 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.993 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.993 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:18.993 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765854+0000", 2024-09-14T23:59:18.993 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00058119599999999997 2024-09-14T23:59:18.993 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.993 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.993 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:18.993 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765869+0000", 2024-09-14T23:59:18.993 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5641000000000002e-05 2024-09-14T23:59:18.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.994 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.994 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:18.994 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765878+0000", 2024-09-14T23:59:18.994 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4519999999999997e-06 2024-09-14T23:59:18.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.994 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.994 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.994 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765942+0000", 2024-09-14T23:59:18.994 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3727999999999997e-05 2024-09-14T23:59:18.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.994 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.994 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:18.994 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.765968+0000", 2024-09-14T23:59:18.995 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6171999999999999e-05 2024-09-14T23:59:18.995 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.995 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.995 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:18.995 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.769319+0000", 2024-09-14T23:59:18.995 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033511629999999999 2024-09-14T23:59:18.995 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.995 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.995 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:18.995 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:58.769358+0000", 2024-09-14T23:59:18.995 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9515999999999999e-05 2024-09-14T23:59:18.995 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.995 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:18.995 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:18.996 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.996 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.996 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.4376.0:160196 3.c e103/49 3:34623e68:::rbd_data.1118b2f495e6.0000000000000005:head v 103'551, mlcod=102'549)", 2024-09-14T23:59:18.996 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:59.131433+0000", 2024-09-14T23:59:18.996 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 79.796143763000003, 2024-09-14T23:59:18.996 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0036932530000000001, 2024-09-14T23:59:18.996 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:18.996 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:18.996 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:18.996 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:18.996 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.996 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:18.997 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.131433+0000", 2024-09-14T23:59:18.997 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.997 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.997 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:18.997 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.131433+0000", 2024-09-14T23:59:18.997 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:18.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.997 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.997 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:18.997 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.131433+0000", 2024-09-14T23:59:18.997 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8500000000000001e-07 2024-09-14T23:59:18.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.998 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:18.998 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.132144+0000", 2024-09-14T23:59:18.998 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00071022800000000003 2024-09-14T23:59:18.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.998 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:18.998 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.132154+0000", 2024-09-14T23:59:18.998 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0301e-05 2024-09-14T23:59:18.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.998 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:18.998 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.132159+0000", 2024-09-14T23:59:18.999 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6280000000000004e-06 2024-09-14T23:59:18.999 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.999 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.999 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:18.999 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.132208+0000", 2024-09-14T23:59:18.999 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9595000000000002e-05 2024-09-14T23:59:18.999 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.999 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.999 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:18.999 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.132231+0000", 2024-09-14T23:59:18.999 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3010000000000002e-05 2024-09-14T23:59:18.999 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:18.999 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:18.999 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:18.999 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.135087+0000", 2024-09-14T23:59:19.000 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0028553239999999998 2024-09-14T23:59:19.000 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.000 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.000 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:19.000 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:59.135126+0000", 2024-09-14T23:59:19.000 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9481999999999999e-05 2024-09-14T23:59:19.000 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.000 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:19.000 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.000 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.000 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.000 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:188871 3.d 3:b35cff4d:::rbd_header.1118c900944f:head [watch reconnect cookie 140670715433840 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-14T23:59:19.000 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.970653+0000", 2024-09-14T23:59:19.001 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 70.956924029999996, 2024-09-14T23:59:19.001 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0727808270000001, 2024-09-14T23:59:19.001 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:19.001 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:19.001 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:19.001 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:19.001 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:19.001 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:19.001 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 188871 2024-09-14T23:59:19.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.001 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:19.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:19.001 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970653+0000", 2024-09-14T23:59:19.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.002 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:19.002 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970653+0000", 2024-09-14T23:59:19.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.002 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:19.002 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970655+0000", 2024-09-14T23:59:19.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.277e-06 2024-09-14T23:59:19.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.002 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:19.003 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970662+0000", 2024-09-14T23:59:19.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9619999999999996e-06 2024-09-14T23:59:19.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.003 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:19.003 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970682+0000", 2024-09-14T23:59:19.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0279999999999999e-05 2024-09-14T23:59:19.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.003 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:19.003 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970691+0000", 2024-09-14T23:59:19.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2790000000000004e-06 2024-09-14T23:59:19.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970762+0000", 2024-09-14T23:59:19.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1019999999999994e-05 2024-09-14T23:59:19.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.004 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-14T23:59:19.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.970770+0000", 2024-09-14T23:59:19.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7999999999999999e-06 2024-09-14T23:59:19.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.004 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.973516+0000", 2024-09-14T23:59:19.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0027456469999999999 2024-09-14T23:59:19.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.005 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.005 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:19.005 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.973525+0000", 2024-09-14T23:59:19.005 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4890000000000001e-06 2024-09-14T23:59:19.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.005 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.005 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.005 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.041451+0000", 2024-09-14T23:59:19.005 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.067926012 2024-09-14T23:59:19.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.005 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.005 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:19.005 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.041673+0000", 2024-09-14T23:59:19.006 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000222048 2024-09-14T23:59:19.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.006 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:19.006 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:09.043433+0000", 2024-09-14T23:59:19.006 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017600129999999999 2024-09-14T23:59:19.006 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.006 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:19.006 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.006 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(osd.0.0:3238 3.1f e122/49 3:f8a59e83:::rbd_data.11187b9f1032.0000000000000006:2f v 122'943, mlcod=120'940)", 2024-09-14T23:59:19.007 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:22.285283+0000", 2024-09-14T23:59:19.007 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 56.642293207000002, 2024-09-14T23:59:19.007 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003756194, 2024-09-14T23:59:19.007 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:19.007 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:19.007 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:19.007 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:19.007 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.007 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:19.007 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.285283+0000", 2024-09-14T23:59:19.007 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.007 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:19.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.285283+0000", 2024-09-14T23:59:19.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:19.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.285284+0000", 2024-09-14T23:59:19.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5600000000000002e-07 2024-09-14T23:59:19.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.009 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:19.009 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.285289+0000", 2024-09-14T23:59:19.009 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9579999999999998e-06 2024-09-14T23:59:19.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.009 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.009 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:19.009 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.285300+0000", 2024-09-14T23:59:19.009 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1688999999999999e-05 2024-09-14T23:59:19.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.009 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.009 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:19.009 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.285309+0000", 2024-09-14T23:59:19.009 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.388e-06 2024-09-14T23:59:19.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.010 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.010 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.010 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.285367+0000", 2024-09-14T23:59:19.010 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8811000000000003e-05 2024-09-14T23:59:19.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.010 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.010 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:19.010 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.285388+0000", 2024-09-14T23:59:19.010 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0117000000000001e-05 2024-09-14T23:59:19.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.010 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.010 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:19.010 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.288994+0000", 2024-09-14T23:59:19.010 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0036064270000000002 2024-09-14T23:59:19.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.011 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.011 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:19.011 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:22.289040+0000", 2024-09-14T23:59:19.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5547999999999998e-05 2024-09-14T23:59:19.011 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.011 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:19.011 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.011 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.011 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(osd.6.0:1517 3.d e127/112 3:b69c12aa:::rbd_data.1118f9c96a7.0000000000000007:2c v 127'452, mlcod=126'449)", 2024-09-14T23:59:19.011 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:27.316914+0000", 2024-09-14T23:59:19.011 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.610662599000001, 2024-09-14T23:59:19.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0052453980000000001, 2024-09-14T23:59:19.012 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:19.012 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:19.012 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:19.012 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:19.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.012 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:19.012 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.316914+0000", 2024-09-14T23:59:19.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.012 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:19.012 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.316914+0000", 2024-09-14T23:59:19.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.013 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:19.013 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.316914+0000", 2024-09-14T23:59:19.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4900000000000002e-07 2024-09-14T23:59:19.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.013 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:19.013 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.316920+0000", 2024-09-14T23:59:19.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7169999999999996e-06 2024-09-14T23:59:19.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.013 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:19.013 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.316928+0000", 2024-09-14T23:59:19.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3769999999999996e-06 2024-09-14T23:59:19.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.014 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:19.014 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.316938+0000", 2024-09-14T23:59:19.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7329999999999998e-06 2024-09-14T23:59:19.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.014 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.014 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.317033+0000", 2024-09-14T23:59:19.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5295000000000002e-05 2024-09-14T23:59:19.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.014 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:19.014 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.317057+0000", 2024-09-14T23:59:19.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3989999999999999e-05 2024-09-14T23:59:19.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.015 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:19.015 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.322122+0000", 2024-09-14T23:59:19.015 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0050642500000000002 2024-09-14T23:59:19.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.015 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:19.015 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.322159+0000", 2024-09-14T23:59:19.015 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7787000000000003e-05 2024-09-14T23:59:19.015 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.015 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:19.015 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.016 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(osd.1.0:1466 3.16 e127/112 3:6c7910cf:::rbd_data.1118f9c96a7.0000000000000006:2c v 127'376, mlcod=121'372)", 2024-09-14T23:59:19.016 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:27.317464+0000", 2024-09-14T23:59:19.016 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.610112565999998, 2024-09-14T23:59:19.016 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0050844499999999999, 2024-09-14T23:59:19.016 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:19.016 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:19.016 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:19.016 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:19.016 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.016 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:19.016 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.317464+0000", 2024-09-14T23:59:19.016 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.016 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.017 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:19.017 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.317464+0000", 2024-09-14T23:59:19.017 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.017 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.017 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:19.017 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.317464+0000", 2024-09-14T23:59:19.017 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.91e-07 2024-09-14T23:59:19.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.017 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.017 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:19.017 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.317469+0000", 2024-09-14T23:59:19.017 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4719999999999997e-06 2024-09-14T23:59:19.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.017 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.018 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:19.018 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.317481+0000", 2024-09-14T23:59:19.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1814000000000001e-05 2024-09-14T23:59:19.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.018 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:19.018 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.317488+0000", 2024-09-14T23:59:19.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2740000000000002e-06 2024-09-14T23:59:19.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.018 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.018 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.317584+0000", 2024-09-14T23:59:19.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6191000000000001e-05 2024-09-14T23:59:19.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:19.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.317605+0000", 2024-09-14T23:59:19.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0554e-05 2024-09-14T23:59:19.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:19.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.322504+0000", 2024-09-14T23:59:19.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0048991089999999996 2024-09-14T23:59:19.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:19.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.322548+0000", 2024-09-14T23:59:19.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4744999999999999e-05 2024-09-14T23:59:19.019 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.020 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:19.020 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.020 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:257723 3.5 3:a794514b:::rbd_header.1118350d5ae5:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e164)", 2024-09-14T23:59:19.020 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:59:04.108044+0000", 2024-09-14T23:59:19.020 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 14.819532531, 2024-09-14T23:59:19.020 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003806233, 2024-09-14T23:59:19.020 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:19.020 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:19.020 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:19.020 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:19.020 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:19.020 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:19.020 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 257723 2024-09-14T23:59:19.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.021 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:19.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.021 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:19.021 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:59:04.108044+0000", 2024-09-14T23:59:19.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.021 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:19.021 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:59:04.108044+0000", 2024-09-14T23:59:19.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.021 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:19.022 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:59:04.108046+0000", 2024-09-14T23:59:19.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4899999999999999e-06 2024-09-14T23:59:19.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.022 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:19.022 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:59:04.108051+0000", 2024-09-14T23:59:19.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.575e-06 2024-09-14T23:59:19.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.022 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:19.022 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:59:04.108062+0000", 2024-09-14T23:59:19.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.111e-05 2024-09-14T23:59:19.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.022 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:19.023 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:59:04.108070+0000", 2024-09-14T23:59:19.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7349999999999996e-06 2024-09-14T23:59:19.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.023 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.023 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:59:04.108120+0000", 2024-09-14T23:59:19.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0420000000000002e-05 2024-09-14T23:59:19.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.023 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op must wait for map", 2024-09-14T23:59:19.024 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:59:04.108128+0000", 2024-09-14T23:59:19.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2849999999999997e-06 2024-09-14T23:59:19.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.024 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.024 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:59:04.109897+0000", 2024-09-14T23:59:19.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001769807 2024-09-14T23:59:19.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.024 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:19.024 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:59:04.110013+0000", 2024-09-14T23:59:19.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011595 2024-09-14T23:59:19.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-14T23:59:19.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:59:04.110462+0000", 2024-09-14T23:59:19.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00044826399999999998 2024-09-14T23:59:19.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:59:19.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:59:04.111415+0000", 2024-09-14T23:59:19.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00095352900000000005 2024-09-14T23:59:19.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:59:19.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:59:04.111793+0000", 2024-09-14T23:59:19.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00037828200000000002 2024-09-14T23:59:19.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:19.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:59:04.111825+0000", 2024-09-14T23:59:19.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1488000000000001e-05 2024-09-14T23:59:19.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:19.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:59:04.111850+0000", 2024-09-14T23:59:19.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5298000000000001e-05 2024-09-14T23:59:19.026 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.026 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:19.026 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.027 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:257823 3.5 3:a4e06f1c:::rbd_data.1118350d5ae5.0000000000000015:head [delete] snapc c=[] ondisk+write+known_if_redirected+supports_pool_eio e164)", 2024-09-14T23:59:19.027 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:59:04.186028+0000", 2024-09-14T23:59:19.027 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 14.741548382, 2024-09-14T23:59:19.027 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0041912110000000002, 2024-09-14T23:59:19.027 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:19.027 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:19.027 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:19.027 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:19.027 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:19.027 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:19.027 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 257823 2024-09-14T23:59:19.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.027 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:19.027 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.027 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:19.028 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:59:04.186028+0000", 2024-09-14T23:59:19.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.028 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:19.028 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:59:04.186028+0000", 2024-09-14T23:59:19.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.028 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:19.028 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:59:04.186030+0000", 2024-09-14T23:59:19.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3740000000000001e-06 2024-09-14T23:59:19.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:19.029 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:59:04.186042+0000", 2024-09-14T23:59:19.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2707e-05 2024-09-14T23:59:19.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:19.029 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:59:04.186063+0000", 2024-09-14T23:59:19.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0823e-05 2024-09-14T23:59:19.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:19.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:59:04.186070+0000", 2024-09-14T23:59:19.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2459999999999997e-06 2024-09-14T23:59:19.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:59:04.186188+0000", 2024-09-14T23:59:19.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000117644 2024-09-14T23:59:19.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:19.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:59:04.186307+0000", 2024-09-14T23:59:19.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011913099999999999 2024-09-14T23:59:19.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2", 2024-09-14T23:59:19.031 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:59:04.186422+0000", 2024-09-14T23:59:19.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011489600000000001 2024-09-14T23:59:19.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.031 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:59:19.031 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:59:04.187555+0000", 2024-09-14T23:59:19.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0011330540000000001 2024-09-14T23:59:19.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.031 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:59:19.031 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:59:04.190127+0000", 2024-09-14T23:59:19.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0025716609999999998 2024-09-14T23:59:19.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:19.032 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:59:04.190176+0000", 2024-09-14T23:59:19.032 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9604000000000001e-05 2024-09-14T23:59:19.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:19.032 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:59:04.190219+0000", 2024-09-14T23:59:19.032 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3071000000000001e-05 2024-09-14T23:59:19.032 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.032 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:19.032 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.032 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.032 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:19.033 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:59:19.033 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T23:59:19.035 INFO:teuthology.orchestra.run.smithi042.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-14T23:59:19.035 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T23:59:19.036 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-14T23:59:19.143 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:59:19.143 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:59:19.143 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:59:19.143 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:59:19.152 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T23:59:19.264 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:59:19.265 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:59:19.265 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:59:19.265 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:59:19.265 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:59:19.273 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T23:59:19.382 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:59:19.382 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:59:19.382 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:59:19.383 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:59:19.383 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.383 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:107472 3.1e 3:7bf21588:::rbd_data.11187de8b0a5.000000000000005d:head [set-alloc-hint object_size 2097152 write_size 2097152,write 317788~1491] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-14T23:59:19.383 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:25.198493+0000", 2024-09-14T23:59:19.383 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 114.181602097, 2024-09-14T23:59:19.383 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.78175149300000002, 2024-09-14T23:59:19.383 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:19.383 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:19.383 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:19.383 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:19.383 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:19.383 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:19.383 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 107472 2024-09-14T23:59:19.384 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.384 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:19.384 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.384 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:19.384 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198493+0000", 2024-09-14T23:59:19.384 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.384 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.384 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.384 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:19.384 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198493+0000", 2024-09-14T23:59:19.384 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.384 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.384 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.384 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:19.385 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198495+0000", 2024-09-14T23:59:19.385 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.826e-06 2024-09-14T23:59:19.385 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.385 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.385 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:19.385 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198503+0000", 2024-09-14T23:59:19.385 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0150000000000003e-06 2024-09-14T23:59:19.385 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.385 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.385 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:19.385 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198521+0000", 2024-09-14T23:59:19.385 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8374000000000001e-05 2024-09-14T23:59:19.385 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.386 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.386 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:19.386 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198533+0000", 2024-09-14T23:59:19.386 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1593000000000001e-05 2024-09-14T23:59:19.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.386 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.386 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.386 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198603+0000", 2024-09-14T23:59:19.386 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0279999999999998e-05 2024-09-14T23:59:19.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.386 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.386 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:19.386 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.198617+0000", 2024-09-14T23:59:19.387 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3517e-05 2024-09-14T23:59:19.387 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.387 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.387 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.387 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.977067+0000", 2024-09-14T23:59:19.387 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.77845065800000002 2024-09-14T23:59:19.387 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.387 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.388 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for missing object", 2024-09-14T23:59:19.388 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.977173+0000", 2024-09-14T23:59:19.388 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010543999999999999 2024-09-14T23:59:19.388 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.388 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.388 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.388 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.978737+0000", 2024-09-14T23:59:19.388 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001564078 2024-09-14T23:59:19.388 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.388 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.388 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:19.388 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.978845+0000", 2024-09-14T23:59:19.388 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000108643 2024-09-14T23:59:19.389 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.389 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.389 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1", 2024-09-14T23:59:19.389 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.978964+0000", 2024-09-14T23:59:19.389 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000119085 2024-09-14T23:59:19.389 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.389 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.389 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:59:19.389 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980109+0000", 2024-09-14T23:59:19.389 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0011442170000000001 2024-09-14T23:59:19.389 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.389 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.389 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:59:19.390 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980130+0000", 2024-09-14T23:59:19.390 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1766000000000001e-05 2024-09-14T23:59:19.390 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.390 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.390 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:19.390 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980171+0000", 2024-09-14T23:59:19.390 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0627999999999998e-05 2024-09-14T23:59:19.390 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.390 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.390 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:19.390 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:25.980244+0000", 2024-09-14T23:59:19.390 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3373000000000004e-05 2024-09-14T23:59:19.390 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.390 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:19.391 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.391 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.391 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.391 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129159 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 5996544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:19.391 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.927783+0000", 2024-09-14T23:59:19.391 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 100.452312206, 2024-09-14T23:59:19.391 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054661122999999999, 2024-09-14T23:59:19.391 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:19.391 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:19.391 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:19.391 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:19.391 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:19.391 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:19.391 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129159 2024-09-14T23:59:19.392 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.392 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:19.392 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.392 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:19.392 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.927783+0000", 2024-09-14T23:59:19.392 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.392 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.392 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.392 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:19.392 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.927783+0000", 2024-09-14T23:59:19.392 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.392 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.392 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.392 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:19.393 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929684+0000", 2024-09-14T23:59:19.393 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0019011080000000001 2024-09-14T23:59:19.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.393 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.393 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:19.393 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929688+0000", 2024-09-14T23:59:19.393 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2610000000000004e-06 2024-09-14T23:59:19.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.393 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.393 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:19.393 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929697+0000", 2024-09-14T23:59:19.393 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0219999999999993e-06 2024-09-14T23:59:19.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.393 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.394 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:19.394 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929702+0000", 2024-09-14T23:59:19.394 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8840000000000002e-06 2024-09-14T23:59:19.394 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.394 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.394 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.394 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.982090+0000", 2024-09-14T23:59:19.394 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052388045000000001 2024-09-14T23:59:19.394 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.394 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.394 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:19.394 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.982208+0000", 2024-09-14T23:59:19.394 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118236 2024-09-14T23:59:19.394 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.395 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.395 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:19.395 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.982444+0000", 2024-09-14T23:59:19.395 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000235567 2024-09-14T23:59:19.395 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.395 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:19.395 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.395 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.395 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.395 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129166 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 8192000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:19.395 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.929794+0000", 2024-09-14T23:59:19.395 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 100.450301455, 2024-09-14T23:59:19.395 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054471973, 2024-09-14T23:59:19.395 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:19.396 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:19.396 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:19.396 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:19.396 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:19.396 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:19.396 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129166 2024-09-14T23:59:19.396 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.396 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:19.396 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.396 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:19.396 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929794+0000", 2024-09-14T23:59:19.396 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.396 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.396 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.397 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:19.397 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929794+0000", 2024-09-14T23:59:19.397 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.397 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.397 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.397 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:19.397 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929794+0000", 2024-09-14T23:59:19.397 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6899999999999997e-07 2024-09-14T23:59:19.397 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.397 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.397 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:19.397 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929797+0000", 2024-09-14T23:59:19.397 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.328e-06 2024-09-14T23:59:19.398 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.398 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.398 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:19.398 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929802+0000", 2024-09-14T23:59:19.398 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7309999999999998e-06 2024-09-14T23:59:19.398 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.398 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.398 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:19.398 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929807+0000", 2024-09-14T23:59:19.398 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4349999999999999e-06 2024-09-14T23:59:19.398 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.398 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.398 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.398 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984063+0000", 2024-09-14T23:59:19.398 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054256331999999997 2024-09-14T23:59:19.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.399 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.399 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:19.399 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984177+0000", 2024-09-14T23:59:19.399 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114225 2024-09-14T23:59:19.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.399 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.399 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:19.399 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984266+0000", 2024-09-14T23:59:19.399 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8352999999999999e-05 2024-09-14T23:59:19.399 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.399 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:19.399 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.400 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.400 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.400 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129167 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 6782976~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:19.400 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.929827+0000", 2024-09-14T23:59:19.400 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 100.45026804600001, 2024-09-14T23:59:19.400 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054637593999999998, 2024-09-14T23:59:19.400 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:19.400 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:19.400 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:19.400 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:19.400 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:19.400 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:19.400 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129167 2024-09-14T23:59:19.400 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.401 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:19.401 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.401 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:19.401 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929827+0000", 2024-09-14T23:59:19.401 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.401 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.401 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.401 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:19.401 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929827+0000", 2024-09-14T23:59:19.401 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.401 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.401 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.401 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:19.401 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929827+0000", 2024-09-14T23:59:19.402 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4000000000000002e-07 2024-09-14T23:59:19.402 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.402 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.402 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:19.402 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929830+0000", 2024-09-14T23:59:19.402 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.419e-06 2024-09-14T23:59:19.402 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.402 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.402 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:19.402 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929837+0000", 2024-09-14T23:59:19.402 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0450000000000003e-06 2024-09-14T23:59:19.402 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.402 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.402 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:19.403 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929842+0000", 2024-09-14T23:59:19.403 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.69e-06 2024-09-14T23:59:19.403 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.403 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.403 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.403 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984278+0000", 2024-09-14T23:59:19.403 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054436516999999997 2024-09-14T23:59:19.403 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.403 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.403 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:19.403 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984331+0000", 2024-09-14T23:59:19.403 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2590999999999998e-05 2024-09-14T23:59:19.403 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.403 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.403 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:19.404 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984465+0000", 2024-09-14T23:59:19.404 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013389200000000001 2024-09-14T23:59:19.404 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.404 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:19.404 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.404 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.404 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.404 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129168 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 8224768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:19.404 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.929849+0000", 2024-09-14T23:59:19.404 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 100.450246488, 2024-09-14T23:59:19.404 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054870368000000003, 2024-09-14T23:59:19.404 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:19.404 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:19.404 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:19.405 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:19.405 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:19.405 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:19.405 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129168 2024-09-14T23:59:19.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.405 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:19.405 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.405 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:19.405 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929849+0000", 2024-09-14T23:59:19.405 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.405 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.405 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:19.405 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.929849+0000", 2024-09-14T23:59:19.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.406 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:19.406 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931681+0000", 2024-09-14T23:59:19.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018326989999999999 2024-09-14T23:59:19.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.406 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:19.406 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931686+0000", 2024-09-14T23:59:19.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7190000000000001e-06 2024-09-14T23:59:19.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.406 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:19.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931697+0000", 2024-09-14T23:59:19.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1287999999999999e-05 2024-09-14T23:59:19.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.407 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:19.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931704+0000", 2024-09-14T23:59:19.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3339999999999998e-06 2024-09-14T23:59:19.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.407 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984505+0000", 2024-09-14T23:59:19.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052801684000000002 2024-09-14T23:59:19.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.408 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:19.408 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984621+0000", 2024-09-14T23:59:19.408 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000116165 2024-09-14T23:59:19.408 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.408 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.408 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:19.408 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.984719+0000", 2024-09-14T23:59:19.408 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7479000000000006e-05 2024-09-14T23:59:19.408 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.408 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:19.408 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.408 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.408 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.408 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129176 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 10452992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:19.409 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.931830+0000", 2024-09-14T23:59:19.409 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 100.44826472699999, 2024-09-14T23:59:19.409 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054513292999999997, 2024-09-14T23:59:19.409 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:19.409 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:19.409 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:19.409 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:19.409 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:19.409 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:19.409 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129176 2024-09-14T23:59:19.409 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.409 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:19.409 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.410 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:19.410 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931830+0000", 2024-09-14T23:59:19.410 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.410 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.410 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:19.410 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.931830+0000", 2024-09-14T23:59:19.410 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.410 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.410 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:19.410 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.933678+0000", 2024-09-14T23:59:19.410 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018472440000000001 2024-09-14T23:59:19.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.411 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.411 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:19.411 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.933686+0000", 2024-09-14T23:59:19.411 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0239999999999997e-06 2024-09-14T23:59:19.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.411 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.411 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:19.411 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.933694+0000", 2024-09-14T23:59:19.411 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8400000000000001e-06 2024-09-14T23:59:19.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.411 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.411 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:19.411 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.933701+0000", 2024-09-14T23:59:19.411 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2870000000000001e-06 2024-09-14T23:59:19.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.412 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.412 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.412 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.986165+0000", 2024-09-14T23:59:19.412 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052463900000000001 2024-09-14T23:59:19.412 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.412 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.412 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:19.412 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.986259+0000", 2024-09-14T23:59:19.412 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4435e-05 2024-09-14T23:59:19.412 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.412 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.412 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:19.412 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.986344+0000", 2024-09-14T23:59:19.412 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4562999999999997e-05 2024-09-14T23:59:19.413 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.413 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:19.413 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.413 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.413 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.413 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129201 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 10289152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:19.413 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.935960+0000", 2024-09-14T23:59:19.413 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 100.444135265, 2024-09-14T23:59:19.413 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054338487999999997, 2024-09-14T23:59:19.413 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:19.413 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:19.413 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:19.413 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:19.413 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:19.414 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:19.414 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129201 2024-09-14T23:59:19.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.414 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:19.414 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.414 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:19.414 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935960+0000", 2024-09-14T23:59:19.414 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.414 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.414 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:19.414 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935960+0000", 2024-09-14T23:59:19.414 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.415 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.415 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:19.415 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935960+0000", 2024-09-14T23:59:19.415 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.72e-07 2024-09-14T23:59:19.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.415 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.415 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:19.415 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935963+0000", 2024-09-14T23:59:19.415 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5390000000000002e-06 2024-09-14T23:59:19.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.415 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.415 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:19.415 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935971+0000", 2024-09-14T23:59:19.415 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4589999999999998e-06 2024-09-14T23:59:19.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.416 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.416 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:19.416 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935977+0000", 2024-09-14T23:59:19.416 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4110000000000004e-06 2024-09-14T23:59:19.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.416 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.416 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.416 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990213+0000", 2024-09-14T23:59:19.416 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054236642000000002 2024-09-14T23:59:19.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.416 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.416 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:19.416 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990257+0000", 2024-09-14T23:59:19.416 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3696000000000003e-05 2024-09-14T23:59:19.417 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.417 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.417 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:19.417 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990298+0000", 2024-09-14T23:59:19.417 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1369000000000003e-05 2024-09-14T23:59:19.417 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.417 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:19.417 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.417 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.417 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.417 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129202 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 11370496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:19.417 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.935986+0000", 2024-09-14T23:59:19.417 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 100.44410864699999, 2024-09-14T23:59:19.417 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054543700000000001, 2024-09-14T23:59:19.418 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:19.418 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:19.418 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:19.418 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:19.418 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:19.418 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:19.418 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129202 2024-09-14T23:59:19.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.418 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:19.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.418 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:19.418 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935986+0000", 2024-09-14T23:59:19.418 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.419 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:19.419 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.935986+0000", 2024-09-14T23:59:19.419 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.419 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.419 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:19.419 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937678+0000", 2024-09-14T23:59:19.419 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016917849999999999 2024-09-14T23:59:19.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.419 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.419 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:19.419 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937682+0000", 2024-09-14T23:59:19.419 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2400000000000001e-06 2024-09-14T23:59:19.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.419 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.420 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:19.420 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937695+0000", 2024-09-14T23:59:19.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2999e-05 2024-09-14T23:59:19.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.420 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.420 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:19.420 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937700+0000", 2024-09-14T23:59:19.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8420000000000004e-06 2024-09-14T23:59:19.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.420 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.420 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.420 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990336+0000", 2024-09-14T23:59:19.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052636060999999998 2024-09-14T23:59:19.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.421 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.421 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:19.421 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990435+0000", 2024-09-14T23:59:19.421 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9015e-05 2024-09-14T23:59:19.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.421 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.421 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:19.421 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.990530+0000", 2024-09-14T23:59:19.421 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4758000000000003e-05 2024-09-14T23:59:19.421 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.421 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:19.421 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.421 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.421 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129214 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 11468800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:19.422 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.937939+0000", 2024-09-14T23:59:19.422 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 100.44215573300001, 2024-09-14T23:59:19.422 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054463733, 2024-09-14T23:59:19.422 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:19.422 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:19.422 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:19.422 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:19.422 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:19.422 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:19.422 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129214 2024-09-14T23:59:19.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.422 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:19.422 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.422 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:19.422 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937939+0000", 2024-09-14T23:59:19.423 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.423 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:19.423 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.937939+0000", 2024-09-14T23:59:19.423 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.423 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:19.423 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939674+0000", 2024-09-14T23:59:19.423 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017342589999999999 2024-09-14T23:59:19.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.423 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:19.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939678+0000", 2024-09-14T23:59:19.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2590000000000002e-06 2024-09-14T23:59:19.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:19.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939699+0000", 2024-09-14T23:59:19.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1143000000000001e-05 2024-09-14T23:59:19.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:19.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939706+0000", 2024-09-14T23:59:19.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0269999999999998e-06 2024-09-14T23:59:19.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.425 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.992221+0000", 2024-09-14T23:59:19.425 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052514981000000002 2024-09-14T23:59:19.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.425 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:19.425 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.992321+0000", 2024-09-14T23:59:19.425 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9625999999999998e-05 2024-09-14T23:59:19.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.426 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:19.426 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.992403+0000", 2024-09-14T23:59:19.426 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2437999999999999e-05 2024-09-14T23:59:19.426 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.426 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:19.426 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.426 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.426 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.426 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129226 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 14090240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:19.426 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.939867+0000", 2024-09-14T23:59:19.426 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 100.44022780500001, 2024-09-14T23:59:19.426 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054523573999999998, 2024-09-14T23:59:19.426 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:19.426 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:19.427 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:19.427 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:19.427 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:19.427 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:19.427 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129226 2024-09-14T23:59:19.427 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.427 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:19.427 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.427 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:19.427 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939867+0000", 2024-09-14T23:59:19.427 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.427 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.427 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.427 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:19.427 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.939867+0000", 2024-09-14T23:59:19.428 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.428 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.428 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.428 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:19.428 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941669+0000", 2024-09-14T23:59:19.428 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018021549999999999 2024-09-14T23:59:19.428 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.428 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.428 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:19.428 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941673+0000", 2024-09-14T23:59:19.428 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1269999999999999e-06 2024-09-14T23:59:19.428 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.428 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.428 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:19.429 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941679+0000", 2024-09-14T23:59:19.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1879999999999999e-06 2024-09-14T23:59:19.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.429 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:19.429 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941682+0000", 2024-09-14T23:59:19.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7110000000000001e-06 2024-09-14T23:59:19.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.429 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.429 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.994201+0000", 2024-09-14T23:59:19.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052518265000000001 2024-09-14T23:59:19.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.430 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:19.430 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.994302+0000", 2024-09-14T23:59:19.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101653 2024-09-14T23:59:19.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.430 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:19.430 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.994391+0000", 2024-09-14T23:59:19.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8474999999999996e-05 2024-09-14T23:59:19.430 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.430 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:19.430 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.431 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.431 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129237 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 12386304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:19.431 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.941825+0000", 2024-09-14T23:59:19.431 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 100.438270043, 2024-09-14T23:59:19.431 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054368076000000001, 2024-09-14T23:59:19.431 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:19.431 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:19.431 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:19.431 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:19.431 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:19.431 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:19.431 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129237 2024-09-14T23:59:19.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.431 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:19.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.432 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:19.432 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941825+0000", 2024-09-14T23:59:19.432 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.432 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:19.432 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941825+0000", 2024-09-14T23:59:19.432 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.432 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:19.432 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941825+0000", 2024-09-14T23:59:19.432 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.29e-07 2024-09-14T23:59:19.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.433 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:19.433 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941826+0000", 2024-09-14T23:59:19.433 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0999999999999997e-07 2024-09-14T23:59:19.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.433 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:19.433 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941834+0000", 2024-09-14T23:59:19.433 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5519999999999999e-06 2024-09-14T23:59:19.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.433 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:19.433 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941843+0000", 2024-09-14T23:59:19.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2960000000000004e-06 2024-09-14T23:59:19.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.434 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.434 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996108+0000", 2024-09-14T23:59:19.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054265200999999999 2024-09-14T23:59:19.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.434 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:19.434 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996152+0000", 2024-09-14T23:59:19.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4066999999999997e-05 2024-09-14T23:59:19.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.434 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:19.435 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996193+0000", 2024-09-14T23:59:19.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0821000000000001e-05 2024-09-14T23:59:19.435 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.435 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:19.435 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.435 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129238 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 13828096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:19.435 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.941851+0000", 2024-09-14T23:59:19.435 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 100.438243719, 2024-09-14T23:59:19.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.05458031, 2024-09-14T23:59:19.435 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:19.435 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:19.435 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:19.436 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:19.436 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:19.436 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:19.436 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129238 2024-09-14T23:59:19.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.436 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:19.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.436 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:19.436 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941851+0000", 2024-09-14T23:59:19.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.436 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:19.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.941851+0000", 2024-09-14T23:59:19.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.437 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:19.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943670+0000", 2024-09-14T23:59:19.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018181930000000001 2024-09-14T23:59:19.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.437 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:19.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943673+0000", 2024-09-14T23:59:19.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3790000000000002e-06 2024-09-14T23:59:19.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:19.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943679+0000", 2024-09-14T23:59:19.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8719999999999999e-06 2024-09-14T23:59:19.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:19.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943683+0000", 2024-09-14T23:59:19.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8539999999999999e-06 2024-09-14T23:59:19.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996233+0000", 2024-09-14T23:59:19.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052550690999999997 2024-09-14T23:59:19.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.439 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:19.439 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996339+0000", 2024-09-14T23:59:19.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105989 2024-09-14T23:59:19.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.439 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:19.439 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.996432+0000", 2024-09-14T23:59:19.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2331999999999996e-05 2024-09-14T23:59:19.439 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.439 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:19.439 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.440 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129249 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 12124160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:19.440 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.943826+0000", 2024-09-14T23:59:19.440 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 100.436269228, 2024-09-14T23:59:19.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054373266000000003, 2024-09-14T23:59:19.440 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:19.440 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:19.440 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:19.440 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:19.440 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:19.440 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:19.440 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129249 2024-09-14T23:59:19.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.440 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:19.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:19.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943826+0000", 2024-09-14T23:59:19.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:19.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943826+0000", 2024-09-14T23:59:19.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:19.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943826+0000", 2024-09-14T23:59:19.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.03e-07 2024-09-14T23:59:19.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:19.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943827+0000", 2024-09-14T23:59:19.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.08e-06 2024-09-14T23:59:19.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:19.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943834+0000", 2024-09-14T23:59:19.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4119999999999998e-06 2024-09-14T23:59:19.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:19.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943836+0000", 2024-09-14T23:59:19.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7369999999999998e-06 2024-09-14T23:59:19.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.443 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.443 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998115+0000", 2024-09-14T23:59:19.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054279040000000001 2024-09-14T23:59:19.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.443 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:19.443 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998159+0000", 2024-09-14T23:59:19.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3486999999999999e-05 2024-09-14T23:59:19.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:19.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998199+0000", 2024-09-14T23:59:19.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0306999999999999e-05 2024-09-14T23:59:19.444 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.444 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:19.444 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.444 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129250 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 13565952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:19.444 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.943840+0000", 2024-09-14T23:59:19.444 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 100.436255278, 2024-09-14T23:59:19.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054573875000000001, 2024-09-14T23:59:19.444 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:19.444 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:19.445 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:19.445 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:19.445 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:19.445 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:19.445 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129250 2024-09-14T23:59:19.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.445 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:19.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:19.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943840+0000", 2024-09-14T23:59:19.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:19.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.943840+0000", 2024-09-14T23:59:19.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.446 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:19.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945669+0000", 2024-09-14T23:59:19.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018289720000000001 2024-09-14T23:59:19.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.446 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:19.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945671+0000", 2024-09-14T23:59:19.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6869999999999999e-06 2024-09-14T23:59:19.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:19.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945680+0000", 2024-09-14T23:59:19.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5259999999999993e-06 2024-09-14T23:59:19.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:19.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945685+0000", 2024-09-14T23:59:19.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8559999999999998e-06 2024-09-14T23:59:19.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998242+0000", 2024-09-14T23:59:19.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052557505999999997 2024-09-14T23:59:19.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:19.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998329+0000", 2024-09-14T23:59:19.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6410999999999996e-05 2024-09-14T23:59:19.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:19.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.998414+0000", 2024-09-14T23:59:19.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4917e-05 2024-09-14T23:59:19.448 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.448 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:19.448 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.448 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129262 3.10 3:0dc863b7:::rbd_data.1118aa300d48.0000000000000000:23 [read 13303808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:19.449 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.945825+0000", 2024-09-14T23:59:19.449 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 100.434270337, 2024-09-14T23:59:19.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054515942999999997, 2024-09-14T23:59:19.449 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:19.449 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:19.449 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:19.449 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:19.449 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:19.449 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:19.449 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129262 2024-09-14T23:59:19.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.449 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:19.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:19.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945825+0000", 2024-09-14T23:59:19.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.450 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:19.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.945825+0000", 2024-09-14T23:59:19.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.450 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:19.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.947670+0000", 2024-09-14T23:59:19.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018451229999999999 2024-09-14T23:59:19.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:19.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.947672+0000", 2024-09-14T23:59:19.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5289999999999998e-06 2024-09-14T23:59:19.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:19.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.947678+0000", 2024-09-14T23:59:19.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9880000000000001e-06 2024-09-14T23:59:19.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:19.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.947682+0000", 2024-09-14T23:59:19.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.63e-06 2024-09-14T23:59:19.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000183+0000", 2024-09-14T23:59:19.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052500770000000002 2024-09-14T23:59:19.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:19.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000266+0000", 2024-09-14T23:59:19.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3133000000000005e-05 2024-09-14T23:59:19.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:19.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.000341+0000", 2024-09-14T23:59:19.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4770000000000004e-05 2024-09-14T23:59:19.453 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.453 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:19.453 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.453 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:129299 3.10 3:0f1e06ee:::rbd_object_map.1118721be852:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-14T23:59:19.453 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:38.952330+0000", 2024-09-14T23:59:19.453 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 100.42776552399999, 2024-09-14T23:59:19.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054412818000000002, 2024-09-14T23:59:19.453 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:19.453 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:19.453 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:19.454 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:19.454 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:19.454 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:19.454 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 129299 2024-09-14T23:59:19.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.454 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:19.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:19.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952330+0000", 2024-09-14T23:59:19.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:19.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952330+0000", 2024-09-14T23:59:19.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:19.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952330+0000", 2024-09-14T23:59:19.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4900000000000001e-07 2024-09-14T23:59:19.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:19.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952332+0000", 2024-09-14T23:59:19.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2879999999999999e-06 2024-09-14T23:59:19.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:19.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952340+0000", 2024-09-14T23:59:19.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9910000000000002e-06 2024-09-14T23:59:19.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.456 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:19.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:38.952344+0000", 2024-09-14T23:59:19.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0280000000000002e-06 2024-09-14T23:59:19.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.456 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.004722+0000", 2024-09-14T23:59:19.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052377932000000002 2024-09-14T23:59:19.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.456 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:19.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.004778+0000", 2024-09-14T23:59:19.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5795999999999999e-05 2024-09-14T23:59:19.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0", 2024-09-14T23:59:19.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.005016+0000", 2024-09-14T23:59:19.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000238006 2024-09-14T23:59:19.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:59:19.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.006366+0000", 2024-09-14T23:59:19.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001349786 2024-09-14T23:59:19.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:59:19.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.006685+0000", 2024-09-14T23:59:19.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00031970000000000002 2024-09-14T23:59:19.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:19.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.006718+0000", 2024-09-14T23:59:19.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2329000000000002e-05 2024-09-14T23:59:19.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:19.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:39.006742+0000", 2024-09-14T23:59:19.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4613e-05 2024-09-14T23:59:19.458 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.458 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:19.459 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.459 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:186210 3.d 3.b2ff3acd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:59:19.459 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.129044+0000", 2024-09-14T23:59:19.459 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 72.251051161000007, 2024-09-14T23:59:19.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.84555443900000005, 2024-09-14T23:59:19.459 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:19.459 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:19.459 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:59:19.459 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:19.459 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:19.459 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:19.459 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 186210 2024-09-14T23:59:19.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.460 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:19.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:19.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129044+0000", 2024-09-14T23:59:19.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:19.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129044+0000", 2024-09-14T23:59:19.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:19.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129046+0000", 2024-09-14T23:59:19.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.742e-06 2024-09-14T23:59:19.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:19.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129052+0000", 2024-09-14T23:59:19.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7390000000000002e-06 2024-09-14T23:59:19.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:19.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129067+0000", 2024-09-14T23:59:19.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.417e-05 2024-09-14T23:59:19.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:19.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.129076+0000", 2024-09-14T23:59:19.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2289999999999992e-06 2024-09-14T23:59:19.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.130947+0000", 2024-09-14T23:59:19.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001871075 2024-09-14T23:59:19.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:19.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.130955+0000", 2024-09-14T23:59:19.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4500000000000004e-06 2024-09-14T23:59:19.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974587+0000", 2024-09-14T23:59:19.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.84363205500000005 2024-09-14T23:59:19.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:19.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974598+0000", 2024-09-14T23:59:19.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0978999999999999e-05 2024-09-14T23:59:19.463 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.463 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:19.463 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.464 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187415 3.d 3.b2ff3acd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:59:19.464 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.254681+0000", 2024-09-14T23:59:19.464 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 72.125413640000005, 2024-09-14T23:59:19.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.71995530699999999, 2024-09-14T23:59:19.464 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:19.464 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:19.464 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:59:19.464 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:19.464 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:19.464 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:19.464 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187415 2024-09-14T23:59:19.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.465 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:19.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:19.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254681+0000", 2024-09-14T23:59:19.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:19.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254681+0000", 2024-09-14T23:59:19.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:19.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254682+0000", 2024-09-14T23:59:19.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2e-07 2024-09-14T23:59:19.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:19.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254684+0000", 2024-09-14T23:59:19.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2730000000000001e-06 2024-09-14T23:59:19.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:19.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254689+0000", 2024-09-14T23:59:19.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4769999999999997e-06 2024-09-14T23:59:19.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:19.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254692+0000", 2024-09-14T23:59:19.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9890000000000001e-06 2024-09-14T23:59:19.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254721+0000", 2024-09-14T23:59:19.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9762999999999999e-05 2024-09-14T23:59:19.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:19.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.254725+0000", 2024-09-14T23:59:19.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9650000000000002e-06 2024-09-14T23:59:19.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974628+0000", 2024-09-14T23:59:19.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.71990235499999999 2024-09-14T23:59:19.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:19.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974637+0000", 2024-09-14T23:59:19.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0650000000000005e-06 2024-09-14T23:59:19.468 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.468 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:19.468 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.469 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:187457 3.d 3.b2ff3acd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-14T23:59:19.469 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:07.257980+0000", 2024-09-14T23:59:19.469 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 72.122115066000006, 2024-09-14T23:59:19.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.71671711199999999, 2024-09-14T23:59:19.469 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:19.469 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:19.469 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:59:19.469 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:19.469 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:19.469 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:19.469 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 187457 2024-09-14T23:59:19.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.469 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:19.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:19.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257980+0000", 2024-09-14T23:59:19.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:19.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257980+0000", 2024-09-14T23:59:19.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:19.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257980+0000", 2024-09-14T23:59:19.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.17e-07 2024-09-14T23:59:19.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:19.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257981+0000", 2024-09-14T23:59:19.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.172e-06 2024-09-14T23:59:19.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:19.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257984+0000", 2024-09-14T23:59:19.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9550000000000001e-06 2024-09-14T23:59:19.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:19.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.257987+0000", 2024-09-14T23:59:19.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5799999999999999e-06 2024-09-14T23:59:19.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.258030+0000", 2024-09-14T23:59:19.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3118999999999998e-05 2024-09-14T23:59:19.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:19.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.258038+0000", 2024-09-14T23:59:19.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6850000000000001e-06 2024-09-14T23:59:19.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974688+0000", 2024-09-14T23:59:19.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.71665016000000004 2024-09-14T23:59:19.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:19.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:07.974697+0000", 2024-09-14T23:59:19.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2240000000000001e-06 2024-09-14T23:59:19.473 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.473 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:19.473 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.473 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:251656 3.c 3:32436721:::rbd_object_map.1118e09062ff:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-14T23:59:19.474 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:27.433419+0000", 2024-09-14T23:59:19.474 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.946676392000001, 2024-09-14T23:59:19.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.55083879499999999, 2024-09-14T23:59:19.474 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:19.474 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:19.474 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T23:59:19.474 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:19.474 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:19.474 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:19.474 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 251656 2024-09-14T23:59:19.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.474 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:19.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:19.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433419+0000", 2024-09-14T23:59:19.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:19.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433419+0000", 2024-09-14T23:59:19.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:19.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433419+0000", 2024-09-14T23:59:19.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1500000000000003e-07 2024-09-14T23:59:19.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:19.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433424+0000", 2024-09-14T23:59:19.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2409999999999997e-06 2024-09-14T23:59:19.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:19.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433435+0000", 2024-09-14T23:59:19.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1009000000000001e-05 2024-09-14T23:59:19.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:19.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433440+0000", 2024-09-14T23:59:19.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1950000000000002e-06 2024-09-14T23:59:19.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433487+0000", 2024-09-14T23:59:19.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7184000000000001e-05 2024-09-14T23:59:19.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:19.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.433494+0000", 2024-09-14T23:59:19.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5490000000000003e-06 2024-09-14T23:59:19.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.981923+0000", 2024-09-14T23:59:19.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.54842898600000001 2024-09-14T23:59:19.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:19.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.982097+0000", 2024-09-14T23:59:19.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017428699999999999 2024-09-14T23:59:19.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5", 2024-09-14T23:59:19.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.982334+0000", 2024-09-14T23:59:19.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00023700899999999999 2024-09-14T23:59:19.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-14T23:59:19.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.983927+0000", 2024-09-14T23:59:19.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0015932710000000001 2024-09-14T23:59:19.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-14T23:59:19.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.984128+0000", 2024-09-14T23:59:19.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020104199999999999 2024-09-14T23:59:19.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-14T23:59:19.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.984212+0000", 2024-09-14T23:59:19.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3709999999999996e-05 2024-09-14T23:59:19.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:19.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:27.984257+0000", 2024-09-14T23:59:19.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5497000000000002e-05 2024-09-14T23:59:19.480 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.480 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:19.480 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.480 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.481 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:19.481 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:59:19.482 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T23:59:19.590 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:59:19.590 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:59:19.590 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-14T23:59:19.590 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:59:19.599 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T23:59:19.706 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:59:19.706 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-14T23:59:19.706 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-14T23:59:19.706 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-14T23:59:19.706 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:59:19.715 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T23:59:19.824 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-14T23:59:19.824 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-14T23:59:19.824 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-14T23:59:19.824 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-14T23:59:19.824 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.824 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69262 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 4800512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:59:19.824 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300811+0000", 2024-09-14T23:59:19.825 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 124.520576433, 2024-09-14T23:59:19.825 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050403796000000001, 2024-09-14T23:59:19.825 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:19.825 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:19.825 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:19.825 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:19.825 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:19.825 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:19.825 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69262 2024-09-14T23:59:19.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.825 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:19.825 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.825 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:19.825 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300811+0000", 2024-09-14T23:59:19.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.826 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.826 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:19.826 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300811+0000", 2024-09-14T23:59:19.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.826 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.826 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:19.826 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300811+0000", 2024-09-14T23:59:19.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5700000000000002e-07 2024-09-14T23:59:19.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.827 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.827 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:19.827 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300815+0000", 2024-09-14T23:59:19.827 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.087e-06 2024-09-14T23:59:19.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.827 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.827 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:19.827 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300823+0000", 2024-09-14T23:59:19.827 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8720000000000002e-06 2024-09-14T23:59:19.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.827 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.827 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:19.827 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300830+0000", 2024-09-14T23:59:19.827 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0589999999999997e-06 2024-09-14T23:59:19.828 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.828 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.828 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.828 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351020+0000", 2024-09-14T23:59:19.828 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050189552999999998 2024-09-14T23:59:19.828 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.828 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.828 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:19.828 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351112+0000", 2024-09-14T23:59:19.828 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2615999999999996e-05 2024-09-14T23:59:19.828 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.828 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.828 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:19.829 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351214+0000", 2024-09-14T23:59:19.829 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101952 2024-09-14T23:59:19.829 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.829 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:19.829 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.829 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.829 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.829 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69263 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 4915200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:59:19.829 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300842+0000", 2024-09-14T23:59:19.829 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 124.520544929, 2024-09-14T23:59:19.829 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050590637000000001, 2024-09-14T23:59:19.829 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:19.829 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:19.830 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:19.830 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:19.830 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:19.830 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:19.830 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69263 2024-09-14T23:59:19.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.830 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:19.830 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.830 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:19.830 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300842+0000", 2024-09-14T23:59:19.830 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.830 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.831 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:19.831 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300842+0000", 2024-09-14T23:59:19.831 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.831 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.831 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:19.831 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300843+0000", 2024-09-14T23:59:19.831 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4000000000000002e-07 2024-09-14T23:59:19.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.831 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.831 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:19.831 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300846+0000", 2024-09-14T23:59:19.831 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8720000000000002e-06 2024-09-14T23:59:19.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.831 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.832 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:19.832 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300853+0000", 2024-09-14T23:59:19.832 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0450000000000003e-06 2024-09-14T23:59:19.832 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.832 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.832 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:19.832 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300859+0000", 2024-09-14T23:59:19.832 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9510000000000003e-06 2024-09-14T23:59:19.832 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.832 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.832 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.832 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351260+0000", 2024-09-14T23:59:19.832 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050401654999999997 2024-09-14T23:59:19.833 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.833 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.833 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:19.833 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351353+0000", 2024-09-14T23:59:19.833 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2785e-05 2024-09-14T23:59:19.833 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.833 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.833 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:19.833 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351433+0000", 2024-09-14T23:59:19.833 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9789000000000005e-05 2024-09-14T23:59:19.833 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.833 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:19.833 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.833 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.833 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.834 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69264 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5029888~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:59:19.834 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300866+0000", 2024-09-14T23:59:19.834 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 124.520520996, 2024-09-14T23:59:19.834 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050794063, 2024-09-14T23:59:19.834 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:19.834 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:19.834 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:19.834 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:19.834 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:19.834 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:19.834 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69264 2024-09-14T23:59:19.834 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.834 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:19.834 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.835 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:19.835 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300866+0000", 2024-09-14T23:59:19.835 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.835 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.835 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.835 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:19.835 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300866+0000", 2024-09-14T23:59:19.835 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.835 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.835 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.835 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:19.835 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300866+0000", 2024-09-14T23:59:19.835 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8799999999999998e-07 2024-09-14T23:59:19.835 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.835 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.836 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:19.836 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300869+0000", 2024-09-14T23:59:19.836 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0880000000000002e-06 2024-09-14T23:59:19.836 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.836 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.836 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:19.836 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300875+0000", 2024-09-14T23:59:19.836 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9280000000000004e-06 2024-09-14T23:59:19.836 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.836 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.836 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:19.836 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300882+0000", 2024-09-14T23:59:19.836 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4740000000000002e-06 2024-09-14T23:59:19.837 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.837 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.837 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.837 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351472+0000", 2024-09-14T23:59:19.837 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050590006999999999 2024-09-14T23:59:19.837 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.837 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.837 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:19.837 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351563+0000", 2024-09-14T23:59:19.837 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0762999999999998e-05 2024-09-14T23:59:19.837 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.837 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.837 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:19.838 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351660+0000", 2024-09-14T23:59:19.838 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7415000000000002e-05 2024-09-14T23:59:19.838 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.838 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:19.838 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.838 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.838 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.838 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69265 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5144576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:59:19.838 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300890+0000", 2024-09-14T23:59:19.838 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 124.520497492, 2024-09-14T23:59:19.838 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050921389999999997, 2024-09-14T23:59:19.838 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:19.838 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:19.839 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:19.839 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:19.839 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:19.839 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:19.839 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69265 2024-09-14T23:59:19.839 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.839 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:19.839 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.839 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:19.839 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300890+0000", 2024-09-14T23:59:19.839 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.839 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.839 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.840 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:19.840 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300890+0000", 2024-09-14T23:59:19.840 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.840 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.840 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.840 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:19.840 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300890+0000", 2024-09-14T23:59:19.840 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2500000000000001e-07 2024-09-14T23:59:19.840 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.840 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.840 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:19.840 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300892+0000", 2024-09-14T23:59:19.840 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2469999999999998e-06 2024-09-14T23:59:19.841 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.841 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.841 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:19.841 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300899+0000", 2024-09-14T23:59:19.841 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5509999999999996e-06 2024-09-14T23:59:19.841 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.841 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.841 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:19.841 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300904+0000", 2024-09-14T23:59:19.841 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3299999999999998e-06 2024-09-14T23:59:19.841 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.841 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.841 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.842 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351707+0000", 2024-09-14T23:59:19.842 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050802673999999999 2024-09-14T23:59:19.842 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.842 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.842 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:19.842 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351755+0000", 2024-09-14T23:59:19.842 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8362000000000002e-05 2024-09-14T23:59:19.842 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.842 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.842 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:19.842 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351811+0000", 2024-09-14T23:59:19.842 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5801000000000003e-05 2024-09-14T23:59:19.842 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.843 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:19.843 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.843 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.843 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.843 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69266 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5259264~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:59:19.843 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300911+0000", 2024-09-14T23:59:19.843 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 124.520476419, 2024-09-14T23:59:19.843 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051125089999999998, 2024-09-14T23:59:19.843 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:19.843 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:19.843 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:19.843 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:19.843 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:19.844 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:19.844 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69266 2024-09-14T23:59:19.844 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.844 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:19.844 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.844 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:19.844 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300911+0000", 2024-09-14T23:59:19.844 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.844 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.844 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.844 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:19.844 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300911+0000", 2024-09-14T23:59:19.844 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.845 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.845 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.845 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:19.845 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300911+0000", 2024-09-14T23:59:19.845 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7e-07 2024-09-14T23:59:19.845 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.845 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.845 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:19.845 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300913+0000", 2024-09-14T23:59:19.845 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9300000000000002e-06 2024-09-14T23:59:19.845 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.845 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.845 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:19.845 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300919+0000", 2024-09-14T23:59:19.846 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.489e-06 2024-09-14T23:59:19.846 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.846 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.846 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:19.846 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300925+0000", 2024-09-14T23:59:19.846 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1719999999999998e-06 2024-09-14T23:59:19.846 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.846 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.846 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.846 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351847+0000", 2024-09-14T23:59:19.846 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050922621000000001 2024-09-14T23:59:19.846 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.847 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.847 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:19.847 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.351938+0000", 2024-09-14T23:59:19.847 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0830999999999998e-05 2024-09-14T23:59:19.847 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.847 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.847 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:19.847 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352036+0000", 2024-09-14T23:59:19.847 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7676999999999993e-05 2024-09-14T23:59:19.847 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.847 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:19.847 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.847 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.847 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.848 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69267 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5373952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:59:19.848 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300945+0000", 2024-09-14T23:59:19.848 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 124.520441624, 2024-09-14T23:59:19.848 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051217934, 2024-09-14T23:59:19.848 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:19.848 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:19.848 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:19.848 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:19.848 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:19.848 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:19.848 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69267 2024-09-14T23:59:19.848 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.848 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:19.848 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.849 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:19.849 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300945+0000", 2024-09-14T23:59:19.849 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.849 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.849 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.849 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:19.849 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300945+0000", 2024-09-14T23:59:19.849 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.849 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.849 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.850 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:19.850 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300946+0000", 2024-09-14T23:59:19.850 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2600000000000002e-07 2024-09-14T23:59:19.850 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.850 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.850 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:19.850 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300948+0000", 2024-09-14T23:59:19.850 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4940000000000002e-06 2024-09-14T23:59:19.851 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.851 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.851 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:19.851 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300956+0000", 2024-09-14T23:59:19.851 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.622e-06 2024-09-14T23:59:19.851 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.851 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.851 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:19.852 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300962+0000", 2024-09-14T23:59:19.852 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5790000000000003e-06 2024-09-14T23:59:19.852 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.852 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.852 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.852 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352063+0000", 2024-09-14T23:59:19.852 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051101281999999998 2024-09-14T23:59:19.852 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.852 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.852 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:19.852 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352110+0000", 2024-09-14T23:59:19.852 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7401000000000002e-05 2024-09-14T23:59:19.852 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.853 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.853 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:19.853 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352163+0000", 2024-09-14T23:59:19.853 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3029999999999999e-05 2024-09-14T23:59:19.853 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.853 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:19.853 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.853 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.853 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.853 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69268 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5488640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:59:19.853 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.300969+0000", 2024-09-14T23:59:19.853 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 124.520417952, 2024-09-14T23:59:19.853 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051413420000000001, 2024-09-14T23:59:19.853 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:19.854 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:19.854 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:19.854 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:19.854 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:19.854 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:19.854 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69268 2024-09-14T23:59:19.854 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.854 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:19.854 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.854 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:19.854 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300969+0000", 2024-09-14T23:59:19.854 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.854 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.854 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.855 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:19.855 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300969+0000", 2024-09-14T23:59:19.855 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.855 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.855 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.855 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:19.855 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300970+0000", 2024-09-14T23:59:19.855 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7899999999999999e-07 2024-09-14T23:59:19.855 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.855 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.855 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:19.855 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300982+0000", 2024-09-14T23:59:19.855 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2741e-05 2024-09-14T23:59:19.855 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.855 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.855 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:19.856 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300990+0000", 2024-09-14T23:59:19.856 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8169999999999999e-06 2024-09-14T23:59:19.856 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.856 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.856 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:19.856 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.300995+0000", 2024-09-14T23:59:19.856 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2240000000000003e-06 2024-09-14T23:59:19.856 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.856 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.856 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.856 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352200+0000", 2024-09-14T23:59:19.856 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051204223 2024-09-14T23:59:19.856 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.856 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.857 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:19.857 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352285+0000", 2024-09-14T23:59:19.857 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5519999999999994e-05 2024-09-14T23:59:19.857 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.857 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.857 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:19.857 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352383+0000", 2024-09-14T23:59:19.857 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7516000000000006e-05 2024-09-14T23:59:19.857 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.857 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:19.857 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.857 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.857 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.858 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69269 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 5603328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:59:19.858 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.301003+0000", 2024-09-14T23:59:19.858 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 124.52038377, 2024-09-14T23:59:19.858 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051497381000000002, 2024-09-14T23:59:19.858 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:19.858 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:19.858 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:19.858 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:19.858 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:19.858 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:19.858 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69269 2024-09-14T23:59:19.858 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.858 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:19.858 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.859 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:19.859 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.301003+0000", 2024-09-14T23:59:19.859 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.859 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.859 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.859 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:19.859 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.301003+0000", 2024-09-14T23:59:19.859 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.859 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.859 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.859 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:19.859 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302671+0000", 2024-09-14T23:59:19.859 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001667785 2024-09-14T23:59:19.859 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.860 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.860 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:19.860 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302676+0000", 2024-09-14T23:59:19.860 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6600000000000003e-06 2024-09-14T23:59:19.860 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.860 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.860 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:19.860 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302697+0000", 2024-09-14T23:59:19.860 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1288999999999999e-05 2024-09-14T23:59:19.860 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.860 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.860 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:19.860 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302716+0000", 2024-09-14T23:59:19.860 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9128999999999999e-05 2024-09-14T23:59:19.861 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.861 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.861 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.861 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352399+0000", 2024-09-14T23:59:19.861 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049682995000000001 2024-09-14T23:59:19.861 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.861 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.861 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:19.861 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352446+0000", 2024-09-14T23:59:19.861 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6341000000000002e-05 2024-09-14T23:59:19.861 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.861 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.861 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:19.862 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.352501+0000", 2024-09-14T23:59:19.862 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5182000000000001e-05 2024-09-14T23:59:19.862 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.862 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:19.862 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.862 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.862 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.862 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69274 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6176768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:59:19.862 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302854+0000", 2024-09-14T23:59:19.862 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 124.51853311000001, 2024-09-14T23:59:19.862 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050579526, 2024-09-14T23:59:19.862 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:19.862 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:19.862 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:19.863 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:19.863 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:19.863 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:19.863 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69274 2024-09-14T23:59:19.863 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.863 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:19.863 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.863 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:19.863 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302854+0000", 2024-09-14T23:59:19.863 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.863 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.863 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.863 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:19.864 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302854+0000", 2024-09-14T23:59:19.864 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.864 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.864 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.864 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:19.864 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302855+0000", 2024-09-14T23:59:19.864 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0200000000000001e-07 2024-09-14T23:59:19.864 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.864 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.864 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:19.864 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302857+0000", 2024-09-14T23:59:19.864 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.447e-06 2024-09-14T23:59:19.864 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.865 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.865 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:19.865 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302864+0000", 2024-09-14T23:59:19.865 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7730000000000002e-06 2024-09-14T23:59:19.865 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.865 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.865 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:19.865 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302869+0000", 2024-09-14T23:59:19.865 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3199999999999999e-06 2024-09-14T23:59:19.865 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.865 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.865 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.865 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353249+0000", 2024-09-14T23:59:19.865 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050379647999999999 2024-09-14T23:59:19.866 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.866 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.866 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:19.866 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353336+0000", 2024-09-14T23:59:19.866 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6767999999999993e-05 2024-09-14T23:59:19.866 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.866 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.866 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:19.866 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353433+0000", 2024-09-14T23:59:19.866 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7868000000000006e-05 2024-09-14T23:59:19.866 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.866 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:19.866 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.867 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.867 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.867 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69275 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6291456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:59:19.867 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302889+0000", 2024-09-14T23:59:19.867 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 124.51849849200001, 2024-09-14T23:59:19.867 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050663118, 2024-09-14T23:59:19.867 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:19.867 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:19.867 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:19.867 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:19.867 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:19.867 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:19.867 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69275 2024-09-14T23:59:19.867 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.868 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:19.868 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.868 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:19.868 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302889+0000", 2024-09-14T23:59:19.868 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.868 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.868 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.868 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:19.868 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302889+0000", 2024-09-14T23:59:19.868 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.868 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.868 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.868 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:19.869 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302889+0000", 2024-09-14T23:59:19.869 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6399999999999998e-07 2024-09-14T23:59:19.869 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.869 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.869 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:19.869 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302891+0000", 2024-09-14T23:59:19.869 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.339e-06 2024-09-14T23:59:19.869 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.869 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.869 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:19.869 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302898+0000", 2024-09-14T23:59:19.869 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0380000000000002e-06 2024-09-14T23:59:19.869 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.869 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.870 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:19.870 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302904+0000", 2024-09-14T23:59:19.870 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2159999999999997e-06 2024-09-14T23:59:19.870 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.870 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.870 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.870 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353449+0000", 2024-09-14T23:59:19.870 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050545386999999997 2024-09-14T23:59:19.870 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.870 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.871 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:19.871 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353494+0000", 2024-09-14T23:59:19.871 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4592999999999998e-05 2024-09-14T23:59:19.871 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.871 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.871 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:19.871 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353552+0000", 2024-09-14T23:59:19.871 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8180999999999997e-05 2024-09-14T23:59:19.871 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.871 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:19.871 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.872 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.872 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.872 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69276 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6406144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:59:19.872 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302911+0000", 2024-09-14T23:59:19.872 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 124.518476079, 2024-09-14T23:59:19.872 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050865753, 2024-09-14T23:59:19.872 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:19.872 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:19.872 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:19.873 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:19.873 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:19.873 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:19.873 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69276 2024-09-14T23:59:19.873 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.873 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:19.873 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.873 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:19.873 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302911+0000", 2024-09-14T23:59:19.874 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.874 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.874 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.874 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:19.874 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302911+0000", 2024-09-14T23:59:19.874 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.874 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.874 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.874 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:19.875 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302912+0000", 2024-09-14T23:59:19.875 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0800000000000004e-07 2024-09-14T23:59:19.875 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.875 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.875 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:19.875 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302914+0000", 2024-09-14T23:59:19.875 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9589999999999998e-06 2024-09-14T23:59:19.875 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.875 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.875 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:19.875 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302920+0000", 2024-09-14T23:59:19.875 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4579999999999998e-06 2024-09-14T23:59:19.876 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.876 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.876 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:19.876 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302925+0000", 2024-09-14T23:59:19.876 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9429999999999999e-06 2024-09-14T23:59:19.876 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.876 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.876 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.876 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353586+0000", 2024-09-14T23:59:19.876 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050660966000000002 2024-09-14T23:59:19.876 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.876 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.876 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:19.877 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353679+0000", 2024-09-14T23:59:19.877 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3163000000000002e-05 2024-09-14T23:59:19.877 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.877 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.877 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:19.877 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353777+0000", 2024-09-14T23:59:19.877 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7555999999999999e-05 2024-09-14T23:59:19.877 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.877 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:19.877 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.877 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.877 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.877 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69277 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6520832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:59:19.878 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302932+0000", 2024-09-14T23:59:19.878 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 124.518454992, 2024-09-14T23:59:19.878 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050977674000000001, 2024-09-14T23:59:19.878 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:19.878 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:19.878 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:19.878 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:19.878 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:19.878 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:19.878 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69277 2024-09-14T23:59:19.878 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.878 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:19.878 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.878 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:19.879 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302932+0000", 2024-09-14T23:59:19.879 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.879 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.879 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.879 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:19.879 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302932+0000", 2024-09-14T23:59:19.879 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.879 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.879 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.879 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:19.879 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302932+0000", 2024-09-14T23:59:19.879 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3299999999999998e-07 2024-09-14T23:59:19.879 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.879 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.880 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:19.880 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302935+0000", 2024-09-14T23:59:19.880 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2139999999999999e-06 2024-09-14T23:59:19.880 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.880 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.880 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:19.880 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302941+0000", 2024-09-14T23:59:19.880 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5039999999999999e-06 2024-09-14T23:59:19.880 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.880 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.880 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:19.880 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302946+0000", 2024-09-14T23:59:19.880 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2079999999999999e-06 2024-09-14T23:59:19.881 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.881 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.881 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.881 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353805+0000", 2024-09-14T23:59:19.881 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050858588000000003 2024-09-14T23:59:19.881 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.881 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.881 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:19.881 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353849+0000", 2024-09-14T23:59:19.881 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4511000000000001e-05 2024-09-14T23:59:19.881 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.881 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.881 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:19.881 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353910+0000", 2024-09-14T23:59:19.881 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0316000000000003e-05 2024-09-14T23:59:19.882 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.882 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:19.882 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.882 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.882 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.882 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69278 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6635520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:59:19.882 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302953+0000", 2024-09-14T23:59:19.882 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 124.518433864, 2024-09-14T23:59:19.882 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051174059000000001, 2024-09-14T23:59:19.882 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:19.882 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:19.882 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:19.882 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:19.882 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:19.883 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:19.883 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69278 2024-09-14T23:59:19.883 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.883 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:19.883 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.883 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:19.883 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302953+0000", 2024-09-14T23:59:19.883 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.883 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.883 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.883 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:19.883 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302953+0000", 2024-09-14T23:59:19.883 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.884 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.884 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.884 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:19.884 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302954+0000", 2024-09-14T23:59:19.884 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5100000000000001e-07 2024-09-14T23:59:19.884 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.884 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.884 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:19.884 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302964+0000", 2024-09-14T23:59:19.884 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0475000000000001e-05 2024-09-14T23:59:19.884 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.884 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.884 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:19.885 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302970+0000", 2024-09-14T23:59:19.885 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3879999999999997e-06 2024-09-14T23:59:19.885 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.885 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.885 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:19.885 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302975+0000", 2024-09-14T23:59:19.885 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8409999999999999e-06 2024-09-14T23:59:19.885 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.885 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.885 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.885 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.353944+0000", 2024-09-14T23:59:19.885 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050968952999999997 2024-09-14T23:59:19.885 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.886 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.886 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:19.886 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354031+0000", 2024-09-14T23:59:19.886 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6537000000000002e-05 2024-09-14T23:59:19.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.886 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.886 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:19.886 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354127+0000", 2024-09-14T23:59:19.886 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6514000000000005e-05 2024-09-14T23:59:19.886 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.886 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:19.886 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.886 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.887 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69279 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6750208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:59:19.887 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.302984+0000", 2024-09-14T23:59:19.887 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 124.518403513, 2024-09-14T23:59:19.887 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051258845999999997, 2024-09-14T23:59:19.887 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:19.887 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:19.887 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:19.887 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:19.887 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:19.887 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:19.887 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69279 2024-09-14T23:59:19.887 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.887 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:19.887 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.888 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:19.888 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302984+0000", 2024-09-14T23:59:19.888 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.888 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.888 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:19.888 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.302984+0000", 2024-09-14T23:59:19.888 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.888 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.888 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:19.888 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303025+0000", 2024-09-14T23:59:19.888 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1690999999999998e-05 2024-09-14T23:59:19.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.889 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.889 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:19.889 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303029+0000", 2024-09-14T23:59:19.889 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6559999999999998e-06 2024-09-14T23:59:19.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.889 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.889 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:19.889 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303037+0000", 2024-09-14T23:59:19.889 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7950000000000008e-06 2024-09-14T23:59:19.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.889 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.889 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:19.889 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303043+0000", 2024-09-14T23:59:19.890 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9669999999999998e-06 2024-09-14T23:59:19.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.890 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.890 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.890 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354142+0000", 2024-09-14T23:59:19.890 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051099072000000002 2024-09-14T23:59:19.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.890 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.890 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:19.890 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354183+0000", 2024-09-14T23:59:19.890 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0834000000000002e-05 2024-09-14T23:59:19.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.890 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.890 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:19.891 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354242+0000", 2024-09-14T23:59:19.891 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9830999999999997e-05 2024-09-14T23:59:19.891 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.891 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:19.891 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.891 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.891 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69280 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 6864896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:59:19.891 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.303052+0000", 2024-09-14T23:59:19.891 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 124.51833497, 2024-09-14T23:59:19.891 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051406463999999999, 2024-09-14T23:59:19.891 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:19.891 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:19.891 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:19.892 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:19.892 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:19.892 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:19.892 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69280 2024-09-14T23:59:19.892 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.892 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:19.892 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.892 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:19.892 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303052+0000", 2024-09-14T23:59:19.892 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.892 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.892 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.892 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:19.893 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.303052+0000", 2024-09-14T23:59:19.893 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.893 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.893 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.893 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:19.893 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304673+0000", 2024-09-14T23:59:19.893 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001621389 2024-09-14T23:59:19.893 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.893 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.893 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:19.893 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304681+0000", 2024-09-14T23:59:19.893 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8439999999999998e-06 2024-09-14T23:59:19.893 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.893 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.894 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:19.894 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304691+0000", 2024-09-14T23:59:19.894 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8339999999999994e-06 2024-09-14T23:59:19.894 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.894 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.894 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:19.894 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304698+0000", 2024-09-14T23:59:19.894 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3629999999999999e-06 2024-09-14T23:59:19.894 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.894 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.894 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.894 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354278+0000", 2024-09-14T23:59:19.894 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049580618 2024-09-14T23:59:19.894 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.894 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.895 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:19.895 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354364+0000", 2024-09-14T23:59:19.895 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6059999999999999e-05 2024-09-14T23:59:19.895 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.895 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.895 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:19.895 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.354459+0000", 2024-09-14T23:59:19.895 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4356000000000003e-05 2024-09-14T23:59:19.895 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.895 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:19.895 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.895 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.895 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.895 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69286 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 7553024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:59:19.896 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.304811+0000", 2024-09-14T23:59:19.896 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 124.516576199, 2024-09-14T23:59:19.896 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050585512999999999, 2024-09-14T23:59:19.896 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:19.896 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:19.896 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:19.896 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:19.896 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:19.896 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:19.896 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69286 2024-09-14T23:59:19.896 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.896 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:19.896 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.897 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:19.897 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304811+0000", 2024-09-14T23:59:19.897 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.897 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.897 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:19.897 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304811+0000", 2024-09-14T23:59:19.897 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.897 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.897 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:19.897 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304811+0000", 2024-09-14T23:59:19.897 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.89e-07 2024-09-14T23:59:19.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.898 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.898 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:19.898 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304814+0000", 2024-09-14T23:59:19.898 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3199999999999998e-06 2024-09-14T23:59:19.898 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.898 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.898 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:19.898 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304821+0000", 2024-09-14T23:59:19.898 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5689999999999999e-06 2024-09-14T23:59:19.898 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.898 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.898 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:19.899 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304826+0000", 2024-09-14T23:59:19.899 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1200000000000001e-06 2024-09-14T23:59:19.899 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.899 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.899 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.899 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355226+0000", 2024-09-14T23:59:19.899 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050399653000000003 2024-09-14T23:59:19.899 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.899 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.899 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:19.899 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355314+0000", 2024-09-14T23:59:19.899 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8389999999999999e-05 2024-09-14T23:59:19.899 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.900 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.900 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:19.900 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355396+0000", 2024-09-14T23:59:19.900 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2071999999999996e-05 2024-09-14T23:59:19.900 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.900 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:19.900 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.900 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.900 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.900 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69287 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 7667712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:59:19.900 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.304833+0000", 2024-09-14T23:59:19.900 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 124.516553601, 2024-09-14T23:59:19.900 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050664740999999999, 2024-09-14T23:59:19.901 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:19.901 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:19.901 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:19.901 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:19.901 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:19.901 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:19.901 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69287 2024-09-14T23:59:19.901 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.901 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:19.901 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.901 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:19.901 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304833+0000", 2024-09-14T23:59:19.901 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.902 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.902 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:19.902 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304833+0000", 2024-09-14T23:59:19.902 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.902 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.902 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:19.902 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304834+0000", 2024-09-14T23:59:19.902 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4999999999999998e-07 2024-09-14T23:59:19.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.902 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.902 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:19.902 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304836+0000", 2024-09-14T23:59:19.903 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0279999999999999e-06 2024-09-14T23:59:19.903 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.903 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.903 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:19.903 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304842+0000", 2024-09-14T23:59:19.903 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8919999999999997e-06 2024-09-14T23:59:19.903 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.903 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.903 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:19.903 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304847+0000", 2024-09-14T23:59:19.903 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6160000000000001e-06 2024-09-14T23:59:19.903 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.903 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.904 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.904 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355411+0000", 2024-09-14T23:59:19.904 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050563540999999997 2024-09-14T23:59:19.904 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.904 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.904 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:19.904 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355459+0000", 2024-09-14T23:59:19.904 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7979000000000003e-05 2024-09-14T23:59:19.904 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.904 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.904 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:19.904 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355498+0000", 2024-09-14T23:59:19.904 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9335000000000002e-05 2024-09-14T23:59:19.905 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.905 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:19.905 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.905 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.905 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.905 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:69288 3.11 3:8cc6c82d:::rbd_data.11181e486ee6.0000000000000004:18 [read 7782400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T23:59:19.905 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:57:15.304854+0000", 2024-09-14T23:59:19.905 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 124.51653316300001, 2024-09-14T23:59:19.905 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050855145999999997, 2024-09-14T23:59:19.905 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:19.905 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:19.905 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-14T23:59:19.905 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:19.906 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:19.906 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:19.906 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 69288 2024-09-14T23:59:19.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.906 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:19.906 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.906 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:19.906 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304854+0000", 2024-09-14T23:59:19.906 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.906 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.906 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:19.906 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.304854+0000", 2024-09-14T23:59:19.907 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.907 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.907 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:19.907 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.306670+0000", 2024-09-14T23:59:19.907 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00181631 2024-09-14T23:59:19.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.907 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.907 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:19.907 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.306675+0000", 2024-09-14T23:59:19.907 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0220000000000004e-06 2024-09-14T23:59:19.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.907 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.907 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:19.908 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.306684+0000", 2024-09-14T23:59:19.908 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7309999999999999e-06 2024-09-14T23:59:19.908 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.908 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.908 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:19.908 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.306689+0000", 2024-09-14T23:59:19.908 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6609999999999999e-06 2024-09-14T23:59:19.908 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.908 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.908 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.908 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355533+0000", 2024-09-14T23:59:19.908 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048844578 2024-09-14T23:59:19.909 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.909 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.909 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-14T23:59:19.909 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355622+0000", 2024-09-14T23:59:19.909 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9006999999999997e-05 2024-09-14T23:59:19.909 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.909 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.909 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:19.909 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:57:15.355709+0000", 2024-09-14T23:59:19.909 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6836999999999995e-05 2024-09-14T23:59:19.909 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.910 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:19.910 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.910 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.910 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.910 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:255885 3.13 3.54329f53 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-14T23:59:19.910 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:48.978834+0000", 2024-09-14T23:59:19.910 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 30.842552577999999, 2024-09-14T23:59:19.910 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.007469078, 2024-09-14T23:59:19.910 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:19.910 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:19.911 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:59:19.911 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:19.911 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:19.911 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:19.911 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 255885 2024-09-14T23:59:19.911 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.911 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:19.911 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.911 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:19.911 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.978834+0000", 2024-09-14T23:59:19.911 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.911 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.911 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.912 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:19.912 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.978834+0000", 2024-09-14T23:59:19.912 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.912 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.912 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:19.912 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.978836+0000", 2024-09-14T23:59:19.912 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.288e-06 2024-09-14T23:59:19.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.912 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.912 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:19.912 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.978844+0000", 2024-09-14T23:59:19.912 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9880000000000004e-06 2024-09-14T23:59:19.913 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.913 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.913 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:19.913 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.978861+0000", 2024-09-14T23:59:19.913 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7020999999999998e-05 2024-09-14T23:59:19.913 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.913 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.913 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:19.913 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.978872+0000", 2024-09-14T23:59:19.913 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1274000000000001e-05 2024-09-14T23:59:19.913 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.913 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.913 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.913 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.982358+0000", 2024-09-14T23:59:19.914 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003486393 2024-09-14T23:59:19.914 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.914 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.914 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:19.914 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.982368+0000", 2024-09-14T23:59:19.914 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0690000000000008e-06 2024-09-14T23:59:19.914 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.914 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.914 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.914 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:49.986277+0000", 2024-09-14T23:59:19.914 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0039092489999999 2024-09-14T23:59:19.914 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.914 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.914 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:19.915 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:49.986304+0000", 2024-09-14T23:59:19.915 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6795999999999999e-05 2024-09-14T23:59:19.915 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.915 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:19.915 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.915 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.915 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.4376.0:255886 3.13 3.54329f53 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-14T23:59:19.915 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-14T23:58:48.981728+0000", 2024-09-14T23:59:19.915 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 30.839658660000001, 2024-09-14T23:59:19.915 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0046192860000001, 2024-09-14T23:59:19.915 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-14T23:59:19.915 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-14T23:59:19.915 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-14T23:59:19.916 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-14T23:59:19.916 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.4376", 2024-09-14T23:59:19.916 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.42:0/59737527", 2024-09-14T23:59:19.916 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 255886 2024-09-14T23:59:19.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.916 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-14T23:59:19.916 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.916 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-14T23:59:19.916 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.981728+0000", 2024-09-14T23:59:19.916 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.916 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.916 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-14T23:59:19.916 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.981728+0000", 2024-09-14T23:59:19.917 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-14T23:59:19.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.917 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.917 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-14T23:59:19.917 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.981730+0000", 2024-09-14T23:59:19.917 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5179999999999999e-06 2024-09-14T23:59:19.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.917 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.917 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-14T23:59:19.917 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.981737+0000", 2024-09-14T23:59:19.917 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.956e-06 2024-09-14T23:59:19.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.917 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.917 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-14T23:59:19.918 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.981748+0000", 2024-09-14T23:59:19.918 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0754000000000001e-05 2024-09-14T23:59:19.918 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.918 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.918 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-14T23:59:19.918 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.981757+0000", 2024-09-14T23:59:19.918 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8850000000000005e-06 2024-09-14T23:59:19.918 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.918 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.918 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.918 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.983225+0000", 2024-09-14T23:59:19.918 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001468605 2024-09-14T23:59:19.918 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.918 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.919 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-14T23:59:19.919 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:48.983232+0000", 2024-09-14T23:59:19.919 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0090000000000001e-06 2024-09-14T23:59:19.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.919 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.919 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-14T23:59:19.919 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:49.986337+0000", 2024-09-14T23:59:19.919 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0031047470000001 2024-09-14T23:59:19.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-14T23:59:19.919 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-14T23:59:19.919 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-14T23:59:19.919 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-14T23:58:49.986348+0000", 2024-09-14T23:59:19.919 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0812000000000001e-05 2024-09-14T23:59:19.919 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.919 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:19.920 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.920 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-14T23:59:19.920 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-14T23:59:19.920 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-14T23:59:19.995 INFO:teuthology.orchestra.run.smithi042.stderr:noscrub is unset 2024-09-14T23:59:19.998 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.1 to 1 (10000) 2024-09-14T23:59:20.018 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T23:59:20.019 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-14T23:59:21.009 INFO:teuthology.orchestra.run.smithi042.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-14T23:59:21.009 INFO:teuthology.orchestra.run.smithi042.stderr:nodeep-scrub is unset 2024-09-14T23:59:21.028 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-14T23:59:22.015 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.2 to 1 (10000) 2024-09-14T23:59:22.033 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-14T23:59:22.989 INFO:teuthology.orchestra.run.smithi042.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-14T23:59:23.007 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-14T23:59:23.990 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.3 to 1 (10000) 2024-09-14T23:59:24.007 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-14T23:59:25.021 INFO:teuthology.orchestra.run.smithi042.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-14T23:59:25.040 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-14T23:59:26.027 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.4 to 1 (10000) 2024-09-14T23:59:26.046 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-14T23:59:27.037 INFO:teuthology.orchestra.run.smithi042.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-14T23:59:27.054 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-14T23:59:27.990 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.5 to 1 (10000) 2024-09-14T23:59:28.005 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-14T23:59:29.044 INFO:teuthology.orchestra.run.smithi042.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-14T23:59:29.064 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-14T23:59:30.056 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.6 to 1 (10000) 2024-09-14T23:59:30.074 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-14T23:59:31.057 INFO:teuthology.orchestra.run.smithi042.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-14T23:59:31.074 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-14T23:59:32.063 INFO:teuthology.orchestra.run.smithi042.stderr:reweighted osd.7 to 1 (10000) 2024-09-14T23:59:32.081 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-09-14T23:59:33.069 INFO:teuthology.orchestra.run.smithi042.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-14T23:59:33.086 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-14T23:59:33.087 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-14T23:59:33.087 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-14T23:59:33.087 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-14T23:59:33.087 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-14T23:59:33.087 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-14T23:59:33.385 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:59:33.386 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":193,"fsid":"21916fcc-f95c-4bfa-8b8b-11ca45845613","created":"2024-09-14T23:56:03.167948+0000","modified":"2024-09-14T23:59:33.063861+0000","last_up_change":"2024-09-14T23:56:14.322882+0000","last_in_change":"2024-09-14T23:59:07.115398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-14T23:56:13.500197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-14T23:56:21.284705+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-14T23:56:30.861504+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"177","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":51,"snap_epoch":177,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"db9d0c67-2d60-4592-9ad3-f99e5294f93f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":314865173},{"type":"v1","addr":"172.21.15.42:6817","nonce":314865173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":314865173},{"type":"v1","addr":"172.21.15.42:6819","nonce":314865173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":314865173},{"type":"v1","addr":"172.21.15.42:6823","nonce":314865173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":314865173},{"type":"v1","addr":"172.21.15.42:6821","nonce":314865173}]},"public_addr":"172.21.15.42:6817/314865173","cluster_addr":"172.21.15.42:6819/314865173","heartbeat_back_addr":"172.21.15.42:6823/314865173","heartbeat_front_addr":"172.21.15.42:6821/314865173","state":["exists","up"]},{"osd":1,"uuid":"6c1a2765-c591-4c57-8ab3-ee6645970aa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":3197912310},{"type":"v1","addr":"172.21.15.42:6801","nonce":3197912310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":3197912310},{"type":"v1","addr":"172.21.15.42:6803","nonce":3197912310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":3197912310},{"type":"v1","addr":"172.21.15.42:6807","nonce":3197912310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":3197912310},{"type":"v1","addr":"172.21.15.42:6805","nonce":3197912310}]},"public_addr":"172.21.15.42:6801/3197912310","cluster_addr":"172.21.15.42:6803/3197912310","heartbeat_back_addr":"172.21.15.42:6807/3197912310","heartbeat_front_addr":"172.21.15.42:6805/3197912310","state":["exists","up"]},{"osd":2,"uuid":"930db893-7be6-4645-a7f1-e043c5ec95eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":3241153660},{"type":"v1","addr":"172.21.15.42:6809","nonce":3241153660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":3241153660},{"type":"v1","addr":"172.21.15.42:6811","nonce":3241153660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":3241153660},{"type":"v1","addr":"172.21.15.42:6815","nonce":3241153660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":3241153660},{"type":"v1","addr":"172.21.15.42:6813","nonce":3241153660}]},"public_addr":"172.21.15.42:6809/3241153660","cluster_addr":"172.21.15.42:6811/3241153660","heartbeat_back_addr":"172.21.15.42:6815/3241153660","heartbeat_front_addr":"172.21.15.42:6813/3241153660","state":["exists","up"]},{"osd":3,"uuid":"45bb3b84-b1c4-4f87-a7d1-0eb77b81741b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":1736416326},{"type":"v1","addr":"172.21.15.42:6825","nonce":1736416326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1736416326},{"type":"v1","addr":"172.21.15.42:6827","nonce":1736416326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1736416326},{"type":"v1","addr":"172.21.15.42:6831","nonce":1736416326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1736416326},{"type":"v1","addr":"172.21.15.42:6829","nonce":1736416326}]},"public_addr":"172.21.15.42:6825/1736416326","cluster_addr":"172.21.15.42:6827/1736416326","heartbeat_back_addr":"172.21.15.42:6831/1736416326","heartbeat_front_addr":"172.21.15.42:6829/1736416326","state":["exists","up"]},{"osd":4,"uuid":"a812e38f-c0f9-4d33-9373-1910fb09103c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1755154427},{"type":"v1","addr":"172.21.15.43:6809","nonce":1755154427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1755154427},{"type":"v1","addr":"172.21.15.43:6811","nonce":1755154427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1755154427},{"type":"v1","addr":"172.21.15.43:6815","nonce":1755154427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1755154427},{"type":"v1","addr":"172.21.15.43:6813","nonce":1755154427}]},"public_addr":"172.21.15.43:6809/1755154427","cluster_addr":"172.21.15.43:6811/1755154427","heartbeat_back_addr":"172.21.15.43:6815/1755154427","heartbeat_front_addr":"172.21.15.43:6813/1755154427","state":["exists","up"]},{"osd":5,"uuid":"0de87e5b-7a21-4fb4-9543-d96c065786f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":3033769028},{"type":"v1","addr":"172.21.15.43:6817","nonce":3033769028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":3033769028},{"type":"v1","addr":"172.21.15.43:6819","nonce":3033769028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":3033769028},{"type":"v1","addr":"172.21.15.43:6823","nonce":3033769028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":3033769028},{"type":"v1","addr":"172.21.15.43:6821","nonce":3033769028}]},"public_addr":"172.21.15.43:6817/3033769028","cluster_addr":"172.21.15.43:6819/3033769028","heartbeat_back_addr":"172.21.15.43:6823/3033769028","heartbeat_front_addr":"172.21.15.43:6821/3033769028","state":["exists","up"]},{"osd":6,"uuid":"c58c1d59-ad02-40f0-98e3-9078ef7e622b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3314282416},{"type":"v1","addr":"172.21.15.43:6801","nonce":3314282416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3314282416},{"type":"v1","addr":"172.21.15.43:6803","nonce":3314282416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":3314282416},{"type":"v1","addr":"172.21.15.43:6807","nonce":3314282416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3314282416},{"type":"v1","addr":"172.21.15.43:6805","nonce":3314282416}]},"public_addr":"172.21.15.43:6801/3314282416","cluster_addr":"172.21.15.43:6803/3314282416","heartbeat_back_addr":"172.21.15.43:6807/3314282416","heartbeat_front_addr":"172.21.15.43:6805/3314282416","state":["exists","up"]},{"osd":7,"uuid":"389bdb04-20e3-4a7a-8504-687e034bd7dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1481350118},{"type":"v1","addr":"172.21.15.43:6825","nonce":1481350118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":1481350118},{"type":"v1","addr":"172.21.15.43:6827","nonce":1481350118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":1481350118},{"type":"v1","addr":"172.21.15.43:6831","nonce":1481350118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1481350118},{"type":"v1","addr":"172.21.15.43:6829","nonce":1481350118}]},"public_addr":"172.21.15.43:6825/1481350118","cluster_addr":"172.21.15.43:6827/1481350118","heartbeat_back_addr":"172.21.15.43:6831/1481350118","heartbeat_front_addr":"172.21.15.43:6829/1481350118","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T23:56:11.077810+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T23:56:11.578838+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T23:56:10.895490+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T23:56:12.290821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T23:56:11.800581+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T23:56:11.883155+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T23:56:11.604790+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T23:56:12.067759+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T23:59:33.398 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-14T23:59:33.399 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-14T23:59:33.690 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:59:33.690 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":193,"fsid":"21916fcc-f95c-4bfa-8b8b-11ca45845613","created":"2024-09-14T23:56:03.167948+0000","modified":"2024-09-14T23:59:33.063861+0000","last_up_change":"2024-09-14T23:56:14.322882+0000","last_in_change":"2024-09-14T23:59:07.115398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-14T23:56:13.500197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-14T23:56:21.284705+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-14T23:56:30.861504+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"177","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":51,"snap_epoch":177,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"db9d0c67-2d60-4592-9ad3-f99e5294f93f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":314865173},{"type":"v1","addr":"172.21.15.42:6817","nonce":314865173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":314865173},{"type":"v1","addr":"172.21.15.42:6819","nonce":314865173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":314865173},{"type":"v1","addr":"172.21.15.42:6823","nonce":314865173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":314865173},{"type":"v1","addr":"172.21.15.42:6821","nonce":314865173}]},"public_addr":"172.21.15.42:6817/314865173","cluster_addr":"172.21.15.42:6819/314865173","heartbeat_back_addr":"172.21.15.42:6823/314865173","heartbeat_front_addr":"172.21.15.42:6821/314865173","state":["exists","up"]},{"osd":1,"uuid":"6c1a2765-c591-4c57-8ab3-ee6645970aa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":3197912310},{"type":"v1","addr":"172.21.15.42:6801","nonce":3197912310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":3197912310},{"type":"v1","addr":"172.21.15.42:6803","nonce":3197912310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":3197912310},{"type":"v1","addr":"172.21.15.42:6807","nonce":3197912310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":3197912310},{"type":"v1","addr":"172.21.15.42:6805","nonce":3197912310}]},"public_addr":"172.21.15.42:6801/3197912310","cluster_addr":"172.21.15.42:6803/3197912310","heartbeat_back_addr":"172.21.15.42:6807/3197912310","heartbeat_front_addr":"172.21.15.42:6805/3197912310","state":["exists","up"]},{"osd":2,"uuid":"930db893-7be6-4645-a7f1-e043c5ec95eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":3241153660},{"type":"v1","addr":"172.21.15.42:6809","nonce":3241153660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":3241153660},{"type":"v1","addr":"172.21.15.42:6811","nonce":3241153660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":3241153660},{"type":"v1","addr":"172.21.15.42:6815","nonce":3241153660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":3241153660},{"type":"v1","addr":"172.21.15.42:6813","nonce":3241153660}]},"public_addr":"172.21.15.42:6809/3241153660","cluster_addr":"172.21.15.42:6811/3241153660","heartbeat_back_addr":"172.21.15.42:6815/3241153660","heartbeat_front_addr":"172.21.15.42:6813/3241153660","state":["exists","up"]},{"osd":3,"uuid":"45bb3b84-b1c4-4f87-a7d1-0eb77b81741b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":1736416326},{"type":"v1","addr":"172.21.15.42:6825","nonce":1736416326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1736416326},{"type":"v1","addr":"172.21.15.42:6827","nonce":1736416326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1736416326},{"type":"v1","addr":"172.21.15.42:6831","nonce":1736416326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1736416326},{"type":"v1","addr":"172.21.15.42:6829","nonce":1736416326}]},"public_addr":"172.21.15.42:6825/1736416326","cluster_addr":"172.21.15.42:6827/1736416326","heartbeat_back_addr":"172.21.15.42:6831/1736416326","heartbeat_front_addr":"172.21.15.42:6829/1736416326","state":["exists","up"]},{"osd":4,"uuid":"a812e38f-c0f9-4d33-9373-1910fb09103c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1755154427},{"type":"v1","addr":"172.21.15.43:6809","nonce":1755154427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1755154427},{"type":"v1","addr":"172.21.15.43:6811","nonce":1755154427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1755154427},{"type":"v1","addr":"172.21.15.43:6815","nonce":1755154427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1755154427},{"type":"v1","addr":"172.21.15.43:6813","nonce":1755154427}]},"public_addr":"172.21.15.43:6809/1755154427","cluster_addr":"172.21.15.43:6811/1755154427","heartbeat_back_addr":"172.21.15.43:6815/1755154427","heartbeat_front_addr":"172.21.15.43:6813/1755154427","state":["exists","up"]},{"osd":5,"uuid":"0de87e5b-7a21-4fb4-9543-d96c065786f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":3033769028},{"type":"v1","addr":"172.21.15.43:6817","nonce":3033769028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":3033769028},{"type":"v1","addr":"172.21.15.43:6819","nonce":3033769028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":3033769028},{"type":"v1","addr":"172.21.15.43:6823","nonce":3033769028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":3033769028},{"type":"v1","addr":"172.21.15.43:6821","nonce":3033769028}]},"public_addr":"172.21.15.43:6817/3033769028","cluster_addr":"172.21.15.43:6819/3033769028","heartbeat_back_addr":"172.21.15.43:6823/3033769028","heartbeat_front_addr":"172.21.15.43:6821/3033769028","state":["exists","up"]},{"osd":6,"uuid":"c58c1d59-ad02-40f0-98e3-9078ef7e622b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3314282416},{"type":"v1","addr":"172.21.15.43:6801","nonce":3314282416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3314282416},{"type":"v1","addr":"172.21.15.43:6803","nonce":3314282416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":3314282416},{"type":"v1","addr":"172.21.15.43:6807","nonce":3314282416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3314282416},{"type":"v1","addr":"172.21.15.43:6805","nonce":3314282416}]},"public_addr":"172.21.15.43:6801/3314282416","cluster_addr":"172.21.15.43:6803/3314282416","heartbeat_back_addr":"172.21.15.43:6807/3314282416","heartbeat_front_addr":"172.21.15.43:6805/3314282416","state":["exists","up"]},{"osd":7,"uuid":"389bdb04-20e3-4a7a-8504-687e034bd7dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1481350118},{"type":"v1","addr":"172.21.15.43:6825","nonce":1481350118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":1481350118},{"type":"v1","addr":"172.21.15.43:6827","nonce":1481350118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":1481350118},{"type":"v1","addr":"172.21.15.43:6831","nonce":1481350118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1481350118},{"type":"v1","addr":"172.21.15.43:6829","nonce":1481350118}]},"public_addr":"172.21.15.43:6825/1481350118","cluster_addr":"172.21.15.43:6827/1481350118","heartbeat_back_addr":"172.21.15.43:6831/1481350118","heartbeat_front_addr":"172.21.15.43:6829/1481350118","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T23:56:11.077810+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T23:56:11.578838+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T23:56:10.895490+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T23:56:12.290821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T23:56:11.800581+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T23:56:11.883155+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T23:56:11.604790+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T23:56:12.067759+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T23:59:33.704 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-14T23:59:33.704 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-14T23:59:33.705 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-14T23:59:33.705 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-14T23:59:33.706 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-14T23:59:33.706 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-14T23:59:33.706 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-14T23:59:33.707 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-14T23:59:33.932 INFO:teuthology.orchestra.run.smithi042.stdout:60129542187 2024-09-14T23:59:33.932 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-14T23:59:33.933 INFO:teuthology.orchestra.run.smithi042.stdout:64424509484 2024-09-14T23:59:33.933 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-14T23:59:33.947 INFO:teuthology.orchestra.run.smithi042.stdout:60129542188 2024-09-14T23:59:33.947 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-14T23:59:33.949 INFO:teuthology.orchestra.run.smithi042.stdout:64424509483 2024-09-14T23:59:33.949 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-14T23:59:33.950 INFO:teuthology.orchestra.run.smithi042.stdout:64424509483 2024-09-14T23:59:33.950 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-14T23:59:33.954 INFO:teuthology.orchestra.run.smithi042.stdout:64424509485 2024-09-14T23:59:33.954 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-14T23:59:34.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-14T23:59:34.086+0000 7fa71f7fe640 0 --2- 172.21.15.42:0/2527089850 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa718031080 0x7fa7180c0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-14T23:59:34.151 INFO:teuthology.orchestra.run.smithi042.stdout:55834574891 2024-09-14T23:59:34.151 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-14T23:59:34.155 INFO:teuthology.orchestra.run.smithi042.stdout:64424509485 2024-09-14T23:59:34.155 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-14T23:59:34.457 INFO:teuthology.orchestra.run.smithi042.stdout:60129542186 2024-09-14T23:59:34.475 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542187 got 60129542186 for osd.0 2024-09-14T23:59:34.486 INFO:teuthology.orchestra.run.smithi042.stdout:64424509483 2024-09-14T23:59:34.500 INFO:teuthology.orchestra.run.smithi042.stdout:60129542187 2024-09-14T23:59:34.506 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509484 got 64424509483 for osd.6 2024-09-14T23:59:34.520 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542188 got 60129542187 for osd.1 2024-09-14T23:59:34.545 INFO:teuthology.orchestra.run.smithi042.stdout:64424509482 2024-09-14T23:59:34.546 INFO:teuthology.orchestra.run.smithi042.stdout:64424509484 2024-09-14T23:59:34.559 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509483 got 64424509482 for osd.5 2024-09-14T23:59:34.564 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509485 got 64424509484 for osd.7 2024-09-14T23:59:34.574 INFO:teuthology.orchestra.run.smithi042.stdout:64424509482 2024-09-14T23:59:34.592 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509483 got 64424509482 for osd.3 2024-09-14T23:59:34.613 INFO:teuthology.orchestra.run.smithi042.stdout:55834574890 2024-09-14T23:59:34.626 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574891 got 55834574890 for osd.2 2024-09-14T23:59:34.671 INFO:teuthology.orchestra.run.smithi042.stdout:64424509484 2024-09-14T23:59:34.684 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509485 got 64424509484 for osd.4 2024-09-14T23:59:35.476 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-14T23:59:35.507 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-14T23:59:35.521 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-14T23:59:35.560 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-14T23:59:35.565 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-14T23:59:35.593 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-14T23:59:35.627 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-14T23:59:35.685 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-14T23:59:35.975 INFO:teuthology.orchestra.run.smithi042.stdout:60129542186 2024-09-14T23:59:35.993 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542187 got 60129542186 for osd.0 2024-09-14T23:59:36.071 INFO:teuthology.orchestra.run.smithi042.stdout:60129542187 2024-09-14T23:59:36.091 INFO:teuthology.orchestra.run.smithi042.stdout:64424509485 2024-09-14T23:59:36.092 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542188 got 60129542187 for osd.1 2024-09-14T23:59:36.113 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509484 got 64424509485 for osd.6 2024-09-14T23:59:36.113 DEBUG:teuthology.parallel:result is None 2024-09-14T23:59:36.113 INFO:teuthology.orchestra.run.smithi042.stdout:55834574891 2024-09-14T23:59:36.123 INFO:teuthology.orchestra.run.smithi042.stdout:64424509484 2024-09-14T23:59:36.132 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574891 got 55834574891 for osd.2 2024-09-14T23:59:36.132 DEBUG:teuthology.parallel:result is None 2024-09-14T23:59:36.137 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509483 got 64424509484 for osd.5 2024-09-14T23:59:36.137 DEBUG:teuthology.parallel:result is None 2024-09-14T23:59:36.160 INFO:teuthology.orchestra.run.smithi042.stdout:64424509486 2024-09-14T23:59:36.174 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509485 got 64424509486 for osd.7 2024-09-14T23:59:36.175 DEBUG:teuthology.parallel:result is None 2024-09-14T23:59:36.181 INFO:teuthology.orchestra.run.smithi042.stdout:64424509484 2024-09-14T23:59:36.192 INFO:teuthology.orchestra.run.smithi042.stdout:64424509486 2024-09-14T23:59:36.194 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509483 got 64424509484 for osd.3 2024-09-14T23:59:36.194 DEBUG:teuthology.parallel:result is None 2024-09-14T23:59:36.206 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509485 got 64424509486 for osd.4 2024-09-14T23:59:36.206 DEBUG:teuthology.parallel:result is None 2024-09-14T23:59:36.993 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-14T23:59:37.093 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-14T23:59:37.303 INFO:teuthology.orchestra.run.smithi042.stdout:60129542187 2024-09-14T23:59:37.316 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542187 got 60129542187 for osd.0 2024-09-14T23:59:37.316 DEBUG:teuthology.parallel:result is None 2024-09-14T23:59:37.390 INFO:teuthology.orchestra.run.smithi042.stdout:60129542188 2024-09-14T23:59:37.403 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542188 got 60129542188 for osd.1 2024-09-14T23:59:37.403 DEBUG:teuthology.parallel:result is None 2024-09-14T23:59:37.403 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-14T23:59:37.403 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T23:59:37.693 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:59:37.694 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-14T23:59:37.705 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2024-09-14T23:59:37.488524+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":1208,"num_object_clones":0,"num_object_copies":2416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1208,"num_whiteouts":1200,"num_read":310777,"num_read_kb":1031638,"num_write":24745,"num_write_kb":672942,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7234,"num_bytes_recovered":334172432,"num_keys_recovered":596,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22327,"ondisk_log_size":22327,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":392704,"kb_used_data":18260,"kb_used_omap":12,"kb_used_meta":374259,"kb_avail":754582016,"statfs":{"total":773094113280,"available":772691984384,"internally_reserved":0,"allocated":18698240,"data_stored":10162226,"data_compressed":66229,"data_compressed_allocated":90112,"data_compressed_original":1163264,"omap_allocated":12752,"internal_metadata":383241776},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.434141"},"pg_stats":[{"pgid":"3.1f","version":"177'1121","reported_seq":6213,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067915+0000","last_change":"2024-09-14T23:59:16.994682+0000","last_active":"2024-09-14T23:59:33.067915+0000","last_peered":"2024-09-14T23:59:33.067915+0000","last_clean":"2024-09-14T23:59:33.067915+0000","last_became_active":"2024-09-14T23:57:00.062116+0000","last_became_peered":"2024-09-14T23:57:00.062116+0000","last_unstale":"2024-09-14T23:59:33.067915+0000","last_undegraded":"2024-09-14T23:59:33.067915+0000","last_fullsized":"2024-09-14T23:59:33.067915+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":1121,"log_dups_size":0,"ondisk_log_size":1121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:38:40.792973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094842500000000003,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":8822,"num_read_kb":37588,"num_write":1354,"num_write_kb":38053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":2327658,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"177'479","reported_seq":6679,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067687+0000","last_change":"2024-09-14T23:59:16.998886+0000","last_active":"2024-09-14T23:59:33.067687+0000","last_peered":"2024-09-14T23:59:33.067687+0000","last_clean":"2024-09-14T23:59:33.067687+0000","last_became_active":"2024-09-14T23:59:11.135384+0000","last_became_peered":"2024-09-14T23:59:11.135384+0000","last_unstale":"2024-09-14T23:59:33.067687+0000","last_undegraded":"2024-09-14T23:59:33.067687+0000","last_fullsized":"2024-09-14T23:59:33.067687+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:24:06.187016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001605626,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":5337,"num_read_kb":37535,"num_write":472,"num_write_kb":39218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":526,"num_bytes_recovered":14308962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"176'609","reported_seq":13697,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067726+0000","last_change":"2024-09-14T23:59:30.060565+0000","last_active":"2024-09-14T23:59:33.067726+0000","last_peered":"2024-09-14T23:59:33.067726+0000","last_clean":"2024-09-14T23:59:33.067726+0000","last_became_active":"2024-09-14T23:59:30.058342+0000","last_became_peered":"2024-09-14T23:59:30.058342+0000","last_unstale":"2024-09-14T23:59:33.067726+0000","last_undegraded":"2024-09-14T23:59:33.067726+0000","last_fullsized":"2024-09-14T23:59:33.067726+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:31:03.745601+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0080108929999999998,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":14616,"num_read_kb":22815,"num_write":536,"num_write_kb":24101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":1249963,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"177'995","reported_seq":6658,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067658+0000","last_change":"2024-09-14T23:59:30.060281+0000","last_active":"2024-09-14T23:59:33.067658+0000","last_peered":"2024-09-14T23:59:33.067658+0000","last_clean":"2024-09-14T23:59:33.067658+0000","last_became_active":"2024-09-14T23:59:30.058275+0000","last_became_peered":"2024-09-14T23:59:30.058275+0000","last_unstale":"2024-09-14T23:59:33.067658+0000","last_undegraded":"2024-09-14T23:59:33.067658+0000","last_fullsized":"2024-09-14T23:59:33.067658+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":995,"log_dups_size":0,"ondisk_log_size":995,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:51:47.471326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086515500000000005,"stat_sum":{"num_bytes":0,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":45,"num_read":9513,"num_read_kb":25550,"num_write":1310,"num_write_kb":9734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":2675491,"num_keys_recovered":40,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"176'524","reported_seq":2402,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.068285+0000","last_change":"2024-09-14T23:59:16.994187+0000","last_active":"2024-09-14T23:59:33.068285+0000","last_peered":"2024-09-14T23:59:33.068285+0000","last_clean":"2024-09-14T23:59:33.068285+0000","last_became_active":"2024-09-14T23:56:32.398329+0000","last_became_peered":"2024-09-14T23:56:32.398329+0000","last_unstale":"2024-09-14T23:59:33.068285+0000","last_undegraded":"2024-09-14T23:59:33.068285+0000","last_fullsized":"2024-09-14T23:59:33.068285+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":524,"log_dups_size":0,"ondisk_log_size":524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:25:07.627636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00103907,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":2076,"num_read_kb":19877,"num_write":623,"num_write_kb":10342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"177'899","reported_seq":8499,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.685085+0000","last_change":"2024-09-14T23:59:16.994978+0000","last_active":"2024-09-14T23:59:33.685085+0000","last_peered":"2024-09-14T23:59:33.685085+0000","last_clean":"2024-09-14T23:59:33.685085+0000","last_became_active":"2024-09-14T23:56:32.399655+0000","last_became_peered":"2024-09-14T23:56:32.399655+0000","last_unstale":"2024-09-14T23:59:33.685085+0000","last_undegraded":"2024-09-14T23:59:33.685085+0000","last_fullsized":"2024-09-14T23:59:33.685085+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":899,"log_dups_size":0,"ondisk_log_size":899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:01:52.825811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001807665,"stat_sum":{"num_bytes":19,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":46,"num_read":10058,"num_read_kb":53829,"num_write":970,"num_write_kb":34825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"177'695","reported_seq":4690,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.684153+0000","last_change":"2024-09-14T23:59:16.994457+0000","last_active":"2024-09-14T23:59:33.684153+0000","last_peered":"2024-09-14T23:59:33.684153+0000","last_clean":"2024-09-14T23:59:33.684153+0000","last_became_active":"2024-09-14T23:59:11.135739+0000","last_became_peered":"2024-09-14T23:59:11.135739+0000","last_unstale":"2024-09-14T23:59:33.684153+0000","last_undegraded":"2024-09-14T23:59:33.684153+0000","last_fullsized":"2024-09-14T23:59:33.684153+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:33:05.725582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099865499999999999,"stat_sum":{"num_bytes":0,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":42,"num_read":5485,"num_read_kb":20536,"num_write":717,"num_write_kb":15834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":318,"num_bytes_recovered":6472298,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"177'742","reported_seq":6564,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067404+0000","last_change":"2024-09-14T23:59:16.998152+0000","last_active":"2024-09-14T23:59:33.067404+0000","last_peered":"2024-09-14T23:59:33.067404+0000","last_clean":"2024-09-14T23:59:33.067404+0000","last_became_active":"2024-09-14T23:59:12.133258+0000","last_became_peered":"2024-09-14T23:59:12.133258+0000","last_unstale":"2024-09-14T23:59:33.067404+0000","last_undegraded":"2024-09-14T23:59:33.067404+0000","last_fullsized":"2024-09-14T23:59:33.067404+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":742,"log_dups_size":0,"ondisk_log_size":742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:43:45.205419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093947500000000005,"stat_sum":{"num_bytes":0,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":46,"num_read":6734,"num_read_kb":59364,"num_write":741,"num_write_kb":26095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":340,"num_bytes_recovered":12763266,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"176'544","reported_seq":11742,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.068027+0000","last_change":"2024-09-14T23:59:16.994889+0000","last_active":"2024-09-14T23:59:33.068027+0000","last_peered":"2024-09-14T23:59:33.068027+0000","last_clean":"2024-09-14T23:59:33.068027+0000","last_became_active":"2024-09-14T23:57:00.062304+0000","last_became_peered":"2024-09-14T23:57:00.062304+0000","last_unstale":"2024-09-14T23:59:33.068027+0000","last_undegraded":"2024-09-14T23:59:33.068027+0000","last_fullsized":"2024-09-14T23:59:33.068027+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:28:05.460487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091006300000000002,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":12421,"num_read_kb":16852,"num_write":491,"num_write_kb":6294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":3345543,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"177'463","reported_seq":7658,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067791+0000","last_change":"2024-09-14T23:59:30.062097+0000","last_active":"2024-09-14T23:59:33.067791+0000","last_peered":"2024-09-14T23:59:33.067791+0000","last_clean":"2024-09-14T23:59:33.067791+0000","last_became_active":"2024-09-14T23:59:30.060469+0000","last_became_peered":"2024-09-14T23:59:30.060469+0000","last_unstale":"2024-09-14T23:59:33.067791+0000","last_undegraded":"2024-09-14T23:59:33.067791+0000","last_fullsized":"2024-09-14T23:59:33.067791+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:38:29.716639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002288432,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":7773,"num_read_kb":27495,"num_write":382,"num_write_kb":13368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":404,"num_bytes_recovered":26159849,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"176'612","reported_seq":9857,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.367048+0000","last_change":"2024-09-14T23:59:17.002582+0000","last_active":"2024-09-14T23:59:33.367048+0000","last_peered":"2024-09-14T23:59:33.367048+0000","last_clean":"2024-09-14T23:59:33.367048+0000","last_became_active":"2024-09-14T23:59:11.135860+0000","last_became_peered":"2024-09-14T23:59:11.135860+0000","last_unstale":"2024-09-14T23:59:33.367048+0000","last_undegraded":"2024-09-14T23:59:33.367048+0000","last_fullsized":"2024-09-14T23:59:33.367048+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":612,"log_dups_size":0,"ondisk_log_size":612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:16:43.430234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001929275,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":9948,"num_read_kb":36137,"num_write":618,"num_write_kb":23694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":516,"num_bytes_recovered":42197286,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"176'1093","reported_seq":8194,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.685233+0000","last_change":"2024-09-14T23:59:16.996306+0000","last_active":"2024-09-14T23:59:33.685233+0000","last_peered":"2024-09-14T23:59:33.685233+0000","last_clean":"2024-09-14T23:59:33.685233+0000","last_became_active":"2024-09-14T23:58:53.983979+0000","last_became_peered":"2024-09-14T23:58:53.983979+0000","last_unstale":"2024-09-14T23:59:33.685233+0000","last_undegraded":"2024-09-14T23:59:33.685233+0000","last_fullsized":"2024-09-14T23:59:33.685233+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":1093,"log_dups_size":0,"ondisk_log_size":1093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:26:54.972175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002903152,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":9679,"num_read_kb":42324,"num_write":1584,"num_write_kb":28908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":312,"num_bytes_recovered":22540127,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":373,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.684825+0000","last_change":"2024-09-14T23:58:49.989598+0000","last_active":"2024-09-14T23:59:33.684825+0000","last_peered":"2024-09-14T23:59:33.684825+0000","last_clean":"2024-09-14T23:59:33.684825+0000","last_became_active":"2024-09-14T23:58:49.989224+0000","last_became_peered":"2024-09-14T23:58:49.989224+0000","last_unstale":"2024-09-14T23:59:33.684825+0000","last_undegraded":"2024-09-14T23:59:33.684825+0000","last_fullsized":"2024-09-14T23:59:33.684825+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:52:25.663126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045105900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"177'718","reported_seq":5318,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.068336+0000","last_change":"2024-09-14T23:59:16.995528+0000","last_active":"2024-09-14T23:59:33.068336+0000","last_peered":"2024-09-14T23:59:33.068336+0000","last_clean":"2024-09-14T23:59:33.068336+0000","last_became_active":"2024-09-14T23:56:32.398664+0000","last_became_peered":"2024-09-14T23:56:32.398664+0000","last_unstale":"2024-09-14T23:59:33.068336+0000","last_undegraded":"2024-09-14T23:59:33.068336+0000","last_fullsized":"2024-09-14T23:59:33.068336+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":718,"log_dups_size":0,"ondisk_log_size":718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:15:18.740717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001169722,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":6635,"num_read_kb":24355,"num_write":754,"num_write_kb":5733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":363,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.684380+0000","last_change":"2024-09-14T23:56:24.534632+0000","last_active":"2024-09-14T23:59:33.684380+0000","last_peered":"2024-09-14T23:59:33.684380+0000","last_clean":"2024-09-14T23:59:33.684380+0000","last_became_active":"2024-09-14T23:56:22.526355+0000","last_became_peered":"2024-09-14T23:56:22.526355+0000","last_unstale":"2024-09-14T23:59:33.684380+0000","last_undegraded":"2024-09-14T23:59:33.684380+0000","last_fullsized":"2024-09-14T23:59:33.684380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:02:16.059627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049910000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"177'528","reported_seq":7754,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067820+0000","last_change":"2024-09-14T23:59:17.000419+0000","last_active":"2024-09-14T23:59:33.067820+0000","last_peered":"2024-09-14T23:59:33.067820+0000","last_clean":"2024-09-14T23:59:33.067820+0000","last_became_active":"2024-09-14T23:59:11.135030+0000","last_became_peered":"2024-09-14T23:59:11.135030+0000","last_unstale":"2024-09-14T23:59:33.067820+0000","last_undegraded":"2024-09-14T23:59:33.067820+0000","last_fullsized":"2024-09-14T23:59:33.067820+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":528,"log_dups_size":0,"ondisk_log_size":528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:12:06.383593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027209629999999999,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":7655,"num_read_kb":43475,"num_write":499,"num_write_kb":17299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":640,"num_bytes_recovered":36520799,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":95,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.366957+0000","last_change":"2024-09-14T23:58:49.988483+0000","last_active":"2024-09-14T23:59:33.366957+0000","last_peered":"2024-09-14T23:59:33.366957+0000","last_clean":"2024-09-14T23:59:33.366957+0000","last_became_active":"2024-09-14T23:58:49.988169+0000","last_became_peered":"2024-09-14T23:58:49.988169+0000","last_unstale":"2024-09-14T23:59:33.366957+0000","last_undegraded":"2024-09-14T23:59:33.366957+0000","last_fullsized":"2024-09-14T23:59:33.366957+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:57:59.569156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"177'622","reported_seq":5510,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.684308+0000","last_change":"2024-09-14T23:59:16.993791+0000","last_active":"2024-09-14T23:59:33.684308+0000","last_peered":"2024-09-14T23:59:33.684308+0000","last_clean":"2024-09-14T23:59:33.684308+0000","last_became_active":"2024-09-14T23:56:32.401089+0000","last_became_peered":"2024-09-14T23:56:32.401089+0000","last_unstale":"2024-09-14T23:59:33.684308+0000","last_undegraded":"2024-09-14T23:59:33.684308+0000","last_fullsized":"2024-09-14T23:59:33.684308+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:44:29.413362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094649999999999997,"stat_sum":{"num_bytes":0,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":43,"num_read":6577,"num_read_kb":26346,"num_write":596,"num_write_kb":13585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":363,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.684310+0000","last_change":"2024-09-14T23:56:24.534510+0000","last_active":"2024-09-14T23:59:33.684310+0000","last_peered":"2024-09-14T23:59:33.684310+0000","last_clean":"2024-09-14T23:59:33.684310+0000","last_became_active":"2024-09-14T23:56:22.527611+0000","last_became_peered":"2024-09-14T23:56:22.527611+0000","last_unstale":"2024-09-14T23:59:33.684310+0000","last_undegraded":"2024-09-14T23:59:33.684310+0000","last_fullsized":"2024-09-14T23:59:33.684310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:15:45.606026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049916299999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"176'729","reported_seq":7600,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067812+0000","last_change":"2024-09-14T23:59:30.062265+0000","last_active":"2024-09-14T23:59:33.067812+0000","last_peered":"2024-09-14T23:59:33.067812+0000","last_clean":"2024-09-14T23:59:33.067812+0000","last_became_active":"2024-09-14T23:59:30.060381+0000","last_became_peered":"2024-09-14T23:59:30.060381+0000","last_unstale":"2024-09-14T23:59:33.067812+0000","last_undegraded":"2024-09-14T23:59:33.067812+0000","last_fullsized":"2024-09-14T23:59:33.067812+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":729,"log_dups_size":0,"ondisk_log_size":729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:19:07.081227+0000","scrub_duration":0,"objects_trimmed":10,"snaptrim_duration":0.012412685999999999,"stat_sum":{"num_bytes":0,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":47,"num_read":9294,"num_read_kb":18810,"num_write":776,"num_write_kb":10083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":401,"num_bytes_recovered":9491176,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"177'834","reported_seq":12831,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.685176+0000","last_change":"2024-09-14T23:59:16.995637+0000","last_active":"2024-09-14T23:59:33.685176+0000","last_peered":"2024-09-14T23:59:33.685176+0000","last_clean":"2024-09-14T23:59:33.685176+0000","last_became_active":"2024-09-14T23:56:32.397883+0000","last_became_peered":"2024-09-14T23:56:32.397883+0000","last_unstale":"2024-09-14T23:59:33.685176+0000","last_undegraded":"2024-09-14T23:59:33.685176+0000","last_fullsized":"2024-09-14T23:59:33.685176+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":834,"log_dups_size":0,"ondisk_log_size":834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:51:15.895964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023599810000000001,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":38,"num_read":13370,"num_read_kb":25781,"num_write":804,"num_write_kb":35448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":87,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067672+0000","last_change":"2024-09-14T23:59:30.058288+0000","last_active":"2024-09-14T23:59:33.067672+0000","last_peered":"2024-09-14T23:59:33.067672+0000","last_clean":"2024-09-14T23:59:33.067672+0000","last_became_active":"2024-09-14T23:59:30.057866+0000","last_became_peered":"2024-09-14T23:59:30.057866+0000","last_unstale":"2024-09-14T23:59:33.067672+0000","last_undegraded":"2024-09-14T23:59:33.067672+0000","last_fullsized":"2024-09-14T23:59:33.067672+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:55:57.947601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"177'518","reported_seq":9365,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.684112+0000","last_change":"2024-09-14T23:59:16.993927+0000","last_active":"2024-09-14T23:59:33.684112+0000","last_peered":"2024-09-14T23:59:33.684112+0000","last_clean":"2024-09-14T23:59:33.684112+0000","last_became_active":"2024-09-14T23:56:32.400278+0000","last_became_peered":"2024-09-14T23:56:32.400278+0000","last_unstale":"2024-09-14T23:59:33.684112+0000","last_undegraded":"2024-09-14T23:59:33.684112+0000","last_fullsized":"2024-09-14T23:59:33.684112+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:26:19.473912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098058499999999996,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":8550,"num_read_kb":62565,"num_write":432,"num_write_kb":30360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":341,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.684325+0000","last_change":"2024-09-14T23:59:23.991858+0000","last_active":"2024-09-14T23:59:33.684325+0000","last_peered":"2024-09-14T23:59:33.684325+0000","last_clean":"2024-09-14T23:59:33.684325+0000","last_became_active":"2024-09-14T23:59:23.991492+0000","last_became_peered":"2024-09-14T23:59:23.991492+0000","last_unstale":"2024-09-14T23:59:33.684325+0000","last_undegraded":"2024-09-14T23:59:33.684325+0000","last_fullsized":"2024-09-14T23:59:33.684325+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:13:20.777068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026863400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"176'788","reported_seq":7241,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067718+0000","last_change":"2024-09-14T23:59:20.000557+0000","last_active":"2024-09-14T23:59:33.067718+0000","last_peered":"2024-09-14T23:59:33.067718+0000","last_clean":"2024-09-14T23:59:33.067718+0000","last_became_active":"2024-09-14T23:59:19.998578+0000","last_became_peered":"2024-09-14T23:59:19.998578+0000","last_unstale":"2024-09-14T23:59:33.067718+0000","last_undegraded":"2024-09-14T23:59:33.067718+0000","last_fullsized":"2024-09-14T23:59:33.067718+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":788,"log_dups_size":0,"ondisk_log_size":788,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:22:03.401033+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.00381613,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":9431,"num_read_kb":22132,"num_write":943,"num_write_kb":5118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":95,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.366890+0000","last_change":"2024-09-14T23:58:49.988483+0000","last_active":"2024-09-14T23:59:33.366890+0000","last_peered":"2024-09-14T23:59:33.366890+0000","last_clean":"2024-09-14T23:59:33.366890+0000","last_became_active":"2024-09-14T23:58:49.988096+0000","last_became_peered":"2024-09-14T23:58:49.988096+0000","last_unstale":"2024-09-14T23:59:33.366890+0000","last_undegraded":"2024-09-14T23:59:33.366890+0000","last_fullsized":"2024-09-14T23:59:33.366890+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:34:48.166954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"176'1105","reported_seq":30205,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067544+0000","last_change":"2024-09-14T23:59:16.999655+0000","last_active":"2024-09-14T23:59:33.067544+0000","last_peered":"2024-09-14T23:59:33.067544+0000","last_clean":"2024-09-14T23:59:33.067544+0000","last_became_active":"2024-09-14T23:59:12.133789+0000","last_became_peered":"2024-09-14T23:59:12.133789+0000","last_unstale":"2024-09-14T23:59:33.067544+0000","last_undegraded":"2024-09-14T23:59:33.067544+0000","last_fullsized":"2024-09-14T23:59:33.067544+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":1105,"log_dups_size":0,"ondisk_log_size":1105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:04:55.112519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002233971,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":31963,"num_read_kb":26382,"num_write":1422,"num_write_kb":17307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":641,"num_bytes_recovered":19031962,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":122,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067683+0000","last_change":"2024-09-14T23:59:30.060260+0000","last_active":"2024-09-14T23:59:33.067683+0000","last_peered":"2024-09-14T23:59:33.067683+0000","last_clean":"2024-09-14T23:59:33.067683+0000","last_became_active":"2024-09-14T23:59:30.059854+0000","last_became_peered":"2024-09-14T23:59:30.059854+0000","last_unstale":"2024-09-14T23:59:33.067683+0000","last_undegraded":"2024-09-14T23:59:33.067683+0000","last_fullsized":"2024-09-14T23:59:33.067683+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:14:35.344939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":277,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.366889+0000","last_change":"2024-09-14T23:58:53.985309+0000","last_active":"2024-09-14T23:59:33.366889+0000","last_peered":"2024-09-14T23:59:33.366889+0000","last_clean":"2024-09-14T23:59:33.366889+0000","last_became_active":"2024-09-14T23:58:53.984830+0000","last_became_peered":"2024-09-14T23:58:53.984830+0000","last_unstale":"2024-09-14T23:59:33.366889+0000","last_undegraded":"2024-09-14T23:59:33.366889+0000","last_fullsized":"2024-09-14T23:59:33.366889+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:14.322882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:14.322882+0000","last_clean_scrub_stamp":"2024-09-14T23:56:14.322882+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:37:48.336097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"177'609","reported_seq":5982,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067398+0000","last_change":"2024-09-14T23:59:17.000201+0000","last_active":"2024-09-14T23:59:33.067398+0000","last_peered":"2024-09-14T23:59:33.067398+0000","last_clean":"2024-09-14T23:59:33.067398+0000","last_became_active":"2024-09-14T23:59:12.136444+0000","last_became_peered":"2024-09-14T23:59:12.136444+0000","last_unstale":"2024-09-14T23:59:33.067398+0000","last_undegraded":"2024-09-14T23:59:33.067398+0000","last_fullsized":"2024-09-14T23:59:33.067398+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:26:06.094223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026622379999999999,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":5935,"num_read_kb":18123,"num_write":528,"num_write_kb":10919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":399,"num_bytes_recovered":9189081,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"176'343","reported_seq":11038,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.685073+0000","last_change":"2024-09-14T23:59:16.994730+0000","last_active":"2024-09-14T23:59:33.685073+0000","last_peered":"2024-09-14T23:59:33.685073+0000","last_clean":"2024-09-14T23:59:33.685073+0000","last_became_active":"2024-09-14T23:56:32.400017+0000","last_became_peered":"2024-09-14T23:56:32.400017+0000","last_unstale":"2024-09-14T23:59:33.685073+0000","last_undegraded":"2024-09-14T23:59:33.685073+0000","last_fullsized":"2024-09-14T23:59:33.685073+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:24:02.170046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015805459999999999,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":10189,"num_read_kb":13014,"num_write":297,"num_write_kb":15704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"176'834","reported_seq":16004,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.684762+0000","last_change":"2024-09-14T23:59:17.069111+0000","last_active":"2024-09-14T23:59:33.684762+0000","last_peered":"2024-09-14T23:59:33.684762+0000","last_clean":"2024-09-14T23:59:33.684762+0000","last_became_active":"2024-09-14T23:56:32.402769+0000","last_became_peered":"2024-09-14T23:56:32.402769+0000","last_unstale":"2024-09-14T23:59:33.684762+0000","last_undegraded":"2024-09-14T23:59:33.684762+0000","last_fullsized":"2024-09-14T23:59:33.684762+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":834,"log_dups_size":0,"ondisk_log_size":834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:36:02.993358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098966399999999991,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":40,"num_read":16916,"num_read_kb":32658,"num_write":1110,"num_write_kb":14962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"176'665","reported_seq":13705,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067583+0000","last_change":"2024-09-14T23:59:16.998126+0000","last_active":"2024-09-14T23:59:33.067583+0000","last_peered":"2024-09-14T23:59:33.067583+0000","last_clean":"2024-09-14T23:59:33.067583+0000","last_became_active":"2024-09-14T23:59:12.132838+0000","last_became_peered":"2024-09-14T23:59:12.132838+0000","last_unstale":"2024-09-14T23:59:33.067583+0000","last_undegraded":"2024-09-14T23:59:33.067583+0000","last_fullsized":"2024-09-14T23:59:33.067583+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":665,"log_dups_size":0,"ondisk_log_size":665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:34:25.667684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087623300000000002,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":13816,"num_read_kb":56666,"num_write":771,"num_write_kb":15952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":258,"num_bytes_recovered":8738708,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"176'705","reported_seq":16265,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067866+0000","last_change":"2024-09-14T23:59:30.062604+0000","last_active":"2024-09-14T23:59:33.067866+0000","last_peered":"2024-09-14T23:59:33.067866+0000","last_clean":"2024-09-14T23:59:33.067866+0000","last_became_active":"2024-09-14T23:59:30.060837+0000","last_became_peered":"2024-09-14T23:59:30.060837+0000","last_unstale":"2024-09-14T23:59:33.067866+0000","last_undegraded":"2024-09-14T23:59:33.067866+0000","last_fullsized":"2024-09-14T23:59:33.067866+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":705,"log_dups_size":0,"ondisk_log_size":705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:18:03.409303+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.021276283,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":15379,"num_read_kb":69494,"num_write":885,"num_write_kb":43376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":30030415,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"176'521","reported_seq":3381,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.367121+0000","last_change":"2024-09-14T23:59:17.003938+0000","last_active":"2024-09-14T23:59:33.367121+0000","last_peered":"2024-09-14T23:59:33.367121+0000","last_clean":"2024-09-14T23:59:33.367121+0000","last_became_active":"2024-09-14T23:58:52.985661+0000","last_became_peered":"2024-09-14T23:58:52.985661+0000","last_unstale":"2024-09-14T23:59:33.367121+0000","last_undegraded":"2024-09-14T23:59:33.367121+0000","last_fullsized":"2024-09-14T23:59:33.367121+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:53:00.607967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030262919999999999,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":3961,"num_read_kb":12312,"num_write":537,"num_write_kb":8315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":453,"num_bytes_recovered":19224487,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"177'532","reported_seq":5732,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.367080+0000","last_change":"2024-09-14T23:59:17.003376+0000","last_active":"2024-09-14T23:59:33.367080+0000","last_peered":"2024-09-14T23:59:33.367080+0000","last_clean":"2024-09-14T23:59:33.367080+0000","last_became_active":"2024-09-14T23:58:53.986149+0000","last_became_peered":"2024-09-14T23:58:53.986149+0000","last_unstale":"2024-09-14T23:59:33.367080+0000","last_undegraded":"2024-09-14T23:59:33.367080+0000","last_fullsized":"2024-09-14T23:59:33.367080+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:15:03.478135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024995130000000001,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":7765,"num_read_kb":21397,"num_write":463,"num_write_kb":11356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":12997809,"num_keys_recovered":45,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"176'663","reported_seq":14079,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.367080+0000","last_change":"2024-09-14T23:59:17.004025+0000","last_active":"2024-09-14T23:59:33.367080+0000","last_peered":"2024-09-14T23:59:33.367080+0000","last_clean":"2024-09-14T23:59:33.367080+0000","last_became_active":"2024-09-14T23:58:53.984808+0000","last_became_peered":"2024-09-14T23:58:53.984808+0000","last_unstale":"2024-09-14T23:59:33.367080+0000","last_undegraded":"2024-09-14T23:59:33.367080+0000","last_fullsized":"2024-09-14T23:59:33.367080+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":663,"log_dups_size":0,"ondisk_log_size":663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:48:22.749991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030575429999999998,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":14675,"num_read_kb":21661,"num_write":655,"num_write_kb":35778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":12964873,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"176'797","reported_seq":6504,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.684762+0000","last_change":"2024-09-14T23:59:17.069019+0000","last_active":"2024-09-14T23:59:33.684762+0000","last_peered":"2024-09-14T23:59:33.684762+0000","last_clean":"2024-09-14T23:59:33.684762+0000","last_became_active":"2024-09-14T23:56:32.402973+0000","last_became_peered":"2024-09-14T23:56:32.402973+0000","last_unstale":"2024-09-14T23:59:33.684762+0000","last_undegraded":"2024-09-14T23:59:33.684762+0000","last_fullsized":"2024-09-14T23:59:33.684762+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":797,"log_dups_size":0,"ondisk_log_size":797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:18:08.492160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088108100000000003,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":7036,"num_read_kb":57979,"num_write":1182,"num_write_kb":51466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"176'569","reported_seq":8379,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.366890+0000","last_change":"2024-09-14T23:59:17.001624+0000","last_active":"2024-09-14T23:59:33.366890+0000","last_peered":"2024-09-14T23:59:33.366890+0000","last_clean":"2024-09-14T23:59:33.366890+0000","last_became_active":"2024-09-14T23:58:52.987786+0000","last_became_peered":"2024-09-14T23:58:52.987786+0000","last_unstale":"2024-09-14T23:59:33.366890+0000","last_undegraded":"2024-09-14T23:59:33.366890+0000","last_fullsized":"2024-09-14T23:59:33.366890+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:55:48.806780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001145566,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":8694,"num_read_kb":36214,"num_write":569,"num_write_kb":28063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":19310765,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"176'376","reported_seq":3270,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067650+0000","last_change":"2024-09-14T23:59:20.000977+0000","last_active":"2024-09-14T23:59:33.067650+0000","last_peered":"2024-09-14T23:59:33.067650+0000","last_clean":"2024-09-14T23:59:33.067650+0000","last_became_active":"2024-09-14T23:59:19.999489+0000","last_became_peered":"2024-09-14T23:59:19.999489+0000","last_unstale":"2024-09-14T23:59:33.067650+0000","last_undegraded":"2024-09-14T23:59:33.067650+0000","last_fullsized":"2024-09-14T23:59:33.067650+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:21:32.054413+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0042424580000000002,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":3170,"num_read_kb":16380,"num_write":287,"num_write_kb":14646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":7078411,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"176'975","reported_seq":4718,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.366901+0000","last_change":"2024-09-14T23:59:17.002239+0000","last_active":"2024-09-14T23:59:33.366901+0000","last_peered":"2024-09-14T23:59:33.366901+0000","last_clean":"2024-09-14T23:59:33.366901+0000","last_became_active":"2024-09-14T23:58:53.987342+0000","last_became_peered":"2024-09-14T23:58:53.987342+0000","last_unstale":"2024-09-14T23:59:33.366901+0000","last_undegraded":"2024-09-14T23:59:33.366901+0000","last_fullsized":"2024-09-14T23:59:33.366901+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:30.333512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001659113,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":7188,"num_read_kb":25892,"num_write":1276,"num_write_kb":14588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":290,"num_bytes_recovered":14372748,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":1204,"num_object_clones":0,"num_object_copies":2408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1204,"num_whiteouts":1200,"num_read":310661,"num_read_kb":1031538,"num_write":24584,"num_write_kb":670524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7228,"num_bytes_recovered":332991677,"num_keys_recovered":596,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22207,"ondisk_log_size":22207,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":1771104,"data_compressed":100161,"data_compressed_allocated":135168,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509486,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51552,"kb_used_data":2316,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320288,"statfs":{"total":96636764160,"available":96583974912,"internally_reserved":0,"allocated":2371584,"data_stored":1713775,"data_compressed":33932,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509485,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33456,"kb_used_data":2268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338384,"statfs":{"total":96636764160,"available":96602505216,"internally_reserved":0,"allocated":2322432,"data_stored":1123407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509484,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47348,"kb_used_data":2272,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324492,"statfs":{"total":96636764160,"available":96588279808,"internally_reserved":0,"allocated":2326528,"data_stored":1123426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509486,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77876,"kb_used_data":2272,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293964,"statfs":{"total":96636764160,"available":96557019136,"internally_reserved":0,"allocated":2326528,"data_stored":1123426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509484,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55668,"kb_used_data":2268,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316172,"statfs":{"total":96636764160,"available":96579760128,"internally_reserved":0,"allocated":2322432,"data_stored":1123407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542188,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41772,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330068,"statfs":{"total":96636764160,"available":96593989632,"internally_reserved":0,"allocated":2318336,"data_stored":1117603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542187,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51576,"kb_used_data":2332,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320264,"statfs":{"total":96636764160,"available":96583950336,"internally_reserved":0,"allocated":2387968,"data_stored":1713775,"data_compressed":32297,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574892,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33456,"kb_used_data":2268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338384,"statfs":{"total":96636764160,"available":96602505216,"internally_reserved":0,"allocated":2322432,"data_stored":1123407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32297,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":33932,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":33932,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T23:59:37.707 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T23:59:37.996 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:59:37.996 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-14T23:59:38.008 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2024-09-14T23:59:37.488524+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":1208,"num_object_clones":0,"num_object_copies":2416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1208,"num_whiteouts":1200,"num_read":310777,"num_read_kb":1031638,"num_write":24745,"num_write_kb":672942,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7234,"num_bytes_recovered":334172432,"num_keys_recovered":596,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22327,"ondisk_log_size":22327,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":392704,"kb_used_data":18260,"kb_used_omap":12,"kb_used_meta":374259,"kb_avail":754582016,"statfs":{"total":773094113280,"available":772691984384,"internally_reserved":0,"allocated":18698240,"data_stored":10162226,"data_compressed":66229,"data_compressed_allocated":90112,"data_compressed_original":1163264,"omap_allocated":12752,"internal_metadata":383241776},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.434141"},"pg_stats":[{"pgid":"3.1f","version":"177'1121","reported_seq":6213,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067915+0000","last_change":"2024-09-14T23:59:16.994682+0000","last_active":"2024-09-14T23:59:33.067915+0000","last_peered":"2024-09-14T23:59:33.067915+0000","last_clean":"2024-09-14T23:59:33.067915+0000","last_became_active":"2024-09-14T23:57:00.062116+0000","last_became_peered":"2024-09-14T23:57:00.062116+0000","last_unstale":"2024-09-14T23:59:33.067915+0000","last_undegraded":"2024-09-14T23:59:33.067915+0000","last_fullsized":"2024-09-14T23:59:33.067915+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":1121,"log_dups_size":0,"ondisk_log_size":1121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:38:40.792973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094842500000000003,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":8822,"num_read_kb":37588,"num_write":1354,"num_write_kb":38053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":2327658,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"177'479","reported_seq":6679,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067687+0000","last_change":"2024-09-14T23:59:16.998886+0000","last_active":"2024-09-14T23:59:33.067687+0000","last_peered":"2024-09-14T23:59:33.067687+0000","last_clean":"2024-09-14T23:59:33.067687+0000","last_became_active":"2024-09-14T23:59:11.135384+0000","last_became_peered":"2024-09-14T23:59:11.135384+0000","last_unstale":"2024-09-14T23:59:33.067687+0000","last_undegraded":"2024-09-14T23:59:33.067687+0000","last_fullsized":"2024-09-14T23:59:33.067687+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:24:06.187016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001605626,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":5337,"num_read_kb":37535,"num_write":472,"num_write_kb":39218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":526,"num_bytes_recovered":14308962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"176'609","reported_seq":13697,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067726+0000","last_change":"2024-09-14T23:59:30.060565+0000","last_active":"2024-09-14T23:59:33.067726+0000","last_peered":"2024-09-14T23:59:33.067726+0000","last_clean":"2024-09-14T23:59:33.067726+0000","last_became_active":"2024-09-14T23:59:30.058342+0000","last_became_peered":"2024-09-14T23:59:30.058342+0000","last_unstale":"2024-09-14T23:59:33.067726+0000","last_undegraded":"2024-09-14T23:59:33.067726+0000","last_fullsized":"2024-09-14T23:59:33.067726+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:31:03.745601+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0080108929999999998,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":14616,"num_read_kb":22815,"num_write":536,"num_write_kb":24101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":1249963,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"177'995","reported_seq":6658,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067658+0000","last_change":"2024-09-14T23:59:30.060281+0000","last_active":"2024-09-14T23:59:33.067658+0000","last_peered":"2024-09-14T23:59:33.067658+0000","last_clean":"2024-09-14T23:59:33.067658+0000","last_became_active":"2024-09-14T23:59:30.058275+0000","last_became_peered":"2024-09-14T23:59:30.058275+0000","last_unstale":"2024-09-14T23:59:33.067658+0000","last_undegraded":"2024-09-14T23:59:33.067658+0000","last_fullsized":"2024-09-14T23:59:33.067658+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":995,"log_dups_size":0,"ondisk_log_size":995,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:51:47.471326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086515500000000005,"stat_sum":{"num_bytes":0,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":45,"num_read":9513,"num_read_kb":25550,"num_write":1310,"num_write_kb":9734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":2675491,"num_keys_recovered":40,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"176'524","reported_seq":2402,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.068285+0000","last_change":"2024-09-14T23:59:16.994187+0000","last_active":"2024-09-14T23:59:33.068285+0000","last_peered":"2024-09-14T23:59:33.068285+0000","last_clean":"2024-09-14T23:59:33.068285+0000","last_became_active":"2024-09-14T23:56:32.398329+0000","last_became_peered":"2024-09-14T23:56:32.398329+0000","last_unstale":"2024-09-14T23:59:33.068285+0000","last_undegraded":"2024-09-14T23:59:33.068285+0000","last_fullsized":"2024-09-14T23:59:33.068285+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":524,"log_dups_size":0,"ondisk_log_size":524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:25:07.627636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00103907,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":2076,"num_read_kb":19877,"num_write":623,"num_write_kb":10342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"177'899","reported_seq":8499,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.685085+0000","last_change":"2024-09-14T23:59:16.994978+0000","last_active":"2024-09-14T23:59:33.685085+0000","last_peered":"2024-09-14T23:59:33.685085+0000","last_clean":"2024-09-14T23:59:33.685085+0000","last_became_active":"2024-09-14T23:56:32.399655+0000","last_became_peered":"2024-09-14T23:56:32.399655+0000","last_unstale":"2024-09-14T23:59:33.685085+0000","last_undegraded":"2024-09-14T23:59:33.685085+0000","last_fullsized":"2024-09-14T23:59:33.685085+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":899,"log_dups_size":0,"ondisk_log_size":899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:01:52.825811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001807665,"stat_sum":{"num_bytes":19,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":46,"num_read":10058,"num_read_kb":53829,"num_write":970,"num_write_kb":34825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"177'695","reported_seq":4690,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.684153+0000","last_change":"2024-09-14T23:59:16.994457+0000","last_active":"2024-09-14T23:59:33.684153+0000","last_peered":"2024-09-14T23:59:33.684153+0000","last_clean":"2024-09-14T23:59:33.684153+0000","last_became_active":"2024-09-14T23:59:11.135739+0000","last_became_peered":"2024-09-14T23:59:11.135739+0000","last_unstale":"2024-09-14T23:59:33.684153+0000","last_undegraded":"2024-09-14T23:59:33.684153+0000","last_fullsized":"2024-09-14T23:59:33.684153+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:33:05.725582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099865499999999999,"stat_sum":{"num_bytes":0,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":42,"num_read":5485,"num_read_kb":20536,"num_write":717,"num_write_kb":15834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":318,"num_bytes_recovered":6472298,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"177'742","reported_seq":6564,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067404+0000","last_change":"2024-09-14T23:59:16.998152+0000","last_active":"2024-09-14T23:59:33.067404+0000","last_peered":"2024-09-14T23:59:33.067404+0000","last_clean":"2024-09-14T23:59:33.067404+0000","last_became_active":"2024-09-14T23:59:12.133258+0000","last_became_peered":"2024-09-14T23:59:12.133258+0000","last_unstale":"2024-09-14T23:59:33.067404+0000","last_undegraded":"2024-09-14T23:59:33.067404+0000","last_fullsized":"2024-09-14T23:59:33.067404+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":742,"log_dups_size":0,"ondisk_log_size":742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:43:45.205419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093947500000000005,"stat_sum":{"num_bytes":0,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":46,"num_read":6734,"num_read_kb":59364,"num_write":741,"num_write_kb":26095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":340,"num_bytes_recovered":12763266,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"176'544","reported_seq":11742,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.068027+0000","last_change":"2024-09-14T23:59:16.994889+0000","last_active":"2024-09-14T23:59:33.068027+0000","last_peered":"2024-09-14T23:59:33.068027+0000","last_clean":"2024-09-14T23:59:33.068027+0000","last_became_active":"2024-09-14T23:57:00.062304+0000","last_became_peered":"2024-09-14T23:57:00.062304+0000","last_unstale":"2024-09-14T23:59:33.068027+0000","last_undegraded":"2024-09-14T23:59:33.068027+0000","last_fullsized":"2024-09-14T23:59:33.068027+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:28:05.460487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091006300000000002,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":12421,"num_read_kb":16852,"num_write":491,"num_write_kb":6294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":3345543,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"177'463","reported_seq":7658,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067791+0000","last_change":"2024-09-14T23:59:30.062097+0000","last_active":"2024-09-14T23:59:33.067791+0000","last_peered":"2024-09-14T23:59:33.067791+0000","last_clean":"2024-09-14T23:59:33.067791+0000","last_became_active":"2024-09-14T23:59:30.060469+0000","last_became_peered":"2024-09-14T23:59:30.060469+0000","last_unstale":"2024-09-14T23:59:33.067791+0000","last_undegraded":"2024-09-14T23:59:33.067791+0000","last_fullsized":"2024-09-14T23:59:33.067791+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:38:29.716639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002288432,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":7773,"num_read_kb":27495,"num_write":382,"num_write_kb":13368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":404,"num_bytes_recovered":26159849,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"176'612","reported_seq":9857,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.367048+0000","last_change":"2024-09-14T23:59:17.002582+0000","last_active":"2024-09-14T23:59:33.367048+0000","last_peered":"2024-09-14T23:59:33.367048+0000","last_clean":"2024-09-14T23:59:33.367048+0000","last_became_active":"2024-09-14T23:59:11.135860+0000","last_became_peered":"2024-09-14T23:59:11.135860+0000","last_unstale":"2024-09-14T23:59:33.367048+0000","last_undegraded":"2024-09-14T23:59:33.367048+0000","last_fullsized":"2024-09-14T23:59:33.367048+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":612,"log_dups_size":0,"ondisk_log_size":612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:16:43.430234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001929275,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":9948,"num_read_kb":36137,"num_write":618,"num_write_kb":23694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":516,"num_bytes_recovered":42197286,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"176'1093","reported_seq":8194,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.685233+0000","last_change":"2024-09-14T23:59:16.996306+0000","last_active":"2024-09-14T23:59:33.685233+0000","last_peered":"2024-09-14T23:59:33.685233+0000","last_clean":"2024-09-14T23:59:33.685233+0000","last_became_active":"2024-09-14T23:58:53.983979+0000","last_became_peered":"2024-09-14T23:58:53.983979+0000","last_unstale":"2024-09-14T23:59:33.685233+0000","last_undegraded":"2024-09-14T23:59:33.685233+0000","last_fullsized":"2024-09-14T23:59:33.685233+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":1093,"log_dups_size":0,"ondisk_log_size":1093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:26:54.972175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002903152,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":9679,"num_read_kb":42324,"num_write":1584,"num_write_kb":28908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":312,"num_bytes_recovered":22540127,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":373,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.684825+0000","last_change":"2024-09-14T23:58:49.989598+0000","last_active":"2024-09-14T23:59:33.684825+0000","last_peered":"2024-09-14T23:59:33.684825+0000","last_clean":"2024-09-14T23:59:33.684825+0000","last_became_active":"2024-09-14T23:58:49.989224+0000","last_became_peered":"2024-09-14T23:58:49.989224+0000","last_unstale":"2024-09-14T23:59:33.684825+0000","last_undegraded":"2024-09-14T23:59:33.684825+0000","last_fullsized":"2024-09-14T23:59:33.684825+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:52:25.663126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045105900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"177'718","reported_seq":5318,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.068336+0000","last_change":"2024-09-14T23:59:16.995528+0000","last_active":"2024-09-14T23:59:33.068336+0000","last_peered":"2024-09-14T23:59:33.068336+0000","last_clean":"2024-09-14T23:59:33.068336+0000","last_became_active":"2024-09-14T23:56:32.398664+0000","last_became_peered":"2024-09-14T23:56:32.398664+0000","last_unstale":"2024-09-14T23:59:33.068336+0000","last_undegraded":"2024-09-14T23:59:33.068336+0000","last_fullsized":"2024-09-14T23:59:33.068336+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":718,"log_dups_size":0,"ondisk_log_size":718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:15:18.740717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001169722,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":6635,"num_read_kb":24355,"num_write":754,"num_write_kb":5733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":363,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.684380+0000","last_change":"2024-09-14T23:56:24.534632+0000","last_active":"2024-09-14T23:59:33.684380+0000","last_peered":"2024-09-14T23:59:33.684380+0000","last_clean":"2024-09-14T23:59:33.684380+0000","last_became_active":"2024-09-14T23:56:22.526355+0000","last_became_peered":"2024-09-14T23:56:22.526355+0000","last_unstale":"2024-09-14T23:59:33.684380+0000","last_undegraded":"2024-09-14T23:59:33.684380+0000","last_fullsized":"2024-09-14T23:59:33.684380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:02:16.059627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049910000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"177'528","reported_seq":7754,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067820+0000","last_change":"2024-09-14T23:59:17.000419+0000","last_active":"2024-09-14T23:59:33.067820+0000","last_peered":"2024-09-14T23:59:33.067820+0000","last_clean":"2024-09-14T23:59:33.067820+0000","last_became_active":"2024-09-14T23:59:11.135030+0000","last_became_peered":"2024-09-14T23:59:11.135030+0000","last_unstale":"2024-09-14T23:59:33.067820+0000","last_undegraded":"2024-09-14T23:59:33.067820+0000","last_fullsized":"2024-09-14T23:59:33.067820+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":528,"log_dups_size":0,"ondisk_log_size":528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:12:06.383593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027209629999999999,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":7655,"num_read_kb":43475,"num_write":499,"num_write_kb":17299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":640,"num_bytes_recovered":36520799,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":95,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.366957+0000","last_change":"2024-09-14T23:58:49.988483+0000","last_active":"2024-09-14T23:59:33.366957+0000","last_peered":"2024-09-14T23:59:33.366957+0000","last_clean":"2024-09-14T23:59:33.366957+0000","last_became_active":"2024-09-14T23:58:49.988169+0000","last_became_peered":"2024-09-14T23:58:49.988169+0000","last_unstale":"2024-09-14T23:59:33.366957+0000","last_undegraded":"2024-09-14T23:59:33.366957+0000","last_fullsized":"2024-09-14T23:59:33.366957+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:57:59.569156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"177'622","reported_seq":5510,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.684308+0000","last_change":"2024-09-14T23:59:16.993791+0000","last_active":"2024-09-14T23:59:33.684308+0000","last_peered":"2024-09-14T23:59:33.684308+0000","last_clean":"2024-09-14T23:59:33.684308+0000","last_became_active":"2024-09-14T23:56:32.401089+0000","last_became_peered":"2024-09-14T23:56:32.401089+0000","last_unstale":"2024-09-14T23:59:33.684308+0000","last_undegraded":"2024-09-14T23:59:33.684308+0000","last_fullsized":"2024-09-14T23:59:33.684308+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:44:29.413362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094649999999999997,"stat_sum":{"num_bytes":0,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":43,"num_read":6577,"num_read_kb":26346,"num_write":596,"num_write_kb":13585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":363,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.684310+0000","last_change":"2024-09-14T23:56:24.534510+0000","last_active":"2024-09-14T23:59:33.684310+0000","last_peered":"2024-09-14T23:59:33.684310+0000","last_clean":"2024-09-14T23:59:33.684310+0000","last_became_active":"2024-09-14T23:56:22.527611+0000","last_became_peered":"2024-09-14T23:56:22.527611+0000","last_unstale":"2024-09-14T23:59:33.684310+0000","last_undegraded":"2024-09-14T23:59:33.684310+0000","last_fullsized":"2024-09-14T23:59:33.684310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:15:45.606026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049916299999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"176'729","reported_seq":7600,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067812+0000","last_change":"2024-09-14T23:59:30.062265+0000","last_active":"2024-09-14T23:59:33.067812+0000","last_peered":"2024-09-14T23:59:33.067812+0000","last_clean":"2024-09-14T23:59:33.067812+0000","last_became_active":"2024-09-14T23:59:30.060381+0000","last_became_peered":"2024-09-14T23:59:30.060381+0000","last_unstale":"2024-09-14T23:59:33.067812+0000","last_undegraded":"2024-09-14T23:59:33.067812+0000","last_fullsized":"2024-09-14T23:59:33.067812+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":729,"log_dups_size":0,"ondisk_log_size":729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:19:07.081227+0000","scrub_duration":0,"objects_trimmed":10,"snaptrim_duration":0.012412685999999999,"stat_sum":{"num_bytes":0,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":47,"num_read":9294,"num_read_kb":18810,"num_write":776,"num_write_kb":10083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":401,"num_bytes_recovered":9491176,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"177'834","reported_seq":12831,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.685176+0000","last_change":"2024-09-14T23:59:16.995637+0000","last_active":"2024-09-14T23:59:33.685176+0000","last_peered":"2024-09-14T23:59:33.685176+0000","last_clean":"2024-09-14T23:59:33.685176+0000","last_became_active":"2024-09-14T23:56:32.397883+0000","last_became_peered":"2024-09-14T23:56:32.397883+0000","last_unstale":"2024-09-14T23:59:33.685176+0000","last_undegraded":"2024-09-14T23:59:33.685176+0000","last_fullsized":"2024-09-14T23:59:33.685176+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":834,"log_dups_size":0,"ondisk_log_size":834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:51:15.895964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023599810000000001,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":38,"num_read":13370,"num_read_kb":25781,"num_write":804,"num_write_kb":35448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":87,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067672+0000","last_change":"2024-09-14T23:59:30.058288+0000","last_active":"2024-09-14T23:59:33.067672+0000","last_peered":"2024-09-14T23:59:33.067672+0000","last_clean":"2024-09-14T23:59:33.067672+0000","last_became_active":"2024-09-14T23:59:30.057866+0000","last_became_peered":"2024-09-14T23:59:30.057866+0000","last_unstale":"2024-09-14T23:59:33.067672+0000","last_undegraded":"2024-09-14T23:59:33.067672+0000","last_fullsized":"2024-09-14T23:59:33.067672+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:55:57.947601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"177'518","reported_seq":9365,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.684112+0000","last_change":"2024-09-14T23:59:16.993927+0000","last_active":"2024-09-14T23:59:33.684112+0000","last_peered":"2024-09-14T23:59:33.684112+0000","last_clean":"2024-09-14T23:59:33.684112+0000","last_became_active":"2024-09-14T23:56:32.400278+0000","last_became_peered":"2024-09-14T23:56:32.400278+0000","last_unstale":"2024-09-14T23:59:33.684112+0000","last_undegraded":"2024-09-14T23:59:33.684112+0000","last_fullsized":"2024-09-14T23:59:33.684112+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:26:19.473912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098058499999999996,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":8550,"num_read_kb":62565,"num_write":432,"num_write_kb":30360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":341,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.684325+0000","last_change":"2024-09-14T23:59:23.991858+0000","last_active":"2024-09-14T23:59:33.684325+0000","last_peered":"2024-09-14T23:59:33.684325+0000","last_clean":"2024-09-14T23:59:33.684325+0000","last_became_active":"2024-09-14T23:59:23.991492+0000","last_became_peered":"2024-09-14T23:59:23.991492+0000","last_unstale":"2024-09-14T23:59:33.684325+0000","last_undegraded":"2024-09-14T23:59:33.684325+0000","last_fullsized":"2024-09-14T23:59:33.684325+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:13:20.777068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026863400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"176'788","reported_seq":7241,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067718+0000","last_change":"2024-09-14T23:59:20.000557+0000","last_active":"2024-09-14T23:59:33.067718+0000","last_peered":"2024-09-14T23:59:33.067718+0000","last_clean":"2024-09-14T23:59:33.067718+0000","last_became_active":"2024-09-14T23:59:19.998578+0000","last_became_peered":"2024-09-14T23:59:19.998578+0000","last_unstale":"2024-09-14T23:59:33.067718+0000","last_undegraded":"2024-09-14T23:59:33.067718+0000","last_fullsized":"2024-09-14T23:59:33.067718+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":788,"log_dups_size":0,"ondisk_log_size":788,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:22:03.401033+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.00381613,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":9431,"num_read_kb":22132,"num_write":943,"num_write_kb":5118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":95,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.366890+0000","last_change":"2024-09-14T23:58:49.988483+0000","last_active":"2024-09-14T23:59:33.366890+0000","last_peered":"2024-09-14T23:59:33.366890+0000","last_clean":"2024-09-14T23:59:33.366890+0000","last_became_active":"2024-09-14T23:58:49.988096+0000","last_became_peered":"2024-09-14T23:58:49.988096+0000","last_unstale":"2024-09-14T23:59:33.366890+0000","last_undegraded":"2024-09-14T23:59:33.366890+0000","last_fullsized":"2024-09-14T23:59:33.366890+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:34:48.166954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"176'1105","reported_seq":30205,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067544+0000","last_change":"2024-09-14T23:59:16.999655+0000","last_active":"2024-09-14T23:59:33.067544+0000","last_peered":"2024-09-14T23:59:33.067544+0000","last_clean":"2024-09-14T23:59:33.067544+0000","last_became_active":"2024-09-14T23:59:12.133789+0000","last_became_peered":"2024-09-14T23:59:12.133789+0000","last_unstale":"2024-09-14T23:59:33.067544+0000","last_undegraded":"2024-09-14T23:59:33.067544+0000","last_fullsized":"2024-09-14T23:59:33.067544+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":1105,"log_dups_size":0,"ondisk_log_size":1105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:04:55.112519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002233971,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":31963,"num_read_kb":26382,"num_write":1422,"num_write_kb":17307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":641,"num_bytes_recovered":19031962,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":122,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067683+0000","last_change":"2024-09-14T23:59:30.060260+0000","last_active":"2024-09-14T23:59:33.067683+0000","last_peered":"2024-09-14T23:59:33.067683+0000","last_clean":"2024-09-14T23:59:33.067683+0000","last_became_active":"2024-09-14T23:59:30.059854+0000","last_became_peered":"2024-09-14T23:59:30.059854+0000","last_unstale":"2024-09-14T23:59:33.067683+0000","last_undegraded":"2024-09-14T23:59:33.067683+0000","last_fullsized":"2024-09-14T23:59:33.067683+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:14:35.344939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":277,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.366889+0000","last_change":"2024-09-14T23:58:53.985309+0000","last_active":"2024-09-14T23:59:33.366889+0000","last_peered":"2024-09-14T23:59:33.366889+0000","last_clean":"2024-09-14T23:59:33.366889+0000","last_became_active":"2024-09-14T23:58:53.984830+0000","last_became_peered":"2024-09-14T23:58:53.984830+0000","last_unstale":"2024-09-14T23:59:33.366889+0000","last_undegraded":"2024-09-14T23:59:33.366889+0000","last_fullsized":"2024-09-14T23:59:33.366889+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:14.322882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:14.322882+0000","last_clean_scrub_stamp":"2024-09-14T23:56:14.322882+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:37:48.336097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"177'609","reported_seq":5982,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067398+0000","last_change":"2024-09-14T23:59:17.000201+0000","last_active":"2024-09-14T23:59:33.067398+0000","last_peered":"2024-09-14T23:59:33.067398+0000","last_clean":"2024-09-14T23:59:33.067398+0000","last_became_active":"2024-09-14T23:59:12.136444+0000","last_became_peered":"2024-09-14T23:59:12.136444+0000","last_unstale":"2024-09-14T23:59:33.067398+0000","last_undegraded":"2024-09-14T23:59:33.067398+0000","last_fullsized":"2024-09-14T23:59:33.067398+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:26:06.094223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026622379999999999,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":5935,"num_read_kb":18123,"num_write":528,"num_write_kb":10919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":399,"num_bytes_recovered":9189081,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"176'343","reported_seq":11038,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.685073+0000","last_change":"2024-09-14T23:59:16.994730+0000","last_active":"2024-09-14T23:59:33.685073+0000","last_peered":"2024-09-14T23:59:33.685073+0000","last_clean":"2024-09-14T23:59:33.685073+0000","last_became_active":"2024-09-14T23:56:32.400017+0000","last_became_peered":"2024-09-14T23:56:32.400017+0000","last_unstale":"2024-09-14T23:59:33.685073+0000","last_undegraded":"2024-09-14T23:59:33.685073+0000","last_fullsized":"2024-09-14T23:59:33.685073+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:24:02.170046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015805459999999999,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":10189,"num_read_kb":13014,"num_write":297,"num_write_kb":15704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"176'834","reported_seq":16004,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.684762+0000","last_change":"2024-09-14T23:59:17.069111+0000","last_active":"2024-09-14T23:59:33.684762+0000","last_peered":"2024-09-14T23:59:33.684762+0000","last_clean":"2024-09-14T23:59:33.684762+0000","last_became_active":"2024-09-14T23:56:32.402769+0000","last_became_peered":"2024-09-14T23:56:32.402769+0000","last_unstale":"2024-09-14T23:59:33.684762+0000","last_undegraded":"2024-09-14T23:59:33.684762+0000","last_fullsized":"2024-09-14T23:59:33.684762+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":834,"log_dups_size":0,"ondisk_log_size":834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:36:02.993358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098966399999999991,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":40,"num_read":16916,"num_read_kb":32658,"num_write":1110,"num_write_kb":14962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"176'665","reported_seq":13705,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067583+0000","last_change":"2024-09-14T23:59:16.998126+0000","last_active":"2024-09-14T23:59:33.067583+0000","last_peered":"2024-09-14T23:59:33.067583+0000","last_clean":"2024-09-14T23:59:33.067583+0000","last_became_active":"2024-09-14T23:59:12.132838+0000","last_became_peered":"2024-09-14T23:59:12.132838+0000","last_unstale":"2024-09-14T23:59:33.067583+0000","last_undegraded":"2024-09-14T23:59:33.067583+0000","last_fullsized":"2024-09-14T23:59:33.067583+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":665,"log_dups_size":0,"ondisk_log_size":665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:34:25.667684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087623300000000002,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":13816,"num_read_kb":56666,"num_write":771,"num_write_kb":15952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":258,"num_bytes_recovered":8738708,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"176'705","reported_seq":16265,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067866+0000","last_change":"2024-09-14T23:59:30.062604+0000","last_active":"2024-09-14T23:59:33.067866+0000","last_peered":"2024-09-14T23:59:33.067866+0000","last_clean":"2024-09-14T23:59:33.067866+0000","last_became_active":"2024-09-14T23:59:30.060837+0000","last_became_peered":"2024-09-14T23:59:30.060837+0000","last_unstale":"2024-09-14T23:59:33.067866+0000","last_undegraded":"2024-09-14T23:59:33.067866+0000","last_fullsized":"2024-09-14T23:59:33.067866+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":705,"log_dups_size":0,"ondisk_log_size":705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:18:03.409303+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.021276283,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":15379,"num_read_kb":69494,"num_write":885,"num_write_kb":43376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":30030415,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"176'521","reported_seq":3381,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.367121+0000","last_change":"2024-09-14T23:59:17.003938+0000","last_active":"2024-09-14T23:59:33.367121+0000","last_peered":"2024-09-14T23:59:33.367121+0000","last_clean":"2024-09-14T23:59:33.367121+0000","last_became_active":"2024-09-14T23:58:52.985661+0000","last_became_peered":"2024-09-14T23:58:52.985661+0000","last_unstale":"2024-09-14T23:59:33.367121+0000","last_undegraded":"2024-09-14T23:59:33.367121+0000","last_fullsized":"2024-09-14T23:59:33.367121+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:53:00.607967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030262919999999999,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":3961,"num_read_kb":12312,"num_write":537,"num_write_kb":8315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":453,"num_bytes_recovered":19224487,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"177'532","reported_seq":5732,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.367080+0000","last_change":"2024-09-14T23:59:17.003376+0000","last_active":"2024-09-14T23:59:33.367080+0000","last_peered":"2024-09-14T23:59:33.367080+0000","last_clean":"2024-09-14T23:59:33.367080+0000","last_became_active":"2024-09-14T23:58:53.986149+0000","last_became_peered":"2024-09-14T23:58:53.986149+0000","last_unstale":"2024-09-14T23:59:33.367080+0000","last_undegraded":"2024-09-14T23:59:33.367080+0000","last_fullsized":"2024-09-14T23:59:33.367080+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:15:03.478135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024995130000000001,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":7765,"num_read_kb":21397,"num_write":463,"num_write_kb":11356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":12997809,"num_keys_recovered":45,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"176'663","reported_seq":14079,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.367080+0000","last_change":"2024-09-14T23:59:17.004025+0000","last_active":"2024-09-14T23:59:33.367080+0000","last_peered":"2024-09-14T23:59:33.367080+0000","last_clean":"2024-09-14T23:59:33.367080+0000","last_became_active":"2024-09-14T23:58:53.984808+0000","last_became_peered":"2024-09-14T23:58:53.984808+0000","last_unstale":"2024-09-14T23:59:33.367080+0000","last_undegraded":"2024-09-14T23:59:33.367080+0000","last_fullsized":"2024-09-14T23:59:33.367080+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":663,"log_dups_size":0,"ondisk_log_size":663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:48:22.749991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030575429999999998,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":14675,"num_read_kb":21661,"num_write":655,"num_write_kb":35778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":12964873,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"176'797","reported_seq":6504,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.684762+0000","last_change":"2024-09-14T23:59:17.069019+0000","last_active":"2024-09-14T23:59:33.684762+0000","last_peered":"2024-09-14T23:59:33.684762+0000","last_clean":"2024-09-14T23:59:33.684762+0000","last_became_active":"2024-09-14T23:56:32.402973+0000","last_became_peered":"2024-09-14T23:56:32.402973+0000","last_unstale":"2024-09-14T23:59:33.684762+0000","last_undegraded":"2024-09-14T23:59:33.684762+0000","last_fullsized":"2024-09-14T23:59:33.684762+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":797,"log_dups_size":0,"ondisk_log_size":797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:18:08.492160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088108100000000003,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":7036,"num_read_kb":57979,"num_write":1182,"num_write_kb":51466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"176'569","reported_seq":8379,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.366890+0000","last_change":"2024-09-14T23:59:17.001624+0000","last_active":"2024-09-14T23:59:33.366890+0000","last_peered":"2024-09-14T23:59:33.366890+0000","last_clean":"2024-09-14T23:59:33.366890+0000","last_became_active":"2024-09-14T23:58:52.987786+0000","last_became_peered":"2024-09-14T23:58:52.987786+0000","last_unstale":"2024-09-14T23:59:33.366890+0000","last_undegraded":"2024-09-14T23:59:33.366890+0000","last_fullsized":"2024-09-14T23:59:33.366890+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:55:48.806780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001145566,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":8694,"num_read_kb":36214,"num_write":569,"num_write_kb":28063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":19310765,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"176'376","reported_seq":3270,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067650+0000","last_change":"2024-09-14T23:59:20.000977+0000","last_active":"2024-09-14T23:59:33.067650+0000","last_peered":"2024-09-14T23:59:33.067650+0000","last_clean":"2024-09-14T23:59:33.067650+0000","last_became_active":"2024-09-14T23:59:19.999489+0000","last_became_peered":"2024-09-14T23:59:19.999489+0000","last_unstale":"2024-09-14T23:59:33.067650+0000","last_undegraded":"2024-09-14T23:59:33.067650+0000","last_fullsized":"2024-09-14T23:59:33.067650+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:21:32.054413+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0042424580000000002,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":3170,"num_read_kb":16380,"num_write":287,"num_write_kb":14646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":7078411,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"176'975","reported_seq":4718,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.366901+0000","last_change":"2024-09-14T23:59:17.002239+0000","last_active":"2024-09-14T23:59:33.366901+0000","last_peered":"2024-09-14T23:59:33.366901+0000","last_clean":"2024-09-14T23:59:33.366901+0000","last_became_active":"2024-09-14T23:58:53.987342+0000","last_became_peered":"2024-09-14T23:58:53.987342+0000","last_unstale":"2024-09-14T23:59:33.366901+0000","last_undegraded":"2024-09-14T23:59:33.366901+0000","last_fullsized":"2024-09-14T23:59:33.366901+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:30.333512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001659113,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":7188,"num_read_kb":25892,"num_write":1276,"num_write_kb":14588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":290,"num_bytes_recovered":14372748,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":1204,"num_object_clones":0,"num_object_copies":2408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1204,"num_whiteouts":1200,"num_read":310661,"num_read_kb":1031538,"num_write":24584,"num_write_kb":670524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7228,"num_bytes_recovered":332991677,"num_keys_recovered":596,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22207,"ondisk_log_size":22207,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":1771104,"data_compressed":100161,"data_compressed_allocated":135168,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509486,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51552,"kb_used_data":2316,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320288,"statfs":{"total":96636764160,"available":96583974912,"internally_reserved":0,"allocated":2371584,"data_stored":1713775,"data_compressed":33932,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509485,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33456,"kb_used_data":2268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338384,"statfs":{"total":96636764160,"available":96602505216,"internally_reserved":0,"allocated":2322432,"data_stored":1123407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509484,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47348,"kb_used_data":2272,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324492,"statfs":{"total":96636764160,"available":96588279808,"internally_reserved":0,"allocated":2326528,"data_stored":1123426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509486,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77876,"kb_used_data":2272,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293964,"statfs":{"total":96636764160,"available":96557019136,"internally_reserved":0,"allocated":2326528,"data_stored":1123426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509484,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55668,"kb_used_data":2268,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316172,"statfs":{"total":96636764160,"available":96579760128,"internally_reserved":0,"allocated":2322432,"data_stored":1123407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542188,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41772,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330068,"statfs":{"total":96636764160,"available":96593989632,"internally_reserved":0,"allocated":2318336,"data_stored":1117603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542187,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51576,"kb_used_data":2332,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320264,"statfs":{"total":96636764160,"available":96583950336,"internally_reserved":0,"allocated":2387968,"data_stored":1713775,"data_compressed":32297,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574892,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33456,"kb_used_data":2268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338384,"statfs":{"total":96636764160,"available":96602505216,"internally_reserved":0,"allocated":2322432,"data_stored":1123407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32297,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":33932,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":33932,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T23:59:38.010 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-14T23:59:38.010 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-14T23:59:38.324 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-14T23:59:38.334 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-14T23:59:38.334 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T23:59:38.625 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:59:38.625 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-14T23:59:38.636 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2024-09-14T23:59:37.488524+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":1208,"num_object_clones":0,"num_object_copies":2416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1208,"num_whiteouts":1200,"num_read":310777,"num_read_kb":1031638,"num_write":24745,"num_write_kb":672942,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7234,"num_bytes_recovered":334172432,"num_keys_recovered":596,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22327,"ondisk_log_size":22327,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":392704,"kb_used_data":18260,"kb_used_omap":12,"kb_used_meta":374259,"kb_avail":754582016,"statfs":{"total":773094113280,"available":772691984384,"internally_reserved":0,"allocated":18698240,"data_stored":10162226,"data_compressed":66229,"data_compressed_allocated":90112,"data_compressed_original":1163264,"omap_allocated":12752,"internal_metadata":383241776},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.434141"},"pg_stats":[{"pgid":"3.1f","version":"177'1121","reported_seq":6213,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067915+0000","last_change":"2024-09-14T23:59:16.994682+0000","last_active":"2024-09-14T23:59:33.067915+0000","last_peered":"2024-09-14T23:59:33.067915+0000","last_clean":"2024-09-14T23:59:33.067915+0000","last_became_active":"2024-09-14T23:57:00.062116+0000","last_became_peered":"2024-09-14T23:57:00.062116+0000","last_unstale":"2024-09-14T23:59:33.067915+0000","last_undegraded":"2024-09-14T23:59:33.067915+0000","last_fullsized":"2024-09-14T23:59:33.067915+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":1121,"log_dups_size":0,"ondisk_log_size":1121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:38:40.792973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094842500000000003,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":8822,"num_read_kb":37588,"num_write":1354,"num_write_kb":38053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":2327658,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"177'479","reported_seq":6679,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067687+0000","last_change":"2024-09-14T23:59:16.998886+0000","last_active":"2024-09-14T23:59:33.067687+0000","last_peered":"2024-09-14T23:59:33.067687+0000","last_clean":"2024-09-14T23:59:33.067687+0000","last_became_active":"2024-09-14T23:59:11.135384+0000","last_became_peered":"2024-09-14T23:59:11.135384+0000","last_unstale":"2024-09-14T23:59:33.067687+0000","last_undegraded":"2024-09-14T23:59:33.067687+0000","last_fullsized":"2024-09-14T23:59:33.067687+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:24:06.187016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001605626,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":5337,"num_read_kb":37535,"num_write":472,"num_write_kb":39218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":526,"num_bytes_recovered":14308962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"176'609","reported_seq":13697,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067726+0000","last_change":"2024-09-14T23:59:30.060565+0000","last_active":"2024-09-14T23:59:33.067726+0000","last_peered":"2024-09-14T23:59:33.067726+0000","last_clean":"2024-09-14T23:59:33.067726+0000","last_became_active":"2024-09-14T23:59:30.058342+0000","last_became_peered":"2024-09-14T23:59:30.058342+0000","last_unstale":"2024-09-14T23:59:33.067726+0000","last_undegraded":"2024-09-14T23:59:33.067726+0000","last_fullsized":"2024-09-14T23:59:33.067726+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:31:03.745601+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0080108929999999998,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":14616,"num_read_kb":22815,"num_write":536,"num_write_kb":24101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":1249963,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"177'995","reported_seq":6658,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067658+0000","last_change":"2024-09-14T23:59:30.060281+0000","last_active":"2024-09-14T23:59:33.067658+0000","last_peered":"2024-09-14T23:59:33.067658+0000","last_clean":"2024-09-14T23:59:33.067658+0000","last_became_active":"2024-09-14T23:59:30.058275+0000","last_became_peered":"2024-09-14T23:59:30.058275+0000","last_unstale":"2024-09-14T23:59:33.067658+0000","last_undegraded":"2024-09-14T23:59:33.067658+0000","last_fullsized":"2024-09-14T23:59:33.067658+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":995,"log_dups_size":0,"ondisk_log_size":995,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:51:47.471326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086515500000000005,"stat_sum":{"num_bytes":0,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":45,"num_read":9513,"num_read_kb":25550,"num_write":1310,"num_write_kb":9734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":2675491,"num_keys_recovered":40,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"176'524","reported_seq":2402,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.068285+0000","last_change":"2024-09-14T23:59:16.994187+0000","last_active":"2024-09-14T23:59:33.068285+0000","last_peered":"2024-09-14T23:59:33.068285+0000","last_clean":"2024-09-14T23:59:33.068285+0000","last_became_active":"2024-09-14T23:56:32.398329+0000","last_became_peered":"2024-09-14T23:56:32.398329+0000","last_unstale":"2024-09-14T23:59:33.068285+0000","last_undegraded":"2024-09-14T23:59:33.068285+0000","last_fullsized":"2024-09-14T23:59:33.068285+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":524,"log_dups_size":0,"ondisk_log_size":524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:25:07.627636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00103907,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":2076,"num_read_kb":19877,"num_write":623,"num_write_kb":10342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"177'899","reported_seq":8499,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.685085+0000","last_change":"2024-09-14T23:59:16.994978+0000","last_active":"2024-09-14T23:59:33.685085+0000","last_peered":"2024-09-14T23:59:33.685085+0000","last_clean":"2024-09-14T23:59:33.685085+0000","last_became_active":"2024-09-14T23:56:32.399655+0000","last_became_peered":"2024-09-14T23:56:32.399655+0000","last_unstale":"2024-09-14T23:59:33.685085+0000","last_undegraded":"2024-09-14T23:59:33.685085+0000","last_fullsized":"2024-09-14T23:59:33.685085+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":899,"log_dups_size":0,"ondisk_log_size":899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:01:52.825811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001807665,"stat_sum":{"num_bytes":19,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":46,"num_read":10058,"num_read_kb":53829,"num_write":970,"num_write_kb":34825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"177'695","reported_seq":4690,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.684153+0000","last_change":"2024-09-14T23:59:16.994457+0000","last_active":"2024-09-14T23:59:33.684153+0000","last_peered":"2024-09-14T23:59:33.684153+0000","last_clean":"2024-09-14T23:59:33.684153+0000","last_became_active":"2024-09-14T23:59:11.135739+0000","last_became_peered":"2024-09-14T23:59:11.135739+0000","last_unstale":"2024-09-14T23:59:33.684153+0000","last_undegraded":"2024-09-14T23:59:33.684153+0000","last_fullsized":"2024-09-14T23:59:33.684153+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:33:05.725582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099865499999999999,"stat_sum":{"num_bytes":0,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":42,"num_read":5485,"num_read_kb":20536,"num_write":717,"num_write_kb":15834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":318,"num_bytes_recovered":6472298,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"177'742","reported_seq":6564,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067404+0000","last_change":"2024-09-14T23:59:16.998152+0000","last_active":"2024-09-14T23:59:33.067404+0000","last_peered":"2024-09-14T23:59:33.067404+0000","last_clean":"2024-09-14T23:59:33.067404+0000","last_became_active":"2024-09-14T23:59:12.133258+0000","last_became_peered":"2024-09-14T23:59:12.133258+0000","last_unstale":"2024-09-14T23:59:33.067404+0000","last_undegraded":"2024-09-14T23:59:33.067404+0000","last_fullsized":"2024-09-14T23:59:33.067404+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":742,"log_dups_size":0,"ondisk_log_size":742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:43:45.205419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093947500000000005,"stat_sum":{"num_bytes":0,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":46,"num_read":6734,"num_read_kb":59364,"num_write":741,"num_write_kb":26095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":340,"num_bytes_recovered":12763266,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"176'544","reported_seq":11742,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.068027+0000","last_change":"2024-09-14T23:59:16.994889+0000","last_active":"2024-09-14T23:59:33.068027+0000","last_peered":"2024-09-14T23:59:33.068027+0000","last_clean":"2024-09-14T23:59:33.068027+0000","last_became_active":"2024-09-14T23:57:00.062304+0000","last_became_peered":"2024-09-14T23:57:00.062304+0000","last_unstale":"2024-09-14T23:59:33.068027+0000","last_undegraded":"2024-09-14T23:59:33.068027+0000","last_fullsized":"2024-09-14T23:59:33.068027+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:28:05.460487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091006300000000002,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":12421,"num_read_kb":16852,"num_write":491,"num_write_kb":6294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":3345543,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"177'463","reported_seq":7658,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067791+0000","last_change":"2024-09-14T23:59:30.062097+0000","last_active":"2024-09-14T23:59:33.067791+0000","last_peered":"2024-09-14T23:59:33.067791+0000","last_clean":"2024-09-14T23:59:33.067791+0000","last_became_active":"2024-09-14T23:59:30.060469+0000","last_became_peered":"2024-09-14T23:59:30.060469+0000","last_unstale":"2024-09-14T23:59:33.067791+0000","last_undegraded":"2024-09-14T23:59:33.067791+0000","last_fullsized":"2024-09-14T23:59:33.067791+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:38:29.716639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002288432,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":7773,"num_read_kb":27495,"num_write":382,"num_write_kb":13368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":404,"num_bytes_recovered":26159849,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"176'612","reported_seq":9857,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.367048+0000","last_change":"2024-09-14T23:59:17.002582+0000","last_active":"2024-09-14T23:59:33.367048+0000","last_peered":"2024-09-14T23:59:33.367048+0000","last_clean":"2024-09-14T23:59:33.367048+0000","last_became_active":"2024-09-14T23:59:11.135860+0000","last_became_peered":"2024-09-14T23:59:11.135860+0000","last_unstale":"2024-09-14T23:59:33.367048+0000","last_undegraded":"2024-09-14T23:59:33.367048+0000","last_fullsized":"2024-09-14T23:59:33.367048+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":612,"log_dups_size":0,"ondisk_log_size":612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:16:43.430234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001929275,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":9948,"num_read_kb":36137,"num_write":618,"num_write_kb":23694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":516,"num_bytes_recovered":42197286,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"176'1093","reported_seq":8194,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.685233+0000","last_change":"2024-09-14T23:59:16.996306+0000","last_active":"2024-09-14T23:59:33.685233+0000","last_peered":"2024-09-14T23:59:33.685233+0000","last_clean":"2024-09-14T23:59:33.685233+0000","last_became_active":"2024-09-14T23:58:53.983979+0000","last_became_peered":"2024-09-14T23:58:53.983979+0000","last_unstale":"2024-09-14T23:59:33.685233+0000","last_undegraded":"2024-09-14T23:59:33.685233+0000","last_fullsized":"2024-09-14T23:59:33.685233+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":1093,"log_dups_size":0,"ondisk_log_size":1093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:26:54.972175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002903152,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":9679,"num_read_kb":42324,"num_write":1584,"num_write_kb":28908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":312,"num_bytes_recovered":22540127,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":373,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.684825+0000","last_change":"2024-09-14T23:58:49.989598+0000","last_active":"2024-09-14T23:59:33.684825+0000","last_peered":"2024-09-14T23:59:33.684825+0000","last_clean":"2024-09-14T23:59:33.684825+0000","last_became_active":"2024-09-14T23:58:49.989224+0000","last_became_peered":"2024-09-14T23:58:49.989224+0000","last_unstale":"2024-09-14T23:59:33.684825+0000","last_undegraded":"2024-09-14T23:59:33.684825+0000","last_fullsized":"2024-09-14T23:59:33.684825+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:52:25.663126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045105900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"177'718","reported_seq":5318,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.068336+0000","last_change":"2024-09-14T23:59:16.995528+0000","last_active":"2024-09-14T23:59:33.068336+0000","last_peered":"2024-09-14T23:59:33.068336+0000","last_clean":"2024-09-14T23:59:33.068336+0000","last_became_active":"2024-09-14T23:56:32.398664+0000","last_became_peered":"2024-09-14T23:56:32.398664+0000","last_unstale":"2024-09-14T23:59:33.068336+0000","last_undegraded":"2024-09-14T23:59:33.068336+0000","last_fullsized":"2024-09-14T23:59:33.068336+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":718,"log_dups_size":0,"ondisk_log_size":718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:15:18.740717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001169722,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":6635,"num_read_kb":24355,"num_write":754,"num_write_kb":5733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":363,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.684380+0000","last_change":"2024-09-14T23:56:24.534632+0000","last_active":"2024-09-14T23:59:33.684380+0000","last_peered":"2024-09-14T23:59:33.684380+0000","last_clean":"2024-09-14T23:59:33.684380+0000","last_became_active":"2024-09-14T23:56:22.526355+0000","last_became_peered":"2024-09-14T23:56:22.526355+0000","last_unstale":"2024-09-14T23:59:33.684380+0000","last_undegraded":"2024-09-14T23:59:33.684380+0000","last_fullsized":"2024-09-14T23:59:33.684380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:02:16.059627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049910000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"177'528","reported_seq":7754,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067820+0000","last_change":"2024-09-14T23:59:17.000419+0000","last_active":"2024-09-14T23:59:33.067820+0000","last_peered":"2024-09-14T23:59:33.067820+0000","last_clean":"2024-09-14T23:59:33.067820+0000","last_became_active":"2024-09-14T23:59:11.135030+0000","last_became_peered":"2024-09-14T23:59:11.135030+0000","last_unstale":"2024-09-14T23:59:33.067820+0000","last_undegraded":"2024-09-14T23:59:33.067820+0000","last_fullsized":"2024-09-14T23:59:33.067820+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":528,"log_dups_size":0,"ondisk_log_size":528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:12:06.383593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027209629999999999,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":7655,"num_read_kb":43475,"num_write":499,"num_write_kb":17299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":640,"num_bytes_recovered":36520799,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":95,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.366957+0000","last_change":"2024-09-14T23:58:49.988483+0000","last_active":"2024-09-14T23:59:33.366957+0000","last_peered":"2024-09-14T23:59:33.366957+0000","last_clean":"2024-09-14T23:59:33.366957+0000","last_became_active":"2024-09-14T23:58:49.988169+0000","last_became_peered":"2024-09-14T23:58:49.988169+0000","last_unstale":"2024-09-14T23:59:33.366957+0000","last_undegraded":"2024-09-14T23:59:33.366957+0000","last_fullsized":"2024-09-14T23:59:33.366957+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:57:59.569156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"177'622","reported_seq":5510,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.684308+0000","last_change":"2024-09-14T23:59:16.993791+0000","last_active":"2024-09-14T23:59:33.684308+0000","last_peered":"2024-09-14T23:59:33.684308+0000","last_clean":"2024-09-14T23:59:33.684308+0000","last_became_active":"2024-09-14T23:56:32.401089+0000","last_became_peered":"2024-09-14T23:56:32.401089+0000","last_unstale":"2024-09-14T23:59:33.684308+0000","last_undegraded":"2024-09-14T23:59:33.684308+0000","last_fullsized":"2024-09-14T23:59:33.684308+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:44:29.413362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094649999999999997,"stat_sum":{"num_bytes":0,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":43,"num_read":6577,"num_read_kb":26346,"num_write":596,"num_write_kb":13585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":363,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.684310+0000","last_change":"2024-09-14T23:56:24.534510+0000","last_active":"2024-09-14T23:59:33.684310+0000","last_peered":"2024-09-14T23:59:33.684310+0000","last_clean":"2024-09-14T23:59:33.684310+0000","last_became_active":"2024-09-14T23:56:22.527611+0000","last_became_peered":"2024-09-14T23:56:22.527611+0000","last_unstale":"2024-09-14T23:59:33.684310+0000","last_undegraded":"2024-09-14T23:59:33.684310+0000","last_fullsized":"2024-09-14T23:59:33.684310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:15:45.606026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049916299999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"176'729","reported_seq":7600,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067812+0000","last_change":"2024-09-14T23:59:30.062265+0000","last_active":"2024-09-14T23:59:33.067812+0000","last_peered":"2024-09-14T23:59:33.067812+0000","last_clean":"2024-09-14T23:59:33.067812+0000","last_became_active":"2024-09-14T23:59:30.060381+0000","last_became_peered":"2024-09-14T23:59:30.060381+0000","last_unstale":"2024-09-14T23:59:33.067812+0000","last_undegraded":"2024-09-14T23:59:33.067812+0000","last_fullsized":"2024-09-14T23:59:33.067812+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":729,"log_dups_size":0,"ondisk_log_size":729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:19:07.081227+0000","scrub_duration":0,"objects_trimmed":10,"snaptrim_duration":0.012412685999999999,"stat_sum":{"num_bytes":0,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":47,"num_read":9294,"num_read_kb":18810,"num_write":776,"num_write_kb":10083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":401,"num_bytes_recovered":9491176,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"177'834","reported_seq":12831,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.685176+0000","last_change":"2024-09-14T23:59:16.995637+0000","last_active":"2024-09-14T23:59:33.685176+0000","last_peered":"2024-09-14T23:59:33.685176+0000","last_clean":"2024-09-14T23:59:33.685176+0000","last_became_active":"2024-09-14T23:56:32.397883+0000","last_became_peered":"2024-09-14T23:56:32.397883+0000","last_unstale":"2024-09-14T23:59:33.685176+0000","last_undegraded":"2024-09-14T23:59:33.685176+0000","last_fullsized":"2024-09-14T23:59:33.685176+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":834,"log_dups_size":0,"ondisk_log_size":834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:51:15.895964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023599810000000001,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":38,"num_read":13370,"num_read_kb":25781,"num_write":804,"num_write_kb":35448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":87,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067672+0000","last_change":"2024-09-14T23:59:30.058288+0000","last_active":"2024-09-14T23:59:33.067672+0000","last_peered":"2024-09-14T23:59:33.067672+0000","last_clean":"2024-09-14T23:59:33.067672+0000","last_became_active":"2024-09-14T23:59:30.057866+0000","last_became_peered":"2024-09-14T23:59:30.057866+0000","last_unstale":"2024-09-14T23:59:33.067672+0000","last_undegraded":"2024-09-14T23:59:33.067672+0000","last_fullsized":"2024-09-14T23:59:33.067672+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:55:57.947601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"177'518","reported_seq":9365,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.684112+0000","last_change":"2024-09-14T23:59:16.993927+0000","last_active":"2024-09-14T23:59:33.684112+0000","last_peered":"2024-09-14T23:59:33.684112+0000","last_clean":"2024-09-14T23:59:33.684112+0000","last_became_active":"2024-09-14T23:56:32.400278+0000","last_became_peered":"2024-09-14T23:56:32.400278+0000","last_unstale":"2024-09-14T23:59:33.684112+0000","last_undegraded":"2024-09-14T23:59:33.684112+0000","last_fullsized":"2024-09-14T23:59:33.684112+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:26:19.473912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098058499999999996,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":8550,"num_read_kb":62565,"num_write":432,"num_write_kb":30360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":341,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.684325+0000","last_change":"2024-09-14T23:59:23.991858+0000","last_active":"2024-09-14T23:59:33.684325+0000","last_peered":"2024-09-14T23:59:33.684325+0000","last_clean":"2024-09-14T23:59:33.684325+0000","last_became_active":"2024-09-14T23:59:23.991492+0000","last_became_peered":"2024-09-14T23:59:23.991492+0000","last_unstale":"2024-09-14T23:59:33.684325+0000","last_undegraded":"2024-09-14T23:59:33.684325+0000","last_fullsized":"2024-09-14T23:59:33.684325+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:13:20.777068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026863400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"176'788","reported_seq":7241,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067718+0000","last_change":"2024-09-14T23:59:20.000557+0000","last_active":"2024-09-14T23:59:33.067718+0000","last_peered":"2024-09-14T23:59:33.067718+0000","last_clean":"2024-09-14T23:59:33.067718+0000","last_became_active":"2024-09-14T23:59:19.998578+0000","last_became_peered":"2024-09-14T23:59:19.998578+0000","last_unstale":"2024-09-14T23:59:33.067718+0000","last_undegraded":"2024-09-14T23:59:33.067718+0000","last_fullsized":"2024-09-14T23:59:33.067718+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":788,"log_dups_size":0,"ondisk_log_size":788,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:22:03.401033+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.00381613,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":9431,"num_read_kb":22132,"num_write":943,"num_write_kb":5118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":95,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.366890+0000","last_change":"2024-09-14T23:58:49.988483+0000","last_active":"2024-09-14T23:59:33.366890+0000","last_peered":"2024-09-14T23:59:33.366890+0000","last_clean":"2024-09-14T23:59:33.366890+0000","last_became_active":"2024-09-14T23:58:49.988096+0000","last_became_peered":"2024-09-14T23:58:49.988096+0000","last_unstale":"2024-09-14T23:59:33.366890+0000","last_undegraded":"2024-09-14T23:59:33.366890+0000","last_fullsized":"2024-09-14T23:59:33.366890+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:34:48.166954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"176'1105","reported_seq":30205,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067544+0000","last_change":"2024-09-14T23:59:16.999655+0000","last_active":"2024-09-14T23:59:33.067544+0000","last_peered":"2024-09-14T23:59:33.067544+0000","last_clean":"2024-09-14T23:59:33.067544+0000","last_became_active":"2024-09-14T23:59:12.133789+0000","last_became_peered":"2024-09-14T23:59:12.133789+0000","last_unstale":"2024-09-14T23:59:33.067544+0000","last_undegraded":"2024-09-14T23:59:33.067544+0000","last_fullsized":"2024-09-14T23:59:33.067544+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":1105,"log_dups_size":0,"ondisk_log_size":1105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:04:55.112519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002233971,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":31963,"num_read_kb":26382,"num_write":1422,"num_write_kb":17307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":641,"num_bytes_recovered":19031962,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":122,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067683+0000","last_change":"2024-09-14T23:59:30.060260+0000","last_active":"2024-09-14T23:59:33.067683+0000","last_peered":"2024-09-14T23:59:33.067683+0000","last_clean":"2024-09-14T23:59:33.067683+0000","last_became_active":"2024-09-14T23:59:30.059854+0000","last_became_peered":"2024-09-14T23:59:30.059854+0000","last_unstale":"2024-09-14T23:59:33.067683+0000","last_undegraded":"2024-09-14T23:59:33.067683+0000","last_fullsized":"2024-09-14T23:59:33.067683+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:14:35.344939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":277,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.366889+0000","last_change":"2024-09-14T23:58:53.985309+0000","last_active":"2024-09-14T23:59:33.366889+0000","last_peered":"2024-09-14T23:59:33.366889+0000","last_clean":"2024-09-14T23:59:33.366889+0000","last_became_active":"2024-09-14T23:58:53.984830+0000","last_became_peered":"2024-09-14T23:58:53.984830+0000","last_unstale":"2024-09-14T23:59:33.366889+0000","last_undegraded":"2024-09-14T23:59:33.366889+0000","last_fullsized":"2024-09-14T23:59:33.366889+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:14.322882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:14.322882+0000","last_clean_scrub_stamp":"2024-09-14T23:56:14.322882+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:37:48.336097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"177'609","reported_seq":5982,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067398+0000","last_change":"2024-09-14T23:59:17.000201+0000","last_active":"2024-09-14T23:59:33.067398+0000","last_peered":"2024-09-14T23:59:33.067398+0000","last_clean":"2024-09-14T23:59:33.067398+0000","last_became_active":"2024-09-14T23:59:12.136444+0000","last_became_peered":"2024-09-14T23:59:12.136444+0000","last_unstale":"2024-09-14T23:59:33.067398+0000","last_undegraded":"2024-09-14T23:59:33.067398+0000","last_fullsized":"2024-09-14T23:59:33.067398+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:26:06.094223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026622379999999999,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":5935,"num_read_kb":18123,"num_write":528,"num_write_kb":10919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":399,"num_bytes_recovered":9189081,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"176'343","reported_seq":11038,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.685073+0000","last_change":"2024-09-14T23:59:16.994730+0000","last_active":"2024-09-14T23:59:33.685073+0000","last_peered":"2024-09-14T23:59:33.685073+0000","last_clean":"2024-09-14T23:59:33.685073+0000","last_became_active":"2024-09-14T23:56:32.400017+0000","last_became_peered":"2024-09-14T23:56:32.400017+0000","last_unstale":"2024-09-14T23:59:33.685073+0000","last_undegraded":"2024-09-14T23:59:33.685073+0000","last_fullsized":"2024-09-14T23:59:33.685073+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:24:02.170046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015805459999999999,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":10189,"num_read_kb":13014,"num_write":297,"num_write_kb":15704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"176'834","reported_seq":16004,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.684762+0000","last_change":"2024-09-14T23:59:17.069111+0000","last_active":"2024-09-14T23:59:33.684762+0000","last_peered":"2024-09-14T23:59:33.684762+0000","last_clean":"2024-09-14T23:59:33.684762+0000","last_became_active":"2024-09-14T23:56:32.402769+0000","last_became_peered":"2024-09-14T23:56:32.402769+0000","last_unstale":"2024-09-14T23:59:33.684762+0000","last_undegraded":"2024-09-14T23:59:33.684762+0000","last_fullsized":"2024-09-14T23:59:33.684762+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":834,"log_dups_size":0,"ondisk_log_size":834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:36:02.993358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098966399999999991,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":40,"num_read":16916,"num_read_kb":32658,"num_write":1110,"num_write_kb":14962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"176'665","reported_seq":13705,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067583+0000","last_change":"2024-09-14T23:59:16.998126+0000","last_active":"2024-09-14T23:59:33.067583+0000","last_peered":"2024-09-14T23:59:33.067583+0000","last_clean":"2024-09-14T23:59:33.067583+0000","last_became_active":"2024-09-14T23:59:12.132838+0000","last_became_peered":"2024-09-14T23:59:12.132838+0000","last_unstale":"2024-09-14T23:59:33.067583+0000","last_undegraded":"2024-09-14T23:59:33.067583+0000","last_fullsized":"2024-09-14T23:59:33.067583+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":665,"log_dups_size":0,"ondisk_log_size":665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:34:25.667684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087623300000000002,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":13816,"num_read_kb":56666,"num_write":771,"num_write_kb":15952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":258,"num_bytes_recovered":8738708,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"176'705","reported_seq":16265,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067866+0000","last_change":"2024-09-14T23:59:30.062604+0000","last_active":"2024-09-14T23:59:33.067866+0000","last_peered":"2024-09-14T23:59:33.067866+0000","last_clean":"2024-09-14T23:59:33.067866+0000","last_became_active":"2024-09-14T23:59:30.060837+0000","last_became_peered":"2024-09-14T23:59:30.060837+0000","last_unstale":"2024-09-14T23:59:33.067866+0000","last_undegraded":"2024-09-14T23:59:33.067866+0000","last_fullsized":"2024-09-14T23:59:33.067866+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":705,"log_dups_size":0,"ondisk_log_size":705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:18:03.409303+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.021276283,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":15379,"num_read_kb":69494,"num_write":885,"num_write_kb":43376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":30030415,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"176'521","reported_seq":3381,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.367121+0000","last_change":"2024-09-14T23:59:17.003938+0000","last_active":"2024-09-14T23:59:33.367121+0000","last_peered":"2024-09-14T23:59:33.367121+0000","last_clean":"2024-09-14T23:59:33.367121+0000","last_became_active":"2024-09-14T23:58:52.985661+0000","last_became_peered":"2024-09-14T23:58:52.985661+0000","last_unstale":"2024-09-14T23:59:33.367121+0000","last_undegraded":"2024-09-14T23:59:33.367121+0000","last_fullsized":"2024-09-14T23:59:33.367121+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:53:00.607967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030262919999999999,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":3961,"num_read_kb":12312,"num_write":537,"num_write_kb":8315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":453,"num_bytes_recovered":19224487,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"177'532","reported_seq":5732,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.367080+0000","last_change":"2024-09-14T23:59:17.003376+0000","last_active":"2024-09-14T23:59:33.367080+0000","last_peered":"2024-09-14T23:59:33.367080+0000","last_clean":"2024-09-14T23:59:33.367080+0000","last_became_active":"2024-09-14T23:58:53.986149+0000","last_became_peered":"2024-09-14T23:58:53.986149+0000","last_unstale":"2024-09-14T23:59:33.367080+0000","last_undegraded":"2024-09-14T23:59:33.367080+0000","last_fullsized":"2024-09-14T23:59:33.367080+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:15:03.478135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024995130000000001,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":7765,"num_read_kb":21397,"num_write":463,"num_write_kb":11356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":12997809,"num_keys_recovered":45,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"176'663","reported_seq":14079,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.367080+0000","last_change":"2024-09-14T23:59:17.004025+0000","last_active":"2024-09-14T23:59:33.367080+0000","last_peered":"2024-09-14T23:59:33.367080+0000","last_clean":"2024-09-14T23:59:33.367080+0000","last_became_active":"2024-09-14T23:58:53.984808+0000","last_became_peered":"2024-09-14T23:58:53.984808+0000","last_unstale":"2024-09-14T23:59:33.367080+0000","last_undegraded":"2024-09-14T23:59:33.367080+0000","last_fullsized":"2024-09-14T23:59:33.367080+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":663,"log_dups_size":0,"ondisk_log_size":663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:48:22.749991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030575429999999998,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":14675,"num_read_kb":21661,"num_write":655,"num_write_kb":35778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":12964873,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"176'797","reported_seq":6504,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.684762+0000","last_change":"2024-09-14T23:59:17.069019+0000","last_active":"2024-09-14T23:59:33.684762+0000","last_peered":"2024-09-14T23:59:33.684762+0000","last_clean":"2024-09-14T23:59:33.684762+0000","last_became_active":"2024-09-14T23:56:32.402973+0000","last_became_peered":"2024-09-14T23:56:32.402973+0000","last_unstale":"2024-09-14T23:59:33.684762+0000","last_undegraded":"2024-09-14T23:59:33.684762+0000","last_fullsized":"2024-09-14T23:59:33.684762+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":797,"log_dups_size":0,"ondisk_log_size":797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:18:08.492160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088108100000000003,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":7036,"num_read_kb":57979,"num_write":1182,"num_write_kb":51466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"176'569","reported_seq":8379,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.366890+0000","last_change":"2024-09-14T23:59:17.001624+0000","last_active":"2024-09-14T23:59:33.366890+0000","last_peered":"2024-09-14T23:59:33.366890+0000","last_clean":"2024-09-14T23:59:33.366890+0000","last_became_active":"2024-09-14T23:58:52.987786+0000","last_became_peered":"2024-09-14T23:58:52.987786+0000","last_unstale":"2024-09-14T23:59:33.366890+0000","last_undegraded":"2024-09-14T23:59:33.366890+0000","last_fullsized":"2024-09-14T23:59:33.366890+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:55:48.806780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001145566,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":8694,"num_read_kb":36214,"num_write":569,"num_write_kb":28063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":19310765,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"176'376","reported_seq":3270,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067650+0000","last_change":"2024-09-14T23:59:20.000977+0000","last_active":"2024-09-14T23:59:33.067650+0000","last_peered":"2024-09-14T23:59:33.067650+0000","last_clean":"2024-09-14T23:59:33.067650+0000","last_became_active":"2024-09-14T23:59:19.999489+0000","last_became_peered":"2024-09-14T23:59:19.999489+0000","last_unstale":"2024-09-14T23:59:33.067650+0000","last_undegraded":"2024-09-14T23:59:33.067650+0000","last_fullsized":"2024-09-14T23:59:33.067650+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:21:32.054413+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0042424580000000002,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":3170,"num_read_kb":16380,"num_write":287,"num_write_kb":14646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":7078411,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"176'975","reported_seq":4718,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.366901+0000","last_change":"2024-09-14T23:59:17.002239+0000","last_active":"2024-09-14T23:59:33.366901+0000","last_peered":"2024-09-14T23:59:33.366901+0000","last_clean":"2024-09-14T23:59:33.366901+0000","last_became_active":"2024-09-14T23:58:53.987342+0000","last_became_peered":"2024-09-14T23:58:53.987342+0000","last_unstale":"2024-09-14T23:59:33.366901+0000","last_undegraded":"2024-09-14T23:59:33.366901+0000","last_fullsized":"2024-09-14T23:59:33.366901+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:30.333512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001659113,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":7188,"num_read_kb":25892,"num_write":1276,"num_write_kb":14588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":290,"num_bytes_recovered":14372748,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":1204,"num_object_clones":0,"num_object_copies":2408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1204,"num_whiteouts":1200,"num_read":310661,"num_read_kb":1031538,"num_write":24584,"num_write_kb":670524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7228,"num_bytes_recovered":332991677,"num_keys_recovered":596,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22207,"ondisk_log_size":22207,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":1771104,"data_compressed":100161,"data_compressed_allocated":135168,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509486,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51552,"kb_used_data":2316,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320288,"statfs":{"total":96636764160,"available":96583974912,"internally_reserved":0,"allocated":2371584,"data_stored":1713775,"data_compressed":33932,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509485,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33456,"kb_used_data":2268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338384,"statfs":{"total":96636764160,"available":96602505216,"internally_reserved":0,"allocated":2322432,"data_stored":1123407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509484,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47348,"kb_used_data":2272,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324492,"statfs":{"total":96636764160,"available":96588279808,"internally_reserved":0,"allocated":2326528,"data_stored":1123426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509486,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77876,"kb_used_data":2272,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293964,"statfs":{"total":96636764160,"available":96557019136,"internally_reserved":0,"allocated":2326528,"data_stored":1123426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509484,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55668,"kb_used_data":2268,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316172,"statfs":{"total":96636764160,"available":96579760128,"internally_reserved":0,"allocated":2322432,"data_stored":1123407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542188,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41772,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330068,"statfs":{"total":96636764160,"available":96593989632,"internally_reserved":0,"allocated":2318336,"data_stored":1117603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542187,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51576,"kb_used_data":2332,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320264,"statfs":{"total":96636764160,"available":96583950336,"internally_reserved":0,"allocated":2387968,"data_stored":1713775,"data_compressed":32297,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574892,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33456,"kb_used_data":2268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338384,"statfs":{"total":96636764160,"available":96602505216,"internally_reserved":0,"allocated":2322432,"data_stored":1123407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32297,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":33932,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":33932,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T23:59:38.639 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T23:59:38.929 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:59:38.929 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-14T23:59:38.941 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2024-09-14T23:59:37.488524+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":1208,"num_object_clones":0,"num_object_copies":2416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1208,"num_whiteouts":1200,"num_read":310777,"num_read_kb":1031638,"num_write":24745,"num_write_kb":672942,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7234,"num_bytes_recovered":334172432,"num_keys_recovered":596,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22327,"ondisk_log_size":22327,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":392704,"kb_used_data":18260,"kb_used_omap":12,"kb_used_meta":374259,"kb_avail":754582016,"statfs":{"total":773094113280,"available":772691984384,"internally_reserved":0,"allocated":18698240,"data_stored":10162226,"data_compressed":66229,"data_compressed_allocated":90112,"data_compressed_original":1163264,"omap_allocated":12752,"internal_metadata":383241776},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.434141"},"pg_stats":[{"pgid":"3.1f","version":"177'1121","reported_seq":6213,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067915+0000","last_change":"2024-09-14T23:59:16.994682+0000","last_active":"2024-09-14T23:59:33.067915+0000","last_peered":"2024-09-14T23:59:33.067915+0000","last_clean":"2024-09-14T23:59:33.067915+0000","last_became_active":"2024-09-14T23:57:00.062116+0000","last_became_peered":"2024-09-14T23:57:00.062116+0000","last_unstale":"2024-09-14T23:59:33.067915+0000","last_undegraded":"2024-09-14T23:59:33.067915+0000","last_fullsized":"2024-09-14T23:59:33.067915+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":1121,"log_dups_size":0,"ondisk_log_size":1121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:38:40.792973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094842500000000003,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":8822,"num_read_kb":37588,"num_write":1354,"num_write_kb":38053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":2327658,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"177'479","reported_seq":6679,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067687+0000","last_change":"2024-09-14T23:59:16.998886+0000","last_active":"2024-09-14T23:59:33.067687+0000","last_peered":"2024-09-14T23:59:33.067687+0000","last_clean":"2024-09-14T23:59:33.067687+0000","last_became_active":"2024-09-14T23:59:11.135384+0000","last_became_peered":"2024-09-14T23:59:11.135384+0000","last_unstale":"2024-09-14T23:59:33.067687+0000","last_undegraded":"2024-09-14T23:59:33.067687+0000","last_fullsized":"2024-09-14T23:59:33.067687+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:24:06.187016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001605626,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":5337,"num_read_kb":37535,"num_write":472,"num_write_kb":39218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":526,"num_bytes_recovered":14308962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"176'609","reported_seq":13697,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067726+0000","last_change":"2024-09-14T23:59:30.060565+0000","last_active":"2024-09-14T23:59:33.067726+0000","last_peered":"2024-09-14T23:59:33.067726+0000","last_clean":"2024-09-14T23:59:33.067726+0000","last_became_active":"2024-09-14T23:59:30.058342+0000","last_became_peered":"2024-09-14T23:59:30.058342+0000","last_unstale":"2024-09-14T23:59:33.067726+0000","last_undegraded":"2024-09-14T23:59:33.067726+0000","last_fullsized":"2024-09-14T23:59:33.067726+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:31:03.745601+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0080108929999999998,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":14616,"num_read_kb":22815,"num_write":536,"num_write_kb":24101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":1249963,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"177'995","reported_seq":6658,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067658+0000","last_change":"2024-09-14T23:59:30.060281+0000","last_active":"2024-09-14T23:59:33.067658+0000","last_peered":"2024-09-14T23:59:33.067658+0000","last_clean":"2024-09-14T23:59:33.067658+0000","last_became_active":"2024-09-14T23:59:30.058275+0000","last_became_peered":"2024-09-14T23:59:30.058275+0000","last_unstale":"2024-09-14T23:59:33.067658+0000","last_undegraded":"2024-09-14T23:59:33.067658+0000","last_fullsized":"2024-09-14T23:59:33.067658+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":995,"log_dups_size":0,"ondisk_log_size":995,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:51:47.471326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086515500000000005,"stat_sum":{"num_bytes":0,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":45,"num_read":9513,"num_read_kb":25550,"num_write":1310,"num_write_kb":9734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":2675491,"num_keys_recovered":40,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"176'524","reported_seq":2402,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.068285+0000","last_change":"2024-09-14T23:59:16.994187+0000","last_active":"2024-09-14T23:59:33.068285+0000","last_peered":"2024-09-14T23:59:33.068285+0000","last_clean":"2024-09-14T23:59:33.068285+0000","last_became_active":"2024-09-14T23:56:32.398329+0000","last_became_peered":"2024-09-14T23:56:32.398329+0000","last_unstale":"2024-09-14T23:59:33.068285+0000","last_undegraded":"2024-09-14T23:59:33.068285+0000","last_fullsized":"2024-09-14T23:59:33.068285+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":524,"log_dups_size":0,"ondisk_log_size":524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:25:07.627636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00103907,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":2076,"num_read_kb":19877,"num_write":623,"num_write_kb":10342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"177'899","reported_seq":8499,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.685085+0000","last_change":"2024-09-14T23:59:16.994978+0000","last_active":"2024-09-14T23:59:33.685085+0000","last_peered":"2024-09-14T23:59:33.685085+0000","last_clean":"2024-09-14T23:59:33.685085+0000","last_became_active":"2024-09-14T23:56:32.399655+0000","last_became_peered":"2024-09-14T23:56:32.399655+0000","last_unstale":"2024-09-14T23:59:33.685085+0000","last_undegraded":"2024-09-14T23:59:33.685085+0000","last_fullsized":"2024-09-14T23:59:33.685085+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":899,"log_dups_size":0,"ondisk_log_size":899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:01:52.825811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001807665,"stat_sum":{"num_bytes":19,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":46,"num_read":10058,"num_read_kb":53829,"num_write":970,"num_write_kb":34825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"177'695","reported_seq":4690,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.684153+0000","last_change":"2024-09-14T23:59:16.994457+0000","last_active":"2024-09-14T23:59:33.684153+0000","last_peered":"2024-09-14T23:59:33.684153+0000","last_clean":"2024-09-14T23:59:33.684153+0000","last_became_active":"2024-09-14T23:59:11.135739+0000","last_became_peered":"2024-09-14T23:59:11.135739+0000","last_unstale":"2024-09-14T23:59:33.684153+0000","last_undegraded":"2024-09-14T23:59:33.684153+0000","last_fullsized":"2024-09-14T23:59:33.684153+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:33:05.725582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099865499999999999,"stat_sum":{"num_bytes":0,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":42,"num_read":5485,"num_read_kb":20536,"num_write":717,"num_write_kb":15834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":318,"num_bytes_recovered":6472298,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"177'742","reported_seq":6564,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067404+0000","last_change":"2024-09-14T23:59:16.998152+0000","last_active":"2024-09-14T23:59:33.067404+0000","last_peered":"2024-09-14T23:59:33.067404+0000","last_clean":"2024-09-14T23:59:33.067404+0000","last_became_active":"2024-09-14T23:59:12.133258+0000","last_became_peered":"2024-09-14T23:59:12.133258+0000","last_unstale":"2024-09-14T23:59:33.067404+0000","last_undegraded":"2024-09-14T23:59:33.067404+0000","last_fullsized":"2024-09-14T23:59:33.067404+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":742,"log_dups_size":0,"ondisk_log_size":742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:43:45.205419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093947500000000005,"stat_sum":{"num_bytes":0,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":46,"num_read":6734,"num_read_kb":59364,"num_write":741,"num_write_kb":26095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":340,"num_bytes_recovered":12763266,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"176'544","reported_seq":11742,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.068027+0000","last_change":"2024-09-14T23:59:16.994889+0000","last_active":"2024-09-14T23:59:33.068027+0000","last_peered":"2024-09-14T23:59:33.068027+0000","last_clean":"2024-09-14T23:59:33.068027+0000","last_became_active":"2024-09-14T23:57:00.062304+0000","last_became_peered":"2024-09-14T23:57:00.062304+0000","last_unstale":"2024-09-14T23:59:33.068027+0000","last_undegraded":"2024-09-14T23:59:33.068027+0000","last_fullsized":"2024-09-14T23:59:33.068027+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:28:05.460487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091006300000000002,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":12421,"num_read_kb":16852,"num_write":491,"num_write_kb":6294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":3345543,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"177'463","reported_seq":7658,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067791+0000","last_change":"2024-09-14T23:59:30.062097+0000","last_active":"2024-09-14T23:59:33.067791+0000","last_peered":"2024-09-14T23:59:33.067791+0000","last_clean":"2024-09-14T23:59:33.067791+0000","last_became_active":"2024-09-14T23:59:30.060469+0000","last_became_peered":"2024-09-14T23:59:30.060469+0000","last_unstale":"2024-09-14T23:59:33.067791+0000","last_undegraded":"2024-09-14T23:59:33.067791+0000","last_fullsized":"2024-09-14T23:59:33.067791+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:38:29.716639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002288432,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":7773,"num_read_kb":27495,"num_write":382,"num_write_kb":13368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":404,"num_bytes_recovered":26159849,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"176'612","reported_seq":9857,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.367048+0000","last_change":"2024-09-14T23:59:17.002582+0000","last_active":"2024-09-14T23:59:33.367048+0000","last_peered":"2024-09-14T23:59:33.367048+0000","last_clean":"2024-09-14T23:59:33.367048+0000","last_became_active":"2024-09-14T23:59:11.135860+0000","last_became_peered":"2024-09-14T23:59:11.135860+0000","last_unstale":"2024-09-14T23:59:33.367048+0000","last_undegraded":"2024-09-14T23:59:33.367048+0000","last_fullsized":"2024-09-14T23:59:33.367048+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":612,"log_dups_size":0,"ondisk_log_size":612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:16:43.430234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001929275,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":9948,"num_read_kb":36137,"num_write":618,"num_write_kb":23694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":516,"num_bytes_recovered":42197286,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"176'1093","reported_seq":8194,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.685233+0000","last_change":"2024-09-14T23:59:16.996306+0000","last_active":"2024-09-14T23:59:33.685233+0000","last_peered":"2024-09-14T23:59:33.685233+0000","last_clean":"2024-09-14T23:59:33.685233+0000","last_became_active":"2024-09-14T23:58:53.983979+0000","last_became_peered":"2024-09-14T23:58:53.983979+0000","last_unstale":"2024-09-14T23:59:33.685233+0000","last_undegraded":"2024-09-14T23:59:33.685233+0000","last_fullsized":"2024-09-14T23:59:33.685233+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":1093,"log_dups_size":0,"ondisk_log_size":1093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:26:54.972175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002903152,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":9679,"num_read_kb":42324,"num_write":1584,"num_write_kb":28908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":312,"num_bytes_recovered":22540127,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":373,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.684825+0000","last_change":"2024-09-14T23:58:49.989598+0000","last_active":"2024-09-14T23:59:33.684825+0000","last_peered":"2024-09-14T23:59:33.684825+0000","last_clean":"2024-09-14T23:59:33.684825+0000","last_became_active":"2024-09-14T23:58:49.989224+0000","last_became_peered":"2024-09-14T23:58:49.989224+0000","last_unstale":"2024-09-14T23:59:33.684825+0000","last_undegraded":"2024-09-14T23:59:33.684825+0000","last_fullsized":"2024-09-14T23:59:33.684825+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:52:25.663126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045105900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"177'718","reported_seq":5318,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.068336+0000","last_change":"2024-09-14T23:59:16.995528+0000","last_active":"2024-09-14T23:59:33.068336+0000","last_peered":"2024-09-14T23:59:33.068336+0000","last_clean":"2024-09-14T23:59:33.068336+0000","last_became_active":"2024-09-14T23:56:32.398664+0000","last_became_peered":"2024-09-14T23:56:32.398664+0000","last_unstale":"2024-09-14T23:59:33.068336+0000","last_undegraded":"2024-09-14T23:59:33.068336+0000","last_fullsized":"2024-09-14T23:59:33.068336+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":718,"log_dups_size":0,"ondisk_log_size":718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:15:18.740717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001169722,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":6635,"num_read_kb":24355,"num_write":754,"num_write_kb":5733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":363,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.684380+0000","last_change":"2024-09-14T23:56:24.534632+0000","last_active":"2024-09-14T23:59:33.684380+0000","last_peered":"2024-09-14T23:59:33.684380+0000","last_clean":"2024-09-14T23:59:33.684380+0000","last_became_active":"2024-09-14T23:56:22.526355+0000","last_became_peered":"2024-09-14T23:56:22.526355+0000","last_unstale":"2024-09-14T23:59:33.684380+0000","last_undegraded":"2024-09-14T23:59:33.684380+0000","last_fullsized":"2024-09-14T23:59:33.684380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:02:16.059627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049910000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"177'528","reported_seq":7754,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067820+0000","last_change":"2024-09-14T23:59:17.000419+0000","last_active":"2024-09-14T23:59:33.067820+0000","last_peered":"2024-09-14T23:59:33.067820+0000","last_clean":"2024-09-14T23:59:33.067820+0000","last_became_active":"2024-09-14T23:59:11.135030+0000","last_became_peered":"2024-09-14T23:59:11.135030+0000","last_unstale":"2024-09-14T23:59:33.067820+0000","last_undegraded":"2024-09-14T23:59:33.067820+0000","last_fullsized":"2024-09-14T23:59:33.067820+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":528,"log_dups_size":0,"ondisk_log_size":528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:12:06.383593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027209629999999999,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":7655,"num_read_kb":43475,"num_write":499,"num_write_kb":17299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":640,"num_bytes_recovered":36520799,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":95,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.366957+0000","last_change":"2024-09-14T23:58:49.988483+0000","last_active":"2024-09-14T23:59:33.366957+0000","last_peered":"2024-09-14T23:59:33.366957+0000","last_clean":"2024-09-14T23:59:33.366957+0000","last_became_active":"2024-09-14T23:58:49.988169+0000","last_became_peered":"2024-09-14T23:58:49.988169+0000","last_unstale":"2024-09-14T23:59:33.366957+0000","last_undegraded":"2024-09-14T23:59:33.366957+0000","last_fullsized":"2024-09-14T23:59:33.366957+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:57:59.569156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"177'622","reported_seq":5510,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.684308+0000","last_change":"2024-09-14T23:59:16.993791+0000","last_active":"2024-09-14T23:59:33.684308+0000","last_peered":"2024-09-14T23:59:33.684308+0000","last_clean":"2024-09-14T23:59:33.684308+0000","last_became_active":"2024-09-14T23:56:32.401089+0000","last_became_peered":"2024-09-14T23:56:32.401089+0000","last_unstale":"2024-09-14T23:59:33.684308+0000","last_undegraded":"2024-09-14T23:59:33.684308+0000","last_fullsized":"2024-09-14T23:59:33.684308+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:44:29.413362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094649999999999997,"stat_sum":{"num_bytes":0,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":43,"num_read":6577,"num_read_kb":26346,"num_write":596,"num_write_kb":13585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":363,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.684310+0000","last_change":"2024-09-14T23:56:24.534510+0000","last_active":"2024-09-14T23:59:33.684310+0000","last_peered":"2024-09-14T23:59:33.684310+0000","last_clean":"2024-09-14T23:59:33.684310+0000","last_became_active":"2024-09-14T23:56:22.527611+0000","last_became_peered":"2024-09-14T23:56:22.527611+0000","last_unstale":"2024-09-14T23:59:33.684310+0000","last_undegraded":"2024-09-14T23:59:33.684310+0000","last_fullsized":"2024-09-14T23:59:33.684310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:15:45.606026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049916299999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"176'729","reported_seq":7600,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067812+0000","last_change":"2024-09-14T23:59:30.062265+0000","last_active":"2024-09-14T23:59:33.067812+0000","last_peered":"2024-09-14T23:59:33.067812+0000","last_clean":"2024-09-14T23:59:33.067812+0000","last_became_active":"2024-09-14T23:59:30.060381+0000","last_became_peered":"2024-09-14T23:59:30.060381+0000","last_unstale":"2024-09-14T23:59:33.067812+0000","last_undegraded":"2024-09-14T23:59:33.067812+0000","last_fullsized":"2024-09-14T23:59:33.067812+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":729,"log_dups_size":0,"ondisk_log_size":729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:19:07.081227+0000","scrub_duration":0,"objects_trimmed":10,"snaptrim_duration":0.012412685999999999,"stat_sum":{"num_bytes":0,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":47,"num_read":9294,"num_read_kb":18810,"num_write":776,"num_write_kb":10083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":401,"num_bytes_recovered":9491176,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"177'834","reported_seq":12831,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.685176+0000","last_change":"2024-09-14T23:59:16.995637+0000","last_active":"2024-09-14T23:59:33.685176+0000","last_peered":"2024-09-14T23:59:33.685176+0000","last_clean":"2024-09-14T23:59:33.685176+0000","last_became_active":"2024-09-14T23:56:32.397883+0000","last_became_peered":"2024-09-14T23:56:32.397883+0000","last_unstale":"2024-09-14T23:59:33.685176+0000","last_undegraded":"2024-09-14T23:59:33.685176+0000","last_fullsized":"2024-09-14T23:59:33.685176+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":834,"log_dups_size":0,"ondisk_log_size":834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:51:15.895964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023599810000000001,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":38,"num_read":13370,"num_read_kb":25781,"num_write":804,"num_write_kb":35448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":87,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067672+0000","last_change":"2024-09-14T23:59:30.058288+0000","last_active":"2024-09-14T23:59:33.067672+0000","last_peered":"2024-09-14T23:59:33.067672+0000","last_clean":"2024-09-14T23:59:33.067672+0000","last_became_active":"2024-09-14T23:59:30.057866+0000","last_became_peered":"2024-09-14T23:59:30.057866+0000","last_unstale":"2024-09-14T23:59:33.067672+0000","last_undegraded":"2024-09-14T23:59:33.067672+0000","last_fullsized":"2024-09-14T23:59:33.067672+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:55:57.947601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"177'518","reported_seq":9365,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.684112+0000","last_change":"2024-09-14T23:59:16.993927+0000","last_active":"2024-09-14T23:59:33.684112+0000","last_peered":"2024-09-14T23:59:33.684112+0000","last_clean":"2024-09-14T23:59:33.684112+0000","last_became_active":"2024-09-14T23:56:32.400278+0000","last_became_peered":"2024-09-14T23:56:32.400278+0000","last_unstale":"2024-09-14T23:59:33.684112+0000","last_undegraded":"2024-09-14T23:59:33.684112+0000","last_fullsized":"2024-09-14T23:59:33.684112+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:26:19.473912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098058499999999996,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":8550,"num_read_kb":62565,"num_write":432,"num_write_kb":30360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":341,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.684325+0000","last_change":"2024-09-14T23:59:23.991858+0000","last_active":"2024-09-14T23:59:33.684325+0000","last_peered":"2024-09-14T23:59:33.684325+0000","last_clean":"2024-09-14T23:59:33.684325+0000","last_became_active":"2024-09-14T23:59:23.991492+0000","last_became_peered":"2024-09-14T23:59:23.991492+0000","last_unstale":"2024-09-14T23:59:33.684325+0000","last_undegraded":"2024-09-14T23:59:33.684325+0000","last_fullsized":"2024-09-14T23:59:33.684325+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:13:20.777068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026863400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"176'788","reported_seq":7241,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067718+0000","last_change":"2024-09-14T23:59:20.000557+0000","last_active":"2024-09-14T23:59:33.067718+0000","last_peered":"2024-09-14T23:59:33.067718+0000","last_clean":"2024-09-14T23:59:33.067718+0000","last_became_active":"2024-09-14T23:59:19.998578+0000","last_became_peered":"2024-09-14T23:59:19.998578+0000","last_unstale":"2024-09-14T23:59:33.067718+0000","last_undegraded":"2024-09-14T23:59:33.067718+0000","last_fullsized":"2024-09-14T23:59:33.067718+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":788,"log_dups_size":0,"ondisk_log_size":788,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:22:03.401033+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.00381613,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":9431,"num_read_kb":22132,"num_write":943,"num_write_kb":5118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":95,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.366890+0000","last_change":"2024-09-14T23:58:49.988483+0000","last_active":"2024-09-14T23:59:33.366890+0000","last_peered":"2024-09-14T23:59:33.366890+0000","last_clean":"2024-09-14T23:59:33.366890+0000","last_became_active":"2024-09-14T23:58:49.988096+0000","last_became_peered":"2024-09-14T23:58:49.988096+0000","last_unstale":"2024-09-14T23:59:33.366890+0000","last_undegraded":"2024-09-14T23:59:33.366890+0000","last_fullsized":"2024-09-14T23:59:33.366890+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:34:48.166954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"176'1105","reported_seq":30205,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067544+0000","last_change":"2024-09-14T23:59:16.999655+0000","last_active":"2024-09-14T23:59:33.067544+0000","last_peered":"2024-09-14T23:59:33.067544+0000","last_clean":"2024-09-14T23:59:33.067544+0000","last_became_active":"2024-09-14T23:59:12.133789+0000","last_became_peered":"2024-09-14T23:59:12.133789+0000","last_unstale":"2024-09-14T23:59:33.067544+0000","last_undegraded":"2024-09-14T23:59:33.067544+0000","last_fullsized":"2024-09-14T23:59:33.067544+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":1105,"log_dups_size":0,"ondisk_log_size":1105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:04:55.112519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002233971,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":31963,"num_read_kb":26382,"num_write":1422,"num_write_kb":17307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":641,"num_bytes_recovered":19031962,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":122,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067683+0000","last_change":"2024-09-14T23:59:30.060260+0000","last_active":"2024-09-14T23:59:33.067683+0000","last_peered":"2024-09-14T23:59:33.067683+0000","last_clean":"2024-09-14T23:59:33.067683+0000","last_became_active":"2024-09-14T23:59:30.059854+0000","last_became_peered":"2024-09-14T23:59:30.059854+0000","last_unstale":"2024-09-14T23:59:33.067683+0000","last_undegraded":"2024-09-14T23:59:33.067683+0000","last_fullsized":"2024-09-14T23:59:33.067683+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:14:35.344939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":277,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.366889+0000","last_change":"2024-09-14T23:58:53.985309+0000","last_active":"2024-09-14T23:59:33.366889+0000","last_peered":"2024-09-14T23:59:33.366889+0000","last_clean":"2024-09-14T23:59:33.366889+0000","last_became_active":"2024-09-14T23:58:53.984830+0000","last_became_peered":"2024-09-14T23:58:53.984830+0000","last_unstale":"2024-09-14T23:59:33.366889+0000","last_undegraded":"2024-09-14T23:59:33.366889+0000","last_fullsized":"2024-09-14T23:59:33.366889+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:14.322882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:14.322882+0000","last_clean_scrub_stamp":"2024-09-14T23:56:14.322882+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:37:48.336097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"177'609","reported_seq":5982,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067398+0000","last_change":"2024-09-14T23:59:17.000201+0000","last_active":"2024-09-14T23:59:33.067398+0000","last_peered":"2024-09-14T23:59:33.067398+0000","last_clean":"2024-09-14T23:59:33.067398+0000","last_became_active":"2024-09-14T23:59:12.136444+0000","last_became_peered":"2024-09-14T23:59:12.136444+0000","last_unstale":"2024-09-14T23:59:33.067398+0000","last_undegraded":"2024-09-14T23:59:33.067398+0000","last_fullsized":"2024-09-14T23:59:33.067398+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:26:06.094223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026622379999999999,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":5935,"num_read_kb":18123,"num_write":528,"num_write_kb":10919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":399,"num_bytes_recovered":9189081,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"176'343","reported_seq":11038,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.685073+0000","last_change":"2024-09-14T23:59:16.994730+0000","last_active":"2024-09-14T23:59:33.685073+0000","last_peered":"2024-09-14T23:59:33.685073+0000","last_clean":"2024-09-14T23:59:33.685073+0000","last_became_active":"2024-09-14T23:56:32.400017+0000","last_became_peered":"2024-09-14T23:56:32.400017+0000","last_unstale":"2024-09-14T23:59:33.685073+0000","last_undegraded":"2024-09-14T23:59:33.685073+0000","last_fullsized":"2024-09-14T23:59:33.685073+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:24:02.170046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015805459999999999,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":10189,"num_read_kb":13014,"num_write":297,"num_write_kb":15704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"176'834","reported_seq":16004,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.684762+0000","last_change":"2024-09-14T23:59:17.069111+0000","last_active":"2024-09-14T23:59:33.684762+0000","last_peered":"2024-09-14T23:59:33.684762+0000","last_clean":"2024-09-14T23:59:33.684762+0000","last_became_active":"2024-09-14T23:56:32.402769+0000","last_became_peered":"2024-09-14T23:56:32.402769+0000","last_unstale":"2024-09-14T23:59:33.684762+0000","last_undegraded":"2024-09-14T23:59:33.684762+0000","last_fullsized":"2024-09-14T23:59:33.684762+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":834,"log_dups_size":0,"ondisk_log_size":834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:36:02.993358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098966399999999991,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":40,"num_read":16916,"num_read_kb":32658,"num_write":1110,"num_write_kb":14962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"176'665","reported_seq":13705,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067583+0000","last_change":"2024-09-14T23:59:16.998126+0000","last_active":"2024-09-14T23:59:33.067583+0000","last_peered":"2024-09-14T23:59:33.067583+0000","last_clean":"2024-09-14T23:59:33.067583+0000","last_became_active":"2024-09-14T23:59:12.132838+0000","last_became_peered":"2024-09-14T23:59:12.132838+0000","last_unstale":"2024-09-14T23:59:33.067583+0000","last_undegraded":"2024-09-14T23:59:33.067583+0000","last_fullsized":"2024-09-14T23:59:33.067583+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":665,"log_dups_size":0,"ondisk_log_size":665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:34:25.667684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087623300000000002,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":13816,"num_read_kb":56666,"num_write":771,"num_write_kb":15952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":258,"num_bytes_recovered":8738708,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"176'705","reported_seq":16265,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067866+0000","last_change":"2024-09-14T23:59:30.062604+0000","last_active":"2024-09-14T23:59:33.067866+0000","last_peered":"2024-09-14T23:59:33.067866+0000","last_clean":"2024-09-14T23:59:33.067866+0000","last_became_active":"2024-09-14T23:59:30.060837+0000","last_became_peered":"2024-09-14T23:59:30.060837+0000","last_unstale":"2024-09-14T23:59:33.067866+0000","last_undegraded":"2024-09-14T23:59:33.067866+0000","last_fullsized":"2024-09-14T23:59:33.067866+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":705,"log_dups_size":0,"ondisk_log_size":705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:18:03.409303+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.021276283,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":15379,"num_read_kb":69494,"num_write":885,"num_write_kb":43376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":30030415,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"176'521","reported_seq":3381,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.367121+0000","last_change":"2024-09-14T23:59:17.003938+0000","last_active":"2024-09-14T23:59:33.367121+0000","last_peered":"2024-09-14T23:59:33.367121+0000","last_clean":"2024-09-14T23:59:33.367121+0000","last_became_active":"2024-09-14T23:58:52.985661+0000","last_became_peered":"2024-09-14T23:58:52.985661+0000","last_unstale":"2024-09-14T23:59:33.367121+0000","last_undegraded":"2024-09-14T23:59:33.367121+0000","last_fullsized":"2024-09-14T23:59:33.367121+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:53:00.607967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030262919999999999,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":3961,"num_read_kb":12312,"num_write":537,"num_write_kb":8315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":453,"num_bytes_recovered":19224487,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"177'532","reported_seq":5732,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.367080+0000","last_change":"2024-09-14T23:59:17.003376+0000","last_active":"2024-09-14T23:59:33.367080+0000","last_peered":"2024-09-14T23:59:33.367080+0000","last_clean":"2024-09-14T23:59:33.367080+0000","last_became_active":"2024-09-14T23:58:53.986149+0000","last_became_peered":"2024-09-14T23:58:53.986149+0000","last_unstale":"2024-09-14T23:59:33.367080+0000","last_undegraded":"2024-09-14T23:59:33.367080+0000","last_fullsized":"2024-09-14T23:59:33.367080+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:15:03.478135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024995130000000001,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":7765,"num_read_kb":21397,"num_write":463,"num_write_kb":11356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":12997809,"num_keys_recovered":45,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"176'663","reported_seq":14079,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.367080+0000","last_change":"2024-09-14T23:59:17.004025+0000","last_active":"2024-09-14T23:59:33.367080+0000","last_peered":"2024-09-14T23:59:33.367080+0000","last_clean":"2024-09-14T23:59:33.367080+0000","last_became_active":"2024-09-14T23:58:53.984808+0000","last_became_peered":"2024-09-14T23:58:53.984808+0000","last_unstale":"2024-09-14T23:59:33.367080+0000","last_undegraded":"2024-09-14T23:59:33.367080+0000","last_fullsized":"2024-09-14T23:59:33.367080+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":663,"log_dups_size":0,"ondisk_log_size":663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:48:22.749991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030575429999999998,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":14675,"num_read_kb":21661,"num_write":655,"num_write_kb":35778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":12964873,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"176'797","reported_seq":6504,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.684762+0000","last_change":"2024-09-14T23:59:17.069019+0000","last_active":"2024-09-14T23:59:33.684762+0000","last_peered":"2024-09-14T23:59:33.684762+0000","last_clean":"2024-09-14T23:59:33.684762+0000","last_became_active":"2024-09-14T23:56:32.402973+0000","last_became_peered":"2024-09-14T23:56:32.402973+0000","last_unstale":"2024-09-14T23:59:33.684762+0000","last_undegraded":"2024-09-14T23:59:33.684762+0000","last_fullsized":"2024-09-14T23:59:33.684762+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":797,"log_dups_size":0,"ondisk_log_size":797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:18:08.492160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088108100000000003,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":7036,"num_read_kb":57979,"num_write":1182,"num_write_kb":51466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"176'569","reported_seq":8379,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.366890+0000","last_change":"2024-09-14T23:59:17.001624+0000","last_active":"2024-09-14T23:59:33.366890+0000","last_peered":"2024-09-14T23:59:33.366890+0000","last_clean":"2024-09-14T23:59:33.366890+0000","last_became_active":"2024-09-14T23:58:52.987786+0000","last_became_peered":"2024-09-14T23:58:52.987786+0000","last_unstale":"2024-09-14T23:59:33.366890+0000","last_undegraded":"2024-09-14T23:59:33.366890+0000","last_fullsized":"2024-09-14T23:59:33.366890+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:55:48.806780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001145566,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":8694,"num_read_kb":36214,"num_write":569,"num_write_kb":28063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":19310765,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"176'376","reported_seq":3270,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067650+0000","last_change":"2024-09-14T23:59:20.000977+0000","last_active":"2024-09-14T23:59:33.067650+0000","last_peered":"2024-09-14T23:59:33.067650+0000","last_clean":"2024-09-14T23:59:33.067650+0000","last_became_active":"2024-09-14T23:59:19.999489+0000","last_became_peered":"2024-09-14T23:59:19.999489+0000","last_unstale":"2024-09-14T23:59:33.067650+0000","last_undegraded":"2024-09-14T23:59:33.067650+0000","last_fullsized":"2024-09-14T23:59:33.067650+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:21:32.054413+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0042424580000000002,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":3170,"num_read_kb":16380,"num_write":287,"num_write_kb":14646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":7078411,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"176'975","reported_seq":4718,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.366901+0000","last_change":"2024-09-14T23:59:17.002239+0000","last_active":"2024-09-14T23:59:33.366901+0000","last_peered":"2024-09-14T23:59:33.366901+0000","last_clean":"2024-09-14T23:59:33.366901+0000","last_became_active":"2024-09-14T23:58:53.987342+0000","last_became_peered":"2024-09-14T23:58:53.987342+0000","last_unstale":"2024-09-14T23:59:33.366901+0000","last_undegraded":"2024-09-14T23:59:33.366901+0000","last_fullsized":"2024-09-14T23:59:33.366901+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:30.333512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001659113,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":7188,"num_read_kb":25892,"num_write":1276,"num_write_kb":14588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":290,"num_bytes_recovered":14372748,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":1204,"num_object_clones":0,"num_object_copies":2408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1204,"num_whiteouts":1200,"num_read":310661,"num_read_kb":1031538,"num_write":24584,"num_write_kb":670524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7228,"num_bytes_recovered":332991677,"num_keys_recovered":596,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22207,"ondisk_log_size":22207,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":1771104,"data_compressed":100161,"data_compressed_allocated":135168,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509486,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51552,"kb_used_data":2316,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320288,"statfs":{"total":96636764160,"available":96583974912,"internally_reserved":0,"allocated":2371584,"data_stored":1713775,"data_compressed":33932,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509485,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33456,"kb_used_data":2268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338384,"statfs":{"total":96636764160,"available":96602505216,"internally_reserved":0,"allocated":2322432,"data_stored":1123407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509484,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47348,"kb_used_data":2272,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324492,"statfs":{"total":96636764160,"available":96588279808,"internally_reserved":0,"allocated":2326528,"data_stored":1123426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509486,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77876,"kb_used_data":2272,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293964,"statfs":{"total":96636764160,"available":96557019136,"internally_reserved":0,"allocated":2326528,"data_stored":1123426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509484,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55668,"kb_used_data":2268,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316172,"statfs":{"total":96636764160,"available":96579760128,"internally_reserved":0,"allocated":2322432,"data_stored":1123407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542188,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41772,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330068,"statfs":{"total":96636764160,"available":96593989632,"internally_reserved":0,"allocated":2318336,"data_stored":1117603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542187,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51576,"kb_used_data":2332,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320264,"statfs":{"total":96636764160,"available":96583950336,"internally_reserved":0,"allocated":2387968,"data_stored":1713775,"data_compressed":32297,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574892,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33456,"kb_used_data":2268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338384,"statfs":{"total":96636764160,"available":96602505216,"internally_reserved":0,"allocated":2322432,"data_stored":1123407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32297,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":33932,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":33932,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T23:59:38.943 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-14T23:59:38.943 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T23:59:39.235 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:59:39.236 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-14T23:59:39.247 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2024-09-14T23:59:37.488524+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":1208,"num_object_clones":0,"num_object_copies":2416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1208,"num_whiteouts":1200,"num_read":310777,"num_read_kb":1031638,"num_write":24745,"num_write_kb":672942,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7234,"num_bytes_recovered":334172432,"num_keys_recovered":596,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22327,"ondisk_log_size":22327,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":392704,"kb_used_data":18260,"kb_used_omap":12,"kb_used_meta":374259,"kb_avail":754582016,"statfs":{"total":773094113280,"available":772691984384,"internally_reserved":0,"allocated":18698240,"data_stored":10162226,"data_compressed":66229,"data_compressed_allocated":90112,"data_compressed_original":1163264,"omap_allocated":12752,"internal_metadata":383241776},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.434141"},"pg_stats":[{"pgid":"3.1f","version":"177'1121","reported_seq":6213,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067915+0000","last_change":"2024-09-14T23:59:16.994682+0000","last_active":"2024-09-14T23:59:33.067915+0000","last_peered":"2024-09-14T23:59:33.067915+0000","last_clean":"2024-09-14T23:59:33.067915+0000","last_became_active":"2024-09-14T23:57:00.062116+0000","last_became_peered":"2024-09-14T23:57:00.062116+0000","last_unstale":"2024-09-14T23:59:33.067915+0000","last_undegraded":"2024-09-14T23:59:33.067915+0000","last_fullsized":"2024-09-14T23:59:33.067915+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":1121,"log_dups_size":0,"ondisk_log_size":1121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:38:40.792973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094842500000000003,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":8822,"num_read_kb":37588,"num_write":1354,"num_write_kb":38053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":2327658,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"177'479","reported_seq":6679,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067687+0000","last_change":"2024-09-14T23:59:16.998886+0000","last_active":"2024-09-14T23:59:33.067687+0000","last_peered":"2024-09-14T23:59:33.067687+0000","last_clean":"2024-09-14T23:59:33.067687+0000","last_became_active":"2024-09-14T23:59:11.135384+0000","last_became_peered":"2024-09-14T23:59:11.135384+0000","last_unstale":"2024-09-14T23:59:33.067687+0000","last_undegraded":"2024-09-14T23:59:33.067687+0000","last_fullsized":"2024-09-14T23:59:33.067687+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:24:06.187016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001605626,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":5337,"num_read_kb":37535,"num_write":472,"num_write_kb":39218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":526,"num_bytes_recovered":14308962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"176'609","reported_seq":13697,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067726+0000","last_change":"2024-09-14T23:59:30.060565+0000","last_active":"2024-09-14T23:59:33.067726+0000","last_peered":"2024-09-14T23:59:33.067726+0000","last_clean":"2024-09-14T23:59:33.067726+0000","last_became_active":"2024-09-14T23:59:30.058342+0000","last_became_peered":"2024-09-14T23:59:30.058342+0000","last_unstale":"2024-09-14T23:59:33.067726+0000","last_undegraded":"2024-09-14T23:59:33.067726+0000","last_fullsized":"2024-09-14T23:59:33.067726+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:31:03.745601+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0080108929999999998,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":14616,"num_read_kb":22815,"num_write":536,"num_write_kb":24101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":1249963,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"177'995","reported_seq":6658,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067658+0000","last_change":"2024-09-14T23:59:30.060281+0000","last_active":"2024-09-14T23:59:33.067658+0000","last_peered":"2024-09-14T23:59:33.067658+0000","last_clean":"2024-09-14T23:59:33.067658+0000","last_became_active":"2024-09-14T23:59:30.058275+0000","last_became_peered":"2024-09-14T23:59:30.058275+0000","last_unstale":"2024-09-14T23:59:33.067658+0000","last_undegraded":"2024-09-14T23:59:33.067658+0000","last_fullsized":"2024-09-14T23:59:33.067658+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":995,"log_dups_size":0,"ondisk_log_size":995,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:51:47.471326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086515500000000005,"stat_sum":{"num_bytes":0,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":45,"num_read":9513,"num_read_kb":25550,"num_write":1310,"num_write_kb":9734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":2675491,"num_keys_recovered":40,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"176'524","reported_seq":2402,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.068285+0000","last_change":"2024-09-14T23:59:16.994187+0000","last_active":"2024-09-14T23:59:33.068285+0000","last_peered":"2024-09-14T23:59:33.068285+0000","last_clean":"2024-09-14T23:59:33.068285+0000","last_became_active":"2024-09-14T23:56:32.398329+0000","last_became_peered":"2024-09-14T23:56:32.398329+0000","last_unstale":"2024-09-14T23:59:33.068285+0000","last_undegraded":"2024-09-14T23:59:33.068285+0000","last_fullsized":"2024-09-14T23:59:33.068285+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":524,"log_dups_size":0,"ondisk_log_size":524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:25:07.627636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00103907,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":2076,"num_read_kb":19877,"num_write":623,"num_write_kb":10342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"177'899","reported_seq":8499,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.685085+0000","last_change":"2024-09-14T23:59:16.994978+0000","last_active":"2024-09-14T23:59:33.685085+0000","last_peered":"2024-09-14T23:59:33.685085+0000","last_clean":"2024-09-14T23:59:33.685085+0000","last_became_active":"2024-09-14T23:56:32.399655+0000","last_became_peered":"2024-09-14T23:56:32.399655+0000","last_unstale":"2024-09-14T23:59:33.685085+0000","last_undegraded":"2024-09-14T23:59:33.685085+0000","last_fullsized":"2024-09-14T23:59:33.685085+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":899,"log_dups_size":0,"ondisk_log_size":899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:01:52.825811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001807665,"stat_sum":{"num_bytes":19,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":46,"num_read":10058,"num_read_kb":53829,"num_write":970,"num_write_kb":34825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"177'695","reported_seq":4690,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.684153+0000","last_change":"2024-09-14T23:59:16.994457+0000","last_active":"2024-09-14T23:59:33.684153+0000","last_peered":"2024-09-14T23:59:33.684153+0000","last_clean":"2024-09-14T23:59:33.684153+0000","last_became_active":"2024-09-14T23:59:11.135739+0000","last_became_peered":"2024-09-14T23:59:11.135739+0000","last_unstale":"2024-09-14T23:59:33.684153+0000","last_undegraded":"2024-09-14T23:59:33.684153+0000","last_fullsized":"2024-09-14T23:59:33.684153+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:33:05.725582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099865499999999999,"stat_sum":{"num_bytes":0,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":42,"num_read":5485,"num_read_kb":20536,"num_write":717,"num_write_kb":15834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":318,"num_bytes_recovered":6472298,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"177'742","reported_seq":6564,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067404+0000","last_change":"2024-09-14T23:59:16.998152+0000","last_active":"2024-09-14T23:59:33.067404+0000","last_peered":"2024-09-14T23:59:33.067404+0000","last_clean":"2024-09-14T23:59:33.067404+0000","last_became_active":"2024-09-14T23:59:12.133258+0000","last_became_peered":"2024-09-14T23:59:12.133258+0000","last_unstale":"2024-09-14T23:59:33.067404+0000","last_undegraded":"2024-09-14T23:59:33.067404+0000","last_fullsized":"2024-09-14T23:59:33.067404+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":742,"log_dups_size":0,"ondisk_log_size":742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:43:45.205419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093947500000000005,"stat_sum":{"num_bytes":0,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":46,"num_read":6734,"num_read_kb":59364,"num_write":741,"num_write_kb":26095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":340,"num_bytes_recovered":12763266,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"176'544","reported_seq":11742,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.068027+0000","last_change":"2024-09-14T23:59:16.994889+0000","last_active":"2024-09-14T23:59:33.068027+0000","last_peered":"2024-09-14T23:59:33.068027+0000","last_clean":"2024-09-14T23:59:33.068027+0000","last_became_active":"2024-09-14T23:57:00.062304+0000","last_became_peered":"2024-09-14T23:57:00.062304+0000","last_unstale":"2024-09-14T23:59:33.068027+0000","last_undegraded":"2024-09-14T23:59:33.068027+0000","last_fullsized":"2024-09-14T23:59:33.068027+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:28:05.460487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091006300000000002,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":12421,"num_read_kb":16852,"num_write":491,"num_write_kb":6294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":3345543,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"177'463","reported_seq":7658,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067791+0000","last_change":"2024-09-14T23:59:30.062097+0000","last_active":"2024-09-14T23:59:33.067791+0000","last_peered":"2024-09-14T23:59:33.067791+0000","last_clean":"2024-09-14T23:59:33.067791+0000","last_became_active":"2024-09-14T23:59:30.060469+0000","last_became_peered":"2024-09-14T23:59:30.060469+0000","last_unstale":"2024-09-14T23:59:33.067791+0000","last_undegraded":"2024-09-14T23:59:33.067791+0000","last_fullsized":"2024-09-14T23:59:33.067791+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:38:29.716639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002288432,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":7773,"num_read_kb":27495,"num_write":382,"num_write_kb":13368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":404,"num_bytes_recovered":26159849,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"176'612","reported_seq":9857,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.367048+0000","last_change":"2024-09-14T23:59:17.002582+0000","last_active":"2024-09-14T23:59:33.367048+0000","last_peered":"2024-09-14T23:59:33.367048+0000","last_clean":"2024-09-14T23:59:33.367048+0000","last_became_active":"2024-09-14T23:59:11.135860+0000","last_became_peered":"2024-09-14T23:59:11.135860+0000","last_unstale":"2024-09-14T23:59:33.367048+0000","last_undegraded":"2024-09-14T23:59:33.367048+0000","last_fullsized":"2024-09-14T23:59:33.367048+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":612,"log_dups_size":0,"ondisk_log_size":612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:16:43.430234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001929275,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":9948,"num_read_kb":36137,"num_write":618,"num_write_kb":23694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":516,"num_bytes_recovered":42197286,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"176'1093","reported_seq":8194,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.685233+0000","last_change":"2024-09-14T23:59:16.996306+0000","last_active":"2024-09-14T23:59:33.685233+0000","last_peered":"2024-09-14T23:59:33.685233+0000","last_clean":"2024-09-14T23:59:33.685233+0000","last_became_active":"2024-09-14T23:58:53.983979+0000","last_became_peered":"2024-09-14T23:58:53.983979+0000","last_unstale":"2024-09-14T23:59:33.685233+0000","last_undegraded":"2024-09-14T23:59:33.685233+0000","last_fullsized":"2024-09-14T23:59:33.685233+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":1093,"log_dups_size":0,"ondisk_log_size":1093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:26:54.972175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002903152,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":9679,"num_read_kb":42324,"num_write":1584,"num_write_kb":28908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":312,"num_bytes_recovered":22540127,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":373,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.684825+0000","last_change":"2024-09-14T23:58:49.989598+0000","last_active":"2024-09-14T23:59:33.684825+0000","last_peered":"2024-09-14T23:59:33.684825+0000","last_clean":"2024-09-14T23:59:33.684825+0000","last_became_active":"2024-09-14T23:58:49.989224+0000","last_became_peered":"2024-09-14T23:58:49.989224+0000","last_unstale":"2024-09-14T23:59:33.684825+0000","last_undegraded":"2024-09-14T23:59:33.684825+0000","last_fullsized":"2024-09-14T23:59:33.684825+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:52:25.663126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045105900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"177'718","reported_seq":5318,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.068336+0000","last_change":"2024-09-14T23:59:16.995528+0000","last_active":"2024-09-14T23:59:33.068336+0000","last_peered":"2024-09-14T23:59:33.068336+0000","last_clean":"2024-09-14T23:59:33.068336+0000","last_became_active":"2024-09-14T23:56:32.398664+0000","last_became_peered":"2024-09-14T23:56:32.398664+0000","last_unstale":"2024-09-14T23:59:33.068336+0000","last_undegraded":"2024-09-14T23:59:33.068336+0000","last_fullsized":"2024-09-14T23:59:33.068336+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":718,"log_dups_size":0,"ondisk_log_size":718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:15:18.740717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001169722,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":6635,"num_read_kb":24355,"num_write":754,"num_write_kb":5733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":363,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.684380+0000","last_change":"2024-09-14T23:56:24.534632+0000","last_active":"2024-09-14T23:59:33.684380+0000","last_peered":"2024-09-14T23:59:33.684380+0000","last_clean":"2024-09-14T23:59:33.684380+0000","last_became_active":"2024-09-14T23:56:22.526355+0000","last_became_peered":"2024-09-14T23:56:22.526355+0000","last_unstale":"2024-09-14T23:59:33.684380+0000","last_undegraded":"2024-09-14T23:59:33.684380+0000","last_fullsized":"2024-09-14T23:59:33.684380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:02:16.059627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049910000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"177'528","reported_seq":7754,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067820+0000","last_change":"2024-09-14T23:59:17.000419+0000","last_active":"2024-09-14T23:59:33.067820+0000","last_peered":"2024-09-14T23:59:33.067820+0000","last_clean":"2024-09-14T23:59:33.067820+0000","last_became_active":"2024-09-14T23:59:11.135030+0000","last_became_peered":"2024-09-14T23:59:11.135030+0000","last_unstale":"2024-09-14T23:59:33.067820+0000","last_undegraded":"2024-09-14T23:59:33.067820+0000","last_fullsized":"2024-09-14T23:59:33.067820+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":528,"log_dups_size":0,"ondisk_log_size":528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:12:06.383593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027209629999999999,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":7655,"num_read_kb":43475,"num_write":499,"num_write_kb":17299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":640,"num_bytes_recovered":36520799,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":95,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.366957+0000","last_change":"2024-09-14T23:58:49.988483+0000","last_active":"2024-09-14T23:59:33.366957+0000","last_peered":"2024-09-14T23:59:33.366957+0000","last_clean":"2024-09-14T23:59:33.366957+0000","last_became_active":"2024-09-14T23:58:49.988169+0000","last_became_peered":"2024-09-14T23:58:49.988169+0000","last_unstale":"2024-09-14T23:59:33.366957+0000","last_undegraded":"2024-09-14T23:59:33.366957+0000","last_fullsized":"2024-09-14T23:59:33.366957+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:57:59.569156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"177'622","reported_seq":5510,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.684308+0000","last_change":"2024-09-14T23:59:16.993791+0000","last_active":"2024-09-14T23:59:33.684308+0000","last_peered":"2024-09-14T23:59:33.684308+0000","last_clean":"2024-09-14T23:59:33.684308+0000","last_became_active":"2024-09-14T23:56:32.401089+0000","last_became_peered":"2024-09-14T23:56:32.401089+0000","last_unstale":"2024-09-14T23:59:33.684308+0000","last_undegraded":"2024-09-14T23:59:33.684308+0000","last_fullsized":"2024-09-14T23:59:33.684308+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:44:29.413362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094649999999999997,"stat_sum":{"num_bytes":0,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":43,"num_read":6577,"num_read_kb":26346,"num_write":596,"num_write_kb":13585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":363,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.684310+0000","last_change":"2024-09-14T23:56:24.534510+0000","last_active":"2024-09-14T23:59:33.684310+0000","last_peered":"2024-09-14T23:59:33.684310+0000","last_clean":"2024-09-14T23:59:33.684310+0000","last_became_active":"2024-09-14T23:56:22.527611+0000","last_became_peered":"2024-09-14T23:56:22.527611+0000","last_unstale":"2024-09-14T23:59:33.684310+0000","last_undegraded":"2024-09-14T23:59:33.684310+0000","last_fullsized":"2024-09-14T23:59:33.684310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:15:45.606026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049916299999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"176'729","reported_seq":7600,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067812+0000","last_change":"2024-09-14T23:59:30.062265+0000","last_active":"2024-09-14T23:59:33.067812+0000","last_peered":"2024-09-14T23:59:33.067812+0000","last_clean":"2024-09-14T23:59:33.067812+0000","last_became_active":"2024-09-14T23:59:30.060381+0000","last_became_peered":"2024-09-14T23:59:30.060381+0000","last_unstale":"2024-09-14T23:59:33.067812+0000","last_undegraded":"2024-09-14T23:59:33.067812+0000","last_fullsized":"2024-09-14T23:59:33.067812+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":729,"log_dups_size":0,"ondisk_log_size":729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:19:07.081227+0000","scrub_duration":0,"objects_trimmed":10,"snaptrim_duration":0.012412685999999999,"stat_sum":{"num_bytes":0,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":47,"num_read":9294,"num_read_kb":18810,"num_write":776,"num_write_kb":10083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":401,"num_bytes_recovered":9491176,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"177'834","reported_seq":12831,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.685176+0000","last_change":"2024-09-14T23:59:16.995637+0000","last_active":"2024-09-14T23:59:33.685176+0000","last_peered":"2024-09-14T23:59:33.685176+0000","last_clean":"2024-09-14T23:59:33.685176+0000","last_became_active":"2024-09-14T23:56:32.397883+0000","last_became_peered":"2024-09-14T23:56:32.397883+0000","last_unstale":"2024-09-14T23:59:33.685176+0000","last_undegraded":"2024-09-14T23:59:33.685176+0000","last_fullsized":"2024-09-14T23:59:33.685176+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":834,"log_dups_size":0,"ondisk_log_size":834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:51:15.895964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023599810000000001,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":38,"num_read":13370,"num_read_kb":25781,"num_write":804,"num_write_kb":35448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":87,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067672+0000","last_change":"2024-09-14T23:59:30.058288+0000","last_active":"2024-09-14T23:59:33.067672+0000","last_peered":"2024-09-14T23:59:33.067672+0000","last_clean":"2024-09-14T23:59:33.067672+0000","last_became_active":"2024-09-14T23:59:30.057866+0000","last_became_peered":"2024-09-14T23:59:30.057866+0000","last_unstale":"2024-09-14T23:59:33.067672+0000","last_undegraded":"2024-09-14T23:59:33.067672+0000","last_fullsized":"2024-09-14T23:59:33.067672+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:55:57.947601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"177'518","reported_seq":9365,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.684112+0000","last_change":"2024-09-14T23:59:16.993927+0000","last_active":"2024-09-14T23:59:33.684112+0000","last_peered":"2024-09-14T23:59:33.684112+0000","last_clean":"2024-09-14T23:59:33.684112+0000","last_became_active":"2024-09-14T23:56:32.400278+0000","last_became_peered":"2024-09-14T23:56:32.400278+0000","last_unstale":"2024-09-14T23:59:33.684112+0000","last_undegraded":"2024-09-14T23:59:33.684112+0000","last_fullsized":"2024-09-14T23:59:33.684112+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:26:19.473912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098058499999999996,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":8550,"num_read_kb":62565,"num_write":432,"num_write_kb":30360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":341,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.684325+0000","last_change":"2024-09-14T23:59:23.991858+0000","last_active":"2024-09-14T23:59:33.684325+0000","last_peered":"2024-09-14T23:59:33.684325+0000","last_clean":"2024-09-14T23:59:33.684325+0000","last_became_active":"2024-09-14T23:59:23.991492+0000","last_became_peered":"2024-09-14T23:59:23.991492+0000","last_unstale":"2024-09-14T23:59:33.684325+0000","last_undegraded":"2024-09-14T23:59:33.684325+0000","last_fullsized":"2024-09-14T23:59:33.684325+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:13:20.777068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026863400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"176'788","reported_seq":7241,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067718+0000","last_change":"2024-09-14T23:59:20.000557+0000","last_active":"2024-09-14T23:59:33.067718+0000","last_peered":"2024-09-14T23:59:33.067718+0000","last_clean":"2024-09-14T23:59:33.067718+0000","last_became_active":"2024-09-14T23:59:19.998578+0000","last_became_peered":"2024-09-14T23:59:19.998578+0000","last_unstale":"2024-09-14T23:59:33.067718+0000","last_undegraded":"2024-09-14T23:59:33.067718+0000","last_fullsized":"2024-09-14T23:59:33.067718+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":788,"log_dups_size":0,"ondisk_log_size":788,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:22:03.401033+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.00381613,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":9431,"num_read_kb":22132,"num_write":943,"num_write_kb":5118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":95,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.366890+0000","last_change":"2024-09-14T23:58:49.988483+0000","last_active":"2024-09-14T23:59:33.366890+0000","last_peered":"2024-09-14T23:59:33.366890+0000","last_clean":"2024-09-14T23:59:33.366890+0000","last_became_active":"2024-09-14T23:58:49.988096+0000","last_became_peered":"2024-09-14T23:58:49.988096+0000","last_unstale":"2024-09-14T23:59:33.366890+0000","last_undegraded":"2024-09-14T23:59:33.366890+0000","last_fullsized":"2024-09-14T23:59:33.366890+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:34:48.166954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"176'1105","reported_seq":30205,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067544+0000","last_change":"2024-09-14T23:59:16.999655+0000","last_active":"2024-09-14T23:59:33.067544+0000","last_peered":"2024-09-14T23:59:33.067544+0000","last_clean":"2024-09-14T23:59:33.067544+0000","last_became_active":"2024-09-14T23:59:12.133789+0000","last_became_peered":"2024-09-14T23:59:12.133789+0000","last_unstale":"2024-09-14T23:59:33.067544+0000","last_undegraded":"2024-09-14T23:59:33.067544+0000","last_fullsized":"2024-09-14T23:59:33.067544+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":1105,"log_dups_size":0,"ondisk_log_size":1105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:04:55.112519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002233971,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":31963,"num_read_kb":26382,"num_write":1422,"num_write_kb":17307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":641,"num_bytes_recovered":19031962,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":122,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067683+0000","last_change":"2024-09-14T23:59:30.060260+0000","last_active":"2024-09-14T23:59:33.067683+0000","last_peered":"2024-09-14T23:59:33.067683+0000","last_clean":"2024-09-14T23:59:33.067683+0000","last_became_active":"2024-09-14T23:59:30.059854+0000","last_became_peered":"2024-09-14T23:59:30.059854+0000","last_unstale":"2024-09-14T23:59:33.067683+0000","last_undegraded":"2024-09-14T23:59:33.067683+0000","last_fullsized":"2024-09-14T23:59:33.067683+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:14:35.344939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":277,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.366889+0000","last_change":"2024-09-14T23:58:53.985309+0000","last_active":"2024-09-14T23:59:33.366889+0000","last_peered":"2024-09-14T23:59:33.366889+0000","last_clean":"2024-09-14T23:59:33.366889+0000","last_became_active":"2024-09-14T23:58:53.984830+0000","last_became_peered":"2024-09-14T23:58:53.984830+0000","last_unstale":"2024-09-14T23:59:33.366889+0000","last_undegraded":"2024-09-14T23:59:33.366889+0000","last_fullsized":"2024-09-14T23:59:33.366889+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:14.322882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:14.322882+0000","last_clean_scrub_stamp":"2024-09-14T23:56:14.322882+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:37:48.336097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"177'609","reported_seq":5982,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067398+0000","last_change":"2024-09-14T23:59:17.000201+0000","last_active":"2024-09-14T23:59:33.067398+0000","last_peered":"2024-09-14T23:59:33.067398+0000","last_clean":"2024-09-14T23:59:33.067398+0000","last_became_active":"2024-09-14T23:59:12.136444+0000","last_became_peered":"2024-09-14T23:59:12.136444+0000","last_unstale":"2024-09-14T23:59:33.067398+0000","last_undegraded":"2024-09-14T23:59:33.067398+0000","last_fullsized":"2024-09-14T23:59:33.067398+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:26:06.094223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026622379999999999,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":5935,"num_read_kb":18123,"num_write":528,"num_write_kb":10919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":399,"num_bytes_recovered":9189081,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"176'343","reported_seq":11038,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.685073+0000","last_change":"2024-09-14T23:59:16.994730+0000","last_active":"2024-09-14T23:59:33.685073+0000","last_peered":"2024-09-14T23:59:33.685073+0000","last_clean":"2024-09-14T23:59:33.685073+0000","last_became_active":"2024-09-14T23:56:32.400017+0000","last_became_peered":"2024-09-14T23:56:32.400017+0000","last_unstale":"2024-09-14T23:59:33.685073+0000","last_undegraded":"2024-09-14T23:59:33.685073+0000","last_fullsized":"2024-09-14T23:59:33.685073+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:24:02.170046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015805459999999999,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":10189,"num_read_kb":13014,"num_write":297,"num_write_kb":15704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"176'834","reported_seq":16004,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.684762+0000","last_change":"2024-09-14T23:59:17.069111+0000","last_active":"2024-09-14T23:59:33.684762+0000","last_peered":"2024-09-14T23:59:33.684762+0000","last_clean":"2024-09-14T23:59:33.684762+0000","last_became_active":"2024-09-14T23:56:32.402769+0000","last_became_peered":"2024-09-14T23:56:32.402769+0000","last_unstale":"2024-09-14T23:59:33.684762+0000","last_undegraded":"2024-09-14T23:59:33.684762+0000","last_fullsized":"2024-09-14T23:59:33.684762+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":834,"log_dups_size":0,"ondisk_log_size":834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:36:02.993358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098966399999999991,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":40,"num_read":16916,"num_read_kb":32658,"num_write":1110,"num_write_kb":14962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"176'665","reported_seq":13705,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067583+0000","last_change":"2024-09-14T23:59:16.998126+0000","last_active":"2024-09-14T23:59:33.067583+0000","last_peered":"2024-09-14T23:59:33.067583+0000","last_clean":"2024-09-14T23:59:33.067583+0000","last_became_active":"2024-09-14T23:59:12.132838+0000","last_became_peered":"2024-09-14T23:59:12.132838+0000","last_unstale":"2024-09-14T23:59:33.067583+0000","last_undegraded":"2024-09-14T23:59:33.067583+0000","last_fullsized":"2024-09-14T23:59:33.067583+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":665,"log_dups_size":0,"ondisk_log_size":665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:34:25.667684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087623300000000002,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":13816,"num_read_kb":56666,"num_write":771,"num_write_kb":15952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":258,"num_bytes_recovered":8738708,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"176'705","reported_seq":16265,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067866+0000","last_change":"2024-09-14T23:59:30.062604+0000","last_active":"2024-09-14T23:59:33.067866+0000","last_peered":"2024-09-14T23:59:33.067866+0000","last_clean":"2024-09-14T23:59:33.067866+0000","last_became_active":"2024-09-14T23:59:30.060837+0000","last_became_peered":"2024-09-14T23:59:30.060837+0000","last_unstale":"2024-09-14T23:59:33.067866+0000","last_undegraded":"2024-09-14T23:59:33.067866+0000","last_fullsized":"2024-09-14T23:59:33.067866+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":705,"log_dups_size":0,"ondisk_log_size":705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:18:03.409303+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.021276283,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":15379,"num_read_kb":69494,"num_write":885,"num_write_kb":43376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":30030415,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"176'521","reported_seq":3381,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.367121+0000","last_change":"2024-09-14T23:59:17.003938+0000","last_active":"2024-09-14T23:59:33.367121+0000","last_peered":"2024-09-14T23:59:33.367121+0000","last_clean":"2024-09-14T23:59:33.367121+0000","last_became_active":"2024-09-14T23:58:52.985661+0000","last_became_peered":"2024-09-14T23:58:52.985661+0000","last_unstale":"2024-09-14T23:59:33.367121+0000","last_undegraded":"2024-09-14T23:59:33.367121+0000","last_fullsized":"2024-09-14T23:59:33.367121+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:53:00.607967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030262919999999999,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":3961,"num_read_kb":12312,"num_write":537,"num_write_kb":8315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":453,"num_bytes_recovered":19224487,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"177'532","reported_seq":5732,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.367080+0000","last_change":"2024-09-14T23:59:17.003376+0000","last_active":"2024-09-14T23:59:33.367080+0000","last_peered":"2024-09-14T23:59:33.367080+0000","last_clean":"2024-09-14T23:59:33.367080+0000","last_became_active":"2024-09-14T23:58:53.986149+0000","last_became_peered":"2024-09-14T23:58:53.986149+0000","last_unstale":"2024-09-14T23:59:33.367080+0000","last_undegraded":"2024-09-14T23:59:33.367080+0000","last_fullsized":"2024-09-14T23:59:33.367080+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:15:03.478135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024995130000000001,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":7765,"num_read_kb":21397,"num_write":463,"num_write_kb":11356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":12997809,"num_keys_recovered":45,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"176'663","reported_seq":14079,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.367080+0000","last_change":"2024-09-14T23:59:17.004025+0000","last_active":"2024-09-14T23:59:33.367080+0000","last_peered":"2024-09-14T23:59:33.367080+0000","last_clean":"2024-09-14T23:59:33.367080+0000","last_became_active":"2024-09-14T23:58:53.984808+0000","last_became_peered":"2024-09-14T23:58:53.984808+0000","last_unstale":"2024-09-14T23:59:33.367080+0000","last_undegraded":"2024-09-14T23:59:33.367080+0000","last_fullsized":"2024-09-14T23:59:33.367080+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":663,"log_dups_size":0,"ondisk_log_size":663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:48:22.749991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030575429999999998,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":14675,"num_read_kb":21661,"num_write":655,"num_write_kb":35778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":12964873,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"176'797","reported_seq":6504,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.684762+0000","last_change":"2024-09-14T23:59:17.069019+0000","last_active":"2024-09-14T23:59:33.684762+0000","last_peered":"2024-09-14T23:59:33.684762+0000","last_clean":"2024-09-14T23:59:33.684762+0000","last_became_active":"2024-09-14T23:56:32.402973+0000","last_became_peered":"2024-09-14T23:56:32.402973+0000","last_unstale":"2024-09-14T23:59:33.684762+0000","last_undegraded":"2024-09-14T23:59:33.684762+0000","last_fullsized":"2024-09-14T23:59:33.684762+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":797,"log_dups_size":0,"ondisk_log_size":797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:18:08.492160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088108100000000003,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":7036,"num_read_kb":57979,"num_write":1182,"num_write_kb":51466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"176'569","reported_seq":8379,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.366890+0000","last_change":"2024-09-14T23:59:17.001624+0000","last_active":"2024-09-14T23:59:33.366890+0000","last_peered":"2024-09-14T23:59:33.366890+0000","last_clean":"2024-09-14T23:59:33.366890+0000","last_became_active":"2024-09-14T23:58:52.987786+0000","last_became_peered":"2024-09-14T23:58:52.987786+0000","last_unstale":"2024-09-14T23:59:33.366890+0000","last_undegraded":"2024-09-14T23:59:33.366890+0000","last_fullsized":"2024-09-14T23:59:33.366890+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:55:48.806780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001145566,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":8694,"num_read_kb":36214,"num_write":569,"num_write_kb":28063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":19310765,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"176'376","reported_seq":3270,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067650+0000","last_change":"2024-09-14T23:59:20.000977+0000","last_active":"2024-09-14T23:59:33.067650+0000","last_peered":"2024-09-14T23:59:33.067650+0000","last_clean":"2024-09-14T23:59:33.067650+0000","last_became_active":"2024-09-14T23:59:19.999489+0000","last_became_peered":"2024-09-14T23:59:19.999489+0000","last_unstale":"2024-09-14T23:59:33.067650+0000","last_undegraded":"2024-09-14T23:59:33.067650+0000","last_fullsized":"2024-09-14T23:59:33.067650+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:21:32.054413+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0042424580000000002,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":3170,"num_read_kb":16380,"num_write":287,"num_write_kb":14646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":7078411,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"176'975","reported_seq":4718,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.366901+0000","last_change":"2024-09-14T23:59:17.002239+0000","last_active":"2024-09-14T23:59:33.366901+0000","last_peered":"2024-09-14T23:59:33.366901+0000","last_clean":"2024-09-14T23:59:33.366901+0000","last_became_active":"2024-09-14T23:58:53.987342+0000","last_became_peered":"2024-09-14T23:58:53.987342+0000","last_unstale":"2024-09-14T23:59:33.366901+0000","last_undegraded":"2024-09-14T23:59:33.366901+0000","last_fullsized":"2024-09-14T23:59:33.366901+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:30.333512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001659113,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":7188,"num_read_kb":25892,"num_write":1276,"num_write_kb":14588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":290,"num_bytes_recovered":14372748,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":1204,"num_object_clones":0,"num_object_copies":2408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1204,"num_whiteouts":1200,"num_read":310661,"num_read_kb":1031538,"num_write":24584,"num_write_kb":670524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7228,"num_bytes_recovered":332991677,"num_keys_recovered":596,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22207,"ondisk_log_size":22207,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":1771104,"data_compressed":100161,"data_compressed_allocated":135168,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509486,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51552,"kb_used_data":2316,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320288,"statfs":{"total":96636764160,"available":96583974912,"internally_reserved":0,"allocated":2371584,"data_stored":1713775,"data_compressed":33932,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509485,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33456,"kb_used_data":2268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338384,"statfs":{"total":96636764160,"available":96602505216,"internally_reserved":0,"allocated":2322432,"data_stored":1123407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509484,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47348,"kb_used_data":2272,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324492,"statfs":{"total":96636764160,"available":96588279808,"internally_reserved":0,"allocated":2326528,"data_stored":1123426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509486,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77876,"kb_used_data":2272,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293964,"statfs":{"total":96636764160,"available":96557019136,"internally_reserved":0,"allocated":2326528,"data_stored":1123426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509484,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55668,"kb_used_data":2268,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316172,"statfs":{"total":96636764160,"available":96579760128,"internally_reserved":0,"allocated":2322432,"data_stored":1123407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542188,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41772,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330068,"statfs":{"total":96636764160,"available":96593989632,"internally_reserved":0,"allocated":2318336,"data_stored":1117603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542187,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51576,"kb_used_data":2332,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320264,"statfs":{"total":96636764160,"available":96583950336,"internally_reserved":0,"allocated":2387968,"data_stored":1713775,"data_compressed":32297,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574892,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33456,"kb_used_data":2268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338384,"statfs":{"total":96636764160,"available":96602505216,"internally_reserved":0,"allocated":2322432,"data_stored":1123407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32297,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":33932,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":33932,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T23:59:39.249 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-14T23:59:39.538 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:59:39.538 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":193,"fsid":"21916fcc-f95c-4bfa-8b8b-11ca45845613","created":"2024-09-14T23:56:03.167948+0000","modified":"2024-09-14T23:59:33.063861+0000","last_up_change":"2024-09-14T23:56:14.322882+0000","last_in_change":"2024-09-14T23:59:07.115398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-14T23:56:13.500197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-14T23:56:21.284705+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-14T23:56:30.861504+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"177","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":51,"snap_epoch":177,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"db9d0c67-2d60-4592-9ad3-f99e5294f93f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":314865173},{"type":"v1","addr":"172.21.15.42:6817","nonce":314865173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":314865173},{"type":"v1","addr":"172.21.15.42:6819","nonce":314865173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":314865173},{"type":"v1","addr":"172.21.15.42:6823","nonce":314865173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":314865173},{"type":"v1","addr":"172.21.15.42:6821","nonce":314865173}]},"public_addr":"172.21.15.42:6817/314865173","cluster_addr":"172.21.15.42:6819/314865173","heartbeat_back_addr":"172.21.15.42:6823/314865173","heartbeat_front_addr":"172.21.15.42:6821/314865173","state":["exists","up"]},{"osd":1,"uuid":"6c1a2765-c591-4c57-8ab3-ee6645970aa5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":3197912310},{"type":"v1","addr":"172.21.15.42:6801","nonce":3197912310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":3197912310},{"type":"v1","addr":"172.21.15.42:6803","nonce":3197912310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":3197912310},{"type":"v1","addr":"172.21.15.42:6807","nonce":3197912310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":3197912310},{"type":"v1","addr":"172.21.15.42:6805","nonce":3197912310}]},"public_addr":"172.21.15.42:6801/3197912310","cluster_addr":"172.21.15.42:6803/3197912310","heartbeat_back_addr":"172.21.15.42:6807/3197912310","heartbeat_front_addr":"172.21.15.42:6805/3197912310","state":["exists","up"]},{"osd":2,"uuid":"930db893-7be6-4645-a7f1-e043c5ec95eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":3241153660},{"type":"v1","addr":"172.21.15.42:6809","nonce":3241153660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":3241153660},{"type":"v1","addr":"172.21.15.42:6811","nonce":3241153660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":3241153660},{"type":"v1","addr":"172.21.15.42:6815","nonce":3241153660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":3241153660},{"type":"v1","addr":"172.21.15.42:6813","nonce":3241153660}]},"public_addr":"172.21.15.42:6809/3241153660","cluster_addr":"172.21.15.42:6811/3241153660","heartbeat_back_addr":"172.21.15.42:6815/3241153660","heartbeat_front_addr":"172.21.15.42:6813/3241153660","state":["exists","up"]},{"osd":3,"uuid":"45bb3b84-b1c4-4f87-a7d1-0eb77b81741b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":1736416326},{"type":"v1","addr":"172.21.15.42:6825","nonce":1736416326}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1736416326},{"type":"v1","addr":"172.21.15.42:6827","nonce":1736416326}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1736416326},{"type":"v1","addr":"172.21.15.42:6831","nonce":1736416326}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1736416326},{"type":"v1","addr":"172.21.15.42:6829","nonce":1736416326}]},"public_addr":"172.21.15.42:6825/1736416326","cluster_addr":"172.21.15.42:6827/1736416326","heartbeat_back_addr":"172.21.15.42:6831/1736416326","heartbeat_front_addr":"172.21.15.42:6829/1736416326","state":["exists","up"]},{"osd":4,"uuid":"a812e38f-c0f9-4d33-9373-1910fb09103c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1755154427},{"type":"v1","addr":"172.21.15.43:6809","nonce":1755154427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1755154427},{"type":"v1","addr":"172.21.15.43:6811","nonce":1755154427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1755154427},{"type":"v1","addr":"172.21.15.43:6815","nonce":1755154427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1755154427},{"type":"v1","addr":"172.21.15.43:6813","nonce":1755154427}]},"public_addr":"172.21.15.43:6809/1755154427","cluster_addr":"172.21.15.43:6811/1755154427","heartbeat_back_addr":"172.21.15.43:6815/1755154427","heartbeat_front_addr":"172.21.15.43:6813/1755154427","state":["exists","up"]},{"osd":5,"uuid":"0de87e5b-7a21-4fb4-9543-d96c065786f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":3033769028},{"type":"v1","addr":"172.21.15.43:6817","nonce":3033769028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":3033769028},{"type":"v1","addr":"172.21.15.43:6819","nonce":3033769028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":3033769028},{"type":"v1","addr":"172.21.15.43:6823","nonce":3033769028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":3033769028},{"type":"v1","addr":"172.21.15.43:6821","nonce":3033769028}]},"public_addr":"172.21.15.43:6817/3033769028","cluster_addr":"172.21.15.43:6819/3033769028","heartbeat_back_addr":"172.21.15.43:6823/3033769028","heartbeat_front_addr":"172.21.15.43:6821/3033769028","state":["exists","up"]},{"osd":6,"uuid":"c58c1d59-ad02-40f0-98e3-9078ef7e622b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3314282416},{"type":"v1","addr":"172.21.15.43:6801","nonce":3314282416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3314282416},{"type":"v1","addr":"172.21.15.43:6803","nonce":3314282416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":3314282416},{"type":"v1","addr":"172.21.15.43:6807","nonce":3314282416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3314282416},{"type":"v1","addr":"172.21.15.43:6805","nonce":3314282416}]},"public_addr":"172.21.15.43:6801/3314282416","cluster_addr":"172.21.15.43:6803/3314282416","heartbeat_back_addr":"172.21.15.43:6807/3314282416","heartbeat_front_addr":"172.21.15.43:6805/3314282416","state":["exists","up"]},{"osd":7,"uuid":"389bdb04-20e3-4a7a-8504-687e034bd7dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1481350118},{"type":"v1","addr":"172.21.15.43:6825","nonce":1481350118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":1481350118},{"type":"v1","addr":"172.21.15.43:6827","nonce":1481350118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":1481350118},{"type":"v1","addr":"172.21.15.43:6831","nonce":1481350118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1481350118},{"type":"v1","addr":"172.21.15.43:6829","nonce":1481350118}]},"public_addr":"172.21.15.43:6825/1481350118","cluster_addr":"172.21.15.43:6827/1481350118","heartbeat_back_addr":"172.21.15.43:6831/1481350118","heartbeat_front_addr":"172.21.15.43:6829/1481350118","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T23:56:11.077810+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T23:56:11.578838+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T23:56:10.895490+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T23:56:12.290821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T23:56:11.800581+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T23:56:11.883155+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T23:56:11.604790+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T23:56:12.067759+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T23:59:40.552 INFO:tasks.ceph:Scrubbing osd.0 2024-09-14T23:59:40.552 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-14T23:59:40.694 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:59:40.694 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T23:59:40.694 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:59:40.705 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-14T23:59:40.995 INFO:teuthology.orchestra.run.smithi042.stderr:instructed osd(s) 0 to deep-scrub 2024-09-14T23:59:41.009 INFO:tasks.ceph:Scrubbing osd.1 2024-09-14T23:59:41.009 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-14T23:59:41.135 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:59:41.135 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T23:59:41.135 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:59:41.146 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-14T23:59:41.434 INFO:teuthology.orchestra.run.smithi042.stderr:instructed osd(s) 1 to deep-scrub 2024-09-14T23:59:41.447 INFO:tasks.ceph:Scrubbing osd.2 2024-09-14T23:59:41.447 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-14T23:59:41.574 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:59:41.575 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T23:59:41.575 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:59:41.585 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-14T23:59:41.876 INFO:teuthology.orchestra.run.smithi042.stderr:instructed osd(s) 2 to deep-scrub 2024-09-14T23:59:41.889 INFO:tasks.ceph:Scrubbing osd.3 2024-09-14T23:59:41.889 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-14T23:59:42.015 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:59:42.016 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T23:59:42.016 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:59:42.027 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-14T23:59:42.319 INFO:teuthology.orchestra.run.smithi042.stderr:instructed osd(s) 3 to deep-scrub 2024-09-14T23:59:42.332 INFO:tasks.ceph:Scrubbing osd.4 2024-09-14T23:59:42.333 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-14T23:59:42.460 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:59:42.460 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T23:59:42.461 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:59:42.472 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-14T23:59:42.779 INFO:teuthology.orchestra.run.smithi042.stderr:instructed osd(s) 4 to deep-scrub 2024-09-14T23:59:42.793 INFO:tasks.ceph:Scrubbing osd.5 2024-09-14T23:59:42.793 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-14T23:59:42.920 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:59:42.921 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T23:59:42.921 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:59:42.931 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-14T23:59:43.228 INFO:teuthology.orchestra.run.smithi042.stderr:instructed osd(s) 5 to deep-scrub 2024-09-14T23:59:43.241 INFO:tasks.ceph:Scrubbing osd.6 2024-09-14T23:59:43.241 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-14T23:59:43.374 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:59:43.374 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "" 2024-09-14T23:59:43.375 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:59:43.385 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-14T23:59:43.676 INFO:teuthology.orchestra.run.smithi042.stderr:instructed osd(s) 6 to deep-scrub 2024-09-14T23:59:43.689 INFO:tasks.ceph:Scrubbing osd.7 2024-09-14T23:59:43.689 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-14T23:59:43.817 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-14T23:59:43.817 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T23:59:43.817 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-14T23:59:43.828 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-14T23:59:44.119 INFO:teuthology.orchestra.run.smithi042.stderr:instructed osd(s) 7 to deep-scrub 2024-09-14T23:59:44.133 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T23:59:44.446 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-14T23:59:44.447 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-14T23:59:44.458 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2024-09-14T23:59:43.489425+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":1208,"num_object_clones":0,"num_object_copies":2416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1208,"num_whiteouts":1200,"num_read":310777,"num_read_kb":1031638,"num_write":24745,"num_write_kb":672942,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7234,"num_bytes_recovered":334172432,"num_keys_recovered":596,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22393,"ondisk_log_size":22393,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":392708,"kb_used_data":18272,"kb_used_omap":12,"kb_used_meta":374259,"kb_avail":754582012,"statfs":{"total":773094113280,"available":772691980288,"internally_reserved":0,"allocated":18710528,"data_stored":10168068,"data_compressed":66229,"data_compressed_allocated":90112,"data_compressed_original":1163264,"omap_allocated":12752,"internal_metadata":383241776},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.423606"},"pg_stats":[{"pgid":"3.1f","version":"177'1121","reported_seq":6214,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067965+0000","last_change":"2024-09-14T23:59:16.994682+0000","last_active":"2024-09-14T23:59:33.067965+0000","last_peered":"2024-09-14T23:59:33.067965+0000","last_clean":"2024-09-14T23:59:33.067965+0000","last_became_active":"2024-09-14T23:57:00.062116+0000","last_became_peered":"2024-09-14T23:57:00.062116+0000","last_unstale":"2024-09-14T23:59:33.067965+0000","last_undegraded":"2024-09-14T23:59:33.067965+0000","last_fullsized":"2024-09-14T23:59:33.067965+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":1121,"log_dups_size":0,"ondisk_log_size":1121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094842500000000003,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":8822,"num_read_kb":37588,"num_write":1354,"num_write_kb":38053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":2327658,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"177'479","reported_seq":6679,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067687+0000","last_change":"2024-09-14T23:59:16.998886+0000","last_active":"2024-09-14T23:59:33.067687+0000","last_peered":"2024-09-14T23:59:33.067687+0000","last_clean":"2024-09-14T23:59:33.067687+0000","last_became_active":"2024-09-14T23:59:11.135384+0000","last_became_peered":"2024-09-14T23:59:11.135384+0000","last_unstale":"2024-09-14T23:59:33.067687+0000","last_undegraded":"2024-09-14T23:59:33.067687+0000","last_fullsized":"2024-09-14T23:59:33.067687+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:24:06.187016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001605626,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":5337,"num_read_kb":37535,"num_write":472,"num_write_kb":39218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":526,"num_bytes_recovered":14308962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"176'609","reported_seq":13697,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067726+0000","last_change":"2024-09-14T23:59:30.060565+0000","last_active":"2024-09-14T23:59:33.067726+0000","last_peered":"2024-09-14T23:59:33.067726+0000","last_clean":"2024-09-14T23:59:33.067726+0000","last_became_active":"2024-09-14T23:59:30.058342+0000","last_became_peered":"2024-09-14T23:59:30.058342+0000","last_unstale":"2024-09-14T23:59:33.067726+0000","last_undegraded":"2024-09-14T23:59:33.067726+0000","last_fullsized":"2024-09-14T23:59:33.067726+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:31:03.745601+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0080108929999999998,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":14616,"num_read_kb":22815,"num_write":536,"num_write_kb":24101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":1249963,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"177'995","reported_seq":6658,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067658+0000","last_change":"2024-09-14T23:59:30.060281+0000","last_active":"2024-09-14T23:59:33.067658+0000","last_peered":"2024-09-14T23:59:33.067658+0000","last_clean":"2024-09-14T23:59:33.067658+0000","last_became_active":"2024-09-14T23:59:30.058275+0000","last_became_peered":"2024-09-14T23:59:30.058275+0000","last_unstale":"2024-09-14T23:59:33.067658+0000","last_undegraded":"2024-09-14T23:59:33.067658+0000","last_fullsized":"2024-09-14T23:59:33.067658+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":995,"log_dups_size":0,"ondisk_log_size":995,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:51:47.471326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086515500000000005,"stat_sum":{"num_bytes":0,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":45,"num_read":9513,"num_read_kb":25550,"num_write":1310,"num_write_kb":9734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":2675491,"num_keys_recovered":40,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"193'549","reported_seq":2437,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:41.615126+0000","last_change":"2024-09-14T23:59:41.615183+0000","last_active":"2024-09-14T23:59:41.615126+0000","last_peered":"2024-09-14T23:59:41.615126+0000","last_clean":"2024-09-14T23:59:41.615126+0000","last_became_active":"2024-09-14T23:56:32.398329+0000","last_became_peered":"2024-09-14T23:56:32.398329+0000","last_unstale":"2024-09-14T23:59:41.615126+0000","last_undegraded":"2024-09-14T23:59:41.615126+0000","last_fullsized":"2024-09-14T23:59:41.615126+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"193'549","last_scrub_stamp":"2024-09-14T23:59:41.615024+0000","last_deep_scrub":"193'549","last_deep_scrub_stamp":"2024-09-14T23:59:41.615024+0000","last_clean_scrub_stamp":"2024-09-14T23:59:41.615024+0000","objects_scrubbed":25,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:57:49.861137+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0.00103907,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":2076,"num_read_kb":19877,"num_write":623,"num_write_kb":10342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"177'899","reported_seq":8499,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.685085+0000","last_change":"2024-09-14T23:59:16.994978+0000","last_active":"2024-09-14T23:59:33.685085+0000","last_peered":"2024-09-14T23:59:33.685085+0000","last_clean":"2024-09-14T23:59:33.685085+0000","last_became_active":"2024-09-14T23:56:32.399655+0000","last_became_peered":"2024-09-14T23:56:32.399655+0000","last_unstale":"2024-09-14T23:59:33.685085+0000","last_undegraded":"2024-09-14T23:59:33.685085+0000","last_fullsized":"2024-09-14T23:59:33.685085+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":899,"log_dups_size":0,"ondisk_log_size":899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:01:52.825811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001807665,"stat_sum":{"num_bytes":19,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":46,"num_read":10058,"num_read_kb":53829,"num_write":970,"num_write_kb":34825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"177'695","reported_seq":4691,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.684204+0000","last_change":"2024-09-14T23:59:16.994457+0000","last_active":"2024-09-14T23:59:33.684204+0000","last_peered":"2024-09-14T23:59:33.684204+0000","last_clean":"2024-09-14T23:59:33.684204+0000","last_became_active":"2024-09-14T23:59:11.135739+0000","last_became_peered":"2024-09-14T23:59:11.135739+0000","last_unstale":"2024-09-14T23:59:33.684204+0000","last_undegraded":"2024-09-14T23:59:33.684204+0000","last_fullsized":"2024-09-14T23:59:33.684204+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099865499999999999,"stat_sum":{"num_bytes":0,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":42,"num_read":5485,"num_read_kb":20536,"num_write":717,"num_write_kb":15834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":318,"num_bytes_recovered":6472298,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"177'742","reported_seq":6564,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067404+0000","last_change":"2024-09-14T23:59:16.998152+0000","last_active":"2024-09-14T23:59:33.067404+0000","last_peered":"2024-09-14T23:59:33.067404+0000","last_clean":"2024-09-14T23:59:33.067404+0000","last_became_active":"2024-09-14T23:59:12.133258+0000","last_became_peered":"2024-09-14T23:59:12.133258+0000","last_unstale":"2024-09-14T23:59:33.067404+0000","last_undegraded":"2024-09-14T23:59:33.067404+0000","last_fullsized":"2024-09-14T23:59:33.067404+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":742,"log_dups_size":0,"ondisk_log_size":742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:43:45.205419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093947500000000005,"stat_sum":{"num_bytes":0,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":46,"num_read":6734,"num_read_kb":59364,"num_write":741,"num_write_kb":26095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":340,"num_bytes_recovered":12763266,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"176'544","reported_seq":11743,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.068075+0000","last_change":"2024-09-14T23:59:16.994889+0000","last_active":"2024-09-14T23:59:33.068075+0000","last_peered":"2024-09-14T23:59:33.068075+0000","last_clean":"2024-09-14T23:59:33.068075+0000","last_became_active":"2024-09-14T23:57:00.062304+0000","last_became_peered":"2024-09-14T23:57:00.062304+0000","last_unstale":"2024-09-14T23:59:33.068075+0000","last_undegraded":"2024-09-14T23:59:33.068075+0000","last_fullsized":"2024-09-14T23:59:33.068075+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091006300000000002,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":12421,"num_read_kb":16852,"num_write":491,"num_write_kb":6294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":3345543,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"177'463","reported_seq":7658,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067791+0000","last_change":"2024-09-14T23:59:30.062097+0000","last_active":"2024-09-14T23:59:33.067791+0000","last_peered":"2024-09-14T23:59:33.067791+0000","last_clean":"2024-09-14T23:59:33.067791+0000","last_became_active":"2024-09-14T23:59:30.060469+0000","last_became_peered":"2024-09-14T23:59:30.060469+0000","last_unstale":"2024-09-14T23:59:33.067791+0000","last_undegraded":"2024-09-14T23:59:33.067791+0000","last_fullsized":"2024-09-14T23:59:33.067791+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:38:29.716639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002288432,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":7773,"num_read_kb":27495,"num_write":382,"num_write_kb":13368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":404,"num_bytes_recovered":26159849,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"176'612","reported_seq":9857,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.367048+0000","last_change":"2024-09-14T23:59:17.002582+0000","last_active":"2024-09-14T23:59:33.367048+0000","last_peered":"2024-09-14T23:59:33.367048+0000","last_clean":"2024-09-14T23:59:33.367048+0000","last_became_active":"2024-09-14T23:59:11.135860+0000","last_became_peered":"2024-09-14T23:59:11.135860+0000","last_unstale":"2024-09-14T23:59:33.367048+0000","last_undegraded":"2024-09-14T23:59:33.367048+0000","last_fullsized":"2024-09-14T23:59:33.367048+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":612,"log_dups_size":0,"ondisk_log_size":612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:16:43.430234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001929275,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":9948,"num_read_kb":36137,"num_write":618,"num_write_kb":23694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":516,"num_bytes_recovered":42197286,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"176'1093","reported_seq":8194,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.685233+0000","last_change":"2024-09-14T23:59:16.996306+0000","last_active":"2024-09-14T23:59:33.685233+0000","last_peered":"2024-09-14T23:59:33.685233+0000","last_clean":"2024-09-14T23:59:33.685233+0000","last_became_active":"2024-09-14T23:58:53.983979+0000","last_became_peered":"2024-09-14T23:58:53.983979+0000","last_unstale":"2024-09-14T23:59:33.685233+0000","last_undegraded":"2024-09-14T23:59:33.685233+0000","last_fullsized":"2024-09-14T23:59:33.685233+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":1093,"log_dups_size":0,"ondisk_log_size":1093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:26:54.972175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002903152,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":9679,"num_read_kb":42324,"num_write":1584,"num_write_kb":28908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":312,"num_bytes_recovered":22540127,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":373,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.684825+0000","last_change":"2024-09-14T23:58:49.989598+0000","last_active":"2024-09-14T23:59:33.684825+0000","last_peered":"2024-09-14T23:59:33.684825+0000","last_clean":"2024-09-14T23:59:33.684825+0000","last_became_active":"2024-09-14T23:58:49.989224+0000","last_became_peered":"2024-09-14T23:58:49.989224+0000","last_unstale":"2024-09-14T23:59:33.684825+0000","last_undegraded":"2024-09-14T23:59:33.684825+0000","last_fullsized":"2024-09-14T23:59:33.684825+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:52:25.663126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045105900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"193'759","reported_seq":5369,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:42.646302+0000","last_change":"2024-09-14T23:59:42.646352+0000","last_active":"2024-09-14T23:59:42.646302+0000","last_peered":"2024-09-14T23:59:42.646302+0000","last_clean":"2024-09-14T23:59:42.646302+0000","last_became_active":"2024-09-14T23:56:32.398664+0000","last_became_peered":"2024-09-14T23:56:32.398664+0000","last_unstale":"2024-09-14T23:59:42.646302+0000","last_undegraded":"2024-09-14T23:59:42.646302+0000","last_fullsized":"2024-09-14T23:59:42.646302+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"193'759","last_scrub_stamp":"2024-09-14T23:59:42.646221+0000","last_deep_scrub":"193'759","last_deep_scrub_stamp":"2024-09-14T23:59:42.646221+0000","last_clean_scrub_stamp":"2024-09-14T23:59:42.646221+0000","objects_scrubbed":41,"log_size":759,"log_dups_size":0,"ondisk_log_size":759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:15:46.598055+0000","scrub_duration":36,"objects_trimmed":0,"snaptrim_duration":0.001169722,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":6635,"num_read_kb":24355,"num_write":754,"num_write_kb":5733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":372,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:42.322050+0000","last_change":"2024-09-14T23:59:42.322072+0000","last_active":"2024-09-14T23:59:42.322050+0000","last_peered":"2024-09-14T23:59:42.322050+0000","last_clean":"2024-09-14T23:59:42.322050+0000","last_became_active":"2024-09-14T23:56:22.526355+0000","last_became_peered":"2024-09-14T23:56:22.526355+0000","last_unstale":"2024-09-14T23:59:42.322050+0000","last_undegraded":"2024-09-14T23:59:42.322050+0000","last_fullsized":"2024-09-14T23:59:42.322050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:59:42.321966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:59:42.321966+0000","last_clean_scrub_stamp":"2024-09-14T23:59:42.321966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:23:38.392091+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00049910000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"177'528","reported_seq":7754,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067820+0000","last_change":"2024-09-14T23:59:17.000419+0000","last_active":"2024-09-14T23:59:33.067820+0000","last_peered":"2024-09-14T23:59:33.067820+0000","last_clean":"2024-09-14T23:59:33.067820+0000","last_became_active":"2024-09-14T23:59:11.135030+0000","last_became_peered":"2024-09-14T23:59:11.135030+0000","last_unstale":"2024-09-14T23:59:33.067820+0000","last_undegraded":"2024-09-14T23:59:33.067820+0000","last_fullsized":"2024-09-14T23:59:33.067820+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":528,"log_dups_size":0,"ondisk_log_size":528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:12:06.383593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027209629999999999,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":7655,"num_read_kb":43475,"num_write":499,"num_write_kb":17299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":640,"num_bytes_recovered":36520799,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":95,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.366957+0000","last_change":"2024-09-14T23:58:49.988483+0000","last_active":"2024-09-14T23:59:33.366957+0000","last_peered":"2024-09-14T23:59:33.366957+0000","last_clean":"2024-09-14T23:59:33.366957+0000","last_became_active":"2024-09-14T23:58:49.988169+0000","last_became_peered":"2024-09-14T23:58:49.988169+0000","last_unstale":"2024-09-14T23:59:33.366957+0000","last_undegraded":"2024-09-14T23:59:33.366957+0000","last_fullsized":"2024-09-14T23:59:33.366957+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:57:59.569156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"177'622","reported_seq":5511,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.684357+0000","last_change":"2024-09-14T23:59:16.993791+0000","last_active":"2024-09-14T23:59:33.684357+0000","last_peered":"2024-09-14T23:59:33.684357+0000","last_clean":"2024-09-14T23:59:33.684357+0000","last_became_active":"2024-09-14T23:56:32.401089+0000","last_became_peered":"2024-09-14T23:56:32.401089+0000","last_unstale":"2024-09-14T23:59:33.684357+0000","last_undegraded":"2024-09-14T23:59:33.684357+0000","last_fullsized":"2024-09-14T23:59:33.684357+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094649999999999997,"stat_sum":{"num_bytes":0,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":43,"num_read":6577,"num_read_kb":26346,"num_write":596,"num_write_kb":13585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":364,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.684386+0000","last_change":"2024-09-14T23:56:24.534510+0000","last_active":"2024-09-14T23:59:33.684386+0000","last_peered":"2024-09-14T23:59:33.684386+0000","last_clean":"2024-09-14T23:59:33.684386+0000","last_became_active":"2024-09-14T23:56:22.527611+0000","last_became_peered":"2024-09-14T23:56:22.527611+0000","last_unstale":"2024-09-14T23:59:33.684386+0000","last_undegraded":"2024-09-14T23:59:33.684386+0000","last_fullsized":"2024-09-14T23:59:33.684386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049916299999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"176'729","reported_seq":7600,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067812+0000","last_change":"2024-09-14T23:59:30.062265+0000","last_active":"2024-09-14T23:59:33.067812+0000","last_peered":"2024-09-14T23:59:33.067812+0000","last_clean":"2024-09-14T23:59:33.067812+0000","last_became_active":"2024-09-14T23:59:30.060381+0000","last_became_peered":"2024-09-14T23:59:30.060381+0000","last_unstale":"2024-09-14T23:59:33.067812+0000","last_undegraded":"2024-09-14T23:59:33.067812+0000","last_fullsized":"2024-09-14T23:59:33.067812+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":729,"log_dups_size":0,"ondisk_log_size":729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:19:07.081227+0000","scrub_duration":0,"objects_trimmed":10,"snaptrim_duration":0.012412685999999999,"stat_sum":{"num_bytes":0,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":47,"num_read":9294,"num_read_kb":18810,"num_write":776,"num_write_kb":10083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":401,"num_bytes_recovered":9491176,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"177'834","reported_seq":12831,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.685176+0000","last_change":"2024-09-14T23:59:16.995637+0000","last_active":"2024-09-14T23:59:33.685176+0000","last_peered":"2024-09-14T23:59:33.685176+0000","last_clean":"2024-09-14T23:59:33.685176+0000","last_became_active":"2024-09-14T23:56:32.397883+0000","last_became_peered":"2024-09-14T23:56:32.397883+0000","last_unstale":"2024-09-14T23:59:33.685176+0000","last_undegraded":"2024-09-14T23:59:33.685176+0000","last_fullsized":"2024-09-14T23:59:33.685176+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":834,"log_dups_size":0,"ondisk_log_size":834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:51:15.895964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023599810000000001,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":38,"num_read":13370,"num_read_kb":25781,"num_write":804,"num_write_kb":35448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":87,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067672+0000","last_change":"2024-09-14T23:59:30.058288+0000","last_active":"2024-09-14T23:59:33.067672+0000","last_peered":"2024-09-14T23:59:33.067672+0000","last_clean":"2024-09-14T23:59:33.067672+0000","last_became_active":"2024-09-14T23:59:30.057866+0000","last_became_peered":"2024-09-14T23:59:30.057866+0000","last_unstale":"2024-09-14T23:59:33.067672+0000","last_undegraded":"2024-09-14T23:59:33.067672+0000","last_fullsized":"2024-09-14T23:59:33.067672+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:55:57.947601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"177'518","reported_seq":9366,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.684167+0000","last_change":"2024-09-14T23:59:16.993927+0000","last_active":"2024-09-14T23:59:33.684167+0000","last_peered":"2024-09-14T23:59:33.684167+0000","last_clean":"2024-09-14T23:59:33.684167+0000","last_became_active":"2024-09-14T23:56:32.400278+0000","last_became_peered":"2024-09-14T23:56:32.400278+0000","last_unstale":"2024-09-14T23:59:33.684167+0000","last_undegraded":"2024-09-14T23:59:33.684167+0000","last_fullsized":"2024-09-14T23:59:33.684167+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098058499999999996,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":8550,"num_read_kb":62565,"num_write":432,"num_write_kb":30360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":350,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:42.055729+0000","last_change":"2024-09-14T23:59:42.055747+0000","last_active":"2024-09-14T23:59:42.055729+0000","last_peered":"2024-09-14T23:59:42.055729+0000","last_clean":"2024-09-14T23:59:42.055729+0000","last_became_active":"2024-09-14T23:59:23.991492+0000","last_became_peered":"2024-09-14T23:59:23.991492+0000","last_unstale":"2024-09-14T23:59:42.055729+0000","last_undegraded":"2024-09-14T23:59:42.055729+0000","last_fullsized":"2024-09-14T23:59:42.055729+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:59:42.055653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:59:42.055653+0000","last_clean_scrub_stamp":"2024-09-14T23:59:42.055653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:46:25.276975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026863400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"176'788","reported_seq":7242,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067783+0000","last_change":"2024-09-14T23:59:20.000557+0000","last_active":"2024-09-14T23:59:33.067783+0000","last_peered":"2024-09-14T23:59:33.067783+0000","last_clean":"2024-09-14T23:59:33.067783+0000","last_became_active":"2024-09-14T23:59:19.998578+0000","last_became_peered":"2024-09-14T23:59:19.998578+0000","last_unstale":"2024-09-14T23:59:33.067783+0000","last_undegraded":"2024-09-14T23:59:33.067783+0000","last_fullsized":"2024-09-14T23:59:33.067783+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":788,"log_dups_size":0,"ondisk_log_size":788,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.00381613,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":9431,"num_read_kb":22132,"num_write":943,"num_write_kb":5118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":95,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.366890+0000","last_change":"2024-09-14T23:58:49.988483+0000","last_active":"2024-09-14T23:59:33.366890+0000","last_peered":"2024-09-14T23:59:33.366890+0000","last_clean":"2024-09-14T23:59:33.366890+0000","last_became_active":"2024-09-14T23:58:49.988096+0000","last_became_peered":"2024-09-14T23:58:49.988096+0000","last_unstale":"2024-09-14T23:59:33.366890+0000","last_undegraded":"2024-09-14T23:59:33.366890+0000","last_fullsized":"2024-09-14T23:59:33.366890+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:34:48.166954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"176'1105","reported_seq":30205,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067544+0000","last_change":"2024-09-14T23:59:16.999655+0000","last_active":"2024-09-14T23:59:33.067544+0000","last_peered":"2024-09-14T23:59:33.067544+0000","last_clean":"2024-09-14T23:59:33.067544+0000","last_became_active":"2024-09-14T23:59:12.133789+0000","last_became_peered":"2024-09-14T23:59:12.133789+0000","last_unstale":"2024-09-14T23:59:33.067544+0000","last_undegraded":"2024-09-14T23:59:33.067544+0000","last_fullsized":"2024-09-14T23:59:33.067544+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":1105,"log_dups_size":0,"ondisk_log_size":1105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:04:55.112519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002233971,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":31963,"num_read_kb":26382,"num_write":1422,"num_write_kb":17307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":641,"num_bytes_recovered":19031962,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":122,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067683+0000","last_change":"2024-09-14T23:59:30.060260+0000","last_active":"2024-09-14T23:59:33.067683+0000","last_peered":"2024-09-14T23:59:33.067683+0000","last_clean":"2024-09-14T23:59:33.067683+0000","last_became_active":"2024-09-14T23:59:30.059854+0000","last_became_peered":"2024-09-14T23:59:30.059854+0000","last_unstale":"2024-09-14T23:59:33.067683+0000","last_undegraded":"2024-09-14T23:59:33.067683+0000","last_fullsized":"2024-09-14T23:59:33.067683+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:21.513263+0000","last_clean_scrub_stamp":"2024-09-14T23:56:21.513263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:14:35.344939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":277,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.366889+0000","last_change":"2024-09-14T23:58:53.985309+0000","last_active":"2024-09-14T23:59:33.366889+0000","last_peered":"2024-09-14T23:59:33.366889+0000","last_clean":"2024-09-14T23:59:33.366889+0000","last_became_active":"2024-09-14T23:58:53.984830+0000","last_became_peered":"2024-09-14T23:58:53.984830+0000","last_unstale":"2024-09-14T23:59:33.366889+0000","last_undegraded":"2024-09-14T23:59:33.366889+0000","last_fullsized":"2024-09-14T23:59:33.366889+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:14.322882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:14.322882+0000","last_clean_scrub_stamp":"2024-09-14T23:56:14.322882+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:37:48.336097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"177'609","reported_seq":5982,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067398+0000","last_change":"2024-09-14T23:59:17.000201+0000","last_active":"2024-09-14T23:59:33.067398+0000","last_peered":"2024-09-14T23:59:33.067398+0000","last_clean":"2024-09-14T23:59:33.067398+0000","last_became_active":"2024-09-14T23:59:12.136444+0000","last_became_peered":"2024-09-14T23:59:12.136444+0000","last_unstale":"2024-09-14T23:59:33.067398+0000","last_undegraded":"2024-09-14T23:59:33.067398+0000","last_fullsized":"2024-09-14T23:59:33.067398+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:26:06.094223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026622379999999999,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":5935,"num_read_kb":18123,"num_write":528,"num_write_kb":10919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":399,"num_bytes_recovered":9189081,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"176'343","reported_seq":11038,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.685073+0000","last_change":"2024-09-14T23:59:16.994730+0000","last_active":"2024-09-14T23:59:33.685073+0000","last_peered":"2024-09-14T23:59:33.685073+0000","last_clean":"2024-09-14T23:59:33.685073+0000","last_became_active":"2024-09-14T23:56:32.400017+0000","last_became_peered":"2024-09-14T23:56:32.400017+0000","last_unstale":"2024-09-14T23:59:33.685073+0000","last_undegraded":"2024-09-14T23:59:33.685073+0000","last_fullsized":"2024-09-14T23:59:33.685073+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:24:02.170046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015805459999999999,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":10189,"num_read_kb":13014,"num_write":297,"num_write_kb":15704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"176'834","reported_seq":16004,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.684762+0000","last_change":"2024-09-14T23:59:17.069111+0000","last_active":"2024-09-14T23:59:33.684762+0000","last_peered":"2024-09-14T23:59:33.684762+0000","last_clean":"2024-09-14T23:59:33.684762+0000","last_became_active":"2024-09-14T23:56:32.402769+0000","last_became_peered":"2024-09-14T23:56:32.402769+0000","last_unstale":"2024-09-14T23:59:33.684762+0000","last_undegraded":"2024-09-14T23:59:33.684762+0000","last_fullsized":"2024-09-14T23:59:33.684762+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":834,"log_dups_size":0,"ondisk_log_size":834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:36:02.993358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098966399999999991,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":40,"num_read":16916,"num_read_kb":32658,"num_write":1110,"num_write_kb":14962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"176'665","reported_seq":13705,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067583+0000","last_change":"2024-09-14T23:59:16.998126+0000","last_active":"2024-09-14T23:59:33.067583+0000","last_peered":"2024-09-14T23:59:33.067583+0000","last_clean":"2024-09-14T23:59:33.067583+0000","last_became_active":"2024-09-14T23:59:12.132838+0000","last_became_peered":"2024-09-14T23:59:12.132838+0000","last_unstale":"2024-09-14T23:59:33.067583+0000","last_undegraded":"2024-09-14T23:59:33.067583+0000","last_fullsized":"2024-09-14T23:59:33.067583+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":665,"log_dups_size":0,"ondisk_log_size":665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:34:25.667684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087623300000000002,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":13816,"num_read_kb":56666,"num_write":771,"num_write_kb":15952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":258,"num_bytes_recovered":8738708,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"176'705","reported_seq":16265,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067866+0000","last_change":"2024-09-14T23:59:30.062604+0000","last_active":"2024-09-14T23:59:33.067866+0000","last_peered":"2024-09-14T23:59:33.067866+0000","last_clean":"2024-09-14T23:59:33.067866+0000","last_became_active":"2024-09-14T23:59:30.060837+0000","last_became_peered":"2024-09-14T23:59:30.060837+0000","last_unstale":"2024-09-14T23:59:33.067866+0000","last_undegraded":"2024-09-14T23:59:33.067866+0000","last_fullsized":"2024-09-14T23:59:33.067866+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":705,"log_dups_size":0,"ondisk_log_size":705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:18:03.409303+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.021276283,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":15379,"num_read_kb":69494,"num_write":885,"num_write_kb":43376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":30030415,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"176'521","reported_seq":3381,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.367121+0000","last_change":"2024-09-14T23:59:17.003938+0000","last_active":"2024-09-14T23:59:33.367121+0000","last_peered":"2024-09-14T23:59:33.367121+0000","last_clean":"2024-09-14T23:59:33.367121+0000","last_became_active":"2024-09-14T23:58:52.985661+0000","last_became_peered":"2024-09-14T23:58:52.985661+0000","last_unstale":"2024-09-14T23:59:33.367121+0000","last_undegraded":"2024-09-14T23:59:33.367121+0000","last_fullsized":"2024-09-14T23:59:33.367121+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:53:00.607967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030262919999999999,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":3961,"num_read_kb":12312,"num_write":537,"num_write_kb":8315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":453,"num_bytes_recovered":19224487,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"177'532","reported_seq":5732,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.367080+0000","last_change":"2024-09-14T23:59:17.003376+0000","last_active":"2024-09-14T23:59:33.367080+0000","last_peered":"2024-09-14T23:59:33.367080+0000","last_clean":"2024-09-14T23:59:33.367080+0000","last_became_active":"2024-09-14T23:58:53.986149+0000","last_became_peered":"2024-09-14T23:58:53.986149+0000","last_unstale":"2024-09-14T23:59:33.367080+0000","last_undegraded":"2024-09-14T23:59:33.367080+0000","last_fullsized":"2024-09-14T23:59:33.367080+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:15:03.478135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024995130000000001,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":7765,"num_read_kb":21397,"num_write":463,"num_write_kb":11356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":12997809,"num_keys_recovered":45,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"176'663","reported_seq":14079,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.367080+0000","last_change":"2024-09-14T23:59:17.004025+0000","last_active":"2024-09-14T23:59:33.367080+0000","last_peered":"2024-09-14T23:59:33.367080+0000","last_clean":"2024-09-14T23:59:33.367080+0000","last_became_active":"2024-09-14T23:58:53.984808+0000","last_became_peered":"2024-09-14T23:58:53.984808+0000","last_unstale":"2024-09-14T23:59:33.367080+0000","last_undegraded":"2024-09-14T23:59:33.367080+0000","last_fullsized":"2024-09-14T23:59:33.367080+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":663,"log_dups_size":0,"ondisk_log_size":663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:48:22.749991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030575429999999998,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":14675,"num_read_kb":21661,"num_write":655,"num_write_kb":35778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":12964873,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"176'797","reported_seq":6504,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.684762+0000","last_change":"2024-09-14T23:59:17.069019+0000","last_active":"2024-09-14T23:59:33.684762+0000","last_peered":"2024-09-14T23:59:33.684762+0000","last_clean":"2024-09-14T23:59:33.684762+0000","last_became_active":"2024-09-14T23:56:32.402973+0000","last_became_peered":"2024-09-14T23:56:32.402973+0000","last_unstale":"2024-09-14T23:59:33.684762+0000","last_undegraded":"2024-09-14T23:59:33.684762+0000","last_fullsized":"2024-09-14T23:59:33.684762+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":797,"log_dups_size":0,"ondisk_log_size":797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:18:08.492160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088108100000000003,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":7036,"num_read_kb":57979,"num_write":1182,"num_write_kb":51466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"176'569","reported_seq":8379,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.366890+0000","last_change":"2024-09-14T23:59:17.001624+0000","last_active":"2024-09-14T23:59:33.366890+0000","last_peered":"2024-09-14T23:59:33.366890+0000","last_clean":"2024-09-14T23:59:33.366890+0000","last_became_active":"2024-09-14T23:58:52.987786+0000","last_became_peered":"2024-09-14T23:58:52.987786+0000","last_unstale":"2024-09-14T23:59:33.366890+0000","last_undegraded":"2024-09-14T23:59:33.366890+0000","last_fullsized":"2024-09-14T23:59:33.366890+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:55:48.806780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001145566,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":8694,"num_read_kb":36214,"num_write":569,"num_write_kb":28063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":19310765,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"176'376","reported_seq":3271,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.067711+0000","last_change":"2024-09-14T23:59:20.000977+0000","last_active":"2024-09-14T23:59:33.067711+0000","last_peered":"2024-09-14T23:59:33.067711+0000","last_clean":"2024-09-14T23:59:33.067711+0000","last_became_active":"2024-09-14T23:59:19.999489+0000","last_became_peered":"2024-09-14T23:59:19.999489+0000","last_unstale":"2024-09-14T23:59:33.067711+0000","last_undegraded":"2024-09-14T23:59:33.067711+0000","last_fullsized":"2024-09-14T23:59:33.067711+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0042424580000000002,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":3170,"num_read_kb":16380,"num_write":287,"num_write_kb":14646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":7078411,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"176'975","reported_seq":4718,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:33.366901+0000","last_change":"2024-09-14T23:59:17.002239+0000","last_active":"2024-09-14T23:59:33.366901+0000","last_peered":"2024-09-14T23:59:33.366901+0000","last_clean":"2024-09-14T23:59:33.366901+0000","last_became_active":"2024-09-14T23:58:53.987342+0000","last_became_peered":"2024-09-14T23:58:53.987342+0000","last_unstale":"2024-09-14T23:59:33.366901+0000","last_undegraded":"2024-09-14T23:59:33.366901+0000","last_fullsized":"2024-09-14T23:59:33.366901+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:56:31.385920+0000","last_clean_scrub_stamp":"2024-09-14T23:56:31.385920+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:30.333512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001659113,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":7188,"num_read_kb":25892,"num_write":1276,"num_write_kb":14588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":290,"num_bytes_recovered":14372748,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":1204,"num_object_clones":0,"num_object_copies":2408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1204,"num_whiteouts":1200,"num_read":310661,"num_read_kb":1031538,"num_write":24584,"num_write_kb":670524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7228,"num_bytes_recovered":332991677,"num_keys_recovered":596,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22273,"ondisk_log_size":22273,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":1771104,"data_compressed":100161,"data_compressed_allocated":135168,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509487,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51552,"kb_used_data":2316,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320288,"statfs":{"total":96636764160,"available":96583974912,"internally_reserved":0,"allocated":2371584,"data_stored":1713775,"data_compressed":33932,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509486,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33456,"kb_used_data":2268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338384,"statfs":{"total":96636764160,"available":96602505216,"internally_reserved":0,"allocated":2322432,"data_stored":1123407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509485,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47348,"kb_used_data":2272,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324492,"statfs":{"total":96636764160,"available":96588279808,"internally_reserved":0,"allocated":2326528,"data_stored":1123426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509487,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77876,"kb_used_data":2272,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293964,"statfs":{"total":96636764160,"available":96557019136,"internally_reserved":0,"allocated":2326528,"data_stored":1123426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509485,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55668,"kb_used_data":2268,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316172,"statfs":{"total":96636764160,"available":96579760128,"internally_reserved":0,"allocated":2322432,"data_stored":1123407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542190,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41788,"kb_used_data":2276,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330052,"statfs":{"total":96636764160,"available":96593973248,"internally_reserved":0,"allocated":2330624,"data_stored":1123445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542189,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51564,"kb_used_data":2332,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320276,"statfs":{"total":96636764160,"available":96583962624,"internally_reserved":0,"allocated":2387968,"data_stored":1713775,"data_compressed":32297,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33456,"kb_used_data":2268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338384,"statfs":{"total":96636764160,"available":96602505216,"internally_reserved":0,"allocated":2322432,"data_stored":1123407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32297,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":33932,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":33932,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T23:59:44.463 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-14T23:56:31.385920+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=56, tm_sec=31, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=59, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T23:59:44.463 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-14T23:56:31.385920+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=56, tm_sec=31, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=59, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T23:59:44.463 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-14T23:56:31.385920+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=56, tm_sec=31, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=59, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T23:59:44.463 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-14T23:56:31.385920+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=56, tm_sec=31, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=59, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T23:59:44.464 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-14T23:56:31.385920+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=56, tm_sec=31, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=59, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T23:59:44.464 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-14T23:56:31.385920+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=56, tm_sec=31, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=59, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T23:59:44.464 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-14T23:56:31.385920+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=56, tm_sec=31, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=59, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T23:59:44.464 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-14T23:56:31.385920+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=56, tm_sec=31, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=59, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T23:59:44.464 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-14T23:56:31.385920+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=56, tm_sec=31, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=59, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T23:59:44.464 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-14T23:56:31.385920+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=56, tm_sec=31, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=59, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T23:59:44.464 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-14T23:56:31.385920+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=56, tm_sec=31, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=59, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T23:59:44.464 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-14T23:56:21.513263+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=56, tm_sec=21, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=59, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T23:59:44.464 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-14T23:56:31.385920+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=56, tm_sec=31, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=59, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T23:59:44.465 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-14T23:56:21.513263+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=56, tm_sec=21, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=59, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T23:59:44.465 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-14T23:56:31.385920+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=56, tm_sec=31, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=59, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T23:59:44.465 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-14T23:56:21.513263+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=56, tm_sec=21, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=59, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T23:59:44.465 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-14T23:56:31.385920+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=56, tm_sec=31, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=59, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T23:59:44.465 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-14T23:56:31.385920+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=56, tm_sec=31, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=59, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T23:59:44.465 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-14T23:56:21.513263+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=56, tm_sec=21, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=59, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T23:59:44.465 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-14T23:56:31.385920+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=56, tm_sec=31, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=59, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T23:59:44.465 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-14T23:56:31.385920+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=56, tm_sec=31, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=59, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T23:59:44.465 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-14T23:56:21.513263+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=56, tm_sec=21, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=59, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T23:59:44.466 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-14T23:56:31.385920+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=56, tm_sec=31, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=59, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T23:59:44.466 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-14T23:56:21.513263+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=56, tm_sec=21, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=59, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T23:59:44.466 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-14T23:56:14.322882+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=56, tm_sec=14, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=59, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T23:59:44.466 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-14T23:56:31.385920+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=56, tm_sec=31, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=59, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T23:59:44.466 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-14T23:56:31.385920+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=56, tm_sec=31, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=59, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T23:59:44.466 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-14T23:56:31.385920+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=56, tm_sec=31, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=59, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T23:59:44.466 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-14T23:56:31.385920+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=56, tm_sec=31, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=59, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T23:59:44.466 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-14T23:56:31.385920+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=56, tm_sec=31, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=59, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T23:59:44.466 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-14T23:56:31.385920+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=56, tm_sec=31, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=59, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T23:59:44.467 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-14T23:56:31.385920+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=56, tm_sec=31, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=59, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T23:59:44.467 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-14T23:56:31.385920+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=56, tm_sec=31, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=59, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T23:59:44.467 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-14T23:56:31.385920+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=56, tm_sec=31, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=59, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T23:59:44.467 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-14T23:56:31.385920+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=56, tm_sec=31, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=59, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T23:59:44.467 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-14T23:56:31.385920+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=56, tm_sec=31, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=59, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T23:59:44.467 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-14T23:56:31.385920+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=56, tm_sec=31, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=23, tm_min=59, tm_sec=39, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T23:59:44.467 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-15T00:00:04.469 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T00:00:04.780 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-15T00:00:04.780 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-15T00:00:04.794 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":302,"stamp":"2024-09-15T00:00:03.493843+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":1208,"num_object_clones":0,"num_object_copies":2416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1208,"num_whiteouts":1200,"num_read":310777,"num_read_kb":1031638,"num_write":24745,"num_write_kb":672942,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7234,"num_bytes_recovered":334172432,"num_keys_recovered":596,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23532,"ondisk_log_size":23532,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":405152,"kb_used_data":18272,"kb_used_omap":12,"kb_used_meta":386739,"kb_avail":754569568,"statfs":{"total":773094113280,"available":772679237632,"internally_reserved":0,"allocated":18710528,"data_stored":10168068,"data_compressed":66229,"data_compressed_allocated":90112,"data_compressed_original":1163264,"omap_allocated":12752,"internal_metadata":396021296},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002686"},"pg_stats":[{"pgid":"3.1f","version":"193'1160","reported_seq":6262,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:43.613135+0000","last_change":"2024-09-14T23:59:43.613200+0000","last_active":"2024-09-14T23:59:43.613135+0000","last_peered":"2024-09-14T23:59:43.613135+0000","last_clean":"2024-09-14T23:59:43.613135+0000","last_became_active":"2024-09-14T23:57:00.062116+0000","last_became_peered":"2024-09-14T23:57:00.062116+0000","last_unstale":"2024-09-14T23:59:43.613135+0000","last_undegraded":"2024-09-14T23:59:43.613135+0000","last_fullsized":"2024-09-14T23:59:43.613135+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"193'1160","last_scrub_stamp":"2024-09-14T23:59:43.613015+0000","last_deep_scrub":"193'1160","last_deep_scrub_stamp":"2024-09-14T23:59:43.613015+0000","last_clean_scrub_stamp":"2024-09-14T23:59:43.613015+0000","objects_scrubbed":39,"log_size":1160,"log_dups_size":0,"ondisk_log_size":1160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:20:38.592714+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0.00094842500000000003,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":8822,"num_read_kb":37588,"num_write":1354,"num_write_kb":38053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":2327658,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"193'510","reported_seq":6720,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:44.272318+0000","last_change":"2024-09-14T23:59:44.272346+0000","last_active":"2024-09-14T23:59:44.272318+0000","last_peered":"2024-09-14T23:59:44.272318+0000","last_clean":"2024-09-14T23:59:44.272318+0000","last_became_active":"2024-09-14T23:59:11.135384+0000","last_became_peered":"2024-09-14T23:59:11.135384+0000","last_unstale":"2024-09-14T23:59:44.272318+0000","last_undegraded":"2024-09-14T23:59:44.272318+0000","last_fullsized":"2024-09-14T23:59:44.272318+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"193'510","last_scrub_stamp":"2024-09-14T23:59:44.272241+0000","last_deep_scrub":"193'510","last_deep_scrub_stamp":"2024-09-14T23:59:44.272241+0000","last_clean_scrub_stamp":"2024-09-14T23:59:44.272241+0000","objects_scrubbed":31,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:31:42.978637+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0.001605626,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":5337,"num_read_kb":37535,"num_write":472,"num_write_kb":39218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":526,"num_bytes_recovered":14308962,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"193'649","reported_seq":13747,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:46.211391+0000","last_change":"2024-09-14T23:59:46.211433+0000","last_active":"2024-09-14T23:59:46.211391+0000","last_peered":"2024-09-14T23:59:46.211391+0000","last_clean":"2024-09-14T23:59:46.211391+0000","last_became_active":"2024-09-14T23:59:30.058342+0000","last_became_peered":"2024-09-14T23:59:30.058342+0000","last_unstale":"2024-09-14T23:59:46.211391+0000","last_undegraded":"2024-09-14T23:59:46.211391+0000","last_fullsized":"2024-09-14T23:59:46.211391+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"193'649","last_scrub_stamp":"2024-09-14T23:59:46.211310+0000","last_deep_scrub":"193'649","last_deep_scrub_stamp":"2024-09-14T23:59:46.211310+0000","last_clean_scrub_stamp":"2024-09-14T23:59:46.211310+0000","objects_scrubbed":40,"log_size":649,"log_dups_size":0,"ondisk_log_size":649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:41:22.042723+0000","scrub_duration":31,"objects_trimmed":2,"snaptrim_duration":0.0080108929999999998,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":14616,"num_read_kb":22815,"num_write":536,"num_write_kb":24101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":1249963,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"193'1040","reported_seq":6714,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:45.240431+0000","last_change":"2024-09-14T23:59:45.240492+0000","last_active":"2024-09-14T23:59:45.240431+0000","last_peered":"2024-09-14T23:59:45.240431+0000","last_clean":"2024-09-14T23:59:45.240431+0000","last_became_active":"2024-09-14T23:59:30.058275+0000","last_became_peered":"2024-09-14T23:59:30.058275+0000","last_unstale":"2024-09-14T23:59:45.240431+0000","last_undegraded":"2024-09-14T23:59:45.240431+0000","last_fullsized":"2024-09-14T23:59:45.240431+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"193'1040","last_scrub_stamp":"2024-09-14T23:59:45.240347+0000","last_deep_scrub":"193'1040","last_deep_scrub_stamp":"2024-09-14T23:59:45.240347+0000","last_clean_scrub_stamp":"2024-09-14T23:59:45.240347+0000","objects_scrubbed":46,"log_size":1040,"log_dups_size":0,"ondisk_log_size":1040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:04:43.083622+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0.00086515500000000005,"stat_sum":{"num_bytes":0,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":45,"num_read":9513,"num_read_kb":25550,"num_write":1310,"num_write_kb":9734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":2675491,"num_keys_recovered":40,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"193'549","reported_seq":2437,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:41.615126+0000","last_change":"2024-09-14T23:59:41.615183+0000","last_active":"2024-09-14T23:59:41.615126+0000","last_peered":"2024-09-14T23:59:41.615126+0000","last_clean":"2024-09-14T23:59:41.615126+0000","last_became_active":"2024-09-14T23:56:32.398329+0000","last_became_peered":"2024-09-14T23:56:32.398329+0000","last_unstale":"2024-09-14T23:59:41.615126+0000","last_undegraded":"2024-09-14T23:59:41.615126+0000","last_fullsized":"2024-09-14T23:59:41.615126+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"193'549","last_scrub_stamp":"2024-09-14T23:59:41.615024+0000","last_deep_scrub":"193'549","last_deep_scrub_stamp":"2024-09-14T23:59:41.615024+0000","last_clean_scrub_stamp":"2024-09-14T23:59:41.615024+0000","objects_scrubbed":25,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:57:49.861137+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0.00103907,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":2076,"num_read_kb":19877,"num_write":623,"num_write_kb":10342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"193'946","reported_seq":8556,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:45.314969+0000","last_change":"2024-09-14T23:59:45.315049+0000","last_active":"2024-09-14T23:59:45.314969+0000","last_peered":"2024-09-14T23:59:45.314969+0000","last_clean":"2024-09-14T23:59:45.314969+0000","last_became_active":"2024-09-14T23:56:32.399655+0000","last_became_peered":"2024-09-14T23:56:32.399655+0000","last_unstale":"2024-09-14T23:59:45.314969+0000","last_undegraded":"2024-09-14T23:59:45.314969+0000","last_fullsized":"2024-09-14T23:59:45.314969+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"193'946","last_scrub_stamp":"2024-09-14T23:59:45.314840+0000","last_deep_scrub":"193'946","last_deep_scrub_stamp":"2024-09-14T23:59:45.314840+0000","last_clean_scrub_stamp":"2024-09-14T23:59:45.314840+0000","objects_scrubbed":47,"log_size":946,"log_dups_size":0,"ondisk_log_size":946,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:44:47.006730+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0.001807665,"stat_sum":{"num_bytes":19,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":46,"num_read":10058,"num_read_kb":53829,"num_write":970,"num_write_kb":34825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"193'737","reported_seq":4742,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:46.387151+0000","last_change":"2024-09-14T23:59:46.387217+0000","last_active":"2024-09-14T23:59:46.387151+0000","last_peered":"2024-09-14T23:59:46.387151+0000","last_clean":"2024-09-14T23:59:46.387151+0000","last_became_active":"2024-09-14T23:59:11.135739+0000","last_became_peered":"2024-09-14T23:59:11.135739+0000","last_unstale":"2024-09-14T23:59:46.387151+0000","last_undegraded":"2024-09-14T23:59:46.387151+0000","last_fullsized":"2024-09-14T23:59:46.387151+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"193'737","last_scrub_stamp":"2024-09-14T23:59:46.387036+0000","last_deep_scrub":"193'737","last_deep_scrub_stamp":"2024-09-14T23:59:46.387036+0000","last_clean_scrub_stamp":"2024-09-14T23:59:46.387036+0000","objects_scrubbed":42,"log_size":737,"log_dups_size":0,"ondisk_log_size":737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:24:05.924247+0000","scrub_duration":34,"objects_trimmed":0,"snaptrim_duration":0.00099865499999999999,"stat_sum":{"num_bytes":0,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":42,"num_read":5485,"num_read_kb":20536,"num_write":717,"num_write_kb":15834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":318,"num_bytes_recovered":6472298,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"193'788","reported_seq":6620,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:47.307283+0000","last_change":"2024-09-14T23:59:47.307340+0000","last_active":"2024-09-14T23:59:47.307283+0000","last_peered":"2024-09-14T23:59:47.307283+0000","last_clean":"2024-09-14T23:59:47.307283+0000","last_became_active":"2024-09-14T23:59:12.133258+0000","last_became_peered":"2024-09-14T23:59:12.133258+0000","last_unstale":"2024-09-14T23:59:47.307283+0000","last_undegraded":"2024-09-14T23:59:47.307283+0000","last_fullsized":"2024-09-14T23:59:47.307283+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"193'788","last_scrub_stamp":"2024-09-14T23:59:47.307195+0000","last_deep_scrub":"193'788","last_deep_scrub_stamp":"2024-09-14T23:59:47.307195+0000","last_clean_scrub_stamp":"2024-09-14T23:59:47.307195+0000","objects_scrubbed":46,"log_size":788,"log_dups_size":0,"ondisk_log_size":788,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:24:09.511382+0000","scrub_duration":37,"objects_trimmed":0,"snaptrim_duration":0.00093947500000000005,"stat_sum":{"num_bytes":0,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":46,"num_read":6734,"num_read_kb":59364,"num_write":741,"num_write_kb":26095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":340,"num_bytes_recovered":12763266,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"193'576","reported_seq":11784,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:44.650671+0000","last_change":"2024-09-14T23:59:44.650700+0000","last_active":"2024-09-14T23:59:44.650671+0000","last_peered":"2024-09-14T23:59:44.650671+0000","last_clean":"2024-09-14T23:59:44.650671+0000","last_became_active":"2024-09-14T23:57:00.062304+0000","last_became_peered":"2024-09-14T23:57:00.062304+0000","last_unstale":"2024-09-14T23:59:44.650671+0000","last_undegraded":"2024-09-14T23:59:44.650671+0000","last_fullsized":"2024-09-14T23:59:44.650671+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"193'576","last_scrub_stamp":"2024-09-14T23:59:44.650596+0000","last_deep_scrub":"193'576","last_deep_scrub_stamp":"2024-09-14T23:59:44.650596+0000","last_clean_scrub_stamp":"2024-09-14T23:59:44.650596+0000","objects_scrubbed":32,"log_size":576,"log_dups_size":0,"ondisk_log_size":576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:14:01.503924+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0.00091006300000000002,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":12421,"num_read_kb":16852,"num_write":491,"num_write_kb":6294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":3345543,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"193'494","reported_seq":7699,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:47.226854+0000","last_change":"2024-09-14T23:59:47.226894+0000","last_active":"2024-09-14T23:59:47.226854+0000","last_peered":"2024-09-14T23:59:47.226854+0000","last_clean":"2024-09-14T23:59:47.226854+0000","last_became_active":"2024-09-14T23:59:30.060469+0000","last_became_peered":"2024-09-14T23:59:30.060469+0000","last_unstale":"2024-09-14T23:59:47.226854+0000","last_undegraded":"2024-09-14T23:59:47.226854+0000","last_fullsized":"2024-09-14T23:59:47.226854+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"193'494","last_scrub_stamp":"2024-09-14T23:59:47.226770+0000","last_deep_scrub":"193'494","last_deep_scrub_stamp":"2024-09-14T23:59:47.226770+0000","last_clean_scrub_stamp":"2024-09-14T23:59:47.226770+0000","objects_scrubbed":31,"log_size":494,"log_dups_size":0,"ondisk_log_size":494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:14:52.164632+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0.002288432,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":7773,"num_read_kb":27495,"num_write":382,"num_write_kb":13368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":404,"num_bytes_recovered":26159849,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"193'657","reported_seq":9912,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:52.567655+0000","last_change":"2024-09-14T23:59:52.567707+0000","last_active":"2024-09-14T23:59:52.567655+0000","last_peered":"2024-09-14T23:59:52.567655+0000","last_clean":"2024-09-14T23:59:52.567655+0000","last_became_active":"2024-09-14T23:59:11.135860+0000","last_became_peered":"2024-09-14T23:59:11.135860+0000","last_unstale":"2024-09-14T23:59:52.567655+0000","last_undegraded":"2024-09-14T23:59:52.567655+0000","last_fullsized":"2024-09-14T23:59:52.567655+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"193'657","last_scrub_stamp":"2024-09-14T23:59:52.567575+0000","last_deep_scrub":"193'657","last_deep_scrub_stamp":"2024-09-14T23:59:52.567575+0000","last_clean_scrub_stamp":"2024-09-14T23:59:52.567575+0000","objects_scrubbed":45,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:30:32.859980+0000","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0.001929275,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":9948,"num_read_kb":36137,"num_write":618,"num_write_kb":23694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":516,"num_bytes_recovered":42197286,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"193'1133","reported_seq":8244,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:46.284036+0000","last_change":"2024-09-14T23:59:46.284082+0000","last_active":"2024-09-14T23:59:46.284036+0000","last_peered":"2024-09-14T23:59:46.284036+0000","last_clean":"2024-09-14T23:59:46.284036+0000","last_became_active":"2024-09-14T23:58:53.983979+0000","last_became_peered":"2024-09-14T23:58:53.983979+0000","last_unstale":"2024-09-14T23:59:46.284036+0000","last_undegraded":"2024-09-14T23:59:46.284036+0000","last_fullsized":"2024-09-14T23:59:46.284036+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"193'1133","last_scrub_stamp":"2024-09-14T23:59:46.283954+0000","last_deep_scrub":"193'1133","last_deep_scrub_stamp":"2024-09-14T23:59:46.283954+0000","last_clean_scrub_stamp":"2024-09-14T23:59:46.283954+0000","objects_scrubbed":40,"log_size":1133,"log_dups_size":0,"ondisk_log_size":1133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:47:52.783448+0000","scrub_duration":37,"objects_trimmed":0,"snaptrim_duration":0.002903152,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":9679,"num_read_kb":42324,"num_write":1584,"num_write_kb":28908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":312,"num_bytes_recovered":22540127,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":382,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:46.270996+0000","last_change":"2024-09-14T23:59:46.271039+0000","last_active":"2024-09-14T23:59:46.270996+0000","last_peered":"2024-09-14T23:59:46.270996+0000","last_clean":"2024-09-14T23:59:46.270996+0000","last_became_active":"2024-09-14T23:58:49.989224+0000","last_became_peered":"2024-09-14T23:58:49.989224+0000","last_unstale":"2024-09-14T23:59:46.270996+0000","last_undegraded":"2024-09-14T23:59:46.270996+0000","last_fullsized":"2024-09-14T23:59:46.270996+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:59:46.270863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:59:46.270863+0000","last_clean_scrub_stamp":"2024-09-14T23:59:46.270863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:19:29.522519+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00045105900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"193'759","reported_seq":5369,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:42.646302+0000","last_change":"2024-09-14T23:59:42.646352+0000","last_active":"2024-09-14T23:59:42.646302+0000","last_peered":"2024-09-14T23:59:42.646302+0000","last_clean":"2024-09-14T23:59:42.646302+0000","last_became_active":"2024-09-14T23:56:32.398664+0000","last_became_peered":"2024-09-14T23:56:32.398664+0000","last_unstale":"2024-09-14T23:59:42.646302+0000","last_undegraded":"2024-09-14T23:59:42.646302+0000","last_fullsized":"2024-09-14T23:59:42.646302+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"193'759","last_scrub_stamp":"2024-09-14T23:59:42.646221+0000","last_deep_scrub":"193'759","last_deep_scrub_stamp":"2024-09-14T23:59:42.646221+0000","last_clean_scrub_stamp":"2024-09-14T23:59:42.646221+0000","objects_scrubbed":41,"log_size":759,"log_dups_size":0,"ondisk_log_size":759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:15:46.598055+0000","scrub_duration":36,"objects_trimmed":0,"snaptrim_duration":0.001169722,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":6635,"num_read_kb":24355,"num_write":754,"num_write_kb":5733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":372,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:42.322050+0000","last_change":"2024-09-14T23:59:42.322072+0000","last_active":"2024-09-14T23:59:42.322050+0000","last_peered":"2024-09-14T23:59:42.322050+0000","last_clean":"2024-09-14T23:59:42.322050+0000","last_became_active":"2024-09-14T23:56:22.526355+0000","last_became_peered":"2024-09-14T23:56:22.526355+0000","last_unstale":"2024-09-14T23:59:42.322050+0000","last_undegraded":"2024-09-14T23:59:42.322050+0000","last_fullsized":"2024-09-14T23:59:42.322050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:59:42.321966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:59:42.321966+0000","last_clean_scrub_stamp":"2024-09-14T23:59:42.321966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:23:38.392091+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00049910000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"193'572","reported_seq":7808,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:43.310424+0000","last_change":"2024-09-14T23:59:43.310495+0000","last_active":"2024-09-14T23:59:43.310424+0000","last_peered":"2024-09-14T23:59:43.310424+0000","last_clean":"2024-09-14T23:59:43.310424+0000","last_became_active":"2024-09-14T23:59:11.135030+0000","last_became_peered":"2024-09-14T23:59:11.135030+0000","last_unstale":"2024-09-14T23:59:43.310424+0000","last_undegraded":"2024-09-14T23:59:43.310424+0000","last_fullsized":"2024-09-14T23:59:43.310424+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"193'572","last_scrub_stamp":"2024-09-14T23:59:43.310296+0000","last_deep_scrub":"193'572","last_deep_scrub_stamp":"2024-09-14T23:59:43.310296+0000","last_clean_scrub_stamp":"2024-09-14T23:59:43.310296+0000","objects_scrubbed":44,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:33:02.022600+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0.0027209629999999999,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":7655,"num_read_kb":43475,"num_write":499,"num_write_kb":17299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":640,"num_bytes_recovered":36520799,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":104,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:45.488787+0000","last_change":"2024-09-14T23:59:45.488810+0000","last_active":"2024-09-14T23:59:45.488787+0000","last_peered":"2024-09-14T23:59:45.488787+0000","last_clean":"2024-09-14T23:59:45.488787+0000","last_became_active":"2024-09-14T23:58:49.988169+0000","last_became_peered":"2024-09-14T23:58:49.988169+0000","last_unstale":"2024-09-14T23:59:45.488787+0000","last_undegraded":"2024-09-14T23:59:45.488787+0000","last_fullsized":"2024-09-14T23:59:45.488787+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:59:45.488693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:59:45.488693+0000","last_clean_scrub_stamp":"2024-09-14T23:59:45.488693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:17:37.581468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"193'665","reported_seq":5563,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:45.337708+0000","last_change":"2024-09-14T23:59:45.337794+0000","last_active":"2024-09-14T23:59:45.337708+0000","last_peered":"2024-09-14T23:59:45.337708+0000","last_clean":"2024-09-14T23:59:45.337708+0000","last_became_active":"2024-09-14T23:56:32.401089+0000","last_became_peered":"2024-09-14T23:56:32.401089+0000","last_unstale":"2024-09-14T23:59:45.337708+0000","last_undegraded":"2024-09-14T23:59:45.337708+0000","last_fullsized":"2024-09-14T23:59:45.337708+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"193'665","last_scrub_stamp":"2024-09-14T23:59:45.337615+0000","last_deep_scrub":"193'665","last_deep_scrub_stamp":"2024-09-14T23:59:45.337615+0000","last_clean_scrub_stamp":"2024-09-14T23:59:45.337615+0000","objects_scrubbed":43,"log_size":665,"log_dups_size":0,"ondisk_log_size":665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:04:08.801287+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0.00094649999999999997,"stat_sum":{"num_bytes":0,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":43,"num_read":6577,"num_read_kb":26346,"num_write":596,"num_write_kb":13585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":372,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:43.295574+0000","last_change":"2024-09-14T23:59:43.295611+0000","last_active":"2024-09-14T23:59:43.295574+0000","last_peered":"2024-09-14T23:59:43.295574+0000","last_clean":"2024-09-14T23:59:43.295574+0000","last_became_active":"2024-09-14T23:56:22.527611+0000","last_became_peered":"2024-09-14T23:56:22.527611+0000","last_unstale":"2024-09-14T23:59:43.295574+0000","last_undegraded":"2024-09-14T23:59:43.295574+0000","last_fullsized":"2024-09-14T23:59:43.295574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:59:43.295468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:59:43.295468+0000","last_clean_scrub_stamp":"2024-09-14T23:59:43.295468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:55:46.057818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049916299999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"193'776","reported_seq":7657,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:48.284099+0000","last_change":"2024-09-14T23:59:48.284142+0000","last_active":"2024-09-14T23:59:48.284099+0000","last_peered":"2024-09-14T23:59:48.284099+0000","last_clean":"2024-09-14T23:59:48.284099+0000","last_became_active":"2024-09-14T23:59:30.060381+0000","last_became_peered":"2024-09-14T23:59:30.060381+0000","last_unstale":"2024-09-14T23:59:48.284099+0000","last_undegraded":"2024-09-14T23:59:48.284099+0000","last_fullsized":"2024-09-14T23:59:48.284099+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"193'776","last_scrub_stamp":"2024-09-14T23:59:48.284021+0000","last_deep_scrub":"193'776","last_deep_scrub_stamp":"2024-09-14T23:59:48.284021+0000","last_clean_scrub_stamp":"2024-09-14T23:59:48.284021+0000","objects_scrubbed":47,"log_size":776,"log_dups_size":0,"ondisk_log_size":776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:14:02.901228+0000","scrub_duration":34,"objects_trimmed":10,"snaptrim_duration":0.012412685999999999,"stat_sum":{"num_bytes":0,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":47,"num_read":9294,"num_read_kb":18810,"num_write":776,"num_write_kb":10083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":401,"num_bytes_recovered":9491176,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"193'872","reported_seq":12879,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:43.213927+0000","last_change":"2024-09-14T23:59:43.213968+0000","last_active":"2024-09-14T23:59:43.213927+0000","last_peered":"2024-09-14T23:59:43.213927+0000","last_clean":"2024-09-14T23:59:43.213927+0000","last_became_active":"2024-09-14T23:56:32.397883+0000","last_became_peered":"2024-09-14T23:56:32.397883+0000","last_unstale":"2024-09-14T23:59:43.213927+0000","last_undegraded":"2024-09-14T23:59:43.213927+0000","last_fullsized":"2024-09-14T23:59:43.213927+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"193'872","last_scrub_stamp":"2024-09-14T23:59:43.213844+0000","last_deep_scrub":"193'872","last_deep_scrub_stamp":"2024-09-14T23:59:43.213844+0000","last_clean_scrub_stamp":"2024-09-14T23:59:43.213844+0000","objects_scrubbed":39,"log_size":872,"log_dups_size":0,"ondisk_log_size":872,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:57:54.298014+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0.0023599810000000001,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":38,"num_read":13370,"num_read_kb":25781,"num_write":804,"num_write_kb":35448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"193'3","reported_seq":97,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:43.232129+0000","last_change":"2024-09-14T23:59:43.232156+0000","last_active":"2024-09-14T23:59:43.232129+0000","last_peered":"2024-09-14T23:59:43.232129+0000","last_clean":"2024-09-14T23:59:43.232129+0000","last_became_active":"2024-09-14T23:59:30.057866+0000","last_became_peered":"2024-09-14T23:59:30.057866+0000","last_unstale":"2024-09-14T23:59:43.232129+0000","last_undegraded":"2024-09-14T23:59:43.232129+0000","last_fullsized":"2024-09-14T23:59:43.232129+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"193'3","last_scrub_stamp":"2024-09-14T23:59:43.232047+0000","last_deep_scrub":"193'3","last_deep_scrub_stamp":"2024-09-14T23:59:43.232047+0000","last_clean_scrub_stamp":"2024-09-14T23:59:43.232047+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:55:57.630364+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"193'551","reported_seq":9408,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:44.352725+0000","last_change":"2024-09-14T23:59:44.352757+0000","last_active":"2024-09-14T23:59:44.352725+0000","last_peered":"2024-09-14T23:59:44.352725+0000","last_clean":"2024-09-14T23:59:44.352725+0000","last_became_active":"2024-09-14T23:56:32.400278+0000","last_became_peered":"2024-09-14T23:56:32.400278+0000","last_unstale":"2024-09-14T23:59:44.352725+0000","last_undegraded":"2024-09-14T23:59:44.352725+0000","last_fullsized":"2024-09-14T23:59:44.352725+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"193'551","last_scrub_stamp":"2024-09-14T23:59:44.352640+0000","last_deep_scrub":"193'551","last_deep_scrub_stamp":"2024-09-14T23:59:44.352640+0000","last_clean_scrub_stamp":"2024-09-14T23:59:44.352640+0000","objects_scrubbed":33,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:50:57.454178+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0.00098058499999999996,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":8550,"num_read_kb":62565,"num_write":432,"num_write_kb":30360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":350,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:42.055729+0000","last_change":"2024-09-14T23:59:42.055747+0000","last_active":"2024-09-14T23:59:42.055729+0000","last_peered":"2024-09-14T23:59:42.055729+0000","last_clean":"2024-09-14T23:59:42.055729+0000","last_became_active":"2024-09-14T23:59:23.991492+0000","last_became_peered":"2024-09-14T23:59:23.991492+0000","last_unstale":"2024-09-14T23:59:42.055729+0000","last_undegraded":"2024-09-14T23:59:42.055729+0000","last_fullsized":"2024-09-14T23:59:42.055729+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:59:42.055653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:59:42.055653+0000","last_clean_scrub_stamp":"2024-09-14T23:59:42.055653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:46:25.276975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026863400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"193'829","reported_seq":7292,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:45.630520+0000","last_change":"2024-09-14T23:59:45.630570+0000","last_active":"2024-09-14T23:59:45.630520+0000","last_peered":"2024-09-14T23:59:45.630520+0000","last_clean":"2024-09-14T23:59:45.630520+0000","last_became_active":"2024-09-14T23:59:19.998578+0000","last_became_peered":"2024-09-14T23:59:19.998578+0000","last_unstale":"2024-09-14T23:59:45.630520+0000","last_undegraded":"2024-09-14T23:59:45.630520+0000","last_fullsized":"2024-09-14T23:59:45.630520+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"193'829","last_scrub_stamp":"2024-09-14T23:59:45.630427+0000","last_deep_scrub":"193'829","last_deep_scrub_stamp":"2024-09-14T23:59:45.630427+0000","last_clean_scrub_stamp":"2024-09-14T23:59:45.630427+0000","objects_scrubbed":41,"log_size":829,"log_dups_size":0,"ondisk_log_size":829,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:58:15.070728+0000","scrub_duration":27,"objects_trimmed":4,"snaptrim_duration":0.00381613,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":9431,"num_read_kb":22132,"num_write":943,"num_write_kb":5118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":104,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:44.450565+0000","last_change":"2024-09-14T23:59:44.450590+0000","last_active":"2024-09-14T23:59:44.450565+0000","last_peered":"2024-09-14T23:59:44.450565+0000","last_clean":"2024-09-14T23:59:44.450565+0000","last_became_active":"2024-09-14T23:58:49.988096+0000","last_became_peered":"2024-09-14T23:58:49.988096+0000","last_unstale":"2024-09-14T23:59:44.450565+0000","last_undegraded":"2024-09-14T23:59:44.450565+0000","last_fullsized":"2024-09-14T23:59:44.450565+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T23:59:44.450468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T23:59:44.450468+0000","last_clean_scrub_stamp":"2024-09-14T23:59:44.450468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:47:50.142696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"193'1146","reported_seq":30256,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:46.276476+0000","last_change":"2024-09-14T23:59:46.276515+0000","last_active":"2024-09-14T23:59:46.276476+0000","last_peered":"2024-09-14T23:59:46.276476+0000","last_clean":"2024-09-14T23:59:46.276476+0000","last_became_active":"2024-09-14T23:59:12.133789+0000","last_became_peered":"2024-09-14T23:59:12.133789+0000","last_unstale":"2024-09-14T23:59:46.276476+0000","last_undegraded":"2024-09-14T23:59:46.276476+0000","last_fullsized":"2024-09-14T23:59:46.276476+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"193'1146","last_scrub_stamp":"2024-09-14T23:59:46.276391+0000","last_deep_scrub":"193'1146","last_deep_scrub_stamp":"2024-09-14T23:59:46.276391+0000","last_clean_scrub_stamp":"2024-09-14T23:59:46.276391+0000","objects_scrubbed":41,"log_size":1146,"log_dups_size":0,"ondisk_log_size":1146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:26:31.427796+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0.002233971,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":31963,"num_read_kb":26382,"num_write":1422,"num_write_kb":17307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":641,"num_bytes_recovered":19031962,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"193'2","reported_seq":132,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:44.186998+0000","last_change":"2024-09-14T23:59:44.187055+0000","last_active":"2024-09-14T23:59:44.186998+0000","last_peered":"2024-09-14T23:59:44.186998+0000","last_clean":"2024-09-14T23:59:44.186998+0000","last_became_active":"2024-09-14T23:59:30.059854+0000","last_became_peered":"2024-09-14T23:59:30.059854+0000","last_unstale":"2024-09-14T23:59:44.186998+0000","last_undegraded":"2024-09-14T23:59:44.186998+0000","last_fullsized":"2024-09-14T23:59:44.186998+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"193'2","last_scrub_stamp":"2024-09-14T23:59:44.186887+0000","last_deep_scrub":"193'2","last_deep_scrub_stamp":"2024-09-14T23:59:44.186887+0000","last_clean_scrub_stamp":"2024-09-14T23:59:44.186887+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:06:37.558237+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"193'119","reported_seq":288,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:48.466844+0000","last_change":"2024-09-14T23:59:48.466878+0000","last_active":"2024-09-14T23:59:48.466844+0000","last_peered":"2024-09-14T23:59:48.466844+0000","last_clean":"2024-09-14T23:59:48.466844+0000","last_became_active":"2024-09-14T23:58:53.984830+0000","last_became_peered":"2024-09-14T23:58:53.984830+0000","last_unstale":"2024-09-14T23:59:48.466844+0000","last_undegraded":"2024-09-14T23:59:48.466844+0000","last_fullsized":"2024-09-14T23:59:48.466844+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"193'119","last_scrub_stamp":"2024-09-14T23:59:48.466743+0000","last_deep_scrub":"193'119","last_deep_scrub_stamp":"2024-09-14T23:59:48.466743+0000","last_clean_scrub_stamp":"2024-09-14T23:59:48.466743+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:08:14.921940+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"193'644","reported_seq":6027,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:48.294034+0000","last_change":"2024-09-14T23:59:48.294074+0000","last_active":"2024-09-14T23:59:48.294034+0000","last_peered":"2024-09-14T23:59:48.294034+0000","last_clean":"2024-09-14T23:59:48.294034+0000","last_became_active":"2024-09-14T23:59:12.136444+0000","last_became_peered":"2024-09-14T23:59:12.136444+0000","last_unstale":"2024-09-14T23:59:48.294034+0000","last_undegraded":"2024-09-14T23:59:48.294034+0000","last_fullsized":"2024-09-14T23:59:48.294034+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"193'644","last_scrub_stamp":"2024-09-14T23:59:48.293952+0000","last_deep_scrub":"193'644","last_deep_scrub_stamp":"2024-09-14T23:59:48.293952+0000","last_clean_scrub_stamp":"2024-09-14T23:59:48.293952+0000","objects_scrubbed":35,"log_size":644,"log_dups_size":0,"ondisk_log_size":644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:00:35.506423+0000","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0.0026622379999999999,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":5935,"num_read_kb":18123,"num_write":528,"num_write_kb":10919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":399,"num_bytes_recovered":9189081,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"193'377","reported_seq":11082,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:44.252265+0000","last_change":"2024-09-14T23:59:44.252307+0000","last_active":"2024-09-14T23:59:44.252265+0000","last_peered":"2024-09-14T23:59:44.252265+0000","last_clean":"2024-09-14T23:59:44.252265+0000","last_became_active":"2024-09-14T23:56:32.400017+0000","last_became_peered":"2024-09-14T23:56:32.400017+0000","last_unstale":"2024-09-14T23:59:44.252265+0000","last_undegraded":"2024-09-14T23:59:44.252265+0000","last_fullsized":"2024-09-14T23:59:44.252265+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"193'377","last_scrub_stamp":"2024-09-14T23:59:44.252148+0000","last_deep_scrub":"193'377","last_deep_scrub_stamp":"2024-09-14T23:59:44.252148+0000","last_clean_scrub_stamp":"2024-09-14T23:59:44.252148+0000","objects_scrubbed":34,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:13:10.840052+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0.0015805459999999999,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":10189,"num_read_kb":13014,"num_write":297,"num_write_kb":15704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"193'874","reported_seq":16054,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:44.211214+0000","last_change":"2024-09-14T23:59:44.211268+0000","last_active":"2024-09-14T23:59:44.211214+0000","last_peered":"2024-09-14T23:59:44.211214+0000","last_clean":"2024-09-14T23:59:44.211214+0000","last_became_active":"2024-09-14T23:56:32.402769+0000","last_became_peered":"2024-09-14T23:56:32.402769+0000","last_unstale":"2024-09-14T23:59:44.211214+0000","last_undegraded":"2024-09-14T23:59:44.211214+0000","last_fullsized":"2024-09-14T23:59:44.211214+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"193'874","last_scrub_stamp":"2024-09-14T23:59:44.211133+0000","last_deep_scrub":"193'874","last_deep_scrub_stamp":"2024-09-14T23:59:44.211133+0000","last_clean_scrub_stamp":"2024-09-14T23:59:44.211133+0000","objects_scrubbed":41,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:29:07.989740+0000","scrub_duration":34,"objects_trimmed":0,"snaptrim_duration":0.00098966399999999991,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":40,"num_read":16916,"num_read_kb":32658,"num_write":1110,"num_write_kb":14962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"193'693","reported_seq":13743,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:45.276833+0000","last_change":"2024-09-14T23:59:45.276863+0000","last_active":"2024-09-14T23:59:45.276833+0000","last_peered":"2024-09-14T23:59:45.276833+0000","last_clean":"2024-09-14T23:59:45.276833+0000","last_became_active":"2024-09-14T23:59:12.132838+0000","last_became_peered":"2024-09-14T23:59:12.132838+0000","last_unstale":"2024-09-14T23:59:45.276833+0000","last_undegraded":"2024-09-14T23:59:45.276833+0000","last_fullsized":"2024-09-14T23:59:45.276833+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"193'693","last_scrub_stamp":"2024-09-14T23:59:45.276745+0000","last_deep_scrub":"193'693","last_deep_scrub_stamp":"2024-09-14T23:59:45.276745+0000","last_clean_scrub_stamp":"2024-09-14T23:59:45.276745+0000","objects_scrubbed":28,"log_size":693,"log_dups_size":0,"ondisk_log_size":693,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:49:54.381837+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0.00087623300000000002,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":13816,"num_read_kb":56666,"num_write":771,"num_write_kb":15952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":258,"num_bytes_recovered":8738708,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"193'740","reported_seq":16310,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:49.231868+0000","last_change":"2024-09-14T23:59:49.231905+0000","last_active":"2024-09-14T23:59:49.231868+0000","last_peered":"2024-09-14T23:59:49.231868+0000","last_clean":"2024-09-14T23:59:49.231868+0000","last_became_active":"2024-09-14T23:59:30.060837+0000","last_became_peered":"2024-09-14T23:59:30.060837+0000","last_unstale":"2024-09-14T23:59:49.231868+0000","last_undegraded":"2024-09-14T23:59:49.231868+0000","last_fullsized":"2024-09-14T23:59:49.231868+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"193'740","last_scrub_stamp":"2024-09-14T23:59:49.231785+0000","last_deep_scrub":"193'740","last_deep_scrub_stamp":"2024-09-14T23:59:49.231785+0000","last_clean_scrub_stamp":"2024-09-14T23:59:49.231785+0000","objects_scrubbed":35,"log_size":740,"log_dups_size":0,"ondisk_log_size":740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:41:42.104692+0000","scrub_duration":25,"objects_trimmed":4,"snaptrim_duration":0.021276283,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":15379,"num_read_kb":69494,"num_write":885,"num_write_kb":43376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":30030415,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"193'555","reported_seq":3425,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:46.470072+0000","last_change":"2024-09-14T23:59:46.470111+0000","last_active":"2024-09-14T23:59:46.470072+0000","last_peered":"2024-09-14T23:59:46.470072+0000","last_clean":"2024-09-14T23:59:46.470072+0000","last_became_active":"2024-09-14T23:58:52.985661+0000","last_became_peered":"2024-09-14T23:58:52.985661+0000","last_unstale":"2024-09-14T23:59:46.470072+0000","last_undegraded":"2024-09-14T23:59:46.470072+0000","last_fullsized":"2024-09-14T23:59:46.470072+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"193'555","last_scrub_stamp":"2024-09-14T23:59:46.469982+0000","last_deep_scrub":"193'555","last_deep_scrub_stamp":"2024-09-14T23:59:46.469982+0000","last_clean_scrub_stamp":"2024-09-14T23:59:46.469982+0000","objects_scrubbed":34,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:19:33.084472+0000","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0.0030262919999999999,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":3961,"num_read_kb":12312,"num_write":537,"num_write_kb":8315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":453,"num_bytes_recovered":19224487,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"193'563","reported_seq":5773,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:50.511679+0000","last_change":"2024-09-14T23:59:50.511715+0000","last_active":"2024-09-14T23:59:50.511679+0000","last_peered":"2024-09-14T23:59:50.511679+0000","last_clean":"2024-09-14T23:59:50.511679+0000","last_became_active":"2024-09-14T23:58:53.986149+0000","last_became_peered":"2024-09-14T23:58:53.986149+0000","last_unstale":"2024-09-14T23:59:50.511679+0000","last_undegraded":"2024-09-14T23:59:50.511679+0000","last_fullsized":"2024-09-14T23:59:50.511679+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"193'563","last_scrub_stamp":"2024-09-14T23:59:50.511580+0000","last_deep_scrub":"193'563","last_deep_scrub_stamp":"2024-09-14T23:59:50.511580+0000","last_clean_scrub_stamp":"2024-09-14T23:59:50.511580+0000","objects_scrubbed":31,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:00:35.485520+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0.0024995130000000001,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":7765,"num_read_kb":21397,"num_write":463,"num_write_kb":11356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":12997809,"num_keys_recovered":45,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"193'698","reported_seq":14124,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:49.542337+0000","last_change":"2024-09-14T23:59:49.542378+0000","last_active":"2024-09-14T23:59:49.542337+0000","last_peered":"2024-09-14T23:59:49.542337+0000","last_clean":"2024-09-14T23:59:49.542337+0000","last_became_active":"2024-09-14T23:58:53.984808+0000","last_became_peered":"2024-09-14T23:58:53.984808+0000","last_unstale":"2024-09-14T23:59:49.542337+0000","last_undegraded":"2024-09-14T23:59:49.542337+0000","last_fullsized":"2024-09-14T23:59:49.542337+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"193'698","last_scrub_stamp":"2024-09-14T23:59:49.542251+0000","last_deep_scrub":"193'698","last_deep_scrub_stamp":"2024-09-14T23:59:49.542251+0000","last_clean_scrub_stamp":"2024-09-14T23:59:49.542251+0000","objects_scrubbed":35,"log_size":698,"log_dups_size":0,"ondisk_log_size":698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:21.603864+0000","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0.0030575429999999998,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":14675,"num_read_kb":21661,"num_write":655,"num_write_kb":35778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":12964873,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"193'842","reported_seq":6559,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:45.258800+0000","last_change":"2024-09-14T23:59:45.258841+0000","last_active":"2024-09-14T23:59:45.258800+0000","last_peered":"2024-09-14T23:59:45.258800+0000","last_clean":"2024-09-14T23:59:45.258800+0000","last_became_active":"2024-09-14T23:56:32.402973+0000","last_became_peered":"2024-09-14T23:56:32.402973+0000","last_unstale":"2024-09-14T23:59:45.258800+0000","last_undegraded":"2024-09-14T23:59:45.258800+0000","last_fullsized":"2024-09-14T23:59:45.258800+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"193'842","last_scrub_stamp":"2024-09-14T23:59:45.258723+0000","last_deep_scrub":"193'842","last_deep_scrub_stamp":"2024-09-14T23:59:45.258723+0000","last_clean_scrub_stamp":"2024-09-14T23:59:45.258723+0000","objects_scrubbed":45,"log_size":842,"log_dups_size":0,"ondisk_log_size":842,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:03:26.717003+0000","scrub_duration":36,"objects_trimmed":0,"snaptrim_duration":0.00088108100000000003,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":7036,"num_read_kb":57979,"num_write":1182,"num_write_kb":51466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"193'601","reported_seq":8421,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:47.505271+0000","last_change":"2024-09-14T23:59:47.505310+0000","last_active":"2024-09-14T23:59:47.505271+0000","last_peered":"2024-09-14T23:59:47.505271+0000","last_clean":"2024-09-14T23:59:47.505271+0000","last_became_active":"2024-09-14T23:58:52.987786+0000","last_became_peered":"2024-09-14T23:58:52.987786+0000","last_unstale":"2024-09-14T23:59:47.505271+0000","last_undegraded":"2024-09-14T23:59:47.505271+0000","last_fullsized":"2024-09-14T23:59:47.505271+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"193'601","last_scrub_stamp":"2024-09-14T23:59:47.505190+0000","last_deep_scrub":"193'601","last_deep_scrub_stamp":"2024-09-14T23:59:47.505190+0000","last_clean_scrub_stamp":"2024-09-14T23:59:47.505190+0000","objects_scrubbed":32,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:18:25.501017+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0.001145566,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":8694,"num_read_kb":36214,"num_write":569,"num_write_kb":28063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":19310765,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"193'402","reported_seq":3306,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:46.596746+0000","last_change":"2024-09-14T23:59:46.596773+0000","last_active":"2024-09-14T23:59:46.596746+0000","last_peered":"2024-09-14T23:59:46.596746+0000","last_clean":"2024-09-14T23:59:46.596746+0000","last_became_active":"2024-09-14T23:59:19.999489+0000","last_became_peered":"2024-09-14T23:59:19.999489+0000","last_unstale":"2024-09-14T23:59:46.596746+0000","last_undegraded":"2024-09-14T23:59:46.596746+0000","last_fullsized":"2024-09-14T23:59:46.596746+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"193'402","last_scrub_stamp":"2024-09-14T23:59:46.596657+0000","last_deep_scrub":"193'402","last_deep_scrub_stamp":"2024-09-14T23:59:46.596657+0000","last_clean_scrub_stamp":"2024-09-14T23:59:46.596657+0000","objects_scrubbed":26,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:43:01.399507+0000","scrub_duration":19,"objects_trimmed":4,"snaptrim_duration":0.0042424580000000002,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":3170,"num_read_kb":16380,"num_write":287,"num_write_kb":14646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":7078411,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"193'1010","reported_seq":4763,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-14T23:59:51.531221+0000","last_change":"2024-09-14T23:59:51.531261+0000","last_active":"2024-09-14T23:59:51.531221+0000","last_peered":"2024-09-14T23:59:51.531221+0000","last_clean":"2024-09-14T23:59:51.531221+0000","last_became_active":"2024-09-14T23:58:53.987342+0000","last_became_peered":"2024-09-14T23:58:53.987342+0000","last_unstale":"2024-09-14T23:59:51.531221+0000","last_undegraded":"2024-09-14T23:59:51.531221+0000","last_fullsized":"2024-09-14T23:59:51.531221+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"193'1010","last_scrub_stamp":"2024-09-14T23:59:51.531137+0000","last_deep_scrub":"193'1010","last_deep_scrub_stamp":"2024-09-14T23:59:51.531137+0000","last_clean_scrub_stamp":"2024-09-14T23:59:51.531137+0000","objects_scrubbed":35,"log_size":1010,"log_dups_size":0,"ondisk_log_size":1010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:53:53.444540+0000","scrub_duration":30,"objects_trimmed":0,"snaptrim_duration":0.001659113,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":7188,"num_read_kb":25892,"num_write":1276,"num_write_kb":14588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":290,"num_bytes_recovered":14372748,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":1204,"num_object_clones":0,"num_object_copies":2408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1204,"num_whiteouts":1200,"num_read":310661,"num_read_kb":1031538,"num_write":24584,"num_write_kb":670524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7228,"num_bytes_recovered":332991677,"num_keys_recovered":596,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23408,"ondisk_log_size":23408,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":1771104,"data_compressed":100161,"data_compressed_allocated":135168,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509491,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51548,"kb_used_data":2316,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320292,"statfs":{"total":96636764160,"available":96583979008,"internally_reserved":0,"allocated":2371584,"data_stored":1713775,"data_compressed":33932,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509490,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33452,"kb_used_data":2268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338388,"statfs":{"total":96636764160,"available":96602509312,"internally_reserved":0,"allocated":2322432,"data_stored":1123407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509489,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51504,"kb_used_data":2272,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320336,"statfs":{"total":96636764160,"available":96584024064,"internally_reserved":0,"allocated":2326528,"data_stored":1123426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509491,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77872,"kb_used_data":2272,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293968,"statfs":{"total":96636764160,"available":96557023232,"internally_reserved":0,"allocated":2326528,"data_stored":1123426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509489,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59820,"kb_used_data":2268,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312020,"statfs":{"total":96636764160,"available":96575508480,"internally_reserved":0,"allocated":2322432,"data_stored":1123407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542194,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41780,"kb_used_data":2276,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330060,"statfs":{"total":96636764160,"available":96593981440,"internally_reserved":0,"allocated":2330624,"data_stored":1123445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542193,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55724,"kb_used_data":2332,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316116,"statfs":{"total":96636764160,"available":96579702784,"internally_reserved":0,"allocated":2387968,"data_stored":1713775,"data_compressed":32297,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574897,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33452,"kb_used_data":2268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338388,"statfs":{"total":96636764160,"available":96602509312,"internally_reserved":0,"allocated":2322432,"data_stored":1123407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32297,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":33932,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":33932,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T00:00:04.797 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-15T00:00:05.107 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-15T00:00:05.108 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-15T00:00:05.108 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-15T00:00:05.109 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T00:00:05.144 INFO:tasks.ceph.osd.0:Stopped 2024-09-15T00:00:05.144 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-15T00:00:05.145 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T00:00:05.223 INFO:tasks.ceph.osd.1:Stopped 2024-09-15T00:00:05.223 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-15T00:00:05.223 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T00:00:05.299 INFO:tasks.ceph.osd.2:Stopped 2024-09-15T00:00:05.300 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-15T00:00:05.300 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T00:00:05.374 INFO:tasks.ceph.osd.3:Stopped 2024-09-15T00:00:05.374 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-15T00:00:05.374 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T00:00:05.409 INFO:tasks.ceph.osd.4:Stopped 2024-09-15T00:00:05.410 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-15T00:00:05.410 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T00:00:05.486 INFO:tasks.ceph.osd.5:Stopped 2024-09-15T00:00:05.486 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-15T00:00:05.486 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T00:00:05.569 INFO:tasks.ceph.osd.6:Stopped 2024-09-15T00:00:05.569 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-15T00:00:05.569 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T00:00:05.611 INFO:tasks.ceph.osd.7:Stopped 2024-09-15T00:00:05.611 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-15T00:00:05.611 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-15T00:00:05.611 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T00:00:05.637 INFO:tasks.ceph.mgr.y:Stopped 2024-09-15T00:00:05.638 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-15T00:00:05.638 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T00:00:05.665 INFO:tasks.ceph.mgr.x:Stopped 2024-09-15T00:00:05.665 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-15T00:00:05.665 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-15T00:00:05.665 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T00:00:05.689 INFO:tasks.ceph.mon.a:Stopped 2024-09-15T00:00:05.689 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-15T00:00:05.689 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T00:00:05.741 INFO:tasks.ceph.mon.c:Stopped 2024-09-15T00:00:05.741 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-15T00:00:05.741 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T00:00:05.755 INFO:tasks.ceph.mon.b:Stopped 2024-09-15T00:00:05.755 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-15T00:00:05.755 DEBUG:teuthology.orchestra.run.smithi042:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v noscrub | egrep -v nodeep-scrub | head -n 1 2024-09-15T00:00:05.824 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi042.front.sepia.ceph.com 2024-09-15T00:00:05.825 DEBUG:teuthology.orchestra.run.smithi042:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-15T00:00:06.086 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi042.front.sepia.ceph.com 2024-09-15T00:00:06.086 DEBUG:teuthology.orchestra.run.smithi042:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-15T00:00:06.214 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi042.front.sepia.ceph.com 2024-09-15T00:00:06.214 DEBUG:teuthology.orchestra.run.smithi042:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-15T00:00:06.272 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi042.front.sepia.ceph.com 2024-09-15T00:00:06.273 DEBUG:teuthology.orchestra.run.smithi042:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-15T00:00:06.341 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi043.front.sepia.ceph.com 2024-09-15T00:00:06.341 DEBUG:teuthology.orchestra.run.smithi043:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-15T00:00:06.650 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi043.front.sepia.ceph.com 2024-09-15T00:00:06.650 DEBUG:teuthology.orchestra.run.smithi043:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-15T00:00:06.791 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi043.front.sepia.ceph.com 2024-09-15T00:00:06.791 DEBUG:teuthology.orchestra.run.smithi043:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-15T00:00:06.923 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi043.front.sepia.ceph.com 2024-09-15T00:00:06.923 DEBUG:teuthology.orchestra.run.smithi043:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-15T00:00:07.074 INFO:tasks.ceph:Archiving mon data... 2024-09-15T00:00:07.075 DEBUG:teuthology.misc:Transferring archived files from smithi042:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904803/data/mon.a.tgz 2024-09-15T00:00:07.076 DEBUG:teuthology.orchestra.run.smithi042:> mktemp 2024-09-15T00:00:07.095 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/tmp.nhey572t8H 2024-09-15T00:00:07.096 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.nhey572t8H 2024-09-15T00:00:07.272 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 0666 /tmp/tmp.nhey572t8H 2024-09-15T00:00:07.357 DEBUG:teuthology.orchestra.remote:smithi042:/tmp/tmp.nhey572t8H is 427KB 2024-09-15T00:00:07.457 DEBUG:teuthology.orchestra.run.smithi042:> rm -fr /tmp/tmp.nhey572t8H 2024-09-15T00:00:07.473 DEBUG:teuthology.misc:Transferring archived files from smithi042:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904803/data/mon.c.tgz 2024-09-15T00:00:07.473 DEBUG:teuthology.orchestra.run.smithi042:> mktemp 2024-09-15T00:00:07.528 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/tmp.HgwULiXE9e 2024-09-15T00:00:07.528 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.HgwULiXE9e 2024-09-15T00:00:07.716 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 0666 /tmp/tmp.HgwULiXE9e 2024-09-15T00:00:07.802 DEBUG:teuthology.orchestra.remote:smithi042:/tmp/tmp.HgwULiXE9e is 569KB 2024-09-15T00:00:07.877 DEBUG:teuthology.orchestra.run.smithi042:> rm -fr /tmp/tmp.HgwULiXE9e 2024-09-15T00:00:07.893 DEBUG:teuthology.misc:Transferring archived files from smithi043:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904803/data/mon.b.tgz 2024-09-15T00:00:07.893 DEBUG:teuthology.orchestra.run.smithi043:> mktemp 2024-09-15T00:00:07.913 INFO:teuthology.orchestra.run.smithi043.stdout:/tmp/tmp.1PHp6UFJF5 2024-09-15T00:00:07.914 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.1PHp6UFJF5 2024-09-15T00:00:08.134 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 0666 /tmp/tmp.1PHp6UFJF5 2024-09-15T00:00:08.219 DEBUG:teuthology.orchestra.remote:smithi043:/tmp/tmp.1PHp6UFJF5 is 569KB 2024-09-15T00:00:08.318 DEBUG:teuthology.orchestra.run.smithi043:> rm -fr /tmp/tmp.1PHp6UFJF5 2024-09-15T00:00:08.334 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-15T00:00:08.334 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-15T00:00:08.337 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-15T00:00:08.417 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-15T00:00:08.456 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-15T00:00:08.456 INFO:tasks.ceph:Archiving crash dumps... 2024-09-15T00:00:08.458 DEBUG:teuthology.misc:Transferring archived files from smithi042:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904803/remote/smithi042/crash 2024-09-15T00:00:08.459 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-15T00:00:08.485 DEBUG:teuthology.misc:Transferring archived files from smithi043:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904803/remote/smithi043/crash 2024-09-15T00:00:08.486 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-15T00:00:08.521 INFO:tasks.ceph:Compressing logs... 2024-09-15T00:00:08.521 DEBUG:teuthology.orchestra.run.smithi042:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-15T00:00:08.529 DEBUG:teuthology.orchestra.run.smithi043:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-15T00:00:08.551 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-15T00:00:08.552 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60256.log 2024-09-15T00:00:08.552 INFO:teuthology.orchestra.run.smithi042.stderr:gzip/var/log/ceph/ceph-osd.0.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.36921.log 2024-09-15T00:00:08.605 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32898.log 2024-09-15T00:00:08.605 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32994.log 2024-09-15T00:00:08.605 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-15T00:00:08.606 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.32898.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32898.log.gz 2024-09-15T00:00:08.606 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32858.log 2024-09-15T00:00:08.606 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.32994.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32994.log.gz 2024-09-15T00:00:08.606 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33114.log 2024-09-15T00:00:08.606 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.32858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32954.log 2024-09-15T00:00:08.606 INFO:teuthology.orchestra.run.smithi043.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32858.log.gz 2024-09-15T00:00:08.606 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33114.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-15T00:00:08.606 INFO:teuthology.orchestra.run.smithi043.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33114.log.gz 2024-09-15T00:00:08.606 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.32954.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 0.0% 2024-09-15T00:00:08.606 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.32954.log.gz 2024-09-15T00:00:08.606 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33034.log 2024-09-15T00:00:08.606 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-15T00:00:08.606 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33034.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33034.log.gz 2024-09-15T00:00:08.606 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-15T00:00:08.606 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-15T00:00:08.607 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-osd.6.log: /var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33154.log 2024-09-15T00:00:08.607 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-15T00:00:08.607 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33154.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-15T00:00:08.607 INFO:teuthology.orchestra.run.smithi043.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33154.log.gz 2024-09-15T00:00:08.607 INFO:teuthology.orchestra.run.smithi043.stderr: 89.4% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-15T00:00:08.607 INFO:teuthology.orchestra.run.smithi043.stderr:gzip/var/log/ceph/ceph-mgr.x.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.33074.log 2024-09-15T00:00:08.607 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-client.admin.33074.log: 84.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-15T00:00:08.607 INFO:teuthology.orchestra.run.smithi043.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33074.log.gz 2024-09-15T00:00:08.607 INFO:teuthology.orchestra.run.smithi043.stderr: 93.2% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-15T00:00:08.681 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.60256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58786.log 2024-09-15T00:00:08.681 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60256.log.gz 2024-09-15T00:00:08.681 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60634.log 2024-09-15T00:00:08.681 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36921.log.gz 2024-09-15T00:00:08.681 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.58786.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58786.log.gz 2024-09-15T00:00:08.681 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44776.log 2024-09-15T00:00:08.681 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59131.log 2024-09-15T00:00:08.681 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.60634.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60634.log.gz 2024-09-15T00:00:08.682 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43451.log 2024-09-15T00:00:08.682 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44776.log.gz 2024-09-15T00:00:08.682 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.59131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53810.log 2024-09-15T00:00:08.682 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59131.log.gz 2024-09-15T00:00:08.682 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41352.log 2024-09-15T00:00:08.682 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43451.log.gz 2024-09-15T00:00:08.682 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.53810.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53810.log.gz 2024-09-15T00:00:08.682 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60886.log 2024-09-15T00:00:08.682 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55700.log 2024-09-15T00:00:08.682 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41352.log.gz 2024-09-15T00:00:08.682 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.60886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58116.log 2024-09-15T00:00:08.682 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60886.log.gz 2024-09-15T00:00:08.682 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.55700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60382.log 2024-09-15T00:00:08.683 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55700.log.gz 2024-09-15T00:00:08.683 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.58116.log: /var/log/ceph/ceph-client.admin.44898.log 2024-09-15T00:00:08.683 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58116.log.gz 2024-09-15T00:00:08.683 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.60382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60676.log 2024-09-15T00:00:08.683 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60382.log.gz 2024-09-15T00:00:08.683 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58260.log 2024-09-15T00:00:08.683 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44898.log.gz 2024-09-15T00:00:08.683 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.60676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37095.log 2024-09-15T00:00:08.683 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60676.log.gz 2024-09-15T00:00:08.683 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.58260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37318.log 2024-09-15T00:00:08.683 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58260.log.gz 2024-09-15T00:00:08.683 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54881.log 2024-09-15T00:00:08.683 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37095.log.gz 2024-09-15T00:00:08.683 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52552.log 2024-09-15T00:00:08.684 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37318.log.gz 2024-09-15T00:00:08.684 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51990.log 2024-09-15T00:00:08.684 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54881.log.gz 2024-09-15T00:00:08.684 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.52552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48684.log 2024-09-15T00:00:08.684 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52552.log.gz 2024-09-15T00:00:08.684 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.51990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47144.log 2024-09-15T00:00:08.684 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51990.log.gz 2024-09-15T00:00:08.684 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.48684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48543.log 2024-09-15T00:00:08.684 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48684.log.gz 2024-09-15T00:00:08.684 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.47144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60928.log 2024-09-15T00:00:08.684 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47144.log.gz 2024-09-15T00:00:08.684 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.48543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56916.log 2024-09-15T00:00:08.685 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48543.log.gz 2024-09-15T00:00:08.685 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.60928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60424.log 2024-09-15T00:00:08.685 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60928.log.gz 2024-09-15T00:00:08.685 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.56916.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56916.log.gz 2024-09-15T00:00:08.685 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52853.log 2024-09-15T00:00:08.685 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37653.log 2024-09-15T00:00:08.685 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.60424.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60424.log.gz 2024-09-15T00:00:08.685 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.52853.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.52853.log.gz 2024-09-15T00:00:08.685 INFO:teuthology.orchestra.run.smithi042.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.59418.log 2024-09-15T00:00:08.685 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57571.log 2024-09-15T00:00:08.685 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37653.log.gz 2024-09-15T00:00:08.685 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.59418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53331.log 2024-09-15T00:00:08.685 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59418.log.gz 2024-09-15T00:00:08.686 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.57571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46524.log 2024-09-15T00:00:08.686 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57571.log.gz 2024-09-15T00:00:08.686 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.53331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37654.log 2024-09-15T00:00:08.686 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53331.log.gz 2024-09-15T00:00:08.686 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.46524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59358.log 2024-09-15T00:00:08.686 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46524.log.gz 2024-09-15T00:00:08.686 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40812.log 2024-09-15T00:00:08.686 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37654.log.gz 2024-09-15T00:00:08.686 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.59358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38187.log 2024-09-15T00:00:08.686 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59358.log.gz 2024-09-15T00:00:08.686 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60970.log 2024-09-15T00:00:08.686 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40812.log.gz 2024-09-15T00:00:08.686 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46683.log 2024-09-15T00:00:08.686 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38187.log.gz 2024-09-15T00:00:08.687 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.60970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38903.log 2024-09-15T00:00:08.687 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60970.log.gz 2024-09-15T00:00:08.687 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.46683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39953.log 2024-09-15T00:00:08.687 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46683.log.gz 2024-09-15T00:00:08.687 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60214.log 2024-09-15T00:00:08.687 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38903.log.gz 2024-09-15T00:00:08.687 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39090.log 2024-09-15T00:00:08.687 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39953.log.gz 2024-09-15T00:00:08.687 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.60214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49951.log 2024-09-15T00:00:08.687 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60214.log.gz 2024-09-15T00:00:08.687 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38439.log 2024-09-15T00:00:08.687 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39090.log.gz 2024-09-15T00:00:08.687 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39516.log 2024-09-15T00:00:08.687 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49951.log.gz 2024-09-15T00:00:08.687 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43986.log 2024-09-15T00:00:08.688 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38439.log.gz 2024-09-15T00:00:08.688 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43908.log 2024-09-15T00:00:08.688 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39516.log.gz 2024-09-15T00:00:08.688 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39331.log 2024-09-15T00:00:08.688 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43986.log.gz 2024-09-15T00:00:08.688 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43908.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-15T00:00:08.688 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43908.log.gz 2024-09-15T00:00:08.688 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54234.log 2024-09-15T00:00:08.688 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39331.log.gz 2024-09-15T00:00:08.688 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41573.log 2024-09-15T00:00:08.688 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59708.log 2024-09-15T00:00:08.688 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54234.log.gz 2024-09-15T00:00:08.688 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46482.log 2024-09-15T00:00:08.689 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41573.log.gz 2024-09-15T00:00:08.689 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.59708.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59708.log.gz 2024-09-15T00:00:08.689 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48786.log 2024-09-15T00:00:08.689 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.46482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61146.log 2024-09-15T00:00:08.689 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46482.log.gz 2024-09-15T00:00:08.689 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.48786.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48786.log.gz 2024-09-15T00:00:08.689 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53868.log 2024-09-15T00:00:08.689 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.61146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53372.log 2024-09-15T00:00:08.689 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61146.log.gz 2024-09-15T00:00:08.689 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.53868.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53868.log.gz 2024-09-15T00:00:08.689 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60592.log 2024-09-15T00:00:08.689 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.53372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59572.log 2024-09-15T00:00:08.689 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53372.log.gz 2024-09-15T00:00:08.689 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.60592.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60592.log.gz 2024-09-15T00:00:08.689 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51691.log 2024-09-15T00:00:08.690 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.59572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52031.log 2024-09-15T00:00:08.690 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59572.log.gz 2024-09-15T00:00:08.690 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.51691.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51691.log.gz 2024-09-15T00:00:08.690 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57911.log 2024-09-15T00:00:08.690 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.52031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37137.log 2024-09-15T00:00:08.690 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52031.log.gz 2024-09-15T00:00:08.690 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.57911.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57911.log.gz 2024-09-15T00:00:08.690 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58744.log 2024-09-15T00:00:08.690 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56596.log 2024-09-15T00:00:08.690 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37137.log.gz 2024-09-15T00:00:08.690 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.58744.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58744.log.gz 2024-09-15T00:00:08.690 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44515.log 2024-09-15T00:00:08.690 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.56596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57354.log 2024-09-15T00:00:08.690 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56596.log.gz 2024-09-15T00:00:08.691 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44515.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.44515.log.gz 2024-09-15T00:00:08.691 INFO:teuthology.orchestra.run.smithi042.stderr: -- /var/log/ceph/ceph-client.admin.37315.log 2024-09-15T00:00:08.691 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.57354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60508.log 2024-09-15T00:00:08.691 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57354.log.gz 2024-09-15T00:00:08.691 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37315.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37315.log.gz 2024-09-15T00:00:08.691 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50391.log 2024-09-15T00:00:08.691 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.60508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42671.log 2024-09-15T00:00:08.691 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60508.log.gz 2024-09-15T00:00:08.691 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50391.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50391.log.gz 2024-09-15T00:00:08.691 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37649.log 2024-09-15T00:00:08.691 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43145.log 2024-09-15T00:00:08.691 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42671.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42671.log.gz 2024-09-15T00:00:08.691 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45586.log 2024-09-15T00:00:08.691 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37649.log.gz 2024-09-15T00:00:08.691 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56216.log 2024-09-15T00:00:08.692 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43145.log.gz 2024-09-15T00:00:08.692 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45586.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45586.log.gz 2024-09-15T00:00:08.692 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51651.log 2024-09-15T00:00:08.692 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.56216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50334.log 2024-09-15T00:00:08.692 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56216.log.gz 2024-09-15T00:00:08.692 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.51651.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51651.log.gz 2024-09-15T00:00:08.692 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59707.log 2024-09-15T00:00:08.692 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46043.log 2024-09-15T00:00:08.692 INFO:teuthology.orchestra.run.smithi042.stderr: 28.8% -- replaced with /var/log/ceph/ceph-client.admin.50334.log.gz 2024-09-15T00:00:08.692 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.59707.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59707.log.gz 2024-09-15T00:00:08.692 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41888.log 2024-09-15T00:00:08.692 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.46043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38640.log 2024-09-15T00:00:08.692 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46043.log.gz 2024-09-15T00:00:08.692 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41888.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41888.log.gz 2024-09-15T00:00:08.693 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48005.log 2024-09-15T00:00:08.693 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38229.log 2024-09-15T00:00:08.693 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38640.log.gz 2024-09-15T00:00:08.693 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.48005.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48005.log.gz 2024-09-15T00:00:08.693 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49093.log 2024-09-15T00:00:08.693 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50550.log 2024-09-15T00:00:08.693 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38229.log.gz 2024-09-15T00:00:08.693 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49093.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49093.log.gz 2024-09-15T00:00:08.693 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38313.log 2024-09-15T00:00:08.693 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39756.log 2024-09-15T00:00:08.693 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50550.log.gz 2024-09-15T00:00:08.693 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38313.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38313.log.gz 2024-09-15T00:00:08.693 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50913.log 2024-09-15T00:00:08.693 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53769.log 2024-09-15T00:00:08.694 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39756.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39756.log.gz 2024-09-15T00:00:08.694 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47807.log 2024-09-15T00:00:08.694 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50913.log.gz 2024-09-15T00:00:08.694 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.53769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47766.log 2024-09-15T00:00:08.694 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53769.log.gz 2024-09-15T00:00:08.694 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.47807.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47807.log.gz 2024-09-15T00:00:08.694 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52950.log 2024-09-15T00:00:08.694 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.47766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45205.log 2024-09-15T00:00:08.694 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47766.log.gz 2024-09-15T00:00:08.694 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.52950.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52950.log.gz 2024-09-15T00:00:08.694 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36816.log 2024-09-15T00:00:08.694 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60004.log 2024-09-15T00:00:08.694 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45205.log.gz 2024-09-15T00:00:08.694 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36816.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36816.log.gz 2024-09-15T00:00:08.694 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49555.log 2024-09-15T00:00:08.695 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.60004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47343.log 2024-09-15T00:00:08.695 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60004.log.gz 2024-09-15T00:00:08.695 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49555.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49555.log.gz 2024-09-15T00:00:08.695 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57154.log 2024-09-15T00:00:08.695 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.47343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59136.log 2024-09-15T00:00:08.695 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47343.log.gz 2024-09-15T00:00:08.695 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.57154.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57154.log.gz 2024-09-15T00:00:08.695 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56097.log 2024-09-15T00:00:08.695 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50712.log 2024-09-15T00:00:08.695 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.59136.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59136.log.gz 2024-09-15T00:00:08.695 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.56097.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.56097.log.gz 2024-09-15T00:00:08.695 INFO:teuthology.orchestra.run.smithi042.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.61097.log 2024-09-15T00:00:08.695 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60718.log 2024-09-15T00:00:08.695 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50712.log.gz 2024-09-15T00:00:08.696 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.61097.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61097.log.gz 2024-09-15T00:00:08.696 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39028.log 2024-09-15T00:00:08.696 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.60718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54333.log 2024-09-15T00:00:08.696 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60718.log.gz 2024-09-15T00:00:08.696 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39028.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39028.log.gz 2024-09-15T00:00:08.696 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45307.log 2024-09-15T00:00:08.696 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60088.log 2024-09-15T00:00:08.696 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54333.log.gz 2024-09-15T00:00:08.696 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40930.log 2024-09-15T00:00:08.696 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45307.log.gz 2024-09-15T00:00:08.696 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.60088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46381.log 2024-09-15T00:00:08.696 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60088.log.gz 2024-09-15T00:00:08.696 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40930.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40930.log.gz 2024-09-15T00:00:08.696 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59592.log 2024-09-15T00:00:08.696 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.46381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37317.log 2024-09-15T00:00:08.697 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46381.log.gz 2024-09-15T00:00:08.697 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.59592.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59592.log.gz 2024-09-15T00:00:08.697 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47385.log 2024-09-15T00:00:08.697 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33338.log 2024-09-15T00:00:08.697 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37317.log.gz 2024-09-15T00:00:08.697 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.47385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39673.log 2024-09-15T00:00:08.697 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47385.log.gz 2024-09-15T00:00:08.697 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57713.log 2024-09-15T00:00:08.697 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33338.log.gz 2024-09-15T00:00:08.697 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39673.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39673.log.gz 2024-09-15T00:00:08.697 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49672.log 2024-09-15T00:00:08.697 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.57713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60802.log 2024-09-15T00:00:08.697 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57713.log.gz 2024-09-15T00:00:08.697 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49672.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49672.log.gz 2024-09-15T00:00:08.697 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51213.log 2024-09-15T00:00:08.698 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.60802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42550.log 2024-09-15T00:00:08.698 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60802.log.gz 2024-09-15T00:00:08.698 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.51213.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51213.log.gz -5 2024-09-15T00:00:08.698 INFO:teuthology.orchestra.run.smithi042.stderr: --verbose -- /var/log/ceph/ceph-client.admin.37987.log 2024-09-15T00:00:08.698 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42550.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-15T00:00:08.698 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42550.log.gz 2024-09-15T00:00:08.698 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37987.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37987.log.gz 2024-09-15T00:00:08.698 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58259.log 2024-09-15T00:00:08.698 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56535.log 2024-09-15T00:00:08.698 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.58259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46301.log 2024-09-15T00:00:08.698 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58259.log.gz 2024-09-15T00:00:08.698 INFO:teuthology.orchestra.run.smithi042.stderr:gzip/var/log/ceph/ceph-client.admin.56535.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.43007.log 2024-09-15T00:00:08.698 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56535.log.gz 2024-09-15T00:00:08.698 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.46301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52168.log 2024-09-15T00:00:08.699 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46301.log.gz 2024-09-15T00:00:08.699 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41749.log 2024-09-15T00:00:08.699 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43007.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43007.log.gz 2024-09-15T00:00:08.699 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.52168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37652.log 2024-09-15T00:00:08.699 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52168.log.gz 2024-09-15T00:00:08.699 INFO:teuthology.orchestra.run.smithi042.stderr:gzip/var/log/ceph/ceph-client.admin.41749.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.43269.log 2024-09-15T00:00:08.699 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41749.log.gz 2024-09-15T00:00:08.699 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55497.log 2024-09-15T00:00:08.699 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37652.log.gz 2024-09-15T00:00:08.699 INFO:teuthology.orchestra.run.smithi042.stderr: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-15T00:00:08.699 INFO:teuthology.orchestra.run.smithi042.stderr:gzip/var/log/ceph/ceph-client.admin.43269.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.48204.log 2024-09-15T00:00:08.699 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43269.log.gz 2024-09-15T00:00:08.699 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.55497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60172.log 2024-09-15T00:00:08.699 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55497.log.gz 2024-09-15T00:00:08.699 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.48204.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48204.log.gz 2024-09-15T00:00:08.700 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43966.log 2024-09-15T00:00:08.700 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.60172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60844.log 2024-09-15T00:00:08.700 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60172.log.gz 2024-09-15T00:00:08.700 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51491.log 0.0% 2024-09-15T00:00:08.700 INFO:teuthology.orchestra.run.smithi042.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43966.log.gz 2024-09-15T00:00:08.700 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.60844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59792.log 2024-09-15T00:00:08.700 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60844.log.gz 2024-09-15T00:00:08.700 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.51491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48301.log 2024-09-15T00:00:08.700 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51491.log.gz 2024-09-15T00:00:08.700 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.59792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45805.log 2024-09-15T00:00:08.700 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59792.log.gz 2024-09-15T00:00:08.700 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.48301.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48301.log.gz 2024-09-15T00:00:08.700 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52266.log 2024-09-15T00:00:08.700 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55659.log 2024-09-15T00:00:08.700 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45805.log.gz 2024-09-15T00:00:08.701 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.52266.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52266.log.gz 2024-09-15T00:00:08.701 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37312.log 2024-09-15T00:00:08.701 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.55659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48642.log 2024-09-15T00:00:08.701 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55659.log.gz 2024-09-15T00:00:08.701 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37312.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37312.log.gz 2024-09-15T00:00:08.701 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39152.log 2024-09-15T00:00:08.701 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.48642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49450.log 2024-09-15T00:00:08.701 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48642.log.gz 2024-09-15T00:00:08.701 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39152.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39152.log.gz 2024-09-15T00:00:08.701 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54939.log 2024-09-15T00:00:08.701 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40591.log 2024-09-15T00:00:08.701 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49450.log.gz 2024-09-15T00:00:08.701 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37002.log 2024-09-15T00:00:08.701 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54939.log.gz 2024-09-15T00:00:08.702 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37634.log 2024-09-15T00:00:08.702 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40591.log.gz 2024-09-15T00:00:08.702 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37002.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37002.log.gz 2024-09-15T00:00:08.702 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48070.log 2024-09-15T00:00:08.702 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54536.log 2024-09-15T00:00:08.702 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37634.log.gz 2024-09-15T00:00:08.702 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.48070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37310.log 2024-09-15T00:00:08.702 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48070.log.gz 2024-09-15T00:00:08.702 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37991.log 2024-09-15T00:00:08.702 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54536.log.gz 2024-09-15T00:00:08.702 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49253.log 2024-09-15T00:00:08.702 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37310.log.gz 2024-09-15T00:00:08.702 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38641.log 2024-09-15T00:00:08.702 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37991.log.gz 2024-09-15T00:00:08.702 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49253.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49253.log.gz 2024-09-15T00:00:08.703 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59750.log 2024-09-15T00:00:08.703 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47425.log 2024-09-15T00:00:08.703 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38641.log.gz 2024-09-15T00:00:08.703 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.59750.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59750.log.gz 2024-09-15T00:00:08.703 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54778.log 2024-09-15T00:00:08.703 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.47425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41510.log 2024-09-15T00:00:08.703 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47425.log.gz 2024-09-15T00:00:08.703 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54778.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54778.log.gz 2024-09-15T00:00:08.703 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44070.log 2024-09-15T00:00:08.703 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41510.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41510.log.gz 2024-09-15T00:00:08.703 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38555.log 2024-09-15T00:00:08.703 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54455.log 2024-09-15T00:00:08.703 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44070.log.gz 2024-09-15T00:00:08.703 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.0.38555.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-15T00:00:08.703 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54455.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54455.log.gz 2024-09-15T00:00:08.704 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44614.log 2024-09-15T00:00:08.704 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43090.log 2024-09-15T00:00:08.704 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-mon.c.log: 95.8%/var/log/ceph/ceph-client.admin.44614.log: -- replaced with /var/log/ceph/ceph-client.0.38555.log.gz 2024-09-15T00:00:08.704 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44614.log.gz 2024-09-15T00:00:08.704 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42871.log 2024-09-15T00:00:08.704 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59351.log 2024-09-15T00:00:08.704 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43090.log.gz 2024-09-15T00:00:08.704 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42871.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.27099.log 2024-09-15T00:00:08.704 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42871.log.gz 2024-09-15T00:00:08.704 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.59351.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59351.log.gz 2024-09-15T00:00:08.704 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44756.log 2024-09-15T00:00:08.704 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph.tmp-client.admin.27099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40972.log 2024-09-15T00:00:08.704 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.27099.log.gz 2024-09-15T00:00:08.704 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38768.log 2024-09-15T00:00:08.704 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44756.log.gz 2024-09-15T00:00:08.705 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45643.log 2024-09-15T00:00:08.705 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40972.log.gz 2024-09-15T00:00:08.705 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58702.log 2024-09-15T00:00:08.705 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38768.log.gz 2024-09-15T00:00:08.705 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45643.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45643.log.gz 2024-09-15T00:00:08.705 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47467.log 2024-09-15T00:00:08.705 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.58702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55359.log 2024-09-15T00:00:08.705 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58702.log.gz 2024-09-15T00:00:08.705 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.47467.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47467.log.gz -5 2024-09-15T00:00:08.705 INFO:teuthology.orchestra.run.smithi042.stderr: --verbose -- /var/log/ceph/ceph-client.admin.46002.log 2024-09-15T00:00:08.705 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.55359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52512.log 2024-09-15T00:00:08.705 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55359.log.gz 2024-09-15T00:00:08.705 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.46002.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.46002.log.gz 2024-09-15T00:00:08.705 INFO:teuthology.orchestra.run.smithi042.stderr: -- /var/log/ceph/ceph-client.admin.49971.log 2024-09-15T00:00:08.706 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.52512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58344.log 2024-09-15T00:00:08.706 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52512.log.gz 2024-09-15T00:00:08.706 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49971.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.59748.log -- replaced with /var/log/ceph/ceph-client.admin.49971.log.gz 2024-09-15T00:00:08.706 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-09-15T00:00:08.706 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.58344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59354.log 2024-09-15T00:00:08.706 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58344.log.gz 2024-09-15T00:00:08.706 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.59748.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59748.log.gz -5 2024-09-15T00:00:08.706 INFO:teuthology.orchestra.run.smithi042.stderr: --verbose -- /var/log/ceph/ceph-client.admin.58576.log 2024-09-15T00:00:08.706 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.59354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46584.log 2024-09-15T00:00:08.706 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59354.log.gz 2024-09-15T00:00:08.706 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.58576.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58576.log.gz 2024-09-15T00:00:08.706 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58492.log 2024-09-15T00:00:08.706 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.46584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40095.log 2024-09-15T00:00:08.706 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46584.log.gz 2024-09-15T00:00:08.706 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.58492.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.58492.log.gz /var/log/ceph/ceph-client.admin.51609.log 2024-09-15T00:00:08.707 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-09-15T00:00:08.707 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40095.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-15T00:00:08.707 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40095.log.gz 2024-09-15T00:00:08.707 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.51609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59132.log 2024-09-15T00:00:08.707 INFO:teuthology.orchestra.run.smithi042.stderr: 28.8% -- replaced with /var/log/ceph/ceph-client.admin.51609.log.gz 2024-09-15T00:00:08.707 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37626.log 2024-09-15T00:00:08.707 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.59132.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59132.log.gz 2024-09-15T00:00:08.707 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38841.log 2024-09-15T00:00:08.707 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60044.log 2024-09-15T00:00:08.707 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37626.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37626.log.gz 2024-09-15T00:00:08.707 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43528.log 2024-09-15T00:00:08.707 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38841.log.gz 2024-09-15T00:00:08.707 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5/var/log/ceph/ceph-client.admin.60044.log: --verbose -- /var/log/ceph/ceph-client.admin.37311.log 2024-09-15T00:00:08.707 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60044.log.gz 2024-09-15T00:00:08.708 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59125.log 2024-09-15T00:00:08.708 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43528.log.gz 2024-09-15T00:00:08.708 INFO:teuthology.orchestra.run.smithi042.stderr:gzip/var/log/ceph/ceph-client.admin.37311.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.49449.log 2024-09-15T00:00:08.708 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37311.log.gz 2024-09-15T00:00:08.708 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.59125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46440.log 2024-09-15T00:00:08.708 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59125.log.gz 2024-09-15T00:00:08.708 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54840.log 2024-09-15T00:00:08.708 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49449.log.gz 2024-09-15T00:00:08.708 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.46440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44191.log 2024-09-15T00:00:08.708 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46440.log.gz 2024-09-15T00:00:08.708 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58534.log 2024-09-15T00:00:08.708 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54840.log.gz 2024-09-15T00:00:08.708 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40273.log 0.0% 2024-09-15T00:00:08.708 INFO:teuthology.orchestra.run.smithi042.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44191.log.gz 2024-09-15T00:00:08.709 INFO:teuthology.orchestra.run.smithi042.stderr:gzip/var/log/ceph/ceph-client.admin.58534.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.45121.log 2024-09-15T00:00:08.709 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58534.log.gz 2024-09-15T00:00:08.709 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48600.log 2024-09-15T00:00:08.709 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40273.log.gz 2024-09-15T00:00:08.709 INFO:teuthology.orchestra.run.smithi042.stderr:gzip/var/log/ceph/ceph-client.admin.45121.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.44555.log 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45121.log.gz 2024-09-15T00:00:08.709 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-09-15T00:00:08.709 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.48600.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48600.log.gz 2024-09-15T00:00:08.709 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-15T00:00:08.709 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44555.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44555.log.gz 2024-09-15T00:00:08.709 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37986.log 2024-09-15T00:00:08.709 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42268.log 2024-09-15T00:00:08.709 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-osd.2.log: /var/log/ceph/ceph-client.admin.37986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59908.log 2024-09-15T00:00:08.709 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37986.log.gz 2024-09-15T00:00:08.709 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60130.log 2024-09-15T00:00:08.709 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42268.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42268.log.gz 2024-09-15T00:00:08.710 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56956.log/var/log/ceph/ceph-client.admin.59908.log: 2024-09-15T00:00:08.710 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59908.log.gz 2024-09-15T00:00:08.710 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.60130.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60130.log.gz 2024-09-15T00:00:08.710 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50453.log 2024-09-15T00:00:08.710 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.56956.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56956.log.gz 2024-09-15T00:00:08.710 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41791.log 2024-09-15T00:00:08.710 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50453.log: gzip -5 --verbose 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50453.log.gz 2024-09-15T00:00:08.710 INFO:teuthology.orchestra.run.smithi042.stderr: -- /var/log/ceph/ceph-client.admin.45017.log 2024-09-15T00:00:08.710 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41791.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41791.log.gz 2024-09-15T00:00:08.710 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42672.log 2024-09-15T00:00:08.710 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45017.log: gzip -5 --verbose 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45017.log.gz 2024-09-15T00:00:08.710 INFO:teuthology.orchestra.run.smithi042.stderr: -- /var/log/ceph/ceph-client.admin.57252.log 2024-09-15T00:00:08.710 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42672.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42672.log.gz 2024-09-15T00:00:08.710 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40193.log 2024-09-15T00:00:08.711 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.57252.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57252.log.gz 2024-09-15T00:00:08.711 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50816.log 2024-09-15T00:00:08.711 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40193.log: gzip -5 --verbose 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40193.log.gz 2024-09-15T00:00:08.711 INFO:teuthology.orchestra.run.smithi042.stderr: -- /var/log/ceph/ceph-client.admin.60340.log 2024-09-15T00:00:08.711 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50816.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50816.log.gz 2024-09-15T00:00:08.711 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-15T00:00:08.711 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.60340.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60340.log.gz 2024-09-15T00:00:08.711 INFO:teuthology.orchestra.run.smithi042.stderr: --verbose -- /var/log/ceph/ceph-client.admin.37989.log 2024-09-15T00:00:08.711 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43867.log 2024-09-15T00:00:08.711 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37989.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55181.log 2024-09-15T00:00:08.711 INFO:teuthology.orchestra.run.smithi042.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37989.log.gz 2024-09-15T00:00:08.711 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53052.log 2024-09-15T00:00:08.711 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43867.log: /var/log/ceph/ceph-client.admin.55181.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43867.log.gz 2024-09-15T00:00:08.711 INFO:teuthology.orchestra.run.smithi042.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55181.log.gz 2024-09-15T00:00:08.711 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58954.log 2024-09-15T00:00:08.712 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45163.log 2024-09-15T00:00:08.712 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.53052.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53052.log.gz 2024-09-15T00:00:08.712 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.58954.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58954.log.gz 2024-09-15T00:00:08.712 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58912.log 2024-09-15T00:00:08.712 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45163.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45163.log.gz 2024-09-15T00:00:08.712 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55961.log 2024-09-15T00:00:08.712 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59668.log 2024-09-15T00:00:08.712 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.58912.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58912.log.gz 2024-09-15T00:00:08.712 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.55961.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-15T00:00:08.712 INFO:teuthology.orchestra.run.smithi042.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55961.log.gz 2024-09-15T00:00:08.712 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.59668.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59668.log.gz 2024-09-15T00:00:08.712 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37313.log 2024-09-15T00:00:08.712 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36977.log 2024-09-15T00:00:08.712 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.37313.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37313.log.gz 2024-09-15T00:00:08.713 INFO:teuthology.orchestra.run.smithi042.stderr: 91.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-15T00:00:08.713 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51270.log 2024-09-15T00:00:08.713 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47281.log 2024-09-15T00:00:08.713 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36977.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36977.log.gz 2024-09-15T00:00:08.713 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.51270.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51270.log.gz 2024-09-15T00:00:08.713 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32935.log 2024-09-15T00:00:08.713 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.47281.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47281.log.gz 2024-09-15T00:00:08.713 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60466.log 2024-09-15T00:00:08.713 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54435.log 2024-09-15T00:00:08.713 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.32935.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32935.log.gz 2024-09-15T00:00:08.713 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.60466.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60466.log.gz 2024-09-15T00:00:08.713 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59130.log 2024-09-15T00:00:08.713 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54435.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54435.log.gz 2024-09-15T00:00:08.713 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60760.log 2024-09-15T00:00:08.714 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59134.log 2024-09-15T00:00:08.714 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.59130.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59130.log.gz 2024-09-15T00:00:08.714 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.60760.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60760.log.gz 2024-09-15T00:00:08.714 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58198.log 2024-09-15T00:00:08.714 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.59134.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59134.log.gz 2024-09-15T00:00:08.714 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58618.log 2024-09-15T00:00:08.714 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38637.log 2024-09-15T00:00:08.714 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.58198.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58198.log.gz 2024-09-15T00:00:08.714 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.58618.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58618.log.gz 2024-09-15T00:00:08.714 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33237.log 2024-09-15T00:00:08.714 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38637.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38637.log.gz 2024-09-15T00:00:08.714 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46905.log 2024-09-15T00:00:08.714 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38397.log 2024-09-15T00:00:08.714 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.33237.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33237.log.gz 2024-09-15T00:00:08.715 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.46905.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46905.log.gz 2024-09-15T00:00:08.715 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52470.log 2024-09-15T00:00:08.715 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38397.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38397.log.gz 2024-09-15T00:00:08.715 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40395.log 2024-09-15T00:00:08.715 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55278.log 2024-09-15T00:00:08.715 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.52470.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52470.log.gz 2024-09-15T00:00:08.715 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40395.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40395.log.gz 2024-09-15T00:00:08.715 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45079.log 2024-09-15T00:00:08.715 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.55278.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55278.log.gz 2024-09-15T00:00:08.715 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41012.log 2024-09-15T00:00:08.715 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42187.log 2024-09-15T00:00:08.715 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45079.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45079.log.gz 2024-09-15T00:00:08.715 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41012.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41012.log.gz 2024-09-15T00:00:08.715 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42090.log 2024-09-15T00:00:08.715 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42187.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42187.log.gz 2024-09-15T00:00:08.716 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56458.log 2024-09-15T00:00:08.716 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58828.log 2024-09-15T00:00:08.716 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42090.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42090.log.gz 2024-09-15T00:00:08.716 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.56458.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56458.log.gz 2024-09-15T00:00:08.716 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54070.log 2024-09-15T00:00:08.716 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.58828.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58828.log.gz 2024-09-15T00:00:08.716 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37650.log 2024-09-15T00:00:08.716 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57790.log 2024-09-15T00:00:08.716 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54070.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54070.log.gz 2024-09-15T00:00:08.716 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37650.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37650.log.gz 2024-09-15T00:00:08.716 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38966.log 2024-09-15T00:00:08.716 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.57790.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57790.log.gz 2024-09-15T00:00:08.716 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60550.log 2024-09-15T00:00:08.716 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39715.log 2024-09-15T00:00:08.717 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38966.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38966.log.gz 2024-09-15T00:00:08.717 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.60550.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60550.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58450.log 2024-09-15T00:00:08.717 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-09-15T00:00:08.717 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39715.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39715.log.gz 2024-09-15T00:00:08.717 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58408.log 2024-09-15T00:00:08.717 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44430.log 2024-09-15T00:00:08.717 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.58450.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58450.log.gz 2024-09-15T00:00:08.717 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.58408.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58408.log.gz 2024-09-15T00:00:08.717 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43588.log 2024-09-15T00:00:08.717 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44430.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44430.log.gz 2024-09-15T00:00:08.717 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52428.log 2024-09-15T00:00:08.717 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48724.log 2024-09-15T00:00:08.717 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43588.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43588.log.gz 2024-09-15T00:00:08.717 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.52428.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44391.log 2024-09-15T00:00:08.717 INFO:teuthology.orchestra.run.smithi042.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52428.log.gz 2024-09-15T00:00:08.718 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.48724.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48724.log.gz 2024-09-15T00:00:08.718 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53248.log 2024-09-15T00:00:08.718 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37985.log 2024-09-15T00:00:08.718 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44391.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44391.log.gz 2024-09-15T00:00:08.718 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.53248.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53248.log.gz 2024-09-15T00:00:08.718 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61012.log 2024-09-15T00:00:08.718 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37985.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37985.log.gz 2024-09-15T00:00:08.718 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49073.log 2024-09-15T00:00:08.718 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50033.log 2024-09-15T00:00:08.718 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.61012.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61012.log.gz 2024-09-15T00:00:08.718 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49073.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49073.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38355.log 2024-09-15T00:00:08.718 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-09-15T00:00:08.718 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50033.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50033.log.gz 2024-09-15T00:00:08.718 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53610.log 2024-09-15T00:00:08.719 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58343.log 2024-09-15T00:00:08.719 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38355.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38355.log.gz 2024-09-15T00:00:08.719 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.53610.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53610.log.gz 2024-09-15T00:00:08.719 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60298.log 2024-09-15T00:00:08.719 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.58343.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58343.log.gz 2024-09-15T00:00:08.719 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46962.log 2024-09-15T00:00:08.719 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37655.log 2024-09-15T00:00:08.719 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.60298.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60298.log.gz 2024-09-15T00:00:08.719 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.46962.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46962.log.gz 2024-09-15T00:00:08.719 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55898.log 2024-09-15T00:00:08.719 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37655.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37655.log.gz 2024-09-15T00:00:08.719 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54254.log 2024-09-15T00:00:08.719 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53727.log 2024-09-15T00:00:08.719 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.55898.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55898.log.gz 2024-09-15T00:00:08.720 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54254.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54254.log.gz 2024-09-15T00:00:08.720 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61054.log 2024-09-15T00:00:08.720 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.53727.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53727.log.gz 2024-09-15T00:00:08.720 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37990.log 2024-09-15T00:00:08.720 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59127.log 2024-09-15T00:00:08.720 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.61054.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61054.log.gz 2024-09-15T00:00:08.720 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37990.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37990.log.gz 2024-09-15T00:00:08.720 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56814.log 2024-09-15T00:00:08.720 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.59127.log: 28.5% -- replaced with /var/log/ceph/ceph-client.admin.59127.log.gz 2024-09-15T00:00:08.720 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37851.log 2024-09-15T00:00:08.720 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41291.log 2024-09-15T00:00:08.720 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.56814.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56814.log.gz 2024-09-15T00:00:08.720 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37851.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37851.log.gz 2024-09-15T00:00:08.720 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56137.log 2024-09-15T00:00:08.721 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41291.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41291.log.gz 2024-09-15T00:00:08.721 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37053.log 2024-09-15T00:00:08.721 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59417.log 2024-09-15T00:00:08.721 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.56137.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56137.log.gz 2024-09-15T00:00:08.721 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37053.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37053.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59135.log 2024-09-15T00:00:08.721 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-09-15T00:00:08.721 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.59417.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59417.log.gz 2024-09-15T00:00:08.721 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42467.log 2024-09-15T00:00:08.721 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40534.log 2024-09-15T00:00:08.721 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.59135.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59135.log.gz 2024-09-15T00:00:08.721 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42467.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42467.log.gz 2024-09-15T00:00:08.721 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38501.log 2024-09-15T00:00:08.721 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40534.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40534.log.gz 2024-09-15T00:00:08.721 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42365.log 2024-09-15T00:00:08.722 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59848.log 2024-09-15T00:00:08.722 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38501.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38501.log.gz 2024-09-15T00:00:08.722 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42365.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42365.log.gz 2024-09-15T00:00:08.722 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38271.log 2024-09-15T00:00:08.722 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.59848.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59848.log.gz 2024-09-15T00:00:08.722 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54696.log 2024-09-15T00:00:08.722 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-15T00:00:08.722 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.38271.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38271.log.gz 2024-09-15T00:00:08.722 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54696.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54696.log.gz 2024-09-15T00:00:08.722 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49793.log 2024-09-15T00:00:08.722 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58660.log 2024-09-15T00:00:08.722 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49793.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49793.log.gz 2024-09-15T00:00:08.722 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37319.log 2024-09-15T00:00:08.722 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.58660.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58660.log.gz 2024-09-15T00:00:08.723 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59411.log 2024-09-15T00:00:08.723 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37319.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37319.log.gz 2024-09-15T00:00:08.723 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58049.log 2024-09-15T00:00:08.723 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.59411.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59411.log.gz 2024-09-15T00:00:08.723 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39274.log 2024-09-15T00:00:08.723 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.58049.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58049.log.gz 2024-09-15T00:00:08.723 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45267.log 2024-09-15T00:00:08.723 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.39274.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39274.log.gz 2024-09-15T00:00:08.723 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54069.log 2024-09-15T00:00:08.723 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45267.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45267.log.gz 2024-09-15T00:00:08.723 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58870.log 2024-09-15T00:00:08.723 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54069.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54069.log.gz 2024-09-15T00:00:08.723 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37988.log 2024-09-15T00:00:08.723 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.58870.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58870.log.gz 2024-09-15T00:00:08.724 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52611.log 2024-09-15T00:00:08.724 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37988.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37988.log.gz 2024-09-15T00:00:08.724 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50993.log 2024-09-15T00:00:08.724 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.52611.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52611.log.gz 2024-09-15T00:00:08.725 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44213.log 2024-09-15T00:00:08.725 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50993.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50993.log.gz 2024-09-15T00:00:08.725 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44213.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44213.log.gz 2024-09-15T00:00:08.726 INFO:teuthology.orchestra.run.smithi042.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-15T00:00:08.776 INFO:teuthology.orchestra.run.smithi042.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-15T00:00:08.800 INFO:teuthology.orchestra.run.smithi043.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-15T00:00:09.014 INFO:teuthology.orchestra.run.smithi042.stderr: 90.3% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-15T00:00:09.355 INFO:teuthology.orchestra.run.smithi042.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-15T00:00:09.806 INFO:teuthology.orchestra.run.smithi043.stderr: 94.6% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-15T00:00:11.073 INFO:teuthology.orchestra.run.smithi043.stderr: 95.9% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-15T00:00:11.249 INFO:teuthology.orchestra.run.smithi042.stderr: 95.3% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-15T00:00:13.023 INFO:teuthology.orchestra.run.smithi043.stderr: 95.7% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-15T00:00:13.257 INFO:teuthology.orchestra.run.smithi042.stderr: 95.4% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-15T00:00:15.869 INFO:teuthology.orchestra.run.smithi043.stderr: 95.6% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-15T00:00:15.871 INFO:teuthology.orchestra.run.smithi043.stderr: 2024-09-15T00:00:15.871 INFO:teuthology.orchestra.run.smithi043.stderr:real 0m7.287s 2024-09-15T00:00:15.871 INFO:teuthology.orchestra.run.smithi043.stderr:user 0m15.029s 2024-09-15T00:00:15.871 INFO:teuthology.orchestra.run.smithi043.stderr:sys 0m0.570s 2024-09-15T00:00:17.397 INFO:teuthology.orchestra.run.smithi042.stderr: 96.1% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-15T00:00:17.399 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-09-15T00:00:17.399 INFO:teuthology.orchestra.run.smithi042.stderr:real 0m8.858s 2024-09-15T00:00:17.399 INFO:teuthology.orchestra.run.smithi042.stderr:user 0m17.002s 2024-09-15T00:00:17.399 INFO:teuthology.orchestra.run.smithi042.stderr:sys 0m0.741s 2024-09-15T00:00:17.399 INFO:tasks.ceph:Archiving logs... 2024-09-15T00:00:17.400 DEBUG:teuthology.misc:Transferring archived files from smithi042:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904803/remote/smithi042/log 2024-09-15T00:00:17.400 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-15T00:00:19.733 DEBUG:teuthology.misc:Transferring archived files from smithi043:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904803/remote/smithi043/log 2024-09-15T00:00:19.734 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-15T00:00:21.478 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-15T00:00:21.498 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-15T00:00:21.498 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-15T00:00:21.501 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-15T00:00:21.574 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-15T00:00:21.643 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-15T00:00:21.643 DEBUG:teuthology.orchestra.run.smithi042:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T00:00:21.646 DEBUG:teuthology.orchestra.run.smithi043:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T00:00:21.661 INFO:teuthology.orchestra.run.smithi042.stderr:bash: line 1: ntpq: command not found 2024-09-15T00:00:21.664 INFO:teuthology.orchestra.run.smithi043.stderr:bash: line 1: ntpq: command not found 2024-09-15T00:00:21.666 INFO:teuthology.orchestra.run.smithi042.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T00:00:21.666 INFO:teuthology.orchestra.run.smithi042.stdout:=============================================================================== 2024-09-15T00:00:21.666 INFO:teuthology.orchestra.run.smithi042.stdout:^* hv01.front.sepia.ceph.com 3 6 377 36 +376us[ +382us] +/- 42ms 2024-09-15T00:00:21.666 INFO:teuthology.orchestra.run.smithi042.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 35 +538us[ +538us] +/- 74ms 2024-09-15T00:00:21.666 INFO:teuthology.orchestra.run.smithi042.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 36 -1399us[-1399us] +/- 79ms 2024-09-15T00:00:21.666 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T00:00:21.668 INFO:teuthology.orchestra.run.smithi043.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T00:00:21.669 INFO:teuthology.orchestra.run.smithi043.stdout:=============================================================================== 2024-09-15T00:00:21.669 INFO:teuthology.orchestra.run.smithi043.stdout:^* hv01.front.sepia.ceph.com 3 6 377 35 +397us[ +407us] +/- 42ms 2024-09-15T00:00:21.669 INFO:teuthology.orchestra.run.smithi043.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 35 +550us[ +550us] +/- 74ms 2024-09-15T00:00:21.669 INFO:teuthology.orchestra.run.smithi043.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 34 -1392us[-1392us] +/- 79ms 2024-09-15T00:00:21.669 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T00:00:21.670 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-15T00:00:21.678 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-15T00:00:21.679 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-15T00:00:21.687 DEBUG:teuthology.orchestra.run.smithi042:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-15T00:00:21.797 DEBUG:teuthology.orchestra.run.smithi043:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-15T00:00:21.879 DEBUG:teuthology.orchestra.run.smithi042:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-15T00:00:21.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T00:00:21.911 DEBUG:teuthology.orchestra.run.smithi043:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-15T00:00:21.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T00:00:21.943 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-15T00:00:21.984 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-15T00:00:21.995 INFO:teuthology.task.internal:Duration was 662.178186 seconds 2024-09-15T00:00:21.995 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-15T00:00:22.004 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-15T00:00:22.004 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-15T00:00:22.006 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-15T00:00:22.047 INFO:teuthology.orchestra.run.smithi042.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T00:00:22.051 INFO:teuthology.orchestra.run.smithi043.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T00:00:22.320 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-15T00:00:22.320 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi042.front.sepia.ceph.com 2024-09-15T00:00:22.320 DEBUG:teuthology.orchestra.run.smithi042:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-15T00:00:22.344 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi043.front.sepia.ceph.com 2024-09-15T00:00:22.344 DEBUG:teuthology.orchestra.run.smithi043:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-15T00:00:22.369 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-15T00:00:22.369 DEBUG:teuthology.orchestra.run.smithi042:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-15T00:00:22.386 DEBUG:teuthology.orchestra.run.smithi043:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-15T00:00:22.440 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-15T00:00:22.440 DEBUG:teuthology.orchestra.run.smithi042:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-15T00:00:22.454 DEBUG:teuthology.orchestra.run.smithi043:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-15T00:00:22.604 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-15T00:00:22.687 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-15T00:00:22.687 DEBUG:teuthology.orchestra.run.smithi042:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-15T00:00:22.713 DEBUG:teuthology.orchestra.run.smithi043:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-15T00:00:22.739 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-15T00:00:22.758 DEBUG:teuthology.orchestra.run.smithi042:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-15T00:00:22.760 DEBUG:teuthology.orchestra.run.smithi043:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-15T00:00:22.782 INFO:teuthology.orchestra.run.smithi042.stdout:kernel.core_pattern = core 2024-09-15T00:00:22.805 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern = core 2024-09-15T00:00:22.824 DEBUG:teuthology.orchestra.run.smithi042:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-15T00:00:22.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T00:00:22.859 DEBUG:teuthology.orchestra.run.smithi043:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-15T00:00:22.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T00:00:22.879 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-15T00:00:22.910 INFO:teuthology.task.internal:Transferring archived files... 2024-09-15T00:00:22.911 DEBUG:teuthology.misc:Transferring archived files from smithi042:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904803/remote/smithi042 2024-09-15T00:00:22.911 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-15T00:00:22.952 DEBUG:teuthology.misc:Transferring archived files from smithi043:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904803/remote/smithi043 2024-09-15T00:00:22.952 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-15T00:00:22.985 INFO:teuthology.task.internal:Removing archive directory... 2024-09-15T00:00:22.985 DEBUG:teuthology.orchestra.run.smithi042:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-15T00:00:22.988 DEBUG:teuthology.orchestra.run.smithi043:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-15T00:00:23.037 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-15T00:00:23.066 INFO:teuthology.task.internal:Not uploading archives. 2024-09-15T00:00:23.066 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-15T00:00:23.077 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-15T00:00:23.078 DEBUG:teuthology.orchestra.run.smithi042:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-15T00:00:23.080 DEBUG:teuthology.orchestra.run.smithi043:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-15T00:00:23.093 INFO:teuthology.orchestra.run.smithi042.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 15 00:00 /home/ubuntu/cephtest 2024-09-15T00:00:23.096 INFO:teuthology.orchestra.run.smithi043.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 15 00:00 /home/ubuntu/cephtest 2024-09-15T00:00:23.097 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-15T00:00:23.108 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-15T00:00:23.129 INFO:teuthology.run:Summary data: description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nocache} duration: 662.1781861782074 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-15T00:00:23.129 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-15T00:00:23.208 INFO:teuthology.run:pass